coursera机器学习第8周的常见问题（原文翻译）

最新推荐文章于 2021-08-08 20:13:39 发布

hanaif

最新推荐文章于 2021-08-08 20:13:39 发布

阅读量542

点赞数

分类专栏：机器学习文章标签： coursera机器学习奇异值分解（SVD）

机器学习专栏收录该内容

10 篇文章 0 订阅

订阅专栏

原文地址：https://www.coursera.org/learn/machine-learning/discussions/weeks/8/threads/XLl24URmEea1pw5frt5utw

The lecture slides are nowavailable in the "Review" section of each week's course materials.

Q1)How do we know which are the most significant features to retain? Why can wechoose just the first K of them? They aren't naturally ordered by significanceare they? Wouldn't it make more sense to select the K most influentialdimensions?

Prof Ng doesn't get into the details of how Singular ValueDecomposition (SVD) works, but it turns out that the output of SVD is exactlywhat we need for the purposes of PCA. It does "spectraldecomposition" of the input matrix and gives it back expressed in a formin which it is obvious which are the important dimensions. The output of SVD is:

[U,S,V] = svd(Sigma);

Where Sigma is the covariance matrix. The output values U and Vare unitary matrices and the columns of U are the eigenvectors of thetransformation. S is a diagonal matrix, containing the correspondingeigenvalues in decreasing order. In order words, the SVD has done the work tofigure out which dimensions are the most significant and gives us the resultsin that order. Prof Ng does discuss this in the video entitled "Choosingthe Number or Principal Components" around the 7:00 mark. There are anumber of good articles on the web that give more information about SVD:

http://mathworld.wolfram.com/SingularValueDecomposition.html

https://en.wikipedia.org/wiki/Singular_value_decomposition

Here's the MATLAB doc for SVD:

http://www.mathworks.com/help/matlab/ref/svd.html

Q2)Why don't we need to compute the matrix inverse of U in order to recover thedata in the original dimensions?

Please have a look at the references about Singular ValueDecomposition above. It turns out that the U matrix that is returned has thespecial property that it is a Unitary Matrix. One of the special properties ofa Unitary Matrix is:

U−1=U∗ wherethe "*" means "conjugate transpose".

Since we are dealing with real numbers here, this is equivalentto:

U−1=UT

So we could compute the inverse and use that, but it would be awaste of energy and compute cycles.

Q3)From the video "Advice for Applying PCA": Why do we only use PCA onthe training set?

If you apply PCA to the entire data set, and then split the setinto training, validation, and test sets, then the data in the original testset will have an impact on the reduced training set.

That causes overfitting.

Ex7Programming Assignment FAQ

Note: Tutorialsand additional Test Cases are available in the Resources menu.

Q1)When I run the image compression step in ex7.m, the image comes out as auniform grey or brown color.

This means that you neglected to fill in the random centroidinitialization code in kMeansInitCentroids. The ex7.pdf file actuallygives you the code to use, but the grader does not grade this routine whichmakes it easy to miss.

Q2)K-Means: For robustness, be sure the initial centroids are unique

(thanksto students Cameron Willden and Paul Nel for raising this issue)

Sometimes when running K-Means, you may get an error message dueto having an empty cluster. This can happen if any of the initial centroids areidentical.

This is most common if you are compressing an image that haslarge areas of a uniform background color.

You can avoid this issue by modifying your kMeanInitCentroids.mscript as follows (updated April 20, 2018):

% create a matrix of only the unique rows

X_unq = unique(X, 'rows');

% create a random permutation of the rows

randidx = randperm(size(X_unq, 1));

% take the first K rows as centroids

centroids = X_unq(randidx(1:K), :);

This code creates a matrix of the unique rows of X, thenrandomly selects from this matrix for the initial centroids.

翻译：

Q1）我们如何知道哪些是最重要的功能要保留？为什么我们只能选择第一个K？他们不是自然地按照意义排序吗？选择K最有影响力的维度不是更有意义吗？

吴教授没有深入讨论奇异值分解（SVD）如何工作的细节，但事实证明，SVD的输出正是我们在PCA中所需要的。它对输入矩阵进行“谱分解”，并以一种明显的形式给出它，这些是重要的维度。SVD的输出是：

[U，S，V] = svd（Sigma）;

其中Sigma是协方差矩阵。输出值U和V是酉矩阵，U的列是变换的特征向量。S是对角矩阵，包含降序的相应特征值。换句话说，SVD已经完成了确定哪些维度最重要的工作，并按照该顺序给出了结果。Ng教授在7点左右的标题为“选择数量或主要部分”的视频中进行了讨论。网络上有许多优秀的文章可以提供有关SVD的更多信息：