[原创]ML吴恩达系列习题解答5_machine_learning_ex7

要求如上，实现如下：
第一部分
1.实现findClosestCentroids，就是找到距离最近的中心点

% ====================== YOUR CODE HERE ======================
% Instructions: Go over every example, find its closest centroid, and store
%               the index inside idx at the appropriate location.
%               Concretely, idx(i) should contain the index of the centroid
%               closest to example i. Hence, it should be a value in the 
%               range 1..K
%
% Note: You can use a for-loop over the examples to compute this.
%


tempK = [];
for i=1:size(X,1)
    for j=1:K
        tempK(j) = (X(i,:) - centroids(j,:))*(X(i,:) - centroids(j,:))';
    end
    [~,idx(i)] = min(tempK,[],2);
end

% =============================================================

方法：对每一行X（即每一个样本）计算其到每个中心点的距离平方，对所有计算结果取最小，并将最小的index赋给idx
看看输出

Closest centroids for the first 3 examples: 
 1 3 2
(the closest centroids should be 1, 3, 2 respectively)
Program paused. Press enter to continue.

2.实现computeCentroids，每个样本有了新的归属，那么根据某一归属内的样本选个中心代表，就是计算均值了。由于有K个，所以叫K均值法（K-Means）

% ====================== YOUR CODE HERE ======================
% Instructions: Go over every centroid and compute mean of all points that
%               belong to it. Concretely, the row vector centroids(i, :)
%               should contain the mean of the data points assigned to
%               centroid i.
%
% Note: You can use a for-loop over the centroids to compute this.
%
for i=1:K
    list = find(idx == i);
    sum = 0;
    for j=1:size(list,1)
        sum = sum + X(list(j),:);
    end
    centroids(i,:) = sum./size(list,1);
end

% =============================================================

方法：由于已经把每个样本分配到最近的中心，其对应的中心标号已记录在idx中，因此先把idx归归类。找到idx中存储中心1，中心2，。。。，中心K的对应样本序号（即X对应的行），赋给list，对当前的list求和取平均，即可的新的中心
看看结果

Computing centroids means.

Centroids computed after initial finding of closest centroids: 
 2.428301 3.157924 
 5.813503 2.633656 
 7.119387 3.616684 

(the centroids should be
   [ 2.428301 3.157924 ]
   [ 5.813503 2.633656 ]
   [ 7.119387 3.616684 ]

Program paused. Press enter to continue.

吴老师给出了随机初始化中心K的方法，简洁明了

% ====================== YOUR CODE HERE ======================
% Instructions: You should set centroids to randomly chosen examples from
%               the dataset X
%
randidx = randperm(size(X,1));
centroids = X(randidx(1:K),:);

% =============================================================

看看运行结果


最后把K-Means应用在图像压缩重建，看看效果（24色 -> 16色）

第二部分
3.实现pca，（Principle Component Analysis，主成分分析）

% ====================== YOUR CODE HERE ======================
% Instructions: You should first compute the covariance matrix. Then, you
%               should use the "svd" function to compute the eigenvectors
%               and eigenvalues of the covariance matrix. 
%
% Note: When computing the covariance matrix, remember to divide by m (the
%       number of examples).
%

sigma = X'* X./m;
[U,S,~] = svd(sigma);

% =========================================================================

方法：计算协方差矩阵，进行奇异值分解获得U&S（这里只是套公式，后续我要专门写一篇SVD实现原理）
看看输出

Running PCA on example dataset.

Top eigenvector: 
 U(:,1) = -0.707107 -0.707107 

(you should expect to see -0.707107 -0.707107)
Program paused. Press enter to continue.

4.实现projectData

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the projection of the data using only the top K 
%               eigenvectors in U (first K columns). 
%               For the i-th example X(i,:), the projection on to the k-th 
%               eigenvector is given as follows:
%                    x = X(i, :)';
%                    projection_k = x' * U(:, k);
%

U_reduce = U(:,1:K);
Z = X * U_reduce;

% =============================================================

到这里，我说一下我的理解：
所谓的PCA降维，之所以能够降维，本质上是坐标构建的问题，准确的说是正交向量选取的问题。比如上图大多数点都映射到一条斜线上（二维点面降成一维点线）。如果以上图原来的横纵坐标作映射，则有好多点会映射到点线的同一个点上，将来就无法重建了。所以，PCA主要是发现了数据可以用新的正交向量近似低维表示，大多数数据点也可以一一映射。重建的时候，做个矩阵逆运算即可。

5.实现recoverData

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the approximation of the data by projecting back
%               onto the original space using the top K eigenvectors in U.
%
%               For the i-th example Z(i,:), the (approximate)
%               recovered data for dimension j is given as follows:
%                    v = Z(i, :)';
%                    recovered_j = v' * U(j, 1:K)';
%
%               Notice that U(j, 1:K) is a row vector.
%               

U_reduce = U(:,1:K);
X_rec = Z * U_reduce';

% =============================================================

这个理解应该是 U_reduce*U_reduce' ≈ I 。因为 X_rec = Z * U_reduce' = X * U_reduce* U_reduce'，疑车无据，还是以后证明一下，但首要问题还是分析奇异值分解的原理。
刷个脸看看：

下图是PCA可视化，图美而加，无其他意思

最后，博客提到的理解，仅仅是我个人推断，未经严谨证明，特此声明。

谢谢大家