[原创]ML吴恩达系列习题解答2_machine_learning_ex3

要求如上，解答如下：
第一部分 正则逻辑回归
1.实现lrCostFunction.m 正则化逻辑回归的代价函数和梯度

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta.
%               You should set J to the cost.
%               Compute the partial derivatives and set grad to the partial
%               derivatives of the cost w.r.t. each parameter in theta
%
% Hint: The computation of the cost function and gradients can be
%       efficiently vectorized. For example, consider the computation
%
%           sigmoid(X * theta)
%
%       Each row of the resulting matrix will contain the value of the
%       prediction for that example. You can make use of this to vectorize
%       the cost function and gradient computations. 
%
% Hint: When computing the gradient of the regularized cost function, 
%       there're many possible vectorized solutions, but one solution
%       looks like:
%           grad = (unregularized gradient for logistic regression)
%           temp = theta; 
%           temp(1) = 0;   % because we don't add anything for j = 0  
%           grad = grad + YOUR_CODE_HERE (using the temp variable)
%

J =(-y'*log(sigmoid(X * theta)) - (1-y')*log(1-sigmoid(X * theta)))/m + lambda.*theta'*theta/(2*m);

grad = X'*(sigmoid(X * theta)- y)./m;
temp = theta;
temp(1) = 0;
grad = grad + lambda.*temp./m;


% =============================================================

结果输出

Loading and Visualizing Data ...
Program paused. Press enter to continue.

Testing lrCostFunction() with regularization
Cost: 3.734819
Expected cost: 2.534819
Gradients:
 0.146561 
 -0.548558 
 0.724722 
 1.398003 
Expected gradients:
 0.146561
 -0.548558
 0.724722
 1.398003
Program paused. Press enter to continue.

我坚信自己的计算结果，我觉得Expected cost 2.534819是错的，梯度倒是挺吻合

2.实现oneVsAll 实现0-9手写数字识别theta调参

% ====================== YOUR CODE HERE ======================
% Instructions: You should complete the following code to train num_labels
%               logistic regression classifiers with regularization
%               parameter lambda. 
%
% Hint: theta(:) will return a column vector.
%
% Hint: You can use y == c to obtain a vector of 1's and 0's that tell you
%       whether the ground truth is true/false for this class.
%
% Note: For this assignment, we recommend using fmincg to optimize the cost
%       function. It is okay to use a for-loop (for c = 1:num_labels) to
%       loop over the different classes.
%
%       fmincg works similarly to fminunc, but is more efficient when we
%       are dealing with large number of parameters.
%
% Example Code for fmincg:
%
% Set Initial theta
initial_theta = zeros(n + 1, 1);

% Set options for fminunc
options = optimset('GradObj', 'on', 'MaxIter', 50);

% Run fmincg to obtain the optimal theta
% This function will return theta and the cost 
for c = 1:num_labels
    [theta] = fmincg (@(t)(lrCostFunction(t, X, (y==c), lambda)),initial_theta, options);
    all_theta(c,:) = theta';
end

% =========================================================================

运行结果

Training One-vs-All Logistic Regression...
Iteration     1 | Cost: 2.802129e-01
Iteration     2 | Cost: 9.454399e-02
Iteration     3 | Cost: 5.704683e-02
Iteration     4 | Cost: 4.688225e-02
Iteration     5 | Cost: 3.759419e-02
Iteration     6 | Cost: 3.522129e-02
***
省略一万字
***
Iteration    41 | Cost: 1.092069e-02
Iteration    42 | Cost: 1.090472e-02
Iteration    43 | Cost: 1.084519e-02
Iteration    44 | Cost: 1.081438e-02
Iteration    45 | Cost: 1.079171e-02
Iteration    46 | Cost: 1.076308e-02
Iteration    47 | Cost: 1.072741e-02
Iteration    48 | Cost: 1.071758e-02
Iteration    49 | Cost: 1.068114e-02
Iteration    50 | Cost: 1.067206e-02

Program paused. Press enter to continue.

3.实现predictOneVsAll 找出预测最大值及其列索引

% ====================== YOUR CODE HERE ======================
% Instructions: Complete the following code to make predictions using
%               your learned logistic regression parameters (one-vs-all).
%               You should set p to a vector of predictions (from 1 to
%               num_labels).
%
% Hint: This code can be done all vectorized using the max function.
%       In particular, the max function can also return the index of the 
%       max element, for more information see 'help max'. If your examples 
%       are in rows, then, you can use max(A, [], 2) to obtain the max 
%       for each row.
%       

[~,p] = max(X*all_theta',[],2);

% =========================================================================

运行结果

Program paused. Press enter to continue.

Training Set Accuracy: 94.640000

吴老师在ex3PDF中提到是94.9%，比较接近了

Once you are done, ex3.m will call your predictOneVsAll function using the learned value of Θ. You should see that the training set accuracy is about 94.9% (i.e., it classifies 94.9% of the examples in the training set correctly).

第二部分 神经网络
4.实现predict 评估神经网络预测准确度

% ====================== YOUR CODE HERE ======================
% Instructions: Complete the following code to make predictions using
%               your learned neural network. You should set p to a 
%               vector containing labels between 1 to num_labels.
%
% Hint: The max function might come in useful. In particular, the max
%       function can also return the index of the max element, for more
%       information see 'help max'. If your examples are in rows, then, you
%       can use max(A, [], 2) to obtain the max for each row.
%

X = [ones(m, 1) X];
A1 = sigmoid(X*Theta1');

A1 = [ones(m, 1) A1];

A2 = sigmoid(A1*Theta2');

[~,p] = max(A2,[],2);



% =========================================================================

运行结果

Loading and Visualizing Data ...
Program paused. Press enter to continue.

Loading Saved Neural Network Parameters ...

Training Set Accuracy: 97.520000
Program paused. Press enter to continue.

Displaying Example Image

Neural Network Prediction: 6 (digit 6)
Paused - press enter to continue, q to exit:

Displaying Example Image

Neural Network Prediction: 6 (digit 6)
Paused - press enter to continue, q to exit:

Displaying Example Image

Neural Network Prediction: 1 (digit 1)
Paused - press enter to continue, q to exit:

Displaying Example Image

Neural Network Prediction: 10 (digit 0)
Paused - press enter to continue, q to exit:

吴老师在ex3PDF中提到是97.5%，比较接近了
Once you are done, ex3 nn.m will call your predict function using the loaded set of parameters for Theta1 and Theta2. You should see that the accuracy is about 97.5%. After that, an interactive sequence will launch displaying images from the training set one at a time, while the console prints out the predicted label for the displayed image.

最后提一下，10和0的问题，这是因为MATLAB下标从1开始，所以1到9下标对应1到9，10下标对应0。这只是个映射关系而已，无伤大雅

谢谢大家