[原创]ML吴恩达系列习题解答3_machine_learning_ex5

我是在网易云课堂看的视频，有点奇怪练习题怎么不连续？
有习题就做吧，不太在意这个问题了

要求如上，解答如下：
1.实现linearRegCostFunction.m 计算代价函数、梯度

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost and gradient of regularized linear 
%               regression for a particular choice of theta.
%
%               You should set J to the cost and grad to the gradient.
%

J =(X * theta - y)'*(X * theta - y)/(2*m) + lambda.*theta'*theta/(2*m);

grad = X'*((X * theta)- y)./m;
temp = theta;
temp(1) = 0;
grad = grad + lambda.*temp./m;

% =========================================================================

结果输出

Loading and Visualizing Data ...
Program paused. Press enter to continue.
Cost at theta = [1 ; 1]: 304.034859 
(this value should be about 303.993192)
Program paused. Press enter to continue.
Gradient at theta = [1 ; 1]:  [-15.303016; 598.250744] 
(this value should be about [-15.303016; 598.250744])
Program paused. Press enter to continue.

弹出个图


一看就是欠拟合，高偏差状态

2.实现learningCurve.m 计算训练误差、验证误差

% ---------------------- Sample Solution ----------------------
for i = 1:m
  theta = trainLinearReg(X(1:i,:), y(1:i,:), 1);
  error_train(i) = linearRegCostFunction(X(1:i,:),y(1:i,:), theta, 0);
  error_val(i) = linearRegCostFunction(Xval, yval, theta, 0);
end
% -------------------------------------------------------------

输出结果

Program paused. Press enter to continue.
Iteration     1 | Cost: 2.702544e-01
Iteration     2 | Cost: 8.897722e-03

Iteration     1 | Cost: 4.291859e-01

Iteration     1 | Cost: 1.021936e+02
Iteration     2 | Cost: 4.554752e+01
Iteration     3 | Cost: 3.917659e+01
Iteration     4 | Cost: 3.425546e+01
Iteration     6 | Cost: 3.404140e+01

Iteration     1 | Cost: 1.438906e+02
Iteration     2 | Cost: 1.035785e+02
Iteration     3 | Cost: 7.687986e+01
Iteration     4 | Cost: 3.562517e+01
Iteration     6 | Cost: 2.408017e+01

Iteration     1 | Cost: 1.592798e+02
Iteration     2 | Cost: 3.973956e+01
Iteration     3 | Cost: 3.965419e+01
Iteration     4 | Cost: 3.937865e+01
Iteration     5 | Cost: 3.890279e+01
Iteration     6 | Cost: 3.877728e+01
Iteration     7 | Cost: 3.874218e+01

Iteration     1 | Cost: 1.531175e+02
Iteration     2 | Cost: 1.351073e+02
Iteration     3 | Cost: 1.142386e+02
Iteration     4 | Cost: 5.008180e+01
Iteration     5 | Cost: 4.060199e+01
Iteration     6 | Cost: 4.017274e+01
Iteration     7 | Cost: 3.700265e+01
Iteration     8 | Cost: 3.677356e+01
Iteration     9 | Cost: 3.657979e+01
Iteration    10 | Cost: 3.653069e+01

Iteration     1 | Cost: 1.383969e+02
Iteration     2 | Cost: 1.210395e+02
Iteration     3 | Cost: 1.017535e+02
Iteration     4 | Cost: 4.032857e+01
Iteration     5 | Cost: 3.638298e+01
Iteration     7 | Cost: 3.477674e+01
Iteration     8 | Cost: 3.454830e+01
Iteration     9 | Cost: 3.451723e+01
Iteration    10 | Cost: 3.402196e+01
Iteration    11 | Cost: 3.302623e+01
Iteration    13 | Cost: 3.296200e+01

Iteration     1 | Cost: 1.237813e+02
Iteration     2 | Cost: 1.202608e+02
Iteration     3 | Cost: 1.195206e+02
Iteration     4 | Cost: 9.398538e+01
Iteration     5 | Cost: 5.681171e+01
Iteration     6 | Cost: 5.551731e+01
Iteration     7 | Cost: 3.108281e+01
Iteration     8 | Cost: 3.084944e+01
Iteration     9 | Cost: 3.005461e+01

Iteration     1 | Cost: 1.090030e+02
Iteration     2 | Cost: 1.064756e+02
Iteration     3 | Cost: 1.054807e+02
Iteration     4 | Cost: 3.198878e+01
Iteration     6 | Cost: 3.198782e+01

Iteration     1 | Cost: 1.108642e+02
Iteration     2 | Cost: 3.208388e+01
Iteration     3 | Cost: 3.208331e+01
Iteration     4 | Cost: 3.207818e+01
Iteration     5 | Cost: 3.204417e+01
Iteration     6 | Cost: 3.192196e+01
Iteration     8 | Cost: 3.191835e+01

Iteration     1 | Cost: 1.023423e+02
Iteration     2 | Cost: 3.147523e+01

Iteration     1 | Cost: 1.052470e+02
Iteration     2 | Cost: 3.008946e+01
Iteration     3 | Cost: 2.982757e+01
Iteration     4 | Cost: 2.941814e+01
Iteration     5 | Cost: 2.941011e+01
Iteration     6 | Cost: 2.939006e+01

# Training Examples	Train Error	Cross Validation Error
  	1		0.000035	204.928109
  	2		0.428234	181.102387
  	3		3.479217	45.428264
  	4		6.485323	58.202967
  	5		14.497483	32.798709
  	6		20.848559	33.425030
  	7		20.944011	33.219292
  	8		19.121174	31.313011
  	9		22.669457	31.549484
  	10		24.100859	30.331855
  	11		24.431052	30.322369
  	12		22.381328	29.586472
Program paused. Press enter to continue.

弹出个图

增加样本个数，并不能改善两个误差，且训练误差和验证误差比较接近，显然是比较高偏差状态

3.实现ployFeatures.m 增加特征输入

% ====================== YOUR CODE HERE ======================
% Instructions: Given a vector X, return a matrix X_poly where the p-th 
%               column of X contains the values of X to the p-th power.
%
% 

for i = 1:p
    X_poly(:,i) = X.^i;
end
% =========================================================================

lambda=0下，用一个高阶多项式拟合数据，会发现训练误差很小，但验证误差很大，典型的高方差过拟合状态

4.实现validationCurve.m 计算不同lambda下的训练误差、验证误差

for i = 1:length(lambda_vec)
  lambda = lambda_vec(i);
  theta = trainLinearReg(X, y, lambda);
  error_train(i) = linearRegCostFunction(X,y, theta, 0);
  error_val(i) = linearRegCostFunction(Xval, yval, theta, 0);
end
% =========================================================================

lambda_vec 范围 [0 0.001 0.003 0.01 0.03 0.1 0.3 1 3 10]’;

在8阶多项式下，尝试不同的lambda，发现误差函数呈现 对号函数的趋势（近似Nike标志），lambda较小时，处于高偏差状态，lambda较大时，处于高方差状态。lambda近似在3处最佳

不妨将lambda取1、3,、10、100看看拟合曲线：




谢谢大家