Advice for Applying Machine Learning
linearRegCostFunction.m
% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost and gradient of regularized linear
% regression for a particular choice of theta.
%
% You should set J to the cost and grad to the gradient.
%
h = X * theta;
theta2 = [0 ; theta(2:size(theta), :)];
J = 1 / (2 * m) * sum((h - y) .^ 2) + lambda / (2 * m) * sum(theta2 .^ 2);
grad = ( X' * (h - y) + lambda * theta2 ) / m;
learningCurve.m
for i=1:m
% train parameter: theta
theta=trainLinearReg(X(1:i,:),y(1:i), lambda);
% Compute train errors using training examples
[error_train(i),grad]=linearRegCostFunction(X(1:i,:), y(1:i), theta, 0);
% cross-validation error
[error_val(i), grad]=linearRegCostFunction(Xval, yval, theta, 0);
end
polyFeatures.m
% ====================== YOUR CODE HERE ======================
% Instructions: Given a vector X, return a matrix X_poly where the p-th
% column of X contains the values of X to the p-th power.
%
%
for i =1:p
X_poly(:,i) = X(:,1).^i;
end
validationCurve.m
for i=1:length(lambda_vec)
lambda = lambda_vec(i);
theta = trainLinearReg(X,y, lambda);
% Compute train / val errors when training linear
[error_train(i), grad] = linearRegCostFunction(X, y, theta, 0);
[error_val(i), grad] = linearRegCostFunction(Xval, yval, theta, 0);
end
(转载)machine learning week6 确诊机器学习算法的性能 各种学习曲线 来判断学习算法是过拟合或欠拟合
%%-------------------------------------------------------------------------
%%Optional (ungraded) exercise: Computing test set
lambda=3;
theta=trainLinearReg(X_poly,y, lambda);
[error_test,grad]=linearRegCostFunction(X_poly_test, ytest, theta, 0);
fprintf(' %f\n',error_test);
%%-------------------------------------------------------------------------
%%Optional (ungraded) exercise: Plotting learning curves with randomly selected examples
% m=size(X_poly,1);
% X_poly_y=[X_poly,y];
% X_poly_val_y=[X_poly_val,yval];
% lambda=0.01;
% error_train = zeros(m, 1);
% error_val = zeros(m, 1);
% for i=1:m
% error_train_sum=0;
% error_val_sum =0;
% for k=1:20 %50次迭代
% rand_seq=round(rand(1,i)*(m-1))+1;%生成i个随机序列 0~m
% rand_X_poly_y=X_poly_y(rand_seq,:);
% rand_X_poly_val_y=X_poly_val_y(rand_seq,:);
% X=rand_X_poly_y(:,1:end-1);
% y=rand_X_poly_y(:,end);
% Xval=rand_X_poly_val_y(:,1:end-1);
% yval=rand_X_poly_val_y(:,end);
% theta=trainLinearReg(X,y,lambda);
% [error_train_val,grad]=linearRegCostFunction(X, y, theta, 0);
% [error_val_val, grad]=linearRegCostFunction(Xval, yval, theta, 0);
% error_train_sum=error_train_sum+error_train_val;
% error_val_sum=error_val_sum+error_val_val;
% end
% error_train(i)=error_train_sum/20;
% error_val(i)=error_val_sum/20;
% end
% plot(1:m, error_train, 1:m, error_val);
% title(sprintf('Polynomial Regression Learning Curve (lambda = %f)', lambda));
% xlabel('Number of training examples')
% ylabel('Error')
% axis([0 13 0 100])
% legend('Train', 'Cross Validation')
% fprintf('Polynomial Regression (lambda = %f)\n\n', lambda);
% fprintf('# Training Examples\tTrain Error\tCross Validation Error\n');
%%-------------------------------------------------------------------------
Machine Learning System Design
- precision = ture positive / #predicted positive
recall = true positive / # actual positive -
- Suppose we want to predict y=1 (cancer) only if very confident
Threshold: higher value
Higher precision, lower recall - avoid missing too many cases of cancer (avoid false negtives)
Threshold: lower value
Higher recall, lower precision
- Suppose we want to predict y=1 (cancer) only if very confident
F1 Score: 2PR/(P+R)
Measure presicion (P) and recall(R) on the cross validation set and choose the value of threshold which maximizes 2PR/(P+R)Large data rationale:
Use a learning algorithm with many parameters: low bias
=> Jtrain(theta) will be small
Use a very large training set(unlikely to overfit): low variance
=> Jtrain(theta) ≈ Jtest(theta)
So Jtest(theta) will be small
Q3错了。。。