这一节主要是使用softmax实现一个手写数字的识别器,难点主要是在代价函数和梯度的矢量化写法。
STEP 2: Implement softmaxCost
function [cost, grad] = softmaxCost(theta, numClasses, inputSize, lambda, data, labels)
% numClasses - the number of classes
% inputSize - the size N of the input vector
% lambda - weight decay parameter
% data - the N x M input matrix, where each column data(:, i) corresponds to
% a single test set
% labels - an M x 1 matrix containing the labels corresponding for the input data
%
% Unroll the parameters from theta
theta = reshape(theta, numClasses, inputSize);
numCases = size(data, 2);
groundTruth = full(sparse(labels, 1:numCases, 1));
cost = 0;
thetagrad = zeros(numClasses, inputSize);
%% ---------- YOUR CODE HERE --------------------------------------
% Instructions: Compute the cost and gradient for softmax regression.
% You need to compute thetagrad and cost.
% The groundTruth matrix might come in handy.
m = theta * data;
m = bsxfun(@minus,m,max(m,[],1));
m = exp(m);
m = bsxfun(@rdivide, m, sum(m));
cost = -sum(sum(groundTruth .* log(m)))/size(data,2) + lambda/2*sum(sum(theta.^2));
thetagrad = -(groundTruth - m)*data'/size(data,2) + lambda*theta;
% ------------------------------------------------------------------
% Unroll the gradient matrices into a vector for minFunc
grad = [thetagrad(:)];
end
其中groundTruth的大小为numClasses * numCases的矩阵,groundTruth横坐标表示的是类别,纵坐标表示的是第几个样本,如果第j样本属于i类,那么groundTruth[i][j]=1,其他groundTruth[x][j]=0(x!=i)
则根据公式
容易知道
groundTruth .* log(m)就等价于
至于thetagrad,根据公式
-(groundTruth - m)*data'得到了一个新的矩阵,它的大小跟theta是一样,它的第j行就等价于下面的公式,因为groundTruth - m的size为numClasses*numCases,data的size为inputSize*numCases,所以(groundTruth - m)*data'就有一个将所有样本累加的作用(想象一下两个矩阵相乘是怎么样的就容易明白了)
STEP 3: Gradient checking
运行它提供的代码即可(DEBUG设为true),下面是我测试的结果
STEP 4&5 :Learning parameters && Testing
softmaxPredict.m
function [pred] = softmaxPredict(softmaxModel, data)
% softmaxModel - model trained using softmaxTrain
% data - the N x M input matrix, where each column data(:, i) corresponds to
% a single test set
%
% Your code should produce the prediction matrix
% pred, where pred(i) is argmax_c P(y(c) | x(i)).
% Unroll the parameters from theta
theta = softmaxModel.optTheta; % this provides a numClasses x inputSize matrix
pred = zeros(1, size(data, 2));
%% ---------- YOUR CODE HERE --------------------------------------
% Instructions: Compute pred using theta assuming that the labels start
% from 1.
m = theta * data;
[~,pred] = max(m);
% ---------------------------------------------------------------------
end
接下来运行它提供的训练和测试代码即可,记得要把DEBUG设为false
结果如下