这一节内容关于logistic regression。名称叫regression,但其实是一个分类算法。logistics regression就是在线性回归的外面套了一个sigmoid函数(也会选择tanh函数)将结果转化到区间[0,1]上([-1,1]对应tanh函数)。
然后关于logistic regression内容比较简单,原理就不写了,主要针对一道题目编写一段代码。
问题就是对周志华的《机器学习》P89页数据用logistic regression进行分类。这一节用SGD方法分类,下一节用牛顿法分类。代码如下(数据在代码里):
clear all;
close all;
clc;
%%input the data;
density = [0.697, 0.774, 0.634, 0.608, 0.556, 0.403, 0.481, 0.437, 0.666, 0.243, 0.245, 0.343, 0.639, 0.657, 0.360, 0.593, 0.719];
density = density';
sugar = [0.460, 0.376, 0.264, 0.318, 0.215, 0.237, 0.149, 0.211, 0.091, 0.267, 0.057, 0.099, 0.161, 0.198, 0.370, 0.042, 0.103];
sugar = sugar';
y = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0];
y = y';
x = [density sugar];
%坐标轴上显示数据
pos = find(y' == 1);
neg = find(y' == 0);
figure,
plot(x(pos,1),x(pos,2),'o');
hold on
plot(x(neg,1),x(neg,2),'*');
hold on
xlabel('density'),ylabel('sugar');
[m,n] = size(x);
x=[ones(m,1),x];
%inatialize the data
theta = zeros(n+1,1);
%training
MaxIter = 18500;
lr = 0.007;%learning rate
J = zeros(MaxIter,1);
for i = 1:MaxIter
hx = sigmoid(x*theta);%hypothesis
grad = (x'*(y-hx));
theta = theta+lr.*grad;
J(i) = 1./m*sum(y.*log(hx)+(1-y).*(1-log(1-hx)));
end
%plot the line
hold on
plot_x=[min(x(:,2)),max(x(:,2))];
plot_y=((-1./theta(3))*(theta(1)+theta(2).*plot_x));
plot(plot_x,plot_y);
% Plot J(损失函数J随着迭代次数的变化)
figure
plot(0:MaxIter-1, J, 'o--', 'MarkerFaceColor', 'r', 'MarkerSize', 4)
xlabel('Iteration'); ylabel('J')
最后,附一下结果: