训练数据:
A = [3 4 5 6 1 10 11 13 15 16 17 18 118 120];
Y = [0 0 0 0 0 0 0 1 1 1 1 1 1 1];
明显地,是一个以12为界的一维分类结果
训练数据结果集合
y 属于{0,1}这个二元集合。
将最大似然表达式表示为:
max(L) = ∏(P(yi|xi)^yi * P(yi|xi) ^ (1 – yi))
求对数
l = ∑(yi * (P(yi|xi)) + (1 –yi) * P(yi|xi))
使用sigmoid函数进行近似
P(y|x) = o = 1 / (1 + exp(- theta * x))
theta * x 是一个多项式:z = theta0 + theta1 * x + theta2 * x^2 + ...
对o求thetai的偏导,可知:
d(o) = o * (1 – o) * x^i
若我们取0次多项式,则
o = 1 / (1 + exp(-theta0)
d(o) = 1 / (1 + exp(-theta0)
若取1次多项式,
o = 1 / (1 + exp(-(theta1 * x + theta0))
则对theta0取偏导:
d(o) = o * (1 – o)
对theta1取偏导:
d(o) = o * (1 – o) * x
对l取偏导,则可以获取梯度方向,以更新权值
theta0 = theta0 + alpha * ∑(yi – oi)
theta1 = theta1 + alpha * ∑(yi – oi) *xi
迭代多次,直到theta0 和 theta1都不大变化为止.
试验结果:
o =
Columns 1 through 10
0.0000 0.0000 0.0000 0.0000 0.0000 0.0267 0.1444 0.8643 0.9959 0.9993
Columns 11 through 14
0.9999 1.0000 1.0000 1.0000
theta = 1.8154
b = -21.7487
input 7
s =
1.1847e-04
说明该数据可以被2维的直线划分.
Matlab程序有:
for j = 1:1:1000sum = [0]
sumb = 0
for k = 1:1:length(A)
sum(k) = 0
end
for i = 1:1:length(A)
exp_list = b
for k = 1:1:size(A)
exp_list = exp_list + theta(k) * A(k,i)
end
for k = 1:1:size(A)
sum(k) = sum(k) + (Y(i) - 1 / (1 + exp(- exp_list))) * A(k,i);
end
sumb = sumb + (Y(i) - 1 / (1 + exp(- exp_list)));
end
for k = 1:1:size(A)
theta(k) = theta(k) + alpha * sum(k)
end
b = b + alpha * sumb;
end
o = [0]
for i = 1:1:length(A)
exp_list = b
for k = 1:1:size(A)
exp_list = exp_list + theta(k) * A(k,i)
end
o(i) = 1 / (1 + exp(- exp_list))
end
display(o);