# 梯度下降法及其Python实现

22362人阅读 评论(6)

SGD有一个缺点是，其噪音较BGD要多，使得SGD并不是每次迭代都向着整体最优化方向。而且SGD因为每次都是使用一个样本进行迭代，因此最终求得的最优解往往不是全局最优解，而只是局部最优解。但是大的整体的方向是向全局最优解的，最终的结果往往是在全局最优解附近。

============ 分割分割 =============

# _*_ coding: utf-8 _*_
# 作者: yhao
# 博客: http://blog.csdn.net/yhao2014
# 邮箱: yanhao07@sina.com

# 训练集
# 每个样本点有3个分量 (x0,x1,x2)
x = [(1, 0., 3), (1, 1., 3), (1, 2., 3), (1, 3., 2), (1, 4., 4)]
# y[i] 样本点对应的输出
y = [95.364, 97.217205, 75.195834, 60.105519, 49.342380]

# 迭代阀值，当两次迭代损失函数之差小于该阀值时停止迭代
epsilon = 0.0001

# 学习率
alpha = 0.01
diff = [0, 0]
max_itor = 1000
error1 = 0
error0 = 0
cnt = 0
m = len(x)

# 初始化参数
theta0 = 0
theta1 = 0
theta2 = 0

while True:
cnt += 1

# 参数迭代计算
for i in range(m):
# 拟合函数为 y = theta0 * x[0] + theta1 * x[1] +theta2 * x[2]
# 计算残差
diff[0] = (theta0 + theta1 * x[i][1] + theta2 * x[i][2]) - y[i]

# 梯度 = diff[0] * x[i][j]
theta0 -= alpha * diff[0] * x[i][0]
theta1 -= alpha * diff[0] * x[i][1]
theta2 -= alpha * diff[0] * x[i][2]

# 计算损失函数
error1 = 0
for lp in range(len(x)):
error1 += (y[lp]-(theta0 + theta1 * x[lp][1] + theta2 * x[lp][2]))**2/2

if abs(error1-error0) < epsilon:
break
else:
error0 = error1

print ' theta0 : %f, theta1 : %f, theta2 : %f, error1 : %f' % (theta0, theta1, theta2, error1)
print 'Done: theta0 : %f, theta1 : %f, theta2 : %f' % (theta0, theta1, theta2)
print '迭代次数: %d' % cnt


 theta0 : 2.782632, theta1 : 3.207850, theta2 : 7.998823, error1 : 7.508687
theta0 : 4.254302, theta1 : 3.809652, theta2 : 11.972218, error1 : 813.550287
theta0 : 5.154766, theta1 : 3.351648, theta2 : 14.188535, error1 : 1686.507256
theta0 : 5.800348, theta1 : 2.489862, theta2 : 15.617995, error1 : 2086.492788
theta0 : 6.326710, theta1 : 1.500854, theta2 : 16.676947, error1 : 2204.562407
theta0 : 6.792409, theta1 : 0.499552, theta2 : 17.545335, error1 : 2194.779569
theta0 : 74.892395, theta1 : -13.494257, theta2 : 8.587471, error1 : 87.700881
theta0 : 74.942294, theta1 : -13.493667, theta2 : 8.571632, error1 : 87.372640
theta0 : 74.992087, theta1 : -13.493079, theta2 : 8.555828, error1 : 87.045719
theta0 : 75.041771, theta1 : -13.492491, theta2 : 8.540057, error1 : 86.720115
theta0 : 75.091349, theta1 : -13.491905, theta2 : 8.524321, error1 : 86.395820
theta0 : 75.140820, theta1 : -13.491320, theta2 : 8.508618, error1 : 86.072830
theta0 : 75.190184, theta1 : -13.490736, theta2 : 8.492950, error1 : 85.751139
theta0 : 75.239442, theta1 : -13.490154, theta2 : 8.477315, error1 : 85.430741
theta0 : 97.986390, theta1 : -13.221172, theta2 : 1.257259, error1 : 1.553781
theta0 : 97.986505, theta1 : -13.221170, theta2 : 1.257223, error1 : 1.553680
theta0 : 97.986620, theta1 : -13.221169, theta2 : 1.257186, error1 : 1.553579
theta0 : 97.986735, theta1 : -13.221167, theta2 : 1.257150, error1 : 1.553479
theta0 : 97.986849, theta1 : -13.221166, theta2 : 1.257113, error1 : 1.553379
theta0 : 97.986963, theta1 : -13.221165, theta2 : 1.257077, error1 : 1.553278
Done: theta0 : 97.987078, theta1 : -13.221163, theta2 : 1.257041

随机梯度下降法

python实现梯度下降算法


1
5

个人资料
• 访问：262927次
• 积分：2514
• 等级：
• 排名：第17231名
• 原创：41篇
• 转载：9篇
• 译文：3篇
• 评论：66条
最新评论
custom
Welcome to yhao's blog~