【梯度下降算法】GD与SGD_参考自B站刘二大人视频中代码+结果

#梯度下降，Gradient Descent
代码部分：

`import matplotlib.pyplot as plt

x_data = [1.0,2.0,3.0]
y_data = [2.0,4.0,6.0]

w = 1.0

def forward(x):
return x*w

def cost(xs,ys):
cost = 0
for x,y in zip(xs,ys):
y_pred = forward(x)
cost += (y_pred-y)**2
return cost/len(xs)

def gradient(xs,ys):
grad=0
for x,y in zip(xs,ys):
grad += 2x(x*w-y)
return grad/len(xs)

cost_list=[]
epoch_list=[]
print(‘Predict (before trainning)’, 4, forward(4))
for epoch in range(1000):#将刘老师的100改为了1000
cost_val = cost(x_data,y_data)
grad_val = gradient(x_data,y_data)
w -= 0.01*grad_val
print(“Epoch:”,epoch, “w=”,w, “loss=”,cost_val)
cost_list.append(cost_val)
epoch_list.append(epoch)

print(“Predict(after training)”,4, forward(4))
plt.plot(epoch_list, cost_list)
plt.ylabel(‘loss’)
plt.xlabel(‘epoch’)
plt.show()
plt.grid()
plt.savefig(‘/home/202301/liu_er_gd.jpg’, bbox_inches=‘tight’)`

GD的结果：

#随机梯度下降，SGD
代码部分：
、、、
import matplotlib.pyplot as plt

x_data = [1.0,2.0,3.0]
y_data = [2.0,4.0,6.0]

w = 1.0

def forward(x):
return x*w

def loss(x,y):
y_pred = forward(x)
return (y_pred-y)**2

def gradient(x,y):
return 2x(x*w-y)

cost_list=[]
epoch_list=[]
print(‘Predict (before trainning)’, 4, forward(4))
for epoch in range(100):
for x,y in zip(x_data,y_data):
grad =gradient(x,y)
w -= 0.01*grad
print(“\tgrad:”,x,y,grad)
l=loss(x,y)
cost_list.append(l)
epoch_list.append(epoch)

print(“Predict(after training)”,4, forward(4))
plt.plot(epoch_list, cost_list)
plt.ylabel(‘loss’)
plt.xlabel(‘epoch’)
plt.show()
plt.grid()
plt.savefig(‘/home/202301/liu_er_sgd.jpg’, bbox_inches=‘tight’)

SGD的结果：

#总结
由结果来看，SGD的结果收敛速度比GD快了不止一点点呀！