梯度下降、反向传播

日萌社

人工智能AI：Keras PyTorch MXNet TensorFlow PaddlePaddle 深度学习实战（不定时更新）

---

nltk 处理文本

注意力机制 SENet、CBAM

卷积神经网络 处理文本：word2vec、TF-IDF、TextRank、字符卷积、词卷积、卷积神经网络文本分类模型的实现(Conv1D一维卷积、Conv2D二维卷积)

反向传播、链式求导

梯度下降

最小二乘法(LS算法)：实际为L2范数的一个具体应用(计算残差平方和)

线性回归 例子

---

【程序3-2】
x = [(2, 0, 3), (1, 0, 3), (1, 1, 3), (1,4, 2), (1, 2, 4)]
y = [5, 6, 8, 10, 11]
epsilon = 0.002 #噪音值
alpha = 0.02 #学习率
diff = [0, 0]
max_itor = 1000
error0 = 0
error1 = 0
cnt = 0
m = len(x)
theta0 = 0 #θ权重参数
theta1 = 0
theta2 = 0

while True:
    cnt += 1
    for i in range(m):
        diff[0] = (theta0 * x[i][0] + theta1 * x[i][1] + theta2 * x[i][2]) - y[i]
        theta0 -= alpha * diff[0] * x[i][0]
        theta1 -= alpha * diff[0] * x[i][1]
        theta2 -= alpha * diff[0] * x[i][2]
    error1 = 0
    for lp in range(len(x)):
        error1 += (y[lp] - (theta0 * x[lp][0] + theta1 * x[lp][1] + theta2 * x[lp][2])) ** 2 / 2
    if abs(error1 - error0) < epsilon: 
        break
    else:
        error0 = error1

#theta0 : -0.510496, theta1 : 1.542565, theta2 : 2.140431, error0 : 0.246543, error1 : 0.244565
print('theta0 : %f, theta1 : %f, theta2 : %f, error0 : %f, error1 : %f' % (theta0, theta1, theta2, error0, error1))
#Done: theta0 : -0.510496, theta1 : 1.542565, theta2 : 2.140431
print('Done: theta0 : %f, theta1 : %f, theta2 : %f' % (theta0, theta1, theta2)) 
print('迭代次数: %d' % cnt) #迭代次数: 77

#最终比较计算的值[5.400301,5.910797,7.453362,9.940628,11.136359] 和 y的值[5, 6, 8, 10, 11] 之间的相近性
for i in range(m):
    result = theta0 * x[i][0] + theta1 * x[i][1] + theta2 * x[i][2]
    print('%f' % result)
#5.400301
#5.910797
#7.453362
#9.940628
#11.136359