李宏毅机器学习2

Regression

推荐系统也可以认为是regression问题，
f(使用者A 商品B )=购买可能性
预测宝可梦的cp值：
找一个function，input是某一只宝可梦xcp，output是宝可梦的cp值y
步骤一：定义function :假设y= b +wxcp*

步骤2：评估模型好快

training data: 10 pokemons


loss 是衡量一组 w b的好坏。
3 选择最好的一组function:将 w, b 带入loss function ，最小的loss 对应的w,b 是最好的


梯度下降法：
1 随机选取一个初始点
2 计算w=w0时loss function 的微分，若切线斜率为负，则最小值应该出现在左边，应该增加w的值；反之减小w的值。踏一步更新的幅度和此时微分大小有关，此时的微分再乘以一个常数n,学习率。越陡峭的地方下降越快，越平坦的地方下降越慢。regression 没有local optimal
3循环计算
如何防止过拟合：regularization,

import numpy as np
x_data = [338, 333, 328, 207, 226, 25, 179, 60, 28, 66]
y_data = [640, 633, 619, 393, 428, 27, 193, 66, 226, 1591]
x = np.arange(-200, -100, 1)
y = np.arange(-5, 5, 0.1)
Z = np.zeros((len(x), len(y)))
X, Y = np.meshgrid(x, y)
for i in range(len(x)):
    for j in range(len(y)):
        b = x[i]
        w = y[j]
        Z[j][i] = 0
        for n in range(len(x_data)):
            Z[j][i] = Z[j][i]+(y_data[n]-b-w*x_data[n])**2
        Z[j][i]=Z[j][i]/len(x_data)

import matplotlib.pyplot as plt
import numpy as np
x_data = [338, 333, 328, 207, 226, 25, 179, 60, 28, 66]
y_data = [640, 633, 619, 393, 428, 27, 193, 66, 226, 1591]
x = np.arange(-200, -100, 1)
y = np.arange(-5, 5, 0.1)
Z = np.zeros((len(x), len(y)))
X, Y = np.meshgrid(x, y)
for i in range(len(x)):
    for j in range(len(y)):
        b = x[i]
        w = y[j]
        Z[j][i] = 0
        for n in range(len(x_data)):
            Z[j][i] = Z[j][i]+(y_data[n]-b-w*x_data[n])**2
        Z[j][i] = Z[j][i]/len(x_data)

b = -120
w = -4
lr =0.0000001
iteration = 100000
b_history = [b]
w_history = [w]
for i in range(iteration):
   b_grad=0.0
   w_grad=0.0
   for n in range(len(x_data)):
       b_grad=b_grad-2.0*(y_data[n]-b-w*x_data[n])*1.0
       w_grad=w_grad-2.0*(y_data[n]-b-w*x_data[n])*x_data[n]
   b = b - lr*b_grad
   w = w - lr*w_grad
   b_history.append(b)
   w_history.append(w)
plt.contourf(x, y, Z, 50, alpha=0.5, cmap=plt.get_cmap("jet"))
plt.plot([-188.4], [2.67], "x", ms=12, markeredgewidth=3, color="orange")
plt.plot(b_history, w_history, "o-", ms=3, lw=1.5, color="black")
plt.xlim(-200, -100)
plt.ylim(-5, 5)
plt.xlabel(r"$b$", fontsize=16)
plt.ylabel(r"$b$", fontsize=16)
plt.show()