感知机

感知机

超平面w⋅x+b=0

当 wx+b>0时 为+1类
当 wx+b<0时 为-1类

距离计算:

$\frac{1}{||w||_{_2}}|w\cdot x_{i}+b| \qquad \text{注:}||w||_2为L_2\text{范式}$

计算错分类的距离:

$$\begin{array}{llllll} &\text{错分时: } w\cdot x_{i} \text{ 与 }y_{i}\text{异号} & \\ &\text{此时可以去掉abs 得:} & -\frac{1}{||w||_{_2}}y_{i}(w\cdot x_{i}+b)>0 \\ &\text{忽略掉}\frac{1}{||w||_{_2}} \text{可得损失函数:}&L(w,b)=-\sum_{i=1}^{N}y_{i}(w\cdot x_{i}+b) \end{array}$$

---

优化:梯度▽下降法

$$\begin{array}{llllllll} &w&\text{的梯度:}&\frac{\partial}{\partial w} L(w,b)=&-\sum_{i=1}^{N}\ y_{i}\ x_{i} \\ &b&\text{的梯度:}&\frac{\partial}{\partial b} L(w,b)=&-\sum_{i=1}^{N} \ y_{i} \\ &&&&\text{i=1,2,3...N} \end{array}$$
$$\begin{array}{lclllllllll} \\ loop: &\\ &计算\triangledown_w,\triangledown_b\\ &w \ \ \gets &w-(-\alpha*\bigtriangledown_w)=w+\alpha*\bigtriangledown_w\\ &b\ \ \gets& b-(-\alpha*\bigtriangledown_b)=b+\alpha*\bigtriangledown_b\\ &&结束条件:1.\ 设定loop次数 \quad or 2.\ 设定前后两次梯度差值<阈值\epsilon\text{时停止}\\\\ \end{array}$$

代码简单实现

import numpy as np

#梯度下降法
def gradDesc(x,y,lr=0.001,loop=400,r=0): #lr=learning rate
    w=np.zeros(len(x[0])+1)
    x=np.column_stack([np.ones([len(x),1]),x])
    for i in range(loop):
        y_hat=x.dot(w)
        misclass=((y*y_hat)<=0)
        grad=np.dot(y[misclass],x[misclass])
        w+=lr*grad #--α▽  =>  +α·▽ 
    return w,misclass

if __name__=='__main__':
    x=np.array([[1,1],[1,0],[0,1],[2,3],[2,2],[3,3]])
    y=np.array([-1,-1,-1,1,1,1])
    w=gradDesc(x,y,lr=0.01)#learning rate 需要根据实际情况调整.
    print(w)
    #画图
    import matplotlib.pyplot as plt
    plt.scatter(x[:,0],x[:,1],c=y)
    y_plot=(-x[:,0]*w[0][1]-w[0][0])/w[0][2]
    plt.plot(x[:,0],y_plot,c='b')
    plt.show()