machine_learning

Machine Learning

1. Linear Regression with One Variable

1. Basic theory

1. Objective Function(Hypothesis)

   ​ $h_{\theta }(x)={\theta }_0+{\theta }_{1}x$

2. Parameters

   ​ ${\theta }_0,{\theta }_1$

3. Cost/Loss Function

   $J({\theta }_0,{\theta }_1)=\frac{1}{2m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)}{)}^2$ (square error function)

4. Goal

​ $minimiz{e}_{{\theta }_0,{\theta }_1}J({\theta }_0,{\theta }_1)$

2. Gradient descent algorithm

​ $repeatuntilconvergence$ { \{ {

​ $${\theta }_j:={\theta }_j-\alpha \frac{\partial }{\partial {\theta }_j}J({\theta }_0,{\theta }_1)$$ $forj=0andj=1$

​ $\}$

$where,\alpha isthelearningrate$

Simultaneous update
$$temp0:={\theta }_0-\alpha \frac{\partial }{\partial {\theta }_0}J({\theta }_0,{\theta }_1)$$

$$temp1:={\theta }_1-\alpha \frac{\partial }{\partial {\theta }_j}J({\theta }_0,{\theta }_1)$$

$${\theta }_0:=temp0$$

$$\theta 1:=temp1$$

r e p e a t u n t i l c o n v e r g e n c e { repeat\quad until\quad convergence\{ repeatuntilconvergence{

​ $${\theta }_0:={\theta }_0-\alpha \frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)})$$

​ KaTeX parse error: Expected group after '_' at position 41: …\frac{1}{m}\sum_̲\limits{i=1} ^m…

$$\begin{gathered} \} \\ updata{\theta }_{0}and{\theta }_{1}simultaneously \end{gathered}$$

2.Linear Regression with Multiple Variables

1. Basic theory

		$h_\theta(x) = \theta_0+\theta_1x_1+\theta_2x_2+\cdot\cdot\cdot+\theta_nx_n$

​ For convenience of notation, define$x_0=1(x_0^{(i)}=1)$.

​ Let

​ $fecturevector\mathbf{x}=\left[\begin{array}{c}{x}_0\\ x_1\\ x_2\\ \cdot \\ \cdot \\ \cdot \\ x_n\end{array}\right]\in {\mathbb{R}}^{n+1}parametervector\mathbf{\theta }=\left[\begin{array}{c}{\theta }_0\\ {\theta }_1\\ {\theta }_2\\ \cdot \\ \cdot \\ \cdot \\ {\theta }_n\end{array}\right]\in {\mathbb{R}}^{n+1}$

​ So,
$$h_{\theta }(x)={\theta }_0{x}_0+{\theta }_1{x}_1+{\theta }_2{x}_2+\cdot \cdot \cdot +{\theta }_n{x}_n={\mathbf{\theta }}^T\mathbf{x}$$

Support Vector Machine

svm 应用实例

1. 简单例子

2. 划分超平面

#sklearn划分超平面
print(__doc__)

import numpy as np
import pylab as pl #绘图功能
from sklearn import svm

#创建 40 个点
np.random.seed(0)#让每次运行程序生成的随机样本点不变
#生成训练实例并保证是线性可分的
#np.r_表示将矩阵在行方向上进行相连
#random.randn(a,b) 表示生成a行b列的矩阵，且随机数服从标准正态分布
#array（20,2）-[2,2] 相当于给每一行的两个数都减去2
X = np.r_[np.random.randn(20,2) - [2,2],np.random.randn(20,2)+[2,2]]
# 两个类别 每类有 20 个点， Y 为 40 行 1 列的列向量
Y = [0]*20+[1]*20

#建立svm模型
clf = svm.SVC(kernel = "linear")
clf.fit(X,Y)

#获得划分超平面
#划分超平面原方程：w0x0+w1x1+b = 0
#将其转化为点斜式方程，并把 x0看作 x, x1看作 y, b看作w2
#点斜式：y = -(w0/w1)x-(w2/w1)
w = clf.coef_[0]# w 是一个二维数据，coef就是w = [w0,w1]
a = -w[0]/w[1] # 斜率
xx = np.linspace(-5,5) #从 -5 到5 产生一些连续的值（随机的）
yy = a * xx-(clf.intercept_[0])/w[1] #带入 x 的值，获得直线方程

#画出和划分超平面平行且经过支持向量的两条线（斜率相同，截距不同）
b = clf.support_vectors_[0] # 取出第一个支持向量点
yy_down = a *xx +(b[1] - a*b[0])
b = clf.support_vectors_[-1]# 取出最后一个支持向量点
yy_up = a*xx +(b[1] - a*b[0])
#注意，b是确定的
#print(clf.support_vectors_)

#查看相关的参数值
print("w:",w)
print("a:",a)
print("support_vectors_:",clf.support_vectors_)
print("clf.coef_:",clf.coef_)
print("X:",X)
print("Y:",Y)
#在scikit-learn中,coef_保存了线性模型中划分超平面的参数向量
#形式为（n_classes,n_features）.若n_classes>1，则为多分类问题
#(1,n_features)为二分类。

#绘制划分超平面，边际平面和样本点
pl.plot(xx,yy,'k-')
pl.plot(xx,yy_down,'k--')
pl.plot(xx,yy_up,'k--')
# 圈出支持向量
pl.scatter(clf.support_vectors_[:,0],clf.support_vectors_[:,1],
           s = 80, facecolors = "none")
pl.scatter(X[:,0],X[:,1],c = Y, cmap = pl.cm.Paired)
pl.axis("tight")
pl.show()

``