学习笔记(Study Note)
线性回归模型(Linear Regression Model)
原理(theory)
线性回归的主要功能为通过对训练集上的数据进行函数拟合,从而得到一个可预测的函数(即模型)
The main function of Linear Regression is to obtain a predction function by fiting the data on training set.
损失函数(Loss Function)
J ( ω , b ) = 1 2 m ∑ i = 1 m ( f w , b ( x ( i ) ) − y ( i ) ) 2 m a s n u m b e r o f t r a i n i n g s e t J(\omega,b)= \frac{1}{2m}\sum_{i=1}^{m}(f_{w,b}(x^{(i)})-y^{(i)})^2 \\ \quad m\ as\ number\ of\ training\ set J(ω,b)=2m1i=1∑m(fw,b(x(i))−y(i))2m as number of training set
梯度下降算法(Gradient Descent)
梯度下降算法用于得到损失函数的局部最小值。
Gradient descent algorithm is used to obtain the local minimum of loss function.
代码实现(Code Implementation)
该代码还存在很多缺陷,如:对损失函数高阶求导无法计算等,等以后更加熟悉了再回来修改。
The code also has many flaws, such as can’t computer higher derivative of loss funciton.Come back to it later when I am more familiar with it.
import torch
import numpy as np
import matplotlib.pyplot as plt
from CommonFunction import GadientDescent as GD
from CommonFunction import fun
x = np.arange(0, 20)
y = np.arange(150,170)
x = fun.scal(x, x.max(), x.min(), x.sum()/x.size)
y = fun.scal(y, y.max(), y.min(), y.sum()/y.size)
plt.figure('1')
plt.title('training set')
plt.xlabel('x')
plt.ylabel('y')
plt.scatter(x, y, color='red')
# set w and b
w = np.arange(0,1)
b = 0
a = 0.01
# set loss function and iterNum
m = x.size
iterNum = 10000
for index in range(iterNum):
tw, tb = GD.GD(w, b, a, x, y, m)
w = tw
b = tb
print('w=', w, 'b=', b)
xx = np.arange(-1, 2)
yy = 0
for i in range(len(w)):
yy += w[i] * np.power(xx, i+1) + b
plt.subplot
plt.xlim(-1, 1)
plt.ylim(-1, 1)
plt.plot(xx, yy)
plt.show()
from CommonFunction import fun
def GD(w, b, a, x, y, m):
tempSumW = 0
tempSumB = 0
for i in range(m):
tempSumW += (fun.fun(w, b, x[i]) - y[i]) * x[i]
tempSumB += (fun.fun(w, b, x[i]) - y[i])
tempW = w - a * (1 / m)*tempSumW
tempB = b - a * (1 / m)*tempSumB
return tempW, tempB
import numpy as np
def fun(w, b, x):
summ = 0
for i in range(len(w)):
summ += w[i] * np.power(x, i+1) + b
return summ
def scal(x, xmax, xmin, xave):
Xnew = (x - xave) / (xmax - xmin)
return Xnew
Iteration = 100
Iteration = 1000
Iteration = 10000