机器学习之线性回归:算法兑现为python代码
步骤
- 采取线性回归模型
- 求出代价函数(cost),求出梯度(gradient),求偏导
- 设定一个学习率迭代参数
- 当前的代价函数与当前的代价函数的差小于阈值,计算结束
参数定义
- model: 线性回归模型
- cost: 代价函数
- gradient: 梯度公式
- theta_update: 参数更新公式
- stop_stratege: 迭代停止策略,代价函数小于阈值法
代码参考(python3)
import numpy as np
def model(theta, X):
return X.dot(theta)
def cost(m, theta, X, y):
ele = y - model(theta, X)
item = ele ** 2
item_sum = np.sum(item)
return item_sum / 2 / m
def gradient(m, theta, X, y, cols):
grad_theta = []
for j in range(cols):
grad = (y - model(theta, X)).dot(X[:, j])
grad_sum = np.sum(grad)
grad_theta.append(- grad_sum / m)
return np.array(grad_theta)
def theta_update(grad_theta, theta, sigma):
return theta - sigma * grad_theta
def stop_stratege(cost, cost_update, threshold):
return cost - cost_update < threshold
def OLS(X, y, threshold):
m
theta = np.array()
iters = 0
cost_record = []
sigma = 0.0001
cost_val = cost(m, theta, X, y)
while True:
grad = gradient(m, theta, X, y)
theta = theta_update(grad, theta, sigma)
cost_update = cost(m, theta, X, y)
if stop_stratege(cost_val, cost_update, threshold):
break
iters += 1
cost_val = cost_update
cost_record.append(cost_val)
print("OLS convergence duration: %f s" % (end - start))
return cost_record, iters, theta