最小二乘法拟合曲线
用目标函数y=sin2πx, 加上一个正态分布的噪音干扰,用多项式去拟合【例1.1 11页】
import numpy as np
import scipy as sp
from scipy.optimize import leastsq
import matplotlib.pyplot as plt
%matplotlib inline
# 目标函数
def real_func(x):
return np.sin(2 * np.pi * x)
# 多项式
def fit_func(p, x):
# 输入一个列表,返回以这个列表中的值为参数的多项式
f = np.poly1d(p)
return f(x)
# 残差
def residuals_func(p, x, y):
ret = fit_func(p, x) - y
return ret
# 十个点
x = np.linspace(0, 1, 10)
x_points = np.linspace(0, 1, 1000)
# 加上正态分布噪音的目标函数的值
y_ = real_func(x)
y = [np.random.normal(0, 0.1) + y1 for y1 in y_]
def fitting(M=0):
"""
M 为 多项式的次数
"""
# 随机初始化多项式参数
p_init = np.random.rand(M + 1)
# 最小二乘法
p_lsq = leastsq(residuals_func, p_init, args=(x, y))
print('Fitting Parameters:', p_lsq[0])
# 可视化
plt.plot(x_points, real_func(x_points), label='real')
plt.plot(x_points, fit_func(p_lsq[0], x_points), label='fitted curve')
plt.plot(x, y, 'bo', label='noise')
plt.legend()
return p_lsq
# m=0
p_lsp_q = fitting(M=0)
# m=3
p_lsq_3 = fitting(M=3)
# m=9
p_lsq_9 = fitting(M=9)
当M=9时,多项式曲线通过了每个数据点,但是造成了过拟合,引入正则化,降低过拟合
Q
(
x
)
=
∑
i
=
1
n
(
h
(
x
i
)
−
y
i
)
2
+
λ
∥
w
∥
∘
2
Q(x)=\sum_{i=1}^{n}\left(h\left(x_{i}\right)-y_{i}\right)^{2}+\lambda\|w\|_{\circ}^{2}
Q(x)=i=1∑n(h(xi)−yi)2+λ∥w∥∘2
回归问题中,损失函数是平方损失,正则化可以是参数向量的L2范数,也可以是L1范数。
L1: regularization*abs( p )
L2: 0.5 * regularization * np.square( p )
regularization = 0.0001
def residuals_func_regularization(p, x, y):
ret = fit_func(p, x) - y
ret = np.append(ret, np.sqrt(0.5 * regularization * np.square(p))) # L2范数作为正则化项
return ret
# 最小二乘法,加正则化项
p_init = np.random.rand(9 + 1)
p_lsq_regularization = leastsq(
residuals_func_regularization, p_init, args=(x, y))
plt.plot(x_points, real_func(x_points), label='real')
plt.plot(x_points, fit_func(p_lsq_9[0], x_points), label='fitted curve')
plt.plot(
x_points,
fit_func(p_lsq_regularization[0], x_points),
label='regularization')
plt.plot(x, y, 'bo', label='noise')
plt.legend()