最小二乘
用于不知道
f
(
x
)
{f(x)}
f(x) 的时候,
[
a
,
b
]
{[a,b]}
[a,b] 只有一堆点。
x
1
∣
x
2
∣
x
3
∣
⋯
∣
x
n
∣
−
−
−
−
−
−
−
−
−
−
f
(
x
1
)
∣
f
(
x
2
)
∣
f
(
x
3
)
∣
⋯
∣
f
(
x
n
)
∣
\begin{array}{cccccc} x_1 &|& x_2 &|& x_3 &|& \cdots &|& x_n &| \\ -&-&-&-&-&-&-&-&-&- \\ f(x_1) &|& f(x_2) &|& f(x_3) &|& \cdots &|& f(x_n) &| \\ \end{array}
x1−f(x1)∣−∣x2−f(x2)∣−∣x3−f(x3)∣−∣⋯−⋯∣−∣xn−f(xn)∣−∣
以
ϕ
0
=
1
,
ϕ
1
=
x
,
ϕ
2
=
x
2
,
ϕ
3
=
x
3
,
⋯
\phi_0=1 ,\phi_1=x,\phi_2=x^2,\phi_3=x^3, \cdots
ϕ0=1,ϕ1=x,ϕ2=x2,ϕ3=x3,⋯
为例。定义
(
ϕ
0
,
ϕ
0
)
=
∑
i
=
1
n
ϕ
0
×
ϕ
0
=
∑
i
=
1
n
1
×
1
=
n
(
ϕ
0
,
ϕ
1
)
=
∑
i
=
1
n
ϕ
0
×
ϕ
1
=
∑
i
=
1
n
1
×
x
i
(
ϕ
1
,
ϕ
2
)
=
∑
i
=
1
n
x
i
×
x
i
2
(
ϕ
0
,
f
)
=
∑
i
=
1
n
1
×
f
(
x
i
)
(
ϕ
1
,
f
)
=
∑
i
=
1
n
x
i
×
f
(
x
i
)
⋯
\begin{align*} (\phi_0,\phi_0)=&\sum_{i=1}^{ n}\phi_0 \times \phi_0= \sum_{i=1}^{ n}1 \times 1=n\\ (\phi_0,\phi_1)=& \sum_{i=1}^{ n}\phi_0 \times \phi_1= \sum_{i=1}^{ n} 1 \times x_i\\ (\phi_1,\phi_2)=&\sum_{i=1}^{ n}x_i \times x_i^2\\ (\phi_0,f)=& \sum_{i=1}^{ n} 1 \times f(x_i)\\ (\phi_1,f)=& \sum_{i=1}^{ n} x_i \times f(x_i)\\ \cdots \end{align*}
(ϕ0,ϕ0)=(ϕ0,ϕ1)=(ϕ1,ϕ2)=(ϕ0,f)=(ϕ1,f)=⋯i=1∑nϕ0×ϕ0=i=1∑n1×1=ni=1∑nϕ0×ϕ1=i=1∑n1×xii=1∑nxi×xi2i=1∑n1×f(xi)i=1∑nxi×f(xi)
求内积要带权函数!
对应的法方程组:
[
(
ϕ
0
,
ϕ
0
)
(
ϕ
0
,
ϕ
1
)
(
ϕ
0
,
ϕ
2
)
(
ϕ
1
,
ϕ
0
)
(
ϕ
1
,
ϕ
1
)
(
ϕ
1
,
ϕ
2
)
(
ϕ
2
,
ϕ
0
)
(
ϕ
2
,
ϕ
1
)
(
ϕ
2
,
ϕ
2
)
]
[
a
b
c
]
=
[
(
ϕ
0
,
f
)
(
ϕ
1
,
f
)
(
ϕ
2
,
f
)
]
\begin{bmatrix} (\phi_0,\phi_0) & (\phi_0,\phi_1) & (\phi_0,\phi_2)\\ (\phi_1,\phi_0) & (\phi_1,\phi_1) & (\phi_1,\phi_2)\\ (\phi_2,\phi_0) & (\phi_2,\phi_1) & (\phi_2,\phi_2) \end{bmatrix} \begin{bmatrix} a\\b\\c \end{bmatrix}= \begin{bmatrix} (\phi_0,f)\\(\phi_1,f)\\(\phi_2,f) \end{bmatrix}
(ϕ0,ϕ0)(ϕ1,ϕ0)(ϕ2,ϕ0)(ϕ0,ϕ1)(ϕ1,ϕ1)(ϕ2,ϕ1)(ϕ0,ϕ2)(ϕ1,ϕ2)(ϕ2,ϕ2)
abc
=
(ϕ0,f)(ϕ1,f)(ϕ2,f)
则二次最小二乘法拟合函数为:
S
∗
(
x
)
=
a
ϕ
0
+
b
ϕ
1
+
c
ϕ
2
S^*(x)=a\phi_0+b\phi_1+c\phi_2
S∗(x)=aϕ0+bϕ1+cϕ2
均方误差为
∣
∣
δ
∣
∣
2
=
∑
i
=
1
n
ρ
(
x
i
)
(
f
(
x
i
)
−
S
∗
(
x
i
)
)
2
|| \delta ||_2= \sqrt{ \sum_{i=1}^{ n} \rho(x_i) (f(x_i)-S^*(x_i))^2}
∣∣δ∣∣2=i=1∑nρ(xi)(f(xi)−S∗(xi))2
最大质误差
∣
∣
δ
∣
∣
∞
=
max
∣
f
(
x
i
)
−
S
∗
(
x
i
)
∣
|| \delta ||_{\infty}=\max|f(x_i)-S^*(x_i)|
∣∣δ∣∣∞=max∣f(xi)−S∗(xi)∣
若拟合公式为
y
=
a
x
b
{y=ax^b}
y=axb ,取对数变成
y
′
=
a
′
+
b
x
′
{y'=a'+bx'}
y′=a′+bx′ 。
[!example]-
已知一组实验数据如下表:
x i ∣ ∣ 1 ∣ 2 ∣ 3 ∣ 4 ∣ 5 ∣ = = = = = = = = = = = = y i ∣ ∣ 4 ∣ 4.5 ∣ 6 ∣ 8 ∣ 8.5 ∣ − − − − − − − − − − − − ρ i ∣ ∣ 2 ∣ 1 ∣ 3 ∣ 1 ∣ 1 ∣ \begin{array}{cccccc} x_i &||& 1 &|& 2 &|& 3 &|& 4 &|& 5&| \\=&=&=&=&=&=&=&=&=&=&=&=\\ y_i &||& 4 &|& 4.5 &|& 6 &|& 8 &|& 8.5 &| \\ -&-&-&-&-&-&-&-&-&-&-&- \\ \rho_i &||& 2 &|& 1 &|& 3 &|& 1 &|& 1 &|\\ \end{array} xi=yi−ρi∣∣=∣∣−∣∣1=4−2∣=∣−∣2=4.5−1∣=∣−∣3=6−3∣=∣−∣4=8−1∣=∣−∣5=8.5−1∣=∣−∣
试用最小二乘法求线性拟合曲线 y = a + b x {y=a+bx} y=a+bx ,并求均方误差。
解:设拟合函数为
ϕ ( x ) = a + b x \phi(x)=a+bx ϕ(x)=a+bx
ϕ 0 ( x ) = 1 , ϕ 1 ( x ) = x \phi_0(x)=1 \,\,,\,\, \phi_1(x)=x ϕ0(x)=1,ϕ1(x)=x
引进向量
ϕ 0 = ( 1 , 1 , 1 , 1 , 1 ) T ϕ 1 = ( 1 , 2 , 3 , 4 , 5 ) T f = ( 4 , 4.5 , 6 , 8 , 8.5 ) T \begin{align*} \phi_0=&(1,1,1,1,1)^ \mathrm T \\ \phi_1=&(1,2,3,4,5)^ \mathrm T \\ f=&(4,4.5,6,8,8.5)^ \mathrm T \end{align*} ϕ0=ϕ1=f=(1,1,1,1,1)T(1,2,3,4,5)T(4,4.5,6,8,8.5)T
求内积,带上权函数
( ϕ 0 , ϕ 0 ) = 8 , ( ϕ 0 , ϕ 1 ) = 22 ( ϕ 1 , ϕ 0 ) = 22 , ( ϕ 1 , ϕ 1 ) = 74 ( f , ϕ 0 ) = 47 , ( f , ϕ 1 ) = 145.5 \begin{align*} (\phi_0,\phi_0)=8 \,\,,\,\, &(\phi_0,\phi_1)=22 \\ (\phi_1,\phi_0)=22 \,\,,\,\, &(\phi_1,\phi_1)=74 \\ (f,\phi_0)=47 \,\,,\,\, &(f,\phi_1)=145.5 \end{align*} (ϕ0,ϕ0)=8,(ϕ1,ϕ0)=22,(f,ϕ0)=47,(ϕ0,ϕ1)=22(ϕ1,ϕ1)=74(f,ϕ1)=145.5
[ 8 22 22 74 ] [ a b ] = [ 47 145.5 ] \begin{bmatrix} 8 & 22 \\ 22 & 74 \end{bmatrix} \begin{bmatrix} a\\ b \end{bmatrix}= \begin{bmatrix} 47\\145.5 \end{bmatrix} [8222274][ab]=[47145.5]
解得
{ a = 2.5648 b = 1.2037 \begin{cases} a=2.5648 \\ \\ b=1.2037 \end{cases} ⎩ ⎨ ⎧a=2.5648b=1.2037
所得线性拟合曲线为(保留四位小数)
y = 2.5648 + 1.2037 x y=2.5648+1.2037x y=2.5648+1.2037x
均方误差
∣ ∣ δ ∣ ∣ 2 = ∑ i = 0 m ρ ( x i ) ( ϕ ∗ ( x i ) − y i ) 2 || \delta ||_2= \sqrt{ \sum_{i=0}^{ m}\rho(x_i)(\phi^*(x_i)-y_i)^2 } ∣∣δ∣∣2=i=0∑mρ(xi)(ϕ∗(xi)−yi)2
matlab最小二乘逼近
matlab可以使用左除来求解最小二乘问题。
最佳一致逼近
就是构造一个 n {n} n 次逼近函数,使得函数与逼近函数的差在区间上有 n + 2 {n+2} n+2 个关于 0 {0} 0 的交错点,且这个误差函数在这几个交错点上的绝对值都相等,都为极值点。
一次最佳一致逼近
[!example]-
求函数 f ( x ) = x 3 {f(x)=x^3} f(x)=x3 在 [ 1 , 3 ] {[1,3]} [1,3] 上的一次最佳一致逼近多项式 P ( x ) {P(x)} P(x) 。
解:设
P 1 ( x ) = a 0 + a 1 x P_1(x)=a_0+a_1x P1(x)=a0+a1x
∵ f ′ ( x ) = 3 x 2 , f ′ ′ ( x ) = 6 x > 0 , x ∈ [ 1 , 3 ] \because f'(x)=3x^2,f''(x)=6x>0,x\in[1,3] ∵f′(x)=3x2,f′′(x)=6x>0,x∈[1,3]
∴ f ( x ) 与 P 1 ( x ) 有三个交错偏差点 \therefore f(x)与P_1(x)有三个交错偏差点 ∴f(x)与P1(x)有三个交错偏差点
x 0 = 1 , x 1 ∈ ( 1 , 3 ) , x 2 = 3 x_0=1 \,\,,\,\, x_1\in(1,3) \,\,,\,\, x_2=3 x0=1,x1∈(1,3),x2=3
由
{ f ( x 0 ) − P 1 ( x 0 ) = − [ f ( x 1 ) − P 1 ( x 1 ) ] = f ( x 2 ) − P 1 ( x 2 ) f ′ ( x 1 ) − P 1 ′ ( x 1 ) = 0 \begin{cases} f(x_0)-P_1(x_0)=-[f(x_1)-P_1(x_1)]=f(x_2)-P_1(x_2)\\\\ f'(x_1)-P_1'(x_1)=0 \end{cases} ⎩ ⎨ ⎧f(x0)−P1(x0)=−[f(x1)−P1(x1)]=f(x2)−P1(x2)f′(x1)−P1′(x1)=0
得
{ 1 3 − ( a 0 + a 1 ⋅ 1 ) = − [ x 1 3 − ( a 0 + a 1 x 1 ) ] = 3 3 − ( a 0 + 3 a 1 ) 3 x 1 2 − a 1 = 0 \begin{cases} 1^3-(a_0+a_1 \cdot 1)=-[x_1^3-(a_0+a_1x_1)]=3^3-(a_0+3a_1)\\\\ 3x_1^2-a_1=0 \end{cases} ⎩ ⎨ ⎧13−(a0+a1⋅1)=−[x13−(a0+a1x1)]=33−(a0+3a1)3x12−a1=0
解得
a 1 = 13 , x 1 = 13 3 , a 0 = − ( 6 + 13 3 13 3 ) a_1=13 \,\,,\,\, x_1= \sqrt{ \frac{13}{3}} \,\,,\,\, a_0=-(6+ \frac{13}{3} \sqrt{\frac{13}{3}}) a1=13,x1=313,a0=−(6+313313)
P 1 ( x ) = − ( 6 + 13 3 13 3 ) + 13 x ≈ − 15.0206 + 13 x P_1(x)=-(6+ \frac{13}{3} \sqrt{\frac{13}{3}})+13x \approx -15.0206+13x P1(x)=−(6+313313)+13x≈−15.0206+13x
二次最佳一致逼近
用低阶多项式来逼近高阶多项式
定理:
考虑区间
[
−
1
,
1
]
{[-1,1]}
[−1,1] 上的函数的最佳逼近,对于
n
{n}
n 次首一多项式
p
(
x
)
∈
H
n
{p(x)\in H_n}
p(x)∈Hn ,满足
min
p
(
x
)
∈
H
n
∣
∣
p
(
x
)
∣
∣
C
[
−
1
,
1
]
{\min_{p(x)\in H_n}||p(x)||_{C[-1,1]}}
minp(x)∈Hn∣∣p(x)∣∣C[−1,1] 的
p
(
x
)
{p(x)}
p(x) 是
n
{n}
n 次首一切比雪夫多项式
T
n
(
x
)
2
n
−
1
\frac{T_n(x)}{2^{n-1}}
2n−1Tn(x)
即误差函数是一个
n
{n}
n 次首一切比雪夫多项式。所以
n
−
1
{n-1}
n−1 次最佳一致逼近多项式
P
n
−
1
(
x
)
{P_{n-1}(x)}
Pn−1(x) 满足
f
(
x
)
−
P
n
−
1
(
x
)
a
n
=
T
n
(
x
)
2
n
−
1
\frac{f(x)-P_{n-1}(x)}{a_n}= \frac{T_n(x)}{2^{n-1}}
anf(x)−Pn−1(x)=2n−1Tn(x)
即:
P
n
−
1
(
x
)
=
f
(
x
)
−
a
n
2
n
−
1
T
n
(
x
)
P_{n-1}(x)=f(x)- \frac{a_n}{2^{n-1}}T_n(x)
Pn−1(x)=f(x)−2n−1anTn(x)
[!example]-
求 f ( x ) = x 3 + 2 x 2 {f(x)=x^3+2x^2} f(x)=x3+2x2 在区间 [ 2 , 4 ] {[2,4]} [2,4] 上的二次最佳一致逼近函数,并估计误差。
解:
x ∈ [ 2 , 4 ] , 令 x = 2 + 4 2 + 4 − 2 2 t = 3 + t , t ∈ [ − 1 , 1 ] x\in[2,4]\,\,,\,\, 令x= \frac{2+4}{2}+ \frac{4-2}{2}t=3+t \,\,,\,\, t\in[-1,1] x∈[2,4],令x=22+4+24−2t=3+t,t∈[−1,1]
有
g ( t ) = ( 3 + t ) 3 + 2 ( 3 + t ) 2 g(t)=(3+t)^3+2(3+t)^2 g(t)=(3+t)3+2(3+t)2
切比雪夫多项式
T 3 = 4 t 3 − 3 t T_3=4t^3-3t T3=4t3−3t
在 t ∈ [ − 1 , 1 ] {t\in[-1,1]} t∈[−1,1] 上关于 0 {0} 0 有 4 {4} 4 个交错偏差点,所以
P 2 ( t ) = g ( t ) − 1 4 T 3 ( t ) = ( t + 3 ) 3 + 2 ( t + 3 ) 2 − 1 4 ( 4 t 3 − 3 t ) = 11 x 2 − 105 4 x + 99 4 = P 2 ∗ ( x ) \begin{align*} P_2(t)=&g(t)- \frac{1}{4}T_3(t)\\ =&(t+3)^3+2(t+3)^2- \frac{1}{4}(4t^3-3t)\\ =&11x^2- \frac{105}{4}x+ \frac{99}{4}=P_2^*(x) \end{align*} P2(t)===g(t)−41T3(t)(t+3)3+2(t+3)2−41(4t3−3t)11x2−4105x+499=P2∗(x)
误差:
max 2 ≤ x ≤ 4 ∣ f ( x ) − P 2 ∗ ( x ) ∣ = max − 1 ≤ t ≤ 1 ∣ 1 4 T 3 ( t ) ∣ = 1 4 \max_{2\le x\le 4}|f(x)-P_2^*(x)|=\max_{-1\le t\le 1}| \frac{1}{4}T_3(t)|= \frac{1}{4} 2≤x≤4max∣f(x)−P2∗(x)∣=−1≤t≤1max∣41T3(t)∣=41