3.1. Let
A
∈
R
m
×
n
,
b
∈
R
n
,
L
∈
R
p
×
n
,
λ
∈
R
+
+
\mathbf{A}\in\mathbb{R}^{m\times n},\mathbf{b}\in\mathbb{R}^n,\mathbf{L}\in\mathbb{R}^{p\times n},\lambda \in \mathbb{R}_{++}
A∈Rm×n,b∈Rn,L∈Rp×n,λ∈R++.Consider the regularizer least squares problem
(RLS)
min
x
∈
R
n
∥
A
x
−
b
∥
2
+
λ
∥
L
x
∥
2
\text { (RLS) } \min _{x \in \mathbb{R}^{n}}\|\mathbf{A} \mathbf{x}-\mathbf{b}\|^{2}+\lambda\|\mathbf{L} \mathbf{x}\|^{2}
(RLS) x∈Rnmin∥Ax−b∥2+λ∥Lx∥2
Show that (RLS) has a unique solution if and only if
N
u
l
l
(
A
)
∩
N
u
l
l
(
L
)
=
{
0
}
\mathrm{Null}\left(\mathbf{A}\right)\cap \mathrm{Null}\left(\mathbf{L}\right)=\left\{0\right\}
Null(A)∩Null(L)={0},where here for a matrix
B
\mathbf{B}
B,
N
u
l
l
(
B
)
\mathrm{Null}\left(\mathbf{B}\right)
Null(B) is the null space of
B
\mathbf{B}
B given by
{
x
:
B
x
=
0
}
\left\{\mathbf{x}:\mathbf{Bx}=0\right\}
{x:Bx=0}.
解:
设
B
=
(
A
L
)
\mathbf{B}=\begin{pmatrix}\mathbf{A}\\\mathbf{L}\\\end{pmatrix}
B=(AL)
C
=
(
A
λ
L
)
\mathbf{C}=\begin{pmatrix}\mathbf{A}\\\sqrt{\lambda}\mathbf{L}\\\end{pmatrix}
C=(AλL)
N
u
l
l
(
A
)
∩
N
u
l
l
(
L
)
=
{
0
}
⇔
N
u
l
l
(
B
)
=
{
0
}
⇔
N
u
l
l
(
C
)
=
{
0
}
\mathrm{Null}\left(\mathbf{A}\right)\cap \mathrm{Null}\left(\mathbf{L}\right)=\left\{0\right\}\Leftrightarrow \mathrm{Null}\left(\mathbf{B}\right)=\left\{0\right\}\Leftrightarrow \mathrm{Null}\left(\mathbf{C}\right)=\left\{0\right\}
Null(A)∩Null(L)={0}⇔Null(B)={0}⇔Null(C)={0}
设
f
(
x
)
=
∥
A
x
−
b
∥
2
+
λ
∥
L
x
∥
2
f\left(\mathbf{x}\right)=\|\mathbf{A} \mathbf{x}-\mathbf{b}\|^{2}+\lambda\|\mathbf{L} \mathbf{x}\|^{2}
f(x)=∥Ax−b∥2+λ∥Lx∥2
∇
2
f
(
x
)
=
2
(
A
T
A
+
λ
L
T
L
)
=
C
T
C
\nabla^2 f\left(\mathbf{x}\right)=2\left(\mathbf{A}^T\mathbf{A}+\lambda\mathbf{L}^T\mathbf{L}\right)=\mathbf{C}^T\mathbf{C}
∇2f(x)=2(ATA+λLTL)=CTC
又因为
r
a
n
k
(
C
)
=
r
a
n
k
(
C
T
C
)
rank\left(\mathbf{C}\right)=rank\left(\mathbf{C}^T\mathbf{C}\right)
rank(C)=rank(CTC)
∇
2
f
(
x
)
≻
0
⇔
r
a
n
k
(
C
T
C
)
=
n
⇔
r
a
n
k
(
C
)
=
n
⇔
N
u
l
l
(
A
)
∩
N
u
l
l
(
L
)
=
{
0
}
\nabla^2 f\left(\mathbf{x}\right)\succ0\Leftrightarrow rank\left(\mathbf{C}^T\mathbf{C}\right)=n\Leftrightarrow rank\left(\mathbf{C}\right)=n\Leftrightarrow \mathrm{Null}\left(\mathbf{A}\right)\cap \mathrm{Null}\left(\mathbf{L}\right)=\left\{0\right\}
∇2f(x)≻0⇔rank(CTC)=n⇔rank(C)=n⇔Null(A)∩Null(L)={0}
所以成立
3.2. Generate thirty points ( x i , y i ) , i = 1 , 2 , … , 30 \left(x_{i}, y_{i}\right), i=1,2, \ldots, 30 (xi,yi),i=1,2,…,30,by the MATLAB code
randn('seed',314);
x=linspace(0,1,30)';
y=2*x.^2-3*x+1+0.05*randn(size(x));
Find the quadratic function
y
=
a
x
2
+
b
x
+
c
y=ax^2+bx+c
y=ax2+bx+c that best fits the points in the least squares sense.Indicate what are the parameters
a
,
b
,
c
a,b,c
a,b,c found by the least squares solution, and plot the points along with the derived quadratic function. The resulting plot should look like the one in Figure3.5.
解
randn('seed',314);
x=linspace(0,1,30)';
y=2*x.^2-3*x+1+0.05*randn(size(x));
A=[x.*x,x,ones(size(x))];
x_ls=(A'*A)\(A'*y);%(1.9055,-2.8678,0.96179)
plot(x,y,'*');
hold on
plot(x,A*x_ls,'b');
实现是拟合的,*是数据
3.3. Write a MATLAB function circle_fi t whose input is an n x m matrix A; the columns of A \mathbf{A} A are the m m m vectors in R n \mathbb{R}^n Rn to which a circle should be fitted. The call to the function will be of the form
[x,r]=circle_fit(A)
The output
(
x
,
r
)
\left(\mathbf{x},r\right)
(x,r) is the optimal solution of (3.6). Use the code in order to find the best circle fit in the sense of (3.6) of the 5 points
a
1
=
(
0
0
)
,
a
2
=
(
0.5
0
)
,
a
3
=
(
1
0
)
,
a
4
=
(
1
1
)
,
a
5
=
(
0
1
)
a_{1}=\left(\begin{array}{l} 0 \\ 0 \end{array}\right), \quad a_{2}=\left(\begin{array}{c} 0.5 \\ 0 \end{array}\right), \quad a_{3}=\left(\begin{array}{l} 1 \\ 0 \end{array}\right), \quad a_{4}=\left(\begin{array}{l} 1 \\ 1 \end{array}\right), \quad a_{5}=\left(\begin{array}{l} 0 \\ 1 \end{array}\right)
a1=(00),a2=(0.50),a3=(10),a4=(11),a5=(01)
解:
(应该是说
a
i
∈
R
n
a_i\in\mathbb{R}^n
ai∈Rn吧)
function [x,r] = circle_fit(A)
b=diag(A'*A);
A_tilde=[2*A',-ones(size(b))];
y=(A_tilde'*A_tilde)\(A_tilde'*b);
x=y(1:2,:);
r=sqrt(x'*x-y(3));
end
a1=[0;0];
a2=[0.5;0];
a3=[1;0;];
a4=[1;1];
a5=[0;1];
A=[a1,a2,a3,a4,a5];
[x,r] = circle_fit(A)
plot(A(1,:),A(2,:),'b*');
hold on
alpha=0:pi/40:2*pi;
plot(x(1)+r*cos(alpha),x(2)+r*sin(alpha),'r--');