Introduction to nonlinear optimization第三章习题

3.1. Let $\mathbf{A}\in {\mathbb{R}}^{m\times n},\mathbf{b}\in {\mathbb{R}}^n,\mathbf{L}\in {\mathbb{R}}^{p\times n},\lambda \in {\mathbb{R}}_{++}$.Consider the regularizer least squares problem
$$\text{ (RLS) }\underset{x\in {\mathbb{R}}^n}{\min }\Vert \mathbf{A}\mathbf{x}-\mathbf{b}{\Vert }^2+\lambda \Vert \mathbf{L}\mathbf{x}{\Vert }^2$$
Show that (RLS) has a unique solution if and only if N u l l ( A ) ∩ N u l l ( L ) = { 0 } \mathrm{Null}\left(\mathbf{A}\right)\cap \mathrm{Null}\left(\mathbf{L}\right)=\left\{0\right\} Null(A)∩Null(L)={0},where here for a matrix $\mathbf{B}$,$\mathrm{N}\mathrm{u}\mathrm{l}\mathrm{l}\left(\mathbf{B}\right)$ is the null space of $\mathbf{B}$ given by { x : B x = 0 } \left\{\mathbf{x}:\mathbf{Bx}=0\right\} {x:Bx=0}.

解：
设$\mathbf{B}=\left(\begin{array}{c}\mathbf{A}\\ \mathbf{L}\end{array}\right)$
$\mathbf{C}=\left(\begin{array}{c}\mathbf{A}\\ \sqrt{\lambda }\mathbf{L}\end{array}\right)$
N u l l ( A ) ∩ N u l l ( L ) = { 0 } ⇔ N u l l ( B ) = { 0 } ⇔ N u l l ( C ) = { 0 } \mathrm{Null}\left(\mathbf{A}\right)\cap \mathrm{Null}\left(\mathbf{L}\right)=\left\{0\right\}\Leftrightarrow \mathrm{Null}\left(\mathbf{B}\right)=\left\{0\right\}\Leftrightarrow \mathrm{Null}\left(\mathbf{C}\right)=\left\{0\right\} Null(A)∩Null(L)={0}⇔Null(B)={0}⇔Null(C)={0}
设$f\left(\mathbf{x}\right)=\Vert \mathbf{A}\mathbf{x}-\mathbf{b}{\Vert }^2+\lambda \Vert \mathbf{L}\mathbf{x}{\Vert }^2$
$${\nabla }^{2}f\left(\mathbf{x}\right)=2\left({\mathbf{A}}^T\mathbf{A}+\lambda {\mathbf{L}}^T\mathbf{L}\right)={\mathbf{C}}^T\mathbf{C}$$
又因为$rank\left(\mathbf{C}\right)=rank\left({\mathbf{C}}^T\mathbf{C}\right)$
∇ 2 f ( x ) ≻ 0 ⇔ r a n k ( C T C ) = n ⇔ r a n k ( C ) = n ⇔ N u l l ( A ) ∩ N u l l ( L ) = { 0 } \nabla^2 f\left(\mathbf{x}\right)\succ0\Leftrightarrow rank\left(\mathbf{C}^T\mathbf{C}\right)=n\Leftrightarrow rank\left(\mathbf{C}\right)=n\Leftrightarrow \mathrm{Null}\left(\mathbf{A}\right)\cap \mathrm{Null}\left(\mathbf{L}\right)=\left\{0\right\} ∇2f(x)≻0⇔rank(CTC)=n⇔rank(C)=n⇔Null(A)∩Null(L)={0}
所以成立

3.2. Generate thirty points $\left(x_i,y_i\right),i=1,2,\dots ,30$,by the MATLAB code

randn('seed',314);
x=linspace(0,1,30)';
y=2*x.^2-3*x+1+0.05*randn(size(x));

Find the quadratic function $y=a{x}^2+bx+c$ that best fits the points in the least squares sense.Indicate what are the parameters $a,b,c$ found by the least squares solution, and plot the points along with the derived quadratic function. The resulting plot should look like the one in Figure3.5.

解

randn('seed',314);
x=linspace(0,1,30)';
y=2*x.^2-3*x+1+0.05*randn(size(x));
A=[x.*x,x,ones(size(x))];
x_ls=(A'*A)\(A'*y);%(1.9055,-2.8678,0.96179)
plot(x,y,'*');
hold on
plot(x,A*x_ls,'b');

实现是拟合的，*是数据

3.3. Write a MATLAB function circle_fi t whose input is an n x m matrix A; the columns of $\mathbf{A}$ are the $m$ vectors in ${\mathbb{R}}^n$ to which a circle should be fitted. The call to the function will be of the form

[x,r]=circle_fit(A)

The output $\left(\mathbf{x},r\right)$ is the optimal solution of (3.6). Use the code in order to find the best circle fit in the sense of (3.6) of the 5 points
$$a_1=\left(\begin{array}{l}0\\ 0\end{array}\right),a_2=\left(\begin{array}{c}0.5\\ 0\end{array}\right),a_3=\left(\begin{array}{l}1\\ 0\end{array}\right),a_4=\left(\begin{array}{l}1\\ 1\end{array}\right),a_5=\left(\begin{array}{l}0\\ 1\end{array}\right)$$

解：
（应该是说$a_i\in {\mathbb{R}}^n$吧）

function [x,r] = circle_fit(A)
    b=diag(A'*A);
    A_tilde=[2*A',-ones(size(b))];
    y=(A_tilde'*A_tilde)\(A_tilde'*b);
    x=y(1:2,:);
    r=sqrt(x'*x-y(3));
end

a1=[0;0];
a2=[0.5;0];
a3=[1;0;];
a4=[1;1];
a5=[0;1];
A=[a1,a2,a3,a4,a5];
[x,r] = circle_fit(A)
plot(A(1,:),A(2,:),'b*');
hold on
alpha=0:pi/40:2*pi;
plot(x(1)+r*cos(alpha),x(2)+r*sin(alpha),'r--');