机器学习的数学基础(二)


三道试题

第二题{Positive definite matrices}

题目

A matrix A ∈ R n × n ​ A \in {R}^{n×n}​ ARn×n is positive semi-definite (PSD), denoted A ≥ 0 ​ A \ge 0​ A0, if A = A T ​ A = {A}^{T}​ A=AT and x T A x ≥ 0 ​ {x}^T Ax \ge 0​ xTAx0 for all x ∈ R n ​ x \in {R}^{n}​ xRn. A matrix A ​ A​ A is positive definite,denoted A > 0 ​ A > 0​ A>0,if A = A T ​ A={A}^{T}​ A=AT and x T A x > 0 ​ {x}^TAx>0​ xTAx>0 for all x ≠ 0 ​ x\ne 0​ x̸=0, that is, all non-zero vectors x.The simplest example of a positive definite matrix is the identity I (the diagonal matrix with 1s on the diagonal and 0s elsewhere), which satisfies x T I x = ∥ x ∥ 2 2 = ∑ i = 1 n x i 2 ​ {x}^TIx={\left\| x\right\|}_2^2 = \sum_{i=1}^{n}{{x}_{i}^2}​ xTIx=x22=i=1nxi2.

第一问

Let z ∈ R n z \in {R}^{n} zRn be an n-vector. Show that A = z z T A = z{z}^T A=zzT is positive semi-definite.

首先: A T = ( z z T ) T = z z T = A A^T = (zz^T)^T = zz^T=A AT=(zzT)T=zzT=A

z z T = [ z 1 z 2 ⋮ z n ] [ z 1 z 2 ⋯ z n ] = [ z 1 z 1 z 1 z 2 ⋯ z 1 z n z 2 z 1 z 2 z 2 ⋯ z 2 z n ⋮ ⋮ ⋱ ⋮ z n z 1 z n z 2 ⋯ z n z n ] ​ zz^T=\begin{bmatrix} z_1 \\ z_2 \\ \vdots \\ z_n \end{bmatrix} \begin{bmatrix} z_1 & z_2 & \cdots & z_n \end{bmatrix}=\begin{bmatrix} z_1z_1 & z_1z_2 & \cdots & z_1z_n \\ z_2z_1 & z_2z_2 & \cdots & z_2z_n \\ \vdots &\vdots &\ddots & \vdots \\ z_nz_1 & z_nz_2 & \cdots &z_nz_n\end{bmatrix}​ zzT=z1z2zn[z1z2zn]=z1z1z2z1znz1z1z2z2z2znz2z1znz2znznzn

∴ A i j = z i z j \therefore A_{ij} = z_iz_j Aij=zizj

由第一题可知:
x T A x = ∑ i = 1 n ∑ j = 1 n A i j x i x j = ∑ i = 1 n ∑ j = 1 n z i z j x i x j = ∑ i = 1 n z i x i ∑ j = 1 n z i x j = ( ∑ i = 1 n z i x i ) 2 ≥ 0 {x}^{T}Ax =\sum_{i=1}^{n}{\sum_{j=1}^{n}{{A}_{ij}{x}_{i}}{x}_{j}}=\sum_{i=1}^{n}{\sum_{j=1}^{n}{z_iz_j{x}_{i}}{x}_{j}}=\sum_{i=1}^{n}z_ix_i\sum_{j=1}^{n}z_ix_j={(\sum_{i=1}^{n}z_ix_i)}^2 \ge 0 xTAx=i=1nj=1nAijxixj=i=1nj=1nzizjxixj=i=1nzixij=1nzixj=(i=1nzixi)20 for x ∈ R n x\in R^n xRn

所以:A is PSD。

第二问

Let z ∈ R n ​ z \in R^n​ zRn be a non-zero n-vector. Let A = z z T ​ A = zz^T​ A=zzT . What is the null-space of A? What is the rank of A?

  • 知识点 null-space
    The nullspace of A consists of all solutions to A x = 0 Ax=0 Ax=0. These vectors x are in R n R^n Rn. The nullspace containing all solutions of A x = 0 Ax=0 Ax=0 is denoted by N ( A ) ​ N(A)​ N(A).

因为:z为非0向量,所以: A x = 0 ⇒ z z T x = 0 ⇒ [ z 1 z 1 z 1 z 2 ⋯ z 1 z n z 2 z 1 z 2 z 2 ⋯ z 2 z n ⋮ ⋮ ⋱ ⋮ z n z 1 z n z 2 ⋯ z n z n ] x = 0 ⇒ [ z 1 z 1 z 1 z 2 ⋯ z 1 z n 0 0 ⋯ 0 ⋮ ⋮ ⋱ ⋮ 0 0 ⋯ 0 ] x = 0 ⇒ [ z 1 z 2 ⋯ z n 0 0 ⋯ 0 ⋮ ⋮ ⋱ ⋮ 0 0 ⋯ 0 ] x = 0 ​ Ax=0 \Rightarrow zz^Tx=0 \Rightarrow \begin{bmatrix} z_1z_1 & z_1z_2 & \cdots & z_1z_n \\ z_2z_1 & z_2z_2 & \cdots & z_2z_n \\ \vdots &\vdots &\ddots & \vdots \\ z_nz_1 & z_nz_2 & \cdots &z_nz_n\end{bmatrix}x=0 \Rightarrow \begin{bmatrix} z_1z_1 & z_1z_2 & \cdots & z_1z_n \\ 0 & 0 & \cdots & 0 \\ \vdots &\vdots &\ddots & \vdots \\ 0 & 0 & \cdots &0 \end{bmatrix}x=0 \Rightarrow \begin{bmatrix} z_1 & z_2 & \cdots & z_n \\ 0 & 0 & \cdots & 0 \\ \vdots &\vdots &\ddots & \vdots \\ 0 & 0 & \cdots &0 \end{bmatrix}x=0​ Ax=0zzTx=0z1z1z2z1znz1z1z2z2z2znz2z1znz2znznznx=0z1z100z1z200z1zn00x=0z100z200zn00x=0

可得 r a n k ( A ) = 1 rank(A) = 1 rank(A)=1.

可知: x 1 x_1 x1为pivot variable,其他为free variables,所以 special solutions如下:

s 1 = [ − z 2 z 1 0 ⋮ 0 ] s_1 = \begin{bmatrix} -z_2 \\ z_1 \\ 0\\ \vdots \\ 0 \end{bmatrix} s1=z2z100

s 2 = [ − z 3 0 z 1 ⋮ 0 ] s_2 = \begin{bmatrix} -z_3 \\ 0\\ z_1 \\ \vdots \\ 0 \end{bmatrix} s2=z30z10

⋮ ​ \vdots​

s n − 2 = [ − z n − 1 0 0 ⋮ z 1 0 ] ​ s_{n-2}= \begin{bmatrix} -z_{n-1} \\ 0 \\ 0\\ \vdots \\ z_1\\ 0 \end{bmatrix}​ sn2=zn100z10

s n − 1 = [ − z n 0 0 ⋮ 0 z 1 ] ​ s_{n-1}= \begin{bmatrix} -z_n \\ 0 \\ 0\\ \vdots \\ 0 \\ z_1 \end{bmatrix}​ sn1=zn000z1

A的Null Space是 s 1 ​ s_1​ s1 s 2 ​ s_2​ s2 s n − 1 ​ s_{n-1}​ sn1共n-1个special solutions的线性组合,即:

N ( A ) = c 1 s 1 + c 2 s 2 + ⋯ + c n − 1 s ( n − 1 ) = [ − ∑ i = 1 n − 1 c i z i + 1 c 1 z 1 ⋮ c n − 1 z 1 ] ​ N(A)= c_1s_1 + c_2s_2 + \cdots + c_{n-1}s_(n-1)= \begin{bmatrix} -\sum_{i=1}^{n-1}c_iz_{i+1} \\ c_1z_1\\ \vdots \\c_{n-1} z_1 \end{bmatrix}​ N(A)=c1s1+c2s2++cn1s(n1)=i=1n1cizi+1c1z1cn1z1

第三问

Let A ∈ R n × n A \in R^{n\times n} ARn×n be positive semidefinite and B ∈ R m × n B \in R^{m \times n} BRm×n be arbitrary, where m , n ∈ N m, n \in N m,nN. Is B A B T BAB^T BABT PSD? If so, prove it. If not, give a counterexample with explicit A, B.

  1. C = B A B T ​ C=BAB^T​ C=BABT,则 C T = B A B T ​ C^T = BAB^T​ CT=BABT, 且 C ∈ R m × m ​ C \in R^{m \times m}​ CRm×m.

  2. x ∈ R m x \in R^m xRm, x T C x = x T B A B T x = ( B T x ) T A ( B T x ) x^TCx = x^TBAB^Tx=(B^Tx)^TA(B^Tx) xTCx=xTBABTx=(BTx)TA(BTx) y = B T x y = B^Tx y=BTx, y ∈ R n y \in R^n yRn, 因为 A是PSD,所以: y T A y ≥ 0 y^TAy \ge 0 yTAy0

    综合1,2,可得 C为PSD,即: B A B T BAB^T BABT为PSD。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值