机器学习的数学基础（二）

最新推荐文章于 2024-10-30 15:22:37 发布

置顶章政兴

最新推荐文章于 2024-10-30 15:22:37 发布

阅读量352

点赞数 2

分类专栏：机器学习文章标签：机器学习数学矩阵人工智能正定矩阵

本文链接：https://blog.csdn.net/weixin_43144524/article/details/87909626

版权

机器学习专栏收录该内容

2 篇文章 1 订阅

订阅专栏

三道试题第二题

- 第二题{Positive definite matrices}

第二题{Positive definite matrices}

题目

A matrix $\in {R}^{n×n}$ is positive semi-definite (PSD), denoted $\ge 0$ , if $A = {A}^{T}$ and ${x}^T Ax \ge 0$ for all $\in {R}^{n}$ . A matrix $A $ is positive definite,denoted $A > 0 $ ,if $A={A}^{T}$ and ${x}^TAx>0$ for all $x\ne 0$ , that is, all non-zero vectors x.The simplest example of a positive definite matrix is the identity I (the diagonal matrix with 1s on the diagonal and 0s elsewhere), which satisfies ${x}^TIx={\left\| x\right\|}_2^2 = \sum_{i=1}^{n}{{x}_{i}^2}$ .

第一问

Let $\in {R}^{n}$ be an n-vector. Show that $A = z{z}^T$ is positive semi-definite.

解

首先: $A^T = (zz^T)^T = zz^T=A$

$zz^T=\begin{bmatrix} z_1 \\ z_2 \\ \vdots \\ z_n \end{bmatrix} \begin{bmatrix} z_1 & z_2 & \cdots & z_n \end{bmatrix}=\begin{bmatrix} z_1z_1 & z_1z_2 & \cdots & z_1z_n \\ z_2z_1 & z_2z_2 & \cdots & z_2z_n \\ \vdots &\vdots &\ddots & \vdots \\ z_nz_1 & z_nz_2 & \cdots &z_nz_n\end{bmatrix}$

$\therefore A_{ij} = z_iz_j$

由第一题可知:
${x}^{T}Ax =\sum_{i=1}^{n}{\sum_{j=1}^{n}{{A}_{ij}{x}_{i}}{x}_{j}}=\sum_{i=1}^{n}{\sum_{j=1}^{n}{z_iz_j{x}_{i}}{x}_{j}}=\sum_{i=1}^{n}z_ix_i\sum_{j=1}^{n}z_ix_j={(\sum_{i=1}^{n}z_ix_i)}^2 \ge 0$ for $x\in R^n$

所以：A is PSD。

第二问

Let $\in R^n$ be a non-zero n-vector. Let $A = zz^T$ . What is the null-space of A? What is the rank of A?

解

知识点 null-space
The nullspace of A consists of all solutions to $A x = 0$ . These vectors x are in $R^n$ . The nullspace containing all solutions of $A x = 0$ is denoted by $N (A) $ .

因为：z为非0向量，所以： $\Rightarrow zz^Tx=0 \Rightarrow \begin{bmatrix} z_1z_1 & z_1z_2 & \cdots & z_1z_n \\ z_2z_1 & z_2z_2 & \cdots & z_2z_n \\ \vdots &\vdots &\ddots & \vdots \\ z_nz_1 & z_nz_2 & \cdots &z_nz_n\end{bmatrix}x=0 \Rightarrow \begin{bmatrix} z_1z_1 & z_1z_2 & \cdots & z_1z_n \\ 0 & 0 & \cdots & 0 \\ \vdots &\vdots &\ddots & \vdots \\ 0 & 0 & \cdots &0 \end{bmatrix}x=0 \Rightarrow \begin{bmatrix} z_1 & z_2 & \cdots & z_n \\ 0 & 0 & \cdots & 0 \\ \vdots &\vdots &\ddots & \vdots \\ 0 & 0 & \cdots &0 \end{bmatrix}x=0$

可得 $r a n k (A) = 1$ .

可知： $x_1$ 为pivot variable，其他为free variables，所以 special solutions如下：

$s_1 = \begin{bmatrix} -z_2 \\ z_1 \\ 0\\ \vdots \\ 0 \end{bmatrix}$

$s_2 = \begin{bmatrix} -z_3 \\ 0\\ z_1 \\ \vdots \\ 0 \end{bmatrix}$

$\vdots$

$s_{n-2}= \begin{bmatrix} -z_{n-1} \\ 0 \\ 0\\ \vdots \\ z_1\\ 0 \end{bmatrix}$

$s_{n-1}= \begin{bmatrix} -z_n \\ 0 \\ 0\\ \vdots \\ 0 \\ z_1 \end{bmatrix}$

A的Null Space是 $s_1$ $s_2$ 到 $s_{n-1}$ 共n-1个special solutions的线性组合，即：

$c_1s_1 + c_2s_2 + \cdots + c_{n-1}s_(n-1)= \begin{bmatrix} -\sum_{i=1}^{n-1}c_iz_{i+1} \\ c_1z_1\\ \vdots \\c_{n-1} z_1 \end{bmatrix}$

第三问

Let $\in R^{n\times n}$ be positive semidefinite and $\in R^{m \times n}$ be arbitrary, where $\in N$ . Is $BAB^T$ PSD? If so, prove it. If not, give a counterexample with explicit A, B.

解

令 $C=BAB^T$ ,则 $C^T = BAB^T$ , 且 $\in R^{m \times m}$ .
令 $\in R^m$ , $x^TCx = x^TBAB^Tx=(B^Tx)^TA(B^Tx)$ 设 $y = B^Tx$ , $\in R^n$ , 因为 A是PSD，所以: $y^TAy \ge 0$ 。

综合1,2,可得 C为PSD，即： $BAB^T$ 为PSD。