机器人学状态估计 第一讲作业

1.证明${\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x-u{)}^2}{2{\sigma }^2}}dx=1$

证：

令 $\frac{x-u}{\sqrt{2}\sigma }=t,t\in (-\infty ,\infty )$

故$x=\sqrt{2}\sigma t+u$，有

$${\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x-u{)}^2}{2{\sigma }^2}}dx={\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-t^2}d(\sqrt{2}\sigma t+u)={\int }_{-\infty }^{\infty }\frac{1}{\sqrt{\pi }}{e}^{-t^2}dt$$

由${\int }_{-\infty }^{\infty }{e}^{-t^2}dt=\sqrt{\pi }$可知

$${\int }_{-\infty }^{\infty }\frac{1}{\sqrt{\pi }}{e}^{-t^2}dt=1$$

故

$${\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}dx=1$$

2.（第二章第1题）$\mu \text{μ}\mu ,v\text{v}v$为维度相同的列向量，证明

$${u\text{u}u}^{\top }v\text{v}v=\mathrm{tr}\left(v\text{v}v{u\text{u}u}^{\top }\right)$$

证：

令 $\mu \text{μ}\mu =[{\mu }_1,\cdots ,{\mu }_M{]}^T$,$v\text{v}v=[v_1,\cdots ,v_M{]}^T$,则
$$v\text{v}v{u\text{u}u}^{\top }=\left[\begin{array}{c}{v}_1\\ ⋮\\ v_m\end{array}\right]\left[u_1\cdots \cdot u_m\right]=\left[\begin{array}{cccccc}{v}_1{u}_1& \cdots & \cdots & \cdots & v_1{u}_m& \\ ⋮& \ddots & & & ⋮& \\ ⋮& & v_k{u}_k& & ⋮& \\ ⋮& & & \ddots & ⋮& \\ v_m{u}_1& \cdots & \cdots & \cdots & v_m{u}_m& \end{array}\right]$$

故
$$\mathrm{tr}\left(v\text{v}v{u\text{u}u}^{\top }\right)=\sum _{i=1}^m{v}_i{u}_i={u\text{u}u}^{\top }v\text{v}v$$

原命题得证

3.（第二章第4题）对于高斯分布随机变量，$x\text{x}x\sim N(\mu \text{μ}\mu ,\Sigma \text{Σ}\Sigma )$,证明

$$\mu \text{μ}\mu =E(x\text{x}x)={\int }_{-\infty }^{\infty }x\text{x}x\frac{1}{\sqrt{(2\pi {)}^{N}det(\Sigma )}}\exp \left(-\frac{1}{2}(x\text{x}x-u\text{u}u{)}^T{\Sigma }^{-1}(x\text{x}x-u\text{u}u)\right)dx\text{x}x$$

证：

因为$\Sigma$对称半正定且$\Sigma$可逆

故$\Sigma$正定且可相似对角化，即存在单位正交矩阵Q使得

$$\Sigma =Q^{\top }\Lambda Q$$

其中$\Lambda =\left[\begin{array}{lll}{\lambda }_1& & \\ & \ddots & {\lambda }_N\end{array}\right]$,${\lambda }_i$为$\Sigma$特征值且${\lambda }_i>0$

故

$$u\text{u}u=E(x\text{x}x)={\int }_{-\infty }^{\infty }x\text{x}x\frac{1}{\sqrt{(2\pi {)}^{N}det(\Sigma )}}\exp \left(-\frac{1}{2}(x\text{x}x-u\text{u}u{)}^{\top }{Q}^{\top }\Lambda Q(x\text{x}x-u\text{u}u)\right)dx\text{x}x$$

令$t\text{t}t=Q(x\text{x}x-u\text{u}u)$， 则$x\text{x}x=Q^{-1}t\text{t}t+\mu \text{μ}\mu$

因为Q的行列式为1，

故
$$\begin{array}{rl}u=E(x)& ={\int }_{-\infty }^{\infty }\frac{Q^{-1}t\text{t}t+u\text{u}u}{\sqrt{(2\pi {)}^{N}det(\Sigma )}}\exp \left(-\frac{1}{2}{t\text{t}t}^{\top }\Lambda t\text{t}t\right)dt\text{t}t\\ & ={\int }_{-\infty }^{\infty }\frac{Q^{-1}t\text{t}t}{\sqrt{(2\pi {)}^N\det (\Sigma )}}\exp \left(-\frac{1}{2}{t\text{t}t}^{\top }\Lambda t\text{t}t\right)dt\text{t}t+{\int }_{-\infty }^{\infty }\frac{u\text{u}u}{\sqrt{(2\pi {)}^{N}det(\Sigma )}}\exp \left(-\frac{1}{2}{t\text{t}t}^{\top }\Lambda t\text{t}t\right)dt\text{t}t\end{array}$$

因第一项中被积项为奇函数，积分后为0

有
$$\begin{array}{rl}u\text{u}u=E(x\text{x}x)& ={\int }_{-\infty }^{\infty }\frac{u\text{u}u}{\sqrt{(2\pi {)}^{N}det(\Sigma )}}\exp \left(-\frac{1}{2}{t\text{t}t}^{\top }\Lambda t\text{t}t\right)dt\text{t}t\\ & ={\int }_{-\infty }^{\infty }\frac{u\text{u}u}{\sqrt{(2\pi {)}^{N}det(\Sigma )}}\exp \left(-\frac{1}{2}\sum _{i=1}^N\frac{1}{{\lambda }_i}{t}_i^2\right)dt\text{t}t\\ & =\frac{u\text{u}u}{\sqrt{(2\pi {)}^N{\prod }_{i=N}^N{\lambda }_i}}\prod _{i=1}^N{\int }_{-\infty }^{\infty }\exp \left(-\frac{1}{2}{\lambda }_i{t}_i^2\right)d{t}_i\end{array}$$

令$z_i=\frac{1}{\sqrt{2{\lambda }_i}}{t}_i$,有

$$\begin{array}{rl}\mu \text{μ}\mu & =\frac{\mu \text{μ}\mu }{\sqrt{(2\pi {)}^N{\prod }_{i=1}^N{\lambda }_i}}\prod _{i=1}^N\sqrt{2{\lambda }_i}{\int }_{-\infty }^{\infty }\exp \left(-z_i^2\right)d{z}_i\\ & =\mu \text{μ}\mu \end{array}$$

原式得证

##E 4.（第二章第5题）对于高斯分布随机变量，$x\text{x}x\sim N(\mu \text{μ}\mu ,\Sigma \text{Σ}\Sigma )$,证明
$$\Sigma =E\left[(x\text{x}x-u\text{u}u)(x\text{x}x-u\text{u}u{)}^{\top }\right]={\int }_{-\infty }^{\infty }(x\text{x}x-u\text{u}u)(x\text{x}x-u\text{u}u{)}^{\top }\frac{1}{\sqrt{(2\pi {)}^N\det (\Sigma )}}\exp \left[-\frac{1}{2}(x\text{x}x-u\text{u}u{)}^{\top }{\Sigma }^{-1}(x\text{x}x-u\text{u}u)\right]dx\text{x}x$$

证：

令$f(x\text{x}x)=\Sigma (x\text{x}x-u\text{u}u)(x\text{x}x-u\text{u}u{)}^{\top }{\Sigma }^{-1}$，先求$f(x\text{x}x)$期望:

$$\begin{array}{rl}E[f(x\text{x}x)]& ={\int }_{-\infty }^{\infty }f(x\text{x}x)p(x\text{x}x)dx\text{x}x\\ & ={\int }_{-\infty }^{\infty }\frac{\sum (x\text{x}x-u\text{u}u)(x\text{x}x-u\text{u}u{)}^T{\Sigma }^{-1}}{\sqrt{(2\pi {)}^N\det (\Sigma )}}\exp \left(-\frac{1}{2}(x\text{x}x-u\text{u}u{)}^{\top }{\Sigma }^{-1}(x\text{x}x-u\text{u}u)\right)dx\text{x}x\\ & ={\int }_{-\infty }^{\infty }\frac{\sum y\text{y}y{y\text{y}y}^T{\sum }^{-1}}{\sqrt{(2\pi {)}^{N}det(\Sigma )}}\exp \left(-\frac{1}{2}{y\text{y}y}^T{\Sigma }^{-1}y\text{y}y\right)dy\text{y}y\\ & ={\int }_{-\infty }^{\infty }\frac{\Sigma y\text{y}y}{\sqrt{(2\pi {)}^N\det (\Sigma )}}{\left(\exp \left(-\frac{1}{2}{y\text{y}y}^{\top }{\Sigma }^{-1}y\text{y}y\right){\Sigma }^{-1}y\text{y}y\right)}^{\top }dy\text{y}y\\ & ={\int }_{-\infty }^{\infty }\frac{\Sigma y\text{y}y}{\sqrt{(2\pi {)}^N\det (\Sigma )}}d{\exp }^{\top }\left(-\frac{1}{2}{y\text{y}y}^{\top }{\Sigma }^{-1}y\text{y}y\right)\\ & =-\Sigma {\int }_{-\infty }^{\infty }\frac{y\text{y}y}{\sqrt{(2\pi {)}^N\det (\Sigma )}}d\exp \left(-\frac{1}{2}{y\text{y}y}^{\top }{\Sigma }^{-1}y\text{y}y\right)\\ & =-\Sigma \left[{\frac{y\text{y}y}{\sqrt{(2\pi {)}^{N}det(\Sigma )}}\exp \left(-\frac{1}{2}{y\text{y}y}^{\top }{\Sigma }^{-1}y\text{y}y\right)|}_{-\infty }^{\infty }-{\int }_{-\infty }^{\infty }\frac{1}{\sqrt{(2\pi {)}^{N}det\Sigma }}\exp \left(-\frac{1}{2}{y\text{y}y}^{\top }{\Sigma }^{-1}y\text{y}y\right)dy\text{y}y\right]\\ & =-\Sigma [0-I]\\ & =\Sigma \end{array}$$

上面推导过程用到了如下性质：

1.若$\Sigma$对称正定

$$\frac{d{y\text{y}y}^{\top }{\Sigma }^{-1}y\text{y}y}{dy\text{y}y}=\left({\Sigma }^{-1}+{\Sigma }^{-T}\right)y\text{y}y=2{\Sigma }^{-1}y\text{y}y$$

2.$\exp \left(-\frac{1}{2}{y\text{y}y}^{\top }{\Sigma }^{-1}y\text{y}y\right)$对称（可从泰勒展开得出）

因
$$\begin{array}{rl}E[f(x\text{x}x)]& =E\left[\Sigma (x\text{x}x-u\text{u}u)(x\text{x}x-u\text{u}u{)}^{\top }{\Sigma }^{-1}\right]\\ & =\Sigma E\left[(x\text{x}x-u\text{u}u)(x\text{x}x-u\text{u}u{)}^{\top }\right]{\Sigma }^{-1}\\ & =\Sigma \end{array}$$

故$E\left[(x\text{x}x-u\text{u}u)(x\text{x}x-u\text{u}u{)}^{\top }\right]=\Sigma$

原命题得证

5.（第二章第6题）对于K个相互独立的高斯变量，$x_k\sim N({\mu }_k,{\Sigma }_k)$,请证明它们的归一化积仍是高斯分布

$$\exp \left(-\frac{1}{2}(x\text{x}x-u\text{u}u{)}^{\top }{\Sigma }^{-1}(x\text{x}x-u\text{u}u)\right)=\eta \prod _{k=1}^K\exp \left(-\frac{1}{2}{\left(x\text{x}x-{u\text{u}u}_k\right)}^{\top }{\Sigma }_k^{-1}\left(x\text{x}x-{u\text{u}u}_k\right)\right)$$

其中
$${\Sigma }^{-1}=\sum _{k=1}^K{\Sigma }_k^{-1}{\Sigma }^{-1}u\text{u}u=\sum _{k=1}^K{\Sigma }_k^{-1}{u}_k$$

证明：
$$\begin{array}{rl}\eta \prod _{k=1}^K\exp \left(-\frac{1}{2}{\left(x\text{x}x-{u\text{u}u}_k\right)}^T{\Sigma }_k^{-1}\left(x\text{x}x-{u\text{u}u}_k\right)\right)& =\eta \exp \left[\sum _{k=1}^K-\frac{1}{2}{\left(x\text{x}x-{u\text{u}u}_k\right)}^{\top }{\Sigma }_k^{-1}\left(x\text{x}x-{u\text{u}u}_k\right)\right]\\ & =\eta \exp \left[-\frac{1}{2}{x\text{x}x}^{\top }\sum _{k=1}^k{\Sigma }_k^{-1}x\text{x}x-{x\text{x}x}^{\top }\sum _{k=1}^k{\Sigma }_k^{-1}{u\text{u}u}_k-\sum _{k=1}^k{u\text{u}u}_k^{\top }{\Sigma }_k^{-1}x\text{x}x+\sum _{k=1}^k{u\text{u}u}_k^{\top }{\Sigma }_k^{-1}{u\text{u}u}_k)\right]\end{array}$$

记$M={\left({\sum }_{k=1}^K{\Sigma }_k^{-1}{U}_k\right)}^{\top }{\left({\sum }_{k=1}^k{\sum }_k^{-1}\right)}^{-1}\left({\sum }_{k=1}^k{\Sigma }_k^{-1}{U}_k\right)$

$$\begin{array}{rl}\eta \prod _{k=1}^K\exp \left(-\frac{1}{2}{\left(x\text{x}x-{u\text{u}u}_k\right)}^T{\Sigma }_k^{-1}\left(x\text{x}x-{u\text{u}u}_k\right)\right)& =\eta \exp \left[-\frac{1}{2}\left({x\text{x}x}^T\sum _{k=1}^K{\Sigma }_k^{-1}x\text{x}x-{x\text{x}x}^{\top }\sum _{k=1}^k{\Sigma }_k^{-1}{u\text{u}u}_k-\sum _{k=1}^k{u\text{u}u}_k^{\top }{\Sigma }_k^{-1}x\text{x}x+M\right)+\frac{1}{2}M+\frac{1}{2}\sum _{k=1}^K{u\text{u}u}_k^{\top }{\Sigma }_k^{-1}{u\text{u}u}_k\right]\\ & =\eta \exp \left[\frac{1}{2}M+\frac{1}{2}\sum _{k=1}^K{u\text{u}u}_k^T{\Sigma }_k^{-1}{u\text{u}u}_k\right]\exp \left[-\frac{1}{2}{x\text{x}x}^{\top }\sum _{k=1}^K{\Sigma }_{k=1}^{-1}x\text{x}x-{x\text{x}x}^T\sum _{k=1}^K{\Sigma }_k^{-1}{u\text{u}u}_k-\sum _{k=1}^T{u\text{u}u}_k^T{\Sigma }_k^{-1}x\text{x}x+M\right]\\ & ={\eta }^{\prime }\exp \left[-\frac{1}{2}{\left(x\text{x}x-{\left(\sum _{k=1}^k{\Sigma }_k^{-1}\right)}^{-1}\sum _{k=1}^k{\Sigma }_k^{-1}{u\text{u}u}_k\right)}^T\sum _{k=1}^K{\Sigma }_k^{-1}\left(x\text{x}x-{\left(\sum _{k=1}^K{\Sigma }_k^{-1}\right)}^{-1}\sum _{k=1}^k{\Sigma }_k^{-1}{u\text{u}u}_k\right)\right]\\ & ={\eta }^{\prime }\exp \left[-\frac{1}{2}(x\text{x}x-u\text{u}u{)}^{\top }{\Sigma }^{-1}(x\text{x}x-u\text{u}u)\right]\left(记{\Sigma }^{-1}=\sum _{k=1}^K{\Sigma }_k^{-1}u\text{u}u={\Sigma }^{-1}\sum _{k=1}^K{\Sigma }_k^{-1}{u\text{u}u}_k\right)\end{array}$$

上式中
$$\{\begin{array}{l}{\Sigma }^{-1}={\sum }_{k=1}^K{\Sigma }_k^{-1}\\ u\text{u}u=\Sigma {\sum }_{k=1}^K{\Sigma }_k^{-1}{u\text{u}u}_k\Rightarrow {\Sigma }^{-1}u\text{u}u={\sum }_{k=1}^k{\Sigma }_k^{-1}{u\text{u}u}_k\\ {\eta }^{\prime }=\eta \exp \left[\frac{1}{2}{u\text{u}u}^{\top }{\Sigma }^{-1}u\text{u}u+\frac{1}{2}{\sum }_{k=1}^k{u\text{u}u}_k^{\top }{\Sigma }_k^{-1}{u\text{u}u}_k\right]\end{array}$$
原命题得证