多元正态分布的条件概率分布(一)

最新推荐文章于 2024-06-16 10:43:40 发布

omadesala

最新推荐文章于 2024-06-16 10:43:40 发布

阅读量2.8w

点赞数 20

分类专栏：基础概念文章标签：多元高斯分布条件分布

本文链接：https://blog.csdn.net/omade/article/details/28408663

版权

基础概念专栏收录该内容

13 篇文章 4 订阅

订阅专栏

多元正态分布的条件概率分布

假设分别有两个多维向量 $\mathbf{X1}=\begin{pmatrix}X_1\\ X_2\\\vdots\\X_R\end{pmatrix}$ 和 $\mathbf{X2}=\begin{pmatrix}Y_1\\ Y_2\\\vdots\\Y_S\end{pmatrix}$

$\mathbf{X}=\begin{pmatrix}\mathbf{X_1}\\ \mathbf{X_2}\\\end{pmatrix}$

$E[\mathbf{X}]=\begin{pmatrix}\boldsymbol{\mu_1}\\ \boldsymbol{\mu_2}\\\end{pmatrix}$

其中

$\boldsymbol{\mu_1}=E[\mathbf{X1}]=\begin{pmatrix}\boldsymbol{\mu_{x1}}\\ \boldsymbol{\mu_{x2}}\\\vdots\\\boldsymbol{\mu_{xR}}\\\end{pmatrix}$

$\boldsymbol{\mu_2}=E[\mathbf{X2}]==\begin{pmatrix}\boldsymbol{\mu_{y1}}\\ \boldsymbol{\mu_{y2}}\\\vdots\\\boldsymbol{\mu_{yS}}\\\end{pmatrix}$

那么 $X_1$ 的协方差矩阵为：

$\boldsymbol{{\sum}_{11}}=\begin{pmatrix}(X_1-\mu_1)(X_1-\mu_1)&(X_1-\mu_1)(X_2-\mu_2)&\hdots&(X_1-\mu_1)(X_j-\mu_j)&\hdots&(X_1-\mu_1)(X_R-\mu_R)\\ (X_2-\mu_2)(X_1-\mu_1)&(X_2-\mu_2)(X_2-\mu_2)&\hdots&(X_2-\mu_2)(X_j-\mu_j)&\hdots&(X_2-\mu_2)(X_R-\mu_R)\\\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\(X_i-\mu_i)(X_1-\mu_1)&(X_i-\mu_i)(X_2-\mu_2)&\hdots&(X_i-\mu_i)(X_j-\mu_j)&\hdots&(X_i-\mu_i)(X_R-\mu_R)\\\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\(X_R-\mu_R)(X_1-\mu_1)&(X_R-\mu_R)(X_R-\mu_R)&\hdots&(X_R-\mu_R)(X_j-\mu_j)&\hdots&(X_R-\mu_R)(X_R-\mu_R)\end{pmatrix}$

那么 $X_2$ 的协方差矩阵为：

$\boldsymbol{{\sum}_{22}}=\begin{pmatrix}(Y_1-\mu_1)(Y_1-\mu_1)&(Y_1-\mu_1)(Y_2-\mu_2)&\hdots&(Y_1-\mu_1)(Y_j-\mu_j)&\hdots&(Y_1-\mu_1)(Y_S-\mu_S)\\ (Y_2-\mu_2)(Y_1-\mu_1)&(Y_2-\mu_2)(Y_2-\mu_2)&\hdots&(Y_2-\mu_2)(Y_j-\mu_j)&\hdots&(Y_2-\mu_2)(Y_S-\mu_S)\\\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\(Y_i-\mu_i)(Y_1-\mu_1)&(Y_i-\mu_i)(Y_2-\mu_2)&\hdots&(Y_i-\mu_i)(Y_j-\mu_j)&\hdots&(Y_i-\mu_i)(Y_S-\mu_S)\\\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\(Y_S-\mu_S)(Y_1-\mu_1)&(Y_S-\mu_S)(Y_S-\mu_S)&\hdots&(Y_S-\mu_S)(Y_j-\mu_j)&\hdots&(Y_S-\mu_S)(Y_S-\mu_S)\end{pmatrix}$

那么 $X_1,X_2$ 的协方差矩阵为： $E[(\mathbf{X_1}-\boldsymbol{\mu1})(\mathbf{X_2}-\boldsymbol{\mu2})^T]$

$\boldsymbol{{\sum}_{12}}=\begin{pmatrix}(X_1-\mu_1)(Y_1-\mu_1)&(X_1-\mu_1)(Y_2-\mu_2)&\hdots&(X_1-\mu_1)(Y_j-\mu_j)&\hdots&(X_1-\mu_1)(Y_S-\mu_S)\\ (X_2-\mu_2)(Y_1-\mu_1)&(X_2-\mu_2)(Y_2-\mu_2)&\hdots&(X_2-\mu_2)(Y_j-\mu_j)&\hdots&(X_2-\mu_2)(Y_S-\mu_S)\\\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\(X_i-\mu_i)(Y_1-\mu_1)&(X_i-\mu_i)(Y_2-\mu_2)&\hdots&(X_i-\mu_i)(Y_j-\mu_j)&\hdots&(X_i-\mu_i)(Y_S-\mu_S)\\\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\(X_R-\mu_R)(Y_1-\mu_1)&(X_R-\mu_R)(Y_2-\mu_2)&\hdots&(X_R-\mu_R)(Y_j-\mu_j)&\hdots&(X_R-\mu_R)(Y_S-\mu_S)\end{pmatrix}$

那么 $X_2,X_1$ 的协方差矩阵为： $E[(\mathbf{X_1}-\boldsymbol{\mu1})(\mathbf{X_2}-\boldsymbol{\mu2})^T]$

$\boldsymbol{{\sum}_{21}}=\begin{pmatrix}(Y_1-\mu_{y1})(X_1-\mu_{x1})&(Y_1-\mu_{y1})(X_2-\mu_{x2})&\hdots&(Y_1-\mu_{y1})(X_j-\mu_{xj})&\hdots&(Y_1-\mu_{y1})(X_R-\mu_{xR})\\ (Y_2-\mu_{y2})(X_1-\mu_{x1})&(Y_2-\mu_{y2})(X_2-\mu_{x2})&\hdots&(Y_2-\mu_{y2})(X_j-\mu_{xj})&\hdots&(Y_2-\mu_{y2})(X_R-\mu_{xR})\\\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\(Y_i-\mu_{yi})(X_1-\mu_{x1})&(Y_i-\mu_{yi})(X_2-\mu_{x2})&\hdots&(Y_i-\mu_{yi})(X_j-\mu_{xj})&\hdots&(Y_i-\mu_{yi})(X_R-\mu_{xR})\\\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\(Y_S-\mu_{yS})(X_1-\mu_{x1})&(Y_S-\mu_{yS})(X_2-\mu_{x2})&\hdots&(Y_S-\mu_{yS})(X_S-\mu_{xS})&\hdots&(Y_S-\mu_{yS})(X_R-\mu_{xR})\end{pmatrix}$

那么向量 $\mathbf{X}$ 的协方差矩阵为

$\boldsymbol{\sum}=\begin{pmatrix}\boldsymbol{{\sum}_{11}} &\boldsymbol{{\sum}_{12}} \\ \boldsymbol{{\sum}_{21}} &\boldsymbol{{\sum}_{22}} \end{pmatrix}$

其中 $\mathbf{x1}\sim \boldsymbol{N}(\boldsymbol{\mu_1},\boldsymbol{{\sum}_{11}})$

其中 $\mathbf{x2}\sim \boldsymbol{N}(\boldsymbol{\mu_2},\boldsymbol{{\sum}_{22}})$

那么条件分布为

$\mathbf{x1}|\mathbf{x2}\sim \boldsymbol{N}(\boldsymbol{\mu_{1|2}},\boldsymbol{{\sum}_{1|2}})$

其中

$\boldsymbol{\mu_{1|2}}=\boldsymbol{\mu_{1}}+\boldsymbol{{\sum}_{12}}\boldsymbol{{\sum}_{22}^{-1}}(\mathbf{x2}-\boldsymbol{\mu_2})$

$\boldsymbol{{\sum}_{1|2}}=\boldsymbol{{\sum}_{11}}-\boldsymbol{{\sum}_{12}}\boldsymbol{{\sum}_{22}^{-1}}\boldsymbol{{\sum}_{21}}$

现在来看一个二维正态分布的例子

均值为 $E[\mathbf{X}]=\begin{pmatrix}1\\2\end{pmatrix}$

协方差矩阵为 $\boldsymbol{\sum}=\begin{pmatrix} \boldsymbol{{\sum}_{11}}& \boldsymbol{{\sum}_{12}} \\ \boldsymbol{{\sum}_{21}}& \boldsymbol{{\sum}_{22}}\end{pmatrix}=\begin{pmatrix}\delta_{11}& \delta_{12} \\ \delta_{21}& \delta_{22}\end{pmatrix}=\begin{pmatrix} 1& 0.1 \\ 0.1& 1\end{pmatrix}$

令 $\mathbf{X1}=(X_1),\mathbf{X2}=(X_2)$ 他们都只包含一个元素

那么条件分布

$\mathbf{x1}|\mathbf{x2}\sim \boldsymbol{N}(\boldsymbol{\mu_{1|2}},\boldsymbol{{\sum}_{1|2}})$

我们来看条件均值

$\boldsymbol{\mu_{1|2}}=\boldsymbol{\mu_{1}}+\boldsymbol{{\sum}_{12}}\boldsymbol{{\sum}_{22}^{-1}}(\mathbf{x2}-\boldsymbol{\mu_2})$

这里 $\boldsymbol{{\sum}_{12}}=\delta_{12},\boldsymbol{{\sum}_{22}}=\delta_{22}$

将其代入条件均值表达式得到

$\boldsymbol{\mu_{1|2}}=\boldsymbol{\mu_{1}}+\frac{\delta_{12}}{\delta_{22}}(\mathbf{x2}-\boldsymbol{\mu_2})$

同样得到，条件协方差矩阵 $\boldsymbol{{\sum}_{1|2}}=\delta_{11}-\frac{\delta_{12}\times \delta_{21}}{\delta_{22}}=\delta_{11}-\frac{\delta_{12}^2}{\delta_{22}}$

我们可以得到条件分布为

$X1|X2\sim \boldsymbol{N}(\mu_{1}+\frac{\delta_{12}}{\delta_{22}}(X2-\mu_2),\delta_{11}-\frac{\delta_{12}^2}{\delta_{22}})$

同样可以得到

$X2|X1\sim \boldsymbol{N}(\mu_{2}+\frac{\delta_{12}}{\delta_{11}}(X1-\mu_2),\delta_{22}-\frac{\delta_{12}^2}{\delta_{11}})$

如果代入具体的数值，我们可以得到

$\mathbf{x1}|\mathbf{x2}\sim \boldsymbol{N}(1+\frac{0.1}{1}(X2-2),1-\frac{0.01}{1})=\boldsymbol{N}(1+0.1(X2-2),0.99)=\boldsymbol{N}(0.8+0.1 \times X2),0.99)$

$\mathbf{x2}|\mathbf{x1}\sim \boldsymbol{N}(2+\frac{0.1}{1}(X1-2),1-\frac{0.01}{1})=\boldsymbol{N}(1.8+0.1\times X1,0.99)$

那么给定标准正态分布

$x \sim \boldsymbol{N}(0,1)$

$X\sim N(\mu,\delta^2),Y=\frac{X-\mu}{\delta}\sim N(0,1)$

证明：

$f(x)=\frac{1}{\sqrt{2\pi}\delta}e^{-\frac{(x-\mu)^2}{2\delta^2}}$

设F(y)为f(x)的累积分布,那么
令 $Y=\frac{X-\mu}{\delta}$ 那么 $F(y)=f(Y\leqslant y)=f(\frac{X-\mu}{\delta}<y)=f(X\leqslant \delta y+\mu)$

那么 $f(X\leqslant \delta y+\mu)=\frac{1}{\sqrt{2\pi}\delta}e^{-\frac{((\delta y+\mu)-\mu)^2}{2\delta^2}}=\frac{1}{\sqrt{2\pi}\delta}e^{-\frac{(y)^2}{2}}\sim N(0,1)$

如果从标准正态分布采样任意均值方差的公式就可以得到了

$X \sim N(0,1),X\times \delta+\mu \sim N(\mu,\delta^2)$

证明过程如下

令 $Y=X\times \delta + \mu$

F(y)是f(y)的累积分布，那么 $F(y)=f(Y\leqslant y)=f(X\times \delta + \mu \leqslant Y)=f(X\leqslant \frac{Y-\mu}{\delta})$

$f(x)=\frac{1}{\sqrt{2\pi}\delta}e^{-X^2}{2}}$

那么 $f(\frac{Y-\mu}{\delta})=\frac{1}{\sqrt{2\pi}\delta}e^{-\frac{(\frac{Y-\mu}{\delta})^2}{2}}=\frac{1}{\sqrt{2\pi}\delta}e^{-\frac{(Y-\mu)^2}{2\delta^2}} \sim N(\mu,\delta^2)$

对二元正态分布的条件分布进行采样就可以转变成对一维正态分布的采样

$X1|X2\sim \boldsymbol{N}(\mu_{1}+\frac{\delta_{12}}{\delta_{22}}(X2-\mu_2),\delta_{11}-\frac{\delta_{12}^2}{\delta_{22}})$

$X2|X1\sim \boldsymbol{N}(\mu_{2}+\frac{\delta_{12}}{\delta_{22}}(X1-\mu_2),\delta_{11}-\frac{\delta_{12}^2}{\delta_{22}})$

$X \sim N(0,1),X \times \delta + \mu \sim N(\mu_1+\frac{\delta_{12}}{\delta_{22}}(X2-\mu_2),\delta_{11}-\frac{\delta_{12}^2}{\delta_{22}})$

$X \times \delta + \mu = X \times(\delta_{11}-\frac{\delta_{12}^2}{\delta_{22}}) + (\mu_1+\frac{\delta_{12}}{\delta_{22}}(X2-\mu_2))$

同理可以得到X1条件下的采样方法。

$X \times \delta + \mu = X \times(\delta_{11}-\frac{\delta_{12}^2}{\delta_{22}}) + (\mu_2+\frac{\delta_{12}}{\delta_{22}}(X1-\mu_1))$

对于Gibbs采样的情况，我们需要考虑k维正态分布的条件概率

我们假定X1为一维变量，X2为k-1维的向量

$\mathbf{x1}|\mathbf{x2}\sim \boldsymbol{N}(\boldsymbol{\mu_{1|2}},\boldsymbol{{\sum}_{1|2}})$

$\boldsymbol{\mu_{1|2}}=\boldsymbol{\mu_{1}}+\boldsymbol{{\sum}_{12}}\boldsymbol{{\sum}_{22}^{-1}}(\mathbf{x2}-\boldsymbol{\mu_2})$

$\boldsymbol{{\sum}_{1|2}}=\boldsymbol{{\sum}_{11}}-\boldsymbol{{\sum}_{12}}\boldsymbol{{\sum}_{22}^{-1}}\boldsymbol{{\sum}_{21}}$

$\boldsymbol{{\sum}_{11}}=\delta_{11}$

$\boldsymbol{{\sum}_{12}}=\begin{pmatrix}(X_1-\mu_1)(Y_1-\mu_1)&(X_1-\mu_1)(Y_2-\mu_2)&\hdots&(X_1-\mu_1)(Y_j-\mu_j)&\hdots&(X_1-\mu_1)(Y_{k-1}-\mu_{k-1})\\ \end{pmatrix}\\=\begin{pmatrix}\delta_{x_1y_1}&\delta_{x_1y_2}&\hdots&\delta_{x_1y_j}&\hdots&\delta_{x_1y_{k-1}}\\ \end{pmatrix}\\$

$\boldsymbol{{\sum}_{21}}=\begin{pmatrix}(Y_1-\mu_{y1})(X_1-\mu_{x1})\\ (Y_2-\mu_{y2})(X_1-\mu_{x1})\\\vdots\\(Y_i-\mu_{yi})(X_1-\mu_{x1})\\\vdots\\(Y_{k-1}-\mu_{y_{k-1}})(X_1-\mu_{x1})\end{pmatrix}$

$=\begin{pmatrix}\delta_{x_1y_1}\\ \delta_{x_1y_2}\\\vdots\\\delta_{x_1y_i}\\\vdots\\\delta_{x_1y_{k-1}\end{pmatrix}$

$\boldsymbol{{\sum}_{22}}=\begin{pmatrix}(Y_1-\mu_1)(Y_1-\mu_1)&(Y_1-\mu_1)(Y_2-\mu_2)&\hdots&(Y_1-\mu_1)(Y_j-\mu_j)&\hdots&(Y_1-\mu_1)(Y_{k-1}-\mu_{k-1})\\ (Y_2-\mu_2)(Y_1-\mu_1)&(Y_2-\mu_2)(Y_2-\mu_2)&\hdots&(Y_2-\mu_2)(Y_j-\mu_j)&\hdots&(Y_2-\mu_2)(Y_{k-1}-\mu_{k-1})\\\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\(Y_i-\mu_i)(Y_1-\mu_1)&(Y_i-\mu_i)(Y_2-\mu_2)&\hdots&(Y_i-\mu_i)(Y_j-\mu_j)&\hdots&(Y_i-\mu_i)(Y_{k-1}-\mu_{k-1})\\\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\(Y_{k-1}-\mu_{k-1})(Y_1-\mu_1)&(Y_S-\mu_S)(Y_S-\mu_S)&\hdots&(Y_S-\mu_S)(Y_j-\mu_j)&\hdots&(Y_S-\mu_S)(Y_{k-1}-\mu_{k-1})\end{pmatrix}$

$=\begin{pmatrix}\delta_{y_1y_1}&\delta_{y_1y_2}&\hdots&\delta_{y_1y_j}&\hdots&\delta_{y_1y_{k-1}}\\ \delta_{y_2y_1}&\delta_{y_2y_2}&\hdots&\delta_{y_2y_j}&\hdots&\delta_{y_2y_{k-1}}\\\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\\delta_{y_iy_1}&\delta_{y_iy_2}&\hdots&\delta_{y_iy_j}&\hdots&\delta_{y_iy_{k-1}}\\\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\\delta_{y_{k-1}y_1}&\delta_{y_{k-1}y_2}&\hdots&\delta_{y_{k-1}y_j}&\hdots&\delta_{y_{k-1}y_{k-1}}\end{pmatrix}$