Exponential family form of Multivariate Gaussian Distribution

最新推荐文章于 2022-10-09 15:32:12 发布

空言葉

最新推荐文章于 2022-10-09 15:32:12 发布

阅读量302

点赞数

分类专栏：数据学习文章标签：指数族多元高斯分布

本文链接：https://blog.csdn.net/kongyanye/article/details/78488091

版权

数据学习专栏收录该内容

1 篇文章 0 订阅

订阅专栏

Question

Exponential Family. Verify that the multivariate Gaussian distribution can be casted in exponential family form and derive expressions for $\eta$ , $b(\eta)$ , $T(\eta)$ , $a(\eta)$ .

Solution

The exponential family form is

p (y; η) = b (y) exp [η T T (y) - a (η)]

$p(y;\eta)=b(y)\exp[\eta^TT(y)-a(\eta)]$
The multivariate Gaussian distribution takes the form

N (x | μ, Σ) = 1 ( 2 π ) D / 2 | Σ | 1 / 2 exp [- 1 2 (x - μ) T Σ - 1 (x - μ)]

$\begin{equation} N(x|\mu, \Sigma)=\frac{1}{(2\pi)^{D/2}|\Sigma|^{1/2}}\exp[-\frac{1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu)] \end{equation}$
Since

- 1 2 (x - μ) T Σ - 1 (x - μ) = - 1 2 (x T - μ T) Σ - 1 (x - μ) = - 1 2 (x T Σ - 1 x - x T Σ - 1 μ - μ T Σ - 1 x + μ T Σ - 1 μ) = - 1 2 (x T Σ - 1 x - 2 μ T Σ - 1 x + μ T Σ - 1 μ) = - 1 2 (v e c (Σ - 1) v e c (x x T) - 2 μ T Σ - 1 x + μ T Σ - 1 μ) = - 1 2 ([v e c (Σ - 1) - 2 μ T Σ - 1] [v e c (x x T) x] + μ T Σ - 1 μ) = [- 1 2 v e c (Σ - 1) μ T Σ - 1] [v e c (x x T) x] - 1 2 μ T Σ - 1 μ

$\begin{equation} \begin{aligned} -\frac{1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu)&=-\frac{1}{2}(x^T-\mu^T)\Sigma^{-1}(x-\mu)\\ &=-\frac{1}{2}(x^T\Sigma^{-1}x-x^T\Sigma^{-1}\mu-\mu^T\Sigma^{-1}x+\mu^T\Sigma^{-1}\mu)\\ &=-\frac{1}{2}(x^T\Sigma^{-1}x-2\mu^T\Sigma^{-1}x+\mu^T\Sigma^{-1}\mu)\\ &=-\frac{1}{2}(vec(\Sigma^{-1})vec(xx^T)-2\mu^T\Sigma^{-1}x+\mu^T\Sigma^{-1}\mu)\\ &=-\frac{1}{2}(\left[\begin{array}{cc}vec(\Sigma^{-1})&-2\mu^T\Sigma^{-1}\end{array}\right] \left[\begin{array}{c} vec(xx^T)\\x\end{array}\right]+\mu^T\Sigma^{-1}\mu )\\ &=\left[\begin{array}{cc}-\frac{1}{2}vec(\Sigma^{-1})&\mu^T\Sigma^{-1}\end{array}\right] \left[\begin{array}{c} vec(xx^T)\\x\end{array}\right]-\frac{1}{2}\mu^T\Sigma^{-1}\mu \end{aligned} \end{equation}$
The 3rd equation is due to
(

Trace $Trace$ of a real number is still a real number and

Σ−1 $\Sigma^{-1}$ is a symmetric matrix)

x T Σ - 1 μ = T r (x T Σ - 1 μ) = T r (x T Σ - 1 μ) T = T r (μ T Σ - T x) = T r (μ T Σ - 1 x) = μ T Σ - 1 x

$\begin{equation} \begin{aligned} x^T\Sigma^{-1}\mu&=Tr(x^T\Sigma^{-1}\mu)=Tr(x^T\Sigma^{-1}\mu)^T\\&=Tr(\mu^T\Sigma^{-T}x)=Tr(\mu^T\Sigma^{-1}x)\\ &=\mu^T\Sigma^{-1}x \end{aligned} \end{equation}$
The 4th equation is due to

xTΣ−1x=vec(Σ−1)Tvec(xxT) $x^T\Sigma^{-1}x=vec(\Sigma^{-1})^Tvec(xx^T)$ , where

vec $vec$ means stack all columns of a matrix into one single column, i.e. if

x=⎡⎣⎢|x1||x2||…||xm|⎤⎦⎥ $x=\left[\begin{array}{cccc}|&|&|&|\\x_1&x_2&\ldots&x_m\\|&|&|&|\end{array}\right]$ , then

v e c (x) = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ | x 1 | x 2 | ⋮ | x m | ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥

$vec(x)=\left[\begin{array}{c}|\\x_1\\|\\x_2\\|\\\vdots\\|\\x_m\\|\end{array}\right]$
So we can derive the parameter of exponential family as below:

b (y) = 1 ( 2 π ) D / 2 η = ⎡ ⎣ - 1 2 v e c (Σ - 1) μ T Σ - 1 ⎤ ⎦ T (y) = [v e c (x x T) x] a (η) = 1 2 μ T Σ - 1 μ - ln | Σ | 1 / 2

$\begin{equation} \begin{aligned} &b(y)=\frac{1}{(2\pi)^{D/2}}\\ &\eta=\left[\begin{array}{c}-\frac{1}{2}vec(\Sigma^{-1})\\\mu^T\Sigma^{-1}\end{array}\right]\\ &T(y)=\left[\begin{array}{c} vec(xx^T)\\x\end{array}\right]\\ &a(\eta)=\frac{1}{2}\mu^T\Sigma^{-1}\mu-\ln|\Sigma|^{1/2} \end{aligned} \end{equation}$

空言葉

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
2
评论
Exponential family form of Multivariate Gaussian Distribution

QuestionExponential Family. Verify that the multivariate Gaussian distribution can be casted in exponential family form and derive expressions for η\eta, b(η)b(\eta), T(η)T(\eta), a(η)a(\eta).
复制链接

扫一扫