高斯分布_三种类别

最新推荐文章于 2024-07-27 12:09:57 发布

rylq56m9

最新推荐文章于 2024-07-27 12:09:57 发布

阅读量5.2k

点赞数 1

分类专栏：数学分析矩阵计算文章标签：数学分析

数学分析同时被 2 个专栏收录

4 篇文章 0 订阅

订阅专栏

矩阵计算

2 篇文章 0 订阅

订阅专栏

Multivariate Gaussian (normal) pdf

defined as

$\displaystyle p(\boldsymbol{x}\vert\boldsymbol{\mu},\boldsymbol{\Sigma}) \trian......\boldsymbol{\Sigma}^{-1}\left(\boldsymbol{x} - \boldsymbol{\mu}\right)\right\}$

evaluated at $D \times 1$ feature vector $\boldsymbol{x}$ , with $D \times 1$ mean vector $\boldsymbol{\mu}$ and $D \times D$ covariance matrix $\boldsymbol{\Sigma}$ satisfying:

$\displaystyle \boldsymbol{\mu} = \mathcal{E}\left[ \boldsymbol{x} \right]$ and $\displaystyle \quad\boldsymbol{\Sigma} = \mathcal{E}\left[ (\boldsymbol{x} - \boldsymbol{\mu})(\boldsymbol{x} - \boldsymbol{\mu})^T \right]$

Also written as $\mathcal{N}(\boldsymbol{\mu},\boldsymbol{\Sigma})\vert _{\boldsymbol{x}}$ or $\boldsymbol{x} \sim \mathcal{N}(\boldsymbol{\mu},\boldsymbol{\Sigma})$ , where symbol $\sim$ means ``has distribution''
- Parameters $\mu$ and $\Sigma$ describe center and spread of data
- Gaussian pdf has several excellent properties:
  1. ``Safest bet'' amongst all pdfs with given $\boldsymbol{\mu}$ and $\boldsymbol{\Sigma}$
  2. Closed under multiplication (within scale factor):
    
    $\displaystyle \mathcal{N}(\boldsymbol{\mu}_1,\boldsymbol{\Sigma}_1)\times \ma......bol{\Sigma}\boldsymbol{\Sigma}_2^{-1}\boldsymbol{\mu}_2,\,\boldsymbol{\Sigma}),$
    
    with $\boldsymbol{\Sigma}^{-1} = \boldsymbol{\Sigma}_1^{-1} + \boldsymbol{\Sigma}_2^{-1}$ and $C = \mathcal{N}(\boldsymbol{0},\,\boldsymbol{\Sigma}_1 + \boldsymbol{\Sigma}_2)\vert _{\boldsymbol{\mu}_1 - \boldsymbol{\mu}_2}$
  3. Closed under convolution (i.e. sum of independent Gaussian variables is Gaussian):
    
    $\displaystyle \mathcal{N}(\boldsymbol{\mu}_1,\boldsymbol{\Sigma}_1)\ast \math......{\mu}_1 + \boldsymbol{\mu}_2,\,\boldsymbol{\Sigma}_1 + \boldsymbol{\Sigma}_2)$
  4. Consequently, for linear systems, $\mathcal{N}$ in $\Longrightarrow$ $\mathcal{N}$ out:
    
    $\displaystyle \boldsymbol{A}\boldsymbol{x} \sim \mathcal{N}(\boldsymbol{A}\boldsymbol{\mu},\boldsymbol{A}\boldsymbol{\Sigma}\boldsymbol{A}^H)$ if $\displaystyle \:\:\:\:\boldsymbol{x} \sim \mathcal{N}(\boldsymbol{\mu},\boldsymbol{\Sigma})$
  5. BUT: $\mathcal{N} + \mathcal{N} \neq \mathcal{N}$ (mixture Gaussian instead)
  6. Marginal and conditional densities of $\mathcal{N}$ are $\mathcal{N}$

Full covariance Gaussian

$\boldsymbol{\Sigma}$ is general matrix $\Longrightarrow$ model has $D + \frac{D(D+1)}{2}$ parameters

$\fbox{\parbox{0.95\textwidth}{Estimate mean $\boldsymbol{\mu}$\ and covariance......- \frac{1}{2}\ln(\left\vert \boldsymbol{\Sigma} \right\vert)\end{displaymath} }}$

Models data as ellipsoid-shaped cloud ``at an angle'' $\Longrightarrow$ allows correlations between feature components
Most powerful Gaussian model, but needs lots of training data to properly estimate parameters, especially if high

$\includegraphics[width=7.8cm]{gaussf2}$ $\includegraphics[width=9cm]{gaussf3}$

Diagonal covariance Gaussian

$\boldsymbol{\Sigma}$ is diagonal matrix $\Longrightarrow$ model has parameters
$\fbox{\parbox{\textwidth}{Estimate mean $\boldsymbol{\mu}$\ and $D \times 1$\ ......ac{D}{2}\ln(2\pi) - \frac{1}{2}\sum_{i=1}^{D}\ln(\sigma_i^2)\end{displaymath} }}$

Models data as ellipsoid-shaped cloud aligned with axes $\Longrightarrow$ correlations between feature components ignored
Good compromise between modelling power and training data hunger, especially when used in mixture models

$\includegraphics[width=7.8cm]{gaussd2}$ $\includegraphics[width=9cm]{gaussd3}$

Spherical covariance gaussian

$\boldsymbol{\Sigma} = \sigma^2\boldsymbol{I}$ (scaled identity) $\Longrightarrow$ model has parameters

$\fbox{\parbox{\textwidth}{Estimate mean $\boldsymbol{\mu}$\ and scalar varianc......} \right\Vert^{2}}{2\sigma^2}-\frac{D}{2}\ln(2\pi\sigma^2)\end{displaymath} }}$

Models data as spherical cloud
Simplest Gaussian pdf $\Longrightarrow$ useful when data is very scarce or computational speed and storage is an issue

$\includegraphics[width=7.8cm]{gausss2}$ $\includegraphics[width=9cm]{gausss3}$

转载地址：http://courses.ee.sun.ac.za/Pattern_Recognition_813/lectures/lecture03/node1.html

rylq56m9

关注

1
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
高斯分布_三种类别

Multivariate Gaussian (normal) pdf defined asevaluated at feature vector , with mean vector and covariance matrix satisfying: andAlso written as or , where symbol m
复制链接

扫一扫

专栏目录