Multivariate Gaussian (normal) pdf
defined as
evaluated at feature vector
, with
mean vector
and
covariance matrix
satisfying:
and
- Also written as
or
, where symbol
means ``has distribution''
-
- Parameters
and
describe center and spread of data
- Gaussian pdf has several excellent properties:
- ``Safest bet'' amongst all pdfs with given
and
- Closed under multiplication (within scale factor):
and
- Closed under convolution (i.e. sum of independent Gaussian variables is Gaussian):
- Consequently, for linear systems,
in
out:
if
- BUT:
(mixture Gaussian instead)
- Marginal and conditional densities of
are
- ``Safest bet'' amongst all pdfs with given
- Parameters
Full covariance Gaussian
is general matrix
model has
parameters
![\fbox{\parbox{0.95\textwidth}{Estimate mean $\boldsymbol{\mu}$\ and covariance......- \frac{1}{2}\ln(\left\vert \boldsymbol{\Sigma} \right\vert)\end{displaymath} }}](http://courses.ee.sun.ac.za/Pattern_Recognition_813/lectures/lecture03/img25.png)
- Models data as ellipsoid-shaped cloud ``at an angle''
allows correlations between feature components
- Most powerful Gaussian model, but needs lots of training data to properly estimate parameters, especially if
high
![\includegraphics[width=7.8cm]{gaussf2}](http://courses.ee.sun.ac.za/Pattern_Recognition_813/lectures/lecture03/img28.png)
![\includegraphics[width=9cm]{gaussf3}](http://courses.ee.sun.ac.za/Pattern_Recognition_813/lectures/lecture03/img27.png)
Diagonal covariance Gaussian
is diagonal matrix
model has
parameters
- Models data as ellipsoid-shaped cloud aligned with axes
correlations between feature components ignored
- Good compromise between modelling power and training data hunger, especially when used in mixture models
![\includegraphics[width=7.8cm]{gaussd2}](http://courses.ee.sun.ac.za/Pattern_Recognition_813/lectures/lecture03/img32.png)
![\includegraphics[width=9cm]{gaussd3}](http://courses.ee.sun.ac.za/Pattern_Recognition_813/lectures/lecture03/img31.png)
Spherical covariance gaussian
(scaled identity)
model has
parameters
![\fbox{\parbox{\textwidth}{Estimate mean $\boldsymbol{\mu}$\ and scalar varianc......} \right\Vert^{2}}{2\sigma^2}-\frac{D}{2}\ln(2\pi\sigma^2)\end{displaymath} }}](http://courses.ee.sun.ac.za/Pattern_Recognition_813/lectures/lecture03/img35.png)
- Models data as spherical cloud
- Simplest Gaussian pdf
useful when data is very scarce or computational speed and storage is an issue
转载地址:http://courses.ee.sun.ac.za/Pattern_Recognition_813/lectures/lecture03/node1.html