常见概率分布

若 $A_1, A_2, A_3$ 相互独立，则 $A_1$ 与 $A_2 \cup A_3$ 相互独立。
证明
$P(A_1 (A_2 \cup A_3)) = P(A_1 A_2 \cup A_1 A_3)$
$= P(A_1 A_2) + P(A_1 A_3) - P(A_1 A_2 A_3)$
$= P(A_1)P(A_2) + P(A_1)P(A_3) - P(A_1)(A_2 A_3)$
$= P(A_1) [P(A_2) + P(A_3) - P(A_2 A_3)]$
$= P(A_1) P(A_2 \cup A_3)$

Discrete Distributions

Binomial Distribution

$\operatorname {Bin} (k ; n, \theta) = \binom { n } { k } \theta ^k (1 - \theta) ^{n - k}, k = 0, ..., n$

Bernoulli Distribution

$\operatorname {Ber} (x ; \theta) = \theta ^k (1 - \theta) ^{1 - k}, k = 0, 1$

Multinomial Distribution

$\operatorname {Mu}(\mathbf{x} | n, \theta) =\binom { n } { x_1, \cdots, x_K } \prod _{j = 1} ^K \theta _j ^{x_j}, \mathbf{x} = \begin{pmatrix} x_1\\ \vdots \\ x_K \end{pmatrix}$

Multinoulli Distribution

$\operatorname {Mu} (x | 1, \theta) = \prod _{j = 1} ^K \theta _j ^{x_j}$

Poisson Distribution

$\operatorname {Poi} (k | \lambda) = e ^{-\lambda} \dfrac {\lambda ^k} {k!}$

Empirical Distribution / Measure

$\operatorname {P} _{emp} (A) = \dfrac {1} {| \mathcal {D} |} \sum _{x \in \mathcal {D} } \Pi (x \in A)$

Continuous Distributions

Gaussian Distribution

$\mathcal {N} (x ; \mu , \sigma ^2) = \dfrac {1} { \sqrt {2 \pi \sigma ^2}} e ^ {- \dfrac {1} {2\sigma ^2} (x - \mu) ^2 }$

$\Phi (x ; \mu , \sigma ^2) = \int _{- \infty} ^x \mathcal {N} (t ; \mu , \sigma ^2) \mathbf{d} t$

$\Phi (x ; \mu , \sigma ^2) = \dfrac {1} {2} [1 + \operatorname {erf} (z / \sqrt 2)], where$
$z = \dfrac {x - \mu } {\sigma}$
$\operatorname {erf} (x) = \dfrac {2} {\sqrt {\pi}} \int _{0} ^x e ^{- t^2} \mathbf{d} t$

Student t Distribution

$\mathcal {T} (t; \mu , \sigma ^2, n)= \dfrac {1} { \sqrt {n \pi } } \dfrac {\Gamma ((n + 1) / 2)}{ \Gamma (n / 2)} { \left [1 + \dfrac {1} {n} ( \dfrac {x - \mu} {\sigma} )^2 \right] } ^ {- \dfrac {n + 1} {2}}$

F Distribution

$U \sim \chi ^2 (n_1), V \sim \chi ^2 (n_2)$
$F = \dfrac {U / n_1} { V / n_2} \sim F(n_1, n_2) :$
$f (x; n_1, n_2) = \dfrac {1} {\operatorname { B } (x; \dfrac {n_1} {2}, \dfrac {n_2} {2})} {\left( \dfrac {n_1} {n_2} \right)} ^ { \dfrac {n_1} {2} } {x} ^ { \dfrac {n_1} {2} - 1} {\left ( 1 + \dfrac {n_1} {n_2} x \right ) } ^ {\dfrac {n_1 + n_2} {2} } \Pi (x \ge 0), where$
$\operatorname {B} (a, b) = \dfrac {\Gamma (a) \Gamma (b)} {\Gamma (a + b)}$

Laplace Distribution

$\operatorname {Lap} (x; \mu , b) = \dfrac {1} {2b} \operatorname { exp } (-\dfrac {| x - \mu |} {b} )$

Gamma Distribution

$\operatorname { Ga } (x; shape = \alpha , rate = \beta ) = \dfrac { \beta ^ \alpha } {\Gamma (\alpha)} x ^ {\alpha - 1} e ^ {- \beta x} \Pi (x \ge 0)$
$\Gamma (x) = \int _{0} ^{+ \infty} t ^ {x - 1} e ^ {- t} d t$

Exponential Distribution

$\operatorname { Expon } (x ; \lambda) = \operatorname { Ga } (x; 1 , \lambda ) = \lambda e ^{- \lambda x} \Pi (x \ge 0)$

Erlang Distribution

$\operatorname { Erlang} (x ; \lambda) = \operatorname { Ga } (x; 2 , \lambda )$

Chi-squared Distribution

$\chi ^ 2 (x; n) = \operatorname { Ga } (x; \dfrac {n} {2}, \dfrac {1} {2}) = \dfrac { 2^ {- \dfrac {n} {2} }} {\Gamma (\dfrac {n} {2})} x ^ {\dfrac {n} {2} - 1} e ^ {- \dfrac {1} {2} x} \Pi (x \ge 0)$

Inverse Gamma Distribution

$If \ X \sim \operatorname { Ga } (x; shape = \alpha , rate = \beta ), then \dfrac {1}{X} \sim \operatorname { IG } (x; shape = \alpha, scale = \beta):$
$\operatorname { IG } (x; shape = \alpha, scale = \beta) = \dfrac { \beta ^ \alpha } {\Gamma (\alpha)} x ^ {- (\alpha + 1)} e ^ {- \beta / x} \Pi (x \gt 0)$

Beta Distribution

$\operatorname { Beta } (x; a, b) = \dfrac {1} {\operatorname {B} (a, b)} x ^ {a - 1} (1 - x) ^ {b - 1}, where$
$\operatorname {B} (a, b) = \dfrac {\Gamma (a) \Gamma (b)} {\Gamma (a + b)}$

Pareto Distribution

$\operatorname { Pareto } (x; k, m) = k m ^ k x ^ {- (k + 1)} \Pi (x \ge m)$

Multivariate Gaussian

$\mathcal {N} (\mathbf {x} ; \mathbf {\mu} , \mathbf {\Sigma}) = \dfrac {1} { { (2 \pi)} ^ {\mathcal {D} / 2} {| \mathbf {\Sigma} |} ^ {1 / 2}} \operatorname {exp} [- \dfrac {1} {2} (\mathbf {x} - \mathbf {\mu}) ^ {\intercal } {\mathbf {\Sigma} } ^ {- 1} (\mathbf {x} - \mathbf {\mu}) ]$

Multivariate Student t Distribution

$\mathcal {T} ((\mathbf {x}; \mathbf {\mu} , \mathbf {\Sigma}, n)= \dfrac {{| \mathbf {\Sigma} |} ^{- 1 / 2} } {{(n \pi )} ^ {\mathcal {D} / 2} } \dfrac {\Gamma ((n + \mathcal {D}) / 2)}{ \Gamma (n / 2) } { \left [1 + \dfrac {1} {n} (\mathbf {x} - \mathbf {\mu}) ^ {\intercal } {\mathbf {\Sigma} } ^ {- 1} (\mathbf {x} - \mathbf {\mu}) \right] } ^ {- \dfrac {n + \mathcal {D}} {2}}$

$\mathbf {\Sigma}: Scale matrix$

Dirichlet Distribution

$\operatorname {Dir} (x ; \alpha) = \dfrac {1} {B(\alpha)} \prod _{k = 1} ^{K} x _k ^{\alpha _k - 1} \Pi (\mathbf {x} \in S _K)$
$S _K = \{ \mathbf {x} : 0 \le x _k \le 1, \sum _{k = 1} ^K x_k = 1 \}$

Multivariate change of variables

$f_\mathbf{y} (\mathbf{y} ) = f _\mathbf{x} (\mathbf{x} ) | \operatorname {det} ( \dfrac {\partial \mathbf{x}} {\partial \mathbf{y}} ) |$