指数分布族的两个性质-CSDN博客

本文链接：https://blog.csdn.net/xyhs2010/article/details/74546316

（《统计推断》3.4）

称一个概率密度函数族或概率质量函数族为指数族（exponential family），如果它能统一表示为：

f (x | θ) = h (x) c (θ) e x p (\sum i = 1 k ω i (θ t i (x)))

$f(x|{\mathbf \theta}) = h(x) c({\mathbf \theta}) exp(\sum_{i=1}^k \omega_i({\mathbf \theta} t_i(x)))$

指数分布族有两个性质：

E (\sum i = 1 k \partial ω i ( θ ) \partial θ j t i (X)) = - \partial \partial θ j l o g c (θ)

$E(\sum_{i=1}^k \frac{\partial\omega_i({\mathbf \theta})}{\partial{\mathbf \theta}_j}t_i(X)) = -\frac{\partial}{\partial {\mathbf \theta}_j} log c({\mathbf \theta})$

V a r (\sum i = 1 k \partial ω i ( θ ) \partial θ j t i (X)) = - \partial 2 \partial θ 2 j l o g c (θ) - E (\sum i = 1 k \partial 2 ω i ( θ ) \partial θ 2 j t i (X))

$Var(\sum_{i=1}^k \frac{\partial\omega_i({\mathbf \theta})}{\partial{\mathbf \theta}_j}t_i(X)) = - \frac{\partial^2}{\partial {\mathbf \theta}_j^2}log c({\mathbf \theta}) - E(\sum_{i=1}^k \frac{\partial^2 \omega_i({\mathbf \theta})}{\partial{\mathbf \theta}_j^2}t_i(X))$

这里给出其证明。首先，记

k (x, θ) = \sum i = 1 k ω i (θ t i (x)

$k(x,{\mathbf \theta}) = \sum_{i=1}^k \omega_i({\mathbf \theta} t_i(x)$

K (x, θ) = e x p (k (x, θ))

$K(x,{\mathbf \theta}) = exp(k(x,{\mathbf \theta}))$

第一个式子

$E(\sum_{i=1}^k \frac{\partial\omega_i({\mathbf \theta})}{\partial{\mathbf \theta}_j}t_i(X))$

$=E(\frac{\partial}{\partial{\mathbf \theta}_j}k(x,{\mathbf \theta}))$

$=\int h(x)c({\mathbf \theta})exp(k(x,{\mathbf \theta})) \frac{\partial}{\partial{\mathbf \theta}_j}k(x,{\mathbf \theta}) dx$

$=\int h(x)c({\mathbf \theta}) \frac{\partial}{\partial{\mathbf \theta}_j} K(x,{\mathbf \theta}) dx$

$=c({\mathbf \theta}) \int \frac{\partial}{\partial{\mathbf \theta}_j} (\frac{1}{c({\mathbf \theta})} c({\mathbf \theta}))h(x)K(x,{\mathbf \theta}) dx$

$=c({\mathbf \theta}) \frac{\partial}{\partial{\mathbf \theta}_j} \frac{1}{c({\mathbf \theta})} \int c({\mathbf \theta})h(x)K(x,{\mathbf \theta}) dx$

$=c({\mathbf \theta}) \frac{\partial}{\partial{\mathbf \theta}_j} \frac{1}{c({\mathbf \theta})}$

接下来考虑到：

$0=\frac{\partial}{\partial x}1$

$=\frac{\partial}{\partial x} (f(x)\frac{1}{f(x)})$

$=\frac{1}{f(x)}\frac{\partial}{\partial x} f(x) + f(x)\frac{\partial}{\partial x} \frac{1}{f(x)}$

$=\frac{\partial}{\partial x} log f(x) + f(x)\frac{\partial}{\partial x} \frac{1}{f(x)}$

因而

$c({\mathbf \theta}) \frac{\partial}{\partial{\mathbf \theta}_j} \frac{1}{c({\mathbf \theta})}$

$=-\frac{\partial}{\partial{\mathbf \theta}_j} log c({\mathbf \theta})$

证毕。

第二个式子

$Var(\sum_{i=1}^k \frac{\partial\omega_i({\mathbf \theta})}{\partial{\mathbf \theta}_j}t_i(X))$

$=Var(\frac{\partial}{\partial{\mathbf \theta}_j}k(x,{\mathbf \theta}))$

$=E((\frac{\partial}{\partial{\mathbf \theta}_j}k(x,{\mathbf \theta}))^2) - E(\frac{\partial}{\partial{\mathbf \theta}_j}k(x,{\mathbf \theta}))^2$

$=\int h(x)c({\mathbf \theta}) exp(k(x, {\mathbf\theta})) (\frac{\partial}{\partial{\mathbf \theta}_j}k(x,{\mathbf \theta}))^2 dx - (c({\mathbf \theta}) \frac{\partial}{\partial{\mathbf \theta}_j} \frac{1}{c({\mathbf \theta})})^2$

减号右边

$(c({\mathbf \theta}) \frac{\partial}{\partial{\mathbf \theta}_j} \frac{1}{c({\mathbf \theta})})^2$

$=(c({\mathbf \theta}) \frac{\partial}{\partial{\mathbf \theta}_j} \frac{1}{c({\mathbf \theta})})(c({\mathbf \theta}) \frac{\partial}{\partial{\mathbf \theta}_j} \frac{1}{c({\mathbf \theta})})$

$=-(\frac{1}{c({\mathbf \theta})} \frac{\partial}{\partial{\mathbf \theta}_j} c({\mathbf \theta}))(c({\mathbf \theta}) \frac{\partial}{\partial{\mathbf \theta}_j} \frac{1}{c({\mathbf \theta})})$

$=-(\frac{\partial}{\partial{\mathbf \theta}_j} c({\mathbf \theta}))(\frac{\partial}{\partial{\mathbf \theta}_j} \frac{1}{c({\mathbf \theta})})$

减号左边，考虑

$\frac{\partial^2}{\partial x^2} exp(f(x))$

$= exp(f(x))(\frac{\partial^2}{\partial x^2} f(x)+(\frac{\partial}{\partial x} f(x))^2)$

因而

$\int h(x)c({\mathbf \theta}) exp(k(x, {\mathbf\theta})) (\frac{\partial}{\partial{\mathbf \theta}_j}k(x,{\mathbf \theta}))^2 dx$

$=\int h(x)c({\mathbf \theta}) \frac{\partial^2}{\partial{\mathbf \theta}_j^2}K(x,{\mathbf \theta}) dx - \int h(x)c({\mathbf \theta}) K(x, {\mathbf\theta}) \frac{\partial^2}{\partial{\mathbf \theta}_j^2}k(x,{\mathbf \theta}) dx$

$=c({\mathbf \theta}) \frac{\partial^2}{\partial{\mathbf \theta}_j^2} \frac{1}{c({\mathbf \theta})} - E(\frac{\partial^2}{\partial{\mathbf \theta}_j^2}k(x,{\mathbf \theta}))$

结合两部分，得到：

$Var(\sum_{i=1}^k \frac{\partial\omega_i({\mathbf \theta})}{\partial{\mathbf \theta}_j}t_i(X))$

$=c({\mathbf \theta}) \frac{\partial^2}{\partial{\mathbf \theta}_j^2} \frac{1}{c({\mathbf \theta})} + (\frac{\partial}{\partial{\mathbf \theta}_j} c({\mathbf \theta})) (\frac{\partial}{\partial{\mathbf \theta}_j} \frac{1}{c({\mathbf \theta})}) - E(\frac{\partial^2}{\partial{\mathbf \theta}_j^2}k(x,{\mathbf \theta}))$

考虑：

$0=\frac{\partial^2}{\partial x^2}1$

$=\frac{\partial}{\partial x} (\frac{1}{f(x)}\frac{\partial}{\partial x} f(x) + f(x)\frac{\partial}{\partial x} \frac{1}{f(x)})$

$=\frac{1}{f(x)}\frac{\partial^2}{\partial x^2} f(x) + 2(\frac{\partial}{\partial x} f(x))(\frac{\partial}{\partial x} \frac{1}{f(x)}) + f(x)\frac{\partial^2}{\partial x^2} \frac{1}{f(x)}$

$=\frac{1}{f(x)}\frac{\partial^2}{\partial x^2} f(x) + (\frac{\partial}{\partial x} f(x))(\frac{\partial}{\partial x} \frac{1}{f(x)}) + \frac{\partial^2}{\partial x^2} log f(x)$

因而

$Var(\sum_{i=1}^k \frac{\partial\omega_i({\mathbf \theta})}{\partial{\mathbf \theta}_j}t_i(X))$

$=- \frac{\partial^2}{\partial {\mathbf \theta}_j^2}log c({\mathbf \theta}) - E(\frac{\partial^2}{\partial{\mathbf \theta}_j^2}k(x,{\mathbf \theta}))$

$= - \frac{\partial^2}{\partial {\mathbf \theta}_j^2}log c({\mathbf \theta}) - E(\sum_{i=1}^k \frac{\partial^2 \omega_i({\mathbf \theta})}{\partial{\mathbf \theta}_j^2}t_i(X))$