Bernoulli-Gaussian分布

本文主要内容来自：S.Som and L.C.Potter, Sparsity Pattern Recovery in Bernoulli-Gaussian Signal Model.

---

  对于稀疏信号$\mathbf{x}\in {\mathbb{R}}^N$，我们考虑概率信号模型。假定集合$\mathcal{S}$的元素是从集合 I = { 1 , 2 , … , N } {\mathcal I}=\{1,2,\ldots,N\} I={1,2,…,N}中抽取出来的，其抽取方式是，集合$\mathcal{I}$中每个元素i.i.d.地被包含于$\mathcal{S}$中（概率为$p\ll 1$）。因此，支持集$S$的基数等于$K$的概率由下式给出
$$\mathbb{P}[|\mathcal{S}|=K]=\left(\genfrac{}{}{0ex}{}{N}{K}\right)p^K(1-p{)}^{N-K}.$$为了强化稀疏性，我们也假定$p<\frac{1}{2}$。如果$\mathbf{x}$中某个元素的序号不在集合$\mathcal{S}$中，则该元素为零；否则，该元素为高斯分布，均值为${\mu }_1$，方差为${\sigma }_1^2$，即：
x i = { 0 ,   i ∈ ̸ S ∼ N ( μ 1 , σ 1 2 ) , i ∈ S ∀ i = 1 , 2 , . . . N . x_i={\Large\{}\begin{aligned} 0,&amp;\ i \in \not{\mathcal S}\\ \sim{\mathcal N}(\mu_1,\sigma_1^2),&amp;i\in {\mathcal S} \end{aligned}\quad \forall i=1,2,...N. xi​={0,∼N(μ1​,σ12​),​ i∉​Si∈S​∀i=1,2,...N.如果用${\mathbf{x}}_{\mathcal{S}}$表示所有序号在$\mathcal{S}$中的元素所构成的集合，则${\mathbf{x}}_{\mathcal{S}}$满足分布：${\mathbf{x}}_{\mathcal{S}}\sim \mathcal{N}({\mu }_1{1}_{|\mathcal{S}|},{\sigma }_1^2{\mathbf{I}}_{|\mathcal{S}|})$。因此，$\mathcal{S}$为信号向量$\mathbf{x}$的支持集，期望基数为$\mathbb{E}[|\mathcal{S}|]=N_p\ll N$，并且$\mathbf{x}$为稀疏的概率很大。这种Bernoulli-Gaussian模型在文献中非常常见[17-19]，用来在不同应用领域中建模稀疏向量，而在压缩领域研究中，也越来越常用[10,20,21]。

[17] W. Cui and E. George, “Empirical Bayes vs. fully Bayes variable selection,” Journal of Statistical Planning and Inference, vol. 138, pp. 888–900, 2008.
[18] E. George and D. Foster, “Calibration and empirical Bayes variable selection,” Biometrika, vol. 88, pp. 731–747, 2000.
[19] I. M. Johnstone and B. W. Silverman, “Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences,” The Annals of Statistics, vol. 32, pp. 1594–1649, Aug. 2004.
[10] D. Guo, D. Baron, and S. Shamai, “A single-letter characterization of optimal noisy compressed sensing,” in Proc. 47th Allerton Conference on Communication, Control, and Computing, (Monticello, IL), September 2009.
[20] P. Schniter, L. C. Potter, and J. Ziniel, “Fast Bayesian matching pursuit: Model uncertainty and parameter estimation for sparse linear models,” Preprint, Aug. 2008.
[21] D. Baron, S. Sarvotham, and R. G. Baraniuk, “Bayesian compressive sensing via belief propagation,” To appear in IEEE Trans. Signal Processing, 2009.