用tensorflow构建多元高斯分布和高斯混合分布

1. 用tensorflow构建多元高斯分布

• 为了更好的使用概率知识，tensorflow专门建立了一个tensorflow_probability库用来封装各种概率模型。

1.1 准备工作

import tensorflow as tf 
import tensorflow_probability as tfp

tfd = tfp.distributions

1.2 构建多元高斯分布的函数

1.2.1 解释文档

tfd.MultivariateNormalDiag(
    loc=None, # 均值
    scale_diag=None, # 方差
    scale_identity_multiplier=None,
    validate_args=False,
    allow_nan_stats=True,
    name='MultivariateNormalDiag',
)

Args:
  loc: Floating-point `Tensor`. If this is set to `None`, `loc` is
    implicitly `0`. When specified, may have shape `[B1, ..., Bb, k]` where
    `b >= 0` and `k` is the event size.
  scale_diag: Non-zero, floating-point `Tensor` representing a diagonal
    matrix added to `scale`. May have shape `[B1, ..., Bb, k]`, `b >= 0`,
    and characterizes `b`-batches of `k x k` diagonal matrices added to
    `scale`. When both `scale_identity_multiplier` and `scale_diag` are
    `None` then `scale` is the `Identity`.
  scale_identity_multiplier: Non-zero, floating-point `Tensor` representing
    a scaled-identity-matrix added to `scale`. May have shape
    `[B1, ..., Bb]`, `b >= 0`, and characterizes `b`-batches of scaled
    `k x k` identity matrices added to `scale`. When both
    `scale_identity_multiplier` and `scale_diag` are `None` then `scale` is
    the `Identity`.
  validate_args: Python `bool`, default `False`. When `True` distribution
    parameters are checked for validity despite possibly degrading runtime
    performance. When `False` invalid inputs may silently render incorrect
    outputs.
  allow_nan_stats: Python `bool`, default `True`. When `True`,
    statistics (e.g., mean, mode, variance) use the value '`NaN`' to
    indicate the result is undefined. When `False`, an exception is raised
    if one or more of the statistic's batch members are undefined.
  name: Python `str` name prefixed to Ops created by this class.

• Docstrings文档
  此函数可以表达K-rank的多元高斯分布。即我们有长度为k的均值，和k*k的标准差矩阵(‘scale matrix’)。covariance = scale @ scale.T where @ denotes matrix-multiplication.
• 需要注意的是，我们可以加入batch维度，就像上面参数解释一样，loc的维度可以为(B1,B2…,Bb,k),k是event_size。一般我们如果在训练模型的时候要考虑batch_size的维度
• 此函数创立的是各维度之间独立，即各维度之间的协方差为0 的高斯分布，也叫diagonal gussian distribution。

1.2.2 举个例子

• 初始化一个batch为2，变量数为3的多元高斯分布

# Initialize a 2-batch of 3-variate Gaussians.
mvn = tfd.MultivariateNormalDiag(
    loc=[[1., 2, 3],
         [11, 22, 33]]   ,        # shape: [2, 3]
    scale_diag=[[1., 2, 3],
                [0.5, 1, 1.5]])  # shape: [2, 3]

# 我们可以评估一点在该多元高斯分布中的概率值
# Evaluate this on a two observations, each in `R^3`, returning a length-2
# vector.
x = [[-1., 0, 1],
     [-11, 0, 11.]]   # shape: [2, 3].
     
# with batch_size
print("某点的概率值：\n",mvn.prob(x).numpy()) #shape:[2]
print("均值：\n",mvn.mean().numpy())
print("标准差：\n",mvn.stddev().numpy())

# output
某点的概率值：
 [0.00069556 0.        ]
均值：
 [[ 1.  2.  3.]
 [11. 22. 33.]]
标准差：
 [[1.  2.  3. ]
 [0.5 1.  1.5]]       

# 当然我们也可以从该分布中采样
samples = mvn._sample_n(16,seed=1)   # 从该分布中采样16个样本 
# 重点看维度
samples.shape   # shape=(16,2,3) 

• 知识点：当从分布中采样n个样本时，我们样本的维度为(n,batch_size,event_size)，event_size指的是数据的维度。

2 用tensorflow构建高斯混合分布

• 待更新