第一章 贝叶斯推断指南

Bayesian Analysis-Chapter1 贝叶斯推断指南

Exercise

1. 修改本章第三个图的代码，在图中增加一条竖线用来表示观测到的正面向上的比例，将该竖线位置与每个子图中后验众数进行比较。

import matplotlib.pyplot as plt
import numpy as np
from scipy import stats
import seaborn as sns
# 参数选择
theta_real = 0.35
trials = [0, 1, 2, 3, 4, 8 , 16, 32, 50, 150]
data =   [0, 1, 1, 1, 1, 4, 6, 9, 13, 48]
# 设计三种不同的先验分布参数
beta_params = [(1, 1), (0.5, 0.5), (20, 20)]
dist = stats.beta
x = np.linspace(0, 1, 100)

for idx, N in enumerate(trials):
    if(idx == 0):
        # 画4行3列子图的第二个
        plt.subplot(4, 3, 2)
    else:
        plt.subplot(4, 3, idx + 3)
    y = data[idx]
    for(a_prior, b_prior), c in zip(beta_params, ('b', 'r', 'g')):
        # 由beta函数后验公式推出：
        p_theta_given_y = dist.pdf(x, a_prior + y, b_prior + N - y)
        plt.plot(x, p_theta_given_y, c)
        # 将曲线和横坐标间填充颜色
        plt.fill_between(x, 0, p_theta_given_y, color = c, alpha = 0.6)
    plt.axvline(theta_real, ymax = 0.5, color = 'k')
    # excerise
    if(idx != 0):
        plt.axvline(data[idx] / trials[idx], ymax = 0.5, color = 'b')
#     plt.plot(0, 0, label = "{:d} experience \n{:d} heads".format(N, y), alpha = 0)
    plt.xlim(0, 1)
    plt.ylim(0, 12)
    plt.xlabel(r"$\theta$")
    plt.legend()
    plt.gca().axes.get_yaxis().set_visible(False)
plt.tight_layout()
plt.show()
plt.savefig('base experience.png', dpi = 300, figsize = (5.5, 5.5))

2. 尝试用不同的先验参数（beta值）和不同的实验数据（正面朝上的次数和实验次数）重新绘制本章的第三个图。
（修改上面代码中的参数选择部分即可）。
3. Cromwell 准则
在wiki上查找克伦威尔准则（Cromwell Rule）得到以下内容：

Cromwell’s rule, named by statistician Dennis Lindley,[1] states that the use of prior probabilities of 1 (“the event will definitely occur”) or 0 (“the event will definitely not occur”) should be avoided, except when applied to statements that are logically true or false, such as 2+2 equaling 4 or 5.

其含义是指：除非是对于逻辑上正确或错误的断言，应当避免使用概率值为1或概率值为0的先验。

As Lindley puts it, assigning a probability should “leave a little probability for the moon being made of green cheese; it can be as small as 1 in a million, but have it there since otherwise an army of astronauts returning with samples of the said cheese will leave you unmoved.”[3] Similarly, in assessing the likelihood that tossing a coin will result in either a head or a tail facing upwards, there is a possibility, albeit remote, that the coin will land on its edge and remain in that position.

我们可以为“月球是由绿色奶酪制成的”这一个断言分配一个非常小的概率（比如1/100000000），但这仍然保持这个断言为真的概率，否则宇航员返回上述样本奶酪会让你无动于衷。同样，在评估投掷硬币会导致头部或尾部朝上的可能性时，有可能虽然很遥远，但仍有可能发生。

If the prior probability assigned to a hypothesis is 0 or 1, then, by Bayes’ theorem, the posterior probability (probability of the hypothesis, given the evidence) is forced to be 0 or 1 as well; no evidence, no matter how strong, could have any influence.

如果将先验概率设定为0或者1，由贝叶斯定理

$$P(H|D)=\frac{P(D|H)\times P(H)}{P(D)}\text{\hspace{1em}(1)}$$

我们得到的后验概率总是为0或者1（i.e. $P(H|D)=1\to P(H)=1$），无论我们得到了关于事实的怎样有力的证据。

A strengthened version of Cromwell’s rule, applying also to statements of arithmetic and logic, alters the first rule of probability, or the convexity rule, 0 ≤ Pr(A) ≤ 1, to 0 < Pr(A) < 1.

克伦威尔规则的强化版本，也适用于算术和逻辑的陈述上。改变概率的第一个性质即$0\le Pr（A）\le 1⟶0<Pr（A）<1$。

4. 探索不同参数下高斯分布，二项分布和beta分布的图像
   这里以beta分布为例：

a_params = np.linspace(1, 20, 20)
b_params = np.linspace(1, 20, 20)
dist = stats.beta
x = np.linspace(0, 1, 100)

for a in a_params:
    for b in b_params:
        print("a = {:.2f},b = {:.2f}".format(a,b))
        y = dist.pdf(x, a, b)
        plt.plot(x, y, 'b')
        plt.plot(0, 0, label="\alpha = {:.2f}\n \beta = {:.2f}".format(a, b), alpha = 0.6)
        plt.xlim(0, 1)
        plt.ylim(0, 12)
        plt.legend()
        plt.show()

这样我们就可以得到我们的 α , β = { 1 , 2 , … 20 } \alpha,\beta = \{1,2,\dots20\} α,β={1,2,…20}的全部beta分布了。