频率主义（Frequentism）与贝叶斯主义（Bayesianism）的哲学辨异与实践（Python仿真）

对概率的认识

两者的根本不同在于对概率的认识：

• frequentists
  probabilities are fundamentally related to frequencies of events.

• Bayesians,
  probabilities are fundamentally related to our own knowledge about an event.

频率

We’ll start with the classical frequentist maximum likelihood approach.

$$P(\mathcal{D}|F_{true})$$

$$\frac{d\log\mathcal{L}}{dF_{true}}=0$$

贝叶斯

The Bayesian approach, as you might expect, begins and ends with probabilities.
It recognizes that what we fundamentally want to compute is our knowledge of the parameters in question, i.e. in this case,

$$P(F_{true}|\mathcal{D})=\frac{P(F_{true})P(\mathcal{D}|P(F_{true}))}{P(\mathcal{D})}$$

$P(F_{true}|\mathcal{D})$中的$\mathcal{D}$ 表示数据或者抽象地说我们掌握的知识，
$P(F_{true})$，模型的先验知识，which encodes what we knew about the model prior to the application of the data $\mathcal D$。

Note that this formulation of the problem is fundamentally contrary to the frequentist philosophy, which says that probabilities have no meaning for model parameters like Ftrue.
Nevertheless, within the Bayesian philosophy this is perfectly acceptable.

noninformative prior like the flat prior

It turns out that in many situations, a truly noninformative prior does not exist!

Frequentism can often be viewed as simply a special case of the Bayesian approach for some (implicit) choice of the prior: a Bayesian would say that it’s better to make this implicit choice explicit, even if the choice might include some subjectivity.

Photon counts: the Bayesian approach

For a one parameter problem like the one considered here, it’s as simple as computing the posterior probability $P(F_{true}|\mathcal{D})$ as a function of $F_{true}$: this is the distribution reflecting our knowledge of the parameter $F_{true}$.

But as the dimension of the model grows, this direct approach becomes increasingly intractable. For this reason, Bayesian calculations often depend on sampling methods such as Markov Chain Monte Carlo (MCMC).

This maximum likelihood value gives our best estimate of the parameters $\mu$ and $\sigma$ governing our model of the source. But this is only half the answer: we need to determine how confident we are in this answer, that is, we need to compute the error bars on $\mu$ and $\sigma$.