MCMC sampling introduction

MCMC是贝叶斯常用的抽样算法之一，主要包括MH算法和Gibbs抽样等。MCMC是基于趋于平稳分布的Markov链，本文主要介绍了Markov链的基本性质和平稳分布。

1 Motivation

1.1 Monte Carlo (MC)

where is iid sample from the posterior distribution .

However, obtaining the iid sample is difficult, and the complete form of the posterior distribution must be known.

1.2 MCMC (Markov Chain MC)

MCMC generally generates a random sequence satisfying markov properties, which is ergodic and the limit distribution is . Based on Markov chains, get "not independent" samples from that have the same effect as iid sample.

2 Basic properties

The Markov chain satisfies: stationary distribution, ergodic (or irreducile and aperiodic).

• Markov property

• Transition Probability (kernel) Homogeneous: the transition kernel are the same for all .

• Marginal Distribution

3. Stationary (Invariant) Chains

A -finite measure is invariant for the transition kernel (and for the associated chain) if

The invariant distribution is also referred to as stationary if is a probability measure, since implies that for every ; thus, the chain is stationary in distribution.

4. Detailed balance condition

Detailed balance condition

A Markov chain with transition kernel satisfies the detailed balance condition (reversible) if there exists a function satisfying

for every .

Proof.

Remark：
Detailed balance condition provides a sufficient but not necessary condition for to be a stationary measure associated with the transition kernel .

Theorem

Suppose that a Markov chain with transition function satisfies the detailed balance condition with a probability density function. Then:
(1) The density is the invariant density of the chain.
(2) The chain is reversible.

Reversible:

5. Ergodicity

To prove the convergence (independence of initial conditions)

• positive recurrent
  State is positive recurrent, if where is the first time to return state .

• aperiodic
  A state has period ,

where gcd is greatest common divisor. Aperiodic means the gcd of any state is 1.

6. Irreducible

For any state , the probability of this chain going from state to state is positive. Namely, for ,

The law of large numbers for Markov chains

Suppose is a Markov chain with countable state space , and the transition probability matrix is , and suppose that it is irreducible and has stationary distribution . Then for any bounded functions and any initial distribution, have

Remark.

• A given Markov chain may have more than one invariant distribution.

• Stationary + ergodicity equilibrium distribution (unique)

• Stationary + irreducible + aperiodic unique(A ergodic Markov chain is irreducible. )