Metropolis-Hasting involves designing a Markov process by constructing Transition Probabilities, which has a unique stationary distribution π ( x ) \pi(x) π(x) if it fulfills two conditions:
- Existitence of Stationary Distribution:
Reason----Detailed Balance:
π ( x ) P ( x ′ ∣ x ) = π ( x ′ ) P ( x ∣ x ′ ) \pi(x) P(x' | x) = \pi(x')P(x|x') π(x)P(x′∣x)=π(x′)P(x∣x′) - Uniqueness of Stationary Distribution:
Reason – Ergodicity: every states satisfies
1 Aperiodic: the system does not return to the same state at fixed intervals----
2 Be positive recurrent - the expected number of steps for returning to the same state is finite.
The derivation of the algorithm starts with the condition of detailed balance.
P ( x ′ ∣ x ) P ( x ) = P ( x ′ , x ) =