Some symbols in stochastic process

$p_{ij}=P(X_{n+1}=j\mid X_n=i)$

If${\alpha }_i=P(X_0=i)$,then $P(X_0=i_0,\dots ,X_n=i_n)={\alpha }_{i_0}{P}_{i_0,i_1}\cdots P_{i_{n-1}{i}_n}$

$P_{i,j}^n$: $(i,j)th$ entry of $P^n$

τ = m i n { n ≥ 1 : X n ∈ A } = \tau=min\{ n\geq 1: X_n \in A\}= τ=min{n≥1:Xn​∈A}=: hitting time

τ j = m i n { n ≥ 1 : X n = j } \tau_j=min \{ n\geq 1: X_n =j \} τj​=min{n≥1:Xn​=j}hitting time, the first time hit j

${\tau }_n(j)$: the nth time to hit j.

$N_j={\sum }_{n=0}^{\infty }I(x_n=j)$: the number of Markov chain hits j.

P i { N j ≥ n } = P i { τ n ( j ) < ∞ } P_i \{ N_j \geq n\}=P_i \{\tau_n(j)<\infty \} Pi​{Nj​≥n}=Pi​{τn​(j)<∞}

{ τ = n } = { X 1 , … , X n − 1 ∉ A , X n ∈ A } \{\tau =n \}=\{ X_1, \dots, X_{n-1}\notin A, X_n \in A \} {τ=n}={X1​,…,Xn−1​∈/​A,Xn​∈A}

${\xi }_n={\tau }_n-{\tau }_{n-1}$: inter-arrival time

f i j n = P i { τ j = n } f_{ij}^n=P_i \{\tau_j=n \} fijn​=Pi​{τj​=n} , the chain starting at i enters j for the first time at the nth step

f i j = P i { τ j < ∞ } = ∑ n = 1 ∞ f i j n f_{ij}=P_i \{\tau_j < \infty \}=\sum_{n=1}^{\infty} f_{ij}^n fij​=Pi​{τj​<∞}=∑n=1∞​fijn​, probability begin at i, it even hits j.

recurrent if $f_{ii}=1$ , transient if $f_{ii}<1$

positive recurrent if $E_i[{\tau }_i]<\infty$, null recurrent if $E_i[{\tau }_i]=\infty$, and ${\mu }_i=E_i[{\tau }_i]$

stationary distribution: ${\pi }_i={\sum }_j{\pi }_j{p}_{ji}$, or $\pi =\pi P$