1.Modes of Convergence
(1).Convergence in Probability
Definition
(Convergence in Probability)
Let X 1 , X 2 , . . . X_1,X_2,... X1,X2,... and X X X be random variables on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},P) (Ω,F,P).
We say X n X_n Xn convergence in probability to X X X if
P ( ∣ X n − X ∣ > ϵ ) → 0 , every ϵ > 0 P(|X_n-X|>\epsilon)\to 0,\text{every}\text{ } \epsilon>0 P(∣Xn−X∣>ϵ)→0,every ϵ>0
as n → ∞ n\to \infin n→∞
常记为 X n → P X , n → ∞ X_n \stackrel{P}{\rightarrow} X,n\to \infin Xn→PX,n→∞
扩展(多元随机变量的依概率收敛)
对于随机的 p p p维随机向量for random p-vectors X 1 , X 2 , . . . and X \mathbf{X_1,X_2},...\text{and}\text{ }\mathbf{X} X1,X2,...and X , we say that X n → P X \mathbf{X}_n \stackrel{P}{\rightarrow}\mathbf{X} Xn→PX
EX
: X n → P X \mathbf{X}_n \stackrel{P}{\rightarrow}\mathbf{X} Xn→PX 当且仅当分量是依概率收敛的 .
Proof. 依据定义证明
如果 X n → P X \mathbf{X}_n \stackrel{P}{\rightarrow}\mathbf{X} Xn→PX,那么 ∀ ϵ > 0 , P ( ∣ X n − X ∣ > ϵ ) → 0 , n → ∞ \forall \epsilon>0, P(|\mathbf{X_n-X}|>\epsilon)\to 0,n\to \infin ∀ϵ>0,P(∣Xn−X∣>ϵ)→0,n→∞
即 P ( ∣ X n 1 − X 1 ∣ 2 + . . . + ∣ X n p − X p ∣ 2 > ϵ ) → 0 , n → ∞ P(\sqrt{|X_{n_1}-X_1|^2+...+|X_{n_p}-X_p|^2}>\epsilon)\to 0,n\to \infin P(∣Xn1−X1∣2+...+∣Xnp−Xp∣2>ϵ)→0,n→∞
也即 P ( ∣ X n 1 − X 1 ∣ 2 + . . . + ∣ X n p − X p ∣ 2 ≤ ϵ ) → 1 , n → ∞ P(\sqrt{|X_{n_1}-X_1|^2+...+|X_{n_p}-X_p|^2}\le \epsilon)\to 1,n\to \infin P(∣Xn1−X1∣2+...+∣Xnp−Xp∣2≤ϵ)→1,n→∞
注意到下述关系:
∀ i , ∣ X n i − X i ∣ 2 ≤ ∣ X n 1 − X 1 ∣ 2 + . . . + ∣ X n p − X p ∣ 2 ≤ ϵ \forall i,\sqrt{|X_{n_i}-X_i|^2}\le \sqrt{|X_{n_1}-X_1|^2+...+|X_{n_p}-X_p|^2}\le\epsilon ∀i,∣Xni−Xi∣2≤∣Xn1−X1∣2+...+∣Xnp−Xp∣2≤ϵ
所以,若 n → ∞ n\to \infin n→∞ 时, 分 量 和 ≤ ϵ \textcolor{red}{分量和\le \epsilon} 分量和≤ϵ 的概率趋向1,那么任意一个分量≤ ϵ \epsilon ϵ自然也是趋于1的.
如果 ∀ i , X n i → P X i \forall i,X_{ni}\stackrel{P}{\rightarrow}X_i ∀i,Xni→PXi ,那么 ∀ ϵ > 0 \forall \epsilon>0 ∀ϵ>0, P ( ( X n i − X i ) 2 > ϵ 2 p ) → 0 P((X_{ni}-X_i)^2> \frac{\epsilon^2}{p})\to 0 P((Xni−Xi)2>pϵ2)→0
注意到: { ∣ X n 1 − X 1 ∣ 2 + . . . + ∣ X n p − X p ∣ 2 > ϵ } \{\sqrt{|X_{n_1}-X_1|^2+...+|X_{n_p}-X_p|^2}>\epsilon\} { ∣Xn1−X1∣2+...+∣Xnp−Xp∣2>ϵ}这个事件成立一定能推出 { ∃ i , ( X n i − X i ) 2 > ϵ 2 p } \{\exist i,(X_{ni}-X_i)^2>\frac{\epsilon^2}{p}\} { ∃i,(Xni−Xi)2>pϵ2},而 P ( { ∃ i , ( X n i − X i ) 2 > ϵ 2 p } ) ≤ ∑ i P ( ( X n i − X i ) 2 > ϵ 2 p ) → 0 P(\{\exist i,(X_{ni}-X_i)^2>\frac{\epsilon^2}{p}\})\le \sum_i P((X_{ni}-X_i)^2>\frac{\epsilon^2}{p})\to 0 P({ ∃i,(Xni−Xi)2>pϵ2})≤∑iP((Xni−Xi)2>pϵ2)→0,所以 P ( { ∣ X n 1 − X 1 ∣ 2 + . . . + ∣ X n p − X p ∣ 2 > ϵ } ) → 0 P(\{\sqrt{|X_{n_1}-X_1|^2+...+|X_{n_p}-X_p|^2}>\epsilon\})\to 0 P({ ∣Xn1−X1∣2+...+∣Xnp−Xp∣2>ϵ})→0
Example1
: For i.i.d. Bernoulli trials with a success probability p = 1 / 2 p = 1/2 p=1/2, let T n T_n Tn denote
the number of times in the first n n n trials that a success is followed by a
failure. Show that T n / n → P 1 / 4 T_n/n\stackrel{P}{\rightarrow} 1/4 Tn/n→P1/4.
Proof. 目标是证明 ∀ ϵ > 0 , P ( ∣ T n / n − 1 / 4 ∣ > ϵ ) → 0 \forall \epsilon>0,P(|T_n/n-1/4|>\epsilon)\to 0 ∀ϵ>0,P(∣Tn/n−1/4∣>ϵ)→0
技巧:利用Markov不等式: For any t > 0 , P ( Z > t ) ≤ E [ Z ] t t>0,P(Z>t)\le \frac{E[Z]}{t} t>0,P(Z>t)≤tE[Z],Where Z Z Z is a positive r.v.
P ( ∣ T n / n − 1 / 4 ∣ > ϵ ) = P ( ( T n / n − 1 / 4 ) 2 > ϵ 2 ) ≤ E ( T n / n − 1 / 4 ) 2 ϵ 2 P(|T_n/n-1/4|>\epsilon)=P((T_n/n-1/4)^2>\epsilon^2) \le \frac{E(T_n/n-1/4)^2}{\epsilon^2} P(∣Tn/n−1/4∣>ϵ)=P((Tn/n−1/4)2>ϵ2)≤ϵ2E(Tn/n−1/4)2
下面就只需要证明 E ( T n / n − 1 / 4 ) 2 → 0 , n → ∞ E(T_n/n-1/4)^2\to 0,n\to \infin E(Tn/n−1/4)2→0,n→∞就可以了:
E [ ( T n / n − 1 / 4 ) 2 ] = E [ T n 2 / n 2 − E [ T n / ( 2 n ) ] + 1 / 16 E[(T_n/n-1/4)^2]=E[T_n^2/n^2-E[T_n/(2n)]+1/16 E[(Tn/n−1/4)2]=E[Tn2/n2−E[Tn/(2n)]+1/16
令 X i = 1 { 第 i 次 成 功 , 但 是 第 i + 1 次 失 败 } X_i=1_{\{第i次成功,但是第i+1次失败\}} Xi=1{ 第i次成功,但是第i+1次失败}, 可知 T n = ∑ i = 1 n − 1 X i T_n=\sum_{i=1}^{n-1}X_i Tn=∑i=1n−1Xi 【这一步是想将 T n T_n Tn写成随机变量和的形式】
注意到: X i X_i Xi其实就是两点分布,取值1的概率是1/4,取值0的概率是3/4.
E ( T n ) = ∑ i = 1 n − 1 E ( X i ) = n − 1 4 E(T_n)=\sum_{i=1}^{n-1}E(X_i)=\frac{n-1}{4} E(Tn)=∑i=1n−1E(Xi)=4n−</