Chapter 5 (Limit Theorems): Convergence in Probability (依概率收敛)

本文为 $Introduction$ $to$ $Probability$ 的读书笔记

Convergence in Probability

• We can interpret the weak law of large numbers as stating that "$M_n$ converges to $\mu$". However. since $M_1,M_2....$ is a sequence of random variables, not a sequence of numbers, the meaning of convergence has to be made precise.
  

---

• Given this definition. the weak law of large numbers simply states that the sample mean converges in probability to the true mean $\mu$.
• More generally, the Chebyshev inequality implies that if all $Y_n$ have the same mean $\mu$, and $var(Y_n)$ converges to 0, then $Y_n$ converges to $\mu$ in probability.
• If the random variables $Y_1,Y_2,....$ have a PMF or a PDF and converge in probability to $a$, then according to the above definition, almost all of the PMF or PDF of $Y_n$ is concentrated within $ϵ$ of a for large values of $n$. It is also instructive to rephrase the above definition as follows: for every $ϵ>0$, and for every $\delta >0$, there exists some $n_0$ such that
  $$P(|Y_n-a|\ge ϵ)\le \delta foralln\ge n_0$$
  • If we refer to $ϵ$ as the accuracy level (精度), and $\delta$ as the confidence level (置信水平), the definition takes the following intuitive form: for any given levels of accuracy and confidence, $Y_n$ will be equal to $a$, within these levels of accuracy and confidence, provided that $n$ is large enough.

---

Example 5.6.

• Consider a sequence of independent random variables $X_n$ that are uniformly distributed in the interval $[0,1]$, and let
  Y n = { X 1 , . . . , X n } Y_n=\{X_1,...,X_n\} Yn​={X1​,...,Xn​}
• In particular.
  $$\underset{n\to \infty }{\lim }P(|Y_n-0|\ge ϵ)=\underset{n\to \infty }{\lim }(1-ϵ{)}^n=0$$Since this is true for every $ϵ>0$, we conclude that $Y_n$ converges to zero, in probability.

---

• One might be tempted to believe that if a sequence $Y_n$ converges to a number $a$, then $E[Y_n]$ must converge to $a$.
  • The following example shows that this need not be the case, and illustrates some of the limitations (局限性) of the notion of convergence in probability.

Example 5.8.

• Consider a sequence of discrete random variables $Y_n$ with the following distribution:
  
• For every $ϵ>0$, we have
  $$\underset{n\to \infty }{\lim }P(|Y_n-0|\ge ϵ)=\underset{n\to \infty }{\lim }\frac{1}{n}=0$$and $Y_n$ converges to zero in probability.
• On the other hand,
  $$E[Y_n]=n^2/n=n$$which goes to infinity as $n$ increases.

---

Problem 5
Let $X_1,X_2,...$ be independent random variables that are uniformly distributed over $[-1,1]$. Show that the sequence $Y_1,Y_2....$ converges in probability to some limit, and identify the limit.
$$Y_n=X_1\cdot X_2...X_n$$
SOLUTION

• We have
  $$E[Y_n]=E[X_1]...E[X_n]=0$$Also
  $$var(Y_n)=E[Y_n^2]=E[X_1^2]...E[X_2^n]=var(X_1{)}^n=(\frac{4}{12}{)}^n$$so $var(Y_n)\to 0$. Since all $Y_n$ have 0 as a common mean, from Chebyshev’s inequality it follows that $Y_n$ converges to 0 in probability.

---

Problem 6.
Consider two sequences of random variables $X_1,X_2,...$ and $Y_1,Y_2,...$, which converge in probability to some constants. Let $c$ be another constant. Show that $c{X}_n$, $X_n+Y_n$, max ⁡ { 0 , X n } \max\{0, X_n \} max{0,Xn​}, $|X_n|$, and $X_n{Y}_n$ all converge in probability to corresponding limits.

SOLUTION

• Let $x$ and $y$ be the limits of $X_n$ and $Y_n$, respectively. Fix some $ϵ>0$ and a constant $c$. If $c=0$, then $c{X}_n$ equals zero for all $n$, and convergence trivially holds. If $c\ne 0$, we observe that $P(|c{X}_n-cx|\ge ϵ)=P(|X_n-x|\ge ϵ/|c|)$, which converges to zero, thus establishing convergence in probability of $c{X}_n$.
• We note that
  $$P(|X_n+Y_n-x-y|\ge ϵ)\le P(|X_n-x|\ge ϵ/2)+P(|Y_n-y|\ge ϵ/2)$$Therefore,
  $$\underset{n\to \infty }{\lim }P(|X_n+Y_n-x-y|\ge ϵ)\le \underset{n\to \infty }{\lim }P(|X_n-x|\ge ϵ/2)+\underset{n\to \infty }{\lim }P(|Y_n-y|\ge ϵ/2)=0$$
• We have
  lim ⁡ n → ∞ P ( { ∣ max ⁡ { 0 , X n } − max ⁡ { 0 , x } ∣ ≥ ϵ } ) = lim ⁡ n → ∞ P ( { ∣ max ⁡ { 0 , X n } − max ⁡ { 0 , x } ∣ ≥ ϵ } ∣ x X n ≥ 0 ) P ( x X n ≥ 0 ) + lim ⁡ n → ∞ P ( { ∣ max ⁡ { 0 , X n } − max ⁡ { 0 , x } ∣ ≥ ϵ } ∣ x X n < 0 ) P ( x X n < 0 ) = lim ⁡ n → ∞ P ( { ∣ max ⁡ { 0 , X n } − max ⁡ { 0 , x } ∣ ≥ ϵ } ∣ x X n ≥ 0 ) \begin{aligned} &\lim_{n\rightarrow\infty}P(\{|\max\{0,X_n\}-\max\{0,x\}|\geq\epsilon\}) \\=&\lim_{n\rightarrow\infty}P(\{|\max\{0,X_n\}-\max\{0,x\}|\geq\epsilon\}|xX_n\geq0)P(xX_n\geq0)+\lim_{n\rightarrow\infty}P(\{|\max\{0,X_n\}-\max\{0,x\}|\geq\epsilon\}|xX_n<0)P(xX_n<0) \\=&\lim_{n\rightarrow\infty}P(\{|\max\{0,X_n\}-\max\{0,x\}|\geq\epsilon\}|xX_n\geq0) \end{aligned} ==​n→∞lim​P({∣max{0,Xn​}−max{0,x}∣≥ϵ})n→∞lim​P({∣max{0,Xn​}−max{0,x}∣≥ϵ}∣xXn​≥0)P(xXn​≥0)+n→∞lim​P({∣max{0,Xn​}−max{0,x}∣≥ϵ}∣xXn​<0)P(xXn​<0)n→∞lim​P({∣max{0,Xn​}−max{0,x}∣≥ϵ}∣xXn​≥0)​
  • If $x>0$, then
    lim ⁡ n → ∞ P ( { ∣ max ⁡ { 0 , X n } − max ⁡ { 0 , x } ∣ ≥ ϵ } ) = lim ⁡ n → ∞ P ( { ∣ X n − x ∣ ≥ ϵ } ∣ x X n ≥ 0 ) = 0 \lim_{n\rightarrow\infty}P(\{|\max\{0,X_n\}-\max\{0,x\}|\geq\epsilon\})=\lim_{n\rightarrow\infty}P(\{|X_n-x|\geq\epsilon\}|xX_n\geq0)=0 n→∞lim​P({∣max{0,Xn​}−max{0,x}∣≥ϵ})=n→∞lim​P({∣Xn​−x∣≥ϵ}∣xXn​≥0)=0
  • If $x\le 0$, then
    lim ⁡ n → ∞ P ( { ∣ max ⁡ { 0 , X n } − max ⁡ { 0 , x } ∣ ≥ ϵ } ) = lim ⁡ n → ∞ P ( { 0 ≥ ϵ } ∣ x X n ≥ 0 ) = 0 \lim_{n\rightarrow\infty}P(\{|\max\{0,X_n\}-\max\{0,x\}|\geq\epsilon\})=\lim_{n\rightarrow\infty}P(\{0\geq\epsilon\}|xX_n\geq0)=0 n→∞lim​P({∣max{0,Xn​}−max{0,x}∣≥ϵ})=n→∞lim​P({0≥ϵ}∣xXn​≥0)=0
  • Thus, we have
    lim ⁡ n → ∞ P ( { ∣ max ⁡ { 0 , X n } − max ⁡ { 0 , x } ∣ ≥ ϵ } ) = 0 \lim_{n\rightarrow\infty}P(\{|\max\{0,X_n\}-\max\{0,x\}|\geq\epsilon\})=0 n→∞lim​P({∣max{0,Xn​}−max{0,x}∣≥ϵ})=0
• We have ∣ X n ∣ = max ⁡ { 0 , X n } + max ⁡ { 0 , − X n } |X_n| = \max\{0, X_n\}+\max\{0, -X_n\} ∣Xn​∣=max{0,Xn​}+max{0,−Xn​}. Since max ⁡ { 0 , X n } \max\{0, X_n\} max{0,Xn​} and max ⁡ { 0 , − X n } \max\{0, -X_n\} max{0,−Xn​} converge, it follows that their sum, $|X_n|$, converges to max ⁡ { 0 , x } + max ⁡ { 0 , − x } = ∣ x ∣ \max\{0, x\}+\max\{0, -x\}=|x| max{0,x}+max{0,−x}=∣x∣ in probability.
• Finally, we have
  $$\begin{array}{rl}P(|X_n{Y}_n-xy|\ge ϵ)& =P(|(X_n-x)(Y_n-y)+x{Y}_n+y{X}_n-2xy|\ge ϵ)\\ & \le P(|(X_n-x)(Y_n-y)|\ge ϵ/2)+P(|x{Y}_n+y{X}_n-2xy|\ge ϵ/2)\\ & \le P(|X_n-x|\ge \sqrt{ϵ/2})P(|Y_n-x|\ge \sqrt{ϵ/2})+P(|x{Y}_n+y{X}_n-2xy|\ge ϵ/2)\end{array}$$Since $x{Y}_n$ and $y{X}_n$ both converge to $xy$ in probability. the last probability in the above expression converges to 0. It will thus suffice to show that
  $$\begin{array}{rl}\underset{x\to \infty }{\lim }P(|X_n{Y}_n-xy|\ge ϵ)& \le 0\end{array}$$

---

Problem 7.
A sequence $X_n$ of random variables is said to converge to a number $c$ in the mean square (均方收敛), if
$$\underset{n\to \infty }{\lim }E[(X_n-c{)}^2]=0$$

• (a) Show that convergence in the mean square implies convergence in probability.
• (b) Give an example that shows that convergence in probability does not imply convergence in the mean square.

SOLUTION

• $(a)$ Suppose that $X_n$ converges to $c$ in the mean square. Using the Markov inequality, we have
  $$P(|X_n-c|\ge ϵ)=P((X_n-c{)}^2\ge {ϵ}^2)\le \frac{E[(X_n-c{)}^2]}{{ϵ}^2}$$Taking the limit as $n\to \infty$. we obtain
  $$\underset{n\to \infty }{\lim }P(|X_n-c|\ge ϵ)=0$$
• (b) In Example 5.8, we have convergence in probability to 0 but $E[Y^2]=n^3$ , which diverges to infinity.