防盗
https://www.cnblogs.com/setdong/p/17325420.html
1. Markov’s inequality
Theorem:
X
X
X 为非负随机变量, 且
E
[
X
]
<
∞
\mathbb{E}[X]<\infty
E[X]<∞. 那么对于任意
t
>
0
t>0
t>0 有
P
[
X
≥
t
E
[
X
]
]
≤
1
t
\mathbb{P}[X\geq t\mathbb{E}[X]]\leq\frac{1}{t}
P[X≥tE[X]]≤t1
或对于任意的
ϵ
>
0
\epsilon>0
ϵ>0 有
P
[
X
≥
ϵ
]
≤
E
[
X
]
ϵ
\mathbb{P}[X\geq\epsilon]\leq\frac{\mathbb{E}[X]}{\epsilon}
P[X≥ϵ]≤ϵE[X]
证明, 以上两式等价, 这里证第一个:
由定义有
P
[
X
≥
t
E
[
X
]
]
=
∑
x
:
x
≥
t
E
[
X
]
P
[
X
=
x
]
\mathbb{P}[X\geq t\mathbb{E}[X]]=\sum_{x:x\geq t\mathbb{E}[X]}\mathbb{P}[X=x]
P[X≥tE[X]]=x:x≥tE[X]∑P[X=x]
对于
x
≥
t
E
[
X
]
x\geq t\mathbb{E}[X]
x≥tE[X] 部分, 有
x
t
E
[
X
]
≥
1
\frac{x}{t\mathbb{E}[X]}\geq 1
tE[X]x≥1, 所以
P
[
X
≥
t
E
[
X
]
]
≤
∑
x
:
x
≥
t
E
[
X
]
P
[
X
=
x
]
x
t
E
[
X
]
\mathbb{P}[X\geq t\mathbb{E}[X]]\leq\sum_{x:x\geq t\mathbb{E}[X]}\mathbb{P}[X=x]\frac{x}{t\mathbb{E}[X]}
P[X≥tE[X]]≤x:x≥tE[X]∑P[X=x]tE[X]x
对于
x
<
t
E
[
X
]
x < t\mathbb{E}[X]
x<tE[X] 部分, 有
P
[
X
=
x
]
x
t
E
[
X
]
≥
0
\mathbb{P}[X=x]\frac{x}{t\mathbb{E}[X]}\geq 0
P[X=x]tE[X]x≥0 , 所以加上这些非负部分, 有:
P
[
X
≥
t
E
[
X
]
]
≤
∑
x
P
[
X
=
x
]
x
t
E
[
X
]
=
E
[
X
]
t
E
[
X
]
=
1
t
\mathbb{P}[X\geq t\mathbb{E}[X]]\leq\sum_{x}\mathbb{P}[X=x]\frac{x}{t\mathbb{E}[X]}=\frac{\mathbb{E}[X]}{t\mathbb{E}[X]}=\frac{1}{t}
P[X≥tE[X]]≤x∑P[X=x]tE[X]x=tE[X]E[X]=t1
2. Chebyshev’s inequality
Theorem:
X
X
X 为随机变量, 且
Var
[
X
]
<
∞
\text{Var}[X]<\infty
Var[X]<∞ (方差). 那么对于任意
t
>
0
t>0
t>0 有
P
[
∣
X
−
E
[
X
]
∣
≥
t
Var
[
X
]
1
2
]
≤
1
t
2
\mathbb{P}[|X-\mathbb{E}[X]|\geq t\text{Var}[X]^{\frac{1}{2}}]\leq\frac{1}{t^2}
P[∣X−E[X]∣≥tVar[X]21]≤t21
或对于任意
ϵ
>
0
\epsilon>0
ϵ>0 有
P
[
∣
X
−
E
[
X
]
∣
≥
ϵ
]
≤
Var
[
X
]
ϵ
2
\mathbb{P}[|X-\mathbb{E}[X]|\geq \epsilon]\leq\frac{\text{Var}[X]}{\epsilon^2}
P[∣X−E[X]∣≥ϵ]≤ϵ2Var[X]
证明:
首先有
P
[
∣
X
−
E
[
X
]
∣
≥
t
Var
[
X
]
1
2
]
=
P
[
(
X
−
E
[
X
]
)
2
≥
t
2
Var
[
X
]
]
\mathbb{P}[|X-\mathbb{E}[X]|\geq t\text{Var}[X]^{\frac{1}{2}}]=\mathbb{P}[(X-\mathbb{E}[X])^2\geq t^2\text{Var}[X]]
P[∣X−E[X]∣≥tVar[X]21]=P[(X−E[X])2≥t2Var[X]]
然后对上式右侧应用 Markov’s inequality, 其中
X
X
X 是
(
X
−
E
[
X
]
)
2
(X-\mathbb{E}[X])^2
(X−E[X])2,
ϵ
\epsilon
ϵ 是
t
2
Var
[
X
]
2
t^2\text{Var}[X]^2
t2Var[X]2
P
[
(
X
−
E
[
X
]
)
2
≥
t
2
Var
[
X
]
]
≤
E
[
(
X
−
E
[
X
]
)
2
]
t
2
Var
[
X
]
=
1
t
2
\mathbb{P}[(X-\mathbb{E}[X])^2\geq t^2\text{Var}[X]]\leq \frac{\mathbb{E}[(X-\mathbb{E}[X])^2]}{t^2\text{Var}[X]}=\frac{1}{t^2}
P[(X−E[X])2≥t2Var[X]]≤t2Var[X]E[(X−E[X])2]=t21
其中
E
[
(
X
−
E
[
X
]
)
2
]
=
Var
[
X
]
\mathbb{E}[(X-\mathbb{E}[X])^2]=\text{Var}[X]
E[(X−E[X])2]=Var[X] 为方差的定义.
3. Weak law of large numbers
Theorem:
(
X
n
)
n
∈
N
(X_n)_{n\in\mathbb{N}}
(Xn)n∈N 是一列独立的随机变量, 它们有相同的期望
μ
\mu
μ 和方差
σ
2
>
∞
\sigma^2>\infty
σ2>∞.
X
ˉ
n
=
1
n
∑
i
=
1
n
X
i
\bar{X}_n=\frac{1}{n}\sum_{i=1}^n X_i
Xˉn=n1∑i=1nXi 为均值. 那么对于任意的
ϵ
>
0
\epsilon>0
ϵ>0 有
lim
n
→
∞
P
[
∣
X
ˉ
n
−
μ
∣
≥
ϵ
]
=
0
\lim_{n\rightarrow\infty}\mathbb{P}[|\bar{X}_n-\mu|\geq\epsilon]=0
n→∞limP[∣Xˉn−μ∣≥ϵ]=0
证明:
已知变量是独立的, 根据期望和方差的特性有,
E
[
X
ˉ
n
]
=
∑
i
=
1
n
μ
n
=
μ
Var
[
X
ˉ
n
]
=
∑
i
=
1
n
σ
2
n
2
=
σ
2
n
\mathbb{E}[\bar{X}_n]=\sum_{i=1}^n \frac{\mu}{n}=\mu\\ \text{Var}[\bar{X}_n]=\sum_{i=1}^n \frac{\sigma^2}{n^2}=\frac{\sigma^2}{n}
E[Xˉn]=i=1∑nnμ=μVar[Xˉn]=i=1∑nn2σ2=nσ2
对上式使用 Chebyshev’s inequality, 其中
t
=
ϵ
/
Var
[
X
ˉ
n
]
1
2
t=\epsilon/\text{Var}[\bar{X}_n]^{\frac{1}{2}}
t=ϵ/Var[Xˉn]21, 有
P
[
∣
X
ˉ
n
−
μ
∣
≥
ϵ
]
≤
σ
2
n
ϵ
2
\mathbb{P}[|\bar{X}_n-\mu|\geq\epsilon]\leq\frac{\sigma^2}{n\epsilon^2}
P[∣Xˉn−μ∣≥ϵ]≤nϵ2σ2
所以当
n
n
n 趋于无穷时, 概率趋于
0
0
0. 可以理解为样本均值依概率收敛于期望值.
防盗
https://www.cnblogs.com/setdong/p/17325420.html