马尔可夫不等式
设随机变量X的取值为非负数,马尔可夫不等式形式为:
P
(
X
≥
ξ
)
≤
E
(
X
)
ξ
P(X \ge \xi) \le \frac{E(X)}{\xi}
P(X≥ξ)≤ξE(X)
p r o o f . proof. proof.
设非负随机变量 X X X的概率密度函数为 f ( x ) f(x) f(x)
E ( X ) = ∫ 0 ∞ x f ( x ) d x = ∫ 0 ξ x f ( x ) d x + ∫ ξ ∞ x f ( x ) d x ≥ ∫ ξ ∞ x f ( x ) d x ≥ ∫ ξ ∞ ξ f ( x ) d x = ξ P ( X ≥ ξ ) \begin{split} E(X)&=\int_{0}^{\infty}xf(x)dx \\ &= \int_{0}^{\xi}xf(x)dx+\int_{\xi}^{\infty}xf(x)dx \\ &\ge \int_{\xi}^{\infty}xf(x)dx \\ &\ge \int_{\xi}^{\infty}\xi f(x)dx \\ &= \xi P(X \ge \xi) \end{split} E(X)=∫0∞xf(x)dx=∫0ξxf(x)dx+∫ξ∞xf(x)dx≥∫ξ∞xf(x)dx≥∫ξ∞ξf(x)dx=ξP(X≥ξ)
所以 E ( X ) ≥ ξ P ( X ≥ ξ ) E(X) \ge \xi P(X \ge \xi) E(X)≥ξP(X≥ξ)经变换得到结论:
P ( X ≥ ξ ) ≤ E ( X ) ξ P(X \ge \xi) \le \frac{E(X)}{\xi} P(X≥ξ)≤ξE(X)
切诺夫界
令 X = ∑ i = 1 n X i X=\sum_{i=1}^{n}X_i X=∑i=1nXi,其中 X 1 , … , X n X_1, \dots,X_n X1,…,Xn之间相互独立,并且 P ( X i = 1 ) = p i , P ( X i = 0 ) = 1 − p i P(X_i=1)=p_i,\ P(X_i=0)=1-p_i P(Xi=1)=pi, P(Xi=0)=1−pi
令 μ = E ( X ) = ∑ i = 1 n p i \mu = E(X)=\sum_{i=1}^{n}p_i μ=E(X)=∑i=1npi
-
Upper Tail:
∀ δ > 0 , P ( X ≥ ( 1 + δ ) μ ) ≤ e − δ 2 2 + δ μ \forall \delta \gt 0,\ P(X \ge (1+\delta)\mu) \le e^{-\frac{\delta^2}{2+\delta}\mu} ∀δ>0, P(X≥(1+δ)μ)≤e−2+δδ2μ -
Lower Tail:
∀ 0 < δ < 1 , P ( X ≤ ( 1 − δ ) μ ) ≤ e − δ 2 2 μ \forall 0 \lt \delta \lt 1,\ P(X \le (1-\delta)\mu) \le e^{-\frac{\delta^2}{2}\mu} ∀0<δ<1, P(X≤(1−δ)μ)≤e−2δ2μ
p r o o f . proof. proof.
由马尔可夫不等式可知,设 X X X为一非负随机变量,对于 ∀ s > 0 \forall s \gt 0 ∀s>0
P ( X ≥ a ) = P ( e s X ≥ e s a ) ≤ E ( e s X ) e s a P(X \ge a)=P(e^{sX} \ge e^{sa}) \le \frac{E(e^{sX})}{e^{sa}} P(X≥a)=P(esX≥esa)≤esaE(esX)
类似可知
P ( X ≤ a ) = P ( e − s X ≥ e − s a ) ≤ E ( e − s X ) e − s a P(X \le a)=P(e^{-sX} \ge e^{-sa}) \le \frac{E(e^{-sX})}{e^{-sa}} P(X≤a)=P(e−sX≥e−sa)≤e−saE(e−sX)
现令 M X ( s ) = E ( e s X ) M_X(s)=E(e^{sX}) MX(s)=E(esX)l e m m a 1 : lemma\ 1: lemma 1:
设随机变量 X = ∑ i = 1 n X i X=\sum_{i=1}^{n}X_i X=∑i=1nXi
M X ( s ) = E ( e s X ) = E ( e s ∑ i = 1 n X i ) = E ( ∏ i = 1 n e s X i ) = ∏ i = 1 n E ( e s X i ) = ∏ i = 1 n M X i ( s ) M_X(s)=E(e^{sX})=E(e^{s\sum_{i=1}^{n}X_i})=E(\prod_{i=1}^{n}e^{sX_i})=\prod_{i=1}^{n}E(e^{sX_i})=\prod_{i=1}^{n}M_{X_i}(s) MX(s)=E(esX)=E(es∑i=1nXi)=E(∏i=1nesXi)=∏i=1nE(esXi)=∏i=1nMXi(s)
即:
M X ( s ) = ∏ i = 1 n M X i ( s ) M_X(s)=\prod_{i=1}^{n}M_{X_i}(s) MX(s)=i=1∏nMXi(s)
l e m m a 2 lemma\ 2 lemma 2:设随机变量 X ∼ B ( p , 1 ) X \sim B(p,1) X∼B(p,1)
M X ( s ) = p e s + ( 1 − p ) = 1 + p ( e s − 1 ) M_X(s)=pe^{s}+(1-p)=1+p(e^{s}-1) MX(s)=pes+(1−p)=1+p(es−1)
因为 e x ≥ 1 + x e^x \ge 1+x ex≥1+x,得到结论
M X ( s ) ≤ e p ( e s − 1 ) M_X(s) \le e^{p(e^{s}-1)} MX(s)≤ep(es−1)
上尾证明:
P ( X ≤ ( 1 + δ ) μ ) ≤ E ( e s X ) e s ( 1 + δ ) μ = M X ( s ) e s ( 1 + δ ) μ = ∏ i = 1 n M X i ( s ) e s ( 1 + δ ) μ ≤ ∏ i = 1 n e p i ( e s − 1 ) e s ( 1 + δ ) μ = e ( e s − 1 ) ∑ i = 1 n p i e s ( 1 + δ ) μ = e ( e s − 1 ) μ e s ( 1 + δ ) μ = [ e ( e s − 1 ) e s ( 1 + δ ) ] μ \begin{split} P(X \le (1+\delta)\mu) &\le \frac{E(e^{sX})}{e^{s(1+\delta)\mu}} \\ &=\frac{M_X(s)}{e^{s(1+\delta)\mu}} \\ &=\frac{\prod_{i=1}^{n} M_{X_i}(s)}{e^{s(1+\delta)\mu}} \\ &\le \frac{\prod_{i=1}^{n} e^{p_i(e^s-1)}}{e^{s(1+\delta)\mu}} \\ &= \frac{e^{(e^s-1)\sum_{i=1}^{n}p_i}}{e^{s(1+\delta)\mu}} \\ &= \frac{e^{(e^s-1)\mu}}{e^{s(1+\delta)\mu}} \\ &= \left[\frac{e^{(e^s-1)}}{e^{s(1+\delta)}}\right]^\mu \\ \end{split} P(X≤(1+δ)μ)≤es(1+δ)μE(esX)=es(1+δ)μMX(s)=es(1+δ)μ∏i=1nMXi(s)≤es(1+δ)μ∏i=1nepi(es−1)=es(1+δ)μe(es−1)∑i=1npi=es(1+δ)μe(es−1)μ=[es(1+δ)e(es−1)]μ
令 s = l n ( 1 + δ ) s=ln(1+\delta) s=ln(1+δ)可得:
P ( X ≤ ( 1 + δ ) μ ) ≤ ( e δ ( 1 + δ ) ( 1 + δ ) ) μ P(X \le (1+\delta)\mu) \le \left(\frac{e^\delta}{(1+\delta)^{(1+\delta)}}\right)^\mu P(X≤(1+δ)μ)≤((1+δ)(1+δ)eδ)μ
又有: l n [ ( e δ ( 1 + δ ) ( 1 + δ ) ) μ ] = μ ( δ − ( 1 + δ ) l n ( 1 + δ ) ) ln\left[\left(\frac{e^\delta}{(1+\delta)^{(1+\delta)}}\right)^\mu\right]=\mu(\delta-(1+\delta)ln(1+\delta)) ln[((1+δ)(1+δ)eδ)μ]=μ(δ−(1+δ)ln(1+δ)),且 ∀ x > 0 , l n ( 1 + x ) ≥ x 1 + x 2 \forall x \gt 0,\ ln(1+x) \ge \frac{x}{1+\frac{x}{2}} ∀x>0, ln(1+x)≥1+2xx可得:
μ ( δ − ( 1 + δ ) l n ( 1 + δ ) ) ≤ − δ 2 2 + δ μ \mu(\delta-(1+\delta)ln(1+\delta)) \le -\frac{\delta^2}{2+\delta}\mu μ(δ−(1+δ)ln(1+δ))≤−2+δδ2μ
即:
( e δ ( 1 + δ ) ( 1 + δ ) ) μ ≤ e − δ 2 2 + δ μ \left(\frac{e^\delta}{(1+\delta)^{(1+\delta)}}\right)^\mu \le e^{-\frac{\delta^2}{2+\delta}\mu} ((1+δ)(1+δ)eδ)μ≤e−2+δδ2μ
所以上尾得证:
P ( X ≤ ( 1 + δ ) μ ) ≤ e − δ 2 2 + δ μ P(X \le (1+\delta)\mu) \le e^{-\frac{\delta^2}{2+\delta}\mu} P(X≤(1+δ)μ)≤e−2+δδ2μ下尾类似:
令 s = l n ( 1 − δ ) s=ln(1-\delta) s=ln(1−δ)
且使用不等式: ∀ 0 < δ < 1 , l n ( 1 + δ ) ≥ − δ + δ 2 2 \forall 0 \lt \delta \lt 1,\ ln(1+\delta) \ge -\delta + \frac{\delta^2}{2} ∀0<δ<1, ln(1+δ)≥−δ+2δ2
非伯努利切诺夫界
在上述切诺夫界中,假设了 X i ∼ B ( p i , 1 ) X_i \sim B(p_i,1) Xi∼B(pi,1),对于不服从伯努利分布的随机变量,同样有
令 X 1 , … , X n X_1, \dots, X_n X1,…,Xn相互独立,其中 a ≤ X i ≤ b a \le X_i \le b a≤Xi≤b,令 X = ∑ i = 1 n X i , μ = E ( X ) , ∀ δ > 0 X=\sum_{i=1}^{n}X_i,\ \mu=E(X),\ \forall \delta \gt 0 X=∑i=1nXi, μ=E(X), ∀δ>0有:
上尾:
P
(
X
≥
(
1
+
δ
)
μ
)
≤
e
−
2
δ
2
μ
2
n
(
b
−
a
)
2
P(X \ge (1+\delta)\mu) \le e^{-\frac{2\delta^2\mu^2}{n(b-a)^2}}
P(X≥(1+δ)μ)≤e−n(b−a)22δ2μ2
下尾:
P
(
X
≤
(
1
−
δ
)
μ
)
≤
e
−
δ
2
μ
2
n
(
b
−
a
)
2
P(X \le (1-\delta)\mu) \le e^{-\frac{\delta^2\mu^2}{n(b-a)^2}}
P(X≤(1−δ)μ)≤e−n(b−a)2δ2μ2