Order Statistic

Order Statistic

The Order Statistic

所谓顺序统计量, 即一族独立的观测$X_1,X_2,\dots ,X_n$的排序后的产物
$$X_{(1)}\le X_{(2)}\le \cdots \le X_{(n)}.$$
用大写的原因, 自然是我们可以将每一个元$X_{(i)}$看成一个随机变量, 实际上它是$X_i,i=1,\dots ,n$的一个函数, $X_{(i)}=X_{(i)}(X_1,X_2,\cdots ,X_n)$.

推导顺序统计量的性质, 需要用到一个非常有用的表示方法, 设$F(x)=P(X\le x)$为分布函数, 定义其逆为
F − 1 ( y ) = inf ⁡ { x : F ( x ) ≥ y } , F^{-1}(y) = \inf \{x: F(x) \ge y\}, F−1(y)=inf{x:F(x)≥y},
有一个很好的性质是, 设$U$为$[0,1]$上的均匀分布, 则
$$F^{-1}(U)=F=X,$$
实际上, 这是因为$P(F^{-1}(U)\le u)\iff P(U\le F(u))=F(u)$.

故, 倘若我们有独立的随机变量$U_1,U_2,\dots ,U_n$以及独立同分布的$X_1,X_2,\dots ,X_n$, 我们有
$$(X_{(1)},X_{(2)},\cdots ,X_{(n)})=(F^{-1}(U_{(1)}),F^{-1}(U_{(2)}),\cdots ,F^{-1}(U_{(n)})).$$

另外, 令$F_n$表示$X$的一个经验分布, 显示为
$$F_n(x)=\frac{1}{n}\sum _{i=1}^n\mathbb{I}(X_i\le x).$$
并令
$${\xi }_p:=F^{-1}(p),{\hat{\xi }}_{pn}:=F_n^{-1}(p).$$

引理1 $F^{-1}$的一些基本性质

引理1: 假设$F$为一分布函数, 则$F^{-1}(t),0<t<1$是非降左连续的且满足

1. $F^{-1}F(x)\le x,-\infty <x<\infty$;
2. $F(F^{-1}(t))\ge t,0<t<1$;
3. $F(x)\ge t$当前仅当$x\ge F^{-1}(t)$.

注: $F(x)$是非降右连续.

顺序统计量的分布

定理1: 设$F(x)$存在密度函数$f(x)$.

1. $$P(X_{(k)}\le x)=\sum _{i=k}^n{\mathrm{C}}_n^i[F(x){]}^i[1-F(x){]}^{n-i},-\infty <x<\infty .$$

2. $X_k$的密度函数为
   $$n{\mathrm{C}}_{n-1}^{k-1}{F}^{k-1}(x)[1-F(x){]}^{n-k}f(x).$$

3. $X_{(k_1)},X_{(k_2)}$的联合密度函数($x_1<x_2,k_1<k_2$)为
   $$\begin{gathered} \frac{n!}{(k_1-1)!(k_2-k_1-1)!(n-k_2)!}[F(x_1){]}^{k_1-1}[F(x_2)-F(x_1){]}^{k_2-k_1-1} \\ [1-F(x_2){]}^{n-k_2}f(x_1)f(x_2). \end{gathered}$$

4. 全体顺序统计量的密度函数为
   $$n!f(x_1)f(x_2)\cdots f(z_n),-\infty <x_1<x_2<\cdots <x_n<\infty .$$

proof: 1, 2的证明是简单的, 3需注意$X_{(k_1)},X_{(k_2)}$的分布函数为
KaTeX parse error: Invalid delimiter: '{"type":"ordgroup","mode":"math","loc":{"lexer":{"input":"\n\\sum_{i=k_2}^n \\mathrm{C}_n^i [1-F(x_2)]^{n-i} \\Big{\\{} \\sum_{j=k_1}^i \\mathrm{C}_{k_2}^j [F(x_1)]^i [F(x_2)-F(x_1)]^{k_2-j} \\Big{\\}}.\n","settings":{"displayMode":true,"leqno":false,"fleqn":false,"throwOnError":true,"errorColor":"#cc0000","macros":{},"colorIsTextColor":false,"strict":"warn","maxSize":null,"maxExpand":1000,"allowedProtocols":["http","https","mailto","_relative"]},"tokenRegex":{},"catcodes":{"%":14}},"start":52,"end":56},"body":[{"type":"atom","mode":"math","family":"open","loc":{"lexer":{"input":"\n\\sum_{i=k_2}^n \\mathrm{C}_n^i [1-F(x_2)]^{n-i} \\Big{\\{} \\sum_{j=k_1}^i \\mathrm{C}_{k_2}^j [F(x_1)]^i [F(x_2)-F(x_1)]^{k_2-j} \\Big{\\}}.\n","settings":{"displayMode":true,"leqno":false,"fleqn":false,"throwOnError":true,"errorColor":"#cc0000","macros":{},"colorIsTextColor":false,"strict":"warn","maxSize":null,"maxExpand":1000,"allowedProtocols":["http","https","mailto","_relative"]},"tokenRegex":{},"catcodes":{"%":14}},"start":53,"end":55},"text":"\\{"}]}' after '\Big' at position 53: …_2)]^{n-i} \Big{̲\̲{̲}̲ \sum_{j=k_1}^i…
此公式进行求导实际上是和1, 2的证明是类似的. 4的证明是平凡的.

顺序统计量的条件分布

定理2: 设$F(x)$存在密度函数$f(x)$, 则 $X_{(j)}|X_{(i)},i<j$的分布等价于以$\frac{F(x)-F(x_i)}{1-F(x_i)},x_i\le x<\infty$为分布函数的 $n-i$个顺序统计量的第$j-i$个分布.

proof:
KaTeX parse error: Invalid delimiter: '{"type":"ordgroup","mode":"math","loc":{"lexer":{"input":"\n\\begin{array}{ll}\nf(x_j|X_{(i)}=x_i)\n&= f_{X_(i), X_{(j)}}(x_i, x_j) / f_{X_{(i)}}(x_i) \\\\\n&= \\frac{(n-i)!}{(j-i-1)!(n-j)!} \\Big{\\{} \\frac{F(x_j)-F(x_i)}{1-F(x_i)} \\Big{\\}}^{j-i-1} \\times \\Big{\\{} \\frac{1-F(x_j)}{1-F(x_i)} \\Big{\\}} \\frac{f(x_j)}{1-F(x_i)} \\\\\n&= (n-i)\\mathrm{C}_{n-i-1}^{j-i-1} [F_i(x_j)]^{j-i-1} [1-F_i(x_j)]^{n-j} [F_i(x_j)]'.\n\\end{array}\n","settings":{"displayMode":true,"leqno":false,"fleqn":false,"throwOnError":true,"errorColor":"#cc0000","macros":{"\\\\":"\\cr"},"colorIsTextColor":false,"strict":"warn","maxSize":null,"maxExpand":1000,"allowedProtocols":["http","https","mailto","_relative"]},"tokenRegex":{},"catcodes":{"%":14}},"start":129,"end":133},"body":[{"type":"atom","mode":"math","family":"open","loc":{"lexer":{"input":"\n\\begin{array}{ll}\nf(x_j|X_{(i)}=x_i)\n&= f_{X_(i), X_{(j)}}(x_i, x_j) / f_{X_{(i)}}(x_i) \\\\\n&= \\frac{(n-i)!}{(j-i-1)!(n-j)!} \\Big{\\{} \\frac{F(x_j)-F(x_i)}{1-F(x_i)} \\Big{\\}}^{j-i-1} \\times \\Big{\\{} \\frac{1-F(x_j)}{1-F(x_i)} \\Big{\\}} \\frac{f(x_j)}{1-F(x_i)} \\\\\n&= (n-i)\\mathrm{C}_{n-i-1}^{j-i-1} [F_i(x_j)]^{j-i-1} [1-F_i(x_j)]^{n-j} [F_i(x_j)]'.\n\\end{array}\n","settings":{"displayMode":true,"leqno":false,"fleqn":false,"throwOnError":true,"errorColor":"#cc0000","macros":{"\\\\":"\\cr"},"colorIsTextColor":false,"strict":"warn","maxSize":null,"maxExpand":1000,"allowedProtocols":["http","https","mailto","_relative"]},"tokenRegex":{},"catcodes":{"%":14}},"start":130,"end":132},"text":"\\{"}]}' after '\Big' at position 130: …1)!(n-j)!} \Big{̲\̲{̲}̲ \frac{F(x_j)-F…

对比定理1中的公式即可知.

定理3: 设$F(x)$存在密度函数$f(x)$, 则$X_{(i)}|X_{(j)},i<j$的分布等价于以$\frac{F(x)}{F(x_j)},-\infty <x\le x_j$为分布的$j-1$个顺序统计量的第$i$个分布.

proof: 证明同上.

特殊分布的特殊性质

定理4: 设$X_1,X_2,\dots ,X_n$独立服从于标准指数分布, 令
$$Z_i:=(n-i+1)(X_{(i)}-X_{(i-1)}),X_{(0)}\equiv 0,$$
则$Z_1,Z_2,\dots ,Z_n$也独立服从于标准指数分布.

proof: 通过变量替换并利用Jacobian行列式从$x$变换到$z$, 需要注意俩个分布的区域的差别.

定理5: 对于$[0,1]$上的均匀分布, 则随机变量$V_1=U_{(i)}/U_{(j)}$ 且$V_2=U_{(j)},1\le i<j\le n$, 独立, 前者服从$Beta(i,j-1)$, 后者服从$Beta(j,n-j+1)$.

proof: 同上利用变量替换.

定理6: 对于$[0,1]$上的均匀分布, 则随机变量
$$V_1^{\ast }=\frac{U_{(1)}}{U_{(2)}},V_2^{\ast }=(\frac{U_{(2)}}{U_{(3)}}{)}^2,\cdots ,V_{n-1}^{\ast }=(\frac{U_{(n-1)}}{U_{(n)}}{)}^2,V_n^{\ast }=U_{(n)}^n,$$
独立且均服从于$[0,1]$的均匀分布.

proof: 同样可以用变量替换来做, 不过文中是转换成指数分布然后利用前面的结论来证明的.

${\hat{\xi }}_{pn}-{\xi }_p$

定理7: 令$0<p<1.$ 假设${\xi }_p$存在唯一解$x$使得$F(x^{-})\le p\le F(x)$, 则
$$P(|{\hat{\xi }}_{pn}-{\xi }_p|>ϵ)\le 2\exp (-2n{\delta }_{ϵ}^2),\forall ϵ>0,n,$$
其中 δ ϵ = min ⁡ { F ( ξ p + ϵ ) − p , p − F ( ξ p − ϵ ) } \delta_{\epsilon} = \min \{F(\xi_p+\epsilon)-p, p-F(\xi_p-\epsilon)\} δϵ​=min{F(ξp​+ϵ)−p,p−F(ξp​−ϵ)}.

proof: 证明拆成并用到了Hoffeding不等式, 感觉挺有技巧性的.

$F_n$

定理11:

1. $\mathbb{E}(F_n(x))=F(x)$;
2. $\mathrm{V}\mathrm{a}\mathrm{r}(F_n(x))=\frac{F(x)(1-F(x))}{n}\to 0.$

proof: 只需注意到, $n{F}_n(x)$实际上服从的是$\mathrm{b}\mathrm{i}\mathrm{n}\mathrm{o}\mathrm{m}\mathrm{i}\mathrm{a}\mathrm{l}(n,F(x))$即可.

定理12:
P { sup ⁡ x ∣ F n ( x ) − F ( x ) ∣ → 0 } = 1. P\{\sup_x |F_n(x) - F(x)| \rightarrow 0\} = 1. P{xsup​∣Fn​(x)−F(x)∣→0}=1.

proof: 令$ϵ>0$, 取$k>1/ϵ$以及
$$-\infty =x_0<x_1<\cdots <x_{k-1}<x_k=\infty$$
使得$F(x_j^{-})\le j/k\le F(x_j),j=1\dots ,k-1$. 若$x_{j-1}<x_j$, 则$F(x_j^{-})-F(x_{j-1})<ϵ$.

根据强大数定律, 有
$$F_n(x_j)\stackrel{a.s.}{\to }F(x_j),F_n(x_j^{-})\stackrel{a.s.}{\to }F(x_j^{-}),j=1,\dots ,k-1.$$
故
$${\Delta }_n=\max (|F_n(x_j)-F(x_j)|,|F_n(x_j^{-})-F(x_j^{-})|,j=1,\dots ,k-1)\stackrel{a.s.}{\to }0.$$
对于$x_{j-1}<x<x_j^{-}$ (注$x=x_j$的情况下面不等式成立是天然的):
$$\begin{gathered} F_n(x)-F(x)\le F_n(x_j^{-})-F(x_{j-1})\le F_n(x_j^{-})-F(x_j^{-})+ϵ\le {\Delta }_n+ϵ \\ {F}_n(x)-F(x)\ge F_n(x_{j-1})-F(x_j^{-})\ge F_n(x_{j-1})-F(x_{j-1})-ϵ\ge {\Delta }_n-ϵ. \end{gathered}$$

故
$$\underset{x}{\sup }|F_n(x)-F(x)|\le {\Delta }_n+ϵ\stackrel{a.s.}{\to }ϵ.$$
对于任意的$ϵ$均成立. 故不等式成立.

注: 这里的证明和文中的有点不同, 感觉这么写更加合理.

注: 文中还讲了不少其它特别是渐进性质, 能力有限只能看个大概, 便不记录了.

---