离散均匀分布的估计| Serial Number Estimation

Serial Number Estimation

Parameter description

  • If θ \theta θ is contained in a function, n n n would be the total sample numbers, and θ ^ \hat{\theta} θ^ would be the estimator for actual maximum ID.
  • If M M M is contained in a function, k k k would be the total sample numbers, and N N N would be the actual maximum ID. As for M M M, that is a random variable of maximum ID in a random sample.

Method 1: Probability of each sample

The estimator used to predict the maximum value can also be determined by assuming that the probability of getting each sample is uniform where θ \theta θ represents the actual maximum ID in each day.

P ( x ) = 1 θ P(x) = \frac{1}{\theta} P(x)=θ1

Method 2: Probability of maximum sample

According to the Assumption 1, we consider the observed maximum ID as a r.v M, and take the maximum ID we encountered in one specific day as m (i.e. x n : n x_{n:n} xn:n). Assume that the N is the actual maximum ID and k represents the number of ill sample, the probability mass function (PMF) of getting the maximum ID can be expressed as follows:

P ( M = m ) = C k − 1 m − 1 C k N P(M = m) = \frac{C_{k-1}^{m-1}}{C_k^N} P(M=m)=CkNCk1m1

Point Estimate

Estimators intuited from discrete uniform distribution

Estimator 1: 2*Mean-1

Consider continuous distribution for this problem, i.e. U N I F ( 0 , θ ) UNIF(0,\theta) UNIF(0,θ)

F o r   U N I F ( 0 , θ ) , E ( X ) = θ 2 , V a r ( X ) = θ 2 12 For~UNIF(0,\theta),E(X)=\frac{\theta}{2},Var(X)=\frac{\theta^2}{12} For UNIF(0,θ),E(X)=2θ,Var(X)=12θ2

We consider the following estimator:
θ ^ 1 = 2 n ∑ i = 1 n X i − 1   f o r   d i s c r e t e   d i s t r u b u t i o n θ ^ 1 = 2 n ∑ i = 1 n X i   f o r   c o n t i n u o u s   d i s t r u b u t i o n E ( θ ^ 1 ) = E ( 2 n ∑ i = 1 n X i − 1 ) = 2 E ( X ‾ ) = θ V a r ( θ ^ 1 ) = 4 n 2 ∑ i = 1 n V a r ( X i ) = 4 n 2 ∑ i = 1 n θ 2 12 = θ 2 3 n ∴ θ ^ 1   i s   a n   u n b i a s e d   e s t i m a t o r   w i t h   V a r = θ 2 3 n \widehat{\theta}_1=\frac{2}{n}\sum^n_{i=1}X_i-1~for~discrete ~distrubution\\ \widehat{\theta}_1=\frac{2}{n}\sum^n_{i=1}X_i~for~continuous ~distrubution\\ E(\widehat{\theta}_1)=E(\frac{2}{n}\sum^n_{i=1}X_i-1)=2E(\overline{X})=\theta\\ Var(\widehat{\theta}_1)=\frac{4}{n^2}\sum_{i=1}^nVar(X_i)=\frac{4}{n^2}\sum_{i=1}^n\frac{\theta^2}{12}=\frac{\theta^2}{3n}\\ \therefore \widehat{\theta}_1~is~an~unbiased~estimator ~with~Var=\frac{\theta^2}{3n}\\ θ 1=n2i=1nXi1 for discrete distrubutionθ 1=n2i=1nXi for continuous distrubutionE(θ 1)=E(n2i=1nXi1)=2E(X)=θVar(θ 1)=n24i=1nVar(Xi)=n24i=1n12θ2=3nθ2θ 1 is an unbiased estimator with Var=3nθ2

Estimator 2: Max + Avg GAP

Consider other form of improvement from MLE estimator, i.e. using average approach to estimate the GAP between maximum and the upper limit:

θ ^ 2 = X n : n + 1 n − 1 ∑ i > j ( X i − X j − 1 ) … f o r   d i s c r e t e   c a s e θ ^ 2 = X n : n + 1 n − 1 ∑ i > j ( X i − X j ) … f o r   c o n t i n u o u s   c a s e \widehat{\theta}_2 = X_{n:n}+\frac{1}{n-1}\sum_{i>j}(X_i-X_j-1)\quad\dots for~discrete~case\\ \widehat{\theta}_2 = X_{n:n}+\frac{1}{n-1}\sum_{i>j}(X_i-X_j) \quad\dots for~continuous~case θ 2=Xn:n+n11i>j(XiXj1)for discrete caseθ 2=Xn:n+n11i>j(XiXj)for continuous case

Calculate the expected value and variance to determine if this estimator is biased or not.
E ( θ ^ 2 ) = E ( X n : n ) + 1 n − 1 ∑ i > j E ( X i − X j ) = n θ n + 1 V a r ( θ ^ 2 ) = n θ 2 ( n + 1 ) ( n − 1 ) ( n + 2 ) E(\widehat{\theta}_2) = E(X_{n:n}) + \frac{1}{n-1}\sum_{i>j}E{(X_i-X_j)} = \frac{n\theta}{n+1} \\ Var(\hat{\theta}_2) = \frac{n\theta^2}{(n+1)(n-1)(n+2)} E(θ 2)=E(Xn:n)+n11i>jE(XiXj)=n+1nθVar(θ^2)=(n+1)(n1)(n+2)nθ2
Therefore, θ 2 \theta_2 θ2 is a biased estimator.

Estimator3: Min+max estimator

We know that maximum sample ID is what’s closed to the upper limit, and we could add more information to it. Intuitively, we first consider minimum sample ID + maximum sample ID:
θ ^ 3 = x 1 : n + x n : n F X n : n ( x ) = [ F X ( x ) ] n = x n θ n , f X n : n ( x ) = n x n − 1 θ n E [ X n : n ] = ∫ x n x n − 1 θ n d x = n n + 1 θ E [ X n : n 2 ] = ∫ x 2 n x n − 1 θ n d x = n n + 2 θ 2 F X 1 : n ( x ) = 1 − [ 1 − F X ( x ) ] n = 1 − ( θ − x θ ) n , f X 1 : n ( x ) = n ( θ − x ) n − 1 θ n E [ X 1 : n ] = ∫ x n ( θ − x ) n − 1 θ n d x = 1 n + 1 θ E [ X 1 : n 2 ] = ∫ x 2 n ( θ − x ) n − 1 θ n d x = 2 n ( n + 1 ) θ 2 E ( θ ^ 3 ) = E ( x 1 : n ) + E ( x n : n ) = θ V a r ( θ ^ 3 ) = V a r ( X 1 : n ) + V a r ( X n : n ) + 2 C o v ( X 1 : n , X n : n ) = 2 n ( n + 1 ) θ 2 − ( 1 n + 1 θ ) 2 + n n + 2 θ 2 − ( n n + 1 θ ) 2 + 2 C o v ( X 1 : n , X n : n ) S i n c e   t h e   j o i n t   d i s t r i b u t i o n   o f   t h e   o r d e r   s t a t i s t i c s   o f   t h e   u n i f o r m   d i s t r i b u t i o n   i s f u i , v j ( u , v ) = n ! u i − 1 ( i − 1 ) ! ( v − u ) j − i − 1 ( j − i − 1 ) ! ( 1 − v ) n − j ( n − j ) ! C o v ( u k , v j ) = j ( n − k − 1 ) ( n − 1 ) 2 ( n + 2 ) V a r ( θ ^ 3 ) = 2 θ 2 n ( n + 2 ) + 2 n 2 θ 2 ( n + 1 ) 2 ( n + 2 ) \widehat{\theta}_3=x_{1:n}+x_{n:n}\\ F_{X_{n:n}}(x) =[F_X(x)]^n=\frac{x^n}{\theta^n},f_{X_{n:n}}(x)=n\frac{x^{n-1}}{\theta^n}\\ E[X_{n:n}]=\int xn\frac{x^{n-1}}{\theta^n}dx=\frac{n}{n+1}\theta\\ E[X_{n:n}^2] =\int x^2n\frac{x^{n-1}}{\theta^n}dx=\frac{n}{n+2}\theta^2\\ F_{X_{1:n}}(x) =1-[1-F_X(x)]^n=1-(\frac{\theta-x}{\theta})^n,f_{X_{1:n}}(x)=\frac{n (\theta-x)^{n-1}}{\theta^n}\\ E[X_{1:n}]=\int x\frac{n (\theta-x)^{n-1}}{\theta^n}dx=\frac{1}{n+1}\theta\\ E[X_{1:n}^2] =\int x^2\frac{n (\theta-x)^{n-1}}{\theta^n}dx=\frac{2}{n(n+1)}\theta^2\\ E(\widehat{\theta}_3)=E(x_{1:n})+E(x_{n:n})=\theta\\ Var(\hat{\theta}_3)=Var(X_{1:n})+Var(X_{n:n})+2Cov(X_{1:n},X_{n:n})\\ =\frac{2}{n(n+1)}\theta^2-(\frac{1}{n+1}\theta)^2 +\frac{n}{n+2}\theta^2-(\frac{n}{n+1}\theta)^2+2Cov(X_{1:n},X_{n:n})\\ Since~the~ joint~ distribution~ of ~the~ order~ statistics~ of~ the~ uniform~ distribution~is\\ f_{u_i,v_j}(u,v)=n!\frac{u^{i-1}}{(i-1)!}\frac{(v-u)^{j-i-1}}{(j-i-1)!}\frac{(1-v)^{n-j}}{(n-j)!}\\ Cov(u_k,v_j)=\frac{j(n-k-1)}{(n-1)^2(n+2)}\\ Var(\hat{\theta}_3)=\frac{2\theta^2}{n(n+2)}+\frac{2n^2\theta^2}{(n+1)^2(n+2)} θ 3=x1:n+xn:nFXn:n(x)=[FX(x)]n=θnxn,fXn:n(x)=nθnxn1E[Xn:n]=xnθnxn1dx=n+1nθE[Xn:n2]=x2nθnxn1dx=n+2nθ2FX1:n(x)=1[1FX(x)]n=1(θθx)n,fX1:n(x)=θnn(θx)n1E[X1:n]=xθnn(θx)n1dx=n+11θE[X1:n2]=x2θnn(θx)n1dx=n(n+1)2θ2E(θ 3)=E(x1:n)+E(xn:n)=θVar(θ^3)=Var(X1:n)+Var(Xn:n)+2Cov(X1:n,Xn:n)=n(n+1)2θ2(n+11θ)2+n+2nθ2(n+1nθ)2+2Cov(X1:n,Xn:n)Since the joint distribution of the order statistics of the uniform distribution isfui,vj(u,v)=n!(i1)!ui1(ji1)!(vu)ji1(nj)!(1v)njCov(uk,vj)=(n1)2(n+2)j(nk1)Var(θ^3)=n(n+2)2θ2+(n+1)2(n+2)2n2θ2
Therefore, θ 3 \theta_3 θ3 is a biased estimator.

Estimator 4: Mean + 3 Std estimator

θ ^ 4 = ∑ i = 1 n X i n + 3 ∑ i = 1 n ( X i − X ˉ ) 2 n − 1 = ∑ i = 1 n X i n + 3 S E ( θ ^ 4 ) = E [ X ‾ ] + 3 ∗ E ( S ) = 1 2 θ + 3 θ 2 3 V a r ( θ ^ 4 ) > V a r ( X ‾ ) = θ 2 12 n ∴ θ ^ 4   i s   a   b i a s e d   e s t i m a t o r \widehat{\theta}_4=\frac{\sum_{i=1}^n X_i}{n}+3\sqrt{\frac{\sum_{i=1}^n(X_i-\bar{X})^2}{n-1}}= \frac{\sum_{i=1}^n X_i}{n}+3S\\ E(\widehat{\theta}_4)=E[\overline{X}]+3*E(S)=\frac{1}{2} \theta+3\frac{\theta}{2\sqrt{3}}\\ Var(\widehat{\theta}_4)>Var(\overline{X})=\frac{\theta^2}{12n}\\ \therefore \widehat{\theta}_4~is~a~biased~estimator\\ θ 4=ni=1nXi+3n1i=1n(XiXˉ)2 =ni=1nXi+3SE(θ 4)=E[X]+3E(S)=21θ+323 θVar(θ 4)>Var(X)=12nθ2θ 4 is a biased estimator

MLE estimator

f x 1 , x 2 , . . . , x n ( x 1 , x 2 , . . . , x n ) = ∏ i = 1 n f x i ( x i ) = ∏ i = 1 n 1 θ = 1 θ n l n ( f x 1 , x 2 , . . . , x n ( x 1 , x 2 , . . . , x n ) ) = − n l n θ ∴ θ ^ M L E = X n : n F X n : n ( x ) = [ F X ( x ) ] n = x n θ n , f X n : n ( x ) = n x n − 1 θ n E [ X n : n ] = ∫ x n x n − 1 θ n d x = n n + 1 θ E [ X n : n 2 ] = ∫ x 2 n x n − 1 θ n d x = n n + 2 θ 2 E ( θ ^ M L E ) = E ( X n : n ) = n n + 1 θ V a r ( θ ^ M L E ) = n n + 2 θ 2 − ( n n + 1 θ ) 2 = n ( n + 1 ) 2 ( n + 2 ) θ 2 ∴ t h e   M L E   e s t i m a t o r   i s   a n   b i a s e d   e s t i m a t o r . f ( x ) = { 1 θ ,   0 < x < θ 0 ,            0. w . f ( x ) = 1 θ I 0 , θ ( x ) f ( x 1 , . . . , x n ) = 1 θ n ∏ i = 1 n I 0 , θ ( x i ) = 1 θ n I 0 , θ ( x n : n ) = g ( x n : n , θ ) ∗ h ( x 1 , . . . , x n ) ∴   S = x n : n   i s   s u f f i c i e n t   f o r   θ ∴ t h e   M L E   e s t i m a t o r   i s   s u f f i c i e n t . f_{x_1,x_2,...,x_n}(x_1,x_2,...,x_n) = \prod_{i=1}^{n}f_{x_i}(x_i)\\ = \prod_{i=1}^{n}\frac{1}{\theta}\\ = \frac{1}{\theta^n}\\ ln(f_{x_1,x_2,...,x_n}(x_1,x_2,...,x_n))=-nln\theta\\ \therefore \widehat{\theta}_{MLE} = X_{n:n}\\ F_{X_{n:n}}(x) =[F_X(x)]^n=\frac{x^n}{\theta^n},f_{X_{n:n}}(x)=n\frac{x^{n-1}}{\theta^n}\\ E[X_{n:n}]=\int xn\frac{x^{n-1}}{\theta^n}dx=\frac{n}{n+1}\theta\\ E[X_{n:n}^2] =\int x^2n\frac{x^{n-1}}{\theta^n}dx=\frac{n}{n+2}\theta^2\\ E(\widehat{\theta}_{MLE})=E(X_{n:n})=\frac{n}{n+1}\theta\\ Var(\widehat{\theta}_{MLE})=\frac{n}{n+2}\theta^2-(\frac{n}{n+1}\theta)^2=\frac{n}{(n+1)^2(n+2)}\theta^2\\ \therefore the ~MLE~estimator~is~an ~biased~estimator.\\ f(x)= \left\{\begin{matrix} \frac{1}{\theta},~0<x<\theta\\ 0,~~~~~~~~~~0.w. \end{matrix}\right.\\ f(x)=\frac{1}{\theta}I_{0,\theta}(x)\\ f(x_1,...,x_n)=\frac{1}{\theta^n}\prod_{i=1}^{n}I_{0,\theta}(x_i)\\ = \frac{1}{\theta^n}I_{0,\theta}(x_{n:n})\\ = g(x_{n:n},\theta)*h(x_1,...,x_n)\\ \therefore~S=x_{n:n}~is~sufficient~for~\theta\\ \therefore the ~MLE~estimator~is~sufficient. fx1,x2,...,xn(x1,x2,...,xn)=i=1nfxi(xi)=i=1nθ1=θn1ln(fx1,x2,...,xn(x1,x2,...,xn))=nlnθθ MLE=Xn:nFXn:n(x)=[FX(x)]n=θnxn,fXn:n(x)=nθnxn1E[Xn:n]=xnθnxn1dx=n+1nθE[Xn:n2]=x2nθnxn1dx=n+2nθ2E(θ MLE)=E(Xn:n)=n+1nθVar(θ MLE)=n+2nθ2(n+1nθ)2=(n+1)2(n+2)nθ2the MLE estimator is an biased estimator.f(x)={θ1, 0<x<θ0,          0.w.f(x)=θ1I0,θ(x)f(x1,...,xn)=θn1i=1nI0,θ(xi)=θn1I0,θ(xn:n)=g(xn:n,θ)h(x1,...,xn) S=xn:n is sufficient for θthe MLE estimator is sufficient.

Estimator 5: An improvement from MLE: UMVUE

UMVUE under method 1

Consider an improved estimator from θ ^ \hat{\theta} θ^, we have:

θ ^ 5 = n + 1 n x n : n − 1 … f o r   d i s c r e t e   c a s e . θ ^ 5 = n + 1 n x n : n … f o r   c o n t i n u o u s   c a s e . \hat{\theta}_5 = \frac{n+1}{n}x_{n:n} - 1 \quad \dots for~discrete~case. \\ \hat{\theta}_5 = \frac{n+1}{n}x_{n:n} \quad \dots for~continuous~case. θ^5=nn+1xn:n1for discrete case.θ^5=nn+1xn:nfor continuous case.
Get the variance to determine if the estimator has been improved to an unbiased estimator where V a r ( x ) = E ( x 2 ) − [ E ( x ) ] 2 Var(x) = \mathbb{E}(x^2) - [\mathbb{E}(x)]^2 Var(x)=E(x2)[E(x)]2.

E ( θ ^ 5 ) = θ E ( X n : n 2 ) = ∫ x 2 ⋅ n ⋅ x n − 1 θ n d x = n θ 2 n + 2 V a r ( θ ^ 5 ) = ( n + 1 n ) 2 ⋅ V a r ( X n : n ) = ( n + 1 n ) 2 ( n θ 2 n + 2 − ( n θ n + 1 ) 2 ) = θ 2 n ( n + 2 ) E(\hat{\theta}_5)=\theta\\ E(X_{n:n}^2) = \int x^2\cdot\frac{n\cdot x^{n-1}}{\theta^n}dx=\frac{n\theta^2}{n+2} \\ Var(\hat{\theta}_5) = \Big(\frac{n+1}{n}\Big)^2\cdot Var(X_{n:n}) \\ = \Big(\frac{n+1}{n}\Big)^2\Big(\frac{n\theta^2}{n+2}-\big(\frac{n\theta}{n+1}\big)^2\Big) = \frac{\theta^2}{n(n+2)} E(θ^5)=θE(Xn:n2)=x2θnnxn1dx=n+2nθ2Var(θ^5)=(nn+1)2Var(Xn:n)=(nn+1)2(n+2nθ2(n+1nθ)2)=n(n+2)θ2
Therefore, θ ^ 5 \hat{\theta}_5 θ^5 is an unbiased estimator.

To obtain the maximum value of θ \theta θ, the best option is to get the maximum value of each random sample m (i.e. θ ^ = x n : n \hat{\theta} = x_{n:n} θ^=xn:n). According to the Ex 3.4, we know that

F x n : n ( x ) = [ F x ( x ) ] n = x n θ n → f x n : n ( x ) = n ⋅ x n − 1 θ n F_{x_{n:n}}(x) = [F_x(x)]^n = \frac{x^n}{\theta^n}\quad\rightarrow\quad f_{x_{n:n}}(x) = \frac{n\cdot x^{n-1}}{\theta^n} Fxn:n(x)=[Fx(x)]n=θnxnfxn:n(x)=θnnxn1

where n is the total number of sample in each day. To obtain the expected value of m (i.e. x n : n x_{n:n} xn:n), we can determine if the estimator is an biased estimator such that:

E ( x n : n ) = ∫ x ⋅ n ⋅ x n − 1 θ n d x = n θ n + 1 = E ( θ ^ ) ≠ θ \mathbb{E}(x_{n:n}) = \int x\cdot \frac{n\cdot x^{n-1}}{\theta^n}dx = \frac{n\theta}{n+1} = \mathbb{E}(\hat{\theta})\neq \theta E(xn:n)=xθnnxn1dx=n+1nθ=E(θ^)=θ

Therefore, the estimator is an biased estimator.
f ( x ) = { 1 θ   0 < x < θ 0 o . w . f ( x ) = 1 θ I 0 , θ ( x ) f ( x 1 , . . . , x n ) = 1 θ n ∏ i = 1 n I 0 , θ ( x i ) = 1 θ n I 0 , θ ( x n : n ) = g ( x n : n , θ ) ⋅ h ( x 1 , . . . , x n ) f(x)=\left\{\begin{matrix} \frac{1}{\theta} ~0<x<\theta\\ 0 o.w. \end{matrix}\right.\\ f(x) = \frac{1}{\theta}I_{0,\theta}(x) \\ f(x_1,...,x_n) = \frac{1}{\theta^n}\prod_{i=1}^{n}I_{0,\theta}(x_i) = \frac{1}{\theta^n}I_{0,\theta}(x_{n:n}) \\ = g(x_{n:n},\theta)\cdot h(x_1,...,x_n) f(x)={θ1 0<x<θ0o.w.f(x)=θ1I0,θ(x)f(x1,...,xn)=θn1i=1nI0,θ(xi)=θn1I0,θ(xn:n)=g(xn:n,θ)h(x1,...,xn)
Therefore, S = x n : n S = x_{n:n} S=xn:n is sufficient for θ \theta θ. According to Lehmann-Scheffe Theorem, we have:

T = θ ^ 2 = n + 1 n X n : n T = \hat{\theta}_2 = \frac{n+1}{n}X_{n:n} T=θ^2=nn+1Xn:n

which is unbiased for τ ( θ ) = θ \tau(\theta)=\theta τ(θ)=θ. Thus, T T T is UMVUE.

UMVUE under method 2

The expected value of M = m M = m M=m can be calculated as follows where C k N = N ! k ! ( N − k ) ! C_k^N = \frac{N!}{k!(N-k)!} CkN=k!(Nk)!N!, ∑ m = k N C k m = C k + 1 N + 1 \sum_{m=k}^NC_k^m = C_{k+1}^{N+1} m=kNCkm=Ck+1N+1:

E ( M = m ) = ∑ m = k N m ⋅ P ( m ) = ∑ m = k N m ⋅ ( m − 1 ) ! ( k − 1 ) ! ( m − k ) ! N ! k ! ( N − k ) ! = ∑ m = k N m ! k ( N − k ) ! k ! k ! ( m − k ) ! N ! = k ( N − k ) ! k ! N ! ⋅ ∑ m = k N C k m = k ( N − k ) ! k ! N ! ⋅ ( N + 1 ) ! ( k + 1 ) ! ( N − k ) ! = k ( N + 1 ) k + 1 \mathbb{E}(M=m) = \sum_{m=k}^Nm\cdot P(m) \\ = \sum_{m=k}^Nm\cdot\frac{\frac{(m-1)!}{(k-1)!(m-k)!}}{\frac{N!}{k!(N-k)!}} \\ = \sum_{m=k}^N\frac{m!k(N-k)!k!}{k!(m-k)!N!} \\ = \frac{k(N-k)!k!}{N!}\cdot\sum_{m=k}^NC_k^m \\ = \frac{k(N-k)!k!}{N!}\cdot \frac{(N+1)!}{(k+1)!(N-k)!} = \frac{k(N+1)}{k+1} E(M=m)=m=kNmP(m)=m=kNmk!(Nk)!N!(k1)!(mk)!(m1)!=m=kNk!(mk)!N!m!k(Nk)!k!=N!k(Nk)!k!m=kNCkm=N!k(Nk)!k!(k+1)!(Nk)!(N+1)!=k+1k(N+1)
Since we are looking for the maximum ID from our observation, the best guess of M should be the maximum ID of THAT particular day m. Therefore, we get:
m = k ( N + 1 ) k + 1 → k N ^ = m k + m − k → N ^ = m + m k − 1 m = \frac{k(N+1)}{k+1}\quad\rightarrow\quad k\hat{N} = mk + m - k\quad\rightarrow\quad \hat{N} = m + \frac{m}{k} - 1 m=k+1k(N+1)kN^=mk+mkN^=m+km1
By Finding the expected value of N ^ \hat{N} N^, we have:

E ( N ^ ) = E [ E ( M ) + E ( M ) k − 1 ] = E ( M ) + E ( M ) k − 1 = k ( N + 1 ) k + 1 + N + 1 k + 1 − k + 1 k + 1 = N ( k + 1 ) k + 1 = N \mathbb{E}(\hat{N}) = \mathbb{E}\Big[\mathbb{E}(M) + \frac{\mathbb{E}(M)}{k} - 1\Big] = \mathbb{E}(M) + \frac{\mathbb{E}(M)}{k} - 1 \\ = \frac{k(N+1)}{k+1} + \frac{N+1}{k+1} - \frac{k+1}{k+1} = \frac{N(k+1)}{k+1} = N E(N^)=E[E(M)+kE(M)1]=E(M)+kE(M)1=k+1k(N+1)+k+1N+1k+1k+1=k+1N(k+1)=N
Therefore, N ^ \hat{N} N^ is proved to be unbiased.

Estimator 6: Bayes Estimator

The Bayesian approach is to consider the credibility P ( N = n ∣ M = m , K = k ) P(N=n|M=m, K=k) P(N=nM=m,K=k) that the maximum random ID N N N is equal to the number n n n, and the maximum observed serial number M M M is equal to the number m m m. Consider Conditional probability rule instead of using a proper prior distribution.

P ( n ∣ m , k ) P ( m ∣ k ) = P ( m ∣ n , k ) P ( n ∣ k ) = P ( m , n ∣ k ) P(n|m, k)P(m|k)=P(m|n,k)P(n|k)=P(m,n|k) P(nm,k)P(mk)=P(mn,k)P(nk)=P(m,nk)

P ( m ∣ n , k ) P(m|n,k) P(mn,k) answers the question: “What is the probability of a specific serial number m m m being the highest number observed in a sample of k k k patients, given there are n n n in total?” The probability of this occurring is:
P ( m ∣ n , k ) = k ( n − k ) ! n ! ( m − 1 ) ! ( m − k ) ! = C k − 1 m − 1 C k n I k ≤ m I m ≤ n P(m|n,k) =k\frac{(n-k)!}{n!}\frac{(m-1)!}{(m-k)!}=\frac{C_{k-1}^{m-1}}{C_k^n}I_{k\leq m}I_{m\leq n} P(mn,k)=kn!(nk)!(mk)!(m1)!=CknCk1m1IkmImn
P ( m ∣ k ) P(m|k) P(mk) is the probability that the maximum serial number is equal to m m m once k k k tanks have been observed but before the serial numbers have actually been observed.
P ( m ∣ k ) = P ( m ∣ k ) ∗ ∑ n = 0 ∞ P ( n ∣ m , k ) = P ( m ∣ k ) ∗ ∑ n = 0 ∞ P ( m ∣ n , k ) P ( n ∣ k ) P ( m ∣ k ) = ∑ n = 0 ∞ P ( m ∣ n , k ) P ( n ∣ k ) P(m|k) = P(m|k)*\sum_{n=0}^\infty P(n|m,k)\\ =P(m|k)*\sum_{n=0}^\infty \frac{P(m|n,k)P(n|k)}{P(m|k)}\\ =\sum_{n=0}^\infty P(m|n,k)P(n|k)\\ P(mk)=P(mk)n=0P(nm,k)=P(mk)n=0P(mk)P(mn,k)P(nk)=n=0P(mn,k)P(nk)
P ( n ∣ k ) P(n|k) P(nk) is the credibility that the total number of tanks, N N N, is equal to n n n when the number K K K patients observed is known to be k k k, but before the serial numbers have been observed. Assume that it is some discrete uniform distribution:

P ( n ∣ k ) = 1 Ω − k I k ≤ Ω I n ≤ Ω w h e r e    Ω   i s   t h e   u p p e r   l i m i t   a n d   i t ′ s   f i n i t e P(n|k) = \frac{1}{\Omega-k}I_{k \leq \Omega }I_{n \leq \Omega}\\ where~~\Omega ~is~the~upper ~limit~and ~it's~finite\\ P(nk)=Ωk1IkΩInΩwhere  Ω is the upper limit and its finite

P ( n ∣ m , k ) = P ( m ∣ n , k ) P ( n ∣ k ) P ( m ∣ k ) = P ( m ∣ n , k ) P ( n ∣ k ) ∑ n = 0 ∞ P ( m ∣ n , k ) P ( n ∣ k ) , k ≤ m , m ≤ n , k ≤ Ω , n ≤ Ω = P ( m ∣ n , k ) ∑ n = m Ω − 1 P ( m ∣ n , k ) I m ≤ n I n ≤ Ω f o r   k ≥ 2 ,   P ( n ∣ m , k ) = P ( m ∣ n , k ) ∑ n = m ∞ P ( m ∣ n , k ) I m ≤ n = C k − 1 m − 1 C k n ∑ n = m ∞ C k − 1 m − 1 C k n I m ≤ n = k − 1 k C k − 1 m − 1 C k n I m ≤ n P ( N > x ∣ M = m , K = k ) = ∑ n = x + 1 ∞ P ( n ∣ m , k ) I m ≤ x = I m < x + I m ≥ x ∑ n = x + 1 ∞ k − 1 k C k − 1 m − 1 C k n = I m < x + I m ≥ x k − 1 k C k − 1 m − 1 1 ∑ n = x + 1 ∞ 1 C k n = I m < x + I m ≥ x k − 1 k C k − 1 m − 1 1 k k − 1 1 C k − 1 x = I m < x + I m ≥ x C k − 1 m − 1 C k − 1 x P ( N ≤ x ∣ M = m , K = k ) = 1 − P ( N > x ∣ M = m , K = k ) = I m ≥ x ( 1 − C k − 1 m − 1 C k − 1 x ) μ B a y e s = ∑ n n P ( n ∣ m , k ) = ∑ n k − 1 k C k − 1 m − 1 C k n I m ≤ n = ∑ n m − 1 n C k − 2 m − 2 C k − 1 n − 1 I m ≤ n = ( m − 1 ) C k − 2 m − 2 ∑ n ≥ n 1 C k − 1 n − 1 = ( m − 1 ) C k − 2 m − 2 k − 1 k − 2 1 C k − 2 m − 2 = ( m − 1 ) ( k − 1 ) k − 2 P(n|m, k)= \frac{P(m|n,k)P(n|k)}{P(m|k)}\\ =\frac{P(m|n,k)P(n|k)}{\sum_{n=0}^\infty P(m|n,k)P(n|k)},k\leq m,m\leq n,k \leq \Omega,n \leq \Omega\\ =\frac{P(m|n,k)}{\sum_{n=m}^{\Omega-1} P(m|n,k)}I_{m\leq n}I_{n\leq \Omega}\\ for~k \geq 2, ~P(n|m, k)= \frac{P(m|n,k)}{\sum_{n=m}^{\infty } P(m|n,k)}I_{m\leq n}\\ =\frac{\frac{C_{k-1}^{m-1}}{C_k^n}}{\sum_{n=m}^{\infty}\frac{C_{k-1}^{m-1}}{C_k^n}}I_{m\leq n}\\ = \frac{k-1}{k}\frac{C_{k-1}^{m-1}}{C_k^n}I_{m\leq n}\\ P(N>x|M=m, K=k)=\sum_{n=x+1}^{\infty }P(n|m, k)I_{m\leq x}\\ =I_{m< x}+I_{m\geq x}\sum_{n=x+1}^{\infty }\frac{k-1}{k}\frac{C_{k-1}^{m-1}}{C_k^n}\\ =I_{m< x}+I_{m\geq x}\frac{k-1}{k}\frac{C_{k-1}^{m-1}}{1}\sum_{n=x+1}^{\infty }\frac{1}{C_k^n}\\ =I_{m< x}+I_{m\geq x}\frac{k-1}{k}\frac{C_{k-1}^{m-1}}{1}\frac{k}{k-1}\frac{1}{C_{k-1}^x}\\ =I_{m< x}+I_{m\geq x}\frac{C_{k-1}^{m-1}}{C_{k-1}^x}\\ P(N\leq x|M=m, K=k)=1-P(N>x|M=m, K=k)\\ =I_{m\geq x}(1-\frac{C_{k-1}^{m-1}}{C_{k-1}^x})\\ \mu_{Bayes}=\sum_{n}nP(n|m, k)\\ =\sum_{n}\frac{k-1}{k}\frac{C_{k-1}^{m-1}}{C_k^n}I_{m\leq n}\\ =\sum_{n}\frac{m-1}{n}\frac{C_{k-2}^{m-2}}{C_{k-1}^{n-1}}I_{m\leq n}\\ =(m-1)C_{k-2}^{m-2}\sum_{n\geq n}\frac{1}{C_{k-1}^{n-1}}\\ =(m-1)C_{k-2}^{m-2}\frac{k-1}{k-2}\frac{1}{C_{k-2}^{m-2}}\\ =\frac{(m-1)(k-1)}{k-2}\\ P(nm,k)=P(mk)P(mn,k)P(nk)=n=0P(mn,k)P(nk)P(mn,k)P(nk),km,mn,kΩ,nΩ=n=mΩ1P(mn,k)P(mn,k)ImnInΩfor k2, P(nm,k)=n=mP(mn,k)P(mn,k)Imn=n=mCknCk1m1CknCk1m1Imn=kk1CknCk1m1ImnP(N>xM=m,K=k)=n=x+1P(nm,k)Imx=Im<x+Imxn=x+1kk1CknCk1m1=Im<x+Imxkk11Ck1m1n=x+1Ckn1=Im<x+Imxkk11Ck1m1k1kCk1x1=Im<x+ImxCk1xCk1m1P(NxM=m,K=k)=1P(N>xM=m,K=k)=Imx(1Ck1xCk1m1)μBayes=nnP(nm,k)=nkk1CknCk1m1Imn=nnm1Ck1n1Ck2m2Imn=(m1)Ck2m2nnCk1n11=(m1)Ck2m2k2k1Ck2m21=k2(m1)(k1)

Hence, μ B a y e s = ( X n : n − 1 ) ( n − 1 ) n − 2 \mu_{Bayes}=\frac{(X_{n:n}-1)(n-1)}{n-2} μBayes=n2(Xn:n1)(n1) using the standard of writing in other chapter.
E ( μ B a y e s ) = n n + 2 θ E(\mu_{Bayes})=\frac{n}{n+2}\theta E(μBayes)=n+2nθ The Bayes estimator is a biased one.
To measure its uncertainty, we calculate its variance:
μ 2 + σ 2 − μ = ∑ n n ( n − 1 ) P ( n ∣ m , k ) = ∑ n n ( n − 1 ) m − 1 n m − 2 n − 1 k − 1 k − 2 C k − 3 m − 3 C k − 2 n − 2 I m ≤ n = ( m − 1 ) ( m − 2 ) k − 1 k − 2 C k − 3 m − 3 ∑ n ≥ m 1 C k − 2 n − 2 = ( m − 1 ) ( m − 2 ) k − 1 k − 2 C k − 3 m − 3 k − 2 k − 3 1 C k − 3 m − 3 = ( m − 1 ) ( m − 2 ) ( k − 1 ) k − 3 σ B a y e s 2 = ( m − 1 ) ( m − 2 ) ( k − 1 ) k − 3 − ( ( m − 1 ) ( k − 1 ) k − 2 ) 2 + ( m − 1 ) ( k − 1 ) k − 2 = ( m − 1 ) ( k − 1 ) ( m + 1 − k ) ( k − 3 ) ( k − 2 ) 2 V a r ( θ ^ B a y e s ) = ( x n : n − 1 ) ( n − 1 ) ( x n : n + 1 − n ) ( n − 3 ) ( n − 2 ) 2 \mu^2+\sigma^2-\mu=\sum_{n}n(n-1)P(n|m, k)\\ =\sum_{n}n(n-1)\frac{m-1}{n}\frac{m-2}{n-1}\frac{k-1}{k-2}\frac{C_{k-3}^{m-3}}{C_{k-2}^{n-2}}I_{m\leq n}\\ =(m-1)(m-2)\frac{k-1}{k-2}C_{k-3}^{m-3}\sum_{n\geq m}\frac{1}{C_{k-2}^{n-2}}\\ =(m-1)(m-2)\frac{k-1}{k-2}C_{k-3}^{m-3}\frac{k-2}{k-3}\frac{1}{C_{k-3}^{m-3}}\\ =\frac{(m-1)(m-2)(k-1)}{k-3}\\ \sigma^2_{Bayes}=\frac{(m-1)(m-2)(k-1)}{k-3}-(\frac{(m-1)(k-1)}{k-2})^2+\frac{(m-1)(k-1)}{k-2}\\ =\frac{(m-1)(k-1)(m+1-k)}{(k-3)(k-2)^2}\\ Var(\widehat{\theta}_{Bayes}) =\frac{(x_{n:n}-1)(n-1)(x_{n:n}+1-n)}{(n-3)(n-2)^2} μ2+σ2μ=nn(n1)P(nm,k)=nn(n1)nm1n1m2k2k1Ck2n2Ck3m3Imn=(m1)(m2)k2k1Ck3m3nmCk2n21=(m1)(m2)k2k1Ck3m3k3k2Ck3m31=k3(m1)(m2)(k1)σBayes2=k3(m1)(m2)(k1)(k2(m1)(k1))2+k2(m1)(k1)=(k3)(k2)2(m1)(k1)(m+1k)Var(θ Bayes)=(n3)(n2)2(xn:n1)(n1)(xn:n+1n)

Point Estimation Conclusion

According to the distribution of the question, we find six possible estimators, in which four of them are intuitive from the distribution and the background of the question, one is maximum likelihood estimator (MLE) and one the Bayes estimator and the improved estimator from MLE. We proved that the improved estimator from MLE is exactly uniformly minimum-variance unbiased estimator (UMVUE) which is unbiased estimator with the smallest variance.

Also, what is most important in our findings is the X n : n X_{n:n} Xn:n plays an important role in estimating the upper limit of th discrete uniform distribution since the maximum sample give the closest information of the upper limit intuitively and we also proved that it’ s the sufficient statistics to estimate N N N.

To compare the unbiasedness, effectiveness of the estimators we find, we summarize the results and give the following table:

No. F u n c t i o n Function Function E ( θ ^ ) E(\widehat{\theta}) E(θ ) V a r ( θ ^ ) Var(\widehat{\theta}) Var(θ )
θ ^ 1 \widehat{\theta}_1 θ 1 2 n ∑ i = 1 n X i − 1 \frac{2}{n}\sum^n_{i=1}X_i-1 n2i=1nXi1 θ \theta θ θ 2 3 n \frac{\theta^2}{3n} 3nθ2
θ ^ 2 \widehat{\theta}_2 θ 2 X n : n + 1 n − 1 ∑ i > j ( X i − X j − 1 ) X_{n:n}+\frac{1}{n-1}\sum_{i>j}(X_i-X_j-1) Xn:n+n11i>j(XiXj1) n θ n + 1 \frac{n\theta}{n+1} n+1nθ n θ 2 ( n + 1 ) ( n − 1 ) ( n + 2 ) \frac{n\theta^2}{(n+1)(n-1)(n+2)} (n+1)(n1)(n+2)nθ2
θ ^ 3 \widehat{\theta}_3 θ 3 x 1 : n + x n : n x_{1:n}+x_{n:n} x1:n+xn:n θ \theta θ 2 θ 2 n ( n + 2 ) + 2 n 2 θ 2 ( n + 1 ) 2 ( n + 2 ) \frac{2\theta^2}{n(n+2)}+\frac{2n^2\theta^2}{(n+1)^2(n+2)} n(n+2)2θ2+(n+1)2(n+2)2n2θ2
θ ^ 4 \widehat{\theta}_4 θ 4 ∑ i = 1 n X i n + 3 ∑ E ( X i − X ‾ ) 2 n − 1 \frac{\sum_{i=1}^n X_i}{n}+3\sqrt{ \frac{\sum E(X_i-\overline{X})^2}{n-1}} ni=1nXi+3n1E(XiX)2 1 2 θ + 2 3 3 θ \frac{1}{2}\theta+\frac{2\sqrt{3}}{3}\theta 21θ+323 θ V a r ( θ ^ 4 ) > θ 2 12 n Var(\widehat{\theta}_4)>\frac{\theta^2}{12n} Var(θ 4)>12nθ2
θ ^ 5 \hat{\theta}_5 θ^5 n + 1 n x n : n − 1 \frac{n+1}{n}x_{n:n} - 1 nn+1xn:n1 θ \theta θ θ 2 n ( n + 2 ) \frac{\theta^2}{n(n+2)} n(n+2)θ2
θ ^ M L E \widehat{\theta}_{MLE} θ MLE X n : n X_{n:n} Xn:n n n + 1 θ \frac{n}{n+1}\theta n+1nθ n ( n + 1 ) 2 ( n + 2 ) θ 2 \frac{n}{(n+1)^2(n+2)}\theta^2 (n+1)2(n+2)nθ2
θ ^ B a y e s \hat{\theta}_{Bayes} θ^Bayes ( X n : n − 1 ) ( n − 1 ) n − 2 \frac{(X_{n:n}-1)(n-1)}{n-2} n2(Xn:n1)(n1) n n + 2 θ \frac{n}{n+2}\theta n+2nθ ( x n : n − 1 ) ( n − 1 ) ( x n : n + 1 − n ) ( n − 3 ) ( n − 2 ) 2 \frac{(x_{n:n}-1)(n-1)(x_{n:n}+1-n)}{(n-3)(n-2)^2} (n3)(n2)2(xn:n1)(n1)(xn:n+1n)

Interval Estimation

In addition to point estimation, interval estimation can be carried out. Based on the observation that the probability that k observations in the sample will fall in an interval covering p of the range (0 ≤ p ≤ 1) is p k p^k pk(assuming in this section that draws are with replacement, to simplify computations; if draws are without replacement, this overstates the likelihood, and intervals will be overly conservative).

Thus the sampling distribution of the quantile of the sample maximum is the graph x 1 / k x^{1/k} x1/k from 0 to 1: the p-th to q-th quantile of the sample maximum m are the interval [ p 1 / k N p^{1/k}N p1/kN, q 1 / k N q^{1/k}N q1/kN]. Inverting this yields the corresponding confidence interval for the population maximum of [ m / q 1 / k m/q^{1/k} m/q1/k, m / p 1 / k m/p^{1/k} m/p1/k].

Reference

欢迎关注二幺子的知识输出通道:
avatar

已标记关键词 清除标记
相关推荐
©️2020 CSDN 皮肤主题: 数字20 设计师:CSDN官方博客 返回首页