# Serial Number Estimation

## Parameter description

• If θ \theta is contained in a function, n n would be the total sample numbers, and θ ^ \hat{\theta} would be the estimator for actual maximum ID.
• If M M is contained in a function, k k would be the total sample numbers, and N N would be the actual maximum ID. As for M M , that is a random variable of maximum ID in a random sample.

### Method 1: Probability of each sample

The estimator used to predict the maximum value can also be determined by assuming that the probability of getting each sample is uniform where θ \theta represents the actual maximum ID in each day.

P ( x ) = 1 θ P(x) = \frac{1}{\theta}

### Method 2: Probability of maximum sample

According to the Assumption 1, we consider the observed maximum ID as a r.v M, and take the maximum ID we encountered in one specific day as m (i.e. x n : n x_{n:n} ). Assume that the N is the actual maximum ID and k represents the number of ill sample, the probability mass function (PMF) of getting the maximum ID can be expressed as follows:

P ( M = m ) = C k − 1 m − 1 C k N P(M = m) = \frac{C_{k-1}^{m-1}}{C_k^N}

## Point Estimate

### Estimators intuited from discrete uniform distribution

#### Estimator 1: 2*Mean-1

Consider continuous distribution for this problem, i.e. U N I F ( 0 , θ ) UNIF(0,\theta)

F o r   U N I F ( 0 , θ ) , E ( X ) = θ 2 , V a r ( X ) = θ 2 12 For~UNIF(0,\theta),E(X)=\frac{\theta}{2},Var(X)=\frac{\theta^2}{12}

We consider the following estimator:
θ ^ 1 = 2 n ∑ i = 1 n X i − 1   f o r   d i s c r e t e   d i s t r u b u t i o n θ ^ 1 = 2 n ∑ i = 1 n X i   f o r   c o n t i n u o u s   d i s t r u b u t i o n E ( θ ^ 1 ) = E ( 2 n ∑ i = 1 n X i − 1 ) = 2 E ( X ‾ ) = θ V a r ( θ ^ 1 ) = 4 n 2 ∑ i = 1 n V a r ( X i ) = 4 n 2 ∑ i = 1 n θ 2 12 = θ 2 3 n ∴ θ ^ 1   i s   a n   u n b i a s e d   e s t i m a t o r   w i t h   V a r = θ 2 3 n \widehat{\theta}_1=\frac{2}{n}\sum^n_{i=1}X_i-1~for~discrete ~distrubution\\ \widehat{\theta}_1=\frac{2}{n}\sum^n_{i=1}X_i~for~continuous ~distrubution\\ E(\widehat{\theta}_1)=E(\frac{2}{n}\sum^n_{i=1}X_i-1)=2E(\overline{X})=\theta\\ Var(\widehat{\theta}_1)=\frac{4}{n^2}\sum_{i=1}^nVar(X_i)=\frac{4}{n^2}\sum_{i=1}^n\frac{\theta^2}{12}=\frac{\theta^2}{3n}\\ \therefore \widehat{\theta}_1~is~an~unbiased~estimator ~with~Var=\frac{\theta^2}{3n}\\

#### Estimator 2: Max + Avg GAP

Consider other form of improvement from MLE estimator, i.e. using average approach to estimate the GAP between maximum and the upper limit:

θ ^ 2 = X n : n + 1 n − 1 ∑ i > j ( X i − X j − 1 ) … f o r   d i s c r e t e   c a s e θ ^ 2 = X n : n + 1 n − 1 ∑ i > j ( X i − X j ) … f o r   c o n t i n u o u s   c a s e \widehat{\theta}_2 = X_{n:n}+\frac{1}{n-1}\sum_{i>j}(X_i-X_j-1)\quad\dots for~discrete~case\\ \widehat{\theta}_2 = X_{n:n}+\frac{1}{n-1}\sum_{i>j}(X_i-X_j) \quad\dots for~continuous~case

Calculate the expected value and variance to determine if this estimator is biased or not.
E ( θ ^ 2 ) = E ( X n : n ) + 1 n − 1 ∑ i > j E ( X i − X j ) = n θ n + 1 V a r ( θ ^ 2 ) = n θ 2 ( n + 1 ) ( n − 1 ) ( n + 2 ) E(\widehat{\theta}_2) = E(X_{n:n}) + \frac{1}{n-1}\sum_{i>j}E{(X_i-X_j)} = \frac{n\theta}{n+1} \\ Var(\hat{\theta}_2) = \frac{n\theta^2}{(n+1)(n-1)(n+2)}
Therefore, θ 2 \theta_2 is a biased estimator.

#### Estimator3: Min+max estimator

We know that maximum sample ID is what’s closed to the upper limit, and we could add more information to it. Intuitively, we first consider minimum sample ID + maximum sample ID:
θ ^ 3 = x 1 : n + x n : n F X n : n ( x ) = [ F X ( x ) ] n = x n θ n , f X n : n ( x ) = n x n − 1 θ n E [ X n : n ] = ∫ x n x n − 1 θ n d x = n n + 1 θ E [ X n : n 2 ] = ∫ x 2 n x n − 1 θ n d x = n n + 2 θ 2 F X 1 : n ( x ) = 1 − [ 1 − F X ( x ) ] n = 1 − ( θ − x θ ) n , f X 1 : n ( x ) = n ( θ − x ) n − 1 θ n E [ X 1 : n ] = ∫ x n ( θ − x ) n − 1 θ n d x = 1 n + 1 θ E [ X 1 : n 2 ] = ∫ x 2 n ( θ − x ) n − 1 θ n d x = 2 n ( n + 1 ) θ 2 E ( θ ^ 3 ) = E ( x 1 : n ) + E ( x n : n ) = θ V a r ( θ ^ 3 ) = V a r ( X 1 : n ) + V a r ( X n : n ) + 2 C o v ( X 1 : n , X n : n ) = 2 n ( n + 1 ) θ 2 − ( 1 n + 1 θ ) 2 + n n + 2 θ 2 − ( n n + 1 θ ) 2 + 2 C o v ( X 1 : n , X n : n ) S i n c e   t h e   j o i n t   d i s t r i b u t i o n   o f   t h e   o r d e r   s t a t i s t i c s   o f   t h e   u n i f o r m   d i s t r i b u t i o n   i s f u i , v j ( u , v ) = n ! u i − 1 ( i − 1 ) ! ( v − u ) j − i − 1 ( j − i − 1 ) ! ( 1 − v ) n − j ( n − j ) ! C o v ( u k , v j ) = j ( n − k − 1 ) ( n − 1 ) 2 ( n + 2 ) V a r ( θ ^ 3 ) = 2 θ 2 n ( n + 2 ) + 2 n 2 θ 2 ( n + 1 ) 2 ( n + 2 ) \widehat{\theta}_3=x_{1:n}+x_{n:n}\\ F_{X_{n:n}}(x) =[F_X(x)]^n=\frac{x^n}{\theta^n},f_{X_{n:n}}(x)=n\frac{x^{n-1}}{\theta^n}\\ E[X_{n:n}]=\int xn\frac{x^{n-1}}{\theta^n}dx=\frac{n}{n+1}\theta\\ E[X_{n:n}^2] =\int x^2n\frac{x^{n-1}}{\theta^n}dx=\frac{n}{n+2}\theta^2\\ F_{X_{1:n}}(x) =1-[1-F_X(x)]^n=1-(\frac{\theta-x}{\theta})^n,f_{X_{1:n}}(x)=\frac{n (\theta-x)^{n-1}}{\theta^n}\\ E[X_{1:n}]=\int x\frac{n (\theta-x)^{n-1}}{\theta^n}dx=\frac{1}{n+1}\theta\\ E[X_{1:n}^2] =\int x^2\frac{n (\theta-x)^{n-1}}{\theta^n}dx=\frac{2}{n(n+1)}\theta^2\\ E(\widehat{\theta}_3)=E(x_{1:n})+E(x_{n:n})=\theta\\ Var(\hat{\theta}_3)=Var(X_{1:n})+Var(X_{n:n})+2Cov(X_{1:n},X_{n:n})\\ =\frac{2}{n(n+1)}\theta^2-(\frac{1}{n+1}\theta)^2 +\frac{n}{n+2}\theta^2-(\frac{n}{n+1}\theta)^2+2Cov(X_{1:n},X_{n:n})\\ Since~the~ joint~ distribution~ of ~the~ order~ statistics~ of~ the~ uniform~ distribution~is\\ f_{u_i,v_j}(u,v)=n!\frac{u^{i-1}}{(i-1)!}\frac{(v-u)^{j-i-1}}{(j-i-1)!}\frac{(1-v)^{n-j}}{(n-j)!}\\ Cov(u_k,v_j)=\frac{j(n-k-1)}{(n-1)^2(n+2)}\\ Var(\hat{\theta}_3)=\frac{2\theta^2}{n(n+2)}+\frac{2n^2\theta^2}{(n+1)^2(n+2)}
Therefore, θ 3 \theta_3 is a biased estimator.

#### Estimator 4: Mean + 3 Std estimator

θ ^ 4 = ∑ i = 1 n X i n + 3 ∑ i = 1 n ( X i − X ˉ ) 2 n − 1 = ∑ i = 1 n X i n + 3 S E ( θ ^ 4 ) = E [ X ‾ ] + 3 ∗ E ( S ) = 1 2 θ + 3 θ 2 3 V a r ( θ ^ 4 ) > V a r ( X ‾ ) = θ 2 12 n ∴ θ ^ 4   i s   a   b i a s e d   e s t i m a t o r \widehat{\theta}_4=\frac{\sum_{i=1}^n X_i}{n}+3\sqrt{\frac{\sum_{i=1}^n(X_i-\bar{X})^2}{n-1}}= \frac{\sum_{i=1}^n X_i}{n}+3S\\ E(\widehat{\theta}_4)=E[\overline{X}]+3*E(S)=\frac{1}{2} \theta+3\frac{\theta}{2\sqrt{3}}\\ Var(\widehat{\theta}_4)>Var(\overline{X})=\frac{\theta^2}{12n}\\ \therefore \widehat{\theta}_4~is~a~biased~estimator\\

### MLE estimator

f x 1 , x 2 , . . . , x n ( x 1 , x 2 , . . . , x n ) = ∏ i = 1 n f x i ( x i ) = ∏ i = 1 n 1 θ = 1 θ n l n ( f x 1 , x 2 , . . . , x n ( x 1 , x 2 , . . . , x n ) ) = − n l n θ ∴ θ ^ M L E = X n : n F X n : n ( x ) = [ F X ( x ) ] n = x n θ n , f X n : n ( x ) = n x n − 1 θ n E [ X n : n ] = ∫ x n x n − 1 θ n d x = n n + 1 θ E [ X n : n 2 ] = ∫ x 2 n x n − 1 θ n d x = n n + 2 θ 2 E ( θ ^ M L E ) = E ( X n : n ) = n n + 1 θ V a r ( θ ^ M L E ) = n n + 2 θ 2 − ( n n + 1 θ ) 2 = n ( n + 1 ) 2 ( n + 2 ) θ 2 ∴ t h e   M L E   e s t i m a t o r   i s   a n   b i a s e d   e s t i m a t o r . f ( x ) = { 1 θ ,   0 < x < θ 0 ,            0. w . f ( x ) = 1 θ I 0 , θ ( x ) f ( x 1 , . . . , x n ) = 1 θ n ∏ i = 1 n I 0 , θ ( x i ) = 1 θ n I 0 , θ ( x n : n ) = g ( x n : n , θ ) ∗ h ( x 1 , . . . , x n ) ∴   S = x n : n   i s   s u f f i c i e n t   f o r   θ ∴ t h e   M L E   e s t i m a t o r   i s   s u f f i c i e n t . f_{x_1,x_2,...,x_n}(x_1,x_2,...,x_n) = \prod_{i=1}^{n}f_{x_i}(x_i)\\ = \prod_{i=1}^{n}\frac{1}{\theta}\\ = \frac{1}{\theta^n}\\ ln(f_{x_1,x_2,...,x_n}(x_1,x_2,...,x_n))=-nln\theta\\ \therefore \widehat{\theta}_{MLE} = X_{n:n}\\ F_{X_{n:n}}(x) =[F_X(x)]^n=\frac{x^n}{\theta^n},f_{X_{n:n}}(x)=n\frac{x^{n-1}}{\theta^n}\\ E[X_{n:n}]=\int xn\frac{x^{n-1}}{\theta^n}dx=\frac{n}{n+1}\theta\\ E[X_{n:n}^2] =\int x^2n\frac{x^{n-1}}{\theta^n}dx=\frac{n}{n+2}\theta^2\\ E(\widehat{\theta}_{MLE})=E(X_{n:n})=\frac{n}{n+1}\theta\\ Var(\widehat{\theta}_{MLE})=\frac{n}{n+2}\theta^2-(\frac{n}{n+1}\theta)^2=\frac{n}{(n+1)^2(n+2)}\theta^2\\ \therefore the ~MLE~estimator~is~an ~biased~estimator.\\ f(x)= \left\{\begin{matrix} \frac{1}{\theta},~0<x<\theta\\ 0,~~~~~~~~~~0.w. \end{matrix}\right.\\ f(x)=\frac{1}{\theta}I_{0,\theta}(x)\\ f(x_1,...,x_n)=\frac{1}{\theta^n}\prod_{i=1}^{n}I_{0,\theta}(x_i)\\ = \frac{1}{\theta^n}I_{0,\theta}(x_{n:n})\\ = g(x_{n:n},\theta)*h(x_1,...,x_n)\\ \therefore~S=x_{n:n}~is~sufficient~for~\theta\\ \therefore the ~MLE~estimator~is~sufficient.

### Estimator 5: An improvement from MLE: UMVUE

#### UMVUE under method 1

Consider an improved estimator from θ ^ \hat{\theta} , we have:

θ ^ 5 = n + 1 n x n : n − 1 … f o r   d i s c r e t e   c a s e . θ ^ 5 = n + 1 n x n : n … f o r   c o n t i n u o u s   c a s e . \hat{\theta}_5 = \frac{n+1}{n}x_{n:n} - 1 \quad \dots for~discrete~case. \\ \hat{\theta}_5 = \frac{n+1}{n}x_{n:n} \quad \dots for~continuous~case.
Get the variance to determine if the estimator has been improved to an unbiased estimator where V a r ( x ) = E ( x 2 ) − [ E ( x ) ] 2 Var(x) = \mathbb{E}(x^2) - [\mathbb{E}(x)]^2 .

E ( θ ^ 5 ) = θ E ( X n : n 2 ) = ∫ x 2 ⋅ n ⋅ x n − 1 θ n d x = n θ 2 n + 2 V a r ( θ ^ 5 ) = ( n + 1 n ) 2 ⋅ V a r ( X n : n ) = ( n + 1 n ) 2 ( n θ 2 n + 2 − ( n θ n + 1 ) 2 ) = θ 2 n ( n + 2 ) E(\hat{\theta}_5)=\theta\\ E(X_{n:n}^2) = \int x^2\cdot\frac{n\cdot x^{n-1}}{\theta^n}dx=\frac{n\theta^2}{n+2} \\ Var(\hat{\theta}_5) = \Big(\frac{n+1}{n}\Big)^2\cdot Var(X_{n:n}) \\ = \Big(\frac{n+1}{n}\Big)^2\Big(\frac{n\theta^2}{n+2}-\big(\frac{n\theta}{n+1}\big)^2\Big) = \frac{\theta^2}{n(n+2)}
Therefore, θ ^ 5 \hat{\theta}_5 is an unbiased estimator.

To obtain the maximum value of θ \theta , the best option is to get the maximum value of each random sample m (i.e. θ ^ = x n : n \hat{\theta} = x_{n:n} ). According to the Ex 3.4, we know that

F x n : n ( x ) = [ F x ( x ) ] n = x n θ n → f x n : n ( x ) = n ⋅ x n − 1 θ n F_{x_{n:n}}(x) = [F_x(x)]^n = \frac{x^n}{\theta^n}\quad\rightarrow\quad f_{x_{n:n}}(x) = \frac{n\cdot x^{n-1}}{\theta^n}

where n is the total number of sample in each day. To obtain the expected value of m (i.e. x n : n x_{n:n} ), we can determine if the estimator is an biased estimator such that:

E ( x n : n ) = ∫ x ⋅ n ⋅ x n − 1 θ n d x = n θ n + 1 = E ( θ ^ ) ≠ θ \mathbb{E}(x_{n:n}) = \int x\cdot \frac{n\cdot x^{n-1}}{\theta^n}dx = \frac{n\theta}{n+1} = \mathbb{E}(\hat{\theta})\neq \theta

Therefore, the estimator is an biased estimator.
f ( x ) = { 1 θ   0 < x < θ 0 o . w . f ( x ) = 1 θ I 0 , θ ( x ) f ( x 1 , . . . , x n ) = 1 θ n ∏ i = 1 n I 0 , θ ( x i ) = 1 θ n I 0 , θ ( x n : n ) = g ( x n : n , θ ) ⋅ h ( x 1 , . . . , x n ) f(x)=\left\{\begin{matrix} \frac{1}{\theta} ~0<x<\theta\\ 0 o.w. \end{matrix}\right.\\ f(x) = \frac{1}{\theta}I_{0,\theta}(x) \\ f(x_1,...,x_n) = \frac{1}{\theta^n}\prod_{i=1}^{n}I_{0,\theta}(x_i) = \frac{1}{\theta^n}I_{0,\theta}(x_{n:n}) \\ = g(x_{n:n},\theta)\cdot h(x_1,...,x_n)
Therefore, S = x n : n S = x_{n:n} is sufficient for θ \theta . According to Lehmann-Scheffe Theorem, we have:

T = θ ^ 2 = n + 1 n X n : n T = \hat{\theta}_2 = \frac{n+1}{n}X_{n:n}

which is unbiased for τ ( θ ) = θ \tau(\theta)=\theta . Thus, T T is UMVUE.

#### UMVUE under method 2

The expected value of M = m M = m can be calculated as follows where C k N = N ! k ! ( N − k ) ! C_k^N = \frac{N!}{k!(N-k)!} , ∑ m = k N C k m = C k + 1 N + 1 \sum_{m=k}^NC_k^m = C_{k+1}^{N+1} :

E ( M = m ) = ∑ m = k N m ⋅ P ( m ) = ∑ m = k N m ⋅ ( m − 1 ) ! ( k − 1 ) ! ( m − k ) ! N ! k ! ( N − k ) ! = ∑ m = k N m ! k ( N − k ) ! k ! k ! ( m − k ) ! N ! = k ( N − k ) ! k ! N ! ⋅ ∑ m = k N C k m = k ( N − k ) ! k ! N ! ⋅ ( N + 1 ) ! ( k + 1 ) ! ( N − k ) ! = k ( N + 1 ) k + 1 \mathbb{E}(M=m) = \sum_{m=k}^Nm\cdot P(m) \\ = \sum_{m=k}^Nm\cdot\frac{\frac{(m-1)!}{(k-1)!(m-k)!}}{\frac{N!}{k!(N-k)!}} \\ = \sum_{m=k}^N\frac{m!k(N-k)!k!}{k!(m-k)!N!} \\ = \frac{k(N-k)!k!}{N!}\cdot\sum_{m=k}^NC_k^m \\ = \frac{k(N-k)!k!}{N!}\cdot \frac{(N+1)!}{(k+1)!(N-k)!} = \frac{k(N+1)}{k+1}
Since we are looking for the maximum ID from our observation, the best guess of M should be the maximum ID of THAT particular day m. Therefore, we get:
m = k ( N + 1 ) k + 1 → k N ^ = m k + m − k → N ^ = m + m k − 1 m = \frac{k(N+1)}{k+1}\quad\rightarrow\quad k\hat{N} = mk + m - k\quad\rightarrow\quad \hat{N} = m + \frac{m}{k} - 1
By Finding the expected value of N ^ \hat{N} , we have:

E ( N ^ ) = E [ E ( M ) + E ( M ) k − 1 ] = E ( M ) + E ( M ) k − 1 = k ( N + 1 ) k + 1 + N + 1 k + 1 − k + 1 k + 1 = N ( k + 1 ) k + 1 = N \mathbb{E}(\hat{N}) = \mathbb{E}\Big[\mathbb{E}(M) + \frac{\mathbb{E}(M)}{k} - 1\Big] = \mathbb{E}(M) + \frac{\mathbb{E}(M)}{k} - 1 \\ = \frac{k(N+1)}{k+1} + \frac{N+1}{k+1} - \frac{k+1}{k+1} = \frac{N(k+1)}{k+1} = N
Therefore, N ^ \hat{N} is proved to be unbiased.

### Estimator 6: Bayes Estimator

The Bayesian approach is to consider the credibility P ( N = n ∣ M = m , K = k ) P(N=n|M=m, K=k) that the maximum random ID N N is equal to the number n n , and the maximum observed serial number M M is equal to the number m m . Consider Conditional probability rule instead of using a proper prior distribution.

P ( n ∣ m , k ) P ( m ∣ k ) = P ( m ∣ n , k ) P ( n ∣ k ) = P ( m , n ∣ k ) P(n|m, k)P(m|k)=P(m|n,k)P(n|k)=P(m,n|k)

P ( m ∣ n , k ) P(m|n,k) answers the question: “What is the probability of a specific serial number m m being the highest number observed in a sample of k k patients, given there are n n in total?” The probability of this occurring is:
P ( m ∣ n , k ) = k ( n − k ) ! n ! ( m − 1 ) ! ( m − k ) ! = C k − 1 m − 1 C k n I k ≤ m I m ≤ n P(m|n,k) =k\frac{(n-k)!}{n!}\frac{(m-1)!}{(m-k)!}=\frac{C_{k-1}^{m-1}}{C_k^n}I_{k\leq m}I_{m\leq n}
P ( m ∣ k ) P(m|k) is the probability that the maximum serial number is equal to m m once k k tanks have been observed but before the serial numbers have actually been observed.
P ( m ∣ k ) = P ( m ∣ k ) ∗ ∑ n = 0 ∞ P ( n ∣ m , k ) = P ( m ∣ k ) ∗ ∑ n = 0 ∞ P ( m ∣ n , k ) P ( n ∣ k ) P ( m ∣ k ) = ∑ n = 0 ∞ P ( m ∣ n , k ) P ( n ∣ k ) P(m|k) = P(m|k)*\sum_{n=0}^\infty P(n|m,k)\\ =P(m|k)*\sum_{n=0}^\infty \frac{P(m|n,k)P(n|k)}{P(m|k)}\\ =\sum_{n=0}^\infty P(m|n,k)P(n|k)\\
P ( n ∣ k ) P(n|k) is the credibility that the total number of tanks, N N , is equal to n n when the number K K patients observed is known to be k k , but before the serial numbers have been observed. Assume that it is some discrete uniform distribution:

P ( n ∣ k ) = 1 Ω − k I k ≤ Ω I n ≤ Ω w h e r e    Ω   i s   t h e   u p p e r   l i m i t   a n d   i t ′ s   f i n i t e P(n|k) = \frac{1}{\Omega-k}I_{k \leq \Omega }I_{n \leq \Omega}\\ where~~\Omega ~is~the~upper ~limit~and ~it's~finite\\

P ( n ∣ m , k ) = P ( m ∣ n , k ) P ( n ∣ k ) P ( m ∣ k ) = P ( m ∣ n , k ) P ( n ∣ k ) ∑ n = 0 ∞ P ( m ∣ n , k ) P ( n ∣ k ) , k ≤ m , m ≤ n , k ≤ Ω , n ≤ Ω = P ( m ∣ n , k ) ∑ n = m Ω − 1 P ( m ∣ n , k ) I m ≤ n I n ≤ Ω f o r   k ≥ 2 ,   P ( n ∣ m , k ) = P ( m ∣ n , k ) ∑ n = m ∞ P ( m ∣ n , k ) I m ≤ n = C k − 1 m − 1 C k n ∑ n = m ∞ C k − 1 m − 1 C k n I m ≤ n = k − 1 k C k − 1 m − 1 C k n I m ≤ n P ( N > x ∣ M = m , K = k ) = ∑ n = x + 1 ∞ P ( n ∣ m , k ) I m ≤ x = I m < x + I m ≥ x ∑ n = x + 1 ∞ k − 1 k C k − 1 m − 1 C k n = I m < x + I m ≥ x k − 1 k C k − 1 m − 1 1 ∑ n = x + 1 ∞ 1 C k n = I m < x + I m ≥ x k − 1 k C k − 1 m − 1 1 k k − 1 1 C k − 1 x = I m < x + I m ≥ x C k − 1 m − 1 C k − 1 x P ( N ≤ x ∣ M = m , K = k ) = 1 − P ( N > x ∣ M = m , K = k ) = I m ≥ x ( 1 − C k − 1 m − 1 C k − 1 x ) μ B a y e s = ∑ n n P ( n ∣ m , k ) = ∑ n k − 1 k C k − 1 m − 1 C k n I m ≤ n = ∑ n m − 1 n C k − 2 m − 2 C k − 1 n − 1 I m ≤ n = ( m − 1 ) C k − 2 m − 2 ∑ n ≥ n 1 C k − 1 n − 1 = ( m − 1 ) C k − 2 m − 2 k − 1 k − 2 1 C k − 2 m − 2 = ( m − 1 ) ( k − 1 ) k − 2 P(n|m, k)= \frac{P(m|n,k)P(n|k)}{P(m|k)}\\ =\frac{P(m|n,k)P(n|k)}{\sum_{n=0}^\infty P(m|n,k)P(n|k)},k\leq m,m\leq n,k \leq \Omega,n \leq \Omega\\ =\frac{P(m|n,k)}{\sum_{n=m}^{\Omega-1} P(m|n,k)}I_{m\leq n}I_{n\leq \Omega}\\ for~k \geq 2, ~P(n|m, k)= \frac{P(m|n,k)}{\sum_{n=m}^{\infty } P(m|n,k)}I_{m\leq n}\\ =\frac{\frac{C_{k-1}^{m-1}}{C_k^n}}{\sum_{n=m}^{\infty}\frac{C_{k-1}^{m-1}}{C_k^n}}I_{m\leq n}\\ = \frac{k-1}{k}\frac{C_{k-1}^{m-1}}{C_k^n}I_{m\leq n}\\ P(N>x|M=m, K=k)=\sum_{n=x+1}^{\infty }P(n|m, k)I_{m\leq x}\\ =I_{m< x}+I_{m\geq x}\sum_{n=x+1}^{\infty }\frac{k-1}{k}\frac{C_{k-1}^{m-1}}{C_k^n}\\ =I_{m< x}+I_{m\geq x}\frac{k-1}{k}\frac{C_{k-1}^{m-1}}{1}\sum_{n=x+1}^{\infty }\frac{1}{C_k^n}\\ =I_{m< x}+I_{m\geq x}\frac{k-1}{k}\frac{C_{k-1}^{m-1}}{1}\frac{k}{k-1}\frac{1}{C_{k-1}^x}\\ =I_{m< x}+I_{m\geq x}\frac{C_{k-1}^{m-1}}{C_{k-1}^x}\\ P(N\leq x|M=m, K=k)=1-P(N>x|M=m, K=k)\\ =I_{m\geq x}(1-\frac{C_{k-1}^{m-1}}{C_{k-1}^x})\\ \mu_{Bayes}=\sum_{n}nP(n|m, k)\\ =\sum_{n}\frac{k-1}{k}\frac{C_{k-1}^{m-1}}{C_k^n}I_{m\leq n}\\ =\sum_{n}\frac{m-1}{n}\frac{C_{k-2}^{m-2}}{C_{k-1}^{n-1}}I_{m\leq n}\\ =(m-1)C_{k-2}^{m-2}\sum_{n\geq n}\frac{1}{C_{k-1}^{n-1}}\\ =(m-1)C_{k-2}^{m-2}\frac{k-1}{k-2}\frac{1}{C_{k-2}^{m-2}}\\ =\frac{(m-1)(k-1)}{k-2}\\

Hence, μ B a y e s = ( X n : n − 1 ) ( n − 1 ) n − 2 \mu_{Bayes}=\frac{(X_{n:n}-1)(n-1)}{n-2} using the standard of writing in other chapter.
E ( μ B a y e s ) = n n + 2 θ E(\mu_{Bayes})=\frac{n}{n+2}\theta The Bayes estimator is a biased one.
To measure its uncertainty, we calculate its variance:
μ 2 + σ 2 − μ = ∑ n n ( n − 1 ) P ( n ∣ m , k ) = ∑ n n ( n − 1 ) m − 1 n m − 2 n − 1 k − 1 k − 2 C k − 3 m − 3 C k − 2 n − 2 I m ≤ n = ( m − 1 ) ( m − 2 ) k − 1 k − 2 C k − 3 m − 3 ∑ n ≥ m 1 C k − 2 n − 2 = ( m − 1 ) ( m − 2 ) k − 1 k − 2 C k − 3 m − 3 k − 2 k − 3 1 C k − 3 m − 3 = ( m − 1 ) ( m − 2 ) ( k − 1 ) k − 3 σ B a y e s 2 = ( m − 1 ) ( m − 2 ) ( k − 1 ) k − 3 − ( ( m − 1 ) ( k − 1 ) k − 2 ) 2 + ( m − 1 ) ( k − 1 ) k − 2 = ( m − 1 ) ( k − 1 ) ( m + 1 − k ) ( k − 3 ) ( k − 2 ) 2 V a r ( θ ^ B a y e s ) = ( x n : n − 1 ) ( n − 1 ) ( x n : n + 1 − n ) ( n − 3 ) ( n − 2 ) 2 \mu^2+\sigma^2-\mu=\sum_{n}n(n-1)P(n|m, k)\\ =\sum_{n}n(n-1)\frac{m-1}{n}\frac{m-2}{n-1}\frac{k-1}{k-2}\frac{C_{k-3}^{m-3}}{C_{k-2}^{n-2}}I_{m\leq n}\\ =(m-1)(m-2)\frac{k-1}{k-2}C_{k-3}^{m-3}\sum_{n\geq m}\frac{1}{C_{k-2}^{n-2}}\\ =(m-1)(m-2)\frac{k-1}{k-2}C_{k-3}^{m-3}\frac{k-2}{k-3}\frac{1}{C_{k-3}^{m-3}}\\ =\frac{(m-1)(m-2)(k-1)}{k-3}\\ \sigma^2_{Bayes}=\frac{(m-1)(m-2)(k-1)}{k-3}-(\frac{(m-1)(k-1)}{k-2})^2+\frac{(m-1)(k-1)}{k-2}\\ =\frac{(m-1)(k-1)(m+1-k)}{(k-3)(k-2)^2}\\ Var(\widehat{\theta}_{Bayes}) =\frac{(x_{n:n}-1)(n-1)(x_{n:n}+1-n)}{(n-3)(n-2)^2}

### Point Estimation Conclusion

According to the distribution of the question, we find six possible estimators, in which four of them are intuitive from the distribution and the background of the question, one is maximum likelihood estimator (MLE) and one the Bayes estimator and the improved estimator from MLE. We proved that the improved estimator from MLE is exactly uniformly minimum-variance unbiased estimator (UMVUE) which is unbiased estimator with the smallest variance.

Also, what is most important in our findings is the X n : n X_{n:n} plays an important role in estimating the upper limit of th discrete uniform distribution since the maximum sample give the closest information of the upper limit intuitively and we also proved that it’ s the sufficient statistics to estimate N N .

To compare the unbiasedness, effectiveness of the estimators we find, we summarize the results and give the following table:

No. F u n c t i o n Function E ( θ ^ ) E(\widehat{\theta}) V a r ( θ ^ ) Var(\widehat{\theta})
θ ^ 1 \widehat{\theta}_1 2 n ∑ i = 1 n X i − 1 \frac{2}{n}\sum^n_{i=1}X_i-1 θ \theta θ 2 3 n \frac{\theta^2}{3n}
θ ^ 2 \widehat{\theta}_2 X n : n + 1 n − 1 ∑ i > j ( X i − X j − 1 ) X_{n:n}+\frac{1}{n-1}\sum_{i>j}(X_i-X_j-1) n θ n + 1 \frac{n\theta}{n+1} n θ 2 ( n + 1 ) ( n − 1 ) ( n + 2 ) \frac{n\theta^2}{(n+1)(n-1)(n+2)}
θ ^ 3 \widehat{\theta}_3 x 1 : n + x n : n x_{1:n}+x_{n:n} θ \theta 2 θ 2 n ( n + 2 ) + 2 n 2 θ 2 ( n + 1 ) 2 ( n + 2 ) \frac{2\theta^2}{n(n+2)}+\frac{2n^2\theta^2}{(n+1)^2(n+2)}
θ ^ 4 \widehat{\theta}_4 ∑ i = 1 n X i n + 3 ∑ E ( X i − X ‾ ) 2 n − 1 \frac{\sum_{i=1}^n X_i}{n}+3\sqrt{ \frac{\sum E(X_i-\overline{X})^2}{n-1}} 1 2 θ + 2 3 3 θ \frac{1}{2}\theta+\frac{2\sqrt{3}}{3}\theta V a r ( θ ^ 4 ) > θ 2 12 n Var(\widehat{\theta}_4)>\frac{\theta^2}{12n}
θ ^ 5 \hat{\theta}_5 n + 1 n x n : n − 1 \frac{n+1}{n}x_{n:n} - 1 θ \theta θ 2 n ( n + 2 ) \frac{\theta^2}{n(n+2)}
θ ^ M L E \widehat{\theta}_{MLE} X n : n X_{n:n} n n + 1 θ \frac{n}{n+1}\theta n ( n + 1 ) 2 ( n + 2 ) θ 2 \frac{n}{(n+1)^2(n+2)}\theta^2
θ ^ B a y e s \hat{\theta}_{Bayes} ( X n : n − 1 ) ( n − 1 ) n − 2 \frac{(X_{n:n}-1)(n-1)}{n-2} n n + 2 θ \frac{n}{n+2}\theta ( x n : n − 1 ) ( n − 1 ) ( x n : n + 1 − n ) ( n − 3 ) ( n − 2 ) 2 \frac{(x_{n:n}-1)(n-1)(x_{n:n}+1-n)}{(n-3)(n-2)^2}

## Interval Estimation

In addition to point estimation, interval estimation can be carried out. Based on the observation that the probability that k observations in the sample will fall in an interval covering p of the range (0 ≤ p ≤ 1) is p k p^k (assuming in this section that draws are with replacement, to simplify computations; if draws are without replacement, this overstates the likelihood, and intervals will be overly conservative).

Thus the sampling distribution of the quantile of the sample maximum is the graph x 1 / k x^{1/k} from 0 to 1: the p-th to q-th quantile of the sample maximum m are the interval [ p 1 / k N p^{1/k}N , q 1 / k N q^{1/k}N ]. Inverting this yields the corresponding confidence interval for the population maximum of [ m / q 1 / k m/q^{1/k} , m / p 1 / k m/p^{1/k} ].

## Reference

• 点赞
• 评论
• 分享
x

海报分享

扫一扫，分享海报

• 收藏
• 打赏

打赏

天天学习的零柒贰幺

你的鼓励将是我创作的最大动力

C币 余额
2C币 4C币 6C币 10C币 20C币 50C币
• 举报
• 一键三连

点赞Mark关注该博主, 随时了解TA的最新博文

04-11 1万+

01-22 9481
08-25 8万+
05-23 920
07-07 393
03-23 3008
05-10 3万+
07-21 4万+
10-12 2万+
10-20 988
05-18 3万+
11-30 1万+
11-13 825
10-27 893