Composite Hypothesis Testing

Motivation:

• Neyman-Pearson detectors require perfect knowledge of the PDFs
• What if this information is unknown?
• Are there detectors for such scenarios? Radar, Sonar

Approach:

• Design the NP detector, assuming the parameters are known
• Manipulate the Test so that it is not dependent on the parameters

Example: DC Level in WGN with Unknown Amplitude (A>0)

Consider the DC level in WGN detection problem
$$\begin{array}{ll}{\mathcal{H}}_0:x[n]=w[n]& n=0,1,\dots ,N-1\\ {\mathcal{H}}_1:x[n]=A+w[n]& n=0,1,\dots ,N-1\end{array}$$
where the value of $A$ is unknown, although a priori we know that $A>0,$ and $w[n]$ is WGN with variance ${\sigma }^2$. Then, the NP test is to decide ${\mathcal{H}}_1$ if
$$\frac{p\left(\mathbf{x};A,{\mathcal{H}}_1\right)}{p\left(\mathbf{x};{\mathcal{H}}_0\right)}=\frac{\frac{1}{{\left(2\pi {\sigma }^2\right)}^{\frac{N}{2}}}\exp \left[-\frac{1}{2{\sigma }^2}{\sum }_{n=0}^{N-1}(x[n]-A{)}^2\right]}{\frac{1}{{\left(2\pi {\sigma }^2\right)}^{\frac{N}{2}}}\exp \left[-\frac{1}{2{\sigma }^2}{\sum }_{n=0}^{N-1}{x}^2[n]\right]}>\gamma$$
Taking the logarithm we have
$$A\sum _{n=0}^{N-1}x[n]>{\sigma }^2\ln \gamma +\frac{N{A}^2}{2}$$
since it is known that $A>0,$ we have
$$\sum _{n=0}^{N-1}x[n]>\frac{{\sigma }^2}{A}\ln \gamma +\frac{NA}{2}$$
Finally, scaling by $1/N$ produces the test
$$T(\mathbf{x})=\frac{1}{N}\sum _{n=0}^{N-1}x[n]>\frac{{\sigma }^2}{NA}\ln \gamma +\frac{A}{2}={\gamma }^{\prime }$$
Clearly, the test statistic, which is the sample mean of the data, does not depend on $A$.

Recall from Chapter 3 that $T(\mathbf{x};{\mathcal{H}}_0)=\bar{x}\sim \mathcal{N}(0,{\sigma }^2/N),T(\mathbf{x};{\mathcal{H}}_1)=\bar{x}\sim \mathcal{N}(A,{\sigma }^2/N)$. Hence,
P F A = Pr ⁡ { T ( x ) > γ ′ ; H 0 } = Q ( γ ′ σ 2 / N ) P D = Pr ⁡ { T ( x ) > γ ′ ; H 1 } = Q ( γ ′ − A σ 2 / N ) = Q ( Q − 1 ( P F A ) − N A 2 σ 2 ) P_{FA}=\Pr\{T(\mathbf x)>\gamma^\prime;\mathcal H_0 \}=Q\left(\frac{\gamma^\prime}{\sqrt{\sigma^2/N}} \right)\\ P_{D}=\Pr\{T(\mathbf x)>\gamma^\prime;\mathcal H_1 \}=Q\left(\frac{\gamma^\prime-A}{\sqrt{\sigma^2/N}} \right)=Q\left(Q^{-1}(P_{FA})-\sqrt{\frac{NA^2}{\sigma^2}}\right) PFA​=Pr{T(x)>γ′;H0​}=Q(σ2/N ​γ′​)PD​=Pr{T(x)>γ′;H1​}=Q(σ2/N ​γ′−A​)=Q(Q−1(PFA​)−σ2NA2​ ​)
Therefore, $P_{FA}$ (and the threshold) does not depend on $A$, although $P_D$ depends on $A$.

The test $(1)$ leads to the highest $P_D$ (remember NP maximizes $P_D$ ) for any value $A$. (as long as $A>0$ ). Such a test is called a Uniformly Most Powerful (UMP) test. Any other test will have a poorer performance.

Unfortunately, UMP tests seldom exist.

Example: DC Level in WGN with Unknown Amplitude

Reconsider the example above with $-\infty <A<\infty$. If we assume perfect knowledge of $A$ to design a NP detector, then it termed as a clairvoyant detector.

When $A$ can take on positive and negative values, the clairvoyant detector decides ${\mathcal{H}}_1$ if
$$\begin{array}{r}\frac{1}{N}\sum _{n=0}^{N-1}x[n]=\bar{x}>{\gamma }_{+}^{\prime }\text{ for }A>0\\ \frac{1}{N}\sum _{n=0}^{N-1}x[n]=\bar{x}<{\gamma }_{-}^{\prime }\text{ for }A<0\end{array}$$
The detector is clearly unrealizable since it is composed of two different NP tests, the choice of which depends upon the unknown parameter $A$. It provides an upper bound on performance, which can be found as follows.
$$\begin{array}{rlrl}& P_{FA}=\mathrm{Pr}\left\{\bar{x}>{\gamma }_{+}^{\prime };{\mathcal{H}}_0\right\}=Q\left(\frac{{\gamma }_{+}^{\prime }}{\sqrt{{\sigma }^2/N}}\right)& & \text{ for }A>0\\ & P_{FA}=\mathrm{Pr}\left\{\bar{x}<{\gamma }_{-}^{\prime };{\mathcal{H}}_0\right\}=1-Q\left(\frac{{\gamma }_{-}^{\prime }}{\sqrt{{\sigma }^2/N}}\right)=Q\left(\frac{-{\gamma }_{-}^{\prime }}{\sqrt{{\sigma }^2/N}}\right)& & \text{ for }A<0\end{array}$$

$$\begin{array}{rlrl}& P_D=\mathrm{Pr}\left\{\bar{x}>{\gamma }_{+}^{\prime };{\mathcal{H}}_1\right\}=Q\left(\frac{{\gamma }_{+}^{\prime }-A}{\sqrt{{\sigma }^2/N}}\right)=Q\left(Q^{-1}\left(P_{FA}\right)-\sqrt{\frac{N{A}^2}{{\sigma }^2}}\right)& & \text{ for }A>0\\ & P_D=1-Q\left(\frac{{\gamma }_{-}^{\prime }-A}{\sqrt{{\sigma }^2/N}}\right)=Q\left(\frac{-{\gamma }_{-}^{\prime }+A}{\sqrt{{\sigma }^2/N}}\right)=Q\left(Q^{-1}\left(P_{FA}\right)+\frac{A}{\sqrt{{\sigma }^2/N}}\right)& & \text{ for }A<0\end{array}$$

Instead of the clairvoyant detector, let’s look at the realizable detector:
$$T(\mathbf{x})=\left|\frac{1}{N}\sum _{n=0}^{N-1}x[n]\right|>{\gamma }^{\prime \prime }$$
Then the detection performance
P F A = Pr ⁡ { ∣ x ˉ ∣ > γ ′ ′ ; H 0 } = 2 Pr ⁡ { x ˉ > γ ′ ′ ; H 0 } = 2 Q ( γ ′ ′ σ 2 / N ) γ ′ ′ = σ 2 / N Q − 1 ( P F A / 2 ) P D = Pr ⁡ { ∣ x ˉ ∣ > γ ′ ′ ; H 1 } = Q ( Q − 1 ( P F A / 2 ) − A σ 2 / N ) + Q ( Q − 1 ( P F A / 2 ) + A σ 2 / N ) \begin{aligned} P_{F A}&=\operatorname{Pr}\left\{|\bar{x}|>\gamma^{\prime \prime} ; \mathcal{H}_{0}\right\}=2 \operatorname{Pr}\left\{\bar{x}>\gamma^{\prime \prime} ; \mathcal{H}_{0}\right\}=2 Q\left(\frac{\gamma^{\prime \prime}}{\sqrt{\sigma^{2} / N}}\right) \\ \gamma^{\prime \prime}&=\sqrt{\sigma^{2} / N} Q^{-1}\left(P_{F A} / 2\right) \\ P_{D}=\operatorname{Pr}\left\{|\bar{x}|>\gamma^{\prime \prime} ; \mathcal{H}_{1}\right\}&=Q\left(Q^{-1}\left(P_{F A} / 2\right)-\frac{A}{\sqrt{\sigma^{2} / N}}\right)+Q\left(Q^{-1}\left(P_{F A} / 2\right)+\frac{A}{\sqrt{\sigma^{2} / N}}\right) \end{aligned} PFA​γ′′PD​=Pr{∣xˉ∣>γ′′;H1​}​=Pr{∣xˉ∣>γ′′;H0​}=2Pr{xˉ>γ′′;H0​}=2Q(σ2/N ​γ′′​)=σ2/N ​Q−1(PFA​/2)=Q(Q−1(PFA​/2)−σ2/N ​A​)+Q(Q−1(PFA​/2)+σ2/N ​A​)​

The performance of this realizable detector is thus not optimal, but close to the optimal clairvoyant detector.

In fact, the proposed detector is an example of a more general approach to composite hypothesis testing, the generalized likelihood ratio test, which is described in the next section.

Composite Hypothesis Testing Approaches

Bayesian Approach

The Bayesian approach assigns prior PDFs to ${\mathbf{\theta }}_0$ and ${\mathbf{\theta }}_1$. In doing so it models the unknown parameters as realizations of a vector random variable. If the prior PDFs are denoted by $p\left({\mathbf{\theta }}_0\right)$ and $p\left({\mathbf{\theta }}_1\right),$ respectively, the PDFs of the data are
$$\begin{array}{rl}p\left(\mathbf{x};{\mathcal{H}}_0\right)& =\int p\left(\mathbf{x}|{\mathbf{\theta }}_0;{\mathcal{H}}_0\right)p\left({\mathbf{\theta }}_0\right)d{\mathbf{\theta }}_0\\ p\left(\mathbf{x};{\mathcal{H}}_1\right)& =\int p\left(\mathbf{x}|{\mathbf{\theta }}_1;{\mathcal{H}}_1\right)p\left({\mathbf{\theta }}_1\right)d{\mathbf{\theta }}_1\end{array}$$
where $p\left(\mathbf{x}|{\mathbf{\theta }}_i;{\mathcal{H}}_i\right)$ is the conditional PDF of $\mathbf{x},$ conditioned on ${\mathbf{\theta }}_i,$ assuming ${\mathcal{H}}_i$ is true. The unconditional PDFs $p\left(\mathbf{x};{\mathcal{H}}_0\right)$ and $p\left(\mathbf{x};{\mathcal{H}}_1\right)$ are now completely specified, no longer dependent on the unknown parameters. With the Bayesian approach the optimal NP detector decides ${\mathcal{H}}_1$ if
$$\frac{p\left(\mathbf{x};{\mathcal{H}}_1\right)}{p\left(\mathbf{x};{\mathcal{H}}_0\right)}=\frac{\int p\left(\mathbf{x}|{\mathbf{\theta }}_1;{\mathcal{H}}_1\right)p\left({\mathbf{\theta }}_1\right)d{\mathbf{\theta }}_1}{\int p\left(\mathbf{x}|{\mathbf{\theta }}_0;{\mathcal{H}}_0\right)p\left({\mathbf{\theta }}_0\right)d{\mathbf{\theta }}_0}>\gamma$$

• Need to choose prior pdf.
• Integration can be difficult.

Generalized Likelihood Ratio Test (GLRT)

The GLRT replaces the unknown parameters by their maximum likelihood estimates (MLEs). In general, the GLRT decides ${\mathcal{H}}_1$ if
$$L_G(\mathbf{x})=\frac{p\left(\mathbf{x};{\hat{\mathbf{\theta }}}_1,{\mathcal{H}}_1\right)}{p\left(\mathbf{x};{\hat{\mathbf{\theta }}}_0,{\mathcal{H}}_0\right)}>\gamma$$
where ${\hat{\mathbf{\theta }}}_1$ is the MLE of ${\mathbf{\theta }}_1$ assuming ${\mathcal{H}}_1$ is true (maximizes $p\left(\mathbf{x};{\theta }_1,{\mathcal{H}}_1\right)),$ and ${\hat{\mathbf{\theta }}}_0$ is the MLE of ${\mathbf{\theta }}_0$ assuming ${\mathcal{H}}_0$ is true (maximizes $ p\left(\mathbf{x} ; \boldsymbol{\theta}{0}, \mathcal{H}{0}\right)$).

The GLRT can also be expressed in another form, which is sometimes more convenient. since ${\hat{\mathbf{\theta }}}_i$ is the MLE under ${\mathcal{H}}_i,$ it maximizes $p\left(\mathbf{x};{\mathbf{\theta }}_i,{\mathcal{H}}_i\right)$ or
$$p\left(\mathbf{x};{\hat{\mathbf{\theta }}}_i,{\mathcal{H}}_i\right)=\underset{{\mathbf{\theta }}_i}{\max }p\left(\mathbf{x};{\mathbf{\theta }}_i,{\mathcal{H}}_i\right)$$
Hence, $L_G(\mathbf{x})$ can be written as
$$L_G(\mathbf{x})=\frac{{\max }_{{\mathbf{\theta }}_1}p\left(\mathbf{x};{\mathbf{\theta }}_1,{\mathcal{H}}_1\right)}{{\max }_{{\mathbf{\theta }}_0}p\left(\mathbf{x};{\mathbf{\theta }}_0,{\mathcal{H}}_0\right)}$$
The approach also provides information about the unknown parameters since the first step in determining $L_G(\mathbf{x})$ is to find the MLEs. We now continue the DC level in WGN example.

Example: DC Level in WGN with Unknown Amplitude - GLRT

In this case we have ${\mathbf{\theta }}_1=A$ and there are no unknown parameters under ${\mathcal{H}}_0$. The hypothesis test becomes
$$\begin{array}{l}{\mathcal{H}}_0:A=0\\ {\mathcal{H}}_1:A\ne 0\end{array}$$
Thus, the GLRT decides ${\mathcal{H}}_1$ if
$$L_G(\mathbf{x})=\frac{p\left(\mathbf{x};\hat{A},{\mathcal{H}}_1\right)}{p\left(\mathbf{x};{\mathcal{H}}_0\right)}>\gamma$$
The MLE of $A$ is found by maximizing
$$p\left(\mathbf{x};A,{\mathcal{H}}_1\right)=\frac{1}{{\left(2\pi {\sigma }^2\right)}^{\frac{N}{2}}}\exp \left[-\frac{1}{2{\sigma }^2}\sum _{n=0}^{N-1}(x[n]-A{)}^2\right]$$
By differentiating the likelihood (or loglikelihood) function and setting the derivative to zero, wen obtain the MLE $\hat{A}=\bar{x}$. Thus,
$$L_G(\mathbf{x})=\frac{\frac{1}{{\left(2\pi {\sigma }^2\right)}^{\frac{N}{2}}}\exp \left[-\frac{1}{2{\sigma }^2}{\sum }_{n=0}^{N-1}(x[n]-\bar{x}{)}^2\right]}{\frac{1}{{\left(2\pi {\sigma }^2\right)}^{\frac{N}{2}}}\exp \left(-\frac{1}{2{\sigma }^2}{\sum }_{n=0}^{N-1}{x}^2[n]\right)}$$
Taking logarithms we have
$$\begin{array}{rl}\ln L_G(\mathbf{x})& =-\frac{1}{2{\sigma }^2}\left(\sum _{n=0}^{N-1}{x}^2[n]-2\bar{x}\sum _{n=0}^{N-1}x[n]+N{\bar{x}}^2-\sum _{n=0}^{N-1}{x}^2[n]\right)\\ & =-\frac{1}{2{\sigma }^2}\left(-2N{\bar{x}}^2+N{\bar{x}}^2\right)\\ & =\frac{N{\bar{x}}^2}{2{\sigma }^2}\end{array}$$
or we decide ${\mathcal{H}}_1$ if
$$|\bar{x}|>{\gamma }^{\prime }$$
This detector is identical to realizable detector we looked at before and the performance has already been given.

Example: DC Level in WGN with Unknown Amplitude and Variance - GLRT

Consider the detection problem
$$\begin{array}{ll}{\mathcal{H}}_0:x[n]=w[n]& n=0,1,\dots ,N-1\\ {\mathcal{H}}_1:x[n]=A+w[n]& n=0,1,\dots ,N-1\end{array}$$
where $A$ is unknown with $-\infty <A<\infty$ and $w[n]$ is WGN with unknown variance ${\sigma }^2$. A UMP test does not exist because the equivalent parameter test is
$$\begin{array}{l}{\mathcal{H}}_0:A=0,{\sigma }^2>0\\ {\mathcal{H}}_1:A\ne 0,{\sigma }^2>0\end{array}$$
which is two-sided. The GLRT decides ${\mathcal{H}}_1$ if
$$L_G(\mathbf{x})=\frac{p\left(\mathbf{x};\hat{A},{\hat{\sigma }}_1^2,{\mathcal{H}}_1\right)}{p\left(\mathbf{x};{\hat{\sigma }}_0^2,{\mathcal{H}}_0\right)}>\gamma$$
where $[\hat{A}{\hat{\sigma }}^2{]}^T$ is the MLE of the vector parameter ${\mathbf{\theta }}_1=[A{\sigma }^2{]}^T$ under ${\mathcal{H}}_1$, and ${\hat{\sigma }}_0^2$ is the MLE of the parameter ${\mathbf{\theta }}_0={\sigma }^2$ under ${\mathcal{H}}_0$. Note that we need to estimate the variance under both hypotheses.

Since
$$\begin{gathered} p\left(\mathbf{x};A,{\sigma }^2,{\mathcal{H}}_1\right)=\frac{1}{{\left(2\pi {\sigma }^2\right)}^{\frac{N}{2}}}\exp \left[-\frac{1}{2{\sigma }^2}\sum _{n=0}^{N-1}(x[n]-A{)}^2\right] \\ p\left(\mathbf{x};A,{\sigma }^2,{\mathcal{H}}_1\right)=\frac{1}{{\left(2\pi {\sigma }^2\right)}^{\frac{N}{2}}}\exp \left[-\frac{1}{2{\sigma }^2}\sum _{n=0}^{N-1}{x}^2[n]\right] \end{gathered}$$
Similar as before, we have
$$\begin{gathered} \hat{A}=\bar{x} \\ {\hat{\sigma }}_1^2=\frac{1}{N}\sum _{n=1}^{N-1}(x[n]-\bar{x}{)}^2 \\ {\hat{\sigma }}_0^2=\frac{1}{N}\sum _{n=1}^{N-1}{x}^2[n] \end{gathered}$$
Thus the GLRT becomes
$$L_G(\mathbf{x})={\left(\frac{{\hat{\sigma }}_0^2}{{\hat{\sigma }}_1^2}\right)}^{N/2}$$
In essence, the GLRT decides ${\mathcal{H}}_1$ if the fit to the data of the signal $\hat{A}=\bar{x}$ produces a much smaller error, as measured by ${\hat{\sigma }}_1^2=(1/N){\sum }_{n=0}^{N-1}(x[n]-\hat{A}{)}^2$ than a fit of no signal or ${\hat{\sigma }}_0^2=(1/N){\sum }_{n=0}^{N-1}{x}^2[n]$. A slightly more intuitive form can be found as follows. Since
$${\hat{\sigma }}_1^2=\frac{1}{N}\sum _{n=1}^{N-1}{x}^2[n]-{\bar{x}}^2={\hat{\sigma }}_0^2-{\bar{x}}^2$$
we have
$$2\ln L_G(\mathbf{x})=N\ln \left(\frac{{\hat{\sigma }}_1^2+{\bar{x}}^2}{{\hat{\sigma }}_1^2}\right)=N\ln \left(1+\frac{{\bar{x}}^2}{{\hat{\sigma }}_1^2}\right)$$

Since $\ln (1+x)$ is monotonically increasing with increasing $x$, an equivalent test statistic is
$$T(\mathrm{x})=\frac{{\hat{A}}_1^2}{{\hat{\sigma }}_1^2}$$