Random Signals

Reference:
Kay S M. Fundamentals of statistical signal processing[M]. Prentice Hall PTR, 1993. (Vol.2 Ch. 5 - 5.6)
Slides of ET4386, TUD

In the previous chapter we were able to detect deterministic signals in the presents of noise. However, in some cases it is unrealistic to assume that the signal is known (e.g., speech). A better approach is to assume that the signal is a random process with a known covariance structure.

Example: Energy Detector

We start with a simplest random signal detection problem:
$$\begin{array}{rl}& {\mathcal{H}}_0:x[n]=w[n]n=0,1,\cdots ,N-1\\ & {\mathcal{H}}_1:x[n]=s[n]+w[n]n=0,1,\cdots ,N-1\end{array}$$
where

• $s[n]$ is a zero mean, WSS Gaussian random process with variance ${\sigma }_s^2$, i.e., $\mathbf{s}\sim \mathcal{N}(0,{\sigma }_s^2\mathbf{I})$
• $w[n]$ is WGN with variance ${\sigma }^2$ and is independent of the signal, i.e., $\mathbf{w}\sim \mathcal{N}(0,{\sigma }^2\mathbf{I})$

Therefore, we can formulate the likelihood ratio as
$$L(\mathbf{x})=\frac{\frac{1}{{\left[2\pi \left({\sigma }_s^2+{\sigma }^2\right)\right]}^{N/2}}\exp \left[-\frac{1}{2\left({\sigma }_s^2+{\sigma }^2\right)}{\sum }_{n=0}^{N-1}{x}^2[n]\right]}{\frac{1}{{\left(2\pi {\sigma }^2\right)}^{N/2}}\exp \left[-\frac{1}{2{\sigma }^2}{\sum }_{n=0}^{N-1}{x}^2[n]\right]}$$
so that the log-likelihood ratio (LLR) becomes
$$\begin{array}{rl}l(\mathbf{x})& =\frac{N}{2}\ln \left(\frac{{\sigma }^2}{{\sigma }_s^2+{\sigma }^2}\right)-\frac{1}{2}\left(\frac{1}{{\sigma }_s^2+{\sigma }^2}-\frac{1}{{\sigma }^2}\right)\sum _{n=0}^{N-1}{x}^2[n]\\ & =\frac{N}{2}\ln \left(\frac{{\sigma }^2}{{\sigma }_s^2+{\sigma }^2}\right)+\frac{1}{2}\frac{{\sigma }_s^2}{{\sigma }^2\left({\sigma }_s^2+{\sigma }^2\right)}\sum _{n=0}^{N-1}{x}^2[n]\end{array}$$
Hence, we decide ${\mathcal{H}}_1$ if
$$T(\mathbf{x})=\sum _{n=0}^{N-1}{x}^2[n]>{\gamma }^{\prime }\text{\hspace{1em}(RS.1)}$$
The NP detector computes the energy in the received data and compares it to a threshold. Hence, it is known as an energy detector.

$T(\mathbf{x})$ under ${\mathcal{H}}_0$ and ${\mathcal{H}}_1$ is distributed as follows
$$\begin{array}{rl}{\mathcal{H}}_0:& \frac{T(\mathbf{x})}{{\sigma }^2}\sim {\chi }_N^2\\ {\mathcal{H}}_1:& \frac{T(\mathbf{x})}{{\sigma }_s^2+{\sigma }^2}\sim {\chi }_N^2\end{array}$$
Therefore, we have
P F A = Pr ⁡ { T ( x ) > γ ′ ; H 0 } = Pr ⁡ { T ( x ) σ 2 > γ ′ σ 2 ; H 0 } = Q χ N 2 ( γ ′ σ 2 ) (RS.2) \begin{aligned} P_{F A} &=\operatorname{Pr}\left\{T(\mathbf{x})>\gamma^{\prime} ; \mathcal{H}_{0}\right\} \\ &=\operatorname{Pr}\left\{\frac{T(\mathbf{x})}{\sigma^{2}}>\frac{\gamma^{\prime}}{\sigma^{2}} ; \mathcal{H}_{0}\right\} \\ &=Q_{\chi_{N}^{2}}\left(\frac{\gamma^{\prime}}{\sigma^{2}}\right) \end{aligned}\tag{RS.2} PFA​​=Pr{T(x)>γ′;H0​}=Pr{σ2T(x)​>σ2γ′​;H0​}=QχN2​​(σ2γ′​)​(RS.2)
and
P D = Pr ⁡ { T ( x ) > γ ′ ; H 1 } = Pr ⁡ { T ( x ) σ s 2 + σ 2 > γ ′ σ s 2 + σ 2 ; H 0 } = Q χ N 2 ( γ ′ σ s 2 + σ 2 ) = Q χ N 2 ( γ ′ / σ 2 σ s 2 / σ 2 + 1 ) (RS.3) \begin{aligned} P_{D} &=\operatorname{Pr}\left\{T(\mathbf{x})>\gamma^{\prime} ; \mathcal{H}_{1}\right\} \\ &=\operatorname{Pr}\left\{\frac{T(\mathbf{x})}{\sigma_{s}^{2}+\sigma^{2}}>\frac{\gamma^{\prime}}{\sigma_{s}^{2}+\sigma^{2}} ; \mathcal{H}_{0}\right\} \\ &=Q_{\chi_{N}^{2}}\left(\frac{\gamma^{\prime}}{\sigma_{s}^{2}+\sigma^{2}}\right)=Q_{\chi_{N}^{2}}\left(\frac{\gamma^{\prime}/\sigma^{2}}{\sigma_{s}^{2}/\sigma^{2}+1}\right) \end{aligned}\tag{RS.3} PD​​=Pr{T(x)>γ′;H1​}=Pr{σs2​+σ2T(x)​>σs2​+σ2γ′​;H0​}=QχN2​​(σs2​+σ2γ′​)=QχN2​​(σs2​/σ2+1γ′/σ2​)​(RS.3)
As ${\sigma }_s^2/{\sigma }^2$ increases, the argument of the $Q_{{\chi }_N^2}$ function decreases and thus $P_D$ increases.

Generalization 1: Signals With Arbitrary Covariance Matrices

Assume:

• $s[n]$ is a zero mean, Gaussian random process with covariance matrix ${\mathbf{C}}_s$, i.e., $\mathbf{s}\sim \mathcal{N}(0,{\mathbf{C}}_s)$
• $w[n]$ is WGN with variance ${\sigma }^2$ and is independent of the signal, i.e., $\mathbf{w}\sim \mathcal{N}(0,{\sigma }^2\mathbf{I})$

Hence:
$$\mathbf{x}\sim \{\begin{array}{cccc}& \mathcal{N}(0,{\sigma }^2\mathbf{I})& & \text{under }{\mathcal{H}}_0\\ & \mathcal{N}(0,{\mathbf{C}}_s+{\sigma }^2\mathbf{I})& & \text{under }{\mathcal{H}}_1\end{array}$$
The NP detector decides ${\mathcal{H}}_1$ if
$$L(\mathbf{x})=\frac{\frac{1}{(2\pi {)}^{N/2}{\det }^{\frac{1}{2}}\left({\mathbf{C}}_s+{\sigma }^2\mathbf{I}\right)}\exp \left[-\frac{1}{2}{\mathbf{x}}^T{\left({\mathbf{C}}_s+{\sigma }^2\mathbf{I}\right)}^{-1}\mathbf{x}\right]}{\frac{1}{{\left(2\pi {\sigma }^2\right)}^{N/2}}\exp \left(-\frac{1}{2{\sigma }^2}{\mathbf{x}}^T\mathbf{x}\right)}>\gamma$$
Taking logarithms and retaining only the data-dependent terms yields
$$T(\mathbf{x})={\sigma }^2{\mathbf{x}}^T\left[\frac{1}{{\sigma }^2}\mathbf{I}-{\left({\mathbf{C}}_s+{\sigma }^2\mathbf{I}\right)}^{-1}\right]\mathbf{x}>2{\gamma }^{\prime }{\sigma }^2$$
Using the matrix inversion lemma
$$(\mathbf{A}+\mathbf{B}\mathbf{C}\mathbf{D}{)}^{-1}={\mathbf{A}}^{-1}-{\mathbf{A}}^{-1}\mathbf{B}{\left(\mathbf{D}{\mathbf{A}}^{-1}\mathbf{B}+{\mathbf{C}}^{-1}\right)}^{-1}\mathbf{D}{\mathbf{A}}^{-1}$$
we have upon letting $\mathbf{A}={\sigma }^2\mathbf{I},\mathbf{B}=\mathbf{D}=\mathbf{I},\mathbf{C}={\mathbf{C}}_s$
$${\left({\sigma }^2\mathbf{I}+{\mathbf{C}}_s\right)}^{-1}=\frac{1}{{\sigma }^2}\mathbf{I}-\frac{1}{{\sigma }^4}{\left(\frac{1}{{\sigma }^2}\mathbf{I}+{\mathbf{C}}_s^{-1}\right)}^{-1}$$
so that
$$T(\mathbf{x})={\mathbf{x}}^T\left[\frac{1}{{\sigma }^2}{\left(\frac{1}{{\sigma }^2}\mathbf{I}+{\mathbf{C}}_s^{-1}\right)}^{-1}\right]\mathbf{x}\text{\hspace{1em}(RS.4)}$$
Now let $\hat{\mathbf{s}}=\left(1/{\sigma }^2\right){\left(\left(1/{\sigma }^2\right)\mathbf{I}+{\mathbf{C}}_s^{-1}\right)}^{-1}\mathbf{x}.$ This may be also written as
$$\hat{\mathbf{s}}={\mathbf{C}}_s({\mathbf{C}}_s+{\sigma }^2\mathbf{I}{)}^{-1}\mathbf{x}\text{\hspace{1em}(RS.5)}$$
Hence, we decide ${\mathcal{H}}_1$ if
$$T(\mathbf{x})={\mathbf{x}}^T\hat{\mathbf{s}}=\sum _{n=0}^{N-1}x[n]\hat{s}[n]\text{\hspace{1em}(RS.6)}$$

Estimator-Correlator

The NP detector correlates the received data with an estimate of the signal, i.e., $\hat{s}[n]$. It is therefore termed an estimator-correlator.

We claim that $\hat{\mathbf{s}}$ is actually a Wiener filter estimator of the signal. To see this, recall that if $\mathbf{\theta }$ is an unknown random vector whose realization is to be estimated based on the data vector $\mathbf{x}$, and $\mathbf{\theta }$ and $\mathbf{x}$ are jointly Gaussian with zero mean, the minimum mean square error (MMSE) estimator is
$$\hat{\mathbf{\theta }}={\mathbf{C}}_{\theta x}{\mathbf{C}}_{xx}^{-1}\mathbf{x}={\mathbf{w}}^T\mathbf{x}$$
where $\mathbf{w}={\mathbf{C}}_{xx}^{-1}{\mathbf{C}}_{\theta x}$, which is known as the Wiener filter. The MMSE estimator is linear due to the jointly Gaussian assumption.

Here we have $\mathbf{\theta }=\mathbf{s}$ and $\mathbf{x}-\mathbf{s}+\mathbf{w}$, with $\mathbf{s}$ and $\mathbf{w}$ uncorrelated. The MMSE estimate of the signal realization is from
$$\hat{\mathbf{s}}=E[\mathbf{s}(\mathbf{s}+\mathbf{w}{)}^T](E[(\mathbf{s}+\mathbf{w})(\mathbf{s}+\mathbf{w}{)}^T]{)}^{-1}\mathbf{x}={\mathbf{C}}_s({\mathbf{C}}_s+{\sigma }^2\mathbf{I}{)}^{-1}\mathbf{x}$$
which is identical to $(RS.5)$.

Canonical Form

For any ${\mathbf{C}}_s\in {\mathbf{S}}_{+}$, let the eigendecomposition of the $N\times N$ covariance matrix ${\mathbf{C}}_s$ be
$${\mathbf{C}}_s=\mathbf{V}{\mathbf{\Lambda }}_s{\mathbf{V}}^T$$
where $\mathbf{V}=[{\mathbf{v}}_0{\mathbf{v}}_1\cdots {\mathbf{v}}_{N-1}]$ for ${\mathbf{v}}_i$ the $i$th eigenvector of ${\mathbf{C}}_s$ and ${\mathbf{\Lambda }}_s=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}({\lambda }_{s_0},{\lambda }_{s_1},\cdots ,{\lambda }_{s_{N-1}})$ for ${\lambda }_{s_i}$ the corresponding eigenvalue.

Instead of expressing the test statistic in terms of $\mathbf{x}$, it is advantageous to let $\mathbf{y}={\mathbf{V}}^T\mathbf{x}$, so that
$$\begin{array}{rl}T(\mathbf{x})& ={\mathbf{x}}^T\mathbf{V}{\mathbf{\Lambda }}_s{\mathbf{V}}^T(\mathbf{V}{\mathbf{\Lambda }}_s{\mathbf{V}}^T+{\sigma }^2\mathbf{V}{\mathbf{V}}^T{)}^{-1}\mathbf{x}\\ & ={\mathbf{x}}^T\mathbf{V}{\mathbf{\Lambda }}_s({\mathbf{\Lambda }}_s+{\sigma }^2\mathbf{I}{)}^{-1}{\mathbf{V}}^T\mathbf{x}\\ & ={\mathbf{y}}^T{\mathbf{\Lambda }}_s({\mathbf{\Lambda }}_s+{\sigma }^2\mathbf{I}{)}^{-1}\mathbf{y}\\ & =\sum _{n=0}^{N-1}\frac{{\lambda }_{s_n}}{{\lambda }_{s_n}+{\sigma }^2}{y}^2[n]\end{array}\text{\hspace{1em}(RS.7)}$$
This is the canonical form of the detector which is shown in Figure 5.3.

The effect of the filter on an input vector $\mathbf{x}$ is equivalent to first transforming $\mathbf{x}$ to $\mathbf{y}={\mathbf{V}}^T\mathbf{x}$, filtering each component of $\mathbf{y}$ by ${\lambda }_{s_n}/({\lambda }_{s_n}+{\sigma }^2)$ to form ${\mathbf{y}}^{\prime }$, and then transforming back to the original space using ${\mathbf{x}}^{\prime }=\mathbf{V}{\mathbf{y}}^{\prime }$.

Remark:

Now assume that $N=2$ and
$${\mathbf{C}}_s={\sigma }_s^2\left[\begin{array}{cc}1& \rho \\ \rho & 1\end{array}\right]=\mathbf{V}{\mathbf{\Lambda }}_s{\mathbf{V}}^T$$
where $\mathbf{V}=\left[\begin{array}{rlr}& \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ & \frac{1}{\sqrt{2}}& -\frac{1}{\sqrt{2}}\end{array}\right]=[{\mathbf{v}}_1{\mathbf{v}}_2],{\mathbf{\Lambda }}_s=\left[\begin{array}{rlr}& {\sigma }_s^2(1+\rho )& 0\\ & 0& {\sigma }_s^2(1-\rho )\end{array}\right]$, $\rho$ is the correlation coefficient between $s[0]$ and $s[1]$. The detector can easily be derived as
$$T(\mathbf{x})=\frac{{\sigma }_s^2(1+\rho )}{{\sigma }_s^2(1+\rho )+{\sigma }^2}{y}^2[0]+\frac{{\sigma }_s^2(1-\rho )}{{\sigma }_s^2(1-\rho )+{\sigma }^2}{y}^2[1]$$
Why the contribution of $y[0]$ to $T(\mathbf{x})$ is weighted more heavily?

To see this, the signal PDF and noise PDF are
$$\begin{gathered} p(\mathbf{s})=\frac{1}{(2\pi {)}^{N/2}{\det }^{\frac{1}{2}}{\mathbf{C}}_s}\exp \left[-\frac{1}{2}{\mathbf{s}}^T{\mathbf{C}}_s^{-1}\mathbf{s}\right] \\ p(\mathbf{w})=\frac{1}{{\left(2\pi {\sigma }^2\right)}^{N/2}}\exp \left(-\frac{1}{2{\sigma }^2}{\mathbf{w}}^T\mathbf{w}\right) \end{gathered}$$
Then we can draw the contours of constant PDF for signal only and noise only:

Note that line ${\xi }_1={\xi }_0$ is along the direction of ${\mathbf{v}}_1$ and the line ${\xi }_1=-{\xi }_0$ is along the direction of ${\mathbf{v}}_2$.

Let $\mathbf{z}={\mathbf{V}}^T\mathbf{s}$, the expression of constant PDF for signal is
$${\mathbf{z}}^T{\mathbf{\Lambda }}_s^{-1}\mathbf{z}=\frac{z^2[0]}{{\sigma }_s^2(1+\rho )}+\frac{z^2[1]}{{\sigma }_s^2(1-\rho )}$$
Therefore, the constant PDF is a rotated ellipsoid.

The component of $\mathbf{x}$ along line ${\hat{\mathbf{v}}}_1$ is likely to be much larger when the signal is present than the component along the orthogonal line ${\hat{\mathbf{v}}}_2$. For no signal there is no preferred direction. Therefore, the contribution of $y[0]$ to $T(\mathbf{x})$ is weighted more heavily. If $\rho \approx 1$, the PDF of the signal is concentrated along ${\hat{\mathbf{v}}}_1$, so $y[0]$ is retained and $y[1]$ is almost discarded.

Generalization 2: General Gaussian Detection

Assume:

• $s[n]$ is a Gaussian random process with mean ${\mathbf{\mu }}_s$ and covariance matrix ${\mathbf{C}}_s$, i.e., $\mathbf{s}\sim \mathcal{N}({\mathbf{\mu }}_s,{\mathbf{C}}_s)$
• $w[n]$ is a zero mean, Gaussian random process with covariance matrix ${\mathbf{C}}_w$ and is independent of the signal, i.e., $\mathbf{w}\sim \mathcal{N}(0,{\mathbf{C}}_w)$

Hence
$$\mathbf{x}\sim \{\begin{array}{cccc}& \mathcal{N}(0,{\mathbf{C}}_w)& & \text{under }{\mathcal{H}}_0\\ & \mathcal{N}({\mathbf{\mu }}_s,{\mathbf{C}}_s+{\mathbf{C}}_w)& & \text{under }{\mathcal{H}}_1\end{array}$$
The NP detector decides ${\mathcal{H}}_1$ if
$$\frac{\frac{1}{(2\pi {)}^{N/2}{\det }^{\frac{1}{2}}\left({\mathbf{C}}_s+{\mathbf{C}}_w\right)}\exp \left[-\frac{1}{2}{\left(\mathbf{x}-{\mathbf{\mu }}_s\right)}^T{\left({\mathbf{C}}_s+{\mathbf{C}}_w\right)}^{-1}\left(\mathbf{x}-{\mathbf{\mu }}_s\right)\right]}{\frac{1}{(2\pi {)}^{N/2}{\det }^{\frac{1}{2}}\left({\mathbf{C}}_w\right)}\exp \left[-\frac{1}{2}{\mathbf{x}}^T{\mathbf{C}}_w^{-1}\mathbf{x}\right]}>\gamma$$
Taking the logarithm, retaining only the data-dependent terms, and scaling produces the test statistic
$$\begin{array}{rl}T(\mathbf{x})=& {\mathbf{x}}^T{\mathbf{C}}_w^{-1}\mathbf{x}-{\left(\mathbf{x}-{\mathbf{\mu }}_s\right)}^T{\left({\mathbf{C}}_s+{\mathbf{C}}_w\right)}^{-1}\left(\mathbf{x}-{\mathbf{\mu }}_s\right)\\ =& {\mathbf{x}}^T{\mathbf{C}}_w^{-1}\mathbf{x}-{\mathbf{x}}^T{\left({\mathbf{C}}_s+{\mathbf{C}}_w\right)}^{-1}\mathbf{x}\\ & +2{\mathbf{x}}^T{\left({\mathbf{C}}_s+{\mathbf{C}}_w\right)}^{-1}{\mathbf{\mu }}_s-{\mathbf{\mu }}_s^T{\left({\mathbf{C}}_s+{\mathbf{C}}_w\right)}^{-1}{\mathbf{\mu }}_s\end{array}$$
But from the matrix inversion lemma
$${\mathbf{C}}_w^{-1}-{\left({\mathbf{C}}_s+{\mathbf{C}}_w\right)}^{-1}={\mathbf{C}}_w^{-1}{\mathbf{C}}_s{\left({\mathbf{C}}_s+{\mathbf{C}}_w\right)}^{-1}$$
so that by ignoring the nondata dependent term and scaling we have
$$T^{\prime }(\mathbf{x})={\mathbf{x}}^T{\left({\mathbf{C}}_s+{\mathbf{C}}_w\right)}^{-1}{\mathbf{\mu }}_s+\frac{1}{2}{\mathbf{x}}^T{\mathbf{C}}_w^{-1}{\mathbf{C}}_s{\left({\mathbf{C}}_s+{\mathbf{C}}_w\right)}^{-1}\mathbf{x}\text{\hspace{1em}(RS.8)}$$
The test statistic consists of a quadratic form as well as a linear form in $\mathrm{x}$. As special cases we have

1. ${\mathbf{C}}_s=0$ or a deterministic signal with $\mathbf{s}={\mathbf{\mu }}_s$. Then
   $$T^{\prime }(\mathbf{x})={\mathbf{x}}^T{\mathbf{C}}_w^{-1}{\mathbf{\mu }}_s\text{\hspace{1em}(RS.9)}$$
   which is our prewhitener and matched filter.

2. ${\mathbf{\mu }}_s=0$ or a random signal with $\mathbf{s}\sim \mathcal{N}\left(0,{\mathbf{C}}_s\right).$ Then
   $$\begin{array}{rl}{T}^{\prime }(\mathbf{x})& =\frac{1}{2}{\mathbf{x}}^T{\mathbf{C}}_w^{-1}{\mathbf{C}}_s{\left({\mathbf{C}}_s+{\mathbf{C}}_w\right)}^{-1}\mathbf{x}\\ & =\frac{1}{2}{\mathbf{x}}^T{\mathbf{C}}_w^{-1}\hat{\mathbf{s}}\end{array}\text{\hspace{1em}(RS.10)}$$
   where $\hat{\mathbf{s}}={\mathbf{C}}_s{\left({\mathbf{C}}_s+{\mathbf{C}}_w\right)}^{-1}\mathbf{x}$ is the MMSE estimator of $\mathbf{s}$. This is a prewhitener followed by an estimator-correlator.