Chapter 4 (Vector Spaces): Applications to difference equations (差分方程中的应用)

本文为《Linear algebra and its applications》的读书笔记

• Difference equations are often the appropriate tool to analyze discrete, or digital data. Even when a differential equation is used to model a continuous process, a numerical solution is often produced from a related difference equation.
• This section highlights some fundamental properties of linear difference equations.

Discrete-Time Signals

离散时间信号

• The vector space $\mathbb{S}$ of discrete-time signals was introduced in Section 4.1. A signal in $\mathbb{S}$ is a function defined only on the integers and is visualized as a sequence of numbers, say, { y k } \{y_k\} {yk​}.
  

Linear Independence in the Space $\mathbb{S}$ of Signals

• To simplify notation, we consider a set of only three signals in $\mathbb{S}$, say, { u k } \{u_k\} {uk​}, { v k } \{v_k\} {vk​}, and { w k } \{w_k\} {wk​}. They are linearly independent precisely when the equation
  $$c_1{u}_k+c_2{v}_k+c_3{w}_k=0forallk(1)$$implies that $c_1=c_2=c_3=0$.
• Suppose $c_1,c_2,c_3$ satisfy (1). Then equation (1) holds for any three consecutive values of $k$, say, $k$, $k+1$, and $k+2$. Thus (1) implies that
  The coefficient matrix in this system is called the Casorati matrix of the signals, and the determinant of the matrix is called the Casoratian of { u k } \{u_k\} {uk​}, { v k } \{v_k\} {vk​}, and { w k } \{w_k\} {wk​}.
  • If the Casorati matrix is invertible for at least one value of $k$, then (2) will imply that $c_1=c_2=c_3=0$, which will prove that the three signals are linearly independent.
  • If a Casorati matrix is not invertible, the associated signals being tested may or may not be linearly dependent.
    • However, it can be shown that if the signals are all solutions of the same homogeneous difference equation, then either the Casorati matrix is invertible for all $k$ and the signals are linearly independent, or else the Casorati matrix is not invertible for all $k$ and the signals are linearly dependent. (Check the appendix)

---

EXAMPLE 2

Verify that $1^k,(-2{)}^k$, and $3^k$ are linearly independent signals.

SOLUTION

• The Casorati matrix is
  Row operations can show fairly easily that this matrix is always invertible. However, it is faster to substitute a value for $k$—say, $k=0$—and row reduce the numerical matrix:
  So $1^k,(-2{)}^k$, and $3^k$ are linearly independent signals.

Linear Difference Equations

• Given scalars $a_0,...,a_n$, with $a_0$ and $a_n$ nonzero, and given a signal { z k } \{z_k\} {zk​}, the equation
  is called a linear difference equation (or linear recurrence relation) of order $\mathbf{n}$ ( $\mathbf{n}$ 阶线性差分方程 ). For simplicity, $a_0$ is often taken equal to $1$. If { z k } \{z_k\} {zk​} is the zero sequence, the equation is homogeneous; otherwise, the equation is nonhomogeneous.

$y_{k+3}+5y_{k+2}+6y_{k+1}=0$ 是 $2$ 阶差分方程 (将 $k$ 用 $k-1$ 替换)

---

EXAMPLE 3

• In digital signal processing, a difference equation such as (3) describes a linear filter (线性滤波器), and $a_0,...,a_n$ are called the filter coefficients. If { y k } \{y_k\} {yk​} is treated as the input and { z k } \{z_k\} {zk​} as the output, then the solutions of the associated homogeneous equation are the signals that are filtered out and transformed into the zero signal.
• Let us feed two different signals into the filter
  Here $.35$ is an abbreviation for $\sqrt{2}/4$.
  
  • The first signal is created by sampling the continuous signal $y=\cos (\pi t/4)$ at integer values of $t$, as in Figure 3(a). The discrete signal is
    For simplicity, write $\pm .7$ in place of $\pm \sqrt{2}/2$, so that
    Table 1 shows a calculation of the output sequence { z k } \{z_k\} {zk​}. The output is { y k } \{y_k\} {yk​} shifted by one term.
    
  • A different input signal is produced from the higher frequency signal $y=\cos (3\pi t/4)$, shown in Figure 3(b). Sampling at the same rate as before produces a new input sequence:
    When { w k } \{w_k\} {wk​} is fed into the filter, the output is the zero sequence. The filter, called a $low-pass$ $filter$, lets { y k } \{y_k\} {yk​} pass through, but stops the higher frequency { w k } \{w_k\} {wk​}.

---

• In many applications, a sequence { z k } \{z_k\} {zk​} is specified for the right side of a difference equation (3), and a { y k } \{y_k\} {yk​} that satisfies (3) is called a solution of the equation. The next example shows how to find solutions for a homogeneous equation.

EXAMPLE 4

Solutions of a homogeneous difference equation often have the form $y_k=r^k$ for some $r$. Find some solutions of the equation

SOLUTION

• Substitute $r^k$ for $y_k$ in the equation and factor the left side:
  Thus $1^k$, $(-2{)}^k$, and $3^k$ are all solutions of (4).
• In general, a nonzero signal $r^k$ satisfies the homogeneous difference equation
  if and only if $r$ is a root of the auxiliary equation
  When the auxiliary equation has a $complexroot$(复数根), the difference equation has solutions of the form $s^k\cos kw$ and $s^k\sin kw$, for constants $s$ and $w$.

Solution Sets of Linear Difference Equations

• Given $a_1,...,a_n$, consider the mapping $T:\mathbb{S}\to \mathbb{S}$ that transforms a signal { y k } \{y_k\} {yk​} into a signal { w k } \{w_k\} {wk​} given by
  It is readily checked that $T$ is a $linear$ transformation. This implies that the solution set of the homogeneous equationis the kernel of $T$, and hence the solution set is a $subspace$ of $\mathbb{S}$. Any linear combination of solutions is again a solution.

---

• The next theorem, a simple but basic result, will lead to more information about the solution sets of difference equations.

PROOF

• If $y_0,...,y_{n-1}$ are specified, use (7) to define
  And now that $y_1,...,y_n$ are specified, use (7) to define $y_{n+1}$. In general, use the recurrence relationto define $y_{n+k}$ for $k\ge 0$. To define $y_k$ for $k<0$, use the recurrence relation
  So the solution of (7) is unique.

---

PROOF

• As was pointed out earlier, $H$ is a subspace of $\mathbb{S}$ because $H$ is the kernel of a linear transformation.
• For { y k } \{y_k\} {yk​} in $H$, let F { y k } F\{y_k\} F{yk​} be the vector in ${\mathbb{R}}^n$ given by $(y_0,y_1,...,y_{n-1})$. It is readily verified that $F:H\to {\mathbb{R}}^n$ is a linear transformation. Given any vector $(y_0,y_1,...,y_{n-1})$ in ${\mathbb{R}}^n$, Theorem 16 says that there is a unique signal { y k } \{y_k\} {yk​} in $H$ such that F { y k } = ( y 0 , y 1 , . . . , y n − 1 ) F \{y_k\}= (y_0, y_1,..., y_{n-1}) F{yk​}=(y0​,y1​,...,yn−1​). This means that $F$ is a one-to-one linear transformation of $H$ onto ${\mathbb{R}}^n$; that is, $F$ is an isomorphism. Thus $dimH=dim{\mathbb{R}}^n=n$.

---

EXAMPLE 5

Find a basis for the set of all solutions to the difference equation

SOLUTION

• We know from Examples 2 and 4 that $1^k$, $(-2{)}^k$, and $3^k$ are linearly independent solutions. Theorem 17 shows that the solution space is exactly three-dimensional. So $1^k$, $(-2{)}^k$, and $3^k$ form a basis for the solution space.

---

• The standard way to describe the “general solution” of the difference equation (10) is to exhibit a basis for the subspace of all solutions. Such a basis is usually called a fundamental set of solutions (基础解系) of (10).

Nonhomogeneous Equations

• The general solution of the nonhomogeneous difference equation
  can be written as one particular solution of (11) plus an arbitrary linear combination of a fundamental set of solutions of the corresponding homogeneous equation (10).
  • This fact is analogous to the result in Section 1.5 showing that the solution sets of $A\mathbf{x}=\mathbf{b}$ and $A\mathbf{x}=0$ are parallel.

Reduction to Systems of First-Order Equations

化简为一阶方程组

• A modern way to study a homogeneous $n$th-order linear difference equation is to replace it by an equivalent system of first-order difference equations, written in the form
  where the vectors ${\mathbf{x}}_k$ are in ${\mathbb{R}}^n$ and $A$ is an $n\times n$ matrix.

---

EXAMPLE 7

Write the following difference equation as a first-order system:

SOLUTION

• For each $k$, set
  So
  That is,
  
• In general, the equation
  can be rewritten as ${\mathbf{x}}_{k+1}=A{\mathbf{x}}_k$ for all $k$, where
  

---

Appendix: The Casorati Test

Let { y 1 , … , y n } \{\boldsymbol y_1, …, \boldsymbol y_n\} {y1​,…,yn​} be a set of signals in $\mathbb{S}$. For $j=1,\dots ,n$ and for any $k$, let ${\mathbf{y}}_j(k)$ denote the $k$th entry in the signal ${\mathbf{y}}_j$ and let

• a. If $C(k)$ is invertible for some $k$, { y l , … , y n } \{\boldsymbol y_l, …, \boldsymbol y_n\} {yl​,…,yn​} is linearly independent.
• b. If ${\mathbf{y}}_l,\dots ,{\mathbf{y}}_n$ all satisfy a homogeneous difference equation of order $n$,
  (with $a_n\ne 0$), and if the Casorati matrix $C(k)$ is not invertible for some $k$, then { y 1 , … , y n } \{\boldsymbol y_1, …, \boldsymbol y_n\} {y1​,…,yn​} is linearly dependent in $\mathbb{S}$, and for all $k$, $C(k)$ is not invertible.

Proof

• (b) Suppose that ${\mathbf{y}}_1,\dots ,{\mathbf{y}}_n$ are in the set $H$ of solutions of $(\ast )$ and $C(k_0)$ is not invertible for some $k_0$. It is readily verified that if $T:H\to R^n$ is defined by
  then $T$ is a linear transformation. The proof of Theorem 16 is easily modified to show that $(\ast )$ has a unique solution $\mathbf{y}$ whenever $\mathbf{y}(k_0),\dots ,\mathbf{y}(k_0+n–1)$ are specified. This means that $T$ is a one-to-one mapping of $H$ onto ${\mathbb{R}}^n$. Furthermore, the images $T({\mathbf{y}}_1),\dots ,T({\mathbf{y}}_n)$ form the columns of the Casorati matrix $C(k_0)$ and hence are linearly dependent, because $C(k_0)$ is not invertible. Since $T$ is one-to-one, { y 1 , … , y n } \{\boldsymbol y_1, …, \boldsymbol y_n\} {y1​,…,yn​} is linearly dependent. This proves the first statement in (b). The second statement follows immediately from part (a), because if $C(k)$ were invertible for some $k$, then { y 1 , … , y n } \{\boldsymbol y_1, …, \boldsymbol y_n\} {y1​,…,yn​} would be linearly independent, which is not true. So $C(k)$ is not invertible for each $k$.