Matrix operations

本文为《Linear algebra and its applications》的读书笔记

• If $A$ is an $m\times n$ matrix, each column of $A$ identifies a vector in ${\mathbb{R}}^m$.
  $$A=[{\mathbf{a}}_1{\mathbf{a}}_2...{\mathbf{a}}_n]$$
• The diagonal entries in an $m\times n$ matrix $A=[a_{ij}]$ are $a_{11}$, $a_{22}$, $a_{33}$… and they form the main diagonal (主对角线) of $A$. A diagonal matrix (对角矩阵) is a square $n\times n$ matrix whose nondiagonal entries are zero. An $m\times n$ matrix whose entries are all zero is a zero matrix (零矩阵) and is written as $0$. The size of a zero matrix is usually clear from the context.

---

• The arithmetic for vectors described earlier has a natural extension to matrices.

Sums and Scalar Multiples

Sum

• The sum $A+B$ is the $m\times n$ matrix whose columns are the sums of the corresponding columns in $A$ and $B$.
• The sum $A+B$ is defined only when $A$ and $B$ are the same size.

---

Scalar Multiple

• If $r$ is a scalar and $A$ is a matrix, then the scalar multiple $rA$ is the matrix whose columns are $r$ times the corresponding columns in $A$.

Matrix Multiplication

Multiplication of matrices corresponds to composition of linear transformations

• When a matrix $B$ multiplies a vector $\mathbf{x}$, it transforms $\mathbf{x}$ into the vector $B\mathbf{x}$. If this vector is then multiplied in turn by a matrix $A$, the resulting vector is $A(B\mathbf{x})$. See Figure 2.
  Thus $A(B\mathbf{x})$ is produced from $\mathbf{x}$ by a $composition$ of mappings. Our goal is to represent this composite mapping as multiplication by a single matrix, denoted by $AB$, so that
  $$A(B\mathbf{x})=(AB)\mathbf{x}$$

---

• If $A$ is $m\times n$, $B$ is $n\times p$, and $\mathbf{x}$ is in ${\mathbb{R}}^p$. Then
  $$B\mathbf{x}=x_1{\mathbf{b}}_1+...+x_p{\mathbf{b}}_p$$By the linearity of multiplication by $A$,
  $$A(B\mathbf{x})=A(x_1{\mathbf{b}}_1)+...+A(x_p{\mathbf{b}}_p)=x_{1}A{\mathbf{b}}_1+...+x_{p}A{\mathbf{b}}_p$$
• The vector $A(B\mathbf{x})$ is a linear combination of the vectors $A{\mathbf{b}}_1$,…,$A{\mathbf{b}}_p$, using the entries in $\mathbf{x}$ as weights. In matrix notation, this linear combination is written as
  $$A(B\mathbf{x})=[A{\mathbf{b}}_{1}A{\mathbf{b}}_2...A{\mathbf{b}}_p]\mathbf{x}$$Thus multiplication by $[A{\mathbf{b}}_{1}A{\mathbf{b}}_2...A{\mathbf{b}}_p]$ transforms $\mathbf{x}$ into $A(B\mathbf{x})$. We have found the matrix we sought!

---

• Each column of $AB$ is a linear combination of the columns of $A$ using weights from the corresponding column of $B$.
• Obviously, the number of columns of $A$ must match the number of rows in $B$ in order for a linear combination such as $A{\mathbf{b}}_1$ to be defined. Also, the definition of $AB$ shows that $AB$ has the same number of rows as $A$ and the same number of columns as $B$.

The definition of $AB$ lends itself well to parallel processing on a computer. The columns of $B$ are assigned individually or in groups to different processors, which independently and hence simultaneously compute the corresponding columns of $AB$.

---

EXAMPLE 3

Compute $AB$, where $A=\left[\begin{array}{cc}2& 3\\ 1& -5\end{array}\right]$ and $B=\left[\begin{array}{ccc}4& 3& 6\\ 1& -2& 3\end{array}\right]$.

SOLUTION

---

• The definition of $AB$ is important for theoretical work and applications, but the following rule provides a more efficient method for calculating the individual entries in $AB$ when working small problems by hand.

• Let $ro{w}_i(A)$ denote the $i$ th row of a matrix $A$. Then
  $$ro{w}_i(AB)=ro{w}_i(A)\cdot B$$

---

Inner product

• View vectors in ${\mathbb{R}}^n$ as $n\times 1$ matrices. For $\mathbf{u}$ and $\mathbf{v}$ in ${\mathbb{R}}^n$, the matrix product ${\mathbf{u}}^T\mathbf{v}$ is a $1\times 1$ matrix, called the scalar product (数量积), or inner product (内积), of $\mathbf{u}$ and $\mathbf{v}$. It is usually written as a single real number without brackets.
• Inner products (${\mathbf{u}}^T\mathbf{v}$ and ${\mathbf{v}}^T\mathbf{u}$) have the transpose symbol in the middle.
• ${\mathbf{u}}^T\mathbf{v}={\mathbf{v}}^T\mathbf{u}$

Outer product

• The matrix product $\mathbf{u}{\mathbf{v}}^T$ is an $n\times n$ matrix, called the outer product (外积) of $\mathbf{u}$ and $\mathbf{v}$.
• Outer products ($\mathbf{u}{\mathbf{v}}^T$ and $\mathbf{v}{\mathbf{u}}^T$) have the transpose symbol on the outside.
• 外积 $\mathbf{u}{\mathbf{v}}^T$ 和 $\mathbf{v}{\mathbf{u}}^T$ 互为对称矩阵

Properties of Matrix Multiplication

• Recall that $I_m$ represents the $m\times m$ identity matrix and $I_m\mathbf{x}=\mathbf{x}$ for all $\mathbf{x}$ in ${\mathbb{R}}^m$.

• 补充性质：
  • (1) 两个同阶上三角矩阵的乘积仍为上三角矩阵
  • (2) 两个同阶下三角矩阵的乘积仍为下三角矩阵
  • (3) 两个同阶对角矩阵的乘积仍为对角矩阵，且新的对角元为两个原对角元的乘积

PROOF

Property (a)

• Property (a) follows from the fact that matrix multiplication corresponds to composition of linear transformations (which are functions), and it is known that the composition of functions is associative.

在离散数学中，函数的复合运算 $f\circ g$ 被定义为关系乘积 $g\ast f$，可以证明关系乘积是满足结合律的，因此函数的复合运算也满足结合律

• Here is another proof of (a) that rests on the “column definition” of the product of two matrices. Let
  $$\begin{gathered} C=[{\mathbf{c}}_1{\mathbf{c}}_2...{\mathbf{c}}_p] \\ BC=[B{\mathbf{c}}_{1}B{\mathbf{c}}_2...B{\mathbf{c}}_p] \\ A(BC)=[A(B{\mathbf{c}}_1)A(B{\mathbf{c}}_2)...A(B{\mathbf{c}}_p)] \end{gathered}$$Recall that the definition of $AB$ makes $A(B\mathbf{x})=(AB)\mathbf{x}$ for all $\mathbf{x}$, so
  $$A(BC)=[(AB){\mathbf{c}}_1...(AB){\mathbf{c}}_p]=(AB)C$$

---

WARNINGS:

• In general, $AB\ne BA$. If $AB=BA$, we say that $A$ and $B$ commute with one another.(可交换的)
• The cancellation laws (消去律) do not hold for matrix multiplication. That is, if $AB=AC$, then it is not true in general that $B=C$.
• If a product $AB$ is the zero matrix, you cannot conclude in general that either $A=0$ or $B=0$.

Tip: When $B$ is square and $C$ has fewer columns than $A$ has rows, it is more efficient to compute $A(BC)$ than $(AB)C$ .

---

$Checkpoint$:

Show that if $\mathbf{y}$ is a linear combination of the columns of $AB$, then $\mathbf{y}$ is a linear combination of the columns of $A$.

$AnswertoCheckpoint$:

• If $\mathbf{y}$ is a linear combination of the columns of $AB$, then there is a vector $\mathbf{x}$ such that $\mathbf{y}$ = $(AB)\mathbf{x}$. By definition of matrix multiplication, $\mathbf{y}=A(B\mathbf{x})$. This expresses $\mathbf{y}$ as a linear combination of the columns of $A$ using the entries in the vector $B\mathbf{x}$ as weights.

---

EXAMPLE 4

Let $x_1,...,x_n$ be fixed numbers. The matrix below, called a Vandermonde matrix (范德蒙德矩阵), occurs in applications such as signal processing, error-correcting codes (纠错码), and polynomial interpolation.
Given $\mathbf{y}=(y_1,...,y_n)$ in ${\mathbb{R}}^n$, suppose $\mathbf{c}=(c_0,...,c_{n-1})$ in ${\mathbb{R}}^n$ satisfies $V\mathbf{c}=\mathbf{y}$, and define the polynomial
$$p(t)=c_0+c_{1}t+c_2{t}^2+...+c_{n-1}{t}^{n-1}$$

• a. Show that $p(x_1)=y_1,...,p(x_n)=y_n$. We call $p(t)$ an $interpolating$ $polynomial$ $for$ $the$ $points$ $(x_1,y_1),...,(x_n,y_n)$ because the graph of $p(t)$ passes through the points.
• b. Suppose $x_1,...,x_n$ are distinct numbers (相异的数). Show that the columns of $V$ are linearly independent.
• c. Prove: “If $x_1,...,x_n$ are distinct numbers, and $y_1,...,y_n$ are arbitrary numbers, then there is an interpolating polynomial of degree $\le n-1$ for $(x_1,y_1),...,(x_n,y_n)$.”

SOLUTION

• (b) [Hint: How many zeros can a polynomial of degree $n-1$ have?] (一个 $n-1$ 次多项式有多少个零点？)

Powers of a Matrix

• If $A$ is an $n\times n$ matrix and if $k$ is a positive integer, then
  
• If $k=0$; then $A^0\mathbf{x}$ should be $\mathbf{x}$ itself. Thus $A^0$ is interpreted as the identity matrix.

The Transpose of a Matrix

---

• The generalization of Theorem 3(d) to products of more than two factors can be stated in words as follows: