Chapter 2 (Matrix Algebra): Partitioned matrices (分块矩阵)

本文链接：https://blog.csdn.net/weixin_42437114/article/details/108354617

本文为《Linear algebra and its applications》的读书笔记

Partitioned matrices

A key feature of our work with matrices has been the ability to regard a matrix $A$ as a list of column vectors rather than just a rectangular array of numbers. This point of view has been so useful that we wish to consider other partitions of $A$ , indicated by horizontal and vertical dividing rules.

EXAMPLE 1

The matrix

can also be written as the $\times 3$ partitioned (or block) matrix

whose entries are the blocks (分块) (or submatrices)

EXAMPLE 2

When a matrix $A$ appears in a mathematical model of a physical system such as an electrical network, a transportation system, or a large corporation, it may be natural to regard $A$ as a partitioned matrix.
- For instance, if a microcomputer circuit board consists mainly of three VLSI (very large-scale integrated) microchips, then the matrix for the circuit board might have the general form
  The submatrices on the “diagonal” of $A$ —namely, $A_{11}$ , $A_{22}$ , and $A_{33}$ —concern the three VLSI chips, while the other submatrices depend on the interconnections among those microchips.

Addition and Scalar Multiplication

If matrices $A$ and $B$ are the same size and are partitioned in exactly the same way, then it is natural to make the same partition of the ordinary matrix sum $A + B$ .
In this case, each block of $A + B$ is the (matrix) sum of the corresponding blocks of $A$ and $B$ . Multiplication of a partitioned matrix by a scalar is also computed block by block.

Multiplication of Partitioned Matrices

Partitioned matrices can be multiplied by the usual row–column rule as if the block entries were scalars, provided that for a product $A B$ , the column partition of $A$ matches the row partition of $B$ .

EXAMPLE 3

Let
The 5 columns of $A$ are partitioned into a set of 3 columns and then a set of 2 columns. The 5 rows of $B$ are partitioned in the same way—into a set of 3 rows and then a set of 2 rows. We say that the partitions of $A$ and $B$ are conformable for block multiplication. It can be shown that the ordinary product $A B$ can be written as
It is important for each smaller product in the expression for $A B$ to be written with the submatrix from $A$ on the left. For instance,
The row–column rule for multiplication of block matrices provides the most general way to regard the product of two matrices.

下面就理解一下为什么像上面这样进行分块矩阵乘法是正确的：
- 参考：https://zhuanlan.zhihu.com/p/133330692
对于分块矩阵 $A$ 和 $B$ ，证明
$\begin{bmatrix} A_{11}&A_{12}\\ A_{21}&A_{22}\end{bmatrix} \begin{bmatrix} B_{11}&B_{12}\\ B_{21}&B_{22}\end{bmatrix}=\begin{bmatrix} A_{11}B_{11}+A_{12}B_{21}&A_{11}B_{12}+A_{12}B_{22}\\ A_{21}B_{11}+A_{22}B_{21}&A_{21}B_{12}+A_{22}B_{22}\end{bmatrix}\ \ \ \ \ \ \ (1)$
- 首先考虑 $A$ 不分块，将 $B$ 按列分为两块
  $AB=A\begin{bmatrix} B_{11}&B_{12}\end{bmatrix}$ 回忆矩阵乘法的定义， $AB=\begin{bmatrix} A\boldsymbol b_{1}&A\boldsymbol b_{2}&...&A\boldsymbol b_p\end{bmatrix}$ 。也就是说， $B$ 的每一列都作为权重，对 $A$ 的列向量进行线性组合，每一列都是单独工作的。因此，就可以很自然地得到
  $AB=A\begin{bmatrix} B_{11}&B_{12}\end{bmatrix}=\begin{bmatrix} AB_{11}&AB_{12}\end{bmatrix}$
  - 由于每一列都是单独工作的，从中还可以总结出一条规则：当对 $B$ 的列进行划分时，只要找到 $A B$ 的第一列的形式，结果中的每一列都保持相同形式，只不过将 $B$ 的列标替换为相应的列标
    - 例如在 $(1)$ 式中，第二列就是第一列将 $B_{11}\rightarrow B_{12},B_{21}\rightarrow B_{22}$
- 再考虑 $B$ 不分块，将 $A$ 按行分为两块
  $AB=\begin{bmatrix} A_{11}\\A_{21}\end{bmatrix}B$ 而 $AB=\begin{bmatrix} row_{1}A\cdot B\\row_{2}A\cdot B\\...\\row_{p}A\cdot B\end{bmatrix}$ 。因此
  $AB=\begin{bmatrix} A_{11}\\A_{21}\end{bmatrix}B=\begin{bmatrix} A_{11}B\\A_{21}B\end{bmatrix}$
  - 类似得，也可以总结出一条规则：当对 $A$ 的行进行划分时，只要找到 $A B$ 的第一行的形式，结果中的每一行都保持相同形式，只不过将 $A$ 的行标替换为相应的行标
    - 例如在 $(1)$ 式中，第二行就是第一行将 $A_{11}\rightarrow A_{21},A_{12}\rightarrow A_{22}$
- 同时对 $A$ 按行进行水平划分，对 $B$ 按列进行垂直划分
  $AB=\begin{bmatrix} A_{11}\\A_{21}\end{bmatrix}\begin{bmatrix} B_{11}&B_{12}\end{bmatrix}=\begin{bmatrix} \begin{bmatrix}A_{11}\\A_{21}\end{bmatrix}B_{11}&\begin{bmatrix}A_{11}\\A_{21}\end{bmatrix}B_{12}\end{bmatrix}=\begin{bmatrix} A_{11}B_{11}&A_{11}B_{12}\\A_{21}B_{11}&A_{21}B_{12}\end{bmatrix}$
- 按照上面总结的两条规则，
  $\begin{aligned}\begin{bmatrix} A_{11}&A_{12}\\ A_{21}&A_{22}\end{bmatrix} \begin{bmatrix} B_{11}&B_{12}\\ B_{21}&B_{22}\end{bmatrix}&= \begin{bmatrix}\begin{bmatrix} A_{11}&A_{12}\\ A_{21}&A_{22}\end{bmatrix}\begin{bmatrix} B_{11}\\B_{21}\end{bmatrix}&\begin{bmatrix} A_{11}&A_{12}\\ A_{21}&A_{22}\end{bmatrix}\begin{bmatrix} B_{12}\\B_{22}\end{bmatrix}\end{bmatrix} \\&=\begin{bmatrix}\begin{bmatrix} A_{11}&A_{12}\end{bmatrix}\begin{bmatrix} B_{11}\\B_{21}\end{bmatrix}&\begin{bmatrix} A_{11}&A_{12}\end{bmatrix}\begin{bmatrix} B_{12}\\B_{22}\end{bmatrix}\\ \begin{bmatrix} A_{21}&A_{22}\end{bmatrix}\begin{bmatrix} B_{11}\\B_{21}\end{bmatrix}&\begin{bmatrix} A_{21}&A_{22}\end{bmatrix}\begin{bmatrix} B_{12}\\B_{22}\end{bmatrix}\end{bmatrix}\ \ \ \ \ \ \ (1) \end{aligned}$ 要证明 $(1)$ 式，其实就只要证明 $\begin{bmatrix} A_{11}&A_{12}\end{bmatrix}\begin{bmatrix} B_{11}\\B_{21}\end{bmatrix}=\begin{bmatrix}A_{11}B_{11}+A_{12}B_{21}\end{bmatrix}$ 。 $(1)$ 式的其余部分可以用两条规则中的替换的方法得到。这部分可以由列的观点说明：本来我们是用 $B$ 的每一列对 $A$ 的列向量线性组合，现在把这个工作分成两截，用 $B$ 的前几行对 $A$ 的前几列线性组合，用 $B$ 的后几行对 $A$ 的后几列线性组合，然后再把它们加到一起，这和用 $B$ 对整个 $A$ 的列线性组合效果是一样的

在这里插入图片描述

EXAMPLE 4

Compute $X^TX$ , where $X$ is partitioned as $\begin{bmatrix} X_1&X_2\end{bmatrix}$ .

SOLUTION
$X^TX=\begin{bmatrix} X_1^T\\X_2^T\end{bmatrix}\begin{bmatrix} X_1&X_2\end{bmatrix}=\begin{bmatrix} X_1^TX_1&X_1^TX_2\\X_2^TX_1&X_2^TX_2\end{bmatrix}$

The partitions of $X^T$ and $X$ are automatically conformable for block multiplication because the columns of $X^T$ are the rows of $X$ .
This partition of $X^TX$ is used in several computer algorithms for matrix computations.

EXAMPLE 5

Use partitioned matrices to prove by induction (数学归纳法) that the product of two lower triangular matrices (下三角矩阵) is also lower triangular.

SOLUTION

[Hint: A $1)\times (k + 1)$ matrix $A_1$ can be written in the form below, where $a$ is a scalar, $\boldsymbol v$ is in $\mathbb R_k$ , and $A$ is a $\times k$ lower triangular matrix. ]
$A_1=\begin{bmatrix} a&\boldsymbol 0^T\\\boldsymbol v&A\end{bmatrix}$
Prove: The product of two $n\times n$ lower triangular matrices is lower triangular. (*)
- The statement is true for $n = 1$ .
- The “induction step” is next. Suppose that (*) is true when $n$ is some positive integer $k$ , and consider any $(k+1)\times(k+1)$ lower-triangular matrices $A_1$ and $B_1$ . Partition these matrices as
  $A_1=\begin{bmatrix} a&\boldsymbol 0^T\\\boldsymbol v&A\end{bmatrix},B_1=\begin{bmatrix} b&\boldsymbol 0^T\\\boldsymbol w&B\end{bmatrix}$ where $A$ and $B$ are $k \times k$ matrices, $\boldsymbol v$ and $\boldsymbol w$ are in $\mathbb R^k$ , and $a$ and $b$ are scalars. Since $A_1$ and $B_1$ are lower triangular, so are $A$ and $B$ . NowAssuming (*) is true for $n = k$ , $A B$ must be lower triangular. The form of $A_1B_1$ shows that it, too, is lower triangular. Thus the statement (*) about lower triangular matrices is true for $n = k + 1$ if it is true for $n = k$ . By the principle of induction, (*) is true for all $\geq 1$ .

Inverses of Partitioned Matrices

EXAMPLE 6

A matrix of the form
$A=\begin{bmatrix} A_{11}& A_{12}\\0& A_{22}\end{bmatrix}$ is said to be $block\ upper\ triangular (分块上三角矩阵)$ . Assume that $A_{11}$ is $\times p$ , $A_{22}$ is $\times q$ , and $A$ is invertible. Find a formula for $A^{-1}$ .

SOLUTION

Denote $A^{-1}$ by $B$ and partition $B$ so that
$\begin{bmatrix} A_{11}& A_{12}\\0& A_{22}\end{bmatrix}\begin{bmatrix} B_{11}& B_{12}\\B_{21}& B_{22}\end{bmatrix}=\begin{bmatrix} I_p& 0\\0& I_q\end{bmatrix}$
This matrix equation provides four equations：
Since $A_{22}$ is square, equation (6) shows that $A_{22}$ is invertible and $B_{22}=A_{22}^{-1}$ . Next, left-multiply both sides of (5) by $A_{22}^{-1}$ and obtain
$B_{21}=0$ so that (3) simplifies to
$A_{11}B_{11} = I_p$ Since $A_{11}$ is square, this shows that $A_{11}$ is invertible and $B_{11}=A_{11}^{-1}$ . Finally, use these results with (4) to find that
$B_{12}=-A_{11}^{-1}A_{12}A_{22}^{-1}$ Thus
$A^{-1}=\begin{bmatrix} A_{11}& A_{12}\\0& A_{22}\end{bmatrix}^{-1}=\begin{bmatrix} A_{11}^{-1}& -A_{11}^{-1}A_{12}A_{22}^{-1}\\0& A_{22}^{-1}\end{bmatrix}$

A block diagonal matrix (分块对角矩阵) is a partitioned matrix with zero blocks off the main diagonal (of blocks).
Such a matrix is invertible if and only if each block on the diagonal is invertible, and the inverse is the block diagonal matrix formed from the inverses of the diagonal blocks.

EXAMPLE 7

Without using row reduction, find the inverse of
$A=\left[\begin{array}{lllll} 1 & 2 & 0 & 0 & 0 \\ 3 & 5 & 0 & 0 & 0 \\ 0 & 0 & 2 & 0 & 0 \\ 0 & 0 & 0 & 7 & 8 \\ 0 & 0 & 0 & 5 & 6 \end{array}\right]$

SOLUTION

View the $5\times5$ matrix in this example as a $3\times3$ block matrix:
Finish by inverting each of the diagonal blocks and use the results to assemble $A^{–1}$ ,