Linear Algebra Lecture 3

Linear Algebra Lecture 3

1.Matrix multiplication in 4 ways

2.Inverse of A, AB, $A^T$

3.Gauss-Jordan idea

---

Matrix multiplication

$$A\begin{bmatrix} \cdots \\ \cdots \end{bmatrix}B\begin{bmatrix} \cdots \\ \cdots \end{bmatrix} = C\begin{bmatrix} \cdots \\ \cdots \end{bmatrix}$$

Standard rule
$C_{ij}$ is from row i of A and column j of B.

$C_{ij} = A_i \cdot B_j = \sum_{k=1}^n a_{ik}b_{kj}$

When are we allowed to multiply matrices?
If they are square, they’ve got to be the same size.
If they are rectangular, not the same size.

$A_{m\times n} B_{n\times p} = C_{m\times p}$

Multiply by column

$$A_{m\times n}\begin{bmatrix} \cdots \\ \cdots \end{bmatrix}B_{n\times p} \begin{bmatrix} \vdots & \end{bmatrix} = C_{m\times p} \begin{bmatrix} \cdots \\ \cdots \end{bmatrix}$$
Multiply a matrix by column one tells the first column of the answer. And multiply matrix A times each column of matrix B can get the columns of the answer.

Columns of C are combinations of columns of A.

Multiply by row

$$A_{m\times n}\begin{bmatrix} \cdots \end{bmatrix}B_{n\times p} \begin{bmatrix} \cdots \\ \cdots \\ \cdots \end{bmatrix} = C_{m\times p} \begin{bmatrix} \cdots \end{bmatrix}$$

Rows of C are combinations of rows of B.

Column times row
When row times a column, it gave a number.
If I multiply a column of A times a row of B, I get a full-size matrix.
$A_{m\times 1} B_{1\times p} = C_{m\times p}$

$\begin{bmatrix}2\\3\\4\end{bmatrix}\begin{bmatrix}1&6\end{bmatrix} = \begin{bmatrix}2&12\\3&18\\4&24\end{bmatrix}$

Times all columns and rows at a time.

$AB = \sum (Cols of A)\times (Rows of B)$
$\begin{bmatrix}2&7\\3&8\\4&9\end{bmatrix}\begin{bmatrix}1&6\\0&0\end{bmatrix} = \begin{bmatrix}2\\3\\4\end{bmatrix}\begin{bmatrix}1&6\end{bmatrix} + \begin{bmatrix}7\\8\\9\end{bmatrix}\begin{bmatrix}0&0\end{bmatrix}$

Block Multiplication
We can also cut the matrix into blocks and do the multiplication by blocks.

Chop a matrix A into four square blocks, and B is a same size square also.
$A\begin{bmatrix} \cdots&\cdots\\ \cdots&\cdots\end{bmatrix} B\begin{bmatrix} \cdots&\cdots\\ \cdots&\cdots\end{bmatrix} \to A\begin{bmatrix} A_1&A_2\\A_3&A_4\end{bmatrix} B\begin{bmatrix}B_1&B_2\\ B_3&B_4\end{bmatrix} = \begin{bmatrix}A_1B_1+A_2B_3&A_1B_2+A_2B_4\\ A_3B_1+A_4B_3&A_3B_2+A_4B_4\end{bmatrix}$

---

Inverses

Inverses of square matrices
If you know a square matrix A, is it invertible or not? If it is invertible, then there is some other matrix, call it A inverse.

For square matrices, a left inverse is also a right inverse. If I can find a matrix on the left that gets the identity, then also that matrix on the right will produce that identity.
$A^{-1}A = I = A A^{-1}$

For rectangular matrices, we’ll see a left inverse isn’t a right inverse.

If A inverse exists?

These matrices are called invertible(可逆) or non-singular(非奇异).
If singular case, no inverse.

$A=\begin{bmatrix}1&2\\ 3&6\end{bmatrix}$

A square matrix won’t have an inverse because I can find a vector X with $Ax=0$. The matrix can’t have an inverse if some combination of columns gives nothing.
Because, I could take $Ax=0$, I could multiply by A inverse. $AA^{-1}=I$ and $0 \times A^{-1} = 0$, so $x=0$.
So our conclusion will be that non-invertible matrices, singular matrices, some combinations of their columns gives the zero column, there is no way A inverse can recover.

How to compute A inverse?

$A \begin{bmatrix}1&2 \\ 3&7 \end{bmatrix}A^{-1}\begin{bmatrix}a&c\\ b&d\end{bmatrix} = I \begin{bmatrix}1&0 \\ 0&1 \end{bmatrix}$

Gauss-Jordan idea solves two equations at once.

$$\begin{cases} \begin{bmatrix}1&2 \\ 3&7 \end{bmatrix} \begin{bmatrix}a \\ b \end{bmatrix} = \begin{bmatrix}1 \\ 0 \end{bmatrix}\\ \begin{bmatrix}1&2 \\ 3&7 \end{bmatrix} \begin{bmatrix}c\\ d \end{bmatrix} = \begin{bmatrix}0 \\ 1 \end{bmatrix} \end{cases}$$

Use augmented matrix, add the identity at the right-hand side of the matrix, then do elimination.

$A\begin{bmatrix}1&3 \\ 2&7 \end{bmatrix}I \begin{matrix} 1&0\\0&1\end{matrix} \to (Upper TriangularForm)\begin{bmatrix}1&3 \\ 0&1 \end{bmatrix}\begin{matrix} 1&0\\-2&1\end{matrix} \to (UpwardElimination)I\begin{bmatrix}1&0 \\ 0&1 \end{bmatrix}\begin{matrix} 7&-3\\-2&1\end{matrix}$

$E\begin{bmatrix}A&I \end{bmatrix} = \begin{bmatrix}I&A^{-1}\end{bmatrix}$
Because $EA=I$ tells us $E=A^{-1}$