Mathematics Basics - Linear Algebra (Matrix Part 2)

Matrix Inverse and Determinant

Matrix Inverse

We will now turn our attention to solving simultaneous equations. Elimination and substitution are the typical methods we employ to solve simultaneous equations. It turns out matrix multiplication offers another approach we can obtain a solution.

This relies on one property of matrix called matrix inverse. Multiplying a matrix by its inverse will result in an identity matrix.

$$A^{-1}\ast A=I$$

where $A^{-1}$ is the inverse of matrix A and I is the identity matrix.

To solve a simultaneous equation $A\ast r=s$ for vector $r$, we can rearrange the equation as follows

$$\begin{array}{rl}A\ast r& =s\\ A^{-1}\ast (A\ast r)& =A^{-1}\ast s\\ r& =A^{-1}\ast s\end{array}$$

Therefore, solution for vector $r$ can be obtained by multiplying $A^{-1}$ with $s$.

However, finding matrix inverse is a non-trivial task. There exists one shortcut to calculate matrix inverse if it is a 2 by 2 square matrix.

$${\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)}^{-1}=\frac{1}{ad-bc}\left(\begin{array}{cc}d& -b\\ -c& a\end{array}\right)\text{\hspace{1em}(1)}$$

To find inverse of matrix in higher dimensions, QR decomposition could be one approach. But this is out of scope for our discussion here.

Matrix Determinant

One concept closely related to matrix inverse is matrix determinant. For a matrix $A=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)$, we can draw a parallelogram with vectors $\left(\begin{array}{c}a\\ c\end{array}\right)$ and $\left(\begin{array}{c}b\\ d\end{array}\right)$. The determinant of this matrix is then defined as the area of this parallelogram.

This area is calculated by formula below.

$$\begin{array}{rl}|A|& =(a+b)\ast (c+d)-ac-bd-2bc\\ & =ad-bc\end{array}$$

The symbol for determinant is two vertical bars (|), just like the modulus operator for vectors.

Recall that $ad-bc$ is also the term we used in matrix inverse calculation for a 2 by 2 matrix (equation 1). Therefore our matrix inverse formula can be simplified as

$$A^{-1}=\frac{1}{|A|}\ast \left(\begin{array}{cc}d& -b\\ -c& a\end{array}\right)\text{\hspace{1em}(2)}$$

We can also see that not all matrices are invertible. In order to find the inverse of a matrix, we have to compute its determinant first. However, for a matrix like $\left(\begin{array}{cc}1& 2\\ 1& 2\end{array}\right)$, the determinant is $1\ast 2-2\ast 1=0$. It will result in a division by 0 when we substitute the determinant value to equation (2). This is because for a matrix to have non-zero determinant, the basis vectors of that matrix must be linearly independent. In our example 2 by 2 matrix $A$, $\left(\begin{array}{c}a\\ c\end{array}\right)$ and $\left(\begin{array}{c}b\\ d\end{array}\right)$ must not lie on the same line to have a valid matrix inverse. In the simultaneous equation $A\ast r=s$, there exists an infinite number of solutions for vector $r$ if matrix $A$ has no inverse.

Matrices Changing Basis

Changing Basis in General

We are going to revisit the topic of changing basis here after we have grasped the concept of matrix transformation on vectors.

First, let’s define 2 new basis vectors $b_1$ and $b_2$ where $b_1=\left(\begin{array}{c}3\\ 1\end{array}\right)$ and $b_2=\left(\begin{array}{c}1\\ 1\end{array}\right)$. Recall from our matrix transformation. The new basis vectors $b_1$ and $b_2$ are in fact the transformation of basis vectors $e_1=\left(\begin{array}{c}1\\ 0\end{array}\right)$ and $e_2=\left(\begin{array}{c}0\\ 1\end{array}\right)$ by matrix $\left(\begin{array}{cc}3& 1\\ 1& 1\end{array}\right)$. Note $b_1$ and $b_2$ can also be expressed in the basis of $e_1$ and $e_2$.

$$\begin{gathered} b_1=\left(\begin{array}{cc}3& 1\\ 1& 1\end{array}\right)\ast \left(\begin{array}{c}1\\ 0\end{array}\right)=\left(\begin{array}{c}3\\ 1\end{array}\right)=3e_1+1e_2 \\ {b}_2=\left(\begin{array}{cc}3& 1\\ 1& 1\end{array}\right)\ast \left(\begin{array}{c}0\\ 1\end{array}\right)=\left(\begin{array}{c}1\\ 1\end{array}\right)=1e_1+1e_2 \end{gathered}$$

Now we have a vector $r$ that is defined in $b_1$ and $b_2$ basis as $r=\frac{3}{2}{b}_1+\frac{1}{2}{b}_2$. How do we get the same vector $r$ expressed in $e_1$ and $e_2$ basis? We can substitute in vectors $b_1$ and $b_2$ in $e_1$ and $e_2$ basis.

$$\begin{array}{rl}{r}_E& =\frac{3}{2}{b}_1+\frac{1}{2}{b}_2\\ & =\frac{3}{2}\left(\begin{array}{c}3\\ 1\end{array}\right)+\frac{1}{2}\left(\begin{array}{c}1\\ 1\end{array}\right)\\ & =\left(\begin{array}{c}5\\ 2\end{array}\right)\end{array}$$

Alternatively, we can convert the vector $r=\frac{3}{2}{b}_1+\frac{1}{2}{b}_2$ from $b_1$ and $b_2$ basis to $e_1$ and $e_2$ basis by multiplying the transformation matrix $\left(\begin{array}{cc}3& 1\\ 1& 1\end{array}\right)$.

$$\begin{array}{rl}{r}_E& =\left(\begin{array}{cc}3& 1\\ 1& 1\end{array}\right)\ast \left(\begin{array}{c}\frac{3}{2}\\ \frac{1}{2}\end{array}\right)\\ & =\left(\begin{array}{c}5\\ 2\end{array}\right)\end{array}$$

This is illustrated in graph below. Note the expression for vectors in $e_1$ and $e_2$ basis is colored black while that for vectors in $b_1$ and $b_2$ basis is colored orange.

That is how we convert a vector from $b_1$ and $b_2$ basis to $e_1$ and $e_2$ basis. But what is more interesting is to convert a vector from $e_1$ and $e_2$ basis to $b_1$ and $b_2$ basis. This should somehow “reverse” our previous process.

We first need to find out where $e_1$ and $e_2$ are in $b_1$ and $b_2$ basis. This is where matrix inverse comes into play.

$${\left(\begin{array}{cc}3& 1\\ 1& 1\end{array}\right)}^{-1}=\frac{1}{3-1}\left(\begin{array}{cc}1& -1\\ -1& 3\end{array}\right)=\frac{1}{2}\left(\begin{array}{cc}1& -1\\ -1& 3\end{array}\right)$$

Therefore, $e_1=\frac{1}{2}{b}_1-\frac{1}{2}{b}_2$ and $e_2=-\frac{1}{2}{b}_1+\frac{3}{2}{b}_2$ in $b_1$ and $b_2$ basis. We can verify this by substituting the values of $b_1$ and $b_2$ back to get the original $e_1$ and $e_2$.

$$e_1=\frac{1}{2}{b}_1-\frac{1}{2}{b}_2=\frac{1}{2}\left(\begin{array}{c}3\\ 1\end{array}\right)-\frac{1}{2}\left(\begin{array}{c}1\\ 1\end{array}\right)=(\begin{array}{c}1\\ 0\end{array}$$