Linear Algebra Review

Linear Algebra Review

Matrices and Vectors

Matrix: Rectangular array of numbers

Dimension of matrix: number of rows x number of columns

Refer to matrix of specific dimension: ${\mathbb{R}}^{2\times 3}$

Refer to specific matrix elements of the matrix: $A_{ij}$("$i,j$ entry" in the $i^{th}$ row, $j^{th}$ column)

Vector: An n$\times$ 1 matrix

4-dimensional vector :${\mathbb{R}}^4$

Refer elements in vector: $y_i$=$i^{th} $ element

1-indexed vs 0-indexed

By conventional, use capital letters (such as A,B,C,X ) to refer matrices, and use lower cases (like a,d,c,x,y) to refer to either numbers or just raw numbers or scalars or vectors.

Addition and Scalar

Matrix addition and subtraction

Simply add or subtract each corresponding elements

The dimensions of two matrices to add or subtract must be the same

Scalar multiplication and division

Simply multiply or divide every element by the scalar value

Matrix Vector Multiplication

Details:

To get $y_i$, multiply $A^{\prime }s$ $i^{th}$ row with elements of vector $x$, and add them up.

The result is a vector. The number of columns of the matrix must equal the number of rows of the vector.

An m x n matrix multiplied by an n x 1 vector results in an m x 1 vector.

prediction=DataMatrix $\ast$ parameters

Matrix Matrix Multiplication

Details:

The $i^{th}$ column of the matrix $C$ is obtained by multiplying $A$ with the $i^{th}$ column of $B$.(for $i$ =1,2…o)

efficiently make prediction with lots of hypotheses

• To multiply two matrices, the number of columns of the first matrix must equal the number of rows of the second matrix.

Matrix Multiplication

Properties

• not commutative

  Let $A,B$ be matrices, then in general, $A\times B\ne B\times A$.(reversing the order of the matrices can change the dimension of outcome)

• enjoy associative property

  $A\times B\times C$

  Let $D=B\times C$. Compute $A\times D$

  Let $E=A\times B$. Compute $E\times C$.

  all give the same answer.

Identity Matrix

Denoted $Ior{I}_{n\times n}$

• Has 1’s on the diagonal (upper left to lower right diagonal) and 0’s elsewhere.

When multiplied by any matrix of the same dimensions, results in the original matrix.
$$A\times I=I\times A=A$$
(The $I^{\prime }s$ in the function above have different dimension)

The dimension of $I$ is implicit from the context.

Inverse and Transpose

Matrix Inverse

If $A$ is an m x m matrix (square matrix), and if it has an inverse,
$$A{A}^{-1}=A^{-1}A=I$$
Matrices that don’t have an inverse is singular or degenerate.

Matrix Transpose

Let $A$ be an m x n matrix, and let $B=A^T$. Then $B$ is an n x m matrix, and
$$B_{ij}=A_{ij}$$

Transpose**

Let $A$ be an m x n matrix, and let $B=A^T$. Then $B$ is an n x m matrix, and
$$B_{ij}=A_{ij}$$