《Deep Learning》花书学习笔记2 - Linear Algebra

Linear Algebra

1 Basic Conception

1. scalar: a single number
2. vector: an array of numbers / arranged in order
3. matrices: 2-D array of numbers
4. tensors: an array with more than two axes
5. transpose operation
6. add operation: the same shape matrices
7. broadcasting: implicit copying of vector to many locations

2 Multiplying

1. multiply matrix A and B = C
   shape of A: m x n
   shape of B: n x p
   shape of C: m x p
   
2. multiply matrix A and vector b
   Ax = b
   

3 Identity Matrix

all of the entries along the main diagonal is 1
all of the other entries are 0

4 Inverse Matrix

Matrix inversion is not defined for matrices that are not square

5 Linear Dependence and Span

the span of a set of vectors is the set of all points obtainable by linear combination of the original vectors

6 square and singular

square: m = n
singular: a square matrix with linearly depentdent columns

7 Norms

use norms to measure the size of a vector

norms: functions mapping vectors to non-negative values
More rigorously, a norm is any function f that satisfies the following properties:

L1 norm:
L max norm:

Frobenius norm: measure the size of a matrix

8 diagonal / symmetric / orthogonal matrix

diagonal matrix: non-zero entries only along the main diagonal
symmetric matrix: matrix equal to its own transpose
orthogonal matrix: rows are mutually orthonormal, columns are mutually orthonormal

9 Eigen-decomposition （特征分解）

eigenvalues
eigenvectors

The scalar λ is known as the eigenvalue corresponding to this eigenvector

eigen-decomposition of A:

positive definite: all eigenvalues are positive
positive semidefinite: all eigenvalues are positive or zero-valued
negative definite: all eigenvalues are negative
negative semidefinite: all eigenvalues are negative or zero-valued

10 singular value decomposition

shape of
A: m x n
U: m x m
a orthogonal matrix
columns of U are known as the left-singular vectors
D: m x n
a diagonal matrix, the elements along the diagonal are the singular values of A
V: n x n
also a orthogonal matrix
columns of V are known as the right-singular vectors

the left-singular vectors: the eigenvectors of
the right-singular vectors: the eigenvectors of

11 The Moore-Penrose Pseudoinverse（Moore-Penrose伪逆）

pseudoinverse is defined as a matrix:

pracrical algorithms for computing the pseudoinverse:

pseudoinverse of D:

（对角矩阵D对角线上，所有非零元素取倒数，再转置得到）

12 Trace

trace: the sum of all of the daigonal entries of a matrix
trace operation properties：

13 Determinant

The determinant of a square matrix, denoted det (A ): a function mapping matrices to real scalars