mit 18.06 linear algebra video note

//
14:15 2014-8-24 Sunday
start "introduction to linear algebra", video I


2 videos * 17 days == 34 videos


14:16 2014-8-24
row picture, column picture


14:30 2014-8-24
linear combination


14:34 2014-8-24
"the big picture"


-----------------------------------
15:43 2014-8-24
introduction to linear algebra, video II


15:43 2014-8-24
elimination


15:57 2014-8-24
Gaussian elimination


15:57 2014-8-24
pivot


15:57 2014-8-24
forward elimination, backward substitution


15:58 2014-8-24
row exchange


16:10 2014-8-24
augmentd matrix


16:21 2014-8-24
the elimination steps I want to express as matrix


16:29 2014-8-24
elimination matrix


16:48 2014-8-24
particular important:


1. matrix * column vector


2. row vector * matrix


16:49 2014-8-24
E  // elimination matrix


P  // permutation matrix


17:11 2014-8-24
row operation, column operation


17:16 2014-8-24
EA = U  // from A to U


A = LU  // from U to A


--------------------------------------------------
17:35 2014-8-24
introduction to linear algebra, video III


17:36 2014-8-24
matrix multiplication


17:40 2014-8-24
block multiplication


18:09 2014-8-24
invertible(nonsingular)


18:19 2014-8-24
Gauss-Jordan idea


18:58 2014-8-24
upper triangular matrix // U


19:05 2014-8-24
matrix inverse with Gauss-Jordan method


/


///
7:03 2014-8-25 Monday
start introduction to linear algebra, video 4


13:45 2014-8-25
A = LU //  EA = U => A = inv(E) * U


13:45 2014-8-25
PA = LU


14:47 2014-8-25
permutation matrix


14:50 2014-8-25
introduction to linear algebra, video 5


15:06 2014-8-25
vector space


15:06 2014-8-25
subspace


15:06 2014-8-25
symmetric matrix


15:28 2014-8-25
vector space


15:38 2014-8-25
zero vector


15:38 2014-8-25
column vector, column space


15:43 2014-8-25
subspace came from matrix A: C(A)


// column space


16:04 2014-8-25
linear combination of columns => column space


16:06 2014-8-25
how to create a subspace from a matrix? // column space


///
14:43 2014-8-26 Tuesday
finish linear algebra text, chapter 3


------------------------------------------------------
today's goal, video 6, 7


14:44 2014-8-26
introduction to linear algebra, video 6 // chapter 3


vector space, subspace, column space


14:47 2014-8-26
C(A) // column space


N(A) // null space


14:48 2014-8-26
vector space // close for linear combinations


14:51 2014-8-26
column space of a matrix


15:03 2014-8-26
which right-hand side allow me to solve this?


15:20 2014-8-26
N(A)   // nullspace


-----------------------------------------------------
16:05 2014-8-26
start introduction to linear algebra, video 7


find nullspace...


rref // reduced row echelon form


16:05 2014-8-26
pivot column, free column


pivot variable, free variable


16:06 2014-8-26
while I'm doing the elimination, I'm 


not changing the nullspace


16:08 2014-8-26
elimination does change the column space!


16:08 2014-8-26
echelon form // U for rectangular matrix


16:13 2014-8-26
rank of matrix:


number of pivots


16:15 2014-8-26
special solution  // linear combination => nullspace solution


particular solution


16:41 2014-8-26
echelon form(staircase, U) => rref // reduced row echelon form


16:47 2014-8-26
I could do elimination upward // echelon form => rref


16:53 2014-8-26
block matrix


17:16 2014-8-26
N // nullspace matrix


/
13:50 2014-8-27 Wednesday


introduction to linear algebra, video 8, 9


13:51 2014-8-27
[A b] // augmented matrix


13:54 2014-8-27
Xcomplete = Xparticular + Xnullspace


14:28 2014-8-27
X = Xr + Xn 


// Xr rowspace solution


// Xn nullspace solution


14:29 2014-8-27
full column rank


full row rank


14:54 2014-8-27
square matrix, rectangular matrix


---------------------------------------------
15:23 2014-8-27
start introduction to linear algebra, video 9


15:23 2014-8-27
independence, span, basis, dimension


15:26 2014-8-27
there is something in the nullspace of A, 


rather than just the zero vector!


15:28 2014-8-27
basis for vector space


15:56 2014-8-27
dim C(A) = r


dim N(A) = n - r


/
9:52 2014-8-28 Thursday
complete introduction to linear algebra text,


chapter, determinant


9:52 2014-8-28
introduction to linear algebra, video 9


the four fundamental subspace


9:53 2014-8-28
standard basis


9:55 2014-8-28
orthogonal complement


10:11 2014-8-28
dimension of a vector space:


#number of basis


10:13 2014-8-28
we have this great fact to establish


10:17 2014-8-28
elimination, row reduction


10:22 2014-8-28
A is "square & invertible" matrix =>


A is rectangular matrix


10:40 2014-8-28
elementary matrix


10:43 2014-8-28
rref == reduced row echelon form


-----------------------------------------
11:04 2014-8-28
start introduction to linear algebra, video 11


11:04 2014-8-28
matrix space


11:14 2014-8-28
solution space


11:38 2014-8-28
rank one matrix


11:45 2014-8-28
How many steps does it take from anybody to anybody?


12:45 2014-8-28
small world graph  // node, edge


///
8:09 2014-8-29 Friday
potential, potential difference, flow


13:40 2014-8-29
graph // node, edge


13:40 2014-8-29
incidence matrix


13:42 2014-8-29
orthogonal vectors,


orthogonal subspace,


orthogonal basis


14:56 2014-8-29
orthogonal vectors => orthogonal subspaces


15:09 2014-8-29
orthogonal complement


15:20 2014-8-29
row space is orthogonal to nullspace


15:20 2014-8-29
fundamental theorem of linear algebra


15:41 2014-8-29
least square:


Ax = b => A'Ax = A'b // normal equation


---------------------------------------------------
16:07 2014-8-29
start introduction to linear algebra, video 15


projections, projection matrix, least square


16:08 2014-8-29
e == error vector


17:00 2014-8-29
square matrix, 


rectangular matrix


square & invertible matrix


/
18:33 2014/8/30 Saturday
regression, linear regression


18:34 2014/8/30
normal equations




///
8:11 2014/8/31
start introductiont to linear algebra, video 17


orthogonal basis, orthogonal matrix, Gram-schmit


8:12 2014/8/31
orthonormal


8:12 2014/8/31
orthogonal matrix: Q


8:13 2014/8/31
Q is orthogonal matrix =>


Q has orthonormal columns


8:46 2014/8/31
independent vectors => orthogonal vectors => orthonormal vectors


9:01 2014/8/31
projection, orthogonal, error vector


9:04 2014/8/31
A = QR 


// R is upper triangular


// Q is orthogonal matrix


// A is square matrix with independent columns


9:33 2014/8/31
start introduction to linear algebra, video 18


determinant of a square matrix


9:36 2014/8/31
the determinant is a number associated with


every square matrix


9:55 2014/8/31
invertible <=> determinant != 0


singular   <=> determinant == 0


9:57 2014/8/31
the 3 basic properties of determinant:


1. det(I) = 1


2. exchange row => reverse sign


3. linear property for each row(column)


10:10 2014/8/31
elimination does not change the determinant


10:14 2014/8/31
property 7:


det(triangular matrix) == (d1)(d2)...(dn) // product of diagonal


10:30 2014/8/31
determinant of "triangular matrix" is just


the product of diagonal entries


10:36 2014/8/31
property 9:  ???


det AB = detA * detB


10:46 2014/8/31
property 10:


det(A) == det(A') // transpose does not change det


/



8:02 2014/9/1 Monday
introduction to linear algebra, video 19


determinant


8:03 2014/9/1
How to find determinant of square matrix?


1. big formula


2. cofactor


3. pivots


8:03 2014/9/1
row exchange reverse sign


8:41 2014/9/1
Why can we use elimination to get upper triangular matrix,


the use the product of pivots to get det(A)?


because elimination does not change determinant!


--------------------------------------------------------------
9:56 2014/9/1
start introduction to linear algebra, video 20


1. formula for determinant


2. Cramer's rule for x = inv(A) * b


9:57 2014/9/1
cofactor matrix


10:26 2014/9/1
inv(A) =  C' / det(A) // C: cofactor matrix


10:27 2014/9/1
How to find the inv(A)?


1. Gauss-Jordan method


2. formula: inv(A) =  C' / det(A)


10:35 2014/9/1
the validity of the formula:


just check, C'A = det(A) I 


10:35 2014/9/1
why det(A*B) == det(A) * det(B)?


why Cramer's rule?


///
7:24 2014/9/2 Tuesday
start introduction to linear algebra, video 21


eigenvalues


10:57 2014/9/2
det(A - lambda * I) = 0


trace = sum of eigenvalues


10:58 2014/9/2
eigenvalues, eigenvectors


10:58 2014/9/2
Ax = λx // x: eigenvectors, λ:eigenvalue


Ax parallel to x, Ax is some multiple of some x!


11:09 2014/9/2
we look for special vectors!


11:09 2014/9/2
following the eigenvector direction


11:22 2014/9/2
projection matrix


11:27 2014/9/2
sum of the diagonal values == sum of the eigenvalues


a11 + a22 + ... + ann == λ1+ λ2 + λ3 + ... + λn


// trace


11:45 2014/9/2
det(A-λI) = 0 


// characteristic equation, eigenvalue equation



7:56 2014/9/3 Wednesday
introduction linear algebra, video 22


diagonalizing a matrix


powers of A / equation Uk+1 = A*Uk


7:57 2014/9/3
diagonalizable


8:38 2014/9/3
distinct eigenvalues => independent eigenvectors


8:54 2014/9/3
Uk = A * Uk+1 // A: transition matrix


8:59 2014/9/3
powers of A


8:59 2014/9/3
system of differential equations


system of equations


8:59 2014/9/3
to really solve, write U0 as a combination


of eigenvectors!


9:02 2014/9/3
"following the eigenvector direction!"


9:02 2014/9/3
eigenvalues == pole ???


9:33 2014/9/3
dorminant pole


9:33 2014/9/3
we're doing problems that evolving,


we're doing "dynamic", things evolving with time,


eigenvalues are crucial numbers!


9:36 2014/9/3
find the eigenvalue & eigenvector of A,


break U0 as combination of eigenvectors of A


///
7:46 2014-09-04
start linear algebra, video 23


differential equations


7:46 2014-09-04
exp(At) ???


7:47 2014-09-04
eigenvalues <=> poles


8:14 2014-09-04
the whole point of eigenvector is to uncouple


8:38 2014-09-04
by uncoupling it, I mean to diagonalize it


8:40 2014-09-04
it's a system of equations, but they're 


not connected


8:44 2014-09-04
matrix exponential: exp(At)


8:47 2014-09-04
power series  // infinite series, series expansion


8:48 2014-09-04
complex plane


--------------------------------------------
15:52 2014-09-04 Thursday
Harvard statistics video I


15:52 2014-09-04
pattern recognition skills


15:52 2014-09-04
sample space


16:47 2014-09-04
experiment


16:47 2014-09-04
event


16:48 2014-09-04
an event is a subset of a sample space


16:51 2014-09-04
permutation, combination // counting


17:31 2014-09-04
multiplication rule


17:31 2014-09-04
sampling


---------------------------------------------
17:45 2014-09-04
mit multivariable calculus, video I


vector


17:45 2014-09-04
law of cosine


18:14 2014-09-04
detect orthogonality



7:31 2014-09-05 Friday
start linear algebra, video 24


Markov matrix


8:41 2014-09-05
2 properties of Markov matrix:


1. all enteries >= 0


2. all columns add to 1


9:01 2014-09-05
eigenvalues:


1. stability    // λ < 0


2. steady state // λ = 0


3. blow up


9:03 2014-09-05
eigenvalues of A == eigenvalues of A'


9:25 2014-09-05
state transition matrix


9:39 2014-09-05
what can you tell me about the population in k steps?


9:40 2014-09-05
eigenvalue, eigenvector, Markov matrix


9:45 2014-09-05
Fourier series projections


9:53 2014-09-05
projection with orthonormal basis:


q1, q2, ..., qn


9:53 2014-09-05
V = x1 * q1 + x2 * q2 + ... + xn * qn


9:55 2014-09-05
but what is the dot product of functions?



7:14 2014-09-06 Saturday
symmetric matrix, positive definite matrix


7:15 2014-09-06
What is special about symmetric matrix?  // A == A'


1. The eigenvalues are REAL


2. The eigenvectors are ORTHOGONAL  // can be chosen


7:22 2014-09-06
the usual case: A == SΛinv(S)


symmetric case: A = QΛQ'  // Q is orthogonal matrix


7:28 2014-09-06
Q: orthogonal matrix // with all columns are orthonormal basis


7:29 2014-09-06
 A = QΛQ'


// spectrum theorem


// principle axis theorem


7:34 2014-09-06
Why real eigenvalues? // symmetric matrix


15:07 2014-09-06
symmetric matrix is a combination of perpendicular 


projection matrix


15:07 2014-09-06
for symmetric matrix(A == A'), the signs


of pivots same as signs of λ(eigenvalue)'s


15:12 2014-09-06
det(A) == product of pivots == product of eigenvalues


15:15 2014-09-06
What is a positive definite matrix?


they're symmetric matrix with all eigenvalues are real


// all eigenvalues are positive


// all pivots are positive


// all subdeterminant are positive


15:18 2014-09-06
start linear algebra, video 26


complex matrix, DFT, FFT


16:51 2014-09-06
Hermitian matrix


17:22 2014-09-06
for Hermitian matrix:


1. symmetric => Hermitian


2. orthogonal => unitary


17:29 2014-09-06
DFT matrix


17:32 2014-09-06
Fourier matrix


17:42 2014-09-06
matrix factorization


//
7:57 2014-09-07
linear algebra, video 27


positive definite matrix, test for minimum...


7:58 2014-09-07
positive definite: x'Ax > 0




/



10:28 2014-09-08 Monday
introduction to linear algebra, video 28


similar matrix


10:28 2014-09-08
positive definite matrix:


x'Ax > 0 except for x = 0


10:38 2014-09-08
where does positive definite matrix come from?


// least square Ax = b => 


// A'Ax = A'b, normal equation


10:38 2014-09-08
A'A is positive definite, just see


x'A'Ax == (Ax)'(Ax)


10:46 2014-09-08
A'A is square, symmetric, positive definite


10:51 2014-09-08
SVD == Singular Value Decomposition


10:52 2014-09-08
singlular value


10:52 2014-09-08
similar matrix:


A & B are both n by n matrix,


17:46 2014-09-08
similar matrices have the same eigenvalue


17:46 2014-09-08
A = UΣV'  


// A is any rectangular matrix


// Σ is diagonal matrix


// U & V are orthogonal matrix


18:22 2014-09-08
AV = UΣ // U, V orthogonal basis


18:36 2014-09-08
start Harvard Gambler's Ruin and Random variables, video 7


19:25 2014-09-08
random variables & their distribution


19:27 2014-09-08
Grambler's ruin


19:28 2014-09-08
LOTP == Law Of Total Probability


19:42 2014-09-08
PMF == Probability Mass Function


PDF == Probability Density Function


20:30 2014-09-08
SVD for symmetric positive definite matrix:


A = QΛQ'




/


13:31 2014-09-11
linear algebra, video 30,  linear transformation


13:32 2014-09-11
the projection is a linear transformation


13:46 2014-09-11
rotation transformation


13:56 2014-09-11
rotation is also a "linear transformation"


13:57 2014-09-11
linear transformation with coordinates // matrix


14:08 2014-09-11
find the matrix behind it(linear transformation)


14:10 2014-09-11
coordinates come from basis


14:30 2014-09-11
component == coordinate * bisis


14:32 2014-09-11
input basis, output basis


14:40 2014-09-11
basis, coordinate, linear transformation


14:51 2014-09-11
A * input coordinate == output coordinate


// matrix does the job!


14:52 2014-09-1111
A * x = λ * x // eigenvector x is a good coordinate!


15:06 2014-09-11
standard basis => eigenvector basis


15:08 2014-09-11
input space => output space


input basis => output basis


input coordinate => output coordinate


15:12 2014-09-11
the linear transformation which takes a derivative


15:27 2014-09-11
inverse matrix gives the inverse of the linear transformation


----------------------------------------------------------
15:36 2014-09-11
start linear algebra, video 31, the last video!


change of basis


15:38 2014-09-11
linear transformation <=> matrix


16:02 2014-09-11
image compression


16:02 2014-09-11
lossless compression, lossy compression


16:07 2014-09-11
JPEG // change of basis, standard basis => Fourier basis


16:13 2014-09-11
standard basis => better basis


16:13 2014-09-11
what basis to choose?  // image compression


16:15 2014-09-11
JPEG choose "Fourier basis"


16:16 2014-09-11
you have to use prediction & correction


16:38 2014-09-11
wavelet basis


16:40 2014-09-11
Fourier basis => wavelet basis


16:41 2014-09-11
standar basis => wavelet basis


16:47 2014-09-11
a good basis has a nice fast inverse!


16:48 2014-09-11
p = Wc  =>


c = inv(W) * P


16:49 2014-09-11
FFT == Fast Fourier Transform


16:49 2014-09-11
the nice property of orthogonal matrix:


inv(Q) = Q' // inverse is just transpose


16:53 2014-09-11
JPEG     // Fourier basis


JPEG2000 // wavelet basis


17:02 2014-09-11
change of basis


17:05 2014-09-11
I have my vector in one basis, and I want to 


change it to another one.


17:06 2014-09-11
eigenvector basis


17:28 2014-09-11
following the eigenvector direction!


---------------------------------------------------------------