Linear Algebra -- Intro to vector

Linear combinations pictures

For vector u, v, w

Sometimes, 3 vectors fill a plane instead of a space, criteria as follows --

For vector u, v, w, if you can find c, d so that w = cu + dv, then w lies on the plane of cu + dv, and u+v+w also forms the plane of cu + dv

Solving linear equation

Elementary Row Operations

Example

Linear Equation Solution Pattern

Possible consequences

Non-unique solutions

Homogeneous systems

• 等号后都是0.
• Consistent -- 永远有解（x1=x2=x3=...=xn=0) -- trivial solution

**如果m>=n, 矩阵有unique solution (trivial) or there are infinitely many nontrivial solutions

Matrix Operation Properties

• 加减法满足交换律，结合律        
•  乘除法满足交换律，结合律      
  • 注意第2条： r，s的顺序不可以调换
• 混合运算

 Transpose

Rows become columns, columns become rows, denoted as 

Symmetric Matrix

 

 OR

symmetric across the main diagonal

Example

Inner & Outer Product

Dot Product

Scalar Projection 

• 矢量S在矢量r上的投影长度 

        

Vector Projection 

• 矢量S在矢量r上的投影矢量
  • i.e. scalar projection * unit vector in r's direction

        

Changing Basis

A vector describing our data isn't tied to the axes that we originally used to describe it at all. We can change the numbers in the vector by taking the dots or projection product with the new basis, in the case where the new basis vectors are orthogonal to each other.

Linearly Independent

For a vector b3​ to be linearly independent to the vectors b1​ and b2​ --

1. 
2. 

The vectors b1, b2 and b3 forms a 3-D space

Matrix Transformation

Scale

• Definition: Stretch or shrink the space by a factor.
• The factor is known as Determinant -- describe how the space scale

Example: 2D scale

 --> 

• The space formed by e1 and e2 are stretched by a factor of ad
• ad is the determinant

Sheer

• Definition: Pushing or pulling the space by an amount.
• When combined with a scale, the sheering amount is NOT associated with the Determinant

Example: Just sheering y-axis 

 --> 

• The x-axis remains unchanged, i.e. the left side of the matrix is an identity matrix
• The y-axis is being pushed to the right, i.e. new location is now (1,1)
• Both axis can be sheered at the same time

Example: sheering & scaling y-axis 

 --> 

• The x-axis is scaled by a factor of a, i.e. the left side of the matrix is (a,0)
• The y-axis is being pushed to the right by b, and scaled to be d above the x-axis, i.e. new location is now (b,d)
• Because the new space has an area of ad, the determinant is not related to b at all.

Example: general sheering & scaling 

 --> 

• Both axis are scaled and sheered
• The y-axis is now (b,d)
• The x-axis is now (a,c)
• The new space has an area of ad-bc, which is the determinant.

0 determinant

• In a 2-d matrix, the determinant is 0 when 2 column vectors are transformed to the same line (collapsed to 1-d).
• In higher dimensions, the determinant is 0 when the some vectors are on the same plain as the others, which also results in a collapsed dimension (that collapsed dimension has no useful information)

Unsolvable: the matrix is unsolvable because the determinant is 0, and when the determinant = 0, we cannot find the inverse.

Threfore, it is important to make sure the base vectors are linearly independent.