vedio 9
Suppose A m by n while m < n, then there are nonzero solutions to AX = 0.
Reason: There will be free variables. (rank < n)
When
v1,…,vn
are columns of A, they are independent if nullspace of A is only {zero vector}. (rank = n)
They are dependent if AC = 0 for some nonzero C.
Vectors
v1,...,vl
span a space means: The space consists all combinations of those vectors.
Basis for a space is a sequence of vectors
v1,...,vd
with 2 properties
1 They are independent
2 They span the space
Every basis for the space has the same number of vectors.
rank(A) = number of pivot columns = dimention of C(A)
dim N(A) = number of free variables = n - r
vedio 10
4 foundermental sub-spaces:
columnspace: C(A) in
Rm
, to be exactly, it is r, and r < m
nullspace: N(A) in
Rn
, it is (n - r)
rowspace = all colums of row = all columns of
AT
:
C(AT)
in
Rn
nullspace of
AT
:
N(AT)
in
Rm
= left null space of A
left null space 名字由来:
ATy=0⇒(ATy)T=0⇒yTA=0
Ex:
A = [1, 1, 1, 1]
dim rowspace of A = 1
dim N(A) = n - r = 3
dim C(A) = 1
dim N(
AT
) = 0
C(A) | N(A) |
---|---|
pivot columns | special columns |
r | n - r |
行变换以后 same row space, different column space
vedio 11
矩阵空间
R3X3
是9维,basis有9个,分别如下:
⎡⎣⎢100000000⎤⎦⎥
⎡⎣⎢000100000⎤⎦⎥
⎡⎣⎢000000100⎤⎦⎥
……
⎡⎣⎢000000001⎤⎦⎥
S是三维对称矩阵,U是上三角矩阵(似乎基的数量等于维数)
dim S = 6, dim U = 6
dim(S
∩
U) = 3
dim(S + U) = 9 (S + U can span
R3
)
Graph = {nodes, edges}
vedio 14
number | name | dim |
---|---|---|
1 | row space | r |
2 | null space | n - r |
3 | column space | r |
4 | null space of AT | m - r |
1 is orthogonal to 2, 3 is orthogonal to 4
if x is orthogonal to y,
xTy=0
, then
xTx+yTy=(x+y)T(x+y)
Proof:
xTx+yTy=xTx+xTy+yTx+yTy
⇔0=xTy+yTx=2xTy
⇔0=xTy
Subspace S is orthogonal to Subspace T means: every vector in S is orthogonal to every vector in T.
Nullspace(n - r) and rowspace(r) are orthogonal Complements(补) in Rn.
Nullspace contains all vectors ⊥ rowspace.
Ax = b(m > n), when there is no solution: ATAx^=ATb
N(ATA)=N(A)
rank of ATA=rank of A
ATA is invertible exactly if A has independent columns.
vedio 15
有关投影,b在a上的投影为p,p = ax,e = b - p,
a 和e正交,因此
aTe=0
⇔aT(b−p)=0
⇔aT(b−ax)=0
⇔aTax=aTb
⇔x=aTbaTa
p=ax=aaTbaTa=aaTaTab
project matric(投影矩阵为) P=aaTaTa
PT=P
从上式就可以看出来
P2=P
投影两次和一次的结果是一样的
为什么要投影,why project?
Because Ax = b may have no solution, then solve
Ax^=P
instead
二维情况下:
P=Ax^, find x^
key:
e=b−Ax^
is perpendicular to plane
aT1(b−Ax^)=0
aT2(b−Ax^)=0
[aT1aT2](b−Ax^)=[00]
⇔AT(b−Ax^)=0
⇔ATAx^=ATb
⇔x^=ATbATA
p=Ax^=A(ATA)−1ATb
投影矩阵
P=A(ATA)−1AT
PT=P, P2=P
vedio 16
P=A(ATA)−1AT
If b in column space
Pb=b
If b
⊥
column space
Pb=0
数学上的解释:
Column space 与 Nullspace of
AT
正交,
如果b in column space,
b=Ax,
Pb=A(ATA)−1ATb=A(ATA)−1ATAx=Ax=b
如果b in Null space of AT,ATb=0⇒Pb=0
对于任意向量b,在C(A)上的投影为p,在N(A)上的投影为e, p + e = b
其中
p=Pb,e=(I−P)b
也很好理解,
e=b−p=b−Pb=(I−P)b
线性规划的本质就是 Minimize
||Ax−b||2=e2
当我们没办法求出Ax = b时,只能求
Ax^=p
, p是b在C(A)上的投影
Ax^=p=Pb=A(ATA)−1ATb
⇔ATAx^=ATb
e=b−p=b−Ax^
If A has independent Columns space, then ATA is invertible.