CS229-LinearAlgebra
向量-向量乘法
给定两个向量
x
,
y
∈
R
n
x, y \in \mathbb{R}^{n}
x,y∈Rn,
x
T
y
x^T y
xTy通常称为向量内积或者点积,结果是个实数。
x
T
y
∈
R
=
[
x
1
x
2
⋯
x
n
]
[
y
1
y
2
⋮
y
n
]
=
∑
i
=
1
n
x
i
y
i
x^{T} y \in \mathbb{R}=\left[\begin{array}{llll}{x_{1}} & {x_{2}} & {\cdots} & {x_{n}}\end{array}\right]\left[\begin{array}{c}{y_{1}} \\ {y_{2}} \\ {\vdots} \\ {y_{n}}\end{array}\right]=\sum_{i=1}^{n} x_{i} y_{i}
xTy∈R=[x1x2⋯xn]⎣⎢⎢⎢⎡y1y2⋮yn⎦⎥⎥⎥⎤=i=1∑nxiyi
注意:
x
T
y
=
y
T
x
x^T y = y^Tx
xTy=yTx 始终成立。
给定向量 x ∈ R m x \in \mathbb{R}^{m} x∈Rm, y ∈ R n y \in \mathbb{R}^{n} y∈Rn (他们的维度是否相同都没关系), x y T ∈ R m × n xy^T \in \mathbb{R}^{m \times n} xyT∈Rm×n叫做**向量外积 ** , 当 ( x y T ) i j = x i y j (xy^T)_{ij} = x_iy_j (xyT)ij=xiyj 的时候,它是一个矩阵。
x
y
T
∈
R
m
×
n
=
[
x
1
x
2
⋮
x
m
]
[
y
1
y
2
⋯
y
n
]
=
[
x
1
y
1
x
1
y
2
⋯
x
1
y
n
x
2
y
1
x
2
y
2
⋯
x
2
y
n
⋮
⋮
⋱
⋮
x
m
y
1
x
m
y
2
⋯
x
m
y
n
]
x y^{T} \in \mathbb{R}^{m \times n}=\left[\begin{array}{c}{x_{1}} \\ {x_{2}} \\ {\vdots} \\ {x_{m}}\end{array}\right]\left[\begin{array}{llll}{y_{1}} & {y_{2}} & {\cdots} & {y_{n}}\end{array}\right]=\left[\begin{array}{cccc}{x_{1} y_{1}} & {x_{1} y_{2}} & {\cdots} & {x_{1} y_{n}} \\ {x_{2} y_{1}} & {x_{2} y_{2}} & {\cdots} & {x_{2} y_{n}} \\ {\vdots} & {\vdots} & {\ddots} & {\vdots} \\ {x_{m} y_{1}} & {x_{m} y_{2}} & {\cdots} & {x_{m} y_{n}}\end{array}\right]
xyT∈Rm×n=⎣⎢⎢⎢⎡x1x2⋮xm⎦⎥⎥⎥⎤[y1y2⋯yn]=⎣⎢⎢⎢⎡x1y1x2y1⋮xmy1x1y2x2y2⋮xmy2⋯⋯⋱⋯x1ynx2yn⋮xmyn⎦⎥⎥⎥⎤
举一个外积如何使用的一个例子:让
1
∈
R
n
1\in R^{n}
1∈Rn表示一个
n
n
n维向量,其元素都等于1,此外,考虑矩阵
A
∈
R
m
×
n
A \in R^{m \times n}
A∈Rm×n,其列全部等于某个向量
x
∈
R
m
x \in R^{m}
x∈Rm。 我们可以使用外积紧凑地表示矩阵
A
A
A:
A = [ ∣ ∣ ∣ x x ⋯ x ∣ ∣ ∣ ] = [ x 1 x 1 ⋯ x 1 x 2 x 2 ⋯ x 2 ⋮ ⋮ ⋱ ⋮ x m x m ⋯ x m ] = [ x 1 x 2 ⋮ x m ] [ 1 1 ⋯ 1 ] = x 1 T A=\left[\begin{array}{llll}{ |} & { |} & {} & { |} \\ {x} & {x} & {\cdots} & {x} \\ { |} & { |} & {} & { |}\end{array}\right] =\left[\begin{array}{cccc}{x_{1}} & {x_{1}} & {\cdots} & {x_{1}} \\ {x_{2}} & {x_{2}} & {\cdots} & {x_{2}} \\ {\vdots} & {\vdots} & {\ddots} & {\vdots} \\ {x_{m}} & {x_{m}} & {\cdots} & {x_{m}}\end{array}\right]=\left[\begin{array}{c}{x_{1}} \\ {x_{2}} \\ {\vdots} \\ {x_{m}}\end{array}\right]\left[\begin{array}{lll}{1} & {1} & {\cdots} & {1}\end{array}\right]=x \mathbf{1}^{T} A=⎣⎡∣x∣∣x∣⋯∣x∣⎦⎤=⎣⎢⎢⎢⎡x1x2⋮xmx1x2⋮xm⋯⋯⋱⋯x1x2⋮xm⎦⎥⎥⎥⎤=⎣⎢⎢⎢⎡x1x2⋮xm⎦⎥⎥⎥⎤[11⋯1]=x1T
矩阵-向量乘法
给定矩阵 A ∈ R m × n A \in \mathbb{R}^{m \times n} A∈Rm×n,向量 x ∈ R n x \in \mathbb{R}^{n} x∈Rn , 它们的积是一个向量 y = A x ∈ R m y = Ax \in R^{m} y=Ax∈Rm。 有几种方法可以查看矩阵向量乘法,我们将依次查看它们中的每一种。
如果我们按行写
A
A
A,那么我们可以表示
A
x
Ax
Ax为:
y
=
A
x
=
[
−
a
1
T
−
−
a
2
T
−
⋮
−
a
m
T
−
]
x
=
[
a
1
T
x
a
2
T
x
⋮
a
m
T
x
]
y=A x=\left[\begin{array}{ccc}{-} & {a_{1}^{T}} & {-} \\ {-} & {a_{2}^{T}} & {-} \\ {} & {\vdots} & {} \\ {-} & {a_{m}^{T}} & {-}\end{array}\right] x=\left[\begin{array}{c}{a_{1}^{T} x} \\ {a_{2}^{T} x} \\ {\vdots} \\ {a_{m}^{T} x}\end{array}\right]
y=Ax=⎣⎢⎢⎢⎡−−−a1Ta2T⋮amT−−−⎦⎥⎥⎥⎤x=⎣⎢⎢⎢⎡a1Txa2Tx⋮amTx⎦⎥⎥⎥⎤
换句话说,第
i
i
i个
y
y
y是
A
A
A的第
i
i
i行和
x
x
x的内积,即:
y
i
=
y
i
=
a
i
T
x
y_i = y_{i}=a_{i}^{T} x
yi=yi=aiTx。
同样的, 可以把
A
A
A 写成列的方式,则公式如下:
y
=
A
x
=
[
∣
∣
∣
a
1
a
2
⋯
a
n
∣
∣
∣
]
[
x
1
x
2
⋮
x
n
]
=
a
1
x
1
+
a
2
x
2
+
⋯
+
a
n
x
n
y=A x=\left[\begin{array}{cccc} | & | & & | \\ a^{1} & a^{2} & {\cdots} & a^{n} \\ | & | & & { |}\end{array}\right]\left[\begin{array}{c}{x_{1}} \\ {x_{2}} \\ {\vdots} \\ {x_{n}}\end{array}\right]= a^1 x_{1}+a^2x_{2}+{\cdots} +a^nx_{n}
y=Ax=⎣⎡∣a1∣∣a2∣⋯∣an∣⎦⎤⎣⎢⎢⎢⎡x1x2⋮xn⎦⎥⎥⎥⎤=a1x1+a2x2+⋯+anxn
换句话说, y y y是 A A A的列的线性组合,其中线性组合的系数由 x x x的元素给出。
到目前为止,我们一直在右侧乘以列向量,但也可以在左侧乘以行向量。
y
T
=
x
T
A
y^T = x^TA
yT=xTA 表示
A
∈
R
m
×
n
A \in \mathbb{R}^{m \times n}
A∈Rm×n,
x
∈
R
m
x \in \mathbb{R}^{m}
x∈Rm,
y
∈
R
n
y \in \mathbb{R}^{n}
y∈Rn。 和以前一样,我们可以用两种可行的方式表达
y
T
y^T
yT,这取决于我们是否根据行或列表达
A
A
A.
第一种情况,我们把 A A A用列表示:
y T = x T A = x T [ ∣ ∣ ∣ a 1 a 2 ⋯ a n ∣ ∣ ∣ ] = [ x T a 1 x T a 2 … x T a n ] y^{T}=x^{T} A=x^{T}\left[\begin{array}{cccc}{ |} & { |} & {} & { |} \\ {a^{1}} & {a^{2}} & {\cdots} & {a^{n}} \\ { |} & { |} & {} & { |}\end{array}\right]=\left[\begin{array}{cccc}{x^{T} a^{1}} & {x^{T} a^{2}} & {\dots} & {x^{T} a^{n}}\end{array}\right] yT=xTA=xT⎣⎡∣a1∣∣a2∣⋯∣an∣⎦⎤=[xTa1xTa2…xTan]
这表明 y T y^T yT的第 i i i个元素等于 x x x和 A A A的第 i i i列的内积。
最后,根据行表示 A A A,我们得到了向量-矩阵乘积的最终表示:
y
T
=
x
T
A
=
[
x
1
x
2
⋯
x
n
]
[
−
a
1
T
−
−
a
2
T
−
⋮
−
a
m
T
−
]
=
x
1
[
−
a
1
T
−
]
+
x
2
[
−
a
2
T
−
]
+
…
+
x
n
[
−
a
n
T
−
]
y^T=x^TA =\left[\begin{array}{llll}{x_{1}} & {x_{2}} & {\cdots} & {x_{n}}\end{array}\right]\left[\begin{array}{c}{-a_{1}^{T}-} \\ {-a_{2}^{T}-} \\ {\vdots} \\ {-a_{m}^{T}-}\end{array}\right] =x_{1}\left[-a_{1}^{T}-\right]+x_{2}\left[-a_{2}^{T}-\right]+\ldots+x_{n}\left[-a_{n}^{T}-\right]
yT=xTA=[x1x2⋯xn]⎣⎢⎢⎢⎡−a1T−−a2T−⋮−amT−⎦⎥⎥⎥⎤=x1[−a1T−]+x2[−a2T−]+…+xn[−anT−]
所以我们看到
y
T
y^T
yT是
A
A
A的行的线性组合,其中线性组合的系数由
x
x
x的元素给出。
矩阵-矩阵乘法
A
m
×
n
B
n
×
m
=
[
∑
i
=
1
n
a
1
i
b
i
1
∑
i
=
1
n
a
1
i
b
i
2
⋯
∑
i
=
1
n
a
1
i
b
i
m
∑
i
=
1
n
a
2
i
b
i
1
∑
i
=
1
n
a
2
i
b
i
2
⋯
∑
i
=
1
n
a
2
i
b
i
m
⋮
⋮
⋱
⋮
∑
i
=
1
n
a
m
i
b
i
1
∑
i
=
1
n
a
m
i
b
i
2
⋯
a
m
i
b
i
m
]
=
[
a
1
T
b
1
a
1
T
b
2
⋯
a
1
T
b
m
a
2
T
b
1
a
2
T
b
2
⋯
a
2
T
b
m
⋮
⋮
⋱
⋮
a
m
T
b
1
a
m
T
b
2
⋯
a
m
T
b
m
]
=
∑
i
=
1
n
[
a
1
i
b
i
1
a
1
i
b
i
2
⋯
a
1
i
b
i
m
a
2
i
b
i
1
a
2
i
b
i
2
⋯
a
2
i
b
i
m
⋮
⋮
⋱
⋮
a
m
i
b
i
1
a
m
i
b
i
2
⋯
a
m
i
b
i
m
]
=
∑
i
=
1
n
a
i
b
i
T
\begin{aligned} A_{m\times n} B_{n\times m}&= \left[\begin{array}{cccc} \sum_{i=1}^n a_{1i}b_{i1} & \sum_{i=1}^n a_{1i}b_{i2} & \cdots & \sum_{i=1}^n a_{1i}b_{im} \\ \sum_{i=1}^n a_{2i}b_{i1} & \sum_{i=1}^n a_{2i}b_{i2} & \cdots & \sum_{i=1}^n a_{2i}b_{im} \\ \vdots & \vdots & \ddots & \vdots \\ \sum_{i=1}^n a_{mi} b_{i1} & \sum_{i=1}^n a_{mi} b_{i2} & \cdots & a_{mi} b_{im} \end{array}\right] =\left[\begin{array}{cccc} a_{1}^Tb_{1} & a_{1}^Tb_{2} & \cdots & a_{1}^Tb_{m} \\ a_{2}^Tb_{1} & a_{2}^Tb_{2} & \cdots & a_{2}^Tb_{m} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m}^Tb_{1} & a_{m}^Tb_{2} & \cdots & a_{m}^Tb_{m} \end{array}\right]\\ &=\sum_{i=1}^n\left[\begin{array}{cccc} a_{1i}b_{i1} & a_{1i}b_{i2} & \cdots & a_{1i}b_{im} \\ a_{2i}b_{i1} & a_{2i}b_{i2} & \cdots & a_{2i}b_{im} \\ \vdots & \vdots & \ddots & \vdots \\ a_{mi}b_{i1} & a_{mi}b_{i2} & \cdots & a_{mi}b_{im} \\ \end{array}\right] =\sum_{i=1}^na_ib_i^T \end{aligned}
Am×nBn×m=⎣⎢⎢⎢⎡∑i=1na1ibi1∑i=1na2ibi1⋮∑i=1namibi1∑i=1na1ibi2∑i=1na2ibi2⋮∑i=1namibi2⋯⋯⋱⋯∑i=1na1ibim∑i=1na2ibim⋮amibim⎦⎥⎥⎥⎤=⎣⎢⎢⎢⎡a1Tb1a2Tb1⋮amTb1a1Tb2a2Tb2⋮amTb2⋯⋯⋱⋯a1Tbma2Tbm⋮amTbm⎦⎥⎥⎥⎤=i=1∑n⎣⎢⎢⎢⎡a1ibi1a2ibi1⋮amibi1a1ibi2a2ibi2⋮amibi2⋯⋯⋱⋯a1ibima2ibim⋮amibim⎦⎥⎥⎥⎤=i=1∑naibiT
将A矩阵分块为列列向量,B矩阵分块为行向量
C
=
A
B
=
[
∣
∣
∣
a
1
a
2
⋯
a
n
∣
∣
∣
]
[
−
b
1
T
−
−
b
2
T
−
⋮
−
b
n
T
−
]
=
∑
i
=
1
n
a
i
b
i
T
C=A B=\left[\begin{array}{cccc}{ |} & { |} & {} & { |} \\ {a_{1}} & {a_{2}} & {\cdots} & {a_{n}} \\ { |} & { |} & {} & { |}\end{array}\right]\left[\begin{array}{c}{-}& {b_{1}^{T}}&{-} \\ {-}& {b_{2}^{T}}&{-} \\ {\vdots} \\{-}& {b_{n}^{T}}&{-}\end{array}\right]=\sum_{i=1}^{n} a_{i} b_{i}^{T}
C=AB=⎣⎡∣a1∣∣a2∣⋯∣an∣⎦⎤⎣⎢⎢⎢⎡−−⋮−b1Tb2TbnT−−−⎦⎥⎥⎥⎤=i=1∑naibiT
将A矩阵分块为行向量
C
=
A
B
=
[
−
a
1
T
−
−
a
2
T
−
⋮
−
a
m
T
−
]
B
=
[
−
a
1
T
B
−
−
a
2
T
B
−
⋮
−
a
m
T
B
−
]
C=A B=\left[\begin{array}{ccc}{-} & {a_{1}^{T}} & {-} \\ {-} & {a_{2}^{T}} & {-} \\ {} & {\vdots} & {} \\ {-} & {a_{m}^{T}} & {-}\end{array}\right] B=\left[\begin{array}{c} {-} & {a_{1}^{T} B} & {-}\\ {-} & {a_{2}^{T} B} & {-} \\ {\vdots} \\ {-} & {a_{m}^{T} B}& {-}\end{array}\right]
C=AB=⎣⎢⎢⎢⎡−−−a1Ta2T⋮amT−−−⎦⎥⎥⎥⎤B=⎣⎢⎢⎢⎡−−⋮−a1TBa2TBamTB−−−⎦⎥⎥⎥⎤