Kronecker Products and Stack Operator

Whitcomb L. Notes on Kronecker Products.

定义

Stack Operator

对于任意的矩阵$A\in {\mathbb{R}}^{m\times n}$,
$$vec(A):=[A_{00},A_{10},\dots ,A_{m-1,n-1}{]}^T\in {\mathbb{R}}^{mn},$$
即按列展开.

Kronecker Product

对于任意的矩阵$A\in {\mathbb{R}}^{m\times n},B\in {\mathbb{R}}^{p\times q}$,
$$A\otimes B:=\left[\begin{array}{ccc}{A}_{00}\cdot B& \cdots & A_{0n-1}\cdot B\\ ⋮& \ddots & ⋮\\ A_{m-1,0}\cdot B& \cdots & A_{m-1,n-1}\cdot B\end{array}\right]\in {\mathbb{R}}^{mp\times nq}$$

性质

Stack Operator

$$\mathrm{T}\mathrm{r}(A^{T}B)=vec(A{)}^{T}vec(B).$$

Kronecker Product

易知,
$$[A\otimes B{]}_{ip+s,jq+t}=A_{i,j}\cdot B_{s,t},i\in [m],s\in [p],j\in [n],t\in [q],$$
这里 [ m ] = { 0 , 1 , … , m − 1 } [m] = \{0, 1, \ldots, m-1\} [m]={0,1,…,m−1}.

$a\otimes b=vec(b{a}^T)$

• $a\in {\mathbb{R}}^m,b\in {\mathbb{R}}^n$, 则
  $$a\otimes b=vec(b{a}^T)$$

$(A\otimes B{)}^T=(A^T\otimes B^T)$

$(A\otimes B{)}^T=(A^T\otimes B^T)$是显然的.

故
$$\begin{gathered} [A\otimes B{]}_{\ast ,jq+t}=A_{\ast ,j}\otimes B_{\ast ,t}=vec(B_{\ast ,t}{A}_{\ast ,j}^T) \\ [A\otimes B{]}_{ip+s,\ast }^T=A_{i,\ast }^T\otimes B_{s,\ast }^T=vec(B_{s,\ast }^T{A}_{i,\ast }). \end{gathered}$$

半线性

• $A\otimes \alpha B=\alpha A\otimes B=\alpha (A\otimes B).$

• $$\begin{gathered} (A+B)\otimes C=A\otimes C+B\otimes C \\ A\otimes (B+C)=A\otimes B+A\otimes C. \end{gathered}$$

• $(A\otimes B)\otimes C=A\otimes (B\otimes C)$:
  $$\begin{array}{ll}(A\otimes B)\otimes C& =[A_{i,j}\cdot B_{s,t}\cdot C]\\ & =A\otimes (B\otimes C).\end{array}$$

• 通常 $(A\otimes B)\ne (B\otimes A)$.

$(A\otimes B)(C\otimes D)=(AC\otimes BD)$

$$\begin{array}{ll}[(A\otimes B)(C\otimes D){]}_{ip+s,jq+t}& =[A\otimes B{]}_{ip+s,\ast }[C\otimes D{]}_{\ast ,jq+t}\\ & =vec(B_{s,\ast }^T{A}_{i,\ast }{)}^{T}vec(D_{\ast ,t}{C}_{\ast ,j}^T)\\ & =\mathrm{T}\mathrm{r}(A_{i,\ast }^T{B}_{s,\ast }{D}_{\ast ,t}{C}_{\ast ,j}^T)\\ & =\mathrm{T}\mathrm{r}(C_{\ast ,j}^T{A}_{i,\ast }^T{B}_{s,\ast }{D}_{\ast ,t})\\ & =A_{i,\ast }{C}_{\ast ,j}\cdot B_{s,\ast }{D}_{\ast ,t}\\ & =[AC{]}_{ij}\cdot [BD{]}_{st}\\ & =[AC\otimes BD{]}_{ip+s,jq+t}.\end{array}$$

$(A\otimes B{)}^{-1}=(A^{-1}\otimes B^{-1})$

条件自然是A, B为满秩方阵:
$$(A\otimes B)(A^{-1}\otimes B^{-1})=(A{A}^{-1}\otimes B{B}^{-1})=I$$

$\mathrm{d}\mathrm{e}\mathrm{t}(A_{n\times n}\otimes B_{m\times m})=\mathrm{d}\mathrm{e}\mathrm{t}(A{)}^m\cdot \mathrm{d}\mathrm{e}\mathrm{t}(B{)}^n$

就像用普通的高斯消去法将矩阵化为对角型一样, 在对$A_{n\times n}\otimes B_{m\times m}$消去的过程中可以发现, $B$不会产生丝毫的影响, 结果便是显而易见的了.

$\mathrm{T}\mathrm{r}(A\otimes B)=\mathrm{T}\mathrm{r}(A)\cdot \mathrm{T}\mathrm{r}(B)$

$$\mathrm{T}\mathrm{r}(A\otimes B)=\sum _{i=1}^m\sum _{j=1}^n{A}_i{B}_j=\mathrm{T}\mathrm{r}(A)\cdot \mathrm{T}\mathrm{r}(B).$$

$vec(ABC)=(C^T\otimes A)vec(B)$

设$A\in {\mathbb{R}}^{m\times n},B\in {\mathbb{R}}^{n\times p},C\in {\mathbb{R}}^{p\times q}$,
$$[vec(ABC){]}_{jm+i}=[ABC{]}_{i,j}=\mathrm{T}\mathrm{r}(A_{i,\ast }B{C}_{\ast ,j})=\mathrm{T}\mathrm{r}(C_{\ast ,j}{A}_{i,\ast }B)=vec(A_{i,\ast }^T{C}_{\ast j}^T{)}^{T}vec(B)=[C^T\otimes A{]}_{jm+i,\ast }vec(B)$$

特例:

$$Ax=IAx=vec(IAx)=(x^T\otimes I)vec(A)$$

这个在处理梯度的时候会比较有用:
$$y=Ax$$
则
$$\mathrm{d}y=(\mathrm{d}A)x+A\mathrm{d}x=(x^T\otimes I)vec(\mathrm{d}A)+A\mathrm{d}x.$$