我们要首先清楚,图拉普拉斯矩阵不是为了图神经网络而生,而在之前就存在了很久很久。
简单介绍
简单地说,拉普拉斯矩阵就是 L = D − A L=D-A L=D−A, A A A是邻接矩阵, D D D是度矩阵。如下例:
A
=
(
0
1
0
0
1
0
1
0
1
0
1
0
0
1
0
1
0
0
0
0
1
0
1
1
1
1
0
1
0
0
0
0
0
1
0
0
)
D
=
(
2
0
0
0
0
0
0
3
0
0
0
0
0
0
2
0
0
0
0
0
0
3
0
0
0
0
0
0
3
0
0
0
0
0
0
1
)
A=\left(\begin{array}{llllll}0 & 1 & 0 & 0 & 1 & 0 \\ 1 & 0 & 1 & 0 & 1 & 0 \\ 0 & 1 & 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 & 1 & 1 \\ 1 & 1 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 & 0\end{array}\right) D=\left(\begin{array}{llllll}2 & 0 & 0 & 0 & 0 & 0 \\ 0 & 3 & 0 & 0 & 0 & 0 \\ 0 & 0 & 2 & 0 & 0 & 0 \\ 0 & 0 & 0 & 3 & 0 & 0 \\ 0 & 0 & 0 & 0 & 3 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1\end{array}\right)
A=⎝⎜⎜⎜⎜⎜⎜⎛010010101010010100001011110100000100⎠⎟⎟⎟⎟⎟⎟⎞D=⎝⎜⎜⎜⎜⎜⎜⎛200000030000002000000300000030000001⎠⎟⎟⎟⎟⎟⎟⎞
可求得拉普拉斯矩阵
L
L
L:
L
=
D
−
A
=
(
2
−
1
0
0
−
1
0
−
1
3
−
1
0
−
1
0
0
−
1
2
−
1
0
0
0
0
−
1
3
−
1
−
1
−
1
−
1
0
−
1
3
0
0
0
0
−
1
0
1
)
L=D-A=\left(\begin{array}{rrrrrr}2 & -1 & 0 & 0 & -1 & 0 \\ -1 & 3 & -1 & 0 & -1 & 0 \\ 0 & -1 & 2 & -1 & 0 & 0 \\ 0 & 0 & -1 & 3 & -1 & -1 \\ -1 & -1 & 0 & -1 & 3 & 0 \\ 0 & 0 & 0 & -1 & 0 & 1\end{array}\right)
L=D−A=⎝⎜⎜⎜⎜⎜⎜⎛2−100−10−13−10−100−12−10000−13−1−1−1−10−130000−101⎠⎟⎟⎟⎟⎟⎟⎞
为什么在图中会用到Laplacian matrix
初次接触图神经网络时,大家肯定会看到图拉普拉斯矩阵 L = I − D − 1 2 A D − 1 2 L = I - D ^ { - \frac { 1 } { 2 } } A D ^ { - \frac { 1 } { 2 } } L=I−D−21AD−21或者 L = D − A L=D-A L=D−A, 那么为什么需要这么定义,下面我们就来仔细推导一下:
我们规定:
- A A A:邻接矩阵, D D D:度矩阵
- f f f:将图 G G G看作一个包含 N N N个变量的离散函数
- x i x_i xi:节点 i i i的取值
- N i N_i Ni:图中节点的一阶所有邻居节点集合, j ∈ N i j∈N_i j∈Ni表示节点 i i i的一个一阶邻居节点
为了对上述节点的邻居节点集合
N
i
N_i
Ni使用邻接矩阵进行替换,需要继续对上述图中的拉普拉斯算子进行矩阵形式的推导;首先,将图
G
G
G中的各个节点的取值表示成矩阵的形式:
X
N
×
1
=
(
x
1
x
2
⋮
x
N
)
X^{N \times 1}=\left(\begin{array}{c}x_{1} \\ x_{2} \\ \vdots \\ x_{N}\end{array}\right)
XN×1=⎝⎜⎜⎜⎛x1x2⋮xN⎠⎟⎟⎟⎞
那么对于节点
i
i
i, 可以进一步进行展开:
Δ
f
x
i
=
∑
j
∈
N
i
(
x
i
−
x
j
)
=
∑
j
∈
N
a
i
j
(
x
i
−
x
j
)
=
∑
j
∈
N
a
i
j
x
i
−
∑
j
∈
N
a
i
j
x
j
=
x
i
∑
j
∈
N
a
i
j
−
∑
j
∈
N
a
i
j
x
j
\begin{aligned} \Delta f_{x_{i}} &=\sum_{j \in N_{i}}\left(x_{i}-x_{j}\right) \\ &=\sum_{j \in N} a_{i j}\left(x_{i}-x_{j}\right) \\ &=\sum_{j \in N} a_{i j} x_{i}-\sum_{j \in N} a_{i j} x_{j} \\ &=x_{i} \sum_{j \in N} a_{i j}-\sum_{j \in N} a_{i j} x_{j} \\ \end{aligned}
Δfxi=j∈Ni∑(xi−xj)=j∈N∑aij(xi−xj)=j∈N∑aijxi−j∈N∑aijxj=xij∈N∑aij−j∈N∑aijxj
注意到,
∑
j
∈
N
a
i
j
\sum_{j \in N} a_{i j}
∑j∈Naij可以看作第
i
i
i个节点的度之和,也可以看作邻接矩阵A的第
i
i
i行
A
i
A_i
Ai,进行替换后:
=
x
i
D
i
−
∑
j
∈
N
a
i
j
x
j
=
x
i
D
i
−
A
i
X
\begin{aligned} &=x_{i} D_{i}-\sum_{j \in N} a_{i j} x_{j} \\ &=x_{i} D_{i}-A_{i} X \end{aligned}
=xiDi−j∈N∑aijxj=xiDi−AiX
推广至所有节点:
Δ
f
=
(
Δ
f
x
1
Δ
f
x
2
⋮
Δ
f
x
N
)
=
(
x
1
D
1
−
A
1
X
x
2
D
2
−
A
2
X
⋮
x
N
D
N
−
A
N
X
)
=
(
x
1
D
1
x
2
D
2
⋮
x
N
D
N
)
−
(
A
1
X
A
2
X
⋮
A
N
X
)
\Delta f=\left(\begin{array}{c}\Delta f_{x_{1}} \\ \Delta f_{x_{2}} \\ \vdots \\ \Delta f_{x_{N}}\end{array}\right)=\left(\begin{array}{c}x_{1} D_{1}-A_{1} X \\ x_{2} D_{2}-A_{2} X \\ \vdots \\ x_{N} D_{N}-A_{N} X\end{array}\right)=\left(\begin{array}{c}x_{1} D_{1} \\ x_{2} D_{2} \\ \vdots \\ x_{N} D_{N}\end{array}\right)-\left(\begin{array}{c}A_{1} X \\ A_{2} X \\ \vdots \\ A_{N} X\end{array}\right)
Δf=⎝⎜⎜⎜⎛Δfx1Δfx2⋮ΔfxN⎠⎟⎟⎟⎞=⎝⎜⎜⎜⎛x1D1−A1Xx2D2−A2X⋮xNDN−ANX⎠⎟⎟⎟⎞=⎝⎜⎜⎜⎛x1D1x2D2⋮xNDN⎠⎟⎟⎟⎞−⎝⎜⎜⎜⎛A1XA2X⋮ANX⎠⎟⎟⎟⎞
继续分别进行简化,以求化成邻接矩阵
A
A
A和度矩阵
D
D
D:
(
A
1
X
A
2
X
:
A
N
X
)
=
A
N
×
N
X
N
×
1
=
A
X
\left( \begin{array} { l } { A _ { 1 } X } \\ { A _ { 2 } X } \\ { : } \\ { A _ { N } X } \end{array} \right) = A ^ { N \times N } X ^ { N \times 1 } = A X
⎝⎜⎜⎛A1XA2X:ANX⎠⎟⎟⎞=AN×NXN×1=AX
( x 1 D 1 x 2 D 2 ⋮ x i D N ) = ( D 1 0 0 ⋯ 0 0 D 2 0 ⋯ 0 ⋮ ⋮ ⋮ ⋱ ⋮ 0 0 0 ⋯ D N ) N × N X N × 1 = D N × N X N × 1 = D X \left(\begin{array}{c}x_{1} D_{1} \\ x_{2} D_{2} \\ \vdots \\ x_{i} D_{N}\end{array}\right)=\left(\begin{array}{ccccc}D_{1} & 0 & 0 & \cdots & 0 \\ 0 & D_{2} & 0 & \cdots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \cdots & D_{N}\end{array}\right)^{N \times N }X^{N\times1} =D ^ { N \times N } X ^ { N \times 1 } = D X ⎝⎜⎜⎜⎛x1D1x2D2⋮xiDN⎠⎟⎟⎟⎞=⎝⎜⎜⎜⎛D10⋮00D2⋮000⋮0⋯⋯⋱⋯00⋮DN⎠⎟⎟⎟⎞N×NXN×1=DN×NXN×1=DX
这样,我们便可得到拉普拉斯算子在图上的等式
Δ
f
=
D
X
−
A
X
=
(
D
−
A
)
X
\begin{array} { l } { \Delta f = D X - A X = ( D - A ) X } \end{array}
Δf=DX−AX=(D−A)X
L
=
D
−
A
{ L = D - A }
L=D−A,
L
L
L被称作为拉普拉斯矩阵
两种形式
1. 标准拉普拉斯矩阵
L = D − A L=D-A L=D−A
各元素的值:
L
i
j
=
{
D
i
i
,
if
i
=
j
−
1
,
if
i
,
j
in edge set
0
,
otherwise
L_{i j}=\left\{\begin{array}{ll}D_{i i}, & \text { if } \mathrm{i}=\mathrm{j} \\ -1, & \text { if } \mathrm{i}, \mathrm{j} \text { in edge set } \\ 0, & \text { otherwise }\end{array}\right.
Lij=⎩⎨⎧Dii,−1,0, if i=j if i,j in edge set otherwise
- 拉普拉斯矩阵的对角线上取值为节点的度
- 如果节点和节点相邻,那么取值为-1
- 显然,拉普拉斯矩阵为对称的矩阵
可以结合刚开始的例子进行理解。
2. 归一化的拉普拉斯矩阵(Symmetric normalized Laplacian)
这就是大家会在各个模型里见到比较多的形式:
L
s
m
m
=
D
−
1
2
L
D
−
1
2
=
I
−
D
−
1
2
A
D
−
1
2
L _ { s m m } = D ^ { - \frac { 1 } { 2 } } L D ^ { - \frac { 1 } { 2 } } = I - D ^ { - \frac { 1 } { 2 } } A D ^ { - \frac { 1 } { 2 } }
Lsmm=D−21LD−21=I−D−21AD−21
各元素的值:
L
i
j
=
{
1
,
if
i
=
j
−
1
D
i
i
D
j
j
,
if
i
,
j
in edge set
0
,
otherwise
L_{i j}=\left\{\begin{array}{ll}1, & \text { if } \mathrm{i}=\mathrm{j} \\ \frac{-1}{\sqrt{D_{i i}} \sqrt{D_{j j}}}, & \text { if } \mathrm{i}, \mathrm{j} \text { in edge set } \\ 0, & \text { otherwise }\end{array}\right.
Lij=⎩⎪⎨⎪⎧1,DiiDjj−1,0, if i=j if i,j in edge set otherwise