# CS224W摘要06.07.GNN

19 篇文章 8 订阅

CS224W: Machine Learning with Graphs

# 复习

1. 𝑶 ( ∣ 𝑽 ∣ ) 𝑶(|𝑽|) parameters are needed:
No sharing of parameters between nodes参数个数与节点个数成正比，无法共享参数
Every node has its own unique embedding
2. Inherently “transductive”:
Cannot generate embeddings for nodes that are not seen during training
3. Do not incorporate node features:
Many graphs have features that we can and should leverage

# Deep Learning for Graphs

## Setup

Assume we have a graph 𝑮 𝑮 :
§ 𝑉 𝑉 is the vertex set
§ 𝑨 𝑨 is the adjacency matrix (assume binary)
§ 𝑿 ∈ R m × ∣ V ∣ 𝑿 ∈ ℝ^{m\times |V|} is a matrix of node features
§ 𝑣 𝑣 : a node in 𝑉 𝑉 ; 𝑁 ( 𝑣 ) 𝑁(𝑣) : the set of neighbors of 𝑣 𝑣 .

Node features:
§ Social networks: User profile, User image
§ Biological networks: Gene expression profiles, gene functional information
§ When there is no node feature in the graph dataset:
Indicator vectors (one-hot encoding of a node)
Vector of constant 1: [1, 1, …, 1]

1.参数量大，和节点数量+特征维度成正比
2.DNN结构训练好后如果图结构变化，邻接矩阵大小也会变化，此时无法适配原DNN
3.DNN对输入顺序是敏感的，而图是无序的，相同图不同的顺序图的邻接矩阵不一样，DNN无法处理无序的结构

## Describe aggregation strategies

Key idea: Generate node embeddings based on local network neighborhoods

Aggregation的基本方式：

Let H ( l ) = [ h 1 ( l ) ⋯ h ∣ V ∣ ( l ) ] T H^{(l)}=[h_1^{(l)}\cdots h_{|V|}^{(l)}]^T ，节点embedding的矩阵形式

D是度矩阵，是一个对角矩阵： D v , v = D e g ( v ) = ∣ N ( v ) ∣ D_{v,v}=Deg(v)=|N(v)|

∑ u ∈ N ( v ) h u l ∣ N ( v ) ∣ → H ( l + 1 ) = D − 1 A H ( l ) \sum_{u\in N(v)}\cfrac{h_u^l}{|N(v)|}\rightarrow H^{(l+1)}=D^{-1}AH^{(l)}

H ( l + 1 ) = σ ( A ~ H ( l ) W l T + H ( l ) B l T ) , A ~ = D − 1 A H^{(l+1)}=\sigma({\color{Red}\tilde AH^{(l)}W_l^T}+{\color{Blue}H^{(l)}B_l^T}),\tilde A=D^{-1}A
Red: neighborhood aggregation
Blue: self transformation

## 如何训练

L = ∑ z u , z v C E ( y u , v , D E C ( z u , z v ) ) \mathcal{L}=\sum_{z_u,z_v}CE(y_{u,v},DEC(z_u,z_v))
Where y u , v = 1 y_{u,v}= 1 when node 𝑢 and 𝑣 are similar
CE is the cross entropy
DEC is the decoder such as inner product

## 单层GNN

GNN Layer = Message + Aggregation

GCN
h v ( l ) = σ ( ∑ u ∈ N ( v ) W ( l ) h u ( l − 1 ) ∣ N ( v ) ∣ ) h_v^{(l)}=\sigma\left(\sum_{u\in N(v)}W^{(l)}\cfrac{h_u^{(l-1)}}{|N(v)|}\right)

GrahphSAGE
h v ( l ) = σ ( w ( l ) ⋅ C O N C A T ( h v ( l − 1 ) , A G G ( { h u ( l − 1 ) , ∀ u ∈ N ( v ) } ) ) ) h_v^{(l)}=\sigma\left(w^{(l)}\cdot CONCAT\left(h_v^{(l-1)},AGG\left(\{h_u^{(l-1)},\forall u\in N(v)\}\right)\right)\right)

GrahphSAGE的AGG有三种形式：Mean、Pool、LSTM。
GAT也讲了一下，不记了，后面还将了一些NN的基本知识（BN，抓爆，激活函数），不记了。

## 多层GNN

### 防止over-smoothing

Pre-processing layers: Important when encoding node features is necessary.
E.g., when nodes represent images/text
Post-processing layers: Important when reasoning / transformation over node embeddings are needed.
E.g., graph classification, knowledge graphs

## 如果让GNN go deep

We can increase the impact of earlier layers on the final node embeddings, by adding shortcuts in GNN.

h v ( l ) = σ ( ∑ u ∈ N ( v ) W ( l ) h u ( l − 1 ) ∣ N ( v ) ∣ + h v ( l − 1 ) ) h_v^{(l)}=\sigma\left(\sum_{u\in N(v)}W^{(l)}\cfrac{h_u^{(l-1)}}{|N(v)|}+h_v^{(l-1)}\right)

• 0
点赞
• 2
收藏
• 打赏
• 0
评论
01-28 1319
10-24 964
05-24
01-04
07-28 28
07-28 27
07-13 31
07-25 24
07-20 22
07-24 21
12-02 177
12-02 309
12-02 181

### “相关推荐”对你有帮助么？

• 非常没帮助
• 没帮助
• 一般
• 有帮助
• 非常有帮助

oldmao_2000

¥2 ¥4 ¥6 ¥10 ¥20

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、C币套餐、付费专栏及课程。