CS224W: Machine Learning with Graphs
Stanford / Winter 2021
05-message
-
给定图中一些已有标签的节点,如何利用半监督学习将其余节点也打上标签
-
Intuition: Correlations exist in networks
-
In other words: Similar nodes are connected
-
Key concept is collective classification: Idea of assigning labels to all nodes in a network together
-
Relational Classifier
-
Key Idea: Y v Y_v Yv表示节点 v v v的类别概率(二分类),对于没有标签的节点赋初值 Y v = 0.5 Y_v = 0.5 Yv=0.5,根据邻居节点(无序)更新所有节点直到收敛
-
节点的更新公式
P ( Y v = c ) = 1 ∑ ( v , u ) ∈ E A v , u ∑ ( v , u ) ∈ E A v , u P ( Y u = c ) P\left(Y_{v}=c\right)=\frac{1}{\sum_{(v, u) \in E} A_{v, u}} \sum_{(v, u) \in E} A_{v, u} P\left(Y_{u}=c\right) P(Yv=c)=∑(v,u)∈EAv,u1(v,u)∈E∑Av,uP(Yu=c)
-
不能保证收敛性
-
无法使用节点本身的属性特征
-
-
Example
-
Initialization
-
1-st Iteration, Update Node 3
-
1-st Iteration, Update Node 4
-
…After 1-st Iteration
-
…After 2-nd Iteration
-
…After 4 Iterations, Convergence
-
Iterative Classifier
-
Key Idea: Relational Classifier无法利用节点特征,Iterative Classifier利用节点以及邻居特征进行分类
-
Iterative Classification分为两阶段
-
Phase-1 (Train): Classify based on node attributes alone
-
在训练集上训练分类器(MLP、DNN等)
-
ϕ 1 ( f v ) \phi_{1}\left(f_{v}\right) ϕ1(fv): to predict Y v Y_v Yv based on f v f_v fv (仅根据节点本身的特征分类)
-
ϕ 2 ( f v , z v ) \phi_{2}\left(f_{v}, z_{v}\right) ϕ2(fv,zv): to predict Y v Y_v Yv based on f v f_v fv and summary z v z_v zv of labels of v v v’s neighbors (根据节点本身的特征以及邻居节点标签的汇总信息进行分类)
-
-
Phase-2 (Test): Iterate till convergence
-
在测试集上,用 ϕ 1 \phi_{1} ϕ1设置每个节点的初始标签,计算 z v z_v zv并用 ϕ 2 \phi_{2} ϕ2预测节点标签
-
Repeat for each node v v v
-
Update z v z_v zv based on Y u Y_u Yu for all u ∈ N v u \in N_v u∈Nv (根据每个节点的邻居标签更新该节点的 z v z_v zv)
-
Update Y v Y_v Yv based on the new z v z_v zv using ϕ 2 \phi_{2} ϕ2 (根据新的 z v z_v zv使用 ϕ 2 \phi_{2} ϕ2更新节点标签)
-
-
迭代直到标签稳定或达到最大循环上限(无法保证收敛性)
-
-
-
Example
-
假设两个分类器已经训练完成,接下来为Phase-2测试阶段
-
使用 ϕ 1 \phi_{1} ϕ1初始化标签
-
Repeat
-
更新 z v z_v zv
-
使用 ϕ 2 \phi_{2} ϕ2重新赋上标签
-
-
迭代直至收敛
-
Loopy Belief Propagation
-
Key Idea: 有点类似于Message Passing
-
Notation
-
Algorithm
-
Problem
-
Advantages