1、知识梳理
1.1 图卷积神经网络
GCN数学定义为:$$\mathbf{X}^{\prime} = \mathbf{\hat{D}}^{-1/2} \mathbf{\hat{A}}\mathbf{\hat{D}}^{-1/2} \mathbf{X} \mathbf{\Theta},$$是对称归一化矩阵,它的节点式表述为:$$\mathbf{x}^{\prime}_i = \mathbf{\Theta} \sum_{j \in \mathcal{N}(v) \cup\{ i \}} \frac{e_{j,i}}{\sqrt{\hat{d}_j \hat{d}_i}} \mathbf{x}_j$$.
1.2 图注意力神经网络
定义:$$\mathbf{x}^{\prime}_i = \alpha_{i,i}\mathbf{\Theta}\mathbf{x}_{i} +\sum_{j \in \mathcal{N}(i)} \alpha_{i,j}\mathbf{\Theta}\mathbf{x}_{j},$$
$$\alpha_{i,j} =\frac{\exp\left(\mathrm{LeakyReLU}\left(\mathbf{a}^{\top}[\mathbf{\Theta}\mathbf{x}_i \, \Vert \, \mathbf{\Theta}\mathbf{x}_j]\right)\right)}{\sum_{k \in \mathcal{N}(i) \cup \{ i \}}\exp\left(\mathrm{LeakyReLU}\left(\mathbf{a}^{\top}[\mathbf{\Theta}\mathbf{x}_i \, \Vert \, \mathbf{\Theta}\mathbf{x}_k]\right)\right)}.$$
2、使用[PyG中不同的图卷积模块在[PyG的不同数据集]上实现节点分类或回归任务。
2.1 数据获取
本次实验使用citeseer数据集,Citeseer数据集来自CiteSeer数据库,行代表科学论文,每个属性表示一个作者。共有3327个节点,9104条边,3703个属性,分为6个类。
from torch_geometric.datasets import Planetoid
from torch_geometric.transforms import NormalizeFeatures
dataset = Planetoid(root='data/Planetoid', name='citeseer', transform=NormalizeFeatures())
print()
print(f'Dataset: {dataset}:')
print('======================')
print(f'Number of graphs: {len(dataset)}')
print(f'Number of features: {dataset.num_features}')
print(f'Number of classes: {dataset.num_classes}')
data = dataset[0] # Get the first graph object.
print()
print(data)
print('======================')
# Gather some statistics about the graph.
print(f'Number of nodes: {data.num_nodes}')
print(f'Number of edges: {data.num_edges}')
print(f'Average node degree: {data.num_edges / data.num_nodes:.2f}')
print(f'Number of training nodes: {data.train_mask.sum()}')
print(f'Training node label rate: {int(data.train_mask.sum()) / data.num_nodes:.2f}')
print(f'Contains isolated nodes: {data.contains_isolated_nodes()}')
print(f'Contains self-loops: {data.contains_self_loops()}')
print(f'Is undirected: {data.is_undirected()}')
Dataset: citeseer():
======================
Number of graphs: 1
Number of features: 3703
Number of classes: 6
Data(edge_index=[2, 9104], test_mask=[3327], train_mask=[3327], val_mask=[3327], x=[3327, 3703], y=[3327])
======================
Number of nodes: 3327
Number of edges: 9104
Average node degree: 2.74
Number of training nodes: 120
Training node label rate: 0.04
Contains isolated nodes: True
Contains self-loops: False
Is undirected: True
2.2 MLP 分类
import torch
from torch.nn import Linear
import torch.nn.functional as F
class MLP(torch.nn.Module):
def __init__(self, hidden_channels):
super(MLP, self).__init__()
torch.manual_seed(12345)
self.lin1 = Linear(dataset.num_features, hidden_channels)
self.lin2 = Linear(hidden_channels, dataset.num_classes)
def forward(self, x):
x = self.lin1(x)
x = x.relu()
x = F.dropout(x, p=0.5, training=self.training)
x = self.lin2(x)
return x
model = MLP(hidden_channels=16)
print(model)
model = MLP(hidden_channels=16)
criterion = torch.nn.CrossEntropyLoss() # Define loss criterion.
optimizer = torch.optim.Adam(model.parameters(), lr=0.01, weight_decay=5e-4) # Define optimizer.
def train():
model.train()
optimizer.zero_grad() # Clear gradients.
out = model(data.x) # Perform a single forward pass.
loss = criterion(out[data.train_mask], data.y[data.train_mask]) # Compute the loss solely based on the training nodes.
loss.backward() # Derive gradients.
optimizer.step() # Update parameters based on gradients.
return loss
for epoch in range(1, 201):
loss = train()
if epoch % 10 == 0:
print(f'Epoch: {epoch:03d}, Loss: {loss:.4f}')
Epoch: 010, Loss: 1.7279
Epoch: 020, Loss: 1.5793
Epoch: 030, Loss: 1.4126
Epoch: 040, Loss: 1.1490
Epoch: 050, Loss: 1.0015
Epoch: 060, Loss: 0.8061
Epoch: 070, Loss: 0.7439
Epoch: 080, Loss: 0.5477
Epoch: 090, Loss: 0.5127
Epoch: 100, Loss: 0.5078
Epoch: 110, Loss: 0.4658
Epoch: 120, Loss: 0.4191
Epoch: 130, Loss: 0.4491
Epoch: 140, Loss: 0.4426
Epoch: 150, Loss: 0.4146
Epoch: 160, Loss: 0.4921
Epoch: 170, Loss: 0.3707
Epoch: 180, Loss: 0.3892
Epoch: 190, Loss: 0.4187
Epoch: 200, Loss: 0.3789
def test():
model.eval()
out = model(data.x)
pred = out.argmax(dim=1) # Use the class with highest probability.
test_correct = pred[data.test_mask] == data.y[data.test_mask] # Check against ground-truth labels.
test_acc = int(test_correct.sum()) / int(data.test_mask.sum()) # Derive ratio of correct predictions.
return test_acc
test_acc = test()
print(f'Test Accuracy: {test_acc:.4f}')
Test Accuracy: 0.5820
2.3 GCN图节点分类器
# 训练GCN图节点分类器
model = GCN(hidden_channels=16)
optimizer = torch.optim.Adam(model.parameters(), lr=0.01, weight_decay=5e-4)
criterion = torch.nn.CrossEntropyLoss()
def train():
model.train()
optimizer.zero_grad() # Clear gradients.
out = model(data.x, data.edge_index) # Perform a single forward pass.
loss = criterion(out[data.train_mask], data.y[data.train_mask]) # Compute the loss solely based on the training nodes.
loss.backward() # Derive gradients.
optimizer.step() # Update parameters based on gradients.
return loss
for epoch in range(1, 201):
loss = train()
if epoch % 10 == 0:
print(f'Epoch: {epoch:03d}, Loss: {loss:.4f}')
Epoch: 010, Loss: 1.7346
Epoch: 020, Loss: 1.6497
Epoch: 030, Loss: 1.5451
Epoch: 040, Loss: 1.4200
Epoch: 050, Loss: 1.2842
Epoch: 060, Loss: 1.1571
Epoch: 070, Loss: 1.0097
Epoch: 080, Loss: 0.9321
Epoch: 090, Loss: 0.8222
Epoch: 100, Loss: 0.7701
Epoch: 110, Loss: 0.7429
Epoch: 120, Loss: 0.7004
Epoch: 130, Loss: 0.6668
Epoch: 140, Loss: 0.6407
Epoch: 150, Loss: 0.5929
Epoch: 160, Loss: 0.6033
Epoch: 170, Loss: 0.5322
Epoch: 180, Loss: 0.5230
Epoch: 190, Loss: 0.5473
Epoch: 200, Loss: 0.4934
# 测试GCN图节点分类器
def test():
model.eval()
out = model(data.x, data.edge_index)
pred = out.argmax(dim=1) # Use the class with highest probability.
test_correct = pred[data.test_mask] == data.y[data.test_mask] # Check against ground-truth labels.
test_acc = int(test_correct.sum()) / int(data.test_mask.sum()) # Derive ratio of correct predictions.
return test_acc
test_acc = test()
print(f'Test Accuracy: {test_acc:.4f}')
Test Accuracy: 0.7120
2.4 构造GAT节点分类神经网络
# 构造GAT节点分类神经网络
import torch
from torch.nn import Linear
import torch.nn.functional as F
from torch_geometric.nn import GATConv
class GAT(torch.nn.Module):
def __init__(self, hidden_channels):
super(GAT, self).__init__()
torch.manual_seed(12345)
self.conv1 = GATConv(dataset.num_features, hidden_channels)
self.conv2 = GATConv(hidden_channels, dataset.num_classes)
def forward(self, x, edge_index):
x = self.conv1(x, edge_index)
x = x.relu()
x = F.dropout(x, p=0.5, training=self.training)
x = self.conv2(x, edge_index)
return x
Epoch: 010, Loss: 1.7339
Epoch: 020, Loss: 1.6383
Epoch: 030, Loss: 1.5224
Epoch: 040, Loss: 1.3374
Epoch: 050, Loss: 1.1420
Epoch: 060, Loss: 0.9853
Epoch: 070, Loss: 0.7943
Epoch: 080, Loss: 0.6670
Epoch: 090, Loss: 0.5781
Epoch: 100, Loss: 0.5379
Epoch: 110, Loss: 0.4792
Epoch: 120, Loss: 0.4451
Epoch: 130, Loss: 0.3572
Epoch: 140, Loss: 0.3719
Epoch: 150, Loss: 0.3148
Epoch: 160, Loss: 0.3107
Epoch: 170, Loss: 0.3433
Epoch: 180, Loss: 0.2881
Epoch: 190, Loss: 0.2794
Epoch: 200, Loss: 0.2010
# 测试GAT图节点分类器
def test():
model.eval()
out = model(data.x, data.edge_index)
pred = out.argmax(dim=1) # Use the class with highest probability.
test_correct = pred[data.test_mask] == data.y[data.test_mask] # Check against ground-truth labels.
test_acc = int(test_correct.sum()) / int(data.test_mask.sum()) # Derive ratio of correct predictions.
return test_acc
test_acc = test()
print(f'Test Accuracy: {test_acc:.4f}')
Test Accuracy: 0.6100
2.5 对比分析
以上训练结果克制,GCN的效果最好,GAT的效果次之,MLP的效果最差。下图依次为原始节点的投影图,GCN训练后的分布图和GAT训练后的分布图。
| | |