【学习笔记】GNN - Task2

本文详细介绍了图神经网络中的消息传递范式,包括节点嵌入、PyG中的MessagePassing基类及其GCNConv子类的实现。重点讲解了GCNConv的数学定义和消息传递的五个关键步骤,以及如何利用MessagePassing基类构建图神经网络。同时,给出了基于MessagePassing的作业,要求总结运行流程并复现一层图神经网络。
摘要由CSDN通过智能技术生成

消息传递

1.消息传递(Message Passing)范式

在这里插入图片描述

  • 节点嵌入(Node Embedding)- 神经网络的生成节点表征的操作

2.MessagePassing类

Pytorch Geometric(PyG)提供了MessagePassing基类,它封装了“消息传递”的运行流程。

3.GCNConv子类

GCNConv的数学定义为 x i ( k ) = ∑ j ∈ N ( i ) ∪ i 1 deg ⁡ ( i ) ⋅ deg ⁡ ( j ) ⋅ ( Θ ⋅ x j ( k − 1 ) ) , \mathbf{x}i^{(k)} = \sum{j \in \mathcal{N}(i) \cup { i }} \frac{1}{\sqrt{\deg(i)} \cdot \sqrt{\deg(j)}} \cdot \left( \mathbf{\Theta} \cdot \mathbf{x}_j^{(k-1)} \right), xi(k)=jN(i)ideg(i) deg(j) 1(Θxj(k1)), 其中,邻接节点的表征 x j ( k − 1 ) \mathbf{x}_j^{(k-1)} xj(k1)首先通过与权重矩阵 Θ \mathbf{\Theta} Θ相乘进行变换,然后按端点的度 deg ⁡ ( i ) , deg ⁡ ( j ) \deg(i), \deg(j) deg(i),deg(j)进行归一化处理,最后进行求和。这个公式可以分为以下几个步骤:

  • 向邻接矩阵添加自环边。
  • 对节点表征做线性转换。
  • 计算归一化系数。
  • 归一化邻接节点的节点表征。
  • 将相邻节点表征相加("求和 "聚合)。

步骤1-3通常是在消息传递发生之前计算的。步骤4-5可以使用MessagePassing基类轻松处理。

作业

  • 总结MessagePassing基类的运行流程
    • 复现一个一层的图神经网络的构造,总结通过继承MessagePassing基类来构造自己的图神经网络类的规范。
def propagate(self, edge_index: Adj, size: Size = None, **kwargs):
    '''
    args
    '''
    size = self.__check_input__(edge_index, size)

    # Run "fused" message and aggregation (if applicable).
    if (isinstance(edge_index, SparseTensor) and self.fuse
            and not self.__explain__):
        coll_dict = self.__collect__(self.__fused_user_args__, edge_index,
                                     size, kwargs)

        msg_aggr_kwargs = self.inspector.distribute(
            'message_and_aggregate', coll_dict)
        out = self.message_and_aggregate(edge_index, **msg_aggr_kwargs)

        update_kwargs = self.inspector.distribute('update', coll_dict)
        return self.update(out, **update_kwargs)

    # Otherwise, run both functions in separation.
    elif isinstance(edge_index, Tensor) or not self.fuse:
        coll_dict = self.__collect__(self.__user_args__, edge_index, size,
                                     kwargs)

        msg_kwargs = self.inspector.distribute('message', coll_dict)
        out = self.message(**msg_kwargs)

        # For `GNNExplainer`, we require a separate message and aggregate
        # procedure since this allows us to inject the `edge_mask` into the
        # message passing computation scheme.
        if self.__explain__:
            edge_mask = self.__edge_mask__.sigmoid()
            # Some ops add self-loops to `edge_index`. We need to do the
            # same for `edge_mask` (but do not train those).
            if out.size(self.node_dim) != edge_mask.size(0):
                loop = edge_mask.new_ones(size[0])
                edge_mask = torch.cat([edge_mask, loop], dim=0)
            assert out.size(self.node_dim) == edge_mask.size(0)
            out = out * edge_mask.view([-1] + [1] * (out.dim() - 1))

        aggr_kwargs = self.inspector.distribute('aggregate', coll_dict)
        out = self.aggregate(out, **aggr_kwargs)

        update_kwargs = self.inspector.distribute('update', coll_dict)
        return self.update(out, **update_kwargs)

上面的代码中,propagate函数会进行信息的传播计算过程。当fuse时,执行message_and_aggregate和update,否则执行message、aggregate和update函数。

import torch
from torch_geometric.nn import MessagePassing
from torch_geometric.utils import add_self_loops, degree
from torch_sparse import SparseTensor

class GCNConv(MessagePassing):
    def __init__(self, in_channels, out_channels):
        super(GCNConv, self).__init__(aggr='add', flow='source_to_target')
        # "Add" aggregation (Step 5).
        # flow='source_to_target' 表示消息从源节点传播到目标节点
        print("`constructor` is called")
        self.lin = torch.nn.Linear(in_channels, out_channels)

    def forward(self, x, edge_index):
        # x has shape [N, in_channels]
        # edge_index has shape [2, E]

        # Step 1: Add self-loops to the adjacency matrix.
        edge_index, _ = add_self_loops(edge_index, num_nodes=x.size(0))

        # Step 2: Linearly transform node feature matrix.
        x = self.lin(x)

        # Step 3: Compute normalization.
        row, col = edge_index
        deg = degree(col, x.size(0), dtype=x.dtype)
        deg_inv_sqrt = deg.pow(-0.5)
        norm = deg_inv_sqrt[row] * deg_inv_sqrt[col]

        # Step 4-5: Start propagating messages.
        adjmat = SparseTensor(row=edge_index[0], col=edge_index[1], value=torch.ones(edge_index.shape[1]))
        # 此处传的不再是edge_idex,而是SparseTensor类型的Adjancency Matrix
        print("`forward` is called")
        return self.propagate(adjmat, x=x, norm=norm, deg=deg.view((-1, 1)))

    def message(self, x_j, norm, deg_i):
        # x_j has shape [E, out_channels]
        # deg_i has shape [E, 1]
        # Step 4: Normalize node features.
        print("`message` is called")
        return norm.view(-1, 1) * x_j * deg_i

    def aggregate(self, inputs, index, ptr, dim_size):
        #print('self.aggr:', self.aggr)
        print("`aggregate` is called")
        return super().aggregate(inputs, index, ptr=ptr, dim_size=dim_size)

    '''
    def message_and_aggregate(self, adj_t, x, norm):
        print('`message_and_aggregate` is called')
        # 没有实现真实的消息传递与消息聚合的操作
    '''
    def update(self, inputs, deg):
        print("`update` is called")
        return inputs
        
from torch_geometric.datasets import Planetoid

dataset = Planetoid(root='dataset/Cora', name='Cora')
data = dataset[0]

net = GCNConv(data.num_features, 64)
h_nodes = net(data.x, data.edge_index)
print(h_nodes.shape)
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值