斯坦福CS224W作业——colab3笔记

2023年CS22Wassignment中的所有colab答案以及注释已经上传到github：https://github.com/yuyu990116/CS224W-assignment
CS224W课程地址：http://web.stanford.edu/class/cs224w/
以下是做colab3的心得笔记

PyG内置的MessagePassing：

https://pytorch-geometric.readthedocs.io/en/latest/notes/create_gnn.html

MessagePassing.propagate(edge_index, size=None, **kwargs): The initial call to start propagating messages. Takes in the edge indices and all additional data which is needed to construct messages and to update node embeddings. Note that propagate() is not limited to exchanging messages in square adjacency matrices of shape [N, N] only, but can also exchange messages in general sparse assignment matrices, e.g., bipartite graphs（我理解为异质图）, of shape [N, M] by passing size=(N, M) as an additional argument. If set to None, the assignment matrix is assumed to be a square matrix. For bipartite graphs with two independent sets of nodes and indices, and each set holding its own information, this split can be marked by passing the information as a tuple, e.g. x=(x_N, x_M).

propagate会调用message和aggregate和update

import torch
from torch.nn import Linear, Parameter
from torch_geometric.nn import MessagePassing
from torch_geometric.utils import add_self_loops, degree

class GCNConv(MessagePassing):
    def __init__(self, in_channels, out_channels):
        super().__init__(aggr='add')  # "Add" aggregation (Step 5).
        self.lin = Linear(in_channels, out_channels, bias=False)
        self.bias = Parameter(torch.Tensor(out_channels))

        self.reset_parameters()

    def reset_parameters(self):
        self.lin.reset_parameters()
        self.bias.data.zero_()

    def forward(self, x, edge_index):
        # x has shape [N, in_channels]
        # edge_index has shape [2, E]

        # Step 1: Add self-loops to the adjacency matrix.
        edge_index, _ = add_self_loops(edge_index, num_nodes=x.size(0))

        # Step 2: Linearly transform node feature matrix.
        x = self.lin(x)

        # Step 3: Compute normalization.
        row, col = edge_index
        deg = degree(col, x.size(0), dtype=x.dtype)
        deg_inv_sqrt = deg.pow(-0.5)
        deg_inv_sqrt[deg_inv_sqrt == float('inf')] = 0
        norm = deg_inv_sqrt[row] * deg_inv_sqrt[col]

        # Step 4-5: Start propagating messages.
        out = self.propagate(edge_index, x=x, norm=norm)

        # Step 6: Apply a final bias vector.
        out += self.bias

        return out

    def message(self, x_j, norm):
        # x_j has shape [E, out_channels]

        # Step 4: Normalize node features.
        return norm.view(-1, 1) * x_j

如果不进行propagate改写的话，默认是“add"propagation（求和）

在message()函数中，我们需要按norm对相邻节点特征x_j进行规范化。其中，x_j表示一个提升张量，它包含每条边的源节点特征，即每个节点的邻居。可以通过将_i或_j附加到变量名来自动提升节点特性。实际上，任何张量都可以这样转换，只要它们具有源节点或目的节点的特征。

以上所有可以用以下代码进行：

conv = GCNConv(16, 32)
x = conv(x, edge_index)

Parameter：https://pytorch.org/docs/stable/generated/torch.nn.parameter.Parameter.html

class torch.nn.parameter.Parameter(data=None, requires_grad=True)

Xavier初始化方法旨在使得输入和输出的方差相等，以避免在深度神经网络中出现梯度消失或梯度爆炸的问题。