UltraGCN: Ultra Simplification of Graph ConvolutionalNetworks for Recommendation(个人笔记)

该文章发表在CIKM'2021,在何向南老师的LightGCN上做出了进一步的改进。

论文链接:https://arxiv.org/abs/2110.15114

代码链接:https://github.com/xue-pai/ultragcn

文章首先指出了LightGCN的三个不足,如下:

1)消息传递期间在边缘上分配的权重违反直觉,这可能不适用于CF。

2)传播过程递归地将不同类型的关系对(包括用户项对、项项项对和用户用户对)组合到模型中,但未能捕捉到它们不同的重要性。这也可能会引入嘈杂且缺乏信息的关系,从而混淆模型训练。

3)过度平滑问题限制了在LightGCN中使用过多的消息传递层。

然后说明他们的模型不执行显式消息传递,而是通过约束损失直接近似无限层图卷积的极限,从而得到了超简化GCN模型------UltraGCN。并指出所做的贡献:

1)实证分析了LightGCN的培训效率低下,并进一步将其原因归因于消息传递机制的关键限制。 

2)我们提出了一种超简化的GCN公式,即UltraGCN,它跳过了无限层的显式消息传递以实现高效的推荐。

3)在四个基准数据集上进行了大量实验,以证明UltraGCN的有效性和效率。


这部分详细说明LCN的不足之处:

在LCN中,各层信息的传递公式表示为:

 本文为用户 u 和项目 i 重写了消息传递公式:

 其中,e_{u}^{_{(l)}} 和 e_{i}^{_{(l)}} 分别代表用户 u 和 项目 i 在第 l 层的嵌入。N\left ( u \right ) 和 N\left ( i \right ) 分别代表用户 u 和 项目 i 的邻接节点。 

  • 1
    点赞
  • 13
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
One possible improvement could be to use vectorized operations instead of loops to improve performance. Here's an updated implementation: ```python def estimate_variance(xs: np.ndarray, ys: np.ndarray, affine: np.ndarray, translation: np.ndarray, responsibility: np.ndarray) -> float: """ Estimate the variance of GMM. For simplification, we assume all the Gaussian distributions share the same variance, and each feature dimension is independent, so the variance can be represented as a scalar. :param xs: a set of points with size (N, D), N is the number of samples, D is the dimension of points :param ys: a set of points with size (M, D), M is the number of samples, D is the dimension of points :param affine: an affine matrix with size (D, D) :param translation: a translation vector with size (1, D) :param responsibility: the responsibility matrix with size (N, M) :return: the variance of each Gaussian distribution, a float """ # Transform xs and ys using affine and translation xs_transformed = xs @ affine.T + translation ys_transformed = ys @ affine.T + translation # Compute the difference between xs and ys for each pair of samples diff = xs_transformed[:, None, :] - ys_transformed[None, :, :] # Compute the squared Euclidean distance for each pair of samples dist_sq = np.sum(diff**2, axis=2) # Compute the weighted sum of squared distances using the responsibility matrix weighted_dist_sq = np.sum(responsibility * dist_sq) # Compute the total weight of responsibility matrix total_weight = np.sum(responsibility) # Compute the variance as the weighted average of squared distances variance = weighted_dist_sq / (total_weight * xs.shape[1]) return variance ``` This implementation uses matrix multiplication and broadcasting to perform the affine transformation and compute the pairwise distances between samples, which should be more efficient than using loops. It also computes the variance directly from the weighted sum of squared distances, without needing to compute the sum of squared differences and then dividing by the number of dimensions.

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值