《Translating Embeddings for Modeling Multi-relational Data》阅读笔记

翻译 2018年04月16日 14:20:54

Abstract

We propose TransE, embedding entities and relationships of multi-relational data in low-dimensional vector space. It significantly outperforms state-of-the-art methods in link prediction on two knowledge bases. It can also be successfully trained on a large scale data set.

Introduction

Our work focuses on modeling multi-relational data from KBs(Knowledge Base), with the goal of providing an efficient tool to complete them by automatically adding new facts, without requiring extra knowledge.

1. Modeling Multi-relational data

In contrast to single-relational data, the difficulty of multi-relational data is that the notion of locality may involve relationships and entities of different types at the same time, so that modeling multi-relational data requires more generic approaches that can choose the appropriate patterns considering all heterogeneous relationships at the same time.

… suggested that even in complex and heterogeneous multi-relational domains simple yet appropriate modeling assumptions can lead to better trade-offs(权衡) between accuracy and scalability(可扩展性).

2. Relationships as translations in the embedding space

TransE, an energy-based model for learning low-dimensional embeddings of entities. In TransE, realtionships are represented as translations in the embedding space: if (h,l,t) holds, then the embedding of the tail entity should be close to embedding of the head entity h plus some vector that depends on the relationship l. This approach relies on a reduced set of parameters as it learns only one low-dimensional vector for each entity and each relationship.

The main motivation behind our translation-based parameterization is that hierarchical relationships are extremely common in KBs and translations are the natural transformations for representing them. Indeed, considering the natural representation of trees(i.e. embeddings of the nodes in dimension 2), the siblings are close to each other and nodes at a given height are organized on the x-axis, the parent-child relationship corresponds to a translation on the y-axis.

ps: a null translation vector corresponds to an equivalence relationship between entities.

Translation-based Model

Given a training set S of triplets (h,l,t) composed of two entities h,tE (the set of entities) and a relationship lL (the set of relationships), the model learns vector embeddings of the entities and the relationships.

Note that for a given entity, its embedding vector is the same when the entity appears as the head or as the tail of a triplet.

We want that h+lt when (h,l,t) holds (t should be a nearest neighbor of h+l), while h+l should be far away from t otherwise.

transe算法

measure d

Following an energy-based framework, the energy of a triplet is equal to d(h+l,t) for some dissimilarity measure d, which we take to be either the L1 or the L2-norm(曼哈顿或欧几里得距离).
d(h+l,t)=||h+tl||22

corrupted triplets S(h,l,t)

either the head or tail replaced by a random entity(but not both at the same time)

S(h,l,t)={(h,l,t)|hE}{(h,l,t)|tE}.

loss function L

Given margin hyperparameter γ>0 1, [x]+ denotes the positive part of x 2.

L=(h,l,t)S(h,l,t)S(h,l,t)[γ+d(h+l,t)d(h+l,t)]+

The loss function favors lower values of the energy for training triplets than for corrupted triplets, and thus a natural implementation of the intended criterion.

The optimization is carried out by stochastic gradient descent3 (in minibatch mode), over the possible h, l, t, with the additional constraints that the L2norm of the embeddings of the entities is 14(no regularization or norm constraints are given to the label embeddings l). It prevents the training process to trivially minimize L by artificially increasing entity embeddings norms.


  1. 一般设置为1
  2. 当值大于零,取本身;小于零,取0
  3. SGD, 随机梯度下降。这里是对一个batch求梯度之后就立即更新theta值
  4. 约束节点的嵌入(向量)的欧几里得距离为1,但是关系的嵌入不用约束

Spring Data JPA精讲教程

Spring 对 JPA 的支持非常强大,开发者只需关心核心业务逻辑的实现代码,无需过多关注 EntityManager 的创建、事务处理等 JPA 相关的处理,这基本上也是作为一个开发框架而言所能做到的极限了。至此,开发者连仅剩的实现持久层业务逻辑的工作都省了,唯一要做的,就只是声明持久层的接口,其他都交给 Spring Data JPA 来帮你完成!
  • 2017年06月29日 16:16

Translating Embeddings for Modeling Multi-relational Data 论文翻译:多元关系数据嵌入

摘要 1简介 2transE模型 3相关工作 4实验 1数据集 2实验设置 3链接预测 4用几个例子学习预测新关系 5总结和展望 摘要: 考虑多元关系数据得实体和关系在低维向量空间的嵌...
  • u012052268
  • u012052268
  • 2018-01-12 16:24:41
  • 190

知识图谱 论文

基于张量分解的知识图谱表示学习: Large-scale factorization of type-constrained multi-relational data Querying fact...
  • guotong1988
  • guotong1988
  • 2016-07-03 20:52:50
  • 2355

综述 | 知识图谱向量化表示

作者丨窦洪健 学校丨中国人民大学在读硕士 研究方向丨推荐系统、文本生成 本文经授权转载自知乎专栏「RUC AI Box」。 前言 知识图谱主要的目标是用来描述...
  • c9Yv2cf9I06K2A9E
  • c9Yv2cf9I06K2A9E
  • 2017-10-25 00:00:00
  • 1032

论文浅尝 |「知识表示学习」专题论文推荐

本文转载自公众号:PaperWeekly。本期论文清单来自清华大学博士生韩旭和北师大本科生曹书林,涵盖了近年知识表示学习方向的重要论文。[ 综述类 ]■ 论文 | Representation Lea...
  • TgqDT3gGaMdkHasLZv
  • TgqDT3gGaMdkHasLZv
  • 2018-02-12 00:00:00
  • 197

学习匈牙利算法总结(求解二分图最大匹配)

匈牙利算法就是求解二分图的最大匹配算法,
  • smileyk
  • smileyk
  • 2014-07-19 11:43:47
  • 564

Numbers in Multi-Relational Data Mining

  • 2008年04月16日 21:54
  • 151KB
  • 下载

Word embeddings in 2017: Trends and future directions (2017年里的词嵌入:趋势和未来方向)

Word embeddings in 2017: Trends and future directions 原文作者:anonymous 原文地址:http://ruder.io/word-emb...
  • Solo95
  • Solo95
  • 2018-02-05 18:16:58
  • 201

【网摘】Data Modeling 简介

原文:【网摘】Data Modeling 简介 Resource: Data Modeling Resource Center: http://infogoal.com/dmc/dmcdmd....
  • wolf96
  • wolf96
  • 2015-10-18 13:12:29
  • 1453

Introduction to multi-relational data mining

  • 2008年04月12日 18:04
  • 832KB
  • 下载
收藏助手
不良信息举报
您举报文章:《Translating Embeddings for Modeling Multi-relational Data》阅读笔记
举报原因:
原因补充:

(最多只允许输入30个字)