Notes

Notes1:

Knowledge Transfer for Out-of-Knowledge-Base Entities:A Graph Neural Network Approach

Introduction

This paper address the out-of-knowledge-base(OOKB) entity problem in KBC: how to answer queries concering
test entities not observed at training time.

Setting and problems

Existing embedding-based KBC models assume that all test entities are available at training time

OOKB entity problem arise when new entities(OOKB entities) occur in the relation triplets that are given to the system after training.

To solve the OOKB entity problem without retraining,we use graph neural networks(Graph-NNs) to compute the embeddings of OOKB entities,exploiting the limited auxiliary knowledge provided at the test time.

OOKB Entity Problem

Let $E$ be a set of entities,and $R$ be a set of relations.

In addtion to knowledge base G observed at training time,new triplets $G_{aux}$ are given at test time,with
$E(G_{aux}) \not \subset E(G)$ and $R(G_{aux}) \subseteq R(G)$.Thus,$G_{aux}$ contains new entities $E_{OOKB} = E(G_{aux}) \setminus E(G)$,but no new relations are involved.

triplets in $G_{aux}$
Every triplet in $G_{aux}$ contains exactly one OOKB entity fro $E_{OOKB}$ and one entity from $E(G)$;that is ,the additional triplets $G_{aux}$ represent edges bridging $E(G)$ and $E_{OOKB}$ in the combined knowledge graph $G \cup G_{aux}$.

Motivation

want to use the information already have in $G$ to deal with OOKB entities,with the help of the added knowledge graph $G_{aux}$

Graph-NN Model

Propagation Model on a Knowledge Graph

Let G be a knowledge graph, $e \in E(G)$ be an entity,and $v_{e} \in R^d$ be the d-dimensional representation vector of e.

The propagation model definition:
$v_e = \sum_{(h,r,e) \in \mathcal{N}_{head}(e) } T_{head}(v_h;h,r,e) + \sum_{(e,r,t) \in \mathcal{N}_{tail}(e)} T_{tail}(v_t;e,r,t)$

where head neighborhood $\mathcal{N}_{head}$ and tail neighborhood $\mathcal{N}_{tail}$ are $\mathcal{N}_{head}(e) = { \{ (h,r,e) \mid (h,r,e) \in G \}}$ and $\mathcal{N}_{tail}(e) = { \{ (e,r,t) \mid (e,r,t) \in G \}}$

Modified Model(in this paper):

$S_{head}(e) = \{ T_{head}(v_h,h,r,e) \mid (h,r,e) \in \mathcal{N}_{h}(e) \}$
$S_{tail}(e) = \{ T_{tail}(v_t,e,r,t) \mid (e,r,t) \in \mathcal{N}_{t}(e) \}$
$v_e = P(S_{head}(e) \cup S_{tail}(e))$
Transition Function used in experiment:
the aim of transition function T is to modify the vector of a neighbor node to reflect the relations between the current node and the neighbor.
$T_{head}(v_h;h,r,e) = ReLU(BN(A_r^{head}v_h))$
$T_{tail}(v_t;h,r,e) = ReLu(BN(A_r^{tail}v_t))$

where $A \in R^{(d\times d)}$ is a matrix of model parameters,and BN indicates batch normalization.

Pooling Function:
To extract share aspects from a set of vectors.

$P(S) = \sum_{i=1}^{N} x_i$ sum pooling
$P(S) = \frac{1}{N} \sum_{i=1}^{N} x_i$ average pooling
$P(S) = max (\{ x_i\}_{i=1}^{N})$ max pooling

Output Model

Using a TransE-based objective function as the output model.

Absolute-Margin Objective Function:

$\mathcal{L} = \sum_{i=1}^{N} f(h_i,r_i,t_i) + [ \tau - f(h_i',r_i',t_i') ]_{+}$

OOKB Entity Experiment

construct dataset

1.choosing OOKB entities.Selected N=1000,3000,5000 triplets from WN11 test file.
Head setting:all head entities in the N triplets are regarded as candidate OOKB entities.
Tail setting is similar,but with the tail entities regarded as candidates.
In the Both setting,all entities appearing as either a head or tail are the candidates.

2.Filtering and spliting triplets.Using the OOKB entities, the original training dataset
was split into the training dataset and the auxiliary dataset.The OOKB training dataset(triplets) did not
contain OOKB entities and the auxiliary dataset(triplets) contain one OOKB entity and non-OOKB entity.

For the test triplets,they used the same first N triplets in the WN11 test file that used in Step 1 and the
triplets did not contain any OOKB entities were removed. For validation triplets they simply removed
the triplets containing OOKB entities from WN11 valid file.

github address: http://github.com/takou-h/GNN-for-OOKB.