-
For example, if we want the model to acquire the knowledge of
“Paracetamol can treat cold”, a large number of cooccurrences of
”Paracetamol” and ”cold” are required in the pre-training
corpus. Instead of this strategy, what else can we do to make
the model a domain expert? The knowledge graph (KG), which was
called ontology in early research, serves as a good solution。
例如,如果我们想让模型获得“扑热息痛可以治疗感冒”的知识,那么在训练前的语料 库中就需要大量的“扑热息痛”和“感冒”同时出现。除了这个策略,我们还能做些什么让模型成为领域专家呢?知识图(KG)在早期的研究中被称为本体论,是一种很好的解决方案。 -
As shown in Figure 1, the model architecture of K-BERT consists of four modules, i.e., knowledge layer, embeddinglayer, seeing layer and mask-transformer.
如图1所示,××模型体系结构由××××层、××××层、××××层和××××四个模块组成 -
To some degree
在某种程度上 -
However, there are two challenges lies in the road of this knowledge integration: (1) Heterogeneous Embedding Space (HES): In general, the embedding vectors of words in text and entities in KG are obtained in separate ways, making their vector-space inconsistent; (2) Knowledge Noise (KN): Too much knowledge incorporation may divert the sentence from its correct meaning. To overcome these challenges, this paper propose a Knowledge-enabled Bidirectional Encoder Representation from Transformers (K-BERT).
然而,在×××××的道路上有两个挑战:(1)×××(2)××××××
English notes for KG
最新推荐文章于 2023-03-25 17:52:45 发布