整体而言,这篇文章做的是很贴近真实情景的,做的也比较细致
。
**未完**
待续
从多模态文本中抽取出N维关系类型。模态可能是纯文本,可能是表格。
涉及到两个问题,一是关系所在位置定位;二关系分类。
RESEL将这一具有挑战性的任务分解为两个较简单的子任务:(1)高层次的组件检索,旨在找到最终目标实体所在的目标段落/表格,以及(2)低层次的实体提取,旨在从选定的组件中选择目标实体。
For high-level component retrieval, we propose a
simple but effective feature-based model that combines multi-level semantic and lexical features between queries and components; (3) For low-level
entity extraction, we propose a multi-view architecture, which fuses graph-based structural relations
with BERT-based semantic information for extraction
Model
component 检索
Component-Level Semantic Features (CS).
(1) Embedding-Based Similarity: the cosine similarities fcs-1(Ci , Qj ) between component and query embeddings
Entailment-Based Score: the classification score fcs-2(Ci , Qj )
between Qj and Ci calculated by feeding them both into a BERT binary sequence classifier as a concatenated sequence
Entity-Level Semantic Features (ES)
entity-level cosine similarities fes(mi,b, ej,a) between the component entity embeddings h(mi,b) and the query elements embeddings h(ej,a). With all these similarity scores,we apply a max-pooling operation over all component entities mi,b, and use the obtained maximum fes(Ci
, ej,a) = maxmi,b∈Cifes(mi,b, ej,a) to represent the relation between the component Ci and one query element ej,a.
Entity-Level Lexical Features (EL)
three text similarities (Appendix A.2):
(1) Levenshtein Distance (Levenshtein et al., 1966); (2) the length of Longest Common Substring; (3) the length of Longest Common Subsequence.
实体关系分类
根据五种关系建议图:
(1)Co-occurence Edge测量两个实体节点vi和vj是否出现在同一个句子或相邻的句子中;(2)Coreference Edge提取两个实体节点vi和vj提及同一概念的关系信息;(3)Reference Edge用参考信息连接表格和文本(例如。“在表3中”);(4)表-结构边缘提取表的列和行的结构信息;(5)表-段落连接通过文本相似性加强表单元和段落实体之间的联系(详见附录A.3)。 有了这五种来自不同模式的边缘类型,几乎涵盖了文档中所有的隐藏关系,多模式实体关联图可以有效地模拟文档级信息。由于所有的边缘类型都在[0, 1]范围内,而且大多数都不重叠,所以我们对它们一视同仁,将图定义为无向同质图。
还有基于构建的图如何分类?
最后,是建立了两个分类器》(1) one classifier based on the concatenation of the entity nodes’ and query elements’ BERT embeddings, and (2) the other classifier based on the
GAT-updated BON features.