KBQA 常用的问答数据集之 ComplexQuestions

目录

1. 论文相关

2. 数据集概述

   2.1 内容介绍 

   2.2 数据统计

3. 模型性能比较

---

1. 论文相关

ComplexQuestions [Bao et al., 2016]

源自论文：Constraint-Based Question Answering with Knowledge Graph

数据集：https://github.com/JunweiBao/MulCQA/tree/ComplexQuestions

2. 数据集概述

   2.1 内容介绍 

   基于Freebase，执行一些操作来选择合适的多约束问题(Multi-Constraint Questions)，供human annotator 进行标注。

    Multi-Constraint Questions被定义为一个需要多个KB关系或特殊操作才能得到答案的问题。基于web查询分析，约束条件分可为以下6类:

    (1) Multi-entity constraint

    (2) Type constraint

    (3) Explicit temporal constraint

    (4) Implicit temporal constraint

    (5) Ordinal constraint

    (6) Aggregation constraint
   

    ComplexQuestions数据集字段：

   2.2 数据统计

     ComplexQuestions 数据集中有2100个问答对，来源于三个地方：

     ① 有596个问答对来自WebQuestions 训练集,有326 来自 WebQuestions 测试集。

     ② 有300个问答对来自 [Yin et al., 2015]。

     ③ 有878个问答对来自人工标注(参考2.1 提到的标注)。

      ComplexQuestions数据集的训练集和测试集的划分情况如下：

ComplexQuestons数据集划分

total	2100
training set	1300
test set	800

3. 模型性能比较

各模型在数据集ComplexQuestions上的表现

模型(年份)	F1	论文	代码链接
MulCG(2016)	40.94	Constraint-Based Question Answering with Knowledge Graph
QUINT(2017)	49.2	Automated Template Generation for Question Answering over Knowledge Graphs
CompQA(2018)	42.8	Knowledge Base Question Answering via Encoding of Complex Query Graphs
STF(2018)	54.3	A State-transition Framework to Answer Complex Questions over Knowledge Base
QGG(2020)	43.3	Query Graph Generation for Answering Multi-hop Complex Questions from Knowledge Bases	GitHub - lanyunshi/Multi-hopComplexKBQA
DAC(2020)	45.0	Hierarchical Query Graph Generation for Complex Question Answering over Knowledge Graph
AQG(2020)	43.1	Formal Query Building with Query Structure Prediction for Complex Question Answering over Knowledge Base	https://github.com/Bahuia/AQGNet

后续将持续更新，欢迎大家评论和补充~