KBQA 常用的问答数据集之WebQuestions

目录

1. 论文相关

2. 数据集概述

2.1 内容介绍

2.2 数据统计

3. 模型性能比较

---

1. 论文相关

WebQuestions [Berant et al., 2013]

源自论文：Semantic Parsing on Freebase from Question-Answer Pairs

数据集：The Stanford Natural Language Processing Group

leaderboard: CodaLab Worksheets

2. 数据集概述

   2.1 内容介绍

    这个数据集经常用于semantic parsing 和question answering；其使用的知识库是Freebase。

    每个examples 有三个fields:

    utterance: 自然语言问句。

    targetValue: 答案。

    url：AMT工作者可以从Freebase 页找到答案。

   2.2 数据统计

    数据集规模虽然较FREE917提高了不少，但有两个突出的缺陷：没有提供对应的查询，不利于基于逻辑表达式模型的训练；另外webquestions中简单问句多而复杂问句少。

WebQuestions数据集划分

total	5,810
train	3,778
test	2,032

3. 模型性能比较

各模型在数据集WebQuestions上的表现

模型(年份)	prec	Rec	Acc	F1	论文	代码链接
GraphParser (2014)	41.9	37.0		39.3	Large-scale Semantic Parsing without Question-Answer Pairs	Downloads | Siva Reddy
STAGG (2015)	52.8	60.7		52.5	Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base	https://github.com/scottyih/STAGG
Aqqu (2015)				49.4	More Accurate Question Answering on Freebase	Publications — Professur für Algorithmen und Datenstrukturen
QAoverFB (2016)				53.3	Question Answering on Freebase via Relation Extraction and Textual Evidence	GitHub - syxu828/QuestionAnsweringOverFB
MulCG (2016)				52.43	Constraint-Based Question Answering with Knowledge Graph
QUINT (2017)				51.0	Automated Template Generation for Question Answering over Knowledge Graphs
CompQA (2018)				52.7	Knowledge Base Question Answering via Encoding of Complex Query Graphs
APVA-TURBO (2018)				63.4	The APVA-TURBO Approach To Question Answering in Knowledge Base
ABWIM (2018)			85.32		An Attention-Based Word-Level Interaction Model: Relation Detection for Knowledge Base Question Answering
STF(2018)				53.6	A State-transition Framework to Answer Complex Questions over Knowledge Base
NFF(2018)				49.6	Answering Natural Language Questions by Subgraph Matching over Knowledge Graphs	https://github.com/pkumod/gAnswer
Tree2Seq (2019)				52.1	Knowledge-based question answering by tree-to-sequence learning
BAMnet (2019)				55.7	Bidirectional Attentive Memory Networks for Question Answering over Knowledge Bases	GitHub - hugochan/BAMnet: Code & data accompanying the NAACL 2019 paper "Bidirectional Attentive Memory Networks for Question Answering over Knowledge Bases"
DAC(2020)				54.8	Hierarchical Query Graph Generation for Complex Question Answering over Knowledge Graph
AQG(2020)				53.4	Formal Query Building with Query Structure Prediction for Complex Question Answering over Knowledge Base	https://github.com/Bahuia/AQGNet

后续将持续更新，欢迎大家评论和补充~