APER: AdaPtive Evidence-driven Reasoning Network for machine reading comprehension with unanswerable

论文全文链接

This is the paper that published in 2021 Knowledge-Based System. Impact Factor, 8.038 (2020).

动机

在解决不可回答任务上，先前的方法有两个问题：

1. First, most of them utilize a simple classifier
   or a verifiable module to determine whether a question is unanswerable, which lacks the explicit process of explanation.

2. Second, these methods treat the answer extraction task and the unanswerable MRC task as two
   independent tasks without considering the logical consistency of their results

贡献

1. An Evidence Refining Reasoner is designed to fuse questionrelevant information globally to refine the key evidence which is a basis for determining the non-answerability.

2. A novel logical consistency training objective is introduced to keep logical consistency of the results between the answer extraction task and the unanswerable MRC task.

3. Experimental results and ablation study on two datasets show that the APER has strong competitiveness and improves the Recall of unanswerable questions significantly.

做法

1. Global Encoder
   • 利用PLMs和task-specific global encoder 对上下文进行编码，获取 task-specific global semantic information
2. Evidence Refining Reasoner （the interaction between the passage and question is sometimes performs multiple times during the rereading process of human comprehension）
   • Evidence Refining Reasoner contains a stack of reasoning and fusion blocks, each of which contains a start and an end sub-block. See the figure below.
   • Finally, Evidence Refining Reasoner refines the key evidence information for subsequent prediction.

3. Answer Consistency Detector

   • outputs an unanswerable score based on the refined evidences (clues)
   • compute a consistency training loss to keep logical consistency between the answer extraction task and the unanswerable MRC task.

4. Training process
   

5. Dataset processing

实验

1. Dureader and Squad 2.0
   

2. Precision §, Recall ® and F1 of unanswerable questions
   

3. ablation study
   

4. Case study
   

5. Different question types

6. Steps of the reasoning