bAbI Question Answering keras实践(1) pure RNN & Memory Network

bAbI Question Answering keras实践(1) pure RNN & Memory Network

参考: keras babi_rnn.py, keras babi_memnn.py, bAbI - Facebook.

bAbI tasks 是为了测试机器学习算法对自然语言文本的理解和逻辑能力(text understanding and reasoning) 的一组数据。有20组数据，分别考察记忆力、方向、词汇关系、逻辑推导等能力。

第一组数据为: single supporting fact，节选如下：

1 Mary moved to the bathroom.
2 John went to the hallway.
3 Where is Mary? 	bathroom	1
4 Daniel went back to the hallway.
5 Sandra moved to the garden.
6 Where is Daniel? 	hallway	4
7 John moved to the office.
8 Sandra journeyed to the bathroom.
9 Where is Daniel? 	hallway	4

其中句首数字为ID，无问号的内容为故事(story)，有问号的一行被 \t 拆分为三部分：问题(query)，答案(answer), 以及确证ID(supporting)即该ID的句子可直接解释结果。

keras的案例中分别使用的纯RNN以及记忆网络(memory network)建立模型，结果为：仅使用LSTM的训练准确度最高只有52%左右，而使用memory network approach的训练准确率达到了96%左右。

数据处理

keras 的例子中建立了四个函数以处理文本，分别是：
tokenize()：将一句话(str) 拆分为由单词组成的list
parse_stories()：输入多行文本，将story，query，answer，supporting拆分并拼装成以多个元组(story, q, a)组成的list
get_stories()：调用上面两个函数转化文本为可处理的列表
vectorize_stories()：将词汇转换成数字编码

Pure RNN

结构如下：

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_3 (InputLayer)            (None, 68)           0                                            
__________________________________________________________________________________________________
input_4 (InputLayer)            (None, 4)            0                                            
__________________________________________________________________________________________________
embedding_4 (Embedding)         (None, 68, 50)       1100        input_3[0][0]                    
__________________________________________________________________________________________________
embedding_5 (Embedding)         (None, 4, 50)        1100        input_4[0][0]                    
__________________________________________________________________________________________________
lstm_2 (LSTM)                   (None, 100)          60400       embedding_4[0][0]                
__________________________________________________________________________________________________
lstm_3 (LSTM)                   (None, 100)          60400       embedding_5[0][0]                
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 200)          0           lstm_2[0][0]                     
                                                                 lstm_3[0][0]                     
__________________________________________________________________________________________________
dense_2 (Dense)                 (None, 22)           4422        concatenate_2[0][0]              
================================================================================&