bAbI Question Answering keras实践(1) pure RNN & Memory Network
参考: keras babi_rnn.py, keras babi_memnn.py, bAbI - Facebook.
bAbI tasks 是为了测试机器学习算法对自然语言文本的理解和逻辑能力(text understanding and reasoning) 的一组数据。有20组数据,分别考察记忆力、方向、词汇关系、逻辑推导等能力。
第一组数据为: single supporting fact,节选如下:
1 Mary moved to the bathroom.
2 John went to the hallway.
3 Where is Mary? bathroom 1
4 Daniel went back to the hallway.
5 Sandra moved to the garden.
6 Where is Daniel? hallway 4
7 John moved to the office.
8 Sandra journeyed to the bathroom.
9 Where is Daniel? hallway 4
其中句首数字为ID,无问号的内容为故事(story),有问号的一行被 \t
拆分为三部分:问题(query),答案(answer), 以及确证ID(supporting)即该ID的句子可直接解释结果。
keras的案例中分别使用的纯RNN以及记忆网络(memory network)建立模型,结果为:仅使用LSTM的训练准确度最高只有52%左右,而使用memory network approach的训练准确率达到了96%左右。
数据处理
keras 的例子中建立了四个函数以处理文本,分别是:
tokenize()
:将一句话(str) 拆分为由单词组成的list
parse_stories()
:输入多行文本,将story,query,answer,supporting拆分并拼装成以多个元组(story, q, a)组成的list
get_stories()
:调用上面两个函数转化文本为可处理的列表
vectorize_stories()
:将词汇转换成数字编码
Pure RNN
结构如下:
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_3 (InputLayer) (None, 68) 0
__________________________________________________________________________________________________
input_4 (InputLayer) (None, 4) 0
__________________________________________________________________________________________________
embedding_4 (Embedding) (None, 68, 50) 1100 input_3[0][0]
__________________________________________________________________________________________________
embedding_5 (Embedding) (None, 4, 50) 1100 input_4[0][0]
__________________________________________________________________________________________________
lstm_2 (LSTM) (None, 100) 60400 embedding_4[0][0]
__________________________________________________________________________________________________
lstm_3 (LSTM) (None, 100) 60400 embedding_5[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, 200) 0 lstm_2[0][0]
lstm_3[0][0]
__________________________________________________________________________________________________
dense_2 (Dense) (None, 22) 4422 concatenate_2[0][0]
================================================================================&