记忆神经网络

记忆神经网络

原来图片是这样的：

现在图片是这样的，就比较好理解了。（当然只是单层的实现）

keras 实现如下：

		input_base = layers.Input(shape=(self.sent_len*5,))
        input_question = layers.Input(shape=(self.sent_len,))

        x_base_embeding = layers.Embedding(input_dim=self.word_dic_len, output_dim=self.embeding_len)(input_base)
        x_q_embeding = layers.Embedding(input_dim=self.word_dic_len, output_dim=self.embeding_len)(input_question)

        x_base = layers.Bidirectional(layers.LSTM(units=self.unit_len, return_sequences=True))(x_base_embeding)
        x_q = layers.Bidirectional(layers.LSTM(units=self.unit_len, return_sequences=True))(x_q_embeding)

        x_base2 = layers.Bidirectional(layers.LSTM(units=15 ,return_sequences=True))(x_base_embeding)

        w2w_dot = layers.dot([x_base, x_q], axes=(2,2))
        w2w_dot_soft = layers.Dense(activation="softmax", units=30)(w2w_dot)

        base_sum = layers.add([x_base2, w2w_dot_soft])
        base_sum = layers.Permute((2,1))(base_sum)
        total = layers.concatenate([base_sum, x_q], axis=-1) #转换

        predict_lstm = layers.Bidirectional(layers.LSTM(units=128))(total)

        predict = layers.Dense(units=self.class_num, activation="softmax")(predict_lstm)

        model = Model([input_base, input_question], predict)
        model.compile(optimizer='rmsprop', loss='categorical_crossentropy',
                      metrics=['accuracy'])
        model.summary()

总结：
记忆网络可以看做一个框架，用于QA或者分类等。