读论文

文本分类：

• 《Bag of Tricks for Efficient Text Classification》，fastText文本分类
• 《Convolutional Neural Networks for Sentence Classification》，CNN文本分类经典论文
• 《A Sensitivity Analysis of (and Practitioners’ Guide to)
  Convolutional Neural Networks for Sentence Classification》，CNN文本分类，分析调参
• 《Recurrent Convolutional Neural Networks for Text Classification》，RCNN文本分类

# RCNN
left_train_word_ids = [[len(vocab)] + x[:-1] for x in X_train_word_ids]
left_test_word_ids = [[len(vocab)] + x[:-1] for x in X_test_word_ids]
right_train_word_ids = [x[1:] + [len(vocab)] for x in X_train_word_ids]
right_test_word_ids = [x[1:] + [len(vocab)] for x in X_test_word_ids]

left_train_padded_seqs = pad_sequences(left_train_word_ids, maxlen=20)
left_test_padded_seqs = pad_sequences(left_test_word_ids, maxlen=20)
right_train_padded_seqs = pad_sequences(right_train_word_ids, maxlen=20)
right_test_padded_seqs = pad_sequences(right_test_word_ids, maxlen=20)

document = Input(shape=(None,), dtype='int32')
left_context = Input(shape=(None,), dtype='int32')
right_context = Input(shape=(None,), dtype='int32')

embedder = Embedding(len(vocab) + 1, 300, input_length=20)
doc_embedding = embedder(document)
l_embedding = embedder(left_context)
r_embedding = embedder(right_context)

forward = LSTM(256, return_sequences=True)(l_embedding)
backward = LSTM(256, return_sequences=True, go_backwards=True)(r_embedding)
together = concatenate([forward, doc_embedding, backward], axis=2)
semantic = TimeDistributed(Dense(128, activation='tanh'))(together)
pool_rnn = Lambda(lambda x: K.max(x, axis=1), output_shape=(128,))(semantic)
output = Dense(10, activation='softmax')(pool_rnn)
model = Model(inputs=[document, left_context, right_context], outputs=output)
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
print(model.summary())

• 《Hierarchical Attention Networks for Document Classification》，HAN文本分类
• 《Enriching Word Vectors with Subword Information》，word vector from subword

自然语言推理：

• 《A Decomposable Attention Model for Natural Language Inference》

• 《Enhanced LSTM for Natural Language Inference》，HIM（Hybrid Inference Model），ESIM（Enhanced Sequential Inference Model），code：https://www.kaggle.com/lamdang/dl-models

• 《Natural Language Inference over Interaction Space》，DIIN（densely-connected co-attentive recurrent neural network），code：https://github.com/YichenGong/Densely-Interactive-Inference-Network ，keras code：https://github.com/YerevaNN/DIIN-in-Keras

• 《Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information》，DRCN（Densely-connected Recurrent and Co-attentive neural Network）

seq2seq

• 《Attention Is All You Need》

embedding

• 《BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding》
• 《Frustratingly Easy Meta-Embedding – Computing Meta-Embeddings by Averaging Source Word Embeddings》
• 《Dynamic Meta-Embeddings for Improved Sentence Representations》
• 《Deep contextualized word representations》