相关参考:
https://www.cnblogs.com/bonelee/p/8534560.html
https://blog.csdn.net/sean2100/article/details/83834565
https://blog.csdn.net/sean2100/article/details/83783632
两篇文章主要给了两个结论:
结论一:CNN 文本分类模型优化经验——关键点:加卷积层和FC可以提高精度,在FC前加BN可以加快收敛,有时候可以提高精度,FC后加dropout,conv_1d的input维度加大可以提高精度,但是到256会出现OOM。
结论二:应该仅在全连接层上使用Dropout操作,并在卷积层之间使用批量标准化(BN)。
*********************************************************************************************************************************************
结论一对应的内容
network = tflearn.input_data(shape=[None, max_len], name='input')
network = tflearn.embedding(network, input_dim=volcab_size, output_dim=32)
network = conv_1d(network, 64, 3, activation='relu', regularizer="L2")
network = max_pool_1d(network, 2)
network = conv_1d(network, 64, 3, activation='relu', regularizer="L2")
network = max_pool_1d(network, 2)
#network = conv_1d(network, 64, 3, activation='relu', regularizer="L2")
#network = max_pool_1d(network, 2)
network = batch_normalization(network)
#network = fully_connected(network, 512, activation='relu')
#network = dropout(network, 0.5)
network = fully_connected(network, 64, activation='relu')
network = dropout(network, 0.5)
network = fully_connected(network, 2, activation='softmax')
迭代一次,acc是98.5%多一点。
如果使用:
# 关于一维CNN的网络,例子较少
# https://github.com/tflearn/tflearn/blob/master/examples/nlp/cnn_sentence_classification.py
# Building convolutional network
network = input_data(shape=[None, 100], name='input')
network = tflearn.embedding(network, input_dim=10000, output_dim=128)
branch1 = conv_1d(network, 128, 3, padding='valid', activation='relu', regularizer="L2")
branch2 = conv_1d(network, 128, 4, padding='valid', activation='relu', regularizer="L2")
branch3 = conv_1d(network, 128, 5, padding='valid', activation='relu', regularizer="L2")
network = merge([branch1, branch2, branch3], mode='concat', axis=1)
network = tf.expand_dims(network, 2)
network = global_max_pool(network)
network = dropout(network, 0.5)
network = fully_connected(network, 2, activation='softmax')
network = regression(network, optimizer='adam', learning_rate=0.001,
loss='categorical_crossentropy', name='target')
# Training
model = tflearn.DNN(network, tensorboard_verbose=0)
acc是95%多一点点。
使用类似 vgg的模型, https://github.com/AhmetHamzaEmra/tflearn/blob/master/examples/images/VGG19.py
network = tflearn.input_data(shape=[None, max_len], name='input')
network = tflearn.embedding(network, input_dim=volcab_size, output_dim=64)
network = conv_1d(network, 64, 3, activation='relu')
network = conv_1d(network, 64, 3, activation='relu')
network = max_pool_1d(network, 2, strides=2)
network = conv_1d(network, 128, 3, activation='relu')
network = conv_1d(network, 128, 3, activation='relu')
network = max_pool_1d(network, 2, strides=2)
network = conv_1d(network, 256, 3, activation='relu')
network = conv_1d(network, 256, 3, activation='relu')
network = conv_1d(network, 256, 3, activation='relu')
network = max_pool_1d(network, 2, strides=2)
network = batch_normalization(network)
network = fully_connected(network, 512, activation='relu')
network = dropout(network, 0.5)
network = fully_connected(network, 2, activation='softmax')
acc是98.5%多一点,稍微比第一种模型高,但是训练时间太长。
其他的,本质上都是加卷积层或者FC:
。。。
network = conv_1d(network, 64, 3, activation='relu', regularizer="L2")
network = max_pool_1d(network, 2)
network = conv_1d(network, 64, 3, activation='relu', regularizer="L2")
network = max_pool_1d(network, 2)
network = conv_1d(network, 64, 3, activation='relu', regularizer="L2")
network = conv_1d(network, 64, 3, activation='relu', regularizer="L2")
network = max_pool_1d(network, 2)
。。。
*****************************************************************************************************************************
结论二对应的内容
地址:https://blog.csdn.net/sean2100/article/details/83783632
应该仅在全连接层上使用Dropout操作,并在卷积层之间使用批量标准化。