转载地址:https://www.cnblogs.com/bonelee/p/8534560.html
https://blog.csdn.net/sean2100/article/details/83834565
两篇文章主要给了两个结论:
结论二:应该仅在全连接层上使用Dropout操作,并在卷积层之间使用批量标准化(BN)。
*********************************************************************************************************************************************
结论一对应的内容
network = tflearn.input_data(shape=[None, max_len], name='input') network = tflearn.embedding(network, input_dim=volcab_size, output_dim=32) network = conv_1d(network, 64, 3, activation='relu', regularizer="L2") network = max_pool_1d(network, 2) network = conv_1d(network, 64, 3, activation='relu', regularizer="L2") network = max_pool_1d(network, 2) #network = conv_1d(network, 64, 3, activation='relu', regularizer="L2") #network = max_pool_1d(network, 2) network = batch_normalization(network) #network = fully_connected(network, 512, activation='relu') #network = dropout(network, 0.5) network = fully_connected(network, 64, activation='relu') network = dropout(network, 0.5) network = fully_connected(network, 2, activation='softmax')
迭代一次,acc是98.5%多一点。
如果使用:
# 关于一维CNN的网络,例子较少 # https://github.com/tflearn/tflearn/blob/master/examples/nlp/cnn_sentence_classification.py # Building convolutional network network = input_data(shape=[None, 100], name='input') network = tflearn.embedding(network, input_dim=10000, output_dim=128) branch1 = conv_1d(network, 128, 3, padding='valid', activation='relu', regularizer="L2") branch2 = conv_1d(network, 128, 4, padding='valid', activation='relu', regularizer="L2") branch3 = conv_1d(network, 128, 5, padding='valid', activation='relu', regularizer="L2") network = merge([branch1, branch2, branch3], mode='concat', axis=1) network = tf.expand_dims(network, 2) network = global_max_pool(network) network = dropout(network, 0.5) network = fully_connected(network, 2, activation='softmax') network = regression(network, optimizer='adam', learning_rate=0.001, loss='categorical_crossentropy', name='target') # Training model = tflearn.DNN(network, tensorboard_verbose=0)
acc是95%多一点点。
使用类似 vgg的模型, https://github.com/AhmetHamzaEmra/tflearn/blob/master/examples/images/VGG19.py
network = tflearn.input_data(shape=[None, max_len], name='input') network = tflearn.embedding(network, input_dim=volcab_size, output_dim=64) network = conv_1d(network, 64, 3, activation='relu') network = conv_1d(network, 64, 3, activation='relu') network = max_pool_1d(network, 2, strides=2) network = conv_1d(network, 128, 3, activation='relu') network = conv_1d(network, 128, 3, activation='relu') network = max_pool_1d(network, 2, strides=2) network = conv_1d(network, 256, 3, activation='relu') network = conv_1d(network, 256, 3, activation='relu') network = conv_1d(network, 256, 3, activation='relu') network = max_pool_1d(network, 2, strides=2) network = batch_normalization(network) network = fully_connected(network, 512, activation='relu') network = dropout(network, 0.5) network = fully_connected(network, 2, activation='softmax')
acc是98.5%多一点,稍微比第一种模型高,但是训练时间太长。
其他的,本质上都是加卷积层或者FC:
。。。 network = conv_1d(network, 64, 3, activation='relu', regularizer="L2") network = max_pool_1d(network, 2) network = conv_1d(network, 64, 3, activation='relu', regularizer="L2") network = max_pool_1d(network, 2) network = conv_1d(network, 64, 3, activation='relu', regularizer="L2") network = conv_1d(network, 64, 3, activation='relu', regularizer="L2") network = max_pool_1d(network, 2) 。。。
*****************************************************************************************************************************
结论二对应的内容
地址:https://blog.csdn.net/sean2100/article/details/83783632
应该仅在全连接层上使用Dropout操作,并在卷积层之间使用批量标准化。