《Python 深度学习》刷书笔记 Chapter 4 关于电影评论模型的进一步探讨

最新推荐文章于 2024-10-04 20:30:00 发布

FeverTwice

最新推荐文章于 2024-10-04 20:30:00 发布

阅读量217

点赞数

分类专栏： # 《Python 深度学习》刷书笔记 Python之路文章标签： python 深度学习 pytorch 神经网络

本文链接：https://blog.csdn.net/LeungSr/article/details/120610389

版权

Python之路同时被 2 个专栏收录

25 篇文章 1 订阅

订阅专栏

《Python 深度学习》刷书笔记

20 篇文章 9 订阅

订阅专栏

电影评论模型的进一步改进

我们将在这一节使用实验的方式探讨电影评论模型的改进方法，在此之前，我们先总结几个知识点

验证集和测试集的区别

为什么不能把他们放在同一个集合里面？—信息泄露
在调节超参数的时候，我们可能会把验证集中的某些特征泄露到这些超参数里面，从而使得模型的性能看起来是提高了，但是在面对完全陌生的数据集的时候，却不一定能够表现很好的泛化性（存在过拟合的可能）
所以我们在训练模型时候，如果基于测试集来调节模型，那么对泛化能力的衡量是不准确的

评估模型时候需要注意的事项

数据代表性：有些样本标签可能完全没有出现在训练集里（需要打乱数据）
时间箭头：有时序性（不能打乱数据）
数据冗余：在样本中可能出现数据缺失的情况，但是我们却把它考虑为0，导致模型不准确

# 加载数据集
import os
os.environ['KERAS_BACKEND']='tensorflow'
from keras.datasets import imdb

# 仅保留训练集数据中前10k个最常出现的单词
(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words = 10000)

# 将整数序列编码为二进制矩阵
import numpy as np

def vectorize_sequence(sequences, dimension = 10000):
    # 创建一个形状为(len(sequences), dimension)的零矩阵
    results = np.zeros((len(sequences), dimension)) # 创建矩阵
    
    for i , sequences in enumerate(sequences):
        results[i, sequences] = 1. # 将results[i]的指定索引设置为1
    
    return results

x_train = vectorize_sequence(train_data) # 将训练数据向量化
x_test = vectorize_sequence(test_data)   # 将测试数据向量化

print(x_train[0])

# 标签向量化
y_train = np.asarray(train_labels).astype('float32')
y_test = np.asarray(test_labels).astype('float32')

[0. 1. 1. ... 0. 0. 0.]

4-3 原始模型

# 原始模型
from keras import models
from keras import layers

model = models.Sequential()
model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))
model.add(layers.Dense(16, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

4-4 容量更小的模型

from keras import models
from keras import layers

model = models.Sequential()
model.add(layers.Dense(4, activation='relu', input_shape=(10000,)))
model.add(layers.Dense(4, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

4-5 容量更大的模型

from keras import models
from keras import layers

model = models.Sequential()
model.add(layers.Dense(512, activation='relu', input_shape=(10000,)))
model.add(layers.Dense(512, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

4-6 向模型中添加L2权重正则化

from keras import regularizers


# l2（0.001）的意思是权重矩阵的每个系数都会使得网络的总损失增加0.001 * weight_coefficient_value
# 注意这里是 L2， 不是十二
# 注意这个惩罚一般只在训练的时候加
l2_model = models.Sequential()
l2_model.add(layers.Dense(16, kernel_regularizer = regularizers.l2(0.001),
                          activation = 'relu', input_shape = (10000,)))

l2_model.add(layers.Dense(16, kernel_regularizer = regularizers.l2(0.001),
                          activation = 'relu'))

l2_model.add(layers.Dense(1, activation = 'sigmoid'))

# 编译器
l2_model.compile(optimizer='rmsprop',
                 loss='binary_crossentropy',
                 metrics=['acc'])

# 训练
l2_model_hist = l2_model.fit(x_train, y_train,
                             epochs=20,
                             batch_size=512,
                             validation_data=(x_test, y_test))

Train on 25000 samples, validate on 25000 samples
Epoch 1/20
25000/25000 [==============================] - 4s 151us/step - loss: 0.5189 - acc: 0.7970 - val_loss: 0.3919 - val_acc: 0.8752
Epoch 2/20
25000/25000 [==============================] - 3s 123us/step - loss: 0.3237 - acc: 0.9018 - val_loss: 0.3371 - val_acc: 0.8889
Epoch 3/20
25000/25000 [==============================] - 3s 124us/step - loss: 0.2725 - acc: 0.9209 - val_loss: 0.3340 - val_acc: 0.8869
Epoch 4/20
25000/25000 [==============================] - 3s 136us/step - loss: 0.2511 - acc: 0.9289 - val_loss: 0.3444 - val_acc: 0.8824
Epoch 5/20
25000/25000 [==============================] - 3s 126us/step - loss: 0.2372 - acc: 0.9320 - val_loss: 0.3748 - val_acc: 0.8729
Epoch 6/20
25000/25000 [==============================] - 3s 128us/step - loss: 0.2288 - acc: 0.9369 - val_loss: 0.4030 - val_acc: 0.8648
Epoch 7/20
25000/25000 [==============================] - 3s 129us/step - loss: 0.2244 - acc: 0.9386 - val_loss: 0.3587 - val_acc: 0.8810
Epoch 8/20
25000/25000 [==============================] - 3s 130us/step - loss: 0.2187 - acc: 0.9414 - val_loss: 0.3753 - val_acc: 0.8764
Epoch 9/20
25000/25000 [==============================] - 3s 127us/step - loss: 0.2099 - acc: 0.9459 - val_loss: 0.4297 - val_acc: 0.8572
Epoch 10/20
25000/25000 [==============================] - 3s 123us/step - loss: 0.2116 - acc: 0.9431 - val_loss: 0.3970 - val_acc: 0.8706
Epoch 11/20
25000/25000 [==============================] - 3s 121us/step - loss: 0.2068 - acc: 0.9465 - val_loss: 0.3857 - val_acc: 0.8754
Epoch 12/20
25000/25000 [==============================] - 3s 123us/step - loss: 0.2029 - acc: 0.9465 - val_loss: 0.3937 - val_acc: 0.8735
Epoch 13/20
25000/25000 [==============================] - 3s 123us/step - loss: 0.1981 - acc: 0.9495 - val_loss: 0.4351 - val_acc: 0.8633
Epoch 14/20
25000/25000 [==============================] - 3s 122us/step - loss: 0.1976 - acc: 0.9471 - val_loss: 0.4027 - val_acc: 0.8700
Epoch 15/20
25000/25000 [==============================] - 3s 124us/step - loss: 0.1883 - acc: 0.9533 - val_loss: 0.4374 - val_acc: 0.8616
Epoch 16/20
25000/25000 [==============================] - 3s 121us/step - loss: 0.1909 - acc: 0.9504 - val_loss: 0.4139 - val_acc: 0.8706
Epoch 17/20
25000/25000 [==============================] - 3s 122us/step - loss: 0.1863 - acc: 0.9524 - val_loss: 0.4338 - val_acc: 0.8640
Epoch 18/20
25000/25000 [==============================] - 3s 121us/step - loss: 0.1861 - acc: 0.9525 - val_loss: 0.4250 - val_acc: 0.8674
Epoch 19/20
25000/25000 [==============================] - 3s 125us/step - loss: 0.1884 - acc: 0.9518 - val_loss: 0.4287 - val_acc: 0.8690
Epoch 20/20
25000/25000 [==============================] - 3s 122us/step - loss: 0.1793 - acc: 0.9550 - val_loss: 0.4548 - val_acc: 0.8645

# 传入定义好的交叉熵函数
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])

from keras import optimizers

model.compile(optimizer=optimizers.RMSprop(lr=0.001),
              loss='binary_crossentropy',
              metrics=['accuracy'])

# 优化配置器
from keras import optimizers

model.compile(optimizer=optimizers.RMSprop(lr=0.001),
              loss='binary_crossentropy',
              metrics=['accuracy'])

# 留出验证集
x_val = x_train[:10000]
partial_x_train = x_train[10000:]

y_val = y_train[:10000]
partial_y_train = y_train[10000:]

# 训练模型
model.compile(optimizer = 'rmsprop',loss = 'binary_crossentropy', metrics = ['acc'])

history = model.fit(partial_x_train, 
                    partial_y_train, 
                    epochs = 20, batch_size = 512, 
                    validation_data = (x_val, y_val))

Train on 15000 samples, validate on 10000 samples
Epoch 1/20
15000/15000 [==============================] - 2s 118us/step - loss: 0.5012 - acc: 0.7831 - val_loss: 0.3668 - val_acc: 0.8692
Epoch 2/20
15000/15000 [==============================] - 1s 98us/step - loss: 0.2911 - acc: 0.9023 - val_loss: 0.3168 - val_acc: 0.8738
Epoch 3/20
15000/15000 [==============================] - 1s 95us/step - loss: 0.2132 - acc: 0.9294 - val_loss: 0.2865 - val_acc: 0.8841
Epoch 4/20
15000/15000 [==============================] - 1s 96us/step - loss: 0.1663 - acc: 0.9468 - val_loss: 0.2770 - val_acc: 0.8894
Epoch 5/20
15000/15000 [==============================] - 2s 105us/step - loss: 0.1330 - acc: 0.9581 - val_loss: 0.3133 - val_acc: 0.8802
Epoch 6/20
15000/15000 [==============================] - 1s 99us/step - loss: 0.1107 - acc: 0.9646 - val_loss: 0.3048 - val_acc: 0.8808
Epoch 7/20
15000/15000 [==============================] - 1s 96us/step - loss: 0.0894 - acc: 0.9723 - val_loss: 0.3368 - val_acc: 0.8804
Epoch 8/20
15000/15000 [==============================] - 1s 98us/step - loss: 0.0745 - acc: 0.9787 - val_loss: 0.3552 - val_acc: 0.8728
Epoch 9/20
15000/15000 [==============================] - 2s 101us/step - loss: 0.0609 - acc: 0.9841 - val_loss: 0.3750 - val_acc: 0.8746
Epoch 10/20
15000/15000 [==============================] - 1s 95us/step - loss: 0.0479 - acc: 0.9877 - val_loss: 0.4036 - val_acc: 0.8724
Epoch 11/20
15000/15000 [==============================] - 1s 95us/step - loss: 0.0382 - acc: 0.9920 - val_loss: 0.5095 - val_acc: 0.8621
Epoch 12/20
15000/15000 [==============================] - 1s 97us/step - loss: 0.0312 - acc: 0.9935 - val_loss: 0.4559 - val_acc: 0.8732
Epoch 13/20
15000/15000 [==============================] - 1s 97us/step - loss: 0.0238 - acc: 0.9955 - val_loss: 0.6070 - val_acc: 0.8572
Epoch 14/20
15000/15000 [==============================] - 1s 96us/step - loss: 0.0164 - acc: 0.9977 - val_loss: 0.5313 - val_acc: 0.8662
Epoch 15/20
15000/15000 [==============================] - 1s 96us/step - loss: 0.0168 - acc: 0.9962 - val_loss: 0.5706 - val_acc: 0.8638
Epoch 16/20
15000/15000 [==============================] - 1s 96us/step - loss: 0.0134 - acc: 0.9973 - val_loss: 0.5975 - val_acc: 0.8682
Epoch 17/20
15000/15000 [==============================] - 1s 100us/step - loss: 0.0063 - acc: 0.9997 - val_loss: 0.6466 - val_acc: 0.8682
Epoch 18/20
15000/15000 [==============================] - 1s 95us/step - loss: 0.0113 - acc: 0.9971 - val_loss: 0.6640 - val_acc: 0.8664
Epoch 19/20
15000/15000 [==============================] - 1s 98us/step - loss: 0.0035 - acc: 0.9999 - val_loss: 0.6934 - val_acc: 0.8655
Epoch 20/20
15000/15000 [==============================] - 1s 97us/step - loss: 0.0095 - acc: 0.9981 - val_loss: 0.7261 - val_acc: 0.8647