一、简单循环神经网络
任务描述
本关任务:搭建一个简单循环神经网络模型。
相关知识
为了完成本关任务,你需要掌握: 1.什么是循环神经网络 2.如何定义简单的循环神经网络 3.代码实现定义网络
循环网络简介
RNN处理序列的方式是,遍历所有序列元素,并保存一个状态,其中包含与已查看内容相关的信息。实际上,RNN是一类具有内部环的神经网络(如下图)。在处理两个不同的独立序列(比如两条不同的IMDB评论)之间,RNN状态会被重置,因此,你仍可以将一个序列看作单个数据点,即网络的单个输入。真正改变的是,数据点不再是在单个步骤中进行处理,相反,网络内部会对序列元素进行遍历。
Keras中的循环层
在Keras中能够通过以下代码实现一个简单的RNN层(该步骤不需要学员实现),请复制代码到.ipynb文件中并运行,代码如下:
from keras.models import Sequential
from keras.layers import Embedding, SimpleRNN
model = Sequential()
model.add(Embedding(10000, 32))
model.add(SimpleRNN(32))
model.summary()
代码执行结果:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
embedding_1 (Embedding) (None, None, 32) 320000
_________________________________________________________________
simple_rnn_1 (SimpleRNN) (None, 32) 2080
=================================================================
Total params: 322,080
Trainable params: 322,080
Non-trainable params: 0
_________________________________________________________________
检测条件:输出是否一致
数据处理
我们还是使用IMDB电影评论的数据,但是我们需要引入一个sequence模块把我们的数据格式化输入到神经网络中,(该步骤不需要学员实现),请复制代码到.ipynb文件中并运行,代码如下:
from keras.datasets import imdb
from keras.preprocessing import sequence
max_features = 10000 #作为特征的单词个数
maxlen = 500
batch_size = 32
#加载数据
print('Loading data...')
(input_train, y_train), (input_test, y_test) = imdb.load_data(num_words=max_features)
print(len(input_train), 'train sequences')
print(len(input_test), 'test sequences')
print('Pad sequences (samples x time)')
input_train = sequence.pad_sequences(input_train, maxlen=maxlen)
input_test = sequence.pad_sequences(input_test, maxlen=maxlen)
print('input_train shape:', input_train.shape)
print('input_test shape:', input_test.shape)
代码执行结果:
Loading data...
25000 train sequences
25000 test sequences
Pad sequences (samples x time)
input_train shape: (25000, 500)
input_test shape: (25000, 500)
检测条件:输出是否一致
绘制结果
请学员完成训练并绘制损失值和准确率的结果图
参考代码:
acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']
epochs = range(len(acc))
plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()
plt.figure()
plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()
plt.show()
绘制结果:
编程要求
根据提示,在右侧编辑器补充代码,搭建自己RNN神经网络。
1)从keras导入全连接层模块 2)定义一个序列模型 3) 添加一个Embedding层,参数是(max_features,32) 4)添加一个SimpleRNN层,输出维度32 5)添加一个全连接层,输出维度1,激活函数‘sigmoid’
测试说明
补充代码,完成对应的输出。
输出:
开始你的任务吧,祝你成功!
代码部分
from keras.models import Sequential
from keras.layers import Dense, Embedding, SimpleRNN, LSTM
from keras.datasets import imdb
from keras.preprocessing import sequence
import matplotlib.pyplot as plt
if __name__ == "__main__":
max_features = 10000
maxlen = 500
print("Loading data...")
##################Begin#########################
"""
加载imdb代码,数据集地址为:/data/workspace/myshixun/imdb.npz
"""
print('Loading data...')
(X_train, y_train), (X_test, y_test) = imdb.load_data(num_words=max_features)
#################END#############################
print(len(X_train), "train sequences")
print(len(X_train), "test sequences")
print("Pad sequences (sample x times)")
X_train = sequence.pad_sequences(X_train, maxlen=maxlen)
X_test = sequence.pad_sequences(X_test, maxlen=maxlen)
print("X_train shape:", X_train.shape)
print("X_test shape:", X_test.shape)
model = Sequential()
model.add(Embedding(max_features, 32))
##################Begin#########################
"""
添加SimpleRNN层,参数为32
"""
model.add(SimpleRNN(32))
#################END#############################
model.add(Dense(1, activation='sigmoid'))
model.summary()
# acc = history.history['acc']
# val_acc = history.history['val_acc']
# loss = history.history['loss']
# val_loss = history.history['val_loss']
# epochs = 1
# plt.plot(epochs, acc, 'bo', label='Training acc')
# plt.plot(epochs, val_acc, 'b', label='Validation acc')
# plt.title('Training and validation accuracy')
# plt.legend()
# plt.figure()
# plt.plot(epochs, loss, 'bo', label='Training loss')
# plt.plot(epochs, val_loss, 'b', label='Validation loss')
# plt.title('Training and validation loss')
# plt.legend()
# plt.show()
二、LSTM循环神经网络
任务描述
本关任务:搭建自己的LSTM模型。
相关知识
1.什么是LSTM模型 2.搭建给定要求的LSTM模型
SimpleRNN的缺陷
SimpleRNN最大的问题是:在时刻t,理论上来说,它应该能够记住许多时间步之前见过的信息,但实际上它是不可能学到这种长期依赖的,其原因在于梯度消失问题,随着层数的增加,网络最终变得无法训练,LSTM层和GRU层都是为了解决这个问题而设计的。本实验主要介绍使用更为广泛的LSTM模型
LSTM层
LSTM层是SimpleRNN层的一种变体,它增加了一种携带信息跨越多个时间步的方法。假设设有一条传送带,其运行方向平行于你所处理的序列。序列中的信息可以在任意位置跳上传送带,然后被传送到更晚的时间步,并在需要是原封不动地跳回来。这实际上就是LSTM的原理:它保存信息以便后面使用,从而防止较早期的信号在处理过程中逐渐消失。
搭建自己的LSTM神经网络
类似1.4节请学员完成以下步骤 1)从kaeras导入LSTM模块 2)定义一个序列模型 3)添加一个Embedding层,参数是(max_features,32) 4)添加一个LSTM层,输出维度32 5)添加一个全连接层,输出维度1,激活函数‘sigmoid’
参考代码:
from keras.layers import LSTM
model = Sequential()
model.add(Embedding(max_features, 32))
model.add(LSTM(32))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['acc'])
history = model.fit(input_train, y_train,
epochs=10,
batch_size=128,
validation_split=0.2)
代码执行结果:
Train on 20000 samples, validate on 5000 samples
Epoch 1/10
20000/20000 [==============================] - 108s - loss: 0.5038 - acc: 0.7574 - val_loss: 0.3853 - val_acc: 0.8346
Epoch 2/10
20000/20000 [==============================] - 108s - loss: 0.2917 - acc: 0.8866 - val_loss: 0.3020 - val_acc: 0.8794
Epoch 3/10
20000/20000 [==============================] - 107s - loss: 0.2305 - acc: 0.9105 - val_loss: 0.3125 - val_acc: 0.8688
Epoch 4/10
20000/20000 [==============================] - 107s - loss: 0.2033 - acc: 0.9261 - val_loss: 0.4013 - val_acc: 0.8574
Epoch 5/10
20000/20000 [==============================] - 107s - loss: 0.1749 - acc: 0.9385 - val_loss: 0.3273 - val_acc: 0.8912
Epoch 6/10
20000/20000 [==============================] - 107s - loss: 0.1543 - acc: 0.9457 - val_loss: 0.3505 - val_acc: 0.8774
Epoch 7/10
20000/20000 [==============================] - 107s - loss: 0.1417 - acc: 0.9493 - val_loss: 0.4485 - val_acc: 0.8396
Epoch 8/10
20000/20000 [==============================] - 106s - loss: 0.1331 - acc: 0.9522 - val_loss: 0.3242 - val_acc: 0.8928
Epoch 9/10
20000/20000 [==============================] - 106s - loss: 0.1147 - acc: 0.9618 - val_loss: 0.4216 - val_acc: 0.8746
Epoch 10/10
20000/20000 [==============================] - 106s - loss: 0.1092 - acc: 0.9628 - val_loss: 0.3972 - val_acc: 0.8758
检测条件:输出准确率接近即可。
绘制结果
请学员完成训练并绘制损失值和准确率的结果图
参考代码:
acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']
epochs = range(len(acc))
plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()
plt.figure()
plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()
plt.show()
绘制结果:
编程要求
根据提示,在右侧编辑器补充代码,搭建自己LSTM神经网络。
1)从kaeras导入全连接层模块 2)定义一个序列模型 3) 添加一个Embedding层,参数是(max_features,32) 4)添加一个LSTM层,输出维度32 5)添加一个全连接层,输出维度1,激活函数‘sigmoid’
测试说明
平台会对你编写的代码进行测试:
预期输出: 补全相应的代码,输出相应的答案。
开始你的任务吧,祝你成功!
代码部分
from keras.models import Sequential
from keras.layers import Dense, Embedding, SimpleRNN, LSTM
from keras.datasets import imdb
from keras.preprocessing import sequence
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
from keras.layers import LSTM
if __name__ == "__main__":
max_features = 10000
maxlen = 500
print("Loading data...")
##################Begin#########################
"""
加载imdb数据,数据路径为'/data/workspace/myshixun/imdb.npz'
"""
(X_train, y_train), (X_test, y_test) = imdb.load_data(num_words=max_features)
#################END#############################
print(len(X_train), "train sequences")
print(len(X_train), "test sequences")
print("Pad sequences (sample x times)")
X_train = sequence.pad_sequences(X_train, maxlen=maxlen)
X_test = sequence.pad_sequences(X_test, maxlen=maxlen)
print("X_train shape:", X_train.shape)
print("X_test shape:", X_test.shape)
model = Sequential()
model.add(Embedding(max_features, 32))
##################Begin#########################
"""
LSTM层,参数为32
"""
model.add(LSTM(32))
model.add(Dense(1, activation='sigmoid'))
model.summary()
model.compile(optimizer='rmsprop',loss='binary_crossentropy',metrics=['acc'])
# history = model.fit(X_train, y_train, epochs=1, batch_size=32, validation_split=20)
# # 绘图
# acc = history.history['acc']
# val_acc = history.history['val_acc']
# loss = history.history['loss']
# val_loss = history.history['val_loss']
# epochs = range(len(acc))
# plt.plot(epochs, acc, 'bo', label='Training acc')
# plt.plot(epochs, val_acc, 'b', label='Validation acc')
# plt.title('Training and validation accuracy')
# plt.legend()
# plt.figure()
# plt.plot(epochs, loss, 'bo', label='Training loss')
# plt.plot(epochs, val_loss, 'b', label='Validation loss')
# plt.title('Training and validation loss')
# plt.legend()
# plt.show()
#################END#############################
# model.add(Dense(1, activation='sigmoid'))
# model.summary()