【Keras-RNN】IMDb

---

MLP或者CNN都只能依照当前的状态进行识别，如果处理时间序列的问题，就需要RNN、LSTM模型了。本博客使用 RNN 对 IMDb 数据集进行分析预测，用MLP进行预测可以参考这篇博客 【TensorFlow-MLP】MNIST

RNN结构如上图所示，

$X_t$ 是 $t$ 时间点神经网络的输入

$O_t$ 是 $t$ 时间点神经网络的输出

$（U,V,W）$都是神经网络的参数

$S_t$ 是隐藏状态，代表着 “记忆”

$S_t=f\left([U]X_t+[W]S_{t-1}\right)$

$f$ 表示 ReLU

1 下载数据集

import urllib.request
import os
import tarfile

#下载数据集
url="http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz"
filepath="data/aclImdb_v1.tar.gz"
if not os.path.isfile(filepath):
    result=urllib.request.urlretrieve(url,filepath)
    print('downloaded:',result)
# 解压
if not os.path.exists("data/aclImdb"):
    tfile = tarfile.open("data/aclImdb_v1.tar.gz", 'r:gz')
    result=tfile.extractall('data/')

2 数据预处理

同 【Keras-MLP】IMDb

from keras.datasets import imdb
from keras.preprocessing import sequence
from keras.preprocessing.text import Tokenizer
import re
re_tag = re.compile(r'<[^>]+>')

def rm_tags(text):
    return re_tag.sub('', text)

import os
def read_files(filetype):
    path = "data/aclImdb/"
    file_list=[]

    positive_path=path + filetype+"/pos/"
    for f in os.listdir(positive_path):
        file_list+=[positive_path+f]
    
    negative_path=path + filetype+"/neg/"
    for f in os.listdir(negative_path):
        file_list+=[negative_path+f]
        
    print('read',filetype, 'files:',len(file_list))
       
    all_labels = ([1] * 12500 + [0] * 12500) 
    
    all_texts  = []
    
    for fi in file_list:
        with open(fi,encoding='utf8') as file_input:
            all_texts += [rm_tags(" ".join(file_input.readlines()))]
            
    return all_labels,all_texts

开始处理

# 读文件
y_train,train_text=read_files("train")
y_test,test_text=read_files("test")

# 建立单词和数字映射的字典
token = Tokenizer(num_words=3800)
token.fit_on_texts(train_text)

#将影评的单词映射到数字
x_train_seq = token.texts_to_sequences(train_text)
x_test_seq  = token.texts_to_sequences(test_text)

# 让所有影评保持在380个数字
x_train = sequence.pad_sequences(x_train_seq, maxlen=380)
x_test  = sequence.pad_sequences(x_test_seq,  maxlen=380)

output

read train files: 25000
read test files: 25000

3 建立模型

from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation
from keras.layers.embeddings import Embedding
from keras.layers.recurrent import SimpleRNN

model = Sequential()

model.add(Embedding(output_dim=32,
                    input_dim=3800, 
                    input_length=380))
model.add(Dropout(0.35))

# 加了一个简单的RNN层
model.add(SimpleRNN(units=16))

model.add(Dense(units=256,activation='relu' ))
model.add(Dropout(0.35))

model.add(Dense(units=1,activation='sigmoid' ))

model.summary()

output

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_1 (Embedding)      (None, 380, 32)           121600    
_________________________________________________________________
dropout_1 (Dropout)          (None, 380, 32)           0         
_________________________________________________________________
simple_rnn_1 (SimpleRNN)     (None, 16)                784       
_________________________________________________________________
dense_1 (Dense)              (None, 256)               4352      
_________________________________________________________________
dropout_2 (Dropout)          (None, 256)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 257       
=================================================================
Total params: 126,993
Trainable params: 126,993
Non-trainable params: 0
_________________________________________________________________

参数计算
3800×32 = 121600

RNN 输入32，输出16 可以理解为3层，入上图
32×16 + 16×16 + 16 = 784

16×256+256 = 4352

256×1 + 1 = 257

4 训练模型

model.compile(loss='binary_crossentropy', 
              optimizer='adam', 
              metrics=['accuracy'])

train_history =model.fit(x_train, y_train,batch_size=100, 
                         epochs=10,verbose=2,
                         validation_split=0.2)

参数说明请参考
【TensorFlow-MLP】MNIST 或者 Keras中文文档

output

Train on 20000 samples, validate on 5000 samples
Epoch 1/10
 - 26s - loss: 0.4915 - acc: 0.7604 - val_loss: 0.2876 - val_acc: 0.9006
Epoch 2/10
 - 23s - loss: 0.3209 - acc: 0.8685 - val_loss: 0.4203 - val_acc: 0.8004
Epoch 3/10
 - 22s - loss: 0.2829 - acc: 0.8879 - val_loss: 0.3567 - val_acc: 0.8744
Epoch 4/10
 - 22s - loss: 0.2391 - acc: 0.9065 - val_loss: 0.3321 - val_acc: 0.8536
Epoch 5/10
 - 22s - loss: 0.2187 - acc: 0.9158 - val_loss: 0.4706 - val_acc: 0.8358
Epoch 6/10
 - 21s - loss: 0.1867 - acc: 0.9290 - val_loss: 0.5693 - val_acc: 0.8118
Epoch 7/10
 - 22s - loss: 0.1622 - acc: 0.9378 - val_loss: 0.6032 - val_acc: 0.8078
Epoch 8/10
 - 22s - loss: 0.1785 - acc: 0.9307 - val_loss: 0.4812 - val_acc: 0.8424
Epoch 9/10
 - 21s - loss: 0.1424 - acc: 0.9478 - val_loss: 0.6116 - val_acc: 0.8094
Epoch 10/10
 - 22s - loss: 0.1196 - acc: 0.9548 - val_loss: 0.9426 - val_acc: 0.7768

可视化结果

import matplotlib.pyplot as plt
def show_train_history(train_history,train,validation):
    plt.plot(train_history.history[train])
    plt.plot(train_history.history[validation])
    plt.title('Train History')
    plt.ylabel(train)
    plt.xlabel('Epoch')
    plt.legend(['train', 'validation'], loc='upper left')
    plt.savefig('1.png')
    plt.show()

调用查看精度变化

show_train_history(train_history,'acc','val_acc')

调用查看损失变化

show_train_history(train_history,'loss','val_loss')

5 评估模型的准确率

scores = model.evaluate(x_test, y_test, verbose=1)
scores[1]

output

25000/25000 [==============================] - 18s 706us/step
0.84692

Note：scores[0] 为损失

6 预测概率和预测结果

查看输出的概率

probility=model.predict(x_test)
probility[:10]

output

array([[0.999658  ],
       [0.9902057 ],
       [0.99913883],
       [0.99437803],
       [0.9984084 ],
       [0.9932498 ],
       [0.99893945],
       [0.9921451 ],
       [0.9777389 ],
       [0.8260359 ]], dtype=float32)

查看输出的结果，大于0.5的为1.小于0.5的为0

predict=model.predict_classes(x_test)
predict[:10]

output

array([[1],
       [1],
       [1],
       [1],
       [1],
       [1],
       [1],
       [1],
       [1],
       [1]], dtype=int32)

7 查看测试集中的文本和其预测结果

SentimentDict={1:'正面的',0:'负面的'}
def display_test_Sentiment(i):
    print(test_text[i])
    print('label:',SentimentDict[y_test[i]],
          '预测结果:',SentimentDict[predict[i][0]])

调用

display_test_Sentiment(2)

output

BLACK WATER is a thriller that manages to completely transcend it’s limitations (it’s an indie flick) by continually subverting expectations to emerge as an intense experience.In the tradition of all good animal centered thrillers ie Jaws, The Edge, the original Cat People, the directors know that restraint and what isn’t shown are the best ways to pack a punch. The performances are real and gripping, the crocdodile is extremely well done, indeed if the Black Water website is to be believed that’s because they used real crocs and the swamp location is fabulous.If you are after a B-grade gore fest croc romp forget Black Water but if you want a clever, suspenseful ride that will have you fearing the water and wondering what the hell would I do if i was up that tree then it’s a must see.
label: 正面的 预测结果: 正面的

8 测试新的影评

http://www.imdb.com/title/tt2771200

def predict_review(input_text):
	# 影评转换为数字列表
    input_seq = token.texts_to_sequences([input_text])
    # 截断数字列表使得所有输入长度为380
    pad_input_seq  = sequence.pad_sequences(input_seq , maxlen=380)
    # 预测分类结果
    predict_result=model.predict_classes(pad_input_seq)
    # 输出结果
    print(SentimentDict[predict_result[0][0]])

调用

predict_review(’’’
As a fan of the original Disney film (Personally I feel it’s their masterpiece) I was taken aback to the fact that a new version was in the making. Still excited I had high hopes for the film. Most of was shattered in the first 10 minutes. Campy acting with badly performed singing starts off a long journey holding hands with some of the worst CGI Hollywood have managed to but to screen in ages.
A film that is over 50% GCI, should focus on making that part believable, unfortunately for this film, it’s far from that. It looks like the original film was ripped apart frame by frame and the beautiful hand-painted drawings have been replaced with digital caricatures. Besides CGI that is bad, it’s mostly creepy. As the little teacup boy will give me nightmares for several nights to come. Emma Watson plays the same character as she always does, with very little acting effort and very little conviction as Belle. Although I can see why she was cast in the film based on merits, she is far from the right choice for the role. Dan Stevens does alright under as some motion captured dead-eyed Beast, but his performance feels flat as well. Luke Evans makes for a great pompous Gaston, but a character that has little depth doesn’t really make for a great viewing experience. Josh Gad is a great comic relief just like the original movie’s LeFou. Other than that, none of the cast stands out enough for me to remember them. Human or CHI creature. I was just bored through out the whole experience. And for a project costing $160 000 000, I can see why the PR department is pushing it so hard because they really need to get some cash back on this pile of wet stinky CGI-fur!
All and all, I might be bias from really loving Disney’s first adaptation. That for me marks the high-point of all their work, perfectly combining the skills of their animators along with some CGI in a majestic blend. This film however is more like the bucket you wash off your paintbrush in, it has all the same colors, but muddled with water and to thin to make a captivating story from. The film is quite frankly not worth your time, you would be better off watching the original one more time.
‘’’)

output

负面的

9 保存模型

model_json = model.to_json()
with open("SaveModel/Imdb_RNN_model.json", "w") as json_file:
    json_file.write(model_json)

model.save_weights("SaveModel/Imdb_RNN_model.h5")
print("Saved model to disk")

---

声明：代码源于《TensorFlow+Keras深度学习人工智能实践应用》 林大贵版，引用、转载请注明出处，谢谢，如果对书本感兴趣，买一本看看吧！！！