course-nlp——5-nn-imdb

本文参考自https://github.com/fastai/course-nlp。这部分是fastai1.0版本的教程，由于现在fastai2.0重构的改变非常大，所以文中的很多api都变了，由于学习目的并不是熟练掌握fastai，因此这里就简单的存一下，本文是用IMDB电影评论数据集在wikitext-103语言模型中微调，构建语言模型和情感分类器。

自然语言建模的迁移学习

为 IMDB 电影评论构建语言模型和情感分类器

几年来，迁移学习已在计算机视觉领域得到广泛应用并取得了巨大成功，但直到最近一年左右，它才成功应用于 NLP（从 ULMFit 开始，我们将在这里使用它，它基于 BERT 和 GPT-2 构建）。

正如 Sebastian Ruder 去年夏天在《The Gradient》中所写，NLP 的 ImageNet 时刻已经到来。

我们将首先为 IMDB 电影评论构建一个语言模型。接下来，我们将构建一个情绪分类器，它将根据文本预测评论是负面的还是正面的。对于这两项任务，我们都将使用迁移学习。从 wikitext-103 语言模型的预训练权重开始，我们将调整这些权重以专门针对 IMDb 电影评论的语言。

语言模型（Language Models）

语言建模可以是一种有趣的创意形式。

Using a GPU

您需要为本课安装 fastai 库，并且需要使用 GPU 来训练您的神经网络。

%reload_ext autoreload
%autoreload 2
%matplotlib inline

from fastai import *
from fastai.text.all import *
from scipy.spatial.distance import cosine as dist

请注意，语言模型可以使用大量 GPU，因此您可能需要在此处减少批量大小。

# bs=192
bs=48
# bs=24

torch.cuda.set_device(0)

1. 准备 IMDb 数据（样本）

首先，让我们下载我们要研究的数据集。IMDb 数据集由 Andrew Maas 等人整理，共包含 100,000 条 IMDB 评论。其中 25,000 条被标记为正面和负面以用于训练，另外 25,000 条被标记为测试（在这两种情况下，它们都高度两极化）。剩下的 50,000 条是额外的未标记数据（但我们仍然会找到它的用途）。

我们将从为您准备的样本开始，以便在浏览整个数据集之前快速运行。

path = untar_data(URLs.IMDB_SAMPLE)
path.ls()

(#1) [Path('C:/Users/wuzhongyanqiu/.fastai/data/imdb_sample/texts.csv')]

它只包含一个 csv 文件，让我们看一下。
它包含每条评论一行，带有标签（“负面”或“正面”）、文本和一个标志，以确定它应该是验证集还是训练集的一部分。如果我们忽略此标志，我们可以在一行代码中创建一个包含此数据的 DataBunch：

加载并预处理数据并形成数据集

添加针对 fastai Text API 中错误的解决方法

%%time

# throws `BrokenProcessPool' Error sometimes. Keep trying `till it works!
count = 0
error = True
while error:
    try: 
        # Preprocessing steps
        data_lm = TextDataBunch.from_csv(path, 'texts.csv')
        error = False
        print(f'failure count is {count}\n')    
    except: # catch *all* exceptions
        # accumulate failure count
        count = count + 1
        print(f'failure count is {count}')
        if count >= 5:
            print('Max attempts reached. Exiting.')
            break  # 达到最大尝试次数后退出循环
        time.sleep(1)  # 等待一段时间后重试，避免立即重试可能的问题

通过执行此行，启动了一个需要一些时间的过程。让我们深入研究一下。图像可以（几乎）直接输入模型，因为它们只是一大堆像素值，这些像素值是 0 到 1 之间的浮点数。文本由单词组成，我们不能直接对它们应用数学函数。我们首先必须将它们转换为数字。这分为两个不同的步骤：标记化和数值化。TextDataBunch 在后台为您完成所有这些工作。

标记化

处理文本的第一步是将原始句子拆分成单词，或者更确切地说是标记。最简单的方法是用空格拆分字符串，但我们可以更聪明：

• 我们需要注意标点符号
• 有些词是两个不同词的缩写，如isn’t或don’t
• 我们可能需要清理文本的某些部分，例如如果有HTML代码
• 要查看标记器在幕后做了什么，让我们看一下一批中的几个文本。

为了提高可读性，文本在100个标记处被截断。我们可以看到，它所做的不仅仅是根据空格和标点符号进行拆分：

• “'s”被组合成一个标记

• 缩写被分开，就像他的：“did”、“n’t”

• 内容已清除所有 HTML 符号并小写

• 有几个特殊标记（所有以 xx 开头的标记），用于替换未知标记（见下文）或引入不同的文本字段（这里我们只有一个）。

数值化

从文本中提取标记后，我们会通过创建所有使用过的单词列表将其转换为整数。我们只保留在列表中出现两次的单词，最大词汇量为 60,000（默认情况下），并用未知标记 UNK 替换未达到要求的单词。

ids 标记的对应关系存储在我们数据集的 vocab 属性中，位于一个名为 itos（用于将 int 转换为字符串）的字典中。

data_lm.vocab.itos[:10]

['xxunk',
 'xxpad',
 'xxbos',
 'xxeos',
 'xxfld',
 'xxmaj',
 'xxup',
 'xxrep',
 'xxwrep',
 'the']

如果我们看看我们的数据集里有什么，我们会看到标记化的文本作为一种表示：

data_lm.train_ds[0][0]

Text xxbos xxmaj this movie is so bad , i knew how it ends right after this little girl killed the first person . xxmaj very bad acting very bad plot very bad movie 
 
  do yourself a favour and xxup don't watch it 1 / 10

但底层数据都是数字

data_lm.train_ds[0][0].data[:10]

array([  2,   5,  21,  31,  16,  52, 107,  10,  19, 669], dtype=int64)

替代方法：使用数据块 API(data block API)

我们可以将数据块 API 与 NLP 结合使用，并且比默认工厂方法提供的灵活性要高得多。例如，在前面的示例中，数据在训练和验证之间随机分割，而不是读取 csv 的第三列。

但是，使用数据块 API，我们必须手动调用标记和数值化步骤。这允许更大的灵活性，如果您不使用 fastai 的默认设置，则要传递的各种参数将出现在它们相关的步骤中，因此它将更具可读性。

加载并预处理数据并形成数据块（datablock）

%%time

# throws `BrokenProcessPool' Error sometimes. Keep trying `till it works!
count = 0
error = True
while error:
    try: 
        # Preprocessing steps
        data = (TextList.from_csv(path, 'texts.csv', cols='text')
                .split_from_df(col=2)
                .label_from_df(cols=0)
                .databunch())        
        error = False
        print(f'failure count is {count}\n')    
    except: # catch *all* exceptions
        # accumulate failure count
        count = count + 1
        print(f'failure count is {count}')

failure count is 0

Wall time: 15.5 s

迁移学习

我们将从 wikitext-103 语言模型的预训练权重开始创建一个 IMDb 语言模型。
现在让我们获取完整的 IMDb 数据集以进行后续操作。

path = untar_data(URLs.IMDB)
path.ls()

[WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/data_clas.pkl'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/data_lm.pkl'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/data_save.pkl'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/finetuned.pth'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/finetuned_enc.pth'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/imdb.vocab'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/imdb_textlist_class'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/ld.pkl'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/ll_clas.pkl'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/ll_lm.pkl'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/lm_databunch'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/models'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/pretrained'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/README'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/test'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/tmp_clas'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/tmp_lm'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/train'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/unsup'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/vocab_lm.pkl')]

(path/'train').ls()

[WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/train/labeledBow.feat'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/train/neg'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/train/pos'),
 WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb/train/unsupBow.feat')]

评论位于遵循 imagenet 结构的训练和测试集中。唯一的区别是 train 中有一个包含未标记数据的 unsup 文件夹。

我们不会从头开始训练对评论进行分类的模型。就像在计算机视觉中一样，我们将使用在更大的数据集（维基百科的清理子集，称为 wikitext-103）上预训练的模型。该模型已经过训练，可以猜测下一个单词是什么，其输入是所有前面的单词。它具有循环结构和隐藏状态，每次看到新单词时都会更新。因此，这个隐藏状态包含有关该点之前的句子的信息。

我们将使用英语的“知识”来构建分类器，但首先，就像计算机视觉一样，我们需要根据特定数据集对预训练模型进行微调。由于人们在 IMDB 上留下的评论的英语与维基百科的英语不同，因此我们需要稍微调整一下模型的参数。另外，该数据集中可能有一些极为常见的词在维基百科中几乎不存在，因此可能不属于模型训练词汇的一部分。

更多信息关于 WikiText-103

我们将使用 Stephen Merity 创建的 WikiText-103 数据集来预训练语言模型。

引用 Stephen 的帖子：

WikiText 语言建模数据集是从 Wikipedia 上经过验证的优质和精选文章集中提取的超过 1 亿个标记的集合。该数据集在 Creative Commons Attribution-ShareAlike 许可下可用。

与 Penn Treebank (PTB) 的预处理版本相比，WikiText-2 大 2 倍以上，WikiText-103 大 110 倍以上。WikiText 数据集还具有更大的词汇量，并保留了原始的大小写、标点符号和数字 - 所有这些都在 PTB 中被删除。由于它由完整的文章组成，因此该数据集非常适合可以利用长期依赖关系的模型。

下载 wikitext-103。将其解压缩到计算机上的 .fastai/data/ 文件夹中。

将 IMDb 数据打包成语言模型数据集(databunch)

这就是未标记数据对我们有用的地方，因为我们可以用它来微调我们的模型。让我们用数据块 API 创建数据对象（这需要几分钟）。

我们将为语言模型使用一种特殊的 TextDataBunch，它会忽略标签（这就是我们在任何地方都放置 0 的原因），在每个时期对文本进行打乱，然后再将它们全部连接在一起（仅用于训练；我们不对验证集进行打乱），并将按顺序发送读取该文本的批次，目标是句子中的下一个单词。

添加一个 try-except 包装器作为 fastai Text API 中错误的解决方法

%%time

# throws `BrokenProcessPool` Error sometimes. Keep trying `till it works!
count = 0
error = True
while error:
    try: 
        # Preprocessing steps
        data_lm = (TextList.from_folder(path)
           #Inputs: all the text files in path
            .filter_by_folder(include=['train', 'test', 'unsup']) 
           # notebook 3-logreg-nb-imbd used .split_by_folder instead of .filter_by_folder
            # and this took less time to run. Can we do the same here?
           #We may have other temp folders that contain text files so we only keep what's in train and test
            .split_by_rand_pct(0.1, seed=42))
           #We randomly split and keep 10% (10,000 reviews) for validation
            #.label_for_lm()           
           #We want to make a language model so we label accordingly
            #.databunch(bs=bs, num_workers=1))
        error = False
        print(f'failure count is {count}\n')    
    except: # catch *all* exceptions
        # accumulate failure count
        count = count + 1
        print(f'failure count is {count}')

failure count is 0

Wall time: 2.79 s

当我在单独的单元格中执行最后两个步骤时，我获得了更快的结果：

%%time

# throws `BrokenProcessPool' Error sometimes. Keep trying `till it works!
count = 0
error = True
while error:
    try: 
        # Preprocessing steps
        #     the next step is the bottleneck
        data_lm = (data_lm.label_for_lm()           
           #We want to make a language model so we label accordingly
            .databunch(bs=bs, num_workers=1))
        error = False
        print(f'failure count is {count}\n')    
    except: # catch *all* exceptions
        # accumulate failure count
        count = count + 1
        print(f'failure count is {count}')

failure count is 4

Wall time: 14min 18s

len(data_lm.vocab.itos),len(data_lm.train_ds)

(60000, 90000)

data_lm.show_batch()

idx	text
0	forgotten until much later , by which time i did not care . xxmaj the character we should really care about is a very cocky , overconfident xxmaj ashton xxmaj kutcher . xxmaj the problem is he comes off as kid who thinks he 's better than anyone else around him and shows no signs of a cluttered closet . xxmaj his only obstacle appears to be winning over xxmaj
1	of an xxunk bad b - movie . xxmaj there are worse movies than this one ( xxmaj titanic for example ) , but this definitely shares the pile of steaming crap movies . \n \n xxup ok this was apparently shot in xxmaj kansas xxmaj city , which explains why everyone is so lame . xxmaj the main guy looks like xxmaj steve xxmaj guttenberg , and is
2	, then this'll be okay for you .. xxmaj however , if you ca n't stand a plot being thrown at you which remains unresolved by the time the credits roll , you should go watch something else . xxwrep 5 xxbos i am not quite sure what to say / think about this movie . xxmaj it is definitely not the worst in the series ( there
3	tension for such a famous race and anyway the race - off at the end seems like another xxunk . \n \n xxmaj actually i 'd have given it another mark if they 'd stuck to the alternative title " xxmaj those xxmaj magnificent xxmaj men in xxmaj their xxmaj jaunty xxmaj xxunk " but in truth the animated series " xxmaj wacky xxmaj races " did this so
4	though . xxmaj rest of the cast are average at best . xxmaj overall xxmaj not worth your time or money . * 1 / 2 out of 5 xxwrep 5 xxbos xxup the xxup protector . xxmaj you hear the name . xxmaj you think , " ah , it 's a crappy xxmaj hong xxmaj kong movie . " xxmaj guess what - it 's not

data_lm.save()

加载已保存的数据

data_lm = load_data(path, 'lm_databunch', bs=bs)

迁移学习步骤

这就是奇迹发生的地方！
AWD_LSTM 对象包含 wikitext-103 语言模型的预训练权重和神经网络架构。这些将在您第一次执行以下行时下载，并存储在 ~/.fastai/models/ 中（如果您在配置文件中指定了不同的路径，则存储在其他地方）。
我们将这些导入到我们的 IMDb 语言模型的 language_model_learner 对象中，如下所示：

learn_lm = language_model_learner(data_lm, AWD_LSTM, drop_mult=0.3)

获取 IMDb 语言模型词汇表(vocabulary)

vocab = data_lm.vocab

vocab.stoi["stingray"]

35639

vocab.itos[vocab.stoi["stingray"]]

'stingray'

vocab.itos[vocab.stoi["mobula"]]

'xxunk'

awd = learn_lm.model[0]

获取 IMDb 语言模型编码器。回想一下，编码器将标记转换为 IMDb 词汇表定义的空间中的数值向量。

enc = learn_lm.model[0].encoder

enc.weight.size()

torch.Size([60000, 400])

IMDB 和 Wikipedia 语言模型的词汇差异

我们将从 wikitext 103 语言模型中加载 wiki_itos（索引到字符串列表）。我们将比较 wikitext-103 和 IMDB 的词汇表。可以预料的是，这两组词汇表中会有一些不同的单词，但这对于迁移学习来说没有问题！

#wiki_itos = pickle.load(open(Config().model_path()/'wt103-1/itos_wt103.pkl', 'rb'))
wiki_itos = pickle.load(open(Config().model_path()/'wt103-fwd/itos_wt103.pkl', 'rb'))

wiki_itos[:10]

['xxunk',
 'xxpad',
 'xxbos',
 'xxeos',
 'xxfld',
 'xxmaj',
 'xxup',
 'xxrep',
 'xxwrep',
 'the']

len(wiki_itos)

60000

len(vocab.itos)

60000

i, unks = 0, []
while len(unks) < 50:
    if data_lm.vocab.itos[i] not in wiki_itos: unks.append((i,data_lm.vocab.itos[i]))
    i += 1

wiki_words = set(wiki_itos)

imdb_words = set(vocab.itos)

wiki_not_imbdb = wiki_words.difference(imdb_words)

imdb_not_wiki = imdb_words.difference(wiki_words)

wiki_not_imdb_list = []

for i in range(100):
    word = wiki_not_imbdb.pop()
    wiki_not_imdb_list.append(word)
    wiki_not_imbdb.add(word)

wiki_not_imdb_list[:15]

['fustat',
 'sterilization',
 'konkani',
 'tvrđa',
 'bruner',
 'brachiopods',
 'aud',
 'i-68',
 'kaafjord',
 'maximian',
 '48th',
 'morphy',
 'freyja',
 'maenas',
 'perthshire']

imdb_not_wiki_list = []

for i in range(100):
    word = imdb_not_wiki.pop()
    imdb_not_wiki_list.append(word)
    imdb_not_wiki.add(word)

imdb_not_wiki_list[:15]

['sceptic',
 'paquin',
 'louche',
 'sodom',
 'atlantean',
 'kron',
 'gourd',
 'tolwyn',
 'black-',
 'catty',
 'bandito',
 'skulduggery',
 'johnathon',
 'fiendishly',
 'malachi']

所有出现在 IMDB 词汇表中但未出现在 wikitext-103 词汇表中的单词都将在模型中初始化为相同的随机向量。在模型训练过程中，我们将了解它们的权重。

vocab.stoi["modernisation"]

0

"modernisation" in wiki_words

True

True

25365

"30-something" in wiki_words, "30-something" in imdb_words

(False, True)

vocab.stoi["linklater"]

16735

"linklater" in wiki_words, "linklater" in imdb_words

(False, True)

"house" in wiki_words, "house" in imdb_words

(True, True)

np.allclose(enc.weight[vocab.stoi["30-something"], :], 
            enc.weight[vocab.stoi["linklater"], :])

True

np.allclose(enc.weight[vocab.stoi["30-something"], :], 
            enc.weight[vocab.stoi["house"], :])

False

new_word_vec = enc.weight[vocab.stoi["linklater"], :]

使用未训练的 IMDb 语言模型生成类似电影评论的虚假文本

TEXT = "The color of the sky is"
N_WORDS = 40
N_SENTENCES = 2

print("\n".join(learn_lm.predict(TEXT, N_WORDS, temperature=0.75) for _ in range(N_SENTENCES)))

The color of the sky is hues in the National Portrait Gallery , London , which was originally the Royal Academy of Art . This display has since been referred to as the Venus Jubilee
The color of the sky is called black , in the form of a white cloth . The color of the body varies through the color of the colours , but the black and blue areas of the wings are black . The white

TEXT = "I hated this movie"
N_WORDS = 30
N_SENTENCES = 2

print("\n".join(learn_lm.predict(TEXT, N_WORDS, temperature=0.75) for _ in range(N_SENTENCES)))

I hated this movie . The first film in the genre , Don ' t Look Now , was what the Academy Awards called a " apologized "
I hated this movie by a Dutch composer . This was a musical method often used by Latin music , which was used by the original French film La

print("\n".join(learn_lm.predict(TEXT, N_WORDS, temperature=0.75) for _ in range(N_SENTENCES)))
```doc(LanguageLearner.predict)

```python
I hated this movie , " Blaxplotation of the Lake " , and other films from the era , and all of them were not pfetten on film . It was
I hated this movie = 
 
  The film is the third film in the Harry Potter series and the third film in the Harry Potter series . The

doc(LanguageLearner.predict)

降低温度将使文本的随机性降低。

print("\n".join(learn_lm.predict(TEXT, N_WORDS, temperature=0.10) for _ in range(N_SENTENCES)))

I hated this movie by John Lennon , and he said he wanted to do it for the American public . He said that he was " a little bit
I hated this movie . It was a film that was a success . It was a success , and it was the first film to be released in the United

print("\n".join(learn_lm.predict(TEXT, N_WORDS, temperature=0.10) for _ in range(N_SENTENCES)))

I hated this movie by Michael Jackson , and he said he was " a fan of the Academy Award for Best Picture " . He said
I hated this movie by John Lennon , and the film was released in the United States on November 11 , 1999 . The film was released on

训练（微调）IMDb 语言模型

从 wikitext-103 预训练权重开始，我们将微调模型以“学习” IMDb 电影评论“语言”的结构。
选择合适的学习率。

learn_lm.lr_find()

LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.

learn_lm.recorder.plot(skip_end=15)

lr = 1e-3
lr *= bs/48

如果有的话，请使用混合精度选项，否则省略此步骤

learn_lm.to_fp16()

LanguageLearner(data=TextLMDataBunch;

Train: LabelList (90000 items)
x: LMTextList
  xxwrep 18 xxbos xxmaj once again xxmaj mr. xxmaj costner has dragged out a movie for far longer than necessary . xxmaj aside from the terrific sea rescue sequences , of which there are very few i just did not care about any of the characters . xxmaj most of us have ghosts in the closet , and xxmaj costner 's character are realized early on , and then forgotten until much later , by which time i did not care . xxmaj the character we should really care about is a very cocky , overconfident xxmaj ashton xxmaj kutcher . xxmaj the problem is he comes off as kid who thinks he 's better than anyone else around him and shows no signs of a cluttered closet . xxmaj his only obstacle appears to be winning over xxmaj costner . xxmaj finally when we are well past the half way point of this stinker , xxmaj costner tells us all about xxmaj kutcher 's ghosts . xxmaj we are told why xxmaj kutcher is driven to be the best with no prior inkling or foreshadowing . xxmaj no magic here , it was all i could do to keep from turning it off an hour in .,  xxwrep 18 xxbos xxmaj this is an example of why the majority of action films are the same . xxmaj generic and boring , there 's really nothing worth watching here . a complete waste of the then barely - tapped talents of xxmaj ice - t and xxmaj ice xxmaj cube , who 've each proven many times over that they are capable of acting , and acting well . xxmaj do n't bother with this one , go see xxmaj new xxmaj jack xxmaj city , xxmaj ricochet or watch xxmaj new xxmaj york xxmaj undercover for xxmaj ice - t , or xxmaj boyz n the xxmaj hood , xxmaj higher xxmaj learning or xxmaj friday for xxmaj ice xxmaj cube and see the real deal . xxmaj ice - t 's horribly cliched dialogue alone makes this film grate at the teeth , and i 'm still wondering what the heck xxmaj bill xxmaj paxton was doing in this film ? xxmaj and why the heck does he always play the exact same character ? xxmaj from xxmaj aliens onward , every film i 've seen with xxmaj bill xxmaj paxton has him playing the exact same irritating character , and at least in xxmaj aliens his character died , which made it somewhat gratifying ... 
 
  xxmaj overall , this is second - rate action trash . xxmaj there are countless better films to see , and if you really want to see this one , watch xxmaj judgement xxmaj night , which is practically a carbon copy but has better acting and a better script . xxmaj the only thing that made this at all worth watching was a decent hand on the camera - the cinematography was almost refreshing , which comes close to making up for the horrible film itself - but not quite . 4 / 10 .,  xxwrep 18 xxbos xxmaj first of all i hate those moronic rappers , who could'nt act if they had a gun pressed against their foreheads . xxmaj all they do is curse and shoot each other and acting like xxunk version of gangsters . 
 
  xxmaj the movie does n't take more than five minutes to explain what is going on before we 're already at the warehouse xxmaj there is not a single sympathetic character in this movie , except for the homeless guy , who is also the only one with half a brain . 
 
  xxmaj bill xxmaj paxton and xxmaj william xxmaj sadler are both hill billies and xxmaj xxunk character is just as much a villain as the gangsters . i did'nt like him right from the start . 
 
  xxmaj the movie is filled with pointless violence and xxmaj walter xxmaj hills specialty : people falling through windows with glass flying everywhere . xxmaj there is pretty much no plot and it is a big problem when you root for no - one . xxmaj everybody dies , except from xxmaj paxton and the homeless guy and everybody get what they deserve . 
 
  xxmaj the only two black people that can act is the homeless guy and the junkie but they 're actors by profession , not annoying ugly brain dead rappers . 
 
  xxmaj stay away from this crap and watch 48 hours 1 and 2 instead . xxmaj at lest they have characters you care about , a sense of humor and nothing but real actors in the cast .,  xxwrep 18 xxbos xxmaj not even the xxmaj beatles could write songs everyone liked , and although xxmaj walter xxmaj hill is no mop - top he 's second to none when it comes to thought provoking action movies . xxmaj the nineties came and social platforms were changing in music and film , the emergence of the xxmaj rapper turned movie star was in full swing , the acting took a back seat to each man 's overpowering regional accent and transparent acting . xxmaj this was one of the many ice - t movies i saw as a kid and loved , only to watch them later and cringe . xxmaj bill xxmaj paxton and xxmaj william xxmaj sadler are firemen with basic lives until a burning building tenant about to go up in flames hands over a map with gold implications . i hand it to xxmaj walter for quickly and neatly setting up the main characters and location . xxmaj but i fault everyone involved for turning out xxmaj lame - o performances . xxmaj ice - t and cube must have been red hot at this time , and while i 've enjoyed both their careers as rappers , in my opinion they fell flat in this movie . xxmaj it 's about ninety minutes of one guy ridiculously turning his back on the other guy to the point you find yourself locked in multiple states of disbelief . xxmaj now this is a movie , its not a documentary so i wo nt waste my time recounting all the stupid plot twists in this movie , but there were many , and they led nowhere . i got the feeling watching this that everyone on set was xxunk of confused and just playing things off the cuff . xxmaj there are two things i still enjoy about it , one involves a scene with a needle and the other is xxmaj sadler 's huge 45 pistol . xxmaj bottom line this movie is like domino 's pizza . xxmaj yeah ill eat it if i 'm hungry and i do n't feel like cooking , xxmaj but i 'm well aware it tastes like crap . 3 stars , meh .,  xxwrep 18 xxbos xxmaj brass pictures ( movies is not a fitting word for them ) really are somewhat brassy . xxmaj their alluring visual qualities are reminiscent of expensive high class xxup tv commercials . xxmaj but unfortunately xxmaj brass pictures are feature films with the pretense of wanting to entertain viewers for over two hours ! xxmaj in this they fail miserably , their undeniable , but rather soft and flabby than steamy , erotic qualities non withstanding . 
 
  xxmaj senso ' 45 is a remake of a film by xxmaj luchino xxmaj visconti with the same title and xxmaj alida xxmaj valli and xxmaj farley xxmaj granger in the lead . xxmaj the original tells a story of senseless love and lust in and around xxmaj venice during the xxmaj italian wars of independence . xxmaj brass moved the action from the 19th into the 20th century , 1945 to be exact , so there are xxmaj mussolini murals , men in black shirts , xxmaj german uniforms or the tattered garb of the partisans . xxmaj but it is just window dressing , the historic context is completely negligible . 
 
  xxmaj anna xxmaj xxunk plays the attractive aristocratic woman who falls for the amoral xxup ss guy who always puts on too much lipstick . xxmaj she is an attractive , versatile , well trained xxmaj italian actress and clearly above the material . xxmaj her wide range of facial expressions ( xxunk boredom , loathing , delight , fear , hate ... and ecstasy ) are the best reason to watch this picture and worth two stars . xxmaj she endures this basically trashy stuff with an astonishing amount of dignity . i wish some really good parts come along for her . xxmaj she really deserves it .
y: LMLabelList
,,,,
Path: C:\Users\cross-entropy\.fastai\data\imdb;

Valid: LabelList (10000 items)
x: LMTextList
xxbos xxup cavite is an example of ultimate independent film , with a very short budget , a very simple concept , an exotic locale , a minimal cast , and a hand - held camera . 
 
  xxmaj the story is simple : xxmaj adam ( xxmaj ian xxmaj gamazon ) is called home to the xxmaj phillipines because of a family crisis . xxmaj instead of his family picking him up , he finds himself forced to follow instructions of a man claiming to have his family . xxmaj there 's no clear reason for the abduction , or what makes xxmaj adam a target ; all xxmaj adam really knows is that his every move is watched , and the kidnappers have no regard for their victims . 
 
  xxmaj as xxmaj adam follows the obscure instructions , and the obstacles in his way , the audience ca n't help but be caught up in his plight . xxmaj the hand held camera and jumpy editing style enhances the sense of desperation and time . xxmaj the scenes of urban xxmaj phillipines , particular the markets and the squatter holdings are a vivid cacophony . 
 
  xxmaj co - directors and xxmaj co - writers xxmaj neill xxmaj xxunk xxmaj xxunk and xxmaj ian xxmaj gamazon have done an outstanding job of making the most out of limited means . xxmaj the economy of the film makes it both intimate and discomfiting , as xxmaj adam is an everyman who only wants his family safe and instead is completely at the whim of an omniscient tormentor . 
 
  xxup cavite is an absolute must for anyone who has an interest in film , as storytelling , in it 's structure , and as an art form .,xxbos xxmaj decades ago , a crate filled with weapons grade plutonium crashes on an island and soaks into the ground . xxmaj today , a team of military men are sent to track down a notorious terrorist ( of ambiguous national origin ) and they track him to this polluted island . xxmaj when their raft is destroyed , the team must spend the night on shore , but soon discover that the plutonium has done something awful to the island -- it has called forth hundreds of bloodthirsty velociraptors . 
 
  xxmaj let me start this with a lesson : do n't lend a movie to your friend before you 've seen it , especially if you are supposed to be reviewing it for the internet 's finest horror movie site . xxmaj it took me almost a year to get this film back , and the person who borrowed it still had not watched it ( though we ended up seeing it together ) . xxmaj and a second , more important , lesson : when you do watch this , keep your expectations as low as humanly possible . xxmaj because this film ranks among the worst i 've ever seen . 
 
  xxmaj my acting in 8th grade was more convincing than the seasoned actors who appear in this film ( xxmaj lorenzo xxmaj lamas , xxmaj stephen xxmaj bauer ) . xxmaj line delivery is very fake , and the words themselves are poorly scripted . xxmaj the opening words come from a man checking out his gun 's scope : " xxmaj boom . xxmaj dead bad guys . " xxmaj yes , that 's pure genius at work . xxmaj the only conversation with any depth has two main characters explaining their histories . xxmaj but it , too , seems unnatural and a poor attempt to provide character background and to fill time . xxmaj we did n't need to know anything about their histories , so why bore us with it ? xxmaj and if you think the conversations are bad , you ai n't seen nothing yet . 
 
  xxmaj the lighting is atrocious . i generally do n't notice lighting , but my friend ( a former film school student ) was practically vomiting in rage at the way more often than not shadows fell on the actors ' faces and the light would be in the background , focused on nothing in particular . xxmaj most lighting looks like a spotlight in a dim room , and many of the scenes involve a deep , subterranean cavern -- which you 'd then expect to be poorly lit , but had lights coming from all sorts of random angles . xxmaj do n't ask me why . 
 
  xxmaj the plot was pretty bad . xxmaj some films can take the idea of military men chasing a terrorist and make a convincing film out of it . xxmaj cat and mouse stories are riveting . xxmaj well , not here . xxmaj the terrorist is really not even part of the story , just an excuse to go to the island . xxmaj and the raptors ? xxmaj and the allosaurus ? xxmaj sure , they came from the plutonium that soaked into the ground . xxmaj but if that makes sense to you , please explain it to me because i have no clue how radiation brings dinosaurs back from millions of years of extinction . 
 
  xxmaj by far the worst part of " xxmaj raptor xxmaj island " is the animation of the raptors . xxmaj that 's right -- the selling point of the film is the worst aspect . xxmaj the animation is n't just bad , it 's subpar . i ca n't even express the hilarity of cartoons this cheesy . xxmaj and when they get shot ? xxmaj red splats like one would see in an old video game . xxmaj even the airplane , helicopter and xxmaj navy ship are cartoons ... how hard is it to get a model plane ? xxmaj please do n't see " xxmaj raptor xxmaj island " unless you need a good laugh or want to get sickeningly drunk . xxmaj sure , you probably want to see it before you see " xxmaj raptor xxmaj island 2 " ( which seems to be getting better reviews ) . xxmaj but just avoiding it entirely is your best bet . xxmaj the closest thing i can compare it to is " xxmaj pinata : xxmaj survival xxmaj island " , and unfortunately this film makes the pinata look good by comparison . xxmaj you have been warned .,xxbos xxmaj this is one creepy underrated xxmaj gem with chilling performances and a fantastic xxunk xxmaj all the characters are great , and the story was awesome , plus i thought the ending was really xxunk xxmaj the plot was great , and it never bored me , plus while the child actors were bad , they gave me the xxunk xxmaj this happened to be on the space channel a while ago , so i decided to check it out and tape it , i read some good reviews from fellow horror fans , i must say i agree with them , it 's very creepy , and suspenseful , plus xxmaj strother xxmaj martin , was fantastic in his role , as the xxmaj satan worshiper . xxmaj it has tons of creepy atmosphere , and it keeps you guessing throughout , plus all the characters were very likable , and you really start to root for xxmaj ben and his xxunk xxmaj it has plenty of disturbing moments , and the film really shocked me at times , plus , it 's extremely well made on a low xxunk xxmaj this is one creepy underrated gem , with chilling performances and a fantastic finale ! , i highly recommend this one!. xxmaj the xxmaj direction is very good!. xxmaj bernard mceveety does a very good job here , with great camera work , creating a lot of creepy atmosphere , and keeping the film at a very fast pace!. xxmaj ther is a little bit of blood and gore . xxmaj we get a severed leg , lots of bloody corpses , bloody slit throat , slicing and dicing , decapitation , and an impaling . xxmaj the xxmaj acting is xxunk xxmaj strother xxmaj martin is fantastic here ! as the xxmaj satan worshiper , he is extremely creepy , very convincing , was quite chilling , was extremely intense , seemed to be enjoying himself , and just did a fantastic job xxunk xxmaj charles xxmaj bateman is great as the xxmaj dad , he was very caring , very likable , and gave a good show ! , i liked him lots . xxup l.q. xxmaj jones is awesome as the xxmaj sheriff , he was funny , on top of things , looked very young , had a cool character , and just did an awesome job xxunk xxmaj ahna xxmaj capri is good as the girlfriend and did what she had to do pretty well . xxmaj charles xxmaj robinson overacted to the extreme as the xxmaj priest and did n't convince me one bit ! , and that laugh of his was especially bad . xxmaj geri xxmaj reischl is actually decent as the daughter , she was somewhat likable , and only got on my nerves a couple times , i rather liked her . xxmaj alvy xxmaj moore was goofy , but very likable in his role as xxmaj tobey i dug him!. xxmaj rest of the cast do good . xxmaj overall i highly recommend it!. * * * 1 / 2 out of 5,xxbos xxmaj aside from the gross factual inaccuracies of this movie ( ie king xxmaj richard having a son and that son replacing xxmaj john on the throne ) this movie was a sweet love tale full of adventure . i 'm sure that this movie will appeal to the younger generation of girls and boys .,xxbos xxmaj the special effects again are superb but this is xxmaj finding xxmaj nemo in reverse i.e the parents get taken away and xxmaj pi ( xxmaj nemo ) is left behind , if you have n't seen xxmaj finding xxmaj nemo you will like it . 
 
  xxmaj it becomes very clear at the beginning of the film what the plot is and takes a long time to reach the end ! 
 
  xxmaj there 's is nothing new or nail biting in this one and there is no humour at all , which is very disappointing . 
 
  xxmaj all in all very disappointing as a follow up movie to xxmaj shark xxmaj tale which there really are no similarities with it should have been called xxmaj finding xxmaj nemo 2 xxrep 56 !
y: LMLabelList
,,,,
Path: C:\Users\cross-entropy\.fastai\data\imdb;

Test: None, model=SequentialRNN(
  (0): AWD_LSTM(
    (encoder): Embedding(60000, 400, padding_idx=1)
    (encoder_dp): EmbeddingDropout(
      (emb): Embedding(60000, 400, padding_idx=1)
    )
    (rnns): ModuleList(
      (0): WeightDropout(
        (module): LSTM(400, 1152, batch_first=True)
      )
      (1): WeightDropout(
        (module): LSTM(1152, 1152, batch_first=True)
      )
      (2): WeightDropout(
        (module): LSTM(1152, 400, batch_first=True)
      )
    )
    (input_dp): RNNDropout()
    (hidden_dps): ModuleList(
      (0): RNNDropout()
      (1): RNNDropout()
      (2): RNNDropout()
    )
  )
  (1): LinearDecoder(
    (decoder): Linear(in_features=400, out_features=60000, bias=True)
    (output_dp): RNNDropout()
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99)), loss_func=FlattenedLoss of CrossEntropyLoss(), metrics=[<function accuracy at 0x00000236A8585048>], true_wd=True, bn_wd=True, wd=0.01, train_bn=True, path=WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb'), model_dir='models', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False)], callbacks=[RNNTrainer
learn: ...
alpha: 2.0
beta: 1.0, MixedPrecision
learn: ...
loss_scale: 65536
max_noskip: 1000
dynamic: True
clip: None
flat_master: False
max_scale: 16777216
loss_fp32: True], layer_groups=[Sequential(
  (0): WeightDropout(
    (module): LSTM(400, 1152, batch_first=True)
  )
  (1): RNNDropout()
), Sequential(
  (0): WeightDropout(
    (module): LSTM(1152, 1152, batch_first=True)
  )
  (1): RNNDropout()
), Sequential(
  (0): WeightDropout(
    (module): LSTM(1152, 400, batch_first=True)
  )
  (1): RNNDropout()
), Sequential(
  (0): Embedding(60000, 400, padding_idx=1)
  (1): EmbeddingDropout(
    (emb): Embedding(60000, 400, padding_idx=1)
  )
  (2): LinearDecoder(
    (decoder): Linear(in_features=400, out_features=60000, bias=True)
    (output_dp): RNNDropout()
  )
)], add_time=True, silent=False)

微调的第一步是仅训练模型的最后一层。

在 NVIDIA RTX-2070 GPU 上，这大约需要半小时

learn_lm.fit_one_cycle(1, lr*10, moms=(0.8,0.7))

由于训练速度相对较慢，我们将保存我们的权重：

learn_lm.save('fit_1')

learn_lm.load('fit_1')

LanguageLearner(data=TextLMDataBunch;

Train: LabelList (90000 items)
x: LMTextList
  xxwrep 18 xxbos xxmaj once again xxmaj mr. xxmaj costner has dragged out a movie for far longer than necessary . xxmaj aside from the terrific sea rescue sequences , of which there are very few i just did not care about any of the characters . xxmaj most of us have ghosts in the closet , and xxmaj costner 's character are realized early on , and then forgotten until much later , by which time i did not care . xxmaj the character we should really care about is a very cocky , overconfident xxmaj ashton xxmaj kutcher . xxmaj the problem is he comes off as kid who thinks he 's better than anyone else around him and shows no signs of a cluttered closet . xxmaj his only obstacle appears to be winning over xxmaj costner . xxmaj finally when we are well past the half way point of this stinker , xxmaj costner tells us all about xxmaj kutcher 's ghosts . xxmaj we are told why xxmaj kutcher is driven to be the best with no prior inkling or foreshadowing . xxmaj no magic here , it was all i could do to keep from turning it off an hour in .,  xxwrep 18 xxbos xxmaj this is an example of why the majority of action films are the same . xxmaj generic and boring , there 's really nothing worth watching here . a complete waste of the then barely - tapped talents of xxmaj ice - t and xxmaj ice xxmaj cube , who 've each proven many times over that they are capable of acting , and acting well . xxmaj do n't bother with this one , go see xxmaj new xxmaj jack xxmaj city , xxmaj ricochet or watch xxmaj new xxmaj york xxmaj undercover for xxmaj ice - t , or xxmaj boyz n the xxmaj hood , xxmaj higher xxmaj learning or xxmaj friday for xxmaj ice xxmaj cube and see the real deal . xxmaj ice - t 's horribly cliched dialogue alone makes this film grate at the teeth , and i 'm still wondering what the heck xxmaj bill xxmaj paxton was doing in this film ? xxmaj and why the heck does he always play the exact same character ? xxmaj from xxmaj aliens onward , every film i 've seen with xxmaj bill xxmaj paxton has him playing the exact same irritating character , and at least in xxmaj aliens his character died , which made it somewhat gratifying ... 
 
  xxmaj overall , this is second - rate action trash . xxmaj there are countless better films to see , and if you really want to see this one , watch xxmaj judgement xxmaj night , which is practically a carbon copy but has better acting and a better script . xxmaj the only thing that made this at all worth watching was a decent hand on the camera - the cinematography was almost refreshing , which comes close to making up for the horrible film itself - but not quite . 4 / 10 .,  xxwrep 18 xxbos xxmaj first of all i hate those moronic rappers , who could'nt act if they had a gun pressed against their foreheads . xxmaj all they do is curse and shoot each other and acting like xxunk version of gangsters . 
 
  xxmaj the movie does n't take more than five minutes to explain what is going on before we 're already at the warehouse xxmaj there is not a single sympathetic character in this movie , except for the homeless guy , who is also the only one with half a brain . 
 
  xxmaj bill xxmaj paxton and xxmaj william xxmaj sadler are both hill billies and xxmaj xxunk character is just as much a villain as the gangsters . i did'nt like him right from the start . 
 
  xxmaj the movie is filled with pointless violence and xxmaj walter xxmaj hills specialty : people falling through windows with glass flying everywhere . xxmaj there is pretty much no plot and it is a big problem when you root for no - one . xxmaj everybody dies , except from xxmaj paxton and the homeless guy and everybody get what they deserve . 
 
  xxmaj the only two black people that can act is the homeless guy and the junkie but they 're actors by profession , not annoying ugly brain dead rappers . 
 
  xxmaj stay away from this crap and watch 48 hours 1 and 2 instead . xxmaj at lest they have characters you care about , a sense of humor and nothing but real actors in the cast .,  xxwrep 18 xxbos xxmaj not even the xxmaj beatles could write songs everyone liked , and although xxmaj walter xxmaj hill is no mop - top he 's second to none when it comes to thought provoking action movies . xxmaj the nineties came and social platforms were changing in music and film , the emergence of the xxmaj rapper turned movie star was in full swing , the acting took a back seat to each man 's overpowering regional accent and transparent acting . xxmaj this was one of the many ice - t movies i saw as a kid and loved , only to watch them later and cringe . xxmaj bill xxmaj paxton and xxmaj william xxmaj sadler are firemen with basic lives until a burning building tenant about to go up in flames hands over a map with gold implications . i hand it to xxmaj walter for quickly and neatly setting up the main characters and location . xxmaj but i fault everyone involved for turning out xxmaj lame - o performances . xxmaj ice - t and cube must have been red hot at this time , and while i 've enjoyed both their careers as rappers , in my opinion they fell flat in this movie . xxmaj it 's about ninety minutes of one guy ridiculously turning his back on the other guy to the point you find yourself locked in multiple states of disbelief . xxmaj now this is a movie , its not a documentary so i wo nt waste my time recounting all the stupid plot twists in this movie , but there were many , and they led nowhere . i got the feeling watching this that everyone on set was xxunk of confused and just playing things off the cuff . xxmaj there are two things i still enjoy about it , one involves a scene with a needle and the other is xxmaj sadler 's huge 45 pistol . xxmaj bottom line this movie is like domino 's pizza . xxmaj yeah ill eat it if i 'm hungry and i do n't feel like cooking , xxmaj but i 'm well aware it tastes like crap . 3 stars , meh .,  xxwrep 18 xxbos xxmaj brass pictures ( movies is not a fitting word for them ) really are somewhat brassy . xxmaj their alluring visual qualities are reminiscent of expensive high class xxup tv commercials . xxmaj but unfortunately xxmaj brass pictures are feature films with the pretense of wanting to entertain viewers for over two hours ! xxmaj in this they fail miserably , their undeniable , but rather soft and flabby than steamy , erotic qualities non withstanding . 
 
  xxmaj senso ' 45 is a remake of a film by xxmaj luchino xxmaj visconti with the same title and xxmaj alida xxmaj valli and xxmaj farley xxmaj granger in the lead . xxmaj the original tells a story of senseless love and lust in and around xxmaj venice during the xxmaj italian wars of independence . xxmaj brass moved the action from the 19th into the 20th century , 1945 to be exact , so there are xxmaj mussolini murals , men in black shirts , xxmaj german uniforms or the tattered garb of the partisans . xxmaj but it is just window dressing , the historic context is completely negligible . 
 
  xxmaj anna xxmaj xxunk plays the attractive aristocratic woman who falls for the amoral xxup ss guy who always puts on too much lipstick . xxmaj she is an attractive , versatile , well trained xxmaj italian actress and clearly above the material . xxmaj her wide range of facial expressions ( xxunk boredom , loathing , delight , fear , hate ... and ecstasy ) are the best reason to watch this picture and worth two stars . xxmaj she endures this basically trashy stuff with an astonishing amount of dignity . i wish some really good parts come along for her . xxmaj she really deserves it .
y: LMLabelList
,,,,
Path: C:\Users\cross-entropy\.fastai\data\imdb;

Valid: LabelList (10000 items)
x: LMTextList
xxbos xxup cavite is an example of ultimate independent film , with a very short budget , a very simple concept , an exotic locale , a minimal cast , and a hand - held camera . 
 
  xxmaj the story is simple : xxmaj adam ( xxmaj ian xxmaj gamazon ) is called home to the xxmaj phillipines because of a family crisis . xxmaj instead of his family picking him up , he finds himself forced to follow instructions of a man claiming to have his family . xxmaj there 's no clear reason for the abduction , or what makes xxmaj adam a target ; all xxmaj adam really knows is that his every move is watched , and the kidnappers have no regard for their victims . 
 
  xxmaj as xxmaj adam follows the obscure instructions , and the obstacles in his way , the audience ca n't help but be caught up in his plight . xxmaj the hand held camera and jumpy editing style enhances the sense of desperation and time . xxmaj the scenes of urban xxmaj phillipines , particular the markets and the squatter holdings are a vivid cacophony . 
 
  xxmaj co - directors and xxmaj co - writers xxmaj neill xxmaj xxunk xxmaj xxunk and xxmaj ian xxmaj gamazon have done an outstanding job of making the most out of limited means . xxmaj the economy of the film makes it both intimate and discomfiting , as xxmaj adam is an everyman who only wants his family safe and instead is completely at the whim of an omniscient tormentor . 
 
  xxup cavite is an absolute must for anyone who has an interest in film , as storytelling , in it 's structure , and as an art form .,xxbos xxmaj decades ago , a crate filled with weapons grade plutonium crashes on an island and soaks into the ground . xxmaj today , a team of military men are sent to track down a notorious terrorist ( of ambiguous national origin ) and they track him to this polluted island . xxmaj when their raft is destroyed , the team must spend the night on shore , but soon discover that the plutonium has done something awful to the island -- it has called forth hundreds of bloodthirsty velociraptors . 
 
  xxmaj let me start this with a lesson : do n't lend a movie to your friend before you 've seen it , especially if you are supposed to be reviewing it for the internet 's finest horror movie site . xxmaj it took me almost a year to get this film back , and the person who borrowed it still had not watched it ( though we ended up seeing it together ) . xxmaj and a second , more important , lesson : when you do watch this , keep your expectations as low as humanly possible . xxmaj because this film ranks among the worst i 've ever seen . 
 
  xxmaj my acting in 8th grade was more convincing than the seasoned actors who appear in this film ( xxmaj lorenzo xxmaj lamas , xxmaj stephen xxmaj bauer ) . xxmaj line delivery is very fake , and the words themselves are poorly scripted . xxmaj the opening words come from a man checking out his gun 's scope : " xxmaj boom . xxmaj dead bad guys . " xxmaj yes , that 's pure genius at work . xxmaj the only conversation with any depth has two main characters explaining their histories . xxmaj but it , too , seems unnatural and a poor attempt to provide character background and to fill time . xxmaj we did n't need to know anything about their histories , so why bore us with it ? xxmaj and if you think the conversations are bad , you ai n't seen nothing yet . 
 
  xxmaj the lighting is atrocious . i generally do n't notice lighting , but my friend ( a former film school student ) was practically vomiting in rage at the way more often than not shadows fell on the actors ' faces and the light would be in the background , focused on nothing in particular . xxmaj most lighting looks like a spotlight in a dim room , and many of the scenes involve a deep , subterranean cavern -- which you 'd then expect to be poorly lit , but had lights coming from all sorts of random angles . xxmaj do n't ask me why . 
 
  xxmaj the plot was pretty bad . xxmaj some films can take the idea of military men chasing a terrorist and make a convincing film out of it . xxmaj cat and mouse stories are riveting . xxmaj well , not here . xxmaj the terrorist is really not even part of the story , just an excuse to go to the island . xxmaj and the raptors ? xxmaj and the allosaurus ? xxmaj sure , they came from the plutonium that soaked into the ground . xxmaj but if that makes sense to you , please explain it to me because i have no clue how radiation brings dinosaurs back from millions of years of extinction . 
 
  xxmaj by far the worst part of " xxmaj raptor xxmaj island " is the animation of the raptors . xxmaj that 's right -- the selling point of the film is the worst aspect . xxmaj the animation is n't just bad , it 's subpar . i ca n't even express the hilarity of cartoons this cheesy . xxmaj and when they get shot ? xxmaj red splats like one would see in an old video game . xxmaj even the airplane , helicopter and xxmaj navy ship are cartoons ... how hard is it to get a model plane ? xxmaj please do n't see " xxmaj raptor xxmaj island " unless you need a good laugh or want to get sickeningly drunk . xxmaj sure , you probably want to see it before you see " xxmaj raptor xxmaj island 2 " ( which seems to be getting better reviews ) . xxmaj but just avoiding it entirely is your best bet . xxmaj the closest thing i can compare it to is " xxmaj pinata : xxmaj survival xxmaj island " , and unfortunately this film makes the pinata look good by comparison . xxmaj you have been warned .,xxbos xxmaj this is one creepy underrated xxmaj gem with chilling performances and a fantastic xxunk xxmaj all the characters are great , and the story was awesome , plus i thought the ending was really xxunk xxmaj the plot was great , and it never bored me , plus while the child actors were bad , they gave me the xxunk xxmaj this happened to be on the space channel a while ago , so i decided to check it out and tape it , i read some good reviews from fellow horror fans , i must say i agree with them , it 's very creepy , and suspenseful , plus xxmaj strother xxmaj martin , was fantastic in his role , as the xxmaj satan worshiper . xxmaj it has tons of creepy atmosphere , and it keeps you guessing throughout , plus all the characters were very likable , and you really start to root for xxmaj ben and his xxunk xxmaj it has plenty of disturbing moments , and the film really shocked me at times , plus , it 's extremely well made on a low xxunk xxmaj this is one creepy underrated gem , with chilling performances and a fantastic finale ! , i highly recommend this one!. xxmaj the xxmaj direction is very good!. xxmaj bernard mceveety does a very good job here , with great camera work , creating a lot of creepy atmosphere , and keeping the film at a very fast pace!. xxmaj ther is a little bit of blood and gore . xxmaj we get a severed leg , lots of bloody corpses , bloody slit throat , slicing and dicing , decapitation , and an impaling . xxmaj the xxmaj acting is xxunk xxmaj strother xxmaj martin is fantastic here ! as the xxmaj satan worshiper , he is extremely creepy , very convincing , was quite chilling , was extremely intense , seemed to be enjoying himself , and just did a fantastic job xxunk xxmaj charles xxmaj bateman is great as the xxmaj dad , he was very caring , very likable , and gave a good show ! , i liked him lots . xxup l.q. xxmaj jones is awesome as the xxmaj sheriff , he was funny , on top of things , looked very young , had a cool character , and just did an awesome job xxunk xxmaj ahna xxmaj capri is good as the girlfriend and did what she had to do pretty well . xxmaj charles xxmaj robinson overacted to the extreme as the xxmaj priest and did n't convince me one bit ! , and that laugh of his was especially bad . xxmaj geri xxmaj reischl is actually decent as the daughter , she was somewhat likable , and only got on my nerves a couple times , i rather liked her . xxmaj alvy xxmaj moore was goofy , but very likable in his role as xxmaj tobey i dug him!. xxmaj rest of the cast do good . xxmaj overall i highly recommend it!. * * * 1 / 2 out of 5,xxbos xxmaj aside from the gross factual inaccuracies of this movie ( ie king xxmaj richard having a son and that son replacing xxmaj john on the throne ) this movie was a sweet love tale full of adventure . i 'm sure that this movie will appeal to the younger generation of girls and boys .,xxbos xxmaj the special effects again are superb but this is xxmaj finding xxmaj nemo in reverse i.e the parents get taken away and xxmaj pi ( xxmaj nemo ) is left behind , if you have n't seen xxmaj finding xxmaj nemo you will like it . 
 
  xxmaj it becomes very clear at the beginning of the film what the plot is and takes a long time to reach the end ! 
 
  xxmaj there 's is nothing new or nail biting in this one and there is no humour at all , which is very disappointing . 
 
  xxmaj all in all very disappointing as a follow up movie to xxmaj shark xxmaj tale which there really are no similarities with it should have been called xxmaj finding xxmaj nemo 2 xxrep 56 !
y: LMLabelList
,,,,
Path: C:\Users\cross-entropy\.fastai\data\imdb;

Test: None, model=SequentialRNN(
  (0): AWD_LSTM(
    (encoder): Embedding(60000, 400, padding_idx=1)
    (encoder_dp): EmbeddingDropout(
      (emb): Embedding(60000, 400, padding_idx=1)
    )
    (rnns): ModuleList(
      (0): WeightDropout(
        (module): LSTM(400, 1152, batch_first=True)
      )
      (1): WeightDropout(
        (module): LSTM(1152, 1152, batch_first=True)
      )
      (2): WeightDropout(
        (module): LSTM(1152, 400, batch_first=True)
      )
    )
    (input_dp): RNNDropout()
    (hidden_dps): ModuleList(
      (0): RNNDropout()
      (1): RNNDropout()
      (2): RNNDropout()
    )
  )
  (1): LinearDecoder(
    (decoder): Linear(in_features=400, out_features=60000, bias=True)
    (output_dp): RNNDropout()
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99)), loss_func=FlattenedLoss of CrossEntropyLoss(), metrics=[<function accuracy at 0x00000236A8585048>], true_wd=True, bn_wd=True, wd=0.01, train_bn=True, path=WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb'), model_dir='models', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False)], callbacks=[RNNTrainer
learn: ...
alpha: 2.0
beta: 1.0, MixedPrecision
learn: ...
loss_scale: 2097152.0
max_noskip: 1000
dynamic: True
clip: None
flat_master: False
max_scale: 16777216
loss_fp32: True], layer_groups=[Sequential(
  (0): WeightDropout(
    (module): LSTM(400, 1152, batch_first=True)
  )
  (1): RNNDropout()
), Sequential(
  (0): WeightDropout(
    (module): LSTM(1152, 1152, batch_first=True)
  )
  (1): RNNDropout()
), Sequential(
  (0): WeightDropout(
    (module): LSTM(1152, 400, batch_first=True)
  )
  (1): RNNDropout()
), Sequential(
  (0): Embedding(60000, 400, padding_idx=1)
  (1): EmbeddingDropout(
    (emb): Embedding(60000, 400, padding_idx=1)
  )
  (2): LinearDecoder(
    (decoder): Linear(in_features=400, out_features=60000, bias=True)
    (output_dp): RNNDropout()
  )
)], add_time=True, silent=False)

为了完成微调，我们解冻所有权重并重新训练

采用 wikitext-103 权重作为初始值，我们的神经网络将通过优化对其进行调整，找到专门针对 IMDb 电影评论“语言”的新值。

learn_lm.unfreeze()

在 NVIDIA RTX-2070 GPU 上，对模型进行微调每个时期大约需要 30 分钟，bs=48
请注意，准确度值相对较低，在第 4 个时期之后没有显着提高。

learn_lm.fit_one_cycle(10, lr, moms=(0.8,0.7))

保存微调后的语言模型和编码器

我们不仅要保存经过微调的 IMDb 语言模型，还要保存其编码器。语言模型是试图猜测下一个单词的部分。编码器是负责创建和更新隐藏状态的部分。

在下一部分中，我们将为 IMDb 电影评论构建一个情感分类器。为此，我们需要从我们构建的 IMDb 语言模型中获取编码器。

learn_lm.save('fine_tuned')

learn_lm.save_encoder('fine_tuned_enc')

加载已保存的模型及其编码器

learn_lm.load('fine_tuned')

LanguageLearner(data=TextLMDataBunch;

Train: LabelList (90000 items)
x: LMTextList
  xxwrep 18 xxbos xxmaj once again xxmaj mr. xxmaj costner has dragged out a movie for far longer than necessary . xxmaj aside from the terrific sea rescue sequences , of which there are very few i just did not care about any of the characters . xxmaj most of us have ghosts in the closet , and xxmaj costner 's character are realized early on , and then forgotten until much later , by which time i did not care . xxmaj the character we should really care about is a very cocky , overconfident xxmaj ashton xxmaj kutcher . xxmaj the problem is he comes off as kid who thinks he 's better than anyone else around him and shows no signs of a cluttered closet . xxmaj his only obstacle appears to be winning over xxmaj costner . xxmaj finally when we are well past the half way point of this stinker , xxmaj costner tells us all about xxmaj kutcher 's ghosts . xxmaj we are told why xxmaj kutcher is driven to be the best with no prior inkling or foreshadowing . xxmaj no magic here , it was all i could do to keep from turning it off an hour in .,  xxwrep 18 xxbos xxmaj this is an example of why the majority of action films are the same . xxmaj generic and boring , there 's really nothing worth watching here . a complete waste of the then barely - tapped talents of xxmaj ice - t and xxmaj ice xxmaj cube , who 've each proven many times over that they are capable of acting , and acting well . xxmaj do n't bother with this one , go see xxmaj new xxmaj jack xxmaj city , xxmaj ricochet or watch xxmaj new xxmaj york xxmaj undercover for xxmaj ice - t , or xxmaj boyz n the xxmaj hood , xxmaj higher xxmaj learning or xxmaj friday for xxmaj ice xxmaj cube and see the real deal . xxmaj ice - t 's horribly cliched dialogue alone makes this film grate at the teeth , and i 'm still wondering what the heck xxmaj bill xxmaj paxton was doing in this film ? xxmaj and why the heck does he always play the exact same character ? xxmaj from xxmaj aliens onward , every film i 've seen with xxmaj bill xxmaj paxton has him playing the exact same irritating character , and at least in xxmaj aliens his character died , which made it somewhat gratifying ... 
 
  xxmaj overall , this is second - rate action trash . xxmaj there are countless better films to see , and if you really want to see this one , watch xxmaj judgement xxmaj night , which is practically a carbon copy but has better acting and a better script . xxmaj the only thing that made this at all worth watching was a decent hand on the camera - the cinematography was almost refreshing , which comes close to making up for the horrible film itself - but not quite . 4 / 10 .,  xxwrep 18 xxbos xxmaj first of all i hate those moronic rappers , who could'nt act if they had a gun pressed against their foreheads . xxmaj all they do is curse and shoot each other and acting like xxunk version of gangsters . 
 
  xxmaj the movie does n't take more than five minutes to explain what is going on before we 're already at the warehouse xxmaj there is not a single sympathetic character in this movie , except for the homeless guy , who is also the only one with half a brain . 
 
  xxmaj bill xxmaj paxton and xxmaj william xxmaj sadler are both hill billies and xxmaj xxunk character is just as much a villain as the gangsters . i did'nt like him right from the start . 
 
  xxmaj the movie is filled with pointless violence and xxmaj walter xxmaj hills specialty : people falling through windows with glass flying everywhere . xxmaj there is pretty much no plot and it is a big problem when you root for no - one . xxmaj everybody dies , except from xxmaj paxton and the homeless guy and everybody get what they deserve . 
 
  xxmaj the only two black people that can act is the homeless guy and the junkie but they 're actors by profession , not annoying ugly brain dead rappers . 
 
  xxmaj stay away from this crap and watch 48 hours 1 and 2 instead . xxmaj at lest they have characters you care about , a sense of humor and nothing but real actors in the cast .,  xxwrep 18 xxbos xxmaj not even the xxmaj beatles could write songs everyone liked , and although xxmaj walter xxmaj hill is no mop - top he 's second to none when it comes to thought provoking action movies . xxmaj the nineties came and social platforms were changing in music and film , the emergence of the xxmaj rapper turned movie star was in full swing , the acting took a back seat to each man 's overpowering regional accent and transparent acting . xxmaj this was one of the many ice - t movies i saw as a kid and loved , only to watch them later and cringe . xxmaj bill xxmaj paxton and xxmaj william xxmaj sadler are firemen with basic lives until a burning building tenant about to go up in flames hands over a map with gold implications . i hand it to xxmaj walter for quickly and neatly setting up the main characters and location . xxmaj but i fault everyone involved for turning out xxmaj lame - o performances . xxmaj ice - t and cube must have been red hot at this time , and while i 've enjoyed both their careers as rappers , in my opinion they fell flat in this movie . xxmaj it 's about ninety minutes of one guy ridiculously turning his back on the other guy to the point you find yourself locked in multiple states of disbelief . xxmaj now this is a movie , its not a documentary so i wo nt waste my time recounting all the stupid plot twists in this movie , but there were many , and they led nowhere . i got the feeling watching this that everyone on set was xxunk of confused and just playing things off the cuff . xxmaj there are two things i still enjoy about it , one involves a scene with a needle and the other is xxmaj sadler 's huge 45 pistol . xxmaj bottom line this movie is like domino 's pizza . xxmaj yeah ill eat it if i 'm hungry and i do n't feel like cooking , xxmaj but i 'm well aware it tastes like crap . 3 stars , meh .,  xxwrep 18 xxbos xxmaj brass pictures ( movies is not a fitting word for them ) really are somewhat brassy . xxmaj their alluring visual qualities are reminiscent of expensive high class xxup tv commercials . xxmaj but unfortunately xxmaj brass pictures are feature films with the pretense of wanting to entertain viewers for over two hours ! xxmaj in this they fail miserably , their undeniable , but rather soft and flabby than steamy , erotic qualities non withstanding . 
 
  xxmaj senso ' 45 is a remake of a film by xxmaj luchino xxmaj visconti with the same title and xxmaj alida xxmaj valli and xxmaj farley xxmaj granger in the lead . xxmaj the original tells a story of senseless love and lust in and around xxmaj venice during the xxmaj italian wars of independence . xxmaj brass moved the action from the 19th into the 20th century , 1945 to be exact , so there are xxmaj mussolini murals , men in black shirts , xxmaj german uniforms or the tattered garb of the partisans . xxmaj but it is just window dressing , the historic context is completely negligible . 
 
  xxmaj anna xxmaj xxunk plays the attractive aristocratic woman who falls for the amoral xxup ss guy who always puts on too much lipstick . xxmaj she is an attractive , versatile , well trained xxmaj italian actress and clearly above the material . xxmaj her wide range of facial expressions ( xxunk boredom , loathing , delight , fear , hate ... and ecstasy ) are the best reason to watch this picture and worth two stars . xxmaj she endures this basically trashy stuff with an astonishing amount of dignity . i wish some really good parts come along for her . xxmaj she really deserves it .
y: LMLabelList
,,,,
Path: C:\Users\cross-entropy\.fastai\data\imdb;

Valid: LabelList (10000 items)
x: LMTextList
xxbos xxup cavite is an example of ultimate independent film , with a very short budget , a very simple concept , an exotic locale , a minimal cast , and a hand - held camera . 
 
  xxmaj the story is simple : xxmaj adam ( xxmaj ian xxmaj gamazon ) is called home to the xxmaj phillipines because of a family crisis . xxmaj instead of his family picking him up , he finds himself forced to follow instructions of a man claiming to have his family . xxmaj there 's no clear reason for the abduction , or what makes xxmaj adam a target ; all xxmaj adam really knows is that his every move is watched , and the kidnappers have no regard for their victims . 
 
  xxmaj as xxmaj adam follows the obscure instructions , and the obstacles in his way , the audience ca n't help but be caught up in his plight . xxmaj the hand held camera and jumpy editing style enhances the sense of desperation and time . xxmaj the scenes of urban xxmaj phillipines , particular the markets and the squatter holdings are a vivid cacophony . 
 
  xxmaj co - directors and xxmaj co - writers xxmaj neill xxmaj xxunk xxmaj xxunk and xxmaj ian xxmaj gamazon have done an outstanding job of making the most out of limited means . xxmaj the economy of the film makes it both intimate and discomfiting , as xxmaj adam is an everyman who only wants his family safe and instead is completely at the whim of an omniscient tormentor . 
 
  xxup cavite is an absolute must for anyone who has an interest in film , as storytelling , in it 's structure , and as an art form .,xxbos xxmaj decades ago , a crate filled with weapons grade plutonium crashes on an island and soaks into the ground . xxmaj today , a team of military men are sent to track down a notorious terrorist ( of ambiguous national origin ) and they track him to this polluted island . xxmaj when their raft is destroyed , the team must spend the night on shore , but soon discover that the plutonium has done something awful to the island -- it has called forth hundreds of bloodthirsty velociraptors . 
 
  xxmaj let me start this with a lesson : do n't lend a movie to your friend before you 've seen it , especially if you are supposed to be reviewing it for the internet 's finest horror movie site . xxmaj it took me almost a year to get this film back , and the person who borrowed it still had not watched it ( though we ended up seeing it together ) . xxmaj and a second , more important , lesson : when you do watch this , keep your expectations as low as humanly possible . xxmaj because this film ranks among the worst i 've ever seen . 
 
  xxmaj my acting in 8th grade was more convincing than the seasoned actors who appear in this film ( xxmaj lorenzo xxmaj lamas , xxmaj stephen xxmaj bauer ) . xxmaj line delivery is very fake , and the words themselves are poorly scripted . xxmaj the opening words come from a man checking out his gun 's scope : " xxmaj boom . xxmaj dead bad guys . " xxmaj yes , that 's pure genius at work . xxmaj the only conversation with any depth has two main characters explaining their histories . xxmaj but it , too , seems unnatural and a poor attempt to provide character background and to fill time . xxmaj we did n't need to know anything about their histories , so why bore us with it ? xxmaj and if you think the conversations are bad , you ai n't seen nothing yet . 
 
  xxmaj the lighting is atrocious . i generally do n't notice lighting , but my friend ( a former film school student ) was practically vomiting in rage at the way more often than not shadows fell on the actors ' faces and the light would be in the background , focused on nothing in particular . xxmaj most lighting looks like a spotlight in a dim room , and many of the scenes involve a deep , subterranean cavern -- which you 'd then expect to be poorly lit , but had lights coming from all sorts of random angles . xxmaj do n't ask me why . 
 
  xxmaj the plot was pretty bad . xxmaj some films can take the idea of military men chasing a terrorist and make a convincing film out of it . xxmaj cat and mouse stories are riveting . xxmaj well , not here . xxmaj the terrorist is really not even part of the story , just an excuse to go to the island . xxmaj and the raptors ? xxmaj and the allosaurus ? xxmaj sure , they came from the plutonium that soaked into the ground . xxmaj but if that makes sense to you , please explain it to me because i have no clue how radiation brings dinosaurs back from millions of years of extinction . 
 
  xxmaj by far the worst part of " xxmaj raptor xxmaj island " is the animation of the raptors . xxmaj that 's right -- the selling point of the film is the worst aspect . xxmaj the animation is n't just bad , it 's subpar . i ca n't even express the hilarity of cartoons this cheesy . xxmaj and when they get shot ? xxmaj red splats like one would see in an old video game . xxmaj even the airplane , helicopter and xxmaj navy ship are cartoons ... how hard is it to get a model plane ? xxmaj please do n't see " xxmaj raptor xxmaj island " unless you need a good laugh or want to get sickeningly drunk . xxmaj sure , you probably want to see it before you see " xxmaj raptor xxmaj island 2 " ( which seems to be getting better reviews ) . xxmaj but just avoiding it entirely is your best bet . xxmaj the closest thing i can compare it to is " xxmaj pinata : xxmaj survival xxmaj island " , and unfortunately this film makes the pinata look good by comparison . xxmaj you have been warned .,xxbos xxmaj this is one creepy underrated xxmaj gem with chilling performances and a fantastic xxunk xxmaj all the characters are great , and the story was awesome , plus i thought the ending was really xxunk xxmaj the plot was great , and it never bored me , plus while the child actors were bad , they gave me the xxunk xxmaj this happened to be on the space channel a while ago , so i decided to check it out and tape it , i read some good reviews from fellow horror fans , i must say i agree with them , it 's very creepy , and suspenseful , plus xxmaj strother xxmaj martin , was fantastic in his role , as the xxmaj satan worshiper . xxmaj it has tons of creepy atmosphere , and it keeps you guessing throughout , plus all the characters were very likable , and you really start to root for xxmaj ben and his xxunk xxmaj it has plenty of disturbing moments , and the film really shocked me at times , plus , it 's extremely well made on a low xxunk xxmaj this is one creepy underrated gem , with chilling performances and a fantastic finale ! , i highly recommend this one!. xxmaj the xxmaj direction is very good!. xxmaj bernard mceveety does a very good job here , with great camera work , creating a lot of creepy atmosphere , and keeping the film at a very fast pace!. xxmaj ther is a little bit of blood and gore . xxmaj we get a severed leg , lots of bloody corpses , bloody slit throat , slicing and dicing , decapitation , and an impaling . xxmaj the xxmaj acting is xxunk xxmaj strother xxmaj martin is fantastic here ! as the xxmaj satan worshiper , he is extremely creepy , very convincing , was quite chilling , was extremely intense , seemed to be enjoying himself , and just did a fantastic job xxunk xxmaj charles xxmaj bateman is great as the xxmaj dad , he was very caring , very likable , and gave a good show ! , i liked him lots . xxup l.q. xxmaj jones is awesome as the xxmaj sheriff , he was funny , on top of things , looked very young , had a cool character , and just did an awesome job xxunk xxmaj ahna xxmaj capri is good as the girlfriend and did what she had to do pretty well . xxmaj charles xxmaj robinson overacted to the extreme as the xxmaj priest and did n't convince me one bit ! , and that laugh of his was especially bad . xxmaj geri xxmaj reischl is actually decent as the daughter , she was somewhat likable , and only got on my nerves a couple times , i rather liked her . xxmaj alvy xxmaj moore was goofy , but very likable in his role as xxmaj tobey i dug him!. xxmaj rest of the cast do good . xxmaj overall i highly recommend it!. * * * 1 / 2 out of 5,xxbos xxmaj aside from the gross factual inaccuracies of this movie ( ie king xxmaj richard having a son and that son replacing xxmaj john on the throne ) this movie was a sweet love tale full of adventure . i 'm sure that this movie will appeal to the younger generation of girls and boys .,xxbos xxmaj the special effects again are superb but this is xxmaj finding xxmaj nemo in reverse i.e the parents get taken away and xxmaj pi ( xxmaj nemo ) is left behind , if you have n't seen xxmaj finding xxmaj nemo you will like it . 
 
  xxmaj it becomes very clear at the beginning of the film what the plot is and takes a long time to reach the end ! 
 
  xxmaj there 's is nothing new or nail biting in this one and there is no humour at all , which is very disappointing . 
 
  xxmaj all in all very disappointing as a follow up movie to xxmaj shark xxmaj tale which there really are no similarities with it should have been called xxmaj finding xxmaj nemo 2 xxrep 56 !
y: LMLabelList
,,,,
Path: C:\Users\cross-entropy\.fastai\data\imdb;

Test: None, model=SequentialRNN(
  (0): AWD_LSTM(
    (encoder): Embedding(60000, 400, padding_idx=1)
    (encoder_dp): EmbeddingDropout(
      (emb): Embedding(60000, 400, padding_idx=1)
    )
    (rnns): ModuleList(
      (0): WeightDropout(
        (module): LSTM(400, 1152, batch_first=True)
      )
      (1): WeightDropout(
        (module): LSTM(1152, 1152, batch_first=True)
      )
      (2): WeightDropout(
        (module): LSTM(1152, 400, batch_first=True)
      )
    )
    (input_dp): RNNDropout()
    (hidden_dps): ModuleList(
      (0): RNNDropout()
      (1): RNNDropout()
      (2): RNNDropout()
    )
  )
  (1): LinearDecoder(
    (decoder): Linear(in_features=400, out_features=60000, bias=True)
    (output_dp): RNNDropout()
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99)), loss_func=FlattenedLoss of CrossEntropyLoss(), metrics=[<function accuracy at 0x00000236A8585048>], true_wd=True, bn_wd=True, wd=0.01, train_bn=True, path=WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb'), model_dir='models', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False)], callbacks=[RNNTrainer
learn: ...
alpha: 2.0
beta: 1.0, MixedPrecision
learn: ...
loss_scale: 2097152.0
max_noskip: 1000
dynamic: True
clip: None
flat_master: False
max_scale: 16777216
loss_fp32: True], layer_groups=[Sequential(
  (0): WeightDropout(
    (module): LSTM(400, 1152, batch_first=True)
  )
  (1): RNNDropout()
), Sequential(
  (0): WeightDropout(
    (module): LSTM(1152, 1152, batch_first=True)
  )
  (1): RNNDropout()
), Sequential(
  (0): WeightDropout(
    (module): LSTM(1152, 400, batch_first=True)
  )
  (1): RNNDropout()
), Sequential(
  (0): Embedding(60000, 400, padding_idx=1)
  (1): EmbeddingDropout(
    (emb): Embedding(60000, 400, padding_idx=1)
  )
  (2): LinearDecoder(
    (decoder): Linear(in_features=400, out_features=60000, bias=True)
    (output_dp): RNNDropout()
  )
)], add_time=True, silent=False)

现在我们已经训练好了我们的模型，对于 IMDb 中存在但不在 wikitext-103 中的单词，我们已经学会了不同的表示形式（记住，一开始我们将它们全部初始化为相同的形式）：

enc = learn_lm.model[0].encoder

np.allclose(enc.weight[vocab.stoi["30-something"], :], 
            enc.weight[vocab.stoi["linklater"], :])

False

np.allclose(enc.weight[vocab.stoi["30-something"], :], new_word_vec)

False

使用经过微调的 IMDb 语言模型生成类似电影评论的文本

将这些文本与第 2A 部分中未经训练的 IMDb 模型生成的文本进行比较。它们看起来质量更好吗？
我们微调后的 IMDb 语言模型效果如何？让我们试着看看，当给出可能出现在电影评论中的短语时，它会预测什么。

TEXT = "i liked this movie because"
N_WORDS = 40
N_SENTENCES = 2

print("\n".join(learn_lm.predict(TEXT, N_WORDS, temperature=0.75) for _ in range(N_SENTENCES)))

i liked this movie because it gave a new perspective on the subject . The story of how to kill a dog was actually a good concept that i found to be very funny . The whole movie is shot with a sense
i liked this movie because it was fun to see Fred Macmurray - a good actor - and this movie had a good cast like Gene Hackman , Gene Hackman , Catherine o'hara , William Hurt

TEXT = "This movie was"
N_WORDS = 30
N_SENTENCES = 2

print("\n".join(learn_lm.predict(TEXT, N_WORDS, temperature=0.75) for _ in range(N_SENTENCES)))

This movie was just awful . i mean , they had a lot of money , i mean , they wanted to make money with a $ 35 million budget . And
This movie was very pleasant to watch , but it was just too dull . It took us a few minutes to catch the Mexican guy , but he seemed to

TEXT = "I hated this movie"
N_WORDS = 40
N_SENTENCES = 2

print("\n".join(learn_lm.predict(TEXT, N_WORDS, temperature=0.75) for _ in range(N_SENTENCES)))

I hated this movie , if only it ever made it to DVD . If you are planning to see this : 1 ) i would recommend that you buy this movie instead of the movie . 2 . It is
I hated this movie . It was a little too distant from the book , but the acting was good and the at least some scenes were funny . The story line was pretty good , but it was n't the best

构建 IMDb 情绪分类器\

我们现在将使用迁移学习来创建分类器，同样从 wikitext-103 语言模型的预训练权重开始。我们还需要之前保存的 IMDb 语言模型编码器。

加载并预处理数据，并形成数据集

使用 fastai 灵活的 API，我们现在将创建一种不同类型的数据集对象，该对象适用于分类器而不是语言模型（如我们在 2A 中所做的那样）。这次我们将保留 IMDb 电影评论数据的标签。

为 fastai Text API 中的错误添加 try-except 包装器解决方法

此处，批处理大小从 48 减少到 8，以避免 CUDA 内存不足错误；您的硬件可能能够处理更大的批处理，在这种情况下训练可能会更快。

同样，这需要一点时间。

bs=8

%%time

# throws `BrokenProcessPool' Error sometimes. Keep trying `till it works!
#    the progress bar has to complete three consecutive steps. Why three? 
#    fails nearly 100 times, and doesn't respond to interrupt
count = 0
error = True
while error:
    try: 
        # Preprocessing steps
        data_clas = (TextList.from_folder(path, vocab=data_lm.vocab)
             #grab all the text files in path
             .split_by_folder(valid='test')
             #split by train and valid folder (that only keeps 'train' and 'test' so no need to filter)
             .label_from_folder(classes=['neg', 'pos']))
             #label them all with their folders
             #.databunch(bs=bs, num_workers=1))
        error = False
        print(f'failure count is {count}\n')   
        
    except: # catch *all* exceptions
        # accumulate failure count
        count = count + 1
        print(f'failure count is {count}')

failure count is 5

Wall time: 9min 15s

将预处理后的数据形成数据束

data_clas = data_clas.databunch(bs=bs, num_workers=1)

保存数据集（因为制作花了很长时间）并加载它

data_clas.save('imdb_textlist_class')

data_clas = load_data(path, 'imdb_textlist_class', bs=bs, num_workers=1)

data_clas.show_batch()

text	target
xxbos xxmaj match 1 : xxmaj tag xxmaj team xxmaj table xxmaj match xxmaj bubba xxmaj ray and xxmaj spike xxmaj dudley vs xxmaj eddie xxmaj guerrero and xxmaj chris xxmaj benoit xxmaj bubba xxmaj ray and xxmaj spike xxmaj dudley started things off with a xxmaj tag xxmaj team xxmaj table xxmaj match against xxmaj eddie xxmaj guerrero and xxmaj chris xxmaj benoit . xxmaj according to the rules	pos
xxbos xxmaj some have praised _ xxunk _ as a xxmaj disney adventure for adults . i do n't think so -- at least not for thinking adults . \n \n xxmaj this script suggests a beginning as a live - action movie , that struck someone as the type of crap you can not sell to adults anymore . xxmaj the " crack staff " of many older	neg
xxbos xxmaj this movie was recently released on xxup dvd in the xxup us and i finally got the chance to see this hard - to - find gem . xxmaj it even came with original theatrical previews of other xxmaj italian horror classics like " xxup xxunk " and " xxup beyond xxup the xxup darkness " . xxmaj unfortunately , the previews were the best thing about this	neg
xxbos xxmaj there are many adaptations of xxmaj charlotte xxmaj brontë 's classic novel " xxmaj jane xxmaj eyre " , and taking into consideration the numerous reviews written about them there is also a lively discussion on which of them is the best . xxmaj the short film adaptations all suffer from the fact that it is simply not possible to cram the whole plot of the novel into	pos
xxbos xxmaj title : xxmaj zombie 3 ( 1988 ) \n \n xxmaj directors : xxmaj mostly xxmaj lucio xxmaj fulci , but also xxmaj claudio xxmaj fragasso and xxmaj bruno xxmaj mattei \n \n xxmaj cast : xxmaj xxunk xxunk , xxmaj massimo xxmaj xxunk , xxmaj beatrice xxmaj ring , xxmaj deran xxmaj xxunk \n \n xxmaj review : \n \n xxmaj to review	neg

创建一个模型来对IMDb评论进行分类，并加载我们之前保存的编码器。

冻结除最后一层之外的所有层的权重，并找到一个合适的学习率值。

learn_c = text_classifier_learner(data_clas, AWD_LSTM, drop_mult=0.3).to_fp16()
learn_c.load_encoder('fine_tuned_enc')
learn_c.freeze()

learn_c.lr_find()

LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.

learn_c.recorder.plot()

训练和微调 IMDb 情绪分类器

训练一个周期，保存中间结果

learn_c.fit_one_cycle(1, 2e-2, moms=(0.8,0.7))

learn_c.save('first')

learn_c.load('first');

解冻最后两层并训练一个周期，保存中间结果。

learn_c.freeze_to(-2)
learn_c.fit_one_cycle(1, slice(1e-2/(2.6**4),1e-2), moms=(0.8,0.7))

learn_c.save('2nd')

解冻最后三层，并训练一个周期，并保存中间结果。

此时我们已经超越了 2017 年（迁移学习之前）的最先进水平！

learn_c.freeze_to(-3)
learn_c.fit_one_cycle(1, slice(5e-3/(2.6**4),5e-3), moms=(0.8,0.7))

learn_c.save('3rd')

learn_c.load('3rd')

RNNLearner(data=TextClasDataBunch;

Train: LabelList (25000 items)
x: TextList
xxbos xxmaj story of a man who has unnatural feelings for a pig . xxmaj starts out with a opening scene that is a terrific example of absurd comedy . a formal orchestra audience is turned into an insane , violent mob by the crazy xxunk of it 's singers . xxmaj unfortunately it stays absurd the xxup whole time with no general narrative eventually making it just too off putting . xxmaj even those from the era should be turned off . xxmaj the cryptic dialogue would make xxmaj shakespeare seem easy to a third grader . xxmaj on a technical level it 's better than you might think with some good cinematography by future great xxmaj vilmos xxmaj zsigmond . xxmaj future stars xxmaj sally xxmaj kirkland and xxmaj frederic xxmaj forrest can be seen briefly .,xxbos xxmaj airport ' 77 starts as a brand new luxury 747 plane is loaded up with valuable paintings & such belonging to rich businessman xxmaj philip xxmaj stevens ( xxmaj james xxmaj stewart ) who is flying them & a bunch of xxup vip 's to his estate in preparation of it being opened to the public as a museum , also on board is xxmaj stevens daughter xxmaj julie ( xxmaj kathleen xxmaj quinlan ) & her son . xxmaj the luxury jetliner takes off as planned but mid - air the plane is hi - jacked by the co - pilot xxmaj chambers ( xxmaj robert xxmaj foxworth ) & his two accomplice 's xxmaj banker ( xxmaj monte xxmaj markham ) & xxmaj wilson ( xxmaj michael xxmaj pataki ) who knock the passengers & crew out with sleeping gas , they plan to steal the valuable cargo & land on a disused plane strip on an isolated island but while making his descent xxmaj chambers almost hits an oil rig in the xxmaj ocean & loses control of the plane sending it crashing into the sea where it sinks to the bottom right bang in the middle of the xxmaj bermuda xxmaj triangle . xxmaj with air in short supply , water leaking in & having flown over 200 miles off course the problems mount for the survivor 's as they await help with time fast running out ... 
 
  xxmaj also known under the slightly different tile xxmaj airport 1977 this second sequel to the smash - hit disaster thriller xxmaj airport ( 1970 ) was directed by xxmaj jerry xxmaj jameson & while once again like it 's predecessors i ca n't say xxmaj airport ' 77 is any sort of forgotten classic it is entertaining although not necessarily for the right reasons . xxmaj out of the three xxmaj airport films i have seen so far i actually liked this one the best , just . xxmaj it has my favourite plot of the three with a nice mid - air hi - jacking & then the crashing ( did n't he see the oil rig ? ) & sinking of the 747 ( maybe the makers were trying to cross the original xxmaj airport with another popular disaster flick of the period xxmaj the xxmaj poseidon xxmaj adventure ( 1972 ) ) & submerged is where it stays until the end with a stark dilemma facing those trapped inside , either suffocate when the air runs out or drown as the 747 floods or if any of the doors are opened & it 's a decent idea that could have made for a great little disaster flick but bad unsympathetic character 's , dull dialogue , lethargic set - pieces & a real lack of danger or suspense or tension means this is a missed opportunity . xxmaj while the rather sluggish plot keeps one entertained for 108 odd minutes not that much happens after the plane sinks & there 's not as much urgency as i thought there should have been . xxmaj even when the xxmaj navy become involved things do n't pick up that much with a few shots of huge ships & helicopters flying about but there 's just something lacking here . xxmaj george xxmaj kennedy as the jinxed airline worker xxmaj joe xxmaj patroni is back but only gets a couple of scenes & barely even says anything preferring to just look worried in the background . 
 
  xxmaj the home video & theatrical version of xxmaj airport ' 77 run 108 minutes while the xxup us xxup tv versions add an extra hour of footage including a new opening credits sequence , many more scenes with xxmaj george xxmaj kennedy as xxmaj patroni , flashbacks to flesh out character 's , longer rescue scenes & the discovery or another couple of dead bodies including the navigator . xxmaj while i would like to see this extra footage i am not sure i could sit through a near three hour cut of xxmaj airport ' 77 . xxmaj as expected the film has dated badly with horrible fashions & interior design choices , i will say no more other than the toy plane model effects are n't great either . xxmaj along with the other two xxmaj airport sequels this takes pride of place in the xxmaj razzie xxmaj award 's xxmaj hall of xxmaj shame although i can think of lots of worse films than this so i reckon that 's a little harsh . xxmaj the action scenes are a little dull unfortunately , the pace is slow & not much excitement or tension is generated which is a shame as i reckon this could have been a pretty good film if made properly . 
 
  xxmaj the production values are alright if nothing spectacular . xxmaj the acting is n't great , two time xxmaj oscar winner xxmaj jack xxmaj lemmon has said since it was a mistake to star in this , one time xxmaj oscar winner xxmaj james xxmaj stewart looks old & frail , also one time xxmaj oscar winner xxmaj lee xxmaj grant looks drunk while xxmaj sir xxmaj christopher xxmaj lee is given little to do & there are plenty of other familiar faces to look out for too . 
 
  xxmaj airport ' 77 is the most disaster orientated of the three xxmaj airport films so far & i liked the ideas behind it even if they were a bit silly , the production & bland direction does n't help though & a film about a sunken plane just should n't be this boring or lethargic . xxmaj followed by xxmaj the xxmaj concorde ... xxmaj airport ' 79 ( 1979 ) .,xxbos xxmaj this film lacked something i could n't put my finger on at first : charisma on the part of the leading actress . xxmaj this inevitably translated to lack of chemistry when she shared the screen with her leading man . xxmaj even the romantic scenes came across as being merely the actors at play . xxmaj it could very well have been the director who miscalculated what he needed from the actors . i just do n't know . 
 
  xxmaj but could it have been the screenplay ? xxmaj just exactly who was the chef in love with ? xxmaj he seemed more enamored of his culinary skills and restaurant , and ultimately of himself and his youthful exploits , than of anybody or anything else . xxmaj he never convinced me he was in love with the princess . 
 
  i was disappointed in this movie . xxmaj but , do n't forget it was nominated for an xxmaj oscar , so judge for yourself .,xxbos xxmaj sorry everyone , , , i know this is supposed to be an " art " film , , but wow , they should have handed out guns at the screening so people could blow their brains out and not watch . xxmaj although the scene design and photographic direction was excellent , this story is too painful to watch . xxmaj the absence of a sound track was brutal . xxmaj the l xxrep 4 o xxrep 5 n g shots were too long . xxmaj how long can you watch two people just sitting there and talking ? xxmaj especially when the dialogue is two people complaining . i really had a hard time just getting through this film . xxmaj the performances were excellent , but how much of that dark , sombre , uninspired , stuff can you take ? xxmaj the only thing i liked was xxmaj maureen xxmaj stapleton and her red dress and dancing scene . xxmaj otherwise this was a ripoff of xxmaj bergman . xxmaj and i 'm no fan f his either . i think anyone who says they enjoyed 1 1 / 2 hours of this is , , well , lying .,xxbos xxmaj when i was little my parents took me along to the theater to see xxmaj interiors . xxmaj it was one of many movies i watched with my parents , but this was the only one we walked out of . xxmaj since then i had never seen xxmaj interiors until just recently , and i could have lived out the rest of my life without it . xxmaj what a pretentious , ponderous , and painfully boring piece of 70 's wine and cheese tripe . xxmaj woody xxmaj allen is one of my favorite directors but xxmaj interiors is by far the worst piece of crap of his career . xxmaj in the unmistakable style of xxmaj ingmar xxmaj berman , xxmaj allen gives us a dark , angular , muted , insight in to the lives of a family wrought by the psychological damage caused by divorce , estrangement , career , love , non - love , xxunk , whatever . xxmaj the film , intentionally , has no comic relief , no music , and is drenched in shadowy pathos . xxmaj this film style can be best defined as expressionist in nature , using an improvisational method of dialogue to illicit a " more pronounced depth of meaning and truth " . xxmaj but xxmaj woody xxmaj allen is no xxmaj ingmar xxmaj bergman . xxmaj the film is painfully slow and dull . xxmaj but beyond that , i simply had no connection with or sympathy for any of the characters . xxmaj instead i felt only contempt for this parade of shuffling , whining , nicotine stained , martyrs in a perpetual quest for identity . xxmaj amid a backdrop of cosmopolitan affluence and baked xxmaj brie intelligentsia the story looms like a fart in the room . xxmaj everyone speaks in affected platitudes and elevated language between cigarettes . xxmaj everyone is " lost " and " struggling " , desperate to find direction or understanding or whatever and it just goes on and on to the point where you just want to slap all of them . xxmaj it 's never about resolution , it 's only about interminable introspective babble . xxmaj it is nothing more than a psychological drama taken to an extreme beyond the audience 's ability to connect . xxmaj woody xxmaj allen chose to make characters so immersed in themselves we feel left out . xxmaj and for that reason i found this movie painfully self indulgent and spiritually draining . i see what he was going for but his insistence on promoting his message through xxmaj prozac prose and distorted film techniques jettisons it past the point of relevance . i highly recommend this one if you 're feeling a little too happy and need something to remind you of death . xxmaj otherwise , let 's just pretend this film never happened .
y: CategoryList
neg,neg,neg,neg,neg
Path: C:\Users\cross-entropy\.fastai\data\imdb;

Valid: LabelList (25000 items)
x: TextList
xxbos xxmaj once again xxmaj mr. xxmaj costner has dragged out a movie for far longer than necessary . xxmaj aside from the terrific sea rescue sequences , of which there are very few i just did not care about any of the characters . xxmaj most of us have ghosts in the closet , and xxmaj costner 's character are realized early on , and then forgotten until much later , by which time i did not care . xxmaj the character we should really care about is a very cocky , overconfident xxmaj ashton xxmaj kutcher . xxmaj the problem is he comes off as kid who thinks he 's better than anyone else around him and shows no signs of a cluttered closet . xxmaj his only obstacle appears to be winning over xxmaj costner . xxmaj finally when we are well past the half way point of this stinker , xxmaj costner tells us all about xxmaj kutcher 's ghosts . xxmaj we are told why xxmaj kutcher is driven to be the best with no prior inkling or foreshadowing . xxmaj no magic here , it was all i could do to keep from turning it off an hour in .,xxbos xxmaj this is an example of why the majority of action films are the same . xxmaj generic and boring , there 's really nothing worth watching here . a complete waste of the then barely - tapped talents of xxmaj ice - t and xxmaj ice xxmaj cube , who 've each proven many times over that they are capable of acting , and acting well . xxmaj do n't bother with this one , go see xxmaj new xxmaj jack xxmaj city , xxmaj ricochet or watch xxmaj new xxmaj york xxmaj undercover for xxmaj ice - t , or xxmaj boyz n the xxmaj hood , xxmaj higher xxmaj learning or xxmaj friday for xxmaj ice xxmaj cube and see the real deal . xxmaj ice - t 's horribly cliched dialogue alone makes this film grate at the teeth , and i 'm still wondering what the heck xxmaj bill xxmaj paxton was doing in this film ? xxmaj and why the heck does he always play the exact same character ? xxmaj from xxmaj aliens onward , every film i 've seen with xxmaj bill xxmaj paxton has him playing the exact same irritating character , and at least in xxmaj aliens his character died , which made it somewhat gratifying ... 
 
  xxmaj overall , this is second - rate action trash . xxmaj there are countless better films to see , and if you really want to see this one , watch xxmaj judgement xxmaj night , which is practically a carbon copy but has better acting and a better script . xxmaj the only thing that made this at all worth watching was a decent hand on the camera - the cinematography was almost refreshing , which comes close to making up for the horrible film itself - but not quite . 4 / 10 .,xxbos xxmaj first of all i hate those moronic rappers , who could'nt act if they had a gun pressed against their foreheads . xxmaj all they do is curse and shoot each other and acting like xxunk version of gangsters . 
 
  xxmaj the movie does n't take more than five minutes to explain what is going on before we 're already at the warehouse xxmaj there is not a single sympathetic character in this movie , except for the homeless guy , who is also the only one with half a brain . 
 
  xxmaj bill xxmaj paxton and xxmaj william xxmaj sadler are both hill billies and xxmaj xxunk character is just as much a villain as the gangsters . i did'nt like him right from the start . 
 
  xxmaj the movie is filled with pointless violence and xxmaj walter xxmaj hills specialty : people falling through windows with glass flying everywhere . xxmaj there is pretty much no plot and it is a big problem when you root for no - one . xxmaj everybody dies , except from xxmaj paxton and the homeless guy and everybody get what they deserve . 
 
  xxmaj the only two black people that can act is the homeless guy and the junkie but they 're actors by profession , not annoying ugly brain dead rappers . 
 
  xxmaj stay away from this crap and watch 48 hours 1 and 2 instead . xxmaj at lest they have characters you care about , a sense of humor and nothing but real actors in the cast .,xxbos xxmaj not even the xxmaj beatles could write songs everyone liked , and although xxmaj walter xxmaj hill is no mop - top he 's second to none when it comes to thought provoking action movies . xxmaj the nineties came and social platforms were changing in music and film , the emergence of the xxmaj rapper turned movie star was in full swing , the acting took a back seat to each man 's overpowering regional accent and transparent acting . xxmaj this was one of the many ice - t movies i saw as a kid and loved , only to watch them later and cringe . xxmaj bill xxmaj paxton and xxmaj william xxmaj sadler are firemen with basic lives until a burning building tenant about to go up in flames hands over a map with gold implications . i hand it to xxmaj walter for quickly and neatly setting up the main characters and location . xxmaj but i fault everyone involved for turning out xxmaj lame - o performances . xxmaj ice - t and cube must have been red hot at this time , and while i 've enjoyed both their careers as rappers , in my opinion they fell flat in this movie . xxmaj it 's about ninety minutes of one guy ridiculously turning his back on the other guy to the point you find yourself locked in multiple states of disbelief . xxmaj now this is a movie , its not a documentary so i wo nt waste my time recounting all the stupid plot twists in this movie , but there were many , and they led nowhere . i got the feeling watching this that everyone on set was xxunk of confused and just playing things off the cuff . xxmaj there are two things i still enjoy about it , one involves a scene with a needle and the other is xxmaj sadler 's huge 45 pistol . xxmaj bottom line this movie is like domino 's pizza . xxmaj yeah ill eat it if i 'm hungry and i do n't feel like cooking , xxmaj but i 'm well aware it tastes like crap . 3 stars , meh .,xxbos xxmaj brass pictures ( movies is not a fitting word for them ) really are somewhat brassy . xxmaj their alluring visual qualities are reminiscent of expensive high class xxup tv commercials . xxmaj but unfortunately xxmaj brass pictures are feature films with the pretense of wanting to entertain viewers for over two hours ! xxmaj in this they fail miserably , their undeniable , but rather soft and flabby than steamy , erotic qualities non withstanding . 
 
  xxmaj senso ' 45 is a remake of a film by xxmaj luchino xxmaj visconti with the same title and xxmaj alida xxmaj valli and xxmaj farley xxmaj granger in the lead . xxmaj the original tells a story of senseless love and lust in and around xxmaj venice during the xxmaj italian wars of independence . xxmaj brass moved the action from the 19th into the 20th century , 1945 to be exact , so there are xxmaj mussolini murals , men in black shirts , xxmaj german uniforms or the tattered garb of the partisans . xxmaj but it is just window dressing , the historic context is completely negligible . 
 
  xxmaj anna xxmaj xxunk plays the attractive aristocratic woman who falls for the amoral xxup ss guy who always puts on too much lipstick . xxmaj she is an attractive , versatile , well trained xxmaj italian actress and clearly above the material . xxmaj her wide range of facial expressions ( xxunk boredom , loathing , delight , fear , hate ... and ecstasy ) are the best reason to watch this picture and worth two stars . xxmaj she endures this basically trashy stuff with an astonishing amount of dignity . i wish some really good parts come along for her . xxmaj she really deserves it .
y: CategoryList
neg,neg,neg,neg,neg
Path: C:\Users\cross-entropy\.fastai\data\imdb;

Test: None, model=SequentialRNN(
  (0): MultiBatchEncoder(
    (module): AWD_LSTM(
      (encoder): Embedding(60000, 400, padding_idx=1)
      (encoder_dp): EmbeddingDropout(
        (emb): Embedding(60000, 400, padding_idx=1)
      )
      (rnns): ModuleList(
        (0): WeightDropout(
          (module): LSTM(400, 1152, batch_first=True)
        )
        (1): WeightDropout(
          (module): LSTM(1152, 1152, batch_first=True)
        )
        (2): WeightDropout(
          (module): LSTM(1152, 400, batch_first=True)
        )
      )
      (input_dp): RNNDropout()
      (hidden_dps): ModuleList(
        (0): RNNDropout()
        (1): RNNDropout()
        (2): RNNDropout()
      )
    )
  )
  (1): PoolingLinearClassifier(
    (layers): Sequential(
      (0): BatchNorm1d(1200, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (1): Dropout(p=0.12, inplace=False)
      (2): Linear(in_features=1200, out_features=50, bias=True)
      (3): ReLU(inplace=True)
      (4): BatchNorm1d(50, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): Dropout(p=0.1, inplace=False)
      (6): Linear(in_features=50, out_features=2, bias=True)
    )
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99)), loss_func=FlattenedLoss of CrossEntropyLoss(), metrics=[<function accuracy at 0x000001BC017E0F28>], true_wd=True, bn_wd=True, wd=0.01, train_bn=True, path=WindowsPath('C:/Users/cross-entropy/.fastai/data/imdb'), model_dir='models', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False)], callbacks=[RNNTrainer
learn: ...
alpha: 2.0
beta: 1.0, MixedPrecision
learn: ...
loss_scale: 65536
max_noskip: 1000
dynamic: True
clip: None
flat_master: False
max_scale: 16777216
loss_fp32: True], layer_groups=[Sequential(
  (0): Embedding(60000, 400, padding_idx=1)
  (1): EmbeddingDropout(
    (emb): Embedding(60000, 400, padding_idx=1)
  )
), Sequential(
  (0): WeightDropout(
    (module): LSTM(400, 1152, batch_first=True)
  )
  (1): RNNDropout()
), Sequential(
  (0): WeightDropout(
    (module): LSTM(1152, 1152, batch_first=True)
  )
  (1): RNNDropout()
), Sequential(
  (0): WeightDropout(
    (module): LSTM(1152, 400, batch_first=True)
  )
  (1): RNNDropout()
), Sequential(
  (0): PoolingLinearClassifier(
    (layers): Sequential(
      (0): BatchNorm1d(1200, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (1): Dropout(p=0.12, inplace=False)
      (2): Linear(in_features=1200, out_features=50, bias=True)
      (3): ReLU(inplace=True)
      (4): BatchNorm1d(50, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): Dropout(p=0.1, inplace=False)
      (6): Linear(in_features=50, out_features=2, bias=True)
    )
  )
)], add_time=True, silent=False)

解冻所有层，训练两个循环，并保存结果。

注意：在此步骤中，我遇到了 CUDA 错误：未指定的启动失败。这是使用 LSTM 时 PyTorch 的一个已知（且未解决）问题。https://github.com/pytorch/pytorch/issues/27837

除了再试一次外别无他法……第二次尝试成功了。

learn_c.unfreeze()
learn_c.fit_one_cycle(2, slice(1e-3/(2.6**4),1e-3), moms=(0.8,0.7))

2017 年该数据集的最佳水平是 94.1%，我们已经打破了它的记录！！！

保存 IMDb 分类器模型

learn_c.save('clas')

让我们看几个例子，只是为了检查分类器是否按我们认为的那样工作。

模型预测的三个输出是标签（pos 或 neg）以及 neg 和 pos 的类概率估计，它们衡量了模型对其预测的信心。正如我们所料，该模型非常确信第一个评论是 pos，并且相当确信第二个评论是 neg。因此它以优异的成绩通过了测试。

learn_c.predict("I really loved that movie, it was awesome!")

C:\Users\cross-entropy\Anaconda3\envs\fastai\lib\site-packages\fastai\torch_core.py:83: UserWarning: Tensor is int32: upgrading to int64; for better performance use int64 input
  warn('Tensor is int32: upgrading to int64; for better performance use int64 input')

(Category pos, tensor(1), tensor([0.0161, 0.9839]))

learn_c.predict("I didn't really love that movie, and I didn't think it was awesome.")

C:\Users\cross-entropy\Anaconda3\envs\fastai\lib\site-packages\fastai\torch_core.py:83: UserWarning: Tensor is int32: upgrading to int64; for better performance use int64 input
  warn('Tensor is int32: upgrading to int64; for better performance use int64 input')

(Category neg, tensor(0), tensor([0.9698, 0.0302]))

现在我们已经建立了模型，下面就是你可以享受乐趣的部分了！！试用一下模型，试试你自己的例子！！