2.7数据-paddlepaddle数据集sentiment

由NLTK提供的movie_reviews数据集
可能会报错：

Resource movie_reviews not found.
  Please use the NLTK Downloader to obtain the resource:
  >>> import nltk
  >>> nltk.download('movie_reviews')
  For more information see: https://www.nltk.org/data.html
  Attempted to load corpora/movie_reviews

可能需要：pip install nltk

import nltk
nltk.download('movie_reviews')

---

paddle.dataset.sentiment:https://www.paddlepaddle.org.cn/documentation/docs/zh/api_cn/data/dataset_cn/sentiment_cn.html

---

import paddle
import paddle.fluid as fluid
import paddle.dataset.sentiment as sentiment

# sentiment.get_word_dict() python3 还会报错，是代码内部错误,cmp在python3不支持，只在python2支持：
'''
TypeError: 'cmp' is an invalid keyword argument for sort()
---> 71     words_sort_list.sort(cmp=lambda a, b: b[1] - a[1])
'''

'''
需要找到上面引用的sentiment代码目录，如anaconda3/envs/paddle15_py37/lib/python3.7/site-packages/paddle/dataset/sentiment.py，
头部引入：from functools import cmp_to_key
在71行改为：words_sort_list.sort(key=cmp_to_key(lambda a, b: b[1] - a[1]))
'''
# 按照样本中出现的单词的频率对单词进行排序
word_dict=sentiment.get_word_dict()

train=sentiment.train() # <generator object reader_creator at 0x7fdb35cd1650>
a_sample=next(train)
len(a_sample),len(a_sample[0]),a_sample[1]
# 2, 304, 0

# transform idx to a document str
' '.join([word_dict[i][0] for i in a_sample[0]])
'''
'the happy bastard \' s quick movie review damn that y2k bug . it \' s got a head start in this movie starring jamie lee curtis and another baldwin brother ( william this time ) in a story regarding a crew of a tugboat that comes across a deserted russian tech ship that has a strangeness to it when they kick the power back on . little do they know the power within . . . going for the gore and bringing on a few action sequences here and there , virus still feels very empty , like a movie going for all flash and no substance . we don \' t know why the crew was really out in the middle of nowhere , we don \' t know the origin of what took over the ship ( just that a big pink flashy thing hit the mir ) , and , of course , we don \' t know why donald sutherland is stumbling around drunkenly throughout . here , it \' s just " hey , let \' s chase these people around with some robots " . the acting is below average , even from the likes of curtis . you \' re more likely to get a kick out of her work in halloween h20 . sutherland is wasted and baldwin , well , he \' s acting like a baldwin , of course . the real star here are stan winston \' s robot design , some schnazzy cgi , and the occasional good gore shot , like picking into someone \' s brain . so , if robots and body parts really turn you on , here \' s your movie . otherwise , it \' s pretty much a sunken ship of a movie .'
'''