用户评论情感分析 nlp_nlp达到95精度的电影评论情感分析

用户评论情感分析 nlp

自然语言处理 (Natural Language Processing)

A few days ago, I published an article that uses this same machine learning module to perform a sentiment analysis on a dataset of tweets reaching 96% accuracy. It is now time to increase complexity and approach more complicated problems. One perfect dataset for this experiment is the movie review dataset that you can download on Kaggle (see the link above).

几天前，我发表了一篇文章，该文章使用相同的机器学习模块对达到96％准确性的推文数据集执行情感分析 。 现在该增加复杂性并解决更复杂的问题了。 该实验的一个理想数据集是电影评论数据集，您可以在Kaggle上下载它(请参见上面的链接)。

机器学习与深度学习 (Machine Learning vs. Deep Learning)

Why am I not using deep learning for these tasks? If I had to use Tensorflow, I would use an Embedding neural network. Unfortunately, this dataset only contains 2000 reviews. Compared with the standard movie reviews in Keras, which contains 50,000 reviews, there might not be enough data for the neural net to perform at its top. Deep Learning only outperforms machine learning when there is a sufficient volume of data.

为什么我不将深度学习用于这些任务？ 如果必须使用Tensorflow，则可以使用嵌入神经网络。 不幸的是，该数据集仅包含2000条评论。 与Keras中包含50,000条评论的标准电影评论相比，可能没有足够的数据供神经网络在其顶部执行。 深度学习仅在有足够数据量时才胜过机器学习。

nltk模块 (nltk module)

I will be using a machine learning library specialized for NLP, called nltk. I prefer using scikit-learn for creating machine learning models, but it is a library specialized for tabular data, rather than natural language processing.

我将使用专门用于NLP的机器学习库，称为nltk。 我更喜欢使用scikit-learn创建机器学习模型，但是它是专门用于表格数据而不是自然语言处理的库。

脚步 (Steps)

In this article, I will follow the following steps. Compared with the Twitter Sentiment analysis in the previous article, the preprocessing of data will be much more troubling.

在本文中，我将遵循以下步骤。 与上一篇文章中的Twitter Sentiment分析相比，数据的预处理将更加麻烦。

1. Importing Modules
   导入模块
2. Looking at the data
   看数据
3. Creating Features and Labels (encoding)
   创建特征和标签(编码)
4. Creating train and test (splitting)
   创建训练和测试(拆分)
5. Using the model: Naive Bayes Classifier
   使用模型：朴素贝叶斯分类器
6. Performance Evaluation
   绩效评估

I will be using a particular kind of encoding: instead of converting words to numbers,