我是一个新的python,正在处理一个文本分类问题。我用不同的在线资源开发了一个代码。但是这个代码并没有做词性标注。有人能帮我找出我的代码中我真正出错的那一行吗。我在代码中做词性标记,但结果中没有显示。我也试过用nltk做词性标注,但这对我也不起作用。如有任何帮助,我们将不胜感激。谢谢。在# Add the Data using pandas
Corpus = pd.read_csv(r"U:\FAHAD UL HASSAN\Python Code\projectdatacor.csv",encoding='latin-1')
# Data Pre-processing - This will help in getting better results through the classification algorithms
# Remove blank rows if any.
Corpus['description'].dropna(inplace=True)
# Change all the text to lower case. This is required as python interprets 'design' and 'DESIGN' differently
Corpus['description'] = [entry.lower() for entry in Corpus['description']]
# Punctuation Removal
Corpus['description'] = Corpus.description.str.replace('[^\w\s]', '')
# Tokenization : In this each entry in the corpus will be broken into set of words
Corpus['description']= [word_tokenize(entry) for entry in Corpus['description']]
# Remove Stop words, Non-Numeric and perfom Word Stemming/Lemmenting.
# WordNetLemmatizer requires Pos tags to understand if the word is noun or verb or adjective etc. By default it is set to Noun
STOPWORDS = set(stopwords.words('english'))
tag_map = defaultdict(lambda : wn.NOUN)
tag_map['J'] = wn.ADJ
tag_map['V'] = wn.VERB
tag_map['R'] = wn.ADV
for index,entry in enumerate(Corpus['description']):
# Declaring Empty List to store the words that follow the rules for this step
Final_words = []
# Initializing WordNetLemmatizer()
word_Lemmatized = WordNetLemmatizer()
# pos_tag function below will provide the 'tag' i.e if the word is Noun(N) or Verb(V) or something else.
for word, tag in pos_tag(entry):
# Below condition is to check for Stop words and consider only alphabets
if word not in STOPWORDS and word.isalpha():
word_Final = word_Lemmatized.lemmatize(word,tag_map[tag[0]])
Final_words.append(word_Final)
# The final processed set of words for each iteration will be stored in 'description_final'
Corpus.loc[index,'description_final'] = str(Final_words)
print(Corpus['description_final'].head())
这些就是我得到的结果。这段代码做了很多事情,比如标记化,删除了stopwords,但是它在我的结果中显示了pos标记。在
^{pr2}$