text processing

Import libraries

from nltk.corpus import stopwords
from textblob import TextBlob
from textblob import Word

Lower casing and removing punctuations

df[‘Text’] = df[‘Text’].apply(lambda x: " “.join(x.lower() for
x in x.split()))
df[‘Text’] = df[‘Text’].str.replace(’[^\w\s]’,”)
df.Text.head(5)

Removal of stop words

stop = stopwords.words(‘english’)
df[‘Text’] = df[‘Text’].apply(lambda x: " ".join(x for x in
x.split() if x not in stop))

Spelling correction

df[‘Text’] = df[‘Text’].apply(lambda x: str(TextBlob(x).
correct()))

Lemmatization

df[‘Text’] = df[‘Text’].apply(lambda x: " ".join([Word(word).
lemmatize() for word in x.split()]))