电影延展性预测

 首先我们先来观察一下这组数据，这组数据总共只有三列，看起来比较简单，但实际上仔细观察你会发现非常多需要操作的地方。

我们所需要的模块如下所示：

import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
import seaborn  as sns
import random
from plotly import tools
import plotly.express as px
from plotly.offline import init_notebook_mode, iplot, plot
import plotly.graph_objs as go  
import ast

接着对我们的数据进行查看和校验

 

 

第一步：数据清洗 

1.我们在rating上可以看到每一个数据后面都带有一个分字，而我们实际分析的时候假如数字后带有一个分子的话我们就会比较难以处理

2.我们可以看到第三列star里面有一些我们想要的信息，但是都被一些无关紧要的要素影响我们的分析了可以在后续对他进行处理

首先去掉分子：

df_movie.rating = df_movie.rating.apply(lambda x : float(x.split("分")[0]))

其次的话去掉无关仅要的信息：

#我们通过观察字符串可以发现演员这个字段里面分别有用（“ ”，“，”，“/”）这三个字段相互分割
def split_star(star_str):
    final_star_list =[]
    split_list = [" ",",",r"/"]
    star_str = star_str.replace("主演","")
    for i in split_list:
        if type(star_str) != "list":
            star_list = star_str.split(i)
    if len(star_list) == 1:
        star_list = star_list[0].replace(" ","").split(",")
    for i in star_list:
        i = i.strip()
        final_star_list.append(i)
    return final_star_list
df_movie.stars = df_movie.stars.apply(split_star)

3.我们已经在我们的思维导图中体现了我们的核心思路了，将star分为

(topstar_count)(topstar_movie_count)(strstar_count)

在做这个工作之前我们要先确定什么是star什么是topstar，暂且订评分有超过4分的为topstar吧

先假定两个字典all_star = {}/alltop_star = {}

all_star = {}
all_top_star = {}
i = 0

for item in df_movie.stars:
    rating = df_movie.loc[i,"rating"]
    # print(item)
    for sub_item in item:
        if sub_item in all_star:
            all_star[sub_item] = all_star[sub_item] +1
        else:
            all_star[sub_item] = 1
    
        if rating >=4:
        # print(sub_item)
            if sub_item in all_top_star:
                all_top_star[sub_item] = all_top_star[sub_item] +1
            else:
                all_top_star[sub_item] = 1
    i +=1

len(all_star),len(all_top_star)

# all_star = {}
# all_top_star = {}
# df_top_rating  = df_movie[df_movie.rating >= 4]
# for item in df_movie.stars:
    
#     for star in item :
#         if star in all_star:
#             all_star[star] = all_star.get(star,0) +1 
            
# for item in df_top_rating.stars:
    
#     for star in item:
#         # print(star)
#         all_top_star[star] = all_top_star.get(star,0) +1
        
# len(all_star),len(all_top_star)
# df_top_rating.head(40)

以上两种方法都可以无非就是字典的相加而已。

我们接下来要计算(topstar_count)(topstar_movie_count)(strstar_count)就得先定义一个函数以便后续对整个dataframe进行一个函数的导入

def get_topstar_count(star_list):
    result = 0
    for i in star_list:
        if i in all_top_star:
            result +=1
    return result         #假如star在topstar里则count的数量加1
def get_topstar_movie(star_list):
    result = 0
    for i in star_list:
        if i in all_top_star:
            result = all_top_star[i] +result
    return result  #出演电影数量相加
def str_star_count(star_list):
    result = 0
    for i in star_list :
        if all_star[i] >3:
            result += 1
    return result  #出演过三部电影演员数量相加

df_movie["topstar_count"] = df_movie.stars.apply(get_topstar_count)
df_movie["topstar_movie_count"]= df_movie.stars.apply(get_topstar_movie)
df_movie["strstar_count"] = df_movie.stars.apply(str_star_count)
df_movie["star_count"] = df_movie.stars.apply(len)

4.标题字数的统计

df_movie["title_count"] = df_movie.title.apply(len)

 

最后清理完成之后的表格将会呈下面样子

 第二步：数据分析

#作可视化数据分析
sns.set()
fig = plt.figure(figsize=(16,6))
fig.add_subplot(2,2,1)
sns.scatterplot(data=df_movie,x="topstar_count",y = "rating")
fig.add_subplot(2,2,2)
sns.scatterplot(data=df_movie,x="topstar_movie_count",y ="rating")
fig.add_subplot(2,2,3)
sns.scatterplot(data = df_movie,x ="strstar_count",y = "rating")
fig.add_subplot(2,2,4)
sns.scatterplot(data = df_movie,x ="star_count",y = "rating")

做一个散点图相关性分析以观察，这些变量与因变量的关系

由上面这张图可以很直观地反映出rating和xx_count之间的关系 

第三步：建立模型

from sklearn.model_selection import train_test_split
df_train,df_test = train_test_split(df_movie,test_size = 0.2) #取出0.2做一个测试
from  xgboost import XGBRegressor

xgb = XGBRegressor(n_estimater = 2700,max_denth = 27,random_size = 0)
y1 = df_train.rating
# x1 = df_train.drop(columns=["title","rating","stars","topstar_count","topstar_movie_count","strstar_count"])

x1 =df_train[["star_count","title_count"]]
res1 = xgb.fit(x1,y1)

均方误差实际上来说越小越好

from sklearn.metrics import mean_squared_error

这是我们第一组预测的结果： 

predict1 = res1.predict(df_test[["star_count","title_count"]])
mean_squared_error(predict1,df_test.rating)

这是我们第二组预测的结果：

xgb = XGBRegressor(n_estimater = 2700,max_denth = 27,random_size = 0)
y2 = df_train.rating
x2 = df_train.drop(columns = ["title","rating","stars","title_count"])
res2 =xgb.fit(x2,y2)
predict2 =xgb.predict(df_test.drop(columns = ["title","rating","stars","title_count"]))
mean_squared_error(predict2,df_test.rating)

 

显然第二组结果更加准确，我们在实际实验中应该更加偏重于第二种方法的用法！