MovieLens 1M之python数据分析练习

数据集来源https://grouplens.org/datasets/movielens/1m/
这里写图片描述


代码区:

import pandas as pd
uname=['user_id','gender','age','occupation','zip']
users=pd.read_table(r'D:\demo1\ml-1m\users.dat',sep='::',header=None,names=uname,engine = 'python')
'''
sep : str, default ‘,’
指定分隔符。如果不指定参数,则会尝试使用逗号分隔。分隔符长于一个字符并且不是‘\s+’,
将使用python的语法分析器。并且忽略数据中的逗号。正则表达式例子:'\r\t'

header : int or list of ints, default ‘infer’指定行数用来作为列名,数据开始行数。

names : array-like, default None
用于结果的列名列表,如果数据文件中没有列标题行,就需要执行header=None。
engine解析器引擎使用。C引擎速度更快,而python引擎目前更加完善。除去警告
'''

rnames=['user_id','movie_id','rating','timestamp']
ratings=pd.read_table(r'D:\demo1\ml-1m\ratings.dat',sep='::',header=None,names=rnames,engine = 'python')
mname=['movie_id','title','genres']
movies=pd.read_table(r'D:\demo1\ml-1m\movies.dat',sep='::',header=None,names=mname,engine = 'python')

data=pd.merge(pd.merge(movies,ratings),users)
print data.loc[0]#ix[0]已经deprecated弃用

结果:
这里写图片描述

这里写图片描述

movie_id                                1
title                    Toy Story (1995)
genres        Animation|Children's|Comedy
user_id                                 1
rating                                  5
timestamp                       978824268
gender                                  F
age                                     1
occupation                             10
zip                                 48067

'''
#枢轴表pandas.pivot_table(data, values=None, 
index=None, columns=None, aggfunc='mean', fill_value=None, margins=False, dropna=True, margins_name='All')
'''
mean_ratings=data.pivot_table('rating',index='title',columns='gender',aggfunc='mean')
print mean_ratings[:5]

result:

这里写图片描述

gender                                F         M
title                                            
$1,000,000 Duck (1971)         3.375000  2.761905
'Night Mother (1986)           3.388889  3.352941
'Til There Was You (1997)      2.675676  2.733333
'burbs, The (1989)             2.793478  2.962085
...And Justice for All (1979)  3.828571  3.689024

#过滤数据不足200条的电影
ratings_groupby_title=data.groupby('title').size()
print ratings_groupby_title[:5]

reslut:

title
$1,000,000 Duck (1971)            37
'Night Mother (1986)              70
'Til There Was You (1997)         52
'burbs, The (1989)               303
...And Justice for All (1979)    199
dtype: int64

这里写图片描述


active_titles=data.groupby('title').size().index[data.groupby('title').size()>=200]
print active_titles

result:

Index([u''burbs, The (1989)', u'10 Things I Hate About You (1999)',
       u'101 Dalmatians (1961)', u'101 Dalmatians (1996)',
       u'12 Angry Men (1957)', u'13th Warrior, The (1999)',
       u'2 Days in the Valley (1996)', u'20,000 Leagues Under the Sea (1954)',
       u'2001: A Space Odyssey (1968)', u'2010 (1984)',
       ...
       u'Year of Living Dangerously (1982)', u'Yellow Submarine (1968)',
       u'Yojimbo (1961)', u'You've Got Mail (1998)',
       u'Young Frankenstein (1974)', u'Young Guns (1988)',
       u'Young Guns II (1990)', u'Young Sherlock Holmes (1985)',
       u'Zero Effect (1998)', u'eXistenZ (1999)'],
      dtype='object', name=u'title', length=1426)

mean_ratings=mean_ratings.loc[active_titles]
#对F列进行降序
top_female_rating=mean_ratings.sort_values(by='F',ascending='False')
print top_female_rating[:10]

result:

gender                                                     F         M
title                                                                 
Battlefield Earth (2000)                            1.574468  1.616949
Barb Wire (1996)                                    1.585366  2.100386
Showgirls (1995)                                    1.709091  2.166667
Jaws 3-D (1983)                                     1.863636  1.851064
Rocky V (1990)                                      1.878788  2.132780
Speed 2: Cruise Control (1997)                      1.906667  1.863014
Avengers, The (1998)                                1.915254  2.017467
Anaconda (1997)                                     2.000000  2.248447
Nightmare on Elm Street 5: The Dream Child, A (...  2.052632  1.981481
Howard the Duck (1986)                              2.074627  2.103542

计算评分分歧

mean_ratings['diff']=mean_ratings['M']-mean_ratings['F']
sorted_by_diff=mean_ratings.sort_values(by='diff')
print sorted_by_diff[:5]

result:

gender                                                     F         M  
title                                                                    
Dirty Dancing (1987)                                3.790378  2.959596   
To Wong Foo, Thanks for Everything! Julie Newma...  3.486842  2.795276   
Jumpin' Jack Flash (1986)                           3.254717  2.578358   
Grease (1978)                                       3.975265  3.367041   
Relic, The (1997)                                   3.309524  2.723077   

gender                                                  diff  
title                                                         
Dirty Dancing (1987)                               -0.830782  
To Wong Foo, Thanks for Everything! Julie Newma... -0.691567  
Jumpin' Jack Flash (1986)                          -0.676359  
Grease (1978)                                      -0.608224  
Relic, The (1997)                                  -0.586447  

记一个笔记:脚本实现txt替换

#把文件内容替换  
#把file3.txt 的 :: 替换为 ,,并保存到file4.txt  
import re  

fp3=open("file3.txt","r")  
fp4=open("file4.txt","w")  

for s in fp3.readlines():#先读出来     
    fp4.write(s.replace("::",",")) #替换 并写入  

fp3.close()  
fp4.close()
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值