吴恩达机器学习课后习题ex8-2 推荐系统（python实现）

在本部分练习中，您将实现协作筛选搜索算法，并将其应用于电影分级数据集。2此数据集由1到5的分级组成。这个数据集有nu=943个用户，nm=1682个电影。

数据集

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from scipy.io import loadmat
movies_mat = loadmat('data/ex8_movies.mat')
#934个用户，1682部电影 存储评级Y（i，j）（从1到5）
Y=movies_mat['Y'] 
#二值指示矩阵，其中，如果用户j对电影i进行了评级，则R（i，j）=1，否则R（i，j）=0。
#预测用户尚未评分的电影的收视率，即R（i，j）=0的条目。
R=movies_mat['R'] 
Y.shape,R.shape

(1682, 943),(1682, 943)

params_data =loadmat('data/ex8_movieParams.mat')
X=params_data['X']
Theta=params_data['Theta']
X.shape,Theta.shape

(1682, 10),(943,10)

协同过滤算法

计算代价函数和梯度计算，类似ex4神经网络计算

def serialize(x,theta):
  return np.r_[x.flatten(),theta.flatten()] #变成一维

def deserialize(seq,nm,nu,nf):
  x=seq[:nm*nf].reshape(nm,nf)
  theta=seq[nm*nf:].reshape(nu,nf)
  return x,theta

计算代价函数

def cost(seq,R,Y,n_features):
  n_movie,n_user=Y.shape
  x,theta=deserialize(seq,n_movie,n_user,n_features)
  inner=(x@theta.T-Y)*R
  J=np.sum(inner**2)/2
  return J

计算梯度

def gradient(seq,R,Y,n_features):
  n_movie,n_user=Y.shape
  x,theta=deserialize(seq,n_movie,n_user,n_features)
  inner=(x@theta.T-Y)*R #1682,943
  grad_x=inner@theta #1682,10
  grad_theta=inner.T@x # 943,10
  return serialize(grad_x,grad_theta)

计算包含正则项的代价函数和梯度

def reg_cost(seq,R,Y,n_features,l=1):
  n_movie,n_user=Y.shape
  x,theta=deserialize(seq,n_movie,n_user,n_features)
  inner=(x@theta.T-Y)*R
  J=np.sum(inner**2)/2
  reg1=(l/2)*(np.sum(theta**2))
  reg2=(l/2)*(np.sum(x**2))
  return J+reg1+reg2
  
def reg_gradient(seq,R,Y,n_features,l=1):
  n_movie,n_user=Y.shape
  x,theta=deserialize(seq,n_movie,n_user,n_features)
  inner=(x@theta.T-Y)*R #1682,943
  grad_x=inner@theta+l*x #1682,10
  grad_theta=inner.T@x+l*theta # 943,10
  return serialize(grad_x,grad_theta)

为自己推荐电影

f=open('data/movie_ids.txt',encoding='gbk')
movie_idx=[]
for line in f:
#把每行的每个字符一个个分开，变成一个list
  token=line.strip().split(' ') 
  movie_idx.append(' '.join(token[1:]))
movid_idx=np.array(movid_idx)

通过我们自己对电影的评分,来为自己推荐电影

ratings = np.zeros((1682, 1))

ratings[0] = 4
ratings[6] = 3
ratings[11] = 5
ratings[53] = 4
ratings[63] = 5
ratings[65] = 3
ratings[68] = 5
ratings[97] = 2
ratings[182] = 4
ratings[225] = 5
ratings[354] = 5

把我们的评分添加到矩阵Y和R中

Y=np.append(Y,ratings,axis=1)  #(1682,944)
R=np.append(R,ratings!=0,axis=1)

对Y进行均值归一化，如果我们要用新训练出的算法来预测评分，则需要将平均值重新加回去，预测
$$\Theta \ast x+\mu$$

Y_temp=Y.sum(axis=1)/R.sum(axis=1).reshape(1682,1)
Y_norm=(Y-Y_temp)*R

随机初始化X，theta

x=np.random.random(size=(1682,10))
theta=np.random.random(size=(943,10))
l=10
n_features=10

使用minimize函数训练

from scipy.optimize import minimize
res=minimize(fun=reg_cost, x0=seq, args=(Y_norm,R,n_features,l), method='TNC', jac=reg_gradient)

X_trained, theta_trained = deserialize(res.x,1682,944,10)
X_trained, theta_trained

使用训练出的数据给出推荐电影

predicition= X_trained @ theta_trained.T
my_pre=predicition[:,-1]+Y_temp.flatten()
##argsort()函数是将x中的元素从小到大排列，提取其对应的index(索引) [::-1] 从后向前取出元素（倒序）
idx=np.argsort(my_pre)[::-1]
idx

array([ 287, 10, 1499, …, 1353, 1407, 1347])
给出的数组是电影评分从高到低的索引

for i in range(10):
  print('Predicting rating %0.1f for movie %s.' %(my_pre[idx[i]],movie_idx[idx[i]]))

Top recommendations for you:
Predicting rating 5.2 for movie Scream (1996).
Predicting rating 5.1 for movie Seven (Se7en) (1995).
Predicting rating 5.0 for movie Santa with Muscles (1996).
Predicting rating 5.0 for movie Prefontaine (1997).
Predicting rating 5.0 for movie They Made Me a Criminal (1939).
Predicting rating 5.0 for movie Entertaining Angels: The Dorothy Day Story (1996).
Predicting rating 5.0 for movie Saint of Fort Washington, The (1993).
Predicting rating 5.0 for movie Marlene Dietrich: Shadow and Light (1996).
Predicting rating 5.0 for movie Aiqing wansui (1994).
Predicting rating 5.0 for movie Great Day in Harlem, A (1994).