Movie recommendation based on collaborative filtering method

Movie recommendation based on collaborative filtering method
Zhen Zhu
• PROPOSAL
Online movie have become increasingly popular in recent ten years, and tremendous volumes movie resources have been upload to internet every day. Also, the recommender system has become a vital part of Internet companies [1]. So, it is quite interesting and important to build an intelligent recommender system to identify the effective of what are user’s real interests. So, we choose the recommendation prediction task on MovieLens datasets for project in this course.
• RELATED WORK
Such recommendation problems are usually solved by the collaborative filtering (CF) [3] approach, which relies only on information about the behavior of users in the past. There are two primary methods in CF: the neighborhood approach [4] and latent factor modeling [5]. Both of these methods have been proven to be successful for the recommendation problem. A key step in CF is to combine these models. Methods such as regression, gradient boosting decision trees, and neural networks have been used to ensemble CF models.
• OUTLINE OF APPROACH
The framework of our approach shown in figure1
 
Fig.1 the framework of the approach
1. We use Movielens datasets for our project and transform the data form for data model.
2. Use several different algorithms to calculate the similarity in users or items.
3 For user-based method, we need to do neighborhood calculation.
4 Use user-based CF and item-based CF to recommendation.
5 Design a evaluator to compare the results of different approaches.
• SIMILARITY CALCULATION
Euclidean similarity
In this way, we calculate similarity between two points by Euclidean distance. The Euclidean distance between point p and q is the length of the line segment connecting them ( ). In Cartesian coordinates, if p = (p1, p2,..., pn) and q = (q1, q2,..., qn) are two points in Euclidean n-space, then the distance (d) from p to q, or from q to p is given by the formula as following:
 
The position of a point in a Euclidean n-space is a Euclidean vector. So, p and q are Euclidean vectors, starting from the origin of the space, and their tips indicate two points. The Euclidean norm, or Euclidean length, or magnitude of a vector measures the length of the vector:
 
Log-likelihood similarity
The natural logarithm of the likelihood function, called the log-likelihood, is more convenient to work for similarity calculation. Because the logarithm is a monotonically increasing function, the logarithm of a function achieves its maximum value at the same points as the likelihood itself, and hence the log-likelihood can be used in place of the likelihood in maximum likelihood estimation and related techniques. Finding the maximum of a function often involves taking the derivative of a function and solving for the parameter being maximized, and this is often easier when the function being maximized is a log-likelihood rather than the original likelihood function.
The distribution has two parameters α and β. The likelihood function is
 .
Finding the maximum likelihood estimate of β for a single observed value x looks rather daunting. Its logarithm is much simpler to work with:
 
Gamma Function:  
Pearson Correlation Similarity
Pearson's correlation coefficient is the covariance of the two variables divided by the product of their standard deviations. Pearson's correlation coefficient when applied to a population is commonly represented by the Greek letter ρ and may be referred to as the population correlation coefficient or the population Pearson correlation coefficient. The formula as following:
 
  is the covariance,  is the standard deviation of  
The formula for ρ can be expressed in terms of mean and expectation. Since
 
  and   are defined as above;   is the mean of ;   is the expectation.
Then the formula for ρ can also be written as
 
Cosine similarity
Cosine similarity is a measure of similarity between two vectors of an inner product space that measures the cosine of the angle between them. The cosine of 0° is 1, and it is less than 1 for any other angle. It is thus a judgment of orientation and not magnitude: two vectors with the same orientation have a cosine similarity of 1, two vectors at 90° have a similarity of 0, and two vectors diametrically opposed have a similarity of -1, independent of their magnitude. Cosine similarity is particularly used in positive space, where the outcome is neatly bounded in [0, 1].
The cosine of two vectors can be derived by using the Euclidean dot product formula:
 
Given two vectors of attributes, A and B, the cosine similarity, cos (θ), is represented using a dot product and magnitude as following:
 
The resulting similarity ranges from −1 meaning exactly opposite, to 1 meaning exactly the same, with 0 indicating de-correlation, and in-between values indicating intermediate similarity or dissimilarity.
For distance calculation, most common way is by following:
  or   
• NEIGHBORHOOD CALCULATION
Neighborhood calculation only use for collaborative filtering based on user. Use this calculation to sort top N similar user for final recommendation.
Fixed-size neighborhoods (K-neighborhoods)
K-neighborhoods means the recommendations are derived from a neighborhood of the K most similar users. We need to define a suitable K. If K were smaller. The recommendation would base on fewer similar users, and it would exclude some less-similar users from consideration. If K were larger. The recommendation would base on more users, and it might be add some less-similar users. Only choose suitable K can improve the affection of recommendation.
 
Fig.2 An illustration of defining a neighborhood of most similar users by picking a fixed number of closest neighbors. Distance illustrates similarity: farther means less similar.
Threshold-based neighborhood
What if we don't want to build a neighborhood of the n most similar users, but rather try to pick the “pretty similar” users and ignore everyone else? We could pick a similarity threshold and take any users that are at least that similar. The threshold should be between -1 and 1, since all similarity metrics return similarity values in this range. In the moment, we may use the similarity metric above. 
 
Fig.3 An illustration of defining a neighborhood of most-similar users with a similarity threshold
• COLLABORATIVE FILTERING
Collaborative filtering (CF) approaches assume that those who agreed in the past tend to agree again in the future. For example, a collaborative filtering or recommendation system for movie tastes could make predictions about which movie a user should like by given a partial list of that user's tastes (likes or dislikes). CF methods have two important steps, firstly, CF collects taste information from many users. In the second step, using information gleaned from many users to predict users’ interest and recommend Item to user. Researchers have devised a number of collaborative filtering algorithms which mainly can be divided into two main categories, User-based and Item-based algorithms.
User-based CF
User-based CF is also called nearest-neighbor based Collaborative Filtering, it utilize the entire user-item data to generate a prediction. These systems use statistical techniques to find users’ nearest-neighbors, who have the similar preference. Once the nearest-neighborhood of users are found, these systems use algorithms to combine the preferences of neighbors to produce a prediction or top-N recommendation for the target user. The techniques are popular and widely used in practice.
User-based CF algorithms have been popular and successful in several years, but the widespread use has revealed some potential challenges. In practice, users only have purchased few percent of all the items, maybe 1% of 2 million items, so that recommender system based on nearest neighbor algorithms may be unable to make any item recommendations for a particular user, and the accuracy of recommendations may be poor. Nearest neighbor algorithms require computation that grows with the number of users. With millions of users, a typical web-based recommender system running existing algorithms will suffer serious scalability problems. 
Item-based CF
Unlike the User-based CF algorithm discussed above, the item-based approach focus on items which the target user had rated. Then, calculate the similarity to the target item and then selects k most similar items. According to the weight on those item. We find the most similar items to target user which have not been used.
One critical step in the item-based collaborative filtering algorithm is to compute the similarity between items. The basic idea in similarity computation between two items i and j is to build co-occurrence matrix. Then, using the Euclidean dot product to calculate with target user’s rating. After that we can find the most similar items which has the highest score.
• EXPERIMENT
Data sets
This datasets were collected by the GroupLens Research Project at the University of Minnesota. The datasets are very suit for do recommendation research. This data set consists of: 100,000 ratings (1-5) from 943 users on 1682 movies and each user has rated at least 20 movies with attributes (age, gender, occupation).The data was collected through the MovieLens web site (movielens.umn.edu) during the seven-month period from September 19th, 1997 through April 22nd, 1998. This data has been cleaned up – users who had less than 20 ratings or did not have complete demographic information were removed from this data set.
Evaluation
We use precision value and recall value to measure the result.
 
 
C is the number of the item target user like and identified by system.
N is the total number user like in test data.
T is the Identified number by system.
Results
We use both User-based CF and Item-based CF to experiments. Set Number of neighborhood is 2. The value of neighborhood threshold is 0.5 and the recommender number is 5. The training data account for 70% and the testing datasets account for 30%. The result as Table1 and Table2.
TABLE I. EVALUATION ON USER-BASED CF
Similarity algorithm User-based CF
Rating type Neighborhood calculation Precision Recall Difference
Euclidean similarity Score(1-10) K-neighborhoods 0.15406236275801483 0.14404609475032024 0.07389905172235824
Log-likelihood similarity Score(1-10) K-neighborhoods 0.11314553990610342 0.13617157490396936 0.8677294206556467
Cosine similarity Score(1-10) K-neighborhoods 0.15008714596949832 0.12059325650874954 1.1111111111111112
Pearson Correlation Similarity Score (1-10) K-neighborhoods 0.12151979565772673 0.10734101579171995 0.5506329113924048
Euclidean similarity Boolean(0,1) K-neighborhoods 0.08128205128205122 0.1065941101152369 2.8610988410290643
Log-likelihood similarity Boolean(0,1) K-neighborhoods 0.11395646606914228 0.13107127614169872 2.324895084732162
Cosine similarity Boolean(0,1) K-neighborhoods 0.09692307692307693 0.11662398634229627 2.6268349661343318
Pearson Correlation Similarity Boolean(0,1) K-neighborhoods 0.0725752508361203 0.07543747332479718 2.616772097320647
Euclidean similarity Score(1-10) Threshold-based neighborhood 0.03057017543859648 0.04244558258642761 0.5040645832107125
Log-likelihood similarity Score(1-10) Threshold-based neighborhood 0.0035851472471190786 0.0036491677336747777 0.8119956200953123
Cosine similarity Score (1-10) Threshold-based neighborhood 0.008903225806451634 0.008578745198463503 0.8116693771906173
Pearson Correlation Similarity Score (1-10) Threshold-based neighborhood 0.04111349036402571 0.03252240717029451 0.6220533633729025
Euclidean similarity Boolean(0,1) Threshold-based neighborhood 0.20906735751295338 0.26429790866410546 6.877570770773614
Log-likelihood similarity Boolean(0,1) Threshold-based neighborhood 0.10653008962868132 0.1333546734955189 82.5830467048691
Cosine similarity Boolean(0,1) Threshold-based neighborhood 0.224358974358974 0.2836961160904817 85.7349085590648
Pearson Correlation Similarity Boolean(0,1) Threshold-based neighborhood 0.17935943060498222 0.1814127187366625 5.8100268562854955
TABLE II. EVALUATION ON ITEM-BASED CF
Similarity algorithm Item-based CF
Rating type Precision Recall Difference
Euclidean similarity Score(1-10) 0.0017925736235595393 0.0017925736235595393 0.7945934427298251
Log-likelihood similarity Score(1-10) 0.0 0.0 0.8183123680491949
Cosine similarity Score(1-10) 0.0 0.0 0.8303853550496187
Pearson Correlation Similarity Score (1-10) 0.00870678617157492 0.009240290226205712 0.6706314833178705
Euclidean similarity Boolean(0,1) 0.0025608194622279107 0.0025608194622279107 47.13259472322781
Log-likelihood similarity Boolean(0,1) 0.12522407170294492 0.1432565087494666 91.85028229060038
Cosine similarity Boolean(0,1) 0.07221510883482725 0.07407170294494238 106.26153096575278
Pearson Correlation Similarity Boolean(0,1) 0.0015364916773367482 0.0015364916773367482 0.7373257024463155
Slope one
Score(1-10) 0.0015364916773367482 0.0015364916773367482 0.7373257024463155
Unexpectedly, the experiments results of User-based are mostly better than those of Item-based. Only prediction on rating on Boolean data are better than rating on score. Use threshold to choose neighborhoods are better than use fixed size. The approach of cosine to calculate the similarity is better than others. Value of P reach to about 22.4% and value of R reach to 28.4%.


• CONCLUSIONS
We find Similarity algorithm and CF approaches are sensitive to the datasets. Item-based method do better in many application scenarios. However, in this datasets are lower than User-based method. So, we may use different methods in different datasets. If it possible, we may use several approaches for testing and make the decision based on experimental results. 
REFERENCES
[1]M. Hao, Z. Dengyong, L. Chao, L. M. R., and K. Irwin. Recommender systems with social regularization. In Proceedings of the fourth ACM International Conference on Web Search and Data Mining, WSDM '11, pages 287{296, New York, NY, USA, 2011. ACM.
[2]Yanzhi Niu, Yi Wang, Gordon Sun, Aden Yue, Brian Dalessandro, Claudia Perlich, Ben Hammer, 2012. The Tencent Dataset and KDD-Cup'12. KDD-Cup Workshop 
[3]Y. Niu, Y. Wang, G. Sun, A. Yue, B. Dalessandro, C. Perlich, and B. Hamner. The Tencent dataset and kdd-cup'12. In KDD-Cup Workshop, 2012., 2012.
[4]Y. Koren. 2009. The bellkor solution to the netflix grand prize. Tech. report 
[5]M. Piotte and M. Chabbert. 2009. The Pragmatic Theory solution to the Netflix Grand prize. Tech. report 
[6] R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. In Advances in Neural Information Processing Systems 20 (NIPS'07), pages 1257-1264, 2008
[7] Y. Koren, R. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems. IEEE Computer, 42(8):30-37, 2009.