PROJECT REPORT
ON
“Movie Recommendation System”
Submitted in Partial Fulfillment of requirements for the Award of
Degree of Bachelor of Engineering in Electronics and
Communication.
Submitted to: Submitted by:
Dr. Ruby Beniwal Ritik Mittal
(17102213)
CERTIFICATE
This is to Certified that this project titled “Movie Recommendation
System “is submitted by “Ritik Mittal(17102213)” in fulfilment for
the reward of degree of B.Tech in Electronic and Communication
of Jaypee Institute of Information Technology, Noida has been
carried out under my supervision.
Dr. Ruby Beniwal,
ECE Department
JIIT, Noida, Sector-62
1
ABSTRACT
A recommendation engine filters the data using different algorithms and recommends the
most relevant items to users. It first captures the past behavior of a customer and based on
that, recommends products which the users might be likely to buy. If a completely new user
visits an e-commerce site, that site will not have any past history of that user. So how does the
site go about recommending products to the user in such a scenario? One possible solution
could be to recommend the bestselling products, i.e. the products which are high in demand.
Another possible solution could be to recommend the products which would bring the
maximum profit to the business. Three main approaches are used for our recommender
systems. One is Demographic filtering i.e. .They offer generalized recommendations to every
user, based on movie popularity and/or genre. The System recommends the same movies to
users with similar demographic features. Since each user is different, this approach is
considered to be too simple. The basic idea behind this system is that movies that are more
popular and critically acclaimed will have a higher probability of being liked by the average
audience. Second is content-based filtering, where we try to profile the users’ interests using
information collected, and recommend items based on that profile. The other is collaborative
filtering, where we try to group similar users together and use information about the group to
make recommendations to the user.
2
ACKNOWLEDGEMENT
It gives me immense pleasure to express my deepest sense of gratitude and sincere thanks
to my respected guide Dr. Ruby Beniwal, for their valuable guidance, encouragement
and help for completing this work. Their useful suggestions for this whole work and co-
operative behavior are sincerely acknowledged.
I also wish to express my indebtedness to my parents as well as my family member whose
blessings and support always helped me to face the challenges ahead.
Ritik Mittal
(17102213)
3
TABLE OF CONTENTS
Introduction and Motivation.....................................................................................................5
Technology Used......................................................................................................................7
Literature Survey.....................................................................................................................9
Approach..................................................................................................................................11
Result.......................................................................................................................................20
Conclusion...............................................................................................................................24
References................................................................................................................................25
4
INTRODUCTION AND MOTIVATION
A recommendation system is a type of information filtering system which attempts to predict
the preferences of a user, and make suggests based on these preferences. There are a wide
variety of applications for recommendation systems. These have become increasingly popular
over the last few years and are now utilized in most online platforms that we use. The content
of such platforms varies from movies, music, books and videos, to friends and stories on social
media platforms, to products on e-commerce websites, to people on professional and dating
websites, to search results returned on Google. Often, these systems are able to collect
information about a users choices, and can use this information to improve their suggestions in
the future. For example, Facebook can monitor your interaction with various stories on your
feed in order to learn what types of stories appeal to you. Sometimes, the recommender systems
can make improvements based on the activities of a large number of people. For example, if
Amazon observes that a large number of customers who buy the latest Apple Macbook also
buy a USB-C-toUSB Adapter, they can recommend the Adapter to a new user who has just
added a Macbook to his cart. Due to the advances in recommender systems, users constantly
expect good recommendations. They have a low threshold for services that are not able to make
appropriate suggestions. If a music streaming app is not able to predict and play music that the
user likes, then the user will simply stop using it. This has led to a high emphasis by tech
companies on improving their recommendation systems. However, the problem is more
complex than it seems. Every user has different preferences and likes. In addition, even the
taste of a single user can vary depending on a large number of factors, such as mood, season,
or type of activity the user is doing. For example, the type of music one would like to hear while
exercising differs greatly from the type of music he’d listen to when cooking dinner. Another
issue that recommendation systems have to solve is the exploration vs exploitation problem.
They must explore new domains to discover more about the user, while still making the most
of what is already known about of the user. Three main approaches are used for our
recommender systems. One is Demographic Filtering i.e
They offer generalized recommendations to every user, based on movie popularity and/or
genre. The System recommends the same movies to users with similar demographic features.
5
Since each user is different , this approach is considered to be too simple. The basic idea behind
this system is that movies that are more popular and critically acclaimed will have a higher
probability of being liked by the average audience. Second is content-based filtering, where we
try to profile the users interests using information collected, and recommend items based on
that profile. The other is collaborative filtering, where we try to group similar users together
and use information about the group to make recommendations to the user.
6
Technology Used
1. Python
Python is an interpreted, high-level, general-purpose programming language. Created by
Guido van Rossum and first released in 1991, Python's design philosophy emphasizes
code readability with its notable use of significant whitespace. Its language constructs and
object-oriented approach aim to help programmers write clear, logical code for small and
large-scale projects.
Python is dynamically typed and garbage-collected. It supports multiple programming
paradigms, including procedural, object-oriented, and functional programming. Python is
often described as a "batteries included" language due to its comprehensive standard library.
2. Jupyter Notebook
The Jupyter Notebook is an open-source web application that allows you to create and share
documents that contain live code, equations, visualizations and narrative text. Uses include:
data cleaning and transformation, numerical simulation, statistical modeling, data
visualization, machine learning, and much more.
7
LITERATURE SURVEY
MOVREC is a movie recommendation system presented by D.K. Yadav et al. based on
collaborative filtering approach. Collaborative filtering makes use of information provided by
user. That information is analyzed and a movie is recommended to the users which are arranged
with the movie with highest rating first.
Luis M Capos et al has analyzed two traditional recommender systems i.e. content based
filtering and collaborative filtering. As both of them have their own drawbacks he proposed a
new system which is a combination of Bayesian network and collaborative filtering. A hybrid
system has been presented by Harpreet Kaur et al. The system uses a mix of content as well as
collaborative filtering algorithm. The context of the movies is also considered while
recommending. The user - user relationship as well as user - item relationship plays a role in
the recommendation.
The user specific information or item specific information is clubbed to form a cluster by
Utkarsh Gupta et al. using chameleon. This is an efficient technique based on Hierarchical
clustering for recommender system. To predict the rating of an item voting system is used.
The proposed system has lower error and has better clustering of similar items.
Urszula Kużelewska et al. proposed clustering as a way to deal with recommender systems.
Two methods of computing cluster representatives were presented and evaluated. Centroid-
based solution and memory-based collaborative filtering methods were used as a basis for
comparing effectiveness of the proposed two methods. The result was a significant increase in
the accuracy of the generated recommendations when compared to just centroid-based method.
Costin-Gabriel Chiru et al. proposed Movie Recommender, a system which uses the
information known about the user to provide movie recommendations. This system attempts to
solve the problem of unique recommendations which results from ignoring the data specific to
the user. The psychological profile of the user, their watching history and the data involving
movie scores from other websites is collected. They are based on aggregate similarity
8
calculation. The system is a hybrid model which uses both content based filtering and
collaborative filtering.
To predict the difficulty level of each case for each trainee Hongli LIn et al. proposed a method
called contentboosted collaborative filtering (CBCF).The algorithm is divided into two stages,
First being the content-based filtering that improves the existing trainee case ratings data and
the second being collaborative filtering that provides the final predictions. The CBCF algorithm
involves the advantages of both CBF and CF, while at the same time, overcoming both their
disadvantages.
9
Approach
There are various types of recommender systems with different approaches and some of them
are classified as below:
1. Demographic Filtering- They offer generalized recommendations to every user, based
on movie popularity and/or genre. The System recommends the same movies to users with
similar demographic features. Since each user is different , this approach is considered to be
too simple. The basic idea behind this system is that movies that are more popular and critically
acclaimed will have a higher probability of being liked by the average audience. Before getting
started with this -
● We need a metric to score or rate movie
● Calculate the score for every movie
● Sort the scores and recommend the best rated movie to the users.
We can use the average ratings of the movie as the score but using this won't be fair enough
since a movie with 8.9 average rating and only 3 votes cannot be considered better than the
movie with 7.8 as as average rating but 40 votes. So, I'll be using IMDB's weighted rating
(wr) which is given as :-
where,
● v is the number of votes for the movie;
● m is the minimum votes required to be listed in the chart;
● R is the average rating of the movie; And
● C is the mean vote across the whole report
10
Fig. 1.1 Demographic Filtering
2. Content-based Filtering Systems: In content-based filtering, items are recommended
based on comparisons between item profile and user profile. A user profile is content that is
found to be relevant to the user in form of keywords(or features). A user profile might be seen
as a set of assigned keywords (terms, features) collected by algorithm from items found relevant
(or interesting) by the user. A set of keywords (or features) of an item is the Item profile. For
example, consider a scenario in which a person goes to buy his favorite cake ‘X’ to a pastry.
Unfortunately, cake ‘X’ has been sold out and as a result of this the shopkeeper recommends
the person to buy cake ‘Y’ which is made up of ingredients similar to cake ‘X’.
This is an instance of content-based filtering.
11
Fig. 2.1 Content Based Filtering
We will be using the cosine similarity to calculate a numeric quantity that denotes the similarity
between two movies. We use the cosine similarity score since it is independent of magnitude
and is relatively easy and fast to calculate. Mathematically, it is defined as follows:
We are now in a good position to define our recommendation function. These are the
following steps we'll follow :-
● Get the index of the movie given its title.
● Get the list of cosine similarity scores for that particular movie with all movies. Convert it
into a list of tuples where the first element is its position and the second is the similarity
score.
● Sort the aforementioned list of tuples based on the similarity scores; that is, the second
element.
● Get the top 10 elements of this list. Ignore the first element as it refers to self (the movie
most similar to a particular movie is the movie itself).
● Return the titles corresponding to the indices of the top elements.
12
While our system has done a decent job of finding movies with similar plot descriptions, the
quality of recommendations is not that great. "The Dark Knight Rises" returns all Batman
movies while it is more likely that the people who liked that movie are more inclined to
enjoy other Christopher Nolan movies. This is something that cannot be captured by the
present system.
Credits, Genres and Keywords Based Recommender
It goes without saying that the quality of our recommender would be increased with the usage
of better metadata. That is exactly what we are going to do in this section. We are going to
build a recommender based on the following metadata: the 3 top actors, the director, related
genres and the movie plot keywords.
From the cast, crew and keywords features, we need to extract the three most important
actors, the director and the keywords associated with that movie. Right now, our data is
present in the form of "stringified" lists , we need to convert it into a safe and usable structure
Fig. 2.2 Credits, Genres and Keywords Based Recommender
Advantages of content-based filtering are:
● They capable of recommending unrated items.
13
● We can easily explain the working of recommender system by listing the Content
features of an item.
● Content-based recommender systems use need only the rating of the concerned user,and
not any other user of the system.
Disadvantages of content-based filtering are:
● It does not work for a new user who has not rated any item yet as enough ratings are
required contentbased recommender evaluates the user preferences and provides accurate
recommendations.
● No recommendation of serendipitous items.
● Limited Content Analysis- The recommend does not work if the system fails to
distinguish the items hat a user likes from the items that he does not like.
3. Collaborative filtering based systems: Our content based engine suffers from some
severe limitations. It is only capable of suggesting movies which are close to a certain movie.
That is, it is not capable of capturing tastes and providing recommendations across genres.
Also, the engine that we built is not really personal in that it doesn't capture the personal
tastes and biases of a user. Anyone querying our engine for recommendations based on a
movie will receive the same recommendations for that movie, regardless of who she/he is.
Therefore, in this section, we will use a technique called Collaborative Filtering to make
recommendations to Movie Watchers. It is basically of two types:-
14
a) User based filtering- These systems recommend products to a user that similar
users have liked. For measuring the similarity between two users we can either use
pearson correlation or cosine similarity. This filtering technique can be illustrated
with an example. In the following matrix's, each row represents a user, while the
columns correspond to different movies except the last one which records the
similarity between that user and the target user. Each cell represents the rating that the
user gives to that movie. Assume user E is the target.
Fig. 3.1User Based Filtering
Although computing user-based CF is very simple, it suffers from several problems. One
main issue is that users’ preference can change over time. It indicates that precomputing the
matrix based on their neighboring users may lead to bad performance. To tackle this problem,
we can apply item-based CF.
15
b) Item Based Collaborative Filtering - Instead of measuring the similarity
between users, the item-based CF recommends items based on their similarity with
the items that the target user rated. Likewise, the similarity can be computed with
Pearson Correlation or Cosine Similarity. The major difference is that, with item-
based collaborative filtering, we fill in the blank vertically, as oppose to the
horizontal manner that user-based CF does. The following table shows how to do so
for the movie Me Before
Fig. 3.2 Item Based Filtering
It successfully avoids the problem posed by dynamic user preference as item-based CF is
more static. However, several problems remain for this method. First, the main issue is
scalability. The computation grows with both the customer and the product. The worst case
complexity is O(mn) with m users and n items. In addition, sparsity is another concern. Take
a look at the above table again. Although there is only one user that rated both Matrix and
Titanic rated, the similarity between them is 1. In extreme cases, we can have millions of
users and the similarity between two fairly different movies could be very high simply
because they have similar rank for the only user who ranked them both.
16
Single Value Decomposition
One way to handle the scalability and sparsity issue created by CF is to leverage a latent
factor model to capture the similarity between users and items. Essentially, we want to turn
the recommendation problem into an optimization problem. We can view it as how good we
are in predicting the rating for items given a user. One common metric is Root Mean Square
Error (RMSE). The lower the RMSE, the better the performance.
Now talking about latent factor you might be wondering what is it ?It is a broad idea which
describes a property or concept that a user or an item have. For instance, for music, latent
factor can refer to the genre that the music belongs to. SVD decreases the dimension of the
utility matrix by extracting its latent factors. Essentially, we map each user and each item into
a latent space with dimension r. Therefore, it helps us better understand the relationship
between users and items as they become directly comparable. The below figure illustrates this
idea.
Fig 3.3 Single Value Decomposition
Now enough said , let's see how to implement this. Since the dataset we used before did not
have userId(which is necessary for collaborative filtering) let's load another dataset. We'll be
using the Surprise library to implement SVD.
17
Advantages of collaborative filtering based systems:
● It is dependent on the relation between users which implies that it is
content-independent.
● CF recommender systems can suggest serendipitous items by observing similar-minded
people’s behavior.
● They can make real quality assessment of items by considering other peoples experience
Disadvantages of collaborative filtering are:
● Early rater problem: Collaborative filtering systems cannot provide recommendations for
new items since there are no user ratings on which to base a prediction.
● Gray sheep: In order for CF based system to work, group with similar characteristics are
needed. Even if such groups exist, it will be very difficult to recommend users who do
not consistently agree or disagree to these groups.
● Sparsity problem: In most cases, the amount of items exceed the number of users by a
great margin which makes it difficult to find items that are rated by enough people.
18
RESULTS
1. Demographic Filtering
Fig. 4.1 Demographic Output
19
2. Content-based Filtering Systems
Fig. 4.2 Content Based Output_1
get_recommendations() function by passing in the new cosine_sim2 matrix is
Fig. 4.3 Content Based Output_2
20
3. Collaborative filtering based systems
Fig. 4.4 Collaborative Based Output_1
Fig. 4.5 Collaborative Based Output_2
21
For movie with ID 302, we get an estimated prediction of 2.851. One startling feature of this
recommender system is that it doesn't care what the movie is (or what it contains). It works
purely on the basis of an assigned movie ID and tries to predict ratings based on how the other
users have predicted the movie.
22
CONCLUSION
A hybrid approach is taken between context based filtering and collaborative filtering to
implement the system. This approach overcomes drawbacks of each individual algorithm and
improves the performance of the system. Techniques like Clustering, Similarity and
Classification are used to get better recommendations thus reducing MAE and increasing
precision and accuracy. In future we can work on hybrid recommender using clustering and
similarity for better performance. Our approach can be further extended to other domains to
recommend songs, video, venue, news, books, tourism and e-commerce sites, etc.
23
References
1. Peng, Xiao, Shao Liangshan, and Li Xiuran. "Improved Collaborative Filtering Algorithm
in the Research and Application of Personalized Movie Recommendations", 2013 Fourth
International Conference on Intelligent Systems Design and Engineering Applications,
2013.
2. Munoz-Organero, Mario, Gustavo A. Ramíez-González, Pedro J. Munoz-Merino, and
Carlos Delgado Kloos. "A Collaborative Recommender System Based on Space- Time
Similarities", IEEE Pervasive Computing, 2010.
3. Al-Shamri, M.Y.H.. "Fuzzy-genetic approach to recommender systems based on a novel
hybrid user model", Expert Systems With Applications, 200810
4. Hu Jinming. "Application and research of collaborative filtering in e-commerce
recommendation system", 2010 3rd International Conference on Computer Science and
Information Technology, 07/2010
5. Xu, Qingzhen Wu, Jiayong Chen, Qiang. "A novel mobile personalized recommended
method based on money flow model for stock exchange.(Researc", Mathematical Problems
in Engineering, Annual 2014 Issue
6. Yan, Bo, and Guanling Chen. "AppJoy : personalized mobile application discovery",
Proceedings of the 9th international conference on Mobile systems applications and
services - MobiSys 11 MobiSys 11, 2011.
7. Davidsson C, Moritz S. Utilizing implicit feedback and context to recommend mobile
applications from first use.In: Proc. of the Ca RR 2011. New York: ACM Press, 2011. 19-
22.http://dl.acm.org/citation.cfm?id=1961639[doi:10.1145/1961634.1961639]
8. Bilge, A., Kaleli, C., Yakut, I., Gunes, I., Polat, H.: A survey of privacy-preserving
collaborative filtering schemes. Int. J. Softw. Eng. Knowl. Eng. 23(08), 1085–
1108 (2013)CrossRefGoogle Scholar
9. Calandrino, J.A., Kilzer, A., Narayanan, A., Felten, E.W., Shmatikov, V.: You might also
like: privacy risks of collaborative filtering. In: Proceedings of the IEEE Symposium on
24
Security and Privacy, pp. 231–246, Oakland, CA, USA (2011)
10. Research.ijcaonline.org
11. Hu Jinming. "Application and research of collaborative filtering in e-commerce
recommendation system", 2010 3rd International Conference on Computer Science and
Information Technology, 07/2010
12. www.pacis2014.org
25
