1 RECOMMENDATION SYSTEMS ÖZNUR KIRMEMİŞ
Dec 20, 2015
1
RECOMMENDATION SYSTEMS
ÖZNUR KIRMEMİŞ
OUTLINE
INTRODUCTION FORMALIZATION OF THE PROBLEM APPROACHES
COLLABORATIVE CONTENT BASED HYBRID
CONCLUSION
PAPERS
1. Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions, Gediminas Adomavicius, Alexander Tuzhilin IEEE Transactions on Knowledge and Data Engineering(June 2005)
2. Content-Boosted Collaborative Filtering for Improved Recommendations, Prem Melville, Raymond J. Mooney and Ramadass Nagarajan, Proceedings of the Eighteenth National Conference on Artificial Intelligence(AAAI-2002)
3. Recommendation as Classification: Using Social and Content-Based Information in Recommendation, Chumki Basu, Haym Hirsh,William Cohen(AAAI-1998)
4
PART 1:
INTRODUCTION
Recommendations
Items
Search Recommendations
Products, web sites, blogs, news items, …
We are in the Information society. The quantity of new information available every day goes over our limited processing capabilities.We face far more choices than we can try in the world, like, which book shall I read, which movie is worth watching, where I shall have dinner tonight, etc.For this reason, we need something able to suggest us only the worthwhile information.Make search space smaller!
Recommendations
Acting upon recommendations from other people is a normal part of life.
By using recommendations we can take a shortcut to the things we like without having to try many things we dislike or without having to acquire all the knowledge to make an informed decision.
Recommender systems(RS) automate this facility.
Recommendation systems are thus a solution for information overload.
DEFINITION OF RS
programs which attempt to predictitems (movies, music, books, news,web pages) that a user may beinterested in, given some informationabout the user's profile
Recommendation Systems
Based on a synthesis of ideas from; Artificial Intelligence Natural Language Processing Human-Computer Interaction Sociology Information Retrieval and the technology of the WWW
GENERIC RS For a typical recommender system, there are three
steps: The user provides some form of input to the system. These
inputs can be both explicit and implicit . Ratings submitted by users are among explicit inputs whereas the URLs visited by a user and time spent reading a web site are among possible implicit inputs.
These inputs are brought together to form a representation of the user's likes and dislikes. This representation could be as simple as a matrix of items-ratings, or as complex as a data structure combining both content and rating information.
The system computes recommendations using these user profiles.
Even though the steps are essentially the same for most recommender systems, there have been different approaches to both step 2 and 3.
Current Examples
MovieLens Movie recommendation makes use of collaborative filtering
technology gathers user preferences by asking the user
to rate movies. searches for similar profiles (i.e. users that
share the same or similar taste) and uses them to generate new suggestions.
Current Examples
Amazon Book recommendations recommends books frequently purchased by
customers who purchased the selected book customers receive text recommendations based
on the opinions of other customers
LIBRA Book recommendations Combines a content-based approach with
machine learning
Current Examples
Cinemax.com Moviecritic: movies again And much more……
13
PART 2:
FORMALIZATION OF THE PROBLEM
Formal Model<C,S,u>
Let C be the set of all users or customers and let S be the set of all possible items that can be recommended, such as books, movies, or
restaurants. S = set of Items C = set of Customers
Let u be a utility function that measures the usefulness of item s to user c Utility function u:
C × S → R,
Utility Function
Utility function u: C × S → R, R
e.g., 0-5 stars, real number in [0,1] u(c1,s1) = r1; u(c1,s2) = r2;..... Recommendation: for each user c є C,
choose such item si є S that maximizes the user’s utility
USER SPACE && ITEM SPACE
USER SPACE(C): can be defined with a profile that includes various
user characteristics, such as age, gender, income, marital status, etc.
ITEM SPACE(S): Similarly, each element of the item space S can be
defined with a set of characteristics. Ex; (in a movie recommendation application):
S: a collection of movies, each movie can be represented not only by its ID, but
also by its title, genre, director, year of release, leading actors, etc.
UTILITY FUNCTION
The central problem of recommender systems lies in that utility u is usually not defined on the whole CXS space, but only on some subset of it.
This means u needs to be extrapolated to the whole space CXS.
The recommendation engine should be able to estimate the ratings of the nonrated item/user combinations and issue appropriate recommendations based on these predictions.
Example: Utility Matrix
0.4
10.2
0.30.5
0.21
King KongKing Kong GarfieldGarfield MatrixMatrix Usual SuspectsUsual Suspects
AyşeAyşe
AliAli
VeliVeli
HasanHasan
Gathering “known” ratings for matrixExtrapolate unknown ratings from known ratings
EXTRAPOLATION
Extrapolations from known ratings are done by Specifying heuristics that defines the utility function
and validating its performance. Estimating the utility function that optimizes certain
performace criterion, such as the mean square error.
Once the unknown ratings are estimated, recommendations to a user are made by selecting the highest rating among all the estimated ratings for that user. Alternatively, we can recommend the N best items to a
user.
20
PART 3:
APPROACHES•Content Based•Collaborative•Hybrid
APPROACHES
Recommender systems are usually classified into the following categories, based on how recommendations are made: Content-based recommendations:
The user will be recommended items similar to the ones the user preferred in the past, similarity between user profile and item profile, or similarity between item profiles.
Collaborative recommendations: aim to identify users that have relevant interests and
preferences by calculating similarities and dissimilarities between user profiles
The user will be recommended items that are preferred by other people with similar tastes and preferences.
Hybrid approaches: These methods combine collaborative and content-based
methods.
CONTENT BASED METHODS
Content-based Methods
Main idea: recommend items to customer C similar to
previous items rated highly by C No similar user information!!
Formalization: the utility u(c,s) of item s for user c is estimated
based on the utilities u(c,si) assigned by user c to items si є S
that are “similar” to item s.
Content-based Methods
has its roots in information retrieval and information filtering research. The improvement over the traditional information
retrieval approaches comes from the use of user profiles that contain information about users’ tastes, preferences, and needs.
The profiling information can be obtained from users explicitly, e.g., through
questionnaires, or implicitly—learned from their transactional behavior
over time. Can use a machine learning algorithm to induce a
profile of the users preferences
Plan of action(Item Profile+User Profile+Prediction Mechanism)
likeslikesItem profilesItem profiles
RedRedCirclesCircles
TrianglesTriangles
User profileUser profile
matchmatch
recommendrecommendbuildbuild
recommend objects with similar content, same color, shape,..
Item Profiles
For each item, create an item profile Let Content(s) be an item profile,
a set of attributes characterizing item s. movies: author, title, actor, director text: set of “important” words in document
attributes are used to determine the appropriateness of the item for recommendation purposes.
Item Profiles
How attributes determined? straightforward
By deciding which slots are important Slots: Author,Title,Editorial Reviews,..etc
By processing texts The “importance” (or “informativeness”) of word kj in
document dj is determined with some weighting measure wij that can be defined in several different ways.
One of the best-known measures for specifying keyword weights in Information Retrieval is the term frequency/inverse document frequency (TF-IDF) measure.
User profiles
Let ContentBasedProfile(c) be the profile of user c containing preferences of this user. These profiles are obtained by analyzing the content of the items previously seen and constructed using keyword analysis techniques from information retrieval.
For example, ContentBasedProfile(c) can be defined as a vector of weights (wc1, . . . , wck), where each weight wci denotes the importance of keyword ki to user c and can be computed from individually rated content vectors using a variety of techniques.
Prediction
In content-based systems, the utility function u(c,s) is usually defined as:
Especially, recommending Web pages, both ContentBasedProfile(c) of user c and Content(s) of document s can be represented as TF-IDF vectors and of keyword weights. Moreover, utility function u(c,s) is usually represented in the information retrieval literature by some scoring heuristic defined in terms of vectors mentioned above, such as the
cosine similarity measure. K is the total number of keywords in the system.
cw
sw
LIBRALearning Intelligent Book Recommending Agent
Content-based recommender for books using information about titles extracted from Amazon.
Uses information extraction from the web to organize text into fields: Author Title Editorial Reviews Customer Comments Subject terms Related authors Related titles
EXAMPLE: LIBRA System
Amazon Pages
Rated Examples
User Profile
Machine Learning
Learner
InformationExtraction
LIBRA Database
Recommendations
1.~~~~~~2.~~~~~~~3.~~~~~::: Predictor
Sample Extracted Information
Title: <The Age of Spiritual Machines: When Computers Exceed Human Intelligence>Author: <Ray Kurzweil>Price: <11.96>Publication Date: <January 2000>ISBN: <0140282025>Related Titles: <Title: <Robot: Mere Machine or Transcendent Mind> Author: <Hans Moravec> > …Reviews: <Author: <Amazon.com Reviews> Text: <How much do we humans…> > …Comments: <Stars: <4> Author: <Stephen A. Haines> Text:<Kurzweil has …> > …Related Authors: <Hans P. Moravec> <K. Eric Drexler>…Subjects: <Science/Mathematics> <Computers> <Artificial Intelligence> …
Libra Content Information
Libra uses this extracted information to form “bags of words” for the following slots: Author Title Description (reviews and comments) Subjects Related Titles Related Authors
Libra Overview
User rates selected titles on a 1 to 10 scale.
Libra uses a naïve Bayesian text-categorization algorithm to learn a profile from these rated examples. Rating 6–10: Positive Rating 1–5: Negative
LIMITATIONS(Content Based)
Finding the appropriate features Overspecialization
Never recommends items outside user’s content profile
introduce some randomness ex: genetic algorithms
the diversity of recommendations is often a desirable feature in recommender systems.
Too similar items should not be recommended, ex:a different news article describing the same
event.
LIMITATIONS(Content Based)
Recommendations for new users How to build a profile?
The user has to rate a sufficient number of items before a content-based recommender system can really understand the user’s preferences. Therefore, a new user, having very few ratings, would not be able to get accurate recommendations.
COLLABORATIVE FILTERING
Collaborative Filtering
Unlike content-based recommendation methods, collaborative recommender systems (or collaborative filtering systems) try to predict the utility of items based on the items previously rated by other similar users.
The utility u(c,s) of item s for user c is estimated based on the utilities u(c,s) assigned to item s by those users cj є C who are “similar” to user c.
Basic Algorithm
Maintain a database of many users’ ratings of a variety of items.
For a given user, find other similar users whose ratings strongly correlate with the current user.
Recommend items rated highly by these similar users, but not rated by the current user.
Almost all existing commercial recommenders use this approach (e.g. Amazon).
Similar Users Let rx be the vector of user x’s ratings Cosine similarity measure
sim(x,y) = cos(rx , ry)
Pearson correlation coefficient
....
Collaborative Filtering
A 9B 3C: :Z 5
A B C 9: :Z 10
A 5B 3C: : Z 7
A B C 8: : Z
A 6B 4C: :Z
A 10B 4C 8. .Z 1
UserDatabase
ActiveUser
CorrelationMatch
A 9B 3C . .Z 5
A 9B 3C: :Z 5
A 10B 4C 8. .Z 1
ExtractRecommendations
C
LIMITATIONS(Collaborative)
New User Problem: same problem as with content-based systems. In order to make accurate recommendations, the system
must first learn the user’s preferences from the ratings that the user gives.
New Item Problem: New items are added regularly to recommender systems. Collaborative systems rely solely on users’ preferences to
make recommendations. Therefore, until the new item is rated by a substantial
number of users, the recommender system would not be able to recommend it.
Not a problem in content based!! Works for any kind of item, No feature selection needed
HYBRID METHODS
Hybrid Methods
Content-based and collaborative methods have complementary strengths and weaknesses.
Combine methods to obtain the best of both.
HOW TO COMBINE?
Implement two separate recommenders and combine predictions, by giving weights
Add content-based methods to collaborative filtering Use content-based predictor to
complete collaborative data. “Content-Boosted Collaborative Filtering for
Improved Recommendations”,Prem Melville and Raymond J. Mooney and Ramadass Nagarajan, 2002,AAAI
Movie Domain
hybrid approach in the domain of movie recommendation the user-movie ratings from the EachMovie dataset
The dataset contains rating data provided by each user for various movies.
User ratings range from zero to five stars. Zero stars indicate extreme dislike for a movie and five stars indicate high praise.
The content information for each movie was collected from IMDb using a simple crawler.
The crawler follows the IMDB link provided for every movie in the EachMovie dataset and collects information.
Content information of every movie is represented by a set of slots (features).
Each slot is represented simply as a bag of words. The slots used for the Each-Movie dataset are: movie title, director, cast,
genre, plot
Content-Boosted CF - I
Content-Based Predictor
Training Examples
Pseudo User-ratings Vector
Items with Predicted Ratings
User-ratings Vector
User-rated ItemsUnrated Items
Content-Boosted CF - II
Compute pseudo user ratings matrix Full matrix – approximates actual full user ratings
matrix Perform CF
Using Pearson corr. between pseudo user-rating vectors
User RatingsMatrix
Pseudo UserRatings Matrix
Content-BasedPredictor
Content-Boosted Collaborative Filtering
IMDbEachMovie Web Crawler
MovieContentDatabase
Full UserRatings Matrix
Collaborative Filtering
Active User Ratings
User RatingsMatrix (Sparse) Content-based
Predictor
Recommendations
50
PART 4:
CONCLUSION
CONCLUSION
Recommendation System is an important technology to combating information overload.
Collaborative filtering has problems. Content-based methods address these
problems (but have problems of their own).
Integrating both is best.
THANK YOU FOR LISTENING
QUESTIONS?QUESTIONS?