Recommender systemms search engines

RECOMMENDER SYSTEMS AND SEARCH ENGINES – TWO SIDES OF THE SAME

COIN?!Bracha Shapira

Lior Rokach

Department of Information Systems Engineering

Ben-Gurion University

CONTENT

Introduction Applications Methods Recommender Systems vs. search engines

ARE YOU BEING SERVED? What are you looking for? Demographic – Age, Gender, etc. Context-

Casual/Event Season Gift

Purchase History Loyal Customer What is the customer currently wearing?

Style Color

Social Friends and Family Companion

4

RECOMMENDER SYSTEMS

A recommender system (RS) helps people that have not sufficient personal experience or competence to evaluate the, potentially overwhelming, number of alternatives offered by a Web site. In their simplest form RSs recommend to their

users personalized and ranked lists of items Provide consumers with information to help

them decide which items to purchase

EXAMPLE APPLICATIONS

6

WHAT BOOK SHOULD I BUY?

7

WHAT MOVIE SHOULD I WATCH?

• The Internet Movie Database (IMDb) provides information about actors, films, television shows, television stars, video games and production crew personnel.

• Owned by Amazon.com since 1998 • 796,328 titles and 2,127,371 people• More than 50M users per month.

08.04.2023

In October 2006, Netflix announced it would give a $1 million to whoever created a movie-recommending algorithm 10% better than its own.

Within two weeks, the DVD rental company had received 169 submissions, including three that were slightly superior to Cinematch, Netflix's recommendation software

After a month, more than a thousand programs had been entered, and the top scorers were almost halfway to the goal

But what started out looking simple suddenly got hard. The rate of improvement began to slow. The same three or four teams clogged the top of the leader-board.

Progress was almost imperceptible, and people began to say a 10 percent improvement might not be possible.

Three years later, on 21st of September 2009, Netflix announced the winner.

abcdThe Nextflix prize story

8

9

WHAT NEWS SHOULD I READ?

10

WHERE SHOULD I SPEND MY VACATION?

Tripadvisor.com

I would like to escape from this ugly an tedious work life and

relax for two weeks in a sunny place. I am fed up with

these crowded and noisy places … just the sand and the

sea … and some “adventure.” I would like to bring my wife and my children on a

holiday … it should not be to expensive. I prefer

mountainous places… not too far from home.

Children parks, easy paths and good cuisine are a

must.I want to experience the contact with a completely different

culture. I would like to be fascinated by the people and

learn to look at my life in a totally different way.

08.04.2023

Usage in the market/products Recommendation State-of-the-art solutions

Method CommonnessExamined Solutions

Jinni Taste Kid Nanocrowd Clerkdogs Criticker IMDb Flixster Movielens Netflix Shazam Pandora LastFM YooChoose Think Analytics Itunes Amazon

Collaborative Filtering v       v v v v v v   v v v v v

Content-Based Techniques v v v v   v   v     v v v v   v

Knowledge-Based Techniques v v v v   v         v     v    Stereotype-Based Recommender Systems

v v v v   v             v v    

Ontologies and Semantic Web Technologies for Recommender Systems

v   v               v          

Hybrid Techniques v   v   v     v v       v v    

Ensemble Techniques for Improving Recommendation

                v future

Context Dependent Recommender Systems

v   v v v               v v    

Conversational/Critiquing Recommender Systems

v                         v    

Community Based Recommender Systems and Recommender Systems 2.0

v           v   v v   v        

11

COLLABORATIVE FILTERING

08.04.2023

kNN - Nearest Neighbor SVD – Matrix Factorization

Similarity Weights Optimization (SWO)

The method of making automatic predictions (filtering) about the interests of a user by collecting taste information from many users (collaborating). The underlying assumption of CF approach is that those who agreed in the past tend to agree again in the future.

Collaborative Filtering1Des

crip

tion

Sel

ecte

d Te

chni

ques

Collaborative Filtering

13

08.04.2023

COLLABORATIVE FILTERING

Trying to predict the opinion the user will have on the different items and be able to recommend the “best” items to each user based on: the user’s previous likings and the opinions of other like minded users

abcdThe Idea

?Positive Rating

Negative Rating

14

08.04.2023

abcdHow it works

How collaborative filtering works?“People who liked this also liked”…

15

Item to

Item

User to User

abcdUser-to-User

Recommendations are made by finding users with similar tastes. Jane and Tim both liked Item 2 and disliked Item 3; it seems they might have similar taste, which suggests that in general Jane agrees with Tim. This makes Item 1 a good recommendation for Tim.This approach does not scale well for millions of users.

Item-to-Item

Recommendations are made by finding items that have similar appeal to many users. Tom and Sandra are two users who liked both Item 1 and Item 4. That suggests that, in general, people who liked Item 4 will also like item 1, so Item 1 will be recommended to Tim. This approach is scalable to millions of users and millions of items.

08.04.2023

Hamming distance

5 6 6 5 4 8

0 Dislike

1 Like

? Unknown

1

?

0

1

1

0

1

1

0

1

1

1

1

0

Current User Users

Item

s

User Model = interaction history

1

1st item rate

14th item rate

KNN - NEAREST NEIGHBOR

NearestNeighbor

Nearest Neighbor

abcd

16

This user did not rate the item. We will try to predict a rating according to hisneighbors.

This user did not rate the item. We will try to predict a rating according to his neighbors.

abcdUnknown Rating

There are other users who rated the same item. We are interested in the Nearest Neighbors.

There are other users who rated the same item. We are interested in the Nearest Neighbors.

abcdOther Users

We are looking for the Nearest Neighbor. The one with the lowest Hamming distance.

We are looking for the Nearest Neighbor. The one with the lowest Hamming distance.

abcdNearest Neighbors

The prediction was made based on the nearest neighbor.

The prediction was made based on the nearest neighbor.

abcdPrediction

The Hamming distance is named after Richard Hamming.

In information theory, the Hamming distance between two strings of equal length is the number of positions at which the corresponding symbols are different.

The Hamming distance is named after Richard Hamming.

In information theory, the Hamming distance between two strings of equal length is the number of positions at which the corresponding symbols are different.

abcdHamming Distance

IMPORTANT ISSUES

Cold Start Implicit/Explicit Rating Sparsity

Long Tail problem - many items in the Long Tail have only few ratings

Portfolio Effect: Non Diversity Problem It is not useful to recommend all movies by Antonio

Banderas to a user who liked one of them in the past Beyond Popularity

Gray sheep problem Iformation Security

Misuse Privacy

CONTENT-BASED RECOMMENDER SYSTEM

19

CONTENT-BASED RECOMMENDATION In content-based recommendations the system

tries to recommend items that matches the User Profile.

The Profile is based on items user has liked in the past or explicit interests that he defines.

A content-based recommender system matches the profile of the item to the user profile to decide on its relevancy to the user.

20

SIMPLE EXAMPLE

Read updat

e User Profile

New books User Profile

Recommender Systems

Match

recommendation

CONTEXT-BASED RECOMMENDER SYSTEMS

08.04.2023

The recommender system uses additional data about the context of an item consumption.

For example, in the case of a restaurant the time or the location may be used to improve the recommendation compared to what could be performed without this additional source of information.

A restaurant recommendation for a Saturday evening when you go with your spouse should be different than a restaurant recommendation on a workday afternoon when you go with co-workers

abcdOverview

Context-Based Recommender Systems

22

08.04.2023

Recommend a vacation Winter vs. summer

Recommend a purchase (e-retailer) Gift vs. for yourself

Recommend a movie To a student who wants to watch it on Saturday

night with his girlfriend in a movie theater.

Motivating Examples

Context-Based Recommender Systems

23

08.04.2023

Recommend music The music that we like to hear is greatly affected by a

context, such that can be thought of a mixture of our feelings (mood) and the situation or location (the theme) we associate it with.

Listen to Bruce Springteen "Born in USA" while driving along the 101.

Listening to Mozart's Magic Flute while walking in Salzburg.

Motivating Examples

Context-Based Recommender Systems

24

08.04.2023

abcdMusicovery.com

An Interactive personalized WebRadio

A mood matrix propose a relationship between music and mood.

Ethnographic studies have shown that people choose music peaces according to their mood or mood change expectation.

abcdDetails

Information Discovery: Example“Tell me the music that I want to listen NOW"

25

08.04.2023

Context-Based Recommender Systems

What is the user doing when asking for a recommendation? Where (and when) the user is located? What does the user really want (e.g., improve his knowledge or

really buy a product)? Is the user alone or with other fellows? Are there many products to choose or only few?

What simple recommendation techniques ignore?

26

08.04.2023

What is the user doing when asking for a recommendation? Where (and when) the user is located? What does the user really want (e.g., improve his knowledge or

really buy a product)? Is the user alone or with other fellows? Are there many products to choose or only few?

Plain recommendation technologies forget to takeinto account the user context.

Context-Based Recommender Systems

What simple recommendation techniques ignore?

27

08.04.2023

Obtain sufficient and reliable data describing the user context

Selecting the right information, i.e., relevant in a particular personalization task

Understand the impact of contextual dimensions on the personalization process

Computational model the contextual dimension in a more classical recommendation technology

For instance: how to extend Collaborative Filtering to include contextual dimensions?

abcdMajor obstacle for contextual computing

Context-Based Recommender Systems

28

08.04.2023

Each item in the data base ( ) is a candidate for splitting Context defines ( ) all possible splits of an item ratings vector

We test all the possible splits – we do not have many contextual features

We choose one split (using a single contextual feature) that maximizesan impurity measure and whose impurity is higher than a threshold

abcdItem Split - Intuition and Approach

Context-Based Recommender Systems

29

SOCIAL (TRUST) BASED RECOMMENDER SYSTEMS

08.04.2023

Intuition – Users tend to receive advice from people they trust, i.e., from their friends.

Trusted friends can be defined explicitly by the users or inferred from social networks they are registered to.

.

abcdOverview

Social Based (Trust based) Recommender Systems

31

?3

Active user

Rating prediction

TRUST- BASED COLLABORATIVE FILTERING

Active users’ trusted friends

TRUST METRICS

Global metrics: computes a single global trust value for every single user (reputation on the network)

Pros: Based on the whole community opinion

Cons: Trust is subjective (controversial users)

a

b

d

c

1 3

32

3

TRUST METRICS (CONT.) Local metrics: predicts (different) trust scores

that are personalized from the point of view of every single user

Pros: More accurate Attack resistance

Cons: Ignoring the “wisdom of the crowd”

a

b

d

c

1 5

32

?

SEARCH ENGINES AND RECOMMENDER SYSTEMS

SEARCH ENGINES VS. RECOMMENDER SYSTEMS –

Search Engines Goal – answer users

ad hoc queries Input – user ad-hoc

need defined as a query

Output- ranked items relevant to user need (based on her preferences???)

Methods - Mainly IR based methods

Recommender Systems Goal – recommend

services or items to user Input - user preferences

defined as a profile

Output - ranked items based on her preferences

Methods – variety of methods, IR, ML, UM

NEW TRENDS…

“Understand” the user actual needs from her context

Personalize results according to the user preferences

Search engines may use some recommender systems methods to achieve these goals

SEARCH ENGINES PERSONALIZATION METHODSADOPTED FROM RECOMMENDER SYSTEMS

Collaborative filtering User-based - Cross domain collaborative filtering is

required??? Content-based

Search history Collaborative content-based

Collaborate on similar queries Context-based

Little research – difficult to evaluate Locality, language, calendar

Social-based Friends I trust relating to the query domain Notion of trust, expertise