Crowd-Based Personalized Natural Language Explanations for Recommendations Shuo (Steven) Chang Maxwell Harper Loren Terveen GroupLens University of Minnesota 1
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Crowd-Based Personalized Natural Language Explanations
for Recommendations Shuo (Steven) Chang
Maxwell Harper Loren Terveen
GroupLens University of Minnesota
1
2 Background Introduction Method Experiment Result Discussion
How do recommenders explain themselves?
3 Background Introduction Method Experiment Result Discussion
4
Herlocker et. al. (2000) Explaining collaborative filtering recommendations.
Background Introduction Method Experiment Result Discussion
5
Can we generate personalized natural language explanations at scale?
Background Introduction Method Experiment Result Discussion
6
Tintarev et. al. (2012) Evaluating the effectiveness of explanations for recommender systems.
Vig. et. al (2008) Tagsplanations.
Background Introduction Method Experiment Result Discussion
7
Can we do better than formulaic explanations?
Background Introduction Method Experiment Result Discussion
8 Background Introduction Method Experiment Result Discussion
9 Background Introduction Method Experiment Result Discussion
10
From your MovieLens profile it seems that you prefer movies tagged as visual, Gravity is unlike what we have seen on a cinema screen before and arguably it has one of the best uses of 3D in a movie.
Background Introduction Method Experiment Result Discussion
11
From your MovieLens profile it seems that you prefer movies tagged as intense, the movie is a pretty intense ninety minutes, with Bullock's character constantly battling one catastrophe after another, and all of it is amazing to see.
Background Introduction Method Experiment Result Discussion
Process Overview
12 Background Introduction Method Experiment Result Discussion
13 Background Introduction Method Experiment Result Discussion
14
From your MovieLens profile it seems that you prefer movies
tagged as [Topical Dimension], [Natural Language Explanation].
1. Model topics of items
2. Generate explanations
3. Model user interest and present matching explanation
Background Introduction Method Experiment Result Discussion
15
Model topics of items
1
Background Introduction Method Experiment Result Discussion
Crowd recruited from Amazon Mechanical Turk
16
Top 20 most relevant tags
Background Introduction Method Experiment Result Discussion
mafia, gangster, gangsters, mob, crime, mentor
violent, narrated, violence, stylish, visceral, stylized, bloody, brutality
masterpiece, storytelling, drama, dialogue
interesting, original
17
Build semantic similarity graph (word2vec trained on
IMDB reviews)
Clustering (Affinity
Propagation)
Input to crowd
Background Introduction Method Experiment Result Discussion
18
Crowds refine clusters
Background Introduction Method Experiment Result Discussion
19
1.Filter tags
2.Pick representative tag
Background Introduction Method Experiment Result Discussion
20 Background Introduction Method Experiment Result Discussion
mafia, gangster, gangsters, mob, crime, mentor
violent, narrated, violence, stylish, visceral, stylized, bloody, brutality
masterpiece, storytelling, drama, dialogue
interesting, original
21
mafia, gangster, gangsters, mob, crime, mentor
violent, narrated, violence, stylish, visceral, stylized, bloody, brutality
masterpiece, storytelling, drama, dialogue
interesting, original
Background Introduction Method Experiment Result Discussion
22
Generate explanations
2
Background Introduction Method Experiment Result Discussion
23
Quotes from IMDB reviews
mafia, gangster, gangsters, mob, crime, mentor
violent, narrated, violence, stylish, visceral, stylized, bloody, brutality
masterpiece, storytelling, drama, dialogue
interesting, original
Background Introduction Method Experiment Result Discussion
24
Map
Background Introduction Method Experiment Result Discussion
25
Reduce
Background Introduction Method Experiment Result Discussion
26
Model user interest and present matching explanation
3
Background Introduction Method Experiment Result Discussion
User Experiment
27 Background Introduction Method Experiment Result Discussion
28
• Generate natural language explanations for 100 movies with $3.90/movie
• Survey 216 MovieLens users
Background Introduction Method Experiment Result Discussion
29
• Within subject design • Two random unseen movies • One with baseline and another with
natural language explanation
Background Introduction Method Experiment Result Discussion
30
• Baseline: user preferred relevant topics
We recommend the movie because you like the following features: [tag1, ..., tag5]
Vig. et. al (2008) Tagsplanations.
Background Introduction Method Experiment Result Discussion
Result
31 Background Introduction Method Experiment Result Discussion
32
Users are more satisfied with natural language explanations
1
Background Introduction Method Experiment Result Discussion
33
15%
25%
66%
51%
19%
25%
13%
22%
76%
61%
12%
17%
19%
26%
68%
51%
13%
23%
I wish MovieLens included explanations like this.
The explanation is easy to understand.
The explanation is useful.
CROWD
TAG
CROWD
TAG
CROWD
TAG
100 50 0 50 100Percentage
Strongly disagree Disagree Neutral Agree Strongly agree
Background Introduction Method Experiment Result Discussion
34
Users are have more trust in natural language explanations
2
Background Introduction Method Experiment Result Discussion
35
15%
17%
66%
57%
19%
26%
24%
22%
48%
42%
28%
36%
I trust the explanation.
The explanation reflects my preferences about this movie.CROWD
TAG
CROWD
TAG
100 50 0 50 100Percentage
Strongly disagree Disagree Neutral Agree Strongly agree
Background Introduction Method Experiment Result Discussion
36
Users perceive natural language explanations to contain more
appropriate amount of information
3
Background Introduction Method Experiment Result Discussion
37
34%
55%
45%
19%
21%
26%
The explanation contains right amount of information.
CROWD
TAG
100 50 0 50 100Percentage
Strongly disagree Disagree Neutral Agree Strongly agree
4%
6%
63%
55%
32%
39%
Changes in response regarding knowledge about a movie
CROWD
TAG
100 50 0 50 100Percentage
Response −2 −1 0 1 2 3 4
Background Introduction Method Experiment Result Discussion
38
4
Little difference in helping users make decision
Background Introduction Method Experiment Result Discussion
39
Discussion
Background Introduction Method Experiment Result Discussion
40 Background Introduction Method Experiment Result Discussion
Mixed computation
41
https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwi34q-j6bTOAhUE2mMKHdnfBcEQjRwIBw&url=https%3A%2F%2Fwww.engadget.com%2F2014%2F09%2F08%2Fgoogle-details-object-recognition-tech%2F&bvm=bv.129389765,d.cGc&psig=AFQjCNEWUMTMw249fiU1MrdLKiztXM_KOA&ust=1470848879786441
Background Introduction Method Experiment Result Discussion
42 Background Introduction Method Experiment Result Discussion
43 Background Introduction Method Experiment Result Discussion
44
Takeaways
Background Introduction Method Experiment Result Discussion
45
• Mixed computation approach • Human effort to
- Refine item topic clusters - Synthesize review quotes
• Better user experience than tag based explanations
Background Introduction Method Experiment Result Discussion
46
Questions?
Shuo (Steven) Chang @ [email protected]
http://www-users.cs.umn.edu/~schang
Crowd-Based Personalized Natural Language Explanations for Recommendations
47 Intro New user Model topics Recommend Explain Discussion
mafia, gangster, gangsters, mob, crime, mentor
violent, narrated, violence, stylish, visceral, stylized, bloody, brutality
masterpiece, storytelling, drama, dialogue
interesting, original
Quotes about drama masterpiece, story-
telling, dialogue :
As much as the true events of Henry’s life have more than likely been dramatised and glamourised to a certain extent, the essence of this film IMO is that it is still a brilliantly damning portrayal of the characters and lifestyle of mobsters.
The consistently fine acting by the large ensemble cast (both known and unknown), the cinematography, editing, dialogue, brilliant use of music, it’s all breathtaking.
The dialogue is incredible.
Storytelling with impeccable pacing, this is what it’s like when a master composer conducts his masterpiece.
If ever the word ‘masterpiece’ was meant to be used, it was for this film. ‘Goodfellas’ is a masterpiece, pure and simple.
48 Background Motivation Expt_1 Expt_2 Discussion
Related Documents
