Analysis, design and implementation of a Multi- Criteria Recommender System based on Aspect Extraction and Sentiment Analysis techniques Instructors: Student: Prof. Giovanni Semeraro Davide GIANNICO Dott. Marco de Gemmis Department of Computer Science Master of Computer Science (Msc)
25
Embed
Analysis, design and implementation of a Multi-Criteria Recommender System based on Aspect Extraction and Sentiment Analysis techniques
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Analysis, design and implementation of a Multi-
Criteria Recommender System based on Aspect
Extraction and Sentiment Analysis techniques
Instructors: Student:
Prof. Giovanni Semeraro Davide GIANNICO
Dott. Marco de Gemmis
Department of Computer ScienceMaster of Computer Science (Msc)
Outline
2
Intro
Multi-Criteria RecSys
Proposed approach
Experimentation
Conclusion
Future work
3
Need: necessity of the user to be supported during
complex decisional processes
Tendency: development of on-line platforms which provide
recommendations to the user and where the community
expresses the own opinion for a item class
Some stats 315M unique views per month & 200M reviews(TripAdvisor, 2014);
168M unique views per month & 35M reviews (Amazon, 2013);
139M unique views per month & 67M reviews (Yelp, 2014).
Scenery
Opportunity
4
Take advantage of the reviews informative power
incorporating such information in the recommendation
process.
Concept: make «value» from raw data.
Recommender Systems
5
Recommender Systems (RecSys) are decision support and
information filtering tools
The main goal is helping users which access to a data
source for discovering information or items that could be
interesting for them
Recently, RecSys area has focused on Multi-Criteria
RecSys[AMK11]
[AMK11] G. Adomavicius, N. Manouselis, Y. Kwon. ‘Multi-Criteria Recommender Systems’. In:
Recommender Systems Handbook, pp 769-803, Springer US, 2011
Multi-Criteria Recommender Systems
6
Techniques which provide recommendations to the user,
modeling the utility concept espressed by the user for an
Q1) #multi-ORE-criteria algorithm overtakes the single-criteria(#single-criteria) and multi-criteria techniques (#multi-static-criteria), which use default criteria, results ?
Q2) Multi-criteria Aspect-based Recommender system based on sentimenT Analysis(#MARTA) algorithm overtakes the state-of-the-art approaches results?
10-Fold Cross Validation. For each run the training set isused for calculating the similarity matrix
Evaluation of the approaches on the respective test set
Evaluation metrics: MAE, RMSE, Precision, Recall and F-Measure
Statistical validation using the Wilcoxon test
Dataset
18
Amazon
TripAdvisor
Yelp
#ratings #users #items #ratings/
user
#ratings/
item
sparsity
355.949 2.850 2.820 124,73 126,06 0,96
#ratings #users #items #ratings/
user
#ratings/
item
sparsity
229.905 45.981 11.537 5 19,9 0,99
#ratings #users #items #ratings/
user
#ratings/
item
sparsity
208.135 1.386 1.580 150,16 131,73 0,90
Q1 Results (#multi-ORE-criteria vs #single-
criteria and #multi-static-criteria)
19
Dataset TripAdvisor
MAE
SVD 0,87
#single-criteria 1,32
#multi-static-criteria 1,10
#Multi-ORE-criteria 0,96
0,87
1,32
1,10
0,96
0,00
0,20
0,40
0,60
0,80
1,00
1,20
1,40
MA
E
F-Measure
SVD 0,59
#single-criteria 0,52
#multi-static-criteria 0,53
#multi-ORE-criteria 0,71
0,59
0,52 0,53
0,71
0,00
0,10
0,20
0,30
0,40
0,50
0,60
0,70
0,80
F-M
easu
re
Q2 Results (#MARTA)
20
DatasetYelp (sparsity: 0,99)
MAE
SVD 0,91
#single-criteria 1,18
#multi-ORE-criteria 1,13
#MARTA 0,85
0,91
1,181,13
0,85
0,00
0,20
0,40
0,60
0,80
1,00
1,20
1,40
MA
E
F-Measure
SVD 0,58
#single-criteria 0,49
#multi-ORE-criteria 0,55
#MARTA 0,63
0,58
0,49
0,55
0,63
0,00
0,10
0,20
0,30
0,40
0,50
0,60
0,70
F-M
easu
re
Q2 Results (#MARTA)
21
Dataset TripAdvisor (sparsity: 0,96)
MAE
SVD 0,87
#single-criteria 1,32
#multi-ORE-criteria 0,96
#MARTA 0,87
0,87
1,32
0,960,87
0,00
0,20
0,40
0,60
0,80
1,00
1,20
1,40
MA
E
F-Measure
SVD 0,59
#single-criteria 0,51
#multi-ORE-criteria 0,71
#MARTA 0,58
0,59
0,51
0,71
0,58
0,00
0,10
0,20
0,30
0,40
0,50
0,60
0,70
0,80
F-M
easu
re
Q2 Results (#MARTA)
22
Dataset Amazon (sparsity: 0,90)
MAE
SVD 0,71
#single-criteria 0,65
#multi-ORE-criteria 0,55
#MARTA 0,71
0,710,65
0,55
0,71
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
MA
E
F-Measure
#SVD 0,62
#single-criteria 0,65
#multi-ORE-criteria 0,74
#MARTA 0,63
0,62
0,65
0,74
0,63
0,54
0,56
0,58
0,60
0,62
0,64
0,66
0,68
0,70
0,72
0,74
0,76
F-M
easu
re
Analysis of the results
23
Q1) #multi-ORE-criteria algorithm overtakes the single-
criteria(#single-criteria) and multi-criteria techniques (#multi-
static-criteria), which use default criteria, results ?
#multi-ORE-criteria showed better results than #multi-
static-criteria and #single-criteria (MAE and F1)
Q2) Multi-criteria Aspect-based Recommender system based on
sentimenT Analysis(#MARTA) algorithm overtakes the state-of-
the-art approaches results?
#MARTA showed good results especially on sparse
dataset
Conclusions
24
Indipendence from the domain
Influenced by the review quality
Good performance especially on the item classification
(relevant or not)
Future work
25
Considering the extracted opinions relevance score
Testing more advanced Aspect Extraction and Sentiment
Analysis techniques
Learning weighing schemes for each user and considering