Analyzing Cross-System User Modeling on the Social Web

DelftUniversity ofTechnology

Analyzing Cross-System User Modeling on the Social WebICWE, Cyprus, June 22, 2011

Fabian Abel, Samur Araujo, Qi Gao, Geert-Jan HoubenWeb Information Systems, TU Delft

2Analyzing Cross-System User Modeling on the Social Web

PersonalizedRecommendations

Personalized Search Adaptive Systems

What we do: Science and Engineering for the Personal Web

Social Web

Analysis and User Modeling

user/usage data

Semantic Enrichment, Linkage and Alignment

domains: news social media cultural heritage public data e-learning


Pitfalls of User-adaptive Systems

System A

time

Hi, I’m your new user. Give me

personalization!

Hi, I have a new-user problem!

profile ?

profile

Hi, I don’t know that your

interests changed!

Hi, I’m back andI have new interests.

System Cprofile

System Dprofile

System Bprofil

e

How can we tackle these problems?


User data on the Social Web

Cross-system user modeling on the Social Web

5Challenge the future

Google Profile URI http://google.com/profile/XY

4. enrich data withsemantics

WordNet®

Semantic Enhancement

Profile Alignment

3. Map profiles totarget user model

FOAF vCard

Blog posts:

Bookmarks:

Other media:

Social networking profiles:

2. aggregate public profile

data

Social Web Aggregator

1. get other accounts of user

SocialGraph API

Account Mapping

Aggregated, enriched profile(e.g., in RDF or vCard)

Analysis and user modeling

5. generate user profiles

Interweaving public user data with Mypes


In this paper: User Modeling across Twitter, Flickr and Delicious

Twitter and Delicious• 1500 users• 80k + 620k TAS

Flickr and Delicious• 1467 users• 890k + 680k TAS

This is #interesting: http://bit.ly/3gt42f #web

http://claimid.com

websocialmediaidentity

travel, google IO

Twitter Delicious Flickr

Bob


Tag-based user profilesTag-based profile of a user u = set of weighted tags:

€

P(u) = {(t,w(u, t)) | t ∈T}

weight indicates to what degreethe user is interested in t

tag of interest

Lightweight weighting scheme:count how often the user applied the tag


Characteristics of tag-based profiles


Characteristics of tag-based profiles

1. What are the characteristics of the individual tag-based profiles in Twitter, Flickr and Delicious?

2. How do the tag-based profiles of individual users overlap between the different systems?


Size of tag-based profiles

Delicious

Flickr

Twitter


Overlap of tag-based profiles

Overlap of tag-based profile is less than 10% for more

than 90% of the users


Entropy of Tag-based profilesDeliciousFlickrTwitter

Flickr & DeliciousTwitter & Delicious

Aggregated profiles reveal wrt entropy significantly more

information than the service specific profiles.

where: - p(t) = probability that t occurs in Tu - Tu = tags in user profile P(u)

€

Entropy(Tu) = p(t)∗(−log2(p(t)))t∈Tu

∑


Observations• Profile size varies from system to system (e.g. tag-

based Twitter profiles are rather sparse)• Tag-based profiles of an individual user overlap only little (e.g. overlap is less than 10% for more than 90% of the users)

• Entropy of tag-based profiles:Twitter < Flickr < Delicious < aggregated profiles


Cross-System User Modeling for Cold-start recommendations


Evaluation: Recommending tags / bookmarks

How does cross-system user modeling impact the recommendation quality (in cold-start situations)?

Hi, I’m your new user. Give me

personalization!

profile ?

delicious

Cosin

e-ba

sed

reco

mm

ende

r

tags to exploreWeb sites to

bookmark

profile

profile

Cros

s-sy

stem

user

mod

elin

g

leave-n-out evaluation

user’s tags and bookmarks

Ground truth:

actual tags and bookmarks of the user


User Modeling Building Blocks

Profile?1. Source

System A System B

?

tags weightst1t2t3

analyze

enrich t4t5

0.10.10.50.20.1

weight

2. Semantic

Enrichment

3. Weighting Scheme

1. Which tags should be contained in the profile?2. Further enrich/align tags?3. How to weight the tags?


User Modeling Building Blocks (in this talk)

1. Source:a) Personal tags from foreign systemb) Popular tags from target system

2. Semantic Enrichment:a) Enrich tags with similar tags (based on Jaro-Winkler similarity)b) Cross-system rules: if tag A was used in foreign system then

add tag B3. Weighting scheme:

a) Personal usage frequency in foreign systemb) Global usage frquency in target system

profile profile ?

Foreign: Target:

requires profile to compute recommendations

web

blog jav

a

a) simJaro(blog, blogs) is high

blogs

b) Cross-system rule: blogforeign nikontarget

france

personalpopular

similaritycross rules

personalglobal


Cross-System User Modeling for Cold-start recommendations

1. Which user modeling strategies performs best in which context?

2. How do the different building blocks of the user modeling strategies (e.g. source of user data) influence the quality of the tag-based profiles?


Tag recommendations: Twitter / Delicious

As you can easily see…:-)


Tag recommendations: Twitter Delicious

profile profile ?

Improvement regarding P@10, but “global Delicious trend”

performs better regarding MRR & S@1.

popularglobal global

personal

similarityglobal

personal

baseline

personalpersonal

Cross-system user modeling

Cross-system strategies lead to significant improvement

(impact of semantic

enrichment is rather low)Significant improvements

regarding all metrics!

profile

user’s tags global

tag frequencies (weights)profile

user profile


Tag recommendations: Delicious Twitter

profile profile ?

Significant improvements regarding all metrics!

user’s tagsand tag frequencies (weights)

user profile

profile

profile

Tag-based profile

information from Delicious seems to be

more valuable than hashtga-based Twitter

profiles

Semantic enrichment (cross-system rules) allow

for significant improvement

regarding P@10

popularglobal

baseline

personalpersonal

Cross-system user modelingglobal

personal

cross rules

globalpersonal


Tag Recommendations: different settings

Cross-system user modeling is also beneficial for cold-start tag recommendations in Flickr.

profile ?

target:profile

Cross-system user modeling allows for cold-start tag recommendations in Delicious: Twitter profiles are more appropriate than Flickr profiles.

profile

profile ?profile

target:

1. Cross-system user modeling has significant impact on the recommendation performance

2. To optimize the performance one adapt to the given application setting


Bookmark Recommendations

baseline Cross UM Cross UM

1. Cross-system user modeling achieves also significant improvements for cold-start bookmark recommendations

2. Twitter is again a more appropriate source than Flickr


Conclusions1. Characteristics of distributed tag-based profiles:

• Overlap of tag-based profiles, which an individual user creates at different services, is low

• Aggregated profiles reveal significantly more information (regarding entropy) than service-specific profiles

2. Performance of cross-system user modeling for cold-start recommendations:• Cross-system UM leads to tremendous (and significant)

improvements of the tag and bookmark recommendation quality

• To optimize the performance one has to adapt the cross-system strategies to the concrete application settinghttp://persweb.org


Thank you!

Fabian Abel, Qi Gao, Geert-Jan Houben, Ke Tao

Twitter: @perswebhttp://persweb.org

Datasets: http://wis.ewi.tudelft.nl/icwe2011/um/