Elizeu Santos-Neto, Flavio Figueiredo Jussara Almeida, Miranda Mowbray Marcos Gonçalves, Matei Ripeanu The 2 nd IEEE SocialCom/SIN -- August 2010.

Elizeu Santos-Neto, Flavio FigueiredoJussara Almeida, Miranda MowbrayMarcos Gonçalves, Matei Ripeanu

Assessing the Value of Contributions in Tagging Systems

The 2nd IEEE SocialCom/SIN -- August 2010

Decentralized, Collaborative & Non-proprietary [1]

2[1] Y. Benkler. “The Wealth of Networks”. Yale University Press (2006).

Commons-based Peer Production Systems

Offline Online

Car poolingInformation

SharingResource Sharing

Volunteer Firefighters

BitTorrent

SETI@Home

OurGrid.org

Wikis

Tagging

Q&A Portals

Online Peer Production Systems

3

Tagging Systems

BandwidthTimeExpertise CPU

Annotation = Tags + Items

Tags are free-form words

Items can be virtually anything URLs, photos, videos,

citation records, etc…Photos Annotations

Why is it important to study tagging systems?

Increasingly popularMillions of users [2]GBytes of content and annotations daily [2]

User-generated metadata = new opportunities To improve existing systems (e.g., social search)To create new mechanisms (e.g., reputation

systems)

Open problem: how to quantify the value of user contributions in these systems?

4[2] R. Ramakrishnan, A. Tomkins. “Toward a PeopleWeb”. IEEE Computer 40(8): 63-72 (2007)

5

To define a method that quantifies the value of users’ contribution in tagging systems that is

accurate, feasible and robust.

What is the tolerance to

malicious users?

What is the computational complexity?

How close is it to the true value of user contribution?

Long Term Goal

Problem formalization

A solution framework

A method to quantify the value of tagsEntropy-based metricAlgorithms to compute the metric

6

Where Are We? (Contributions to Date)

Solution FrameworkIntroduction& Motivation

Solution Framework

Value of Tags Evaluation

Conclusions &

Future Work

Value of User Contributions inTagging Systems

Contribution = Tags + Items

Value of tags Context: navigation/search Intuition: value ≈ improvement on navigation/search

Value of an itemContext: item usage Intuition: value ≈ usefulness to a user

8

Content Annotations

9

Info Seeker

Relevant Item Set Finder

Contribution Value

Item Usage Monitor

Tag Value Calculator

Info Producer

Item Value Calculator

Tag Value Aggregator

Item Value Aggregator

Information Needs

Past activity

Tags

Items

Items

UsageValues

Values

The Value of User ContributionInformation flow

Value of TagsIntroduction& Motivation

Solution Framework

Value of Tags Evaluation

Conclusions &

Future Work

Intuition: tags are valuable if they narrow the scope of navigation, while retrieving relevant items.

11

Value of Tags

Items in the system

Relevant items to an info seeker

Items retrieved by a set of tags

Value of tags is proportional to this

intersection

Tags published by an info producer

Evaluation CriteriaIntroduct

ion&

Motivation

Solution Framew

ork

Value of Tags

Evaluation

Conclusions & Future Work

Feasibility: is the method efficient?

Accuracy: is the estimation close to the real value?

Robustness: can users boost their contributions maliciously?

13

Evaluation Criteria

14

Preliminary ResultsFeasibility – Part 1

Only 4% of users have more than 100 unique tag

assignments.

15

Preliminary ResultsFeasibility – Part 2

80% of users have NOT produced tags/items in 1

moth or more.

Conclusions & Future Work

Introduction& Motivation

Solution Framework

Value of Tags Evaluation

Conclusions &

Future Work

ConclusionsAssessing the value of user contributions in

social tagging systems is a relevant and challenging problem

This work ……proposes a solution framework…provides preliminary results on feasibility

Current efforts…Evaluating techniques that estimate relevant itemsDesigning algorithms to calculate and aggregate

value17

Future WorkAlgorithms

Exploit user activity similarity Aggregation method that exploits implicit social relations

Evaluation Accuracy – does the estimated value match a ground

truth? Robustness – what about spammers?

Value of items

Build mechanisms that harness the value of contributions Spam detection Social search

18

[email protected]

http://www.ece.ubc.ca/~elizeus

Comments? Questions?

Assessing the Value of Contributions in Tagging Systems