Introduction System Recommendation Algorithms Experiments Conclusions SuggestBot: Using Intelligent Task Routing to Help People Find Work in Wikipedia Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl presented by: Gianluca Demartini July 6, 2007 Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
26
Embed
SuggestBot: Using Intelligent Task Routing to Help People Find ...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
SuggestBot: Using Intelligent Task Routing toHelp People Find Work in Wikipedia
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl
presented by: Gianluca Demartini
July 6, 2007
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Outline
1 Introduction
2 System
3 Recommendation Algorithms
4 Experiments
5 Conclusions
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Scenario
Wikipedia lives thank to its contributorsIt is made of community-maintained artefacts of lasting value(CALV)Similar communities are iMDB, slashdot.org
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Wikipedia
A full dump (with history of pages) was 700GB in 2006Reasons for participating are similar to open source: learning,status, belongingBots: automated or semi-automated editing of pagesPeople tag articles they think need work
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Motivation
Member-maintained communities need contributionsReducing the cost of contribution increase motivationGoal: make it easy to find work to do
interestingthat need work
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Intelligent task routing definition
Intelligent task routingreduces the cost of finding workmatches people with tasks they are likely to care about
as a mechanism for increasing contribution
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Intelligent task routing on Wikipedia
With Intelligent task routing usinghistory of editstext matchinglink followingcollaborative filtering
people edit four time more often.
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Outline
1 Introduction
2 System
3 Recommendation Algorithms
4 Experiments
5 Conclusions
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
SuggestBot
SuggestBot provides recommendations only on requestSuggestBot edits the user talk pages addingrecommendations
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Architecture
Four steps:Pre-processing WikipediaModelling user’s interestsFinding candidate articlesMake recommendations
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Figure: SuggestBot architecture
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Pre-processing
SuggestBot processes the wikipedia looking for users’ annotation.SuggestBot considers 6 types of work
Stub: short article needs more infoCleanup: need rewritingMerge: articles need to be combinedSource: need citationWikify: the text is not in the correct styleExpand: long article needs more info
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Modelling interests
The User’s Interests profileis build implicitly (no user participation)is the set of article titles that have been edited by the userdoes not contain more than 500 articlesdoes not consider “vandalism reversion” editsconsiders multiple edits as single edit
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Outline
1 Introduction
2 System
3 Recommendation Algorithms
4 Experiments
5 Conclusions
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Finding candidate articles
SuggestBot does not recommend already edited articlesSuggestBot finds related articles based on
similarity of text: user’s profile as query against full textconnections through linksconnections through co-editing
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Connections through links
SuggestBot uses links created by the users (citation network)SuggestBot ignores date-related linksALGO:
Initialize items in the profile to have a score of 1
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Remove items from original profile
Penalization function:
score :=score
log( #art|BestL−L|)
(1)
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Connections through co-editing
Find people whose history is similar usingJaccard metric for similarity between profilesGive credit to items based on the users’ similarity
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Connections through co-editing
Remove items edited by few others, or edited by the user t
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Filtering results
Both algorithms tend to recommend popular or controversialarticlesSuggestBot drops the top 1% of the most edited articles
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Outline
1 Introduction
2 System
3 Recommendation Algorithms
4 Experiments
5 Conclusions
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Setup
In 6 months 1200 people got recommendationsComparison based on number of edited recommendationsThree different experiments:
compared to random suggestions: 4 times more editedcomparing text-similarity, links, co-edit.Similar performances with differences:
text reco: focuses on rare wordlinks:biased by categories (link circles)co-edit: often edited articlesUsing meta-search techniques to combine results would help
removing noise: not considering minor edits does not improve
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Outline
1 Introduction
2 System
3 Recommendation Algorithms
4 Experiments
5 Conclusions
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
Conclusions
Simple algorithms gave strong resultsIntelligent task routing
can dramatically increase members’ contributionsis most useful where the tasks are numerous andheterogeneous
Future steps:incorporate meta-search techniquesremove noisegive to the user the possibility to edit the profile with low cost
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing
IntroductionSystem
Recommendation AlgorithmsExperimentsConclusions
The End
Q&A
Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing