Global Budgets for Local Recommendations

• 1. Thomas Sandholm, Hang Ung, Christina Aperjis, Bernardo Huberman
  Hewlett-Packard, HP Labs,Social Computing Lab
  RecSys, Barcelona September 27, 2010
  Global budgets for local recommendations
  

2. Why Vote?
How do we get more people to contribute their opinions?
3. 4. 500
30
6
Users
EXPERIMENTS
LOCATIONS
Paris
Chicago
Athens
Palo Alto
Mumbai
Bangalore
5. Many people consume content FEW LEAVE OPINIONS
May 2008
300M+
April 2010
8M
April 2010
27M
Sample taken
Users
6. RATING Budgets and RANKING
Restaurants
Restaurants
$3
Pizza
Bob:$10
$4
Pizza
http://roundtable.com
http://dominos.com
Alice: $11
Restaurants
$6
Burgers
http://burgerking.com
Restaurants
$8
Diners
http://applebees.com
Top Channels
Friend Channels
$8 Applebees
$6 Burger King
$4 Dominos
$3RoundTable
$8 Diners
$7 Pizza
$6 Burgers
$6 Burgers
$4 Pizza
John
7. RATING REWARDS
Alice
$5
http://dominos.com
Bob
$4
http://dominos.com
A
$3
http://roundtable.com
B
$3
http://roundtable.com
http://roundtable.com
$3
John
Top Rewards
(reward factor 10)$40Bob Dominos @ A
$30AliceRoundTable @ B
8. CLICK TO RATING Ratio
July 2010
4K
May 2008
300M+
April 2010
8M
April 2010
27M
Sample taken
Users
9. RECOMMENDATION SUCCESS
Success = proportion of query sessions ending with clicks/ratings
10. System Coverage
11. Mechanical Turk Experiments
Setup-5 Surveys
-10 URLs
-6 Locations
-3 Continents
-500 Users
Surveys
-1-5 Star
-Budget
-Star Bonus
-Budget Bonus
-Gloe
12. Mechanism Results
Kendall Tau Rank Correlation (higher better)
RMSE (lower better)
13. Bonus effect on participation
Probability of signals
Number of surveys taken
14. Lessons LEARNED
Amazon Mechanical Turk
Workers not random geographic sample
Sensitive to task complexity
Respond well to small incentives
Budget Mechanism
Higher quality recommendations with incentives
Social/Economic/Status valueextract more opinions
Tuned based on usage, e.g. reward factor
15. FUTURE WORK
Projects
AfricaMap: map annotation in remote parts for disaster relief UNOSAT/Uni. Geneva
Mobile print provider recommendations via HP ePrint
Research
GSP Auction for commercial bidding
LMSR Market for recommendation arbitrage
Enhance reward mechanism to both encourage and identify high quality contributions
16. Papers at www.hpl.hp.com/research/scl
Live system at www.hpgloe.com
Social Computing Lab
THANK YOU
17. Backup
18. System Architecture
19. Scalability
20. HP Gloe: STATUS
~4k* users on Android, iPhone, BlackBerry, WebOS, Web
~7m* recommendations at http://hpgloe.com
*Sept 2010
21. Lessons LEARNED CONTINUED
Gloe system
Geohash location partitioning simple and efficient
HTTP(S) GET/JSON(P) has served us well on all platforms
MySQL & Sharded architecture flexible and fast