Reputation Systems Talk at Ebay

Social (Reputation) System Design:Opportunities and Challenges

A Practitioners’ View

F Randall FarmerRosenfeld Media

(Coauthor: Bryce Glass)

@eBay, November 14th, 2013

Setting Context:Who am I to speak to eBay?

Definitions

Reputation is information used to make a value judgment about an object or a person – in a specific context.

Reputation about a person (physical or legal) has special features and pitfalls, and rates a special term: Karma

Why Web Reputation Systems?

Two words:

Information Overload

Searching, organizing, filtering, advertizing—all built on top of web reputation systems—is

driving an economy worth $ trillions.

Challenges and Fallacies

But, if web reputation systems are so important, why do so many of them suck?

– “Any content reputation system is better than none.”– “Content quality is a rich-man’s problem.”– “The other guys have thumbs-up/down, we should too.”– “What cool tools do we have on the shelf?”

Time-to-market trumps correctness, testing, and even research. Reputation is not, primarily, a technical problem.

Contexts for Reputation

Individual Context

Community Context

Corporate Context

Industrial Context

Challenges:Reputation is Individual

What does this mean?

What does it actually do?

Like

Personal Meaning & Utility of Ratings

Karma is Hard• Karma is useful for building trust between users, and between a user and the site• Karma can be an incentive for participation and contributions• Karma is contextual and has limited utility globally.• Karma comes in several flavors - Participation, Quality and Robust• Karma is complex (via indirect evaluations), and formulation is often opaque• Personal karma is displayed only to the owner, good for measuring progress• Corporate karma is used by the site to find the very best and very worst users• Public karma is displayed to other users, which makes it the hardest to get right

– It should be used sparingly– It is hard to understand, isn't expected, and easily confused with content

ratings.– It shouldn't be have a socially negative value.– It encourages competition in some users, and may discourage others

The FICO Fallacy: Universal Karma

Credit Score Factors

On-time PaymentsCapacity UsedLength of Credit HistoryTypes of Credit UsedPast Credit Applica-tions

Public Karma is Identity (and Personal)

Scott McCloud, Understanding Comics, pp. 38.

“We built a point system into Consumating because we thought giving direct feedback to people about their conduct on the site would encourage them to be nice to one another—you get a thumbs up when you are nice (treat!), and a thumbs down when you are a douche (electric shock!)”

Ben Brown, Internet Rockstar

Words are Weapons: Public Karma

However…

Dead

Challenges:Reputation is a Community

Communities Have Temperaments

Caring

Collaborative

Cordial

Competitive

Combative

ALTRUISTIC

COMMERCIAL

EGOCENTRIC

The Virtuous Circle

CONTRIBUTORS

THECOMMUNITY

THE SITE

CREATE CONTENT FOR

RANKSIT FOR

REWARDS AND INCENTIVIZES

GRINDING, POINT WHORING

POINTS!

ACTIVITY OVER QUALITY

The Vicious Spiral

POORINCENTIVES

WRONGFOCUS

ILL-BEHAVIOR

FACTIONS DEPARTURES

BAD NORMS

BEST USERS LEAVE

“HOW IS BABBY FORMED!?”

Tragedy of the Commons

Challenges:Reputation is Corporate

The Corporate Virtuous Circle (eBay)

SELLERS

BUYERS

EBAY(THE CORPORATION)

CREATE LISTINGS,SELL ITEMS

BUY,EVALUATESELLERS

REWARDS AND INCENTIVIZES

Yelp: Multi-sided Market…

Yelp: Multi-sided Market with Many Contexts

eBay as Multi-Sided Market:Drop-shippers and Feedback

When is this true? 100 – 5 < 0

What is a Negative Feedback worth?(in U.S. Dollars)

Google’s Virtuous Circle

PUBLISHERS

CONSUMERS(SEARCHERS &

LINKERS)

SEARCH RESULTS(VIA PAGERANK)

CREATE WEB PAGES FOR

PROVIDEINPUTSTO RANK

PROVIDES TRAFFIC BACK TO

ADVERTISING• SALES• ADS• ATTENTION• WHATEVER…

Challenges:Reputation is an Industry

Challenge: Systems that model real life

Honestly.com? Not so much.

Dead

Fallacy: Karma as influence & currency

Dead

Our Ultimate Challenge:Short Institutional Memory

From Fuzzy to Focused

A++++++ audience, would visit again…

Randy Farmer@frandallfarmer

[email protected] http://about.me/frandallfarmer