A cognitive theory of reputation - unibo.itpaolucci/files/ReputationSPO_Part2.pdf · (Paolucci, 2000). • Norm-abiders exchanging reputation of violators Prudence rule • Cooperation

A cognitive theory of reputation

2

Some simplifications, again

● Is good reputation a good predictor of cooperation?● Why noise = bad reputation?● Not only prevent meta-level cheating

– To what extent is cheating harmful?– To what extent Online Reputation Systems differ from

reputation in natural societies, and about what?– What is the effect of meta-level cheating in natural

societies? Is it prevented and how?– Are there just two parts (buyer and seller) in the general

game of reputation?

3

Image vs. Reputation

• Both social evaluations concern other agents' (targets) attitudes toward socially desirable behavior, and may be shared by a multitude of agents.

• Image is an evaluative belief– it tells that the target is “good” when it displays behavior b, and that it is

“bad” in the opposite case.– Most software implementation describe image only

• Reputation is instead a shared voice, i.e. a belief about others saying that a given target enjoys or suffers from a shared image.– Reputation is true when it’s actually spread, not when it is accurate

4

Image & Reputation

●image :

global evaluation owned by an agent about a target's property in relation with

an agent's (or others') goal , e.g. a social norm (Miceli & Castelfranchi 2000)

●reputation :

● process and effect of the transmission of (a sub-set of) originally image-

based evalutation(s) (or forged ones)

● meta-belief, i.e. a belief about others' beliefs (either personal:”agent x

says...”, or impersonal “it is said that...”)

● emergent from singol agents' communication (from micro to meso level)

● immergent as an individual property (from meso to micro level)

5

• mental decisions based upon them can be analyzed:– epistemic– pragmatic-strategic– memetic

• Reputation does not bind the speaker to commit himself to the truth value of the evaluation conveyed but only to the existence of rumours about it

• only the acceptance of a meta-belief is required

• And, unlike ordinary deception, reputation implies– no personal commitment (by no means stating that t deserved it)– no responsibility

Image vs. Reputation

6

• Reputation co-evolved with human language and social organization as a multi-purpose social and cognitive artefact (Dunbar, 1996)– incentives cooperation and norm abiding and discourages defection and

free-riding

– allows retaliation on transgressors by cooperating at the level of information exchange

Reputation and uncertainty

7

Necessity of a cognitive approach

• What cannot be represented outside of this approach?– power and responsibility as the main factor for spreading

– correct meta-reasoning

• what will happen when I answer this question sincerely/insincerely?

– Contagion

8

Social Cognitive Phenomenon(Reputation)

Cognitive Object(Image)

Social Process(Gossip)

A Social Cognitive Model of Reputation

ElementsImage• Belief

– Target evaluation– Accepted: the evaluation is believed

true• Agent Roles:

– Evaluator (E)– Target (T)– Beneficiary (B)

• Decisions– Epistemic: (not) to accept– Strategic: how to act w.r. to target.

• Levels involved– Agent cognition

Reputation• Meta-Belief

– How others evaluate the target– Evaluation is not necessarily

accepted! • Agent Roles:

– Gossiper (G)• Decisions

– Memetic: (not) to communicate• No commitment• No responsibility.

Diffusion without condivision!

• Levels involved– Population– Agent

• Cognitive• Objective

10

Sets of Agents

4 non-empty sets of agents are defined by their functional role:

set E = evaluators

set B = beneficiaries (of the social norm)

set T = target

set M = memetic agents, gossipers

image reputation

●Usually

✔ Set E Ç Set B ≠ 0

✔ Set E < set M●.

11

Factors of Gossip

● Whether and Why– Reputation of the source

– Acceptance of evaluation

– Responsibility and accountability for effects

– Benevolence toward the beneficiary as opposed to benevolence toward the target.

• To Whom

• About whom

• How

12

Whether and Why.Responsibility

Gossip decreases with growing responsibility (perception) and accountability:● Increasing perception of effects (Willcox, 2001):

– No “chain-like” structure

– No opacity of reports to targets

– Low “distance” of targets (Barkow, 1992; Boehm, 1999)

• Increasing severity of effects:

• Increasing commitment (firsthand evaluations: G = E) (anthropological evidence)

• Increasing publicity (vs. Anonimity) of reports (Willcox, 2001)

• Small number of gossipers (Latané & Darley, 1970)

13

Whether and Why.Benevolence

● Beneficiary oriented: negativity bias (Skowronsky and Carlston, 1989)– Transmit– Transmit bad even if uncertain

● Target oriented: leniency bias– Transmit if good reputation– Do not transmit if bad.

14

To Whom

• Reciprocity: likelihood to return information

• Discretion about the source: likelihood to protect identity of source and avoid retaliation

• Usefulness: likelihood to provide relevant information

• Persuasibility: likelihood to accept reputation information. – with regard to the source,

– with regard to the information, etc.

• Co-interestedness in sharing information, etc.

15

About Whom

• Salience:– Diversity (Elias, 1985)

• Distance (Barkow, 1992):– Social hierarchy: elite aversion (Bohem, 1999; Karstedt, 1997)

– Cannot retaliate● Bounded perception: long chains, or wide networks.● Limited power

• Credibility of evaluation.

16

How

• Chain-like Vs one-to-many

• Anonymity of gossipers

• Second-hand Vs own evaluation

• Opacity Vs Accessibility of reports by targets

• Choice of recipients Vs public feedback

Operationalise Social Cognitive Factors...

● G E: – Commitment (no secondhand

report) -> responsibility● G B (and E B):

– Beneficiary-oriented benevolence -> negativity bias

● G T (and E T): – Fear of retaliation. ->

responsibility– Target-oriented benevolence ->

leniency bias

● T B:

– No opacity -> responsibility

G E G B G T B T

H Underprovision

ProvisionUnderrat.

Underprovision

OrOverrat.

Underprovision

OrOverrat.

L Provision Underprovision

Provision(Underrating)

Provision(Underrating)

18

Combined Intersections:Two (Extreme) Examples

E G G BE B

G TE T

B T

Students&

Professors L H L L

EBay H H H H

ProvisionUnderrating

Underprovisionoverrating

What is the effect of • Overrating • Underrating?

19

• Positive effect of underrating (Paolucci, 2000).

• Norm-abiders exchanging reputation of violators

Prudence rule• Cooperation is not necessarily a function of good reputation

Gossip > no gossip (unless overrating)

Suggestion from Simulation

Prudence Rule: Negativity Bias in Social Evaluation

• Prudence: common sense or consolidated knowledge?

• In impression formation (Skowronsky and Carlston, 1989), evidence of negativity bias

• Confirming experimental evidence in social evaluation (Ciacci et al., 2002):

● Pairs of stories about one character● x does bad action a &● x does good action b

● and their dual● x does good action a &● x does bad action b

• Results: negative evaluations prevail.

NP

What About Transmission?

● Polarised explanations of same event (lecturer missing) injected to two subgroups of agents:

bad good

G EL

G BH

G TL

B TL

Back To Examples

Courtesy alg. (Dellarocas, 2001)• Pass on good reputation if you can• Otherwise don’t pass

Prudence alg. (Conte and Paolucci, 2002):• Pass on bad reputation, even if uncertain• Don’t pass good one unless certain

ProvisionUnderrating

Underprovisionoverrating

E G G BE B

G BE T

B T

Students&

Professors L H L L

EBay H H H H

23

To sum up

● Different levels of description can be useful in different contexts

● Simulation can contrast the tendency to excessive simplification– GT’s “for simplicity”

● But– Data analysis, targeted simulation experiments, and field

experiments are still in the doing: the theory is waiting for falsification

24

A review of existing system

● We start now with a review of existing online systems, also called Internet User-driven Reputation Applications (IURAs) from Marmo, 2006

● Systems will be analyzed under the light of the theory presented

25

IURAs: contexts of use

Recommender systems: signaling quality about experts / professionists or brands / products

Electronic Auctions: partner selection, quality of exchanges, balancing info. asymmetry

Social Networks: social connection, status, recognizability

Discussion Forums: signaling quality of post, on topic authors, relevance

26

Mechanism taxonomy (1 of 5)

Factual

mere labels assigned top-down from system to target

examples: top reviewer, power seller, premium account

they're not user-driven reputation mechanisms

Amazon:

eBay PayPal Guru

27

Mechanism taxonomy (2 of 5)

●Rating

●evaluation as choice of user out of an options list (vote-like)

● binary versus non binary values

● Numeric (bounded, real) versus verbal values

● global versus multi-attribute parameters

output:

● Algebric sum

● Arithmetic average (weighted versus

plain)

● Percentage of ratings

● Total number of ratings

● List of all ratings

28

Mechanism taxonomy (3 of 5)

Comment

verbal or alphanumeric drawing comment. Used with voting mechanisms,

or with connection lists in social networks. Used to personalize

evaluations.(Friendster)

FLipFLop | 28/9/2006

===THANKS FOR ADD US ====

+YOUR THE MEMBER OF OUR+

++++=PRODUCTION=+++++

=WAIT FOR OUR NEXT EVENT=

+++===CERTIFIED===+++

>>>BIG RESPECT TO YOU

(Yahoo)

29

Mechanism taxonomy (4 of 5)

Connection list

users agree to include each other in their connection list. This may state

“friendship”, “partnership”, general support or acquaintance between

users. Inclusion may as well be one-sided, or even of a negative nature

(list of enemies)Isabel’s Connections (3)

BeateHaertl

Researcher

SamueleMarmo

Ph.d. student at Università di Siena

NoaRiedel

Research Associate at Jack Russell Consulting

(LinkedIn)

30

Mechanism taxonomy (5 of 5)

Messages

- generally available function of messaging between users

- they are within-domain electronic mail

- may be used in many ways, including circulating reputation.

messages have generally a completely private status (hidden

narrowcasting)

31

Mechanism functionality (1 of 3)

●Directionality●

✔ Unilateral : user releases unilateral evaluation of a target.

Expected effect: prudence rule

✔ Bilateral : users release reciprocal evaluation.

Expected effect: courtesy rule

directionality includes both synchronous and asynchronous evaluations

32

Mechanism functionality (2 of 3)

●Access●

✔ Broadcasting : user can not choose the recipient(s) of evaluation, all

evaluations are public domain.

Expected effect: underprovision

✔ Narrowcasting : user addresses evaluation, which is privately

accessed.

Expected effect: provision

33

Mechanism functionality (3 of 3)

●Dynamism✔ Single evaluation : only last evaluation from user on a specific target

is counted.

Expected effect: dynamic change of reputation

✔ Multiple evaluation : users' evaluations on a specific target are

cumulated.

Expected effect: inertial reputation

✔ Time discount of evaluations

Expected effect: dynamic change of reputation

✔ No time discount

Expected effect: inertial reputation

34

a theoretical categorization

●A theoretical categorization of IURAs can be based on

● Access (broad- versus narrowcasting)

● Transmission of personal evaluation (image) versus

personal + meta-evaluation (image and reputation)

There are four possibilities:

(a) centralized image public access, personal evaluation only

(b) distributed reputation similar to natural reputation, private access, personal

+ meta-evaluation

(c) centralized reputation public access, personal + meta-evaluation

(d) distributed image private access, personal evaluation only

35

HypothesisPostulate: roles of agents are exactly defined by implementation

Main hypothesis

E subset of B ; E ≠ T no courtesy (prudence?)

E = T = B courtesy rule

Context hypothesis

(after observation of typical implementations)

Auctions: courtesy rule, centralized image (bidirectional, M = E, broadcasting)

Social Networks: courtesy rule, distributed reputation (bidirectional, M >

E,narrowcasting)

Discussion Forums: centralized image (unilateral/bidirectional, M = E,

broadcasting)

Recommender systems: no courtesy, centralized image (unilateral, M = E,

broadcasting)

36

Method

Use of systems under analysis during year 2005

Compiling a detailed description of the system

Collecting users' evaluation sample

Methods of collecting: alphabetical user profile search, temporal, random

EXTREME SCENARIO

Under a general assumption of inaccurate reputation, we considered a

discretionary threshold of 10% for negative evaluations: we would think of a

courtesy rule situation when negatives are below this value.

We considered a broad concept of negative evaluation, which included

“neutrals”

37

Sample Details

Ebay: 100 profiles, 7379 evaluations, alphabetic

Guru: 2175 profiles (of which 70% empty), random

Amazon Auctions: 50 profiles, 21539 evaluations, alphabetic

Friendster: 100 profiles, alphabetic, random

LinkedIn: 50 profiles, random

Omidyar: 50 profiles, 59625 evaluations, alphabetic, random

Slashdot: 50 moderators, 101 evaluations, temporal

Epinions: 15 profiles, 949 evaluations, random (+ survey from P.

Massa, B. Bhattacharjee with 664824 evaluations)

RateMyTeachers: 50 profiles, 479 evaluations, alphabetic

38

Results

Omidyar-net

Amazon A.

Guru

eBay

0.00% 25.00% 50.00% 75.00% 100.00%

courtesy rule E = T = B

negatives

positives

RateMyTeachers

Epinions

Slashdot

0.00% 25.00% 50.00% 75.00% 100.00%

no courtesy (prudence) E subset of B ; E ≠ T ; T ≠ B

negatives

positives

39

Auctions

40

Discussion Forums

41

Recommender Systems

42

Social Networks

43

Synoptic table

44

IURAs & natural reputation

●Natural reputation:

● both word of mouth and personal evaluation

● provision, low commitment (mostly narrowcasting)

● negative anchorage (Ciacci 2002)

●IURAs:

● only direct evaluation (image)

● underprovision, high commitment (mostly broadcasting)

● emphasis on positive ratings, risk of system ineffectivess

45

Conclusions on the review

a. Social cognitive theory refinement of reputation concept explains

IURAs dynamics (e.g. the courtesy rule application, or underprovision)

b. Reputation tendencies are consequences of (implementation of) role

overlapping and of commitment factors (anonymity, opacity)

c. Actual IURAs are mainly centralized image ones, which limits

artefact's effectiveness

d. Actual IURAs are generic, based on a naïve concept of reputation

(bidirectional, broadcasting, image only)

46

Appendix: Examined DomainsElectronic Auctions

Qxl, Yahoo Auctions, AaandS, AuctionAddict, BuySellBid, iBidFree, Elance, Universal

Freelancer, Artelino, AuctionandBarter, Auctiondawg, Auction Warehouse, BidMonkey,

Bidalot, Bid4assets, Bidville, Buyselltrades, Cqout, Furbid, Markplaats, Amazon Auctions,

Guru.com, eBay.

Social Networks

RepCheck, Artistnow, Tribe, OpenBC, Backwash, Gorillaexchange, Visiblepath, iKarma,

Isnotwhatyouknow, RitualiUrbani, Ryze, Amicidegliamici, Wowfriends, Asiansocialnetwork,

Netrelate, Hi5, Orkut, Amityzone, Pandora, Friendster, LinkedIn, Studenti.it.

Discussion Forums

Advogato, Kuro5hin, Chiefdelphi, Invisionize, Daniweb, MajesticForum, Isohunt, ArnieAirsoft,

HTML.it, Gratitude.net, Omidyar.net, Slashdot, Cableforum.

Recommender Systems

Amazon, ExpertCentral, AllExperts, MovieLens, Cnet, Bizrate, Barnes&Noble, DontDateHim,

DatingPsycho, Truedater, HotorNot, Ciao.it, Keen, Experts-exchange, Questico.de, Epinions,

RateMyTeachers.