Page 1
Prediction Market Alternatives for ComplexEnvironments
Paul J. Healy (OSU) John Ledyard (Caltech)Sera Linardi (Caltech) Richard Lowery (CMU)
Decentralization ConferenceTulane University
Apr. 5, 2008
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 1 / 73
Page 2
The Success of Prediction Markets
Wall St. market: 1848–1940 (Rhode & Strumpf 2004)
11/15 correct in mid-October, only 1 very wrong (Wilson 1916)
Iowa Electronic Markets (Berg et al. 2003)
See figure...
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 2 / 73
Page 3
The Success of Prediction Markets
Avg. Error: 1.5% vs. 2.1%. Source: Berg,Forsythe, Nelson & Rietz (2003)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 3 / 73
Page 4
The Success of Prediction Markets
Wall St. market: 1848–1940 (Rhode & Strumpf 2004)
11/15 correct in mid-October, only 1 wrong (W. Wilson)
Iowa Electronic Markets (Berg et al. 2003)
See figure...But... Erikson & Wlezien use trends in polls
TradeSports (Tetlock, Wolfers, Zitzewitz, others...)
Trade volume during Davidson vs. Kansas ≈ 7,700 $10 tickets
NewsFutures, Hollywood Stock Exchange (Pennock et al. 2001)
See figure...
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 4 / 73
Page 5
The Success of Prediction Markets
Source: Wolfers & Zitzewitz (2004)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 5 / 73
Page 6
Corporate Applications
Predicting printer sales at Hewlett-Packard (K-Y Chen & Plott 2002)
Companies claiming to use prediction markets:
Abbot Labs Arcelor Mittal Best Buy ChryslerCorning Electronic Arts Eli Lilly Frito Lay
General Electric Google Hewlett-Packard IntelInterContinental Hotels Masterfoods Microsoft Motorola
Nokia Pfizer Qualcomm SiemensTNT
Are they doing it ‘right’? Volume? Complexity??
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 6 / 73
Page 7
The Policy Analysis Market (PAM)
2001–2003 DARPA (DoD) => NetExchange (Ledyard, Polk, Hanson)
Goal: Predict events the DoD might care about
NetExchange focus: political instability in Middle East
A subset of the issues:
Correlation blows up the state spaceManipulation? (Camerer 98, Strumpf & Rhode 07)Moral Hazard? (Hanson et. al 07)Moral repugnance & P.R. (Roth 07, Hanson 07)
Aug 03: Shut Down, DARPA audited, Poindexter ‘retired’
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 7 / 73
Page 8
This Paper
Questions:
1 Can markets actually work when the environment gets ‘complicated’?
2 Would other mechanisms do better?
Answers:
Test markets vs. 3 other mechs in complex lab environments
1 Market performs poorly; incentived, iterated polls perform better
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 8 / 73
Page 9
Behavioral Mechanism Design
Methodology of comibining experiments & theory to design bettermechanisms for real-world use
Short run goal: find a better mechanism
1 Propose alternative mechanisms
Existing theory & behavioral data as guides
2 Testbed proposed mechanisms
The control of the laboratory
3 Tweak if necessary
Long run goal: improve the design process
1 Identify general principles while testbedding2 Add new constraints, etc., to the design problem
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 9 / 73
Page 10
This Paper
Questions:
1 Can markets actually work when the environment gets complicated?
2 Would other mechanisms do better?
Answers:
Test markets vs. 3 other mechs in complex lab environments
1 Market falls apart, simple iterated polls perform better
2 Why the poll seems to do better in this environment
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 10 / 73
Page 11
Easy vs. Hard Environments
Example similar to our experiment:
1 Simple: Will UNC beat Kansas tonight?
Two states: UNC,Kansas, one security
2 Hard: Who will win each of the last 3 games (2 semi’s and final)?
Three events, not independentEight states: UNC,Kansas×Memphis,UCLA×East,West“Eastern finalist wins” is correlated with other 2 eventsIncomplete set of securities is typically used
TradeSports offers 6 securities (1+1+4)
We will use a complete set of 8
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 11 / 73
Page 12
2-State Environment
Two coins: θ ∈ Θ = X ,Y
Two flip outcomes: ω ∈ Ω = H,T
State of the world = (θ, ω)
θ and ω are correlated, but we care only about ω
p (H |X ) = 0.2p (H |Y ) = 0.4
One coin θ is randomly drawn (50/50)
Each subject sees flips of chosen coin
si = (H , T , H , H)#si ∈ 2, 3, 4
Observe sample ω’s, infer about true θ, predict true ω
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 12 / 73
Page 13
Distributions: 2 States
p0
0
1
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 13 / 73
Page 14
Distributions: 2 States
p2
p1
p0
0
1
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 14 / 73
Page 15
Distributions: 2 States
p2
p1
p0p2
p1
p3
0
1
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 15 / 73
Page 16
Distributions: 2 States
p2
p1
p0p2
p1
p3
pFI
0
1
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 16 / 73
Page 17
Distributions: 2 States
p2
p1
p0p2
p1
p3
pFIh
0
1
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 17 / 73
Page 18
Distributions: 2 States
p2
p1
p0p2
p1
p3
pFIh
0
1
E
R
R
O
R
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 18 / 73
Page 19
8-State Environment
Three coins, ordered: θ ∈ Θ = XYZ ,XZY ,YXZ ,YZX ,ZXY ,ZYX
Eight flip outcomes:ω ∈ Ω = TTT ,TTH,THT ,THH,HTT ,HTH,HHT , HHH
Pr [X = H ] = 0.2Pr [Y = H ] = 0.4Pr [Z = H ] = 0.4Pr [Y = X ] = 2/3 (correlation distinguishes Y , Z )
One ordering θ is randomly drawn (uniformly)
Each subject sees flips from chosen coin ordering
si = (THT , THT , HHH , TTT , HTH)#si ∈ 3, 5, 7
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 19 / 73
Page 20
Distributions: 8 States
p6
p1
p2
p3
p4
p5
p0
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 20 / 73
Page 21
Distributions: 8 States
p6
p1
p2
p3
p4
p5
p0
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 21 / 73
Page 22
Distributions: 8 States
p6
p1
p2
p3
p4
p5
p3
p1
p2
p0
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 22 / 73
Page 23
Distributions: 8 States
p6
p1
p2
p3
p4
p5
pFI p3
p1
p2
p0
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 23 / 73
Page 24
Distributions: 8 States
p6
p1
p2
p3
p4
p5
pFI p3
p1
p2
h
p0
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 24 / 73
Page 25
Distributions: 8 States
p6
p1
p2
p3
p4
p5
pFI
h
p0
ERROR
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 25 / 73
Page 26
The Mechanisms
1 Double Auction (prediction market)
2 Pari-mutuel (horse track)
3 Iterated Poll (‘Delphi method’: RAND/USAF)
4 Market Scoring Rule (Hanson 2003)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 26 / 73
Page 27
Alternative Mechanisms: Pari-Mutuel
Bettors buy tickets on each event
nj = # of tickets purchased on event j
Payoff odds of event-j tickets = (nj/ ∑k nk)−1
Still need 2k securities
Still have a no-trade theorem
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 27 / 73
Page 28
Alternative Mechanisms: Poll
Players announce a belief distribution P i over the 8 events
P = (1/n) ∑i Pi is shown
Repeat 5 times
Everone paid based on final average distribution P
Incentive compatible scoring rule:
Everyone receives(
ln[
Pj
]
− ln [1/8])
event-j securitiesIf event k is true, event-k security pays $1.
There exist many seq. equil. with full info aggregation
There exist babbling seq. equil. with “almost” no aggregation
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 28 / 73
Page 29
Alternative Mechanisms: Market Scoring Rule (Hanson)
A public distribution is shown: (1/8, . . . , 1/8)
Individuals may ‘move’ the distribution to(
P i1, . . . ,P i
8
)
Move from (Q1, . . . ,Q8) to(
P i1, . . . ,P i
8
)
=⇒
Receive(
ln[
P ij
]
− ln[
Q ij
])
event-j securities for each j
Moving Pj up means buying, down means sellingIf event k is true, event-k security pays $1Incentive compatible: you should move to your best guess
Subsidized =⇒ avoids no-trade theorem
Incentive compatible =⇒ myopic players reveal truthfully
Incentive to misrepresent? Depends on move timing...
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 29 / 73
Page 30
Methodology
Run experiments using Caltech undergrads paid ≈ $35
No experience
Crossover design: DA-Poll, Poll-DA, MSR-Pari, Pari-MSR
3 subjects per group
8 periods with each mechanism
No rematching
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 30 / 73
Page 31
Period & Order Effects
1 2 3 4 5 6 7 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
l 2 Dis
tanc
e
Period
2 States
1 2 3 4 5 6 7 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
l 2 Dis
tanc
e
Period
8 States
1 20
0.1
0.2
0.3
0.4
0.5
0.6
0.7
l 2 Dis
tanc
e
Order
2 States
1 20
0.1
0.2
0.3
0.4
0.5
0.6
0.7
l 2 Dis
tanc
e
Order
8 States
No significant period or order effects (good!)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 31 / 73
Page 32
2 States: Error
Comparison of l2 distances with 2 states:
Avg Wilcoxon p-valuesDist. DblAuctn MSR Parimutuel Poll
Avg Dist. − 0.262 0.419 0.295 0.266
DblAuctn 0.262 − 0.092 0.646 0.663MSR 0.419 − − 0.225 0.098
Parimutuel 0.295 − − − 0.519Poll 0.266 − − − −
MSR ≥ Parimutuel ≥ Poll ≥ DblAuctnMSR > Poll ≥ DblAuctn
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 32 / 73
Page 33
2 States: Confusion & Mirages
p2
p1
p0
pFI
h
0
1
C
O
N
F
U
S
E
D
C
O
N
F
U
S
E
D
p2
p1
p0
pFI
h
0
1
M
I
R
A
G
E
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 33 / 73
Page 34
2 States: Catastrophes
Periods with catastrophes:
(32 pers. total) DblAuc MSR Pari Poll
No Trade 0 1 4 0Confusion 5 7 6 11
Mirage 13 14 10 12Confused Mirage 0 1 1 3
None 14 12 13 12
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 34 / 73
Page 35
2 States: Summary of Results
2 States 8 StatesMech Err NoTrd Mirg Conf Err NoTrd Mirg Conf
DblAuc X X X X
MSR × X × X
Pari X × X X
Poll X X X ×
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 35 / 73
Page 36
8 States: Error
Comparison of l2 distances with 8 states:
Avg Wilcoxon p-valuesl2 Dist. DblAuc MSR Parimutuel Poll
Avg l2 Dist. − 0.696 0.527 0.605 0.418
DblAuc 0.696 − 0.002 0.093 < 0.001
MSR 0.527 − − 0.083 0.324Parimutuel 0.605 − − − 0.001
Poll 0.418 − − − −
DblAuc > Parimutuel > MSR ≥ Poll
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 36 / 73
Page 37
8 States: Catastrophes: No Trade
DblAuc MSR Parimutuel Poll
Periods w/ No Trade 0 0 9/32 0
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 37 / 73
Page 38
8 States: Confusion
p6
p1
p2
p3
p4
p5
pFI
h
p0
CONFUSED
Pr (TTT |θ) = 24/75 & Pr (HHH|θ) = 4/75 ∀θ
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 38 / 73
Page 39
8 States: Confusion
p6
p1
p2
p3
p4
p5
pFI
p0
hCONFUSED
Confusion occurs in every period of every mechanism
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 39 / 73
Page 40
8 States: Catastrophes: Confusion
l2 distance to convex hull, conditional on trade occurring:
Avg Dist. DblAuc MSR Pari. Poll
Avg. Dist. 0.447 0.362 0.398 0.312# Trade Pers. 32 32 23 32
DblAuc 0.447 − 0.001 0.107 < 0.001
MSR 0.362 − 0.180 0.257Pari 0.398 − 0.008
Poll 0.312 −
DblAuc ≥ Pari ≥ MSR ≥ PollDblAuc > MSR ≥ PollDlbAuc ≥ Pari > Poll
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 40 / 73
Page 41
8 States: Catastrophes: Mirages
Frequency of Mirages:
Pers. w/ No. ofTrade Mirages Frequency
DblAuc 32 13 0.406MSR 32 7 0.219Pari. 23 7 0.304Poll 32 3 0.094
DblAuc > MSR > PollPari > Poll
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 41 / 73
Page 42
8 States: Summary
2 States 8 StatesMech Err NoTrd Mirg Conf Err NoTrd Mirg Conf
DblAuc X X X X × X × ×MSR × X × X X X X X
Pari X × X X × × X ×Poll X X X × X X X X
Increased complexity: Double auction fails, MSR & Poll work
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 42 / 73
Page 43
Declaring a Winner?
Poll’s only failing: confusion in 2-states. How bad is it?
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
1
2
3
4
5
6
7
Pr(H)
Fre
quen
cyPoll Output: 2 States
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 43 / 73
Page 44
Beating the Prior
Percentage of periods where mechanism outperformed the “informed”prior:
2 States 8 States
DblAuc 0.375 0.000MSR 0.355 0.250Pari 0.393 0.044Poll 0.406 0.313
Poll looks good (relatively)...
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 44 / 73
Page 45
Observations
Why does the poll out-perform the market?
Observation 1: Preferences are aligned in the poll, so traders haveno incentive to misrepresent
‘Misrepresentor’: Move away from full info, then move toward
Number of misrepresentors per mechanism:
DblAuc MSR Pari Poll
14 5 12 3
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 45 / 73
Page 46
Observations
Observation 2: Traders have an incentive to participate in the poll
No-trade theorem in DblAuc and Parimutuel
MSR and poll are subsidized
25.9 cents/trader/period in 2-state35.0 cents/trader/period in 8-state
Pari-mutuel no trade: 4/32 and 9/32 pers.
DblAuc: 1 inactive trader in 4/64 periods
MSR: 1 period of no trade (1st period confusion?)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 46 / 73
Page 47
Observations
Observation 3: Attention is ‘spread thin’ in the DblAuc
States Txns/Min. Vol./Min.
2 5.00 6.488 2.60 14.47
% of txns on 2 most active securities: 46%
% of txns on 2 least active securities: 8%
Low-hanging fruit is missed:
p (TTT ) = 24/75 and p (HHH) = 4/75 regardless of pvt infoAvg |p (TTT )− 24/75| and Avg |p (HHH)− 4/75| are greater thanany other mechanismSignificantly greater than MSR and Poll
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 47 / 73
Page 48
Observations
Observation 4: Poll averages traders’ announcements, mitigatingeffects of a single aberrant trader
Frequency of worse-than-average final reports & predictions
2 States 8 StatesMech Last Report Prediction Last Report Prediction
DblAuc 11 11 24 24MSR 18 18 9 9
Pari-mutuel 11 11 9 9Poll 28 8 21 8
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 48 / 73
Page 49
Summary
Double auction works fine with 2 states, not 8
Observation: think markets problem (focus on 2 securities)Note: not market power problem
Pari-mutuel hurt by delay and no trade
MSR helps ‘unfocus’ attention, but prone to bad outcomes
Single ‘bad’ player can damage performance
Poll performs best
Aligned incentives, participation incentives, averaging smoothsbehavior, completely ‘unfocused’
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 49 / 73
Page 50
The End
The End
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 50 / 73
Page 51
p(TTT) and p(HHH)
Recall that p (TTT ) = 24/75 and p (HHH) = 4/75 regardless of signals
DblAuc MSR Pari Poll
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4D
ista
nce
Box Plot of | p(TTT) − 24/75 |
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 51 / 73
Page 52
p(TTT) and p(HHH)
Recall that p (TTT ) = 24/75 and p (HHH) = 4/75 regardless of signals
DblAuc MSR Pari Poll
0
0.05
0.1
0.15
0.2D
ista
nce
Box Plot of | p(HHH) − 4/75 |
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 52 / 73
Page 53
Double Auction: Market Thinness
8 states vs. 2 states:
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 53 / 73
Page 54
Double Auction: Market Thinness
8 states vs. 2 states:
Fewer transactions per minute (2.60 vs. 5.00), despite more markets!
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 53 / 73
Page 55
Double Auction: Market Thinness
8 states vs. 2 states:
Fewer transactions per minute (2.60 vs. 5.00), despite more markets!More total volume per minute (14.47 units vs. 6.48 units)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 53 / 73
Page 56
Double Auction: Market Thinness
8 states vs. 2 states:
Fewer transactions per minute (2.60 vs. 5.00), despite more markets!More total volume per minute (14.47 units vs. 6.48 units)
Fraction of trades in each market:
Session TTT TTH THT THH HTT HTH HHT HHH
1 0.18 0.13 0.15 0.04 0.01 0.15 0.12 0.232 0.25 0.17 0.07 0.03 0.15 0.05 0.14 0.123 0.34 0.04 0.26 0.07 0.14 0.02 0.02 0.114 0.16 0.07 0.23 0.12 0.17 0.09 0.07 0.09
All 0.27 0.11 0.16 0.05 0.14 0.05 0.09 0.12
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 53 / 73
Page 57
Double Auction: Market Thinness
8 states vs. 2 states:
Fewer transactions per minute (2.60 vs. 5.00), despite more markets!More total volume per minute (14.47 units vs. 6.48 units)
Fraction of trades in each market:
Session TTT TTH THT THH HTT HTH HHT HHH
1 0.18 0.13 0.15 0.04 0.01 0.15 0.12 0.232 0.25 0.17 0.07 0.03 0.15 0.05 0.14 0.123 0.34 0.04 0.26 0.07 0.14 0.02 0.02 0.114 0.16 0.07 0.23 0.12 0.17 0.09 0.07 0.09
All 0.27 0.11 0.16 0.05 0.14 0.05 0.09 0.12
‘FOCUS’ = standard deviation of these distributions
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 53 / 73
Page 58
Double Auction: Market Thinness
8 states vs. 2 states:
Fewer transactions per minute (2.60 vs. 5.00), despite more markets!More total volume per minute (14.47 units vs. 6.48 units)
Fraction of trades in each market:
Session TTT TTH THT THH HTT HTH HHT HHH
1 0.18 0.13 0.15 0.04 0.01 0.15 0.12 0.232 0.25 0.17 0.07 0.03 0.15 0.05 0.14 0.123 0.34 0.04 0.26 0.07 0.14 0.02 0.02 0.114 0.16 0.07 0.23 0.12 0.17 0.09 0.07 0.09
All 0.27 0.11 0.16 0.05 0.14 0.05 0.09 0.12
‘FOCUS’ = standard deviation of these distributions
Focus, # transactions, and trading volume don’t predict accuracywell.
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 53 / 73
Page 59
MSR: Market Thinness
Is MSR ‘trading’ focused on a small number of securities?
Session TTT TTH THT THH HTT HTH HHT HHH
1 0.23 0.12 0.11 0.09 0.17 0.12 0.09 0.072 0.36 0.14 0.17 0.02 0.14 0.06 0.02 0.083 0.22 0.16 0.13 0.08 0.10 0.03 0.07 0.214 0.20 0.13 0.06 0.10 0.16 0.10 0.10 0.16
All 0.24 0.13 0.11 0.08 0.14 0.08 0.08 0.14
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 54 / 73
Page 60
MSR: Market Thinness
Is focus predictive of market accuracy?
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 55 / 73
Page 61
MSR: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.007 + 0.202× FOCUSt
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 55 / 73
Page 62
MSR: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.007 + 0.202× FOCUSt
p-value < 0.0001, R2 = 0.52
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 55 / 73
Page 63
MSR: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.007 + 0.202× FOCUSt
p-value < 0.0001, R2 = 0.52Avg. focus in MSR = 0.290
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 55 / 73
Page 64
MSR: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.007 + 0.202× FOCUSt
p-value < 0.0001, R2 = 0.52Avg. focus in MSR = 0.290Avg. focus in Dbl Auction = 0.475
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 55 / 73
Page 65
MSR: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.007 + 0.202× FOCUSt
p-value < 0.0001, R2 = 0.52Avg. focus in MSR = 0.290Avg. focus in Dbl Auction = 0.475
Is activity in a security predictive of its price accuracy?
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 55 / 73
Page 66
MSR: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.007 + 0.202× FOCUSt
p-value < 0.0001, R2 = 0.52Avg. focus in MSR = 0.290Avg. focus in Dbl Auction = 0.475
Is activity in a security predictive of its price accuracy?
ERRORt,ω = 0.659− 0.053× NUMMOVESt,ω
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 55 / 73
Page 67
MSR: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.007 + 0.202× FOCUSt
p-value < 0.0001, R2 = 0.52Avg. focus in MSR = 0.290Avg. focus in Dbl Auction = 0.475
Is activity in a security predictive of its price accuracy?
ERRORt,ω = 0.659− 0.053× NUMMOVESt,ω
p-value = 0.002, R2 = 0.03
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 55 / 73
Page 68
MSR: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.007 + 0.202× FOCUSt
p-value < 0.0001, R2 = 0.52Avg. focus in MSR = 0.290Avg. focus in Dbl Auction = 0.475
Is activity in a security predictive of its price accuracy?
ERRORt,ω = 0.659− 0.053× NUMMOVESt,ω
p-value = 0.002, R2 = 0.03Avg. # moves/period in MSR = 2.03
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 55 / 73
Page 69
MSR: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.007 + 0.202× FOCUSt
p-value < 0.0001, R2 = 0.52Avg. focus in MSR = 0.290Avg. focus in Dbl Auction = 0.475
Is activity in a security predictive of its price accuracy?
ERRORt,ω = 0.659− 0.053× NUMMOVESt,ω
p-value = 0.002, R2 = 0.03Avg. # moves/period in MSR = 2.03Avg. # txns/period in DA = 1.63
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 55 / 73
Page 70
Mechanism Comparison: Summary
DblAuc’n works with 2 states, fails with 8
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 56 / 73
Page 71
Mechanism Comparison: Summary
DblAuc’n works with 2 states, fails with 8
Thin markets problem
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 56 / 73
Page 72
Mechanism Comparison: Summary
DblAuc’n works with 2 states, fails with 8
Thin markets problemNOT a market power problem
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 56 / 73
Page 73
Mechanism Comparison: Summary
DblAuc’n works with 2 states, fails with 8
Thin markets problemNOT a market power problem
MSR performs poorly for 2 states, OK with 8
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 56 / 73
Page 74
Mechanism Comparison: Summary
DblAuc’n works with 2 states, fails with 8
Thin markets problemNOT a market power problem
MSR performs poorly for 2 states, OK with 8
Less focusing on securities
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 56 / 73
Page 75
Mechanism Comparison: Summary
DblAuc’n works with 2 states, fails with 8
Thin markets problemNOT a market power problem
MSR performs poorly for 2 states, OK with 8
Less focusing on securitiesCompetitive =⇒ may hide info
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 56 / 73
Page 76
Mechanism Comparison: Summary
DblAuc’n works with 2 states, fails with 8
Thin markets problemNOT a market power problem
MSR performs poorly for 2 states, OK with 8
Less focusing on securitiesCompetitive =⇒ may hide info
Parimutuel is okay when people trade
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 56 / 73
Page 77
Mechanism Comparison: Summary
DblAuc’n works with 2 states, fails with 8
Thin markets problemNOT a market power problem
MSR performs poorly for 2 states, OK with 8
Less focusing on securitiesCompetitive =⇒ may hide info
Parimutuel is okay when people trade
No-trade theorem is most obvious here?
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 56 / 73
Page 78
Mechanism Comparison: Summary
DblAuc’n works with 2 states, fails with 8
Thin markets problemNOT a market power problem
MSR performs poorly for 2 states, OK with 8
Less focusing on securitiesCompetitive =⇒ may hide info
Parimutuel is okay when people trade
No-trade theorem is most obvious here?
Poll works best with 8 states, good with 2 states
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 56 / 73
Page 79
Mechanism Comparison: Summary
DblAuc’n works with 2 states, fails with 8
Thin markets problemNOT a market power problem
MSR performs poorly for 2 states, OK with 8
Less focusing on securitiesCompetitive =⇒ may hide info
Parimutuel is okay when people trade
No-trade theorem is most obvious here?
Poll works best with 8 states, good with 2 states
Incentives are aligned
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 56 / 73
Page 80
Conclusions
Dbl Auction is not best for all environments
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 57 / 73
Page 81
Conclusions
Dbl Auction is not best for all environments
Thin market problem
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 57 / 73
Page 82
Conclusions
Dbl Auction is not best for all environments
Thin market problem
Want to ‘move’ many prices at once
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 57 / 73
Page 83
Conclusions
Dbl Auction is not best for all environments
Thin market problem
Want to ‘move’ many prices at once
No-trade theorem and subsidies
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 57 / 73
Page 84
Conclusions
Dbl Auction is not best for all environments
Thin market problem
Want to ‘move’ many prices at once
No-trade theorem and subsidies
Parimutuel vs. Dbl Auction??
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 57 / 73
Page 85
Conclusions
Dbl Auction is not best for all environments
Thin market problem
Want to ‘move’ many prices at once
No-trade theorem and subsidies
Parimutuel vs. Dbl Auction??MSR and Poll are subsidized
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 57 / 73
Page 86
Conclusions
Dbl Auction is not best for all environments
Thin market problem
Want to ‘move’ many prices at once
No-trade theorem and subsidies
Parimutuel vs. Dbl Auction??MSR and Poll are subsidizedBetter performance worth the cost?
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 57 / 73
Page 87
General Environment
States of the world: (θ, ω) ∈ Θ × Ω
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 58 / 73
Page 88
General Environment
States of the world: (θ, ω) ∈ Θ × Ω
θ is an unknown parameter
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 58 / 73
Page 89
General Environment
States of the world: (θ, ω) ∈ Θ × Ω
θ is an unknown parameterω is randomly drawn, given θ
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 58 / 73
Page 90
General Environment
States of the world: (θ, ω) ∈ Θ × Ω
θ is an unknown parameterω is randomly drawn, given θ
Prior beliefs (given)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 58 / 73
Page 91
General Environment
States of the world: (θ, ω) ∈ Θ × Ω
θ is an unknown parameterω is randomly drawn, given θ
Prior beliefs (given)
f (θ, ω): Joint distribution
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 58 / 73
Page 92
General Environment
States of the world: (θ, ω) ∈ Θ × Ω
θ is an unknown parameterω is randomly drawn, given θ
Prior beliefs (given)
f (θ, ω): Joint distributionq0 (θ): Prior distribution on Θ
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 58 / 73
Page 93
General Environment
States of the world: (θ, ω) ∈ Θ × Ω
θ is an unknown parameterω is randomly drawn, given θ
Prior beliefs (given)
f (θ, ω): Joint distributionq0 (θ): Prior distribution on Θ
p0 (ω): Prior distribution on Ω
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 58 / 73
Page 94
General Environment
States of the world: (θ, ω) ∈ Θ × Ω
θ is an unknown parameterω is randomly drawn, given θ
Prior beliefs (given)
f (θ, ω): Joint distributionq0 (θ): Prior distribution on Θ
p0 (ω): Prior distribution on Ω
p (ω|θ) = pθ (ω): Conditional distribution
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 58 / 73
Page 95
General Environment
States of the world: (θ, ω) ∈ Θ × Ω
θ is an unknown parameterω is randomly drawn, given θ
Prior beliefs (given)
f (θ, ω): Joint distributionq0 (θ): Prior distribution on Θ
p0 (ω): Prior distribution on Ω
p (ω|θ) = pθ (ω): Conditional distribution
Signal: s = (s1, . . . , sn)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 58 / 73
Page 96
General Environment
States of the world: (θ, ω) ∈ Θ × Ω
θ is an unknown parameterω is randomly drawn, given θ
Prior beliefs (given)
f (θ, ω): Joint distributionq0 (θ): Prior distribution on Θ
p0 (ω): Prior distribution on Ω
p (ω|θ) = pθ (ω): Conditional distribution
Signal: s = (s1, . . . , sn)
si =(
si1, . . . , siKi
)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 58 / 73
Page 97
General Environment
States of the world: (θ, ω) ∈ Θ × Ω
θ is an unknown parameterω is randomly drawn, given θ
Prior beliefs (given)
f (θ, ω): Joint distributionq0 (θ): Prior distribution on Θ
p0 (ω): Prior distribution on Ω
p (ω|θ) = pθ (ω): Conditional distribution
Signal: s = (s1, . . . , sn)
si =(
si1, . . . , siKi
)
sik ∼ p (·|θ) (sik ∈ Ω)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 58 / 73
Page 98
General Environment
States: (ω, θ) ∈ Θ × Ω with pdf f (ω, θ) (correlated)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 59 / 73
Page 99
General Environment
States: (ω, θ) ∈ Θ × Ω with pdf f (ω, θ) (correlated)
Care about ω, but only have information on θ
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 59 / 73
Page 100
General Environment
States: (ω, θ) ∈ Θ × Ω with pdf f (ω, θ) (correlated)
Care about ω, but only have information on θ
Step 1: Draw ‘true’ state (ω∗, θ
∗)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 59 / 73
Page 101
General Environment
States: (ω, θ) ∈ Θ × Ω with pdf f (ω, θ) (correlated)
Care about ω, but only have information on θ
Step 1: Draw ‘true’ state (ω∗, θ
∗)
Step 2: Draw signals si = (si1, . . . , siKi) of ω
∗ given θ∗
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 59 / 73
Page 102
General Environment
States: (ω, θ) ∈ Θ × Ω with pdf f (ω, θ) (correlated)
Care about ω, but only have information on θ
Step 1: Draw ‘true’ state (ω∗, θ
∗)
Step 2: Draw signals si = (si1, . . . , siKi) of ω
∗ given θ∗
sik ∼ p (ω|θ∗) (sik ∈ Ω)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 59 / 73
Page 103
General Environment
States: (ω, θ) ∈ Θ × Ω with pdf f (ω, θ) (correlated)
Care about ω, but only have information on θ
Step 1: Draw ‘true’ state (ω∗, θ
∗)
Step 2: Draw signals si = (si1, . . . , siKi) of ω
∗ given θ∗
sik ∼ p (ω|θ∗) (sik ∈ Ω)
Step 3: Use Bayes’s Law to form various beliefs about θ given signals
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 59 / 73
Page 104
General Environment
States: (ω, θ) ∈ Θ × Ω with pdf f (ω, θ) (correlated)
Care about ω, but only have information on θ
Step 1: Draw ‘true’ state (ω∗, θ
∗)
Step 2: Draw signals si = (si1, . . . , siKi) of ω
∗ given θ∗
sik ∼ p (ω|θ∗) (sik ∈ Ω)
Step 3: Use Bayes’s Law to form various beliefs about θ given signals
Individual: q (θ|si ) ‘Full-info’: q (θ|s1, . . . , sn)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 59 / 73
Page 105
General Environment
States: (ω, θ) ∈ Θ × Ω with pdf f (ω, θ) (correlated)
Care about ω, but only have information on θ
Step 1: Draw ‘true’ state (ω∗, θ
∗)
Step 2: Draw signals si = (si1, . . . , siKi) of ω
∗ given θ∗
sik ∼ p (ω|θ∗) (sik ∈ Ω)
Step 3: Use Bayes’s Law to form various beliefs about θ given signals
Individual: q (θ|si ) ‘Full-info’: q (θ|s1, . . . , sn)
Step 4: Form posteriors on Θ given signals:
pi (ω) = ∑θ∈Θ
p (ω|θ) q (θ|si )
pFI (ω) = ∑θ∈Θ
p (ω|θ) q (θ|s1, . . . , sn) .
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 59 / 73
Page 106
General Environment
States: (ω, θ) ∈ Θ × Ω with pdf f (ω, θ) (correlated)
Care about ω, but only have information on θ
Step 1: Draw ‘true’ state (ω∗, θ
∗)
Step 2: Draw signals si = (si1, . . . , siKi) of ω
∗ given θ∗
sik ∼ p (ω|θ∗) (sik ∈ Ω)
Step 3: Use Bayes’s Law to form various beliefs about θ given signals
Individual: q (θ|si ) ‘Full-info’: q (θ|s1, . . . , sn)
Step 4: Form posteriors on Θ given signals:
pi (ω) = ∑θ∈Θ
p (ω|θ) q (θ|si )
pFI (ω) = ∑θ∈Θ
p (ω|θ) q (θ|s1, . . . , sn) .
Step 5: Run the mechanism, which generates h (ω) over Ω.
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 59 / 73
Page 107
Order Effects
1 20
0.1
0.2
0.3
0.4
0.5
0.6
0.7
l 2 Dis
tanc
e
Order
2 States
1 20
0.1
0.2
0.3
0.4
0.5
0.6
0.7
l 2 Dis
tanc
e
Order
8 States
Pairwise Wilcoxon tests: p = 0.82 or 0.93
By mechanism: p ≥ 0.39 or ≥ 0.07 (Pari. worse)
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 60 / 73
Page 108
2 States: Catastrophes: Confusion
Average l2 distance to convex hull, conditional on confusion:
DblAuc 0.0011MSR 0.0340Pari 0.0185Poll 0.0050
Double Auction & Poll don’t get as ‘badly’ confused
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 61 / 73
Page 109
2 States: Catastrophes: Confusion
Average l2 distance to convex hull, conditional on confusion:
DblAuc 0.0011MSR 0.0340Pari 0.0185Poll 0.0050
Double Auction & Poll don’t get as ‘badly’ confused
Is slight confusion any better than bad confusion??
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 61 / 73
Page 110
2 States: Mechanism Error Without Confusion
Average mechanism error, conditional on NO confusion:
No Confusion All Periods
DblAuc 0.128 0.131MSR 0.136 0.210Pari 0.110 0.148Poll 0.093 0.133
No significant differences in pairwise tests
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 62 / 73
Page 111
8 States: Catastrophes: Confusion
p6
p1
p2
p3
p4
p5
pFI
h
p0
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 63 / 73
Page 112
8 States: Catastrophes: Confusion
p6
p1
p2
p3
p4
p5
pFI
h
p0
CONFUSED
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 64 / 73
Page 113
8 States: Catastrophes: Confusion
p6
p1
p2
p3
p4
p5
pFI
p0
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 65 / 73
Page 114
8 States: Catastrophes: Confusion
p6
p1
p2
p3
p4
p5
pFI
p0
h
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 66 / 73
Page 115
8 States: Catastrophes: Confusion
p6
p1
p2
p3
p4
p5
pFI
p0
hCONFUSED
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 67 / 73
Page 116
8 States: Catastrophes: Mirages
p6
p1
p2
p3
p4
p5
pFI
p0
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 68 / 73
Page 117
8 States: Catastrophes: Mirages
p6
p1
p2
p3
p4
p5
pFI
hp0
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 69 / 73
Page 118
8 States: Catastrophes: Mirages
p6
p1
p2
p3
p4
p5
pFI
hp0
MIRAGE
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 70 / 73
Page 119
8 States: Catastrophes: Mirages
An alternative definition:
Of the 8 probabilities, 6 should move from p0
How many (out of 6) move the right way?
Mean MSR Pari Poll
DblAuc 3.03 0.049 0.046 0.034
MSR 3.69 0.798 0.239Pari 3.70 0.467Poll 3.97
Wilcoxon p-values
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 71 / 73
Page 120
8 States: Mirages
Wilcoxon test on ‘angles’ between(
h − p0)
and(
pFI − p0)
:
MSR Pari Poll
DblAuc 0.025 0.490 0.290MSR 0.180 0.773Pari 0.286Poll
Wilcoxon test p-values
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 72 / 73
Page 121
Double Auction: Market Thinness
Is focus predictive of market accuracy?
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 73 / 73
Page 122
Double Auction: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.218− 0.070× FOCUSt , p-value = 0.222
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 73 / 73
Page 123
Double Auction: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.218− 0.070× FOCUSt , p-value = 0.222
Is trading volume in a security predictive of its price accuracy?
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 73 / 73
Page 124
Double Auction: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.218− 0.070× FOCUSt , p-value = 0.222
Is trading volume in a security predictive of its price accuracy?
ERRORt,ω = 0.954− 0.003× VOLUMEt,ω, p-value = 0.200
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 73 / 73
Page 125
Double Auction: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.218− 0.070× FOCUSt , p-value = 0.222
Is trading volume in a security predictive of its price accuracy?
ERRORt,ω = 0.954− 0.003× VOLUMEt,ω, p-value = 0.200
Is # of txns predictive of price accuracy?
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 73 / 73
Page 126
Double Auction: Market Thinness
Is focus predictive of market accuracy?
ERRORt = 0.218− 0.070× FOCUSt , p-value = 0.222
Is trading volume in a security predictive of its price accuracy?
ERRORt,ω = 0.954− 0.003× VOLUMEt,ω, p-value = 0.200
Is # of txns predictive of price accuracy?
ERRORt,ω = 0.941− 0.011× NUMTXNSt,ω, p-value = 0.367
P.J. Healy (OSU) Prediction Mechanisms Decentralization 2008 73 / 73