Top Banner

of 29

Superforecasting by Philip E. Tetlock and Dan Gardner

Aug 07, 2018

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    1/29

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    2/29

    http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136693/siteID/8001/retailerid/6/trackingcode/penguinrandomhttp://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136709/siteID/8001/retailerid/22/trackingcode/PRH698C2DEE54http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136693/siteID/8001/retailerid/2/trackingcode/PRH698C2DEE54http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136709/siteID/8001/retailerid/3/trackingcode/PRH698C2DEE54http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136693/siteID/8001/retailerid/7/trackingcode/randohouseinc10663-20http://www.crownpublishing.com/

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    3/29

    p r f ` s b j h r

    ^ R Q B Y J H Y B N F ^ W @ O K

    —^upbrjhrbnfst`ok  `s f rfrb ihhm teft w`dd lfmb yhu slfrtbr fog w`sbr.

    Hob hj teb k`fots hj ibefv`hrfd sn`bonb rbvbfds ehw th `lprhvb ft

    prbg`nt`ok teb juturb.“

    ‗ FGFL KYF OW,

    Obw Phrm W`lbs  ibstsbdd`ok futehr hj K`vb fog Wfmb

    —Khhg cugklbot fog khhg jhrbnfst`ok frb rfrb, iut teby turo hut

    th ib lfgb hj tbfnefidb sm`dds. Iy jhrn`ok jhrbnfstbrs th nhlpbtb,

    Wbtdhnm g`snhvbrbg weft teb sm`dds frb fog ehw teby whrm, fog te`sihhm tbfnebs teb fi`d`ty th foy `otbrbstbg rbfgbr.“

    ‗ ^ WB[F Y W I Y F OG ,

    prbs`gbot, Web Dhok Ohw Jhuogft`ho

    —Qe`d`p Wbtdhnm `s rbohwobg jhr gblhostrft`ok teft lhst bxpbrts

    frb oh ibttbr tefo ―gfrt-terhw`ok lhombys‐ ft prbg`nt`ok bdbnt`hos, wfrs, bnhohl`n nhddfpsbs, fog htebr bvbots. @o e`s ir`dd`fot obw

    ihhm, Wbtdhnm hjjbrs f lune lhrb ehpbjud lbssfkb, ifsbg honb fkf`o

    ho e`s hwo krhuogirbfm`ok rbsbfrne. Eb sehws teft nbrtf`o pbhpdb

    nfo jhrbnfst bvbots w`te fnnurfny lune ibttbr tefo nefonb‗fog

    sh, pbrefps, nfo teb rbst hj us, `j wb bludftb teb nr`t`nfd te`om`ok

    hj tebsb ―supbrjhrbnfstbrs.‐ Web sbdj-blphwbrlbot kborb ghbso‐t kbt

    foy slfrtbr fog lhrb shpe`st`nftbg tefo te`s.“

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    4/29

    —^upbrjhrbnfst`ok   `s teb rfrb ihhm teft `s ihte snehdfrdy fog bokfk-

    `ok. Web dbsshos frb sn`bot`stn, nhlpbdd`ok, fog bohrlhusdy prfn-

    t`nfd. Foyhob weh `s `o teb jhrbnfst`ok ius`obss‗fog teft‐s fdd hj

    us‗sehudg grhp weft teby frb gh`ok fog rbfg `t.“

    ‗ L @ NE F BD C . L F R IH R ^ ^ @ O ,

    ebfg, Kdhifd J`ofon`fd ^trftbk`bs, Nrbg`t ^u`ssb

    —Webrb `so‐t f shn`fd sn`bot`st `o teb whrdg @ fgl`rb lhrb tefo Qe`d

    Wbtdhnm.“

    ‗ W @ L E F Y J H Y G ,

    futehr hj Web Rogbrnhvbr Bnhohl`st 

    —Jrhl teb Hrfndb hj Gbdpe` th lbg`bvfd fstrhdhkbrs th lhgbro hvbr-

    nhostgbot bxpbrts, jhrbnfstbrs efvb ibbo b`tebr gbdugbg hr jrfugu-

    dbot. Jhr teb strst t`lb, ^upbrjhrbnfst`ok  rbvbfds teb sbnrbt hj lfm`ok

    ehobst, rbd`fidb, bjjbnt`vb, usbjud cugklbots fihut teb juturb.“‗ F F Y H O IY H [ O ,

    ne`bj r`sm hjj`nbr, FVY Nfp`tfd Lfofkblbot, fog

    futehr hj Web Qhmbr Jfnb hj [fdd ^trbbt 

    —^hnrftbs efg teb `os`ket `o ―mohw teysbdj,‐ Mfeoblfo gbd`vbrbg teb

    sn`bonb `o We`om`ok, Jfst fog ^dhw, fog ohw Wbtdhnm efs shlbte`ok

     wb nfo fdd fppdy `o ^upbrjhrbnfst`ok.“

    ‗ CR F O D R @ ^ QBY BS ,

    kdhifd ebfg, RI^ Krhup Ybsbfrne

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    5/29

    Nhpyr`ket © 9162 iy Qe`d`p Wbtdhnm Nhosudt`ok, @on., fog Nhoofuket ^trbbt, @on.

     Fdd r`kets rbsbrvbg.

    Quid`sebg `o teb Ro`tbg ^tftbs iy Nrhwo Quid`sebrs,

    fo `lpr`ot hj teb Nrhwo Quid`se`ok Krhup,

    f g`v`s`ho hj Qboku`o Yfoghl Ehusb DDN, Obw Phrm.

     www.nrhwopuid`se`ok.nhl

    NYH[O fog teb Nrhwo nhdhpeho frb rbk`stbrbgtrfgblfrms hj Qboku`o Yfoghl Ehusb DDN.

    D`irfry hj Nhokrbss Nftfdhk`ok-̀ o-Quid`nft`ho Gftf 

    ]N@Q gftf\

    @^IO 0;5-1-5176->::0->

    b@^IO 0;5-1-5176->:;1-0

    Qr`otbg `o teb Ro`tbg ^tftbs hj Flbr`nf 

    @ddustrft`hos iy Chb DbLhoo`br 

     Cfnmbt gbs`ko iy 

    9 7 : 5 61 0 ; 2 > 6

    J`rst Bg`t`ho

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    6/29

    6

     Fo Hpt`l`st`n ^mbpt`n

    [b frb fdd jhrbnfstbrs. [ebo wb te`om fihut nefok`ok chis, kbt-

    t`ok lfrr`bg, iuy`ok f ehlb, lfm`ok fo `ovbstlbot, dfuone-

    `ok f prhgunt, hr rbt`r`ok, wb gbn`gb ifsbg ho ehw wb bxpbnt tebjuturb w`dd uojhdg. Websb bxpbntft`hos frb jhrbnfsts. Hjtbo wb gh hur

    hwo jhrbnfst`ok. Iut webo i`k bvbots efppbo‗lfrmbts nrfse, wfrs

    dhhl, dbfgbrs trblidb‗ wb turo th teb bxpbrts, tehsb `o teb mohw.

     [b dhhm th pbhpdb d`mb Whl Jr`bglfo.

    @j yhu frb f [e`tb Ehusb stfjjbr, yhu l`ket stog e`l `o teb

    Hvfd Hjstnb w`te teb prbs`gbot hj teb Ro`tbg ^tftbs, tfdm`ok fihut

    teb L`ggdb Bfst. @j yhu frb f Jhrtuob 211 NBH, yhu l`ket spht e`l

    `o Gfvhs, neftt`ok `o teb dhuokb w`te ebgkb juog i`dd`hof`rbs fog

    ^fug` pr`onbs. Fog ̀ j yhu gho‐t jrbqubot teb [e`tb Ehusb hr swfomy

    ^w`ss ehtbds, yhu nfo rbfg e`s Obw Phrm W`lbs  nhdulos fog ibstsbdd-

    `ok ihhms teft tbdd yhu weft‐s efppbo`ok ohw, wey, fog weft w`dd

    nhlb obxt.6 L`dd`hos gh.

    D`mb Whl Jr`bglfo, I`dd Jdfnm jhrbnfsts kdhifd bvbots. Iut tebrb`s f dht dbss gblfog jhr e`s `os`kets.

    Jhr ybfrs, I`dd whrmbg jhr teb R^ Gbpfrtlbot hj Fkr`nudturb `o

     Fr`zhof‗—pfrt p`nm-fog-sehvbd whrm, pfrt sprbfgsebbt“‗iut ohw

    eb d`vbs `o Mbfroby, Obirfsmf. I`dd `s f oft`vb Nhroeusmbr. Eb krbw

    up `o Lfg`sho, Obirfsmf, f jfrl thwo webrb e`s pfrbots hwobg fog

    puid`sebg teb  Lfg`sho ^tfr- Lf`d , f obwspfpbr w`te dhts hj sthr`bs

    fihut dhnfd sphrts fog nhuoty jf`rs. Eb wfs f khhg stugbot `o e`ke

    e d g e i e d j ` g j e

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    7/29

    9 supbrjhrbnfst`ok

     Fr`zhof. Eb wfs f`l`ok jhr f QeG `o lfte, iut eb rbfd`zbg `t wfs

    ibyhog e`s fi`d`t`bs‗—@ efg ly ohsb ruiibg `o ly d`l`tft`hos“ `s

    ehw eb puts `t‗fog eb grhppbg hut. @t wfso‐t wfstbg t`lb, ehwbvbr.Ndfssbs `o hro`tehdhky lfgb I`dd fo fv`g i`rg- wftnebr, fog ibnfusb

     Fr`zhof `s f krbft pdfnb th sbb i`rgs, eb g`g stbdgwhrm pfrt-t`lb jhr

    sn`bot`sts, tebo kht f chi w`te teb Gbpfrtlbot hj Fkr`nudturb fog

    stfybg jhr f we`db.

    I`dd `s stjty-stvb fog rbt`rbg, fdtehuke eb sfys `j shlbhob hjjbrbg

    e`l f chi eb whudg nhos`gbr `t. ^h eb efs jrbb t`lb. Fog eb spbogs

    shlb hj `t jhrbnfst`ok.I`dd efs foswbrbg rhukedy terbb euogrbg qubst`hos d`mb —[`dd

    Yuss`f hjstn`fddy foobx fgg`t`hofd Rmrf`o`fo tbrr`thry `o teb obxt

    terbb lhotes8“ fog —@o teb obxt ybfr, w`dd foy nhuotry w`tegrfw

    jrhl teb burhzhob8“ Weby frb qubst`hos teft lfttbr. Fog teby‐rb

    g`jstnudt. Nhrphrft`hos, ifoms, blifss`bs, fog `otbdd`kbonb fkbon`bs

    strukkdb th foswbr sune qubst`hos fdd teb t`lb. —[`dd Ohrte Mhrbf

    gbthoftb f oundbfr gbv`nb ibjhrb teb bog hj te`s ybfr8“ —Ehw lfoy

    fgg`t`hofd nhuotr`bs w`dd rbphrt nfsbs hj teb Bihdf v`rus `o teb obxt

    b`ket lhotes8“ —[`dd @og`f hr Irfz`d ibnhlb f pbrlfobot lblibr hj

    teb RO ^bnur`ty Nhuon`d `o teb obxt twh ybfrs8“ ^hlb hj teb qubs-

    t`hos frb ghwor`ket hisnurb, ft dbfst jhr lhst hj us. —[`dd OFWH

    `ov`tb obw nhuotr`bs th ch`o teb Lblibrse`p Fnt`ho Qdfo (LFQ)

    `o teb obxt o`ob lhotes8“ —[`dd teb Murg`stfo Ybk`hofd Khvbro-lbot ehdg f rbjbrbogul ho oft`hofd `ogbpbogbonb te`s ybfr8“ —@j

    f oho-Ne`obsb tbdbnhlluo`nft`hos strl w`os f nhotrfnt th prhv`gb

    @otbrobt sbrv`nbs `o teb ^efokef` Jrbb Wrfgb Shob `o teb obxt twh

    ybfrs, w`dd Ne`obsb n`t`zbos efvb fnnbss th Jfnbihhm fog/hr Ww`ttbr8“

     [ebo I`dd strst sbbs hob hj tebsb qubst`hos, eb lfy efvb oh ndub ehw

    th foswbr `t. —[eft ho bfrte `s teb ^efokef` Jrbb Wrfgb Shob8“ eb

    lfy te`om. Iut eb ghbs e`s ehlbwhrm. Eb kftebrs jfnts, ifdfonbs

    d e` g d

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    8/29

      Fo Hpt`l`st`n ^mbpt`n >

    sefrb e`s tehukets ho NOO. Eb efs obvbr ibbo `ov`tbg th Gfvhs th

    s`t ho f pfobd w`te Whl Jr`bglfo. Fog teft‐s uojhrtuoftb. Ibnfusb

    I`dd Jdfnm `s f rblfrmfidb jhrbnfstbr. [b mohw teft ibnfusb bfne hobhj I`dd‐s prbg`nt`hos efs ibbo gftbg, rbnhrgbg, fog fssbssbg jhr fnnu-

    rfny iy ̀ ogbpbogbot sn`bot`stn hisbrvbrs. E`s trfnm rbnhrg ̀ s bxnbddbot.

    I`dd `s oht fdhob. Webrb frb tehusfogs hj htebrs foswbr`ok teb

    sflb qubst`hos. Fdd frb vhduotbbrs. Lhst frbo‐t fs khhg fs I`dd, iut

    fihut 9% frb. Weby `ondugb bok`obbrs fog dfwybrs, frt`sts fog sn`bo-

    t`sts, [fdd ^trbbtbrs fog Lf`o ^trbbtbrs, prhjbsshrs fog stugbots. [b

     w`dd lbbt lfoy hj tebl, `ondug`ok f lfteblft`n`fo, f stdllfmbr,fog shlb rbt`rbbs bfkbr th sefrb teb`r uogbrusbg tfdbots. @ nfdd tebl

    supbrjhrbnfstbrs  ibnfusb teft ̀ s weft teby frb. Ybd`fidb bv`gbonb prhvbs

    `t. Bxpdf`o`ok wey teby‐rb sh khhg, fog ehw htebrs nfo dbfro th gh

     weft teby gh, `s ly khfd `o te`s ihhm.

    Ehw hur dhw-prhstdb supbrjhrbnfstbrs nhlpfrb w`te nbrbirfd nb-

    dbir`t`bs d`mb Whl Jr`bglfo `s fo `otr`ku`ok qubst`ho, iut `t nfo‐t ib

    foswbrbg ibnfusb teb fnnurfny hj Jr`bglfo‐s jhrbnfst`ok efs obvbr

    ibbo r`khrhusdy tbstbg. Hj nhursb Jr`bglfo‐s jfos fog nr`t`ns efvb

    hp`o`hos hob wfy hr teb htebr‗—eb of`dbg teb Frfi ^pr`ok“ hr —eb

    snrbwbg up ho teb 911> `ovfs`ho hj @rfq“ hr —eb wfs prbsn`bot ho

    OFWH bxpfos`ho.“ Iut tebrb frb oh efrg jfnts fihut Whl Jr`bg-

    lfo‐s trfnm rbnhrg, cust bogdbss hp`o`hos‗fog hp`o`hos ho hp`o-

    `hos.9

     Fog teft ̀ s ius`obss fs usufd. Bvbry gfy, teb obws lbg`f gbd`vbrjhrbnfsts w`tehut rbphrt`ok, hr bvbo fsm`ok, ehw khhg teb jhrbnfst-

    brs weh lfgb teb jhrbnfsts rbfddy frb. Bvbry gfy, nhrphrft`hos fog

    khvbrolbots pfy jhr jhrbnfsts teft lfy ib prbsn`bot hr whrtedbss hr

    shlbte`ok `o ibtwbbo. Fog bvbry gfy, fdd hj us‗dbfgbrs hj oft`hos,

    nhrphrftb bxbnut`vbs, `ovbsthrs, fog vhtbrs‗lfmb nr`t`nfd gbn`s`hos

    ho teb ifs`s hj jhrbnfsts wehsb qufd`ty `s uomohwo. Ifsbifdd lfo-

    fkbrs whudgo‐t grbfl hj kbtt`ok hut teb nebnmihhm th e`rb f pdfybr

    ` e d ` j ` ` B j

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    9/29

    7 supbrjhrbnfst`ok

    th teb jhrbnfstbrs weh ebdp us lfmb gbn`s`hos teft lfttbr jfr lhrb

    tefo foy ifsbifdd kflb, wb‐rb nhotbot th ib `kohrfot.>

    @o teft d`ket, rbdy`ok ho I`dd Jdfnm‐s jhrbnfsts dhhms qu`tb rbfsho-fidb. @ogbbg, rbdy`ok ho teb jhrbnfsts hj lfoy rbfgbrs hj te`s ihhm

    lfy prhvb qu`tb rbfshofidb, jhr `t turos hut teft jhrbnfst`ok `s oht f

    —yhu efvb `t hr yhu gho‐t“ tfdbot. @t `s f sm`dd teft nfo ib nudt`vftbg.

    We`s ihhm w`dd sehw yhu ehw.

    WEB HOB FIHRW WEB NE@LQ

    @ wfot th sph`d teb chmb, sh @‐dd k`vb fwfy teb puone d`ob3 teb fvbrfkb

    bxpbrt wfs rhukedy fs fnnurftb fs f gfrt-terhw`ok ne`lpfozbb.

     Phu‐vb prhifidy ebfrg teft hob ibjhrb. @t‐s jflhus‗`o shlb n`r-

    ndbs, `ojflhus. @t efs phppbg up `o teb Obw Phrm W`lbs , teb [fdd

    ^trbbt Churofd , teb J`ofon`fd W`lbs , teb Bnhohl`st , fog htebr hutdbts

    frhuog teb whrdg. @t khbs d`mb te`s3 F rbsbfrnebr kftebrbg f i`k krhup

    hj bxpbrts‗fnfgbl`ns, puog`ts, fog teb d`mb‗th lfmb tehusfogs

    hj prbg`nt`hos fihut teb bnhohly, sthnms, bdbnt`hos, wfrs, fog htebr

    `ssubs hj teb gfy. W`lb pfssbg, fog webo teb rbsbfrnebr nebnmbg teb

    fnnurfny hj teb prbg`nt`hos, eb jhuog teft teb fvbrfkb bxpbrt g`g

    fihut fs wbdd fs rfoghl kubss`ok. Bxnbpt teft‐s oht teb puone d`ob

    ibnfusb —rfoghl kubss`ok“ `so‐t juooy. Web puone d`ob `s fihut fgfrt-terhw`ok ne`lpfozbb. Ibnfusb ne`lpfozbbs frb juooy.

    @ fl teft rbsbfrnebr fog jhr f we`db @ g`go‐t l`og teb chmb. Ly

    stugy wfs teb lhst nhlprbebos`vb fssbsslbot hj bxpbrt cugklbot `o

    teb sn`bot`stn d`tbrfturb. @t wfs f dhok sdhk teft thhm fihut twboty ybfrs,

    jrhl 6057 th 9117, fog teb rbsudts wbrb jfr r`nebr fog lhrb nhostrun-

    t`vb tefo teb puone d`ob sukkbstbg. Iut @ g`go‐t l`og teb chmb ibnfusb

    `t rf`sbg fwfrbobss hj ly rbsbfrne (fog, ybs, sn`bot`sts sfvhr teb`r stj-

    ` j j ) F g @ dj e g g e dg —g

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    10/29

      Fo Hpt`l`st`n ^mbpt`n 2

    @ fdsh g`go‐t l`og ibnfusb teb chmb lfmbs f vfd`g ph`ot. Hpbo

    foy obwspfpbr, wftne foy WZ obws sehw, fog yhu stog bxpbrts weh

    jhrbnfst weft‐s nhl`ok. ^hlb frb nfut`hus. Lhrb frb ihdg fog nhost-gbot. F efogjud ndf`l th ib Hdylp`fo v`s`hofr`bs fidb th sbb gbnfgbs

    `oth teb juturb. [`te jbw bxnbpt`hos, teby frb oht `o jrhot hj teb

    nflbrfs ibnfusb teby phssbss foy prhvbo sm`dd ft jhrbnfst`ok. Fnnu-

    rfny ̀ s sbdghl bvbo lbot`hobg. Hdg jhrbnfsts frb d`mb hdg obws‗shho

    jhrkhttbo‗fog puog`ts frb fdlhst obvbr fsmbg th rbnhon`db weft

    teby sf`g w`te weft fntufddy efppbobg. Web hob uogbo`fidb tfdbot

    teft tfdm`ok ebfgs efvb `s teb`r sm`dd ft tbdd`ok f nhlpbdd`ok sthry w`tenhov`nt`ho, fog teft `s bohuke. Lfoy efvb ibnhlb wbfdtey pbggd`ok

    jhrbnfst`ok hj uotbstbg vfdub th nhrphrftb bxbnut`vbs, khvbrolbot hj-

    stn`fds, fog hrg`ofry pbhpdb weh whudg obvbr te`om hj swfddhw`ok

    lbg`n`ob hj uomohwo bjstnfny fog sfjbty iut weh rhut`obdy pfy jhr

    jhrbnfsts teft frb fs gui`hus fs bd`x`rs shdg jrhl teb ifnm hj f wfkho.

    Websb pbhpdb‗fog teb`r nusthlbrs‗gbsbrvb f ougkb `o teb r`is. @

     wfs efppy th sbb ly rbsbfrne usbg th k`vb t̀ th tebl.

    Iut @ rbfd`zbg teft fs whrg hj ly whrm sprbfg, ̀ ts fppfrbot lbfo-

    `ok wfs lutft`ok. [eft ly rbsbfrne efg sehwo wfs teft teb fvbrfkb

    bxpbrt efg ghob d`ttdb ibttbr tefo kubss`ok ho lfoy hj teb phd`t`nfd

    fog bnhohl`n qubst`hos @ efg phsbg. —Lfoy“ ghbs oht bqufd fdd. @t

     wfs bfs`bst th ibft nefonb ho teb sehrtbst-rfokb qubst`hos teft hody

    rbqu`rbg dhhm`ok hob ybfr hut, fog fnnurfny jbdd hjj teb jurtebr hut bx-pbrts tr`bg th jhrbnfst‗fpprhfne`ok teb gfrt-terhw`ok-ne`lpfozbb

    dbvbd terbb th stvb ybfrs hut. Weft wfs fo `lphrtfot stog`ok. @t tbdds

    us shlbte`ok fihut teb d`l`ts hj bxpbrt`sb `o f nhlpdbx whrdg‗fog

    teb d`l`ts ho weft `t l`ket ib phss`idb jhr bvbo supbrjhrbnfstbrs th

    fne`bvb. Iut fs `o teb ne`dgrbo‐s kflb hj —tbdbpehob,“ `o we`ne f

    perfsb `s we`spbrbg th hob ne`dg weh pfssbs `t ho th fohtebr, fog

    sh ho, fog bvbryhob `s sehnmbg ft teb bog th g`snhvbr ehw lune `t

    e e g e d id g ` e dd

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    11/29

    : supbrjhrbnfst`ok

    bxpbrt jhrbnfsts frb usbdbss,“ we`ne `s ohosbosb. ^hlb vfr`ft`hos wbrb

    bvbo nrugbr‗d`mb —bxpbrts mohw oh lhrb tefo ne`lpfozbbs.“ Ly

    rbsbfrne efg ibnhlb f ifnmsthp rbjbrbonb jhr o`e`d`sts weh sbb tebjuturb fs `oebrbotdy uoprbg`ntfidb fog mohw-ohte`ok phpud`sts weh

    `os`st ho prbnbg`ok —bxpbrt“ w`te —sh-nfddbg.“

    ^h @ t`rbg hj teb chmb. Ly rbsbfrne g`g oht supphrt tebsb lhrb

    bxtrblb nhondus`hos, ohr g`g @ jbbd foy fjsto`ty jhr tebl. Whgfy, teft

    `s fdd teb lhrb trub.

    Webrb ̀ s pdboty hj rhhl th stfmb hut rbfshofidb phs`t`hos ibtwbbo

    teb gbiuombrs fog teb gbjbogbrs hj bxpbrts fog teb`r jhrbnfsts. Hoteb hob efog, teb gbiuombrs efvb f ph`ot. Webrb frb sefgy pbggdbrs

    hj qubst`hofidb `os`kets `o teb jhrbnfst`ok lfrmbtpdfnb. Webrb frb

    fdsh d`l`ts th jhrbs`ket teft lfy cust oht ib surlhuotfidb. Hur gbs`rb

    th rbfne `oth teb juturb w`dd fdwfys bxnbbg hur krfsp. Iut gbiuom-

    brs kh thh jfr webo teby g`sl`ss fdd jhrbnfst`ok fs f jhhd‐s brrfog.

    @ ibd`bvb `t `s phss`idb th sbb `oth teb juturb, ft dbfst `o shlb s`tuf-

    t`hos fog th shlb bxtbot, fog teft foy ̀ otbdd`kbot, hpbo-l`ogbg, fog

    efrgwhrm`ok pbrsho nfo nudt`vftb teb rbqu`s`tb sm`dds.

    Nfdd lb fo —hpt`l`st`n smbpt`n.“

    WEB ^MBQW@N

    Wh uogbrstfog teb —smbpt`n“ efdj hj teft dfibd, nhos`gbr f yhuok Wu-

    o`s`fo lfo puse`ok f whhgbo efognfrt dhfgbg w`te jru`ts fog vbk-

    btfidbs ghwo f gusty rhfg th f lfrmbt `o teb Wuo`s`fo thwo hj ^`g`

    Ihuz`g. [ebo teb lfo wfs terbb, e`s jftebr g`bg. Eb supphrts e`s

    jfl`dy iy ihrrhw`ok lhoby th stdd e`s nfrt, ehp`ok th bfro bohuke

    sbdd`ok teb prhgunb th pfy hjj teb gbit fog efvb f d`ttdb dbjt hvbr. @t‐s

    teb sflb kr`og bvbry gfy. Iut te`s lhro`ok, teb phd`nb fpprhfne teb

    g e ‐ ` m e` d i e e ` d g

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    12/29

      Fo Hpt`l`st`n ^mbpt`n ;

    eb efs oh lhoby. F phd`nbwhlfo sdfps e`l fog `osudts e`s gbfg jf-

    tebr. Weby tfmb e`s snfdbs fog e`s nfrt. Web lfo khbs th f thwo hjstnb

    th nhlpdf`o. Eb `s thdg teb hjstn`fd `s iusy `o f lbbt`ok. Eul`d`ftbg,jur`hus, phwbrdbss, teb lfo dbfvbs.

    Eb rbturos w`te jubd. Huts`gb teb thwo hjstnb eb ghusbs e`lsbdj,

    d`kets f lftne, fog iuros.

    Hody teb nhondus`ho hj te`s sthry `s uousufd. Webrb frb nhuotdbss

    phhr strbbt vboghrs `o Wuo`s`f fog fnrhss teb Frfi whrdg. Qhd`nb nhr-

    rupt`ho `s r`jb, fog eul`d`ft`hos d`mb tehsb `o`ntbg ho te`s lfo frb

    f gf`dy hnnurrbonb. Weby lfttbr th oh hob fs`gb jrhl teb phd`nb fogteb`r v`nt`ls.

    Iut te`s pfrt`nudfr eul`d`ft`ho, ho Gbnblibr 6;, 9161, nfusbg

    Lheflbg Ihufz`z`, fkbg twboty-s`x, th sbt e`lsbdj ho strb, fog

    Ihufz`z`‐s sbdj-`llhdft`ho spfrmbg prhtbsts. Web phd`nb rbsphogbg

     w`te typ`nfd irutfd`ty. Web prhtbsts sprbfg. Ehp`ok th fssufkb teb

    puid`n, teb g`ntfthr hj Wuo`s`f, Qrbs`gbot S`ob bd- Fi`g`ob Ibo Fd`,

    v`s`tbg Ihufz`z` `o teb ehsp`tfd.

    Ihufz`z` g`bg ho Cfoufry 7, 9166. Web uorbst krbw. Ho Cfoufry

    67, Ibo Fd` bg th f nusey bx`db `o ^fug` Frfi`f, bog`ok e`s twboty-

    terbb-ybfr mdbpthnrfny.

    Web Frfi whrdg wftnebg, stuoobg. Webo prhtbsts bruptbg `o

    Bkypt, D`iyf, ^yr`f, Chrgfo, Muwf`t, fog Iferf`o. Fjtbr terbb gb-

    nfgbs `o phwbr, teb Bkypt`fo g`ntfthr Ehso` Luifrfm wfs gr`vbojrhl hjstnb. Bdsbwebrb, prhtbsts swbddbg `oth rbibdd`hos, rbibdd`hos `oth

    n`v`d wfrs. We`s wfs teb Frfi ^pr`ok‗fog `t stfrtbg w`te hob phhr

    lfo, oh g`jjbrbot jrhl nhuotdbss htebrs, ib`ok efrfssbg iy phd`nb, fs

    sh lfoy efvb ibbo, ibjhrb fog s`onb, w`te oh fppfrbot r`ppdb bjjbnts.

    @t `s hob te`ok th dhhm ifnmwfrg fog smbtne f ofrrft`vb frn, fs @

    g`g ebrb, nhoobnt`ok Lheflbg Ihufz`z` th fdd teb bvbots teft hwbg

    hut hj e`s dhobdy prhtbst. Whl Jr`bglfo, d`mb lfoy bd`tb puog`ts, `s

    m`dd g e j ` ` d d ` e L`ggd B

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    13/29

    5 supbrjhrbnfst`ok

    Obw Phrm W`lbs   nhrrbsphogbot `o Dbifoho. Iut nhudg bvbo Whl

    Jr`bglfo, `j eb efg ibbo prbsbot teft jftfd lhro`ok, efvb pbbrbg

    `oth teb juturb fog jhrbsbbo teb sbdj-`llhdft`ho, teb uorbst, teb thp-pd`ok hj teb Wuo`s`fo g`ntfthr, fog fdd teft jhddhwbg8 Hj nhursb oht.

    Oh hob nhudg. Lfyib, k`vbo ehw lune Jr`bglfo mobw fihut teb

    rbk`ho, eb whudg efvb lusbg teft phvbrty fog uoblpdhylbot wbrb

    e`ke, teb oulibr hj gbspbrftb yhuok pbhpdb wfs krhw`ok, nhrrup-

    t`ho wfs rflpfot, rbprbss`ho wfs rbdbotdbss, fog tebrbjhrb Wuo`s`f

    fog htebr Frfi nhuotr`bs wbrb phwgbr mbks wf`t`ok th idhw. Iut

    fo hisbrvbr nhudg efvb grfwo bxfntdy teb sflb nhondus`ho teb ybfribjhrb. Fog teb ybfr ibjhrb teft. @ogbbg, yhu nhudg efvb sf`g teft

    fihut Wuo`s`f, Bkypt, fog sbvbrfd htebr nhuotr`bs jhr gbnfgbs. Weby

    lfy efvb ibbo phwgbr mbks iut teby obvbr idbw‗uot`d Gbnbl-

    ibr 6;, 9161, webo teb phd`nb pusebg teft hob phhr lfo thh jfr.

    @o 60;9 teb Flbr`nfo lbtbhrhdhk`st Bgwfrg Dhrboz wrhtb

    f pfpbr w`te fo frrbst`ok t`tdb3 —Qrbg`ntfi`d`ty3 Ghbs teb Jdfp hj f

    Iuttbry‐s [`oks `o Irfz`d ^bt Hjj f Whrofgh `o Wbxfs8“ F gbnfgb

    bfrd`br, Dhrboz efg g`snhvbrbg iy fnn`gbot teft t`oy gftf botry vfr`f-

    t`hos `o nhlputbr s`ludft`hos hj wbftebr pfttbros‗d`mb rbpdfn`ok

    1.21:69; w`te 1.21:‗nhudg prhgunb grflft`nfddy g`jjbrbot dhok-

    tbrl jhrbnfsts. @t wfs fo ̀ os`ket teft whudg ̀ osp`rb —nefhs tebhry“3 ̀ o

    ohod`obfr systbls d`mb teb ftlhspebrb, bvbo slfdd nefokbs `o `o`t`fd

    nhog`t`hos nfo luserhhl th bohrlhus prhphrt`hos. ^h, `o pr`on`pdb,f dhob iuttbry `o Irfz`d nhudg fp `ts w`oks fog sbt hjj f throfgh `o

    Wbxfs‗bvbo tehuke swfrls hj htebr Irfz`d`fo iuttbr`bs nhudg fp

    jrfot`nfddy teb`r wehdb d`vbs fog obvbr nfusb f oht`nbfidb kust f jbw

    l`dbs fwfy. Hj nhursb Dhrboz g`go‐t lbfo teft teb iuttbry —nfusbs“

    teb throfgh `o teb sflb sbosb teft @ nfusb f w`obkdfss th irbfm webo

    @ e`t `t w`te f efllbr. Eb lbfot teft `j teft pfrt`nudfr iuttbry

    efgo‐t fppbg `ts w`oks ft teft lhlbot, teb uojftehlfidy nhlpdbx

    m j e ` ` g ` dg e i e g

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    14/29

      Fo Hpt`l`st`n ^mbpt`n 0

     Frfi ^pr`ok l`ket obvbr efvb efppbobg, ft dbfst oht webo fog fs `t

    g`g, `j teb phd`nb efg cust dbt Lheflbg Ihufz`z` sbdd e`s jru`ts fog

    vbkbtfidbs teft lhro`ok `o 9161.Bgwfrg Dhrboz se`jtbg sn`bot`stn hp`o`ho thwfrg teb v`bw teft

    tebrb frb efrg d`l`ts ho prbg`ntfi`d`ty, f gbbpdy pe`dhshpe`nfd qubs-

    t`ho.7  Jhr nbotur`bs, sn`bot`sts efg supphsbg teft krhw`ok mohwd-

    bgkb lust dbfg th krbftbr prbg`ntfi`d`ty ibnfusb rbfd`ty wfs d`mb f

    ndhnm‗fo fwbshlbdy i`k fog nhlpd`nftbg ndhnm iut st`dd f ndhnm‗

    fog teb lhrb sn`bot`sts dbfrobg fihut `ts `oofrgs, ehw teb kbfrs

    kr`og thkbtebr, ehw teb wb`kets fog spr`oks juont`ho, teb ibttbr tebynhudg nfpturb `ts hpbrft`hos w`te gbtbrl`o`st`n bquft`hos fog prbg`nt

     weft t̀ whudg gh. @o 6567 teb Jrbone lfteblft`n`fo fog fstrhohlbr

    Q`brrb-^`lho Dfpdfnb thhm te`s grbfl th `ts dhk`nfd bxtrblb3

    [b lfy rbkfrg teb prbsbot stftb hj teb uo`vbrsb fs teb bjjbnt

    hj `ts pfst fog teb nfusb hj `ts juturb. Fo `otbddbnt we`ne ft f

    nbrtf`o lhlbot whudg mohw fdd jhrnbs teft sbt ofturb `o lh-

    t`ho, fog fdd phs`t`hos hj fdd `tbls hj we`ne ofturb `s nhlphsbg,

    `j te`s `otbddbnt wbrb fdsh vfst bohuke th suil`t tebsb gftf th

    fofdys`s, `t whudg blirfnb `o f s`okdb jhrludf teb lhvblbots

    hj teb krbftbst ihg`bs hj teb uo`vbrsb fog tehsb hj teb t`o`bst

    fthl= jhr sune fo `otbddbnt ohte`ok whudg ib uonbrtf`o fog teb

     juturb cust d`mb teb pfst whudg ib prbsbot ibjhrb `ts bybs.

    Dfpdfnb nfddbg e`s `lfk`ofry bot`ty f —gblho.“ @j `t mobw bvbry-

    te`ok fihut teb prbsbot, Dfpdfnb tehuket, t̀ nhudg prbg`nt bvbryte`ok

    fihut teb juturb. @t whudg ib hlo`sn`bot.2

    Dhrboz phurbg nhdg rf`owftbr ho teft grbfl. @j teb ndhnm syl-

    ihd`zbs pbrjbnt Dfpdfnbfo prbg`ntfi`d`ty, `ts hpphs t̀b `s teb Dhrboz`fo

    ndhug. E`ke snehhd sn`bonb tbdds us teft ndhugs jhrl webo wftbr

    d g g ` d We` g ` d i

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    15/29

    61 supbrjhrbnfst`ok

    ho nhlpdbx jbbgifnm `otbrfnt`hos flhok grhpdbts. Wh nfpturb tebsb

    `otbrfnt`hos, nhlputbr lhgbdbrs obbg bquft`hos teft frb e`kedy sbo-

    s`t`vb th t`oy iuttbry-bjjbnt brrhrs `o gftf nhddbnt`ho. ^h bvbo `j wbdbfro fdd teft `s mohwfidb fihut ehw ndhugs jhrl, wb w`dd oht ib fidb

    th prbg`nt teb sefpb f pfrt`nudfr ndhug w`dd tfmb. [b nfo hody wf`t

    fog sbb. @o hob hj e`sthry‐s krbft `rho`bs, sn`bot`sts thgfy mohw vfstdy

    lhrb tefo teb`r nhddbfkubs f nbotury fkh, fog phssbss vfstdy lhrb

    gftf-nruone`ok phwbr, iut teby frb lune dbss nhostgbot `o teb prhs-

    pbnts jhr pbrjbnt prbg`ntfi`d`ty.

    We`s `s f i`k rbfsho jhr teb —smbpt`n“ efdj hj ly —hpt`l`st`n smbp-t`n“ stfonb. [b d`vb `o f whrdg webrb teb fnt`hos hj hob obfrdy phw-

    brdbss lfo nfo efvb r`ppdb bjjbnts frhuog teb whrdg‗r`ppdbs teft

    fjjbnt us fdd th vfry`ok gbkrbbs. F whlfo d`v`ok `o f Mfosfs N`ty

    suiuri lfy te`om Wuo`s`f `s fohtebr pdfobt, fog ebr d`jb efs oh nho-

    obnt`ho th `t, iut `j seb wbrb lfrr`bg th fo f`r jhrnb ofv`kfthr weh

    `bs hut hj teb obfriy [e`tblfo F`r Jhrnb Ifsb, seb l`ket ib sur-

    pr`sbg th dbfro teft hob hisnurb Wuo`s`fo‐s fnt`hos dbg th prhtbsts,

    teft dbg th r`hts, teft dbg th teb thppd`ok hj f g`ntfthr, teft dbg th

    prhtbsts `o D`iyf, teft dbg th f n`v`d wfr, teft dbg th teb 9169 OFWH

    `otbrvbot`ho, teft dbg th ebr eusifog ghgk`ok fot`f`rnrfjt strb hvbr

    Wr`phd`. Weft‐s fo bfs`dy trfnbfidb nhoobnt`ho. Hjtbo teb nhoobnt`hos

    frb efrgbr th spht, iut teby frb fdd frhuog us, `o te`oks d`mb teb pr`nb

     wb pfy ft teb kfs stft`ho hr teb dfyhjjs ghwo teb strbbt. @o f whrdg webrb f iuttbry `o Irfz`d nfo lfmb teb g`jjbrbonb ibtwbbo cust fo-

    htebr suooy gfy `o Wbxfs fog f throfgh tbfr`ok terhuke f thwo, `t‐s

    l`sku`gbg th te`om foyhob nfo sbb vbry jfr `oth teb juturb.:

    WEB HQW@L@^W

    I ` ` e` ` e d` ` g` i`d` g `

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    16/29

      Fo Hpt`l`st`n ^mbpt`n 66

    Nrfom up teb l`nrhsnhpb ho f gfy `o teb d`jb hj teft whlfo

    d`v`ok `o f Mfosfs N`ty suiuri3 Ft :3>1 `o teb lhro`ok, seb grhps

    pfpbrs `oth f ir`bjnfsb, kbts `o ebr nfr, gr`vbs ebr usufd rhutb th whrm,fog pfrms ghwothwo. Fs seb ghbs bvbry wbbmgfy lhro`ok, seb wfdms

    pfst teb stftubs hj d`hos fog `oth teb Krbbm-`osp`rbg hjstnb iu`dg`ok

    hj teb Mfosfs N`ty D`jb @osurfonb Nhlpfoy. Ft ebr gbsm, seb whrms

    ho sprbfgsebbts jhr f we`db, pfrt`n`pftbs `o f nhojbrbonb nfdd ft 613>1,

    spbogs f jbw l`outbs irhws`ok ho teb Flfzho wbis`tb, fog foswbrs

    b-lf`ds uot`d 66321. Webo seb wfdms th f d`ttdb @tfd`fo rbstfurfot th

    efvb duone w`te ebr s`stbr.We`s whlfo‐s d`jb `s `oubonbg iy lfoy uoprbg`ntfidb jfnthrs‗

    jrhl teb dhttbry t`nmbt `o ebr pursb th teb Frfi ^pr`ok teft rbsudts `o

    ebr eusifog y`ok l`ss`hos hvbr D`iyf th teb jfnt teft teb pr`nb hj

    kfs cust wbot up stvb nbots f kfddho ibnfusb tebrb wfs f nhup `o shlb

    nhuotry seb‐s obvbr ebfrg hj‗iut tebrb `s fs lune hr lhrb teft `s

    qu`tb prbg`ntfidb. [ey g`g seb dbfvb ehlb ft :3>18 ^eb g`go‐t wfot

    th kbt stunm `o ruse ehur. Hr th put teft fohtebr wfy, seb prbg`ntbg

    teft trfjstn whudg ib lune ebfv`br dftbr‗fog seb wfs fdlhst nbr-

    tf`ody r`ket ibnfusb ruse ehur `s e`kedy prbg`ntfidb. [ebo seb grhvb,

    seb fot`n`pftbg htebr gr`vbrs‐ ibefv`hr nhostfotdy3 teby w`dd sthp ft

    teb `otbrsbnt`ho webo teb d`ket `s rbg= teby w`dd stfy `o teb`r dfobs

    fog s`kofd ibjhrb turo`ok. ^eb bxpbntbg teb pbhpdb weh sf`g teby

     whudg ch`o teb 613>1 nhojbrbonb nfdd th gh sh, fog seb wfs r`ket.^eb frrfokbg th lbbt ebr s`stbr ft ohho ft teb rbstfurfot ibnfusb teb

    rbstfurfot‐s phstbg ehurs `og`nftbg `t whudg hpbo tebo, fog phstbg

    ehurs frb f rbd`fidb ku`gb.

     [b lfmb luogfob prbg`nt`hos d`mb tebsb rhut`obdy, we`db htebrs

     cust fs rhut`obdy lfmb prbg`nt`hos teft sefpb hur d`vbs. [ebo teb

     whlfo turobg ebr nhlputbr ho, seb `onrbfsbg bdbntr`n`ty nhosulp-

    t`ho ̀ o Mfosfs N`ty iy f i`t, fs bvbry htebr whrmbr ibb g`g teft lhro-

    ` g dd ` d e g g g e g

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    17/29

    69 supbrjhrbnfst`ok

    prhidbls ibnfusb bdbntr`n`ty prhgunbrs fot`n`pftb tebsb surkbs fog

    vfry teb`r hutput fnnhrg`okdy. [ebo teb whlfo wbot th Flfzho,

    teb wbis`tb e`ked`ketbg nbrtf`o prhgunts `t tehuket seb whudg d`mb, fjhrbnfst gbr`vbg jrhl ebr pfst purnefsbs fog irhws`ok fog teft hj

    l`dd`hos hj htebrs. [b nhostfotdy bonhuotbr prbg`nt`vb hpbrft`hos d`mb

    teft ho teb @otbrobt‗Khhkdb pbrshofd`zbs sbfrne rbsudts iy putt`ok

     weft `t te`oms yhu w`dd stog lhst `otbrbst`ok ho thp‗iut teby hpbr-

    ftb sh slhhtedy wb rfrbdy oht`nb. Fog tebo tebrb‐s teb whlfo‐s whrm-

    pdfnb. Web Mfosfs N`ty D`jb @osurfonb Nhlpfoy `s `o teb ius`obss hj

    jhrbnfst`ok g`sfi`d`ty fog gbfte, fog `t ghbs f khhg chi. Weft ghbso‐tlbfo `t mohws prbn`sbdy webo @ w`dd g`b, iut `t ghbs efvb f khhg `gbf

    hj ehw dhok shlbhob hj ly fkb fog prhstdb‗sbx, ̀ onhlb, d`jbstydb‗`s

    d`mbdy th d`vb. Mfosfs N`ty D`jb wfs jhuogbg `o 6502. @j `ts fntufr`bs

     wbrbo‐t khhg jhrbnfstbrs, t̀ whudg efvb khob ifomrupt dhok fkh.

    ^h lune hj hur rbfd`ty `s te`s prbg`ntfidb, hr lhrb sh. @ cust

    Khhkdbg thlhrrhw‐s suor`sb fog suosbt t`lbs jhr Mfosfs N`ty, L`s-

    shur`, fog kht tebl ghwo th teb l`outb. Wehsb jhrbnfsts frb rbd`fidb,

     webtebr teby frb jhr thlhrrhw, teb gfy fjtbr, hr stjty ybfrs jrhl ohw.

    Web sflb `s trub hj t`gbs, bnd`psbs, fog pefsbs hj teb lhho. Fdd nfo

    ib prbg`ntbg jrhl ndhnmd`mb sn`bot`stn dfws w`te bohuke prbn`s`ho th

    sft`sjy Dfpdfnb‐s jhrbnfst`ok gblho.

    Hj nhursb bfne hj tebsb phnmbts hj prbg`ntfi`d`ty nfo ib firuptdy

    puonturbg. F khhg rbstfurfot `s vbry d`mbdy th hpbo `ts ghhrs webo `tsfys `t w`dd, iut `t lfy oht, jhr foy oulibr hj rbfshos, jrhl f lfo-

    fkbr sdbbp`ok dftb, th strb, ifomruptny, pfogbl`n, oundbfr wfr, hr f

    peys`ns bxpbr`lbot fnn`gbotfddy nrbft`ok f idfnm ehdb teft sunms up

    teb shdfr systbl. Web sflb `s trub hj foyte`ok bdsb. Bvbo tehsb stjty-

    ybfr suor`sb fog suosbt jhrbnfsts nhudg ib hjj shlbweft `j, shlb-

    t`lb `o teb obxt stjty ybfrs, f lfss`vb spfnb rhnm iulps Bfrte hjj

    `ts hri`t frhuog teb suo. Webrb frb oh nbrtf`ot`bs `o d`jb‗oht bvbo

    g e g `j ` i i`d` e ` `

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    18/29

      Fo Hpt`l`st`n ^mbpt`n 6>

    f ndhug-nhlput`ok obtwhrm fog teb blbrkbonb hj f juturb shn`bty

    sh puid`n-sp`r`tbg fog prhspbrhus teft teb stftb nfo ib juogbg w`te

    nefr`tfidb ghoft`hos.^h `s rbfd`ty ndhnmd`mb hr ndhug-d`mb8 @s teb juturb prbg`ntfidb hr

    oht8 Websb frb jfdsb g`nehthl`bs, teb strst hj lfoy wb w`dd bonhuotbr.

     [b d`vb `o f whrdg hj ndhnms fog ndhugs fog f vfst culidb hj htebr

    lbtfpehrs. Roprbg`ntfi`d`ty fog prbg`ntfi`d`ty nhbx`st uobfs`dy `o

    teb `otr`nftbdy `otbrdhnm`ok systbls teft lfmb up hur ihg`bs, hur

    shn`bt`bs, fog teb nhslhs. Ehw prbg`ntfidb shlbte`ok `s gbpbogs ho

     weft wb frb try`ok th prbg`nt, ehw jfr `oth teb juturb, fog uogbr weft n`rnulstfonbs.

    Dhhm ft Bgwfrg Dhrboz‐s stbdg. [bftebr jhrbnfsts frb typ`nfddy

    qu`tb rbd`fidb, uogbr lhst nhog`t`hos, dhhm`ok f jbw gfys febfg,

    iut teby ibnhlb `onrbfs`okdy dbss fnnurftb terbb, jhur, fog stvb

    gfys hut. Lune ibyhog f wbbm, wb l`ket fs wbdd nhosudt teft gfrt-

    terhw`ok ne`lpfozbb. ^h wb nfo‐t sfy teft wbftebr `s prbg`ntfidb

    hr oht, hody teft wbftebr `s prbg`ntfidb th shlb bxtbot uogbr shlb

    n`rnulstfonbs‗fog wb lust ib vbry nfrbjud webo wb try th ib lhrb

    prbn`sb tefo teft. Wfmb shlbte`ok fs sbbl`okdy s`lpdb fs teb rbdft`ho-

    se`p ibtwbbo t`lb fog prbg`ntfi`d`ty3 `t `s kbobrfddy trub teft teb jur-

    tebr wb try th dhhm ̀ oth teb juturb, teb efrgbr ̀ t ̀ s th sbb. Iut tebrb nfo

    ib prhdhokbg bxnbpt`hos th teb rudb. Qrbg`nt`ok teb nhot`ouft`ho hj f

    dhok iudd lfrmbt ̀ o sthnms nfo prhvb prhsttfidb jhr lfoy ybfrs‗uot`d`t suggbody prhvbs th ib yhur uogh`ok. Fog prbg`nt`ok teft g`ohsfurs

     whudg nhot`oub th prbs`gb ft teb thp hj teb jhhg nef`o wfs f sfjb ibt

    jhr tbos hj l`dd`hos hj ybfrs‗uot`d fo fstbrh`g sbt hjj f nftfndysl

    teft hpbobg up bnhdhk`nfd o`nebs jhr f t`oy lfllfd teft bvbotufddy

    bvhdvbg `oth f spbn`bs teft tr`bs th prbg`nt teb juturb. Dfws hj peys`ns

    fs`gb, tebrb frb oh uo`vbrsfd nhostfots, sh sbpfrft`ok teb prbg`ntfidb

    jrhl teb uoprbg`ntfidb ̀ s g`jstnudt whrm. Webrb‐s oh wfy frhuog ̀ t.

    L d ` m e i e We m d

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    19/29

    67 supbrjhrbnfst`ok

     wey wb mohw teft hob- fog twh-gfy jhrbnfsts frb typ`nfddy qu`tb fn-

    nurftb we`db b`ket-gfy jhrbnfsts frb oht. [`te tebsb fofdysbs, lbtbh-

    rhdhk`sts frb fidb th sefrpbo teb`r uogbrstfog`ok hj ehw wbftebr whrmsfog twbfm teb`r lhgbds. Webo teby try fkf`o. Jhrbnfst, lbfsurb, rb-

    v`sb. Ybpbft. @t‐s f obvbr-bog`ok prhnbss hj `onrblbotfd `lprhvblbot

    teft bxpdf`os wey wbftebr jhrbnfsts frb khhg fog sdhwdy kbtt`ok ibttbr.

    Webrb lfy ib d`l`ts th sune `lprhvblbots, ehwbvbr, ibnfusb wbftebr

    `s teb tbxtihhm `ddustrft`ho hj ohod`obfr`ty. Web jurtebr hut teb jhrb-

    nfstbr tr`bs th dhhm, teb lhrb hpphrtuo`ty tebrb `s jhr nefhs th fp `ts

    iuttbry w`oks fog idhw fwfy bxpbntft`hos. I`k dbfps `o nhlput`okphwbr fog nhot`oubg rbstoblbot hj jhrbnfst`ok lhgbds lfy ougkb teb

    d`l`ts f d`ttdb jurtebr `oth teb juturb iut tehsb fgvfonbs krfgufddy kbt

    efrgbr fog teb pfyhjjs ser`om thwfrg zbrh. Ehw khhg nfo `t kbt8 Oh

    hob mohws. Iut mohw`ok teb nurrbot d`l`ts `s t̀sbdj f sunnbss.

    @o sh lfoy htebr e`ke-stfmbs bogbfvhrs, jhrbnfstbrs frb krhp`ok

    `o teb gfrm. Weby efvb oh `gbf ehw khhg teb`r jhrbnfsts frb `o teb

    sehrt, lbg`ul, hr dhok tbrl‗fog oh `gbf ehw khhg teb`r jhrbnfsts

    nhudg ibnhlb. Ft ibst, teby efvb vfkub euonebs. Weft‐s ibnfusb teb

    jhrbnfst-lbfsurb-rbv`sb prhnbgurb hpbrftbs hody w`te`o teb rfrbstbg

    nhostobs hj e`ke-tbne jhrbnfst`ok, sune fs teb whrm hj lfnrhbnhoh-

    l`sts ft nbotrfd ifoms hr lfrmbt`ok fog stofon`fd prhjbss`hofds `o

    i`k nhlpfo`bs hr hp`o`ho phdd fofdysts d`mb Oftb ^`dvbr.; Lhrb hjtbo

    jhrbnfsts frb lfgb fog tebo . . . ohte`ok. Fnnurfny `s sbdghl gbtbr-l`obg fjtbr teb jfnt fog `s fdlhst obvbr ghob w`te sujstn`bot rbkudfr-

    `ty fog r`khr teft nhondus`hos nfo ib grfwo. Web rbfsho8 Lhstdy ̀ t‐s f

    gblfog-s`gb prhidbl3 Web nhosulbrs hj jhrbnfst`ok‗khvbrolbots,

    ius`obss, fog teb puid`n‗gho‐t gblfog bv`gbonb hj fnnurfny. ^h

    tebrb `s oh lbfsurblbot. [e`ne lbfos oh rbv`s`ho. Fog w`tehut

    rbv`s`ho, tebrb nfo ib oh `lprhvblbot. @lfk`ob f whrdg `o we`ne

    pbhpdb dhvb th ruo, iut teby efvb oh `gbf ehw jfst teb fvbrfkb pbrsho

    e j e i dg i e

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    20/29

      Fo Hpt`l`st`n ^mbpt`n 62

    teb kuo `s strbg, bog `t fjtbr f spbn`stbg g`stfonb‗fog tebrb frb oh

    `ogbpbogbot rfnb hjstn`fds fog t`lbmbbpbrs lbfsur`ok rbsudts. Ehw

    d`mbdy `s `t teft ruoo`ok t`lbs frb `lprhv`ok `o te`s whrdg8 Oht vbry. Frb teb ibst ruoobrs ruoo`ok fs jfst fs eulfo ib`oks frb peys`nfddy

    nfpfidb8 Fkf`o, prhifidy oht.

    —@ efvb ibbo strunm iy ehw `lphrtfot lbfsurblbot `s th `l-

    prhv`ok teb eulfo nhog`t`ho,“ I`dd Kftbs wrhtb. —Phu nfo fne`bvb

    `onrbg`idb prhkrbss `j yhu sbt f ndbfr khfd fog stog f lbfsurb teft w`dd

    gr`vb prhkrbss thwfrg teft khfd. . . . We`s lfy sbbl ifs`n, iut `t `s

    flfz`ok ehw hjtbo `t `s oht ghob fog ehw efrg `t `s th kbt r`ket.“5 Eb`s r`ket fihut weft `t tfmbs th gr`vb prhkrbss, fog `t `s surpr`s`ok ehw

    rfrbdy `t‐s ghob `o jhrbnfst`ok. Bvbo teft s`lpdb strst stbp‗sbtt`ok f

    ndbfr khfd‗efso‐t ibbo tfmbo.

     Phu l`ket te`om teb khfd hj jhrbnfst`ok `s th jhrbsbb teb juturb

    fnnurftbdy, iut teft‐s hjtbo oht teb khfd, hr ft dbfst oht teb shdb khfd.

    ^hlbt`lbs jhrbnfsts frb lbfot th botbrtf`o. We`om hj NOIN‐s C`l

    Nrflbr w`te e`s —ihhyfe!“ set`nm, hr Cheo LnDfuked`o, teb ehst

    hj Web LnDfuked`o Krhup, ibddhw`ok ft e`s pfobd`sts th prbg`nt teb

    d`mbd`ehhg hj fo bvbot —ho f snfdb jrhl zbrh th tbo, w`te zbrh rbprb-

    sbot`ok zbrh phss`i`d`ty fog tbo rbprbsbot`ok nhlpdbtb lbtfpeys`nfd

    nbrt`tugb!“ ^hlbt`lbs jhrbnfsts frb usbg th fgvfonb phd`t`nfd fkbo-

    gfs fog kfdvfo`zb fnt`ho‗fs fnt`v`sts ehpb th gh webo teby wfro

    hj dhhl`ok ehrrhrs uodbss wb nefokb hur wfys. Webrb `s fdsh grbss-th-`lprbss jhrbnfst`ok‗ we`ne `s weft ifoms gbd`vbr webo teby pfy

    f jflhus puog`t th tbdd wbfdtey nd`bots fihut teb kdhifd bnhohly `o

    9121. Fog shlb jhrbnfsts frb lbfot th nhljhrt‗iy fssur`ok teb

    fug`bonb teft teb`r ibd`bjs frb nhrrbnt fog teb juturb w`dd uojhdg fs

    bxpbntbg. Qfrt`sfos frb jhog hj tebsb jhrbnfsts. Weby frb teb nhko`-

    t`vb bqu`vfdbot hj sd`pp`ok `oth f wfrl ifte.

    We`s culidb hj khfds `s sbdghl fnmohwdbgkbg, we`ne lfmbs `t

    g`jst d m` g g @ ‐

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    21/29

    6: supbrjhrbnfst`ok

     Fog ybt te`s stfkoft`ho `s f i`k rbfsho wey @ fl fo hpt`l`st`n  

    smbpt`n. [b mohw teft `o sh lune hj weft pbhpdb wfot th prbg`nt‗

    phd`t`ns, bnhohl`ns, stofonb, ius`obss, tbneohdhky, gf`dy d`jb‗prbg`nt-fi`d`ty bx`sts, th shlb gbkrbb, `o shlb n`rnulstfonbs. Iut tebrb `s sh

    lune bdsb wb gh oht mohw. Jhr sn`bot`sts, oht mohw`ok ̀ s bxn`t`ok. @t‐s

    fo hpphrtuo`ty th g`snhvbr= teb lhrb teft ̀ s uomohwo, teb krbftbr teb

    hpphrtuo`ty. Wefoms th teb jrfomdy qu`tb flfz`ok dfnm hj r`khr `o sh

    lfoy jhrbnfst`ok ghlf`os, te`s hpphrtuo`ty `s eukb. Fog th sb`zb `t,

    fdd wb efvb th gh ̀ s sbt f ndbfr khfd‗fnnurfny!‗fog kbt sbr`hus fihut

    lbfsur`ok.@‐vb ibbo gh`ok teft jhr lune hj ly nfrbbr. Web rbsbfrne teft

    prhgunbg teb gfrt-terhw`ok-ne`lpfozbb rbsudt wfs pefsb hob. Qefsb

    twh stfrtbg `o teb sullbr hj 9166, webo ly rbsbfrne (fog d`jb)

    pfrtobr Ifrifrf Lbddbrs fog @ dfuonebg teb Khhg Cugklbot Qrhcbnt

    fog `ov`tbg vhduotbbrs th s`ko up fog jhrbnfst teb juturb. I`dd Jdfnm

    rbsphogbg. ^h g`g f nhupdb hj tehusfog htebrs teft strst ybfr, fog

    tehusfogs lhrb `o teb jhur ybfrs teft jhddhwbg. Nuludft`vbdy, lhrb

    tefo twboty tehusfog `otbddbntufddy nur`hus dfypbhpdb tr`bg th stkurb

    hut `j prhtbsts `o Yuss`f whudg sprbfg, teb pr`nb hj khdg whudg pdul-

    lbt, teb O`mmb` whudg ndhsb fihvb 0,211, wfr whudg brupt ho teb

    Mhrbfo pbo`osudf, fog lfoy htebr qubst`hos fihut nhlpdbx, nefd-

    dbok`ok kdhifd `ssubs. Iy vfry`ok teb bxpbr`lbotfd nhog`t`hos, wb

    nhudg kfukb we`ne jfnthrs `lprhvbg jhrbs`ket, iy ehw lune, hvbr we`ne t`lb jrflbs, fog ehw khhg jhrbnfsts nhudg ibnhlb `j ibst

    prfnt`nbs wbrb dfybrbg ho bfne htebr. Df`g hut d`mb teft, `t shuogs

    s`lpdb. @t wfso‐t. @t wfs f gblfog`ok prhkrfl teft thhm teb tfdbots

    fog efrg whrm hj f ludt`g`sn`pd`ofry tbfl ifsbg ft teb Ro`vbrs`ty hj

    Nfd`jhro`f, Ibrmbdby, fog teb Ro`vbrs t̀y hj Qboosydvfo`f.

    I`k fs `t wfs, teb Khhg Cugklbot Qrhcbnt (KCQ) wfs hody pfrt hj

    f lune dfrkbr rbsbfrne bjjhrt sphoshrbg iy teb @otbdd`kbonb Fgvfonbg

    Y e Q c F ` ` (@FYQF) G ‐ i jj i e id g

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    22/29

      Fo Hpt`l`st`n ^mbpt`n 6;

    rbphrts th teb g`rbnthr hj Oft`hofd @otbdd`kbonb fog `ts chi `s th sup-

    phrt gfr`ok rbsbfrne teft prhl`sbs th lfmb Flbr`nfo `otbdd`kbonb

    ibttbr ft weft `t ghbs. Fog f i`k pfrt hj weft Flbr`nfo `otbdd`kbonbghbs `s jhrbnfst kdhifd phd`t`nfd fog bnhohl`n trbogs. Iy hob rhuke

    bst`lftb, teb Ro`tbg ^tftbs efs twboty tehusfog `otbdd`kbonb fofdysts

    fssbss`ok bvbryte`ok jrhl l`outb puzzdbs th lfchr bvbots sune fs

    teb d`mbd`ehhg hj fo @srfbd` sobfm fttfnm ho @rfo`fo oundbfr jfn`d`-

    t`bs hr teb gbpfrturb hj Krbbnb jrhl teb burhzhob.0 Ehw khhg `s fdd

    te`s jhrbnfst`ok8 Weft `s oht bfs`dy foswbrbg ibnfusb teb `otbdd`kbonb

    nhlluo`ty, d`mb sh lfoy lfchr prhgunbrs hj jhrbnfst`ok, efs obvbribbo mbbo ho spbog`ok lhoby th stkurb teft hut. Webrb frb vfr`hus

    rbfshos jhr teft rbduntfonb, shlb lhrb rbspbntfidb tefo htebrs, iut

     wb‐dd kbt `oth teft dftbr. [eft lfttbrs `s teft te`s jhrbnfst`ok `s nr`t`-

    nfd th oft`hofd sbnur`ty fog ybt d`ttdb nfo ib sf`g w`te foy nhostgbonb

    fihut ehw khhg `t `s, hr bvbo webtebr `t‐s fs khhg fs f ludt`i`dd`ho-

    ghddfr hpbrft`ho w`te twboty tehusfog pbhpdb sehudg ib. Wh nefokb

    teft, @FYQF nrbftbg f jhrbnfst`ok thuroflbot `o we`ne stvb sn`bo-

    t`stn tbfls dbg iy thp rbsbfrnebrs `o teb stbdg whudg nhlpbtb th kbo-

    brftb fnnurftb jhrbnfsts ho teb shrts hj thuke qubst`hos `otbdd`kbonb

    fofdysts gbfd w`te bvbry gfy. Web Khhg Cugklbot Qrhcbnt wfs hob

    hj tehsb stvb tbfls. Bfne tbfl whudg bjjbnt`vbdy ib `ts hwo rbsbfrne

    prhcbnt, jrbb th `lprhv`sb weftbvbr lbtehgs `t tehuket whudg whrm,

    iut rbqu`rbg th suil`t jhrbnfsts ft 0 f.l. bfstbro stfogfrg t`lbbvbry gfy jrhl ^bptblibr 9166 th Cuob 9162. Iy rbqu`r`ok tbfls

    th jhrbnfst teb sflb qubst`hos ft teb sflb t`lb, teb thuroflbot nrb-

    ftbg f dbvbd pdfy`ok stbdg‗fog f r`ne trhvb hj gftf fihut weft whrms,

    ehw wbdd, fog webo. Hvbr jhur ybfrs, @FYQF phsbg obfrdy stvb euo-

    grbg qubst`hos fihut whrdg fjjf`rs. W`lb jrflbs wbrb sehrtbr tefo `o

    ly bfrd`br rbsbfrne, w`te teb vfst lfchr`ty hj jhrbnfsts bxtbog`ok hut

    lhrb tefo hob lhote fog dbss tefo hob ybfr. @o fdd, wb kftebrbg hvbr

    `dd` ` g` `g d c g i e j

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    23/29

    65 supbrjhrbnfst`ok

     wb ibft teb nhotrhd krhup iy ;5%. KCQ fdsh ibft `ts uo`vbrs`ty-

    fjstd`ftbg nhlpbt`thrs, `ondug`ok teb Ro`vbrs`ty hj L`ne`kfo fog

    L@W, iy ebjty lfrk`os, jrhl >1% th ;1%, fog bvbo hutpbrjhrlbgprhjbss`hofd `otbdd`kbonb fofdysts w`te fnnbss th ndfss`stbg gftf. Fjtbr

    twh ybfrs, KCQ wfs gh`ok sh lune ibttbr tefo `ts fnfgbl`n nhlpbt`-

    thrs teft @FYQF grhppbg teb htebr tbfls.61

    @‐dd gbdvb `oth gbtf`ds dftbr, iut dbt‐s ohtb twh mby nhondus`hos teft

    blbrkb jrhl te`s rbsbfrne. Hob, jhrbs`ket `s rbfd. ^hlb pbhpdb‗

    pbhpdb d`mb I`dd Jdfnm‗efvb ̀ t ̀ o spfgbs. Weby frbo‐t kurus hr hrfndbs

     w`te teb phwbr th pbbr gbnfgbs `oth teb juturb, iut teby gh efvb frbfd, lbfsurfidb sm`dd ft cugk`ok ehw e`ke-stfmbs bvbots frb d`mbdy th

    uojhdg terbb lhotes, s`x lhotes, f ybfr, hr f ybfr fog f efdj `o fg-

    vfonb. Web htebr nhondus`ho `s weft lfmbs tebsb supbrjhrbnfstbrs sh

    khhg. @t‐s oht rbfddy weh teby frb. @t `s weft teby gh. Jhrbs`ket `so‐t f

    lystbr`hus k`jt ibsthwbg ft i`rte. @t `s teb prhgunt hj pfrt`nudfr wfys

    hj te`om`ok, hj kftebr`ok `ojhrlft`ho, hj upgft`ok ibd`bjs. Websb

    efi`ts hj tehuket nfo ib dbfrobg fog nudt`vftbg iy foy `otbdd`kbot,

    tehuketjud, gbtbrl`obg pbrsho. @t lfy oht bvbo ib fdd teft efrg th

    kbt stfrtbg. Hob rbsudt teft pfrt`nudfrdy surpr`sbg lb wfs teb bjjbnt

    hj f tuthr`fd nhvbr`ok shlb ifs`n nhonbpts teft wb‐dd bxpdhrb `o te`s

    ihhm fog frb sullfr`zbg `o teb Wbo Nhllfoglbots fppbog`x. @t

    thhm hody fihut s`xty l`outbs th rbfg fog ybt `t `lprhvbg fnnurfny

    iy rhukedy 61% terhuke teb bot`rb thuroflbot ybfr. Pbs, 61% lfyshuog lhgbst, iut `t wfs fne`bvbg ft sh d`ttdb nhst. Fog obvbr jhrkbt

    teft bvbo lhgbst `lprhvblbots `o jhrbs`ket lf`otf`obg hvbr t`lb

    fgg up. @ sphmb fihut teft w`te Ffrho Irhwo, fo futehr, f [fdd

    ^trbbt vbtbrfo, fog teb ne`bj r`sm lfofkbr ft FVY Nfp`tfd Lfofkb-

    lbot, f ebgkb juog w`te hvbr $611 i`dd`ho `o fssbts. —@t‐s sh efrg th

    sbb ibnfusb `t‐s oht grflft`n,“ eb sf`g, iut `j `t `s sustf`obg —`t‐s teb

    g`jjbrbonb ibtwbbo f nhos`stbot w`oobr weh‐s lfm`ok f d`v`ok, hr teb

    e ‐ ` i m dd e ` “66 F dg d m d

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    24/29

      Fo Hpt`l`st`n ^mbpt`n 60

     wb`kets fog flftburs, seb sf`g, `s teft teb ebfvywb`kets mohw teb

    g`jjbrbonb ibtwbbo f :1 ⁇ 71 ibt fog f 71 ⁇ :1 ibt.

     Fog ybt, `j `t‐s phss`idb th `lprhvb jhrbs`ket s`lpdy iy lbfsur-`ok, fog `j teb rbwfrgs hj `lprhvbg jhrbs`ket frb suistfot`fd, wey

    `so‐t lbfsur`ok stfogfrg prfnt`nb8 F i`k pfrt hj teb foswbr th teft

    qubst`ho d`bs `o teb psynehdhky teft nhov`onbs us wb mohw te`oks

     wb rbfddy gho‐t‗te`oks d`mb webtebr Whl Jr`bglfo `s fo fnnurftb

    jhrbnfstbr hr oht. @‐dd bxpdhrb te`s psynehdhky `o nefptbr 9. Jhr nbo-

    tur`bs, `t ehiidbg prhkrbss `o lbg`n`ob. [ebo peys`n`fos stofddy fn-

    nbptbg teft teb`r bxpbr`bonb fog pbrnbpt`hos wbrb oht rbd`fidb lbfoshj gbtbrl`o`ok webtebr f trbftlbot whrms, teby turobg th sn`bot`stn

    tbst`ok‗fog lbg`n`ob stofddy stfrtbg th lfmb rfp`g fgvfonbs. Web

    sflb rbvhdut`ho obbgs th efppbo `o jhrbnfst`ok.

    @t who‐t ib bfsy. Nefptbr > bxfl`obs weft `t tfmbs th tbst jhrb-

    nfst`ok fs r`khrhusdy fs lhgbro lbg`n`ob tbsts trbftlbots. @t‐s f i`k-

    kbr nefddbokb tefo `t lfy fppbfr. @o teb dftb 6051s @ whrmbg hut f

    lbtehghdhky fog nhoguntbg weft wfs, ft teb t`lb, teb i`kkbst tbst hj

    bxpbrt phd`t`nfd jhrbnfst`ok fnnurfny bvbr. Hob rbsudt, gbd`vbrbg lfoy

    ybfrs dftbr, wfs teb puone d`ob teft ohw lfmbs lb squ`rl. Iut fo-

    htebr g`snhvbry hj teft rbsbfrne g`go‐t rbnb`vb obfrdy fs lune fttbo-

    t`ho bvbo tehuke `t wfs jfr lhrb `lphrtfot3 hob krhup hj bxpbrts

    efg lhgbst iut rbfd jhrbs`ket. [eft lfgb teb g`jjbrbonb ibtwbbo teb

    bxpbrts w`te jhrbs`ket fog tehsb weh wbrb sh ehpbdbss teby grfkkbgteb fvbrfkb ghwo th teb dbvbd hj f gfrt-terhw`ok ne`lp8 @t wfso‐t

    shlb lyst`nfd k`jt hr fnnbss th `ojhrlft`ho htebrs g`go‐t efvb. Ohr

     wfs `t foy pfrt`nudfr sbt hj ibd`bjs. @ogbbg, w`te`o f qu`tb w`gb rfokb

    hj v`bws, weft  teby tehuket g`go‐t lfttbr. @t wfs ehw  teby tehuket.

    @osp`rbg `o pfrt iy teft `os`ket, @FYQF nrbftbg `ts uoprbnb-

    gbotbg jhrbnfst`ok thuroflbot. Nefptbr 7 `s teb sthry hj ehw teft

    efppbobg‗fog teb g`snhvbry hj supbrjhrbnfstbrs. [ey frb teby sh

    g We ` e e e 2 e e [e

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    25/29

    91 supbrjhrbnfst`ok

    l`ket suspbnt `t‐s `otbdd`kbonb teft lfmbs fdd teb g`jjbrbonb. @t‐s oht.

    Weby‐rb fdsh rblfrmfidy oulbrftb. D`mb I`dd Jdfnm, lfoy efvb fg-

    vfonbg gbkrbbs `o lfteblft`ns fog sn`bonb. ^h `s teb sbnrbt frnfoblfte8 Oh. Bvbo supbrjhrbnfstbrs weh frb nfrg-nfrry`ok lfteblft`-

    n`fos rfrbdy usb lune lfte. Weby fdsh tbog th ib obwscuom`bs weh

    stfy ho thp hj teb dftbst gbvbdhplbots fog rbkudfrdy upgftb teb`r jhrb-

    nfsts, sh yhu l`ket ib tblptbg th fttr`iutb teb`r sunnbss th spbog`ok

    bogdbss ehurs ho teb chi. Pbt teft thh whudg ib f l`stfmb.

    ^upbrjhrbnfst`ok ghbs rbqu`rb l`o`lul dbvbds hj `otbdd`kbonb,

    oulbrfny, fog mohwdbgkb hj teb whrdg, iut foyhob weh rbfgs sbr`-hus ihhms fihut psynehdhk`nfd rbsbfrne prhifidy efs tehsb prbrbq-

    u`s`tbs. ^h weft `s `t teft bdbvftbs jhrbnfst`ok th supbrjhrbnfst`ok8 Fs

     w`te teb bxpbrts weh efg rbfd jhrbs`ket `o ly bfrd`br rbsbfrne, weft

    lfttbrs lhst `s ehw  teb jhrbnfstbr te`oms. @‐dd gbsnr`ib te`s `o gbtf`d,

    iut irhfgdy spbfm`ok, supbrjhrbnfst`ok gblfogs te`om`ok teft `s

    hpbo-l`ogbg, nfrbjud, nur`hus, fog‗fihvb fdd‗sbdj-nr`t`nfd. @t fdsh

    gblfogs jhnus. Web m`og hj te`om`ok teft prhgunbs supbr`hr cugk-

    lbot ghbs oht nhlb bjjhrtdbssdy. Hody teb gbtbrl`obg nfo gbd`vbr `t

    rbfshofidy nhos`stbotdy, we`ne `s wey hur fofdysbs efvb nhos`stbotdy

    jhuog nhll`tlbot th sbdj-̀ lprhvblbot th ib teb strhokbst prbg`nthr

    hj pbrjhrlfonb.

    @o teb stofd nefptbrs, @‐dd rbshdvb fo fppfrbot nhotrfg`nt`ho ib-

    twbbo teb gblfogs hj khhg cugklbot fog bjjbnt`vb dbfgbrse`p,rbsphog th weft @ te`om frb teb twh strhokbst nefddbokbs th ly rb-

    sbfrne, fog nhondugb‗fpprhpr`ftbdy jhr f ihhm fihut jhrbnfst`ok‗

     w`te f nhos`gbrft`ho hj weft nhlbs obxt.

    F JHYBNF^W FIHRW JHYBNF^W@OK

    I i e` m e` ` dd e d d g g Fj dd

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    26/29

      Fo Hpt`l`st`n ^mbpt`n 96

    fdkhr`tels, fog I`k Gftf. Ft `ts nhrb, teb jhrbnfst`ok @ stugy `ovhdvbs

    suicbnt`vb cugklbot3 `t `s pbhpdb te`om`ok fog gbn`g`ok, ohte`ok

    lhrb. @so‐t teb t`lb jhr sune sdhppy kubsswhrm grfw`ok th f ndhsb8@o 6027, f ir`dd`fot psynehdhk`st, Qfud Lbbed wrhtb f slfdd ihhm

    teft nfusbg f i`k st`r.69 @t rbv`bwbg twboty stug`bs sehw`ok teft wbdd-

    `ojhrlbg bxpbrts prbg`nt`ok hutnhlbs‗ webtebr f stugbot whudg

    sunnbbg `o nhddbkb hr f pfrhdbb whudg ib sbot ifnm th pr`sho‗ wbrb

    oht fs fnnurftb fs s`lpdb fdkhr`tels teft fggbg up hicbnt`vb `og`nf-

    thrs d`mb fi`d`ty tbst snhrbs fog rbnhrgs hj pfst nhogunt. Lbbed‐s ndf`l

    upsbt lfoy bxpbrts, iut suisbqubot rbsbfrne‗ohw lhrb tefo twheuogrbg stug`bs‗efs sehwo teft `o lhst nfsbs stft`st`nfd fdkhr`tels

    ibft suicbnt`vb cugklbot, fog `o teb efogjud hj stug`bs webrb teby

    gho‐t, teby usufddy t`b. K`vbo teft fdkhr`tels frb qu`nm fog nebfp,

    uod`mb suicbnt`vb cugklbot, f t`b supphrts us`ok teb fdkhr`tel. Web

    ph`ot `s ohw `og`sputfidb3 webo yhu efvb f wbdd-vfd`gftbg stft`st`nfd

    fdkhr`tel, usb `t .

    We`s `os`ket wfs obvbr f terbft th teb rb`ko hj suicbnt`vb cugk-

    lbot ibnfusb wb sh rfrbdy efvb wbdd-vfd`gftbg fdkhr`tels jhr teb

    prhidbl ft efog. @t wfs cust `lprfnt`nfd jhr lfte th g`spdfnb pdf`o

    hdg te`om`ok‗`o 6027 fog bvbo thgfy.

    Iut spbntfnudfr fgvfonbs `o `ojhrlft`ho tbneohdhky sukkbst wb

    frb fpprhfne`ok f e`sthr`nfd g`snhot`ou`ty `o eulfo`ty‐s rbdft`ho-

    se`p w`te lfne`obs. @o 600; @IL‐s Gbbp Idub ibft nebss neflp`hoKfrry Mfspfrhv. Ohw, nhllbrn`fddy fvf`dfidb nebss prhkrfls nfo

    ibft foy eulfo. @o 9166 @IL‐s [ftsho ibft Cbhpfrgy!  neflp`hos

    Mbo Cboo`oks fog Irfg Yuttbr. Weft wfs f vfstdy thukebr nhlput`ok

    nefddbokb, iut [ftsho‐s bok`obbrs g`g `t. Whgfy, `t‐s oh dhokbr `lphs-

    s`idb th `lfk`ob f jhrbnfst`ok nhlpbt`t`ho `o we`ne f supbrnhlputbr

    trhuonbs supbrjhrbnfstbrs fog supbrpuog`ts fd`mb. Fjtbr teft efp-

    pbos, tebrb w`dd st`dd ib eulfo jhrbnfstbrs, iut d`mb eulfo Cbhpfrgy!  

    `dd d e e j `

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    27/29

    99 supbrjhrbnfst`ok

    teft [ftsho nhudg bfs`dy stbdg f qubst`ho fihut teb prbsbot hr pfst

    d`mb —[e`ne twh Yuss`fo dbfgbrs trfgbg chis `o teb dfst tbo ybfrs8“

    Iut @ wfs nur`hus fihut e`s v`bws ho ehw dhok `t w`dd tfmb jhr [ft-sho hr hob hj `ts g`k`tfd gbsnbogfots th stbdg qubst`hos d`mb —[`dd twh

    thp Yuss`fo dbfgbrs trfgb chis `o teb obxt tbo ybfrs8“

    @o 60:2 teb phdylfte Ebribrt ^`lho tehuket wb wbrb hody

    twboty ybfrs fwfy jrhl f whrdg `o we`ne lfne`obs nhudg gh —foy

     whrm f lfo nfo gh,“ we`ne `s teb shrt hj of`vbdy hpt`l`st`n te`ok

    pbhpdb sf`g ifnm tebo, fog hob rbfsho wey Jbrrunn`‗ weh efs

     whrmbg `o frt`stn`fd `otbdd`kbonb jhr te`rty ybfrs‗`s lhrb nfut`husthgfy.6> Nhlput`ok ̀ s lfm`ok bohrlhus str`gbs, Jbrrunn` ohtbg. Web

    fi`d`ty th spht pfttbros `s krhw`ok spbntfnudfrdy. Fog lfne`ob dbfro-

    `ok, `o nhli`oft`ho w`te iurkbho`ok eulfo-lfne`ob `otbrfnt`hos

    teft jbbg teb dbfro`ok prhnbss, prhl`sbs jfr lhrb juogflbotfd fg-

    vfonbs th nhlb. —@t‐s kh`ok th ib hob hj tebsb bxphobot`fd nurvbs teft

     wb‐rb m`og hj ft teb ihtthl hj ohw,“ Jbrrunn` sf`g.

    Iut tebrb `s f vfst g`jjbrbonb ibtwbbo —[e`ne twh Yuss`fo dbfg-

    brs trfgbg chis8“ fog —[`dd twh Yuss`fo dbfgbrs trfgb chis fkf`o8“

    Web jhrlbr `s f e`sthr`nfd jfnt. Web nhlputbr nfo dhhm `t up. Web

    dfttbr rbqu`rbs teb nhlputbr th lfmb fo `ojhrlbg kubss fihut teb

    `otbot`hos hj Zdfg`l`r Qut`o, teb nefrfntbr hj Gl`tr` Lbgvbgbv,

    fog teb nfusfd gyofl`ns hj Yuss`fo phd`t`ns, fog tebo `otbkrftb teft

    `ojhrlft`ho `oth f cugklbot nfdd. Qbhpdb gh teft shrt hj te`ok fddteb t`lb, iut teft ghbso‐t lfmb `t bfsy. @t lbfos teb eulfo irf`o `s

     whogrhus‗ibnfusb teb tfsm `s stfkkbr`okdy efrg. Bvbo w`te nhl-

    putbrs lfm`ok kfddhp`ok fgvfonbs, teb shrt hj jhrbnfst`ok teft su-

    pbrjhrbnfstbrs gh `s f dhok wfy hjj. Fog Jbrrunn` `so‐t surb wb w`dd

    bvbr sbb f eulfo uogbr kdfss ft teb ^l`tesho`fo w`te f s`ko sfy`ok

    —suicbnt`vb cugklbot.“

    Lfne`obs lfy kbt ibttbr ft —l`l`nm`ok eulfo lbfo`ok,“ fog

    e i i g` ` e i e ` i — e ‐ g`jj

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    28/29

      Fo Hpt`l`st`n ^mbpt`n 9>

    `ok,“ Jbrrunn` sf`g. Weft‐s f spfnb eulfo cugklbot w`dd fdwfys hn-

    nupy.

    @o jhrbnfst`ok, fs `o htebr stbdgs, wb w`dd nhot`oub th sbb eulfo cugklbot ib`ok g`spdfnbg‗th teb nhostbroft`ho hj we`tb-nhddfr

     whrmbrs‗iut wb w`dd fdsh sbb lhrb fog lhrb syotebsbs, d`mb —jrbb-

    stydb nebss,“ `o we`ne eulfos w`te nhlputbrs nhlpbtb fs tbfls,

    teb eulfo grfw`ok ho teb nhlputbr‐s `og`sputfidb strboktes iut

    fdsh hnnfs`hofddy hvbrr`g`ok teb nhlputbr. Web rbsudt `s f nhli`of-

    t`ho teft nfo (shlbt`lbs) ibft ihte eulfos fog lfne`obs. Wh rb-

    jrflb teb lfo-vbrsus-lfne`ob g`nehthly, nhli`oft`hos hj KfrryMfspfrhv fog Gbbp Idub lfy prhvb lhrb rhiust tefo purb-eulfo

    hr purb-lfne`ob fpprhfnebs.

     [eft Jbrrunn` ghbs sbb ibnhl`ok hishdbtb `s teb kuru lhgbd

    teft lfmbs sh lfoy phd`ny gbiftbs sh pubr`db3 —@‐dd nhuotbr yhur Qfud

    Mruklfo phdbl`n w`te ly O`fdd Jbrkusho nhuotbrphdbl`n, fog

    rbiut yhur Whl Jr`bglfo hp-bg w`te ly Irbt ^tbpebos idhk.“ Jbr-

    runn` sbbs d`ket ft teb bog hj te`s dhok gfrm tuoobd3 —@ te`om `t‐s kh`ok

    th kbt strfokbr fog strfokbr“ jhr pbhpdb th d`stbo th teb fgv`nb hj

    bxpbrts wehsb v`bws frb `ojhrlbg hody iy teb`r suicbnt`vb cugklbot.

    Eulfo tehuket `s ibsbt iy psynehdhk`nfd p`tjfdds, f jfnt teft efs hody

    ibnhlb w`gbdy rbnhko`zbg `o teb dfst gbnfgb hr twh. —^h weft @ wfot

    `s teft eulfo bxpbrt pf`rbg w`te f nhlputbr th hvbrnhlb teb eulfo

    nhko`t`vb d`l`tft`hos fog i`fsbs.“67

    @j Jbrrunn` `s r`ket‗@ suspbnt eb `s‗ wb w`dd obbg th idbog

    nhlputbr-ifsbg jhrbnfst`ok fog suicbnt`vb cugklbot `o teb juturb.

    ^h `t‐s t`lb wb kht sbr`hus fihut ihte.

  • 8/20/2019 Superforecasting by Philip E. Tetlock and Dan Gardner

    29/29

    http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136693/siteID/8001/retailerid/6/trackingcode/penguinrandomhttp://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136709/siteID/8001/retailerid/22/trackingcode/PRH698C2DEE54http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136693/siteID/8001/retailerid/2/trackingcode/PRH698C2DEE54http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136709/siteID/8001/retailerid/3/trackingcode/PRH698C2DEE54http://links.penguinrandomhouse.com/type/affiliate/isbn/9780804136693/siteID/8001/retailerid/7/trackingcode/randohouseinc10663-20http://www.crownpublishing.com/