Top Banner
Argumentation and Reasoned Action Proceedings of the 1 st European Conference on Argumentation, Lisbon 2015 Volume I Edited by Dima Mohammed and Marcin Lewiński
24

Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

Jul 06, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

Argumentation and Reasoned Action

Proceedings of the 1st European Conference on Argumentation,

Lisbon 2015

Volume I

Edited by

Dima Mohammed and

Marcin Lewiński

Page 2: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

© Individual author and College Publications 2016 All rights reserved. ISBN 978-1-84890-211-4 College Publications Scientific Director: Dov Gabbay Managing Director: Jane Spurr http://www.collegepublications.co.uk Original cover design by Orchid Creative www.orchidcreative.co.uk Printed by Lightning Source, Milton Keynes, UK All rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form, or by any means, electronic, mechanical, photocopying, recording or otherwise without prior permission, in writing, from the publisher.

Page 3: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

D.Mohammed&M.Lewiński(eds.)(2016).ArgumentationandReasonedAction:Proceedingsofthe1stEuropeanConferenceonArgumentation,Lisbon,2015.Vol.I,807-820.London:CollegePublications.

807

35GivingReasonsProEtContraasaDebiasingTechnique

inLegalDecisionMaking

FRANKZENKERDepartmentofPhilosophy&CognitiveScience,LundUniversity,

[email protected]

CHRISTIANDAHLMAN

LawFaculty,LundUniversity,[email protected]

RASMUSBÅÅTH

DepartmentofPhilosophy&CognitiveScience,LundUniversity,Sweden

[email protected]

FARHANSARWARDepartmentofPsychology,LundUniversity,Sweden

[email protected]

Wereportontheresultsofdeployingthedebiasingtechnique“giving reasons pro et contra” among professional judges atSwedishmunicipalcourts (n=239).Experimentalparticipantsassessedtherelevanceofaneyewitness’spreviousconvictiontohiscredibility inthepresentcase.Resultsarecomparedtodatafromlayjudges(n=372).Thetechniqueproducedasmallpositive debiasing effect in the sample of Swedish judges,whiletheeffectwasnegativeamonglayjudges.

KEYWORDS: debiasing technique, heuristics and biases, legaldecision-making,priorconviction,witnessscenario

1.INTRODUCTIONHowtoimprovedecisionsisapertinentquestionwhenever judgmentsare unavoidable. The decisions that judges and juries must reach

Page 4: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

FrankZenkeretal.

808

virtually every day provide a case in point, a fortioriwhen these bearstrongly on the fates of individual and collective agents. Since biasedreasoninganddecisionmakingis(rightly)thoughttooccuralsoinlegalcontexts(see,e.g.,Langevoort,1998forareview;cf.Mitchell,2002),noargumentseemsrequiredthatitoughttobereduced.Rather,empiricalknowledgeiswantedhowreliablereductionsmaybeachieved.

Professional judges tend to assume of themselves, firstly, thatnon-jurist decisionmakers regularly err in assessing the relevance oflegal evidence; and, secondly, that judges reason inways that reliablyavoidsucherror.Forsomefivedecades,however,empiricalresearchinthe heuristics and biases tradition has supported the first assumptionalso for judges. Relevance-assessments may therefore be assumed todifferwidelybetweenintuitiveanddeliberativemodesofreasoninganddecisionmaking,bothbetweenandwithin(groupsof)agents.

Our research focuses on the second assumption, above. Itaddresses four related questions through controlled experimentationandinterpretativeanalysis:

(1) Whatistheaccuracy-differencebetweenjudges’andlaypersons’assessments of the relevance of legal evidence (or: howmuchbetterarejudgesatactivating‘system2’insuchassessments)?

(2) Do relevance-assessments improve across both groupssubsequenttobeinginstructedtodeployadebiasingtechnique?

(3) Whatisanoptimalallocationbetweendebiasingtechniquesandthebias(es)thusmitigated?

(4) Howcandebiasingtechniquesbeimproved?This paper reports empirical results regarding the first two questionsforthcoming from a pilot-studywith a sample of professional Swedishjudges and a sample of Swedish lay-judges (nämdeman). Experimentalparticipantswereaskedtoassessaspectsofamocklegal-casethathadbeen manipulated to contain bias-triggering information. In theexperimental subgroup, the mock case was followed by explicitinstructions“togivereasonspro/con”;inthecontrolgroupitwasnot.

Thepurposeofthisexperimentalset-upistoassessthepositive,negative, or neutral effect(-size) of instructions to deploy a debiasingtechnique inahypothetical legaldecisionmaking scenariovis-à-vis anestablishedcognitivebias,ononehand,andadebiasingmethod,ontheother—wherethelattermaycountasfarlessestablished.Therelevantbiasisa“devileffect”, insofarasaninformation-itemaboutapersonisof exaggerated importance in gauging her general credibility (seebelow).Suchresearchcontributestoassessingtheaverageeffectivenessofadebiasing technique, itselfan instanceofprescriptiveameliorative

Page 5: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

Adebiasingtechniqueinlegaldecisionmaking

809

intervention,ifandinsofarasatechniqueconstitutesanefficientcausewhose effect shows as an improved, or perfect, alignment between anormativestandardandthedecisionoutcome.

Followingabrief introductiontobiasesanddebiasing(Sect.2),wepresentthemethod(3)andmainresults(4),offeradiscussion(5),andclosewithbriefconclusions(6).2.BIASESANDDEBIASINGBiases are generally considered latent, that is, subjects tend to beunaware of them. By definition, a technique does debias when itsdeployment brings forth a decision that (i) differsmarkedly from onebrought forth by deploying a heuristics, and (ii) also complies with anormativestandard,e.g.,assetforthbythelaw.

Broadly speaking, what authors such as Kahneman & Tversky(1982; 1996), or Kahneman (2011) call biases, philosophers andscholars of law associate with the fallacies. The latter fields share atradition in Aristotelian scholarship, specifically the critiques of theSophistic mode of audience persuasion. The 16th century FrancisBacon’sdeliveringhisidolatryorthe17thcenturyJohnLockenamingofa range of fallacies fronted by “ad” (e.g.,ad hominem) have continuedthis tradition into themodern age. SinceHamblin (1970), fallacies arestandard fare in speech communication, rhetoric, and argumentationstudies,amongothers.Aroundthattime,moreover,theinterpretationoffallaciesasreasoningerrorsbecameseparatedfromviewingfallaciesasproblematicarguments(e.g.,vanEemeren&Grootendorst,1984).Mostpsychologist and cognitive scientists, however, continue to strictlyendorsethefirstinterpretation.

Despite a vast number of empirical studies confirming theassumed operation of such biases for various groups of subjects, fewstudies pertain to contexts of legal decision making. Exceptions are,amongothers,Guthrieetal.’s(2007)studyofanchoring,hindsightbiasandbaserateneglect,andEnglishetal.’s(2006)studyoftheanchoringeffect.Bothparticularlysupportthatbiasesalsoinfluencelegaldecisionmaking(forfurtherreferencesseeZenkeretal.,2015).

Extant research moreover strongly suggests that humans areespeciallychallengedintheapplicationofdebiasingmethods,andmoresoinself-application(Pronin&Kugler,2007;Pronin,Lin&Ross,2002;Willingham,2007;Kahneman,2011;Kenyon,2014).Self-assessmentforbiasedthinkinggenerallycountsasadifficultcognitiveabilitytomaster;theprimarychallengeisthesuspensionof latency.Butextantresearch(e.g., Guthrie et al., 2007; Irwin et al., 2010) also identifies debiasingtechniques for legal decisionmaking contexts, including the following.

Page 6: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

FrankZenkeretal.

810

Some of their underlying principles are already incorporated intoprocedural and substantial law. Debiasing effects thus brought shouldhenceproducedecisionsthatfallwithinthelaw.

• Accountability: legal decisions are subject to review by higher courts(Arkes,1991).

• Devil’sAdvocate:Remindingsubjectsof thehypotheticalpossibilityoftheoppositestandpoint(Lordetal.,1984;Mussweileretal.,2000).

• Giving Reasons (Larrick, 2004, p. 323; Hodgkinson, 1999; Mumma &Wilson,1995;Koriatetal.,1980).

• Censorship: When evidence counts as inadmissible, this may avoidbiasestriggeredbysuchevidence.

• ReducingDiscretion:Formulatinglegalnormsthatleavelessroomforajudge’s interpretation (e.g., explicit checklists, or a pre-set damageamount).

Anoverviewofextantresearchondebiasinginlegalcontextsincludingkey methodological issues and additional references is provided inZenker, Dahlman, and Sarwar (2015). As is argued there, successfuldebiasingtechniquesmustsimultaneouslyaddressaspectsofcognition,motivation,andtechnology.Theyneedtoraisetheagent’sawarenessofthe bias (cognition) in ways that sustain or increase her impetus toavoid biased reasoning (motivation),while providing information thatagentscaninfactdeploytocorrectextantreasoning(technology).

Empirically testing a debiasing technique vis-à-vis a bias-triggeringmockcaseservesto(i)empiricallyassesstheextenttowhicha hypothetical (yet realistic) legal decision can be subject to biases, ifandinsofarasjudges’andlaypersons’hypotheticaldecisions“inthelab”arerepresentativeofthose“outsidethelab.”Researchfurtherservesto(ii)estimatethepotentialofsuchinstructionsatmitigatingbiases,ifandinsofarasmitigationinthelabindicatesthatthesamesucceedsoutsidethe lab. Finally, research eventually yields (iii) information on theoptimal point at, and the optimal manner in, which decision makerswouldreasonablywanttodeployadebiasingtechnique.3.METHODBy regular mail, all 667 professional judges at municipal courts inSweden were asked to answer a pen-and-paper questionnaire thatsoughttoassesswhether,andifsotowhatextent,apreviousconvictionaffectsawitness’scredibility.Bywayofacourt’schiefjudge,moreover,738 lay judges were asked to assess what onemay generally call the“priorconvictionrelevance”inthefollowingmockcase.

Page 7: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

Adebiasingtechniqueinlegaldecisionmaking

811

Sebastian P is charged for assault. According to theprosecutor’scharge,SebastianPassaultedVictorA,onJuly20,2012at23:30outsideacinemaincentralMalmö,byrepeatedblows to the head. Sebastian P testifies that he acted in self-defense and denies the charges. One of the witnesses in thetrialisTonyT,whowasatthesiteonthatparticularevening.DuringtheexaminationofthewitnessTonyT,itemergesthathehad recently serveda two-yearprison sentence for illegalpossessionofweaponsandarmstrafficking.

Which of the following best describes your assessment? (Tickoneoptiononly)

- Tony T’s previous conviction for illegal possession ofweapons and arms trafficking affects the assessment of hiscredibility as a witness in the current trial. When variousfactors are weighed, the fact that he had previously beenconvictedofillegalpossessionofweaponsandarmstraffickingisstronglytohisdisadvantage.

- Tony T’s previous conviction for illegal possession ofweapons and arms trafficking affects the assessment of hiscredibility as a witness in the current trial. When variousfactors are weighed, the fact that he had previously beenconvictedofillegalpossessionofweaponsandarmstraffickingisclearlytohisdisadvantage.

- Tony T’s previous conviction for illegal possession ofweapons and arms trafficking affects the assessment of hiscredibility as a witness in the current trial. When variousfactors are weighed, the fact that he had previously beenconvictedofillegalpossessionofweaponsandarmstraffickingissomewhattohisdisadvantage.

- Tony T’s previous conviction for illegal possession ofweaponsandarmstraffickingdoesnotaffecttheassessmentofhiscredibilityasawitnessinthecurrenttrial.

In the experimental groups of both samples (professional and layjudges)—afterthescenario,butbeforethecentralquestionandthefouralternative answerswerepresented—participantswere asked to statereasons why Tony T’s convictions would affect his credibility as awitness in the present trial and to state reasons why his convictionswouldnotaffecthiscredibilityinthepresenttrial.Nosuchinstructionswereincludedinthequestionnairegiventocontrolgroup-participants.

Page 8: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

FrankZenkeretal.

812

Of the professional judges, 40% returned the questionnaire(n=239), where 143 participants, i.e., 59.8% of the sample, had notreceived instruction to deploy any debiasing technique beforeansweringthecase(controlgroup),while96participants,i.e.,40.2%ofthe sample, were instructed to state reasons for their assessment(experimental group; later referred to as “debias group”). Among layjudges, 52% returned the questionnaire (n=372), of which 171, i.e.,45.9%,belongedto theexperimentalgroupand201, i.e.,54,1%, to thecontrol group. In both samples, the response rate is unbalanced sinceparticipants were at liberty to return the questionnaire; they did notreceivefinancialorothercompensationforparticipatinginthisstudy.

Typical responses in both samples included the followingpro/conreasons:

Priorconvictionisrelevant(pro)• TonyT.hasnobarriertobreakingthelaw• TonyT.mayhaveaninterest(e.g.,revenge)• TonyT.hasreduced“citizenship-capital”• TonyT.hasapro-attitudetoviolence

Notrelevant(con)• Unrelatedevent/circumstances• Noevidencethatpriorconvictionmatters• Priorconvictionshouldbeirrelevant• Currenttestimonyoccursunderoath

Prior to deploying the questionnaire, we did not formulate a point-hypothesistocodeanormativelycorrectresponse.Rather,weassumedthat obtaining differences between the experimental and the controlgroupsuggeststhat“givingreasonsproetcontra”hasadebiasingeffectprovided thatparticipants in thisgroupdoonaveragedisplaya lowerassessmentofthepriorconvictionrelevance.4.RESULTSThe effect of deploying the debiasing technique “giving reasonspro etcontra”wasprima facieminiscule.First lookingatprofessional judges,fewerparticipantsinthedebiasgroupthaninthecontrolgrouptookthewitness’s previous conviction to be clearly or strongly to hisdisadvantageinthepresentcase.Expressedinnumbers,theseweresixand respectively one vs. zero participants (4.2% and 0.7% of thesample). This can provide at best some reason to believe that the

Page 9: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

Adebiasingtechniqueinlegaldecisionmaking

813

debiasingtechniquehadanamelioratingeffectonjudges.Moreover,28judges in thecontrolgroup(19.6%of the judges in thecontrolgroup)register as finding the witness’s prior conviction to be somewhatnegativelyrelevant.Finally,20judgesintheexperimentalgroup(12.8%of the judges in the control group) so register despite a debiasingtechniquebeingdeployed.

Turningnow to lay judges, by contrast, hardly anynoteworthydifferencesarosebetweenthecontrolandtheexperimentalgroup:7%and8%ofthetotalnumberof lay judgesfoundthepriorconvictiontobe clearly or, respectively, strongly relevant; 30% in each group foundthe conviction to be somewhat relevant; 61% and 63%, respectively,foundthepriorconvictiontobenotrelevant.Table1andFig.1givethefullresultsofthequestionnaire.

Responses were coded on a four point ordinal scale (as notrelevant, somewhat, clearly and strongly to the witness’s disadvantage;see Table 1). In order to investigate the difference between the fourgroups,that is, thecontrolandexperimentalgroups,eachconsistingofeitherprofessionalorlayjudgesdatawasthensubjectedtoanorderedprobitanalysis.1

notrelevant

somewhatrelevant

clearlyrelevant

stronglyrelevant N

Judges Control 108(76%) 28(20%) 6(4%) 1(1%) 141

Debias 76(79%) 20(21%) 0(0%) 0(0%) 96

Total 184(77%) 48(20%) 6(3%) 1(0%) 239

LayJudges Control 126(63%) 60(30%) 12(6%) 3(1%) 201

Debias 105(61%) 52(30%) 11(6%) 2(2%) 171

Total 231(62%) 112(30%) 23(6%) 5(2%) 372

Table1.ResponsesfromSwedishjudgesandlayjudges(n=numberofsubjects) 1SeeDaykinandMoffat(2002)forparadigmaticapplicationsoforderedprobitanalysisanditsadvantagesoverfar-betterknown,butalsolesswell-suited,linearregressionanalyses.Forinstance,orderedprobitanalysisisnotopentotheobjectionthatthedistancesbetweenanytwoordinaldatapointsareimplicitlytreatedasbeingequal.TheprobitanalysiswasdoneintheRstatisticalenvironmentusingthepolrfunctionintheMASSpackage(Venables&Ripley,2002).

Page 10: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

FrankZenkeretal.

814

This analysis assumes that underlying the ordinal scale, on whichparticipants’ responsesaremeasured, isacontinuousrandomvariablerepresenting participants’ assessment of prior-conviction relevance(PCR).ThevalueofthislatentvariablehasnodirectinterpretationbutisarelativemeasureofPCR,whereahighervalue implies thatapriorconvictionisdeemedmorerelevant.Crucialforthefollowingstatisticalanalysis, the expected value of the latent variable can be taken as ameasureofthegeneralsentimentofthegroupandcanthusbeusedincomparingthegroups.

Thedistributionalparametersofthelatentvariableweregaugedthrough maximum likelihood estimation, yielding the parameterestimates under which the ordered probit model is most likely togeneratetheobserveddatainTable1.

Fig.1ProportionofresponsesfromSwedishjudgesandlayjudges

In virtue of being maximally consistent with the original data, thehypotheticalmodelmaybeinterpretedasthemostprobablecontinuousdistributionofthelatentPCR-variableamongrespondents.Inthissense,thehypotheticalmodel canbe viewed to haveprobably generated theoriginal data. The shaded curves in Figure 2 show the maximumlikelihood estimates of the latent PCR-variable among judges and layjudgesinthecontrolandtheexperimentalgroup.Thefigureisdividedinto four regions corresponding to the four possible responses in thesurvey.Thepercentageof theareaunder thecurvewithineachregioncorrespondstothemodel'sestimateoftheprobabilitythatamemberofthesegroupsproducesthecorrespondingsurveyresponse.ThedashedverticallinesmarktheexpectedvaluesofthelatentPCR-variables,heretakenasameasureofthegeneralsentimentofgroups.

Page 11: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

Adebiasingtechniqueinlegaldecisionmaking

815

ComparingpanelsA-BandC-DinFig.2,thedisplacementoftheexpected values of the PCR-variables indicates the impact of thedebiasingintervention.WhilethereisavisibledifferenceinthegeneralassessmentofPCRbetweenjudgesintheexperimentalandjudgesinthecontrolgroup, there ishardlyanydifferencebetweenthe lay judges inthe experimental group and lay judges in the control group.But therewas nevertheless a substantial overall difference between judges andlay judges: the former judged the prior conviction to be less relevantthanthelatter.

Figure2.Probabilitydistributionofthelatent“priorconvictionrelevance”-variableforjudgesandlayjudgesinthedebiasand

thecontrolgroups.

Page 12: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

FrankZenkeretal.

816

ABayesiananalysiswasperformedtogauge theuncertainty intheestimatesfromtheorderedprobitanalysis,andtoquantifywhetherthe joint data from judges and lay judges in the experimental and thecontrolgroupsupport,orundermine,thehypothesisthatthedebiasingtechnique“givingreasonspro/con”hadanamelioratingeffect.2

Fig.3.Distributionofprobabilitiesgivenmodelandevidence

fromprofessionalandlayjudges

2TheanalysiswasperformedintheRstatisticalenvironmentusingtheMCMCoprobitfunctionintheMCMCpackpackage(Andrewetal.,2011).ThedefaultpriorsoftheMCMCoprobitfunctionwasused,whichwerenon-informativeuniformpriorsoverallparameters.

Page 13: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

Adebiasingtechniqueinlegaldecisionmaking

817

Figure 3 shows the probable difference in the expected values of thePCR-variable(markedbyadashedline;Fig.2)betweenallfourgroups.Given model and data, there is a 87% probability that judges in theexperimental find the prior conviction less relevant (Fig. 3, panel A),compared to a 38% probability that lay judges in the experimentalgroupfindthepriorconvictionlessrelevant(Fig.3,panelB).

This may be interpreted as ratherweak positive evidence thatdeployingtherelevantdebiasingtechniquehasadebiasingeffectamongjudges,butnotamong lay judges.Moreover, comparing judgesand layjudgesinthecontrolgroup(Fig.3,panelC)andthedebiasgroup(Fig.3,panel D) shows a probability larger than 99%—which may beinterpretedasverystrongevidence—thatinboththecontrolandintheexperimental condition lay-judges assign a higher prior convictionrelevance than judges,with evidence from the experimental conditionregistering slightly stronger yet. This, in fact, amounts to havingobservedan interactionof theprofessionalstatuswith theassessmentofpriorconvictionrelevance.4.DISCUSSIONIn theexperimentaldata, strongevidence foramitigatingeffectof thedebiasing method “stating reasons pro/con” onto participants’responseshasnotbeenforthcoming.Rather,thestudyfoundan87.1%probability for a mitigating effect. This can at best count as weakevidence.Inthemockcase,layjudgesdidoverallassignagreaterweightto the previous conviction of the witness than professional judges.Moreover—and perhaps disturbingly—compared to the relevantcontrol group lay judges in the experimental group displayed anincreasedmeanscore.

Resultsarebroadlynegativeinthesensethatthe“TonyT”mockcasefailedtotriggerastrongbiasamongprofessionalorlayjudges.Byand large, professional judges merely assigned some weight to theprevious conviction, while lay judges assigned a greater weight. Thedebiasingtechnique“statingreasonsproetcontra”inotherwordsfailedto meet with a strongly biased sample of judges and lay judges. Thetechniqueneverthelessappearstosucceedin“takingtheedgeoff,”asitwere.Afterall, compared to the relevantcontrolgroup, thenumberofextremejudgementsintheexperimentalgroupofprofessionaljudgesisreduced.Itstandstoreason,ofcourse,that“removing”butoneextremejudgementthroughadebiasinginterventiondoesalreadyconstituteanimportant and desirable outcome. This nonetheless remains a verysmall effect.And as thedebiasing techniquemetwith a comparativelymore biased sample of lay judges, its deployment not only failed to

Page 14: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

FrankZenkeretal.

818

mitigate thebias;rather, itslightlyworsenedthe judgementcomparedto the control group of lay judges. But also this result remainsstatistically insignificant, and so cannot easily be accounted for as aneffectofdeployingthetechnique.

Toaddress theobjection thatadditionaldatashouldhavebeencollectedinordertoassesswhetherastatisticallysignificantdebiasing-effectwould after all have been observed, consider that the sample ofSwedish judges in the present study (n=239) represents no less than40% of the relevant population(!). To increase this numberwould nodoubtpresentgreaterpracticaldifficulties.Itremainscorrect,ofcourse,that small experimental effects must always be confronted with largedata-samples. But for the small effect here reported to potentiallyregisterasstatisticallysignificantdoesnecessarilyrequireasample-sizethatexceedsthesizeoftherelevantpopulation!Thisfacthenceentailsthat there might be biases whose presence, and debiasing techniqueswhose effect, can principallynot be demonstrated by obtaining strongevidence for a difference between the control and the experimentalgroup whenever the effect is too small to register as significant evenagainst the size of the relevant population. For this reason, the “needmoredata”-objectionisparticularlyweakinthepresentcontext.

Demonstrating the effectiveness of a debiasing technique atconventionallyacceptedlevelsofsignificancecouldinsteadbeservedbymaximizing the difference between participants’ ratings in the controland the experimental group. In the present study, as we saw, bothgroupsdisplayedratherlowdegreesofbiasedness.Itthereforeremainsa challenge for future research to create experimental set-ups thatinduce stronger biases. In view of the idea that already a smallamelioratingeffect,ifitisreal,shouldbeviewedasadesirableoutcomeofdeployingadebiasingtechnique,wesuggestthatitcanbereasonabletoacceptweakerformsofevidentialsupport,ratherthaninferringthatthe debiasing technique was probably ineffective. Since this stance isunlikely to meet with wide acceptance, however, the key-task wouldremaintoinduceastrongerbias.5.CONCLUSIONAmong Swedish judges at municipal courts, the “Tony T” mock casefailedtomeetwith“sufficientlybiased”respondents,sincefewassigneda great(er) weight to the witness’s prior conviction regarding hiscredibilityinthepresentcase.Thedebiasingtechnique“givingreasonspro et contra” could thus at best produce a small effect—too small tocount as strong evidence relative to the sample or even the relevantpopulation. Rather than inferring that the technique probably had no

Page 15: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

Adebiasingtechniqueinlegaldecisionmaking

819

effect, however, we submit these results asweak positive evidence infavoroftheeffectivenessofthisdebiasingtechnique.

Aswealsosaw,resultsdiffered—yetinthenormatively“wrong”direction—when the same techniquewas deployed vis-à-vis the samemock case among lay judges, who seem to have constituted acomparatively more biased sample than the professional judges. Thedebiasingtechniquehadaweakadverseeffectonlayjudges;subsequentto deploying it, the latter assigned a slightly increased weight to therelevance of previous conviction. As we have stressed, however, thisinterpretationissubjecttocaveatsastheeffectremainedtoosmall.

Among all measures taken, we obtained very strong evidencemerelyforarelationbetweentheprofessionandthelevelofbiasedness,therebeing aprobability greater than99% that lay judgesweremorebiased than professional judges. To test the effectiveness of debiasingmethodsagainststandardstatisticalassumptions,futurestudiesseekingtoproducestrong(er)positiveevidencearechallenged to findwaysoftriggeringstrong(er)biases.ACKNOWLEDGEMENTS: We thank audience members at the FirstEuropean Conference on Argumentation, 9-12 June 2015, Lisbon,Portugal, for discussion and Fabrizio Macagno for his commentary.Research was funded by the Ragnar Söderberg Foundation. RasmusBååth acknowledges funding through Swedish Research Council grantnumber349-2007-8695.REFERENCESArkes, H.R. (1991). Costs and benefits of judgement errors: Implications for

debiasing.PsychologicalBulletin,110,486–498.Daykin, A.R., &Moffat, P.G. (2002). Analyzing ordered responses: a review of

theorderedprobitmodel.UnderstandingStatistics,1(3),157–166.Eemeren, F. H. van, & Grootendorst, R. (1984). Speech acts in argumentative

discussions: A theoreticalmodel for the analysis of discussions directedtowardssolvingconflictsofopinion.Amsterdam:WalterdeGruyter.

English, B., Mussweiler, T., & Strack, F. (2006). Playing dice with criminalsentences: The influence of irrelevant anchors on experts’ judicialdecisionmaking.PersonalityandSocialPsychologyBulletin,32(2),188–200.

Guthrie,C.,Rachlinski,J.J.,&Wistrich,A.J.(2007).Blinkingonthebench:Howjudgesdecidecases.CornellLawReview,1,1–44.

Hamblin,C.(1970).Fallacies.London:Methuen.Irwin, J., & Daniel, L.R. (2010). Unconscious influences on judicial decision-

making.McGeorgeLawReview,43,1–20.

Page 16: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

FrankZenkeretal.

820

Kahneman, D., & Tversky, A. (1982). On the study of cognitive illusions.Cognition,11,1123–1141.

Kahneman,D.&Tversky,A.(1996).Ontherealityofcognitiveillusions:AreplytoGigerenzer’scritique.PsychologicalReview,103,582–591.

Kahneman,D. (2011).Thinking, Fast and Slow. NewYork,NY: Farrar, StraussandGiroux.

Kenyon,T.(2014).Falsepolarization:Debiasingasappliedsocialepistemology.Synthese,191(11),2529–2547.

Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980). Reasons for confidence.JournalofExperimentalPsychology:Humanlearningandmemory,6(2),107–118.

Langevoort,D.C.(1998).Behavioraltheoriesofjudgmentanddecisionmakingin legal scholarship: A literature review. Vanderbilt Law Review, 51,1499–1540.

Lord, C.G., Lepper, M.R., & Preston, E. (1984). Considering the opposite: Acorrective strategy for social judgment. Journal of Personality andSocialJudgment,47(6),1231–1243.

Martin, Andrew D., Quinn, Kevin M., & Park, Jong Hee (2011). MCMCpack:MarkovChainMonteCarloinR.JournalofStatisticalSoftware,42(9),1-21.

Mitchell,G. (2002).Why lawandeconomics’perfect rationality shouldnotbetraded for behavioral law and economics’ equal incompetence.GeorgetownLawJournal,91,67–167.

Mumma, G.H., & Wilson, S.B. (1995). Procedural debiasing of primacy/anchoring effects in clinical-like judgments. Journal of ClinicalPsychology,51(6),841–853.

Mussweiler, T., Strack, F., & Pfeifer, T. (2000). Overcoming the inevitableanchoring effect: Considering the opposite compensates for selectiveaccessibility. Personality and Social Psychology Bulletin, 26(9), 1142–1150.

Pronin,E.,Lin,D.,&Ross,L.(2002).Thebiasblindspot:Perceptionsofbiasinselfversusothers.PersonalityandSocialPsychologyBulletin,28,369–381.

Pronin, E., & Kugler, M. (2007). Valuing thoughts, ignoring behavior: Theintrospection illusion as a source of the bias blind spot. Journal ofExperimentalSocialPsychology,434,565–578.

Venables,W.N.&Ripley,B.D.(2002)ModernAppliedStatisticswithS.FourthEdition.Springer:NewYork.

Willingham,D. (2007).Critical thinking:Why is it sohard to teach?AmericanEducator,31(2),8–19. (reprintedas:Willingham,D.T. (2008).Criticalthinking: Why is it so hard to teach? Arts Education Policy Review,109(4),21–32.)

Zenker,F.,Dahlman,C.(2015).Reliabledebiasingtechniquesinlegalcontexts?Weaksignalsfromadarkercornerofthesocialscienceuniverse.InF.Paglieri (Ed.). The psychology of argument: Cognitive approaches toargumentation and persuasion (pp. xx-yy). London: CollegePublications(forthcoming).

Page 17: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

D.Mohammed&M.Lewiński(eds.)(2016).ArgumentationandReasonedAction:Proceedingsofthe1stEuropeanConferenceonArgumentation,Lisbon,2015.Vol.I,821-827.London:CollegePublications.

821

CommentaryonZenker,Dahlman,BååthandSarwar’sGivingReasonsProEtContraasaDebiasingTechnique

inLegalDecisionMaking

FABRIZIOMACAGNOArgLab,UniversidadeNovadeLisboa,Portugal

fabrizio.macagno@fcsh.unl.pt1.INTRODUCTIONThepaperpresentsaninsightfulandgroundbreakingapproachtolegalreasoning and argumentation. The fundamental assumption of thiswork is that biases, or rather latent fallacious reasoning (Zenker,Dahlman, Bååth& Sarwar, 2016, p. 811-812) affect legal reasoning aswell, and can result in unwarranted conclusions to be reached. Suchbiasescanbeun-triggeredbyspecifictechniques,andinthispapertheauthorsassesstheeffectofastrategyofdebiasingcharacterattacks inwitness testimony. To this purpose, the authors run a mock test inwhich in a hypothetical scenario inwhich awitness is shown to havebeen convicted for previous crimes. The decision-makers (judges andperspective jurors) are divided in two groups (the control and theexperimentalgroup),inwhichtheexperimentalgroupissubjectedtoadebiasingtechnique(togivereasonsfortheirdecision).Theauthorsusequantitativemethodstoassesstheeffectivenessofthisstrategy,butastheyreport, theresultsarestatisticallyweak,eventhoughrelevant forthe purpose of discussing the relationship between the technique anditseffects.Despitetheeffortsoftheauthorstounderminetherelevanceof thepaper for legalargumentation, thisworksheds lightonrelevanttheoreticalandpractical issuesthatneedtobetakenintoaccount,andintroducesanextremelyinterestingmethodofinvestigation.2.THEBIASEDREASONING:CHARACTERASSASSINATIONThe authors took into account a specific type of biased reasoning, thecommonly called “character assassination” in law that can be“devastatinglyeffective” (Cantrell,2003,p.534;Solomon,2003,pp.7–8). By showing that awitness (or a defendant) committed a previouscrime,thedecision-maker(thejudgeorthejury)isledtoconcludethat

Page 18: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

FabrizioMacagno

822

histestimonyislessreliable(orthathecommittedalsothecrimeheisaccusedof).Thistypeofattackiscommonlyanalyzedinargumentationtheoryastheadhominemargument(Walton,1998,pp.198–199,p.217;2002, p. 51). Ad hominem arguments consist in showing that theinterlocutor’s argument should not be accepted based on a negativejudgment on different aspects of his or her character, such as logicalreasoning, perception, veracity, or cognitive skills (Macagno, 2013).Clearly,thereasonablenessoftypeofargumentdependsonthetypeofargument it is aimed at undermining. Ad hominem attacks are oftenreasonablewhentheyundermineargumentsbasedontheexpertiseorthe position to know of a source, namely authoritative arguments. Inthese cases, if the truth, or rather acceptability, of the testimonydependsonsomeof thecharacter featuresattacked, theargumentcanbereasonable.Otherwise,wouldbesimplyirrelevanttotheconclusion.

Despite their unreasonableness, often irrelevant ad hominemargumentshavegreatimpactontheevaluatorofanargument,andmorespecifically in law on the judge or especially the jury. Attackingcharacterdoesnotsimplyamounttoshowingaflawinanargument.Itmeans showing that a person’s character is somehow negative,whichcan lead to negative emotions. Emotions such as indignation, fear,contempt, or hate divert the interlocutor’s attention from the rationalandsystematicassessmentof theattack, leadingtoaconclusionbasedon fast associations between the emotion and a possible immediatereaction (Blanchette & Richards, 2004; Blanchette, 2006; Macagno,2014).

Attacking a witness’s character is allowed by the rules ofevidenceatcommonlaw.Accordingtorule609oftheFederalRulesofEvidence, it is possible to introduce evidence of the witness’s pastconvictionsinordertoimpeachhischaracterfortruthfulness:“Onewayofdiscrediting thewitness is to introduceevidenceofaprior criminalconvictionofthewitness,whichaffordsthejuryabasistoinferthatthewitness'scharacterissuchthathewouldbelesslikelythantheaveragetrustworthy citizen tobe truthful inhis testimony” (State v.Nash, 475So. 2d 752, at 754, 1985). In this sense, evidence of priormisconductcanbearationalgroundforassessingthetrustworthinessofawitness,oneofthepossibledimensionsthatneedtobetakenintoaccountwhenjudging his testimony. However, this evidence often risks becoming atriggerof a fallacious conclusion reachedbymeansof “fast” reasoning(Kahneman, Slovic, & Tversky, 1982; Tversky & Kahneman, 1974). AspointedoutbyMcCormick(McCormick,1972,p.104)“aslashingcross-examination may carry strong accusations of misconduct and badcharacter, which the witness's denial will not remove from the jury'smind.”Oneoftheclearestandmostfamousexamplesoftheforceofthis

Page 19: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

CommentaryonZenkeretal.

823

typeofcharacterattackisPeoplev.Simpson(No.BA097211,1995), inwhichthepreviousmisconductof thewitnessingdetective(Fuhrman),combinedwithproofof racialbehavior, led the jury tobelieve thathewasnotreliable.ThischaracterassassinationledtotheacquittalofO.J.Simpson.3.DEBIASINGCHARACTERASSASSINATIONBiasedreasoninginlawcanoccurbothwhenthejudgmentismadebyaprofessional judge, andwhen it is rendered by a jury of laypeople. Inthislattercase,inparticular,thepossibilityofthejurorsbeingunawareof the possible fallacious or weak reasoning becomes even higher, astheyarenottrainedtomakelegaldecisionsandevaluateobjectivelythevarious factors of the case. To this purpose, the authors have firstanalyzed the so-called debiasing techniques, namely strategy used toun-trigger the latent mechanisms leading to a fallacious conclusion.Such strategieshavedifferent foci, dependingon thedimensionof theautomaticreasoningthatthey intendtoaddress.Theycanbeaimedatturningalatent(implicit)mechanismintoanexplicitone,orleadingthedecision-makertoacarefulassessmentoftheforceofhisconclusion,orsimply preventing the decision-making frommaking some inferences.WecanclassifythetechniquesinTable1:

Making the reasoning explicit

Assessing the conclusion carefully Preventing inferences

Giving reasons Accountability Devil’s advocate

Censorship Reducing discretion

Table1:Debiasingtechniques

Thesetechniquesclearlyaresomeofthepossibleonesthatcanbeusedtoreducethepossibilitythatthedecision-makercomestoaconclusionbased on problematic implicit arguments. Other possible techniquesusedinlawareconfrontingtheinterlocutorwithabiasedbutcontraryconclusionorargument,sothatthereasoningprocessbecomessubjecttocarefulassessment(Macagno&Walton,2012).

The authors chose to test the debiasing technique of givingreasons, and they proved that its effects, even though not statisticallysignificant, however indicate that there is a high probability that thedifference between the control group and the one subjected to the

Page 20: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

FabrizioMacagno

824

debiasing intervention isduetothe intervention itself(5.6timesmoreprobable than not). However, as the authors point out, this outcomedoes not meet the statistical requirements for significance. Moreover,when they took intoaccountonly the lay judges, theynoticed that thedebiasing technique worsened the judgment compared to the controlgroup.4.POSSIBLEPROBLEMSOneofthepossiblecriticismsontheexperimentcanbeaddressedtotheverymockcasethatthejudgeshadtoassess.Perhapsoneofthecausesof a lower variability is due to external factors and variables that theauthors could not control, due to the vagueness of the case. The casereadsasfollows:

Sebastian P is charged for assault. According to theprosecutor’scharge,SebastianPassaultedVictorA,onJuly20,2012at23:30outsideacinemaincentralMalmö,byrepeatedblows to the head. Sebastian P testifies that he acted in self-defenseanddeniesthecharges.OneofthewitnessesinthetrialisTonyT,whowasatthesiteonthatparticularevening.DuringtheexaminationofthewitnessTonyT,itemergesthathe had recently served a two-year prison sentence forillegalpossessionofweaponsandarmstrafficking.

The judges had to assess whether the previous conviction of Tony Taffects the credibility of his testimony (in a strong, clear, some, or noway). However, the case is too generic and leaves too much room tonarratives and possible reconstructions of background information.Consideringthatavery lowpercentageof judges indicatedastrongorclear effect on the credibility of the witness, a low one cannot beexcludedifthejudgeisallowedtoreconstructinformationthatthecasedoesnotspecify.Didthewitnessknowthedefendant?Wasthewitnessinvolved in other criminal activities after his conviction? Was thewitnesssomehowrelatedtowiththedefendantorthevictim?Allsuchfactorscannotbeexcluded,andare likelytobereconstructedortakenintoaccountaspossiblereasonsofabiasedtestimony.Atcommonlaw,a real case usually involves a cross-examination of the keywitnesses,especiallywhentheircredibilitycanbeunderminedbycharacterissues.Whileprofessionaljudgesaretrainedtoevaluatethevariousfactorsofthe case without making additional hypotheses, this could be not thecaseforlaypeoplewhosimplyrelatethestorywiththemostaccessiblenarratives (the witness may know the defendant, since they are

Page 21: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

CommentaryonZenkeretal.

825

allegedlybothviolent,andthewitnessmaywanttocoverforhisfriend).If the debiasing technique consists in giving reason, some factors thatcan be used in such reasons need to be taken into account andcontrolled.

Asecondpossibleproblemisthelanguageofthevariables.Theauthors indicate a four-point scale, but they fail todefine clearlywhat“somehow”meanswhenreferredto“affectingthewitness’scredibility.”As pointed out by common law cases, evidence of prior convictions isallowed because it is an element that the jurymaywant to take intoaccount when assessing a witness’s trustworthiness compared to anaveragecitizen.Clearly,whennootherelementsarepresent,thispieceof evidence canbe irrelevant for evaluating character.However,whencombinedwithanextremelysuccinctnarrationofthecircumstances,itisnotunreasonabletothinkthatanuntrainedjudgemayfindpossibleadditional reasons making the testimony somehow less reliable.Perhaps the authors could refine their tests introducing morevariability. They could formulate clearer hypotheses, less subject topersonal interpretations, including or excluding some circumstantialfactor, and then use a Likert scale with more or more definite levels(strongly disagree… strongly agree) to assess them. For example, thetestcouldreadasfollows:

•“TonyT’spreviousconvictionforillegalpossessionofweaponsand arms trafficking strongly affects the assessment of hiscredibility as awitness in the current trial.” (strongly agree…stronglydisagree).•…•“Considering thatTonyThadnevermet thedefendantor thevictimbefore,TonyT’spreviousconvictionforillegalpossessionofweaponsandarmstraffickingstronglyaffectstheassessmentof his credibility as a witness in the current trial.” (stronglyagree…stronglydisagree).

Inthisfashion,theycouldmeasurehowmuchtheinterpretationoftheeventcanaffectjudgmentandmoreimportantlythereasonsunderlyingit.5.CONCLUSIONTheauthorshavefocusedtheirself-criticismsonthescarcesignificanceoftheoverallresults.However,thisstudyshowsclearlyhowlayjudgesandprofessionalonesdiffer concerning theassessmentof a case.Thisdifferencebecomesevenmorerelevantwhenweconsiderthefactthat

Page 22: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

FabrizioMacagno

826

the debiasing technique has opposite effects on laypeople, whoprovidedevenworseresultswhentheyhadtogivereasons.Thiseffectshouldbe analyzed indepth, and related to theproblemof prejudicesand background knowledge. How do prejudices affect thereconstruction of a state of affairs? Perhaps itwouldbe interesting toinvestigate how a layperson can reconstruct the narrative underlyingthewholecase,includingtherelationshipbetweenthewitnessandthedefendant, and compare themwith the reconstruction of professionaljudges.

To conclude, the authors perhaps failed to make a statisticalpoint, but opened a very broad range of fundamental questions thatshould be addressed with the method that they used and that isrevolutionaryinthefieldoflegalargumentation.ACKNOWLEDGEMENTS: I would like to thank the Fundação para aCiênciaeaTecnologiafortheresearchgrantno.IF/00945/2013.REFERENCESBlanchette, I. (2006).Theeffectofemotionon interpretationofand logic ina

conditionalreasoningtask.MemoryandCognition,34,1112–1125.Blanchette, I., & Richards, A. (2004). Reasoning about emotional and neutral

materials-Is logicaffectedbyemotion?PsychologicalScience,15(11),745–752.http://doi.org/10.1111/j.0956-7976.2004.00751.x

Cantrell, C. (2003). Prosecutorial Misconduct: Recognizing Errors In ClosingArgument.AmericanJournalofTrialAdvocacy,26,535–562.

Kahneman, D., Slovic, P., & Tversky, A. (Eds.). (1982). Judgment underUncertainty: Heuristics & Biases. Cambridge: Cambridge UniversityPress.

Macagno,F.(2013).StrategiesofCharacterAttack.Argumentation,27(4),369–401.http://doi.org/10.1007/s10503-013-9291-1

Macagno, F. (2014). Manipulating Emotions. Value-Based Reasoning AndEmotiveLanguage.Argumentation&Advocacy,51(2),103–122.

Macagno, F.,&Walton,D. (2012). CharacterAttacks as Complex Strategies ofLegal Argumentation. International Journal of Law, Language &Discourse,2(3),1–58.

McCormick, C. (1972).McCormick’s handbook of the law of evidence. St. Paul,Minnesota:WestPublishingCompany.

Solomon,R.(2003).NotPassion’sSlave.NewYork:OxfordUniversityPress.Tversky, A., & Kahneman, D. (1974). Judgment under Uncertainty: Heuristics

and Biases. Science (New York, N.Y.), 185(4157), 1124–1131.http://doi.org/10.1126/science.185.4157.1124

Page 23: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are

CommentaryonZenkeretal.

827

Walton,D.(1998).AdHominemArguments.Tuscaloosa:UniversityofAlabamaPress.

Walton, D. (2002). Legal argumentation and Evidence. University Park: ThePennsylvaniaStateUniversityPress.

Zenker,F.Dahlman,C.Bååth,R.&Sarwar,F.2016.GivingReasonsProetContraasaDebiasingTechniqueinLegalDecisionMaking.InD.Mohammed&M.Lewiński(eds.),ArgumentationandReasonedAction:Proceedingsofthe 1st European Conference on Argumentation, Lisbon, 2015. Vol. I,809-822.London:CollegePublications.

Page 24: Argumentation and Reasoned Actiona range of fallacies fronted by “ad” (e.g., ad hominem) have continued this tradition into the modern age. Since Hamblin (1970), fallacies are