Can Experts be Trusted and what can be done about it? Insights from the Biases and Heuristics Literature Oren Perez, Bar Ilan University Faculty of Law [email protected] LSE-CARR, 14 October 2014
Jun 21, 2015
Can Experts be Trusted and what can be done about it?
Insights from the Biases and Heuristics Literature
Oren Perez, Bar Ilan University Faculty of
Law
[email protected] LSE-CARR, 14 October 2014
Motivation
O Experts play a critical role in the modern
regulatory system;
O Expert groups (Commission is advised by
about 1000 expert groups, which assemble
more than 30,000 experts);
O Regulatory Agencies - e.g., the European
Food Safety Authority (EFSA), the European
Securities and Markets Authority (ESMA) or
the European Integrated Pollution Prevention
and Control (IPPC) Bureau (EIPPCB)
2
Normative Background
O The objectivity ideal :
O "Experts should provide opinions which are
independent, regardless of the pressures of
litigation. In this context, a useful test of
‘independence’ is that the expert would express the
same opinion if given the same instructions by an
opposing party. Experts should not take it upon
themselves to promote the point of view of the
party instructing them or engage in the role of
advocates".
O UK Civil Justice Council. "Protocol for the Instruction of
Experts to give Evidence in Civil Claims." 2009
3
Can experts be Trusted (1)?
O Should I trust this guy?
4
So should experts be Trusted (2)?
O Misaligned incentives due to external
economic pressures (Conflict of Interests,
COI);
O cognitive failures.
O The key distinction between the two
categories of judgment failure is that the
latter can occur even in the absence of
COI
5
Current regulatory framework :
O Focus on conflict of interest:
O Register of Expert Groups
O the European Ombudsman investigation into
the Commission’s expert groups.
O Judicial review of expert advise (via
administrative law or tort law) is weak - strong
deference to expert opinion.
O informal rules of the scientific community -
vague + the normal checking mechanisms of
science are not applicable to the regulatory
context (e.g., peer review)
6
Changing the focus: Cognitive biases as a matter of regulatory concern
O Experts are not immune: their susceptibility
to bias originates in a fundamental human
tendency to couple Type 1 intuitive
processing and Type 2 analytical processing
in reasoning processes.
7
Key Cognitive Biases
O Motivated Reasoning:
O confirmation bias - "the seeking or
interpreting of evidence in ways that are
partial to existing beliefs, expectations, or
a hypothesis in hand"
O disconfirmation bias - people are unable
to ignore their prior beliefs when
processing counter-arguments or counter-
evidence
8
Examples of confirmation bias
O studies of diagnostic decision-making have
demonstrated that doctors are subject to
confirmation bias when they interpret evidence
(e.g., symptoms or lack of symptoms),
O "prosecutorial bias" - documented in the work of
forensic experts that work in laboratories located
in law enforcement agencies or prosecutors'
offices - and reflected in a selective treatment of
the evidence in ways that support the agenda of
the institution in which the laboratory is situated.
9
Hindsight and outcome biases
O Hindsight bias - finding out that an outcome
has occurred increases its perceived
likelihood.
O Outcome bias - the influence of outcome
knowledge upon evaluations of decision
quality and the potential responsibility or
culpability of the decision-maker to the
outcome.
10
The De-biasing Project (and nudging)
O De-biasing constitutes a more extensive
interventionist strategy than nudging.
O A nudge is "any aspect in the framing of a
decision problem that can affect people
decisions without changing economic
incentives" (changing, e.g., the way
information is presented or by changing
default rules).
11
De-biasing may go beyond Nudging
O E.g., by changing the agent's set of choices,
increasing the transaction costs associated
with a certain decision, or by requiring the
development of new institutional structures.
O In the context of expert-decision making the
literature on de-biasing has focused on ways
to shift cognitive processing from a System
1 mode of thinking (automatic, heuristic) to
a System 2 (controlled, rule-governed) mode
of thinking. 12
Debiasing techniques
(a) Introspective techniques, encouraging the
agent to self-reflect on his reasoning process,
revising it as needed (general bias awareness);
(b) Cognitive-forcing techniques - influencing the
decision-maker indirectly by changing some
features of the task or the decision-making
environment;
(c) Introducing deliberative elements into the
decision-making process - 'forcing' the expert to
consider and cope with other viewpoints.
13
Checklists (1)
14
Checklists (2)
15
Process-conditioning rules - Design of Automatic Teller Machines
16
Introspective, meta-cognitive techniques
17
The fallibility of introspective de-biasing
O State 1:{A, B, C, D};
O State 2: {Eva1{A, B, C, D}-> A, B, C, ~D};
O State 3: {Eva2{Eva1{A, B, C, D}-> A, B, C, ~D}-
>A, B, ~C, D};
O …
O (n) State n: {Evan-1{Eva n-2{ Eva n-3 …
18
Cognitive forcing techniques - some problems
O CFTs can generate new cognitive problems,
which potentially can undermine any
benefits achieved through the introduction
of the CFT.
19
The Design of ATMs
20
The pitfalls of checklists
21
E-deliberation and open-gov
O The Problem of Mass E-mails/mass petitions.
O https://petitions.whitehouse.gov/
22
From Theory to Policy
O The move from theory to policy is far from
trivial - requires differential approach ,
experimentation and realistic calibration of
our goals and expectations.
23
Some Examples
O Methodological guidelines - examples:
O GRADE system - Grading of Recommendations
Assessment, Development, and Evaluation;
O American Medical Association’s ("AMA") Guides to
the Evaluation of Permanent Impairment (Sixth
Edition, 2007)
O Forensic Science (Report of the House of Commons
Science and Technology Committee on Forensic Science
(Second Report of Session 2013–14)
O Challenges?
24
Encouraging perspective taking
O By creating a pluralistic decision-making
environment:
O revising the expert selection process
O IPCC Procedures for the Preparation, Review,
Acceptance, Adoption, Approval and
Publication of IPCC Reports,
O Blind selection and randomization
O Public deliberation (Regulation room;
regulation.gov)
25
The ultimate objective of the de-biasing project?
From cognitive sterility - to the creation of
reflexive and epistemologically complex
decision making structures.
26