Effects of prior information and opinions when considering evidence Nigel Harvey Psychology.

Effects of prior information and opinions when considering evidence

Nigel Harvey

Psychology

Using judgment to assess evidence

• Psychologists (eg Kahneman & Tversky) have argued that people often lack the cognitive resources to make judgments in an optimal or rational manner. Instead they use heuristics.

• They assume that heuristics are used both when people assess evidence for and against different hypotheses and when they use data to formulate hypotheses.

Plan

• Classic work on judgment biases that suggests that people use heuristics to make judgments.

• Brief summary of criticisms and some recent developments emphasizing usefulness of heuristics.

• Judgment in forecasting: research implicating heuristics.• The ‘whys’ and ‘hows’ of using heuristics: Some theory.• Conservatism in assessing evidence.• Is conservatism a motivational or cognitive phenomenon?

Initial demonstrations of underadjustment from cognitive anchors

• People saw a number randomly chosen. They first judged whether the % of African countries in the UN was greater or less than it. They then estimated this percentage. When the random number was 10, they estimated 25%; when it was 65, they estimated 45%.

• The effect does not disappear with monetary incentives for accuracy or when anchors are absurdly extreme. Quatronne found it for textbook prices with a $7128 anchor and for San Francisco’s temperatures with a 558 degree anchor.

More examples of underadjustment

• Bar-Hillel asked people to estimate the probability of getting at least one red ball in seven independent draws from an urn containing only 10% red balls. They severely underestimated this probability (.52), presumably because they underadjusted from the 10% anchor.

• Northcraft and Neale gave estate agents exactly the same property details except that selling prices varied from 12% below to 12% above the true asking prices. Agents estimated true values, recommended selling prices, reasonable purchase prices and lowest offers. All were affected by anchoring on the asking prices.

• Preceding questions can supply anchors in opinion polls.

Representativeness

• Likelihood is judged by the degree to which something represents or resembles something else.

• Which birth order is more likely? BBBGGG or GBBGBG

• Hence explains gambler’s and hot-hand fallacies• Probability matching: People match their response

probabilities of forecasting red and black to the probabilities with which they have encountered those colours in the past - eg giving 68% not 80% correct.

• Explains the conjunction fallacy (Linda).• Explains base-rate neglect (lawyer/engineer problem).

Availability

• Availability is used to ‘assess the frequency of a class or the probability of an event by the ease with which instances or occurrences can be brought to mind’ (T & K, 1974).

• Are words of 3+ letters more likely a) to start with ‘k’ or b) to have ‘k’ as their third letter? 105/152 people answered a) but b) is twice as frequent as a).

• Slovic showed that the public overestimate the frequency of rare risks (much publicity) but underestimate the frequency of common ones (no publicity). Public vs private transport; smoking vs illegal drugs.

Criticisms of the question-answering paradigm

• Lack of incentives: This should add noise not bias. Also effects are not eliminated by the introduction of substantial incentives (eg Grether & Plott, 1979; Lichtenstein & Slovic, 1973).

• People may not make the same inferences after being told a fact and after observing a fact: K & T could not exclude this possibility. But work on forecasting from ‘facts’ shows the same effects.

• We would not have evolved heuristics that lead to systematically biased judgment: Heuristics may have been adaptive. Also not all of them lead to biases.

Recent developments

• Gigerenzer argues that problems associated with use of heuristics has been over-emphasized.

• He has shown that use of a ‘familiarity’ heuristic can produce accurate judgments and can lead to an ‘inverse expertise’ effect (German cities, equities).

• This heuristic appears akin to the availability heuristic identified by Kahneman & Tversky.

• Gigerenzer has identified a number of other ‘simple heuristics that make us smart’.

Judgment in forecasting

• In formal forecasting, judgment enters into the model building or model selection process.

• In many cases, judgment is used to make adjustments to the output of the formal model. In some domains, this appears beneficial (economics, meteorology); in others, it is not (eg sales).

• Many surveys have shown that use of judgment alone is the most commonly used forecasting method in medium and small sized enterprises.

Heuristics in judgmental forecasting and judgmental adjustment of forecasts

• A large body of research on the use of judgment in forecasting has established a number of effects (‘biases’) that are explicable in terms of the heuristics suggested by K & T.

• I’ll focus on anchoring and representativeness but availability influences forecasts too. (Will there be more murders or suicides in London next year?)

• See Wright G & Goodwin P (1998) Forecasting with judgment. New York: Wiley.

Anchoring leading to trend-damping

0

4

8

0 7

Months

Sale

s (0

00s)

People are presented with data (red) and make a forecast (black). On average, the last datum is on the trend line. People use this as an anchor and make an insufficient adjustment for trend. Thus damping is observed.

Anchoring leading to non-independent forecasts

0

1

2

3

4

5

6

0 5 10

Months

Sale

s (0

00s)

People are presented with independent data points (red) and make a forecast (black). They use the last data point as an anchor and make an insufficient adjustment towards the mean. Thus they appear to perceive sequential dependence in the data when there is none.

Anchoring leading to ‘overconfidence’ in forecasts

0

2.5

5

0 4 8

Months

Sale

s (0

00

s)

People are presented with data (red), make a forecast (black), and then set an (eg 80%) prediction interval. They use their forecast as an anchor. Underadjustment from this point results in the interval being too narrow. Thus they appear overconfident.

Using statistical forecasts to modify judgmental ones: Underadjustment again

0

5

0 5 10 15

Months

Sale

s (0

00s)

People are presented with data (red), make an initial forecast (black), are given a good statistical forecast that has proved better than their own in the past (blue), and make a final forecast (green). Underadjustment away from their initial forecast depresses their performance. (Their own forecast is weighted twice as much as the statistical one.)

Representativeness leading to noise in a sequence of forecasts

0

3

6

0 5 10 15

Months

Sale

s (0

00s)

People are presented with data (red) and make several forecasts (black). These are not in the most likely positions for future points (the regression line); they represent a sequence that is as noisy as the one that will occur. Noisier data give noisier forecast sequences.

The ‘whys’ and ‘hows’ of heuristics: Anchoring

• In nature, linear trends are part of long-term cycles. Trend damping is more appropriate for cyclical series.

• Surveys suggest that natural series often contain first-order sequential dependence. Incomplete regression to the mean is appropriate in such cases.

• Heuristics may be less appropriate when applied to ‘unnatural’ (eg financial) series characteristic of modern technological societies.

• Anchoring appears to occur because a memory representation of the anchor remains active and primes responses in its vicinity. Extreme anchors have less effect.

The ‘whys’ and ‘hows’ of heuristics: Representativeness and availability

• Representativeness can be appropriate. For example, probability matching allows effective sharing of resources across consumers (cf Milinski’s fish).

• Gigerenzer’s work on familiarity shows how effective use of the availability heuristic can be. The exponential decay characteristics of the forgetting may also be useful for weighting past data when using them to make forecasts.

Conservatism and possible realisations of it

• Conservatism is a judgment bias originally reported by Phillips & Edwards (1966) in this type of experiment:

Bag 1: 30% red & 70% blue balls Bag 2: 70% red & 30% blue balls A bag is chosen randomly and, of 12 balls drawn from it

randomly, 8 are red and 4 are blue. What is the percentage probability balls were drawn from Bag 2? People estimate 70-80% when Bayes indicates 97%. They are conservative in changing their beliefs from the original 50% probability.

• Incrementalism in policy change (Lindblom, 1959).• Primacy effects in jury decision making.• Managerial resistance to change (Langevoort, 2000).• Often attributed to stress-reducing and motivational factors.

Is conservatism distinct from anchoring?

• Poulton (1994) and Kleindorfer et al (1993) argue conservatism is just an example of underadjustment from a mental anchor.

• Baron (1988) regards anchoring merely as one of a number of phenomena showing we are biased in favour of our current opinions. Conservatism subsumes anchoring.

• Harvey & Harries (In press) propose that they are separate and distinct phenomena.

The distinction between anchoring and conservatism

• Anchoring does not occur where the interval between context and target is long and filled with other judgments or items (Parducci, 1995).

• Only conservatism predicts that people will be influenced by an opinion that they believe they held earlier even though they did not. Anchors must really have been present.

• Harvey & Harries (In press) designed judgmental forecasting experiments to produce effects that could only be characterised as conservatism.

Harvey & Harries Experiment 1

• People made forecasts for 45 consecutive outcomes in a time series. Then they combined forecasts from four unbiased advisors for the same outcomes. In a control condition, tasks were in the opposite order.

• Combination performance was worse in the forecast-then-combine condition. Only in that condition did combination responses show the trend-damping that was evident in their forecasting.

• Forecasts had a long-term contextual effect.

Harvey & Harries Experiment 2

• People made forecasts. Then combined forecasts for the same events. Three of these forecasts were from unbiased advisors. The fourth was either the subject’s own forecast labelled as their own or as Dan’s or else it was a yoked subject’s forecast labelled as their own or as Dan’s.

• People put less weight on the fourth forecast when it was someone else’s labelled as someone else’s. They put more weight on it when it was their own or they believed it to be their own. Pro conservatism.

Experiment 2: Further interpretation

• In terms of conservatism, why did they put more weight on their own forecasts labelled as someone else’s? Presumably, they retrieved their own forecast, found it was the same as Dan’s, and, being conservative, put additional weight on a forecast that was the same a their own.

• There was no indication that people noticed the deceit. Presumably, to avoid needless cognitive effort, they retrieved their own forecasts from memory only when they were not displayed for them. Thus the conflict was never noticed.

Summary

• People use heuristics when assessing evidence in relation to hypotheses and when using data to generate hypotheses.

• People making judgments tend to stick too closely to their initial views or are too influenced by prior information.

• These effects may arise either through use of anchoring (a cognitive heuristic) or because of a tendency towards conservatism (likely to have a motivational basis).

• These accounts have different implications. They need to be distinguished. This is difficult but not impossible.

FIN