Top Banner
Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew University of Jerusalem Pragmatic analysis of the Asian Disease problem
69

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Dec 22, 2015

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected]

1

Szklarska Poreba -9 – 2008By Oshri Weiss

The IJN and the Hebrew University of Jerusalem

Pragmatic analysis of the Asian Disease problem

Page 2: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 2

Talk outline

• Introduction The Rationality Debate

• Empirical study Goal Methodology Main predictions Experiment 1 Experiment 2 Experiment 3 Experiment 4

• Open questions and Further study

• Appendices Appendix A: References Appendix B: The invariance rule Appendix C: Results Appendix D: Quotes

• Pragmatic considerations Related to the Asian disease

problem

Page 3: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 3

Introduction - The Rationality Debate

The standard methodology used for reasoning studies:• A:

Present a reasoning task with a given construction in mind • B:

Use the responses in order to infer and evaluate the reasoning process

The problem: The inference is valid only if the respondents are

using the same construction of the task as intended by the author

Page 4: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 4

Introduction - The Rationality Debate

The task:• Hypothetical statement:

If it is raining (p) than it is cloudy (q)• Categorical premise:

It is raining (p); It is cloudy (q); It is not raining (~p); It is not cloudy (~q)

• Conclusion: 1. p, 2. q, 3. ~p, 4.~q

The inadequacy of the Standard Methodology

Models:• Standard model: p →q• Alternative models: A: q → p B: p ↔q

A demonstration by a typical task of deductive reasoning:

Page 5: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 5

Introduction - The Rationality Debate

Standard model:• Valid inferences: (MP,MT)• Invalid inferences: (AC, DA)

Alternative model A:• Valid inferences: (AC,DA)• Invalid inferences: (MP,MT)

Alternative model B:• Valid inferences: (MP,MT, AC, DA)

In addition to misevaluating the validity of the respondents’ answers,

the standard model may assign to the respondents the wrong inferences scheme

The inadequacy of the Standard Methodology

Evaluating the respondent’s answers:

Page 6: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 6

Introduction - The Rationality Debate

There is asystematic diversity from normative models of decision - making and rational judgment

Conclusion

Often, the respondents are using a different construction of the task than the constrictions intended by the authors. Considering these alternative constructions the respondents answers do not violate the rules of rational thinking

and the criticism The “standard methodology”

Page 7: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 7

Introduction - The Rationality Debate

To evaluate the respondents’ reasoning according their own interpretation of the task and the given information

Alternative methodology

Considering the respondents’ own interpretation, the inference from their answer to the reasoning task and the underlying cognitive process is valid

Page 8: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 8

Pragmatic Considerations

The Asian disease problem

Page 9: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 9

Pragmatic Considerations

• The Experiment A choice between a certain and a probable outcome is presented in

two versions once from a positive perspective and once from a negative perspective

The two versions share the same cover story and differ only in the choice of alternatives

• The Findings Majority of subjects have chosen the certain option when the

outcomes are phrased in positive terms 72%

Majority of subjects have chosen the probable outcome when the outcomes are phrased in negative terms

78%• The Interpretation

Since the two sure and the two probable outcomes are equivalent this answer pattern is violating the

invariance rule

The Asian disease problem By Tversky and Kahneman (1981)

Page 10: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 10

The Asian disease problem

• According to K&T’s standard construction of the problem the two certain outcomes are equivalent thus the outcome that 200 people will be saved implies the outcome that 400

people will die and vice versa

• The information given for the sure outcomes is incomplete thus in the positive framing condition it is only mentioned explicitly that

200 people will be saved and in the negative framing condition it is only mentioned explicitly that 400

people will die.

The argument of the opposing view

• This leads to alternative constructions of the problem the certain outcomes allows that more than 200 people will be

saved in the positive framing and more than 400 people will die in the negative framing

Page 11: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 11

The Asian disease problem

• Under the alternative construction of the problem: the certain outcome in the positive framing condition is

better in terms of expected utility than the certain outcome in the negative framing condition.

The argument of the opposing view

The answer pattern observed by K&T does not indicate a violation of the invariance rule

Therefore

Page 12: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 12

The empirical study

Relating interpretations

to subjects’ decisions

Page 13: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 13

The empirical study

To answer the question:

The goal of the study

“In what extent subjects’ interpretation of the given information

determines their choices?”

Page 14: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 14

The empirical studyThe methodology of the study

• The methods I have used to address this question:

1. Manipulating the interpretation of the disputed information (the certain outcomes) and testing the effect of this manipulation on the subjects' responses

2. Detecting the actual interpretation of disputed information by the respondents

• Combining the two methods would provide a stronger evidence for the alleged relation between the interpretation and the choice, than using either one of these two methods separately

Page 15: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 15

The methodology of the empirical study

Changes to the original problem

• Encouraging the respondents to make the “standard” interpretation of the problem by adding the word "exactly" to the description

of the certain outcomes the certain outcome in the positive framing was

changed to: ▪ If program A is adopted, exactly 200 people will be saved

the certain outcome in the negative framing was cahnged to:

▪ If program C is adopted, exactly 400 people will die

Page 16: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 16

The methodology of the empirical study

Detecting the subject’s interpretation

Following Dulany & Hilton the phrasing of this question was:What do you think the writer meant by the last statement: positive framing : “200 (exactly 200) people will be saved”

negative framing : “400 (exactly 400) people will die”

• To detect the interpretation of the certain outcome of each subject: I have used a multiple-choice question this method provides clear answer to the question

whether the subject had an alternative interpretation for the certain option

as opposed to other methods, which do not limit the subjects to a predetermined answer set

Page 17: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 17

The methodology of the empirical study

Detecting the subject’s interpretation 2

• The subjects were then presented with a set of options representing possible interpretations of the certain outcomes

• Two types of options were presented: “standard” interpretation “alternative” interpretations of the certain outcome

Page 18: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 18

The empirical study

Main predictions

1. Adding the word “exactly” would encourage the respondents to make the standard interpretation

2. This predicted influence on the respondents’ interpretation would be followed by a change in their choices.

3. Subjects who have the alternative interpretation will be more likely to choose the certain outcome in the positive framing and less likely to choose it in the negative framing.

Page 19: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 19

The empirical study: Experiment 1

Specific characteristics

• This experiment tested only the positive framing condition Conditions

The subjects were randomly assigned to one of the two conditions: a. the “original positive” version of the problem b. “exactly positive” version (the word “exactly was added to the description of the

certain outcome) There were no other differences between the two versions

Language Hebrew

Time and place The Hebrew University of Jerusalem, 2006

Subjects 191 participants

Material Pen and pencil test

The order of questions: To choose between the two programs To answer the question designed to detect their interpretation

Page 20: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 20

The empirical study: Experiment 1

The set of possible interpretations

• The complete phrasing of the question was designed to detect the respondent’s interpretation: What do you think the author meant by the sentence: (exactly) 200 people will

be saved? There is no "correct" answer to this question, just choose the interpretation Which is most suited to your understanding of the author's intention when you

were making your choice? Please mark one option:

1. 200 people will be saved now, and 400 people will die now.2. It is certain that 200 people will be saved, and it is likely that 400 people

will die3. 200 people will be saved, and it is possible that more people will be saved4. It is certain that 200 people will be saved, and it is certain that 400 people

will die.

• Choosing options 1 or 4 was considered as indicating the standard interpretation of the certain outcome 200 people will be saved and 400 people will die

• Choosing options 2 or 3 was considered as indicating the alternative interpretation of the certain outcome it is possible that more than 200 people will be saved.

Page 21: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 21

The empirical study: Experiment 1

• Prediction A There will be more “perhaps more” interpretations in the

“positive original” version than in the “positive exactly” version.

• Prediction B More respondents will be choosing the certain outcome in

the “positive original” version than in the “positive exactly” version.

• Prediction C The participants who interpret the certain outcome as

allowing that more than 200 people will be saved (as indicated by choosing options 2 or 3) will be more likely to choose the certain outcome than the participants who have the standard interpretation of the certain outcome (as indicated by choosing options 1 or 4)

Predictions

Page 22: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 22

The empirical study: Experiment 1

• Prediction A - Confirmed Subjects were more likely to make the: “perhaps more”

interpretation in the: “positive original” version than in the: “positive exactly” version

• Prediction B - Confirmed Subjects were more likely to choose the certain outcome in

the: “positive original” version than in the: “positive exactly” version

• Prediction C - Confirmed The participants who interpreted the certain outcome as

allowing that more than 200 people will be saved were more likely to choose the certain outcome than the participants who make the standard interpretation of the certain outcome

Results for experiment 1

Page 23: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 23

The empirical study: Experiment 1

Discussion

• The confirmation of all the three predictions of the Pragmatic analysis support the claim The preference of the certain alternative in the positive

framing of the Asian Disease problem is due to the phrasing of this alternative.

• But there is an alternative explanation to the findings of experiment 1: Since the "interpretation question" was presented

after the choice, the subjects' answers to the "interpretation question" could be influenced by the desire to justify their choice.

Page 24: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 24

The empirical study: Experiment 2The goal:

• The method it was assured that the respondents’ answers to

the “interpretation question” would not be influenced by the desire to justify their choice

• The technique

the “interpretation question” was presented immediately after the problem and before presenting the choice between the two alternative programs

to rule out the alternative explanation of the findings of experiment 1

Page 25: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 25

The empirical study: Experiment 2

Specific characteristics

• The changed order of questions is the only difference between the designs of experiments 2 and experiment 1 Conditions

The subjects were randomly assigned to one of two conditions: a. the “original positive” version of the problem b. the “exactly positive” version ▪ There were no other differences between the two versions

Language Hebrew

Time and place The Hebrew University of Jerusalem, 2006

Subjects 74 participants

Material Pen and pencil test

The order of questions: To answer the question designed to detect their interpretation To choose between the two programs

Page 26: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 26

The empirical study: Experiment 2

• Prediction A - Confirmed Subjects were more likely to make the “perhaps more”

interpretation in the “positive original” version than in the “positive exactly” version

• Prediction B - Confirmed Subjects were more likely to choose the certain outcome in

the “positive original” version than in the “positive exactly” version

• Prediction C - Confirmed The participants who interpreted the certain outcome as

allowing that more than 200 people will be saved were more likely to choose the certain outcome than the participants who make the standard interpretation of the certain outcome

Results for experiment 2

Page 27: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 27

The empirical study: Experiment 2

Discussion

• The findings of experiment 2 support the findings of experiment 1 by:

Establishing the independence between the confirmation of predictions A and B

Replicating the results of experiment 1

Page 28: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 28

The empirical study - The first 2 experiments

Conclusions

1. The alternative interpretation of the certain outcome as allowing that more than 200 people will be saved accounts (at least partially) for the subjects' preference of the certain outcome in the positive framing condition

2. These findings questions the claim that the seemingly inconsistent responses observed by K&T are reflecting a violation of the invariance rule

Page 29: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 29

The empirical study - The first 2 experiments

Problems and criticism

1. What about the negative framing? The first 2 experiments have not tested the negative framing

condition and therefore they are not sufficient for answering the question of whether or not the subjects’ answers violate invariance

2. Interpreting the numbers as a maximum amount: Numbers are sometimes used to indicate an upper limit

Mother to the child: “You can eat 3 cookies” Some of the subjects may have such interpretation of the

certain outcomes If subjects who make the “at least 200 people will be saved”

interpretation in the positive framing also make the “at most 400 people will die” interpretation in the negative framing

Then choosing the certain outcome in the positive framing and the probable outcome in the negative framing is still a violation of the invariance rule

Page 30: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 30

The empirical study: Experiment 3

Goal:

• Addressing the negative framing as well by: Adding 2 additional conditions

• Testing the “at most 200(400)” interpretations: adding an option reflecting the

“200 (400) and maybe less” interpretation to the set of multiple choices of the “interpretation question”

To address the limitations of experiments 1 and 2

Page 31: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 31

The empirical study: Experiment 3

Specific characteristics

• In this experiment both, the positive and the negative framing conditions were tested. Conditions

The subjects were randomly assigned to one of four conditions: 1. the “original positive” version of the problem 2. the “exactly positive” version 3. The “original negative” version 4. the “exactly negative” version

Language English

Time and place Various gatherings of the English speaking community in Paris, 2007

Subjects 129 participants

Material Pen and pencil test

The order of questions: To choose between the two programs To answer the question designed to detect their interpretation

Page 32: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 32

The empirical study: Experiment 3

The set of possible interpretations

1. 200 people will be saved now and 400 people will die now2. It is certain that 200 people will be saved, and it is likely that 400 people will

die3. 200 people will be saved, and it is possible that more than 200 people will be

saved4. It is certain that 200 people will be saved, and it is certain that 400 people

will die.5. 200 people will be saved, and it is possible that less than 200 people

will be saved

• Choosing options 1 or 4 were considered as indicating the standard interpretation of the certain outcome

that 200 people will be saved and 400 people will die• Choosing options 2 or 3

were considered as indicating interpretation that at least 200 people will be saved

• Choosing option 5 was considered as indicating interpretation that at most 200 people will be saved.

The positive framing conditions

Page 33: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 33

The empirical study: Experiment 3

1. 400 people will die now and 200 people will be saved now.2. It is certain that 400 people will die, and it is likely that 200 people will be

saved.3. 400 people will die , and it is possible that more than 400 people will die.4. It is certain that 400 people will die , and it is certain that 200 people will be

saved.5. 400 people will die , and it is possible that less than 400 people will

die.

• Choosing options 1 or 4 were considered as indicating the standard interpretation of the certain outcome

that 400 people will die and that 200 people will be saved. • choosing options 2 or 3

were considered as indicating interpretation that at least 400 people will die • choosing option 5

was considered as indicating interpretation that at most 400 people will die

The set of possible interpretationsThe negative framing conditions

Page 34: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 34

The empirical study - Experiment 3

• Positive framing The influence of adding the word “exactly” to the

interpretation of the certain outcome: Reduction in the number of: “at least 200” and: “at most 200”

interpretation An increase in the number of: “exactly 200” interpretations The reduction in the number of: “at least 200” interpretations

was smaller than in experiments 1 and 2

Results – experiment 3

Interpretations

• Negative framing The influence of adding the word “exactly” on the

interpretation of the certain outcome There was not any considerable difference between the two

negative versions

Page 35: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 35

The empirical study – Experiment 3

• The proportion of risk averse subjects Contrary to the predictions of the Pragmatic Analysis

subjects were more likely to choose the certain outcome in the “negative original” version

▪ 24% than in the “negative exactly” version

▪ 20%

Negative framing

Results – experiment 3

• The correlation between the interpretation and choice Contrary to the predictions of the Pragmatic Analysis

Page 36: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 36

The empirical study: Experiment 3

Discussion

• The results of the positive framing provide additional evidence to the claimed relation between the interpretation and choice

The fact that the subjects are much more likely to make the “at least 200” interpretation in the positive framing than to make the “at most 400” interpretation in the negative framing suggests that there is a considerable group of respondents that considering their own interpretation of the certain outcomes do not violate invariance

On the other hand, the findings of the negative framing does not confirm the hypothesis that the preference of the probable outcome in the negative framing condition is related to the interpretation of the certain outcome.

Page 37: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 37

The empirical study – Experiment 3

Open questions

Why are the negative and positive framings different?

• Suggested answer It is possible that some respondents view the description of

the programs with the certain outcomes as a commitment made by the proposing scientists manifesting a worse case scenario.

A respondent with this construction in mind would have both the “at least 200” and the “at most 400” interpretations

The existence of a group of subjects with such a construction could explain the findings that adding the word exactly reduces the proportion of risk averse choices in the positive framing condition but does not increases the proportion of risk averse choices in the negative framing condition

• Nevertheless the fact that the “at most 400” interpretation in the negative framing is not as common as the “at least 200” interpretation in the positive framing raises some doubts

Page 38: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 38

The empirical study: Experiment 3

Problems with the set of possible interpretations

• The phrasing of the options representing the “standard interpretation”

• The task of comparing between these 5 options might be too demanding for some of the subjects

• The set includes only 1 option reflecting the “perhaps less” interpretation whereas there are 2 options representing each of the two other interpretations

Standing issues:

Page 39: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 39

The empirical study: Experiment 4Goal:

• Addressing the first issue The phrasing of the option reflecting the standard

interpretation was changed

• Addressing the second and third issues: The answer set includes only 3 options in order to:

reduce the cognitive load involved in comparing the alternatives equalize the number of options representing each possible

interpretation

In addition, the respondents could specify their interpretation in their own words.

To simplify and to improve the set of possible interpretations.

Page 40: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 40

The empirical study: Experiment 4Specific characteristics

• Conditions The subjects were randomly assigned to one of four conditions:

1. “positive original 2. “positive exactly”3. “negative original”4.“negative exactly”

• Language English

• Time and place Various gatherings of the English speaking community in Paris, 2007-2008

• Subjects 118 participants.

• Material Pen and pencil test

• The order of the questions For half of the subjects the order was:

To answer the question designed to detect their interpretation To choose between the two programs

For the other half of the subjects the order was reversed: To choose between the two programs To answer the question designed to detect their interpretation

Page 41: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 41

The empirical study: Experiment 4

1. 200 people, and perhaps more, will be saved2. 200 people, no more and no less, will be saved3. 200 people, and perhaps less, will be saved4. Other (please specify):

• Option 1 reflects the: “at least 200” interpretation

• Option 2 reflects the: “exactly 200” interpretation

• Options 3 reflects the: “at most 200” interpretation

The set of possible interpretationsThe positive framing conditions

Page 42: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 42

The empirical study: Experiment 4

1. 400 people, and perhaps more, will die2. 400 people, no more and no less, will die3. 400 people, and perhaps less, will die4. Other (please specify):

• Option 1 reflects the: “at least 400” interpretation,

• Option 2 reflects the: “exactly 400” interpretation

• Options 3 reflects the: “at most 400” interpretation

The set of possible interpretationsThe negative framing conditions

Page 43: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 43

The empirical study - Experiment 4

• Positive framing The influence of adding the word “exactly” on the

interpretation of the certain outcome: Reducing the number of: “at least 200” “at “most 200”

and: “more or less 200” (other). Increasing the number of: “exactly 200”

Results – experiment 4

Interpretations

• Negative framing The influence of adding the word “exactly” on the

interpretation of the certain outcome: Reducing the number of: “at least 400” and “at “most

400” Increasing the number of: “exactly 400”

Page 44: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 44

The empirical study - Experiment 4

• Positive framing Contrary to the previous studies:

The participants with the: “at least 200” interpretation were as likely to choose the certain outcome as the participants with the: “exactly 200” interpretation

Subjects from both groups were more likely to choose the certain outcome than the participants who make the: “at most 200” interpretation

The correlation between the interpretation and choice

Results – experiment 4

• Negative framing The results were mixed:

The participants who had the: “at least 400” interpretation were more likely to choose the certain outcome.

Than the participants who make the standard interpretation of the certain outcome.

Contrary to the predictions of the Pragmatic Analysis But less likely than the participants who make the interpretation that

it is possible that less than 400 people will die In accordance with the predictions of the Pragmatic Analysis

Page 45: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 45

The empirical study – Experiment 4

• Negative framing Contrary to the predictions of the Pragmatic

Analysis subjects were more likely to choose the certain

outcome in the “negative original” version▪ 31%

than in the “negative exactly” version ▪ 10%

The proportion of risk averse subjects

Results – experiment 4

Page 46: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 46

The empirical study: Experiment 4

Summary of the results

• The influence of adding the word “exactly” on the proportion of risk averse choices in the positive framing condition was replicated

• As opposed to the previous studies there was not a strong relation between the interpretation and choice neither in the positive nor in the negative conditions

• Comparing to the findings of experiment 3 the most striking difference is a reduction in the number of subjects making the “at least n” interpretation, particularly in the positive framing condition

Page 47: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 47

The empirical study – Experiments 3 and 4

Open questions

Which one of the two set of possible interpretations is better?

One might argue that the set used in experiment 3 over estimated the proportion of subjects making the “at least 200 (400)” interpretations

Therefore, the set used in experiment 4 is more adequate for reflecting the respondent’s construction of the task

But in that case ,the correlation between the interpretation and choice found in experiments 1-3 should be even stronger in experiment 4

Thus, there is a dissonance between the theoretical adequacy of the set used in experiment 4 and the empirical confirmation of the set used in experiment 3

Page 48: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 48

Further study 1

Using a between-subject design

• The Method Experimentally manipulating the alternative

interpretations by adding two additional phrasing: “At least 200 (400) people will be saved (die)” “At most 200 (400) people will be saved (die)”

• Advantages: To provide a measure of the relation between having

the alternative interpretations and the choices which does not rely on the phrasing of the interpretation question

• Disadvantages: Requires many respondents

Additional experiment

Page 49: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 49

Further study 2

• K&T study and my own experiments were using a between- subject design: in which each participant is presented with either the

positive or the negative version of the problem. In within-subjects design each subject is presented with both

the positive and the negative version of the problem.

• The method: To detect each one of the respondents’ interpretation of the

presented problem in a with-in subject design

• The benefits: Estimating the proportion of subjects who violate invariance

considering their own interpretation Studying the relation between the tendency to have

alternative interpretation to several factors that underlines inter personal variety in reasoning task, such as general intelligence Stanovich & West (1998)

Using a within-subject design

Page 50: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 50

Appendix A: References

• Berkeley, D. & Humphreys, P. “Structuring decision problems and the "bias heuristic“ “. Acta Psychologica, 50, 201-252, 1982

• Dulany, D.L & Hilton D.J.:"Conversational Implicature, Conscious Representation, and the Conjunction Fallacy." Social cognition 9, (85-109), 1991

• Hilton D.J.: "The Social Context of Reasoning: Conversational Inference and Rational Judgment".

Psychological Bulletin, 118, No 2 (248-271), 1995 • Kahneman D. and Tversky A :• "The Framing of Decisions and the Psychology of Choice". Science , 211, No 30 (453-458), 1981• "Choices, Values and Frames" American Psychologist, 39 No 4 (341-350), 1984 • "Theoretical Notes On the Reality of Cognitive Illusions". Psychological Review, 103, No 3,

(582-591), 1996• Kühberger A.: "The Framing of Decisions: A new look at Old Problems". Organizational Behavior and Human

Decision Processes, 62, No2, (230-240), 1995 • Politzer G.: "Reasoning, Judgment and Pragmatics" in Dan Sperber and Ira A. Noveck (ed.) Experimental

Prgmatics, Palgrave Macmillan New York (94-115), 2004 • Politzer G. and Noveck I.:

"Are Conjunction Rule Violations the Result of Conversational Rule Violations?" Journal of Psycholinguistics Research, 20, No.2 (83-103), 1991

• Stanovich, K. E., & West, R. F. • "Individual differences in framing and conjunction effects." Thinking and Reasoning, No 4,

(289–317), 1998 • "Individual differences in reasoning: Implications for the rationality debate?" Behavioral and

Brain Sciences, 23 (645–726), 2000

Page 51: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 51

Appendix B: The Asian disease problem

• The cover story Imagine that the U.S. is preparing for the outbreak of an unusual

Asian disease, which is expected to kill 600 people. Two alternative programs to combat the disease have been proposed. Assume that the exact scientific estimates of the consequences of the programs are as follows:

Positive framing If Program A is adopted, 200 people will be saved.  If Program B is adopted, there is a 1/3 probability that 600 people will

be saved, and a 2/3 probability that no people will be saved.  Which of the two programs would you favor? 

Negative framing If Program C is adopted, 400 people will die.  If Program D is adopted, there is a 1/3 probability that nobody will

die, and a 2/3 probability that 600 people will die.

by Tversky & Kahneman

Page 52: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 52

Appendix B: The Asian disease problem

• The choice between outcomes should not be dependent on the way in which they are described.

The invariance rule

Page 53: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 53

Appendix C: Results

• Confirmed Subjects were more likely to make the: “perhaps more”

interpretation in the: “positive original” version 57%

than in the: “positive exactly” version 32%

prediction A

• This difference was found to be statistically significant in a logistics regression analysis: with the version as the predicting variable and the

interpretation as the predicted variable Chi square =12.1, p=0.001

The empirical study - Experiment 1

Page 54: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 54

• Confirmed Subjects were more likely to choose the certain outcome in

the: “positive original” version 68%

than in the: “positive exactly” version 52%

Results – prediction B

• This difference was found to be statistically significant in a logistics regression analysis: with the version as the predicting variable and the choice as

the predicted variable Chi square =5.07, p=0.024

Appendix C: Results

The empirical study - Experiment 1

Page 55: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 55

• Confirmed The participants who interpreted the certain outcome as

allowing that more than 200 people will be saved were more likely to choose the certain outcome 69%

than the participants who make the standard interpretation of the certain outcome 53%

Results – prediction C

• This difference was found to be statistically significant in a logistics regression analysis: with the interpretation as the predicting variable and the

choice as the predicted variable Chi square =5.39, p=0.02

Appendix C: ResultsThe empirical study - Experiment 1

Page 56: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 56

• Confirmed Subjects were more likely to make the “perhaps more”

interpretation in the “positive original” version 58%

than in the “positive exactly” version 25%

Results – prediction A

• This difference was found to be statistically significant in a logistics regression analysis: with the interpretation as the predicting variable and the

choice as the predicted variable Chi square =7.77, p=0.005

Appendix C: ResultsThe empirical study - Experiment 2

Page 57: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 57

• Confirmed Subjects were more likely to choose the certain outcome in

the “positive original” version 61%

than in the “positive exactly” version 47%

Results – prediction B

• This difference was not found to be statistically significant in a logistics regression analysis: with the version as the predicting variable and the choice

as the predicted variable Chi square = 2.11, p=0.146

Appendix C: ResultsThe empirical study - Experiment 2

Page 58: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 58

• Confirmed The participants who interpreted the certain outcome as

allowing that more than 200 people will be saved were more likely to choose the certain outcome 72%

than the participants who make the standard interpretation of the certain outcome 46%

Results – prediction C

• This difference was found to be statistically significant in a logistics regression analysis: with the interpretation as the predicting variable and the

choice as the predicted variable Chi square = 4.83, p=0.028

Appendix C: ResultsThe empirical study - Experiment 2

Page 59: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 59

• The influence of adding the word “exactly” to the interpretation of the certain outcome:

Results – Positive framing

Interpretation/version

“Positive original”

“Positive exactly”

“At least 200” 57% 50%

“Exactly 200” 29% 47%

“At most 200” 14% 3%

Interpretations

Appendix C: ResultsThe empirical study - Experiment 3

Page 60: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 60

• As in the previous studies

Subjects were more likely to choose the certain outcome in the “positive original” version 63%

than in the “positive exactly” version 53%

The proportion of risk averse subjectsResults – Positive framing

Appendix C: ResultsThe empirical study - Experiment 3

Page 61: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 61

• As in the previous studies The participants who interpreted the certain

outcome as allowing that more than 200 people will be saved were more likely to choose the certain outcome 80%

than the participants who make the standard interpretation of the certain outcome 38%

and than the participants who make the interpretation that it is possible that less than 200 people will be saved 17%

The correlation between the interpretation and choice Results – Positive framing

Appendix C: ResultsThe empirical study - Experiment 3

Page 62: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 62

• The influence of adding the word “exactly” on the interpretation of the certain outcome

Interpretations

Interpretation/version

“Negative original”

“Negative exactly”

“At least 400” 42% 43%

“Exactly 400” 45% 40%

“At most 400” 13% 17%

Results – Negative framing

Appendix C: ResultsThe empirical study - Experiment 3

Page 63: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 63

• Contrary to the predictions of the Pragmatic Analysis the participants who interpreted the certain

outcome as allowing that more than 400 people will die, were more likely to choose the certain outcome 28%

than the participants who make the standard interpretation of the certain outcome 19%

and than the participants who make the interpretation that it is possible that less than 400 people will die 11%

The correlation between the interpretation and choice Results – Negative framing

Appendix C: ResultsThe empirical study - Experiment 3

Page 64: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 64

• The influence of adding the word “exactly” on the interpretation of the certain outcome:

Results – Positive framing

Interpretation/version

“Positive original”

“Positive exactly”

“At least 200” 17% 12%

“Exactly 200” 45% 81%

“At most 200” 17% 3%

Other 21% 3%

Interpretations

Appendix C: ResultsThe empirical study - Experiment 4

Page 65: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 65

• As in the previous studies

Subjects were more likely to choose the certain outcome in the “positive original” version 62%

than in the “positive exactly” version 53%

The proportion of risk averse subjectsResults – Positive framing

Appendix C: ResultsThe empirical study - Experiment 4

Page 66: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 66

the participants who interpreted the certain outcome as allowing that more than 200 people will be saved were not more likely to choose the certain outcome. 56%

than the participants who make the standard interpretation of the certain outcome. 59%

And both groups were more likely to choose the certain outcome tan the participants with the: “at most 200” interpretation 40%

Appendix C: ResultsThe empirical study - Experiment 4Results – Positive framingThe correlation between the interpretation and choice

Page 67: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 67

• The influence of adding the word “exactly” on the interpretation of the certain outcome

Interpretations

Interpretation/version

“Negative original”

“Negative exactly”

“At least 400” 30% 0%

“Exactly 400” 48% 90%

“At most 400” 15% 3%

Other 7% 7%

Results – Negative framing

Appendix C: ResultsThe empirical study - Experiment 4

Page 68: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 68

The empirical study - Experiment 4

the participants who interpreted the certain outcome as allowing that more than 400 people will die were more likely to choose the certain outcome. 31%

than the participants who make the standard interpretation of the certain outcome. 12%

But less likely than the participants who make the interpretation that it is possible that less than 400 people will die 40%

The correlation between the interpretation and choice

Results – Negative framing

Page 69: Pragmatic analysis of the Asian Disease problem by Oshri Weiss oshriweiss@gmail.com 1 Szklarska Poreba -9 – 2008 By Oshri Weiss The IJN and the Hebrew.

Pragmatic analysis of the Asian Disease problem by Oshri Weiss [email protected] 69

Appendix C: The Empirical study

• Why did you choose this program? (experiment 3, version: “positive original ”)

Subject A: “Because at least in program A there is an certainty that at least 1/3 of the likely victims will be saved…”

Subject B: “….Other 400 people affect uncertain to die in A are not necessarily predicted to die.”

Subject C (version: “positive exactly ”): “….also perhaps another measure could be found to supplement program A”

Quotes from subjects with alternative constructions