Top Banner
1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics. Friday 17 June 2022 02:23 AM
81

1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

Jan 15, 2016

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

1

Smoking Data

The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

Friday 21 April 2023 09:29 AM

Page 2: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

2

Smoking Data

ALA – American Lung Association

Variable

Name

Description

id participant identification number

gender 1 = female, 2 = male

program

3

2

1

= Standard ALA program plus nicotine anonymous program,

= Behavioural counselling plus exercise program,

= Behavioural counselling plus nicotine gum

pre Mean number of cigarettes smoked per day pre-intervention

post Mean number of cigarettes smoked following intervention

follow6 Mean number of cigarettes smoked at six month follow up

follow12 Mean number of cigarettes smoked at 12 month follow up

Page 3: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

3

Smoking Data

Note that gender has been coded

Page 4: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

4

Two-sample Correlated t-test

Do rates of smoking decrease from pre-intervention to post-intervention?

Page 5: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

5

Two-sample Correlated t-test Analyze > Compare Means > Paired-Samples t Test

Page 6: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

6

Two-sample Correlated t-test Highlight pre, move across. You will see that pre now appears as Variable 1 in the Paired Variables box.

Page 7: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

7

Two-sample Correlated t-test Highlight post, move across. You will see that post now appears as Variable 2 in the Paired Variables box.

Page 8: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

8

Two-sample Correlated t-test Click on OK to run the analysis or Paste to preserve the syntax.

Page 9: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

9

Two-sample Correlated t-test Syntax

GET FILE='\\Client\f$\spss\1\1.sav'.DATASET NAME DataSet1 WINDOW=FRONT.T-TEST PAIRS = pre WITH post (PAIRED) /CRITERIA = CI(.95) /MISSING = ANALYSIS.

Note, you can even include an option to load the data file.

Page 10: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

10

Two-sample Correlated t-test The first table of the printout contains descriptive statistics while the second table contains inferential statistics. Study the printout you can identify n, Mean, and Std for each group.

Paired Samples Statistics

30.53 30 5.406 .987

5.43 30 6.067 1.108

pre

post

Pair1

Mean N Std. DeviationStd. Error

Mean

Page 11: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

11

Two-sample Correlated t-test The second table contains inferential statistics. You can identify t, df, and p (p is in the column labelled "Sig. (2-tailed)").

Paired Samples Test

25.100 7.402 1.351 22.336 27.864 18.574 29 .000pre - postPair 1Mean Std. Deviation

Std. ErrorMean Lower Upper

95% ConfidenceInterval of the

Difference

Paired Differences

t df Sig. (2-tailed)

In this case, the mean number of cigarettes smoked prior to the intervention programs was significantly higher than the number of cigarettes smoked after the intervention programs, t29 = 18.57, p < 0.001.

Page 12: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

12

Caution In Psychology there is great reliance on the “p” value. Over the past twenty year serious flaws have been pointed out with this reliance. In general reporting of a confidence interval is recommended.

R. Hubbard and R.M. Lindsay, 2008, “Why p Values Are Not a Useful Measure of Evidence in Statistical Significance Testing” Theory and Psychology 18 69-88.

J.L. Moran et al., 2004, “A farewell to p-values” Critical Care and Resuscitation 6 130-137.

Page 13: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

13

Caution As pointed out by Johansson1. p is uniformly distributed under the null

hypothesis and can therefore never indicate evidence for the null.

2. p is conditioned solely on the null hypothesis and is therefore unsuited to quantify evidence, because evidence is always relative in the sense of being evidence for or against a hypothesis relative to another hypothesis.

3. p designates probability of obtaining evidence (given the null), rather than strength of evidence.

Page 14: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

14

Caution 4. p depends on unobserved data and subjective

intentions and therefore implies, given the evidential interpretation, that the evidential strength of observed data depends on things that did not happen and subjective intentions.

T. Johansson, 2011,“Hail the impossible: p-values, evidence, and likelihood” Scandinavian Journal of Psychology, 52, 113–125.

Page 15: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

15

Caution For an alternate but equally jaundiced view see

Valen E. JohnsonRevised standards for statistical evidenceProceedings of the National Academy of Sciences of the United States of America 2013 110(48) 19313–19317.

Which is nicely summarised in

Erika Check HaydenWeak statistical standards implicated in scientific irreproducibilityNature 11 November 2013.

and

Geoff CummingThe problem with p values: how significant are they, really?

Page 16: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

16

Caution Regina Nuzzo Scientific method: Statistical errorsNature Volume: 506, Pages: 150–152 13 February 2014

P values, the 'gold standard' of statistical validity, are not as reliable as many scientists assume.

However it seems to get the explanation of hypothesis testing wrong!

Page 17: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

17

Caution The misuse of asterisks in hypothesis testing

Dieter Rasch, Klaus D. Kubinger, Jörg Schmidtke and Joachim HäuslerPsychology Science, Volume 46(2), 2004, p. 227-242.

This paper serves to demonstrate that the practise of using one, two, or three asterisks (according to a type-I-risk α either 0.05, 0.01, or 0.001) in significance testing as given particularly with regard to empirical research in psychology is in no way in accordance with the Neyman-Pearson theory of statistical hypothesis testing. Claiming a-posteriori that even a low type-I-risk α leads to significance merely discloses a researcher’s self-deception. Furthermore it will be emphasised that by using sequential sampling procedures instead of fixed sample sizes the practice of asterisks” would not arise.

Page 18: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

18

Repeated Measures Analysis of Variance (ANOVA)

Do rates of smoking decrease across the four data collection periods. That is, does smoking not only decrease from pre-intervention to post-intervention but also does the rate continue to decrease during a 6-month and 12-month follow up?

Page 19: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

19

Repeated Measures Analysis of Variance (ANOVA)

Analyze > General Linear Model > Repeated Measures

Page 20: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

20

Repeated Measures Analysis of Variance (ANOVA)

In the Within Subject Factor Name box designate a name for the repeated measure factor, let’s call it rate.

Page 21: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

21

Repeated Measures Analysis of Variance (ANOVA)

In the Number of Levels window type in the number of time periods measured. In this case it is 4.

Page 22: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

22

Repeated Measures Analysis of Variance (ANOVA)

Click on Add.To generate rate(4)

Click on Define

Page 23: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

23

Repeated Measures Analysis of Variance (ANOVA)

Click on Define. A Repeated Measures box will appear.

Page 24: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

24

Repeated Measures Analysis of Variance (ANOVA)

Highlight your first time variable, pre, from the list of variables on the left, and click on the upper arrow button to move it into the Within Subject Variables window.

Page 25: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

25

Repeated Measures Analysis of Variance (ANOVA)

Add the remaining three time variables, post, follow6, and follow12, in the same fashion.

Finally Click on the Options button.

Page 26: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

26

Repeated Measures Analysis of Variance (ANOVA)

Click on the square next to the word Descriptive Highlight rate in the Factor(s) and Factor Interaction box. Click on the arrow button and click on the square next to the word Compare main effects. Finally click on Continue.

Page 27: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

27

Repeated Measures Analysis of Variance (ANOVA)

Finally click on OK to run the analysisor Paste to preserve the syntax.

Page 28: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

28

Repeated Measures Analysis of Variance (ANOVA)

Syntax

GLM pre post follow6 follow12 /WSFACTOR = rate 4 Polynomial /METHOD = SSTYPE(3) /EMMEANS = TABLES(rate) COMPARE ADJ(LSD) /PRINT = DESCRIPTIVE /CRITERIA = ALPHA(.05) /WSDESIGN = rate.

Page 29: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

29

Repeated Measures Analysis of Variance (ANOVA)

You can identify n, Mean, and Std for each of the four time periods in the descriptive statistics output.

Descriptive Statistics

30.53 5.406 30

5.43 6.067 30

7.77 6.095 30

8.20 6.509 30

pre

post

follow6

follow12

Mean Std. Deviation N

Page 30: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

30

Repeated Measures Analysis of Variance (ANOVA) -

Sphericity ANOVAs with repeated measures (within-subject factors) are particularly susceptible to the violation of the assumption of sphericity. Sphericity is the condition where the variances of the differences between all combinations of related groups (levels) are equal. Violation of sphericity is when the variances of the differences between all combinations of related groups are not equal. Sphericity can be likened to homogeneity of variances in a between-subjects ANOVA.

Page 31: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

31

Repeated Measures Analysis of Variance (ANOVA) -

Sphericity When the probability of Mauchly's test statistic is greater than or equal to .05 (i.e., p > .05), we fail to reject the null hypothesis that the variances are equal. Therefore we could conclude that the assumption has not been violated. However, when the probability of Mauchly's test statistic is less than or equal to .05 (i.e., p < .05), sphericity cannot be assumed and we would therefore conclude that there are significance differences between the variances.

It should be noted that sphericity is always met for two levels of a repeated measure factor and it is, therefore, unnecessary to evaluate.

Page 32: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

32

Repeated Measures Analysis of Variance (ANOVA)

Examine Mauchly's test of Sphericity to determine if the homogeniety of variance assumption is met.

Mauchly's Test of Sphericityb

Measure: MEASURE_1

.037 91.590 5 .000 .427 .438 .333Within Subjects Effectrate

Mauchly's WApprox.

Chi-Square df Sig.Greenhouse-Geisser Huynh-Feldt Lower-bound

Epsilona

Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables isproportional to an identity matrix.

May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed inthe Tests of Within-Subjects Effects table.

a.

Design: Intercept Within Subjects Design: rate

b.

Page 33: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

33

Repeated Measures Analysis of Variance (ANOVA)

For Mauchly's test if the p-value is significant (look under Sig.) then the assumption has been violated. This will determine which values you interpret on the ANOVA table (Tests of Within-Subject Effects).

Which is the case here, so use values associated with Huynh-Feldt test.

Page 34: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

34

Repeated Measures Analysis of Variance (ANOVA)

Examine the Tests of Within-Subject Effects table (ANOVA table) to determine the significance of your omnibus test.

Tests of Within-Subjects Effects

Measure: MEASURE_1

12452.967 3 4150.989 256.847 .000

12452.967 1.280 9727.653 256.847 .000

12452.967 1.313 9482.095 256.847 .000

12452.967 1.000 12452.967 256.847 .000

1406.033 87 16.161

1406.033 37.125 37.873

1406.033 38.086 36.917

1406.033 29.000 48.484

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sourcerate

Error(rate)

Type III Sumof Squares df Mean Square F Sig.

Page 35: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

35

Repeated Measures Analysis of Variance (ANOVA)

When the Sphericity assumption is not violated, you can interpret the top set of values (i.e., Sum of Squares, df, Mean Sum of Squares, F, and p (Sig.)).

Page 36: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

36

Repeated Measures Analysis of Variance (ANOVA)

When the Sphericity assumption is violated, you can interpret the values associated with Huynh-Feldt test.

In this case, there is a significant difference in smoking rates across the time periods, F(1.31,38.09) = 256.85, p < 0.001 (Huynh-Feldt). Since the results of the repeated measures ANOVA are significant, you will want to examine the post-hoc tests to determine between which time periods significant differences are occurring by using the Multiple Comparison table.

Page 37: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

37

Repeated Measures Analysis of Variance (ANOVA)

The Pairwise Comparisons table provides detailed information concerning the post-hoc results.

Pairwise Comparisons

Measure: MEASURE_1

25.100* 1.351 .000 22.336 27.864

22.767* 1.423 .000 19.857 25.677

22.333* 1.413 .000 19.444 25.223

-25.100* 1.351 .000 -27.864 -22.336

-2.333* .519 .000 -3.396 -1.271

-2.767* .548 .000 -3.888 -1.646

-22.767* 1.423 .000 -25.677 -19.857

2.333* .519 .000 1.271 3.396

-.433 .218 .056 -.879 .013

-22.333* 1.413 .000 -25.223 -19.444

2.767* .548 .000 1.646 3.888

.433 .218 .056 -.013 .879

(J) rate2

3

4

1

3

4

1

2

4

1

2

3

(I) rate1

2

3

4

MeanDifference

(I-J) Std. Error Sig.a

Lower Bound Upper Bound

95% Confidence Interval forDifference

a

Based on estimated marginal means

The mean difference is significant at the .05 level.*.

Adjustment for multiple comparisons: Least Significant Difference (equivalent to noadjustments).

a.

Page 38: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

38

Repeated Measures Analysis of Variance (ANOVA)

The table shows all possible comparisons between the four time periods.

In the first row, the pre-intervention smoking rate is compared to the post-intervention smoking rates. The mean difference for this comparison is 25.1000 (i.e., the average smoking rate for pre-intervention, 30.5333, is subtracted from the average post-intervention smoking rate, 5.4333). To determine whether this mean difference is statistically significant examine the "Sig. Column" which represents the p-value. The p-value is (p <) 0.001 suggesting that the groups are significantly different from one another.

Page 39: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

39

Repeated Measures Analysis of Variance (ANOVA)

This is also supported by the 95% confidence interval which indicates that zero is outside the bounds. Following this comparison, a comparison is made between the pre-intervention smoking rates and smoking rates at a six month follow-up which shows a significant difference between the two groups, p < 0.001. You will notice that SPSS places a star next to mean difference scores that differ significantly. The remaining rows provide the results for the other comparisons.

Page 40: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

40

Mixed Factorial ANOVA

Do the smoking rates differ across the three types of smoking cessation program over time? That is, does one program lead to greater reductions in smoking rates among smokers?

Page 41: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

41

Mixed Factorial ANOVA Analyze > General Linear Model > Repeated Measures

Page 42: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

42

Mixed Factorial ANOVA Note the reset button used to remove redundant factors from old analysis.

In the Within Subject Factor Name box designate a name for the repeated measure factor. In this case, let's call it time.

Page 43: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

43

Mixed Factorial ANOVA In the Number of Levels window type in the number of time periods measured. In this case it is 2.

Page 44: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

44

Mixed Factorial ANOVA Click on Add. Click on Define

Page 45: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

45

Mixed Factorial ANOVA Having clicked on Define.

Page 46: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

46

Mixed Factorial ANOVA Highlight your first time variable, pre, from the list of variables on the left, and click on the upper arrow button to move it into the Within Subject Variables window.

Page 47: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

47

Mixed Factorial ANOVA Add the remaining time variable, post, in the same fashion.

Page 48: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

48

Mixed Factorial ANOVA Highlight program and click on the arrow button in front of the Between-Subjects Factor(s) box.

Click on the Options button.

Page 49: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

49

Mixed Factorial ANOVA

Click on the square next to the word Descriptive.Highlight time in the Factor(s) and Factor Interaction box. Click on the arrow button and click on the square next to the word Compare main effects. Click on Continue to return.

Page 50: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

50

Mixed Factorial ANOVA Click on the Post Hoc... button

Page 51: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

51

Mixed Factorial ANOVA Highlight program in the Factor(s) box. Click on the arrow button to move program to the “Post Hoc tests for:” box. Select a Tukey test by clicking on the box.

Page 52: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

52

Mixed Factorial ANOVA Click on Continue to return to the Repeated Measures window.

Click on OK to run the analysis.

Syntax

GLM pre post BY program /WSFACTOR = time 2 Polynomial /METHOD = SSTYPE(3) /POSTHOC = program ( TUKEY ) /EMMEANS = TABLES(time) COMPARE ADJ(LSD) /PRINT = DESCRIPTIVE /CRITERIA = ALPHA(.05) /WSDESIGN = time /DESIGN = program.

Page 53: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

53

Mixed Factorial ANOVA You can identify n, Mean, and Std for each of the two time periods across the three interventions using the descriptive statistics output.

Descriptive Statistics

27.80 5.160 10

31.70 5.376 10

32.10 5.109 10

30.53 5.406 30

7.00 5.981 10

3.00 4.372 10

6.30 7.319 10

5.43 6.067 30

program1

2

3

Total

1

2

3

Total

pre

post

Mean Std. Deviation N

Page 54: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

54

Mixed Factorial ANOVA Examine the Tests of Within-Subject Effects table (ANOVA table) to determine the significance of your omnibus test.

Tests of Within-Subjects Effects

Measure: MEASURE_1

9450.150 1 9450.150 402.039 .000

9450.150 1.000 9450.150 402.039 .000

9450.150 1.000 9450.150 402.039 .000

9450.150 1.000 9450.150 402.039 .000

159.700 2 79.850 3.397 .048

159.700 2.000 79.850 3.397 .048

159.700 2.000 79.850 3.397 .048

159.700 2.000 79.850 3.397 .048

634.650 27 23.506

634.650 27.000 23.506

634.650 27.000 23.506

634.650 27.000 23.506

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sphericity Assumed

Greenhouse-Geisser

Huynh-Feldt

Lower-bound

Sourcetime

time * program

Error(time)

Type III Sumof Squares df Mean Square F Sig.

Page 55: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

55

Mixed Factorial ANOVA

The interaction effect should be examined first to determine if it is significant. In this case, the interaction effect is significant, F(1, 27) = 3.397, p = 0.05 (It should be noted that sphericity is always met for two levels of a repeated measure factor and it is, therefore, unnecessary to evaluate.) This suggests that there is a significant difference in the interventions programs across time.

Since the interaction is significant, the main effect for time should not be interpreted.

That completes the relevant ANOVA analysis.

Page 56: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

56

Mixed Factorial ANOVA It should be noted that sphericity is always met for two levels of a repeated measure factor and it is, therefore, unnecessary to evaluate.

Page 57: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

57

Caution“Like elaborately plumed birds…we preen and strut and display our t-values.” That was Edward Leamer’s uncharitable description of his profession in 1983. Mr. Leamer, an economist at the University of California in Los Angeles, was frustrated by empirical economists’ emphasis on measures of correlation over underlying questions of cause and effect, such as whether people who spend more years in school go on to earn more in later life.

Cause and defect The Economist, 13 August 2009, p. 68“Instrumental variables help to isolate causal relationships. But they can be taken too far.”

Page 58: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

58

CautionHardly anyone, he wrote gloomily, “takes anyone else’s data analyses seriously”.

To make his point, Mr. Leamer showed how different (but apparently reasonable) choices about which variables to include in an analysis of the effect of capital punishment on murder rates could lead to the conclusion that the death penalty led to more murders, fewer murders, or had no effect at all.

“Let’s take the con out of econometrics”, by Edward Leamer, American Economic Review 73(1), March 1983

Page 59: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

59

CautionConfidence intervals have frequently been proposed as a more useful alternative to null hypothesis significance testing, and their use is strongly encouraged in the APA Manual (American Psychological Association 2009 Publication Manual of the American Psychological Association (6th ed.). Washington, DC).

The misunderstandings surrounding p-values and confidence intervals are particularly unfortunate because they constitute the main tools by which psychologists draw conclusions from data.

Robust misinterpretation of confidence intervals

Page 60: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

60

CautionRobust misinterpretation of confidence intervalsHoekstra, R., Morey, R., Rouder, J., & Wagenmakers, E. (2014). Robust misinterpretation of confidence intervals Psychonomic Bulletin & Review, 21 (5), 1157-1164 DOI: 10.3758/s13423-013-0572-3

Reformers say psychologists should change how they report their results, but does anyone understand the alternative? BPS Research Digest

Page 61: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

61

Caution

Please mark each of the statement below “true” or “false”. False means that the statement does not follow logically from the above statement. Also note that all, several, or none of the statements may be correct:

Hoekstra, R., Morey, R., Rouder, J., & Wagenmakers, E. (2014). Robust misinterpretation of confidence intervals Psychonomic Bulletin & Review, 21(5), 1157-1164 DOI: 10.3758/s13423-013-0572-3

Page 62: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

62

CautionThe 95% confidence interval for the mean ranges from 0.1 to 0.4!

1. The probability that the true mean is greater than 0 is at least 95%. [] true/false []

2. The probability that the true mean equals 0 is smaller than 5%. [] true/false []

3. The “null hypothesis” that the true mean equals 0 is likely to be incorrect. [] true/false []

4. There is a 95% probability that the true mean lies between 0.1 and 0.4. [] true/false []

5. We can be 95% confident that the true mean lies between 0.1 and 0.4. [] true/false []

6. If we were to repeat the experiment over and over, then 95% of the time

the true mean falls between 0.1 and 0.4.

[] true/false []

Page 63: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

63

CautionThe 95% confidence interval for the mean ranges from 0.1 to 0.4!

1. The probability that the true mean is greater than 0 is at least 95%. [] true/false []

2. The probability that the true mean equals 0 is smaller than 5%. [] true/false []

3. The “null hypothesis” that the true mean equals 0 is likely to be incorrect. [] true/false []

4. There is a 95% probability that the true mean lies between 0.1 and 0.4. [] true/false []

5. We can be 95% confident that the true mean lies between 0.1 and 0.4. [] true/false []

6. If we were to repeat the experiment over and over, then 95% of the time

the true mean falls between 0.1 and 0.4.

[] true/false []

Assign probabilities to parameters or hypotheses, something that is not allowed within the frequentist framework.

Page 64: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

64

CautionThe 95% confidence interval for the mean ranges from 0.1 to 0.4!

1. The probability that the true mean is greater than 0 is at least 95%. [] true/false []

2. The probability that the true mean equals 0 is smaller than 5%. [] true/false []

3. The “null hypothesis” that the true mean equals 0 is likely to be incorrect. [] true/false []

4. There is a 95% probability that the true mean lies between 0.1 and 0.4. [] true/false []

5. We can be 95% confident that the true mean lies between 0.1 and 0.4. [] true/false []

6. If we were to repeat the experiment over and over, then 95% of the time

the true mean falls between 0.1 and 0.4.

[] true/false []

Assign probabilities to parameters or hypotheses, something that is not allowed within the frequentist framework.

Page 65: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

65

CautionThe 95% confidence interval for the mean ranges from 0.1 to 0.4!

1. The probability that the true mean is greater than 0 is at least 95%. [] true/false []

2. The probability that the true mean equals 0 is smaller than 5%. [] true/false []

3. The “null hypothesis” that the true mean equals 0 is likely to be incorrect. [] true/false []

4. There is a 95% probability that the true mean lies between 0.1 and 0.4. [] true/false []

5. We can be 95% confident that the true mean lies between 0.1 and 0.4. [] true/false []

6. If we were to repeat the experiment over and over, then 95% of the time

the true mean falls between 0.1 and 0.4.

[] true/false []

Assign probabilities to parameters or hypotheses, something that is not allowed within the frequentist framework.

Page 66: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

66

CautionThe 95% confidence interval for the mean ranges from 0.1 to 0.4!

1. The probability that the true mean is greater than 0 is at least 95%. [] true/false []

2. The probability that the true mean equals 0 is smaller than 5%. [] true/false []

3. The “null hypothesis” that the true mean equals 0 is likely to be incorrect. [] true/false []

4. There is a 95% probability that the true mean lies between 0.1 and 0.4. [] true/false []

5. We can be 95% confident that the true mean lies between 0.1 and 0.4. [] true/false []

6. If we were to repeat the experiment over and over, then 95% of the time

the true mean falls between 0.1 and 0.4.

[] true/false []

Assign probabilities to parameters or hypotheses, something that is not allowed within the frequentist framework.

Page 67: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

67

CautionThe 95% confidence interval for the mean ranges from 0.1 to 0.4!

1. The probability that the true mean is greater than 0 is at least 95%. [] true/false []

2. The probability that the true mean equals 0 is smaller than 5%. [] true/false []

3. The “null hypothesis” that the true mean equals 0 is likely to be incorrect. [] true/false []

4. There is a 95% probability that the true mean lies between 0.1 and 0.4. [] true/false []

5. We can be 95% confident that the true mean lies between 0.1 and 0.4. [] true/false []

6. If we were to repeat the experiment over and over, then 95% of the time

the true mean falls between 0.1 and 0.4.

[] true/false []

Mentions the boundaries of the confidence interval (i.e., 0.1 and 0.4), whereas a confidence interval can be used to evaluate only the procedure and not a specific interval.

Page 68: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

68

CautionThe 95% confidence interval for the mean ranges from 0.1 to 0.4!

1. The probability that the true mean is greater than 0 is at least 95%. [] true/false []

2. The probability that the true mean equals 0 is smaller than 5%. [] true/false []

3. The “null hypothesis” that the true mean equals 0 is likely to be incorrect. [] true/false []

4. There is a 95% probability that the true mean lies between 0.1 and 0.4. [] true/false []

5. We can be 95% confident that the true mean lies between 0.1 and 0.4. [] true/false []

6. If we were to repeat the experiment over and over, then 95% of the time

the true mean falls between 0.1 and 0.4.

[] true/false []

Mentions the boundaries of the confidence interval (i.e., 0.1 and 0.4), whereas a confidence interval can be used to evaluate only the procedure and not a specific interval.

Page 69: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

69

CautionThe 95% confidence interval for the mean ranges from 0.1 to 0.4!

1. The probability that the true mean is greater than 0 is at least 95%. [] true/false []

2. The probability that the true mean equals 0 is smaller than 5%. [] true/false []

3. The “null hypothesis” that the true mean equals 0 is likely to be incorrect. [] true/false []

4. There is a 95% probability that the true mean lies between 0.1 and 0.4. [] true/false []

5. We can be 95% confident that the true mean lies between 0.1 and 0.4. [] true/false []

6. If we were to repeat the experiment over and over, then 95% of the time

the true mean falls between 0.1 and 0.4.

[] true/false []

To sum up, all six statements are incorrect. Note that all six err in the same direction of wishful thinking.

Page 70: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

70

CautionThe 95% confidence interval for the mean ranges from 0.1 to 0.4!

7. If we were to repeat the experiment over and over, then 95 % of the time

the confidence intervals contain the true mean.

[] true/false []

The correct statement, which was absent from the list, is the following:

Page 71: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

71

CautionSuppose you have a treatment that you suspect may alter performance on a certain task.

You compare the means of your control and experimental groups (say 20 subjects in each sample).

Further, suppose you use a simple independent means t-test and your result is significant (t = 2.7, d.f. = 18, p = 0.01).

Please mark each of the statements below as “true” or “false.” “False” means that the statement does not follow logically from the above premises. Also note that several or none of the statements may be correct (Gigerenzer 2004).

Gigerenzer, G. (2004). Mindless statistics. The Journal of Socio-Economics, 33, 587–606.

Page 72: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

72

Caution

1. You have absolutely disproved the null hypothesis (that is, there is no

difference between the population means).

[] true/false []

2. You have found the probability of the null hypothesis being true. [] true/false []

3. You have absolutely proved your experimental hypothesis (that there

is a difference between the population means).

[] true/false []

4. You can deduce the probability of the experimental hypothesis being

true.

[] true/false []

5. You know, if you decide to reject the null hypothesis, the probability

that you are making the wrong decision.

[] true/false []

6. You have a reliable experimental finding in the sense that if,

hypothetically, the experiment were repeated a great number of times,

you would obtain a significant result on 99% of occasions.

[] true/false []

Suppose you have a treatment that you suspect may alter performance on a certain task. You compare the means of your control and experimental groups (say 20 subjects in each sample). Further, suppose you use a simple independent means t-test and your result is significant (t = 2.7, d.f. = 18, p = 0.01).

Page 73: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

73

Caution

1. You have absolutely disproved the null hypothesis (that is, there is no

difference between the population means).

[] true/false []

2. You have found the probability of the null hypothesis being true. [] true/false []

3. You have absolutely proved your experimental hypothesis (that there

is a difference between the population means).

[] true/false []

4. You can deduce the probability of the experimental hypothesis being

true.

[] true/false []

5. You know, if you decide to reject the null hypothesis, the probability

that you are making the wrong decision.

[] true/false []

6. You have a reliable experimental finding in the sense that if,

hypothetically, the experiment were repeated a great number of times,

you would obtain a significant result on 99% of occasions.

[] true/false []

Which statements are in fact true? Recall that a p-value is the probability of the observed data (or of more extreme data points), given that the null hypothesis H0 is true, defined in symbols as p(D|H0).This definition can be rephrased in a more technical form by introducing the statistical model underlying the analysis (Gigerenzer et al., 1989, chapter 3).

Gigerenzer, G., Swijtink, Z., Porter, T., Daston, L., Beatty, J., Krüger, L., 1989. The Empire of Chance. How Probability Changed Science and Every Day Life. Cambridge University Press, Cambridge, UK.

Page 74: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

74

Caution

1. You have absolutely disproved the null hypothesis (that is, there is no

difference between the population means).

[] true/false []

2. You have found the probability of the null hypothesis being true. [] true/false []

3. You have absolutely proved your experimental hypothesis (that there

is a difference between the population means).

[] true/false []

4. You can deduce the probability of the experimental hypothesis being

true.

[] true/false []

5. You know, if you decide to reject the null hypothesis, the probability

that you are making the wrong decision.

[] true/false []

6. You have a reliable experimental finding in the sense that if,

hypothetically, the experiment were repeated a great number of times,

you would obtain a significant result on 99% of occasions.

[] true/false []

Suppose you have a treatment that you suspect may alter performance on a certain task. You compare the means of your control and experimental groups (say 20 subjects in each sample). Further, suppose you use a simple independent means t-test and your result is significant (t = 2.7, d.f. = 18, p = 0.01).

Is easily detected as being false, because a significance test can never disprove the null hypothesis or the (undefined) experimental hypothesis. They are instances of the illusion of certainty (Gigerenzer, 2002).Gigerenzer, G., 2002. Calculated Risks: How to Know When Numbers Deceive You. Simon & Schuster, New York (UK edition: Reckoning with Risk: Learning to Live with Uncertainty. Penguin, London).

Page 75: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

75

Caution

1. You have absolutely disproved the null hypothesis (that is, there is no

difference between the population means).

[] true/false []

2. You have found the probability of the null hypothesis being true. [] true/false []

3. You have absolutely proved your experimental hypothesis (that there

is a difference between the population means).

[] true/false []

4. You can deduce the probability of the experimental hypothesis being

true.

[] true/false []

5. You know, if you decide to reject the null hypothesis, the probability

that you are making the wrong decision.

[] true/false []

6. You have a reliable experimental finding in the sense that if,

hypothetically, the experiment were repeated a great number of times,

you would obtain a significant result on 99% of occasions.

[] true/false []

Suppose you have a treatment that you suspect may alter performance on a certain task. You compare the means of your control and experimental groups (say 20 subjects in each sample). Further, suppose you use a simple independent means t-test and your result is significant (t = 2.7, d.f. = 18, p = 0.01).

Is easily detected as being false, because a significance test can never disprove the null hypothesis or the (undefined) experimental hypothesis. They are instances of the illusion of certainty (Gigerenzer, 2002).Gigerenzer, G., 2002. Calculated Risks: How to Know When Numbers Deceive You. Simon & Schuster, New York (UK edition: Reckoning with Risk: Learning to Live with Uncertainty. Penguin, London).

Page 76: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

76

Caution

1. You have absolutely disproved the null hypothesis (that is, there is no

difference between the population means).

[] true/false []

2. You have found the probability of the null hypothesis being true. [] true/false []

3. You have absolutely proved your experimental hypothesis (that there

is a difference between the population means).

[] true/false []

4. You can deduce the probability of the experimental hypothesis being

true.

[] true/false []

5. You know, if you decide to reject the null hypothesis, the probability

that you are making the wrong decision.

[] true/false []

6. You have a reliable experimental finding in the sense that if,

hypothetically, the experiment were repeated a great number of times,

you would obtain a significant result on 99% of occasions.

[] true/false []

Suppose you have a treatment that you suspect may alter performance on a certain task. You compare the means of your control and experimental groups (say 20 subjects in each sample). Further, suppose you use a simple independent means t-test and your result is significant (t = 2.7, d.f. = 18, p = 0.01).

Is also false. The probability p(D|H0) is not the same as p(H0|D), and more generally, a significance test does not provide a probability for a hypothesis. The statistical toolbox, of course, contains tools that would allow estimating probabilities of hypotheses, such as Bayesian statistics.

Page 77: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

77

Caution

1. You have absolutely disproved the null hypothesis (that is, there is no

difference between the population means).

[] true/false []

2. You have found the probability of the null hypothesis being true. [] true/false []

3. You have absolutely proved your experimental hypothesis (that there

is a difference between the population means).

[] true/false []

4. You can deduce the probability of the experimental hypothesis being

true.

[] true/false []

5. You know, if you decide to reject the null hypothesis, the probability

that you are making the wrong decision.

[] true/false []

6. You have a reliable experimental finding in the sense that if,

hypothetically, the experiment were repeated a great number of times,

you would obtain a significant result on 99% of occasions.

[] true/false []

Suppose you have a treatment that you suspect may alter performance on a certain task. You compare the means of your control and experimental groups (say 20 subjects in each sample). Further, suppose you use a simple independent means t-test and your result is significant (t = 2.7, d.f. = 18, p = 0.01).

Is also false. The probability p(D|H0) is not the same as p(H0|D), and more generally, a significance test does not provide a probability for a hypothesis. The statistical toolbox, of course, contains tools that would allow estimating probabilities of hypotheses, such as Bayesian statistics.

Page 78: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

78

Caution

1. You have absolutely disproved the null hypothesis (that is, there is no

difference between the population means).

[] true/false []

2. You have found the probability of the null hypothesis being true. [] true/false []

3. You have absolutely proved your experimental hypothesis (that there

is a difference between the population means).

[] true/false []

4. You can deduce the probability of the experimental hypothesis being

true.

[] true/false []

5. You know, if you decide to reject the null hypothesis, the probability

that you are making the wrong decision.

[] true/false []

6. You have a reliable experimental finding in the sense that if,

hypothetically, the experiment were repeated a great number of times,

you would obtain a significant result on 99% of occasions.

[] true/false []

Suppose you have a treatment that you suspect may alter performance on a certain task. You compare the means of your control and experimental groups (say 20 subjects in each sample). Further, suppose you use a simple independent means t-test and your result is significant (t = 2.7, d.f. = 18, p = 0.01).

Also refers to a probability of a hypothesis. This is because if one rejects the null hypothesis, the only possibility of making a wrong decision is if the null hypothesis is true. Thus, it makes essentially the same claim as Statement 2 does, and both are incorrect.

Page 79: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

79

Caution

1. You have absolutely disproved the null hypothesis (that is, there is no

difference between the population means).

[] true/false []

2. You have found the probability of the null hypothesis being true. [] true/false []

3. You have absolutely proved your experimental hypothesis (that there

is a difference between the population means).

[] true/false []

4. You can deduce the probability of the experimental hypothesis being

true.

[] true/false []

5. You know, if you decide to reject the null hypothesis, the probability

that you are making the wrong decision.

[] true/false []

6. You have a reliable experimental finding in the sense that if,

hypothetically, the experiment were repeated a great number of times,

you would obtain a significant result on 99% of occasions.

[] true/false []

Suppose you have a treatment that you suspect may alter performance on a certain task. You compare the means of your control and experimental groups (say 20 subjects in each sample). Further, suppose you use a simple independent means t-test and your result is significant (t = 2.7, d.f. = 18, p = 0.01).

Amounts to the replication fallacy (Gigerenzer, 1993, 2000). Here, p=1% is taken to imply that such significant data would reappear in 99% of the repetitions. Statement 6 could be made only if one knew that the null hypothesis was true. In formal terms, p(D|H0) is confused with 1−p(D).Gigerenzer, G., 1993. The superego, the ego, and the id in statistical reasoning. In: Keren, G., Lewis, C. (Eds.), A Handbook for Data Analysis in the Behavioral Sciences: Methodological Issues. Erlbaum, Hillsdale, NJ, pp. 311–339.Gigerenzer, G., 2000. Adaptive Thinking: Rationality in the Real World. Oxford University Press, New York.

Page 80: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

80

Caution

1. You have absolutely disproved the null hypothesis (that is, there is no

difference between the population means).

[] true/false []

2. You have found the probability of the null hypothesis being true. [] true/false []

3. You have absolutely proved your experimental hypothesis (that there

is a difference between the population means).

[] true/false []

4. You can deduce the probability of the experimental hypothesis being

true.

[] true/false []

5. You know, if you decide to reject the null hypothesis, the probability

that you are making the wrong decision.

[] true/false []

6. You have a reliable experimental finding in the sense that if,

hypothetically, the experiment were repeated a great number of times,

you would obtain a significant result on 99% of occasions.

[] true/false []

Suppose you have a treatment that you suspect may alter performance on a certain task. You compare the means of your control and experimental groups (say 20 subjects in each sample). Further, suppose you use a simple independent means t-test and your result is significant (t = 2.7, d.f. = 18, p = 0.01).

Gigerenzer, G., 1993. The superego, the ego, and the id in statistical reasoning. In: Keren, G., Lewis, C. (Eds.), A Handbook for Data Analysis in the Behavioral Sciences: Methodological Issues. Erlbaum, Hillsdale, NJ, pp. 311–339.Gigerenzer, G., 2000. Adaptive Thinking: Rationality in the Real World. Oxford University Press, New York.

Page 81: 1 Smoking Data The investigation was based on examining the effectiveness of smoking cessation programs among heavy smokers who are also recovering alcoholics.

81

SPSS TipsNow you should go and try for yourself.

Each week our cluster (5.05) is booked for 2 hours after this session. This will enable you to come and go as you please.

Obviously other timetabled sessions for this module take precedence.