Class 13 and 14 Jacobson et al (1996) APA (2006)Evidence Based Practice in Psychology 1.

Class 13 and 14

Jacobson et al (1996)

APA (2006)Evidence Based Practice in Psychology

1

2

Experimental Between Group Designs

1. Post-Test Only Control

2. Pre-Test -- Post-Test Control3. Solomon Four Group (combination of 1 and 2 above)

Factorial Design more than one independent variable; interactions gender x treatment

Dependent Sample Design (Matching)

2

3

Experimental Between Group Designs

Post-Test Only Control Pre-Test -- Post-Test Control Solomon Four Group

Factorial Design (Treatment X Therapist) Dependent Sample Design (Matching)

# of previous episodes and severity depression, presence of dysthimia, gender, marital stauts

3

Types Outcome Studies Kazdin (chap 18)

1. Treatment Package Strategy

2. Dismantling Strategy

3. Constructive Strategy

4. Parametric Strategy (structural components)

5. Common factors Control Group

6. Comparative Outcome Strategy

7. Client and Therapist Variation Strategy Moderation Designs

Types Outcome Studies Kazdin (chap 18)

1. Treatment Package Strategy2. Dismantling Strategy

What are the active ingredients ?1. Constructive Strategy 2. Parametric Strategy (structural components)3. Common factors Control Group4. Comparative Outcome Strategy5. Client and Therapist Variation Strategy

Moderation Designs

Research FocusHow treatment effects change:

identify change mechanisms

VS.

How well treatment works

6

Change Mechanisms Activation Hypotheses ( BA) (activation)

Change behaviors- become active and access sources of reinforcement- occurs early in therapy

Coping Skills Hypotheses (AT) Learn effective coping strategies

(coping + activation)

Beck Hypotheses (CT) Change cognitive structures or core schemas

(cognitive schemas + activation + coping) 7

Chapter 7 8

Treatment Groups BA Condition: Behavioral Activation Hypotheses:

Behavioral Activation

AT Condition: Coping Skills Hypotheses Behavioral Activation Coping Skills - Automatic Thoughts

CT Condition: Cognitive Therapy Hypotheses Behavioral Activation Automatic Thoughts Core Schemas

8

Chapter 7 9

Mechanisms of Change Construct Measure


Dysfunctional Thinking

Cognitive Schemas

Pleasant Events Schedule

Automatic Thoughts Q.

Expanded Attributional Styles

How did the authors examine if the mechanisms of change worked in therapy as predicted by theory?

9

How Treatment Effects Change

To what extent treatment groups differed in post-test measures of each change mechanism

Measure Ttreat. G Behavioral activation – Pleasant Event SCd BA Coping skills- - AT (Dist Thinking) AT Core schemas - Exp Attrb Style CT

To what extent was there change from pre-to post- treatment in each change mechanism

10

How Well Treatments Work Which of the three treatment conditions

yielded better outcomes regarding Depression at termination and at 6 month follow up? HRSD BDI Recovered Improved Rates

11

Research Questions Analyses

Treatment Outcomegroup differences Depression

Type of Analyses

CVsIVSDVs

Mechanism of Change:Treatment groups differed in post-test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas)

Type of Analyses

CVsIVSDVs

Mechanism of Change:Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS)

Type and # of Analyses

12


Treatment Outcomegroup differences in Depression

3 x 4 MANCOVACVs Pre-test BDI HRSDIVs Treatments: CT, AT, BA Therapists (n=4) Treatment X TherapistDVs Post-Test BDI HRSD



13

How Well Treatments Work Which of the three treatment conditions

yielded better Depression outcomes at termination and at 6 month follow up?3 X 4 MANCOVA

IVS

3 Treatments (CT, AT, BA)

4 Therapists

Treatment X Therapist (Interaction)

DVs - Depression

HRSD – Clinician

BDI - Self- Report

14

Which group to use in outcome analyses

Group SessionsIntent-to treat Total sample

n = 149

Dropouts (not included in MANCOVA analyses; p. 299)

1<Sessions <12 n = 12

Completers At least 12/20 sessions n = 137

Maximum Completers 20 sessions n = 129

15

Results MANCOVAsTreatment Outcome: Post- Test Depression

Main Effect Treatment ?

Main Effect Therapist ?

Treatment X Therapist Interaction ?

Results p. 299 -- ANOVAs Table 3

16

Results MANCOVAs Treatment Outcome: Post- Test BID & HRSD Treatment Group N= 149 137 129

Main Effect Treatment NS NS NS

Main Effect Therapist NS NS NS

Treatment X Therapist NS NS NS

Results p. 299 -- ANOVAs Table 3

17

Table 3 -- ANCOVAS ??

18

19

ANCOVA Post-TestMain Effects for Therapists

Post-Test Measures

Therapists BDI HRSD

TH-1 BDIT1 HRSDT1

TH-2 BDIT2 HRSDT2

TH-3 BDIT3 HRSDT3

TH-4 BDIT4 HRSDT4

19

20

ANCOVA - Post-Test

Effects Therapist X Treatment Interaction

TherapistBDI

Post-test Scores by Treatment

HRSDPost-test Scores by

Treatment

BA AT CT BA AT CT

TH-1 BDIT1 BDIT1 BDIT1 HRSDT1 HRSDT1 HRSDT1

TH-2 BDIT2 BDIT2 BDIT2 HRSDT2 HRSDT2 HRSD T2

TH-3 BDIT3 BDIT3 BDIT3 HRSDT3 HRSDT3 HRSDT3

TH-4 BDIT4 BDIT4 BDIT4 HRSDT4 HRSDT4 HRSD T4

20

Outcome: Follow-up 6 months Overall Impact of Therapy

ANCOVAS IV:Treatments DVs: Follow-Up HRSD BDI Covariate: Pretest Score

Changes in Follow-up Time ANCOVAS IV: Treatments DVs: Follow-Up HRSD BDI Covariate: Post-test Score

21

Clinical Significance

ImprovedNo major depression at post-test

RecoveredNo major depression and BDI<8

22

23

Clinical Significance: Mean Improvement/Recovered Rates across Treatments – Post -Test (p.299)

Group Improved Recovered

Intent-to treat Total sample (149)

62.3% 51.5%

Completers 12/20 N=37

58.3% 58.3%

Maximum CompletersN = 129

66.0% 54.5%

DropoutsN=12

16.7% 5.6%

24

25

Chapter 7 26

Mechanisms of Change Construct Measure


Dysfunctional Thinking

Cognitive Schemas


Automatic Thoughts Q.


How did the authors examine to what extent the mechanisms of change worked in therapy as predicted by theory?

26

How Treatment Effects Change

To what extent treatment groups differed in post-test measures of each change mechanism

Measure Ttreat. Grps Behavioral activation – Pleasant Event SCd BA Coping skills- - AT (Dist Thinking) AT Core schemas - Exp Attrb Style CT

To what extent was there change from pre-to post- treatment in each change mechanism

27



3 x 4 MANCOVACVs Pre-test BDI HRSDIVs Treatments: CT, AT, BA Therapists (n=4) Treatment X TherapistDVs Post-Test BDI HRSD



28



3 x 4 MANCOVACVs Pre-test BDI HRSDIVs Treatments: CT, AT, BA Therapists Treatment X TherapistDVs Post-Test BDI HRSD


3 ANCOVAS (one CV & DV per analysis)

CV Pre-test: E Attrb Q, ATQ, PES IV Treatments: (CT, AT, BA)DV Post-test: E Attrb Q, ATQ, PES


29


Treatment Outcomegroup differences in Depression

3 x 4 MANCOVACVs Pre-test BDI HRSDIVs Treatments: CT, AT, BA Therapists Treatment X TherapistDVs Post-Test BDI HRSD


3 ANCOVAS (one CV & DV per analysis)

CV Pre-test: E Attrb Q, ATQ, PES IV Treatments: (CT, AT, BA)DV Post-test: E Attrb Q, ATQ, PES


3 Paired T Tests per Treatment Pre-post differences in each mechanism of change for each treatment : E Attrb Q, ATQ, PES

30

Paired T Tests: Pre-test/Post-test Change

Treatment


Automatic Thoughts Questionnaires


Behavioral Activation PESch** ATQ EAS

Automatic Thoughts PESch** ATQ** EAS

Cognitive Therapy PESch** ATQ** EAS**

Clients in all conditions significantly improved on the three measures (p. 301)

31

Residual change score in Mechanism of Change from Pre-to Mid-T and RCS in Depression from Mid-T to Post-T (p.301)

32

Threats to Statistical Conclusion Validity Jacobson et al. (1996)

Are the observed relations among variables accurate?

1. Power

3. Unreliability of Treatment Implementation

33

Threats to Statistical Conclusion Validity


1. Power

•N=149 and 3 groups/about 50 participants per group. Large N for an Exp study (+)•Outcome measures are well-known – high internal reliability (+) •No information is given about alphas with study sample Power analyses not reported

3. Unreliability of Treatment Implementation

•Therapists were experienced in CT and trained for study (+)•Training followed a manual prepared for each treatment group (+)•Therapy tapes were listened on an on-going basis and therapists flagged if they deviated (+) •Analyses of adherence based on 27 taped sessions showed that treatments were distinct and consistent with their respective protocols (+)

34



4. Extraneous Variance in the Experimental Setting

5. Heterogeneity of Participants

35



4. Extraneous Variance in the Experimental Setting

•Do not appear to be any – The same four therapists administered all treatments•However procedures and settings are not described in detail

5. Heterogeneity of Participants

•Study had many exclusion criteria including co-morbidity, taking psychotropic medication, suicidal… p. 296. (+)•However, don’t know #s in pool of volunteers from which the 152 accepted patients were taken from•72% were women (+)•No info regarding race/ethnicity, SES (-)

36

Threats to Internal Validity

Can we conclude that there is a causal relation between the IV and the DV?

1. Selection to Treat. Groups

2. History

3. Attrition

4. Repeated Testing Effects

5. Reaction to Control Group Assignment

37

Threats to Internal Validity

Can we conclude that there is a causal relation between the IV and the DV?

1. Selection to Treat. Groups

•Used Randomization after matching for episodes dep., dysthimia, severity of depression, gender , marital status (+)

2. History •Therapy appeared to occur for everyone at once – but this is not addressed

3. Attrition •Small -only 15 out of 152 - 8% -Attrition during acute treatment was comparable across treatment conditions – p.296 right (+)

4. Repeated Testing Effects

•Five administration of measures – at pre-test; post –test and 6, 12, 18 month follow-up (-)

5. Reaction to Control Group Assignment

•No placebo or no-treatment control group (+)•Every one received treatment (+). •Maybe some realized not receiving whole treatment (?) 38

Threats to Construct Validity

To what extent variables capture desired constructs

1.Mono-Operation Bias

2. Mono-Method Bias

3. Experimenter Expectancies

39

Threats to Construct Validity

To what extent variables capture desired constructs

1.Mono-Operation Bias

•Used two measures to assess outcome: Depression (+)

2. Mono-Method Bias

•HRSD is interview based and BDI is self-report (+)

3. Experimenter Expectancies

•Potential risk (-)•Should have worked in favor of the CT condition that included all aspects of CBT-- therapists and researchers were aligned with CBT

40

Threats to External Validity

Can we generalize observed relations across persons, settings and times

1.Person-Units

2. Outcome Measures

4. Settings

41

Threats to External Validity

Can we generalize observed relations across persons, settings and times

1.Person-Units

•Highly selected sample – p. 296 (-)•Exclusionary criteria (-)•Primarily female = 73% (-)•No info regarding race/ethnicity, SES (-)

2. Outcome Measures

•They used interview based and self report measures of Depression +•Examined outcome using clinical significance index +

4. Settings

•Empirical Question…..

42

Empirically Supported Therapies - EST

1993 APA Division 12Identify Efficacious treatments for

specific disorders FDA Criteria: Specificity

Specific ingredients vs. placebo effects

Randomized double-blind, placebo control design

43

44

What components of treatment are responsible for therapy effects?

Specific Ingredients unique to each therapy approach

Common Factors --- Placebo underlie most approaches

Scientific advances that lead to theRandomized Placebo Control Group Design

Germany: 1850 Wundt’s experimental method in psychology – 1st psychology lab

Britain:1800 Galton & Pearson – use of normal

curve in assessment of distribution mental abilities

France: Physicians start comparing treatments across groups of patients: treatment groups

45

Scientific advances that lead to theRandomized Placebo Control Group Design

1930s Fisher- ANOVA statistics - randomization

1930- 1950 Placebo treatment - physiochemical ingredients vs. patients’ expectations, hopes, beliefs

Mesmerized…. Placebo

46

Well-established treatments Division 12 Task Force Criteria

I. At least two good between-group design experiments must demonstrate efficacy in one or more of the following ways:

A. Superiority to pill or psychotherapy placebo, or to

other treatment (wait list control is not enough)

B. Equivalence to already established treatment with

adequate sample sizes

48

ORWell-established treatments (2)

II. A large series of single-case design experiments must demonstrate efficacy with

A. Use of good experimental design and

B. Comparison of intervention to another treatment

49

And,Well-established treatments (3)

III. Experiments must be conducted with treatment manuals or equivalent clear description of treatment

IV. Characteristics of samples must be specified

V. Effects must be demonstrated by at least two different investigators or teams

50

Probably efficacious treatments

I. Two experiments must show that the treatment is superior to waiting-list control group

ORII. One or more experiments must meet well-established criteria IA or IB, III, and IV

above, but V is not met (2 investigator teams)OR A small series of single-case design experiments must meet well-established-

treatment criteria

51

Treatment: Efficacious or Probably Efficacious ?

Family Intervention (Smith et al.) Anorexia in adolescent and young adult

women Superior cognitive behavioral intervention

(well established treatment) in 2 randomized clinical studies

Both studies conducted by Smith et al.

52

53

Evidence Based Psych. Practice

Best available research Results from randomized Controlled Clinical Trials

and other types of empirical studies

Clinical Expertise

Client Characteristics

Concerns EVT movement:

Brief, straightforward manualized treatments: Cognitive Behavioral

Emphasis on specific effects/ignore common factors

Lack of applicability to wide range of clients; comorbidity, males, race, ethnicity, social class

Mandates to use EVT- restriction to choice of treatment

54

APA Task Force Acceptable Research Designs Clinical observation Qualitative studies Systematic case studies Single-case experimental designs Public health and ethnographic research Process–outcome studies Studies in naturalistic settings Randomized Controlled Clinical Trials Meta-analysis

55

APA Task Force Research Designs

Clinical observation Qualitative studies Systematic case studies Single-case experimental designs Public health and ethnographic research Process–outcome studies Studies in naturalistic settings Randomized Controlled Clinical Trials Meta-analysis

56

Evidenced Based Practice Empirically Supported Tmts.

Effectiveness/Clinical utility: Efficacy;

Main Focus:

Main Focus:

Knowledge from clinical expertise

Knowledge from clinical expertise

Clients’ individual differences

Clients’ individual differences:

Therapists’ individual differences:

Therapists’ individual differences; 57

Evidenced Based Practice Empirically Supported Tmts.

Effectiveness: does it work; generalizability; feasibility

Efficacy; does treatment cause outcome

Main Focus: Client

Main Focus: Intervention

Knowledge from clinical expertise is directly applied

Knowledge fromclinical expertiseIndirect- to generate hypotheses

Clients’ individual differencesCentral

Clients’ individual differences: Nuisance

Therapists’ individual differences: Central

Therapists’ individual differences; Nuisance

58

EBP or EST??

Know the person who has the disorder

Know the disorder the person has

59

EBP or EST??

Know the person who has the disorder

Know the disorder the person has

60

61

Components Shared by Approaches to Psychotherapy (Wampold, 2000)

1. Emotionally charged, confiding relationship

2. In a healing setting

3. Rationale, conceptual scheme, or myth to explains patient’s symptoms

4. There is a ritual or procedure that engages client and therapist based on the rationale (Frank

& Frank, 1991).

62

Common Elements in Therapeutic Rituals and Procedures (1/2)

5. The relationship helps combat clients’ feelings of alienation

6. The process of therapy provides hope for improvement

7. Therapist provides new learning experiences

63

Common Elements in Therapeutic Rituals and Procedures (2/2)

8. Client’s emotions are aroused as a result of therapy and the clients expects to improve

9. Therapist enhances client’s sense of mastery and self-efficacy

10. Therapist provides opportunities for practice

64

Wampold concluded that: Specific theoretical ingredients are not

differentially related to outcome

Specific ingredients are necessary to construct a coherent treatment In which therapists have faith, and that provides a convincing rational for clients

Therefore, knowledge of theoretical approaches is necessary to build coherent interventions and treatments

Class 13 and 14 Jacobson et al (1996) APA (2006)Evidence Based Practice in Psychology 1.

Documents

treatment effects change

treatment package strategy

treatment conditions

post treatment

dismantling strategy

constructive strategy

extent treatment groups

treatment groups ch