Experiments and Dynamic Treatment Regimes S.A. Murphy Univ. of Michigan January, 2006.

Experiments and Dynamic Treatment Regimes

S.A. MurphyUniv. of Michigan

January, 2006

2

• Joint work with– Derek Bingham (Simon Fraser)– Linda Collins (PennState)

• And informed by discussions with– Vijay Nair (U. Michigan)– Bibhas Chakraborty (U. Michigan)– Vic Strecher (U. Michigan)

3

Outline

• Dynamic Treatment Regimes

• Challenges in Experimentation

• Defining Effects and Aliasing

• Examples

4

Dynamic treatment regimes are individually tailored treatments, with treatment type and dosage changing with ongoing subject need. Mimic Clinical Practice.

•High variability across patients in response to any one treatment

•Relapse is likely without either continuous or intermittent treatment for a large proportion of people.

•What works now may not work later

•Exacerbations in disorder may occur if there are no alterations in treatment

5

The Big Questions

•What is the best sequencing of treatments?

•What is the best timings of alterations in treatments?

•What information do we use to make these decisions?

6

Two stages of treatment for each individual

Observation available at jth stage

Treatment (vector) at jth stage

Primary outcome Y is a specified summary of decisions and observations

7

A dynamic treatment regime is a vector of decision rules, one per decision

where each decision rule

inputs the available information

and outputs a recommended treatment decision.

8

Long Term Goal: Construct decision rules that lead to a maximal mean Y.

An example of a decision rule is:

stop treatment if

otherwise maintain on current treatment.

Analysis methods for observational data dominate statistical literature (Murphy, Robins, Moodie & Richardson, Tsiatis)

9

Challenges in Experimentation

10

Dynamic Treatment Regimes (review)

Constructing decision rules is a multi-stage decision problem in which the system dynamics are unknown.

Better data provided by sequential multiple assignment randomized trials: randomize at each decision point— à la full factorial.

But often there are many potential components……

11

Reality

Unknown UnknownCauses Causes

X1 T1 X2 T2 Y

12

Challenges in ExperimentationDynamic Treatment Regimes are multi-component treatments:

many possible components

• decision options for improving patients are often different from decision options for non-improving patients (T2 differs by outcomes observed during initial treatment)

• multiple components employed simultaneously

• medications, adjunctive treatments, delivery mechanisms, behavioral contingencies, staff training, monitoring schedule…….

• Future: series of screening/refining, randomized trials prior to confirmatory trial --- à la Fisher/Box

13

Screening experiments (review)1) Goal is to eliminate inactive factors (e.g. components) and

inactive effects.

2) Each factor at 2 levels

3) Screen marginal causal effects

4) Design experiment using working assumptions concerning the negligibility of certain effects. (Think ANOVA)

5) Designs and analyses permit one to determine aliasing (caused by false working assumptions)

6) Minimize formal assumptions

14

Six Factors:

M1, E, C, T, A2(only for R=1), M2(only for R=0), each with 2 levels

(26= 64 simple dynamic treatment regimes)

The budget permits 16 cells --16 simple dynamic treatment regimes.

Simple Example for Two Stages

15

Two Stage Design: I=M2M1ECT=A2M1ECT

M1 E C T A2=M2

- - - - +

- - - + -

- - + - -

- - + + +

- + - - -

- + - + +

- + + - +

- + + + -

+ - - - -

+ - - + +

+ - + - +

+ - + + -

+ + - - +

+ + - + -

+ + + - -

+ + + + +

16

Screening experiments

Can we:

design screening experiments using working assumptions concerning the marginal causal effects

&

provide an analysis method that permits the determination of the aliasing??

17

Defining the Effects

18

Defining the stage 2 effects

Two decisions (two stages): (R=1 if quick response to T1)

Define effects involving T2 in an ANOVA decomposition of

19

Defining the stage 2 effects

Define effects involving T2 in an ANOVA decomposition:

20

Defining the stage 1 effects (T1)

Unknown UnknownCauses Causes

X1 T1 R T2 Y

21

Defining the stage 1 effects

Unknown UnknownCauses Causes

X1 T1 R T2 Y

22

Defining the stage 1 effects

Define

Define effects involving only T1 in an ANOVA decomposition of

23

Defining the stage 1 effects

Intuition: If T2 were randomized with probability ½ among responders (R=1) and T2 were randomized with probability ½ among nonresponders (R=0) then

(“ignore” R and future treatment).

24

Why marginal, why uniform?Define effects involving only T1 in an ANOVA

decomposition of

1) The defined effects are causal.

2) The defined effects are consistent with tradition in experimental design for screening.

– The main effect for one treatment factor is defined by marginalizing over the remaining treatment factors using a discrete uniform distribution.

25

Why marginal, why uniform?

When no R the main effect for treatment T1 in classical experimental design is

If there is no effect of T1 on R, then our definition of the main effect for treatment T1 reduces to the above.

26

An Aside: Ideally you’d like to replace

by

(X2 is a vector of intermediate outcomes)

in defining the effects of T1.

27

Use an ANOVA-like decomposition:

Representing the effects

28

where

Causal effects:

Nuisance parameters: and

29

General FormulaNew ANOVA

Z1 matrix of stage 1 factor columns, Z2 is the matrix of stage 2 factor columns, Y is a vector

Classical ANOVA

30

Aliasing{Z1, Z2} is determined by the experimental design

The defining words (associated with a fractional factorial experimental design) identify common columns in the collection {Z1, Z2}

ANOVA

31

Aliasing

ANOVA

Consider designs with a shared column in both Z1 and Z2 only if the column in Z1 can be safely assumed to have a zero η coefficient or if the column in Z2 can be safely assumed to have a zero β, α coefficient. The defining words inform the aliasing in this case.

32

Simple Examples

33

Six Factors:

M1, E, C, T, A2(only for R=1), M2(only for R=0), each with 2 levels

(26= 64 simple dynamic treatment regimes)

The budget permits 16 cells --16 simple dynamic treatment regimes.

Simple Example

34

Assumptions

A2C, A2T, M2E, M2T and CE along with the main effects in stage 1 and 2 are of primary interest.

• Working Assumption: All remaining causal effects are likely negligible.

• Formal Assumption: Consider designs for which a shared column in Z1 and Z2 occurs only if the column in Z1 can be safely assumed to have a zero η coefficient (concerns interactions of stage 1 factors with R) or if the column in Z2 can be safely assumed to have a zero β/α coefficient (stage 2 effects).

35

Design 1

• No formal assumptions. I=M1ECT

• The design column for A2=M2 is crossed with stage 1 design.

• A2T is aliased with A2M1EC. The interaction A2T is of primary interest and the working assumption was that A2M1EC is negligible.

• CE is aliased with M1T. The interaction CE is of primary interest and the working assumption was that M1T is negligible.

36

Design 2

• Formal assumption: No three way and higher order stage 2 causal effects & no four way and higher order effects involving R and stage 1 factors.

I=M2M1ECT=A2M1ECT

• A2T are aliased with M1CE; the interaction A2T is of primary interest and the working assumption was that M1CE is negligible.

• M2M1T is negligible so CE is not aliased.

37

Interesting Result in Simulations

• In simulations formal assumption are violated.

• Response rates (probability of R=1) across 16 cells range from .55 to .73

• Results are surprisingly robust to a violation of formal assumptions.

•The maximal value of the correlation between 32 estimators of effects was .12 and average absolute value is .03

•Why? Binary response variables can not vary that much. If response rate is constant, then the effect estimators are uncorrelated as in classical experimental design.

38

Discussion

In classical screening experiments we

• Screen marginal causal effects

• Design experiment using working assumptions concerning the negligibility of the effects.

• Designs and analyses permit one to determine aliasing

• Minimize formal assumptions

We can do this as well when screening for multi-stage decision problems.

39

Discussion

• Compare this to using observational studies to construct dynamic treatment regimes– Uncontrolled selection bias (causal misattributions)

– Uncontrolled aliasing.

• Secondary analyses would assess if variables collected during treatment should enter decision rules.

• This seminar can be found at: http:// www.stat. lsa.umich.edu/~samurphy/seminars/MichStatistics01.06.ppt

40

Reality

Unknown UnknownCauses Causes

X1 T1 X2 T2 Y