What size of trial do I need? Peter T. Donnan Professor of Epidemiology and Biostatistics Co-Director of TCTU Statistics for Health Research.

What size of trial do I need?

Peter T. Donnan

Professor of Epidemiology and Biostatistics

Co-Director of TCTU

Statistics for Health Statistics for Health ResearchResearch

What size of study do I need?

10 10,000

or

AnswerAnswer•As large as possible!As large as possible!

•Data is information, so Data is information, so the more data the sounder the more data the sounder the conclusionsthe conclusions

•In real world, data limited In real world, data limited by resources: access to by resources: access to patients, money, time, etc.. patients, money, time, etc..

What size of study do I What size of study do I need?need?

Expand the question:Expand the question:

What size of study do I need to What size of study do I need to answer the question posedanswer the question posed, , given the size of my practice / given the size of my practice / clinic, or no. of samples, given clinic, or no. of samples, given the amount of resources (time the amount of resources (time and money) I have to collect and money) I have to collect the information?the information?

What is the question?What is the question?

•Trial is Comparative: new Trial is Comparative: new drugs (CTIMP), drugs (CTIMP), management of patients, management of patients, etc… etc…

•EfficacyEfficacy

•EquivalenceEquivalence

•Non-inferiorityNon-inferiority

Why bother?Why bother?

1.1.You will not get your You will not get your study past ethics!study past ethics!

2.2.You will not get your You will not get your proposal past a statistical proposal past a statistical review by funders!review by funders!

3.3.It will be difficult to It will be difficult to publish your results! publish your results!

Why bother?Why bother?

•Is the study feasible?Is the study feasible?

•Is likely sample size Is likely sample size enough to show enough to show meaningfulmeaningful differences differences with statistical with statistical significance?significance?

•Does number planned give Does number planned give enough power or need enough power or need larger number? larger number?

OBJECTIVES OBJECTIVES

•Understand issues Understand issues involved in estimating involved in estimating sample sizesample size

•Sample size is dependent Sample size is dependent on on designdesign and and type of type of analysis analysis

•Parameters needed for Parameters needed for sample size estimation sample size estimation

OBJECTIVESOBJECTIVES

•Understand what is Understand what is necessary to carry out necessary to carry out some simple sample size some simple sample size calculationscalculations

•Carry out these Carry out these calculations with softwarecalculations with software

•Note SPSS does not yet Note SPSS does not yet have a sample size have a sample size calculator calculator

What is the measure of What is the measure of outcome?outcome?

•Difference in Change in :Difference in Change in :

•Scores, physiological Scores, physiological measures (BP, Chol), measures (BP, Chol), QOL, hospitalisations, QOL, hospitalisations,

mortality, etc…. mortality, etc….

•Choose PRIMARY OUTCOMEChoose PRIMARY OUTCOME

•A number of secondary but A number of secondary but not too many!not too many!

Intervention – Intervention – Randomised Controlled Randomised Controlled

TrialTrial1) Randomisation by 1) Randomisation by patient-patient-

RCTRCT

Crossover trialCrossover trial

2) Randomisation by 2) Randomisation by practice practice (Cluster (Cluster randomisation) randomisation)

RANDOMISED CONTROLLED RANDOMISED CONTROLLED TRIAL (RCT)TRIAL (RCT)

Gold standard Gold standard method to method to assess assess EfficacyEfficacy of treatmentof treatment

RANDOMISED CONTROLLED RANDOMISED CONTROLLED TRIAL (RCT)TRIAL (RCT)

Random allocation to Random allocation to intervention or control so intervention or control so likely balance of all likely balance of all factors affecting outcomefactors affecting outcome

Hence any difference in Hence any difference in outcome ‘caused’ by the outcome ‘caused’ by the interventionintervention

Randomised Controlled TrialRandomised Controlled Trial

RANDOMISED

Eligible subjects

Intervention

Control

INTERVENTION:INTERVENTION:

•new drug/therapy new drug/therapy

•patient educationpatient education

•Health professional Health professional educationeducation

•organisational changeorganisational change

To improve patient care To improve patient care and/or efficiency of care and/or efficiency of care deliverydelivery

ExampleExample

RCT of new statin vs. oldRCT of new statin vs. old

Evaluate cost-Evaluate cost-effectiveness of new effectiveness of new statinstatin

Randomise eligible Randomise eligible individuals to either individuals to either receive new statin or old receive new statin or old statinstatin

Eligible subjectsEligible subjects

Evaluate cost-Evaluate cost-effectiveness of new effectiveness of new statin on:statin on:

Men ? Aged over 50?Men ? Aged over 50?

Cardiovascular disease?Cardiovascular disease?

Previous MI?Previous MI?

Requires precise INCLUSION Requires precise INCLUSION and EXCLUSION criteria in and EXCLUSION criteria in protocolprotocol

WHAT IS THE OUTCOME?WHAT IS THE OUTCOME?

•Improvement in patients healthImprovement in patients health

•Reduction in CV hospitalisationsReduction in CV hospitalisations

•More explicitly a greater More explicitly a greater reduction in mean lipid levels reduction in mean lipid levels in those receiving the new statin in those receiving the new statin compared with the old statincompared with the old statin

•Reduction in costsReduction in costs

Effect size?Effect size?

Sounds a bit chicken and Sounds a bit chicken and egg!egg!

Likely size of effect: Likely size of effect:

What is the minimum What is the minimum effect size you will accept effect size you will accept as being as being clinicallyclinically or or scientificallyscientifically meaningful? meaningful?

Effect size?Effect size?

Change in Percentage with Change in Percentage with Total Cholesterol < 5 Total Cholesterol < 5 mmol/lmmol/l

• NewNew OldOldDifference Difference

• 40%40% 20%20% 20%20%

• 30%30% 20%20% 10%10%

• 25%25% 20%20% 5%5%

Variability of effect?Variability of effect?

Variability of size of effect: Variability of size of effect:

Obtained from previous Obtained from previous published studies published studies and/orand/or

Obtained from pilot work Obtained from pilot work prior to main studyprior to main study

Variability of effect?Variability of effect?

For a comparison of two For a comparison of two proportions the variability proportions the variability of size of effect is of size of effect is dependent on:dependent on:

1) the size of the study and 1) the size of the study and

2) the size of the 2) the size of the proportions or percentagesproportions or percentages

How many subjects?How many subjects?

•1) Likely size of effect 1) Likely size of effect

•2) Variability of effect 2) Variability of effect

•3) Statistical significance 3) Statistical significance level level

•4) Power4) Power

•5) 1 or 2-sided tests5) 1 or 2-sided tests

Statistical significance or Statistical significance or type I errortype I error

Type I error – rejecting null Type I error – rejecting null hypothesis when it is true: False hypothesis when it is true: False positive (Prob=positive (Prob= ) )

Generally use 5% level (Generally use 5% level ( = 0.05) = 0.05) i.e. accept evidence that null i.e. accept evidence that null hypothesis unlikely with p< 0.05hypothesis unlikely with p< 0.05

May decrease this for multiple May decrease this for multiple testing e.g. with 10 tests accept p testing e.g. with 10 tests accept p < 0.005< 0.005

1 or 2-sided?1 or 2-sided?

Generally use 2-sided Generally use 2-sided significance tests unless significance tests unless VERYVERY strong belief one strong belief one treatment could not be worse treatment could not be worse than the otherthan the other

e.g. Weakest NSAID e.g. Weakest NSAID compared with new Cox-2 compared with new Cox-2 NSAIDNSAID

How many subjects?How many subjects?

•1) Likely size of effect 1) Likely size of effect

•2) Variability of effect 2) Variability of effect

•3) Statistical significance 3) Statistical significance level level

•4) Power4) Power

•5) 1 or 2-sided tests 5) 1 or 2-sided tests

Power and type II errorPower and type II error

Type II error (False Type II error (False negative):negative):

Not rejecting the null Not rejecting the null hypothesis (non-hypothesis (non-significance) when it is significance) when it is falsefalse

Probability of type II error - Probability of type II error -

Power = 1 - Power = 1 - , typically , typically 80%80%

Type I and Type II errorsType I and Type II errors

ErrorError Prob.Prob. Screening Screening

Type I (Type I ()) False False 1-1-specificity specificity positivepositive

Type IIType II (()) FalseFalse 1-1-sensitivitysensitivity negativenegative

Analogy with sensitivity and Analogy with sensitivity and specificityspecificity

PowerPower

Acceptable power 70% - 99%Acceptable power 70% - 99%

If sample size is not a If sample size is not a problem go for 90% or 95% problem go for 90% or 95% power; power;

If sample size could be If sample size could be problematic go for lower problematic go for lower power but still sufficient power but still sufficient e.g. 80%e.g. 80%

PowerPower

In some studies finite limit In some studies finite limit on the possible size of the on the possible size of the study then question is study then question is rephrased:rephrased:

What likely effect size will I What likely effect size will I be able to detect given a be able to detect given a fixed sample size?fixed sample size?

How many subjects?How many subjects?

•1) Likely size of effect 1) Likely size of effect

•2) Variability of effect 2) Variability of effect

•3) Statistical significance 3) Statistical significance level level

•4) Power 4) Power

•5) 1 or 2-sided tests 5) 1 or 2-sided tests

Sample size for difference Sample size for difference in two proportions in two proportions

Number needed for Number needed for comparison depends on comparison depends on statistical test used statistical test used For comparison of two For comparison of two proportions or proportions or percentages use Chi-percentages use Chi-Squared (Squared (22) test) test

Comparison of two Comparison of two proportionsproportions

Number in each arm = Number in each arm =

221

22112

2

pp

p100pp100pzzn

Where pWhere p11 and p and p22 are the are the percentages in group 1 and group percentages in group 1 and group 2 respectively2 respectively

Assume 90% power and 5% Assume 90% power and 5% statistical significance (2-statistical significance (2-

sided)sided)Number in each arm = Number in each arm =

221

2211

ppp100pp100p507.10

n

zz = 1.96 (5% significance level, 2- = 1.96 (5% significance level, 2-sided) zsided) z22ββ = 1.28 ( 90% power) are = 1.28 ( 90% power) are obtained from Normal distributionobtained from Normal distribution

Assume 40% reach lipid target Assume 40% reach lipid target on new statin and 20% on old on new statin and 20% on old

drugdrug

Number in each arm = Number in each arm = 105105

Total = 210Total = 210

22040

80206040507.10n

Comparison of two Comparison of two proportionsproportions

Repeat for different effects Repeat for different effects

•NewNew OldOld DifferenceDifference nn Total Total

•40%40% 20%20% 20%20% 105105 210 210

•30%30% 20%20%10%10%•25%25% 20%20%

5% 5% 19641964 3928 3928

472472 944 944

n.b. Halving effect size increases size by factor 4!n.b. Halving effect size increases size by factor 4!

Increase in sample size with decrease in difference

Group 2 proportion, ÒÂ

0.10 0.12 0.14 0.16 0.18 0.20 0.22 0.24 0.26 0.28 0.30 0.32 0.34

n pe

r gro

up

0

200

400

600

800

1000

1200

1400

1600

1800

Two group ׿ test of equal proportions (odds ratio = 1) (equal n's)Æ = 0.050 ( 2) ÒÁ= 0.400 Pow= 90

Sample Size per Group

100 200 300 400 500

Power

50

60

70

80

90

100

Two group ׿ test of equal proportions (odds ratio = 1) (equal n's)Æ = 0.050 ( 2) ÒÁ= 0.400 ÒÂ= 0.300

Increase in power with sample size

Comparison of two means Comparison of two means has a similar formulahas a similar formula

Number in each arm = Number in each arm =

( )( )221

22β2α

x-x

σz+z2=n

Where and are the means in Where and are the means in group 1 and group 2 respectively group 1 and group 2 respectively and and is the assumed standard is the assumed standard deviationdeviation

1x 2x

Allowing for loss to follow-Allowing for loss to follow-up / non-compliance up / non-compliance

The number estimated for The number estimated for statistical purposes may need statistical purposes may need to be inflated if likely that a to be inflated if likely that a proportion will be lost to proportion will be lost to follow-upfollow-upFor example if you know For example if you know approx. 20% will drop-out approx. 20% will drop-out inflate sample size by 1/(1-inflate sample size by 1/(1-0.2) = 1.25 0.2) = 1.25

Software and other sample Software and other sample size estimation size estimation

The formula depends on the The formula depends on the nature of the outcome and nature of the outcome and likely statistical testlikely statistical test

Numerous texts with sample Numerous texts with sample size tables and formulasize tables and formula

Software – nQuery Software – nQuery Advisor®Advisor®

SUMMARYSUMMARY

In planning consider:In planning consider:

design, type of design, type of intervention, outcomes, intervention, outcomes, sample size, power, and sample size, power, and ethics together at the ethics together at the design stagedesign stage

SUMMARYSUMMARY

Invaluable information is Invaluable information is gained from pilot work and gained from pilot work and also more likely to be also more likely to be funded (CSO)funded (CSO)

Sample size follows from Sample size follows from type of analysis which type of analysis which follows from designfollows from design

SUMMARYSUMMARY

Pilot also gives information Pilot also gives information on recruitment rateon recruitment rate

You may need to inflate You may need to inflate sample size due to:sample size due to:

Loss of follow-up/ drop-Loss of follow-up/ drop-outout

Low complianceLow compliance

Remember the checklistRemember the checklist

•1) Likely size of effect 1) Likely size of effect

•2) Variability of effect 2) Variability of effect

•3) Statistical significance 3) Statistical significance level level

•4) Power 4) Power

•5) 1 or 2-sided tests 5) 1 or 2-sided tests

SUMMARYSUMMARY

Remember in Scientific Remember in Scientific Research:Research:

Size Size MattersMatters