Randomised Controlled Trials Prof. Craig Jackson Head of Psychology Division School of Social Sciences BCU health.bcu.ac.uk/craigjackson #WPwork @Workplace_Prof.

Randomised Controlled TrialsRandomised Controlled Trials

Prof. Craig Jackson Head of Psychology Division

School of Social SciencesBCU

health.bcu.ac.uk/craigjackson#WPwork

@Workplace_Prof@bcu_psychology

[email protected]@bcu.ac.uk

ObjectivesObjectives

Experimental studiesExperimental studieswithin-subjects studieswithin-subjects studiesbetween-subjects studiesbetween-subjects studies

Observational studiesObservational studiescase-controlscase-controlscohortscohortsRCTsRCTs

BiasBias

PlaceboPlacebo

Control GroupsControl Groups!!

IntroductionIntroduction

Types of researchTypes of research Experimental vs. ObservationalExperimental vs. Observational

Longitudinal vs. Cross-sectionalLongitudinal vs. Cross-sectional

Prospective vs. RetrospectiveProspective vs. Retrospective

LongitudinalLongitudinal

ProspectiveProspective

ExperimentalExperimental

Randomised Controlled TrialRandomised Controlled Trial

Qualititative VS Quantitative ResearchQualititative VS Quantitative Research

False oppositionFalse opposition

Observational methods equally validObservational methods equally valid

Complementary rolesComplementary roles

Qualitative equally as hard to doQualitative equally as hard to do

Quantitative

Qualitative

Experimental StudiesExperimental Studies

Investigator makes interventionInvestigator makes intervention

A “manipulation”A “manipulation”

Then studies the effects of that intervention Then studies the effects of that intervention

Features:Features:

Comparison e.g. before vs. afterComparison e.g. before vs. after control vs. treatmentcontrol vs. treatment

Always longitudinal Always longitudinal

Always prospective Always prospective

Experimental Experimental

Clinical Clinical

TrialsTrials

RCTsRCTs

Rationale of Experimental StudiesRationale of Experimental Studies

Evaluate effectiveness of intervention / therapyEvaluate effectiveness of intervention / therapy

Use similar samples – comparable groupsUse similar samples – comparable groups

Samples reflect populationSamples reflect population

Differences in outcomes due to interventions (not differences Differences in outcomes due to interventions (not differences between groups)between groups)

Independent Variable (IV) alters Dependent Variable (DV)Independent Variable (IV) alters Dependent Variable (DV)

Best evidence of cause and effectBest evidence of cause and effect

Sometimes inconclusiveSometimes inconclusive

Types of Experimental StudiesTypes of Experimental Studies

Between Subjects StudiesBetween Subjects Studies

Each group receives different treatmentEach group receives different treatment

Groups comparedGroups compared

Within Subjects StudiesWithin Subjects Studies

Each individual is measured before & after interventionEach individual is measured before & after intervention

Advantage that each participant is own controlAdvantage that each participant is own control Between subject variability removedBetween subject variability removed

Within Subjects StudiesWithin Subjects Studies

Cross-over-studies Cross-over-studies Each patient receives treatment in sequenceEach patient receives treatment in sequence““Washout” period between treatmentsWashout” period between treatmentsOrder of treatments randomisedOrder of treatments randomised

Matched-pairs studyMatched-pairs studyParallel studyParallel studyPatient in arm 1 matched with patient in arm 2Patient in arm 1 matched with patient in arm 2Matched based on prognostic factorsMatched based on prognostic factorsData is linkedData is linkedPaired individualsPaired individuals

Treatment Treatment 11Treatment 2Treatment 2

Gp AGp A

Gp BGp B

Treatment 2Treatment 2

Treatment Treatment 11

Avoiding BiasAvoiding Bias

Validity of study depends on Validity of study depends on avoiding biasavoiding bias

Bias = “Systematic distortion of results due to unforeseen factors”Bias = “Systematic distortion of results due to unforeseen factors”

e.g.e.g.gp1 = pillgp1 = pill gp2 = no pillgp2 = no pill

How will the “no pill”group progress?How will the “no pill”group progress?Any effects of them knowing they have no treatment?Any effects of them knowing they have no treatment?

Handling differences may influence + complicate trial resultsHandling differences may influence + complicate trial results

Known as Known as confounding factorsconfounding factors

To minimize bias… To minimize bias… control groupcontrol group randomisationrandomisation blindingblinding

Placebo effect – it really does work!Placebo effect – it really does work!

Most effectiveMost effective medication known medication known

In approx. 30% of pop.In approx. 30% of pop.

Subjected to more clinical trialsSubjected to more clinical trials than any other medicament than any other medicament

Nearly always does better thanNearly always does better than anticipatedanticipated

The range of susceptible conditions seems limitlessThe range of susceptible conditions seems limitless

Does not always occurDoes not always occur

Present in subjective and objective outcomesPresent in subjective and objective outcomes

Negative outcomes can occur (Nocebo effect)Negative outcomes can occur (Nocebo effect)

PlaceboPlacebo•Big pills better than smaller pillsBig pills better than smaller pills•Red pills better than blueRed pills better than blue•4 pills better than 24 pills better than 2•30% of pop.30% of pop. Patient’s “knowledge” of their treatment causes bias

Patient’s “knowledge” of their treatment causes bias

e.g. Benedetti & the Turin study

e.g. Benedetti & the Turin study

Control GroupsControl Groups

Allow comparison in Between Group studies Allow comparison in Between Group studies

Evaluations without comparison?Evaluations without comparison?

Patient knowledge of their treatment causes biasPatient knowledge of their treatment causes bias

e.g. Benedetti & the Turin studye.g. Benedetti & the Turin study

Types of Control GroupsTypes of Control Groups

•““no treatment” group no treatment” group likely to be confounded by conditionlikely to be confounded by condition

•““placebo” groupplacebo” group ethically dodgy? ethically dodgy?

•““low dose” grouplow dose” group avoids ethical issuesavoids ethical issues

•““standard treatment” groupstandard treatment” group avoids ethical issues avoids ethical issues

•““gold standard” group gold standard” group avoids ethical issuesavoids ethical issues

•““historical controls”historical controls” too many confounderstoo many confounders

Drug XDrug X

53 yrs53 yrs

80% male80% male20% fem20% fem

Gp AGp A(500)(500)

Gp BGp B(500)(500)

Drug YDrug Y

27 years27 years

50% male50% male50% fem50% fem

PopulationPopulation(60 million)(60 million)

SampleSample(1000)(1000)

Control Groups: Random Control Groups: Random AllocationAllocation

Doesn’t guarantee groups will be Doesn’t guarantee groups will be homogonoushomogonous

Ensures allocation independent of patient Ensures allocation independent of patient featuresfeatures

Avoids (sub)conscious allocation biasAvoids (sub)conscious allocation biase.g. severely sick into treatment groupse.g. severely sick into treatment groups

Guarantees Guarantees allocationallocation to be bias-freeto be bias-free

Non-homogenous groups may still occur Non-homogenous groups may still occur due to chance – random errorsdue to chance – random errors

Comparison Groups: Random SamplingComparison Groups: Random Sampling

Ensures generalizability of findings to larger pop.Ensures generalizability of findings to larger pop.e.g. in-custody sample limitationse.g. in-custody sample limitations

Treatment effects best detected if little between-group variabilityTreatment effects best detected if little between-group variability

Exclusion Criteria & Inclusion Criteria keep groups comparableExclusion Criteria & Inclusion Criteria keep groups comparable

Paradox:Paradox:greater uniformity of sample = less generalizable to gen. popgreater uniformity of sample = less generalizable to gen. pop

Blinding: Importance of doing itBlinding: Importance of doing it

Investigator or Subject know treatment = BiasInvestigator or Subject know treatment = Bias

Observations and Judgements become less reliableObservations and Judgements become less reliable

Patient responses change:Patient responses change:Positive outcomes in active armPositive outcomes in active armNegative outcomes in passive armNegative outcomes in passive arme.g. known cancer diagnoses and deterioratione.g. known cancer diagnoses and deterioration

Use max. degree of blindness possibleUse max. degree of blindness possiblee.g. make subject and investigator both blind if possiblee.g. make subject and investigator both blind if possiblee.g. A.A.Mason & Congenital Ichthyosis and Hypnosis 1951e.g. A.A.Mason & Congenital Ichthyosis and Hypnosis 1951

Blinding: MethodsBlinding: Methods

Double-blindDouble-blind patient & investigator blindpatient & investigator blind

Single-blindSingle-blind patient blindpatient blind

Triple-blindTriple-blind patient & investigator & data monitor blindpatient & investigator & data monitor blind

Double-dummyDouble-dummy 2 treatments2 treatments patients get 2 pills (1 active, 1 dummy)patients get 2 pills (1 active, 1 dummy)

Open trialsOpen trials patient & investigator aware of treatmentpatient & investigator aware of treatment

Randomisation in a double-blind trialRandomisation in a double-blind trialEnvelope technique commonEnvelope technique common

Un-blinding – ethical necessityUn-blinding – ethical necessity

Subject Variables that confound researchSubject Variables that confound researchSTABLE FACTORSSTABLE FACTORS SITUATIONAL FACTORSSITUATIONAL FACTORS

AgeAge Alcohol (recent useAlcohol (recent useEducationEducation Caffeine (recent use)Caffeine (recent use)SexSex Nicotine (recent use)Nicotine (recent use)SocioeconomicsSocioeconomics Medication (recent use)Medication (recent use)LanguageLanguage Paints, glues, pesticides Paints, glues, pesticides HandednessHandedness Time of day Time of dayPhysical activity Time of yearPhysical activity Time of yearNear visual acuityNear visual acuity Screen luminanceScreen luminanceComputer experienceComputer experience Restricted movement (injury)Restricted movement (injury)Caffeine (habitual use)Caffeine (habitual use) Cold / fluCold / fluAlcohol (habitual use)Alcohol (habitual use) StressStressNicotine (habitual use)Nicotine (habitual use) Arousal / FatigueArousal / FatigueMedication (habitual use)Medication (habitual use) SleepSleepPaints, glues, pesticides Paints, glues, pesticides DiabetesDiabetesEpilepsyEpilepsyOther CNS / PNS diseaseOther CNS / PNS diseaseHead injury (out >1 hr)Head injury (out >1 hr)Alcohol / drug addictionAlcohol / drug addiction

Randomized Controlled Trials in PracticeRandomized Controlled Trials in Practice

90% consultations take place in GP surgery90% consultations take place in GP surgery

RCT is really 50 years oldRCT is really 50 years old

Potential problemsPotential problems

2 Key areas:2 Key areas: Recruitment BiasRecruitment Bias

Randomisation BiasRandomisation Bias

Over-focus on failings of RCTsOver-focus on failings of RCTs

RCTs in PracticeRCTs in Practice

RCTs justified inRCTs justified in situations of genuine clinical uncertaintysituations of genuine clinical uncertainty

Samples large enough to establish any worthwhile benefit Samples large enough to establish any worthwhile benefit

(effectiveness or cost, or both) (effectiveness or cost, or both)

Need for larger numbers of participants Need for larger numbers of participants

More than are availableMore than are available to single practicesto single practices

Requires “club together” approachRequires “club together” approach

Practitioners: no contractual obligationPractitioners: no contractual obligation

(i)(i) unwilling to take part if no immediate benefit for clientsunwilling to take part if no immediate benefit for clients

(ii)(ii) while possibly disruptingwhile possibly disrupting the delivery of service/carethe delivery of service/care

RCTs in PracticeRCTs in Practice

Conflict of interest between:Conflict of interest between:Role and Wish to benefit patientsRole and Wish to benefit patientsAcademic meritAcademic merit

Long term nature of practitionerLong term nature of practitioner and client relationship and client relationship may engender loyaltiesmay engender loyaltiesunfairly coerce clientsunfairly coerce clients to give consentto give consent

Patients' fearsPatients' fears about:about:confidentialityconfidentialityrisks of the interventionrisks of the interventionapparent disadvantage of being allocated to a control groupapparent disadvantage of being allocated to a control group

may further inhibit recruitmentmay further inhibit recruitmentFail to recruit consecutiveFail to recruit consecutive clients may introduce selection biasclients may introduce selection bias

RCTs in PracticeRCTs in Practice

Provides rigorous,Provides rigorous, sound basis for evaluating treatmentssound basis for evaluating treatments

May disrupt care May disrupt care Too much disruption = no reflection of real practiceToo much disruption = no reflection of real practice

MethodologicalMethodological problems reduce reliability of the results problems reduce reliability of the results (Recruitment & Randomisation) (Recruitment & Randomisation)

Practice not a laboratoryPractice not a laboratoryPeoplePeople are not experimental animalsare not experimental animals

Case-control studies, retrospectiveCase-control studies, retrospective, , prospective cohort prospective cohort studies,observation and descriptive studies arestudies,observation and descriptive studies are all acceptable all acceptable

methods. methods.

Accept alternative methodsAccept alternative methods when RCT difficult or flawedwhen RCT difficult or flawed

RCT DeficienciesRCT Deficiencies

Trials too smallTrials too small

Trials too shortTrials too short

Poor qualityPoor quality

Poorly presentedPoorly presented

Address wrong questionAddress wrong question

Methodological inadequaciesMethodological inadequacies

Inadequate measures of quality of life (changing)Inadequate measures of quality of life (changing)

Cost-data poorly presentedCost-data poorly presented

Ethical neglectEthical neglect

Participants given limited understandingParticipants given limited understanding

Poor trial managementPoor trial management

PoliticsPolitics

MarketeeringMarketeering

Why still the dominant model?Why still the dominant model?

Observational StudiesObservational Studies

Investigator observes existing situationInvestigator observes existing situation

DescribesDescribes

Analyses Analyses

Interprets Interprets

No influence on events No influence on events

Longitudinal observation studiesLongitudinal observation studies

case-control studies: retrospectivecase-control studies: retrospective

cohort-studies: prospective cohort-studies: prospective

Cross-sectional observation studiesCross-sectional observation studies

surveys examining subjects at one point in timesurveys examining subjects at one point in time

based on random sample of interest populationbased on random sample of interest population

Observational Studies:Observational Studies:

Look for associationsLook for associations• Cause --> EffectCause --> Effect• Exposure --> IllnessExposure --> Illness• EpidemiologicalEpidemiological• IncidenceIncidence• CauseCause• PreventionPrevention

No controlNo controlCannot use classical experimentationCannot use classical experimentationNo randomisationNo randomisationBiasBias

Case-Control StudyCase-Control Study

Identify group with condition / offence (cases)Identify group with condition / offence (cases)

Identify group without condition / offence (controls)Identify group without condition / offence (controls)

Both groups compared for exposure to (hypothesized) risk factorsBoth groups compared for exposure to (hypothesized) risk factors

Greater exposure to risk factor in cases = “causal relation”Greater exposure to risk factor in cases = “causal relation”

BiasesBiases

Lead time biasLead time bias

Recruitment of cases at similar points in timeRecruitment of cases at similar points in time

Newly labelled casesNewly labelled cases

Selection of ControlsSelection of Controls

Cases have lung cancerCases have lung cancer

Controls could be other patients or “normals”Controls could be other patients or “normals”

Matched Cases & Controls for age & genderMatched Cases & Controls for age & gender

Option of 2 Controls per CaseOption of 2 Controls per Case

Smoking years of cases and controls Smoking years of cases and controls (matched for age and sex)(matched for age and sex)

CasesCases ControlsControls

n=456n=456 n=456n=456

FF PPSmoking yrsSmoking yrs 13.7513.75 6.126.12 7.57.5 0.040.04

(± 1.5)(± 1.5) (± 2.1)(± 2.1)

Case-Control Study: Other BiasesCase-Control Study: Other Biases

Recall BiasRecall BiasCases > associations with exposures / risk factorsCases > associations with exposures / risk factors

Unreliable MemoriesUnreliable MemoriesRetrospective natureRetrospective natureOver-reliance on recallOver-reliance on recall

Unreliable RecordsUnreliable RecordsPoor hospital recordsPoor hospital recordsRepetitive, incomplete, inaccurate, irretrievable, interpretationRepetitive, incomplete, inaccurate, irretrievable, interpretation

Interview BiasInterview BiasDifferent interviewersDifferent interviewers

Cohort StudyCohort Study

ID and examination of a group (cohort)ID and examination of a group (cohort)

Followed over time (20 years common!)Followed over time (20 years common!)

Looking for condition development / other end-pointLooking for condition development / other end-point

Aetiology of condition (based on data collected)Aetiology of condition (based on data collected)Data more reliable than case-control studiesData more reliable than case-control studies

• Requires large NRequires large N• Requires long follow upRequires long follow up• InefficientInefficient• Expensive (espec. rare outcomes)Expensive (espec. rare outcomes)

Cohort Study: MethodsCohort Study: Methods

Volunteers in 2 groups e.g. exposed vs non-exposedVolunteers in 2 groups e.g. exposed vs non-exposed

All complete attitude survey every 12 monthsAll complete attitude survey every 12 months

End point at 5 years: groups compared for Health StatusEnd point at 5 years: groups compared for Health Status

Comparison of general health between users and non-users Comparison of general health between users and non-users of mobile phonesof mobile phones

illill healthyhealthy

mobile phone usermobile phone user 292292 108108 400400

non-phone usernon-phone user 8989 313313 402402

381381 421421 802802

Cohort Study: Other BiasesCohort Study: Other Biases

Lost to follow upLost to follow upBias if reason related to exposureBias if reason related to exposureValidity affectedValidity affected

Group sizes changeGroup sizes changeMembership changes e.g ex-smokersMembership changes e.g ex-smokersDifferential mortalityDifferential mortality

Change in circumstanceChange in circumstancee.g. job changee.g. job changeExposures need calculation or re-calculationExposures need calculation or re-calculation

Surveillance biasSurveillance biasInvestigator aware of group membershipInvestigator aware of group membershipInvestigating exposed members moreInvestigating exposed members more

Cross Sectional StudyCross Sectional Study

Subjects contacted & surveyed just onceSubjects contacted & surveyed just once

Questionnaire (post, email, phone)Questionnaire (post, email, phone)

Random sample of defined pop.Random sample of defined pop.

Limited causalityLimited causalityNot temporal relationshipsNot temporal relationshipsLittle insight into aetiologyLittle insight into aetiologySource of descriptive dataSource of descriptive dataPrevalence ratesPrevalence ratesVolunteer biasVolunteer biasNon responsesNon responsesSelf-selectionSelf-selectionUnrepresentative sampleUnrepresentative sample

Further ReadingFurther Reading

Altman, D.G. “Designing Research”. In: Altman, D.G., (ed.) Altman, D.G. “Designing Research”. In: Altman, D.G., (ed.) Practical Practical Statistics For Medical ResearchStatistics For Medical Research. London, Chapman and Hall, 1991; . London, Chapman and Hall, 1991; 74-106.74-106.

Bland, M. “The design of experiments”. In: Bland, M., (ed.) Bland, M. “The design of experiments”. In: Bland, M., (ed.) An An introduction to medical statisticsintroduction to medical statistics. Oxford, Oxford Medical . Oxford, Oxford Medical Publications, 1995; 5-25.Publications, 1995; 5-25.

Daly, L.E., Bourke, G.J. “Epidemiological and clinical research Daly, L.E., Bourke, G.J. “Epidemiological and clinical research methods”. In: Daly L.E., Bourke, G.J., (eds.) methods”. In: Daly L.E., Bourke, G.J., (eds.) Interpretation and uses Interpretation and uses of medical statistics.of medical statistics. Oxford, Blackwell Science Ltd, 2000; 143-201. Oxford, Blackwell Science Ltd, 2000; 143-201.

Jackson, C.A. “Study Design” & “Sample Size and Power”. In: Gao Jackson, C.A. “Study Design” & “Sample Size and Power”. In: Gao Smith, F. and Smith, J. (eds.)Smith, F. and Smith, J. (eds.) Key Topics in Clinical Research Key Topics in Clinical Research. . Oxford, BIOS scientific Publications,Oxford, BIOS scientific Publications, 2002 2002. .

Further ReadingFurther Reading

Jackson, C.A. “Planning Health & Safety Research Projects in the Jackson, C.A. “Planning Health & Safety Research Projects in the Workplace”. Workplace”. Croner Health and Safety at Work Special ReportCroner Health and Safety at Work Special Report 2002; 2002; 62: 1-16. 62: 1-16.

Kumar, R. Kumar, R. Research Methodology: a step by step guide for Research Methodology: a step by step guide for beginnersbeginners. Sage, London 1999.. Sage, London 1999.

Abbott, P. and Sapsford. Abbott, P. and Sapsford. Research methods for nurses and the Research methods for nurses and the caring professions.caring professions. Open University Press, Buckingham 1988. Open University Press, Buckingham 1988.

Bowling, A. Measuring Health. Open University Press, Milton Keynes Bowling, A. Measuring Health. Open University Press, Milton Keynes 19941994

Polit, D. & Hungler, B. Nursing research: Principles and methods (7th Polit, D. & Hungler, B. Nursing research: Principles and methods (7th ed.). Philadelphia: Lippincott, Williams & Wilkins 2003.ed.). Philadelphia: Lippincott, Williams & Wilkins 2003.