Ethics Committees and Quantitative Research: Confessions of a jobbing researcher or “How I learned to stop worrying, and just enjoy doing the research”

Ethics Committees and Ethics Committees and Quantitative Research:Quantitative Research:

Confessions of a Confessions of a jobbing researcherjobbing researcher

oror

““How I learned to stop How I learned to stop worrying, and just enjoy doing worrying, and just enjoy doing

the research”the research”

Dr. Craig JacksonDr. Craig JacksonSenior Lecturer in Health PsychologySenior Lecturer in Health Psychology

School of Health and Policy StudiesSchool of Health and Policy Studies

www.hcc.uce.ac.uk/craig_jacksonwww.hcc.uce.ac.uk/craig_jackson

Brief Research HistoryBrief Research History

Mental Health of UK Farmers using OP Pesticides (X2) -- 1997-2000Mental Health of UK Farmers using OP Pesticides (X2) -- 1997-2000(Epidemiological Surveys)(Epidemiological Surveys)

Neurobehavioural Performance of desert-based Oil Drillers -- 1998-2000Neurobehavioural Performance of desert-based Oil Drillers -- 1998-2000(Clinical assessment)

Temporary Hearing Loss in Student Bar Staff – 2000-2002 Temporary Hearing Loss in Student Bar Staff – 2000-2002 (Epidemiological Survey)

Benefits of Occupational Health Advice in Primary Care Settings -- 2001-2004Benefits of Occupational Health Advice in Primary Care Settings -- 2001-2004(Randomised Controlled Trial)

Smaller-Scale projects – (Tri-Services, NHS Personnel, NHS Patients)Smaller-Scale projects – (Tri-Services, NHS Personnel, NHS Patients)(Cross-sectional Surveys, Clinical Trials)

Multiple roles of psychologist, statistician, and methodology designerMultiple roles of psychologist, statistician, and methodology designer

Good research should be...Good research should be...

JustifiedJustified

Well plannedWell planned

Appropriately designedAppropriately designed

Ethically approvedEthically approved

Ethical misconduct not to meet this standard?Ethical misconduct not to meet this standard?

Design & Ethical ApprovalDesign & Ethical Approval

Research should be driven by protocolResearch should be driven by protocol

Pilot studies should have a written rationalePilot studies should have a written rationale

Protocols should answer specific questionsProtocols should answer specific questions

Not just “collecting data”Not just “collecting data”

Protocols must be agreed by all contributors & participantsProtocols must be agreed by all contributors & participants

Keep the protocol as part of the Research record / logKeep the protocol as part of the Research record / log

Design & Ethical ApprovalDesign & Ethical Approval

Statistical issues should be considered before data collectionStatistical issues should be considered before data collection

Power calculations are becoming essentialPower calculations are becoming essential

Formal documented ethical approval is required for all research involving Formal documented ethical approval is required for all research involving

(i) (i) peoplepeople(ii)(ii) medical recordsmedical records(iii)(iii) anonymous human tissue (Nuffield Council on Bioethics)anonymous human tissue (Nuffield Council on Bioethics)

Fully informed consent should always be soughtFully informed consent should always be sought

If not possible (deceptive studies) a research ethics committee should decideIf not possible (deceptive studies) a research ethics committee should decide

Research Ethics ChecklistResearch Ethics Checklist

• people’s rights and claimspeople’s rights and claims• different sorts of interestsdifferent sorts of interests and their relative strengthand their relative strength• human well-beinghuman well-being• loss of lifeloss of life

• what would be good or bad for peoplewhat would be good or bad for people

• democratic acceptancedemocratic acceptance

• consultationconsultation

• sensitive momentssensitive moments• benefits and harmsbenefits and harms• griefgrief and distressand distress• an obligation to make sacrifices for the community;an obligation to make sacrifices for the community;

• entitlement of the community to deny autonomy and violate bodily integrity in public interestentitlement of the community to deny autonomy and violate bodily integrity in public interest• the systemthe system of justiceof justice• public safetypublic safety• public policy considerationspublic policy considerations• dangerdanger• civil libertiescivil liberties• individual autonomyindividual autonomy• lives and liberties of citizenslives and liberties of citizens

Quantitative Research DesignsQuantitative Research Designs

approach

Observational

Experimental

Survey

Epidemiology

Case - control

Cohort study

Postal questionnaire

Laboratory

RCT

PatientsStaff

Healthy

Ethical Committees (My understanding of them anyway…)Ethical Committees (My understanding of them anyway…)

NHS NHS LRECLREC must be applied to if research involves: must be applied to if research involves:

1. NHS Patients1. NHS Patients

2. NHS Staff *2. NHS Staff *

3. People (not patients) recruited in via NHS3. People (not patients) recruited in via NHS

* Some LRECs do not include this criteria* Some LRECs do not include this criteria

Military Ethics Committees must be applied to if research involves:Military Ethics Committees must be applied to if research involves:

1. Any serving military personnel1. Any serving military personnel

Ethical Committees (My understanding of them anyway…)Ethical Committees (My understanding of them anyway…)

NHS NHS MRECMREC must be applied to if research involves: must be applied to if research involves:

1. NHS Patients1. NHS Patients

2. NHS Staff *2. NHS Staff *

3. People (not patients) recruited in via NHS3. People (not patients) recruited in via NHS

4. Data collected from >1 Health Authority Area4. Data collected from >1 Health Authority Area

5. Project coordinated by agents based in >1 Health Authority Area5. Project coordinated by agents based in >1 Health Authority Area

What Quantitative Research RequiresWhat Quantitative Research Requires

Numerical / Quantifiable dataNumerical / Quantifiable data

Probability-basedProbability-based

NomotheticsNomothetics

Sufficiently large sample size (to detect statistically significant effects)Sufficiently large sample size (to detect statistically significant effects)

Randomised sampling of a population (cannot guarantee a random sample)Randomised sampling of a population (cannot guarantee a random sample)

Statistical analyses of dataStatistical analyses of data

Pet Hates #1Pet Hates #1The Application ProcedureThe Application Procedure

Too longToo long

PrescriptivePrescriptive

Expertise of LREC panel?Expertise of LREC panel?

Monthly meetingsMonthly meetings

““Interview Panel”Interview Panel”

Assumes that “Assumes that “the researcher doesn’t know bestthe researcher doesn’t know best” or at best is unscrupulous” or at best is unscrupulous

Outsourcing for specialist informationOutsourcing for specialist information

Confusion between Anonymity and ConfidentialityConfusion between Anonymity and Confidentiality

Pet Hates #2Pet Hates #2Correspondence and RebuttalCorrespondence and Rebuttal

Power imbalancePower imbalance

Reiterating application and protocolReiterating application and protocol

Time consumingTime consuming

Unfair and unreasonable demandsUnfair and unreasonable demands

Invitation to meetings – Invitation to meetings – impracticalimpracticalunproductiveunproductivekangaroo stylekangaroo style

Correspondence and Rebuttal:Correspondence and Rebuttal:The Placental Blood Flow ExampleThe Placental Blood Flow Example

1.1. Student applies to committee (application, protocol, 2 supervisors)Student applies to committee (application, protocol, 2 supervisors)

2.2. Committee loses applicationCommittee loses application

3.3. Committee contacts student – raises 4 questionsCommittee contacts student – raises 4 questions

4.4. Student addresses 4 questionsStudent addresses 4 questions

5.5. Committee responds – satisfied with 3 answersCommittee responds – satisfied with 3 answersNot happy with statistical proceduresNot happy with statistical procedures

6.6. Student responds with clarificationsStudent responds with clarifications

7.7. Committee responds – final satisfactionCommittee responds – final satisfactionCaveat: student must report to LREC after 1 year Caveat: student must report to LREC after 1 year

get student’s name wrongget student’s name wrong

statistical procedures not statistical procedures not effecting sample sizeeffecting sample size

any jurisdiction?any jurisdiction?

Correspondence and Rebuttal:Correspondence and Rebuttal:The Post-Partum Depression StudyThe Post-Partum Depression Study

1.1. Students apply to committee (application, protocol, supervisor)Students apply to committee (application, protocol, supervisor)

2.2. Committee concerned over 2 widely used psychometric questionnairesCommittee concerned over 2 widely used psychometric questionnairesGHQ 28GHQ 28Edinburgh Post-Natal Depression ScaleEdinburgh Post-Natal Depression Scale

3.3. Concerned the “depressive nature” of items may be “too much”Concerned the “depressive nature” of items may be “too much”Q.25 “Have you thought of the possibility you might make away with yourself?”Q.25 “Have you thought of the possibility you might make away with yourself?”

4.4. Students respond to LREC with references of previous work involving Students respond to LREC with references of previous work involving such psychometricssuch psychometrics

When you’re right . . . You’re right!When you’re right . . . You’re right!The Strengths of LRECsThe Strengths of LRECs

1.1. Sample Size IssuesSample Size Issues

2.2. Power of Study CalculationsPower of Study Calculations

3.3. Statistical Analyses Statistical Analyses

4.4. Student Sub-Committees (e.g South Birmingham)Student Sub-Committees (e.g South Birmingham)

The Importance of Sample SizeThe Importance of Sample Size

• Apparently forgotten in many studiesApparently forgotten in many studies

• Little consideration givenLittle consideration given

• Appropriate size needed to confirm / refute hypothesesAppropriate size needed to confirm / refute hypotheses

• Small samples far too small to detect anything but the grossest differenceSmall samples far too small to detect anything but the grossest difference

• Non-significant results are reported – Type 2 errors occurNon-significant results are reported – Type 2 errors occur

• Too large a sample – unnecessary waste of (clinical) resourcesToo large a sample – unnecessary waste of (clinical) resources

• Ethical considerations – waste of patient time, inconvenience, discomfortEthical considerations – waste of patient time, inconvenience, discomfort

• Essential to make assessment of optimal sample size before starting Essential to make assessment of optimal sample size before starting investigationinvestigation

Issues LRECs Need to Understand No. 1Issues LRECs Need to Understand No. 1

Offering Incentives to ParticipantsOffering Incentives to Participants

Sheep Farmers and OP Study 1996:Sheep Farmers and OP Study 1996:farmers and Quarry workers contactedfarmers and Quarry workers contactedconsent forms entered into Raffle for £200 consent forms entered into Raffle for £200 not permissible under LREC / MRECnot permissible under LREC / MREC

Student Hearing Loss Study 2000:Student Hearing Loss Study 2000:student bar staff contactedstudent bar staff contactedparticipants offered 5 lottery ticketsparticipants offered 5 lottery ticketsnot permissible under Birmingham University Guidelinesnot permissible under Birmingham University Guidelines

Evidence shows incentives do not unduly bias choice of participantsEvidence shows incentives do not unduly bias choice of participants(Edwards (Edwards et al.et al. 2002) 2002)

Issues LRECs Need to Understand No. 1Issues LRECs Need to Understand No. 1

Some “unethical” practices proved to increase response ratesSome “unethical” practices proved to increase response rates

TechniqueTechnique Likelihood of responseLikelihood of response

Cash incentive Cash incentive X 2X 2 (Brown, (Brown, et al.et al. 1997, Roberts 1997, Roberts et al.et al. 2000) 2000)

Warn respondents of follow up (need linkage)Warn respondents of follow up (need linkage) X 1.4X 1.4

Drop out must be explained by the respondentDrop out must be explained by the respondent X 1.3X 1.3

Choice to opt out given to respondentsChoice to opt out given to respondents X 0.7X 0.7

(Edwards (Edwards et al.et al. 2002) 2002)

Issues LRECs Need to Understand No. 2Issues LRECs Need to Understand No. 2

Diminishing ReturnsDiminishing Returns

LRECs insisting on:LRECs insisting on:

Contacting sample and requesting consent BEFORE any questionnaires sent

Serious implications for reduction in sample sizes

Any process involving respondents’ effort will lose participants

Issues LRECs Need to Understand No. 2Issues LRECs Need to Understand No. 2

Diminishing ReturnsDiminishing Returns

ResearcherResearcher Potential SamplePotential Sample

1000 people

540 consents

540 questionnaires

Under-powered studyn = 210 210 questionnaires

Issues LRECs Need to Understand No. 3Issues LRECs Need to Understand No. 3

Knowledge of the Non-RespondersKnowledge of the Non-Responders

Postal Surveys may accrue poor response rates (e.g. 20%) from pop.Postal Surveys may accrue poor response rates (e.g. 20%) from pop.

May need to re-write to pop. to re-recruit bigger sampleMay need to re-write to pop. to re-recruit bigger sample

Inefficient to write to all pop. againInefficient to write to all pop. again

Need to re-write to non-responders and NOT respondersNeed to re-write to non-responders and NOT responders

Impossible in anonymous studies with no linkageImpossible in anonymous studies with no linkage

Can be done with confidential studiesCan be done with confidential studies

Issues LRECs Need to Understand No. 3Issues LRECs Need to Understand No. 3

Knowledge of the Non-RespondersKnowledge of the Non-Responders

SteganographySteganography

Secret communication of a message by hiding it’s Secret communication of a message by hiding it’s existenceexistence

SteganosSteganos, meaning , meaning coveredcovered. . GkGk GrapheinGraphein, meaning , meaning to write, Gkto write, Gk

If message is discovered it is easily read because of no encryptionIf message is discovered it is easily read because of no encryption

CryptographyCryptography

Secret communication of a message by hiding it’s Secret communication of a message by hiding it’s meaningmeaning

KryptosKryptos, meaning , meaning hiddenhidden. . GkGk

Message established using known protocol, to be decrypted by the receiverMessage established using known protocol, to be decrypted by the receiver

Steganography & Cryptography can be combined together if neededSteganography & Cryptography can be combined together if needed

Steganogrpahy arouses less suspicion in questionnaire respondentsSteganogrpahy arouses less suspicion in questionnaire respondents

Issues LRECs Need to Understand No. 4Issues LRECs Need to Understand No. 4

Ethical ConcernsEthical Concerns

Indications from participant of clinical concern:Indications from participant of clinical concern:

Psychiatric problems / thresholdsPsychiatric problems / thresholds

Social problemsSocial problems

Clinical / Well-being issuesClinical / Well-being issues

Need method of linkage to connect concerning data with respondentNeed method of linkage to connect concerning data with respondent

My LREC Wish-listMy LREC Wish-list

1.1. Better understanding of novel small-scale pilot studiesBetter understanding of novel small-scale pilot studies

2.2. More allowance for deceptive studiesMore allowance for deceptive studies

3.3. Greater freedom for “one-stop approach” to surveysGreater freedom for “one-stop approach” to surveys

4.4. Better understanding of multiple measurementsBetter understanding of multiple measurements

5.5. Freedom to choose between anonymity and confidentiality of participantsFreedom to choose between anonymity and confidentiality of participants

6.6. LRECs to act less like a gateway and more like advice-giversLRECs to act less like a gateway and more like advice-givers

7.7. Openness re: membership and qualificationsOpenness re: membership and qualifications

8.8. More consistency between different LRECsMore consistency between different LRECs

ConclusionsConclusions

Defer to nobody – you’re the expert!Defer to nobody – you’re the expert!

““A little humility would suit you better”A little humility would suit you better”

Spend time getting the application right first timeSpend time getting the application right first time

““Kill ‘em with kindness”Kill ‘em with kindness”

Use the following references . . .Use the following references . . .

ReferencesReferences

Council for International Organizations of Medical Sciences (CIOMS). Council for International Organizations of Medical Sciences (CIOMS). International Guidelines for Ethical Review of Epidemiological Studies International Guidelines for Ethical Review of Epidemiological Studies World Health Organisation, Geneva 1991.World Health Organisation, Geneva 1991.

Nuffield Council on Bioethics. Nuffield Council on Bioethics. Human tissue: Ethical and legal issues.Human tissue: Ethical and legal issues.Nuffield Council on Bioethics, London 1995.Nuffield Council on Bioethics, London 1995.

World Medical Association. World Medical Association. Ethical Principles for Medical Research Ethical Principles for Medical Research Involving Human Subjects. Involving Human Subjects. Declaration of Helsinki, 2002. (Washington Declaration of Helsinki, 2002. (Washington Amendment).Amendment).