Introduction to Study Designs for EBM Michelle Howard, MSc, PhD (Candidate), Assistant Professor.

Introduction to Study Designs for EBM

Michelle Howard, MSc, PhD (Candidate), Assistant Professor

Levels of Evidence

1: Systematic review of RCT or individual RCT

2: Cohort study, SR of cohort studies

3: Case-control study, SR of case-control study

4: Case-series

5: Expert opinion, physiology, bench research

Does HRT Prevent or Cause Heart Disease?

Exposed to HRT Not Exposed

RCT: subjects are assigned to intervention, do not get to choose

% no heart disease

% heart disease

Case-Control: Subjects with disease identified (case), matched to those without disease (control), exposures measured RETROSPECTIVELY

Cohort: Subjects choose intervention, exposure measured when exposed, outcomes measured PROSECTIVELY

Cross Sectional: disease measured and exposure measured at same time RETROSPECTIVELY

Main Study Types: Experimental, Observational, Qualitative

The question dictates the type of study, eg:–EXPERIMENTAL

–OBSERVATIONAL (Case Series, Cross Sectional, Cohort or case-control )

Treatment effectiveness = RCT Screening effectiveness = RCT

Harmful exposure (e.g. smoking and cancer) Etiology (e.g. gene and CF) Prognosis (e.g. microalbuminuria and coronary event)

Not all questions can be answered with RCT

Harm– “Antidepressant treatment and the risk of fatal and non-fatal self harm in first episode

depression: nested case-control study” Etiology

– “Dietary intakes of fruit, vegetables, and fiber, and risk of colorectal cancer in a prospective cohort of women (United States)”

Diagnosis– Accuracy of physical examination in the diagnosis of hypothyroidism: a cross-sectional,

double-blind study

For ALL studies ask yourself

“Are the results VALID- or is there another explanation?”

VALID = TRUE

Case-Control Study on HARMAntidepressant treatment and the risk of fatal and non-fatal self harm in first episode depression: nested case-control study

Cases are depressed patients in emergency room who have attempted self-harm. They self-report their anti-depressant use on a survey

Controls are depressed patients from family practices who have not attempted self-harm. A chart audit is done on their anti-depressant prescribing

#1 #2

Cases identified as non-fatal self harm by using relevant medical terms in the EMR

and review of the patient's free text notes.

Controls were a random sample patients with no self-harm in their records, matched to cases on sex, year of birth within one year, and duration of cohort membership

We derived the duration of prescriptions from the quantity of drug prescribed and the daily dose from the EMR

Prospective Cohort Study on EtiologyDietary intakes of fruit, vegetables, and fiber, and risk of colorectal cancer in a prospective cohort of women (United States)

Diet assessed yearly by self-reported questionnaire (e.g. “never” to “more than 6 per day”)

Outcome: Colorectal cancer determined by self-report questionnaire-cases followed up with pathology reports

Exposure: Low, Medium, High Fiber

10 Years

Results: No association between intakes of fruit, vegetables, and fiber, and colorectal cancer risk

How “valid” are the results?

What if women with higher fiber diets are more health conscious and undergo more cancer screening tests compared to women with lower fiber diets?

What if high fiber diets are more accurately reported than low fiber diets?

0

1

2

3

4

5

6

7

8

low med high

Ris

k o

f c

an

ce

r

expect

observed

Diagnostic study validity

Did all patients, regardless of screening result have the “gold standard” test?

Gold standard blinded? Test and gold standard reproducible?

Diagnostic Accuracy StudyFecal DNA versus fecal occult blood for colorectal-cancer screening in an average-risk population.

All subjects first provided a fecal sample for DNA testing and then completed three Hemoccult II cards before undergoing screening colonoscopy. All tests were conducted in a blinded fashion. Stool samples were analyzed for DNA abnormalities without knowledge of Hemoccult II or colonoscopy results; colonoscopy was performed without knowledge of the results of fecal DNA testing. Since Hemoccult II testing was conducted at the study sites, the results were potentially available to the colonoscopists. A clinical research organization (Parexel) received the results of Hemoccult II tests and colonoscopy directly from the clinical sites and received the results of fecal DNA analyses from the clinical laboratory (Exact Sciences).

Subjects were given three Hemoccult II cards and instructions regarding dietary and medication modifications to comply with current recommendationsSubjects could be evaluated only if the specimen for fecal DNA analysis was adequate, all six Hemoccult II panels had been completed, and colonoscopy was adequate.

The colonoscopist documented the extent of the colon that was visualized and the quality of the bowel preparation. Adequate colonoscopy required visualization of the cecum and a minimum of 90 percent of the mucosa. The size and location of any lesions were recorded. Biopsy and surgical resection specimens were examined histopathologically at each site; no centralized pathological review was performed

All samples analyzed for fecal DNA were processed in a single laboratory. Laboratory handling of all samples was fully automated.

Quality Criteria for RCT

Randomization Allocation concealment Double-blinding Drop-outs

Randomised controlled trial of nurse practitioner versus general practitioner care for patients requesting "same day" consultations in primary care BMJ 2000;320:1043-1048 ( 15 April )

In practices using randomisation by day, all patients consulting on a particular day saw the same type of practitioner. Practices

were supplied with a calendar of their study period with the days

allocated at random as nurse practitioner or general practitioner

days by block randomisation.

Truly Random?

Blinded?

Concealed?

Outcome measurement blinded?

After the consultation, patients completed the consultation satisfaction

questionnaire8 and answered yes or no to questions on the information

provided by the clinician during the consultation (the cause of the illness, what the patient could do to relieve symptoms, likely duration, how to reduce chances of recurrence, and what the patient should do if the problem didn't improve)

Four weeks after the initial consultations, patients' medical records were checked for reattendance or hospital admission for the same problem. The results were recorded on an `audit sheet'.

Effects of remote, retroactive intercessory prayer on outcomes in patients with bloodstream infection: randomised controlled trialBMJ 2001;323:1450-1451 ( 22-29 December )

A random number generator was used to randomise the patients into two groups. A list of the first names of the patients in the intervention group was given to a person who said a prayer for the well being and full recovery of the group as a whole. There was no sham intervention. Chart audit of days in hospital was done by auditors that did not know the patients’ groups.

Truly Random?

Blinded?

Concealed?

Threats to Validity: Bias

Definition: “Any systematic error in the design, conduct or analysis of a study that results in a mistaken estimate of an exposure’s effect on the risk of disease”

Bias is always a problem with how data is measured or analyzed

It can over- or under-estimate an exposure’s effect on disease

Examples are: selection bias, misclassification bias, recall bias, “wish bias”.

Threats to Validity: Bias

Example: In a case-control study of the risk of high cardiac output exercise on spontaneous abortion, women with abortion (cases) and those with healthy term pregnancies (controls) were asked about exercise history.

Women with abortion will seek to remember many more details than those without

The problem is SYSTEMATIC, ie. It sorts out by disease group rather than randomly

The problem is with measurement, not with a characteristic of the subject, so it is bias and not confounding.

Threats to Validity: Confounding

Definition: “An observed association between an exposure and disease that is more likely caused by a third factor that that is associated with both the exposure and disease”

Confounding is not a problem with measurement (as with bias) but is a problem with what factors are being considered

It can lead to false conclusions about whether the measured exposure is actually causing disease, or a third factor is causing disease.

Threats to Validity: Confounding

Example: In Alabama, it was noted that with increases in ice cream sales came increases in death. Does ice cream cause death?

Ice Cream Death

HEAT

The End