ebm handbook

7/30/2019 ebm handbook

1/52

EVIDENCE-BASED MEDICINE

USydMP HANDBOOK

Compiled by:

Evidence-Based Medicine Resource GroupUniversity of Sydney Medical Program (5th Ed.)


2/52

1

Contents

Evidence-based Medicine Resource Group Objectives......................................................... 2Checklist of Evidence-based Medicine Skills......................................................................... 5Evidence-based Medicine Course Overview ......................................................................... 7

What is EBM and why do we need it? ..................................................................................................................... 7How EBM is taught in the USydMP.......................................................................................................................... 8

Assessment of EBM................................................................................................................................................11Evidence-based Medicine Glossary .................................................................................... 13

A humorous look at EBM .................................................................................................... 18Asking the right questions and literature searching ............................................................. 19Study types......................................................................................................................... 30Causality............................................................................................................................. 33Frequency and Prognosis ................................................................................................... 35Intervention studies............................................................................................................. 39Diagnostic tests .................................................................................................................. 43Meta-analysis and systematic reviews ................................................................................ 49

Additional resources ........................................................................................................... 51


3/52

2

Evidence-based Medicine Resource Group Objectives

The teaching objectives of the EBM Resource Group focus on the identification,

critical appraisal and application of evidence from clinical epidemiological research.

Students should have covered each objective at the end of the year indicated in

brackets. Students should find opportunities to revisit each objective in subsequent

EBM theme sessions and PBL discussions in years one and two, and to reinforce

and apply these objectives in the clinical years three and four.

Overall Goals of Evidence-Based Medicine

Our aim is that graduates make decisions about health problems on the basis of the

best available evidence.

As part of their role in health care, graduates should have the willingness and ability

to:

1. Define a problem in a way that can be addressed by research evidence in the

form of epidemiological data. (Year One)

2. Identify and critically appraise research evidence. (Years One and Two)

3. Integrate this evidence with the details and preferences of individual patients in

the form of a management plan and then determine how the success of the plan

will be assessed. (Years Three and Four)

4. Interpret information on the benefits and costs of health interventions. (Year

Four)

A. Critical Appraisal of the Evidence

To achieve the above aims, students will need to be able to critically appraise the

literature. This will include primary research studies, review articles, including

systematic reviews and meta-analyses (Years One and Two) and clinical practice or

management guidelines.


4/52

3

By the time of graduation, students should be able to critically appraise the literature

on the following:

interventions for treatment and prevention of health problems

diagnostic tests

screening programs

causes of health problems (aetiology)

natural history of health problems (prognosis)

prevalence and incidence of health problems

And be familiar with some introductory principles of decision analysis and economic

evaluations.

B. Skills Required For Critical Appraisal

The following skills are needed for critical appraisal:

a) Conduct computerised literature searches to identify relevant high quality

research evidence and information from Medline, other data bases (eg. Cochrane

Library) and Internet sites.

b) Identify the following study types and be able to explain their strengths and

weaknesses: randomised controlled trials, cohort studies, case-control studies

(population and hospital-based), cross-sectional analytic and other descriptive

studies.

c) Identify major sources of bias in medical research: selection bias, confounding,

measurement bias, lead time bias and length bias (studies of screening).

d) Calculate and interpret the following measures of disease frequency and effect:

prevalence, incidence, relative risk, absolute risk reduction (risk difference),

number needed to treat and odds ratio (interpret only).

e) Interpret confidence intervals and P values.

f) Explain the difference between statistical significance and clinical significance.


5/52

4

g) Understand and apply criteria for establishing causality in the study of health

problems.

h) Understand the importance of pre-test probability for the interpretation of test

results.

i) Calculate and interpret the following features of diagnostic tests: sensitivity,

specificity, likelihood ratios, post-test probability.

j) For economic evaluations, identify the study question and the viewpoint from

which costs and outcomes have been measured. Explain what is meant by

opportunity cost, the margin and discounting.

Note: By interpret we mean understand and be able to explain in your own words.

The critical appraisal enabling skills are concerned with understanding research and

using results. Only those students interested in actually doing research will need to

be able to perform tests.

C. Application of Evidence to Decision Making

By graduation students should be able to interpret the results from clinical and

population research and decide how to apply these results to individuals or groups of

people. This is introduced in years one and two as results from critically appraised

papers are considered in relation to the PBL problem of the week, and will be

reinforced in the clinical years.


6/52

5

Checklist of Evidence-based Medicine Skills

Critical appraisal of literature Comments

Interventions for treatment and prevention of health problems

Diagnostic tests

Screening programs

Causes of health problems (aetiology)

Natural history of health problems (prognosis)

Prevalence and incidence of health problems

Skill

1. Framing appropriate clinical questions

2. Literature searching

Medline

Cochrane Library Internet sites

3. Identify the following study types and explain their strengths andweaknesses

Randomised controlled trials

Cohort studies Case-control studies (population and hospital-based)

Cross-sectional (analytic and descriptive)


7/52

6

4. Calculate and interpret measures of disease frequency Incidence Prevalence

5. Calculate and interpret measures of treatment effect Relative risk

Absolute risk reduction (risk difference)

Odds ratio (interpret only)

Number needed to treat

6. Interpret confidence intervals and P-values

7. Explain the difference between statistical significance and clinicalsignificance

8. Understand and apply criteria for establishing causality in the study ofhealth problems

9. Understand the importance of pre-test probability for the interpretation oftest results

10. Calculate and interpret the following features of diagnostic tests:

Sensitivity Specificity

Likelihood ratios

Post-test probability

11. For economic evaluations, identify the study question and the viewpoint

from which costs and outcomes have been measured. Explain what ismeant by:

Opportunity cost

Margin

Discounting


8/52

7

Evidence-based Medicine Course Overview

What is EBM and why do we need it?

Evidence-based Medicine is the conscientious, explicit and judicious use of current

best evidence in making decisions about the care of individual patients.1

By best evidence we mean patient-centred clinical research into (for example) the

accuracy and precision of diagnostic tests (including the clinical examination), the power

of prognostic markers and the efficacy and safety of therapeutic and preventive

interventions.

The practice of EBM entails a process of life-long self-directed learning in which caring

for patients creates the need for clinically important information and we then

Convert these information needs into answerable questions

Track down efficiently the best evidence with which to answer them

Critically appraise that evidence for its validity (closeness to truth) and usefulness

Apply the results to patient care.

EBM is new and rapidly growing. There are many reasons why EBM has developednow. These include:

The explosion of medical information. There are over 2 million biomedical papers

published every year -- textbooks can become out-of-date before they are

available in bookstores.

Patients have much greater access to information (eg on the Internet) and

patients are becoming increasingly aware of their rights and responsibilities as

consumers of health services.

1Sackett DL et al. Evidence-based medicine: what it is and what it isnt. BMJ 1996;312:71-72.
http://opac.library.usyd.edu.au/record=b3195783~S4


9/52

8

How EBM is taught in the USydMP

Evidence-based Medicine is part of the Personal and Professional Development

Theme. It is taught in all four years of the USydMP. In Years 1 and 2, students

acquire basic skills in EBM (see EBM Objectives). In Years 3 and 4, these skillsare refined and applied in clinical practice.

In Years 1 and 2, EBM is part of the integrated learning in PBL tutorials.

Students are progressively encouraged to consider the kinds of questions

patients might ask of their health care provider in each case, to search and to

critically appraise the research literature for evidence on which to base answers

to these questions. Students are particularly referred to the Cochrane Database

of Systematic Reviews and Best Evidence. In general, EBM encourages students

to think critically about their learning and to consider the evidence base for

clinical decisions.

EBM teaching in Years 1 and 2 is supplemented by EBM theme sessions.

There are two EBM theme sessions in each block. The first session of each pair

is a large group session in which there will be a presentation of EBM concepts

and skills. The second session of each pair is a small group session in which

students practice the skills demonstrated in the previous session. The small

group sessions are facilitated by tutors with high-level EBM skills and students

are expected to spend some time in preparation before the session. Over time,

there is an increasing expectation on students to think of relevant clinical

questions, to search the literature, and bring worthy papers for appraisal and

discussion in their small group sessions.

In between the formal EBM theme sessions, skills in EBM are reinforced and

consolidated by participation in EBM online activities. There are one to three

EBM activities in each block, each of which is linked to a specific PBL case. The

activities are self-contained, computer based activities and often involve framing

searchable clinical questions followed by a computerized literature search for


10/52

9

relevant articles. Participation in the activities is prompted by PBL tutors.

EBM Theme Sessions Years 1 and 2

Block 1 Asking Questions I Large Group SessionAsking Questions II Online Tutorial

Block 2 Study Designs I Large Group SessionStudy Designs II Small Group Tutorial*

Block 3 Causality I Large Group SessionCausality II Small Group Tutorial*

Block 4 Clinical Schools

Block 5 Frequency and Prognosis I Large Group SessionFrequency and Prognosis II Small Group Tutorial*

Block 6 Intervention Studies I Large Group SessionIntervention Studies II Small Group Tutorial*

Block 7 Diagnostic Tests I Large Group SessionDiagnostic Tests II Small Group Tutorial*

Block 8 Meta-analysis and

Systematic Reviews I

Large Group Session

Meta-analysis andSystematic Reviews II

Small Group Tutorial*

Block 9 Clinical Schools

*Indicates that student preparation is required for the session.


11/52

10

In years 3 and 4, the EBM curriculum is integrated with the clinical attachments

that make up the medical program. The aim is for students to apply the results of

clinical research to the problems of individuals. Each clinical attachment has its

own integrated EBM activity tailored to its characteristics and timing. The EBM

activities are supported by clinicians with high-level EBM and content expertise.

The clinical attachments and EBM activities are summarized in the tables below.

Details of each activity are available on the GMP website.

Attachment Activity Aim AssessmentIntegrated ClinicalAttachments(ICA)

Student PEARLS To experience using the tools of EBM toanswer real clinical questions

Summative rating ofpresentation + reporton audience feedback

PsychologicalMedicine

(PM)

Journal Club To enhance wider understanding of theevidence on which the practice of

psychological medicine is based.

Participation.Required formative

assessment

Children &Adolescents Health(CA)

Integration inPBL

To incorporate high quality evidence fromguidelines and systematic reviews inmanagement discussions with patients andtheir families.

Participation.Integration in exams.

Perinatal & WomensHealth(PW)

Integration inPBL

To integrate and apply information fromdifferent sources into clinical decision-making.

ParticipationIntegration in exams.

Community Practice(Comm)

CommunityPEARLS - casecommentaries

To apply research findings from groups to theproblems of an individual.

Summative rating of1000 word report

Pre-Internship

(PRINT)

Pre-Intern

PEARLS

To apply the principles of EBM within the

constraints of routine clinical practice

Patient rating of

discussion.Required formativeassessment.

Student PEARLS is the model used in the integrated clinical attachments of

GMP3. Students prepare and give a short, evidence-based presentation

addressing a focused clinical question raised by contact with a specific patient

seen during their integrated clinical attachments. Preparation of the presentation

is facilitated by a series of3 tutorials that the students attend in the preceding 3

weeks supported by a clinician with expertise in EBM. Each PEARLS

presentation lasts 15 minutes with an additional 15 minutes for questions and

feedback.


12/52

11

A similar model is used in year 4, except that students develop their

presentations more independently. PEARLS in Psychological Medicine uses a

journal club format. PEARLS in the Community Practice Rotation and Pre-

Internship focus on explaining evidence from research to patients. EBM in

Children & Adolescents Health and Perinatal & Womens Health focuses on

integrating evidence in decision making.

Specific objectives and their corresponding clinical attachments(see appropriate column)

Rotating PracticePlacementsStudents will demonstrate their ability to ICA

Comm CA PW PM

PRINT

1. Translate real people's problems into answerableclinical questions

2. Find, appraise, and interpret pertinent clinicalresearch

3. Discuss the generalisability of research to clinical

practice

4. Discuss the applicability of research to an individual

5. Discuss evidence with patients, relatives, doctors,

the public

Dr Pt Rel All Pub Pt

6. Consider the preferences of patients, relatives,doctors & the community

Dr Pt Rel All Pub Pt

7. Integrate clinical data, evidence and preferences in

decision making8. Appraise and use systematic reviews and evidence

based guidelines

9. Cope with clinical time constraints

10. Appraise and use imperfect evidence

11. Dealing with conflicts between evidence and

prejudice

(hash - all attachments; black - that attachment)

Assessment of EBM

EBM is a component of Personal and Professional Development. It is necessary

to achieve satisfactory performance in EBM in order to achieve satisfactory

performance in PPD and thus to be able to progress from Years 2 to 3,

Years 3 to 4 and to complete Year 4.


13/52

12

In Years 1 and 2, EBM is assessed summatively in a critical appraisal exercise

which is integrated into the year 2 summative assessment. Critical appraisal

exercises are also provided in the formative assessments. In addition there will

be EBM multiple choice questions (SBAs) in the block formative assessments

and in the summative assessment. EBM questions are also included in the on-

line voluntary formative assessments.

In Year 3, the Student PEARLS presentation is assessed summatively by

EBMedicos and formatively by fellow students. In Year 4, there are discrete

required formative assessments in Psychological Medicine (PEARLS

Presentation) and the Pre-Internship (written report); and a discrete summative

assessment in Community Practice (written report). EBM assessment isintegrated in the summative assessments at the end of the rotations in Children

and Adolescent Health, and in Perinatal and Womens Health.

Please see reference:

Sackett DL, Rosenberg WM, Gray J A M Haynes R B, Richardson W S.Evidence based medicine: what it is and what it isnt. BMJ 1996;312:71-72.
http://opac.library.usyd.edu.au/record=b3195783~S4


14/52


15/52

14

Cost-Utility Analysis converts effects into personal preferences (or utilities) and

describes how much it costs for some additional quality gain (e.g. cost per

additional quality-adjusted life-year, or QALY).

Crossover Study Design is the administration of two or more experimental

therapies one after the other in a specified or random order to the same group of

patients.

Cross-Sectional Study the observation of a defined population at a single point

in time or time interval. Exposure and outcome are determined simultaneously.

Decision Analysis is the application of explicit, quantitative methods to analyse

decisions under conditions of uncertainty.

Ecological Survey is based on aggregated data for some population as it exists

at some point or points in time; to investigate the relationship of an exposure to a

known or presumed risk factor for a specified outcome.

EER Experimental Event Rate: see Event Rate.

Event Rate is the proportion of patients in a group in whom an event is

observed. Thus, if out of 100 patients, the event is observed in 27, the event rate

is 0.27. Control Event Rate (CER) and Experimental Event Rate (EER) are

used to refer to this in control and experimental groups of patients

respectively.

Evidence-Based Health Care extends the application of the principles of

Evidence-Based Medicine (see below) to all professions associated with health

care, including purchasing and management.


16/52

15

Evidence-Based Medicine is the conscientious, explicit and judicious use of

current best evidence in integrating individual clinical expertise with the best

available external clinical evidence from systematic research. See also the article

on EBM: What it is and what it isnt.

Likelihood Ratio is the likelihood of a given test result in a patient with the target

disorder compared to the likelihood of the same result in a patient without that

disorder.

Meta-analysis is an overview which uses quantitative methods to summarise the

results.

N-of-1 Trials The patient undergoes pairs of treatment periods organised so that

one period involves the use of the experimental treatment and one period

involves the use of an alternate or placebo therapy. The patients and physician

are blinded, if possible, and outcomes are monitored. Treatment periods are

replicated until the clinician and patient are convinced that the treatments are

definitely different or definitely not different.

Negative Predictive Value (-PV) is the proportion of people with a negative test

who are free of disease. See also SpPins and SnNouts.

Number Needed to Treat (NNT) is the number of patients who need to be

treated to prevent one bad outcome. It is the inverse of the ARR: NNT = 1/ARR.

Odds are a ratio of events to non-events, e.g., if the event rate for a disease is

0.2 (20%), its non-event ratio is 0.8 (80%), then its odds are 0.2./0.8 = 0.25 (see

also Odds Ratio).

Odds Ratio is the odds of an experimental patient suffering an event relative to

the odds of a control patient.


17/52

16

Overview is a systematic review and summary of the medical literature.

Positive Predictive Value (+PV) is the proportion of people with a positive test

who have disease. Also called thepost-test probability of disease after a positive

test. See also SpPins and SnNouts.

Randomised Controlled Clinical Trial a group of patients is randomised into an

experimental group and a control group. These groups are followed up for the

variables/outcomes of interest.

Relative Risk Reduction (RRR) is the percent reduction in events in the treated

group event rate (EER) compared to the control group event rate (CER):

RRR = (CER EER)/CER*100

Risk Ratio is the ratio of risk in the treated group (EER) to the risk in the control

group (CER): RR = EER/CER. RR is used in randomised trials and cohort

studies.

Sensitivity is the proportion of people with disease who have a positive test. See

also SpPins and SnNouts.

SnNout when a sign/test has a high sensitivity, a negative result rules out the

diagnosis; e.g. the sensitivity of a history of ankle swelling for diagnosing ascites

is 92 per cent, therefore if a person does not have a history of ankle swelling, it is

highly unlikely that the person has ascites.

Specificity is the proportion of people free of a disease who have a negative

test.


18/52

17

SpPin when a sign/test has a high specificity, a positive result rules in the

diagnosis; e.g. the specificity of fluid wave for diagnosing ascites is 92 per cent.

Therefore, if a person has a fluid wave, it is highly likely that the person has

ascites.

Glossary taken from: http://www.cebm.net/glossary.asp


19/52

18

A humorous look at EBM

Please see references: Down End Research Group. Polythenia gravis: the downside of evidence

based medicine. BMJ 1995:311;1666-8. Isaacs D, Fitzgerald D. Seven alternatives to evidence based medicine.

BMJ 1999;319:1618.
http://opac.library.usyd.edu.au/record=b3009800~S4http://opac.library.usyd.edu.au/record=b3610267~S4


20/52


21/52


22/52

21

Literature Searching Strategies

Important note:

When searching for articles on treatment (intervention) start at Search A,

however when searching for articles on prognosis, aetiology or diagnosis start at

Search B. This is because at the present time the Cochrane Database of

Systematic Reviews only includes systematic reviews of interventions.

Advanced search:

The new Ovid gives you a choice of doing a basic search or an advanced search.

For the purposes of EBM searching it is more efficient to use the advanced

search feature which searches using subject headings. You will find that when

you use the basic search feature you will be given a large number of

unnecessary articles that can be time consuming to go through!

Demonstration search topic:

Is Glucosamine sulphate an effective agent in the short-term treatment of

osteoarthritis?

Structure of clinical question:

Population/Disease = People with osteoarthritis

Intervention (Study factor) = Glucosamine sulphate

Comparator (Often the control) = Placebo

Outcome (Outcome factor) = e.g pain, function

N.B. The words in bold in the above PICO format are the ones which you will use

to do your search. Note that most commonly only the population and intervention

are used to search and there is no need to put in terms such as placebo or any

outcome factor. This is because if you put in too many terms it will limit your

search too much.


23/52


24/52

23

3. Enter 1st term Osteoarthritis (population/disease). Medline will map your

term to a MeSH term (Medical Subject Heading).

4. Check the explode box to the right of osteoarthritis to widen your search

then click continue.

5. On the next page, do not choose any subheadings and click continue.

6. Enter 2nd term Glucosamine Sulphate (treatment/intervention). Once

again Medline will map your term to an appropriate MeSH term.

7. Check the explode box to the right ofGlucosamine and click continue.

8. Do not choose any subheadings. Click on continue.

9. Combine searches 1 and 2 by ticking the boxes next to them and clicking

AND.

10. Click on limits and then additional limits. Scroll down to publication

types and select meta analysis.

11. Click on limit a search.

An alternative to the last two steps is to type in meta analysis.pt. This will

select all articles referenced as a meta analysis in Medline. pt stands for

publication type. You can then combine this with your other results by using

AND.

Search C

If search B did not produce useful results, move on to search C in which you

limit your search with a Medline Clinical Queries filter.

Filter terms that limit the search differ for different clinical questions i.e.

diagnosis, aetiology, therapy, or prognosis. Emphasis can also be placed on

specificity, sensitivity, or optimization.

A specific filter will retrieve most relevant articles but will probably omit a few.

A sensitive filter will retrieve mostly relevant articles but will probably include

some less relevant ones as well.


25/52


26/52

25

Step by step instructions.

1. Steps 1-9 as above.

2. Click on limits, choose the EBM Reviews limit and click search.

PubMed Search

This search is an easy alternative to Search C, and is useful if you are off

campus and dont have access to the library databases. PubMed is the US

National Library of Medicines version of Medline and is available free of

charge worldwide.

Search PubMed Clinical Queries using keywords. The software automatically

combines search terms with AND. You do not need to type Boolean

connectors unless you wish to use the OR connector.

Step by step instructions:

1. Select PubMed from the Databases list or go to

http://www.ncbi.nlm.nih.gov/entrez

2. Click on Clinical Queries (on the blue sidebar).

3. UnderSearch byClinical Study Category select Therapy and narrow,

specific.

4. Enter keywords osteoarthritis glucosamine sulphate.

5. Click on Go.

The two tables over the page show you the search strategies that OVID Medline

uses in its clinical queries limits.


27/52


28/52

27

Question type Best study type Optimised search Most specific search More sensitive search

Aetiology

Cohort study

Case controlstudy

risk.mp

ORmortality.mp

ORcohort.tw

relative risk$.tw

OR

risks.tw

ORcohort stud$.mp

risk$.mp

ORexp cohort studies

ORbetween group$.tw

PrognosisCohort study

Prognosis/ORdiagnosed.twORCohort$.mpORpredictor$.twORdeath.twORexp models, statistical

prognos$.tw

OR

first episode.tw

OR

cohort.tw

Incidence/ORexp mortalityORfollow-up studies/ORMortality/ORPrognos$.twORpredict$.twORCourse$.tw

Filter terms on this table are taken from OVID Medline Clinical Queries and these are based on the work of R. Brian Haynes MD,PhD et al. of the Health Information Research Unit (HIRU) at McMaster University.http://www.ovid.com/site/products/ovidguide/haynes.htm


29/52

28

Bibliography

Haynes RB. Wilczynski NL. Optimal search strategies for retrieving scientifically

strong studies of diagnosis from Medline: analytical survey.BMJ2004:

328(7447):1040.Available from: http://www.bmj.com/cgi/content/full/328/7447/1040

Haynes RB. McKibbon KA. for the Hedges Team. Optimal search strategies for

retrieving scientifically strong studies of treatment from Medline: analytical survey.

BMJ2005: 330(7501):1179.Available from:

http://www.bmj.com/cgi/content/full/330/7501/1179

Wilczynski NL. Haynes RB for the Hedges Team. Developing optimal searchstrategies for detecting clinically sound prognostic studies in MEDLINE: an analytic

survey. BMC Med2004; (1):23. Available from:

http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=441418.

Wilczynski NL, Haynes RB for the Hedges Team. Developing optimal search

strategies for detecting clinically sound causation studies in MEDLINE. Proc AMIA

Annu Symp 2003:719-23.

Extra InformationMedline classes:

For students who do not feel confident searching Medline the Medical Library

offers hands-on tutorials. For more information and to register go to

http://www.library.usyd.edu.au/subjects/medicine/medIlitregistration.html

Online Tutorials:

Medline (Ovid version) tutorial:

http://www.library.usyd.edu.au/subjects/medicine/tutorials/medlinetut/index.html

EBM Literature Searching Guide:

http://www.library.usyd.edu.au/subjects/medicine/tutorials/ebmtut/index.html


30/52

29

Free on the Web:

PubMed

http://www.ncbi.nlm.nih.gov/entrez

Cochrane EBM Reviews databases

http://www.thecochranelibrary.com/

Link to other useful EBM sites:

http://www.library.usyd.edu.au/subjects/medicine/links/ebp.html

Centre for Evidence-Based Medicine

http://www.cebm.net/searching.asp

Netting the evidence (provides links to many useful websites)

http://www.shef.ac.uk/scharr/ir/netting/

PICO Tutorial

http://www.cwml-tutorials.blogspot.com/2008/01/formulating-your-questions-

using-pico.html

Many thanks to Dianne van Sommers, Jeremy Cullis and Kathy Thornton for their contribution to thischapter.

Please see references:

Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinicalquestion: a key to evidence-based decisions. [editorial] ACP Journal Club1995;123:A-12.

Oxman AD, Sackett DL, Guyatt GH. Users guides to the medical literature. I.How to get started. JAMA 1993;270:2093-2095.


31/52


32/52


33/52


34/52

33

Causality

Objectives

i. Understand the difference between a cohort and a case-control study

ii. Interpret odds ratios

iii. Understand and apply criteria for establishing causality in the study of health

problems

Odds ratio

An odds ratio is the odds of a person with the disease being exposed divided by the

odds of a control (person without the disease) being exposed. The odds ratio is

usually a good approximation of the relative risk.

ie OR=RR

Interpreting odds ratios and relative risks

For questions of harm where the outcome is disease or death, an RR>1 means that

the risk factor is harmful and an RR


35/52


36/52

35

Frequency and Prognosis

Objectives

i. Interpret confidence intervals and P values

ii. Understand the difference between statistical significance and clinical

significance

iii. Critically appraise cross-sectional and cohort studies

Incidence

No. of persons experiencing a new event during a time period/No. of persons at risk

at beginning of time period.

Therefore, incidence can only be calculated by following a group of people over time

(i.e. in an RCT or cohort study).

Prevalence

Total no. of persons with attribute at a given time/Total population at a given time.

This can be assessed by a cross-sectional study but to be representative the study

participants need to be selected at random (not volunteers).

Cohort studies on prognosis do not need a control group as the study follows

diseased people over time to determine their outcome.

P Value

A P value refers to the statistical significance of the results. Another way of thinking

of it is that it refers to the probability that the observed results are due to chance

rather than an actual effect.

The role of chance in the results is assessed by what is known as the null

hypothesis. This is the hypothesis that there is no difference between the two

groups. The statistical test looks at the probability of getting the observed result (or

one even more extreme) by chance, if the truth is that the two groups are actually


37/52


38/52

37

How are 95% CIs and P values related?

If a confidence interval includes no difference then it means that the null hypothesis

(i.e. that there is no real difference between groups) is possible. As the P value is a

measure of the null hypothesis being disproved, this would mean that the results

would not be statistically significant when the CI includes no difference.

NB: When looking at results such as a relative risk which is a ratio, an RR of 1 is

equivalent to no difference between the two groups and therefore if the CI crosses 1

the results will not be statistically significant. However, if the results do not refer to a

ratio but subtraction (e.g. the difference in mean weight loss between 2 groups), then

a value of 0 would indicate no difference and hence a CI crossing 0 would not be

statistically significant.

What P values and CIs dont tell you

These two statistical measures do not tell you how believable the result is. If the

results come from a study that is heavily biased then even statistically significant

results could not be relied upon. This is why it is important to assess a study

according to the JAMA validity criteria before looking at the results.

Statistical significance also does not tell you anything about whether the effect is

important i.e. whether the results are clinically significant. For example, it is possible

that a study looking at a new antihypertensive medication discovers a statistically

significant effect but that the difference in mean BP between the treatment and

control group is only 0.5mmHg which obviously is not clinically important.

There is a further explanation of confidence intervals in the attached paper Primer

on 95% Confidence Intervals.


39/52


40/52

39

Intervention studies

Objectives

i. Calculate and interpret the following measures of disease frequency and effect:

prevalence, incidence, relative risk, absolute risk reduction (risk difference),

number needed to treat (NTT).

ii. Discuss potential sources of bias in randomised controlled trials

Allocation Concealment

Allocation concealment is a way of protecting the randomisation process from bias. If

allocation is concealed it means that the people involved in the study (investigators,

clinicians, patients) cannot identify which group the patient will be allocated to (i.e.

treatment or control) during randomisation and can therefore in no way influence the

group the patient will be put into. For example, a study might decide to allocate

patients to groups using non-opaque envelopes. Clinicians may hold the envelope

up to the light, see what the allocation is, decide that the allocation is not the best for

the patient (e.g. they are very sick so should get the treatment not placebo) and then

pick the next envelope. Obviously if this were to happen then the study would not be

truly randomised. Allocation concealment differs from blinding in that blinding refers

to lack of knowledge about which group patients are in after the randomisation

process.

Analysis by intention to treat

Analysing patient outcomes based on the group into which they were randomized

regardless of whether they actually received the planned intervention. This analysis

ensures randomization is preserved.

In the diagram below, the 8 patients lost to follow up (5 in the treatment group, 3 in

the control group) cannot be included in the analysis as they will have no available

data. The 95 left in the treatment group should all be analysed in the treatment group


41/52


42/52

41

Calculation of estimates of effect

Problem 1. Problem 2.

WHEN RELATIVE RISK1

Of the 750 people randomized to receive 17% of people exposed developed the

treatment, 350 people developed the outcome. outcome whereas only 5% of peopleOf the 800 people randomized to no treatment who had not been exposed developed the500 developed the outcome. outcome.

Experimental event rate =EER (incidence in exposed)=350/750=0.47

ie 47% of treated people experienced the outcomeMay be expressed as 47 events per 100

Experimental event rate =EER (incidence in exposed)=17%=0.17

ie 17%=17/100ie 17% of exposed people experienced the outcome (given)May be expresses as 17 events per 100 etc

Control event rate =CER (incidence in non-exposed)

=baseline risk=500/800=0.625

ie 63% of controls experienced the outcomeMay be expressed as 625 events per 1000 etc

Control event rate =CER (incidence in non-exposed)

=baseline risk=5%=0.05

ie 5% of controls experienced the outcome (given )May be expressed as 5 events per 100 etc

Relative Risk=RRRR =EER/CERRR =(350/750)/(500/800)

=0.467/0.625=0.75

From the study it is estimated that people who undergotreatment have 0.75 times the risk of the outcome compared tothose who did not receive the treatment. (ie treatment ISPROTECTIVE)OrIt is estimated that people who undergo the treatment havethree quarters of the risk of the outcome compared to thosewho did not receive the treatment.

OrIt is estimated that people who undergo the treatment have75% of the risk of the outcome compared to those who did notreceive the treatment

Relative Risk=RRRR =EER/CERRR =0.17/0.05

=3.33

From the study it is estimated that people who were exposedhave 3.33 times the risk of the outcome compared to thosenot exposed.OrIt is estimated that people who have been exposed havethree and a third times the risk of the outcome compared tothose not exposed.OrIt is estimated that people who were exposed have 333% of

the risk of the outcome compared to those not exposed.

Relative Risk Reduction=RRR (expressed as apercentage)RRR =(1-RR)x100

=(1-0.75)x100=0.25x100=25%

It is estimated that people who undergo the treatment have a25% relative reduction in the risk of the outcome compared tothose who did not receive the treatment.

Do not calculate RRR when RR>1 ie it is a relative riskincrease

Absolute Risk Reduction=ARR (or RD)May be expressed as a risk reduction or an event rate

reductionARR =CER-EER

=500/800-350/750=0.625-0.467=0.16

=16 events per 100

If a person undergoes treatment he reduces his absolute riskby 16 events per 100Treatment prevents 16 events per 100

When RR>1 better to change the name from Absoluterisk reduction to Risk Difference (RD)

RD =CER-EER=0.05-0.17= - 0.12

Take the absolute value and interpret it with respect to yourresults.

=12 events per 100

If a person is exposed she INCREASES her absolute risk by12 events per 100Exposure results in a risk difference of 12 events per 100

Number Needed to Treat=NNTNNT =1/ARR

=1/0.16

=6.37 (6.3) people need to be treated to prevent one outcome.


43/52

42


Guyatt GH, Sackett DL, Cook DJ. Users guides to the medical literature. II.How to use an article about therapy or prevention. A. Are the results of thestudy valid? JAMA 1993;270:2598-2601.

Guyatt GH, Sackett DL, Cook DJ. Users guides to the medical literature. II.

How to use an article about therapy or prevention. B. What were the resultsand will they help me in caring for my patients. JAMA 1994;271:59-63.


44/52

43

Diagnostic tests

Objectives

i. Understand the importance of pre-test probability for the interpretation of test

results

ii. Calculate and interpret the following features of diagnostic tests:sensitivity,

specificity, likelihood ratios, pre-test probability, post-test probability.

Gold Standard

Disease

Yes No TOTAL

Diagnostic Test Positive True positive (TP) False positive (FP) TP + FP

Negative False negative (FN) True negative (TN) FN + TN

TOTAL TP + FN FP + TN

Sensitivity and Specificity of tests

Sensitivity is the proportion of all truly diseased persons (TP + FN) who test positiveusing the diagnostic test.

i.e. SN = TP/(TP+FN)

Specificity is the proportion of all truly non-diseased persons (TN + FP) who test

negative with diagnostic test.

i.e. SP = TN/(TN+FP)

Numerical example:

Gold Standard

Disease

Yes No TOTAL

Diagnostic Test Positive 90 20 110

Negative 10 80 90

TOTAL 100 100 200


45/52

44

SN = TP / (TP + FP)

= 90 / (90 + 10)

= 90%

i.e. of those with the disease 90% will test positive (and 10% will test negative)

SP = TN / (TN + FP)

= 80 / (80 + 20)

= 80%

i.e. of those without the disease 80% will test negative (and 20% will test positive)

Predictive values

Predictive values tell you the probability of the test result being correct.

Positive Predictive value=PPV

If you have a patient with a positive test result, this will tell you the probability of that

patient actually having the disease.

As PPV refers to the proportion of people testing positive that are true positives, it is

calculated by:

PPV = TP/(TP+FP)

Negative Predictive Value=NPV

If you have a patient with a negative test result, this will tell you the probability of that

patient actually not having the disease.

As NPV refers to the proportion of people testing negative that are true negatives, it

is calculated by:

NPV = TN/(TN+FN)

Numerical example:

If we look at the previous numerical example for sensitivity and specificity:

PPV = 90 / (90 + 20)

= 82%


46/52

45

i.e. of those with a positive test result 82% will have the disease

NPV = 80 / (80 + 10)

= 89%

i.e. of those with a negative test result 89% will not have the disease

Notice that in the above example the prevalence of disease is 50% (as 100 people

have the disease and 100 people do not have the disease). If we change the

prevalence of disease to 10%, we would get the following table (note that the

sensitivity and specificity of the test do not change).

Gold Standard

Disease

Yes No TOTAL

Diagnostic Test Positive 9 20 29

Negative 1 80 81

TOTAL 10 100 110

This would mean that the predictive values would now be:PPV = 9 / (9 + 20)

= 31%

NPV = 80 / (80 + 1)

= 98%

Notice how affected predictive values are by the prevalence of disease. So for

example, the probability of a test result being correct might be very different when

you conduct the test in a situation with low prevalence (e.g. screening test at a

shopping centre) compared to one with a high prevalence (e.g. specialist clinic).


47/52


48/52


49/52

48


Jaeschke R, Guyatt G, Sackett DL. Users guides to the medical literature. III.How to use an article about a diagnostic test. A. Are the results of the studyvalid? JAMA 1994;271:389-391.

Jaeschke R, Guyatt G, Sackett DL. Users guides to the medical literature. III.How to use an article about a diagnostic test. B. What are the results and willthey help me in caring for my patients? JAMA 1994;271:703-707.


50/52

49

Meta-analysis and systematic reviews

In all cases systematic reviews of the stated best study type would be better than

any individual study.

Meta-analysis

Meta analysis refers to the quantitative methods for combining the results of several

studies into a single pooled or summary statistic.

Systematic reviewSystematic review is a critical evaluation of research that attempts to address a focused

clinical question using methods designed to reduce the likelihood of bias. It may include a

meta analysis (summary statistic).

Narrative review

Narrative Reviewis a good place to start when learning about a topic however it is subject to

substantial biases and limitations so is insufficient for clinical decision making.

Publication bias

Publication biasrefers to the preferential publication ofstudies with a statistically

significant positive result.

Duplication bias

Duplication biasrefers to the inclusion of the same study more than once in the

systematic review.


51/52


52/52

Additional resources

Some useful Internet sites

Cochrane Library: http://www.thecochranelibrary.com

PubMed: http://www.ncbi.nlm.nih.gov/entrez

Oxford Centre for Evidence Based Medicine: http://www.cebm.net/

Netting the Evidence:

http://www.shef.ac.uk/scharr/ir/netting/

Users guides to evidence based practice:http://www.cche.net/usersguides/main.asp

Sydney University Library links to useful EBM sites:http://www.library.usyd.edu.au/subjects/medicine/links/ebp.html

Duke University Introduction to EBM:

http://www.hsl.unc.edu/services/tutorials/ebm/welcome.htm

University of Colorado: A students guide to the medical literature:http://grinch.uchsc.edu/sg/ Some useful references

Evidence-based medicine: how to practice and teach EBM. 3rd edSharon E Strauss, W. Scott Richardson, Paul Glasziou and R Brian Haynes.Churchill Livingstone, 2005.

Evidence-based healthcare.

J.A. Muir Gray.Churchill Livingstone, 2001.

Clinical Epidemiology; a basic science for clinical medicine. 2nd ed

David Sackett et al.Little, Brown, 1991.

A Dictionary of Epidemiology. 5th ed.John Last.Oxford University Press, 2008

Clinical Epidemiology: the essentials. 4th ed.Robert Fletcher and Suzanne Fletcher.Williams and Wilkins, 2005
http://opac.library.usyd.edu.au/record=b2761934~S4http://opac.library.usyd.edu.au/record=b2860849~S4http://opac.library.usyd.edu.au/record=b3557114~S4http://opac.library.usyd.edu.au/record=b1695307~S4http://opac.library.usyd.edu.au/record=b2744845~S4http://opac.library.usyd.edu.au/record=b2863738~S4http://opac.library.usyd.edu.au/record=b2761934~S4

ebm handbook

Documents