An introduction to Evidence-Based Medicine (critical thinking in medicine) Akbar Soltani. MD,MSc Tehran University of Medical Sciences (TUMS) Shariati Hospital www.soltaniebm.com www.ebm.ir www.avicennact.ir
Jan 03, 2016
An introduction to Evidence-Based
Medicine(critical thinking in medicine)Akbar Soltani. MD,MSc
Tehran University of Medical Sciences (TUMS)Shariati Hospital
www.soltaniebm.comwww.ebm.ir
www.avicennact.ir
Educational Activities
• Whole spectrum of the medical profession
• From 2000 to 2006 we had more than 200 lectures in EBM, MDM, Methodology,
• From 2006 to 2007 we had more than 50 lectures in CT
• More than 7000 slides have been prepared
• 10 books have been compiled • www.soltaniebm.com or www.ebm.ir and www.avicennact.ir
Some assumptions
• You, the audience, between you know much more than I do about this
• Lao Tzu said: “Those who know do not speak/Those who speak do not know.”
• Kafka: What a silence had been established in the world if every person talk correlated with his/her knowledge
Workshop objectives
• Problems of conventional medicine• Definition and philosophy of EBM/IM• Different concepts such as
– answerable question, systematic review, NNT,NNH,…
• Search methods • Most popular EBM data bases• Critical appraisal skills
What is Critical Thinking?
• What is the best way of walking?
• What is the best way of thinking?
Agenda
• Definitions: Definitions: Science and EBMScience and EBM
• Dimensions of the problemDimensions of the problem1. Information management (mastery)
2.2. Limitations of current clinical practiceLimitations of current clinical practice
3.3. Heuristic and errorsHeuristic and errors
4. Problems of communication
• EBM
Definition:• Science is devoted to formulating and
testing naturalistic explanations for natural phenomena. It is a process for systematically collecting and recording data about the physical world, then categorizing and studying the collected data in an effort to infer the principles of nature that best explain the observed phenomena.72 Nobel laureates. (From the Amicus Curiae presented in the US Supreme
Court Case of Edwards vs Agullard, 1986)
Philosophers in science:
Trace back to the development of EBM.
• 1972:Archie Cochrane told about the role of randomized control trial in scientific medicine.
• 1980's: Dave Sackett
• 1990s :The term was generated by Gordon Guyatt from McMaster University
Trace back to the development of EBM
What evidence-based medicine is
• “The conscientious (careful), explicit (clear,
unambiguous) and judicious (sensible) , use of current best evidence in making clinical decisions about the care of individual patients.”
Sackett et al, 2000
What evidence-based medicine is:
The practice of EBM is the integration of
• Individual clinical expertise
with the
• Best available external clinical evidence from systematic research.
and
• Patient’s values and expectations
Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS: Evidence based medicine:
what it is and what it isn’t. BMJ 1996;312:71-2.
I.Individual Clinical Expertise:
• Experience: Relates to what we’ve done and to knowledge.
• “An expert is a person who has made all the mistakes that can be made in a very narrow field” (Niels Bohr)
1. Clinical skills2. Clinical judgment3. Vital for determining whether the evidence applies
to the individual patient at all and, if so, how
Hypothesis generating
Confirms or denies hypothesis
Matches research result to specific patients
Clinical judgment
Outcomes research
Patient seen in practice
?
Expertise for
Diagnosis,Procedures
Interventions (diagnostic or
therapeutic)
Helps clinicians
Need accurate/precise
information
Do not need accurate/precise
information
Not adequate Adequate
II. Best External Evidence:
• From real clinical research amongintact patients.
• Has a short doubling-time (10 years).
• Replaces currently accepted diagnostic tests and treatments with new ones that are more powerful, more accurate, more efficacious, and safer.
III. Patients’ Values & Expectations
• Have always played a central role in determining whether and which interventions take place
A model for evidence-based clinical A model for evidence-based clinical decisionsdecisionsSackett et al, 2000
Bayesian approach: background knowledge + evidence= decision making
Current bestCurrent best evidenceevidence
Model of Evidence-Based Medicine
Evidence
Patient’s Preferences
Clinical Expertise
Clinical Setting
Conventional medicine
Expertise(intuition…)
Pathophysiology, references, tradition…
Patient value
Agenda
• Definitions: Definitions: Science and EBMScience and EBM
• Dimensions of the problemDimensions of the problem1. Information management (mastery)
2.2. Limitations of current clinical practiceLimitations of current clinical practice
3.3. Heuristic and errorsHeuristic and errors
4. Problems of communication
• EBM
• The database of the National Library of Medicine MEDLINE has approximately 6 million references from 4.000 journals with about 400.000 new entries added each year.
Why Is It So Hard to BeWhy Is It So Hard to BeUp-to-date?Up-to-date?
• Doubling time of biomedical Doubling time of biomedical science is about 20 months science is about 20 months in in 20012001
Increasing Knowledge
0
2000
4000
6000
8000
1966 1976 1986 1996
Number of articles on Hypertension cited in Medline by Year
Articles
How many original articles should a
specialist read each week to remain
up to date in his/her own field only ?
5 10 20 40 100
Dr.S.Naserimoghaddam
How many original articles should a specialist read each week to remain up to date in his/her own field only ?
5 10 20 40 100The story is different for a generalist: 17 /day!
Dr.S.Naserimoghaddam
RCT23 Patients
1960
1 20.5
Odds Ratio
Treatment Control
Thrombolytic Therapy & MI mortality
Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of results of meta-analyses of randomised control trials and recommendations of clinical experts. JAMA 1992;268:240-8
CumulativeYear RCTs Pts
1960
1965
1 20.5
1 232 653 1494 3167 1793
Odds Ratio
Treatment Control
Antman JAMA 92
Thrombolytic Therapy & MI mortality
CumulativeYear RCTs Pts
1960
1965
1970
1975
1980
1985
1990
1 20.5
1 232 653 1494 3167 179310 254411 265115 331117 392922 545223 576727 612533 657165 4718570 48154
Odds Ratio
Treatment Control
p < 0.01
p < 0.001
p < 0.00001
Antman JAMA 92
Thrombolytic Therapy & MI mortality
Thrombolytic Therapy & MI mortalityCumulative
Year RCTs
Pts1960
1965
1970
1975
1980
1985
1990
1 20.5
1 232 653 1494 3167 179310 254411 265115 331117 392922 545223 576727 612533 657165 4718570 48154
Odds Ratio
Treatment Control
p < 0.01
p < 0.001
p < 0.00001
Textbook Recommendations
Rout Specif Exp NOT
21 510 2 8 7 812 4 3 1 1
112
872
111281
515 6
Antman JAMA 92
Antman JAMA 92
Some parts of textbooks are out-of-date
• Fail to recommend Rx up to ten years after it’s been shown to be efficacious.
• Continue to recommend therapy up to ten years after it’s been shown to be useless.
• Different textbooks, different recommendations.
• Textbooks are fact or opinion?• Textbooks are appraisable?
Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of results of meta-analyses of randomised control trials and recommendations of clinical experts. JAMA 1992;268:240-8
The Prognosis of Ignorance is Poor
Worse with “duration in practice”
Interesting ExampleInteresting Example
Dr Naserimoghaddam
182 Health authorities selected
2 Articles: 1 on cardiac rehabilitation 1 on breast Ca screening Results of each presented in 4 ways:
RRR (Relative Risk Reduction) ARR (Absolute Risk Reduction) PEFP (Proportion of Event Free Patients) NNT ( Number Needed to Treat)
Dr NaserimoghaddamEvidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
They were told that these were the results of 4 articles on each topic
Question: According to which set of data you may choose to adopt the method as part of your regional practice policy?
Dr NaserimoghaddamEvidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
Interesting Results !Interesting Results !
N=140N=140 MammographyMammographyCardiac Cardiac RehabilitationRehabilitation
RRRRRR 79%79% 76%76%
ARRARR 38%38% 56%56%
PEFPPEFP 38%38% 53%53%
NNTNNT 51%51% 62%62%
Dr NaserimoghaddamEvidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
Only 3 noted that all 4 sets of data are the same!
None were clinicians!
Dr NaserimoghaddamEvidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
Hypothesis?
Sample size estimation
None!
Failure to detect a difference
=Equivalence?
Assume non-inferiority if the lower limit of 95% CI is
less than –5%,N=904 per group!
Percent correct answers for knowledge questions
0
10
20
30
40
50
60
70
80
90
100
Sen90
Rule
In/O
ut
LR
NN
T
Hip
RCT
Agenda
• Definitions Definitions
• Dimensions of the problemDimensions of the problem1. Information management (mastery)
2.2. Limitations of current clinical practiceLimitations of current clinical practice
3.3. Heuristic and errorsHeuristic and errors
4. Problems of communication
• EBM
• A pervasive problem for primary care physicians attempting to appraise clinical information is the conflicting recommendations by experts.
Global judgment by expertsGlobal judgment by experts
Bloor M. Bishop Berkeley and the adenotonsillectomy enigma: anexploration of the social construction of medical disposals. Sociology1976; 10: 43–61. EVIDENCE-BASED PRACTICE 2000
Variation in current practiceVariation in current practice
Pathophysiologic approachPathophysiologic approach
• Resident: Do you recommend HRT fore Resident: Do you recommend HRT fore high LDL in postmenopausal patients? high LDL in postmenopausal patients?
• Attending: Attending: YES because estrogen increase HDL and decrease LDL, Lpa,and ………..
Evidence Based Fallacy
Answering question
logic, mathematics, philosophy,
social science?
Empirical scienceMedicine…
Analyticthinking
Synthetic:Re/search,reading…
WHI: Coronary Heart Disease
years 1 2 3 4 5 6
Does CME Work?
– Traditional CME in a nice place with pleasant after lecture diversions is, unfortunately, completely ineffective in changing our behavior.
Davis DA, Thompson MA, Oxman AD, Haynes RB: Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA 1995;274:700-5.
The Slippery Slope
years since graduation
r = -0.54p<0.001
...
...
. ... . .... .
....
....
.....
...knowledgeof current best care
100%
0%
50%
Choudhry, Fletcher and Soumerai, Ann Intern Med 2005;142:260-73-94% of 62 studies found decreasing competence for at least some tasks, with increasing physician age.
Agenda
• Definitions: Definitions: Science and EBMScience and EBM
• Dimensions of the problemDimensions of the problem1. Information management (mastery)
2.2. Limitations of current clinical practiceLimitations of current clinical practice
3.3. Heuristic and errorsHeuristic and errors
4. Problems of communication
• EBM
A quick assessment
Consider the following list of words:
goiter ,weight loss, sweating, hair loss, proptosis, lid lag, dyspnea, wide pulse pressure, weakness, hyperphagia, staring, diarrhea ,anxiety
Write down as many as you can remember…
A quick assessment:
• Did you include tremor or palpitation in the list of words you thought you heard?
• Results : based on nonrandom sampling (N=600), error proportion was 20% (unpublished!)
• What is the validity of the estimation of frequency (or other measures) of the clinical findings?
Heuristical errors
• Heuristic = rule of thumb; mental process used to learn, recall, or understand knowledge
• Some examples:– Recency– Rarity– “burned” by missing a case– Regression towards the mean– …
(Tversky& Kahneman, 1974)
Agenda
• Definitions Definitions
• Dimension of problemsDimension of problems1. Information management (mastery)
2.2. Limitations of current clinical practiceLimitations of current clinical practice
3.3. Heuristic and errorsHeuristic and errors
4. Problems of communication
• EBM
Probability estimates of various qualitative verbal expressions
Probable
Likely
Possible
Certain
Low probability
Suggests
High probability
Unlikely
Moderate probability
Pathognomonic
classic
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0Evidence-Based Medicine: A Framework for Clinical Practice by Friedland et. al
West Vs East – Language
American ways-A guide for foreignersEloquent=expressive
Why Evidence-based Medicine?
• Science is a process for systematically collecting and recording data
• Time not available to find and assimilate evidence into practice
• Doubling time of biomedical science is about oubling time of biomedical science is about 20 months in 200120 months in 2001– Medical Journals: too voluminous– Scientific chaos
Dr Naserimoghadam
Why Evidence-based Medicine?• Traditional sources of info:
– Textbooks :partially (10-30%) outdated before publication
– Experts:• Pathophysiologic approach
• Conflicting recommendations
• Biased towards their own works & knowledge
• Heuristic and errors• Problems of communication
– CME: ineffective
• Clinical judgment / diagnostic skills increase with time, but up-to-date clinical knowledge declines
Dr Naserimoghadam
End of part one
An introduction to Evidence-Based
Medicine(critical thinking in medicine)Akbar Soltani. MD,MSc
Tehran University of Medical Sciences (TUMS)Shariati Hospital
www.soltaniebm.comwww.ebm.ir
www.avicennact.ir
Part-2
Agenda
• Definitions: Definitions: Science and EBMScience and EBM
• Dimensions of the problemDimensions of the problem1. Information management (mastery)
2.2. Limitations of current clinical practiceLimitations of current clinical practice
3.3. Heuristic and errorsHeuristic and errors
4. Problems of communication
• EBM
Evidence Based
Medicine
EBM process
1. Ask
2. Search
3. Appraise
4. Apply
5. EvaluateEvidence-Based Medicine: How to Practice and Teach EBM by David Sackett
Ask
• We need it twice for every 3 outpatients and 2 times for every inpatient
• Questions are most likely to be about treatment
• Most of the questions generated in consultations go unanswered.
P: Among patients with NIDDM who are having MI
I: does tight control of their blood sugar
C: in comparison to conventional methods
O: reduce their risk of dying?"
EBM process
EBM process
1. Ask
2. Search
3. Appraise
4. Apply
5. Evaluate
EBM process
2Efficient track-down of the best evidence –Secondary (pre-appraised) sources e.g.,– Cochrane (systematic reviews)
– E-B Journals
–primary literature
Example of a search strategy
#1 RANDOMIZED-CONTROLLED-TRIAL in PT#2 CONTROLLED-CLINICAL-TRIAL in PT#3 RANDOMIZED-CONTROLLED-TRIALS#4 RANDOM-ALLOCATION#5 DOUBLE-BLIND-METHOD#6 SINGLE-BLIND-METHOD#7 #1 or #2 or #3 or #4 or #5 or #6#8 TG=ANIMAL not (TG=HUMAN andTG=ANIMAL)#9 #7 not #8#10 CLINICAL-TRIAL in PT#11 explode CLINICAL-TRIALS#12 (clin* near trial*) in TI#13 (clin* near trial*) in AB#14 (singl* or doubl* or trebl* or tripl*) near (blind*or mask*)#15 (#14 in TI) or (#14 in AB)#16 PLACEBOS#17 placebo* in TI#18 placebo* in AB#19 random* in TI#20 random* in AB
#21 RESEARCH-DESIGN#22 #10 or #11 or #12 or #13 or #15 or #16 or #17 or#18 or #19 or #20 or #21#23 TG=ANIMAL not (TG=HUMAN andTG=ANIMAL)#24 #22 not #23#25 #24 not #9#26 TG=COMPARATIVE-STUDY#27 explode EVALUATION-STUDIES#28 FOLLOW-UP-STUDIES#29 PROSPECTIVE-STUDIES#30 control* or prospectiv* or volunteer*#31 (#30 in TI) or (#30 in AB)#32 #26 or #27 or #28 or #29 or #31
#33 TG=ANIMAL not (TG=HUMAN andTG=ANIMAL)#34 #32 not #33#35 #34 not (#9 or #25)#36 #9 or #25 or #35
Validity: Find the Best Valid Evidence First
Cochrane Library
Specialty-specificPOEMs
Best Evidence
Clinical EvidenceClinical Inquiries
Textbooks, Up-to-Date, 5-Minute Clinical Consult
Use
fuln
ess
Medline
search for the best (prevalidated) database information first.
EBM process
1. Ask
2. Search
3. Appraise
4. Apply
5. Evaluate
EBM process
3-Critical appraisal of the evidence for its validity and clinical applicability
Level of evidence for treatmentLevel of evidence for treatment
A
B
C
D
Why do I have to bother?Can’t I trust the editors?
Percent of articles meeting quality criteria
NEJM 12.6
Ann Int Med 7.6
JAMA 7.2
Lancet 6.2
BMJ 4.4
Arch Int Med 2.4
EBM process
1. Ask
2. Search
3. Appraise
4. Apply
5. Evaluate
EBM process4 Integration of that critical appraisal
with clinical expertise and the patient’s unique biology and beliefs apply.
5 Evaluation: evaluating our effectiveness and efficiency in executing steps 1–4
Which doctor do you want?
William Osler, 1900 Smart young doctor
Which doctor do you want?
Wise & experienced smart young doctor
What Proportion of Healthcare is Evidence-Based ?
• BMJ Editorial: about 15%
• Archie Cochrane: less than 10%
• NIH : Diagnostic technology 20 %
Smith R: Where is the wisdom...? The poverty of medical evidence. BMJ 1991;303:798-9.
What are the Challenges
• time
• access
• skill in critical appraisal
• language of research
• sense of control over practice
• environment/culture
• applicability
Criticisms of Evidence-Based Medicine
• EBM has been or might be used by payers as an excuse to deny payment and limit clinician autonomy.
• Evidence-based treatment recommendations tend towards the nihilistic
• EBM over-values randomized blinded trials and denigrates other forms of evidence, including clinical experience.
Reference based medicine?
• First, idea second, references
• Vague questions
• systematic search is not usual
• systematic critical appraisal is not usual
• Inadequate evaluation
It’s like pseudoscience, isn’t it?
Agenda
• Definitions: Definitions: Science and EBMScience and EBM
• Dimensions of the problemDimensions of the problem1. Information management (mastery)
2.2. Limitations of current clinical practiceLimitations of current clinical practice
3.3. Heuristic and errorsHeuristic and errors
4. Problems of communication
• EBM
• Summary
Practicing EBM: New Developments
• New strategies for finding and evaluating evidence
• New tools:• Meta-analyses• Systematic reviews /Cochrane Collaboration
• Evidence-based journals of secondary publication
• Information systems bring info in seconds
Dr Naserimoghadam
ConventionalConventional medicine
experiencesPathophysiology,
references,…
Patient value
A model for evidence-based clinical A model for evidence-based clinical decisionsdecisionsSackett et al, 2000
Current bestCurrent best evidenceevidence
Evidence-Based JokeEvidence-Based Joke
A
B
C
D
Class0:Things I believe
Class0a:Things I believe despite the available data
Class1:Randomised controlled clinical trials that agree with what I believe
Class2:Other prospectively collected data that agree with what I believe
Class3:Expert opinion that agree with what I believe
Class4:Randomised controlled clinical trials that do not agree with what I believe
Class5:What you believe that I do not
Evidence-Based JokeEvidence-Based Joke
egocentrism/ narcissistic trait/personality
• Imagine your life and the lives of your friends and family placed in the hands of juries and judges who let their biases and stereotypes govern their decisions, who do not attend to the evidence, who are not interested in reasoned inquiry, who do not know how to draw an inference or evaluate one.
Critical Thinking:What It Is and Why It Counts
Peter A. Facione Dean of the College of Arts and Sciences Santa Clara University 1998
think about
What are the alternatives to EBM?
Isaacs, BMJ
References• Cook DJ, Meade MO, Fink MP: How to keep up with the
critical care literature and avoid being buried alive. Crit Care Med 24:1757-1768, 1996
• Evidence-Based Medicine: A Framework for Clinical Practice by Friedland et. al
• Evidence-Based Medicine: How to Practice and Teach EBM by David Sackett
• How to Read a Paper: The Basics of evidence based medicine by Trisha Greenhalgh
• Studying a Study and Testing a Test by Richard Riegelman and Robert Hirsch
• Smith R: Where is the wisdom...? The poverty of medical evidence. BMJ 1991;303:798-9.
References• Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of
results of meta-analyses of randomised control trials and recommendations of clinical experts. JAMA 1992;268:240-8
• Bero L, Rennie D. The Cochrane Collaboration. JAMA 1995;274:1935–8. • Villanueva EV, Burrows EA, Fennessy PA, Rajendran M, Anderson JN. Improving
question formulation for use in evidence appraisal in a tertiary care setting: a randomised controlled trial. BMC Med Inform Decis Mak. 2001;1(1):4. Epub 2001 Nov 08.
• Booth A, O'Rourke AJ, Ford NJ. Structuring the pre-search reference interview: a useful technique for handling clinical questions. Bull Med Libr Assoc. 2000 Jul;88(3):239-46
• Haynes RB. Clinical review articles. BMJ. 1992;304:330-1.• Oxman AD, Cook DJ, Guyatt GH. Users’ guides to the medical literature. VI. How
to use an overview. Evidence-Based Medicine Working Group. JAMA. 1994;272:1367-71.
• Evidence-based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA 1992;268:2420-5
• Guyatt GH, Rennie D. Users' guides to the medical literature. JAMA 1993;270:2096-7
• Oxman AD, Sackett DL, Guyatt GH. Users' guides to the medical literature. I. How to get started. JAMA 1993;270:2093-5.
• Evidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
Thank you
COMPARE TWO paradigm in iran and weast
• Intuition• Personal• Not transferable• applicable in religeon and
philosophy • Powerful• Rooye mien dar jang• Vague• Introspective/deep• Robust/negotiation is
impossible
• Fact as a sourse of maarefat
• Interpersonal• Apply to scince and
experiences• Rooye mein nemireh• Clear accurate • Factual • Probabilistic• Negotiation is possible
West Vs East - Language• In the West, logical and
semantic clarity are among the most celebrated of the ideals of Reason.
• These ideas are associated with univocal definition guaranteeing unambiguous usage.
• In this sense, the opposite of clarity is confusion – a state of unarticulated ideas or feelings.
• In classical Eastern texts, allusive (indirect) and connotatively rich language is more highly prized than clarity, precision, and argumentative rigor.
• We must attempt to avoid ‘the Fallacy of the Perfect Dictionary.’
Paradigm Shift
• Changing between original and review
• Evidence-based in clinical practice
• Learning according to levels of evidence
• Skills to make critical appraisal topics
• From Sp / Sn to NNT / LR / OR
• Resources on the internet
• Review a clinical question through RCT’s