ESIM Summer School June 20 Friday, 2014 Less is more: Guidelines Primiano Iannone, MD Head of Emergency Department Ospedali del Tigullio, Lavagna (GE) Italy
ESIM Summer School June 20 Friday, 2014
Less is more: Guidelines
Primiano Iannone, MD Head of Emergency Department Ospedali del Tigullio, Lavagna (GE) Italy
• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines
layout
2011
Farquhar CM, et al. Med J Aus 2002
How clinical guidelines are percieved
• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines
Physicians take decisions about (with) their patients
We take clinical decisions every day, several times per day. Often, these decisions are semiautomatic choices dictated by consolidated practices, previous professional experiences and knowledge background, local clinical habits and policies, with an heuristic approach typical of type 1 thinking («fast thinking») according to Kahneman. Often we consider also what our patients say and think about their illness. Sometimes we don’t.
However, we face often serious uncertainty about the quality of evidences on which to base our decisions, as well as to what extent individual patients’ conditions related to age, gender, morbidity, personal preferences and beliefs could modify the picture.
In these cases a sound, slow and complex rational approach («type 2 thinking», referring again to Kahneman’s terminology) is required.
So, we need searching, appraising and staying up-to-date with the best evidence, integrating it with our personal knowledge and experiences, as well aswith cost considerations, weighting risksand benefits carefeully, patientspreferences, with a clever clinicalreasoning
but can we do this efficiently ?
the exponential growth of randomized controlled trials
…we need leaner and moreefficient methods of staying up-to-datewith the evidence. Using current methods,the Cochrane Collaboration has not beenable to keep even half of its reviews up-to-date…
RCTs indexed on PubMED
1978-2013 355.272
1978: 1787 RCTs
2013: 16944 RCTsHeart failure [MeSH] RCTs 2010-2013: 1104
So, what do we need ?
• Raise the right questions in an answerable manner(PICO)
• Search for evidences efficiently• Appraising critically evidences and rating them• Integrating evidence with our experiences and previous
knowledge• Adapting evidences and deciding whehter it is worth
applying them to individual patients
JAMA, 1992; 268: 2420-25
Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise withthe best available external clinical evidence from systematic research.
Evidence based medicine: what it is and what it isn't
David L Sackett, William M C Rosenberg, J A Muir Gray, R Brian Haynes, W Scott RichardsonBMJ 1996;312:71-72 (13 January)
Evidence based medicine
Evidence Based Clinical Guidelines
• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines
Systematic review of literature
Studies have shown that the balance of disciplines within a guideline development group has considerable influence on theguideline recommendations
Multidisciplinary development
Guidelines based on a consensus of expert opinion or on unsystematic literature surveys have been widely criticised as not reflecting current medical knowledge and being liable to bias.
Graded recommendations
Guideline recommendations are graded to differentiate between those based on strong evidence and those based on weak evidence
Miller J, Petrie J. Development of a practice guideline. Lancet 2000; 355:82–3.
What does it mean systematic review of literature?
To minimise potential sources of bias in the guideline recommendations, the literature should be identified according to an explicit search strategy, selectedaccording to defined inclusion criteria,and assessed againstconsistent methodological standards
Graded recommendations
Certainty(Level of evidence)
Strenghtof recommendations
strong
weaklow
high
Many guidelines derive(d) level of evidence almost exclusively from study type
Moreover classification of level of evidences with letters, numbers, or symbols was chaotic
ESC/AHA
SIGN
ERC 2010 guidelines
Type of study
Quality of evidenceStrength
of recommendation
And so, no RCT, no strong recommendation?
?
Sometimes trials are unethical or impossible
Sometimes trials are unethical or impossible
yet some treatments are quite effective
DC shock for ventricular fibrillation
Insulin for diabetic coma
Blood trasfusion for haemorrhagic shock
Type of study
Quality of evidence Strenghtof recommendation
Other factors ?
1.Relevance of outcomesImportance of the outcome that treatment prevents
Deep vein thrombosis : Postflebitic syndrome vs death from PulmonaryEmbolism
Atrial Fibrillation: Palpitations Vs stroke
2. Magnitude of treatment effectthe lower the NNT (=1/ARR) , more effective the treatment is
Relative risk reduction overestimates effect of treatment
3.Risk of BiasSystematic error leading to overestimate or
underestimateof true treatment effect
Also RCTs may be affected by several biases that weaken their quality
Selection biasDetection biasAttrition bias
Reporting bias…..
4. Precision
Reliable measurement of the effect size of the treatment
95 %Confidence intervals
ASA vs Placebo for stroke prevention in Atrial Fibrillation has wider 95% CIthan in Transient Ischemic Attacks
5. inconsistency
Conflicting results across trials
6. directness
differences between studied and target populationas regard of
Interventions
Patients (applicability)
Outcomes (hard vs surrogate)
Absence of head to head comparisons
type of study
Quality of evidence Strenght of recommendation
magnitude of effectrisk of bias
precision
consistency
directness
relevance
Balance of all favorable /unfavorable outcomes
patients values & preferences
resources ?
a more complex approach is needed
According to GRADE Quality of evidence must be summarized in a table
Iannone et Al,JAMA Intern Med, 2014
a more complex approach
http://www.gradeworkinggroup.org/
There are goodguidelines
However many medicalspecialty societies
haven’t adopted GRADE yet
• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines
Grilli et al : Lancet, 2000
Low quality of early guidelines
We have also another problem
Conflict of interests
Bias almost alwaysresults in an overestimation of benefit
and an underestimation ofharm
Is not a source of a random error
COI generates BIAS
A COI is a set of conditions in which professional judgment concerning a primary interest (such as the health and well being of a patient or the validity of research), is unduly influenced by a secondary interest -The secondary interests may be financial or nonfinancial.
Thompson DF (1993) Understanding financial conflicts of interest. NEJM 329: 573–576.
Managing COI within a guideline panel is of
paramount importanceto warrant trustworthy
recommendations
Lenzer et Al, BMJ 2013
Lenzer et Al, BMJ 2013
• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines
• How to decide whether a guideline istrustworthy
• Evaluation frameworks (AGREE, GIN, IOM standards)• Concordance between guidelines
Traditional approach
2011
1. Establishing Transparency
2. Management of Conflict of Interest (COI)
3. Guideline Development Group Composition
4. Clinical Practice Guideline–Systematic Review Intersection
5. Establishing Evidence Foundations for and Rating Strength ofRecommendations
6. Articulation of Recommendations
7. External Review
8. Updating
Evaluation frameworks explore the qualityof producing and reporting guidelines
NOT the trustworthiness of their recommendations
Concordance of recommendationsbetween (among) differentguidelines
Proxy of trustiworthiness
Adherence to quality standards
Or not ?
how many guidelinesfor a disease ?
A case study
• Three renowned medical specialty societies• Three guidelines on the same disease (why ?)• Same evidence base about a given drug X• One of three guideline declares to comply with GRADE• Full disclosures of conflict of interests• Another guideline declares to comply with AGREE criteria• Substantial agreement among them about the effectiveness
of drug X
Who could doubt it?
Can dronedarone be recommended for preventing recurrencesof Atrial Fibrillation ?
• Three renowned medical specialty societies (AHA, ESC, CCS)
• Three guidelines on the same disease (why ?)• Same evidence base (6 RCTs) about dronedarone• One guideline declared to comply with GRADE• Full disclosures of conflict of interests• Another guideline declared to comply with AGREE
criteria• Substantial agreement among them about the
effectiveness of dronedarone
However applying GRADE methods to the same evidencebase considered by these three guidelines….
We didn’t find relevant favorable outcomes, we found unexplained heterogeneity of results, and we could not exclude an unfavorable effect of dronedarone on mortality, with an excess of 13 (95%CI, −15 to 61) deaths per 1000 patients treated with it
Iannone et Al, JAMA Internal Medicine, 2014
Can dronedarone be recommended for AtrialFibrillation ?
• Three renowned medical specialty societies (AHA, ESC, CCS)• Three guidelines on the same disease (why ?)• Same evidence base (6 RCTs) about dronedarone• One guideline declares to comply with GRADE• Full disclosures of conflict of interests• Another guideline declares to comply with AGREE criteria• Substantial agreement among them about the effectiveness of dronedarone
NO
in presence of
• flawed methods (no GRADE guidelines)• uncontrolled conflict of interests• restricted panel compositions
Concordance of recommendations betweenguidelines and declared adherence to qualitystandards do not warrant their trustworthiness
A roadmap I would suggest…
(a very modest & weak recommendation…)
Have You a clinical problem ?
Search whether a guideline addressing relevant outcomes does exsist
Sound methodology ?(GRADE fully exploited)
No/Negligible conflict of Interest ?
Multidisciplinary involvement ?
Low risk of untrustworthiness
PICO conceptualisation
YES
YES
IOM criteriahelpful
Evaluate primary evidences carefully in case of any doubt
YES
NO
NO
NO
NO
Search for other
evidences
YES
• Overall quality of evidences• Relevance of outcomes• Type of studies• Precision• Consistency• Directness• Risk of bias• Modifiers
• Balance across all favourable and unfavourableoutcomes
• Patients’ values and preferences• Resources’ use
Follow GRADE conceptualisation
assessing their trustworthiness, too….
• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines
If a recommendation is trustworthy, the main issue is deciding whetherIt can be applied to our patient(s)
It is a matter of clinical judgement consideringdifferences beween ideal study conditions whereevidences were produced and real life settings in terms of
• Patients• Type of intervention• Outcomes considered• absence or presence of head-to-head comparisons
There is often some uncertainty about this
If a recommendation is untrustworthy
• It should be openly and widely presented and discussed to avoid unnecessary harm to the patients and resources’ wasting
• Reasons of untrustworthiness should be clarifiedand addressed
• An in depth GRADE based, multidisciplinar, unconflicted reassessment of flawedrecommendations should be urgently carried out to produce more firm guidelines
clinical research agenda should be prioritizedto fulfill these gaps, if relevant for our patients
problem driven research vs
curiosity driven research
When guidelines highlight the absence of firm evidences
EBM helps ethical integrity of biomedical research
• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines
Clinical guidelines at their crossroad
Evolution or extinction
More trustworthy, more evidence based,unconflicted, balanced tools to inform wiseclinical decisions and manage uncertainty
Their transformation into
I didn’t mean to confuse You
But Evidence Based Medicine isan eminently creative
methodology which emphasizescritical reasoning
and not the robotic application of rules and recommendations…