Standards for Developing Trustworthy Clinical Practice Guidelines Institute of Medicine January 11, 2010 Sandra Zelman Lewis, PhD Asst VP, Health & Science Policy American College of Chest Physicians
Jan 13, 2016
Standards for Developing Trustworthy
Clinical Practice Guidelines
Standards for Developing Trustworthy
Clinical Practice Guidelines
Institute of MedicineJanuary 11, 2010
Institute of MedicineJanuary 11, 2010
Sandra Zelman Lewis, PhD
Asst VP, Health & Science Policy
American College of Chest Physicians
Sandra Zelman Lewis, PhD
Asst VP, Health & Science Policy
American College of Chest Physicians
Most Important Challenge: DefinitionMost Important Challenge: DefinitionMost Important Challenge: DefinitionMost Important Challenge: Definition
1. Need for a universally accepted definition of evidence-based guidelines (EBGs)
The problem Consensus statements and “even less structured”
documents are often designated as “guidelines” Physicians are misled into thinking that these are
evidence-based and methodologically rigorous Patient care can be adversely impacted
Most Important Challenge: DefinitionMost Important Challenge: DefinitionMost Important Challenge: DefinitionMost Important Challenge: Definition
Most Important Challenge: DefinitionMost Important Challenge: DefinitionMost Important Challenge: DefinitionMost Important Challenge: Definition
Another Important Challenge: FundingAnother Important Challenge: FundingAnother Important Challenge: FundingAnother Important Challenge: Funding
2. Funding guidelines is the biggest challenge that guidelines developers face.
The problem EBGs are very expensive if done correctly Little funding available, difficult to obtain
Charge to this committeeIdentify sources for funding guidelines produced according to the established standards
When Evidence is Nonexistent or Poor QualityWhen Evidence is Nonexistent or Poor QualityWhen Evidence is Nonexistent or Poor QualityWhen Evidence is Nonexistent or Poor Quality
o What does ACCP do when the scientific evidence is absent or poor?
• Sets minimal threshold for evidence: Must be published in a peer-reviewed journal
• Downgrade recs when poor quality evidence
• Allow consideration of indirect evidence if justification (described in text) acceptable but downgrade appropriately
• If evidence is not sufficient, then discuss in text but do not provide recommendation
Disagreements in Interpretations of EvidenceDisagreements in Interpretations of EvidenceDisagreements in Interpretations of EvidenceDisagreements in Interpretations of Evidence
o How do you reconcile disagreements in evidence interpretation among guidelines?
• Assess rigor of methodology/adherence to the evidence
• Request invited associations to review and comment on our guideline recommendations
• Offer competing guideline organizations opportunity to appoint a member to our next edition or update panel (providing he/she passes COI vetting and approval process)
Accommodating SubgroupsAccommodating SubgroupsAccommodating SubgroupsAccommodating Subgroups
o How do guidelines accommodate subgroups (ex. older populations or persons with multi-morbidities) whose treatment outcomes may differ from the average patient?
• If data exist for subgroups, use it.
• If not, use indirect evidence from similar groups and downgrade appropriately
• Always describe patient population in each recommendation based on the patient population in the original studies
• A major challenge is multiple co-morbidities
Setting StandardsSetting StandardsSetting StandardsSetting Standards
o What topics and/or processes do you think the committee should consider in deriving quality standards for clinical practice guidelines?
• Establish a definition of evidence-based guidelines, possibly with rating scales
• Standard setting topics and criteria. PCPI (Physician Consortium for Performance
Improvement) criteria for guidelines permitted as basis of performance measures
. AGREE instrument (AGREE III is in development)
. COGS (Conference on Guideline Standardization)
. GLIA – for implementation purposes
Panel CompositionPanel CompositionPanel CompositionPanel Composition
o What should the composition of CPG development panels, in particular the balance of methodologists, topical experts, and consumers, look like?
• Depends on structure of panel but should have methodologists reviewing evidence and developing evidence tables or profiles
• Consumers must be educated in EBM
• Content experts provide credibility but all must go through rigorous review, including COIs
• Other considerations: health economists, frontline clinicians, patient preference consultants
Grading SystemGrading SystemGrading SystemGrading System
o Is there an available assessment tool that adequately rates both the level of the scientific evidence and strength of clinical recommendations that should be used as standard practice in guideline development?
• The ACCP Grading System . A modification of GRADE (major difference: restricts
evidence to approved threshold)
. Based on (1) assessment of the quality of the original studies and (2) a balance of the risks to benefits
. Has been adopted by other guideline developers
Grading SystemGrading SystemGrading SystemGrading System
Grade of rec
Benefit versus risk and burdens
Methodologic quality of supporting evidence
1A Benefits clearly outweigh risks/burdens, or vice versa (negative recs)
Consistent evidence from randomized controlled trials without important limitations, or exceptionally strong evidence from observational studies
1B Benefits clearly outweigh risks/burdens, or vice versa (negative recs)
Evidence from randomized controlled trials with important limitations (inconsistent results, methodologic flaws, indirect or imprecise), or very strong evidence from observational studies
1C Benefits clearly outweigh risks/burdens, or vice versa (negative recs)
Evidence for at least one critical outcome from observational studies, case series, or from randomized, controlled trials with serious flaws or indirect evidence
2A Benefits closely balanced with risks/burdens
Consistent evidence from randomized controlled trials without important limitations, or exceptionally strong evidence from observational studies
2B Benefits closely balanced with risks/burdens
Evidence from randomized, controlled trials with important limitations (inconsistent results, methodologic flaws, indirect or imprecise), or very strong evidence from observational studies
2C Benefits closely balanced with risks/burdens
Evidence for at least one critical outcome from observational studies, case series, or from randomized, controlled trials with serious flaws or indirect evidence
Recommendations for PMsRecommendations for PMsRecommendations for PMsRecommendations for PMs
o What methods might be developed for determining which recommendations among those in a guideline should be applied to quality measures or electronic medical record decision prompts?
• Evidence should dictate direction and strength of recs
• Suggest some 1A and 1B recs for PMs (although not all)
• 1C, 2A, 2B, and 2C recs should generally not be used
• However, all should be evaluated based on feasibility, usability, scientific importance, practicality, and applicability
QI and HarmonizationQI and Harmonization
o What administrative (eg, accreditation) or legal approaches might improve the quality of clinical practice guidelines?• Published guideline quality rating scale
• Listed on NGC Web site • Currency rating also listed in NGC• PMs & CMS policies should be based on highly rated
EBGs
o What explicit approaches might harmonize guideline developers and increase guidelines convergence?• Funding requires multiple societies to collaborate• Require compliance with evidence-based standards,
rating scale
EBG PromotionEBG PromotionEBG PromotionEBG Promotion
o What types of strategies might promote greater utilization of guidelines?
• Requirements for implementation into:. EMRs. Registries. PMs, including PQRI
• Education to allow developers to learn about new techniques and processes (see last slide)
• Healthcare providers need to know how to find good guidelines and good guidelines should address the needs of providers (ask frontline clinicians what they need!)
Other ConsiderationsOther ConsiderationsOther ConsiderationsOther Considerations
o Are there other characteristics of guideline standards you think are important for the committee to consider?
• Most important: address funding• Also should move the field toward incorporating
resource considerations into the EBG recs
Promotion of IOM StandardsPromotion of IOM StandardsPromotion of IOM StandardsPromotion of IOM Standards
The IOM report and new standards should be presented to appropriate audiences:
Guideline developers will be attending the Guidelines International Network conference and guideline methodology course
Guidelines International Network (G-I-N)
2010 Conference - ChicagoAugust 26-28, 2010 – Conference Dates
August 25, 2010 – Pre-meeting Course in Guideline Methodology
Chicago, Illinois, USA
www.GIN2010.org
Host: American College of Chest Physicians