March 2, 2004: I. Sim Clinical Decision Support Systems Medical Informatics Clinical Decision Support Systems Ida Sim, MD, PhD March 2, 2004 Division of General Internal Medicine, and the Program in Biological and Medical Informatics UCSF Copyright Ida Sim, 2004. All federal and state rights reserved for all original material presented in this course through any medium, including lecture or print.
44
Embed
March 2, 2004: I. Sim Clinical Decision Support Systems Medical Informatics Clinical Decision Support Systems Ida Sim, MD, PhD March 2, 2004 Division of.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Clinical Decision Support Systems
Ida Sim, MD, PhD
March 2, 2004
Division of General Internal Medicine, and the Program in Biological and Medical Informatics
UCSF
Copyright Ida Sim, 2004. All federal and state rights reserved for all original material presented in this course through any medium, including lecture or print.
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Guest Lecture
• Thursday, March 4, 1:30 to 3pm• Paul Tang, MD Chief Information Officer, Palo
Alto Medical Foundation– state of the art electronic medical record
• same one that Kaiser is spending $2.8 billion on
– the promise and the reality
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Outline
• The quality chasm• Clinical decision support systems (CDSS)
– definition– how they work
• Effectiveness of CDSSs– improving quality– reducing errors
• Fundamental barrier
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Crossing the Quality Chasm
• Quality care for the 21st century • safe, effective• patient-centered, timely, efficient• equitable
• Evidence-based practice is means to quality– practice based on currently best available
evidence from clinical research
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Hardest Way to Practice
• Over 1,000 guidelines in National Guideline Clearinghouse
• Over 4,600 journals indexed in Medline– over 10,000 RCTs per year– over 2700 systematic reviews per year
• Logistically impossible to “keep up”– let alone make sense of and apply the evidence
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Informatics to the Rescue• Information technology touted to improve
quality of care– legible information more available
• chart, lab results, allergies
– care is more efficient• visit level coding, charting using macros, e-
prescribing
– improvement in intermediate measures• vaccination and screening rates
– improvement in patient outcomes• ...
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Outline
• The quality chasm• Clinical decision support systems (CDSS)
– definition– how they work
• Effectiveness of CDSSs– improving quality– reducing errors
• Fundamental barrier
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
What is a CDSS?
• Software that is designed to be a direct aid to clinical decision-making
• in which the characteristics of an individual patient are matched to a computerized clinical knowledge base
• and patient-specific assessments or recommendations are then presented to the clinician and/or the patient for a decision (Sim et al, JAMIA, 2001)
• Ground principle: CDSSs should be based on best current evidence, which continually evolves
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
CDSS Methods• Vast majority of clinically-used CDSSs use rule-
based reasoning– problem of combinatorial explosion of rules
• Major limitations– how to represent some data (e.g., “looks sick”)– formal, reproducible methods for making clinical
decisions
• Other major limitation is source of input data– manual input of data by doctors will not work– EMR can enable a new era of CDSSs
• But lots can be done with current technology
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Outline
• The quality chasm• Clinical decision support systems (CDSS)
– definition– how they work
• Effectiveness of CDSSs– improving quality– reducing errors
• Fundamental barriers
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
CDSS Effectiveness
• In controlled trials, only occasional modest benefit found (Hunt, JAMA 1998; updated RB Haynes 2000)
– diagnosis: 1/5 studies beneficial– drug dosing: 9/15– preventive care reminders: 19/26
• Few studies looked at patient outcomes– 6 of 14 showed benefit
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Shortcomings of CDSS Eval Literature
• Variable study quality– 35% rate >8 on 10 point quality scale (mean ~6.2)– more recent studies better quality
• Low power– 5 of 8 studies of patient outcome had low power
• Unit of randomization was the patient– physicians treated some patients with CDSS, and some
without CDSS– this would tend to …. any effects of the CDSS
• Probably publication bias
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Shortcomings of Study Designs
• E.g., evaluate a CDSS for hypertension treatment• Is RCT best design for determining effectiveness?
– should randomize MDs, usually low power– intervention is usually more than just the CDSS
• e.g., “buy-in” sessions to HTN guideline underlying CDSS
– limited generalizability• applies only to this particular CDSS• integration of CDSS into existing workflow often unique to
study site
– if CDSS shows no effect, standard RCT gives little insight into why
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Heterogeneity of CDSSs
• Preventive care reminder CDSS– A clerk routinely abstracts preventive care interventions
from paper chart into a database. Before each clinic session, nurse runs the CDSS for patients coming in that day. Guideline-based recommendations are printed out on paper and attached to front of chart. Doctor orders preventive care in usual way using paper-based methods.
• Hypertension treatment CDSS– Clinic has an EMR. During patient visit, CDSS notes that BP
and trend is too high. It checks patient’s Cr, diabetes status, cardiac status, current meds and allergies and recommends drug therapy change according to JNC VI guidelines. Presents e-prescription for MD to verify. If verified, order is sent directly to pharmacy and medication list updated.
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Understanding CDSS Effectiveness• Hunt counted the number of systems in each category
(e.g., drug dosing) that were “effective”• How would you improve on Hunt’s systematic review?
– CDSSs are very heterogeneous– does the heterogeneity explain any variation in benefit?
• How to meaningfully characterize CDSSs?– target decision maker(s) (MD, nurse, patient)– urgency of decision (stat result, outpatient screening)– method of delivery (paper, EMR, pager)– force of recommendation (suggestion, requirement) ...
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Typology of CDSSs
CONTEXTTarget task typeTarget patient settingPoint of careWorkflow integration
Data source-system
intermediary
CDSSCustomizationClinical knowledge source Update mechanismUnit of analysis
INPUTData source
OUTPUTAction complexityAction embeddedCompliance urgencyForce of action recommendationRecommendation explicitness
ProcessorTarget decision maker
System user
System-user interface
System user/ Processor/Target decision maker
OR• F
orm
/Mod
e of
info
rmat
ion
gene
rati
on
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
CDSS Characteristics: Highlights• Reviewed and classified 42 RCT-evaluated systems• Tremendous variation in decision-maker/context,
how recommendation delivered, staff needed to make system run, complexity of recommended actions– 45% targeted to clinician, 55% patient, 5% both– 62% based on national guidelines or literature– 69% “pushed” recommendation to decision maker– 43% collected data directly from the EMR
• 45% required data input intermediary (11% MD), 58% of time requiring clinical knowledge
– 26% required an output intermediary
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
CDSS Effectiveness Summary• Current data suggests CDSSs can improve the process of
care and perhaps clinical outcomes– most effective at preventive care reminders– modest at best for drug dosing and active care– generally not helpful for improving diagnosis except with trainees
• Findings limited by– methodological problems and design type of studies– insufficient appreciation of workflow component of CDSSs– insufficient appreciation of heterogeneity of systems
• Meta-regression on predictors of CDSS success using taxonomy, pending
• Bottom line: equivocal evidence, limited use
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Outline
• The quality chasm• Clinical decision support systems (CDSS)
– definition– how they work
• Effectiveness of CDSSs– improving quality– reducing errors
• Fundamental barrier
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
A Case...
• 51 y/o F prescribed camphorated tincture of opium• Given tincture of opium, died of morphine OD• Sons suing CVS for “pharmaceutical negligence and
wanton and reckless misconduct” for failing to “utilize, test and maintain” computer software capable of alerting employees to the dangers of confusing the two medications (Hartford Courant, Feb. 28, 2002)
• Was CVS negligent and reckless?– CVS could have used a rule-based system
• if Tincture of Opium => verify not Paregoric• if Tincture of Opium AND (dose > xyz) => verify dose
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Prescription Error Checking
• Major errors can still slip through (Inst. Safe Medication Practices, 99)
– only 38% detect lethal cisplatin, vincristine doses– only 42% of systems linked between pharmacy and lab
• 87% did not detect excessive tobra dose for Cr level
• Drug-drug interaction detection– false positive rates in the 30% range currently– no commercial systems well accepted
• Alert fatigue is a real problem; other industries know “safety” systems can promote error
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
CPOE to Reduce Medication Errors• Computerized Physician Order Entry (CPOE)
• Rush to implement CPOE– ahead/independent of EMR– may be complemented by bar coding, automated
dispensing carts, pharmacy decision support, etc.
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Cedars-Sinai Experience• Patient Care Expert (PCX) CPOE
– developed ~$30 million system in-house– compares orders to recommendations, checks for allergies/drug
interactions, alerts to alternatives– expected 60-80% reduction in medical errors– rolled out October 2002
• A failure– CHF pt. did not receive prescribed pills, pt. got walker 3 days late,
baby got anesthetic day before circumcision– 2 mins. to order Vanco: sign on, search for screen, enter, justify,
verify; vs. 5 secs. previously– MDs delaying writing orders, writing them imprecisely– perception PCX was compromising patient safety
• Plug pulled mid-January 2003 (400 MD insurrection)• No plans to reintroduce CPOE until 2006 at the earliest
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Outline
• The quality chasm• Clinical decision support systems (CDSS)
– definition– how they work
• Effectiveness of CDSSs– improving quality– reducing errors
• Fundamental barrier
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Fundamental Barrier
• Better quality care <-- better decision support• Better decision support <-- “smarter” systems
– “know” more about the patient, evidence, context
• “Smarter” systems <-- more richly coded data– for EMR: SNOMED, standard EMR structure– for knowledge: coding, structures for guidelines, RCTs…
• Coded data <-- Coding of data entry• Coding of data entry <-- Greater physician time• Greater physician time --> no play --> no gain
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Implications
• Clear trade-off between physician coding effort and “smarter” decision support
• Don’t expect more decision support than coding supports– generally successful decision support
• preventive care: age, last mammogram, etc.• allergies: Yes/No on specific drugs• drug dosing: weight, height, creatinine, age
– generally unsuccessful decision support• diagnostic assistance• complicated therapies (e.g., management of hypertension)
• Unrealistic to think of CDSSs as improving evidence-based practice in general
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
The CVS Case
• Family recently settled for $1.8 million• Pharmacist had to take “intensive” error classes• State law changed, pharmacies must
– maintain error records– report errors to prescribing doctor, patient, or family
• No mention of having to use a computer system, etc.– error prevention needs a system approach; CPOE is but
one part of that
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Summary on CDSSs• Equivocal evidence for improving quality
– limited by methodological and other shortcomings• Can reduce errors
– use in context of system approach– weigh costs and challenges against Absolute Risk Reduction
• Barriers in the short term– knowing which decisions to support (preventive care reminders)– data exchange among legacy systems– integrating all into a cohesive EMR solution– organizational challenges, incentivize for quality, etc. (3/16 class)
• Barriers in the long term– efficient coding: of what, by whom, when, why– clinical vocabulary with adequate semantic coverage– better reasoning methods
March 2, 2004: I. Sim Clinical Decision Support SystemsMedical Informatics
Teaching Points• CDSSs have great promise for improving
quality/reducing error– but promise essentially not yet realized
• CDSSs are heterogeneous in design and use– is a social/organizational intervention as much as a
technical one– need to improve methods for evaluating CDSSs
• Near-term difficulties in coding – should limit our sights on what to expect
• Need fundamental informatics research to crack coding barrier