IBM Medical Records Text Analytics Solution Helps UNC Healthcare Improve the Quality of Hospital Discharges Session Number ECA-1419A Carlton Moore, MD UNC Healthcare Fiodar Zboichyk IBM
Dec 18, 2015
IBM Medical Records Text Analytics Solution Helps UNC Healthcare Improve
the Quality of Hospital DischargesSession Number ECA-1419A
Carlton Moore, MDUNC Healthcare
Fiodar ZboichykIBM
2
Overview
• Hospital Readmission Rates
– Medical and Economic Impact
• Reasons for High Readmission Rates
– Importance of discharge summary
• Proposed NLP solution
– Development issues (example, unstructured, inconsistent)
• Results (sensitivity, specificity)
• Future directions
3
30-Day Hospital Readmission Rates by State
Jencks S, Williams M, Coleman E. N Engl J Med 2009; 360 (14): 1418-28
Estimated annual cost to Medicare = $17.4B
4
Economic Impact on Hospitals
• In 2013 Medicare will start applying financial penalties to hospital with higher than expected readmission rates
• Other health insurers are likely to follow Medicare’s lead!!
Condition #of Patients
Average Reimbursement
%Higher than Expected
Potential Penalty
Heart Failure 600 $5,000 20% $600,000
Heart Attack 400 $4,000 20% $320,000
Pneumonia 350 $3,000 15% $157,500
$1,107,500
Sample Hospital
Potential Penalty = (# of patients with condition) x (Avg. reimbursement for condition) x (% Higher than expected)
6
Conceptual Framework
Discharge instructions not carried out
Adverse Event
Hospital Readmission
Patient discharged with unresolved medical issues that need to be addressed after leaving hospital
follow-up physician visitsfollow-up tests and procedures
Discharge Instructions a concise action plan
describing what needs to occur after a patient
leaves the hospital
Definition: condition worsens because of inappropriate or inadequate medical care
Only 50% of discharge summaries are ever received by patients’
physicians
Poor communication of discharge instructions
7
Discharge Instructions
Diagnostic Pro-cedures
Physician Referrals Lab Tests0%
10%
20%
30%
40%
50%
48%
35%
17%
Types of Discharge Instructions(693 hospital discharges)
50% not completed 27% no completed 15% not completed
Moore C, McGinn T, Halm E. Arch Intern Med. 2007;167:1305-1311
8
Examples of Discharge Instructions not Completed
Types of Procedure Reasons for Procedures
CT of Chest Lung mass found on previous x-ray
CT scan of the abdomen Abdominal abscess and kidney mass
Chest x-ray Lung nodule on admission chest x-ray
Colonoscopy Gastrointestinal bleeding
Physician Referrals Reasons for Referrals
Psychiatry Suicidal Ideation
Neurology Seizures
Nephrology Kidney failure
Surgery Infected wound
Moore C, McGinn T, Halm E. Arch Intern Med. 2007;167:1305-1311
9
Adverse Events after Hospital Discharge
• 1 in 5 (20%) patients has an adverse event shortly after hospital discharge
Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. Ann Intern Med. 2003
ADE Procdeure Related
Other Infection Fall0%
10%
20%
30%
40%
50%
60%
70%62%
16% 14%
5% 4%
Types of Adverse Events, %
ADE: adverse drug eventOther: incorrect treatment and/or missed diagnosis
10
Example of an Adverse Event
• A patient with heart failure started receiving spironolactone in the hospital. The patient was sent home with a prescription for this medication in addition to previous use of ramipril and potassium supplements.
• Blood tests were not monitored after hospital discharge even though it was clearly documented in the discharge summary that the patient needed follow-up blood tests.
• Two weeks later the patient developed extreme weakness and went to the emergency room. Blood tests revealed a potassium level >7.5 mmol/L (normal = 4.5 mmol/L).
Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. Ann Intern Med. 2003
11
Purpose of Project
• Extract key elements of the discharge instructions:
– Discharge medications
– Discharge diagnosis
– Follow-up appointments
• Convert the extracted data into structured format that can be:
– electronically transmitted to healthcare providers responsible for care after hospital discharge
– used to generate reminders and alerts to healthcare providers
12
Discharge Instructions
Clinic Physician Scheduled Date/Time
Internal Medicine
Joseph Morgan
Yes 8/10/200916:10
Cardiology - EP Null No Null
Anticoagulation Null Yes 8/4/20098:45
13
Study Design
• Discharge instructions (Name, Type/Location, Time Frame) were extracted from free-text hospital discharge summaries:
– Manual review (physician)
– IBM Content Analytics (ICA)
• Accuracy of ICA was calculated using manual physician review as the “gold standard”
– Sensitivity, specificity
– Positive predictive value, negative predictive value
14
Measurement
• Overall Accuracy = (TP +TN)/(Total)
• Sensitivity
– % of records containing follow-up elements that were identified via text analytics.
• Specificity
– % of records lacking follow-up elements that were not flagged via text analytics.
• Positive Predictive Value (PPV)
– % of records flagged as containing follow-up elements using text analytics that actually contained follow-ups
15
Results: Accuracy of Text Analytics in Identifying Follow-up Appointments and Diagnoses
Element Overall Accuracy
Precision Sensitivity (Recall)
Specificity PPV
Diagnoses 78% 90% 80% 68% 90%
Followup 79% 95% 74% 91% 95%
16
IBM
Co
nte
nt
An
aly
tic
s Document Server
(UIMA Pipeline)
Extended ICA JDBC Crawler
IBM InfoSphere Guardium Data Redaction
UNC Health Care Clinical Data Warehouse
Apache Lucene Search Engine
ICA-Text Miner Web Application
LuceneIndex
JDB
C U
IMA
CA
S
Con
sum
er
Pathology ReportsDischarge Summary ReportsEchocardiogram Reports
Med
ical
Ann
otat
ors
ICA
Ann
otat
ors
ICA-LanguageWare Resource Workbench
Health Language Inc.Language Engine
SNOMED, RxNorm, ICD-9ICD-10, CPT-4
Medical Terminology
UNC Health CareTerminology
UNC Health Care Solution Component Architecture
Discharge Follow-up Reporting
Business Intelligence Tool
17
Project Lessons Learned
• Medical texts are more complicated than we thought… again.
• Standard terminology (RxNorm, SNOMED CT, ICD9, …)
– Absolutely required, but not good enough for dictionary matches
– “tick-born disease”, but not “tick borne illness”.
• Diagnoses
– Negation is actually just part of the range – “rule out”, “possible”.
– “Left femur fracture” and “fracture, left femur”.
– “Discharge diagnosis: same as above”.
• Follow-ups. Sometimes just “fup”.
– Usually “Dr. Good”, but sometimes “her cardiologist”.
– Usually “Vascular Surgery Clinic”, but sometimes “heme-onc”.
18
Summary
• NLP will improve communication of discharge instructions:
– Improve patient care (reduce hospital readmissions)
– Reduce risk of Medicare penalties to the hospital