1 Database & Methods Cyberseminar Series Chart Review using National EHR Tools July 10, 2017 Linda S. Williams, MD Co-PI, Precision Monitoring (PRIS-M) QUERI HSR&D Center for Health Information and Communication Professor of Neurology, Indiana University School of Medicine Research Scientist, Regenstrief Institute, Inc. Indianapolis, IN
48
Embed
Chart Review using National EHR Tools · Chart Review using National EHR Tools . July 10, 2017 . Linda S. Williams, MD . Co-PI, Precision Monitoring (PRIS-M) QUERI HSR&D Center for
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Database amp Methods Cyberseminar Series
Chart Review using National EHR Tools
July 10 2017
Linda S Williams MD
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Research Scientist Regenstrief Institute Inc
Indianapolis IN
2
Database amp Methods Cyberseminar Series
Informational seminars to help VA researchers understand how to use VA and non-VA data in research
and quality improvement
Topics
bull Application of VA and non-VA data to research and quality
improvement questions
bull Limitations of secondary data use
bull Resources to support VA data use
7102016
3
Date Topic
10316 Overview of VA Data amp Research Uses
1172016 Requesting Access to VA Data
1252016 Healthcare Utilization with MedSAS amp CDW
192017 VA Medicare Data (VACMS)
262017 Assessing Pharmacy Utilization with VA Data
362017 Mortality Ascertainment amp Cause of Death
432017 Assessing Race amp Ethnicity
652017 Pharmacy Data
7102017 Chart Review Using National EHR Tools
872017 Applying Comorbidity Measures Using VA and CMS
(MedicareMedicaid) Data
9112017 Using CDW Microbiology and Pharmacy Data in
Outcomes Research
7102016
FY lsquo17 Database amp Methods Schedule First Monday of the month 100pm-200pm ET
Visit our Education page for more information amp
registration links wwwvirecresearchvagov
Schedule shifts by one week in event of VA holiday
4
Chart Review Using National EHR Tools
Database amp Methods Cyberseminar Series
Linda S Williams MD
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Research Scientist Regenstrief Institute Inc
Indianapolis IN
Learning Objectives
By the end of this cyberseminar attendees will be able to
5
bull Identify when chart review is an effective data collection methodology
bull Compare different strategies for conducting chart review
bull Anticipate common steps needed to plan and conduct a rigorous chart
review project
bull Be familiar with tools that can support electronic chart review activities
7102016
6
7102016
Poll 1 What is your role in the VA
bull Research investigatorPI
bull Data manager analyst or programmer
bull Project coordinator
bull Clinical or operations staff
bull Other ndash please describe via the QampA function
7
7102016
Poll Question 2 What is your previous experience using chart review for research
bull Never did chart review
bull Used paper charts
bull Used my local electronic health record only (CPRS)
bull Used VistAWeb or CAPRI
bull Used some other platform for central chart review (please
describe using central QampA)
8
Topics
bull When to use chart review for research
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
9
Project examples
bull INSPIRE SDP (L Williams PI)
bull CARE TIMe SDP (D Bravata PI)
bull Operational projectsOffice of Clinical Analytics and Reporting
References
bull Williams LS et al A cluster-randomized quality improvement study to
improve two inpatient stroke quality indicators BMJ Qual Safety 2015
doi 101136bmjqs-2015-004188
bull Phipps M et al Validation of stroke meaningful use measures in a
national EHR system J Gen Internal Med 201631(Suppl 1)S46-52
bull Bravata D Myers L Cheng E et al Quality of Care for Veterans with
TIA and Minor Stroke Stroke 201546(Supp 1)ATMP73
7102016
10
Topics
bull When to use chart review for research
bull Confirm data in VA administrative datasets
bull Capture data not available in VA administrative datasets
bull Local vs Central Chart Review
bull Costaccuracy vs local chart reviews
bull Using notes in the CDW
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
11
8792
7102016
Confirm data in VHA administrative datasets
bull Validate case ascertainment strategies
bull Assess the accuracy of your administrative data-based
case ascertainment or outcome assessment and adjust if
needed for a retrospective administrative data analysis
bull Chart review serves as ldquocriterion standardrdquo
bull TIA cohort identified
by ICD-9 TIA code
in ED or hospital
discharge
bull Minor stroke cohort
identified by ICD-9
hospital discharge
codes and other
administrative data
(clinical severity
data not available)
12
Type of error
(of 1948 eligible admissions)
Reason
False negative (13) 1- discharge date incorrect
12-admin data did not
capture consult
Confirm data in VHA administrative datasets
Assess clinical vs administrative completion of an action
Example How many patients received a rehabilitation consult during their
stroke admission
bull Joint Commission stroke quality indicator included in VA facility SAIL reports
bull Research question How accurate are administrative measures of this quality indicator
bull Administrative data includes completed consult information
bull However consults may be completed without an actual rehab evaluation
taking place or may be done without administrative documentation
If infrequent it may not be a
problem but assessing a random
sample of cases may be important
to understand the variability in your
estimate of the of patients
receiving this care
False positive (12) 1- consult after discharge
3- patient not seen
4- consult cancelled
4- consult was for
diagnostic test only
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
2
Database amp Methods Cyberseminar Series
Informational seminars to help VA researchers understand how to use VA and non-VA data in research
and quality improvement
Topics
bull Application of VA and non-VA data to research and quality
improvement questions
bull Limitations of secondary data use
bull Resources to support VA data use
7102016
3
Date Topic
10316 Overview of VA Data amp Research Uses
1172016 Requesting Access to VA Data
1252016 Healthcare Utilization with MedSAS amp CDW
192017 VA Medicare Data (VACMS)
262017 Assessing Pharmacy Utilization with VA Data
362017 Mortality Ascertainment amp Cause of Death
432017 Assessing Race amp Ethnicity
652017 Pharmacy Data
7102017 Chart Review Using National EHR Tools
872017 Applying Comorbidity Measures Using VA and CMS
(MedicareMedicaid) Data
9112017 Using CDW Microbiology and Pharmacy Data in
Outcomes Research
7102016
FY lsquo17 Database amp Methods Schedule First Monday of the month 100pm-200pm ET
Visit our Education page for more information amp
registration links wwwvirecresearchvagov
Schedule shifts by one week in event of VA holiday
4
Chart Review Using National EHR Tools
Database amp Methods Cyberseminar Series
Linda S Williams MD
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Research Scientist Regenstrief Institute Inc
Indianapolis IN
Learning Objectives
By the end of this cyberseminar attendees will be able to
5
bull Identify when chart review is an effective data collection methodology
bull Compare different strategies for conducting chart review
bull Anticipate common steps needed to plan and conduct a rigorous chart
review project
bull Be familiar with tools that can support electronic chart review activities
7102016
6
7102016
Poll 1 What is your role in the VA
bull Research investigatorPI
bull Data manager analyst or programmer
bull Project coordinator
bull Clinical or operations staff
bull Other ndash please describe via the QampA function
7
7102016
Poll Question 2 What is your previous experience using chart review for research
bull Never did chart review
bull Used paper charts
bull Used my local electronic health record only (CPRS)
bull Used VistAWeb or CAPRI
bull Used some other platform for central chart review (please
describe using central QampA)
8
Topics
bull When to use chart review for research
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
9
Project examples
bull INSPIRE SDP (L Williams PI)
bull CARE TIMe SDP (D Bravata PI)
bull Operational projectsOffice of Clinical Analytics and Reporting
References
bull Williams LS et al A cluster-randomized quality improvement study to
improve two inpatient stroke quality indicators BMJ Qual Safety 2015
doi 101136bmjqs-2015-004188
bull Phipps M et al Validation of stroke meaningful use measures in a
national EHR system J Gen Internal Med 201631(Suppl 1)S46-52
bull Bravata D Myers L Cheng E et al Quality of Care for Veterans with
TIA and Minor Stroke Stroke 201546(Supp 1)ATMP73
7102016
10
Topics
bull When to use chart review for research
bull Confirm data in VA administrative datasets
bull Capture data not available in VA administrative datasets
bull Local vs Central Chart Review
bull Costaccuracy vs local chart reviews
bull Using notes in the CDW
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
11
8792
7102016
Confirm data in VHA administrative datasets
bull Validate case ascertainment strategies
bull Assess the accuracy of your administrative data-based
case ascertainment or outcome assessment and adjust if
needed for a retrospective administrative data analysis
bull Chart review serves as ldquocriterion standardrdquo
bull TIA cohort identified
by ICD-9 TIA code
in ED or hospital
discharge
bull Minor stroke cohort
identified by ICD-9
hospital discharge
codes and other
administrative data
(clinical severity
data not available)
12
Type of error
(of 1948 eligible admissions)
Reason
False negative (13) 1- discharge date incorrect
12-admin data did not
capture consult
Confirm data in VHA administrative datasets
Assess clinical vs administrative completion of an action
Example How many patients received a rehabilitation consult during their
stroke admission
bull Joint Commission stroke quality indicator included in VA facility SAIL reports
bull Research question How accurate are administrative measures of this quality indicator
bull Administrative data includes completed consult information
bull However consults may be completed without an actual rehab evaluation
taking place or may be done without administrative documentation
If infrequent it may not be a
problem but assessing a random
sample of cases may be important
to understand the variability in your
estimate of the of patients
receiving this care
False positive (12) 1- consult after discharge
3- patient not seen
4- consult cancelled
4- consult was for
diagnostic test only
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
3
Date Topic
10316 Overview of VA Data amp Research Uses
1172016 Requesting Access to VA Data
1252016 Healthcare Utilization with MedSAS amp CDW
192017 VA Medicare Data (VACMS)
262017 Assessing Pharmacy Utilization with VA Data
362017 Mortality Ascertainment amp Cause of Death
432017 Assessing Race amp Ethnicity
652017 Pharmacy Data
7102017 Chart Review Using National EHR Tools
872017 Applying Comorbidity Measures Using VA and CMS
(MedicareMedicaid) Data
9112017 Using CDW Microbiology and Pharmacy Data in
Outcomes Research
7102016
FY lsquo17 Database amp Methods Schedule First Monday of the month 100pm-200pm ET
Visit our Education page for more information amp
registration links wwwvirecresearchvagov
Schedule shifts by one week in event of VA holiday
4
Chart Review Using National EHR Tools
Database amp Methods Cyberseminar Series
Linda S Williams MD
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Research Scientist Regenstrief Institute Inc
Indianapolis IN
Learning Objectives
By the end of this cyberseminar attendees will be able to
5
bull Identify when chart review is an effective data collection methodology
bull Compare different strategies for conducting chart review
bull Anticipate common steps needed to plan and conduct a rigorous chart
review project
bull Be familiar with tools that can support electronic chart review activities
7102016
6
7102016
Poll 1 What is your role in the VA
bull Research investigatorPI
bull Data manager analyst or programmer
bull Project coordinator
bull Clinical or operations staff
bull Other ndash please describe via the QampA function
7
7102016
Poll Question 2 What is your previous experience using chart review for research
bull Never did chart review
bull Used paper charts
bull Used my local electronic health record only (CPRS)
bull Used VistAWeb or CAPRI
bull Used some other platform for central chart review (please
describe using central QampA)
8
Topics
bull When to use chart review for research
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
9
Project examples
bull INSPIRE SDP (L Williams PI)
bull CARE TIMe SDP (D Bravata PI)
bull Operational projectsOffice of Clinical Analytics and Reporting
References
bull Williams LS et al A cluster-randomized quality improvement study to
improve two inpatient stroke quality indicators BMJ Qual Safety 2015
doi 101136bmjqs-2015-004188
bull Phipps M et al Validation of stroke meaningful use measures in a
national EHR system J Gen Internal Med 201631(Suppl 1)S46-52
bull Bravata D Myers L Cheng E et al Quality of Care for Veterans with
TIA and Minor Stroke Stroke 201546(Supp 1)ATMP73
7102016
10
Topics
bull When to use chart review for research
bull Confirm data in VA administrative datasets
bull Capture data not available in VA administrative datasets
bull Local vs Central Chart Review
bull Costaccuracy vs local chart reviews
bull Using notes in the CDW
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
11
8792
7102016
Confirm data in VHA administrative datasets
bull Validate case ascertainment strategies
bull Assess the accuracy of your administrative data-based
case ascertainment or outcome assessment and adjust if
needed for a retrospective administrative data analysis
bull Chart review serves as ldquocriterion standardrdquo
bull TIA cohort identified
by ICD-9 TIA code
in ED or hospital
discharge
bull Minor stroke cohort
identified by ICD-9
hospital discharge
codes and other
administrative data
(clinical severity
data not available)
12
Type of error
(of 1948 eligible admissions)
Reason
False negative (13) 1- discharge date incorrect
12-admin data did not
capture consult
Confirm data in VHA administrative datasets
Assess clinical vs administrative completion of an action
Example How many patients received a rehabilitation consult during their
stroke admission
bull Joint Commission stroke quality indicator included in VA facility SAIL reports
bull Research question How accurate are administrative measures of this quality indicator
bull Administrative data includes completed consult information
bull However consults may be completed without an actual rehab evaluation
taking place or may be done without administrative documentation
If infrequent it may not be a
problem but assessing a random
sample of cases may be important
to understand the variability in your
estimate of the of patients
receiving this care
False positive (12) 1- consult after discharge
3- patient not seen
4- consult cancelled
4- consult was for
diagnostic test only
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
4
Chart Review Using National EHR Tools
Database amp Methods Cyberseminar Series
Linda S Williams MD
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Research Scientist Regenstrief Institute Inc
Indianapolis IN
Learning Objectives
By the end of this cyberseminar attendees will be able to
5
bull Identify when chart review is an effective data collection methodology
bull Compare different strategies for conducting chart review
bull Anticipate common steps needed to plan and conduct a rigorous chart
review project
bull Be familiar with tools that can support electronic chart review activities
7102016
6
7102016
Poll 1 What is your role in the VA
bull Research investigatorPI
bull Data manager analyst or programmer
bull Project coordinator
bull Clinical or operations staff
bull Other ndash please describe via the QampA function
7
7102016
Poll Question 2 What is your previous experience using chart review for research
bull Never did chart review
bull Used paper charts
bull Used my local electronic health record only (CPRS)
bull Used VistAWeb or CAPRI
bull Used some other platform for central chart review (please
describe using central QampA)
8
Topics
bull When to use chart review for research
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
9
Project examples
bull INSPIRE SDP (L Williams PI)
bull CARE TIMe SDP (D Bravata PI)
bull Operational projectsOffice of Clinical Analytics and Reporting
References
bull Williams LS et al A cluster-randomized quality improvement study to
improve two inpatient stroke quality indicators BMJ Qual Safety 2015
doi 101136bmjqs-2015-004188
bull Phipps M et al Validation of stroke meaningful use measures in a
national EHR system J Gen Internal Med 201631(Suppl 1)S46-52
bull Bravata D Myers L Cheng E et al Quality of Care for Veterans with
TIA and Minor Stroke Stroke 201546(Supp 1)ATMP73
7102016
10
Topics
bull When to use chart review for research
bull Confirm data in VA administrative datasets
bull Capture data not available in VA administrative datasets
bull Local vs Central Chart Review
bull Costaccuracy vs local chart reviews
bull Using notes in the CDW
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
11
8792
7102016
Confirm data in VHA administrative datasets
bull Validate case ascertainment strategies
bull Assess the accuracy of your administrative data-based
case ascertainment or outcome assessment and adjust if
needed for a retrospective administrative data analysis
bull Chart review serves as ldquocriterion standardrdquo
bull TIA cohort identified
by ICD-9 TIA code
in ED or hospital
discharge
bull Minor stroke cohort
identified by ICD-9
hospital discharge
codes and other
administrative data
(clinical severity
data not available)
12
Type of error
(of 1948 eligible admissions)
Reason
False negative (13) 1- discharge date incorrect
12-admin data did not
capture consult
Confirm data in VHA administrative datasets
Assess clinical vs administrative completion of an action
Example How many patients received a rehabilitation consult during their
stroke admission
bull Joint Commission stroke quality indicator included in VA facility SAIL reports
bull Research question How accurate are administrative measures of this quality indicator
bull Administrative data includes completed consult information
bull However consults may be completed without an actual rehab evaluation
taking place or may be done without administrative documentation
If infrequent it may not be a
problem but assessing a random
sample of cases may be important
to understand the variability in your
estimate of the of patients
receiving this care
False positive (12) 1- consult after discharge
3- patient not seen
4- consult cancelled
4- consult was for
diagnostic test only
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
Learning Objectives
By the end of this cyberseminar attendees will be able to
5
bull Identify when chart review is an effective data collection methodology
bull Compare different strategies for conducting chart review
bull Anticipate common steps needed to plan and conduct a rigorous chart
review project
bull Be familiar with tools that can support electronic chart review activities
7102016
6
7102016
Poll 1 What is your role in the VA
bull Research investigatorPI
bull Data manager analyst or programmer
bull Project coordinator
bull Clinical or operations staff
bull Other ndash please describe via the QampA function
7
7102016
Poll Question 2 What is your previous experience using chart review for research
bull Never did chart review
bull Used paper charts
bull Used my local electronic health record only (CPRS)
bull Used VistAWeb or CAPRI
bull Used some other platform for central chart review (please
describe using central QampA)
8
Topics
bull When to use chart review for research
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
9
Project examples
bull INSPIRE SDP (L Williams PI)
bull CARE TIMe SDP (D Bravata PI)
bull Operational projectsOffice of Clinical Analytics and Reporting
References
bull Williams LS et al A cluster-randomized quality improvement study to
improve two inpatient stroke quality indicators BMJ Qual Safety 2015
doi 101136bmjqs-2015-004188
bull Phipps M et al Validation of stroke meaningful use measures in a
national EHR system J Gen Internal Med 201631(Suppl 1)S46-52
bull Bravata D Myers L Cheng E et al Quality of Care for Veterans with
TIA and Minor Stroke Stroke 201546(Supp 1)ATMP73
7102016
10
Topics
bull When to use chart review for research
bull Confirm data in VA administrative datasets
bull Capture data not available in VA administrative datasets
bull Local vs Central Chart Review
bull Costaccuracy vs local chart reviews
bull Using notes in the CDW
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
11
8792
7102016
Confirm data in VHA administrative datasets
bull Validate case ascertainment strategies
bull Assess the accuracy of your administrative data-based
case ascertainment or outcome assessment and adjust if
needed for a retrospective administrative data analysis
bull Chart review serves as ldquocriterion standardrdquo
bull TIA cohort identified
by ICD-9 TIA code
in ED or hospital
discharge
bull Minor stroke cohort
identified by ICD-9
hospital discharge
codes and other
administrative data
(clinical severity
data not available)
12
Type of error
(of 1948 eligible admissions)
Reason
False negative (13) 1- discharge date incorrect
12-admin data did not
capture consult
Confirm data in VHA administrative datasets
Assess clinical vs administrative completion of an action
Example How many patients received a rehabilitation consult during their
stroke admission
bull Joint Commission stroke quality indicator included in VA facility SAIL reports
bull Research question How accurate are administrative measures of this quality indicator
bull Administrative data includes completed consult information
bull However consults may be completed without an actual rehab evaluation
taking place or may be done without administrative documentation
If infrequent it may not be a
problem but assessing a random
sample of cases may be important
to understand the variability in your
estimate of the of patients
receiving this care
False positive (12) 1- consult after discharge
3- patient not seen
4- consult cancelled
4- consult was for
diagnostic test only
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
6
7102016
Poll 1 What is your role in the VA
bull Research investigatorPI
bull Data manager analyst or programmer
bull Project coordinator
bull Clinical or operations staff
bull Other ndash please describe via the QampA function
7
7102016
Poll Question 2 What is your previous experience using chart review for research
bull Never did chart review
bull Used paper charts
bull Used my local electronic health record only (CPRS)
bull Used VistAWeb or CAPRI
bull Used some other platform for central chart review (please
describe using central QampA)
8
Topics
bull When to use chart review for research
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
9
Project examples
bull INSPIRE SDP (L Williams PI)
bull CARE TIMe SDP (D Bravata PI)
bull Operational projectsOffice of Clinical Analytics and Reporting
References
bull Williams LS et al A cluster-randomized quality improvement study to
improve two inpatient stroke quality indicators BMJ Qual Safety 2015
doi 101136bmjqs-2015-004188
bull Phipps M et al Validation of stroke meaningful use measures in a
national EHR system J Gen Internal Med 201631(Suppl 1)S46-52
bull Bravata D Myers L Cheng E et al Quality of Care for Veterans with
TIA and Minor Stroke Stroke 201546(Supp 1)ATMP73
7102016
10
Topics
bull When to use chart review for research
bull Confirm data in VA administrative datasets
bull Capture data not available in VA administrative datasets
bull Local vs Central Chart Review
bull Costaccuracy vs local chart reviews
bull Using notes in the CDW
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
11
8792
7102016
Confirm data in VHA administrative datasets
bull Validate case ascertainment strategies
bull Assess the accuracy of your administrative data-based
case ascertainment or outcome assessment and adjust if
needed for a retrospective administrative data analysis
bull Chart review serves as ldquocriterion standardrdquo
bull TIA cohort identified
by ICD-9 TIA code
in ED or hospital
discharge
bull Minor stroke cohort
identified by ICD-9
hospital discharge
codes and other
administrative data
(clinical severity
data not available)
12
Type of error
(of 1948 eligible admissions)
Reason
False negative (13) 1- discharge date incorrect
12-admin data did not
capture consult
Confirm data in VHA administrative datasets
Assess clinical vs administrative completion of an action
Example How many patients received a rehabilitation consult during their
stroke admission
bull Joint Commission stroke quality indicator included in VA facility SAIL reports
bull Research question How accurate are administrative measures of this quality indicator
bull Administrative data includes completed consult information
bull However consults may be completed without an actual rehab evaluation
taking place or may be done without administrative documentation
If infrequent it may not be a
problem but assessing a random
sample of cases may be important
to understand the variability in your
estimate of the of patients
receiving this care
False positive (12) 1- consult after discharge
3- patient not seen
4- consult cancelled
4- consult was for
diagnostic test only
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
7
7102016
Poll Question 2 What is your previous experience using chart review for research
bull Never did chart review
bull Used paper charts
bull Used my local electronic health record only (CPRS)
bull Used VistAWeb or CAPRI
bull Used some other platform for central chart review (please
describe using central QampA)
8
Topics
bull When to use chart review for research
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
9
Project examples
bull INSPIRE SDP (L Williams PI)
bull CARE TIMe SDP (D Bravata PI)
bull Operational projectsOffice of Clinical Analytics and Reporting
References
bull Williams LS et al A cluster-randomized quality improvement study to
improve two inpatient stroke quality indicators BMJ Qual Safety 2015
doi 101136bmjqs-2015-004188
bull Phipps M et al Validation of stroke meaningful use measures in a
national EHR system J Gen Internal Med 201631(Suppl 1)S46-52
bull Bravata D Myers L Cheng E et al Quality of Care for Veterans with
TIA and Minor Stroke Stroke 201546(Supp 1)ATMP73
7102016
10
Topics
bull When to use chart review for research
bull Confirm data in VA administrative datasets
bull Capture data not available in VA administrative datasets
bull Local vs Central Chart Review
bull Costaccuracy vs local chart reviews
bull Using notes in the CDW
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
11
8792
7102016
Confirm data in VHA administrative datasets
bull Validate case ascertainment strategies
bull Assess the accuracy of your administrative data-based
case ascertainment or outcome assessment and adjust if
needed for a retrospective administrative data analysis
bull Chart review serves as ldquocriterion standardrdquo
bull TIA cohort identified
by ICD-9 TIA code
in ED or hospital
discharge
bull Minor stroke cohort
identified by ICD-9
hospital discharge
codes and other
administrative data
(clinical severity
data not available)
12
Type of error
(of 1948 eligible admissions)
Reason
False negative (13) 1- discharge date incorrect
12-admin data did not
capture consult
Confirm data in VHA administrative datasets
Assess clinical vs administrative completion of an action
Example How many patients received a rehabilitation consult during their
stroke admission
bull Joint Commission stroke quality indicator included in VA facility SAIL reports
bull Research question How accurate are administrative measures of this quality indicator
bull Administrative data includes completed consult information
bull However consults may be completed without an actual rehab evaluation
taking place or may be done without administrative documentation
If infrequent it may not be a
problem but assessing a random
sample of cases may be important
to understand the variability in your
estimate of the of patients
receiving this care
False positive (12) 1- consult after discharge
3- patient not seen
4- consult cancelled
4- consult was for
diagnostic test only
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
8
Topics
bull When to use chart review for research
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
9
Project examples
bull INSPIRE SDP (L Williams PI)
bull CARE TIMe SDP (D Bravata PI)
bull Operational projectsOffice of Clinical Analytics and Reporting
References
bull Williams LS et al A cluster-randomized quality improvement study to
improve two inpatient stroke quality indicators BMJ Qual Safety 2015
doi 101136bmjqs-2015-004188
bull Phipps M et al Validation of stroke meaningful use measures in a
national EHR system J Gen Internal Med 201631(Suppl 1)S46-52
bull Bravata D Myers L Cheng E et al Quality of Care for Veterans with
TIA and Minor Stroke Stroke 201546(Supp 1)ATMP73
7102016
10
Topics
bull When to use chart review for research
bull Confirm data in VA administrative datasets
bull Capture data not available in VA administrative datasets
bull Local vs Central Chart Review
bull Costaccuracy vs local chart reviews
bull Using notes in the CDW
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
11
8792
7102016
Confirm data in VHA administrative datasets
bull Validate case ascertainment strategies
bull Assess the accuracy of your administrative data-based
case ascertainment or outcome assessment and adjust if
needed for a retrospective administrative data analysis
bull Chart review serves as ldquocriterion standardrdquo
bull TIA cohort identified
by ICD-9 TIA code
in ED or hospital
discharge
bull Minor stroke cohort
identified by ICD-9
hospital discharge
codes and other
administrative data
(clinical severity
data not available)
12
Type of error
(of 1948 eligible admissions)
Reason
False negative (13) 1- discharge date incorrect
12-admin data did not
capture consult
Confirm data in VHA administrative datasets
Assess clinical vs administrative completion of an action
Example How many patients received a rehabilitation consult during their
stroke admission
bull Joint Commission stroke quality indicator included in VA facility SAIL reports
bull Research question How accurate are administrative measures of this quality indicator
bull Administrative data includes completed consult information
bull However consults may be completed without an actual rehab evaluation
taking place or may be done without administrative documentation
If infrequent it may not be a
problem but assessing a random
sample of cases may be important
to understand the variability in your
estimate of the of patients
receiving this care
False positive (12) 1- consult after discharge
3- patient not seen
4- consult cancelled
4- consult was for
diagnostic test only
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
9
Project examples
bull INSPIRE SDP (L Williams PI)
bull CARE TIMe SDP (D Bravata PI)
bull Operational projectsOffice of Clinical Analytics and Reporting
References
bull Williams LS et al A cluster-randomized quality improvement study to
improve two inpatient stroke quality indicators BMJ Qual Safety 2015
doi 101136bmjqs-2015-004188
bull Phipps M et al Validation of stroke meaningful use measures in a
national EHR system J Gen Internal Med 201631(Suppl 1)S46-52
bull Bravata D Myers L Cheng E et al Quality of Care for Veterans with
TIA and Minor Stroke Stroke 201546(Supp 1)ATMP73
7102016
10
Topics
bull When to use chart review for research
bull Confirm data in VA administrative datasets
bull Capture data not available in VA administrative datasets
bull Local vs Central Chart Review
bull Costaccuracy vs local chart reviews
bull Using notes in the CDW
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
11
8792
7102016
Confirm data in VHA administrative datasets
bull Validate case ascertainment strategies
bull Assess the accuracy of your administrative data-based
case ascertainment or outcome assessment and adjust if
needed for a retrospective administrative data analysis
bull Chart review serves as ldquocriterion standardrdquo
bull TIA cohort identified
by ICD-9 TIA code
in ED or hospital
discharge
bull Minor stroke cohort
identified by ICD-9
hospital discharge
codes and other
administrative data
(clinical severity
data not available)
12
Type of error
(of 1948 eligible admissions)
Reason
False negative (13) 1- discharge date incorrect
12-admin data did not
capture consult
Confirm data in VHA administrative datasets
Assess clinical vs administrative completion of an action
Example How many patients received a rehabilitation consult during their
stroke admission
bull Joint Commission stroke quality indicator included in VA facility SAIL reports
bull Research question How accurate are administrative measures of this quality indicator
bull Administrative data includes completed consult information
bull However consults may be completed without an actual rehab evaluation
taking place or may be done without administrative documentation
If infrequent it may not be a
problem but assessing a random
sample of cases may be important
to understand the variability in your
estimate of the of patients
receiving this care
False positive (12) 1- consult after discharge
3- patient not seen
4- consult cancelled
4- consult was for
diagnostic test only
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
10
Topics
bull When to use chart review for research
bull Confirm data in VA administrative datasets
bull Capture data not available in VA administrative datasets
bull Local vs Central Chart Review
bull Costaccuracy vs local chart reviews
bull Using notes in the CDW
bull Planning and conducting chart review studies
bull Examples of VA EHR tools for chart review
bull Lessons learned
bull Additional Resources
7102016
11
8792
7102016
Confirm data in VHA administrative datasets
bull Validate case ascertainment strategies
bull Assess the accuracy of your administrative data-based
case ascertainment or outcome assessment and adjust if
needed for a retrospective administrative data analysis
bull Chart review serves as ldquocriterion standardrdquo
bull TIA cohort identified
by ICD-9 TIA code
in ED or hospital
discharge
bull Minor stroke cohort
identified by ICD-9
hospital discharge
codes and other
administrative data
(clinical severity
data not available)
12
Type of error
(of 1948 eligible admissions)
Reason
False negative (13) 1- discharge date incorrect
12-admin data did not
capture consult
Confirm data in VHA administrative datasets
Assess clinical vs administrative completion of an action
Example How many patients received a rehabilitation consult during their
stroke admission
bull Joint Commission stroke quality indicator included in VA facility SAIL reports
bull Research question How accurate are administrative measures of this quality indicator
bull Administrative data includes completed consult information
bull However consults may be completed without an actual rehab evaluation
taking place or may be done without administrative documentation
If infrequent it may not be a
problem but assessing a random
sample of cases may be important
to understand the variability in your
estimate of the of patients
receiving this care
False positive (12) 1- consult after discharge
3- patient not seen
4- consult cancelled
4- consult was for
diagnostic test only
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
11
8792
7102016
Confirm data in VHA administrative datasets
bull Validate case ascertainment strategies
bull Assess the accuracy of your administrative data-based
case ascertainment or outcome assessment and adjust if
needed for a retrospective administrative data analysis
bull Chart review serves as ldquocriterion standardrdquo
bull TIA cohort identified
by ICD-9 TIA code
in ED or hospital
discharge
bull Minor stroke cohort
identified by ICD-9
hospital discharge
codes and other
administrative data
(clinical severity
data not available)
12
Type of error
(of 1948 eligible admissions)
Reason
False negative (13) 1- discharge date incorrect
12-admin data did not
capture consult
Confirm data in VHA administrative datasets
Assess clinical vs administrative completion of an action
Example How many patients received a rehabilitation consult during their
stroke admission
bull Joint Commission stroke quality indicator included in VA facility SAIL reports
bull Research question How accurate are administrative measures of this quality indicator
bull Administrative data includes completed consult information
bull However consults may be completed without an actual rehab evaluation
taking place or may be done without administrative documentation
If infrequent it may not be a
problem but assessing a random
sample of cases may be important
to understand the variability in your
estimate of the of patients
receiving this care
False positive (12) 1- consult after discharge
3- patient not seen
4- consult cancelled
4- consult was for
diagnostic test only
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
12
Type of error
(of 1948 eligible admissions)
Reason
False negative (13) 1- discharge date incorrect
12-admin data did not
capture consult
Confirm data in VHA administrative datasets
Assess clinical vs administrative completion of an action
Example How many patients received a rehabilitation consult during their
stroke admission
bull Joint Commission stroke quality indicator included in VA facility SAIL reports
bull Research question How accurate are administrative measures of this quality indicator
bull Administrative data includes completed consult information
bull However consults may be completed without an actual rehab evaluation
taking place or may be done without administrative documentation
If infrequent it may not be a
problem but assessing a random
sample of cases may be important
to understand the variability in your
estimate of the of patients
receiving this care
False positive (12) 1- consult after discharge
3- patient not seen
4- consult cancelled
4- consult was for
diagnostic test only
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
13
Other types of errors found when confirming administrative data via chart review
bull Incorrect ICD-9ICD-10 code used
bull Admission with stroke ICD9 primary discharge code in VA
administrative data is actually for an episode of non-VA care paid
for by the VA
bull A medication noted as given in VA Bar Code Medication
Administration (BCMA) data has a note entered that says ldquoheld patient off floorrdquo
bull An outpatient medication is active electronically but provider note
records instruction to stop the medication
7102016
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
14
Capture data not available in VA administrative
datasets
bull Unstructured data
bull Written orders
bull Comment fields
bull Data elements that reflect complex aspects of care
bull Discussion of comfort care or advanced directives
bull Coordination of care between providers
bull Data elements that reflect clinician judgment
bull Documentation of reasons of providing or not providing care (patient
declines treatment ineligibility etc)
bull Scanned records (many VA and non-VA examples)
7102016
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
Example of need for unstructured data in quality measurement
VTE prophylaxis by hospital day 2 flowchart
4 Were appropriate
medications or
prophylaxis given by
2 Was the patient
ambulatory by hospital day
(HD) 2 3 Were ldquocomfort measures
onlyrdquo documented by HD2
1 Was the patient
hospitalized for 2 days
5 Were any contraindications to
meds and mechanical
prophylaxis recorded by the
15
mechanical
HD2
provider
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
969
527
977
737
993
938
978
747
Using chart review to develop electronic clinical quality
measures for use in clinical quality management N
STK-1
VTE Prophylaxis Denominator
2130 993 995
333
Numerator 2113 864 995
506
STK-5
Antithrombotic by
hospital day 2
Denominator 2130 987
Numerator 2036 984 991
852
STK-10
Consider for
rehabilitation
Denominator 2130 975
Numerator 1948 993
STK-2
Antithrombotic at
discharge
Denominator 2130 976
Numerator 1948 983 984
195
NIHSS by 24 hours Denominator
2130 997 1000
364
Numerator 2126 987 969
988
Compared to
chart review
Only one lt 975
matched due to
non-standard
namesorders for
mechanical
devices
16
matched PPVNPV
7102016
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
17
7102016
Using chart review to develop electronic quality measures for use in implementation trials
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
18
Strategies for Chart Review Local vs Central
bull Local (CPRS) chart review
bull Effective if few sites
bull Best for simple chart review questions
bull Example Retrospective cohort study of Veterans at two VAMCs that received sleep apnea screening post-stroke
bull Small (two VA facilities)
bull Focused (diagnostic case ascertainment receipt of sleep apnea screening within a given time frame)
bull Chart review needed because completion and results of non-VA consults not easily tracked with administrative data
bull Central (national EHR tools) chart review
bull Cheaper for large studies
bull Optimal training and quality control
7102016
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
19
Chart review expense example
bull INSPIRE SDP
bull 11-site cluster randomized study reviewing 25 years of stroke admissions
bull Primary outcome performance on various stroke quality indicators (were patients eligible to receive a process of care and did they receive it)
bull Site level volume approximately 75 stroke admissions per year
bull If prospectively reviewing cases volume small
bull Difficult to find sites willing and able to hire some small of a research assistant to conduct the chart reviews
bull 11 staff at 025 FTE vs 3 full time central staff
bull Still need central EHR review to assess local chart review accuracy due to complexity of review
bull Training maintaining retaining the off-site personnel over a 3-year study is not feasible
7102016
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
20
7102016
ICC_1k kapp 1
Central Chart Review Quality Example
bull INSPIRE SDP
bull 1600 admissions with full chart review
bull Random 10 inter-rater reliability (~ 160 admissions)
bull Essential to track inter-rater reliability throughout the project to detect and
address variability in reviews
bull 118 variable chart review form 11 quality indicators
bull 113118 variables gt 08 ICCkappa
bull QI result agreement (ineligible passed failed) excellent k = 084-096
1
096
098
085 09
095
094 08
09
092
065 07
075
06
Gen
der
co
mfo
nly
non
str
dx6
dca
gg
dos
dcd
ocm
ed
dclm
hdo
s
dcth
rom
6
dcw
arf
re
lipda
tdc
dyscrn
ty
norx
pro
tpa
co
n2
c
wic
thro
m1
wic
thro
m8
pasth
x6
pasth
x1
6
pasth
x2
4
pasth
x3
1
pasth
x9
_1
1
atte
nt
lta
rm
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
bull Subscribe by visiting httpvawwvirecresearchvagovSupportH
SRData-Lhtm (VA Intranet)
bull Individualized support
virecvagov
(708) 202-2413
7102016
47
Contact information
Linda S Williams MD
lindawilliams6vagov
Co-PI Precision Monitoring (PRIS-M) QUERI
HSRampD Center for Health Information and Communication
Professor of Neurology Indiana University School of Medicine
Indianapolis IN
VHA Joint Legacy Viewer Team
Amanda Cournoyer
AmandaCournoyervagov
Jerry Rutherford
jeraldrutherfordvagov
7102016
48
Database amp Methods Cyberseminar Series
Next session August 7 2017
1 PM ET
Applying Comorbidity Measures Using VA and CMS (MedicareMedicaid) Data
Denise M Hynes MPH PhD RN
Director amp Research Career Scientist
VIReC amp CINCCH
Department of Veterans Affairs Hines IL
Professor College of Medicine amp School of Public Health
University of Illinois Chicago IL
21
Using notes in the CDW
bull TIU (Text Integration Utilities) notes are available in the CDW
bull These notes can be useful for extracting unstructured (text) data but also have some disadvantages
bull Not inherently chronologically oriented (requires data manipulation if sequence of notesinformation was important)
bull Date and time of note entry is stripped from TIU notes so you canrsquot see this when you view them
bull Need data management skills (SQL) to manipulate although some tools are being developed to aid in human-assisted review of text strings in TIU notes as well as for annotation in NLP projects
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this
CAPRI(Compensation and Pension Record Interchange)
bull Developed to facilitate coordination between the Veterans Benefit Administration (VBA) and the Veterans Health Administration (VHA) in the determination of Veteran benefits
bull Read-only access to EHR data for individual patients at one specific site at a time
bull Requires special software
bull In the ldquoClinical
Documentsrdquo view
click on the ldquoNotesrdquo
tab
bull All notes from one
facility are shown in
chronological order
Other tabs at the bottom have
specific data categories similar
to CPRS
33
Example of notes in CAPRI
34
Notes are searchable for text
Example looking for documentation
of carotid artery stenosis
bull The search term ldquocarotidrdquo (bottom right corner) returns notes containing this