Measures in Health Services Research: From Constructs to Care Jacob Kean, PhD CCC-SLP Research Health Scientist, Center for Health Information and Communication, Roudebush VA Medical Center Research Scientist, Regenstrief Institute Assistant Research Professor, Department of Physical Medicine and Rehabilitation, Indiana University School of Medicine 1
35
Embed
Measures in Health Services Research: From Constructs to Care Jacob Kean, PhD CCC-SLP Research Health Scientist, Center for Health Information and Communication,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Measures in Health Services Research: From Constructs to Care
Jacob Kean, PhD CCC-SLP
Research Health Scientist, Center for Health Information and Communication, Roudebush VA Medical Center
Research Scientist, Regenstrief InstituteAssistant Research Professor, Department of Physical Medicine and
Rehabilitation, Indiana University School of Medicine
2
High-Level Motivation:• To improve the delivery,
quality, cost, access to, and outcomes of care.
Approaches:• Development and
implementation of measures
• Health information exchange
Contexts:• Medical and psychiatric
rehabilitation• Primary care
Constructs
Models
Interpretations
Actions
3
CONSTRUCTS: DELIRIUM/POST-TRAUMATIC AMNESIA
4
Construct: Delirium/PTA
• Post-traumatic amnesia (PTA)– Duration– Measured retrospectively, and later prospectively– Index of injury severity and prognosis
• Prospective measures operationalized the end point
• Prospective measures used to log severity, but poor construct validity when used that way
PTA/PTCS = DeliriumLipowski’s (1987) definition of delirium
Stuss et al. (1999) definition of PTCS
• “Delirium is a transient organic mental syndrome of acute onset, characterized by global impairment of cognitive functions, a reduced level of consciousness, attentional abnormalities, increased or decreased psychomotor activity, and a disordered sleep-wake cycle.”
• “A confusional state can be defined as a transient organic mental syndrome with acute onset characterized by a global impairment of cognitive functions with a concurrent disturbance of consciousness, attentional abnormalities, reduced or increased psychomotor activity, and a disrupted sleep-wake cycle.”
• Diagnostic accuracy – ROC analysis at cutoff (≤6) resulted in AUC=0.994
(35/36) classified accurately as referenced against the DSM-IV gold standard
• Duration– Correlation between DRS-R98 and DDT-Pro
duration estimates r = 0.975
9
Construct: Delirium/PTA
• Three construct-valid items can accurately – Identify the end of delirium/PTCS following TBI– Measure severity
• Representative papers– Kean & Ryan (2008)– Kean et al. (2010)– Seel et al. (2010)
10
MODELS: ITEM RESPONSE THEORY
11
Models: Item Response Theory
• Resistance to delirium construct in brain injury settings• Our simple, short measure (i.e., DDT-Pro) was limited
– Ordinal-level (nonparametric statistics)• Spurious interactions, underestimation of effect sizes, and impact
on gain scores.
– Person measures are sample-dependent – Focus on group-level metrics (e.g., reliability, error)– Scores obtained from different sets of items (measuring the
same construct) are not directly comparable– Few techniques for validating response patterns and
systematic variation
12
Models: Item Response Theory
• Build on existing PTA measure (Orientation Log)
• Incorporate IRT model to:– Allow better understanding of items and construct– Achieve interval-level measurement– Differentiate ability
Methods (Kean et al., 2011)
• 257 (321) ratings of 90 patients admitted for inpatient rehabilitation following TBI
• 48.25 years (SD 18.87; range from 17 to 93), 75% were male
• Twenty unique items from three scales were administered: O-Log, C-Log, DDT-Pro
• Analyses were conducted with WINSTEPS version 3.6 using a partial credit model
Hybrid Measure Items
Person-Item Maps
•Hybrid measure has improved targeting vs. O-Log
•Separation of both measures suffer due to the poor targeting of more impaired persons
16
Models: Item Response Theory
• IRT model-driven approach– Improves measurement quality and precision– Construct-relevant items contribute to
differentiation of person “ability”• Representative Papers– Kean et al. (2011a, 2011b)– Malec, Kean et al. (2012)– Kean, Malec et al. (2013)– McGuire, Kean et al. (2014)
17
INTERPRETATIONS: RESPONSIVENESS AND SENSITIVITY TO CHANGE
18
Patient-Reported Outcome Measurement Information System (PROMIS)
• The PROMIS assessment system is based on a comprehensive (i.e., physical, mental, social) self-reported health framework composed of many domains
• Domains are represented as unidimensional hierarchies of dozens of items, called “item banks”– Item banks include many items to represent fully the range
of impairment in a given domain• Developed using IRT
– Item banks that can be administered adaptively or assembled as static “short forms”
19
SCOPE Trial (Kroenke et al., 2014)
• Enrolled 250 veterans with moderate to severe and persistent musculoskeletal pain– Mean age of 55.1 years (range, 28 to 65)– 83% were men– Duration of pain was 1 year or longer in 98% of
participants • Tested a telemedicine/collaborative care
intervention• 244 patients who completed both baseline and 3-
month assessments
20
Comparative Responsiveness
• Measures– PROMIS Pain Interference Short Forms– Brief Pain Inventory– SF-36 Bodily Pain– Reference Standard – Patient-reported global
• Implementation (CFIR constructs)– Outer setting– Inner setting– Characteristics of individuals involved– Process
34
Process Factors• Patient Education and Management Strategies• Interventions and surgeries • Medications
Patient Factors• Psychosocial/demographic Factors• Co-occurring Conditions• Severity of Illness and Injury• Genetic information• Measured at Multiple Points in Time