Aligning Measurement-Based QI with Evidence-Based Practice Implementation Richard Hermann, MD, MS Associate Professor of Medicine and Psychiatry Tufts.

Aligning Aligning

Measurement-Based QI with Measurement-Based QI with

Evidence-Based Practice ImplementationEvidence-Based Practice Implementation

Richard Hermann, MD, MS

Associate Professor of Medicine and PsychiatryTufts University School of Medicine

Center for Quality Assessment & Improvement in Mental Healthat Tufts-New England Medical Center

www.cqaimh.org

2

OverviewOverview

How does evidence-based practice implementation (EBPI) relate to measurement-based quality improvement (MBQI)?– different paradigms

– similarities and differences

Potential for convergence & synergy What obstacles need to be addressed? Current research study on QI

3

Evidence-Based PracticesEvidence-Based Practices

EBP Rating

ACT / ICM AEvidence-based Psychotherapies AFamily Psychoeducation ASupported Employment AIntegrated Dual Diagnosis Treatment AMedication Management AMulti-Systemic Therapy A

A = RCTs B = less rigorous studies C = consensus or opinion

4

EBP Implementation: a Top-Down ModelEBP Implementation: a Top-Down Model

Research: Controlled trial of clinical intervention↓

Development: Codification of EBP by experts↓

Commercialization: Packaging: tools, scales, materials↓

Diffusion: Social marketing, training, support↓

Adoption: Local provider organizations↓

Consequences: Change to practice & outcomes

Rogers, Diffusion of Innovations, 2003

5

Measurement-Based QIMeasurement-Based QI

A “bottom-up” model– Activities conducted by local provider organizations– Influenced by external groups

MBQI is in wide use: – 90-98% of hospitals report formal programs

MBQI is costly: – estimated cost ~$200,000 per hospital per year

6

Principles of Measurement-Based QIPrinciples of Measurement-Based QI

Quality as problems in “processes”

Measurement & analysis

Broad participation

Inductive reasoning

Trial and error

Intervene Measure Plan Diagnose

Aim

Model for Measurement-based QIModel for Measurement-based QI

8

Commonalities between MBQI and EBPICommonalities between MBQI and EBPI

Both address important problems—some overlap

Both employ measurement – MBQI: rates of EBP use, appropriateness– EBPI: fidelity to evidence-based model

Both start with an understanding of underlying processes– MBQI: determined locally, informed externally– EBPI: studied externally, expanded locally

Both involve systematic intervention to change practice– MBQI: determined locally, informed by research & experience– EBPI: developed by experts, customized to local circumstances

9

Potential for MBQI to Enhance Potential for MBQI to Enhance Evidence-Based Practice ImplementationEvidence-Based Practice Implementation

Promotes local organizational development– system perspective– team work– analytic skills– experience implementing change

Increases awareness of gapsPrompts investigationMotivates exploration of available interventions

→ Potential for uptake of EBPs

Integrating MBQI with EBPI Requires Integrating MBQI with EBPI Requires Alignment Across Healthcare SystemAlignment Across Healthcare System

Environment

(eg, payers, accreditors)

Local Organization

(eg, hospital)

Micro-system

(eg, hospital inpatient unit)

11

Conditions for Successful AlignmentConditions for Successful Alignment

1. Local organizations need to select QI objectives that address gaps between actual & evidence-based practice

2. External organizations mandating measures also need to emphasize measures of EBPs

3. Microsystems within local organizations need to execute these QI activities effectively

12

1. Do Quality Measures Used for Local MBQI 1. Do Quality Measures Used for Local MBQI Address Evidence-Based Practices? Address Evidence-Based Practices?

Reviewed measures developed for mental health QI

308 measures identified & evaluated:– 9% supported by RCTs– 30% supported by less rigorous evidence– 61% not supported by evidence

Evidence-based measures less likely to be adopted

Pilot study of QI objectives adopted by MA hospitals:< 10% of hospital objectives address EBPs

National Inventory of Mental Health Quality Measures (www.cqaimh.org)

13

2. Do Mandated Quality Measures Address 2. Do Mandated Quality Measures Address Evidence-based Processes of Care?Evidence-based Processes of Care?

Measures established by:– Accreditor requirements – Government reporting requirements– Benchmarking collaboratives

Results increasingly linked to:– Pay for performance incentives – Public disclosure– Employer purchasing decisions

14

2. Do Mandated Quality Measures Address 2. Do Mandated Quality Measures Address Evidence-based Processes of Care?Evidence-based Processes of Care?

Illustrative Measures RatingRestraint / seclusion rates CElopement rate CInjury rate CNumber of medications CReadmission rate CMedication errors BAntipsychotic dose AAntidepressant Adherence AA = RCTs B = less rigorous studies C = consensus or opinion

15

Evidence-Based PracticesEvidence-Based Practices

EBP Rating

ACT / ICM AEvidence-based Psychotherapies AFamily Psychoeducation ASupported Employment AIntegrated Dual Diagnosis Treatment AMedication Management AMulti-Systemic Therapy A

A = RCTs B = less rigorous studies C = consensus or opinion

16

Meaningful

stakeholder needs

clinically important

evidence-based

valid

comprehensible

Feasible

precisely specified

data available

affordable

accurate

reliable

case mix adjustment

pt. confidentiality

Actionable

quality problem

under user’s control

interpretable

results

norms

benchmarks

standards

Domains of Process (prevention, detection, access, assessment, treatment, continuity, coordination,

safety/errors)

Clinical Population (diagnostic groups, comorbidities, prevalence, morbidity)

Vulnerable Groups (children, elderly, racial/ethnic minorities)

Modalities (medication, psychotherapy, other somatic, other psychosocial)

Clinical Setting (inpatient, ambulatory, residential, partial, emergency service)

Purpose of Measurement (internal QI, external QI, consumer selection, purchasing, research)

Level of Health Care System (population, plan, delivery system, facility, provider, patient)

Attributes Informing Quality-Measure Selection

Represent Mental Health System Broadly

Maximize Measure Attributes

17

Evidence-Based Objectives for Inpatient QI: SchizophreniaEvidence-Based Objectives for Inpatient QI: Schizophrenia

↑ use of antipsychotic drugs w/in recommended dose range

↓ use of multiple antipsychotics without adequate rationale

↑ % receiving adequate drug trials for refractory sx

↑ assessment/detection for EPS, akathisia or TD; ↑ rate of evidence-based treatment

↑ enrolled/referred to ACT among inpatients at high risk for relapse

↑ family members provided/referred to psychoeducation

↑ fidelity of inpatient psychoeducation program.

18

Evidence-Based Objectives for Inpatient QI: DepressionEvidence-Based Objectives for Inpatient QI: Depression

↑ use of antidepressant drugs w/in recommended dosage range

↑ assessment/detection of psychosis among depressed inpatients; ↑ use of adequate pharmacotherapy or ECT for psychotic depression

↓ use of anticholinergic antidepressants among depressed elderly inpatients

↑ % of inpatients w/ major depression referred to OP clinicians providing evidence-based psychotherapy

19

Other Evidence-Based Objectives for Inpatient QIOther Evidence-Based Objectives for Inpatient QI

↑ assessment & detection of medical conditions

↑ % receiving appropriate inpatient medical care, outpatient referral & communication between IP & OP clinicians

↑ assessment/detection of SUD; ↑ % receiving inpatient treatment & OP referral

20

3. Do Local Healthcare Organizations 3. Do Local Healthcare Organizations Execute QI Activities Effectively?Execute QI Activities Effectively?

Effectiveness in controlled trialsShortell (1998) reviewed 55 studies finding “pockets of

improvement” rather than evidence of widespread change

Effectiveness of routine QINot well studiedCase reports of successful initiativesAnecdotal evidence suggests much of local QI is ineffective

21

Macro Model of Organizational Predictors of QIMacro Model of Organizational Predictors of QI

Environment

Culture

Organizational Factors

Technical

Hospital QI Implementation

QI Outcomes

Stategic

Structure

Shortell, 1995

22

Predictors of QI ImplementationPredictors of QI Implementation

Cultural: beliefs, values & behaviors relative to QI+ organizational culture emphasizing teamwork & innovation+ commitment of senior managers & physicians

Structural: individual & group responsibilities+ Decentralized decision-making+ Longer experience + Greater number of teams & projects

Strategic: approach to QI+ “prospector” approach

Technical: resources + presence of organization-wide information systems

23

NIMH-funded study of 32 hospitals in MA & CA What are inpatient psychiatry units trying to improve?

– effectiveness -- patient-centered care– access -- safety– equity -- efficiency

To what extent do these objectives address EBPs?– Facilitators & barriers to adoption

To what extent do hospitals achieve measurable change? Hypothesis

– Fit between organization & predict QI effectiveness

Study of MBQI in Inpatient Psychiatric UnitsStudy of MBQI in Inpatient Psychiatric Units

Micro Model of Organizational Predictors of QIMicro Model of Organizational Predictors of QI

Environment

Culture

Organizational Factors

Resources

Selected Aims

& Measures

QI Progress

Diagnose

Measure Plan

Intervene

QI Outcomes

Leadership

Structure

Hermann, 2005

25

CultureCulture

Inpatient clinicians’ knowledge & beliefs about evidence basis for QI objective

Inpatient clinicians’ beliefs about the value of the QI objective to their patients’ care & outcomes

26

StructureStructure

Course of QI objective as tracer of organizational structure:– serial reports of results disseminated to inpatient clinicians?– are interventions attempted?– reports of progress (or barriers) to appropriate committees?– participation / coordination among necessary departments?

27

LeadershipLeadership

Selecting objectives that are priority of hospital

leaders?

Responsive to external pressures?

Leaders actively involved or monitoring progress?

28

ResourcesResources

Availability of resources for achieving QI objective– training – tools– time – support (eg, data collection & analysis)

29

ConclusionConclusion

Greater progress toward implementing EBPs may be achieved by aligning organizations’ QI activities with EBP goals

Components of alignment:– Provider organizations need to select evidence-based QI objectives – External groups need to reinforce emphasis on EBPs– Local MBQI needs to be more effective

Ongoing research aimed at:– understanding barriers to adopting evidence-based QI objectives– understanding organizational factors influencing QI progress– developing interventions to improve effectiveness of local QI