Development and Testing of the AHRQ QI Toolkit for Hospitals Donna O. Farley, PhD Peter Hussey, PhD RAND Corporation.

Development and Testing of the AHRQ QI Toolkit for Hospitals

Donna O. Farley, PhDPeter Hussey, PhDRAND Corporation

Set of tools that hospitals can use to help improve performance in quality and patient safety

The AHRQ Quality Indicators (QIs) – Inpatient Quality Indicators (IQIs)– Patient Safety Indicators (PSIs)

Targeted to wide range of hospitals– Independent or system-affiliated– Varying quality improvement experience

What Is the Toolkit?

2

Toolkit Development

Developed through a Task Order in the AHRQ ACTION program

RAND partnered with UHC to develop and test the toolkit

AHRQ will continue toolkit support

3

Applicable for hospitals with differing knowledge, skills, and needs

Serves as a “resource inventory” from which hospitals can select tools

Different audiences for each tool (e.g., quality officer, finance officer, programmer)

How Hospitals Can Use the Toolkit

4

What Are the Quality Indicators?

Inpatient Quality Indicators – 28 indicators of quality in four sets– Volume, counts (6)– Mortality for conditions, rates (7)– Mortality for procedures, rates (8)– Utilization, rates (7)

Patient Safety Indicators – – 17 indicators and a composite indicator– Screen for adverse events for inpatients– Expressed as rates

5

The Development Process

Develop Alpha Toolkit• Identify tools to include• Develop draft tools

Field Test Alpha Toolkit

Revise and Finalize Toolkit for

Dissemination

Perform Evaluation• Improvement experiences• Usability of toolkit• Effects on QI values

6

Established principles to guide toolkit development

Reviewed literature to guide design Developed outline of toolkit based on

steps of a quality improvement process

Identified and developed specific tools for each step

Tool Development Steps

7

Technical Advisory Panel

8

Six-member panel Brought various skills and perspectives

– Hospital experience– Quality improvement– Relevant research skills

Provided guidance throughout toolkit development– Toolkit design principles– Content of the tools

Parsimony in tool choice and design Target the most important factors

for implementation Provide tools that offer most value

for a range of hospitals Readily accessible contents Enable hospitals to assess

effectiveness of their actions

Principles Guiding Toolkit Development

9

Structure of the Toolkit

Introduction and RoadmapA. Readiness to ChangeB. Applying QIs to the Hospital DataC. Identifying Priorities for Quality ImprovementD. Implementation MethodsE. Monitoring Progress and Sustainability

of ImprovementsF. Return-on-Investment AnalysisG. Existing Quality Improvement Resources

10

The Roadmap

A navigational guide through the toolkit For each tool, it summarizes:

– Action step being taken– Brief description of the tool– Key audience(s) to use the tool– Position with lead role responsibility

11

Quality improvement collaborative Conducted by UHC 11 hospitals participated Structured implementation process

for improvements on the QIs Evaluation performed by RAND

Field Test Design

12

WHAT WE LEARNED

Evaluation Design

Six hospitals participated in evaluation Designed to learn:

– Hospital implementation strategies– Experiences in Improvement effort– Usefulness and usability of the tools

Data collection– Pre/post interviews– Regular update calls during study period– Three post-interviews in site visits

14

Overall, Positive Feedback

The tools were judged by the hospitals to be usable and useful

Hospitals varied widely in how many, and which, tools they chose to apply

Toolkit was useful for achieving staff consensus on the extent of quality gaps and on evidence-based practices

15

PSI 4: Death among surgical inpatients w/ serious complications

PSI 7: Central venous catheter-related bloodstream infection

PSI 12: Postoperative pulmonary embolism or deep vein thrombosis

PSI 13: Postoperative sepsis PSI 15: Accidental puncture/laceration PSI 19: Obstetric trauma-vaginal delivery

w/o instrument

All the Hospitals Chose to Address PSIs

16

Three Key Learnings

Hospitals need to trust their data Priority-setting is challenging Keep the tools short and simple

17

Need to Trust Your Data

The IQI or PSI rates have to be credible: “If we’re running reports over coding

information, we have to be mindful of coding issues before engaging medical staff. Need to be sure that we’re not wasting their time.”

18

Priority-Setting is Challenging

Many hospitals commented on prioritization:“It’s a great benefit to look at data and

explore it to see if it’s an issue… I don’t know if [hospitals] have the time to do that, unless it’s driven by corporate leadership or pay structure.

[There are] so many other things that we’re mandated to report and improve, it’s hard to look for something else.”

19

Keep Tools Simple

Users should be able to easily find the tools they need:

“People have so much going on that it’s hard…”

“I think we have to come up with simpler versions…”

20

Revised Toolkit To Address These Issues

Added a documentation and coding tool to improve PSI validity

Made prioritization matrix tools flexible so a hospital can tailor it with factors it considers in priority-setting

Simplified tools and instructions to increase usability

21

QUESTIONS?