Funded by HRSA HIV/AIDS Bureau Selecting an Indicator & Collecting Performance Data Barbara M Rosa, RN-C, MS
Funded by HRSAHIV/AIDS Bureau
Selecting an Indicator & Collecting Performance Data
Barbara M Rosa, RN-C, MS
National Quality Center (NQC)2
Objectives
Understand the criteria in selecting an indicator for improvement
Develop an indicator with numerator & denominator for a network Part D program or a single site Part D program
Identify data sources and develop a data collection plan &/or manual data collection tool
Understand sampling methodology
National Quality Center (NQC)3
Agenda
15 min—Overview of indicator development and data display/analysis
15 min—Individual programs develop an appropriate indicator with numerator & denominator defined.
10 min—Share indicators with group 10 min—Breakout in 2 or 3 groups: networks,
single sites, and/or programs with electronic data collection to develop a data collection plan.
10 min—Share plans with group
National Quality Center (NQC)4
What Is A Quality Indicator?
A quality indicator is tool to measure specific aspects of care and services that are optimally linked to better health outcomes while being consistent with current professional knowledge and meeting client needs.
National Quality Center (NQC)5
What Makes a Good Indicator?
Relevance Does the indicator affect a lot of people or programs? Does the indicator have a great impact on the
programs or patients/clients in your network or clinic?
Measurability Can the indicator realistically and efficiently be
measured given finite resources?
National Quality Center (NQC)6
What Makes a Good Indicator?
Accuracy/Validity Is the indicator based on accepted guidelines or
developed through formal group-decision making methods?
Improvability Can the performance rate associated with the
indicator realistically be improved given the limitations of your services and population?
National Quality Center (NQC)7
Define Indicator
Who is eligible to be evaluated? What part of this population should have
received the care being measured?
(Who should be counted in the denominator?) What part of those who should have received
the care did receive the recommended care? (Who should be counted in the numerator?)
National Quality Center (NQC)8
Example
Eligibility definition by the National HIVQUAL Project: HIV+ ambulatory patients with at least 2 medical visits
during the study period (i.e., 1/1/08 through 12/31/08)
with at least one visit in each six-month period (January-
June/July-December)
National Quality Center (NQC)9
Examples
Location: all sites, or only some? Gender: men, women, or both? Age: any limits? Client conditions: all HIV-infected clients, or only
those with a specific diagnosis? Treatment status?
National Quality Center (NQC)10
Base the indicator on guidelines and standards of care when possible
Include staff and consumers when
developing an indicator to create ownership Be clear in terms of patient / program characteristics
(gender, age, patient condition, provider type, etc.) Set specific time-frames in indicator definitions
Tips for Defining Indicators
National Quality Center (NQC)11
Types of Measures Outcome measures are the voice of the
customer
Outcome measures reflect the impact of our work on the care provided to our clients
Examples: CD4/VL, number of medical provider visits, incidence of disease rates, functional status indicators, mortality & morbidity rates, client/care team satisfaction rates, or cost of care delivered
National Quality Center (NQC)12
Types of Measures Process measures are the voice of
the system
Process measures tell us whether parts or steps in the system are working as planned
Examples: Waiting time for appointments, no show rates, percent of clients with self-management goals in place, the number of clients organized by acuity level in case management
National Quality Center (NQC)13
Types of Measures
Balance measures are used to monitor the effect one change might have on multiple parts of the system
Example: Mental Health screening & referral is improved. To prevent a potential negative outcome of the increase in referrals leading to lengthy waits for appointments, a balancing measure might be to monitor time between the referral and the mental health appointment
National Quality Center (NQC)14
Collecting & Managing Data
“Data In” characteristics: Data that requires manual chart
review should be completed as efficiently as possible (pull the chart only once)
Include the data collectors (e.g., HIV case managers) in the feedback process: this validates their work and improves data quality
Streamline potential duplication of data entry by systematic alignment
National Quality Center (NQC)15
Collecting & Managing Data
“Data out” characteristics: Data should be displayed clearly; graphs and
charts are effective means Data should be shared throughout the system
along with program goals and improvement activities
Data should be evaluated and acted upon as outlined in the QM Plan
Share data, as appropriate, with consumers, partners, funders, and other stakeholders
Use quality improvement data to inform operational and strategic planning efforts; alignment within the organization and system is smart business
National Quality Center (NQC)16
Constructing A Sample--Definitions
Sampling: allows teams to make inferences about a total population based on observations of a smaller subset of that group
Over sampling: may be required if measurement pertains exclusively to one group within a population
National Quality Center (NQC)17
Constructing A Sample--Methodology
Define the selection criteria Gender Age Patient condition Treatment status
Identify eligible cases Randomly select cases
National Quality Center (NQC)18
Constructing A Sample--Sampling Chart
Total Eligible Population
Minimum Total Records
Charts to Pull*
140-159 64 83
160-179 67 87
180-199 70 91
200-249 75 98
250-299 79 103
300-349 82 107
350-399 85 111
National Quality Center (NQC)19
Sampling Methodology
Create a numbered list of all eligible patients Go to web site: www.randomizer.org Click on link to Randomize Put in total number of eligible patients, e.g. 200 Put in number of sample size, e.g., 41 Click randomize now, and you will get a list of
41 random numbers between 1 and 200 You may have to do women and men separately
because of over sampling for women
National Quality Center (NQC)20
www.randomizer.org
National Quality Center (NQC)21
• How many sets of number would you like?• How many numbers per set?• Number range? From
To• Do you wish each number in a set to remain unique?• Do you wish to sort your outputted numbers?• How do you wish to view the output? • Randomize Now
www.randomizer.org
41
1
200
1
YES
NO
Place Markers Off
National Quality Center (NQC)22
Contact Information
National Quality Center (NQC)
212-417-4730212-417-4730NationalQualityCenter.orgNationalQualityCenter.org
HIVQUAL-US
212-417-4620212-417-4620HIVQUAL.orgHIVQUAL.org