“The public is concerned about hospital quality but has less information about selecting a hospital than they do when purchasing a toaster or a car” (Malone, 1999)
Mar 27, 2015
“The public is concerned about hospital quality but
has less information about selecting a hospital than
they do when purchasing a toaster or a car” (Malone, 1999)
“Puffery” “…representation, statement, or
conduct that clearly over-exaggerates…
…is not intended to be an offer to be relied upon.”
Consumer protection
(2000 Legal on-line Dictionary)
Methods
1. Solucient’s “Top 100 Hospitals”
2. US News & World Report’s “Best Hospitals”
3. American Nurses Credentialing Center’s “Magnet Recognition Program”
Descriptive statistics Spatial mapping Review of empirical literature
US News & World Report’s “Best” Hospitals
“If you are looking for the ‘best’ in medical care…”
—web-based guide.
“designed to assist you in your search for the highest level of medical care.”
(US News & World Report)
US News: Selecting the “Best” National Opinion Research Center Each July since 1990 Ranked by specialties (17) “Only rigorous assessment of all US hospitals” Eligible -- “tertiary care hospitals”
COTH membership or Affiliation with medical school or Availability of 9 out of 17 prescribed
technologies Hospital Quality Index (HQI) Honor Roll – score well in at least 6 specialties
US News continued…
Measures: produce HQI Structure = AHA Annual Survey data
• FTE “on-staff” Nurse to bed staffing ratios• Services available (technology index)
– # of specialty-specific technology services Process = (“proxy measure”)
• Physician nominations• Newest specialties reputation only
Outcomes = • Medicare risk-adjusted mortality by specialty• Pool most recent 3 years• Mortality ratio = # inpatient deaths/expected
Solucient’s “Top 100” Hospitals
“Identifies industry benchmarks by recognizing hospitals that demonstrate superior clinical, operational, and financial performance.”
Selecting the “Top 100” Since 1993 -- December “Management results that separate these
hospitals from the pack.” “Which hospitals offer superior care at lower
costs.” “Data obsession--common trait.” Enterprise = benchmark reports for purchase Eligibility restricted: 5 categories (n=20) Overall + select specialties (ICU, CV, Ortho, Stroke
benchmarks) Data source – Medicare Cost Report
Solucient “TOP” – 5 strata
Major Teaching (400 or more beds & resident to bed ratio of at least .25)
Teaching (at least 5 residents or resident to bed ratio of .01-.24)
Large Community (250 or more beds) Medium Community (100-249 beds) Small Community (25-99 beds)
“Top 100”selection… Ranked by strata on following measures—20 best rankings per
strata selected Financial management =
• Expense/adjusted discharge: case mix and wage adjusted
• Profitability• Proportion of outpatient revenues• Index of total facility occupancy
Clinical performance =• Medicare risk-adjusted mortality index• Risk-adjusted complication index
Operational performance =• Severity-adjusted average LOS• Case-mix and wage-adjusted expense per discharge
ANCC Magnet Recognition Program for Excellence in Nursing
(1994)
ANCC Recognizes “excellence” in 4 target areas
1. Management philosophy & practices
2. Adherence to standards for improving the quality of care
3. Leadership of the CNO in supporting professional practice and continued nursing competence
4. Attention to the cultural and ethnic diversity of patients, their significant others, and providers
Magnet Recognition Process
Self evaluation: ANA’s Scope & Standards ANA quality indicator data (NDNQI) Site visit Community input Could range from $30,000 - $75,000 Reevaluated every 4 years or more often with
leadership change
Continuous Process: submit data annually
Diffusion of “Magnetism”by year and total number recognized
0
20
40
60
80
100
120
1994 1996 1998 2000 2002 2004
“Creep” & Overlap - 2001 & 2003
Designation 2001 2003
Best 173 203
Top 101 101
Magnet 47 91
Best & Magnet 14 25
Best & Top 15 13
Top & Magnet 2 12
Best, Top & Magnet
1 6
Best, Top, & Magnet
1. University of CA Irvine2. Poudre Valley Hospital, Fort Collins
CO3. Mayo Clinic, Rochester MN4. NC Baptist, Wake Forest Un., NC5. Methodist Hospital, Houston, TX6. Inova Fairfax Hospital, Falls Church
VA
Summary – Best, Top & Magnet
Best (14yrs) Top (11yrs) Magnet (10yrs)
Measurement Framework
norm referenced norm referenced
criterion referenced
Focus structure, process, outcomes
outcomes structure, process
Self-nomination no no yes
Self-appraisal no no yes
Site visit no no yes
Nursing yes no yes
Purpose “inform consumer choice”
“excellence in management”
“excellence in nursing care”
Empirical evaluations
3 3 4
U.S. News “Best Hospitals” - 2003
Solucient’s “Top 100 Hospitals” - 2003
ANCC’s Magnet Hospitals - 2003
Best, Top, & Magnets - 2003
Regional dominance = “Pockets of excellence” ?? High preponderance – northeast, south 6 states – “empty” Mountain, west north central, west south
central regions Some states in highly designated regions
had fewer hospitals than those in the “empty regions” CA -- 450 hospitals (3 magnets) = .7% NJ – 74 hospitals (14 magnets) = 19%
The US NEWS “Best” -- 3 Empirically based publications
Different rankings when different decision models used (Teasley, 1996)
CABG outcomes in PA and NY (Hartz et al., 1997) - “Best did not perform better than rest”
Significantly lower AMI mortality rates (Chen et al. (1999)Clinical practice processesAsprin & beta blockers
Numerous reports of methodological “flaws” & conceptual issues What is examined Who does the nomination “best colleges” “best party schools”
Solucient “Top” -- 3 empirically based publications
1. CHEN et al., 1999• AMI patients over 65 Significantly lower LOS and
costs than “rest”• No significant difference in mortality rates
2. Griffith & Alexander 2002• Found that 8 of the Solucient measures of
hospital performance were valid and reliable
3. Griffith, Knutzen & Alexander, 2002• “a disjuncture” between “Top” outcome
measures and JCAHO evaluations in non-federal hospitals
ANCC Magnet Hospitals – 4 Empirically based publications
Aiken, Havens, & Sloane, 2000 support for nursing practice nurse reported quality nurse satisfaction better nurse staffing ratios more highly educated nurses nurse burnout
Havens, 2001 strong departments of nursing quality ratings organizational support for nursing practice difficulty recruiting nurses patient and family complaints
Upenieks, 2002, 2003 nurse satisfaction in ANCC magnet
hospitals -- linked to stronger nurse manager leadership
levels of RN reported empowerment
Havens, (Forthcoming) use of “outside” nurses vacancy rates
Comparison of JCAHO HospitalEvaluation Score Means
(2002 JCAHO file)
Hospital Type mean score
SD
All Community Hospitals (Federal, psych and other specialties eliminated)
90.84 4.43
U.S. News “Best” 91.66 3.68
Solucient “Top 100” 93.13 3.05
ANCC Magnet Hospitals 94.43 1.27
Hospitals with Multiple Designations 96.34 2.04
JCAHO “Commended” 96.94 1.61
Overall Evaluation ScoresComparative Data - 2002
Do JCAHO scores contribute to designation?
No statistically significant differences between designated hospitals and the rest on JCAHO scores (t-test)
Logistic regression models of designation (does JCAHO score predict designation or not?)
Unrelated to receiving best, top, or magnetHigher JCAHO scores only slightly (and not
significantly) predict being designated. Model fit is poor.
Havens, Vasey, & Kellogg 2003,Unpublished
What is the take home? Concern about the quality of hospital
care Adverse events and staffing deficiencies
cause alarm Scant information to assess performance No universal, “consumer friendly”
measure of quality Growing # of ratings / rankings Competition and marketing campaigns
AHRQ advice re: selecting quality care…
“Look for a hospital that: Is accredited by JCAHO.
Is rated highly by the state, consumer groups, or other organizations.”
http://ahrq.gov/
“PR Bonanzas”
“Americans have a love affair with rankings.” “Public relations goldmine” Marketing campaigns News media pick up Web-sites “Get a 2nd opinion from the ‘best’ hospitals” “Helps our development efforts” US News – “hottest marketing tool in the healthcare
industry” Financial boon for awarding organizations… And ….more
Research Implications Paucity of research Mixed findings Are the “Best really better than the rest”? Are some of the best better than other bests? Best on what? Is this meaningful to consumers - The big picture
paradigm? Reliance on structure and process only What are the key questions?
Safety Patient perspectives
Policy Implications A case for moving research to inform policy Consumer info, mis-info, or dis-info? As competition grows—release of comparative information will
increase “Desire to hold healthcare organizations accountable…need
accurate measures.” (Devers) How to uniformly gauge quality? Some argue that public does not understand or will not use…
the public is requesting “top 10” is part of American culture How to depoliticize – big $ maker for magazines Use of e-health sites is growing Being touted as a decision guide for consumers
Web sites Ladies Home Journal Reader’s Digest
We are on the job!
Stay tuned