1 Alternative Methods for Practice-Based Evidence Harnessing Natural Variation for Effectiveness Research Practice-Based Evidence for Clinical Practice.

1

Alternative Methods forAlternative Methods for Practice-Based Evidence Practice-Based Evidence

Harnessing Natural Variation for Harnessing Natural Variation for

Effectiveness ResearchEffectiveness Research

Practice-Based Evidence for Clinical Practice ImprovementPractice-Based Evidence for Clinical Practice Improvement

byby

Susan D. Horn, Ph.DSusan D. Horn, Ph.DInstitute for Clinical Outcomes ResearchInstitute for Clinical Outcomes Research

699 East South Temple, Suite 100 699 East South Temple, Suite 100 Salt Lake City, Utah 84102Salt Lake City, Utah 84102

801-466-5595 (V) 801-466-6685 (F)801-466-5595 (V) 801-466-6685 (F)[email protected] [email protected] www.isisicor.comwww.isisicor.com

http://www.isisicor.com/

2

Outline of PresentationOutline of Presentation

• Brief description of PBEBrief description of PBE--CPI, a practice-based CPI, a practice-based

evidence approach, evidence approach, and how it differs from and how it differs from

other study methodologiesother study methodologies

• PBEPBE-- CPI examples CPI examples showing breadth of showing breadth of

findings from comprehensive data setsfindings from comprehensive data sets

• Application to pressure ulcer prevention in LTCApplication to pressure ulcer prevention in LTC

3

Practice-Based Evidence for Practice-Based Evidence for Clinical Practice ImprovementClinical Practice Improvement Study DesignStudy Design

Analyzes the Analyzes the content and timingcontent and timing of individual of individual

steps of a health care process, in order to steps of a health care process, in order to

determine how to achieve:determine how to achieve:

• superior medical outcomessuperior medical outcomes for thefor the

• least necessary cost least necessary cost over theover the

• continuumcontinuum of a patient’s care of a patient’s care

4


Process FactorsProcess Factors•Management StrategiesManagement Strategies•InterventionsInterventions•MedicationsMedications

Patient FactorsPatient Factors•Psychosocial/demographic FactorsPsychosocial/demographic Factors•Disease(s)Disease(s)•Severity of Disease(s)Severity of Disease(s)

› physiologic signs and symptomsphysiologic signs and symptoms•Multiple Points in TimeMultiple Points in Time

OutcomesOutcomes•ClinicalClinical•Health StatusHealth Status•Cost/LOS/EncountersCost/LOS/Encounters

Improve/Standardize:Improve/Standardize:

Control for:Control for:

Measure:Measure:

5

Outcomes ResearchOutcomes Research

Uses large administrative databases to Uses large administrative databases to evaluate effectiveness ofevaluate effectiveness of

• specific treatment methodologiesspecific treatment methodologies

• medical technologiesmedical technologies

• providersproviders

6


• PBE-CPI goes beyond outcomes research byPBE-CPI goes beyond outcomes research by

- identifying outcomesidentifying outcomes- examining detailed process stepsexamining detailed process steps- adjusting for severity of illnessadjusting for severity of illness

7


• PBE-PBE-CPI goes beyond guidelines, which CPI goes beyond guidelines, which often areoften are- not decidablenot decidable: give a vague description of : give a vague description of

patients patients- not executablenot executable: give a menu of process : give a menu of process

steps to followsteps to follow- not connected to outcomesnot connected to outcomes

8

Example from Stroke GuidelineExample from Stroke Guideline

Stroke patients with diagnosed depression Stroke patients with diagnosed depression

should be offered a course of treatment with should be offered a course of treatment with

antidepressant drug therapy.antidepressant drug therapy.

9

Practice-based Evidence for Clinical Practice ImprovementPractice-based Evidence for Clinical Practice Improvement (PBE-CPI) compared to(PBE-CPI) compared to

Randomized Controlled Trial (RCT)Randomized Controlled Trial (RCT)

PBE-CPIPBE-CPII. Select Key Conditions I. Select Key Conditions

to Studyto Study

RCTRCTI. Define StudyI. Define Study

10



PBE-CPIPBE-CPI

II. Data CollectionII. Data Collection A. Patient VariablesA. Patient Variables

- - Patient eligibility and Patient eligibility and

stratification factorsstratification factors

- Use severity of illness to - Use severity of illness to

measure:measure:

- comorbidities- comorbidities

- disease severity- disease severity

- All patients qualify- All patients qualify

RCTRCT

II. Data CollectionII. Data Collection A. Patient VariablesA. Patient Variables

- - Patient eligibility andPatient eligibility and

stratification factorsstratification factors

- Eliminate patients who could- Eliminate patients who could

bias results:bias results:

- comorbidities- comorbidities

- more serious disease- more serious disease

~ 15% of patients qualify~ 15% of patients qualify

11



PBE-CPIPBE-CPIII. Data CollectionII. Data Collection

B. Process VariablesB. Process Variables

- - Methods for StabilizationMethods for Stabilization

- Measure all processes and use - Measure all processes and use analysis findings to develop analysis findings to develop protocol associated with better protocol associated with better outcomesoutcomes

RCTRCT

II. Data CollectionII. Data Collection

B. Process VariablesB. Process Variables

- - Treatment ProtocolTreatment Protocol

- Specify explicitly every- Specify explicitly every

important element of the important element of the

process of care for bothprocess of care for both

treatment and controltreatment and control

armsarms

12



PBE-CPIPBE-CPIIII. Data Analysis III. Data Analysis Outcome VariablesOutcome Variables

- Dynamic improvement - Dynamic improvement based on factbased on fact

IV. ResultIV. Result

- - Effectiveness researchEffectiveness research

RCTRCT

III. Data Analysis III. Data Analysis Outcome VariablesOutcome Variables

- Change based on fact- Change based on fact

IV. ResultIV. Result

- Efficacy research- Efficacy research

13

Practice-based Evidence for Clinical Practice ImprovementPractice-based Evidence for Clinical Practice Improvement (PBE-CPI)(PBE-CPI)

• PBE-CPI is a comprehensive analysis of PBE-CPI is a comprehensive analysis of patient, process, and outcome variablespatient, process, and outcome variables

• PBE-CPI studies are based on everyday PBE-CPI studies are based on everyday clinical practice, not controlled clinical practice, not controlled circumstances.circumstances.

14


• Led by trans-disciplinary team that Led by trans-disciplinary team that Develops and frames questionsDevelops and frames questionsGathers dataGathers dataInterprets findingsInterprets findingsImplements findingsImplements findings

• Results in more generalizable and transportable Results in more generalizable and transportable findingsfindings

15

PBE-CPI vs. RCTPBE-CPI vs. RCT

• RCTs are considered to be evidence of the RCTs are considered to be evidence of the highest grade.highest grade.

• Observational (CPI) studies are viewed as Observational (CPI) studies are viewed as having less validity because they having less validity because they reportedly over-estimate treatment reportedly over-estimate treatment effects.*effects.*

* New England Journal of Medicine 2000; (June 22, 2000) 342:1887-92.* New England Journal of Medicine 2000; (June 22, 2000) 342:1887-92.

16


Results from 2 NEJM studiesResults from 2 NEJM studies

““Average results of the observational Average results of the observational

studies were remarkably similar to those of studies were remarkably similar to those of

the randomized, controlled trials.”the randomized, controlled trials.”

* New England Journal of Medicine 2000; (June 22, 2000) 342:1878-92.* New England Journal of Medicine 2000; (June 22, 2000) 342:1878-92.

17


Results from JAMA StudyResults from JAMA StudyComparing results on 45 topics with binary Comparing results on 45 topics with binary

outcomes, found "very good correlation …between outcomes, found "very good correlation …between summary odds ratios of randomized and non-summary odds ratios of randomized and non-randomized studies"randomized studies"

r = 0.75, p < .001 for all studies, r = 0.75, p < .001 for all studies,

r = 0.83, p < .001 for prospective studies.r = 0.83, p < .001 for prospective studies.JAMA (Aug 2001) 286;7:821-830JAMA (Aug 2001) 286;7:821-830

18


ConclusionsConclusionsWell-designed observational studies do not Well-designed observational studies do not

systematically over-estimate the magnitudesystematically over-estimate the magnitude

of the effects of treatment as compared of the effects of treatment as compared

with those in randomized, controlled trialswith those in randomized, controlled trials

on the same topic.*on the same topic.** New England Journal of Medicine 2000; (June 22, 2000) 342:1887-92.* New England Journal of Medicine 2000; (June 22, 2000) 342:1887-92.

19

Assigned vs. Assumed CausalityAssigned vs. Assumed Causality

• Assigned causalityAssigned causality: RCT : RCT –Known confounders are eliminatedKnown confounders are eliminated

• Assumed causalityAssumed causality: PBE-CPI : PBE-CPI –No added confounders cause the significant No added confounders cause the significant

association to disappearassociation to disappear–A change in outcomes follows a change in A change in outcomes follows a change in

treatment as predicted by the PBE-CPI modeltreatment as predicted by the PBE-CPI model–Repeated studies on the same topic yield the Repeated studies on the same topic yield the

same findingssame findings

20

PBE-CPI and RCTPBE-CPI and RCT

RCTRCT

PBE-CPIPBE-CPI

Progenitor of Progenitor of RCTsRCTs

Practice effects Practice effects of RCT resultsof RCT results

21

Experimental DesignsExperimental Designs

Experimental designs are rarely Experimental designs are rarely

feasible to evaluate complex feasible to evaluate complex

interventions in the real world.interventions in the real world.

22

PBE-CPIPBE-CPI RCTRCT

•Hypothesis many and vagueHypothesis many and vague •Hypothesis clearHypothesis clear

•Alternatives not discreteAlternatives not discrete •Alternatives discreteAlternatives discrete

•Local knowledge Local knowledge contributescontributes

•Not depend on local Not depend on local knowledgeknowledge

23

PBE-CPI PBE-CPI RCTRCT

Dr. Don Berwick, HSR, April 2005Dr. Don Berwick, HSR, April 2005

•Confounders affect Confounders affect

outcomes and are outcomes and are

interestinginteresting

•Confounders not Confounders not interestinginteresting

•Effects largeEffects large •Effects smallEffects small

24

PBE-CPI offers PBE-CPI offers opportunities toopportunities to

•Focus on clinical realityFocus on clinical reality

•Discover best practicesDiscover best practices

•Test clinical questions and Test clinical questions and intermediate outcomesintermediate outcomes

•Gain some control over variable Gain some control over variable patient, process, and outcome data patient, process, and outcome data

25


• Connects outcomes with detailed process Connects outcomes with detailed process stepssteps

• Adjusts for severity of illnessAdjusts for severity of illness

26

Criteria for Selection Criteria for Selection of a Severity Indexing Systemof a Severity Indexing System

• Disease-specificDisease-specific

• Independent of treatmentsIndependent of treatments

• Comprehensive (i.e., all diseases)Comprehensive (i.e., all diseases)

• Clinically credibleClinically credible

• Able to measure severity at multiple points in the care Able to measure severity at multiple points in the care

processprocess

• Statistically valid in explaining costs/outcomesStatistically valid in explaining costs/outcomes

27

Comprehensive Severity Index Comprehensive Severity Index (CSI(CSI®®))

Severity SystemsSeverity SystemsDiagnostic/Procedure Based Diagnostic/Procedure Based

SystemsSystems

•AIM by IameterAIM by Iameter

•Disease Staging by MedStatDisease Staging by MedStat

•APR DRGs by 3mAPR DRGs by 3m

•Patient ManagementPatient Management CategoriesCategories

Physiologic/Clinically Based Physiologic/Clinically Based SystemsSystems

•ApacheApache

•Atlas by MediqualAtlas by Mediqual

CSICSI®®

28

Comprehensive Severity Index CSIComprehensive Severity Index CSI®®

Over 2,200 individual criteria subdivided into more than Over 2,200 individual criteria subdivided into more than 5,500 disease-specific groups5,500 disease-specific groups

No treatments used as criteriaNo treatments used as criteria

Computes disease-specific and overall severity levels on a Computes disease-specific and overall severity levels on a scale of 0-4 and continuousscale of 0-4 and continuous

Fixed times for inpatient reviewsFixed times for inpatient reviews- Admission review--first 24 hoursAdmission review--first 24 hours- Maximum review--any time during stayMaximum review--any time during stay- Discharge review--last 24 hoursDischarge review--last 24 hours

29

Pneumonia Criteria SetPneumonia Criteria Set480.0-486; 506.3; 507.0-507.1; 516.8; 517.1; 518.3; 518.5; 668.00-668.04; 997.3; 112.4; 136.3; 055.1480.0-486; 506.3; 507.0-507.1; 516.8; 517.1; 518.3; 518.5; 668.00-668.04; 997.3; 112.4; 136.3; 055.1

CATEGORY 1 2 3 4

Cardiovascular

pulse rate 51-100; STsegment changes-EKG;systolic BP 90mmHg

pulse rate 100-129; 41-50; PACs, PAT,PVCs-EKG;systolic BP 80-89mmHg

pulse rate 130; 31-40;systolic BP 61-79mmHg

pulse rate 30;asystole, VT, VF,V flutter;systolic BP 60 mmHg

Fever 96.8-100.4 and/or chills 100.5-102.0 oral; 94.0-96.7

102.1-103.9; 90.1-93.9 and/or rigors

104.0 90.0

LabsABGs

Hematology

pH 7.35-7.45

pO2 61mmHg

WBC 4.5-11.0K/cu mm;bands <10%;

pH >7.46 7.25-7.34

WBC 11.1-20.0K/cu mm;2.4-4.4K/cu mm;bands 10-20%

pH 7.10-7.24

pO2 51-60mmHg

WBC 20.1-30.0K/cu mm;1.0-2.3K/cu mm;bands 21-40%

pH 7.09;

pO2 50mmHg

WBC 30.1K/cu mm;1.0K/cu mm;bands 40%

Neuro Status

Lowest Glasgowcoma score

12

chronic confusion

9-11

acute confusion

6-8

unresponsive

5

Radiology ChestX-Ray or CTScan

infiltrate and/orconsolidation in 1lobe; pleural effusion

infiltrate and/orconsolidation in >1 but3 lobes;

infiltrate and/orconsolidation in >3lobes; cavitation orlung necrosis

Respiratory

white, thin, mucoidsputum

dyspnea on exertion;stridor; rales 50%/3lobes; decreased breathsounds 50%/3 lobes;positive for fremitus;stridorhemoptysis NOS;blood tinged or purulentor frothy sputum

cyanosis presentdyspnea at rest; rales>50%/ 3 lobes;decreased breathsounds >50%/ 3 lobes

frank hemoptysis

apneaabsent breath sounds>50%/ 3 lobes

Copyright 1998. Susan D. Horn. All rights reserved. Do not quote, copy or cite without permission.

30

How Does PBE-CPI Differ?How Does PBE-CPI Differ?

• Severity adjustment methodologySeverity adjustment methodology

• Three-dimensional, comprehensive Three-dimensional, comprehensive measurement framework: patient, process, and measurement framework: patient, process, and outcomesoutcomes

• Balance of rigorous science with a pragmatic Balance of rigorous science with a pragmatic operational focusoperational focus

31

Nursing Home Study (NPULS)Nursing Home Study (NPULS)1996-19971996-1997

• 6 long-term care provider organizations6 long-term care provider organizations

• 109 facilities109 facilities

• 2,490 residents studied2,490 residents studied

• 1,343 residents with pressure ulcer; 1,147 at risk1,343 residents with pressure ulcer; 1,147 at risk

• 70% female, 30% male70% female, 30% male

• Average age = 79.8 yearsAverage age = 79.8 years

Funded by Ross Products Division, Abbott LaboratoriesFunded by Ross Products Division, Abbott Laboratories

32

NPULS OutcomesNPULS Outcomes

• Developed pressure ulcersDeveloped pressure ulcers

• Healed pressure ulcers Healed pressure ulcers

• HospitalizationHospitalization

• Systemic infectionsSystemic infections

33

Effects of Nutritional SupportEffects of Nutritional Supportin Long Term Carein Long Term Care

Nutritional Treatment Strategies

N Pressure

Ulcer Develop Rate

Oral Supplement / Standard Medical Nutritional

134 21.6%

Enteral Formula 210 23.8%

Fluid Order 396 25.0%

Snacks, House Shakes 403 27.3%

No Nutritional Risk -- No Nutritional Treatment

195 27.2%

At Nutritional Risk -- No Nutritional Support

323 35.6%

34

Bladder Incontinence Management Bladder Incontinence Management in Long Term Carein Long Term Care

Treatments N PU Develop Rate

Incontinent-Use one or more of following treatments: 1,441 34.2% Briefs, disposable 501 23.6% Toileting program 549 23.9% Briefs, reusable 118 26.3% Topical Treatment 1,159 29.1% Bed pads, disposable 193 29.5% Bed pads, reusable 221 32.1% Use of catheter 195 51.3% Continent-No incontinence treatment 209 26.3%

35

Long-Term Care Residents with Agitation in DementiaLong-Term Care Residents with Agitation in DementiaRecommended PracticeRecommended Practice

• Use fewest number of medications possible Use fewest number of medications possible (OBRA 1987)(OBRA 1987)

• Minimize use of benzodiazepinesMinimize use of benzodiazepines

• Use atypical over typical antipsychoticsUse atypical over typical antipsychotics

• Use SSRIs over tertiary amine antidepressantsUse SSRIs over tertiary amine antidepressants

• Avoid combination therapyAvoid combination therapy

36

Medications from NPULS StudyMedications from NPULS Study

Optimal MedicationsOptimal MedicationsDementia & Agitation n = 803Dementia & Agitation n = 803

No Psych MedsNo Psych Meds 32.5%32.5% Anti-psychoticsAnti-psychotics 31.5%31.5% Anti-depressantsAnti-depressants 34.6%34.6% Anti-anxietyAnti-anxiety 34.9%34.9%

Combinations in 42% of treated residentsCombinations in 42% of treated residents

37

Medication Use and Outcomes for Elderly Medication Use and Outcomes for Elderly with Dementia with Agitationwith Dementia with Agitation

12.6**12.6** 12.3*12.3* 9.9**9.9**SSRI + AntipsychoticSSRI + Antipsychotic

24.0**24.0** 24.024.017.217.2MonotherapyMonotherapy

37.237.219.919.920.020.0No Psych MedicationsNo Psych Medications

% Pressure % Pressure UlcersUlcers

% Restraints% Restraints% Hospital + % Hospital + ERER

Medication Medication

Monotherapy includes antipsychotic only, antidepressant only, or antianxiety onlyMonotherapy includes antipsychotic only, antidepressant only, or antianxiety onlySSRI + antipsychotic medications concurrently. SSRI + antipsychotic medications concurrently. *p*p<<.05.05 **p**p<<.01.01

Horn, Drug Benefit Trends 2003; 15 (Supplement 1, December): 12-18Horn, Drug Benefit Trends 2003; 15 (Supplement 1, December): 12-18

38

Long Term Care CPI ResultsLong Term Care CPI ResultsOutcome: Develop Pressure UlcerOutcome: Develop Pressure Ulcer

GeneralGeneral AssessmentAssessment

IncontinenceIncontinenceInterventionsInterventions

Pressure ReliefPressure ReliefInterventionsInterventions

StaffingStaffingInterventionsInterventions

+ Age + Age 85 85

+ Male+ Male

+ Severity of Illness+ Severity of Illness

+ History of PU+ History of PU

+ Dependency in + Dependency in >= 7 ADLs>= 7 ADLs

+ Diabetes+ Diabetes

+ History of tobacco use+ History of tobacco use

+ Mechanical devices for + Mechanical devices for the containment of urine the containment of urine (catheters) (catheters)

- Disposable briefs- Disposable briefs

- Toileting Program- Toileting Program

+Static pressure +Static pressure reduction: protective reduction: protective device device

+Positioning: protective +Positioning: protective

devicedevice

- RN hours per resident - RN hours per resident day >=0 .25day >=0 .25

- CNA hours per resident - CNA hours per resident day >= 2day >= 2

-LPN hours per resident -LPN hours per resident day >=0.75day >=0.75

MedicationsMedications

- SSRI + Antipsychotic

39

NutritionalNutritional AssessmentAssessment

NutritionalNutritional InterventionsInterventions

+ Dehydration signs and + Dehydration signs and

symptoms: symptoms: low systolic blood low systolic blood

pressure, high temperature, pressure, high temperature,

dysphagia, high BUN, diarrhea, dysphagia, high BUN, diarrhea,

dehydration dehydration

+ Weight Loss: + Weight Loss: >=5% in last 30 >=5% in last 30

days or >=10% in last 180 daysdays or >=10% in last 180 days

- Fluid Order- Fluid Order

- Nutritional Supplements- Nutritional Supplements

• standard medicalstandard medical

- Enteral Supplements- Enteral Supplements

• disease-specificdisease-specific• high calorie/high high calorie/high protein protein

Horn et al, Horn et al, J. Amer Geriatr SocJ. Amer Geriatr Soc March 2004 March 2004

Long Term Care CPI ResultsLong Term Care CPI ResultsOutcome: Develop Pressure UlcerOutcome: Develop Pressure Ulcer

40

Integrate sustainable quality improvement into daily operationsIntegrate sustainable quality improvement into daily operations• Incorporate practice-based evidence for pressure ulcer prevention Incorporate practice-based evidence for pressure ulcer prevention

• Integrate into daily work versus ‘add-on’ projectIntegrate into daily work versus ‘add-on’ project

Focus on critical data elements and information flowFocus on critical data elements and information flow• Eliminate redundant documentationEliminate redundant documentation

• Reduce paperwork and streamline documentationReduce paperwork and streamline documentation

• Improve accuracy of informationImprove accuracy of information

• Improve communication among trans-disciplinary care teamsImprove communication among trans-disciplinary care teams

Translate documentation into data & data into trans-disciplinary Translate documentation into data & data into trans-disciplinary clinical reportsclinical reports

AHRQ Partnership for QualityAHRQ Partnership for QualityReal-time Optimal Care Plans for Nursing Home QIReal-time Optimal Care Plans for Nursing Home QI

41

ResultsResults Decrease Pressure Ulcer DevelopmentDecrease Pressure Ulcer Development on average 33%on average 33%

Increase Adherence to Best PracticesIncrease Adherence to Best Practices

Increase Staff Accountability and SatisfactionIncrease Staff Accountability and Satisfaction– Inclusion of front-line workers in QI effortsInclusion of front-line workers in QI efforts– Comprehensive documentation at point of care more completeComprehensive documentation at point of care more complete– Communication among care team improvedCommunication among care team improved

Reduce InefficienciesReduce Inefficiencies– # documentation forms for CNAs decreased 50% or more# documentation forms for CNAs decreased 50% or more– CNA time looking for documentation book decreasedCNA time looking for documentation book decreased– Time to compile reports for State Regulators and MDS decreasedTime to compile reports for State Regulators and MDS decreased– Time for Wound RN to summarize and report data decreasedTime for Wound RN to summarize and report data decreased

Improve State Survey ProcessImprove State Survey Process Establish a foundation for EMREstablish a foundation for EMR

42

Q4 03 (Pre-Implementation) to Q3 05 (Post-Intervention Review) Combined Facilities Average

0.0

5.0

10.0

15.0

20.0

% H

igh

Ris

k R

esid

en

ts

Facilities Average 14.0 13.0 12.9 10.6 9.6 9.4 12.0 9.1 8.7

National Norm 14.0 14.0 14.0 13.0 13.0 13.0 14.0 14.0 13.0

Q3 03 Q4 03 Q1 04 Q2 04 Q3 04 Q4 04 Q1 05 Q2 05 Q3 05

Impact On Pressure Ulcer QMsImpact On Pressure Ulcer QMs

Source: CMS Nursing Home Compare; Facility QM data reports

The combined facilities’ average shows an overall reduction of 33% in the QM % of high risk residents with pressure ulcer from pre-implementation to initial post-implementation time periods

Combined Facilities

National Norm

Q4 03 – Q3 05% Change = - 33%

43

DEVELOP PRESSURE ULCER by RNDEVELOP PRESSURE ULCER by RN TimeTime

38.1%

31.8%

25.1%

9.4%

0%

5%

10%

15%

20%

25%

30%

35%

40%

<10 min 10 - <20 min 20 - <30 min 30 - <40 min

Chi-Square (6 df) = 50.86, p<.0001, n=1,376Chi-Square (6 df) = 50.86, p<.0001, n=1,376

% P

ress

ure

Ulc

ers

RN Time Per Resident Per Day

44

Value of NursesValue of Nurses

Parameter Parameter Estimate Chi-Square Pr > ChiSq Estimate Chi-Square Pr > ChiSq

ADLs_78 ADLs_78 0.28 0.28 4.68 4.68 0.0305 0.0305 CSI Severity 0.01 CSI Severity 0.01 18.19 18.19 <.0001 <.0001 MDS PU_hx MDS PU_hx 0.75 0.75 15.00 0.0001 15.00 0.0001 Wt loss Wt loss 0.34 0.34 6.04 0.0140 6.04 0.0140Oral_eat probOral_eat prob 0.39 0.39 9.33 0.0023 9.33 0.0023 Catheter Catheter 0.78 0.78 16.98 <.0001 16.98 <.0001Entcalpr Entcalpr -0.55 -0.55 6.77 0.0093 6.77 0.0093 Ent_dis Ent_dis -0.98 -0.98 6.00 0.0143 6.00 0.0143 Fluid order Fluid order -0.43 -0.43 8.43 0.0037 8.43 0.0037 RN 10-20m -0.41 RN 10-20m -0.41 7.84 7.84 0.0051 0.0051 RN 20-30m -0.62 RN 20-30m -0.62 13.12 13.12 0.0003 0.0003 RN 30-40m -1.86 42.82 RN 30-40m -1.86 42.82 <.0001 <.0001 CNA >2.25h -0.64 5.76 CNA >2.25h -0.64 5.76 0.0164 0.0164 LPN >=45m -0.64 LPN >=45m -0.64 8.74 8.74 0.00310.0031

C = 0.727C = 0.727

Logistic Regression: DEVELOP PU-- RN/LPN/CNA Time and Other EffectsLogistic Regression: DEVELOP PU-- RN/LPN/CNA Time and Other Effects

45

HOSPITALIZATION by RN TimeHOSPITALIZATION by RN Time

18.4%

11.1%9.6%

6.1%

0%

2%

4%6%

8%

10%

12%14%

16%

18%

20%

<10 min 10 - <20 min 20 - <30 min 30 - <40 min

Chi-Square (4 df) = 35.17, p<.0001, n=1,376Chi-Square (4 df) = 35.17, p<.0001, n=1,376

% H

osp

ital

izat

ion

RN Time Per Resident Per Day

46

Effects of RN TimeEffects of RN TimeHorn SD, et el. Assoc. between registered nurse staffing time and outcomes of long-stay nursing home Horn SD, et el. Assoc. between registered nurse staffing time and outcomes of long-stay nursing home residents. Amer J Nursing (Nov 2005 to appear)residents. Amer J Nursing (Nov 2005 to appear)

RN time of 30-40 min/resident/dayRN time of 30-40 min/resident/day

is associated withis associated with

• Fewer UTIs Fewer UTIs • Fewer catheterizationsFewer catheterizations• Less weight lossLess weight loss• Less decline in ADLsLess decline in ADLs

• More nutrition supplementsMore nutrition supplements

04/21/23 47

Cost of additional 30 min Cost of additional 30 min

RN care per resident day RN care per resident day

$472,814$472,814

Societal PerspectiveSocietal PerspectiveEconomic Value of NursesEconomic Value of Nurses

Dorr DA, Horn SD, Smout RJ.Dorr DA, Horn SD, Smout RJ. Cost analysis of nursing home registered nurse staffing times. J American Geriatrics Society 2005; 53:656-661 Cost analysis of nursing home registered nurse staffing times. J American Geriatrics Society 2005; 53:656-661

Savings in avoided PU treatment costSavings in avoided PU treatment cost

$242,426$242,426Savings in avoided hospitalizationsSavings in avoided hospitalizations

$518,627$518,627

Savings in avoided UTI costsSavings in avoided UTI costs

30,88230,882

Cost/Benefit Analysis of More RN TimeCost/Benefit Analysis of More RN Time$ Per 100 at-risk residents per year (FY2001 dollars)$ Per 100 at-risk residents per year (FY2001 dollars)

Assumptions: $1,727 wtd avg to treat PU across stages, Assumptions: $1,727 wtd avg to treat PU across stages,

$8,523 avg for Medicare hospitalization, $53,900K RN salary & FB/yr$8,523 avg for Medicare hospitalization, $53,900K RN salary & FB/yr

Net Savings $319,121Net Savings $319,121

48

Post-Stroke Rehabilitation StudyPost-Stroke Rehabilitation Study

STUDY QUESTIONSSTUDY QUESTIONS1.1. Which Which patient characteristicspatient characteristics are associated are associated

with improved post-stroke outcomes?with improved post-stroke outcomes?

2.2. Controlling for patient characteristics, Controlling for patient characteristics, which which treatment interventionstreatment interventions or or combinations are associated with improved combinations are associated with improved outcomes?outcomes?

49


STUDY QUESTIONSSTUDY QUESTIONS

3.3. What is the What is the optimal intensity and durationoptimal intensity and duration of various of various

post-stroke treatment interventions?post-stroke treatment interventions?

4.4. What What staffing characteristicsstaffing characteristics are associated with better are associated with better

outcomes?outcomes?

50

• PhysiciansPhysicians

• NursesNurses

• Social WorkersSocial Workers

• PsychologistsPsychologists

• Physical TherapistsPhysical Therapists

• Occupational TherapistsOccupational Therapists

• Recreation TherapistsRecreation Therapists

• Speech/Language Speech/Language PathologistsPathologists


Project Clinical TeamProject Clinical Team

51


Patient CharacteristicsPatient Characteristics

1,161 U.S. Patients1,161 U.S. Patients

52% Male: 58% White, 26% Black52% Male: 58% White, 26% Black

66.0 Mean age (18.6 - 95.5 yrs)66.0 Mean age (18.6 - 95.5 yrs)

52


Examples of OUTCOME VARIABLESExamples of OUTCOME VARIABLES

• Change in FIM scoreChange in FIM score

• Length of rehab stayLength of rehab stay

• Discharge disposition Discharge disposition

• ContractureContracture

• DeathDeath

• Deep vein thrombosisDeep vein thrombosis

• Major bleedingMajor bleeding

• Pulmonary embolismPulmonary embolism

• Pressure ulcerPressure ulcer

• PneumoniaPneumonia

53


Examples of PROCESS VARIABLESExamples of PROCESS VARIABLES

• MedicationsMedications

• Nutritional processNutritional process

• Pain managementPain management

• Time to first rehabTime to first rehab

• Intensity, frequency, and Intensity, frequency, and duration of PT duration of PT interventionsinterventions

• Intensity, frequency, and Intensity, frequency, and duration of OT duration of OT interventionsinterventions

• Intensity, frequency, and Intensity, frequency, and duration of SLP duration of SLP interventionsinterventions

• Other therapy Other therapy interventions and dosageinterventions and dosage

54


DEVELOP WORKABLE INSTRUMENTSDEVELOP WORKABLE INSTRUMENTS

MULTIPLE AXESMULTIPLE AXES

• FUNCTIONAL ACTIVITIESFUNCTIONAL ACTIVITIESTask-specific approachTask-specific approach

• THERAPY INTERVENTIONSTHERAPY INTERVENTIONSOrganized around core functional activitiesOrganized around core functional activities

• TIME SPENTTIME SPENTInterventions in activities, formal assessment, home evaluationInterventions in activities, formal assessment, home evaluation

55

• Pre-FunctionalPre-Functional

• Bed MobilityBed Mobility

• SittingSitting

• TransfersTransfers

• Sit-to-StandSit-to-Stand

• Wheelchair MobilityWheelchair Mobility

• Pre-gaitPre-gait

• GaitGait

• Advanced GaitAdvanced Gait

• Community MobilityCommunity Mobility• Intervention not related to Intervention not related to

functional activityfunctional activity


PTPT FUNCTIONALFUNCTIONAL ACTIVITIESACTIVITIES

56

Post-Stroke Physical Therapy FormPost-Stroke Physical Therapy Form

Patient ID:

S a m p l eTherapist:

Date of Therapy Session:

/ /

I N T E R V E N T I O N C O D E SNeuromuscular Interventions:01. Balance training02. Postural awareness03. Motor learning04. PNF05. NDT06. Gait with body weight support07. Involved upper extremity addressed08. Constrained induced movement therapyMusculoskeletal Interventions:09. Strengthening10. Mobilization11. PROM/Stretching12. Manual Therapy13. Motor ControlCardiopulmonary Intervention:14. Breathing15. Aerobic/Conditioning exercisesCognitive/Perceptual/Sensory Interventions:16. Cognitive training17. Perceptual training18. Visual training19. Sensory trainingEducation Interventions:20. Patient21. Family/Caregiver22. StaffEquipment Interventions:23. Prescription/Selection24. Application25. Fabrication

26. OrderingModality Interventions:27. Electrical Stimulation28. Biofeedback29. UltrasoundPet Therapy:30. Use of dog31. Use of other animalAssistive Device:32. Ankle dorsi flex assist33. Cane - Large base34. Cane - Small base35. Cane - Straight36. Crutches - Axillary37. Crutches - Forearm38. Crutches - Small base forearm39. Dowel40. Grocery cart41. Hemirail42. Ironing board43. KAFO44. Lite gait

45. Mirror46. Parallel bars

Pre-Functional Activity

Duration of Activity:Enter in 5 minute increments.

Interventions:Enter one intervention code per group of boxes.

Bed Mobility

Wheelchair Mobility

Sitting

Transfers

Sit-to-Stand

Pre-gait

Gait

Advanced Gait

Community Mobility

Intervention not related

Intervention #2 not related

Co-Treat:

No. of minutes:

47. Platform (parallel bars or FWW)

48. Standing frame49. Steps (various heights)50. Step ladder51. Swedish knee cage52. Swiss ball53. Tray table54. Walker - FWW55. Walker - Hemiwalker56. Walker - Rising Star57. Walker - Standard58. WheelchairOther:

59.

Area Involved/non-functional:60. Upper Extremity61. Lower Extremity62. Trunk63. Head/Neck

Disciplines:

Patient Assessment:Formal Assessment (initial, re-evaluation, discharge): minutes

Home Evaluation: minutes

Work Site Evaluation: minutes

Group Physical Therapy Time:PT Group/Dovetail: minutes

Patients Therapists Assistants

Enter the number of each that participated in the Group PT:

Aides/Techs Students

Physical Therapy Rehabilitation Activities

Physical Therapy Time:PT Assistant

minutes

PT Aide/Tech

minutes

PT Student

minutes

to functional activity

to functional activity

Time session begins:

:

Physical Therapist

minutes

6649

57

Predicting Outcomes Predicting Outcomes Severe Stroke - CMGs 112+114 Severe Stroke - CMGs 112+114

OutcomeOutcome LOSLOS Discharge Discharge HomeHome

Patient Variables alonePatient Variables alone RR2 2 = .39= .39 cc = .70= .70

Add Total PT&OT Time/LOSAdd Total PT&OT Time/LOS RR2 2 = .39= .39 cc = .70= .70

Instead add PT&OT - Specific Instead add PT&OT - Specific Activity Time/LOSActivity Time/LOS

RR2 2 = .61= .61 cc = .85= .85

58

Predicting Outcomes Predicting Outcomes Severe Stroke - CMGs 112+114 Severe Stroke - CMGs 112+114

OutcomeOutcome Discharge Discharge Total FIMTotal FIM

Discharge Discharge Motor FIMMotor FIM

Patient Variables alonePatient Variables alone RR2 2 = .64= .64 RR2 2 = .58= .58

Add Total PT&OT Time/LOSAdd Total PT&OT Time/LOS RR2 2 = .64= .64 RR2 2 = .59= .59

Instead add PT&OT - Specific Instead add PT&OT - Specific Activity Time/LOSActivity Time/LOS

RR2 2 = .78= .78 RR2 2 = .76= .76

59

Post-Stroke Rehabilitation Outcomes ProjectPost-Stroke Rehabilitation Outcomes ProjectOutcome: Increase in Motor FIM Outcome: Increase in Motor FIM CMGs 112+114

GeneralGeneral AssessmentAssessment

GeneralGeneralInterventionsInterventions

MedicationsMedications

PTPTInterventionsInterventions

–– Age Age

–– Adm Severity of IllnessAdm Severity of Illness

++ No dysphagia No dysphagia

+ Chronic confusion+ Chronic confusion

+ Monoplegia or Normal+ Monoplegia or Normal

– – Days onset to rehabDays onset to rehab

++ Rehab length of stay Rehab length of stay

+ Enteral feeding+ Enteral feeding

–– Old SSRIsOld SSRIs

– – Anti-ParkinsonsAnti-Parkinsons

–– ModafinilModafinil

+ Gait time in 1+ Gait time in 1stst 3 hrs 3 hrs

+ Adv gait time in 1+ Adv gait time in 1stst 3 hrs 3 hrs

–– Bed mobility time in 1Bed mobility time in 1stst 3 3 hrshrs

– – Wheelchair time Wheelchair time in 1in 1stst 3 3

hrshrs

General CareGeneral Care

OTOTInterventionsInterventions

+ Home mgt time + Home mgt time in 1in 1stst 3 hrs 3 hrs

+ Comm integr + Comm integr in 1in 1stst 3 hrs 3 hrs

–– Bed mobility time in 1Bed mobility time in 1stst 3 hrs 3 hrs

60


12 papers published in12 papers published in

Supplement to Archives of Physical Medicine and Supplement to Archives of Physical Medicine and

RehabilitationRehabilitationDecember 2005December 2005

Opening the “Black Box” of Stroke Rehabilitation Opening the “Black Box” of Stroke Rehabilitation

And What It Means for Rehabilitation ResearchAnd What It Means for Rehabilitation Research

8 additional papers in other journals8 additional papers in other journals

61

AssessmentAssessment SurgerySurgery Pain Pain ManagementManagement

NutritionNutrition Wound Wound ManagementManagement PT / RTPT / RT DischargeDischarge

CPI Model - Length of StayCPI Model - Length of StayBowel surgeryBowel surgery

+ Admission + Admission CSI CSI (severity)(severity)

+ COPD+ COPD

+ Bowel Prep- Go-+ Bowel Prep- Go-LightlyLightly

+ Skin to skin+ Skin to skin

+ Drain - JP+ Drain - JP

+ Drain - Penrose+ Drain - Penrose

+ Post-op TPN+ Post-op TPN

PROCESS OF CAREPROCESS OF CARE

+ Discharge + Discharge to SNFto SNF

62

Abdominal Surgery Nutrition StudyAbdominal Surgery Nutrition StudyDisease CSI ScoreDisease CSI Score

Intervention SubgroupIntervention Subgroup NN MeanMean

Early & SufficientEarly & Sufficient 4242 50.7 50.7

Not Early & Not SufficientNot Early & Not Sufficient 6161 49.349.3

Not Early & SufficientNot Early & Sufficient 2525 48.848.8

Early & Not Sufficient Early & Not Sufficient 55 55 41.841.8

63

Abdominal Surgery Nutrition StudyAbdominal Surgery Nutrition StudyNutrition CSI Score (Deaths and Transfers Removed)Nutrition CSI Score (Deaths and Transfers Removed)


Early & SufficientEarly & Sufficient 2929 9.8 9.8



Early & Not SufficientEarly & Not Sufficient 4343 7.77.7

64

Abdominal Surgery Nutrition StudyAbdominal Surgery Nutrition StudyLength of Stay (Deaths and Transfers Removed)Length of Stay (Deaths and Transfers Removed)




Early & Not SufficientEarly & Not Sufficient 4343 13.313.3

Early & SufficientEarly & Sufficient 2929 11.911.9

65

Abdominal Surgery Nutrition StudyAbdominal Surgery Nutrition StudyTotal Charges (Deaths & Transfers Removed)Total Charges (Deaths & Transfers Removed)


Not Early & SufficientNot Early & Sufficient 1313 39,88339,883Not Early & Not SufficientNot Early & Not Sufficient 3535 38,57838,578Early & Not SufficientEarly & Not Sufficient 3535 36,54236,542Early & SufficientEarly & Sufficient 2020 34,60234,602

Neumayer LA, et al. Journal of Surgical Research 2001;95:1:73-77Neumayer LA, et al. Journal of Surgical Research 2001;95:1:73-77

Intended and Unintended Intended and Unintended Consequences of HMOConsequences of HMO

Cost-Containment Strategies Cost-Containment Strategies

Results from the Managed Care Outcomes ProjectResults from the Managed Care Outcomes Project

American Journal of Managed CareAmerican Journal of Managed Care

March 1996March 1996

67

Main Study QuestionMain Study Question

““When one looks across multiple managed care When one looks across multiple managed care

organizations at a year’s worth of actual data on organizations at a year’s worth of actual data on

the care of thousands of typical patients treated by the care of thousands of typical patients treated by

their regular doctors, how is the amount of health their regular doctors, how is the amount of health

care services used associated with cost-care services used associated with cost-

containment efforts by the HMO?”containment efforts by the HMO?”

68

Managed Care Outcomes ProjectManaged Care Outcomes Project

This represented more This represented more

than:than:

•99,000 office visits99,000 office visits•480 emergency room visits480 emergency room visits•1,000 hospitalizations1,000 hospitalizations•240,000 prescriptions240,000 prescriptions

Length of study period:Length of study period:

•One yearOne year

Patient PopulationPatient Population::

Nearly 13,000 patientsNearly 13,000 patients

were included in the were included in the

study:study:

•1,309 - 3,938 patients1,309 - 3,938 patients

for each disease groupfor each disease group

•1,876 - 2,663 patients1,876 - 2,663 patients

studied at each HMO sitestudied at each HMO site

69


Patient variablesPatient variables•Severity of patientSeverity of patient

illnessillness

•Age and genderAge and gender

•Time in studyTime in study

•Number ofNumber of

physicians seen byphysicians seen by

patientpatient

Cost-ContainmentCost-ContainmentPractice VariablesPractice Variables•Second-opinionSecond-opinion requirementsrequirements•Strictness of site’sStrictness of site’s gatekeepergatekeeper•Strictness of case mgt.Strictness of case mgt.•Drug and visit co-paysDrug and visit co-pays•Restrictions of Restrictions of formularyformulary•Extent of genericExtent of generic drug usedrug use

HMO Site VariablesHMO Site Variables

•Physician paymentPhysician payment

methodmethod

•HMO profit statusHMO profit status

•GeographicalGeographical

locationlocation

Study controlled for patient, cost-containment practice, and HMO Study controlled for patient, cost-containment practice, and HMO site variablessite variables

70


FindingsFindings

With increased formulary restrictiveness, the study found:With increased formulary restrictiveness, the study found:

• More patient visits to physiciansMore patient visits to physicians

• More emergency room visitsMore emergency room visits

• More hospitalizationsMore hospitalizations

• Greater estimated cost of prescriptions per yearGreater estimated cost of prescriptions per year

• Greater number of prescriptions per yearGreater number of prescriptions per year

71


8.411.4 10.9

2426.3

8.8

16.9

29 28.3

35.8

18

34.639.1

48.446.1

0

5

10

15

20

25

30

35

40

45

50

Nu

mb

er

of

Pre

sc

rip

tio

ns

Pe

r P

ati

en

t P

er

Ye

ar

Number of Prescriptions for Asthma

Site 1(0%)Site 2(65.2%)Site 3(65.2%)Site 4(75%)Site 5(76.1%)

Low Severity Medium Severity High Severity

% Formulary Limitation

72


132 159

332 344

501

245

486587604

726628

858

1010

12851236

$0

$200

$400

$600

$800

$1,000

$1,200

$1,400

Co

st o

f P

resc

rip

tio

ns

Per

Pa

tien

t P

er Y

ear

Cost of Prescriptions for Arthritis

Site 1(0%)Site 2(42.5%)Site 3(47.5%)Site 4(55%)Site 5(62.5%)



73


1.92.3

3.0 3.33.6 3.8

2.9

3.9

5.0 5.0

6.8

6.2 5.9

8.4

11.0

9.5

11.7 11.9

0

2

4

6

8

10

12

Nu

mb

er

of

Vis

its

Pe

r P

ati

en

t P

er

Ye

ar

Number of Visits for Ulcers

Site 1(0%)

Site 2(12.5%)

Site 3(12.5%)

Site 4(25%)

Site 5(25%)

Site 6(37.5%)



74

Cost-Containment MeasureCost-Containment MeasureFormulary LimitationFormulary Limitation

Disease Group

OutcomesHypertension

n=3477Otitis Media

n=3862Arthritisn=2632

EpigastricPain/ Ulcer

n=1511

Asthman=1290

RxCount +0.280 (0.15) +0.270(0.79) +1.460 *** +1.600 ** +0.800 *

DisGpRxCt +0.270 (0.19) +2.390 ** +1.125 *** +1.000 (0.09) +0.550 (0.17)

RxCost +0.780 * +2.590(0.16) +1.420 * +2.280 * +0.420 (0.49)

DisGpRxCst +2.840 *** +2.650(0.84) +4.800 *** +3.790 ** +0.530 (0.55)

Visits +0.320 (0.35) -3.740 (0.09) +0.740(0.39) +2.130 ** +1.110 (0.31)

ED Visits +0.125 *** -1.280 *** +0.290 *** +0.287 *** +0.289 ***

Hosp Adms +0.419 *** -2.350 *** +1.020 *** +0.490 *** +0.431 **

*** indicates p <0.001

** indicates 0.001<p<0.01

* indicates 0.01<p<0.05

Shaded cells represent unexpected direction

+ indicates increased utilization

- indicates decreased utilization

75

Summary of FindingsSummary of Findings

Curtailing access to medications via Curtailing access to medications via

cost-control mechanisms can adverselycost-control mechanisms can adversely

affect other healthcare utilization:affect other healthcare utilization:

• Additional office visits for dose titration/ Additional office visits for dose titration/ monitoringmonitoring

• ER/hospital visitsER/hospital visits

• Concomitant medicationsConcomitant medications

and increase total healthcare costs. and increase total healthcare costs.

76

• Do not scientifically Do not scientifically prove causality of prove causality of underlying relationships underlying relationships - “Association is not - “Association is not causation.”causation.”

• Accuracy and Accuracy and completeness of current completeness of current documentationdocumentation

• Labor-intensive manual Labor-intensive manual data abstractiondata abstraction

• Incidence and type of test Incidence and type of test ordering and availability ordering and availability of information is not of information is not uniform across sitesuniform across sites

• Complexity of analysisComplexity of analysis

Limitations of PBE-CPI StudiesLimitations of PBE-CPI Studies

77

• The strength of an observational study The strength of an observational study

depends on the study’s ability to control for depends on the study’s ability to control for

patient differences that would otherwise be patient differences that would otherwise be

addressed through randomization. There is addressed through randomization. There is

always the chance that some unknown critical always the chance that some unknown critical

variable may have been overlookedvariable may have been overlooked

Limitations of CPI StudiesLimitations of CPI Studies

78

Comprehensive Approaches to Care Save Comprehensive Approaches to Care Save 30% to 50% of Health Costs30% to 50% of Health Costs

Process FactorsProcess Factors•Management StrategiesManagement Strategies•InterventionsInterventions•MedicationsMedications

Patient FactorsPatient Factors•Psychosocial/demographic FactorsPsychosocial/demographic Factors•Disease(s)Disease(s)•Severity of Disease(s)Severity of Disease(s)

› physiologic signs and symptomsphysiologic signs and symptoms•Multiple Points in TimeMultiple Points in Time

OutcomesOutcomes•ClinicalClinical•Health StatusHealth Status•Cost/LOS/EncountersCost/LOS/Encounters

Improve/StandardizeImprove/Standardize

Control for:Control for:

Measure:Measure:

1 Alternative Methods for Practice-Based Evidence Harnessing Natural Variation for Effectiveness Research Practice-Based Evidence for Clinical Practice.

Documents