MDS 3.0 Presenter: Debra Saliba, MD, MPH AHCA Webinar May 14, 2008 Making Assessment Work.

MDS 3.0MDS 3.0

Presenter: Debra Saliba, MD, MPHPresenter: Debra Saliba, MD, MPH

AHCA WebinarAHCA WebinarMay 14, 2008May 14, 2008

Making Making Assessment WorkAssessment Work

2 5/14/08

MDS 3.0 Information Web SitesMDS 3.0 Information Web Sites

MDS 3.0 general information site:MDS 3.0 general information site:  http://www.cms.hhs.gov/nursinghomequalityinits/25_nhqimds30.asphttp://www.cms.hhs.gov/nursinghomequalityinits/25_nhqimds30.asp  MDS 3.0 draft version--1/15/08MDS 3.0 draft version--1/15/08  http://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30DraftVersion.pdfhttp://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30DraftVersion.pdf  Timeline--MDS 3.0Timeline--MDS 3.0  http://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30Timeline.pdfhttp://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30Timeline.pdf  RE: PHQ-9 (one of many reference sites available)RE: PHQ-9 (one of many reference sites available)http://www.depression-primarycare.org/clinicians/toolkits/materials/forms/phq9/http://www.depression-primarycare.org/clinicians/toolkits/materials/forms/phq9/  Link to this and other webinars:Link to this and other webinars:http://www.ahcancal.org/events/Pages/Webinars.aspxhttp://www.ahcancal.org/events/Pages/Webinars.aspx

3 5/14/08

MDS 3.0 Development & TestingMDS 3.0 Development & Testing

Funded by:Funded by:

Centers for Medicare & Medicaid ServicesCenters for Medicare & Medicaid Services

Veteran’s Administration Health Services Veteran’s Administration Health Services Research and Development Research and Development

4 5/14/08

Learning ObjectivesLearning Objectives

By the end of the session, participants will be able toBy the end of the session, participants will be able to

1.1. Report main advances in MDS 3.0Report main advances in MDS 3.0

2.2. Describe how advances were identified and Describe how advances were identified and testedtested

3.3. Identify contribution of resident voice to Identify contribution of resident voice to improved assessment for cognition, mood, improved assessment for cognition, mood, preferences and painpreferences and pain

4.4. Discuss role of facility leaders in supporting Discuss role of facility leaders in supporting improved MDS 3.0 assessmentimproved MDS 3.0 assessment

5 5/14/08

Main Advances in MDS 3.0Main Advances in MDS 3.0

Gives Resident VoiceGives Resident Voice

Increases clinical relevanceIncreases clinical relevance

Increases accuracy (validity & reliability)Increases accuracy (validity & reliability)

Increases clarityIncreases clarity

Reduces time to complete by 45%Reduces time to complete by 45%

6 5/14/08

Why Resident Voice?Why Resident Voice?

CMS’s goal is to increase resident-CMS’s goal is to increase resident-centered care centered care Respect for individual voiceRespect for individual voice Fundamental to high quality & culture changeFundamental to high quality & culture change Residents and families want care to be Residents and families want care to be

individualized and accurateindividualized and accurate

Improves accuracy, feasibility, efficiencyImproves accuracy, feasibility, efficiency General, unfocused questions do not elicit General, unfocused questions do not elicit

meaningful reportsmeaningful reports Detailed daily observations of all behaviors for Detailed daily observations of all behaviors for

all residents is time consuming and not all residents is time consuming and not feasiblefeasible

How did we identify How did we identify and test these and test these

advances?advances?

The Evaluation Team Had The Evaluation Team Had 6 Sets of Players6 Sets of Players

Lead research and administrative teamLead research and administrative team RAND: Debra Saliba, MD, MPH RAND: Debra Saliba, MD, MPH Harvard: Joan Buchanan, PhDHarvard: Joan Buchanan, PhD Administrative Lead: Malia JonesAdministrative Lead: Malia Jones

National VA Nursing Home Research CollaborativeNational VA Nursing Home Research Collaborative Los Angeles, CALos Angeles, CA –– Atlanta, GA Atlanta, GA Philadelphia, PAPhiladelphia, PA – – Bedford, MABedford, MA

Lead Quality Improvement Organization: Lead Quality Improvement Organization: Colorado Foundation for Medical CareColorado Foundation for Medical Care

Instructions, Guides and Form Design: Instructions, Guides and Form Design: Carelink, RRS Consulting, Kleimann Communications Carelink, RRS Consulting, Kleimann Communications

GroupGroup

Centers for Medicare & Medicaid ServicesCenters for Medicare & Medicaid Services Workgroups, consultants, content expertsWorkgroups, consultants, content experts

MDS 3.0 Development Proceeded MDS 3.0 Development Proceeded in 4 Phasesin 4 Phases

Townhall Meeting & Open Comment

VA Validation Protocol Research

Expert Panel Meetings

CMS Revised Draft MDS 3.0 2003

2007

Phase 1: Stakeholder and Expert Feedback

Phase 2: MDS 3.0 Item Development

Integration of Phase 1 Feedback

Phase 2 Improved Key MDS Sections Phase 2 Improved Key MDS Sections and Revised MDS Itemsand Revised MDS Items

The VA Pilot Developed and Tested MDS Items in 8 AreasThe VA Pilot Developed and Tested MDS Items in 8 Areas

MoodMood

Behavior disorders Behavior disorders

Mental statusMental status

DeliriumDelirium

PainPain

FallsFalls

Quality of lifeQuality of life

Diagnostic codingDiagnostic coding

Funded by VA HSR&DFunded by VA HSR&D

Findings of VA Research Findings of VA Research

Depression Depression Self-report is feasible & efficientSelf-report is feasible & efficient Yields more valid estimatesYields more valid estimates

PainPain Self-report is feasible & efficientSelf-report is feasible & efficient Yields more valid estimates than observationYields more valid estimates than observation Ascertaining impact on function is feasible and Ascertaining impact on function is feasible and

provides useful informationprovides useful information

Customary Routine and ActivitiesCustomary Routine and Activities As recommended by TEP and Validation panel, asking As recommended by TEP and Validation panel, asking

importance is feasibleimportance is feasible

Funded by VA HSR&DFunded by VA HSR&D

Findings of VA ResearchFindings of VA Research

DiagnosesDiagnoses Algorithms to define active diagnosis improve Algorithms to define active diagnosis improve

identification compared to administrative dataidentification compared to administrative data

DeliriumDelirium Revised protocol and instruction improved agreementRevised protocol and instruction improved agreement

CognitionCognition Structured interview is feasible and welcomed by staffStructured interview is feasible and welcomed by staff

FallsFalls Simplified response options can be used by NH staff Simplified response options can be used by NH staff

to classify fallsto classify falls

BehaviorBehavior Items can consider impact on residentItems can consider impact on resident

Funded by VA HSR&DFunded by VA HSR&D

13 5/14/08

MDS 3.0 Development Proceeded MDS 3.0 Development Proceeded in 4 Phasesin 4 Phases

Townhall Meeting & Open Comment

VA Validation Protocol Research

Expert Panel Meetings

CMS Revised Draft MDS 3.0

National Pilot Testing

National Test of MDS 3.0

2003

2007

Phase 1: Stakeholder and Expert Feedback

Phase 2: MDS 3.0 Item Development

Integration of Phase 1 Feedback

Phase 3: MDS 3.0 Integration

Workgroup Review

Develop form & Instruction

Phase 4: National Testing

Final Revisions

Data Analysis

14 5/14/08

Revised Form DesignRevised Form Design

Form structured to improve usabilityForm structured to improve usability

Important definitions put on formImportant definitions put on form

Larger font Larger font

Logical breaks, fewer items to a pageLogical breaks, fewer items to a page

Items that were confusing or not needed for Items that were confusing or not needed for programming deletedprogramming deleted

MDS 3.0 Was Tested 71 NHs in 8 States MDS 3.0 Was Tested 71 NHs in 8 States

CACO

TXGA

NC

IL

PANJ

3800 residents participated in different 3800 residents participated in different parts of the evaluationparts of the evaluation

16 5/14/08

Different Types of Data Collectors Different Types of Data Collectors Tested MDS 3.0Tested MDS 3.0

Each state had 2 gold Each state had 2 gold standard nurse data standard nurse data collectorscollectors

Each nursing home Each nursing home had 1 facility nurse had 1 facility nurse data collector data collector

The National Test Measured The National Test Measured Reliability and Validity of MDS 3.0Reliability and Validity of MDS 3.0

Reliability of MDS 3.0Reliability of MDS 3.0 Inter-rater reliability measures the extent to Inter-rater reliability measures the extent to

which two data collectors achieve the same which two data collectors achieve the same results when assessing the same eventresults when assessing the same event

1.1. Gold-standard to Gold StandardGold-standard to Gold Standard

2.2. Gold-standard to Facility NurseGold-standard to Facility Nurse

Validity of MDS 3.0Validity of MDS 3.0 Validity assesses the degree to which items Validity assesses the degree to which items

measure the intended concept measure the intended concept

18 5/14/08

Additional Evaluation ElementsAdditional Evaluation Elements

Time to completeTime to complete Recorded all start and stop times for both MDS 2.0 and Recorded all start and stop times for both MDS 2.0 and

MDS 3.0MDS 3.0

Two Anonymous SurveysTwo Anonymous Surveys Mailed to all nurses who participated Mailed to all nurses who participated MDS 2.0 survey firstMDS 2.0 survey first MDS 3.0 survey completed at end of studyMDS 3.0 survey completed at end of study Provided feedback onProvided feedback on

Clinical usefulness of measuresClinical usefulness of measures Clarity and ease of completionClarity and ease of completion Satisfaction with assessment instrumentSatisfaction with assessment instrument

MDS 2.0 collected to allow cross walk between instruments MDS 2.0 collected to allow cross walk between instruments and into payment cellsand into payment cells

Review of 5 Sections with Review of 5 Sections with Major RevisionsMajor Revisions

20 5/14/08

1. MDS 3.0 Cognitive Assessment1. MDS 3.0 Cognitive Assessment

Brief Interview for Mental Status (BIMS)Brief Interview for Mental Status (BIMS) New structured test replaces staff assessment for New structured test replaces staff assessment for

residents who can be understoodresidents who can be understood

Staff Assessment for Mental StatusStaff Assessment for Mental Status Only completed for residents who cannot complete Only completed for residents who cannot complete

interviewinterview

Validated Confusion Assessment Method (CAM)Validated Confusion Assessment Method (CAM) Replaces old delirium itemsReplaces old delirium items

Rationale for Cognitive ChangesRationale for Cognitive Changes

Old cognitive item:Old cognitive item: Providers express discomfort with observation-based scoring Providers express discomfort with observation-based scoring

““long term memory OK” and “short term memory OK” items long term memory OK” and “short term memory OK” items are not recognized by most providersare not recognized by most providers

Only 29% thought MDS 2.0 easy to complete accuratelyOnly 29% thought MDS 2.0 easy to complete accurately

Instructs to use a formal assessment, but does not provide Instructs to use a formal assessment, but does not provide assessment or cross walk from standard assessment to 2.0assessment or cross walk from standard assessment to 2.0

CPS and COGs scales are not readily completed by NH staffCPS and COGs scales are not readily completed by NH staff

New cognitive item:New cognitive item: Directly tests domains common to most cognitive tests in Directly tests domains common to most cognitive tests in

other settings –registration, temporal orientation, recallother settings –registration, temporal orientation, recall Partial credit for close answers & response to prompts makes Partial credit for close answers & response to prompts makes

more relevant for population more relevant for population Supports validated delirium assessment protocolsSupports validated delirium assessment protocols

22 5/14/08

Rationale for Delirium ChangesRationale for Delirium Changes

Delirium is a serious condition associated with increased Delirium is a serious condition associated with increased mortality, morbidity, costs and institutionalizationmortality, morbidity, costs and institutionalization

Old delirium items:Old delirium items: Reliability in some studies worse than chanceReliability in some studies worse than chance Independent evaluations show significant under-Independent evaluations show significant under-

detection with unstructured observationdetection with unstructured observation

New delirium items = Confusion Assessment Method New delirium items = Confusion Assessment Method (CAM)(CAM)

CAM is cited as appropriate tool by Royal College of CAM is cited as appropriate tool by Royal College of Physicians, NCQA, other guidelinesPhysicians, NCQA, other guidelines

Improved sensitivity & specificity for detecting deliriumImproved sensitivity & specificity for detecting delirium

BIMS Feedback Survey ResultsBIMS Feedback Survey Results

80% thought BIMS improved 80% thought BIMS improved ability to calculate score and ability to calculate score and trigger RAPstrigger RAPs

78% preferred BIMS interview to 78% preferred BIMS interview to old assessment itemsold assessment items

88% reported that BIMS 88% reported that BIMS provided new insights into provided new insights into resident’s cognitive abilitiesresident’s cognitive abilities

24 5/14/08

Results: Cognitive Item Performance Results: Cognitive Item Performance

BIMS showed excellent reliability BIMS showed excellent reliability (kappa for score = .95)(kappa for score = .95)

Completion rates were highCompletion rates were high 90% of residents were able to 90% of residents were able to

complete complete Scores ranged from 0-15Scores ranged from 0-15

25 5/14/08

BIMS had excellent performance as a BIMS had excellent performance as a test to detect impairmenttest to detect impairment

BIMS was more highly correlated with gold-BIMS was more highly correlated with gold-standard measurestandard measure MDS. 3.0 BIMS = 0.91 (< .0001)MDS. 3.0 BIMS = 0.91 (< .0001)

MDS 2.0 CPS = - 0.74 (<.0001) MDS 2.0 CPS = - 0.74 (<.0001)

26 5/14/08

Delirium Feedback Survey ResultsDelirium Feedback Survey Results

85% found definitions on form 85% found definitions on form clearclear

71% felt that CAM helped them 71% felt that CAM helped them do a better job of screening for do a better job of screening for delirium delirium (7% disagreed)(7% disagreed)

64% reported that BIMS led them 64% reported that BIMS led them to observe new delirium to observe new delirium behaviors that differed from behaviors that differed from those in medical recordthose in medical record

27 5/14/08

Delirium showed very good reliabilityDelirium showed very good reliability

Item reliabilities ranged from Item reliabilities ranged from

kappa = .75 to .89kappa = .75 to .89

28 5/14/08

Delirium prevalence Delirium prevalence more consistent with expected ratesmore consistent with expected rates

2.0 and 3.0 Delirium Prevalence

7%

3%

7%

0%

5%

10%

15%

20%

MDS 3.0 (N=3234) MDS 2.0 (N=3262)

Subdelirium

Delrium

29 5/14/08

2. MDS 3.0 Depression Assessment2. MDS 3.0 Depression Assessment

PHQ-9PHQ-9New resident interview replaces staff observations for New resident interview replaces staff observations for residents who can report mood symptomsresidents who can report mood symptoms

Staff Assessment of PHQ-9-OVStaff Assessment of PHQ-9-OVNew observational items replace old staff assessment New observational items replace old staff assessment and only completed for residents who cannot self-and only completed for residents who cannot self-reportreport

Includes irritability itemIncludes irritability item

30 5/14/08

Rationale for Replacing Mood ItemsRationale for Replacing Mood Items

Old mood item:Old mood item: Repeatedly shown to have poor correspondence Repeatedly shown to have poor correspondence

with independent mood assessmentwith independent mood assessment Does not comport with accepted standard of self-reportDoes not comport with accepted standard of self-report Requires time consuming systematic observations of Requires time consuming systematic observations of

ALL residents across all shifts. Difficult to achieve.ALL residents across all shifts. Difficult to achieve. Only 22% reported that 2.0 section was easy to complete Only 22% reported that 2.0 section was easy to complete

accuratelyaccurately

Questionable utility for gauging response to treatment, Questionable utility for gauging response to treatment, since appropriate approach is targeting DSM-IV signs and since appropriate approach is targeting DSM-IV signs and symptomssymptoms

31 5/14/08

Rationale for Replacing Mood ItemsRationale for Replacing Mood Items

New mood item (PHQ-9) New mood item (PHQ-9)

Based on DSM-IV criteriaBased on DSM-IV criteria

Validity well established in other settingsValidity well established in other settings

Increasing use and recognition by cliniciansIncreasing use and recognition by clinicians

Allows threshold definition AND rapid sum of a Allows threshold definition AND rapid sum of a severity score that can track change over timeseverity score that can track change over time

Has been used in outpatient elders, hospital, Has been used in outpatient elders, hospital, rehabilitation (post stroke) and home health rehabilitation (post stroke) and home health populations in addition to younger adult populationspopulations in addition to younger adult populations

32 5/14/08

Mood Feedback ResultsMood Feedback Results

87% nurses rated the mood section 87% nurses rated the mood section as improved over 2.0 sectionas improved over 2.0 section

88% felt PHQ-9 interview was better 88% felt PHQ-9 interview was better than 2.0 observation for capturing than 2.0 observation for capturing resident moodresident mood

84% felt the items could inform care 84% felt the items could inform care plansplans

86% reported that items provided 86% reported that items provided new insights into moodnew insights into mood

33 5/14/08

Feedback Feedback Staff Mood Assessment Staff Mood Assessment

90% felt that detection and 90% felt that detection and communication about mood would communication about mood would improve if staff learned to watch for improve if staff learned to watch for these signs and symptomsthese signs and symptoms

72% found PHQ-9-OV assessment 72% found PHQ-9-OV assessment easier than MDS 2.0easier than MDS 2.0

34 5/14/08

Results: PHQ-9 PerformanceResults: PHQ-9 Performance

PHQ-9 had excellent reliabilityPHQ-9 had excellent reliability

Resident Mood Interview Resident Mood Interview Kappa: 0.94Kappa: 0.94

Staff Mood Observations Staff Mood Observations Kappa: 0.93Kappa: 0.93

86% of non-comatose residents 86% of non-comatose residents were able to complete interviewwere able to complete interview

35 5/14/08

PHQ-9 Interview was More Valid PHQ-9 Interview was More Valid than MDS 2.0than MDS 2.0

0.69

0.52

0.15

0

0.2

0.4

0.6

0.8

PHQ9 GDS MDS 2.0

Agreement withgold standard

36 5/14/08

PHQ-9 OV was More Valid for PHQ-9 OV was More Valid for Residents unable to self-reportResidents unable to self-report

0.84

0.28

0

0.2

0.4

0.6

0.8

1

PHQ9OV MDS 2.0

Correlation withGold StandardMeasure

37 5/14/08

3. MDS 3.0 Behavior Items 3. MDS 3.0 Behavior Items

Hallucinations and psychosis Hallucinations and psychosis moved from checklist in section J & definitions put on moved from checklist in section J & definitions put on

formform

Behaviors Behaviors Revised language clearer, linked to operational definitions Revised language clearer, linked to operational definitions Revised symptom groupings to match constructsRevised symptom groupings to match constructs Replaced “alterability” with specific impact questionsReplaced “alterability” with specific impact questions Replaced “resisting care” with “reject care” and Replaced “resisting care” with “reject care” and

refocused on resident’s goals of carerefocused on resident’s goals of care

Wandering rated separately from the 3 behavioral symptoms Wandering rated separately from the 3 behavioral symptoms groups, and impact replaces alterabilitygroups, and impact replaces alterability

38 5/14/08

Rationale For ChangesRationale For Changes

Old behavior item groupings were not consistent Old behavior item groupings were not consistent with recognized factorswith recognized factors

Only 41% of nurses rated MDS 2.0 items as easy Only 41% of nurses rated MDS 2.0 items as easy to complete accuratelyto complete accurately

Consumers viewed old behavior items as pejorativeConsumers viewed old behavior items as pejorative

Staff varied widely in definition of “alterability”Staff varied widely in definition of “alterability”

Did not identify behaviors requiring interventionDid not identify behaviors requiring intervention

New specific impact items give insight into severity New specific impact items give insight into severity and potential need for treatment/interventionand potential need for treatment/intervention

39 5/14/08

Behavior Behavior Feedback Survey ResultsFeedback Survey Results

90% rated behavioral symptoms as 90% rated behavioral symptoms as easy to complete accuratelyeasy to complete accurately

91% nurses preferred the 3.0 91% nurses preferred the 3.0 behavior item section behavior item section (1% disagreed)(1% disagreed)

90-94% rated new behavioral 90-94% rated new behavioral symptoms items as clearsymptoms items as clear

88% rated impact items as providing 88% rated impact items as providing important severity informationimportant severity information

40 5/14/08

Behavior section Behavior section reliability was excellentreliability was excellent

Psychosis, kappa = 0.96Psychosis, kappa = 0.96

Overall kappa for all other Overall kappa for all other behavioral items = 0.94behavioral items = 0.94

41 5/14/08

Behavior & Psychoses Items Behavior & Psychoses Items Had Higher Agreement with Gold StandardHad Higher Agreement with Gold Standard

Gold-StandardGold-Standard MDS 3.0 KappaMDS 3.0 Kappa

(95% CI)(95% CI)

MDS 2.0 KappaMDS 2.0 Kappa

(95% CI)(95% CI)

Physical toward Physical toward othersothers

.86 .86 (.74, .97)(.74, .97) .23 .23 (.03, .43)(.03, .43)

Verbal toward Verbal toward othersothers

.73 .73 (.61, .84)(.61, .84) .31 .31 (.16, .45)(.16, .45)

Other BehaviorOther Behavior .53 .53 (.42, .66)(.42, .66) .22 .22 (.12, .31)(.12, .31)

HallucinationsHallucinations .92 .92 (.81, 1.00)(.81, 1.00) .23 .23 (.03, .43)(.03, .43)

DelusionsDelusions .88 .88 (.79, .98)(.79, .98) .31 .31 (.16, .45)(.16, .45)

42 5/14/08

Type of Impact on Resident VariesType of Impact on Resident Varies

MDS 3.0 Behavioral Symptoms: Impact on Resident (N=317)

24%

33%36%

0%

10%

20%

30%

40%

Puts resident atrisk

Interferes withcare

Interferes withactivities

43 5/14/08

4. MDS 3.0 Customary Routine & 4. MDS 3.0 Customary Routine & ActivitiesActivities

Preferred RoutinePreferred Routine New interview replaces 20 Customary Routine staff New interview replaces 20 Customary Routine staff

assessment items for residents who can be interviewedassessment items for residents who can be interviewed Current importance rating replaces check all that applied in Current importance rating replaces check all that applied in

past yearpast year

New interview for activities preferences replaces 12 staff New interview for activities preferences replaces 12 staff assessment items for residents who can be interviewedassessment items for residents who can be interviewed

Want to talk to someone about returning to community Want to talk to someone about returning to community

Staff Assessment of Activity and Daily PreferencesStaff Assessment of Activity and Daily Preferences Only completed for residents who cannot complete interviewOnly completed for residents who cannot complete interview Major changes to several items; instructed to observe resident Major changes to several items; instructed to observe resident

response during exposure to activityresponse during exposure to activity

44 5/14/08

Rationale for changesRationale for changes

Old itemsOld items Not perceived as helping with care planningNot perceived as helping with care planning

Prior practice could be related to ability, illness, Prior practice could be related to ability, illness, access, not to preferenceaccess, not to preference

Only 30% rated 2.0 as helping care planningOnly 30% rated 2.0 as helping care planning

TEP and Validation Panels both recommended TEP and Validation Panels both recommended replace with importance scalesreplace with importance scales

New Items (Preference Assessment Tool or PAT)New Items (Preference Assessment Tool or PAT) Grounded in residential care quality Grounded in residential care quality Map to U Minnesota QoL domainsMap to U Minnesota QoL domains Focuses on resident as central to determining Focuses on resident as central to determining

activitiesactivities

45 5/14/08

Residents were able to complete Residents were able to complete

Primary Respondent for Preferred Routine & Activities (N=3246)

4%11%

84%

SelfSignificant OtherNot completed

Customary Routine Feedback ResultsCustomary Routine Feedback Results

81% rated the interview items as more 81% rated the interview items as more useful for care planninguseful for care planning

80% found that the interview changed 80% found that the interview changed their impression of resident’s wantstheir impression of resident’s wants

More likely to report that post-acute More likely to report that post-acute residents appreciated being askedresidents appreciated being asked

Only 1% felt that some residents who Only 1% felt that some residents who responded didn’t understand the itemsresponded didn’t understand the items

• Activity Feedback was similarActivity Feedback was similar

47 5/14/08

Agreement for Customary Routine & Agreement for Customary Routine & Activities was ExcellentActivities was Excellent

Preferred Routine Preferred Routine kappa = 0.97kappa = 0.97

Activities 0.96Activities 0.96kappa = 0.96kappa = 0.96

Overall scores were similar across Overall scores were similar across cognitive groups for daily routinecognitive groups for daily routine

Cognitive groupCognitive group RangeRange Mean (sd)Mean (sd)

IntactIntact(n=1384)(n=1384)

0 - 40 - 4 2.442.44 (1.08) (1.08)

ImpairedImpaired(n = 734)(n = 734)

0 - 40 - 4 2.602.60 (1.05) (1.05)

Severely impairedSeverely impaired(n=827)(n=827)

0 - 40 - 4 2.462.46 (1.09) (1.09)

Same pattern was seen with activity itemsSame pattern was seen with activity items

49 5/14/08

5. MDS 3.0 Pain Assessment Items5. MDS 3.0 Pain Assessment Items

Treatment items addedTreatment items added

Resident interview replaces staff observations for Resident interview replaces staff observations for residents who can report pain symptomsresidents who can report pain symptoms

Section expanded to capture effect on functionSection expanded to capture effect on function

Staff assessment of pain changed to an Staff assessment of pain changed to an observational checklist of pain behaviors and only observational checklist of pain behaviors and only completed for residents who cannot self-reportcompleted for residents who cannot self-report

50 5/14/08

Rationale for Replacing Pain ItemsRationale for Replacing Pain Items

Old pain itemOld pain item Repeatedly shown to have poor correspondence Repeatedly shown to have poor correspondence

with independent pain assessmentswith independent pain assessments Does not comport with accepted standard of self reportDoes not comport with accepted standard of self report

Requires time consuming systematic observations of Requires time consuming systematic observations of all residents across all shiftsall residents across all shifts

Detection bias penalizes more vigilant facilitiesDetection bias penalizes more vigilant facilities

Providers and consumers frustrated that section Providers and consumers frustrated that section addresses limited characteristics, insufficient to addresses limited characteristics, insufficient to capture pain experience capture pain experience

3 point severity response insufficient and not match 3 point severity response insufficient and not match commonly used pain scales. Want severity response commonly used pain scales. Want severity response between “moderate” & “horrible or excruciating”between “moderate” & “horrible or excruciating”

51 5/14/08

Rationale for New Pain ItemsRationale for New Pain Items

Self-report is the gold standard for pain Self-report is the gold standard for pain assessment assessment

Pilot test showed ability to recall over 5 Pilot test showed ability to recall over 5 daysdays

With pain being reported as “5With pain being reported as “5thth vital sign” vital sign” providers have increasingly used 0-10 providers have increasingly used 0-10 scales in NHs & other settingsscales in NHs & other settings

0-10 scale would allow comparison 0-10 scale would allow comparison across settingsacross settings

Pain Feedback was PositivePain Feedback was Positive

88% rated MDS 3.0 pain items as 88% rated MDS 3.0 pain items as improved over MDS 2.0improved over MDS 2.0

94% reported that new pain items 94% reported that new pain items could inform care planscould inform care plans

Even during testing, pain interview Even during testing, pain interview provided new insights into provided new insights into resident’s pain (85%)resident’s pain (85%)

90% felt that all residents who 90% felt that all residents who responded, understood (3% responded, understood (3% disagreed)disagreed)

85% felt the observable behaviors 85% felt the observable behaviors would improve reporting of possible would improve reporting of possible painpain

53 5/14/08

Results Pain Item PerformanceResults Pain Item Performance

Pain Items showed excellent reliabilityPain Items showed excellent reliability Pain treatment & interview (J1-J7)Pain treatment & interview (J1-J7)

kappa = 0.92kappa = 0.92 Staff assessment of pain (J9) Staff assessment of pain (J9)

kappa = 0.97kappa = 0.97

Completion rates were highCompletion rates were high 85% of non-comatose residents were 85% of non-comatose residents were

able to complete the pain interviewable to complete the pain interview

54 5/14/08

Pain PresencePain Presence

Pain Presence Validation Sample

64%

50%

0%

20%

40%

60%

80%

MDS 3.0

MDS 2.0

55 5/14/08

Other Measures of AgreementOther Measures of Agreement

Temporal reliabilityTemporal reliability Interview 24 hours later (different assessor Interview 24 hours later (different assessor

also)also)

kappa = .9242 (.8837, .9647)kappa = .9242 (.8837, .9647)

Agreement with MDS 2.0Agreement with MDS 2.0

kappa = .4812 (.3962, .5662)kappa = .4812 (.3962, .5662)

56 5/14/08

Staff Assessment of Pain Staff Assessment of Pain

18%

27%29%

20%

0%

5%

10%

15%

20%

25%

30%

Non VerbalSounds

VocalComplaints

FacialExpressions

BodyMovements

3.0 Staff Assessment of Pain (N=45)

43 % with any symptom43 % with any symptom

57 5/14/08

Other Sections with Other Sections with Important ChangesImportant Changes

Pressure ulcer Pressure ulcer eliminated reverse stagingeliminated reverse staging adds present on admitadds present on admit

Balance Balance refocused on movement and transitionsrefocused on movement and transitions

Falls Falls introduced type of injuryintroduced type of injury

Bowel & bladderBowel & bladder no longer rate catheter as continentno longer rate catheter as continent improved toileting program itemimproved toileting program item

58 5/14/08

Other Sections with Other Sections with Important ChangesImportant Changes

Activities of daily living – single response scaleActivities of daily living – single response scale

Goals of care item addedGoals of care item added

Oral/dental item improvedOral/dental item improved

Swallowing item - checklist of observable signs Swallowing item - checklist of observable signs and symptomsand symptoms

Restraints – separate bed and chairRestraints – separate bed and chair

59 5/14/08

Overall, Nurses Judged MDS 3.0 Overall, Nurses Judged MDS 3.0 Clinical Utility & Clarity ImprovedClinical Utility & Clarity Improved

85% rated MDS 3.0 as likely to help identify 85% rated MDS 3.0 as likely to help identify unrecognized problemsunrecognized problems

81% rated MDS 3.0 as more relevant than 2.081% rated MDS 3.0 as more relevant than 2.0

84% reported that MDS 3.0 interview items 84% reported that MDS 3.0 interview items improved their knowledge of the residentimproved their knowledge of the resident

85% rated MDS 3.0 questions as more clearly 85% rated MDS 3.0 questions as more clearly wordedworded

60 5/14/08

Nurses also rated Nurses also rated Validity HighValidity High

89% rated MDS 3.0 as providing a more 89% rated MDS 3.0 as providing a more accurate report of resident characteristics accurate report of resident characteristics than MDS 2.0than MDS 2.0

76% rated MDS 3.0 as better reflecting best 76% rated MDS 3.0 as better reflecting best clinical practice or standardsclinical practice or standards

61 5/14/08

MDS 3.0 Took Less TimeMDS 3.0 Took Less Time

MDS 3.0 TimeMDS 3.0 Time

Average time: Average time:

62 Min62 Min

MDS 2.0 TimeMDS 2.0 Time

Average time: Average time:

112 Min112 Min

<<

62 5/14/08

Given these Advances, What Role Given these Advances, What Role Might Facility Leaders play?Might Facility Leaders play?

1.1. Assist staff in understanding clinical utility & Assist staff in understanding clinical utility & efficiency of more accurate assessmentefficiency of more accurate assessment

2.2. Educate primary care providers and medical Educate primary care providers and medical consultants on availability and relevance of new consultants on availability and relevance of new assessment itemsassessment items

3.3. Assist staff in learning how to interviewAssist staff in learning how to interview

4.4. Help identify linkages with culture change Help identify linkages with culture change activitiesactivities

63 5/14/08

Take-Home Message: Take-Home Message: MDS 3.0 Revisions are Based onMDS 3.0 Revisions are Based on

Feedback from usersFeedback from users

Input from ExpertsInput from Experts

Advances in assessment scienceAdvances in assessment science

Improve clinical care in nursing homeImprove clinical care in nursing home

Improve communication with providersImprove communication with providers

Improve ability to track care and Improve ability to track care and patient progress across settings patient progress across settings

Testing of performance in NH populationsTesting of performance in NH populations

64 5/14/08

Take Home Messages: Take Home Messages: MDS 3.0 GainsMDS 3.0 Gains

National testing showed increased resident voice and National testing showed increased resident voice and refined measures in MDS 3.0:refined measures in MDS 3.0:

Increase measurement reliability & validityIncrease measurement reliability & validity Together these improve clinical detection and Together these improve clinical detection and

assessment accuracyassessment accuracy

Both Facility and Study Nurses from 71 NHs who used Both Facility and Study Nurses from 71 NHs who used MDS 3.0 reported higher satisfaction due to:MDS 3.0 reported higher satisfaction due to:

Increased clinical relevanceIncreased clinical relevance Increased clarityIncreased clarity Increased knowledge about residentIncreased knowledge about resident

National testing showed reduced completion time by 45%National testing showed reduced completion time by 45%

MDS 3.0MDS 3.0

Presenter: Debra Saliba, MD, MPHPresenter: Debra Saliba, MD, MPH

AHCA WebinarAHCA WebinarMay 14, 2008May 14, 2008

Making Making Assessment WorkAssessment Work

PointClickCare is the World’s #1 OnDemand PointClickCare is the World’s #1 OnDemand Long Term Care Software solution, offering Long Term Care Software solution, offering Providers of all sizes a comprehensive suite of Providers of all sizes a comprehensive suite of integrated applications specifically designed for integrated applications specifically designed for LTC.LTC.

RUG Tools' software helps nursing facilities RUG Tools' software helps nursing facilities increase PPS and case mix reimbursement and increase PPS and case mix reimbursement and correct MDS errors by highlighting the regulatory correct MDS errors by highlighting the regulatory impacts of each MDS assessment.impacts of each MDS assessment.

The MDS 3.0The MDS 3.0

Questions and Comments:Questions and Comments:

www.cms.hhs.gov/nursinghomequalityinits/25_nhqimds30.aspwww.cms.hhs.gov/nursinghomequalityinits/25_nhqimds30.asp

Improving Improving AssessmentAssessment

69 5/14/08

MDS 3.0 Information Web SitesMDS 3.0 Information Web Sites

MDS 3.0 general information site:MDS 3.0 general information site:  http://www.cms.hhs.gov/nursinghomequalityinits/25_nhqimds30.asphttp://www.cms.hhs.gov/nursinghomequalityinits/25_nhqimds30.asp  MDS 3.0 draft version--1/15/08MDS 3.0 draft version--1/15/08  http://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30DraftVersion.pdfhttp://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30DraftVersion.pdf  Timeline--MDS 3.0Timeline--MDS 3.0  http://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30Timeline.pdfhttp://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30Timeline.pdf  RE: PHQ-9 (one of many reference sites available)RE: PHQ-9 (one of many reference sites available)http://www.depression-primarycare.org/clinicians/toolkits/materials/forms/phq9/http://www.depression-primarycare.org/clinicians/toolkits/materials/forms/phq9/  Link to this and other webinars:Link to this and other webinars:http://www.ahcancal.org/events/Pages/Webinars.aspxhttp://www.ahcancal.org/events/Pages/Webinars.aspx