3 rustia casnur-ifa2012

Greater

healthcare

demands !!

Healthcare meltdown

Aging and nursing

shortage

400,000

shortage

by 2020

1:20 vs

ideal of

1:10

INCREASING DEMANDS on SENIOR NURSES Retention of Workers Strategy

At least 1

adverse

effect

70% of

deaths

INCREASED INCIDENCES of ERROR

Heterogeneous concept that must be broken down

There is no specialized concept or metric for the

cognitive aspect of work ability and there are also no

methods at present to measure / assess this.

Measure Cognitive Work Ability

Job reassignment & interventions

Predicting stress levels and likelihood of

adverse effects (ADEs)

Existing methods are insufficient &

subject to bias.

Automated and Performance-Based

CASNUR: Developing a software to assess cognitive

work ability among aging nurses

by Maria Dominique Rustia & Rosemary Seva

Human Factors and Ergonomics Center, De La Salle University, Manila, Phil.

[email protected]

May 28, 2012

IFA 2012 Global Conference on Ageing

mailto:[email protected]

PROBLEM STATEMENT

• Existing assessment methods for work ability among nurses are

subject to response bias and fail to consider the cognitive

dimension of tasks.

• There is a need for a performance-based assessment method

for cognitive work ability (CWA) to be established, especially in

the context of ICU nursing.

1 Background Method Concept & Design Conclusion Testing

OBJECTIVES, SCOPE & LIMITATIONS

• to provide a reliable and valid measure of CWA among hospital nurses

by developing a software application that can assess CWA among them.


1. To test the validity of the CWA assessment software

2. To perform an initial evaluation of the usability of the software

3. To determine the reliability & effectiveness of software in capturing data for CWA

metric derivation

4. To identify areas for improvement in the software

SCOPE & LIMITATIONS

Hospital nurses, ages 40-65, Intensive Care Unit (ICU) setting

Philippine setting

Will only test variables of CWA model of Rustia & Seva (2011)

Will not propose any interventions

Conceptualization and design of software

Gender, affective & physical needs not considered

OBJECTIVES

COGNITIVE WORK ABILITY 1

Cognitive Work Ability (CWA) the ability of a person to fulfill

mentally demanding tasks efficiently and effectively

Background Method Concept & Design Conclusion Testing

Work Ability Index (WAI) “how good workers are at present and in the

near future, and how they will be able to do their work with respect to work

demands, health and mental resources” (Ilmarinen & Tuomi, 2003)

speed accuracy

EXISTING ASSESSMENT METHODS 1

• Reliability and validity have

been verified but coverage of

components of CWA is poor

• Questionnaire-based, risk of

biased and inaccurate answers

• No performance-based

evaluation methods to date

SIMULATION OF THE

NURSING ENVIRONMENT

(low cost software)


Work Ability Index (Ilmarinen & Tuomi, 1992)

WHO Quality of Life (WHO,

1991)

Expanded Nursing Stress

Scale (French et al., 2007)

TISS-28 (Miranda et al.,

1996)

Revised Nursing Work Index

(Aiken & Patrician, 2000)

CASNUR

Cognitive work ability Assessment

Software for NURses

METHODOLOGY 2


METHODOLOGY 2

Systems Development Life Cycle


SOFTWARE DESIGN- Software Concept 3

SIMULATION

DATA-GATHERING

EVALUATION

1

2

3


SOFTWARE DESIGN – Requirements Planning

• Applied Cognitive Task Analysis (ACTA) applied to determine cognitive

processes in regular ICU nursing tasks.

3

ACCURACY SPEED

PERFORMANCE

QUALITY


SIMULATION INTERVIEW &

KNOWLEDGE AUDIT

MONITOR and ALARM

hard to say immediately

where the problem is

EMERGENCIES lead to

improvising

presence of mind and

alertness, prioritization, good

memory, speed

checking, finding

discrepancies in medicine

arrangement and labeling

of medicine, better systems





3

SIMULATION 1

DATA-GATHERING 2

Abnormal event generation

Errors, Actions, Responses



Other Characteristics:

• Simulation must also have the same layout as actual ICU in the Philippines

• Emergency ICU case feeling of “time pressure” and urgency

Constantly

changing

E-cart to

concentrate on

individual skill

Computerized to

remove bias

related to

handwriting &

technique

SOFTWARE DESIGN - Conceptual Model 3


SOFTWARE DESIGN - Software Features


SOFTWARE DESIGN – Software Features 3


EXPERIMENTATION- Factor Selection 3

Measurable by CASNUR


and eye-tracker software


and eye-tracker software



PILOT TEST – Subjects & Assumptions 4

ASSUMPTIONS

The participants selected are representative of the

targeted users of CASNUR.

The medications are assumed to be correct even if

they medically are not.

The frequency of errors is assumed to be correct.

The subjects were physically, mentally and

emotionally fit before testing.

• Filipino nurses, ages 45+

• Fluent in English and Filipino

• Capable of using computers

• No physical or mental diseases / impairments

• Must have had experience in ICU / emergencies

• Day shift to avoid influence of night shifts

• Pre-scheduled appointments to avoid rushing

SUBJECTS & ENVIRONMENT


PILOT TEST - Methodology 4

CASNUR software

Usability Plan, CASNUR

Usability Questionnaire

Dikablis Eye-Tracker & Software

CASNUR + Eye-Tracker + Morae

Simulation Evaluation Survey

Subject Profile Survey


PILOT TEST - Methodology 4

RELIABILITY VALIDITY USABILITY

Testing of successful

generation of variables

and events

How close the simulation

is to real-life situations

EFFICIENCY

EFFECTIVENESS

SATISFACTION


PILOT TEST - Results and Analysis 4

RELIABILITY

SIMULATION

Vital Signs Monitor changes,

alarms and indicates abnormality

• Emergency cart with full

functionality of selecting and

adding medicine to table

• Medicine Table with maximum

capacity of 3 medications

• Administration of oxygen,

dextrose and medicine (input of

dosage)

DATA-

GATHERING

Error identification,

counting and recording

Response / action

identification, counting and

recording

Visual timer on interface

Export feature to EXCEL



RELIABILITY


10 variables measured


VALIDATION OF CWA MODEL FROM RESULTS

Attentive Resources

Perception

Working Memory

Capacity

Responsiveness

Task structure

TESTS

CONDUCTED: Collinearity

Heteroskedasticity

Normality



VALIDITY

Shapiro-Wilk W-test for normality

Correlation analysis

Simulation evaluation survey

Rating of how realistic simulation is

Mostly normal in distribution

4 / 5 (5 = very realistic)

Number of wrong medicine types = positive

correlation with unnecessary actions

Response time = +correlation with suffering

duration

Correct responses = -correlation w/ wrong

type errors



USABILITY

EFFECTIVENESS High Learnability (12.03%

unnecessary actions)

• 40.66% of abnormal events

solved by the users

EFFICIENCY “Normal” task completion

times



USABILITY

SATISFACTION 50% satisfaction rating

5.25 / 6 in ease of use

Second trial slightly easier than first



DEBRIEFING RESULTS



AREAS FOR IMPROVEMENT

PREPARATION 1. Only qualitative testing for presence

of mental impairments

2. Inconsistencies in pacing and tone of

briefing voice.

Use of COGLAB Memory Test

Design of a briefing video for CASNUR

NORMALITY OF DATA

SOFTWARE 1. Unstoppable alarms make nurses more irritable or tense.

2. Nurses tend to forget to select the medicine first before clicking on GIVE MEDICINE.

3. Confusion with color on vital signs monitor.

4. Nurses forget to click DONE / CANCEL to return to main interface from e-cart.

5. Scattering of physician’s orders during experiment

6. Too many abnormal events occurring within the span of 5 minutes.

More runs / replications




SOFTWARE

ON/OFF toggling of alarm

Reprogramming of functions

Use of brighter color (YELLOW)

Placing of “GO BACK TO PATIENT”

button in e-cart and changing CANCEL

to “CANCEL addition”




EXPERIMENT 1. Difficulty moving mouse around.

2. More than one run per setting is

needed.

3. Subjects tend to get significantly

better after 4th / 5th try.

4. Need a basis / ideal performance

value for the task times.

5. Usability testing needs more

standardization.

Touchscreens

More subjects and runs

Do not use beyond 3 tries per

subject.

Compute for performance

value through control limit

computation & interviews with

nursing educators.

Inclusion of NASA TLX test.


CONCLUSION & RECOMMENDATIONS 5

• Cognitive Work Ability (CWA) is a specialized

concept for the ability of a person to fulfill

mentally demanding tasks efficiently and

effectively.

• Simulation as performance-based method for

assessing CWA among ICU nurses.

• A simulation software has been successfully

designed with acceptable usability ratings,

reliability in data collection & simulation, and

validity (face and construct).

• Recommended use of software by the

academe, health professionals, nurses,

hospitals.

• Larger experiment with more

samples and actual derivation

and evaluation of CWA index.

• Incorporation of eye-tracking and

screen-recording features in

system to reduce CASNUR’s

usage costs.

• Testing by wider demographic

(young and old).

• Longitudinal testing of software

and CWA index.


“With society becoming older and more active, will

we change the way we work…? Does it make any

sense to stop being productive at a particular age?”

– Ken Dychtwald (1990)

TRY OUT CASNUR Look for me after the presentation / session /

anytime during the IFA 2012 Conference.

Thank you. Questions / Comments? Feel free to e-mail them to:

[email protected]

3 rustia casnur-ifa2012

Technology

design of software gender

specialized concept

heterogeneous concept

work demands

cognitivework ability

software application

components of cwa

cwa assessment software2