Transcript
Greater
healthcare
demands !!
Healthcare meltdown
Aging and nursing
shortage
400,000
shortage
by 2020
1:20 vs
ideal of
1:10
INCREASING DEMANDS on SENIOR NURSES Retention of Workers Strategy
At least 1
adverse
effect
70% of
deaths
INCREASED INCIDENCES of ERROR
Heterogeneous concept that must be broken down
There is no specialized concept or metric for the
cognitive aspect of work ability and there are also no
methods at present to measure / assess this.
Measure Cognitive Work Ability
Job reassignment & interventions
Predicting stress levels and likelihood of
adverse effects (ADEs)
Existing methods are insufficient &
subject to bias.
Automated and Performance-Based
CASNUR: Developing a software to assess cognitive
work ability among aging nurses
by Maria Dominique Rustia & Rosemary Seva
Human Factors and Ergonomics Center, De La Salle University, Manila, Phil.
rustiamd@gmail.com
May 28, 2012
IFA 2012 Global Conference on Ageing
PROBLEM STATEMENT
• Existing assessment methods for work ability among nurses are
subject to response bias and fail to consider the cognitive
dimension of tasks.
• There is a need for a performance-based assessment method
for cognitive work ability (CWA) to be established, especially in
the context of ICU nursing.
1 Background Method Concept & Design Conclusion Testing
OBJECTIVES, SCOPE & LIMITATIONS
• to provide a reliable and valid measure of CWA among hospital nurses
by developing a software application that can assess CWA among them.
1 Background Method Concept & Design Conclusion Testing
1. To test the validity of the CWA assessment software
2. To perform an initial evaluation of the usability of the software
3. To determine the reliability & effectiveness of software in capturing data for CWA
metric derivation
4. To identify areas for improvement in the software
SCOPE & LIMITATIONS
Hospital nurses, ages 40-65, Intensive Care Unit (ICU) setting
Philippine setting
Will only test variables of CWA model of Rustia & Seva (2011)
Will not propose any interventions
Conceptualization and design of software
Gender, affective & physical needs not considered
OBJECTIVES
COGNITIVE WORK ABILITY 1
Cognitive Work Ability (CWA) the ability of a person to fulfill
mentally demanding tasks efficiently and effectively
Background Method Concept & Design Conclusion Testing
Work Ability Index (WAI) “how good workers are at present and in the
near future, and how they will be able to do their work with respect to work
demands, health and mental resources” (Ilmarinen & Tuomi, 2003)
speed accuracy
EXISTING ASSESSMENT METHODS 1
• Reliability and validity have
been verified but coverage of
components of CWA is poor
• Questionnaire-based, risk of
biased and inaccurate answers
• No performance-based
evaluation methods to date
SIMULATION OF THE
NURSING ENVIRONMENT
(low cost software)
Background Method Concept & Design Conclusion Testing
Work Ability Index (Ilmarinen & Tuomi, 1992)
WHO Quality of Life (WHO,
1991)
Expanded Nursing Stress
Scale (French et al., 2007)
TISS-28 (Miranda et al.,
1996)
Revised Nursing Work Index
(Aiken & Patrician, 2000)
CASNUR
Cognitive work ability Assessment
Software for NURses
METHODOLOGY 2
Background Method Concept & Design Conclusion Testing
METHODOLOGY 2
Systems Development Life Cycle
Background Method Concept & Design Conclusion Testing
SOFTWARE DESIGN- Software Concept 3
SIMULATION
DATA-GATHERING
EVALUATION
1
2
3
Background Method Concept & Design Conclusion Testing
SOFTWARE DESIGN – Requirements Planning
• Applied Cognitive Task Analysis (ACTA) applied to determine cognitive
processes in regular ICU nursing tasks.
3
ACCURACY SPEED
PERFORMANCE
QUALITY
Background Method Concept & Design Conclusion Testing
SIMULATION INTERVIEW &
KNOWLEDGE AUDIT
MONITOR and ALARM
hard to say immediately
where the problem is
EMERGENCIES lead to
improvising
presence of mind and
alertness, prioritization, good
memory, speed
checking, finding
discrepancies in medicine
arrangement and labeling
of medicine, better systems
3 Background Method Concept & Design Conclusion Testing
SOFTWARE DESIGN – Requirements Planning
3 Background Method Concept & Design Conclusion Testing
SOFTWARE DESIGN – Requirements Planning
3
SIMULATION 1
DATA-GATHERING 2
Abnormal event generation
Errors, Actions, Responses
Background Method Concept & Design Conclusion Testing
SOFTWARE DESIGN – Requirements Planning
Other Characteristics:
• Simulation must also have the same layout as actual ICU in the Philippines
• Emergency ICU case feeling of “time pressure” and urgency
Constantly
changing
E-cart to
concentrate on
individual skill
Computerized to
remove bias
related to
handwriting &
technique
SOFTWARE DESIGN - Conceptual Model 3
Background Method Concept & Design Conclusion Testing
SOFTWARE DESIGN - Software Features
3 Background Method Concept & Design Conclusion Testing
SOFTWARE DESIGN – Software Features 3
Background Method Concept & Design Conclusion Testing
EXPERIMENTATION- Factor Selection 3
Measurable by CASNUR
Measurable by CASNUR
and eye-tracker software
Measurable by CASNUR
and eye-tracker software
Measurable by CASNUR
Background Method Concept & Design Conclusion Testing
PILOT TEST – Subjects & Assumptions 4
ASSUMPTIONS
The participants selected are representative of the
targeted users of CASNUR.
The medications are assumed to be correct even if
they medically are not.
The frequency of errors is assumed to be correct.
The subjects were physically, mentally and
emotionally fit before testing.
• Filipino nurses, ages 45+
• Fluent in English and Filipino
• Capable of using computers
• No physical or mental diseases / impairments
• Must have had experience in ICU / emergencies
• Day shift to avoid influence of night shifts
• Pre-scheduled appointments to avoid rushing
SUBJECTS & ENVIRONMENT
Background Method Concept & Design Conclusion Testing
PILOT TEST - Methodology 4
CASNUR software
Usability Plan, CASNUR
Usability Questionnaire
Dikablis Eye-Tracker & Software
CASNUR + Eye-Tracker + Morae
Simulation Evaluation Survey
Subject Profile Survey
Background Method Concept & Design Conclusion Testing
PILOT TEST - Methodology 4
RELIABILITY VALIDITY USABILITY
Testing of successful
generation of variables
and events
How close the simulation
is to real-life situations
EFFICIENCY
EFFECTIVENESS
SATISFACTION
Background Method Concept & Design Conclusion Testing
PILOT TEST - Results and Analysis 4
RELIABILITY
SIMULATION
Vital Signs Monitor changes,
alarms and indicates abnormality
• Emergency cart with full
functionality of selecting and
adding medicine to table
• Medicine Table with maximum
capacity of 3 medications
• Administration of oxygen,
dextrose and medicine (input of
dosage)
DATA-
GATHERING
Error identification,
counting and recording
Response / action
identification, counting and
recording
Visual timer on interface
Export feature to EXCEL
Background Method Concept & Design Conclusion Testing
PILOT TEST - Results and Analysis 4
RELIABILITY
Background Method Concept & Design Conclusion Testing
10 variables measured
PILOT TEST - Results and Analysis 4
VALIDATION OF CWA MODEL FROM RESULTS
Attentive Resources
Perception
Working Memory
Capacity
Responsiveness
Task structure
TESTS
CONDUCTED: Collinearity
Heteroskedasticity
Normality
Background Method Concept & Design Conclusion Testing
PILOT TEST - Results and Analysis 4
VALIDITY
Shapiro-Wilk W-test for normality
Correlation analysis
Simulation evaluation survey
Rating of how realistic simulation is
Mostly normal in distribution
4 / 5 (5 = very realistic)
Number of wrong medicine types = positive
correlation with unnecessary actions
Response time = +correlation with suffering
duration
Correct responses = -correlation w/ wrong
type errors
Background Method Concept & Design Conclusion Testing
PILOT TEST - Results and Analysis 4
USABILITY
EFFECTIVENESS High Learnability (12.03%
unnecessary actions)
• 40.66% of abnormal events
solved by the users
EFFICIENCY “Normal” task completion
times
Background Method Concept & Design Conclusion Testing
PILOT TEST - Results and Analysis 4
USABILITY
SATISFACTION 50% satisfaction rating
5.25 / 6 in ease of use
Second trial slightly easier than first
Background Method Concept & Design Conclusion Testing
PILOT TEST - Results and Analysis 4
DEBRIEFING RESULTS
Background Method Concept & Design Conclusion Testing
PILOT TEST - Results and Analysis 4
AREAS FOR IMPROVEMENT
PREPARATION 1. Only qualitative testing for presence
of mental impairments
2. Inconsistencies in pacing and tone of
briefing voice.
Use of COGLAB Memory Test
Design of a briefing video for CASNUR
NORMALITY OF DATA
SOFTWARE 1. Unstoppable alarms make nurses more irritable or tense.
2. Nurses tend to forget to select the medicine first before clicking on GIVE MEDICINE.
3. Confusion with color on vital signs monitor.
4. Nurses forget to click DONE / CANCEL to return to main interface from e-cart.
5. Scattering of physician’s orders during experiment
6. Too many abnormal events occurring within the span of 5 minutes.
More runs / replications
Background Method Concept & Design Conclusion Testing
PILOT TEST - Results and Analysis 4
AREAS FOR IMPROVEMENT
SOFTWARE
ON/OFF toggling of alarm
Reprogramming of functions
Use of brighter color (YELLOW)
Placing of “GO BACK TO PATIENT”
button in e-cart and changing CANCEL
to “CANCEL addition”
Background Method Concept & Design Conclusion Testing
PILOT TEST - Results and Analysis 4
AREAS FOR IMPROVEMENT
EXPERIMENT 1. Difficulty moving mouse around.
2. More than one run per setting is
needed.
3. Subjects tend to get significantly
better after 4th / 5th try.
4. Need a basis / ideal performance
value for the task times.
5. Usability testing needs more
standardization.
Touchscreens
More subjects and runs
Do not use beyond 3 tries per
subject.
Compute for performance
value through control limit
computation & interviews with
nursing educators.
Inclusion of NASA TLX test.
Background Method Concept & Design Conclusion Testing
CONCLUSION & RECOMMENDATIONS 5
• Cognitive Work Ability (CWA) is a specialized
concept for the ability of a person to fulfill
mentally demanding tasks efficiently and
effectively.
• Simulation as performance-based method for
assessing CWA among ICU nurses.
• A simulation software has been successfully
designed with acceptable usability ratings,
reliability in data collection & simulation, and
validity (face and construct).
• Recommended use of software by the
academe, health professionals, nurses,
hospitals.
• Larger experiment with more
samples and actual derivation
and evaluation of CWA index.
• Incorporation of eye-tracking and
screen-recording features in
system to reduce CASNUR’s
usage costs.
• Testing by wider demographic
(young and old).
• Longitudinal testing of software
and CWA index.
Background Method Concept & Design Conclusion Testing
“With society becoming older and more active, will
we change the way we work…? Does it make any
sense to stop being productive at a particular age?”
– Ken Dychtwald (1990)
TRY OUT CASNUR Look for me after the presentation / session /
anytime during the IFA 2012 Conference.
Thank you. Questions / Comments? Feel free to e-mail them to:
rustiamd@gmail.com
top related