1 Information Overload and the Design and Implementation of Computerized Decision Support Charlene R. Weir, PhD Associate Director Education and Evaluation,

1

Information Overload and the Design and Implementation of

Computerized Decision Support

Charlene R. Weir, PhDAssociate Director Education and Evaluation, SLC GRECC

Associate Professor

Department of Biomedical Informatics

University of Utah

OBJECTIVES

Identify concepts and causes of Information Overload in terms of computerized clinical environments.

Discuss the relationship between information overload and different memory systems.

Describe how to match clinical task characteristics and implementation strategies in order to minimize information overload in the clinical setting.

OVERVIEW

Definition of Computerized Decision Support

The Goal and the Mandate

Are we meeting the goal?

Central Thesis

The Adaptive User

Memory, attention and control

Recommendations

The Socio-Technical Perspective

“Providing clinicians, patients or individuals

with knowledge and person specific or

population information, intelligently filtered or

presented at appropriate times, to foster

better health processes, better individual

patient care, and better population health.” Consensus Definition of DSS by A Roadmap for

National Action on Clinical Decision Support, AMIA 2006

Types of Decision Support

Alerts

Reminders

Guidelines

Information Displays

Provider Order Entry

Electronic Text/Templates

“to ensure that optimal, usable and effective

clinical decision support is widely available to

providers, patients, and individuals where and

when they need it to make health care decisions.”

A Roadmap for National Action on Clinical Decision Support

AMIA 2006

THE GOAL

ARE WE MEETING THE GOAL?Systematic Reviews - Outcomes

Garg, et al.(2005) Effects of CDSS on Practitioner Performance and Patient Outcomes: . . . . . Sig improvements in practitioner performance in

64% of studies and improvements in patient outcomes for only 13%.

Unable to aggregate due to significant unexplained variation

Recommendations: “Important issues include CDSS user acceptance, workflow integration . . . (p. 1236)

ARE WE MEETING THE GOAL?Systematic Reviews - Outcomes

Kawamoto, et al (2005). Improving clinical practice using clinical decision support systems: a systematic review. Sig improvements in practitioner performance in

68%. Patient outcomes were not examined. Unable to aggregate effects due to significant

unexplained variation Recommendations: “The promise of evidence

based medicine will be fulfilled only when strategies for implementing best practice are rigorously evidence-based themselves.” (p. 7)

ARE WE MEETING THE GOAL?Systematic Reviews - Outcomes

Shekelle P. Costs and Benefits of Health Information Technology. AHRQ Publication. Santa Monica, CA. Unable to aggregate due to significant

unexplained variation

Also published in: Chaudhry B, Want J, Wu S, et al. Systematic review: Impact of HIT on quality, efficiency, and costs of medical care. Annals of Internal Medicine 2006;144:E12-E22

CONLUSION “In summary, we identified no study or collection

of studies, outside of those from a handful of HIT leaders, that would allow a reader to make a determination about the generalizable knowledge of the system’s reported benefit.

Even if further randomized, controlled trials are performed, the generalizability of the evidence would remain low unless additional systematic, comprehensive, and relevant descriptions and measurements are made regarding how the technology is utilized, the individuals using it, and the environment it is used in.” (Shekelle, et al, p. 4)

“What do users say?”The Results of Qualitative

Studies

Types of Unintended Consequences Related to CPOE

“Great care must be taken to balance the risks of over-alerting with not alerting.

Developers should re-work clinical system interfaces to: 1) reduce collection of

redundant information; b) display relevant information in logical locations . . . “ (p.

553)

Campbell, Sittig, Ash, Guappone and Dykstra, JAMIA. 13:547-556.

Information-System Related Errors

Interface not suitable for highly interruptive context

Causes cognitive overload due to overly structured information entry and retrieval

Misrepresenting collective, interactive work as a linear, clear-cut and predictable workflow

Misrepresenting communication as information transfer.

Ash, J, Berg,M. Coiera, E. JAMIA 2004; 11:104-112

Barriers to Effective Use of VA Clinical Reminders

Patterson, et al (2004) Workload was the primary barrier Inapplicability to the situation Lack of utility and ease of use Workflow - not related to core work - duplication “Assembly line medicine” “Having physicians do clerical entry tasks”

Issues in Electronic Documentation

“overwhelmed”

“takes too much effort to sort through everything”

“I avoid reading nursing notes, they just have pages and

pages of blank fields”

“There is so much stuff put into a note, I can’t find what I

need.”

Access versus Availability

Tools to identify relevance not available

Accuracy goals are competing with efficiency goals

Weir, C and Nebeker, J (2007). Critical Issues in an Electronic Documentation System. AMIA Proceedings.

**CENTRAL THESIS** Inattention to work processes in the

implementation process is experienced as information overload to clinicians.

Because: Deviations in work-flow are perceived as

interruptions

Changes in information location and timing increases cognitive effort

The process of adaptation results in the creation of innovative strategies to decrease cognitive burden

Information Overload is really a “mismatch” between available cognitive resources and the task

Changing Work Processes

Information Overload

Adaptive Strategies

TASK ANALYSISInformation Management Strategies

Systematic selection of 13 / 133 VA sites Random selection of a primary care clinic Procedures

Site visit, observations and interviews Goal-based interviews (“in order to . .”; “by . .”)

88 participants (14 nurses, 53 ordering providers, 8 pharmacists, 2 dieticians)

About 60 hours of observation Qualitative Analysis ( tasks, common components,

and goals)

Weir, CR, et al (2007). A cognitive task analysis of information management strategies in a computerized provider order entry environment. JAMIA 14(1):65-75

Information Management Goals

Relevance Screening

Ensuring Accuracy

Minimizing Memory Load

Negotiating Responsibility

COPING STRATEGIES FOR INFORMATION INPUT OVERLOAD

Omission

Reduced Precision

Queuing

Filtering

Cutting Categories

De-centralization

Escape

Hollnagel, E and Woods, D (2005) Joint Cognitive Systems. CRC Press (p. 80)

CONCEPTS of “INFORMATION OVERLOAD”

Mismatch between us and context

Disorientation/ lost

Inability to determine relevance

Distracting/forget goal

High Effort

Lack of situational awareness

Inability to “think” or analyze

MEMORYA Tale of Two Processes

Associative Processing

Symbolic/Rule-Based Processing

ASSOCIATIVE MEMORY PROCESSING

Associative Learning: Gradual accretion of knowledge through progressive associations; expert performance is an example.

Thinking: fast, pattern-completion, effortless Awareness: Not required for performance Errors: common heuristics or “rules of thumb” Change: change is slow,hard; like “breaking

bad habits.”

VERY RESISTANT TO IMPACT OF COGNITIVE LOAD

SYMBOLIC MEMORY PROCESSING

Symbolic Learning: Fast increase in

knowledge through rules/symbols/language.

Thinking: slow, effortful, requires attention

Awareness: Required for performance

Errors: miss-identifed task, not understanding

CHANGE: change may be fast

HIGHLY SENSITIVE TO COGNITIVE LOAD

IMPLICATIONS Both types of cognition are “working”

simultaneously.

Humans prefer to minimize cognitive load, hence their behavior will likely be under the control of what they know as much as possible.

Adaptive strategies or “work-arounds” are geared to “think less.”

Experts do much more with less attention and effort.

MOTIVATION

As cognitive load increases (work, distractions), then the work will be taken care of by the associative system (less thinking, more automatic, pattern-recognition processing).

As motivation to be accurate increases, more of the work will be done by the symbolic system (new material, high patient acuity, social pressure)

RECOMMENDATIONSTask-Person-Technology Fit

Decision support for easy tasks should not require attention (they will be seen as interruptions). Increase Control Order sets and protocols Standing Orders Administrative Control (e.g. formulary) Documentation / Order Combinations Embedded tracking of behavior “just in time” heuristic

RECOMMENDATIONSTask-Person-Technology Fit

Decision support for hard/complex tasks should assist the human in active problem-solving - not replace him/her. Provide information early in the planning phase Display information by tasks (e.g.problems) Slow down the process in order to facilitate

“deeper processing” Info buttons, access to other experts/consults;

and/or scientific authoritative sources Enhance team communication

RECOMMENDATIONSTask-Person-Technology Fit

Humans vary in expertise, current conditions of cognitive load and need for choice and flexibility. User expertise and role identification - based

views.

Search, query and custom views

Tailor views to situations - busy settings may have different views

EXAMPLES Antibiotic prescribing

support Assessing decision-

making capacity Patient Education Diagnoses Identification of

Adverse Drug Events Management of HPT

Flu vaccine Depression Screening DVT prevention

protocols Prevention of

constipation resulting from narcotics

Fall Screening

High Task Complexity/DifficultyLow Task Complexity/Difficulty

Embed Interventions and Increase Control

Increase Information/High Flexibility

Current VA Research Examples

Minimizing Harm from ADEs by Improving Nurse-Patient Communication - Weir (PI)

VA funded research project

Medication Management is a complex, multi-

disciplinary activity

Analyze Medication Management

Communication Patterns

Design an Intervention that enhances

information exchange between providers on

medication management issues

“VA Integrated Medication Manager” Nebeker (PI)

VA-based development project

Improves information display for management of hypertension

Includes symptoms, medication history changes and relevant lab data.

Uses human factor methods to identify task characteristics.

THE END