The High 5s Project

The High 5s Project Interim Report

The High 5s Project Interim Report

December 2013

2The High 5s Project Interim Report

WHO Library Cataloguing-in-Publication Data

The high 5s project: interim report.

1. Patient Safety – standards. 2. Hospital Administration. 3. Medical Errors – prevention

and control. 4. Delivery of Health Care. 5. Safety Management – standards. 6.Hospitals.

7. Data Collection. 8. Program Evaluation. I.World Health Organization.

ISBN 978 92 4 150725 7 (NLM classification: WX 167)

© World Health Organization 2014

All rights reserved. Publications of the World Health Organization are available on the WHO

website (www.who.int) or can be purchased from WHO Press, World Health Organization,

20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22 791 4857;

e-mail: [email protected]).

Requests for permission to reproduce or translate WHO publications –whether for sale or for

non-commercial distribution– should be addressed to WHO Press through the WHO website

(www.who.int/about/licensing/copyright_form/en/index.html).

The designations employed and the presentation of the material in this publication do not imply

the expression of any opinion whatsoever on the part of the World Health Organization

concerning the legal status of any country, territory, city or area or of its authorities,

or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent

approximate border lines for which there may not yet be full agreement.

The mention of specific companies or of certain manufacturers’ products does not imply

that they are endorsed or recommended by the World Health Organization in preference

to others of a similar nature that are not mentioned. Errors and omissions excepted, the names

of proprietary products are distinguished by initial capital letters.

All reasonable precautions have been taken by the World Health Organization to verify

the information contained in this publication. However, the published material is being

distributed without warranty of any kind, either expressed or implied. The responsibility for

the interpretation and use of the material lies with the reader. In no event shall the World Health

Organization be liable for damages arising from its use.

Design by CommonSense, Greece

3 Acknowledgements

This Interim Report was carried out as part of

the High 5s Project set up by the World Health

Organization in 2007 and coordinated globally by

the WHO Collaborating Centre for Patient Safety,

The Joint Commission in the United States of

America, with the participation of the following

Lead Technical Agencies including: Australian

Commission on Safety and Quality in Health Care,

Australia; Canadian Patient Safety Institute, Canada

and the Institute for Safe Medication Practices

Canada, Canada; National Authority for Health-

HAS, France, with CEPPRAL (Coordination pour

L’ Evaluation des pratiques professionnelles en

santé en Rhône-Alpes), France, OMEDIT Aquitaine

(Observatoire du Medicament, Dispositifs medicaux

et Innovation Therapeutique), France (from 2012-

2015) and EVALOR (EVAluation LORraine), France

(from 2009-2011); German Agency for Quality

in Medicine, Germany and the German Coalition

for Patient Safety, Germany; CBO Dutch Institute

for Healthcare Improvement, the Netherlands;

Singapore Ministry of Health, Singapore; Trinidad

and Tobago Ministry of Health, Trinidad & Tobago;

Former National Patient Safety Agency, United

Kingdom of Great Britain and Northern Ireland;

and the Agency for Healthcare Research and

Quality, USA.

This work is a part of the High 5s Project which

has been supported by the Agency for Healthcare

Research and Quality, USA, WHO, and the

Commonwealth Fund, USA.

Acknowledgements

This report is dedicated in memoriam to Jerod M. Loeb, PhD, friend,

colleague, master of performance measurement and quality improvement.

The unique feature of the work presented here is its multidimensional

system of evaluation, for which Jerod was the guiding force. His intellect,

integrity and joie de vivre brought out the best in each of us.

5

Executive Summary ...................................................................................................................... 9

1. The purpose and objectives of the High 5s Project ........................................................................14

2. The history, concepts, and overall strategy of the High 5s Project ................................................16

1. Historical background ............................................................................................................16

2. Scope of the High 5s Project....................................................................................................17

3. Unique features of the High 5s Project ....................................................................................18

3.1. Standardization ............................................................................................................18

3.2. Evaluation....................................................................................................................19

3. The High 5s Project governance, structures, participating LTAs and hospitals, description

of the work of the Steering Group and its several committees ......................................................22

1. Overall governance ................................................................................................................ 22

2. Lead Technical Agency governance ........................................................................................ 22

3. Lead Technical Agency structure ............................................................................................ 23

4. Participating hospitals ............................................................................................................ 25

5. High 5s Steering Group .......................................................................................................... 30

6. High 5s Project Committees .................................................................................................. 30

4. The design of the Project and its protocols and evaluation framework, including an overview

of methodologies, instruments and processes ............................................................................ 34

1. Design of the Project.............................................................................................................. 34

2. Standard Operating Protocols (SOPs)...................................................................................... 34

3. High 5s evaluation framework ................................................................................................ 34

3.1. SOP implementation experience .................................................................................. 35

3.2. SOP-specific performance measures.............................................................................. 35

3.3. Event analysis .............................................................................................................. 38

3.4. Patient safety culture survey ........................................................................................ 40


Contents

5. Qualitative and quantitative interim findings................................................................................ 44

1. Summary of culture survey findings ........................................................................................ 44

1.1. Culture surveys in participating Member States ............................................................ 45

2. Context survey of Medication Reconciliation .......................................................................... 48

2.1. Medication Reconciliation context survey findings in Member States ............................ 49

3. Context survey of Correct Site Surgery.................................................................................... 50

4. Interim findings ...................................................................................................................... 51

4.1. Summary of qualitative findings .................................................................................. 51

4.2. Member State implementation experiences and qualitative findings .............................. 56

4.3 Summary of event analysis findings .............................................................................. 65

4.4. Summary of quantitative findings ................................................................................ 67

4.5. Member State quantitative and event analysis findings .................................................. 80

6. Interim outcomes ...................................................................................................................... 90

1. Feasibility .............................................................................................................................. 90

1.1. Overall issues .............................................................................................................. 90

1.2. Summary of in-country experiences .............................................................................. 92

2. Impact .................................................................................................................................. 99

2.1. Overall issues .............................................................................................................. 99

2.2. Summary of Member State experiences ...................................................................... 101

7. Developing networks and awareness raising for the High 5s Project ............................................110

1. Developing global networks and awareness raising ................................................................110

2. National networks and awareness raising................................................................................112

8. High 5s Steering Group meetings and international hospital meeting ..........................................116

1. Steering Group meetings........................................................................................................116

2. International hospital meeting ..............................................................................................116

9. Next steps ................................................................................................................................118

1. Overview ..............................................................................................................................118

2. The cross High 5s Medication Reconciliation map ..................................................................118

3. Next steps by LTAs ................................................................................................................118

Annexes ............................................................................................................................................125

7 Contents

ACS American College of Surgeons

ADE Adverse Drug Event

ACSQHC Australian Commission on Safety and Quality in Health Care

AHRQ Agency for Healthcare Research & Quality

APS German Coalition for Patient Safety

AZQ / AQuMed German Agency for Quality in Medicine

CBO Dutch Institute for healthcare Improvement (a TNO company)

CC WHO Collaborating Centre for Patient Safety

CEPPRAL Qualité et Sécurité en santé, France

CI Concentrated Injectables

CPSI Canadian Patient Safety Institute

CI Concentrated Injectables

CSHP Canadian Society of Hospital Pharmacists

CSS Correct Site Surgery

BPMH Best Possible Medication History

EA Event Analysis

ED Emergency Department

EHR Electronic Health Record

EMGO Institute for Health and Care Research, The Netherlands

EMR Electronical Medical Record

EVALOR French EVAluation en LORraine

FMEA Failure Mode and Effects Analysis

HAS French National Authority for Health

HCO Healthcare Organization

HRO High Reliability Organization

IfPS Institute for Patient Safety

IMS Information Management System

ISMP Canada Institute for Safe Medication Practices Canada

JCI Joint Commission International USA

LTA Lead Technical Agency

MDS Event Analysis – Minimum Data Set

Med Rec Medication Reconciliation

MoH Ministry of Health

MMP Medication Management Plan

MMS Medicare & Medicaid Services

NIAZ The Netherlands Institute for Accreditation in Healthcare

NIVEL The Netherlands Institute for Health Services Research

OR/OT Operating Room/ Operating Theatre

PCEHR Personally Controlled Electronic Health Record

RHAs Trinidad & Tobago Regional Health Authorities

ROP Required Organizational Practices

SOP Standard Operating Protocol

WHO World Health Organization


Acronyms

Experience with standardization in non-health care

industries clearly shows that standardized processes

lead to improved safety and service excellence.

In health care, the standardization of hospital

processes may enable trained health-care providers

to perform more consistently and reduce latent

tendencies for processes to fail, that is, to reduce

errors.

The High 5s Project (the Project) was launched

in 2007 to examine the concept of standardization

in clinical processes by implementing targeted

patient safety improvement strategies. The High 5s

name derives from the Project’s original intent to

significantly reduce the frequency of 5 challenging

patient safety problems in 5 countries over 5 years.

A key objective was to develop and assess Standard

Operating Protocols (SOPs) in a range of health-

care systems and cultures, and to provide evidence

of the benefits of standardization in health care

as one of the means to effect safety and excellence

in performance. To this end, the Project initiated

the development of SOPs for certain priority

patient safety problems, undertook a

comprehensive evaluation, and then refined the

SOPs and its measurement strategies for later use

by interested countries around the world.

The specific goals of the Project were to determine

the feasibility of implementing SOPs in hospitals

in multiple countries, and to measure the impact

on patient safety. Standardization on such a large

scale is challenging, and to be globally relevant,

it needs to be applicable in different cultures and

health-care environments. If this goal proves to be

feasible, it will create leverage to spread both the

concept of standardization and the SOPs themselves

across the broader health care community.

To achieve these goals, a group of WHO Member

States – Australia, Canada, France, Germany, the

Netherlands, Singapore, Trinidad & Tobago,

and the USA–the World Health Organization

(WHO) Patient Safety Programme and the WHO

Patient Safety Collaborating Centre (The Joint

Commission), have joined forces to develop and

implement SOPs that address recognized patient

safety challenges.

Two unique features of the High 5s Project were

the use of standardization across multi-country

settings, and a carefully-designed, multi-pronged

approach to evaluation. The major components

of the High 5s Project work included:

ñ developing and implementing Standard Operating

Protocols;

ñ developing an Impact Evaluation Strategy;

ñ collecting, analyzing and reporting data;

ñ learning from data through a Collaborative

Learning Community;

ñ disseminating knowledge globally.

Five SOPs were initially drafted but due to resource

constraints, only two were fully developed

and implemented. These were the ‘Medication

Accuracy at Transitions in Care’ (Med Rec) and

‘Correct Procedure at the Correct Body Site’

(Correct Site Surgery) SOPs. Concurrent

performance measurement of these SOPs started

in 2010 and will continue until 2014.

Governance and Lead Technical Agencies

The Project’s activities were planned and

implemented by a Steering Group and five

technical subcommittees, composed of

representatives from the participating institutions

and countries. Each Member State nominated

a Lead Technical Agency (LTA) to coordinate

and support the implementation of the SOPs

in hospitals, and to monitor their impact using

evaluation tools developed by the technical

9

Executive Summary

subcommittees. Each LTA was responsible for

coordinating the Project in its country, including

hospital training, support for evaluation and event

analysis and hospital data submission into a secure

web-based Information Management system (IMS)

developed by the Collaborating Centre to be used

for data collection, analysis and reporting.

Design of the High 5s Project

An early challenge for the Project was to apply

established best practices and evidence based

interventions in quality improvement and patient

safety to the development of standardized

processes and tools. Steering Group members

and other experts provided input to the process

designs, which included development of a

comprehensive system of evaluation that made

possible the production of comparative results

locally, nationally and internationally.

Each SOP summarizes a safety problem, proposes

a solution, presents the evidence for the solution,

identifies potential barriers to adoption, and

delineates potential unintended consequences

of the solution. Through step-by-step instructions

and tools for implementing a defined patient care

process, each SOP can be used by multiple users

in a consistent and measurable way. The SOPs are

intended to achieve systems change and health

care professionals’ behavioral change by applying

standardized, evidence-based quality improvement

methods.

With regard to evaluation, the High 5s Project set

out to answer two key questions.

1. Is it feasible to implement standardized

processes in health care within individual hospitals,

among multiple hospitals within individual Member

States, and across country boundaries?

2. What is the impact of standardization on the

safety problems that the project targets?

The basis for answering these questions was the

‘Impact Evaluation Strategy’ designed to assess

the feasibility and impact of implementing the SOPs

using both quantitative and qualitative approaches.

This strategy applied evaluation approaches that

allowed impact of the interventions to be measured

from different perspectives, including:

1. qualitative evaluation of the SOP

implementation experiences;

2. SOP-specific performance measures;

3. event analysis to identify and investigate

occurrences that may represent SOP failures;

and

4. hospital culture surveys.

An SOP implementation evaluation process

was used to determine if the SOPs could be

implemented as designed in diverse hospital

settings, to identify barriers to implementation,

and to identify ways to overcome these barriers.

Self-reported narrative and interview data were

(and continue to be) collected from the hospitals

for a deeper understanding of the feasibility and

impact of SOP implementation in multi-country

settings and environments.

The second component of the evaluation strategy

was performance measurement. Data, normally

collected on a monthly basis, were used to assess

the degree to which the SOPs were followed

by the participating hospitals.

The third component employed event analysis.

Each participating hospital was directed to

actively seek and investigate a pre-determined set

of patient safety problems (events) that should

have been prevented by the SOP. A systematic

analysis of the facts and contributing factors

of a patient safety incident, and whether it was

linked to the design and/or implementation

of the SOP took place.

The fourth and complementary area of action

was the voluntary administration of a patient safety

culture survey in hospitals. These surveys were

done to provide insight into the patient safety

culture of hospitals participating in the High 5s

Project and to sensitize frontline staff and hospital

management about staff members’ perception

of the hospital’s patient safety culture.

Interim Findings

All country-specific sections in this report reflect

the status of 2nd quarter 2013.

Patient safety culture surveys

Overall, culture survey results showed the

highest scores on teamwork issues within units

(70% positive), and the lowest scores on handoffs

and transitions (34% positive). The percentage

of respondents giving the hospital’s work area/unit

a patient safety grade of an ‘A-Excellent’ or

‘B-Very Good’ ranged from 70% to 21%. Percent

of hospitals, by country, reporting one or more


events ranged from 78% to 37%. Culture survey

results were limited by inconsistent statistical

selection of samples of hospitals, variation in

number of respondents per country, variable

administration of the surveys, and variable quality

of submitted data.

Context surveys

It was considered that the ‘context surveys’

would be useful to understand what external

factors were influencing the uptake of the SOPs

in the different Member States.

Qualitative findings

Hospitals shared their implementation experiences

through quarterly and then six-monthly reports.

For purposes of analyzing and reporting evaluation

data, the hospitals were asked to determine

whether implementation of a SOP was ‘full’–

when all required components of the SOP were

in place across all eligible locations and patient

populations within the hospital– or ‘partial’.

For the Correct Site Surgery SOP implementation,

challenges related to assembling a representative

group to oversee implementation, lack of resources,

resistance to change, poor communication or lack

of leadership were identified. Identification of areas

of potential breakdown through a risk- assessment

allowed them to implement controls, warnings or

protections to minimize identified problem areas.

Much was learned about SOP implementation from

the initial pilot testing which most hospitals did

before starting full SOP implementation. During

piloting, the hospitals were able to identify barriers

and correct them prior to full implementation

through increased communication, enlisting staff

and leadership support, and providing additional

training. Further, many hospitals reported that they

were not using the prescribed High 5s data quality

methodology. However, most hospitals were

performing some form of data quality monitoring,

conducting audits, soliciting information from staff,

engaging in project oversight meetings and

identifying opportunities for improvement.

Hospitals’ implementation experiences for the

Medication Reconciliation SOP emphasized lack of

resources and challenges in assembling an oversight

group. Most hospitals did not complete a formal

risk assessment because of lack of resources or

because it was considered a low priority. Those that

did conduct risk assessment reported that it was

beneficial in helping to identify potential areas

for process breakdown. Although some hospitals

reported that the SOP implementation was an

extension of their existing medication reconciliation

process, many barriers to implementation were

reported including limited resources, data

collection burden, insufficient communication, and

resistance to change. Most responding hospitals

indicated technical challenges and technical issues

with the SOP performance measures.

A detailed description of implementation

experiences for both SOPs in Member States is

presented in the main body of the Interim Report.

Overall, they point to leadership and stakeholder

commitment and support as well as an

organizational culture of safety, adequate

resources, team cohesion, and ongoing training

to be among the key factors for successful SOP

implementation. The principle barrier was the

burden of data collection, although hospitals

themselves appreciated the value of measurement

as a means to successful improvement. Another

major challenge to implementation was resistance

to change by providers and leaders.

Quantitative findings

Hospital performance measurement data were

collected to provide a profile of each SOP

implementation in Member States. Details

are presented in section 5.4 of this report.

Interim Conclusions

Feasibility

In this context, feasibility refers to the degree

to which an SOP can be implemented as it was

originally defined, in a standardized way across

hospitals within a country and in multiple countries.

Over the course of the High 5s Project it became

clear that some aspects of the SOPs were easier

to implement than others. When participants

encountered issues with the implementation of

an SOP, they were invited to submit requests to

adapt or revise components of the SOP. Overall,

relatively few adaptations and revisions were required,

suggesting that a high degree of standardization

of health care processes was seen as feasible.

Impact

The SOPs had a significant impact in hospitals in

all participating Member States. As a result of

implementation of the SOP, certain process steps

were introduced or improved in the Member States

which were not part of everyday care previously.

For example, site-marking was introduced in French

11 Executive summary

hospitals. In Singapore, the improvement in clinical

practice and staff cooperation was reflected in

the performance measure data, which showed 98%

compliance with complete preoperative verification

and Time Outs, and improvement in surgical

site-marking from 40% to 97%. Safety components

are now built into surgical work flows and a culture

of shared ideas and learning among hospitals has

been promoted.

In the case of the Med Rec SOP, the results were

encouraging in that some hospitals managed

to reduce medication inaccuracies substantially.

Implementation of the SOP had an impact in

improving the quality of the medication histories

obtained, resolving discrepancies, and other

related patient care activities.

In the case of the Correct Site Surgery SOP,

the qualitative evaluation showed positive effects

in stimulating organizational safety cultures,

improving communication and teamwork in the OR

and beyond, optimizing surgical care, improving

the quality of patient records and demonstrating

that hospital processes can be re-designed to

provide better patient care. Because of the

infrequency of incorrect surgeries, it is not possible

at this point to demonstrate a measureable

decline in the number of these events. However,

implementing this SOP has resulted in an increase

in the number of identified and resolved

discrepancies in the surgical preparation process,

each of which carries a risk of incorrect surgery.

Developing networks

A major goal of the High 5s Project was to build

national and global networks. The initiative’s early

focus concentrated on building government and

ministerial support for implementing standardised

patient safety solutions by encouraging Member

States’ expert institutions to participate in the

Project. In later stages, the design of all steps in

the SOP development and the implementation

and evaluation plan included engagement of

networks of hospitals. The networks and the

learning communities they comprised have been

successful in raising awareness of the SOPs and

the benefits they bring to institutions and patients.

A good networking achievement was the

participation of 31 hospitals at the ‘International

Hospital Meeting’ at WHO Headquarters in

October 2012, to exchange knowledge and

implementation experiences and further strengthen

and motivate hospitals’ commitment.

Another example of an effective networking tool

is the interactive online map developed by the

Institute for Safe Medication Practices (ISMP)

Canada, the Med Rec SOP lead, to profile High 5s

Project achievements in the participating Member

States which have implemented the Med Rec

SOP. The map includes Med Rec implementation

site information and highlights High 5s Med Rec

implementation publications and national supports

such as accreditation standards. The anticipated

spread of the Med Rec SOP will be reflected in

this map.

Next steps

There is growing consensus that implementation

and evaluation of the SOPs is progressing

successfully in participating hospitals. Next steps

for the LTAs involve the active spread of the SOPs

and implementation experiences to more hospitals

within each Member State and beyond. Plans to

disseminate the SOPs and share the evaluation

methodologies with more WHO Member States

are also being developed. It is expected that

at the last Steering Group meetings in 2014,

discussions will focus on how to expand

the networks to the developing world.


1The purpose and objectivesof the High 5sProject

The High 5s Project is a collaboration among

a group of WHO Member States, the World

Health Organization (WHO) Patient Safety

Programme and the WHO Patient Safety

Collaborating Centre - The Joint Commission

to achieve measurable and sustainable reductions

in challenging patient safety problems through

the implementation of standardized operating

protocols.

The basic assumption being tested in the High 5s

Project is that process standardization, with

minimal variation, can improve patient safety

across national boundaries. This is being evaluated

by assessing the feasibility and impact of

implementing standardized approaches to specific

patient safety problems across multiple WHO

Member States and cultures.

Achieving process consistency, while retaining

the ability to recognize and accommodate variation

in the input (for example, the patient’s severity

of illness, co-morbidities), is one of the major

challenges to standardization in health care.

Process variation to meet individual patient needs

is essential in care delivery. But variation to meet

the preferences of health-care organizations or

individual practitioners need not be. Standardization

supersedes “best practice” when it comes to safety.

To test this, the High 5s Project has taken this

thesis to another level by standardizing certain

processes within multiple organizations in different

WHO Member States around the world.


The purpose and objectives of the High 5s Project

2The history,concepts and overallstrategy of the High 5sProject

The Project is a global health sector collaboration

reflecting the efforts of stakeholders in patient

safety from the government agencies of Australia,

Canada, France, Germany, the Netherlands, Singapore,

Trinidad & Tobago and the USA, as well as research

and patient safety organizations from these Member

States, and other international organizations.

Working together, the High 5s partners achieve

objectives that no single organization or patient

safety agency could achieve individually. These

include:

ñ accelerating the development and

implementation of standardized patient safety

solutions and addressing risk areas having

a high burden of morbidity and mortality;

ñ testing them across different settings and

country cultures;

ñ introducing robust tools for implementing

and evaluating the solutions; and

ñ generating learning and sharing knowledge

globally.

In action, the High 5s Project draws on the specific

strengths of its current and former partners. Since

the Project’s initiation the following types of

organizations have contributed to its activities:

ñ Lead Technical Agencies and technical institutes

from participating Member States;

ñ WHO Collaborating Centre for Patient Safety

designated as The Joint Commission and the Joint

Commission International;

ñ WHO.

The Member States involved in the initiation of the

High 5s Project were Australia, Canada, Germany,

the Netherlands, New Zealand, the United Kingdom

of Great Britain and Northern Ireland, and the

United States of America. France and Singapore

joined the High 5s Project in 2008 and 2009

respectively. In 2010, Trinidad and Tobago joined

as well. Several technical institutes/agencies from

these countries have contributed to the work of

the High 5s Project. These include:

ñ The Canadian Patient Safety Institute (CPSI);

ñ The Institute for Safe Medication Practices

Canada, Canada (ISMP Canada);

ñ The former National Patient Safety Agency, UK.

The Project has been supported by the Agency

for Healthcare Research and Quality, WHO, and

the Commonwealth Fund. It is being coordinated

by the WHO Collaborating Centre for Patient

Safety- The Joint Commission and JCI (hereafter

the Collaborating Centre).

1. Historical background

In 2002, the World Health Assembly recognized

the need to promote patient safety as a fundamental

principle of all health systems and called on the

World Health Organization (WHO) to develop

global norms and standards; promote the framing

of evidence-based policies and mechanisms to

recognize excellence in patient safety globally;

encourage research on patient safety; and support

efforts by Member States to improve the safety

of care. Subsequently, WHO initiated efforts to

address patient safety and established the World

Alliance for Patient Safety in 2004, renamed as

the Patient Safety Programme in 2009.

At the 2005 annual Commonwealth Fund

International Symposium on Health Care Policy,

the need for developing and implementing

standardized patient safety processes to reduce

the magnitude of adverse events was discussed.

The High 5s Project emerged as a result of:

ñ a shift in priorities to address the growing

problems of unsafe care;


The history, concepts and overall strategy of the High 5s Project

ñ an increasing recognition of the importance

of patient safety process improvement as the

means to safer care;

ñ a desire to see if standardized approaches might

be the key to improving and maintaining safety

across different settings and cultures;

ñ a growing interest among WHO, The Joint

Commission, and the Ministers of Health of

Australia, Canada, Germany, the Netherlands,

the United Kingdom and the United States of

America to develop, implement and evaluate

standardized approaches for making care safer.

The High 5s name derives from the Project’s

original intent to significantly reduce the frequency

of 5 challenging patient safety problems in 5

Member States over 5 years.

At the 2006 annual Commonwealth Fund

Symposium, Ministers of Health of Canada,

Germany, the Netherlands, New Zealand,

the United Kingdom and the United States of

America signed formal letters of commitment

to participate in and support the High 5s Project

together with WHO, the Commonwealth Fund,

and the Collaborating Centre. The Project began

to take shape in late 2006 with the nomination

of representatives from the Lead Technical

Agencies (LTAs) of participating Member States

who subsequently constituted the nucleus of

the High 5s Steering Group.

Lead Technical Agencies

Each participating Member State nominated

a LTA which subsequently coordinated and

supported the implementation of Standard

Operating Protocols (SOPs) at hospitals and

monitored their impact through applying evaluation

tools developed by the High 5s Project Steering

Group members. LTAs underwent intense initial

training and were given high visibility and

recognition for their willingness to implement

and evaluate the SOPs and for their leadership

in working to standardize patient care processes

in their country’s health-care system.

Each LTA was responsible for identifying at least

10 hospitals per SOP it decided to implement;

providing hospital training in the use of the SOPs

and Project evaluation process and tools; and

collecting data from participating hospitals and

submitting it to the Collaborating Centre (CC)

for analysis and tracking. Following completion

of LTA and hospital training, the SOPs began to be

implemented in some participating hospitals

in the autumn of 2009. Data collection began

in 2010 and will finish by September of 2014.

2. Scope of the High 5s Project

The mission of the High 5s Project is to facilitate

implementation and evaluation of standardized

patient safety solutions within a global learning

community to achieve measurable, significant,

and sustainable reductions in challenging patient

safety problems.

The major components of the High 5s work

included:

ñ developing and implementing SOPs;

ñ developing an Impact Evaluation Strategy;

ñ collecting, reporting, and analyzing data;

ñ developing a Collaborative Learning Community;

ñ disseminating knowledge learned globally.

The Project was designed to generate learning

that would permit the continuous refinement

and improvement of the SOPs within selected

hospitals of participating Member States, as well

as assessment of the feasibility and impact of


patient safety problems across multiple Member

States and cultures. The Project has built on

existing structures in the participating Member

States. SOP implementation was expected

to provide valuable lessons and knowledge

to support the advancement of patient safety

around the world.

Five SOPs and associated evaluation instruments

were developed between 2007 and 2009 to address

the following challenges:

1. Medication Accuracy at Transitions in Care;

2. Correct Procedure at the Correct Body Site;

3. Use of Concentrated Injectable Medicines;

4. Communication During Patient Care Handovers;

5. Health Care-Associated Infections.

The first two SOPs, ‘Medication Accuracy at

Transitions in Care (Medication Reconciliation)’

and ‘Correct Procedure at the Correct Body Site

(Correct Site Surgery)’ have been implemented by

the participating Member States. Implementation

of the third SOP ‘Use of Concentrated Injectable

Medicines’ was initiated in the UK, but testing

ceased in 2010 when this Member State disengaged

from the High 5s Project. Planning towards

17 The history, concepts and overall strategy of the High 5s Project

implementation of this SOP has been re-initiated

in 2013 by the Dutch Lead Technical Agency.

Extensive investigation of the potential to

implement a standardized approach to

‘Communication During Patient Care Handovers’

found that this SOP was heavily influenced by

cultural and environmental issues that were not

measurable and easy to standardize.

Therefore, work on this SOP was abandoned

in 2010. The ‘Health care-associated infections’

SOP was withdrawn from further development

and testing because parallel work towards

standardizing procedures to address health

care-associated infections was taking place

independently by WHO and other global

organizations.


Table 1: SOP implementation by WHO Member States between 2009-2014

Member State CSS SOP Med Rec SOP Conc. Inject. SOP

Australia �

Germany � �

France � �

The Netherlands � � (initiated in 2013)

Singapore �

Trinidad & Tobago �

U.K. � (stopped in 2010)

U.S.A. � � (initiated in 2013)

3. Unique features of the High 5s Project

Two unique features of the High 5s Project are

the use of standardization across multi-country

settings, and a carefully-designed, multi-pronged

approach to evaluation.

3.1. Standardization

Standardization is the process of developing,

agreeing upon, and implementing technical or

uniform specifications, criteria, methods, processes,

designs or practices that can increase compatibility,

interoperability, safety, repeatability, and quality.

Process standardization is the specification and

communication of a process at a level of detail

sufficient to permit consistent and verifiable

implementation by different users at different

times and in different settings. Standardization

reduces variation: ‘The tendency for a process to

fail is also diminished in relation to the consistency

with which it is carried out; that is, the degree

to which it is standardized.’1 This enables shared

learning, facilitates multi-disciplinary teamwork

and improves efficiency in interactions by

establishing optimum conditions. It also allows

for consistency during changes in scale and the

transfer of processes between people or

organizations – attributes which are essential

for global interactions.

Standardization in health care

The most commonly referenced examples of

standardization are by high reliability organizations

(HROs) in industries such as nuclear energy or

aviation which have well ingrained organizational

safety cultures and standardized management

approaches toward risk. These HROs serve as

an example for improving safety by focusing

on standardizing the systems involved.

Experience with standardization in these industries

is decades ahead of health care. Their standardized

safety approaches acknowledge the prospect

of failure from the start of the process,

and efforts are made to build in safety from

the beginning to the end. This contrasts with

health care, where successful outcomes

are often assumed to be the baseline. Recent

efforts to standardize health-care processes

have been slow to demonstrate their impact on

the delivery of care processes.

Standardization in patient safety

There are examples of standardization in the

area of patient safety informatics2, classification

and definitions3,4,5, and some recent development

in standardizing interventions6, protocols7,8,

outcome measurement9, data collection10,

patient handovers11, medication use12, and patient

monitoring13.

The key to optimizing patient safety is to design

systems that prevent the inevitability of human

error from actually reaching and harming the

patient14. Standardization takes the sharing

of success stories and best practices to a new

level – allowing sharing of new approaches

among health-care workers. Health-care providers

themselves have underscored the need for

standardization in patient safety.

The benefits of implementing standardized patient

safety processes and evaluating that implementation

and its impact are outlined in Box 1.

19 The history, concepts and overall strategy of the High 5s Project

Box 1: Benefits of implementing and evaluating standardized protocols

ñ Standardization provides policy and decision-makers, and health-care workers a means to compare

actions and outcomes implemented within or between groups.

ñ Standardization better enables investigators to compare data and to interpret the relevance and

efficacy of an intervention.

ñ Through standardization, more health-care workers will be able to relate to one another in

meaningful ways (including the standardization of terms used).

ñ As more and more hospitals begin to use the same protocols with the same data fields, the ability

to analyze risk will be enhanced.

ñ Health-care workers who become proficient with SOPs will be constantly building on a solid

foundation, rather than struggling to grasp the range of safety concepts that might otherwise arise

in an unstructured environment.

ñ Standardization will allow health-care workers to learn from each other’s experiences (i.e., new ideas

on how to address problems – what has worked, what has not and why).

3.2. Evaluation

The second unique feature of the High 5s Project

relates to its integrated, multi-pronged approach

to evaluation. The triangulated impact evaluation

approach being used in this Project was necessary

because of the difficulties in assessing the impact

of preventive patient safety interventions,

particularly where the targeted adverse event can

occur relatively infrequently, such as in the case of

wrong- site surgery. The High 5s Project evaluation

approach addresses the impact measurement

challenge from different perspectives, including

qualitative data collection regarding the SOP

implementation experiences, quantitative

performance measurement results, event analysis,

and organization culture assessment. These methods

were used to assess the feasibility and impact of

implementing standardized patient safety protocols.

A secure web-based Information Management

System (IMS) was developed to facilitate the

storage, analysis, dissemination and exchange

of data. The system was designed around a secure

web-based application called TWiki (version

TWiki-4.2.4, Plugin API version 1.2). Several levels

of security were incorporated into the IMS

to protect data submitted. Three types of data

were gathered through the IMS:

1) narrative descriptions of the implementation

experience;

2) aggregate counts of hospital-level data used

to calculate performance measure results; and

3) de-identified and aggregated data from event

analyses conducted by participating hospitals.

This design permits participants to come together

on a global electronic learning platform.

Participating hospitals and LTAs were also asked

to use a standardized approach - the High 5s Data

Quality Management Programme - to assess

and ensure the quality of their data. To manage

data quality, hospitals, LTAs and the CC assumed

the following responsibilities:

ñ the hospitals were responsible for re-collecting

data through independent observation or

‘re-abstraction’ to determine the “data element

agreement rate” or to identify inconsistencies

among the different components of the

evaluation process;

ñ the LTAs were responsible for overseeing

the hospitals’ implementation of data quality

activities, identifying and evaluating results

from and among their participating hospitals,

and reporting to the CC “the extent and nature

of unresolved discrepancies and any identified

data collection problems”;

ñ the CC was responsible for overseeing the LTAs’

data quality activities. Assessing the data quality

information received from the LTAs, reporting

the reliability and completeness of submitted

data to the Evaluation Committee and presenting

its views on the implications of the findings

on achieving data quality goals of the High 5s

initiative.

Assessment of data quality focuses on the

completeness and reliability of the data for each

component of the High 5s Impact Evaluation

Strategy, including narrative data, performance

measurement data, and event analysis data.

The standardization of data collection and

reporting contributes to the following objectives:

ñ ensure comparability of performance and quality

measures across health-care systems;

ñ improve health-care quality15;

ñ share information/data across health care

systems and increase the usefulness, integration

and exchange of data16;

ñ improve efficiency of performance measurement

over time; and

ñ facilitate coordination and cooperation among

all parties in performance measurement and

health care quality improvement17.


1 Croteau RJ, Schyve PM, Chapter 8, “Proactively Error-Proofing Health Care Processes” Error Reduction in Health

Care, 2nd Edition Patrice Spath, editor Jossey-Bass Publishers San Francisco, 2010

2 Bakken S, Cimino J, Hripcsak G. Promoting patient safety and enabling evidence-based practice through informatics

Medical Care, 2004. 42(2):49-56, (sup, Feb 2004).

3 Odwazny R, Hasler S, Abrams R, McNutt R. Organizational and cultural changes for providing safe patient care.

Quality Management in Health Care, 2005, 14/3:132-143.

4 Hellings J, Vleugels A. Patient safety in hospitals: Context and concepts Tijdschriftvoor Geneeskunde, 2007;

63(16):743-750.

5 Chang A, Schyve PM et al. The JCAHO patient safety event taxonomy: a standardised terminology and classification

scheme for near misses and adverse events. Int J for Qual in Health Care 2005.

6 The Health Roundtable Ltd. Australia. SAFE Patient Care Program 2009 – Standardising Actions for Excellent Patient

Care.

7 O’Connor, EJ, Fiol, CM. Resistance to Patient Safety Initiatives. Physician Executive, 01 November 2005; 31(6):64-67.

8 Rozich JD, Howard Ramona J et al. Standardization as a Mechanism to Improve Safety in Health Care. Joint

Commission Journal on Quality and Patient Safety. 2004; 30(1): 5-14.

9 Weiser TG, Makary MA et al; Standardised metrics for global surgical surveillance The Lancet 2009; 374: 1113–17.

10 Runciman, WB, Baker GR, Michel P, Larizgoitia I, Lilford, RJ, Andermann A, Flin, R, Weeks WB. The epistemology

of patient safety research. International Journal of Evidence-Based Healthcare. 2008; 6(4):476-486.

11 Patterson ES. Structuring flexibility: the potential good, bad and ugly in standardisation of handovers. QualSaf

Health Care. 2008;17(1):4-5.

12 Kazandjian VA, Ogunbo S, Wicker KG, Vaida AJ, Pipesh F. Enhancing medication use safety: benefits of learning

from your peers.QualSaf Health Care. 2009;18(5):331-5.

13 Davies M, Tales H. Enhancing Patient Safety Through a Standardized Model of Physiologic Monitoring Healthcare

Quarterly, 8(Sp) 2005: 49-52.

14 O’Leary DS. Patient safety: the search for global solutions. World Hosp Health Serv. 2008;44(1):19-21.29.

Franklin, B.D., Standardization is key to improving patient safety. Pharmaceutical Journal, 2003; 271.

15 O’Leary DS. Patient safety: the search for global solutions. World Hosp Health Serv. 2008;44(1):19-21.

16 Standardization for Health Care Quality Improvement. http://www.ahrq.gov/research/iomracereport/reldata5.htm

17 AHRQ Conference on Health Care Data Collection and Reporting, AHRQ Publication No. 07-0033-EF, March 2007.

3The High 5sProjectgovernance,structures andparticipatingorganizatons

1. Overall governance

The High 5s Project is a collaborative whose

participants include representatives from the LTAs

of the participating Member States, WHO, and

the Collaborating Centre, and experts from

associated national institutions or agencies

which are not LTAs. In addition to the development

of the SOPs and associated implementation and

evaluation instruments, collaborative actions

include participation in strategy setting, advocacy,

fund-raising, Member State and hospital

mobilization, Project implementation, data

collection, evaluation and knowledge dissemination.

The Project’s plans and activities are governed

by a Steering Group composed of representatives

from the WHO Patient Safety Programme,

the Collaborating Centre, the LTAs and associated

experts. The Steering Group meets twice a year

to review progress achieved; set future direction

for the Project; approve technical plans, activities

and products; review data; and review and act

on committee recommendations.

The websites of the High 5s Project organizations,

agencies, institutions and LTAs are presented in

Annex 1.

2. Lead Technical Agency (LTA)governance

The Australian LTA: The Australian Commission on

Safety and Quality in Health Care (the Commission)

is a government agency which was established to

lead and coordinate national improvements in

safety and quality in health care across Australia.

It provides a number of programmes, publications

and resources to support health-care professionals,

health-care organizations and health-care policy

makers, and works with patients and care- givers

to deliver safe, high quality health care across

Australia. The Commission is responsible for the

conduct and governance of the High 5s Project

in Australia. The SOP selected for implementation

in Australian hospitals was ‘Assuring medication

accuracy at transitions of care’.

The French LTA: The French National Authority

for Health (Haute Autorité de Santé, HAS), is an

independent public authority, located near Paris.

HAS contributes to the regulation of health system

quality. Its mission is in the field of evaluation of

health products, professional practices, organization

of care and in public health. With the commitment

of the French Ministry of Health HAS joined the

Project in 2008 as the French Lead Technical Agency

The SOPs selected for implementation were the

‘Assuring medication accuracy at transitions of care’

(Med Rec) and the Correct Site Surgery (CSS) SOPs.

HAS is responsible for the overall management

and has partnered with two regional organizations.

These receive funding from HAS for supporting

participant, hospitals and monitoring

implementation and evaluation of the two SOPs:

ñ CEPPRAL, a regional quality and security of care

organization, has been responsible for follow-up

and evaluation of the CSS SOP implementation;

ñ EVALOR (EVAluation en LORraine) was

responsible for the first two years of the Med Rec

SOP implementation. In 2012, this task was

turned over to OMEDIT (Observatory of Drugs,

Medical Devices and Therapeutic Innovations

of Aquitaine) whose mission addresses the quality

and safety of medication throughout the care

pathway.

The German LTAs: The German Federal Ministry

of Health has been funding the High 5s Project in

Germany since the end of 2007. Two organizations

were appointed to share the tasks in implementing

the High 5s Project on a national level:


The High 5s Project governance, structures,participating LTAs and hospitals, descriptionof the work of the Steering Group and itsseveral committees

ñ The German Agency for Quality in Medicine

(AQuMed, ÄZQ): Located in Berlin, AQuMed

is a non-profit organization owned by the

German Medical Association and the National

Association of Statutory Health Insurance

Physicians. AQuMed coordinates health-care

quality programmes with a special focus on

evidence-based medicine, clinical practice

guidelines, patient empowerment, patient safety

programmes, and quality management;

ñ The German Coalition for Patient Safety (APS):

The German Coalition for Patient Safety is

a non-profit association of health-care

professionals, institutions and patient organizations

whose mission is to improve patient safety

in Germany. The German Coalition for Patient

Safety has appointed the Institute for Patient

Safety (IfPS) of the University of Bonn to carry

out its tasks in the High 5s Project.

The SOPs selected to be implemented by German

LTAs were ‘Assuring medication accuracy at

transitions of care’ and Correct Site Surgery (CSS).

The Dutch (Netherlands) LTA: With the support

of the Dutch Ministry of Health, CBO Dutch

Institute for Healthcare Improvement (a TNO

company) located in Utrecht, assumed responsibility

as the LTA in 2009. The purpose of Dutch

participation in this international collaboration

was primarily to introduce and put into practice

international expertise in Dutch hospitals

and provide technical support to national and

international development of patient safety

solutions. The SOP selected for implementation in

Dutch hospitals was ‘Assuring medication accuracy

at transitions of care’ (Med Rec). In 2013, the LTA

started the ‘Concentrated Injectable Medicines’

SOP for implementation in Dutch hospitals.

The Singaporean LTA was formed to support

the High 5s Project following the expression

of interest to participate in the High 5s project

by Singapore’s Minister of Health in May 2009.

The LTA comprises representatives from the

Standards and Quality Improvement Division in

the Singapore Ministry of Health. The SOP

selected to be implemented by Singapore’s LTA

was Correct Site Surgery (CSS).

The Trinidad & Tobago LTA: In May 2011,

Trinidad & Tobago participated at the High 5s

Steering Group Meeting in Berlin to obtain first

hand experiences from Member States on

implementation of the SOPs. The Collaborative

Action Statement was signed by the Minister

of Health in August 2011, signaling the Ministry’s

support to the Project, and confirming the Ministry

of Health as the LTA for Trinidad and Tobago.

The SOP selected to be implemented by the Trinidad

& Tobago LTA was Correct Site Surgery (CSS).

The United States of America (U.S.A.) LTA:

The Agency for Healthcare Research and Quality

(AHRQ) serves as the official U.S.A. LTA. Since it

has no formal health care delivery responsibilities,

AHRQ has delegated SOP implementation

responsibilities to private sector groups. The first

of these is the American College of Surgeons

(ACS). The ACS is a scientific and educational

association of surgeons that was founded in 1913

to improve the quality of care for the surgical

patient by setting high standards for surgical

education and practice. The ACS is headquartered

in Chicago, Illinois. AHRQ is located in Rockville,

Maryland, is committed to improving care safety

and quality by developing successful partnerships

and generating the knowledge and tools required

for long-term improvement. AHRQ has been

a financial supporter for the High 5s Project.

Thus the ACS has assumed some of the LTA

responsibilities.

The initial SOP selected to be implemented by

the U.S.A. LTA was Correct Site Surgery (CSS).

At the beginning of 2013, The Wheaton Franciscan

system of hospitals volunteered to assume

responsibility for implementing the Med Rec SOP.

3. Lead Technical Agency structure

The Australian LTA: The High 5s Project sits

within the Commission’s Medication Safety Program

and is overseen by an expert advisory group,

The Medication Continuity Expert Advisory Group,

which comprises representatives from medical,

pharmacy, nursing and consumer organizations,

state and territory departments of health, academia

and individual experts from hospital, residential

aged care and community health-care sectors.

The group advises on the conduct of the Project

and reports through the Commission’s Medication

Reference Group to the Commission Board.

The High 5s Project is supported by a senior

project officer and a project manager who are

responsible for the day-to-day management

of the project activities.

23 The High 5s Project governance, structures and participating organizations

The French LTAs: The National Authority

for Health (Haute Autorité de Santé, HAS) and

the EVALOR/OMEDIT and CEPPRAL teams provide

support and assistance to participating hospitals

with respect to training, SOP implementation,

and data collection, analysis and evaluation.

A national pilot committee, led by HAS, was set up

in 2010 and includes representatives from the

Ministry of Health, sanitary agencies, the patient

community, hospital federations, and scientific

societies. This group met annually during the two

first years of the Project. Two committees (one

for each of the SOPs) – co- chaired by the LTA

and including participating hospital representatives

and scientific experts – have been set up and

convene workshops every six months.

The German LTA: The two LTA partner

organizations in Germany share the tasks and

work closely together in implementing the High 5s

Project on a national level:

ñ AQuMed is responsible for overall project

management, recruiting hospitals,

implementation of the SOPs, supporting the

hospitals, and representing the German LTA on

the international level of the High 5s Project;

ñ on behalf of the German Coalition for Patient

Safety, the Institute for Patient Safety (IfPS)

of the University of Bonn carries out its tasks

in the High 5s Project. The IfPS is responsible

for the evaluation tasks which encompass data

management, analysis and feedback to hospitals.

The Ministry of Health convened an expert advisory

committee consisting of leading national experts

on the SOP topics. The expert advisory committee,

representatives from the Federal Ministry of Health

and the project teams from AQuMed and IfPS meet

yearly to discuss strategic aspects of their

involvement in the High 5s Project.

The Dutch (The Netherlands) LTA: The Dutch

Institute for Healthcare Improvement (CBO)

nominated a project leader and an advisor to

oversee the tasks and work of the High 5s project

in the Netherlands. National implementation

networks for ‘Assuring medication accuracy at

transitions of care’ and for Concentrated Injectable

Medicines SOPs was then established. The

implementation strategy for the SOPs was based

on the ‘Breakthrough Series’ developed by the

Institute for Healthcare Improvement in 1994.

The objective of participating hospitals was to

strengthen both the implementation of the national

guidelines for Medication Accuracy at Transitions

in Care (2008) and High Risk Medication (preparing

for administration) (2009) the Medication

Reconciliation and High Risk themes of the national

patient safety programme (2008-2012).

The Singaporean LTA set up three key structures

that aided the facilitation of the High 5s Project.

First, it engaged two leading anaesthetists from

two of the largest public hospitals in the country as

part-time consultants to help in workflow redesign

of operating theatres. Second, it set up a local High

5s Network comprising surgeons, anaesthetists,

and OT nurse managers from all public hospitals.

Each public hospital has two representatives

participating in this Network. The Network was

co-chaired by the LTA’s two part time consultants.

Third, with regard to implementation of the

Correct Site Surgery (CSS) SOP, the LTA funded

each hospital with an operational executive to

adapt the SOP checklists, collect data, conduct

staff education and address identified gaps.

The Trinidad & Tobago LTA: Letters of invitation

to participate in the High 5s Project were sent from

the Ministry of Health to all five Regional Health

Authorities (RHAs) to participate in the

implementation of the Correct Site Surgery SOP in

October 2011. All five RHAs indicated willingness

to implement this SOP in the surgical units of their

secondary care hospitals. The Health Care Protocol

Officer of the Directorate of Quality Management

of the Ministry of Health is the focal Project person

and liaison with the project teams at the RHAs.

Each project team is headed by a designated Project

Team Leader whose responsibility is to ensure

implementation of the SOP at the surgical units of

the respective hospitals. The Quality Improvement

Units at the RHAs and the Quality Directorate

at the Ministry of Health provide technical and

logistical support.

The U.S.A. LTA: AHRQ has supported the High 5s

Project through an expert consultant, in-house

technical contributions and funding of the global

and technical activities of the Project over a period

of four years. The American College of Surgeons

provides support to the High 5s Project through

two of its staff members who serve as consultants

and provide technical expertise on the Correct Site

Surgery SOP.


4. Participating hospitals

Table 2: Participating hospitals in each Member State


Hospital /health service Description

The Alfred Hospital Acute care, 400 bed public tertiary referral hospital in Victoria

Armadale Health Service Acute care, 250 bed public hospital in Western Australia

Epworth HealthCare Acute care, 550 bed private hospital in Victoria

Greater Southern Area Health 9 rural public hospitals in southern New South Wales

Service (9 hospitals) (total of 810 beds across 9 sites)

Logan Hospital* Acute care 390 bed public hospital in Queensland

Mater Health Services (3 hospitals) Tertiary hospital with public and private beds

(in total approximately 1000 beds) consisting of

Mater Adult Hospital (Public)

Mater Children’s Hospital (Public and Private)

Mater Mothers’ Hospital (Public and Private)

Mater Private Hospital in Queensland

Noosa Hospital Acute care, 92 bed private hospital in Queensland

North West Regional Hospital Acute care, 120 bed public hospital in Tasmania

Prince of Wales Hospital Acute care, 550 bed public tertiary referral hospital in New

South Wales

Redland Hospital Acute care, 150 bed public hospital in Queensland

Rockingham Peel Group Acute care, 180 bed public hospital in Western Australia

Royal North Shore Hospital Acute care, 560 bed public tertiary referral hospital in New

South Wales

The Wesley Hospital Acute care, 530 bed private hospital in Queensland

Australia

The Australian collaborative commenced in January

2010 with 18 health services comprising 28 hospitals

recruited through an expression of interest process.

Five health services have since withdrawn from the

project. Reasons given for withdrawal included

changes in priorities, loss of project champions, lack

of resources, and the need to prioritize work to

direct patient care. Participants comprise a mix of

public and private hospitals of differing sizes and

complexity from regional centres and capital cities

in five states. See table 1 for list and description of

the participating hospitals.

(Withdrew from project in October 2012.)

France

A total of 18 health-care organizations (HCO)

were selected (November 2009) and entered

the Project in 2010. The LTA defined selection

criteria* and developed a questionnaire which

was followed-up by an interview conducted

with each potential hospital project coordinator.

Altogether, nine hospitals were recruited for


each SOP. In 2011, one hospital withdrew

from the CSS project following the retirement

of the initial surgical lead. Likewise, one hospital

withdrew from Med Rec SOP implementation

because of competing priorities, lack of leadership,

and lack of resources. Currently, eight hospitals

are participating in the implementation and

evaluation of the two SOPs.

CSS hospitals and health facilities Description

St Joseph St Luc Hospital in Lyon Urban, private, non-profit hospital with 344 beds;

multi specialties surgery, medicine and gynecology/obstetrics

Joseph Ducuing Hospital in Toulouse Urban, private, non-profit hospital with 146 beds;

multi specialties surgery, medicine and gynecology/obstetrics

Clinique du Cambresis in Cambrai Private for profit surgical clinic with 50 beds; surgical specialties:

orthopaedic, vascular, ophthalmology, visceral surgery

Centre Léon Bérard Cancer center Urban private non-profit center with 250 beds;

in Lyon multi-specialties surgery including reconstructive surgery

Centre hospitalier de Cornouaille Urban public hospital with 1 044 beds; multi specialties surgery,

in Quimper medicine and gynecology/obstetrics

Centre hospitalier de Chambéry Urban public hospital with 1643 beds; multi specialties surgery,

in Chambéry medicine and gynecology/obstetrics

Centre hospitalier de Bourg en Bresse Urban, public hospital with 584 beds;

Fleyriat in Bourg en Bresse multi specialties surgery, medicine, gynecology/obstetrics

Pasteur hospital, a subdivision of the Urban, public, academic, affiliated hospital with one surgical

Nice University Hospital in Nice specialty (neurosurgery) and 49 beds

Med Rec hospitals Description

Association clinique la Croix Blanche Private health care facilities specialized in surgery;

Moutier Rozeille acute care: 48 beds

Centre Hospitalier Universitaire Nîmes University hospital. acute care: 722 beds; Long Term Care,

Complex Continuing Care: 295 beds; Psychiatry: 265 beds

Centre Hospitalier Saint Marcellin General hospital; acute care: 30 beds; Long Term Care,

Complex Continuing Care: 38; Home Care/Ambulatory: 30/132

Hôpitaux Universitaire Paris Nord Val University hospital Bichat Claude Bernard; acute care: 1 718 beds.

de Seine Beaujon - Bichat-Claude Long Term Care, Complex Continuing Care: 596 beds.

Bernard - Bretonneau - Louis Mourier Psychiatry: 85 beds.

Centre Hospitalier Universitaire University hospital; acute care: 1 388 beds; Long Term Care,

Grenoble Complex Continuing Care: 419; Psychiatry: 54

Centre Hospitalier Compiègne General hospital acute care; acute care: 418 beds; Long Term

Care, Complex Continuing Care: 142 beds; Home Care /

Ambulatory: 37 beds

Hôpitaux Universitaires de Strasbourg University hospital; acute care: 1 854 beds; Long Term Care,

Complex Continuing Care: 287 beds; Psychiatry: 92 beds

Centre Hospitalier de Lunéville General hospital; acute care: 162 beds; Long Term Care,

Complex Continuing Care: 30 beds; Home Care/Ambulatory:

30 beds /132 beds

Germany

Germany currently has 16 hospitals implementing CSS and seven hospitals implemnting the Med Rec SOP.

CSS hospitals Description

Allgemeines Krankenhaus An urban, non-profit, general hospital with 698 beds and nine

Celle Operating Rooms (ORs) in Celle, a town in the northern part of Germany

Altmark Klinikum Gardelegen A small rural, non-profit, general hospital with with 200 beds and three

ORs in Gardelegen, a small town in the north eastern part of Germany

Altmark Klinikum Salzwedel A small rural, non-profit, general hospital with 218 beds, and three ORs.

It is closely linked with the other Altmark Klinikum mentioned above

Evangelische Elisabeth Klinik A small urban, non-profit, general hospital with 160 beds in the center

Berlin of Berlin

Evangelisches Krankenhaus A small urban, non-profit, general hospital with 210 beds and two ORs

Hubertus on the outskirts of Berlin

Evangelische Lungenklinik A small urban, non-profit hospital with 164 beds, two ORs, with

Berlin a specialized thoracic centre on the outskirts of Berlin

Evangelisches Krankenhaus Paul An urban, non-profit, general hospital with 412 beds and eight ORs in

Gerhardt Stift Lutherstadt Wittenberg, a town in the eastern part of Saxony-Anhalt

Evangelisches Waldkrankenhaus An urban, non-profit, general hospital with 474 beds and eight ORs

Spandau on the outskirts of Berlin

Herzogin Elisabeth Hospital A small urban non-profit hospital with 215 beds, six ORs and

an orthopaedic focus in Braunschweig, a town in Lower Saxony

Klinikum Chemnitz An urban, non-profit, general hospital with more 1720 beds and 23 ORs

in Chemnitz, a town in Saxony

Klinikum Coburg An urban, non-profit, general hospital with 522 beds and six ORs

in Coburg, a town in Bavaria

GRN-Klinik Sinsheim A rural, non-profit, general hospital with 225 beds and four ORs

in Sinsheim, a small town in Baden-Württemberg

Martin-Luther-Krankenhaus An urban, non-profit, general hospital with 285 beds and six ORs

in Berlin

Städtisches Klinikum Solingen An urban, non-profit, general hospital with 716 beds and eight

ORs in Solingen, a town in North Rhine-Westphalia

University Hospital Aachen A large university medical center with 1 282 beds and 30 ORs

in Aachen, a town at the border to the Netherlands and Belgium

University Hospital Freiburg A large university medical center with 1 484 beds and 27 ORs

in Freiburg, a town at the border to Switzerland and France

Med Rec hospitals Description

Diakoniekrankenhaus An urban, non-profit, general hospital with 444 beds in Hannover,

Friederikenstift a town in Lower Saxony

Klinikum Lüneburg An urban, non-profit, general hospital with 472 beds in Lüneburg,

a town in Lower Saxony

Klinikum Coburg An urban, non-profit, general hospital with 522 beds in Coburg,

a town in Bavaria. (Note: This is also a CSS hospital)

Städtische Kliniken- An urban, non-profit, general hospital with 577 beds in

Mönchengladbach Elisabeth Mönchengladbach, a town in North Rhine-Westphalia

Krankenhaus

University Hospital Aachen A large university medical center with 1 282 beds and 30 ORs

in Aachen, a town at the border to the Netherlands and Belgium.

(Note: This is also a CSS hospital)

University Medical Center A large university medical center with 1 484 beds and 27 ORs in Freiburg,

Freiburg a town at the border to Switzerland and France.

(Note: This is also a CSS hospital)

University Medical Center A large university medical center with 1 250 beds in Hamburg

Hamburg-Eppendorf


The Netherlands

The Netherlands currently has 15 hospitals

implementing the Med Rec SOP. A total

of 16 hospitals, of which five are also


Med Rec hospitals, are implementing the SOP

for Safe Management of Concentrated Injectable

Medicines.

Med Rec hospitals - group 1/2011 Description

Antonius Hospital, Sneek General hospital

Diakonessenhuis, Utrecht, Zeist, Doorn General hospital

Fransiscus Hospital, Roosendaal General hospital

HAGA Teaching Hospital, The Hague Large teaching hospital

Medical Center Alkmaar, Alkmaar Large teaching hospital

Rivas Beatrix Hospital, Gorinchem General hospital

Sint Fransiscus Gasthuis, Rotterdam Large teaching hospital

Tergooi, Hilversum, Blaricum General hospital

Radboud University Nijmegen Medical Centre, Nijmegen Academic hospital

VU University Medical Centre, Amsterdam Academic hospital

Rivierenland Hospital, Tiel General hospital

Med Rec hospitals -group 2/ 2011 Description

Hospital Group Twente (ZGT), Almelo, Hengelo General hospital

Elkerliek Hospital, Helmond Non-teaching hospital

Med Rec hospitals -group 3/ 2012 Description

Gelderse Vallei, Ede Teaching hospital

University Medical Centre, Groningen Academic hospital

Med Rec experts involved in group 1 support Description

Erasmus Medical Centre, hospital pharmacist, Rotterdam Academic hospital

Medical Centre Haaglanden, location Westeinde, Large teaching hospital

ED Nurse Practitioner, The Hague

TweeSteden Hospital, hospital pharmacy technician, Tilburg General hospital

Wilhelmina Hospital, ED doctor, Assen General hospital

CIM hospitals involved in group 1 support Description

Albert Schweitzer Hospital Dordrecht General hospital

Antonius Hospital, Sneek General hospital

Elkerliek Hospital, Helmond Non-teaching hospital

Erasmus Medical Centre, Rotterdam Academic hospital

Flevo Hospital, Almere General hospital

HAGA Teaching Hospital, The Hague Large teaching hospital

Maasstad Hospital, Rotterdam General hospital

Martini Hospital, Groningen Large teaching hospital

Meander Medical Centre, Amersfoort Large teaching hospital

Medical Center Alkmaar, Alkmaar Large teaching hospital

Tergooi, Hilversum, Blaricum General hospital

Rijnland Ziekenhuis Leiderdorp Large teaching hospital

Reinier de Graaf Hospital, Delft Large teaching hospital

St. Jansdal Hospital, Harderwijk General hospital

Van Weel Bethesda Hospital, Dirksland General hospital

Westfries Gasthuis, Hoorn General hospital


Singapore

The following list provides a snapshot of the hospital sizes in Singapore. The smallest is Jurong Health

Services with 400 beds; this hospital will be relocating in a few years’ time to a bigger establishment that will

house 700 beds. The largest hospital is Singapore General Hospital at close to 1 500 beds.

CSS hospitals Description

Changi General Hospital 753 beds www.cgh.com.sg

KK Women’s and Children’s Hospital 739 beds www.kkh.com.sg

National University Hospital 1 007 beds www.nuh.com.sg

Singapore General Hospital 1 449 beds www.sgh.com.sg

Tan Tock Seng Hospital 1 293 beds www.ttsh.com.sg

Khoo Teck Puat Hospital 550 beds www.ktph.com.sg

Jurong Health Services -operating at former 400 beds

Alexandra Hospital site

Trinidad & Tobago

The five public hospitals provide 24-hour general, specialized surgical and medical services to the population.

The patient populations vary among the RHAs. Currently, only four of the five hospitals have piloted the CSS

SOP in their respective surgical units.

CSS hospitals / facilities Description

San Fernando General Hospital An urban hospital of more than 300 beds, with12 in-patient operating

rooms, and 12 344 surgical cases performed in the latest calendar year

Port of Spain General Hospital A regional governmental hospital of more than 300 beds, with six

in-patients operating rooms, and 5 561 surgical cases performed

in the latest calendar year

Sangre Grande Hospital A suburban hospital of 100-299 beds, with three in-patient operating

rooms, and 2 694 surgical cases performed in the latest calendar year

Scarborough General Hospital A regional governmental hospital of 100-299 beds, with two in-patient

operating rooms, and 2 342 surgical cases performed in the latest

calendar year

Eric Williams Medical Sciences An urban academic hospital of more than 300 beds, with 8 in-patient

Complex operating rooms and 4 908 surgical cases performed in the latest

calendar year

United States of America

The following list provides a snapshot of the hospitals participating in the High 5s Project in the U.S.A.

CSS hospitals / facilities Description

Baylor University Medical Center Large Teaching and Research Hospital

Washington University School of Medicine Large Teaching and Research Hospital

Med Rec hospitals / facilities Description

Wheaton Fransiscan Elmbrook Small hospital; part of the Wheaton Fransciscan System

Wheaton Fransiscan Healthcare Franklin Small hospital; part of the Wheaton Fransciscan System

Hospital

Wheaton Fransciscan Healthcare St. Francis Small hospital; part of the Wheaton Fransciscan System

Hospital

Wheaton Fransiscan St. Joseph Hospital Small hospital; part of the Wheaton Fransciscan System

Wheaton Fransiacan Wisconsin Heart Hospital Small hospital; part of the Wheaton Fransciscan System

5. High 5s Steering Group

The Steering Group is responsible for overseeing

the High 5s Project planning, management,

coordination and technical direction as outlined

below:

1) Strategic planning

ñ Political and technical direction.

ñ Strategic planning.

ñ Global and country-level Project advocacy and

capacity building at ministerial levels.

ñ Coordination with associated national institutions

or agencies (not LTAs).

2) Management & coordination

ñ Global coordination of planning and activities;

provision of logistics support and services to

LTAs; and High 5s Project biannual meetings.

ñ Country-level coordination of planning and

activities.

ñ Global communications.

3) Technical functions

ñ Development of Standard Operating Protocols

(SOPs) and the Impact Evaluation Strategy and

related tools.

ñ Support and coordination of High 5s Project

national planning, implementation, data

collection, analysis, and training activities.

ñ Development of a global learning community

and associated tools and services as deemed

necessary for implementation, data collection

and analysis, evaluation and dissemination.

ñ Compilation and active dissemination of findings,

outcomes, improved practices and lessons

learned from the High 5s Project for worldwide

publication.

6. High 5s Project Committees

There are five committees composed of

representatives from the WHO, the CC, and LTAs.

The committees advise, plan, develop and execute

activities and are overseen by the Steering Group.

The committees are:

1. Collaborative Learning;

2. Communications;

3. Evaluation;

4. Event Analysis; and

5. Publications.

Participation within each committee varies

according to the task and activity to be achieved.

The Collaborative Learning Committee was

established to develop an implementation plan

for a High 5s Project core learning strategy

including consideration of target audiences and

levels and mechanisms of learning. Strategies

were collaboratively designed and scheduled

to prepare LTAs and hospitals for the

implementation and evaluation of the three SOPs.

They included in-person train-the-trainer workshops,

provision of online resources, and online posting

of Questions and Answers.

The Communications Committee was established to

develop and implement a High 5s communications

strategy that included consideration of target

audiences and creation of common messages/

themes for global use and for High 5s LTA members

to use within their own countries.

The committee’s objectives are to communicate

the concepts, strategies and actions of the High 5s

Project in an effective manner with particular

attention to:

ñ presenting the political profile of the High 5s

Project to external audiences;

ñ supporting LTA activities with in-country

communications and advocacy;

ñ meeting communication needs of the High 5s

Steering Committee members; and

ñ engaging new partners to participate as part

of the High 5s Project expansion strategy.

The Evaluation Committee has overseen the

development of a consistent, integrated approach

to the High 5s Project evaluation across the SOPs.

Specific activities have included:

ñ development of materials and tools necessary

to carry out the evaluation strategy;

ñ development of performance measures as well

as a method for evaluating the SOP

implementation experience;

ñ development of a process for maintaining

the confidentiality of data and comply with LTA

privacy requirements; and

ñ development and monitoring of the overall

evaluation plan.

The Event Analysis Committee is a subcommittee

of the Evaluation committee which was established

to develop the process by which each participating

hospital would actively identify and investigate


a pre-determined set of patient safety problems

(events) that are or could be related to the SOP.

These findings help to determine the effectiveness

of the SOP and are intended to guide future

revisions of the SOPs, as appropriate. Key activities

include:

ñ delineation of SOP-specific prompts for the

identification of SOP-related events for analysis;

ñ establishment of criteria for assessing the

thoroughness and credibility of event analysis;

ñ determination of a minimum data set for the

results of event analyses;

ñ develop a process for submission and review

of the event analyses by the LTA;

ñ creation of a process for submitting an event

analysis report to the Collaborating Center.

The Publications Committee was established to:

ñ identify a potential list of peer-reviewed High 5s

publications;

ñ establish procedures for publishing data under

the High 5s Project; and

ñ develop guidelines that govern publications,

copyright issues, and data ownership and sharing.


4The design of the Project,its protocolsand evaluationframework

1. Design of the Project

The High 5s Project has applied established best

practices and interventions to support the

development of standardized processes and tools.

These processes have included input from all levels

of team members from management to individual

health-care workers. This has also permitted

development of a consistent system of monitoring

that makes the production of comparable results

possible locally, nationally and internationally.

2. Standard Operating Protocols (SOPs)

Each SOP summarizes the problem, proposes

a solution, presents the evidence for the solution,

identifies potential barriers to adoption, and

delineates potential unintended consequences

of the solution. It contains a set of instructions

for implementing a defined patient care process

by multiple users in a consistent and measurable

way. The SOPs were designed to be systematically

implemented using quality improvement methods

that include the following components:

1. oversight of the implementation;

2. project workplan;

3. risk assessment of the proposed process;

4. pilot testing;

5. spread methodology;

6. communication plan;

7. evaluation Strategy;

8. maintenance and Improvement strategy.

Institutions which led the development

of the SOP

In this multi-country collaboration, the Canadian

Patient Safety Institute and the Institute for Safe

Medication Practices in Canada led the

development of the Medication Accuracy at

Transitions in Care SOP. The former National

Patient Safety Agency in the United Kingdom led

the development of the Concentrated Injectable

Medicines SOP. The Joint Commission and the

Agency for Healthcare Research and Quality in

the USA led the development of the Correct Site

Surgery SOP. Participants from all of the LTAs,

the Collaborating Centre, and consultants

associated with Joint Commission International

and the Institute for Safe Medication Practices in

Canada provided technical expertise to support

development of the implementation and evaluation

frameworks that are integral to the SOPs.

3. High 5s evaluation framework

The High 5s Project aims to answer two main

questions:

1. Is it feasible to implement standardized

processes in health care within individual hospitals,

among multiple hospitals within individual

Member States, and across country boundaries?

2. What is the impact of standardization on

the safety problems that the project targets?

The High 5s Project combines process

standardization and intensive evaluation of

the feasibility and impact of implementing SOPs

on a global scale.

A standardized approach to evaluation, the ‘Impact

Evaluation Strategy’ was designed to assess the

feasibility and impact of implementing the SOPs.

Both quantitative and qualitative approaches were

used in this process. The Impact Evaluation Strategy

included structured qualitative and quantitative

evaluation of the SOP implementation experience,

on-site application of SOP-specific performance

measures, use of an event analysis framework to

identify occurrences that may represent SOP

failures, and a baseline hospital patient safety

culture survey. The evaluation strategy identifies

the factors underlying patient safety problems,

matches these problems against those that the


The design of the Project, its protocolsand evaluation framework, including anoverview of methodologies, instrumentsand processes

SOPs are trying to prevent, and tracks

improvements in patient safety in the participating

hospitals.

Following LTA and hospital training on SOP

implementation and use of the evaluation

instruments, the materials and tools necessary to

implement each SOP were pre-tested in individual

representative hospitals in participating Member

States for a two-month period. This was done

to permit revisions and adaptations of the SOPs

and their related evaluation processes prior

to full-scale implementation of the SOPs in 2010.

3.1. SOP implementation experience

The goals of the SOP implementation evaluation

process were to determine if one or more SOPs

could be implemented as designed in diverse

hospital settings, to identify barriers to

implementation, and to identify ways to overcome

said barriers. In order to evaluate the fidelity and

portability of SOP implementation in multi-country

settings and environments, self-reported narrative

data was collected at the beginning of the Project,

at six-monthly intervals throughout the project.

Related interview data is also being collected at

selected hospitals annually.

Methodologies, instruments and processes

employed

A narrative of real-world experiences from

individuals engaged in SOP-implementation

provides a deeper understanding of the feasibility

and impact of SOP implementation. On a biannual

(every six months) basis, participating hospitals

from each Member State were asked to complete

a 67-item questionnaire that was divided into

ten sections. These included implementation

status, oversight, work plan, risk assessment,

pilot test, spread methodology, communication

plan, evaluation strategy, events analysis and

maintenance. A summary of this data was compiled,

classified by specific SOP (either Correct Site

Surgery or Medication Reconciliation), and

reported. Hospitals that had already completed

the initial questionnaire were given the option

from the 4th quarter 2012 to complete a brief

implementation update form. This form included

five open-ended questions which were designed

to capture any new information regarding parts

of the implementation experience that had most

likely changed, such as implementation status,

communication strategies, new strategies or

barriers regarding maintenance, efforts to measure

the effectiveness of the SOP, and any experiences

of the hospital that might benefit other

organizations. This brief update form was designed

to reduce the reporting burden on hospitals

and allowed for participants to describe, in only

a few sentences, any significant changes over

a period of six months. Use of this brief update

form was optional; hospitals were given the

option of completing the questionnaire in full.

3.2. SOP-specific performance measures

Implementation of each SOP is evaluated using

performance measures that determine the extent

to which the protocol was followed during

implementation by the participating hospitals

in multiple Member States and enable national

and international comparability of data.

The standard performance measures include SOP

process measures and outcome measures.

Standardization of SOP performance measures

increases data reliability and comparability.

Performance measurement results are presented

in various formats within the High 5s Information

Management System to support LTA and hospital

needs and enable sites to benchmark and trend

their data over time.

35 The design of the Project, its protocols and evaluation framework


Performance measures for ‘Medication Reconciliation’

H5sMR-1 Percent of patients with medications reconciled within 24 hours of the decision to admit

the patient

H5sMR-2 The mean number of outstanding undocumented intentional medication discrepancies

per patient

H5sMR-3 The mean number of outstanding unintentional medication discrepancies per patient

H5sMR-4 Percent of patients with at least one outstanding unintentional discrepancy

Performance measures for ‘Correct Site Surgery’

H5sCS-0 Proportion of verification checklists for all eligible surgical cases

H5sCS-1 Proportion of eligible surgical cases with a complete preoperative verification process

(exclusive of site marking and Time Out)

H5sCS-2 Proportion of cases with properly marked surgical site

H5sCS-3 Proportion of cases with complete final Time Out

H5sCS-4 Proportion of cases with discrepancy noted at final Time Out

H5sCS-5 Proportion of cases undergoing surgery with unresolved Time Out

H5sCS-6 The proportion of surgical cases that are cancelled or postponed due to discrepancies

identified at any point in the conduct of the SOP

H5sCS-7 Proportion of cases with incorrect surgery (wrong site, procedure or person cases).

Performance measures for ‘Concentrated Injectables’

H5sCI-01 Time between concentrated injectable adverse events

H5sCI-02 Time between adverse drug events delay or omission of administration of injectable medicines

H5sCI-03 The number of adverse events for specified concentrated injectables per 1 000 patient days

H5sCI-P1 Concentrated injectable medicines stored in unauthorized clinical areas

H5sCI-P2 Concentrated injectable medicines supplied to unauthorized clinical areas

H5sCI-P3 Ready-to-administer and ready-to-use injectable medicines supplied to unauthorized clinical

area

H5sCI-P4 The number of clinical areas storing concentrated injectable medicines according to selected

SOP specifications

Although performance measures for the implementation of the ‘Concentrated Injectables’ SOP were

developed, implementation data were not collected.

Data quality

Data and information provided and collected

have to be accurate to ensure a credible evaluation.

The aim is to reach a level of data quality

consistent with the limits of precision that are

achievable with respect to the analytic tools

and sample sizes defined for use in the High 5s

Project. Only data that has been confirmed to

meet the data quality criteria by both the hospital

and the LTA is included in performance measure

calculations.

Methodologies, instruments and processes

employed

The performance measure data is normally

collected at monthly intervals in the secure

High 5s IMS and international evaluation reports

are generated at quarterly intervals. A list of the

performance measures for each SOP is provided

below.

National and international comparison

data calculations

a) Country-level data (national comparison)

Country-level data points are calculated using

a similar approach to the one employed for

calculating an individual hospital’s performance

on a specific measure. The major difference is

that for rate and ratio measures, all the hospital

numerator cases (from within a specific Member

State) are summed, and all of the hospital

denominator cases (from within a specific Member

State) are summed before calculating the measure

rate or ratio. The measure is calculated in the

aggregate for all hospitals in a Member State

during the specific time period. This calculation

creates a weighted mean (weighted by the number

of cases contributed by each hospital) rather than

a grand mean (simply taking the average of the

calculated hospital rates). The approach is slightly

different for continuous variable measures (used

for the Concentrated Injectables SOP). For these

measures, the comparison group value is:

b) International-level data (international

comparison groups)

International-level data points are calculated

using a similar approach to the one employed

for calculating an individual hospital’s performance

on a specific measure. The major difference is

that for rate and ratio measures, all the hospital

numerator cases (from all Member States) are

summed and all of the hospital denominator cases

(from all Member States) are summed before

calculating the measure rate or ratio. The measure

is then calculated in the aggregate for all hospitals

contributing data during the specific time period.

This calculation creates a weighted mean (weighted

by the number of cases contributed by each

hospital) rather than a grand mean (simply taking

the average of the calculated hospital rates or

the average of the national rates). The approach

is slightly different for continuous variable

measures (used for the Concentrated Injectables

SOP). For these measures, the comparison group

value is:

c) Requests for measure revisions

Over the course of the Project, a number of

requests to revise the measures were submitted.

The Evaluation Committee considered each request

and made recommendations to accept or reject

the modification based upon the nature of the

request and the degree to which it would impact

longitudinal analyses. To date, there have been

nine requests for measure revision: four were for

measures related to Medication Reconciliation,

three were for Correct Site Surgery, and two were

for Concentrated Injectables.

In order to reduce data collection burden, Australia

requested to reduce the frequency of data

collection for measures MR 2, 3 and 4 for hospitals

that had met an established performance threshold.

The request was approved and hospitals that

had consistently met a defined threshold were

permitted to submit data quarterly. If a hospital

later failed to meet the threshold, it would then

return to monthly submissions until it could meet

the expected benchmark for three consecutive

months.

Canada requested a 24-hour time boundary for

measures MR 2, 3 and 4. This request was approved

because it allows for standardization and ease

of data collection across all participating Member

States.

Australia also requested changes to measures

MR 2, 3 and 4. Specifically, they requested a tighter

definition of what are to be included as documented

intentional discrepancies and unintentional

discrepancies. They further asked for specific

examples of inclusions/exclusions. Finally, they

requested additional language that would state

that “medications prescribed for the first time

on admission are intentional discrepancies and

shouldn’t be included.” These requested changes

were approved.

A request was made by the United Kingdom,

and approved, to revise the measure statements

for measures CI-02 and CI-03 to include the

intended patient populations. In order to increase

understanding of the SOP and to be consistent

with measure CI-P1, that measure statement

was changed to indicate that measure events

are restricted to only those that caused harm

to patients.

The United Kingdom also made a request to

change the value statement for measure P1


Σ (each hospital value* the number of cases upon

which that value was based)

Σ of hospital cases

Σ (each hospital value* the number of cases upon

which that value was based)

Σ of hospital cases

to read “It is important to monitor whether

concentrated products are held in stock in

unauthorized clinical areas.” Because measure

P1 is not intended to measure the number of

concentrated products held in stock, but instead

measures the number of unauthorized clinical

areas storing concentrated products, this change

was approved and the language was amended

in the performance measure value statement.

The U.S.A. requested that a new measure, CS-0,

be created. This measure would capture the percent

of eligible cases having a pre-operative verification

checklist (total number of checklists/total number

of eligible cases). This measure was created based

on the assumption that it will help ensure that the

correct data are used in the performance measure

calculations.

Because of ambiguity in the labeling of the

numerator data element for CS-1, Germany

requested that the name of the numerator

data element be changed from “Completed

Pre-Operative Verification Checklist” to “Number

of Eligible Surgical Cases with a Completed

Pre-Operative Verification Process”. Because

this change clarified the intent of the measure

to ensure standardized data collection, this change

to the data element was approved.

3.3. Event analysis

An ‘event’ is defined as an occurrence that happens

to or involves a patient and which could have

resulted, or did result, in unnecessary harm to the

patient.

The evaluation framework includes an Event

Analysis (EA) component that directs each

participating hospital to actively seek and

investigate a pre-determined set of patient safety

problems (events) that should have been prevented

in the clinical setting by the SOP. Hospitals

are then asked to report specific de-identified

information regarding the case to their LTA for

review and submission to the High 5s Collaborating

Centre.

Event analysis is undertaken after a patient safety

incident takes place and aims to determine

whether there is a link to the SOP implementation

and, if so, whether factors in the design and/or

implementation of the SOP contributed to

the occurrence of that incident. The event analysis

process yields important information as to

the functioning of the SOP, as well as how

to improve the SOP and its implementation.

In order to effectively identify events for analysis

beyond those that are independently reported,

participating hospitals have been expected to use

the specifically designed methods, including chart

audits and checklist reviews (where applicable).

Minimum Data Set (MDS) forms and checklists

were also developed to ensure thorough and

credible event analysis and methodical de-identified

reporting of specific SOP-related events. Where

appropriate, aggregate or cluster event analysis

processes were applied where certain events

appeared to have common characteristics, in order

to identify patterns of performance.

There are four types of events:

1. Hazard: a circumstance, agent or action

with the potential to cause harm;

2. Near miss/Close Call/Good Catch: an event

which did not reach the patient;

3. No-harm: an event which reached a patient

but no discernible harm resulted;

4. Adverse: an event which resulted in harm

to a patient.

Event analysis is defined as a systematic process

whereby the facts, contributing factors and

resultant recommendations are identified and

reported as a means of investigating an event.

This aspect of the evaluation framework was

intended to go beyond fact-finding, and to help

understand the applicable contributing factors.

The EA process provided an unprecedented

opportunity to obtain and analyze findings

regarding the safety and effectiveness of the SOPs.

The complexity of developing and implementing

the SOPs delayed the High 5s Steering Group

in finalizing and implementing the EA evaluation

process. The Event Analysis Subcommittee

played a key role in formalizing the proposed EA

instructions for the participating hospitals.

After the approval by the Steering Group,

the instructions were posted on the High 5s

Information Management System (see Annex 4A

and 4B for the EA Overview for the Medication

Reconciliation and Correct Site Surgery SOPs).

Each LTA provided specific instructions to

participating hospitals regarding the methodology

to be used by their participating hospitals to

perform the event analysis. Where an LTA did not


have a preferred methodology, they could adopt

one of the established event methodologies

identified by the Event Analysis Subcommittee

and approved by the Evaluation Committee.

The sources of these methodologies were:

1. National Patient Safety Agency

http://www.nrls.npsa.nhs.uk/resources/?entryid45=5

9901

2. United States Department of Veterans Affairs

National Center for Patient Safety

hhttp://www.patientsafety.gov/CogAids/RCA/

3. Canadian Patient Safety Institute

http://www.patientsafetyinstitute.ca/English/toolsR

esources/IncidentAnalysis/Pages/default.aspx

4. The Joint Commission

http://www.jointcommission.org/Framework_for_Co

nducting_a_Root_Cause_Analysis_and_Action_Plan

The Collaborating Centre offered specific

training in any of the above methodologies

requested by an LTA. Most, but not all,

participating LTAs had in-country EA expertise

and a standardized tool for analyzing events after

they were identified. Some LTAs needed to first

invest time and resources in building this capability

before attempting to implement the High 5s EA

evaluation process.

The EA principles were specified to facilitate a

consistent, timely, thorough and credible approach.

The goals of EA were to determine:

1. what happened?

2. why did it happen?

3. how can the likelihood of recurrence of this

event be reduced?

Four types of ‘Event analysis’ were defined:

1. Concise

Event analysis of these occurrences is usually

conducted by someone who is aware of the details

of the event and is knowledgeable about

the involved process and effective solutions.

The EA process is often completed within hours

or days because of the less intensive approach.

Events without patient harm may receive Concise

Event Analysis or Cluster Analysis.

2. Comprehensive

Event analysis is undertaken by an inter-disciplinary

group of staff and physicians that is facilitated

by a person(s) who is knowledgeable about the

involved process(es), human factors and effective

solutions development. The process may take up

to 90 days because of the depth and breadth of

the analysis. Events resulting in patient harm

should receive ‘Comprehensive Event Analysis’.

3. Aggregate

This is the optional but recommended process

of analyzing data combined from the findings

of several completed event analyses (concise

or comprehensive) of similar event characteristics,

in order to identify patterns of causation

and enhance the effectiveness of actions for

improvement.

4. Cluster analysis

This is a process for analyzing three or more no-

harm events having similar event characteristics as

a group in order to identify patterns in causation

and enhance the effectiveness of actions for

improvement. Cluster analysis may be used instead

of concise analysis if there are three or more

no-harm SOP-related events of the same type

within a month.

Identification of events for analysis

Initial development of the EA evaluation involved

creation of a trigger tool for chart reviews in order

to identify wrong site surgeries or medication

reconciliation failures. The Australian LTA provided

key expertise and leadership in pilot testing this

strategy. Eventually an agreed upon strategy was

formalized and implemented for the identification

of events.

A. Medication Reconciliation events

Med Rec events are process issues or outcomes

that are possible SOP-related events. Examples

include:

ñ ‘Best Possible Medication History’ (BPMH) not

obtained and/or reconciled within 24 hours

of decision to admit;

ñ inaccurate or incomplete BPMH;

ñ inaccurate or incomplete resolution of any

discrepancies;

ñ all discrepancies not resolved within 48 hours

of decision to admit.

Two primary sources of SOP events for analysis are:

1. independently reported adverse events during

or after the SOP is implemented;

2. independent ‘Observer Chart Audits’.


Chart audits are a defined independent double-

check strategy to assess the quality of the

Medication Reconciliation “process” that is

performed in hospitals as part of the evaluation

strategy to collect performance measures. During

audits by the independent observer to measure MR-

2 – MR-4, medication reconciliation process

events/issues identified are flagged for EA.

B. Correct Site Surgery events

CSS SOP events are process issues or

outcomes that are possible SOP-related events.

Examples include:

ñ wrong site surgery;

ñ wrong limb prepped for surgery but error

discovered before incision occurred.

Two primary sources of SOP events for analysis are:

1. independently reported adverse events during

or after the SOP is implemented;

2. retrospective SOP checklist review.

The SOP checklist is a defined requirement to

document steps in preparing each patient for

surgery and for recording the outcomes relevant

to the SOP. There are four outcomes requiring

analysis that may be identified by a member

of the health-care team concurrently and/or, a

reviewer of the checklists on a retrospective basis:

ñ incorrect surgery (wrong patient, procedure,

site or implant)

– Hospital completes a Comprehensive Event

Analysis;

ñ case that proceeds to incision with unresolved

discrepancy

– Hospital completes a minimum of a Concise

Event Analysis;

ñ case cancelled because of SOP-related

discrepancy

– Hospital completes a minimum of Concise

Event Analysis;

ñ case with discrepancy resolved at final Time Out

– Hospital completes a minimum of Concise

Event Analysis.

Hospitals can decide to complete a Comprehensive

Event Analysis of certain events at their discretion.

For these cases, they are also asked to consider

aggregate analysis of any group of related

individual event analyses. For no-harm cases of

the same type in these categories, which occur

at a frequency of three or more per month, cluster

analysis can be conducted.

Data collection and reporting

The ‘Event Analysis- Minimum Data Set’

(MDS) forms were developed to capture the key

findings of the analysis. Hospitals were asked

to complete all fields of the applicable forms,

and attach any additional information at their

discretion, before submission to their LTA.

The LTA would subsequently review the submission

and follow-up with the applicable hospital

to ensure that all information was properly

de-identified, understandable and complete.

Criteria for accuracy and completeness are defined

in Annex 5A and 5B. The LTA would then submit

this data to the Collaborating Centre, where

the information would be combined for analysis

and de-identified High 5s reporting. The

information would also be combined with

other evaluation findings to determine if any

changes should be made to the SOP.

3.4. Patient safety culture surveys

The ‘Hospital Survey on Patient Safety Culture’

was developed by Westat under contract with

the Agency for Healthcare Research and Quality

(AHRQ). The survey and accompanying toolkit

materials are available from the AHRQ website

www.ahrq.gov/qual/hospculture. In the context

of the High 5s Project the culture survey was

conducted as an additional voluntary evaluation

component to provide insight into the patient

safety culture of hospitals participating in the High

5s Project and to sensitize frontline staff and

hospital management about staff members’

perception of the hospital’s patient safety culture.

At the time the survey was administered,

participating Member States included Australia,

France, Germany, the Netherlands, Singapore

and the United Kingdom. The Project then included

the Medication Reconciliation, Correct Site Surgery

and Concentrated Injectables SOPs.

The survey, designed to assess hospital staff

opinions about patient safety issues, medical error,

and event reporting, included 42 items that

measure 12 areas or composites of patient safety

culture. Each of the 12 patient safety culture

composites is listed and defined in Table 3.

The survey also included two questions that

asked respondents to provide an overall grade

for patient safety for their work area/unit and

to indicate the number of events they had reported

over the past 12 months. In addition, respondents


Table 3: Patient safety culture composites and definitions

Patient safety culture Definition: The extent to which…composite

1. Communication openness Staff freely speak up if they see something that may negatively

affect a patient, and feel free to question those with more authority

2. Feedback and communication Staff are informed about errors that happen, given feedback about

about error changes implemented, and discuss ways to prevent errors

3. Frequency of events reported Mistakes of the following types are reported: (1) mistakes caught

and corrected before affecting the patient, (2) mistakes with no

potential to harm the patient, and (3) mistakes that could harm

the patient, but do not

4. Hand-offs and transitions Important patient care information is transferred across hospital units

and during shift changes

5. Management support for Hospital management provides a work climate that promotes patient

patient safety safety and shows that patient safety is a top priority

6. Non-punitive response Staff feel that their mistakes and event reports are not held against

to error them, and that mistakes are not kept in their personnel file

7. Organizational learning– Mistakes have led to positive changes and changes are evaluated

continuous improvement for effectiveness

8. Overall perceptions Procedures and systems are good at preventing errors and there

of patient safety is a lack of patient safety problems

9. Staffing There are enough staff to handle the workload and work hours are

appropriate to provide the best care for patients

10. Supervisor/manager Supervisors/managers consider staff suggestions for improving

expectations and actions patient safety, praise staff for following patient safety procedures,

promoting safety and do not overlook patient safety problems

11. Teamwork across units Hospital units cooperate and coordinate with one another to provide

the best care for patients

12. Teamwork within units Staff support each other, treat each other with respect, and work

together as a team


Participating hospitals expressed interest in

comparing their results with those hospitals

in other participating Member States, as it provided

a new source of information to help them identify

and understand potential differences in patient

safety cultures across Member States. For this

reason AHRQ produced the High 5s ‘International

Comparative Report’ (see Annex 2).

The report presented survey administration and

respondent statistics across the Member States,

as well as percent-positive results on the patient

safety culture composites and survey items.

Comparative results from U.S.A. hospitals included

in the AHRQ Hospital Survey on Patient Safety

Culture 2011 Comparative Database Report were

also provided.

were asked to provide limited background

demographic information about themselves

(e.g., their work area/unit, staff position, whether

they had direct interaction with patients).

France, the Netherlands, and Germany translated

the survey for administration in their countries.

The United Kingdom, Singapore, and Australia

administered the survey in English, using an edited

version of the U.S. English version.

5Qualitative andquantitativeinterim findings

1. Summary of culture survey findings

Data from 6 Member States with a total of 59

hospitals was collected and analyzed for reporting

purposes (Table 4). The High 5s participating

hospitals administered the hospital survey to their

staff between November 2009 and November

2010 and voluntarily submitted their data for

inclusion in the database. The U.S.A. database

numbers are based on the AHRQ Hospital Survey

on Patient Safety Culture 2011 Comparative

Database Report.


Qualitative and quantitative interim findings

Table 4: Survey administration statistics

Member State Survey administration statistics High 5s SOP

No. of Total No. Total No. Overall

hospitals of completed of staff asked response

surveys to complete survey rate

United Kingdom 12 2 657 Not Available Not Available Concentrated Injectables

Singapore 7 18 204 23 792 77% Correct Site Surgery

France 7 621 1 000 62% Correct Site Surgery

Germany 5 1 083 3 881 28% Correct Site Surgery

Australia 24 3 485 8 410 41% Medication Reconciliation

The Netherlands 4 127 253 50% Medication Reconciliation

High 5s Total 59 23 520* 37 336* 63%*

* High 5s ‘Totals’ in these columns do not include United Kingdom due to missing data.

On average, hospitals across Member States scored

highest on teamwork within units (70% positive),

and scored lowest on handoffs and transitions

(34% positive). The hospitals in Singapore were

the most positive on seven of the 12 composites,

and were also the least positive on two of the

composites. The hospitals in the Netherlands were

the most positive on five of the 12 composites,

and were also the least positive on two of

the composites. The hospitals in Germany were

the least positive on four of the 12 composites.

The hospitals in Singapore had the smallest

variability on all 12 composites across participating

hospitals. Singaporean hospitals also had the

highest average composite score (58%) while

German hospitals had the lowest (46%) average

composite score. The U.S.A database hospitals

were more positive on four of the 12 composites

and more positive or equal on all 12 composites

in comparison to the High 5s average scores.

The hospitals in the United Kingdom had

the highest percentage of respondents giving

the work area/unit a patient safety grade of

an ‘A-Excellent’ or ‘B-Very Good’ (60%), while

hospitals in the Netherlands had the lowest

percentage (21%). Hospitals in the Netherlands

had the highest average percentage of respondents

reporting one or more events (78%), while

hospitals in Singapore had the lowest (37%).

Culture survey results by staff position were as

follows:

ñ the High 5s average composite score for all

physicians/consultants/medical officers was 51%,

with a low of 47% in Germany to a high of 54%

in Singapore;

ñ the High 5s average composite score for nursing

was 53%, ranging from 45% for the Netherlands

to 60% for Singapore;

ñ the High 5s average composite score for allied

health professionals was 55%, ranging from 49%

for Germany to 58% for France;

ñ the High 5s average composite score for

administration/ management was 57%, ranging

from 49% for France to 69% for Netherlands.

Survey and data limitations

The survey results represented a large compilation

of international patient safety culture survey data

and therefore provided a useful reference for

international comparisons. However, the following

limitations apply to these data and should be kept

in mind when considering the data:

1. The High 5s hospitals included for the report

were not a statistically selected sample of all

hospitals within each participating Member State

and therefore were not necessarily representative

of the status of the patient safety culture within

each Member State.

2. The number of hospitals and the number

of respondents varied greatly by Member State.

For instance, Singapore submitted data from

seven hospitals and 18 204 respondents, while

Australia submitted data from 24 hospitals

and 3 485 respondents. Given this wide range

of participation, the ability to generalize

the comparability of the results is limited.

3. Hospitals administered the survey in different

ways. Most hospitals used paper only, while

others used web only, and still others used

a combination of these two methods of data

collection. It is possible that these different

methods led to mode effects in the survey

scores. In addition, some hospitals surveyed all

hospital staff, while others administered the

survey to a sample of staff in selected work

units/areas. In cases in which a sample was used,

no information was available to determine

the method used to select the sample.

4. Differences in scores across Member States may

relate to a number of factors other than true

differences in patient safety culture, making

direct comparisons among Member States

complicated. For instance, there may be

significant cultural differences across Member

States in how respondents interpreted the survey

items and the ways in which they used the

response scales. In addition, there are differences

in the way health care is structured and provided

across Member States that could affect

responses. Furthermore, survey translations

were developed by individuals within the

Member States, but were not tested or externally

validated before administration.

5. The data was cleaned for out-of-range values

(e.g., invalid response values because of data

entry errors) and blank records (where responses

to all survey items were missing). Otherwise,

data was presented as submitted. No additional

attempts were made to verify or audit the

accuracy of the data submitted.

1.1. Culture surveys in participating Member

States

Australia

All participating hospitals were requested to

complete the AHRQ Patient Safety Culture Survey.

Seven hospitals chose to administer the survey

throughout the hospital, while the remainder

administered it in their emergency departments,

wards and services participating in the Project.

All hospitals administered the survey as hard copies

with one exception where a mix of paper and

online surveys was used. Twenty four hospitals

participated in the survey.

The survey was conducted between May 2010

and December 2010.

Australian hospitals reported high positive

responses to three patient safety culture

composites: teamwork within units (77%);

management support for patient safety (71%); and

organizational learning – continuous improvement

(69%). Conversely, only 34% reported a positive

response on handoffs and transitions; 39% of staff

surveyed believed that staffing levels were

sufficient for patient care; and only half (49%)

reported a positive response to teamwork across

units (Table 1 in Annex 6).

45 Qualitative and quantitative interim findings

Analysis of the data for the hand-offs and

transitions composite found a variation in patient

safety culture across work areas and among

the different categories of staff. Clinical handover,

staffing, teamwork across units, and engaging with

junior medical staff were identified as potential

areas for improvement. Hospitals were encouraged

to review their individual survey results and to:

ñ identify:

– lowest scoring safety culture composites

across items, unit areas, and staff areas and

look for linkages;

– areas for improvement;

– places where additional qualitative data

needed to be collected; and

– stakeholders who should be engaged in

improvement activities

ñ discuss possible improvement strategies; and

ñ develop and implement an action plan for

improvement and evaluate the plan impact.

France

The French LTA decided that only the ‘Correct Site

Surgery (CSS)’ participating hospitals would

complete the AHRQ culture survey, as the workload

of the ‘Medication Reconciliation’ participating

hospitals for the implementation and evaluation

of this SOP was considered too great to permit

their participation in the culture survey.

All of the CSS participating hospitals (nine in 2011)

were asked to complete the AHRQ culture survey,

but two of them did not participate: one because

of organizational issues (delay in the implementation

of the project) and the other because management

did not approve the administration of the survey.

In the seven participating hospitals, 1 000 surveys

were sent out and 620 completed responses were

received (42% to 95% in the various hospitals).

Questionnaires translated in French were distributed

either hospital-wide (three hospitals) or in the units

directly involved in SOP implementation in

the four other hospitals. The LTA trained all

the project coordinators and supplied to each

hospital a tool-kit with communication models.

The challenge encountered by the project hospital

coordinators was the lack of time to explain the

safety culture survey and to collect the surveys.

The areas with the three top scores were teamwork

within units (59%), organizational learning –

continuous improvement (58%) and openness of

communication (56%). Areas with the three lowest

scores were:

ñ handoffs and transitions (39%);

ñ non-punitive response to error (32%); and

ñ management support for patient safety (28%).

The overall level of patient safety was reported

to be between acceptable to excellent by 94% of

respondents. Feedback of the results was provided

to the front line staff and managers by the LTA via

conference call and site visit. Participating in the

High 5s Project was considered to reflect positively

on the safety culture.

Germany

Participation in the AHRQ culture survey was

voluntary, but was encouraged by the LTA. Only

the hospitals implementing the Correct Site Surgery

SOP were invited to take part in the survey, because

at the time the culture survey was administered

internationally, the ‘Medication Reconciliation’

hospitals had not yet been recruited in Germany.

Some of the participating hospitals were keen

to administer the survey a second time in order

to compare the results.

All hospitals had concerns regarding data

protection and had to obtain consent from

the works council and management approval.

Therefore the last section of the questionnaire

(Section H: Background Information) had to be

modified in a way that made identification of

an responder impossible.

Five of the 16 CSS hospitals participated in the

first survey and a total of 1 083 questionnaires

were completed (response rate of approximately

30%). In the second survey three of these five

hospitals participated and 252 questionnaires

were completed.

For the summary of culture survey findings from

Germany, see Annex 14.

The Netherlands

Dutch hospital participation in the culture survey

was not mandatory. Four hospitals completed

the culture survey in 2010 and after SOP

implementation in 2011. They adapted the culture

survey as an implementation intervention to create

awareness among healthcare professionals.


Table 5: Returned questionnaires and response rates

Before SOP implementation After SOP implementation

Distributed & returned Response rate Distributed & returned Response ratequestionnaires questionnaires

Hospital A 12/37 32% 22/50 44%

Hospital B 12/40 30% 20/42 48%

Hospital C 23/26 88% 13/25 52%

Hospital D 80/150 53% 91/160 57%

Total 207/403 51% 146/277 53%

In addition, Singapore tied with United Kingdom

for the lowest score of 36% for staffing.

Nonetheless, Singapore hospitals had the highest

average composite score of 58% with the lowest

score being 46% by German hospitals. In terms

of the average percentage respondents reporting

one or more events, Singapore had the lowest

reporting culture at 37%, while the Netherlands

had the highest at 78%.

It would seem that the drivers for Singapore’s

patient safety culture lie in its strong organizational

learning-continuous improvement culture with

a score of 81%, management support for patient

safety with a score of 75%, and teamwork across

units with a score of 62%.

The Singaporean LTA is currently working

to encourage a culture of reporting mishaps

by issuing patient safety alerts with each report

so that institutions see the value in reporting

as the reports are used to help other institutions

avoid the same errors. Every six months, a sharing

session is organized so institutions can learn from

each error report and treat reporting as a learning

opportunity rather than a punitive event.

Trinidad & Tobago

The aim of the AHRQ culture survey was to gain

an understanding of hospital priorities and

strategies regarding surgical safety. All five

hospitals were required to complete the survey,

and all five submitted completed survey forms

to the Directorate, Health Services Quality

Management, of the Ministry of Health. Only

one survey was conducted in May 2012 at each

of the five participating hospitals.


No particular challenges were encountered in

evaluating the patient safety culture in participating

hospitals. After SOP implementation, hospitals

scored higher on most Patient Safety Culture

Composites. On the three composites of

communication openness, feedback and

communication about error, and frequency

of events reported, hospitals had low scores.

Before and after SOP implementation, management

support for patient safety was unchanged.

To define drivers for patient safety, positively

worded items, which about 75% of respondents

endorsed by answering “Agree/Strongly agree,”

or “Most of the time/Always”, were identified

as strengths. The Netherlands did not utilize

other mechanisms to evaluate the patient safety

culture in the 15 participating hospitals, but there

are many patient safety activities and projects

in Dutch hospitals.

Singapore

The Singaporean LTA facilitated the AHRQ

culture survey by asking all participating hospitals

to complete the questionnaire in March 2010.

Overall, there were not many challenges

encountered in requiring hospitals to complete

the survey. Singapore obtained a total of 18,204

respondents, a 77% overall response rate.

The overall survey results for Singapore hospitals

showed that they were the most positive on seven

of the 12 composites, though they were the least

positive on two of the composites, namely:

1. non-punitive response to error (27% as

compared to the highest score of 62% in the

Netherlands); and

2. communication openness (44% as compared

to the highest score of 77% in the Netherlands).

Table 5 shows the distributed and returned questionnaires and response rates.

2. Context survey of MedicationReconciliation

The context for implementation of the Med Rec

SOP in the participating Member States is very

different and its implementation varies greatly in

terms of numbers of hospitals, teams and support.

The context survey focused on identifying barriers

encountered by the participating Member States,

as well as the role of external drivers and supports

such as accreditation requirements and national

policies. Despite many differences in the context

of implementation, most Member States, reported

similar barriers and facilitators when implementing

the SOP.

The most significant barriers to Med Rec

implementation reported by all Member States

were a lack of staff and financial resources.

The most significant facilitators included

the presence of an accreditation system,

pharmaceutical reforms and state policies,

standards of practice for Med Rec, and internal

leadership/internal ambassadors for Med Rec

within a hospital. A variety of health-care

professionals have been involved in creating

the ‘Best Possible Medication History’ (BPMH) in

the different Member States.

Although the High 5s SOPs were expected to be

implemented in acute care settings, there were

efforts to extend Med Rec SOP implementation

across the system in several Member States.

Canada’s experience in this area suggests that there


Box 2: The context survey

At the September 2011 High 5s Steering Group

meeting, it was agreed that it would be useful

to examine the context of the implementation

of the Med Rec SOP in those Member States

implementing the SOP – Australia, France,

Germany and the Netherlands.

The objective of the context survey was to

identify:

ñ the current landscape of Med Rec in the

individual Member States;

ñ barriers and drivers affecting implementation

in the participating Member States;

ñ successful tools and resources which could be

shared globally.

Information was collected using an online survey.

is a real need for improved communication

and medication reconciliation from each health-

care setting through to home care. There are

many differences of health-care systems across

the participating Member States. For example,

in Germany, there are few, if any, pharmacists

in hospitals, which makes implementation of this

SOP challenging, while in the Netherlands a link

with community pharmacy dispensing databases

supports effective communication about

medications. In light of such differences it

is particularly commendable that four Member

States have chosen to implement this SOP.

Limitations

Participating Member States and hospitals across

and within Member States are in different stages

of implementation of the Med Rec SOP and operate

within very different health-care systems. Within

the context survey, it was difficult to differentiate

Med Rec activities implemented in health-care

settings outside the High 5s Project and those

within the High 5s Project. The survey results reflect

a point in time and not a continuous timeline.

Some general considerations

The primary objective of the context survey was

to provide information on the context in which

the High 5s Med Rec SOP was implemented to

understand what external factors were influencing

the uptake of Med Rec in the different Member States.

This information was collected in addition to

the quantitative performance data and hospital

implementation experience data collection.

The survey did not attempt to correlate hospital

results and accreditation performance, but rather

to determine whether sharing standards, resources,

and experiences learned from the context survey

could assist LTA’s to implement the SOP more

effectively.

Further, best practice tools and resources

obtained from the implementing teams of some

LTAs are available to be shared among the other

participating LTAs and hospitals. For example,

Canadian tools which are part of the High 5s

toolkit are uploaded onto the High 5s Wiki portal

as they are updated by ISMP Canada. Member

States with relevant Med Rec policies or

accreditation standards were able to share policy-

related information for the benefit of other

Member States.

reconciliation. The certification/accreditation

guidelines encourage hospitals to implement

several practices that should eventually evolve

towards a more generalized implementation

of medication reconciliation. For example,

the management system for pharmaceutical care

should take into account the need to ensure

continuity of medication reconciliation from

admission to discharge, including internal transfers.

Patients should be encouraged to pass on

information regarding their medications to

health-care professionals on admission and every

time they receive care. The latter depends on how

proactive a patient is.

The national guidelines for drug management

are consistent with the certification/accreditation

procedure of the National Authority for Health

The safety of drug therapy, in particular, regarding

the prevention of medication errors has become

a major focus of national policy. A February 2012

circular recommends understanding the therapeutic

management of the patient as a process and calls

on focusing on high risk patients and high risk

medication, as well as addressing the prevention

of ‘never-events’ related to drugs and organizations.

Germany

Medication reconciliation is a relatively new

activity in Germany which, prior to involvement

in the High 5s Project, has not been systematically

implemented and evaluated in German health-care

facilities. Medication safety is a key priority for

the Federal Ministry of Health, as evidenced by

its issuance of national “action plans for the

improvement of medication safety in Germany”

for the years 2008-2009 and 2010-2012. These

action plans list specific medication safety

measures, including Med Rec elements, to be

developed and tested in these time frames.

In addition, the German Society of Hospital

Pharmacists has published a statement in which

Med Rec is described as a practice of pharmacists

to support physicians and nurses.

In Germany, hospitals have a legal obligation to

introduce and use a quality management system.

Additionally hospitals can voluntarily be accredited

according to a large variety of programmes.

The degree to which medication safety standards

are included in the quality management systems

or certification criteria varies by programme.


2.1. Medication Reconciliation context survey

findings in Member States

Australia

Major drivers for Australian hospitals to participate

in the High 5s Project included national and state

policy, public hospital pharmaceutical reforms,

and National Safety and Quality Health Service

Standards1. The existing Australian Pharmaceutical

Advisory Council Guiding principles to achieve

continuity of medication management, 2005

(Guiding principles)2 aligned with the High 5s Med

Rec SOP. Australian states and territories

implementing hospital pharmaceutical reforms

required participating hospitals to implement

the Guiding principles and report performance

measure data. In some states, there were incentives

for hospitals to perform Med Rec.

National Safety and Quality Health Service

Standards, 2011, contain two criteria relating

to medication reconciliation. In January 2013,

it became mandatory for all hospitals to be

assessed against these standards as part of the

new Australian Health Service Safety and Quality

Accreditation scheme.

Clinical pharmacy practice is well established in

Australia, and professional practice standards

for medication reconciliation have been available

since 20073. Medication reconciliation in hospitals

is almost exclusively performed by pharmacists.

The increasing number of pharmacists employed in

emergency departments has contributed to the spread

of medication reconciliation in Australian hospitals.

The Australian Commission:

ñ continues to develop resource materials to

support the spread of medication reconciliation;

ñ works with stakeholder organisations to gain

multi-disciplinary buy in; and

ñ advocates improved access to patient medication

information through information systems such as

the Personally Controlled Electronic Health

Record (PCEHR).

France

Medication reconciliation is not one of the

requirements of the certification/ accreditation

procedure of the National Authority for Health.

However, it is mandatory for all health-care

organizations to be fully involved in medication

The Netherlands

The Dutch Government developed a National

Patient Safety Programme (2008-2012) to support

hospitals in obtaining a certified system by

December 2012. Hospitals are required to

implement ten improvement themes to reduce

preventable harm and reach the theme goals.

Prevention of medication errors through medication

reconciliation for elective patients and at discharge

was one of those themes. The Netherlands have

a national guideline for medication accuracy at

transitions in care which recommends a current

medication history within 24 hours for every patient

at every transition. The Med Rec SOP supports

the implementation of the national guideline for

medication accuracy at transitions in care (2008).

The application of the guideline of the Dutch

Inspectorate of Health has been mandatory since

January 2011. In addition, health insurance

companies are playing a greater role in negotiations

with hospitals about safe care for clients.

There is an accreditation and/or certification

initiative overseen by the national patient safety

programme (2008-2012). Another accreditation

body is the Netherlands Institute for Accreditation

in Healthcare (NIAZ); however, hospitals are not

required to be accredited. The first (non-High 5s)

hospital was accredited by Joint Commission

International (JCI) in 2012 and the second in 2013.

3. Context survey of Correct Site Surgery

The context survey on Correct Site Surgery

was conducted to better understand the existing

supporting structures for this SOP. Two Member

States returned completed CSS context surveys.

These were France and Germany.

Key questions on the CSS context survey addressed

reasons for selecting the CSS SOP, formal

expectations for CSS-related procedures which

existed prior to the introduction of the High 5s

Project, requirements for reporting events outside

of the hospital, and surgical safety requirements

of accreditation systems.

Germany and France selected CSS as one of

the SOPs to implement because of an increased

awareness of the topic of wrong site surgery

resulting from the WHO Safe Surgery campaign

and expert/media exposure. In Germany, more

and more hospitals started to implement surgical

safety checklists and the responsible quality

officers in the hospitals were looking to exchange

implementation experiences and advice. In line

with this, hospitals interested in joining the High 5s

Project were asked which SOPs they were interested

in, and CSS was the top topic chosen. In France,

health-care professions were putting an increased

emphasis on serious adverse events.

While there were no formal expectations of

having a surgical checklist in operating theatres

in Germany, a national surgical safety checklist,

inspired by the WHO Surgical Safety Checklist,

was implemented in France and mandated in 2010.

This checklist also became a requirement for

accreditation in France. In Germany,

implementation of surgical checklists was driven

by published recommendations by the Coalition

for Patient Safety and various scientific and medical

societies.

In Germany, while it is not mandatory to report

events outside of hospitals, hospitals can

voluntarily report ‘no harm’ events and

occasionally harm-events to inter-institutional

Critical Incident Reporting Systems databases.

It is mandatory in France to report any serious

adverse events and voluntary to report near misses.

France has a universal accreditation system.

Pre-operative verification and final time-out for

any surgery are required as part of the mandated

process. There were no financial incentives offered


in either Member State for quality improvement.

However, negative consequences such as

conditional accreditation, was associated with

poor performance in France. In Germany, various

certification programmes exist for hospitals,

and each one applies different criteria relating

to surgical safety.

4. Interim findings

4.1 Summary of qualitative findings

Implementation Status

The full scope of implementation of an SOP

is defined as follows:

1. all of the required steps in the process are

standardized;

2. all of the locations where those steps are to be

put into effect are involved;

3. the population of patients to which those steps

will apply (the eligible population) is involved.

A hospital is considered to be at ‘Full

implementation’ when it has put all required

components of the SOP into effect across all

eligible locations and patient populations

within the hospital. Once the SOP has been fully

implemented, the degree to which it has been

successfully implemented is determined primarily

through hospital’s self-reporting the experience

(i.e., the ‘Implementation Experience’

questionnaire) and through the High 5s

performance measure results.

For purposes of analyzing and reporting evaluation

data, the count of hospitals reporting their level

of implementation at ‘Full implementation’ is based

upon their self-reported status at the time each

quarterly report is generated (typically two weeks

prior to the release of the report).

To date, six Member States have been

implementing either the Correct Site Surgery SOP

or the Medication Reconciliation SOP or both.

One Member State initiated implementation

of the Concentrated Injectables SOP in 2013.

Implementation of this SOP was terminated

because the United Kingdom withdrew from

the High 5s Project. Table 6 shows the

implementation status of the SOPs.


Table 6: Implementation status reported by LTAs

Standard No. of No. of No. of hospitals No. of hospitals No. of hospitals

Operating Member States hospitals indicating indicating with no indication

Protocol implementing implementing ‘Full ‘Not Full of Implementation

SOP SOP Implementation’ Implementation’ status

Correct Site Surgery 4 34 24 9 1

Medication Reconciliation 5 47 11 34 2

Concentrated Injectables 1 16

The primary reasons reported for not having

achieved full SOP implementation in all

participating hospitals and for both SOPs have

been:

ñ insufficient resources;

ñ lack of existing policies;

ñ competing priorities;

ñ inadequate support from leadership; and

ñ resistance to change.

Qualitative findings on Correct Site Surgery

Oversight of SOP implementation

The majority of hospitals assembled

a multidisciplinary group as a means to implement

the SOP. Membership of the oversight group

typically included nurses (operating theater nurses,

anesthesia nurses, nurses from various surgical

subspecialties, nursing directors), surgeons,

physicians (various specialties), patient safety

officers, quality directors, quality assurance

officers and other quality personnel, controllers,

risk managers, department chairs/chiefs of

services/directors of care, hospital administrators,

consultants, secretaries, and other administrative

personnel. Frequently, a principle leader in the

oversight group was identified as the individual

who guided the SOP implementation process.

In addition, more than half of hospitals indicated

that they had role models or “champions”

who assisted in the implementation process.

The most common challenge or barrier to

assembling the oversight group was the lack

of resources (funding and/or time). Resistance

to change and staff buy-in were also frequently

cited as significant barriers. Other barriers noted

by participating hospitals included poor

communication; lack of organizational leadership

buy-in/support, and lack of department leadership

buy-in/support. One participating hospital noted

that the data collection and validation processes

are labor intensive and required at least one

dedicated full-time employee. Another hospital

described difficulty in scheduling meetings,

especially when physicians and surgeons were

involved.

To overcome these barriers, hospitals employed

the following strategies:

ñ improving communication;

ñ obtaining leadership support;

ñ providing training/education on the SOP;

ñ developing strategies to manage change, involve

staff, improve the safety culture, increase

communication, and provide recognition; and

ñ requesting assistance from the LTA.

Risk assessment

Half of the participating hospitals reported that

they conducted a risk assessment as part of the SOP

implementation process. Of those that conducted

a risk assessment, the majority indicated that

they followed the process described in the SOP

materials. Most hospitals reported that the risk

assessment process helped them identify potential

areas for breakdown and allowed them to

implement controls, warnings or protections

to minimize identified problem areas. Through

the risk assessment process, one hospital identified

emergency cases and patients in non-surgical

departments as potential areas for breakdown/

failure. Controls were then put in place to include

a monthly check of operating room cases, use

of pre-operative checklists, and instructions that

only patients with a completed checklist can enter

the operating room theatre. Other hospitals

developed printed guides and posters to remind

staff to adhere to the SOP, and implemented new

processes for administrative staff (as well as

surgeons) to verify the operating room log.

Lack of staff involvement, resistance to change,

lack of experience in conducting risk assessments,

and lack of resources were the primary barriers to

conducting risk assessments in approximately half

of the hospitals. Others claimed that the process

was already implemented, hospitals were already

aware of their risks, the event was considered too

rare to make it necessary, or they did not think that

it was a requirement to conduct a risk assessment.

Pilot testing

Almost all of the hospitals reported that pilot

testing was conducted prior to full SOP

implementation. Pilot testing was conducted to

help hospitals determine the feasibility and impact

of SOP implementation on a small scale before

widespread implementation occurred. Pilot tests

were carried out in several units/wards including

the operating theatre/surgical ward and orthopedic

surgery, thoracic surgery, and gynecology services

as well as other general wards. The major

challenges and barriers to conducting pilot tests

were resistance to change, data collection burden,

and lack of resources. To address these barriers,

hospitals increased communication, enlisted staff

and leadership support, and provided education/

training. Despite challenges, the majority of

hospitals that conducted pilot testing reported

that it positively impacted related activities and

patient care.

Hospitals noted that the pilot tests increased

awareness of safety before the surgical operation,

assisted in encouraging pre-surgical operative

site-marking, and increased attention to verifying

patient identity.

Spread methodology

Approximately half of the responding hospitals

implemented the SOP sequentially in various

hospital locations. The other half implemented

the SOP concurrently in all sites. Three quarters

of responding hospitals integrated the SOP process

into or added it to existing systems processes.

Lack of staff buy-in, resistance to change, lack

of resources and data collection burden were

frequently cited barriers to spreading SOP

implementation. Increased communication, training

and education on the SOP, and leadership support

were the most commonly employed strategies

to address these barriers. One hospital reported


that the project was discussed in the medical

commission, and this led to additional resource

allocation for the project. Another hospital

provided continued training on the checklist

procedure, conducted audits, and utilized the

checklist training kit. Still some indicated that,

despite efforts, the barriers and challenges

continued to be a struggle.

High 5s Project information has been most

commonly shared during department and project-

specific meetings. Staff feedback has often been

solicited directly by High 5s project coordinators.

Most hospitals reported some formal

acknowledgement of staff members that actively

participated in the implementation of the SOP.

Recognition often occurred at project meetings,

grand rounds and in printed materials such as

newsletters.

Evaluation

Approximately three quarters of responding

hospitals indicated that they experienced challenges

and barriers in implementing the performance

measures. The primary barriers were lack of

resources (funding and time) to gather the data,

difficulty understanding the performance measure

definitions and/or methodology, and lack of staff

buy-in. For this reason, many hospitals reported

that they were not using the prescribed High 5s

data quality methodology.

Strategies to address barriers included improved

communication with staff that had access

to needed data, engaging staff, and enlisting

the help and guidance of those in leadership

positions. Additional steps taken to evaluate

SOP implementation included observation audits,

development of training programmes, computerized

checklists, and development of databases to assist

in the extraction of performance measure data.

One hospital noted, “With the High 5s SOP,

between 2010 and 2012, we were able to prevent

12 potential events in pre-operative patients.

We analyzed these events and found that nine

errors occurred during surgical consultation,

two errors occurred during site marking in

the surgical unit and one error from the primary

care physician (wrong side written on the letter

to the surgeon).”

Strategies for maintenance and improvement

Most of the hospitals reported that they had been

engaging in regular and ongoing monitoring

of key aspects and activities related to the SOP.

Hospitals were performing data quality monitoring,

conducting audits (chart and electronic),

soliciting information from staff, and engaging

in project oversight meetings. Many hospitals

identified opportunities for improvement that

included better involvement of staff, more training,

better definition of the site-marking procedure,

simplification of the checklist, IT system

enhancements, improved communication with

surgeons, and revision of the time-out process.

In many cases, opportunities for improvement

had been identified but not resolved. Many

hospitals reported ‘drifting’ from the intended

SOP as time went on. The drifting was identified

through data quality monitoring. To address

drifting, hospitals provided increased

educational/training opportunities and encouraged

participation in project-related meetings.

Qualitative findings on Medication

Reconciliation

Oversight of SOP implementation

Hospitals which responded to this issue reported

that they had assembled an oversight group to help

implement the SOP. Oversight groups usually

consisted of pharmacists, nurses, physicians,

quality managers, and a variety of other staff

working in the patient safety area. The majority

of respondents said that the principal leader of

their oversight group was a pharmacist or pharmacy

manager. Lack of resources (funding/time) was

the most common barrier to assembling the

oversight group. More effective communication,

training and education on the SOP, and securing

leadership “buy-in” were common strategies to

address the challenges associated with assembly

of an oversight group. Several hospitals noted

the importance of expanding participation of key

stakeholders beyond the pharmacy.

Risk assessment

Less than half of responding hospitals completed

a risk assessment. The primary reason for not

conducting a risk assessment was lack of resources

(time and funding). In addition, risk assessment

was considered a low priority activity for some

hospitals. These hospitals already had processes

in place for medication reconciliation, so found it

unnecessary. However, those who did conduct


a risk assessment reported that it was beneficial in

helping to identify potential areas for breakdown.

In response to the risk assessment, a number

of hospitals implemented controls, warnings

or protections to minimize the potential for

breakdown or failure. Examples of controls and

protections that were implemented as a result of

a risk assessment include improved communication

through the hospital quality committee

and the medication management committee,

multi-disciplinary events analysis, and monthly

pharmacist meetings to implement actions

and monitor and review progress.

Pilot testing

Almost all of the hospitals completing the

questionnaire conducted a pilot test of the SOP.

For those that did not, lack of resources, data

collection burden, lack of organizational support

and resistance to change were cited as the most

common barriers. Pilot tests occurred in a variety

of units, including geriatrics, internal medicine,

rheumatology, emergency and cardiac care.

Routine auditing, use of independent observers,

development of specific forms and procedures,

and training and education on the SOP were

the most common methods hospitals used to

assure the consistency, timeliness and accuracy

of the implementation. Most hospitals reported

that implementation of the SOP had a positive

impact on related activities and on patient care.

Increased communication and more collaboration

among hospitals, physicians and community

health-care providers was a result of the pilot

test noted by many hospitals. Many pharmacists

indicated that they enjoyed being more directly

involved in patient care and that implementation

of the SOP improved the visibility, importance

and understanding of the role of the pharmacist in

the patient care process. Although some indicated

that implementation of the SOP increased staff

workload, most reported that it was a positive

experience that led to improved electronic

prescribing systems, new patient safety projects

related to medication reconciliation, and funding

for additional studies.

Spread methodology

Approximately half of the respondents implemented

the SOP sequentially in their hospitals. Many noted

that the SOP was just an extension of their existing

medication reconciliation process. Several

challenges to implementation were noted. These

included lack of resources, data collection burden,

insufficient communication, and resistance to

change. To overcome these barriers, hospitals

most frequently provided training and education,

increased communication, involved staff in the

process, and provided greater leadership support.

Despite these efforts, several hospitals noted

that many barriers still existed to successful SOP

implementation.

High 5s Project information has been most

commonly shared during department and

project-specific meetings. Staff feedback was

often solicited directly by High 5s Project

coordinators. Most hospitals reported some

formal acknowledgement of staff members

who actively participated in the implementation

of the SOP. Recognition often occurred at

project meetings and grand rounds and in printed

materials such as newsletters.

Evaluation

More than three quarters of responding hospitals

indicated that they experienced challenges or

barriers when implementing the SOP performance

measures. The primary barriers/challenges were

lack of resources dedicated to evaluation, lack of

staff to perform the required independent observer

role, lack of understanding/clarity regarding the

performance measure definitions and methodology,

and difficulty accessing medical records for

data collection purposes. Some hospitals also

experienced technical issues with sampling

and achievement of high inter-rater reliability.

In addition, some hospitals noted that nurses,

physicians and patients were dissatisfied with

the process because patients were asked twice

about current medications with no clear advantage.

One hospital noted that quality improvement,

alone, is insufficient to motivate behavioral change.

To address these barriers, most hospitals attempted

to better engage staff and provide training and

educational support on the evaluation component

of the project. Others provided additional

resources and staff such as a dedicated pharmacist

to work on the project and assist with timely

data collection. One hospital utilized students for

data collection and maintained a log of clinical

scenarios that are encountered and are outside

the realm of the SOP.

Many hospitals have taken additional steps to

evaluate the implementation of the SOP. These

include analysis of interventions by pharmacists

in response to unintentional medication

discrepancies, analysis of the number of errors

detected through timely medication reconciliation,


and attempts to build a business case for

an extended-hours model to present to hospital

executives and clinical staff. One hospital noted

that, in the future, they intend to measure patient

and physician satisfaction with the medication

reconciliation process.

Strategies for maintenance and improvement

Participating hospitals took many different

approaches to monitor the implementation

process, including data quality monitoring, project

oversight meetings, audits (direct observation

and chart review), soliciting information from staff,

and participation in LTA interviews. Several

hospitals indicated that they continue to struggle

with concurrent data collection and are several

months behind.

Hospitals provided examples of improvement

strategies that included development of data

collection forms, development of written

instructions/processes, training for data collection,

and increasing documentation required of pharmacy

staff. Through the auditing process, many hospitals

noted “drifting” from the intended SOP process.

To address drifting, additional project meetings

occurred and supplemental education/training

was made available to staff.

Summary of qualitative findings: successes

and lessons learned

At this stage of the High 5s Project, it is clear

that it is possible to implement specific patient

safety protocols with some degree of

standardization across multiple hospitals and

multiple Member States.

Participating hospitals were able to successfully

implement the key components of both the CSS

and Med Rec SOPs. However, the SOPs could not

be implemented as “one-size-fits-all” solutions.

Some local customization of each protocol

was necessary. This local customization (at both

the country-level and hospital-level) was essential

to secure initial buy-in. Customization was equally

important for sustaining post-implementation

changes.

Analysis of project data and the qualitative

experiences of project participants will be used

to produce a final set of SOPs that highlight

key components of the SOP, as well as offering

recommendations for local customization.

Interim lessons learned are summarized below.

A. General lessons learned

ñ Data collection is burdensome. Many

organizations have struggled to identify

adequate resources for data collection. Very

few organizations added resources for the

High 5s Project, thus providing little earmarked

staff time for SOP implementation, much less

the associated data collection activities.

ñ Data collection and measurement are absolutely

necessary for successful implementation

(i.e., they helped demonstrate the need for SOP

implementation, they provided the implementation

team with a tool for demonstrating the impact

of implementation efforts, and the act of data

collection and feedback of measure results helped

hospitals maintain attention on the patient safety

area).

ñ Resistance to change (staff and leadership)

is a challenge to implementation.

ñ Ongoing communication, education, and training

are important to achieve and sustain compliance

with the SOPs.

ñ Senior-level hospital leadership and support is

critical to the full and successful implementation

of the SOPs.

ñ On-site champions are critical to successful

implementation.

ñ The exchange of information among hospitals

(within Member States) and among LTAs (across

Member States) helped to build and maintain

enthusiasm for SOP implementation.

B. Med Rec SOP

ñ Implementation of this SOP was most successful

when a pharmacist (or pharmacy staff) was

available to perform the medication reconciliation.

ñ Pharmacists and pharmacy technicians tended

to produce timely and accurate BPMHs.

ñ Limited pharmacy resources were a major

limiting factor for implementation.

ñ Additional performance measures (or different

measures) were useful. In particular, measures

that identified the number of medication

discrepancies within 48 hours of admission,

and those discrepancies unresolved at 48 hours

from admission, were used to demonstrate

the importance of the SOP to clinicians and

decision makers.


C. CSS SOP

ñ Surgeon resistance was a common challenge

to SOP implementation. Thus, placing a surgeon

in the role of SOP champion tended to facilitate

implementation.

ñ Concerns that patients may be resistant to site

marking were unfounded.

ñ In the absence of specific direction from

the Ministry of Health, the High 5s checklists

were not adopted “as-is.” Successful and

sustained implementation required customization

of the checklist so that it could be incorporated

into existing tools (i.e., once adopted, it is much

more difficult to terminate implementation if

the checklist is built into existing processes).

4.2. Member State implementation experiences

and qualitative findings

Australia

LTA role in implementation

Hospitals were recruited through an expression

of interest process, and 18 health services were

selected to participate. The project commenced

in January 2010 with the first teleconference

which was attended by over 60 participants from

the 18 sites. Teleconferences continued to be held

monthly throughout the first two years. In year

three, they were reduced to every second month.

A two-day train-the-trainer workshop was held

in Sydney on 19- 20 April 2010 and was attended

by 64 participants from the 18 health services.

A second two-day workshop was conducted in

October 2010 and two further one-day workshops

were conducted in 2011. Several webinars were

also held to provide further detail on the project

methodology. A table of the activities conducted

for the Australian sites is available in Annex 7.

Copies of all presentations and materials used

were placed on the Australian page of the High 5s

website. The sites were supported by a dedicated

project officer.

Resources

The Australian Commission developed a number of

resources to assist hospitals to implement the SOP.

These included:

ñ A national Medication Management Plan (MMP)

form to record the ‘Best Possible Medication

Histories’ (BPMH), and document the reconciliation

process, discrepancies and their resolution;

ñ Educational materials for health-care

professionals using the MATCH UP Medicines

theme;

ñ Information for consumers.

Examples of the resources are available in Annex 9.

Evaluation

Implementation experience was evaluated

through the use of the implementation experience

questionnaire, hospital interviews, and direct

feedback from sites (see Annex 8). All sites

were requested to complete the implementation

questionnaire as per the schedule set by

the Steering Group. Response rates were high.

Five hospitals were identified to participate in LTA

implementation interviews. A mix of hospitals

was selected from four states: a major teaching

hospital, a private hospital, two metropolitan

hospitals and a rural and regional health service

(nine hospitals). The first round of interviews

was conducted in 2011.

Challenges encountered

Ten out of 11 health services responding to

the implementation experience questionnaire

in June 2012 had implemented all the steps in the

medication reconciliation (Med Rec) process; five

health services had yet to spread the process to all

locations. Insufficient resources and competing

priorities were the most frequently identified reasons

for not implementing the SOP in all locations.

Eight sites identified lack of resources as a barrier

to implementing the SOP. Hospitals said they

underestimated the resources they would need,

especially for education and the evaluation process

(the independent observer role, collection of

performance measure data, and the event analysis).

The burden of the data collection for performance

measures MR 2-4 was an ongoing issue for all sites.

Lack of resources for data collection delayed the

start of data collection at several sites.

The five sites interviewed indicated that additional

resources were needed to implement and sustain

the SOP. This was particularly the case for hospitals

that introduced the process in pilot wards and then

moved to spread the process throughout the facility.

Hospitals opting to employ nursing and medical


staff to take the BPMH found they had to allocate

extensive time to initial and ongoing training.

Resistance of health-care professionals other

than pharmacists to engage in the process

was another challenge for Australian hospitals.

Prior to commencement of the High 5s Project,

several hospitals had an existing medication

reconciliation process in place that had been

provided by pharmacy staff. These hospitals had

a culture of medication reconciliation being

“pharmacy business”. The hospitals experienced

challenges on weekends and after hours when

pharmacists were unavailable and when

pharmacists were on leave.

In those hospitals where attempts were made

to involve nursing staff and training was provided,

there was resistance among the nursing

staff to take on increased activities as well as

a lack of confidence in their ability to participate

in the process – it was seen as ‘not their job’.

Two examples of barriers to Med Rec

implementation are presented in Box 3.


Initially, independent observers had difficulty with

understanding the performance measures and

measuring consistently. Tightening the definitions

of the types of discrepancies, providing additional

training, and creation of a list of examples helped

overcome this problem.

Other challenges experienced in implementing

the SOP were those commonly experienced when

implementing quality improvement projects:

ñ competing priorities in the organization;

ñ lack of support from hospital leaders;

ñ loss or transfer of key personnel;

ñ initial lack of clinician buy-in; and

ñ the introduction of an electronic health record

that did not support the SOP.

‘Drivers’ of patient safety

Sites utilised a number of strategies to overcome

the barriers to implementing the SOP and effect

change in their organizations.

a) Engaging staff and obtaining ‘buy-in’.

Hospitals actively engaged staff in the project.

All sites assembled a project oversight group.

All groups included pharmacists and most included

nurses. Doctors, quality and safety personnel,

educators, and hospital leaders also participated.

Nine of the 11 health services designated at least

one role model or champion from each discipline

involved in SOP related activities. Most hospitals

developed a communication plan and used a variety

of methods to communicate with staff about

the project. These included regular feedback

on performance measures and periodic reports on

specific aspects of the project.

b) Training and education

Educating staff about the medication reconciliation

SOP and training staff on taking a BPMH and

reconciling medicines were key to the successful

implementation of the SOP. Hospitals devoted

considerable resources to training and employed

a number of different approaches to education.

Two examples of training and education on taking

a BPMH are presented in Box 4.

Box 3: Barriers to Med Rec SOP implementation in Australian hospitals

Hospital 1. Eight emergency department registered nurse “Pioneers” were trained to take a BPMH

and conduct medication reconciliation however one year later these ‘Pioneers’ were not performing

these processes, anecdotally because of time constraints.

Hospital 2. Nursing staff were not confident in conducting medication reconciliation on their own

despite the training provided. Barriers to successful implementation by nursing staff were cited as time

constraints, inadequate staff to perform the process, and lack of in-depth pharmaceutical knowledge.

c) Leadership support

Securing leadership support was cited by several

sites as one of the key strategies they employed for

overcoming the challenges and barriers experienced

in trying to spread the SOP throughout the

organisation.

d) Actions taken based on data

Data was used in a number of ways by the different

sites including:

ñ obtaining buy-in to the project and engaging

staff outside the pharmacy, including senior

clinicians and the hospital executive;

ñ building a business case to extend the hours

that medication reconciliation is performed;

ñ showcasing the quality of the process performed

by staff and addressing any training needs

identified;

ñ identifying gaps in processes and addressing

these gaps; and

ñ demonstrating the hospital’s compliance with

the National Safety and Quality Health Service

(NSQHS) Standards for accreditation purposes.

Two examples of positive outcomes based on data

are presented in Box 5.


Box 4: Training and education on taking a BPMH in Australian hospitals

Hospital 3 introduced medication history taking using BPMH principles as part of medical intern

orientation each year, followed by more focused case-based workshops on medication matching

for interns, residents and registrars.

Hospital 4 developed a simulation exercise to train clinicians (doctors, nurses and pharmacists) and

students to take a BPMH. The hospital had access to a simulation ward and experienced facilitators

who worked with the High 5s team at the hospital to develop the simulation exercise. A number

of different scenarios were developed, and the participants were able to access different sources

of information during the exercise, such as calling the patient’s community pharmacist – played by one

of the hospital pharmacists. The use of simulation has the advantage of providing a safe environment

for clinicians to practice their skills with no risk to patient care. It also allows participants to receive

immediate feedback and to learn from others, including the patients.

Box 5: Positive outcomes based on data in Australian hospitals

Hospital 3. The dissemination of Med Rec performance measure results from the top levels in the

hospital down through the Patient Safety Sub-Committee, including to hospital executives, has resulted

in executive engagement.

Hospital 5 used the performance measure data to restructure their clinical pharmacy duties by changing

to a team-based structure and increasing the number of BPMHs completed and medication

reconciliations documented.

e) Effect on patient care

About half the hospitals indicated that

implementation of the SOP had an impact on

related or interacting activities and on patient care.

Most were considered positive and included:

ñ having a standard and improved protocol for

medication reconciliation;

ñ improving processes through raising awareness

of the processes of obtaining a BPMH, reconciling

medicines and following up on discrepancies; and

ñ using data collected to plan interventions to

follow up on discrepancies.

Implementation Surprises

The first surprise was the realisation by the sites

that the SOP could not be implemented with

existing resources and additional resources would

be required to perform the medication reconciliation

process and spread the process to a greater number

of patients. Several hospitals were able to secure

funding to employ additional pharmacists to

introduce clinical pharmacy services to Emergency

Departments and extend clinical pharmacy hours.

Box 6 presents two examples of pharmacist services

that were extended over weekends.

Private hospitals experienced difficulty implementing

the SOP, as their staffing and work practices

differ from public hospitals. Pharmacy services

are limited, and there are no junior medical staff.

Another surprise was the time commitment

required for education and training, particularly

at those sites implementing a multidisciplinary

approach to medication reconciliation. In addition,

junior medical staff regularly rotate through

a number of hospitals, requiring a new cohort

to be trained every three months.

All sites were surprised that collection

and management of the data for the performance

measures and the event analysis was so

burdensome. This resulted in delays in data

collection and reporting, particularly when

there were staff changes and departing staff

were not replaced.

Several sites were surprised by the number

of discrepancies identified by the independent

observer that remained unresolved 48 hours

after admission. Initially, up to as many as 53%

of patients at these sites had unresolved

discrepancies. Sometimes these discrepancies

were still unresolved at the time of discharge,

causing delays in discharge. Two additional

performance measures were developed to enable

hospitals to collect data on these discrepancies

(see section 5.4.5)

The introduction of an electronic health record

(EHR) in the ED in some hospitals that did not

support the medication reconciliation SOP

resulted in duplication of work and threatened

the continuation of the project in one site.

This was completely unexpected and highlighted

the importance of project teams being aware

of information technology initiatives and engaging

with IT services when introducing system changes.

Lessons learned

Successful implementation of the medication

reconciliation SOP hospital wide requires:

ñ recognition that medication reconciliation

is a patient safety priority;

ñ senior leadership support;

ñ resources;

ñ ongoing training; and

ñ integration of the process with hospital

information systems.

Further information technology systems on the

Australian medication reconciliation resources

is provided in Annex 9.

France


Hospitals were recruited to implement the Correct

Site Surgery (CSS) and Medication Reconciliation

(Med Rec) SOPs through an expression of interest

resulting in commitments by 18 health-care

organizations – nine hospitals for each SOP–

to participate in and initiate the Project in 2010.

Subsequently, one hospital withdrew from the CSS

project following the retirement of the initial

surgical lead. Likewise, one hospital withdrew

from Med Rec SOP implementation because of

competing priorities, lack of leadership, and lack

of resources. Monthly communication with

the hospitals was done through teleconferences.

Currently, eight hospitals are participating in

the implementation and evaluation of each SOP.

A. Correct Site Surgery SOP

Evaluation

Monthly performance data were edited in an Excel file

created to generate numerators and denominators.

A sample of 20 anonymous checklists per hospital

was reviewed by the LTA every month. The LTA


Box 6: Pharmacist services extended over weekends in Australian hospitals

Hospital 7: Pharmacy clinical services have increased over the past three years to include weekends

and pharmacist services to the emergency department have been extended to 21:00 hours.

Hospital 2: Initiation of a Saturday ward-based clinical pharmacist service responsible for medication

reconciliation on new, high-risk admissions at a metropolitan hospital significantly improved

performance measure MR-1 rates.

provided monthly feedback to the project hospital

coordinators through a conference call and

a report. This feedback aimed to promote the

correct use of the checklist and to help the hospital

teams to define the actions required to maintain

or improve reliable performance measurement.

Time Out observational audits were conducted in

addition to the monthly questionnaire, interviews

and event analyses required by the SOP.


During the first two years of the High 5s Project

(2010-2011) the challenges included:

ñ Difficulties in integrating the High 5s Checklist

into the WHO Surgical Safety Checklist.

Promoted by HAS, the WHO Checklist became

mandatory in the French accreditation/certification

process in 2010. This checklist does not include

an expectation of site-marking, while the High 5s

Checklist does. With the support of CEPPRAL,

each participating hospital team worked towards

integrating the checklists, and developed an

individualized checklist that included all mandatory

SOP items, while still retaining other items and

the format of their choice. Some health-care

organizations (HCOs) had worked on the patient

clinical pathway and had developed a booklet that

included similar procedures already implemented

before initiation of the High 5s Project.

ñ Surgical site marking as required by the CSS SOP

was new to all hospital teams. Its implementation

caused organizational challenges. The reluctance

of some surgeons to comply with this

requirement was one of the challenges.

ñ It took one year before CEPPRAL was able

to transmit data from participating hospitals

to the Collaborating Center. Only four HCOs

had valid data in 2011, but all eight HCOs had

valid data in 2012.

Disseminating information and lessons learned

remains a challenge, as half of the participating

hospitals are still at partial implementation.

With regard to the four hospitals at full

implementation, the lack of specific resources

dedicated to the project has been a persisting

challenge for evaluation – in terms of review

of performance measure data and analysis

of events – and communication. The duration

of the project and the turnover of staff (e.g., key

leaders, surgeons, risk managers) has also been

a challenge.

For most hospitals, the main barriers encountered

have been lack of time to form multidisciplinary

teams; develop leadership; achieve cultural

changes; and mobilize surgeons and anesthetists.

In addition, challenges encountered by a few

hospitals have included competing priorities and

lack of logistic support.

‘Drivers’ of patient safety

Across all hospitals the following stood out

as strong drivers for patient safety:

ñ performance measures (see example at Box 7);

ñ benchmarking; and

ñ generation of “success stories” (see example

at Box 8).


Box 7: Communicating results of performance measures by two French hospitals

Because of the resistance of some surgeons to implementing site-marking and Time Outs as required

in High 5s CSS SOP, and in an effort to motivate participation, two hospitals communicated on

a monthly basis the results of performance measures for a given surgeon. This was done by e-mail

(individual results) and also posted outside the operating theatre (anonymously).

Box 8: Success stories are about errors stopped during the pre-operative phase

Teams were requested to report errors not evaluated in the High 5s Project to the LTA which assessed

their frequency and time of occurrence and identified the steps where the errors had been intercepted.

Up to the present, about 20 success stories have been collected. Annex 13 presents the framework used

to collect such success stories.

Implementation surprises and lessons learned

ñ Commitment of paramedical staff (technicians

and administrative staff) to the High 5s Project.

ñ Rapid uptake of the High 5s checklist by

substitute nurses working during summer period

in ORs and on the wards (reported by one

HCO).

ñ Streamlining the Time Out (e.g., through

the use of key words) provided teams with

better communication and valuable Time Out

(1 HCO).

ñ About 70% of all errors were intercepted in

the pre-operative verification phase.

ñ Site-marking was well-accepted by the patients,

none refused the site-markings.

ñ A culture change was seen where members

of the non-medical staff felt empowered

to speak up when discrepancies arose.

ñ Great involvement of frontline staff; and

ñ Greater participation of patients in the

verification procedures.

B. Medication Reconciliation


ñ The appropriation of the Med Rec protocol

by various teams, as well as inadequate

understanding of the performance measures.

ñ Resistance to the BPMH completion deadline

of 24 hours after patient admission.

Reconciliation at 48 hours was considered

more realistic and reflected better the work

and progress made by the hospitals.

ñ The burden of data collection. The workload

associated with the implementation of this SOP

was a challenge and resulted in the application

of the process to a limited number of care units

in six out of the eight participating hospitals.

Implementation surprises or successes

ñ The increased understanding of the importance

of the problem achieved through demonstrating

the magnitude of the discrepancies between

the medications prescribed and those actually

taken and of the potential harm to patients

resulting from those discrepancies.

ñ The development of tools to measure the

potential significance of discrepancies.

ñ The ability of some participating hospitals to

streamline their medication management process

in order to perform on a regular basis a proactive

medication reconciliation in the emergency room.

ñ The development of information technology

tools to follow medication administration

and changes throughout the patient course

within the hospital and at discharge.

ñ The development of a clinical pharmacology

capacity integrated into the care units.

Germany

Summary how LTA evaluated implementation

The below description applies only to the

Correct Site Surgery SOP because no evaluation

of implementation experiences has yet been

conducted for the Med Rec SOP.

The evaluation component of interviews and

implementation questionnaires was conducted

as a part of a diploma thesis.

In addition to the High 5 Project requirements,

the German LTA analysed additional matters of

interest. This was because the implementation

questionnaire was felt to be rather general in scope.

Thus, the German LTA included SOP-specific

questions in the questionnaire which focused on

challenges encountered when implementing

the specific SOP process steps. The expanded

questionnaire was administered from December

2011 to February 2012 by the project coordinators


Box 9: Examples of actions taken by LTAs based on data

1) Training of nurses to use the checklist correctly (one hospital).

2) Development of a national surgical site-marking guide in collaboration with the HCOs

and The Joint Commission (Annex 12-France).

3) Fostering leadership of directors: conference calls or meetings with leader of HCOs having

difficulties in the implementation process. A quarterly newsletter which includes results and

benchmarking data is being developed by the LTA.

in all 16 project hospitals; responses were received

from 11 hospitals.

Semi-structured interviews were conducted in

March 2012 with the project coordinators in three

hospitals to gain more detailed information about

barriers, drivers and the resources required for

implementation, as well as the potential for future

sustainability. The interview findings were analysed

using a systematic qualitative evaluation method

(method of Meuser & Nagel).


The main challenges for the hospitals’ implementation

processes were resistance to change, oncompliance

of staff/management, and a lack of resources

(funding/time). Hospitals described how individual

“sceptics” could torpedo progress in implementation

and that these sceptics needed to be addressed

individually.

In relation to the SOP process steps, the German

LTA identified the following main barriers:

resistance to change; frontline staff not sufficiently

acquainted with written standards on verification

process; difficulties in adapting to new process;

conflict about the preferred site mark; and team

Time Out not taken seriously and not exemplified

by role models.

Summary of findings

Key strategies for overcoming the challenges

encountered in the implementation process

were improved communication, involving staff/

management, and training/education on the SOP.

Essential strategies for overcoming barriers

related to the SOP process steps were the provision

of written instructions on the SOP, individual

‘pep talks’, retraining and further education, and

adjustment of the checklist.

Important drivers in implementing the CSS SOP

were the similar process design in different

departments; employees having previous experience

with standardization; interdisciplinary and cross-

hierarchical composition of the implementation

oversight group; regular meetings of the project

work group; and the fact that clinical leaders

used the checklist, thereby providing a role model

function.

Important factors identified in the interviews

included:

ñ intense support is necessary especially for the

implementation start and training phases;

ñ pilot site testing paves the way and unlocks

drivers for implementation; and

ñ substantial time and effort are needed for data

management because there is variable checklist

documentation quality.

There is no information available as to how the

hospitals used the data from the implementation

questionnaires.

On the LTA side, at the end of the project,

recommendations will be derived from these

results and published in order to produce guidance

for other health care organisations planning

on implementing similar processes.

Lessons learned

Some important lessons learned relating to

the implementation process:

ñ feedback of some hospitals: “It’s not just

a checklist, it has helped us to fundamentally

change the way we look at our process”. Thus

the systematic implementation of the SOP

radiates to other not yet standardised processes;

ñ the local tailoring of the instruments (i.e.,

the checklists) was relevant to the acceptance

by the participating hospitals;

ñ departments (and hospitals) ‘infect’ one another

with enthusiasm in relation to the SOP; and

ñ acceptance for the new process and completing

the checklists is generally higher in nursing staff

than among physicians.

For more information see Annex 15.


The Netherlands

Implementation surprises and lessons learned

The implementation questionnaires from 2010

to 2012 showed no differences in six hospitals

after successful implementation. One hospital

wrote that the SOP is continuously under review

and minor changes have been made. With no

changes in the SOP implementation status in 2012,

the SOP was implemented in two additional

departments. The increase in the number of patients

being verified within 24 hours was because extra

staff had been assigned to support medication

reconciliation.

In one hospital, a hospital-wide medication

multidisciplinary medication committee was

established at the end of 2012, and all

communication about Med Rec was processed

through this committee. Another hospital noted

that staff members did not always want to sign

for the medication reconciliation done by

the pharmacy. New doctors are now being trained

on the SOP implementation process and in using

the electronic prescribing system in this hospital.

Another hospital plans to expand the SOP

implementation to patients at discharge (i.e.

beyond the targeted High 5 population) in 2013,

provide feedback of data to the units, provide

a further educational programme, and continue

with the qualitative measurements according

to the High 5s SOP.

Box 11 presents some important lessons learned.


Box 10: Two quotes about the High 5s Project

ñ ‘Yes, it is proven that it works. It was a positive experience, because of the interaction with other

participating hospitals and it really worked to go on with implementation activities and measurements.’

ñ ‘For our hospital it was clear that it would be easier to implement medication reconciliation when

being part of an international project, because of the status.’

Fifteen Dutch Med Rec hospitals are participating.

The first-group of 11 hospitals completed the

qualitative structured and web-based

implementation questionnaires starting six months

after SOP implementation in the summer of 2010

to March 2011. In measuring the follow-up to

these questionnaires, most hospitals showed no

significant changes, except for a few hospitals

which restarted performance measurement

or other activities when expanding the SOP

implementation hospital-wide. Hospitals where

the implementation progress was strong,

continued to spread their activities successfully

and experienced a sustained implementation of

the SOP to become part of daily practice.


All hospitals intend to expand the implementation

of the SOP to internal transfers and discharges,

and patients younger than 65. Full implementation

of the SOP has varied according to hospital size.

‘The best implementation and sustainability

strategy for our hospital is ‘to make it not

a hospital pharmacy thing.’

Dutch hospital leader

All hospitals assembled oversight groups and

created a work plan that was evaluated by the LTA.

The main challenges and barriers to spreading

results hospital-wide included limited resources

(funding and time); insufficient communication;

lack of access to medical records or incident

management systems; lack of leadership/staff

buy-in; and resistance to change. The principal

leader of the implementation process in all

participating hospitals was a hospital pharmacist.

Most hospitals chose to start the implementation

on a small scale first before spreading the SOP

hospital wide.

The six Dutch hospitals interviewed highly

recommend the Med Rec SOP developed by

the High 5s Project. Some of their quotes include:

ñ implementation for other hospitals is not

only recommended; it should be considered

a necessity;

ñ we surely recommend implementation of this

SOP to other hospitals. You can try to do this

on your own, but you might encounter pitfalls;

ñ the exchange of knowledge and experience is very

helpful; however, you still have to translate the

SOP to your own local setting;

ñ most of the hospitals advice was to let the

pharmacy technician interview patients: ‘…..it is

proven that interviews done by pharmacy

technicians are of better quality than when done

by doctors’;

ñ the hospitals have no doubt about continuing

to pursue the Med Rec SOP implementation

hospital-wide. One of the hospital staff said:

‘Oh yes! We want the SOP for all patients and

all departments for emergency patients, elective

admissions and discharged patients’;

ñ the strategy of one hospital has been never

to worry too much about resources and budgets.

As shared by one of its staff, ‘You always need

to be proactive. Just start and do instead

of believing something is impossible’;

ñ the key lessons learned from one hospital is

that it was important to keep most of the SOP

implementation in-house so there is no need

to depend on others or other systems nor to

compete with projects of other departments;

ñ another hospital mentioned that quality

improvement is not about implementing the SOP

only. Showing the benefits to all stakeholders

is needed to create support from others.

Generating awareness and communication are

the most important aspects of success, as many

people are involved in the SOP implementation.

Trinidad and Tobago


The Correct Site Surgery SOP was selected for

implementation as a means of standardizing

patient safety solutions within surgical units

in selected hospitals. There were no formal

expectations for correct site surgery procedures

before introduction of the High 5s Project.

The LTA facilitated the piloting of this SOP

at four of the participating hospitals during the

latter half of 2012. The orthopaedic department

was the chosen area for the pilot of the SOP.

There was a total of 27 849 surgical cases

performed at all five participating hospitals

in-patient surgical facilities in the year 2011.

Of this number, there was an estimated one

to two wrong site surgeries. Reporting of wrong

site surgery is specified in the guidelines set out in

the Ministry of Health’s Adverse Events Policy.

This Policy was developed in collaboration with

the Pan American Health Organization (PAHO)

and issued by the Ministry of Health to both public

and private health-care facilities in December 2011.

Additionally, the WHO Surgical Safety Checklist

was adopted and implemented at all public

hospitals in 2010.

At each participating hospital, the project team

leader completed the ‘Hospital Implementation

Experience Questionnaire’ in collaboration

with team members. The completed questionnaires

were then forwarded to the Directorate Health

Services Quality Management, Ministry of Health.

Additionally, the team leader or representative

reported on the status of implementation at

the monthly High 5s team meeting. Meeting venues

were rotated among the participating hospitals.


Box 11: Lessons learned

ñ Integrate the SOP into routine care processes.

ñ Inform all involved about the process and

progress of implementation through regular

project meetings.

ñ Be enthusiastic.

ñ Be sensitive to the workload of the staff.

ñ Accept imperfection at the beginning

and work towards perfecting the operational

process. This is extremely important

to maintain buy-in and dedication.

ñ Be aware that the SOP requires input from

various sources to become practical. Work

toward integrating these sources.

ñ Implement step-by-step by creating internal

success stories to spread the SOP hospital-

wide and Keep It Stupidly Simple (KISS).

ñ Involve people: show results and apply.

www.high5s.org/Netherlands/2013.02.01


Major challenges common to all participating

hospitals included the frequent turnover of team

members, resistance to change by surgeons and

nurses, lack of staff buy-in, and real or perceived

implementation/data collection burden.

The support of the ‘Quality Improvement Unit’

staff at each hospital and regional health authority

proved to be instrumental in partially addressing

some of these challenges.

The United States of America


Even with automated data collection, a large

commitment of time by participating hospital

staff was needed to collect the data. The U.S.A.

LTA found there was initial interest in the SOP;

however, this was not enough once the burden

on resource commitments became apparent.

Another complicating factor was confusion

of purpose. Within the U.S.A., quality

improvement efforts were already underway

to work on this problem. The Joint Commission’s

Center for Transforming Healthcare ran one

in Rhode Island and produced materials and

tool kits to combat the problem of wrong site

surgery.

For the second round of recruitment of hospitals,

the LTA tried to alleviate some of these issues.

A modest hospital stipend is now being provided.

In addition, the LTA is emphasizing the international

aspects of the programme, as well as use the first

hospital as a “mentor” hospital for the new

hospitals interested in joining the Project.

4.3. Summary of event analysis findings

Overall interim findings

Five of the seven participating LTAs self-report

that some in-country hospitals are completing

event analyses. A number of participating hospitals

and LTAs have used de-identified analysis reports

and findings as the basis for case studies for

teaching purposes.

A total of nine EA reports were submitted

by LTAs to the Collaborating Centre in 2012

that related to 125 patients. Reports for three

Concise and one Cluster analyses related to the

Medication Reconciliation SOP (two of the patients

reportedly experienced temporary harm and the

other nine did not experience harm). Reports for

one Concise and four Cluster analyses related to

the Correct Site Surgery SOP (none of the 114

patients experienced harm).

Contributing factors identified by the hospitals

included the following:

a. lack of effective oral/written communication,

including inaccurate or incomplete

documentation;

b. lack of effective teamwork;

c. individual provider/patient factors;

d. insufficient education and training;

e. lack of clarity related to policies and procedures.

It is anticipated that the number of event analysis

reports submitted to the Collaborating Centre

for analysis and reporting will increase during

the last phase of the Project.

a) Medication Reconciliation

A high level summary of the four event analysis

(EA) reports (total of 11 patients involved)

received between January 1st to December 31st,

2012 is included in Graphic A. See Annex 5B

for a complete summary of the EA submissions.


b) Correct Site Surgery

A high level summary of the five EA reports (total of 114 patients involved) received between January 1st and

December 31st, 2012 is included in Graphic B. See Annex 5A for a complete summary of the EA submissions.


Graphic A: Summary of Medication Reconciliation EA Reports Submitted to the CC in 2012

Graphic B: Summary of Correct Site Surgery EA Reports Submitted to the CC in 2012

a) Analysis

A small number of hospitals are reporting EA data

for both Medication Reconciliation (n = 3) and

Correct Site Surgery (n= 5) SOPs. Possible reasons

include limited capability and/or capacity for

identifying and analysing SOP-related events at

participating hospitals. The data collection burden

has been repeatedly identified as a challenge across

all participating Member States, so it is likely

that limited resources are being focused on

implementation rather than evaluation.

Nonetheless, it is clear from this limited number

of patients (n = 125) that process issues with SOP

implementation have occurred in all reporting

hospitals, with two patients experiencing temporary

harm (Medication Reconciliation SOP). Only one

hospital recommended making changes to the

applicable SOP (Correct Site Surgery) to ensure

the proper conduct of the Time Out, data

collection and data documentation. All hospitals

described local strategies to improve the

implementation of the SOPs through targeted

communication, policies and procedures, education,

and teamwork, among other factors.

EA data remains a potentially rich source

of information on the specific local challenges

involved in implementing a clinical SOP.

4.4. Summary of quantitative findings

Correct Site Surgery

Table 7 and Figure 1 present quantitative interim

data of CSS implementation. The line graphs

include two displays of High 5s Project data,

one by time period (i.e., monthly data points)

and the other by cohort (i.e., data points aligned

based upon the first instance of data submission

for each organization).

The line graphs with monthly data points in the

y-axis summarize hospital data submitted for each

individual month, independent of when the hospital

started the project. The country-level and

international comparison lines are calculated

as follows:

a) Country-level data (national comparison

groups)

Country-level data points for an individual measure

are calculated by aggregating all the hospital

numerator cases (from within a specific country)

and dividing this value by the sum of all of

the hospital denominator cases (from within

the country). The measure is calculated in the

aggregate for all hospitals in the country during

the specific time period. This calculation creates

a weighted mean (weighted by the number of cases

contributed by each hospital) rather than a grand

mean (simply taking the average of the calculated

hospital rates).

b) International-level data (international

comparison groups)

International-level data points are calculated

using a similar approach to the one employed

to calculate a country’s performance on a specific

measure. The major difference is that for rate

and ratio measures, all the hospital numerator

cases (from all countries) are summed and all of

the hospital denominator cases (from all countries)

are summed before calculating the measure rate

or ratio. The measure is then calculated in the

aggregate for all hospitals in contributing data

during the specific time period. This calculation

creates a weighted mean (weighted by the number

of cases contributed by each hospital) rather than

a grand mean (simply taking the average of

the calculated hospital rates or the average

of the national rates).

c) Confidence Intervals

The shaded areas around each line on the graph

represent a 95% confidence interval. The 95%

confidence interval indicates the estimated

probability (95%) that the population parameter

lies within the reported range of values (i.e.,

the degree of certainty that the true value lies

within a range of values). The confidence intervals

are calculated using a slightly different formula

for rate-based proportion measures and ratio

measures. The formula for each calculation is

provided below (p = the rate, r = the ratio, n =

the number of cases, SE = Standard Error).


The cohort graphs use the same formula to

calculate measure rates/ratios, but they aggregate

data based upon each hospital’s first data point

(rather than the month) as its first measureable

experience with SOP implementation. Every

subsequent monthly data point is aligned

accordingly. For example, if Hospital A submitted

its first data point in January of 2011, and Hospital

B submitted its first data point in July of 2012,

both points would aligned as the initial data

points for the hospitals.

Each subsequent monthly point was then aligned

based upon the time from initial SOP

implementation. This approach was used to track

time since implementation and account for gaps

in data collection. Continuing the example above,

if Hospital A submitted data for February,

March and June of 2011, the data points would be

assigned as month 2, month 3, and month 6

(as opposed to point 2, point 3 and point 4).

In this manner, the gap associated with the missing

data points (i.e., April and May) is addressed.

In order to aggregate hospital data within each

country (and across the project), each data point

was aggregated using the same approach,

beginning with the initial data point for each

hospital and calculating subsequent data points

(while correcting for gaps in data collection).

Note: For both graphs, it is important to consider

that the number of contributing hospitals could

change from month to month, based upon

the actual contributions of each hospital. The most

current three months represented in each graph

tend to contain fewer hospital contributions,

as many hospitals have a considerable lag time

in their data collection efforts. As a result,

the confidence intervals for later months tend

to be wider, and the rates more variable.


Table 7: General quantitative data on CSS

Member State Total No. Total No. of hospital submitted

hospitals data to High 5s IMS

implementing Average number of hospitals per month

SOP1,2 (07/2010-04/2013)

MUN8NS1Z 8 4

KEC43YH1 7 6

FDQ17XVY 1 1

RJ94137U 16 12

International Comparison 32 23

1 Implementing = Eligible to enter data in the High 5s Information Management System

2 Only data that has been approved by both the hospital and the LTA meets the criteria to be included in performance

measure calculations.

Figure 1: Interim quantitative data on CSS implementation

H5sCS-0 - Proportion of verification checklists for all eligible surgical cases

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(All

case

s sc

hed

ule

d t

o b

e per

form

ed)

H5sCS-0 – By cohort

Time period

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%


H5sCS-1 – Proportion of eligible surgical cases with a complete preoperative verification process

(exclusive of site marking and Time Out)


Time period

Time period

All Member States RJ94137U KEC43YH1 MUN8NS1Z

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(All

case

s sc

hed

ule

d t

o b

e per

form

ed)

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(All

case

s sc

hed

ule

d t

o b

e per

form

ed)

Jul 10 Oct 10 Ian 11 Apr 11 Jul 11 Oct 11 Jan 12 Apr 12 Jul 12 Oct 12 Jan 13 Apr 13

1

0.8

0.6

0.4

0.2

0

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%


H5sCS-2 – Proportion of cases with properly marked surgical site

Time period

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(eli

gib

le f

or

site

mark

ing

)Pr

opo

rtio

n o

f el

igib

le s

urg

ical

case

s

(eli

gib

le f

or

site

mark

ing

)


Time period



1

0.8

0.6

0.4

0.2

0

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%


H5sCS-3 – Proportion of cases with complete final ‘Time Out’

H5sCS-3 – By Cohort

Time period

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(eli

gib

le f

or

a T

ime

Out)

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(eli

gib

le f

or

a T

ime

Out)

Time period



1

0.8

0.6

0.4

0.2

0

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%


H5sCS-4 – Proportion of cases with discrepancy noted at final Time Out

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(eli

gib

le f

or

a T

ime

Out)

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(eli

gib

le f

or

a T

ime

Out)

Time period



1

0.8

0.6

0.4

0.2

0

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%


Time period


H5sCS-5 – Proportion of cases undergoing surgery with unresolved ‘Time Out’ discrepancies

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(Wit

h D

iscr

epancy

at

Final

Tim

e O

ut)

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(Wit

h D

iscr

epancy

at

Final

Tim

e O

ut)

Time period



1

0.8

0.6

0.4

0.2

0

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%


Time period


H5sCS-6 – The proportion of surgical cases that are cancelled or postponed due

to discrepancies identified at any point in the conduct of the SOP

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(eli

gib

le f

or

a T

ime

Out)

Time period



1

0.8

0.6

0.4

0.2

0

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(All

case

s sc

hed

ule

d t

o b

e per

form

ed)

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%


Time period


H5sCS-7 – Proportion of cases with incorrect surgery (wrong site, procedure or person cases)

Time period



Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(All

case

s sc

hed

ule

d t

o b

e per

form

ed)

1

0.8

0.6

0.4

0.2

0

Pro

po

rtio

n o

f el

igib

le s

urg

ical

case

s

(All

case

s sc

hed

ule

d t

o b

e per

form

ed)

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%


Time period

Medication Reconciliation

Figure 2 presents quantitative interim data of MR implementation. Please see the graph explanation in

section 5.4.1 under CSS to understand the differences between the line graphs and cohort displays.

Table 8: General quantitative data on Medication Reconciliation

Member State Total No. Total No. of hospital submitted

hospitals data to High 5s IMS

implementing Average number of hospitals per month

SOP1 (07/2010-04/2012)2

DVLR6PNK 14 6

OFI427W6 15 5

2K01AJF0 9 6

VES234TZ 7 0

BH4537QS 2 0

International Comparison 47 17

1 Implementing = Eligible to enter data in the High 5s Information Management System. One Member State has

recently started to implement the Med Rec SOP but is not yet depicted in this count.

2 Only data that has been approved by both the hospital and the LTA meets the criteria to be included in performance

measure calculations.

Figure 2: Interim quantitative data on Med Rec implementation


H5sMR-1 – Percent of patients with medications reconciled within 24 hours

of the decision to admit the patient

Perc

ent

of

Pati

ents

Time period

All Member States 2K01AJF0 OFI427W6 DVLR6PNK


100%

80%

60%

40%

20%

0%


H5sMR-2 – The mean number of outstanding undocumented intentional medication discrepancies per patient

H5sMR-1 – By cohort

Perc

ent

of

Pati

ents

Num

ber

of

Dis

crep

anci

es p

er P

ati

ent

Time period


3.5

3

2.5

2

1.5

1

0.5

0

120%

100%

80%

60%

40%

20%

0%

Time period




H5sMR-3 – The mean number of outstanding unintentional medication discrepancies per patient

Num

ber

of

Dis

crep

anci

es p

er P

ati

ent

Num

ber

of

Dis

crep

anci

es p

er P

ati

ent

Time period

2

1.8

1.6

1.4

1.2

1

0.8

0.6

0.4

0.2

0

Time period



2

1.5

1

0.5

0



H5sMR-4 – Percent of patients with at least one outstanding unintentional discrepancy

Num

ber

of

Dis

crep

anci

es p

er P

ati

ent

Perc

ent

of

Pati

ents

Time period

Time period



80

60

40

20

4.5. Member State quantitative and event

analysis findings

Australia

Following the evaluation plan has been a challenge

for all hospitals because of the burden of data

collection for the performance measures, data

quality requirements, and requirements for the

event analysis. Ongoing collection of performance

measure data (MR 2-4) is seen as not sustainable

by some sites. The value of continuing to monitor

measures MR 2-4 once the process was stable and

the target had been reached was also questioned.

One site withdrew from the project in late 2012,

citing the need to prioritize their work because

of resource constraints and preferring to provide

direct patient care over data collection.

Performance measures

Performance measure data has been collected since

mid-2010. As of November 2012, nine Australian

hospitals were submitting data into the High 5s

IMS. In addition, two hospitals in the group

are conducting medication reconciliation but

are not collecting data because of resource

constraints. One health service with multiple

hospitals has stopped collecting performance

measure data for MR 2-4 because of lack of staff

to conduct the independent observer role and meet

data quality requirements.

Data submission into the High 5s IMS has been

staggered, with hospitals starting data collection

at different time points. A number of hospitals are

now collecting MR 2-4 data on a quarterly basis,

with three sites now moving to 6-monthly data

collection. Rates can vary substantially when

the comparative data is produced because of

the number of hospitals contributing data in

a particular month. Sample sizes across the period

from August 2010 to November 2012 ranged

from 2 to 10 hospitals. See figures in Annex 10.

Rates for measure MR-1 vary across hospitals

depending on the stage of implementation,

resources available, and the spread of the

intervention in the organization. Between August

2010 and November 2012, rates for the percentage

of eligible patients with medications reconciled

within 24 hours of admission ranged from 16%

to 96% across participating hospitals, with an

average of around 51%. The trend line for MR-1

has been stable over this period. As medication

reconciliation remains largely pharmacist-driven,

there is a close association between MR-1 rates and

available clinical pharmacist resources. Hospitals

report that without additional clinical pharmacist

resources, MR-1 rates are not likely to improve.

Rates for MR-2 are generally stable and low

(less than 0.3) with a trend towards zero over time

for the majority of hospitals. Australian hospitals



Perc

ent

of

Pati

ents

100%

90%

80%

70%

60%

50%

40%

30%

20%

0%

Time period

report they have used this measure to provide

feedback to medication prescribers around the

importance of clearly documenting intended

medication changes. Similarly, rates for MR-3

are generally less than the SOP absolute target

of 0.3 outstanding unintentional discrepancies

per patient with a trend towards zero over time

for several hospitals. The percentage of patients

with at least one outstanding discrepancy (MR-4),

also continues to decrease with time; however,

there is variation in MR-4 rates across hospitals

which is highlighted in the figures in Annex 10.

Event analysis

The collection of performance measure data has

overshadowed the event analysis (EA) component

of the evaluation with few sites reporting EAs.

Twelve event analysis reports have been received

from four hospitals, including 11 individual reports

and one cluster analysis. All individual reports were

concise analyses, with an assigned level of harm

ranging from temporary harm to no harm. There

have been no serious harm events reported.

In a sub-analysis of seven events, the most

common contributing factors were identified

as teamwork, education and training, and

communication. There were no requests for changes

to the SOP.

All events were the result of a failure to follow

the SOP medication reconciliation process. Most

events occurred during the BPMH stage (five) and

when reconciling the medicines (four); only one

report related to the resolution of a discrepancy.

Individual hospitals have used the event analysis

reports as the basis for case studies for teaching

purposes. Actions resulting from the reports have

resulted in changes to the medication reconciliation

form (one hospital), and regular orientation

sessions for junior medical staff on the medication

reconciliation form and process. Learning from

the events has been shared among the hospitals

participating in the High 5s Project.


Other

Two additional performance measures were

developed to allow hospitals to collect data on

the discrepancies identified by the independent

observer that remained unresolved 48 hours after

admission. By late 2012, five hospitals were

collecting data on the number of these outstanding

unresolved discrepancies. In addition, three of these

hospitals measured the potential for harm from

the discrepancies using the National Coordinating

Council for Medication Error Reporting and

Prevention (NCC MERP) severity scale to classify

potential level of harm. The most common

discrepancies were classified as NCC MERP

Category D - an error occurred that reached

the patient and required monitoring to confirm

that it resulted in no harm to the patient and/or

required intervention to preclude harm. This data,

along with examples of actual discrepancies,

has been used by sites to provide feedback

to prescribers and encourage early resolution

of discrepancies. (See Figure 3.)

Box 12: Learning from error

Hospital 5: An event analysis was conducted on a patient prescribed the incorrect medicines on

admission to hospital. These medications were administered to the patient for a week until a pharmacist

completed a BPMH for the patient and the errors were noted. All of the health-care professionals

involved were informed of the errors, and this has been used as a teaching example to prevent this from

happening again. This case was also presented at facility quality and safety meetings.

France

Measures for Correct Site Surgery

Regarding the first months of evaluation, and

based on the CSS performance measures, it appears

that nearly all the hospitals made progress. The

most striking improvement involves site-marking.

Number of CL initiated / eligible cases

During 2012, the measure CS-0 national average

is 84% (Min = 55%, Max = 100%). Two hospitals

with low CS-0 rates are facing persisting difficulties

in making progress.

Pre-op verification CS-1

Significant variability has been observed among

the different health-care organizations (HCOs)

because of the reluctance of some professionals

(i.e., surgeons, anesthetists) to conform to

the process requirements. However, HCOs with

an electronic checklist integrated in the data

records have achieved good results, as traceability

is less time-consuming and the acceptability of

this preoperative component is better. In 2012,

the French national average for measure CS-1

was 29% (Min = 2%, Max = 69%).

Site-marking

All of the hospitals except one improved their

site-marking results. The mean value for measure

CS-2 was 62% in 2012 (Min=26%, Max=87%).

The hospital which did not show improvements

demonstrated poor Project leadership for the,

inappropriate site markings, such as marking

the site with a cross on the palm of the hand,

only partial implementation, and poor feedback

to frontline staff.

Time Outs

The mean value for measure CS-3 was 66%

(Min=32%, Max=89%). Discrepancies in the OR

were mainly because of a lack of traceability.

In addition evaluation of the Time Out by

observational audits started in June 2012.

Up to the present, the LTA has conducted four

observational audits in the operating rooms

of hospitals (audit grid designed by CEPPRAL).

Event analysis

Since the end of 2011, six hospitals have provided

analysis forms to the LTA. Only two were correctly

analyzed and therefore eligible for posting on the

High 5s Project website. Event analysis was difficult

because it was time consuming and there were

insufficient resources to support this activity.


Figure 3:

Mean number of unresolved medication discrepancies/patient > 48 hours after admission

Interviews with participating hospitals

Since January 2012, nine annual visits which

include interviews (individual and focus group)

have been performed by HAS and CEPPRAL.

About 45 interviews have been conducted thus

far and have involved 90 professionals (front-line

staff and managers, physicians, surgeons, war

nurses, secretaries, and anaesthetists). These

interviews have been invaluable in communicating

with the teams and HCOs as well as recognizing

the efforts they have achieved.

Measures for Medication Reconciliation

During the first months of evaluation of

medication reconciliation practice in France using

the High 5s performance measures, it appears

that all of the hospitals progressed as much

in the number of reconciled patients as in the

identification of discrepancies.

In France, the principal data used by the LTA was

measure MR-1 because it highlights the level of

medication reconciliation implementation for each

hospital. Indeed, from the beginning of the Project,

the other measure results were very low, especially

because the number of eligible patients reconciled

was not a priority. It was felt to be more relevant

to focus on measure MR-1 and on the number

of unintentional intercepted discrepancies as it

provides telling information for health-care

professionals, the CEOs and the nation as a whole.

For example, the medication reconciliation pilot

hospital in France has, over a period of 18 months,

succeeded in intercepting 1.664 unintentional

discrepancies thanks to the medication reconciliation

process. This result has a lot of teaching impact

and allows the promotion of this practice with

other professionals.

In France, the number of reconciled patients

initially varied from one hospital to another, as

shown by MR-1 rates which ranged between 2.5%

and 63.4%. This heterogeneity is explained by

the fact that two of the hospitals had a very low

number of eligible patients because of the hospital

size. However, over the first 13 months of

implementation, the number of patients who had

received formal medication reconciliation increased

for all of the hospitals. From August 2011 to May

2012, the aggregated MR-1 results increased

from 19.6% to 30.2%. In parallel, the number

of discrepancies decreased from 321 to 159

over the same period, which evidences better

understanding of the protocol by the teams as

well as an improvement in their daily practices.

Since the Med Rec teams do not encounter many

unintentional discrepancies, it is difficult to involve

them in event analysis. The Med Rec teams have

been more interested in focusing on the analysis

of the intercepted discrepancies and sharing

the experience gathered by stopping them.

Germany

Measures for Correct Site Surgery

German hospitals collect data for the performance

measures on carbon copies of the surgical checklist

and send one copy to the LTA. To date, over

115.000 checklists from 16 hospitals have been

scanned and imported into the national database.

In addition to the High 5s measures data,

key national data can also be analyzed (e.g.

documentation quality, completeness of checklists,

subgroup analysis of emergency cases) because

of the availability of the complete checklist data.

The German LTA has established a national feedback

system consisting of individual hospital quarterly

reports and bi-annual national benchmark reports

which is much appreciated by the participants.

The German LTA conducted an analysis of the High

5s measures results for those checklists which

were evaluated in the year 2012 (n=61.684).

Four hospitals consistently performed at a high

level for the measures CS-0 through CS-3. Two

of these four hospitals were also part of the top

25% for measure CS-4. These hospitals, however,

ranged completely differently for measures CS-5

and CS-6. The German LTA identified two hospitals

which continuously ranked in the lower 25% for

all of the first four performance measures. These

hospitals also ranked in the lower 33% for measure

CS-5. Both had presented a very low rate of case

cancellations because of discrepancies (for CS-6,

0.00% and 0.01%). Two years after starting SOP

CSS implementation, hospitals have reached

consistent levels of performance on the measures.

From review of checklists, it is evident that with

regard to the first four measures, Project hospitals

are divided into groups of high and low performers.

For the measures dealing with discrepancies (CS-4

through CS-6), high and low performers cannot

clearly be identified.

Measures for Medication Reconciliation

To date, the hospital which has officially started

with the Med Rec SOP has calculated its

performance measures for the months of March


through October 2012. In this hospital, since

the beginning of implementation, Med Rec has been

conducted on an average of 36 patients per month.

The number of documentation errors has decreased

slightly. The number of medication errors

(unintentional discrepancies) has been reduced

by 78%. The proportion of patients with

at least one medication error has been reduced

by 69%.

Event analysis

CSS: The German LTA has learned that Event

Analyses have been conducted in some of

the hospitals, but the results have not yet been

shared with the LTA or the CC.

Other

By way of a baseline survey which the German

LTA conducted, it was found that hospitals

which had already established similar processes

experienced a much smoother transition compared

to those hospitals for which the SOP was a new

process. This is also reflected in the process

measure results.

The Netherlands

Process measures

All of the fifteen participating hospitals established

baseline performance levels before implementing

the SOP. Hospitals usually measured their

performance for at least three months after

implementation to show sustainability in the

reduction of medication discrepancies. Hospitals

would gather data for measures MR-1 through

MR-4 until achieving at least a 75% reduction of

discrepancies. Thereafter, most of the hospitals

only measured MR-1 monthly and measures MR-2

through MR-4 occasionally (e.g., every six months).

Two hospitals targeted almost 100% of their

patients for measure MR-1. Hospitals generally felt

no need to measure monthly only for the specific

High 5s target population (patients 65 years

and older admitted through ER) after the SOP

performance had demonstrated sustainability.

One hospital pharmacist said that, ‘As a hospital

pharmacy, we do see every adolescent clinical

patient within 24 hours. Probably now and then

a patient will slip through the process, but

we achieve 96-100% monthly. There’s no need

to keep on measuring MR-1 monthly, as the

outcome will remain the same.’

Hospitals spread the SOP implementation to all

age groups as this made implementation in daily

practice easier. Some hospitals started

implementation step-by-step, with a few categories

of patients first. For each hospital, the impact

of the SOP was all about the qualitative context

of SOP implementation underlying the quantitative

measures. An academic hospital highlighted that,

‘When we have no performance measurements

in our hospital it does not mean that our

reconciliation process is not of good quality.

Measurements to monitor the process are also

for the national patient safety programme. During

the upcoming months, we will have internal

sessions to provide feedback about how to create

an effective dashboard to monitor reconciliation

and to sustain the reconciliation process.’

Highlights of event analysis

The Dutch hospitals contributed to the development

of the event analysis methodology and

materials. Some hospitals piloted the draft event

analysis process in the spring of 2011, which

showed that the triggers identified in the medical

records did not appear frequently enough to be

used to support the event analysis process.

In 2011, the event analysis methodology was

modified and the new triggers and forms became

available in 2012. One hospital in the Netherlands

finished an event analysis and identified events

potentially related to the SOP. All required forms

were analyzed and reported to the LTA. This

hospital experienced resource-based challenges/

barriers, including the lack of staff to identify

events for analysis, difficulty defining an

independent observer role, and lack of staff and/or

staff time to perform event analysis after event

was identified. Action plans for improvement were

formulated after completing the event analysis,

and the hospital learned important information

about the SOP implementation processes on local

wards through carrying out the event analysis.

A second hospital completed an event analysis

at the end of 2012 and also identified events

potentially related to the SOP.


Singapore

Measurement of improvement

The effects of the planned changes were measured

by gathering compliance data collected according

the following High 5s measures:

ñ CS-1. Completed Preoperative Verification Check

List;

ñ CS-2. Properly Marked Surgical Site;

ñ CS-3. Complete Final ‘Time Out’;

ñ CS-4. Cases with Discrepancy Noted at Final

‘Time Out’;

ñ CS-5. Cases Undergoing Surgery with Unresolved

‘Time Out’ Discrepancies;

ñ CS-6. Case Cancellation Resulting From SOP

Implementation;

ñ CS-7. Incorrect Surgery (wrong site, procedure

or person cases).

The main points to highlight from the Singapore

hospitals’ performance versus other Member States

are that:

ñ Singapore’s national average consistently showed

higher compliance rates than the international

benchmark.

ñ Though hospitals have not all achieved 100%

compliance for measures CS-1 through CS-3, their

current rates have improved to a plateau at >90%.

ñ Measure CS-5 result suggested that Singapore

hospitals tend to let discrepancies pass

unresolved. However, on further investigation

the discrepancies were found to be mostly related

to documentation errors, such as ticking ‘No’

when it should be ‘NA’ and not because of

non-compliance with the SOP. The High 5s

Network is working within its own hospitals

to close the gaps for these discrepancies.

One of the ways they have chosen to do this

is through events analyses which they are just

starting to do now that the data collection

and submission processes have been stabilized.

ñ Measure results for CS-6 and CS-7 have

consistently been 0%.

Trinidad & Tobago

Implementation of the CSS SOP took place at four

participating hospitals. Despite the challenges

experienced, staff in the orthopaedic theatres were

still willing to advance the SOP to other theatres.

Figure 4 presents measure results in participating

hospitals in Trinidad and Tobago.


CS-0 Eligible Surgical Cases

YES NO NA

TRHA 105 0 0

SWRHA 100 0 0

NCRHA 100 0 0

ERHA 62 38 0

Figure 4: Measures in participating hospitals

CS-0 Eligible Surgical Cases

TRHA

110

100

90

80

70

60

50

40

30

20

10

0

YES NO NA

Pati

ent

%

SWRHA

NCRHA

ERHA


CS-1 Completed Preoperative Verification Checklist

CS-1 Completed Preoperative

Verification Checklist

YES NO NA

TRHA 85 14 1

SWRHA 82 18 0

NCRHA 80 20 0

ERHA 90 10 0

TRHA

100

90

80

70

60

50

40

30

20

10

0

YES NO NA

CS-2 Properly Marked

Surgical Site

YES NO NA

TRHA 23 70 8

SWRHA 66 34 0

NCRHA 60 40 0

ERHA 100 0 0

CS-3 Complete Final Time Out

CS-3

Complete Final Time Out

YES NO NA

TRHA 61 38 1

SWRHA 67 33 0

NCRHA 60 40 0

ERHA 95 5 0

CS-2 Properly Marked Surgical Site

Pati

ent

%

100

90

80

70

60

50

40

30

20

10

0

Pati

ent

%

SWRHA

NCRHA

ERHA

TRHA

100

90

80

70

60

50

40

30

20

10

0

YES NO NA

Pati

ent

%

SWRHA

NCRHA

ERHA

TRHA

SWRHA

NCRHA

ERHA

YES NO NA


CS-4 Cases with Discrepancies

noted at final time-out

YES NO NA

TRHA 0 100 0

SWRHA 21 79 0

NCRHA 2 98 0

ERHA 5 95 0

CS-5 Surgery with

Unresolved Discrepancies

YES NO NA

TRHA 0 100 0

SWRHA 5 95 0

NCRHA 0 100 0

ERHA 0 0 0

CS-6 Case cancellation resulting

from SOP Implementation

YES NO NA

TRHA 1 99 0

SWRHA 0 0 0

NCRHA 0 100 0

ERHA 0 0 0

TRHA

100

90

80

70

60

50

40

30

20

10

0

YES NO NA

CS-5 Surgery with Unresolved Discrepancies

Pati

ent

%

SWRHA

NCRHA

ERHA

CS-4 Cases with Discrepancies noted at final time-out

TRHA

100

90

80

70

60

50

40

30

20

10

0

YES NO NA

Pati

ent

%

SWRHA

NCRHA

ERHA

CS-6 Case cancellation resulting

from SOP Implementation

0

0100

90

80

70

60

50

40

30

20

10

0

Pati

ent

%

TRHA

SWRHA

NCRHA

ERHA

YES NO NA


CS-7 Incorrect Surgery

YES NO NA

TRHA 0 100 0

SWRHA 0 0 0

NCRHA 0 100 0

ERHA 0 0 0

CS-7 Incorrect Surgery

100

90

80

70

60

50

40

30

20

10

0

Pati

ent

%

TRHA

SWRHA

NCRHA

ERHA

YES NO NA

1 Australian Commission on Safety and Quality in Health Care. National Safety and Quality Health Service Standards.

Sydney: Australian Commission on Safety and Quality in Health Care, 2011.

2 Australian Pharmaceutical Advisory Council. Guiding principles to achieve continuity in medication management.

Canberra: Commonwealth of Australia, 2005.

3 Society of Hospital Pharmacists of Australia. SHPA Standards of practice for the provision of medication

reconciliation. Journal of Pharmacy Practice and Research 2007;37(3):231-3.

6Interimoutcomes

1. Feasibility

Feasibility refers to the degree to which an SOP

could be implemented as it was originally defined,

in a standardized way across hospitals within

a country or in multiple countries.

1.1. Overall issues

One of the primary objectives of the High 5s

Project has been to assess the feasibility of


patient safety problems across multiple countries

and cultures. The rigorous, multidimensional High

5s evaluation strategy is providing a qualified “yes”

to the first of the key questions for the Project:

is process standardization across different hospitals

in different Member States feasible? Over the

course of the High 5s Project it became clear

that some aspects of the SOPs were more easily

implemented in some participating Member States

than in others. When participants encountered

issues with the implementation of a SOP, they

were invited to submit requests to adapt or modify

components of the SOPs that were problematic

or not working well. These requests were reviewed

by the Collaborating Centre and the High 5s Steering

Group and a decision was made to approve or

reject the modification request. To date, 15 SOP

modification requests have been submitted–two

for Medication Reconciliation, 11 for Correct Site

Surgery, and two for Concentrated Injectables.

Each participating Member State submitted at least

one request for modification. However, while there

have been several country-specific adaptations

to the SOP, the majority of participating hospitals

have achieved full implementation of the protocol.

That is, hospitals implemented all steps of the

protocol in all locations and for the entire eligible

population as specified in the SOP.

In some cases, modification requests led to

revisions of the SOP. These revisions included

changes to key definitions which helped to clarify

the intent of a process or a concept across the

varied languages and cultures of participating

Member States. In other cases, revision requests

identified opportunities for flexibility or local

customization of the SOP. Such requests required

the High 5s Steering Group to reconsider

which elements of the protocol needed to be

“standardized” and which aspects should be left

to the discretion of the hospital. For example, the

Correct Site Surgery protocol originally required

that site marking be performed directly by the

surgeon who was performing the procedure.

The SOP was revised to allow the responsibility

for site-marking to be delegated to another health

care professional, as long as that individual was

qualified, identified in the patient record, and

directly involved in the procedure. In other words,

the SOP continued to require a standardized

site-marking process, but the individual hospitals

could implement it in a way that fit best into

its existing processes.

Modifications to the SOPs are presented in

additional detail below:

Medication Reconciliation

Canada requested that the definition of ‘Best

Possible Medication History’ be revised to include

an expanded definition of ‘medication’ and

to define what is meant by ‘currently taking’

a medication. The requested revisions were

approved based on the assumption that they

provided clarity and a more uniform understanding

of the SOP across participating Member States.

The Netherlands requested to reduce the number

of times the patient/family is consulted to obtain

the patient’s current and past medication history.

The request was that the patient/family be consulted

only one time (instead of two) in order to more

effectively and efficiently utilize staff time.


Interim outcomes

The request was approved, and the SOP was revised

to allow for a more generalized model that allows

more freedom and flexibility for hospitals to define

their own process for obtaining the patient’s best

possible medication history. This change helped

to improve understanding of the SOP, reduce

redundancy, and create a more efficient work flow.

Concentrated Injectables (CIs)

The United Kingdom requested a proactive reduction of

authorized ward storage of concentrated injectables.

The requested language change focused on the

reduction of the number of clinical areas authorized

to store CIs and, where authorized, put limitations on

the types, amounts, concentrations and preparations

of CI drugs stored in those areas (outside of the

pharmacy). These language changes were approved,

and the SOP was modified accordingly.

The United Kingdom also requested that information

be added to the SOP that would help clarify the

intended population for the SOP. This request was

approved, and the wording was amended to read,

“This SOP applies to inpatient clinical areas only.”

Correct Site Surgery

Singapore, Germany and France all asked that

the procedure for site-marking be revised to allow

for a delegated health-care professional (other than

the operating surgeon) to mark the correct surgical

site. The request was approved, and the language

in the SOP was changed to reflect that:

1) another physician or registered nurse

participating in the procedure or directly

involved in patient surgical preparation can

mark the site;

2) hospital policy must include the minimum

qualifications and the role of the person

delegated to perform site-marking;

3) the individual performing the site-marking

is identified in the medical record or

pre-operative verification checklist.

Singapore also requested a waiver of the verification

of site and procedure during pre-operative testing

since surgical site and procedure are not usually

verified during this time. This request was approved,

and the pre-operative verification checklist was

revised accordingly.

In a few cases, requests for modifications to

the SOP did not lead to revision of the SOP,

but rather resulted in a country-wide adaptation

of the SOP. These requests generally involved

a conflict between an individual Member State’s

existing standard, and a critical component of

the SOP. In these cases, the Steering Group allowed

an individual Member State to adapt

the SOP to maintain consistency with the existing

standard. There were two such instances, both

involving the CSS SOP and standard practices

in Germany.

In order to avoid the possibility of misinterpretation,

the SOP prohibits the use of abbreviations in

surgical documentation. However, because there

are many established and commonly understood

abbreviations currently used in Germany, as well

as many long words in the German language that

look similar, Germany requested permission to use

abbreviations in the electronic operating room

(OR) log. In participating German hospitals,

these established abbreviations help distinguish

words and also require less space in the OR log.

A decision was made to make a country-specific

adaptation rather than a revision to the SOP that

would allow participating German hospitals to use

abbreviations in electronic (only) OR logs.

The SOP also specifies a number of approaches

to site-marking, including a prohibition on using “X”

to mark a site. In many cases, “X” can be variously

interpreted as a positive (i.e., “‘X’ marks the spot”)

or negative indication (i.e., “Do NOT…”), which

creates the potential for confusion and error.

In Germany, however, the “X” has been

recommended as a standard marking for the

correct/intended site for surgery by the German

Coalition for Patient Safety. This adaption was

approved for participating German hospitals

to use “X” for the intended surgical site marking.

Rejected requests

A number of additional requests were also

considered but rejected by the Steering Group.

These requests all related to the CSS SOP;

two related to site-marking and one related

to patient positioning.

Germany asked to use only the intraoperative

radiographic technique, not skin marking to mark

the surgical site for spinal procedures. Because both

skin marking and intraoperative marking are required

as part of the SOP, this request was rejected.

Singapore asked to exempt obstetric and

gynecological procedures from site marking

91 Interim outcomes

because of the fact that the many ob/gyn

procedures require exploratory surgery to determine

the correct site for the surgery. This request

was rejected based on the fact that the expected

or anticipated surgical site should be marked in

all surgical cases, regardless of specialty area.

Singapore also requested to waive the patient

position verification during the Time Out because

of the fact that the patient would be checked

during prepping and draping and it is unlikely

that the patient position would be inaccurate

by the time the time-out occurs. This request

was rejected because the verification of the

patient position is part of a minimum set of items

selected to be included in the final time-out.

This list was determined by expert consensus

to include the most measures for avoiding

wrong-site surgery.

1.2. Summary of in-country experiences

Australia

By December 2012, all 12 hospitals in the

collaborative had implemented the medication

reconciliation process. Nine of these hospitals

were submitting data into the High 5s IMS.

Only one hospital was able to perform medication

reconciliation on over 80% of its eligible patients

within 24 hours of admission. This hospital

had a seven-day-a-week pharmacy service with

extended hours in the emergency department (ED).

One hospital implemented a multidisciplinary

process with doctors and nurses trained in taking

a BPMH and reconciling medicines. In the other

hospitals, pharmacy staff, mainly pharmacists,

were responsible for taking the BPMH and

reconciling medicines for admissions through

the ED. Apart from the lack of engagement

by medical and nursing staff in performing

medication reconciliation the actual steps

of the medication reconciliation process were

implemented without change. Hospitals used

a mix of paper and electronic tools to document

the BPMH and record medicines reconciled.

However, not all hospitals followed all

components of the implementation strategy.

Hospitals with a medication reconciliation

process already in place did not run a pilot and

those hospitals that had medication reconciliation

already in place or that lacked expertise in

the failure modes and effects analysis (FMEA)

process did not complete the risk assessment.

Australia requested some modifications to the

performance measures. They affected all hospitals.

See table 9 for the modifications sought.


Table 9: Modifications to SOP sought by Australia

Performance Issues Requested change Outcomemeasure

MR-1 Hospitals could not Use sampling A sample of 50

collect data on all to collect MR-1 was approved*

admissions of eligible

patients

MR 2-4 Monthly data Collect 3 monthly 3 monthly data collection approved

collection or 6 monthly after after meeting target for 12 months,

unsustainable consistently meeting 2011* 6 monthly collection

target of < 0.3 outstanding approved after 12 months of quarterly

discrepancies/patient reports meeting target, 2012*

MR-1 Difficult to perform Extend timeframe Not approved.

Med Rec within for Med Rec Australia will collect own data

24 hours to < 48 hours on Med Rec completed within

48 hours in addition to MR 1-4.

MR 2- 4 Definitions of Definitions be more descriptive Table of discrepancy examples

discrepancies were and include inclusions/ created and added to Volume 4

unclear and there exclusions and provide examples, – ‘Getting Started’ kit. A local

was concern about particularly in relation to PRN “How to guide” was also developed.

inter-rater reliability and complementary, medicines

* Approved by High 5s Steering Group

In addition to the evaluation plan another

challenging aspect of the SOP implementation

was changing the organizational culture from one

where medication reconciliation was regarded as

being only “pharmacy business” to one involving

medical and nursing staff in the process (Box 13)

93 Interim outcomes

Despite the difficulties engaging disciplines other than pharmacy in the medication reconciliation process

sites reported that the initiative was accepted and valued by other staff (Box 14).

Sustainability

Sites with weekend pharmacy services reported

the SOP was sustainable.

The five sites interviewed suggested sustainability

of the SOP was dependent on a number of factors

including:

ñ reducing the requirement for hospital staff

to provide ongoing education and including

training within university curricula;

ñ availability of electronic tools to support

the medication reconciliation process that

are integrated with electronic-prescribing/

medication management systems;

ñ continued engagement and support from

stakeholders;

ñ extending pharmacy services to 7 days a week;

and

ñ provision of ongoing/additional funding to

continue collection of performance measure

data.

In summary

It was feasible to implement the medication

reconciliation SOP in Australian hospitals. It is

a complex process, and successful implementation

requires support from the hospital executive

and senior clinicians, adequate resources and

a commitment to on-going training.

Box 13: A quote on sharing ownership of medication reconciliation

Hospital 1. ‘…another challenge we have faced is in engaging staff to share ownership of medication

reconciliation. While we have tried to increase awareness of medication reconciliation with posters,

competitions, staff education, etc., pharmacists still have ownership of the process, and are the only

ones formally doing and documenting medication reconciliation’.

Box 14: A quote on the value of medication reconciliation

Hospital 2: The implementation of the High 5s Project in the Emergency Department has seen

increased cohesion among ED nursing staff, ward nursing staff and pharmacy staff with the initiative

being welcomed by the ED staff.

France

Experiences implementing the Correct Site

Surgery SOP

This SOP was fully implemented in four out of eight

participating hospitals, and among these four

hospitals at full implementation, three have a

computerized High 5s checklist which is extremely

useful.

ñ The most common strategy used to facilitate

implementation was initially to begin with one

or two surgical specialties for implementation.

Three of the four hospitals, having chosen this

option, encountered two kinds of difficulties:

existence of multiple verification tools and no

sense of urgency for change.

ñ All participating French hospitals except one

(having no out-patient surgery in the in-patient

area) adapted the SOP to permit the delegation

of site-marking to nurses, after training by the

surgeons. This adaptation was necessary because

of the high number of out-patients having surgery

in in-patient ORs (was allowed by the SOP) or

of in-patients who are admitted on the morning

of their surgical interventions.

ñ The most challenging aspects of the SOP were:

Data collection for all the hospitals except

for one who integrated the High 5s checklist

into its electronic record from the beginning.

The workload was heavy, and no resources were

dedicated to support this effort most of the time.

Currently, five of the eight hospitals are working

on integrating the High 5s checklist into their

electronic record projects.

Site-marking faced resistance from surgeons in

six out of the eight hospitals. Working together

with the LTA to create a High 5s site-marking

guide was helpful. Each hospital has developed

procedures for site-marking in accordance

with the requirements of the SOP. Site-marking,

as a new practice in France not included in

the WHO Surgical Safety Checklist promoted

by HAS and mandated in the accreditation

process since 2010, has raised many

organizational, cultural, and technical questions,

as well as debates on the legality of a surgeon

delegating the marking responsibility to a nurse.

Lack of involvement of anesthesiologists in

the pre-op verification process was a common

challenge.


Some lessons learned

ñ The importance of contextual conditions, such

as team instability and reduced cohesion, lack

of leadership, and physician resistance.

ñ The critical role of an organization approach

to safety and the emergence of credible opinion

leaders who are seen as influential in effecting

changes in clinical practices.

ñ The need to include surgeons who are willing

to be champions.

ñ The importance of achieving full implementation

within a reasonable timeline in order to impact

a sufficient number of health-care professionals.

Experiences implementing the Medication

Reconciliation SOP

At the beginning of the project and after having

solved the first difficulties related to understanding

the protocol, the teams involved in medication

reconciliation started the implementation

by following the protocol and respecting all

of the eligibility criteria. However, the teams

were quickly confronted with barriers linked

to the organization of care. The main difficulties

related to:

Box 15: Interview findings in two French hospitals

Nurses working in a surgical ward: ‘At the beginning of the project, site marking looked like

an additional task and responsibility. I was against the delegation of this task by surgeons, but then

I understood that we are one of the links in the chain.’

Project coordinator: ‘Marking was intended for all patients admitted to a surgical unit starting April

2012. However, the majority of patients are admitted on the same day or for ambulatory surgery;

very few are admitted the day before. The marking procedure was put in place. The task was delegated

to a resident and then a nurse, under the surgeon’s responsibility. In approximately 70% of cases,

the task was delegated to a nurse. The borderline between what can be delegated and what cannot

was not always clear to the teams.’

ñ The expectation of formal medication

reconciliation within 24 hours after patient

admission when most organizations strive

to meet a 48-hour timeline which itself

was felt to be challenging.

– Currently, three hospitals are monitoring

Med Rec with a measurement of reconciliation

at 48 hours and at 24 hours.

ñ Questions regarding the most convenient place

to reconcile the patients; should this be on

the medical ward or directly in the emergency

department:

– Regarding the place where medication

reconciliation has to be performed, the teams

had to define whether it was more convenient

in their own organization to reconcile directly

in the emergency department or on the medical

ward. Currently, all except one hospital are

conducting medication reconciliation on medical

wards. This hospital center was particularly

successful in reconciling all of its patients

within 24 hours.

ñ The difficulty in extracting the data to satisfy

the Project’s performance measures:

– Initially, the information systems did not allow

extraction of the data needed for the

performance measures. Currently, teams involved

with the Med Rec SOP are using an Excel file

which was elaborated by the LTA. In addition,

some hospitals are struggling to access the

measure data for the Project measures where

medication reconciliation is fully integrated

into pharmaceutical care and is not easily

identifiable as such in the care process. This

barrier explains why a hospital does not presently

send any data because the design of its process

does not allow it to extract exhaustive data

on the number of reconciled patients within

48 hours (see Box 16).

95 Interim outcomes

Box 16: Example on how to overcome the data collection burden in Med Rec

In order to facilitate the data collection that allows monitoring of the medication reconciliation

practice, a general hospital, in collaboration with the LTA, is currently working on integration of

the medication reconciliation sheet into the information systems. As the computerized physician order

entry (CPOE) system does not support the iterative steps in the Med Rec SOP, the hospital developed

a tailor-made application directly connected to the inpatient electronic medical record. This tool can be

accessed from a reconciliation tab on the medical record screen.

Med Rec data can then be displayed across multiple medical episodes. Physicians and pharmacists can

consult or change the data at any time, and a historical record of changes can be made available.

Data indicators support automated calculation through a supervisor tool. To increase visibility for

the clinician, a summary of discrepancies, warnings and additional information can be made available

when the software is opened. Patient data can be extended to see more specific data (patient ID,

compliance, sources, and author). Med Rec screens, available for admission or discharge, display only

the most useful information in order to increase visibility and allow quick checking. A warning check

box can be manually enabled to display warning logos (e.g., no compliance warning), and free text can

be entered to describe which drug is available. Additional information about drugs stopped before

hospitalization can also be entered in a free text box at the bottom of the screen.

Medication reconciliation performed outside

Med Rec criteria

Currently, French hospitals are performing

medication reconciliation beyond the eligibility

criteria of the Med Rec SOP. Therefore, those

patients not fulfilling Med Rec criteria who are

reconciled are not included in the High 5s Project

database.

For example, several hospitals have started

to spread Med Rec to other medical units.

In the different units, all patients are now eligible

for medication reconciliation- not simply patients

65 years and older entering through the Emergency

Department. From the point of view of the

hospitals teams, it has helped having a better

integration of the medication reconciliation process

into daily medical practices. The pharmaceutical

teams in three hospitals have also started

to implement medication reconciliation upon

discharge.

Germany

To what degree was the SOP implemented

in all hospitals as it was originally defined?

A total of 14 of the 16 hospitals implementing

the CSS SOP have reached full implementation.

Of the two hospitals that have not reached full

implementation status, one has implemented the

SOP in all but one department (Ophthalmology).

The other hospital is the only JCI-accredited

hospital of the 16 hospitals, and it has not

implemented the High 5s checklist evaluation

in all departments. In one hospital which started

with Med Rec implementation, the SOP has been

partially implemented, but it is unlikely that full

implementation will be achieved in the course of

the project because of a lack of human resources.

It is difficult to reach overall conclusions regarding

the feasibility of the SOPs in the German context

because of the likely selection bias which exists

among the German hospitals who decided to

implement the High 5s SOPs. These hospitals

participate in the High 5s Project on a voluntary

basis and thus represent a more dedicated and

experienced group of hospitals than their

counterparts.

How did in-country hospitals modify the SOP

to facilitate implementation?

a) Correct Site Surgery

The adaptation of the international patient safety

tools to the German hospital context involved a

close collaboration between hospitals and the LTA.

The surgical checklist is used in Germany for both

implementation of the SOP and collection of data

for evaluation purposes. After translation and

reduction of the complexity and text density (e.g.,

by creating an instruction leaflet) a national model

version of the checklist was created that contained

the minimum criteria needed to adhere to the High

5s standardization requirements. Therefore,

a process step that is not part of routine clinical

care in German hospitals was removed from

the checklist, and two adaptation requests from

German hospitals relating to the site-marking

process were accepted by the High 5s Steering

Group: (1) It is now permissible to mark the site

with an “X” (adaptation on a national level), and

(2) medical professionals other than surgeons

are allowed to mark the site as well.

As a further result of the joint hospital workshops

it also became clear that it was necessary to

incorporate the checklist into already existing

preoperative processes in the participating

hospitals. Furthermore, more than half of

the hospitals were using surgical checklists when

the project started and did not want to start using

a completely new checklist. Thus 11 different

hospital-individualized checklists were developed

that – depending on hospital needs – contain

additional elements while maintaining High 5s

standardization.

The hospital-individualized checklists differ from

each other with regard to:

ñ comprehensiveness of depicted processes (e.g.,

in some checklists additional items from the

WHO Surgical Safety Checklist are included);

ñ level of detail of the instructions;

ñ design/structure/grouping of the items.

Furthermore, in relation to the use of the High 5s

checklist as an evaluation tool, the hospital

coordinators required the German LTA to remove

the section “Outcomes” from the checklist because

of legal concerns (e.g., documentation whether

an incorrect or potentially incorrect surgery was

identified, information on the degree of harm).

This is the reason why the German LTA does not

gather data for measure CS-7.See also Annex 16.

b) For Medication Reconciliation

Because of the fact that Med Rec is a new process

in Germany, some hospitals did not wish to start

with the “more difficult” population of the older

(65+) emergency patients but rather wanted to

begin with patients who have a planned or elective

admission. The German LTA offered to conduct

data analysis and produce data feedback reports

on these additional cases to the hospital, while

submitting only the data for the internationally

targeted population to the Collaborating Centre.

On the other hand, German hospitals, especially

the large university medical centers, had great

interest in collecting more data than the required

High 5s performance measures. Therefore, a

comprehensive data set was developed which

included additional information on patients (e.g.,

gender, age, diagnosis), type of staff involved

in the SOP, and type of drug along with more

detailed classification information (e.g. change

of dose, discontinuation of medication) for each

discrepancy.

In general, implementing the evaluation

components for both SOPs is the most challenging

part of the High 5s Project for all German

hospitals.

Specific issues relating to the SOPs themselves

a) Correct Site Surgery

Site-marking by the person who is conducting

the procedure was not feasible for the majority

of the German hospitals. Because of workflow

considerations, especially in larger hospitals,

one person is responsible for all admission

processes and has responsibilities like obtaining

consent from the patient and site-marking, while

a separate team performs the procedure.

Further concerns that were identified by the

hospital project coordinators in relation to the

site-marking process were staff hygienic concerns

relating to the markers used as well as the fact

that the skin mark often washes off during skin

preparation.


Implementing the team Time Out was challenging

for those hospitals which had no experience

with this process. A process step for which many

hospitals reported problems was the verification

of special equipment and implants prior to

the procedure. This process step does not occur

in proximity to the patient and the hospitals

therefore found it difficult to integrate it into

the workflow of completing the checklist.

b) Medication Reconciliation

Performing Med Rec within 24 hours after

admission is a great challenge for all Med Rec

hospitals because of a lack of resources (staff,

time). In Germany, there are only 0.3 pharmacists

per 100 hospital beds, which is one of the lowest

ratios in Europe. Therefore, it is very challenging

to achieve 100% Med Rec in the target group.

In the majority of the hospitals only a small number

of the patients constitute the target population

being evaluated.

The Netherlands

Dutch hospitals implemented all SOP steps

of the flow chart, except for one process step

of the Med Rec SOP for the targeted population.

A modification was requested for the process step

of consulting the patient. The SOP required asking

patients for their current medication list directly

after entering the Emergency Department. After

receiving a current medication list from the patient,

other sources had to be consulted (community

pharmacist or primary care physician list). Based

on implementation experiences, Dutch hospitals

preferred consulting patients only once: after

obtaining all possible other medication sources.

The SOP was subsequently revised by the High 5s

Steering Group to provide for a more generalized

model for medication reconciliation that gives

clinicians and facilities the freedom to define their

own process to obtain the BPMH and incorporate

this into their existing processes and workflow.

The most challenging aspects in the SOP

implementation included:

ñ interview with the patient;

ñ a sufficient IT infrastructure;

ñ continuous CEO support;

ñ data collection for SOP evaluation and SOP

monitoring;

ñ hospitals expected additional time for pharmacy

technicians and more staff to be needed

to implement and sustain the SOP, but these

resources were not readily available.

97 Interim outcomes

Box 17: Staff quotes regarding implementation issues of the Med Rec SOP

ñ “We created this by pilot projects and business cases. The High 5s Project has played a role in this.

We discussed business cases with the CEO of the hospital, we lobbied a lot, we shared results about

the effect, and we created a sense of urgency about the implementation of the national guideline

for medication accuracy at transitions in care.”

ñ “You really need manpower to carry out the project. For the sustainability of results you need

the same budget in the end. The initial costs during implementation are based on enough time for

the project leader. After implementation, these costs are free.”

ñ “We have chosen to implement the SOP without extra manpower from the hospital pharmacy.

Departments address themselves to this.”

Issues relating to the SOP implementation

ñ Hospitals tried to create more time for current

staff and find other solutions than demanding

more capacity for reconciling medications within

24 hours after admission during nights and

weekends.

ñ IT modifications were necessary to obtain needed

information to create a BPMH. Hospitals faced

challenges in obtaining licenses, workspace (for

computers) and extensions for current software.

When IT requirements were not met, interventions

and processes had to be adjusted in order to meet

the Med Rec goals.

ñ Although hospitals received permission for High

5s participation from their CEOs, a few hospitals

could not spread the SOP hospital-wide because

of budget restrictions during the financial crisis.

However, most hospitals have managed to

successfully spread the SOP hospital-wide.

The spreading of the SOP takes time depending

on the size of the hospital, awareness of

medication reconciliation, and other competing

priorities.

ñ With increased national awareness of medication

safety, some project leaders were able to restart

SOP implementation and to spread the SOP

to other inpatient services.

ñ Hospitals experienced challenges in implementing

the labor-intensive data collection for the

performance measures mainly because of time

constraints, lack of access to medical records

and lack of resources (staff, funding) to carry out

the work. Performance measures have been useful

in creating a sense of urgency (baseline

measurement) and to show improvements

(measurements after implementation). A hospital

staff member said that, ‘the method of measuring

was very supportive and it was great to use

evidenced based tools and indicators in daily

practice.’

ñ Although project leaders were able to use

the performance results to boost implementation,

the trade-off always was interviewing more

patients for the SOP. Another hospital staff

shared that, “When we have no performance

measurements in our hospital it does not mean

that our reconciliation process is not of good

quality. Measurements to monitor the process

also support the national patient safety

programme. During the upcoming months,

we will have internal sessions to provide feedback

about how to create an effective dashboard

to monitor reconciliation and to sustain the

reconciliation process.”

The issues and challenges mentioned above were

experienced in all Dutch hospitals. Medication

interviews with patients are an ongoing challenge,

which has generated difficult discussions among

hospital leaders, pharmacy representatives,

and physicians in participating hospitals.

Annex 18 shows a summary of the implementation

questionnaires after SOP implementation with

participating hospitals in which issues related

to the SOP implementation are investigated.

Singapore

The Correct Site Surgery SOP was intended for

implementation from Jan 2011 onwards in all seven

participating hospitals. Local surgical checklists

were modified to conform to the SOP, and staff

education sessions were conducted to familiarize

hospitals with this SOP.

There were several challenges noted:

1. Difficulty in changing conventional surgical

practice: In the initial stage of the project,

the bulk of non-compliance related to resistance

by surgeons in complying with site-marking

requirements and Time Out drills, in part because

of refusal to change from conventional practice

and a lack of understanding of the rationale

and requirements of the SOP. Their compliance

eventually improved through hard driving

by OT nurses who refused patient entry into

the operating theatre without a site mark

and refused to pass the instruments and proceed

with the operation until the Time Out was

properly conducted. However, not all nurses

felt empowered to stand up to surgeons.

To maintain compliance, High5 leaders have

found it useful to let data drive performance

by providing monthly feedback on performance

data to surgical heads. In addition, regular staff

education helped new staff understand the SOP.

2. Unpredictable pre-operative scenarios not

covered in SOP: Nurses had difficulty following

pre-operative verification components when

complicated scenarios not covered in the SOP

arose, such as language barriers when asking

open-ended questions to check the correct site

of surgery. To overcome this, posters and videos

were used to offer alternative follow-up actions

and improve compliance with the expectations

of the SOP.

3. Varied adaptation of SOP: Almost all

participating hospitals reported that if they used

a surgical skin marker for site marking, the site

mark invariably got washed off by Chlorhexidine

solution after cleansing, making it impossible

to see the site mark right before the Time Out

which was to occur after draping and cleansing.

It was noted that if Chlorhexidine was not used,

the site mark would not be washed away;

however, Chlorhexidine is the recommended

choice of antiseptic for surgical cleansing

in Singapore’s hospitals. It was then agreed


collectively that for cases where the site mark

is not visible after draping because a) the window

to the operative site is too small to view the site

mark, and/or b) the site mark has been cleansed

off, an acceptable alternative would be to

verbally confirm the laterality of the operative

site with the surgeon.

One of the hospitals also chose, in the alternative,

to conduct the Time Out before draping so that

the whole team could visualize and confirm the site

mark right before draping, even though the SOP

requires this to be done just before skin incision.

The above adaptations of the SOP were shared

during the High 5s webinar held in Dec 2011.

It was then suggested that data be collected

to substantiate the evidence for or against

the practice variations.

There have been no reported cases of wrong-site

surgery in operating theatres since the

implementation of the SOP in Jan 2011.

Trinidad & Tobago

The CSS SOP was piloted and is being implemented

at four of the five participating hospitals in its

original format. No changes have been suggested.

However, Project Team Leaders have noted

the challenge staff are experiencing in coping with

the perceived data collection burden given the fact

that many other data collection forms were already

in use within the various surgical units. A review of

all of the data collection tools used in the surgical

units was recognised as being urgently required.

All project team leaders agreed that the CSS SOP

will have a positive impact when fully implemented.

The hospitals that piloted the SOP have already

identified positive impacts such as:

ñ increased compliance with site marking;

ñ increased staff awareness and vigilance for

surgical safety;

ñ provision of an opportunity for staff to double

check the information given before surgery;

ñ encouragement/reinforcement of the importance

of teamwork; and

ñ encouragement of communication among

members of the surgical team.


The US hospital successfully incorporated the

High 5s checklist into its pre-surgical process

and received Institutional Review Board approval

for participating in the High 5s Project. Further,

it incorporated the ‘Event Analysis’ documents

into its data collection processes. The IT

department at the hospital was able to build an

electronic bridge from the OR electronic medical

record (EMR) Centricity® to a database accessible

to the datacollector for both the pre-operative

records and operative records. However, the

hospital later switched its pre-operative EMR

to another system, and the data collector had

to then hand-filter the relevant records.

The only modification the hospital requested

was to start sampling its cases because of the large

volume of the surgeries that would need to be

reviewed in accordance with the SOP. It provided

the LTA with a request to the Steering Group

to sample (see Annex 21 for the request), and

the High 5s Steering Group approved the request.

Currently, the hospital samples at least 261 cases

per month.

2. Impact

2.1. Overall issues

The SOPs have had a significant impact in all

participating Member States. As a result

of implementation of the SOP, certain process

steps were implemented. Safety components

are now built into surgical and medication

management workflows, and a culture of shared

ideas and learning among hospitals is promoted.

Impact of the Medication Reconciliation SOP

Thirteen hospitals in the Netherlands that

implemented the Med Rec SOP have demonstrated

equally significant medication communication

safety improvements. Medication inaccuracies

were reduced by 90% within the first five months

of the SOP’s introduction in six hospitals.

The percentage of patients reconciled within

24 hours rose by 40% in nine hospitals after

the first year of implementation and has been

increasing to 60% and even 90% following

a few years of implementation. The role of the

pharmacist in the Netherlands gained more visibility

and importance. The care process was unified

through better communication. Electronic

99 Interim outcomes

prescribing systems were introduced in some Dutch

hospitals and others have been able to access

community dispensing records electronically, as

a result of SOP implementation in an effort to

reduce errors. Sites universally achieved a reduction

in unintentional discrepancies.

When all of the LTAs implementing the Med Rec

SOP met, the positive impacts of different models

of implementation, e.g., pharmacist vs. technicians

creating the BPMH, were apparent, as shown by

the process measure, percent reconciled. As a

result, the French LTA was encouraged to consider

the use of pharmacy technicians in the Med Rec

process, as had occurred in the Netherlands. Lastly,

the comparison of and learning from each Member

State’s data and implementation processes was felt

to be beneficial by all of the Member States.

Impact of the Correct Site Surgery SOP

After several years of designing and implementing

the High 5s CSS SOP, positive impacts can be seen

in organizational safety cultures, process

improvement, and outcomes. Clearly, among

the health-care leadership in participating LTAs

and in hospitals, the High 5s Project has achieved

a heightened awareness of the value of

standardization and the risks associated with use

of non-standardized processes; of the actual level

of performance of defined processes (often lower

than had been assumed prior to measurement);

and of the value of teamwork in achieving higher

levels of performance. Comparison of culture

survey results (average composite scores) with

performance measure results (CS-0 through CS-4)

suggests a positive influence of an organization’s

safety culture on the success of its implementation

and level of performance of the standardized

procedures. Yet to be demonstrated is the reverse

influence; that is, the impact of SOP

implementation on the organization’s culture.

Precisely defined, collected, transmitted and

analyzed performance data suggest an

improvement in the key patient care processes:

preoperative verification, surgical site-marking,

and the Time Out before surgery. For example,

in France, site-marking was not routinely done prior

to introduction of the High 5s Correct Site Surgery

SOP. Since then, performance of this step in the

participating hospitals has gained traction with

performance levels ranging between 52% and 89%

for all eligible cases. In Singapore, a Member State

recognized for its rules-based culture, site-marking

performance improved from 40% to 97%.

Meanwhile, compliance with complete preoperative

verification and the pre-operative Time Out has

been consistently in the high 90% range.

If implementation of standardized operating

protocols is successful, does it have an impact on

the patient safety problem that the SOP is designed

to address? Measuring outcomes in patient safety

initiatives is always challenging and especially

so when the outcomes to be measured, in this case,

incorrect surgeries, are relatively infrequent.

The small sample of hospitals, the limited

timeframe for the Project, and the uncertainty

of complete reporting of all wrong-site surgery

cases all add to the difficulty of demonstrating

an actual decrease in the number of incorrect

surgeries. By the end of the Project, there may be

sufficient data to answer this key question.

However, in the meantime, a surrogate measure

is available by considering the results of CSS

performance measure No. 4: Proportion of Cases

with Discrepancy Noted at Final Time Out and

performance measure No.5: Proportion of Cases

Undergoing Surgery with Unresolved Time Out

Discrepancies. Together, these measures indicate

cases at risk for incorrect surgery because of

identified discrepancies that remain unresolved

when the surgical procedure is begun. Certainly

not all of those cases would have resulted in wrong

site surgeries, but some of them may well have.

The relative proportion of the types of unresolved

discrepancies–factual, procedural, or documentary–

is, at this time, unclear but it is reasonable to assert

that a decrease in the number of unresolved

discrepancies is a proxy for a reduction in

the number of incorrect surgeries. However,

this approach is limited in that these measures

apply only to discrepancies identified during

pre-operative Time Out. Ideally, any discrepancies

that arise throughout the process of preparing

patients for surgery would be identified and

reconciled long before the patient gets to the

operating room.

Unfortunately, these “good catches” are not

routinely measured as part of the High 5s

evaluation strategy. Recently France has recognized

this deficiency and instituted a supplemental

measurement activity designed to identify cases

in which discrepancies have been identified and

reconciled at all stages of the preoperative process.

Data showing types of discrepancies, steps in the


process where they most frequently occur,

categories of health professionals who identify

them, and other useful data are emerging along

with compelling stories of individual cases that

were impacted by performance of the standardized

procedures of the SOP. Other participating

Member States have been encouraged to emulate

this measurement approach initiated by the French

LTA in hopes that the database for assessing impact

on surgical outcomes as a result of the High 5s

Project will be enhanced in time for the final

project report.

2.2. Summary of Member State experiences

Australia

Impact of SOP implementation or participation in

this Project

a) Leadership

At some sites, the performance measure data

has proven useful in obtaining executive and senior

clinician engagement with the SOP.

b) Hospital

Several sites have reported a change in the hospital

culture, with medication prescribers actively

looking for and responding to the pharmacist’s

BPMH and recommended actions in the medication

management plan (see Box 18).

101 Interim outcomes

c) Hospital processes

Eight sites integrated the medication reconciliation

SOP into their existing processes to some extent.

Box 18: Medication reconciliation culture change across the organization

“The successful implementation of medication reconciliation in an organization requires an entire

culture change across the organization involving all disciplines. Implementation is complex and difficult

and it’s hard not to get discouraged. However the positives have been a much greater awareness

of medication safety across the organization.”

Box 19: Raising awareness about best practice of medication reconciliation

Hospital 7: Raised awareness of the process of obtaining a BPMH, medication reconciliation and

follow up of discrepancies ultimately resulted in better processes.

Hospital 2: The High 5s Project has resulted in an increased awareness of best practice medication

reconciliation throughout the hospital.

The SOP enabled hospitals to have a standard,

defined and improved protocol for medication

reconciliation (see Box 19).

Several hospitals chose to use the national

Medication Management Plan (MMP) to

standardize the documentation of medication

reconciliation. This enabled pharmacists to record

outstanding discrepancies requiring resolution in

a standard place in the patient’s notes and readily

available when medicines were prescribed.

Since implementing the SOP the MMP has been

viewed as a multidisciplinary tool for

communication rather than a pharmacy document,

as had been the case prior to the High 5s Project

(see Box 20).

Box 20: MMP for reconciliation of medications

Hospital 5: The medical specialties within the hospital now look for the MMP for reconciliation of

the medications when reviewing the patient and completing the medication discharge summary. It has

enabled the doctors to provide complete medication summaries to the primary care providers, including

documenting the status of medications, e.g., continued, dose increased/decreased, ceased. This has led

to improved communication in the transitions of care and also with the patient.

One hospital reported implementation of

the SOP had effects beyond the medication

reconciliation process (see Box 21).

ñ improving patient safety.

Sites reported that the implementation of the

SOP has improved the quality of the medication

histories obtained and documented by pharmacy

staff. The low figures reported for the MR 2- 4

performance measures demonstrate the benefit

of having a standard process on the quality

of the medication reconciliation performed.

See section 5.4.5 for results.

Resolving discrepancies early in the admission

allowed for timely resolution of medication issues

on discharge, thus reducing delays in discharging

patients. It also freed up time for pharmacists to

educate patients about their medicines (see Box 22).


Box 21: SOP impact beyond the medication reconciliation process

Hospital 2: When reviewing existing processes, the hospital identified multiple problems with their

current medication management processes. As a result, there has been a greater focus on

standardization of medication management, prescribing and patient safety, including the introduction

of the National Inpatient Medication Chart in the Emergency Department.

Box 22: Comprehensive counseling to patients

Hospital 5: Having nursing staff aware that all discharge summaries must be reviewed by a pharmacist

prior to discharging a patient has enabled pharmacists to provide comprehensive counselling (written

and verbal) to patients with the opportunity for patients to ask questions.

d) Patient care/clinical impact

Around half of the hospitals indicated that

implementation of the SOP had an impact on

related or interacting activities and on patient care.

Most were considered positive and included:

ñ having a standard and improved protocol

for medication reconciliation;

ñ improving processes through raising awareness

of the processes of obtaining a BPMH, reconciling

medicines and following up on discrepancies;

ñ using data collected to plan interventions

to follow up on discrepancies;

ñ collection of data not previously accessible;

A downside of increasing the quantity and quality

of medication reconciliation performed when

resources are limited is that it is done at the

expense of other activities such as patients

receiving a regular review of their medicines. One

of the reasons given by hospitals for withdrawing

from the Project related to competing priorities.

e) Communication

Communication about the High 5s Project and

the organization’s commitment to the Project

was disseminated primarily through announcements

and progress reports made at department/project

specific meetings, newsletters, and information

posted in public areas. Staff updates on SOP

implementation are primarily provided through

regular reporting of performance measures,

quarterly reports, and periodic reports of specific

aspects of the Project. Lack of resources and

organizational support or guidance were

the primary challenges to communication.

f) Resolving discrepancies

By auditing the quality of the medication

reconciliation process hospitals were able

to identify that some discrepancies remained

unresolved for some days and sometimes until

the time of discharge. As a result, discharges were

often delayed while the discrepancy was resolved.

Initially identified as a problem by one hospital,

other hospitals soon reported similar problems.

This led to the development of two additional

performance measures to assist hospitals collect

data on outstanding discrepancies unresolved after

48 hours following admission. Hospitals used this

data to provide feedback to prescribers, thereby

reducing the number of unresolved discrepancies.

Impact of multi-pronged approach to SOP

evaluation

a) Patient care /clinical impact

Hospitals reported that implementation of the

SOP has resulted in a more standardized process

for obtaining a BPMH, training staff, reconciling

medicines and resolving discrepancies. The benefit

of using a standardized approach to medication

reconciliation is demonstrated in the results of

the audits undertaken by the independent observer.

Hospitals have found the performance measure

data useful for identifying gaps in practice and staff

training needs and for further improving their

processes. Overall the Australian performance

measures show patients that receive medication

reconciliation at admission are at low risk of having

an adverse medication event as the result of

a discrepancy between the medicines taken prior to

admission and those prescribed during the hospital

admission. This risk has continued to decrease over

the duration of the project. See figures in Annex 10.

A limited number of event analysis reports have

been submitted, none of them for a medication

reconciliation incident causing serious harm.

The four hospitals that have submitted event

analyses report that the process has provided

them with valuable information on their medication

reconciliation processes and failures (see Box 22).

This information has been used:

ñ to improve the medication reconciliation process;

ñ as case studies in education sessions to highlight

to other clinicians what can go wrong when the

medication reconciliation SOP is not followed.

See section 5.4.3 for further information on event

analysis.

b) Engagement of frontline providers and/or

senior leaders

Several sites reported that the performance

data has been useful in engaging executive staff

and senior leaders (see Box 23).


Box 23: Engagement of frontline providers

Hospital 1: Intermittent reporting of Med Rec rates increases knowledge and awareness, and

reinforces processes. All executive staff within the facility are now aware of the MMP and its use.

There is now heightened awareness of the benefits and value of obtaining a BPMH and performing

medication reconciliation.

Performance measure MR-1 allows hospitals to

measure the proportion of the eligible population

receiving medication reconciliation and has been

useful in some hospitals for making a successful

case for more resources (see Box 24).

Box 24: Improving services

Hospital 1: Significant and sustained improvement in MR-1 rates was realized with the initiation of

a Saturday ward-based clinical pharmacist service that provided medication reconciliation for new, high

risk admissions.

frontline clinicians about the project and

encouraging them to follow the SOP (see Box 25).

The performance measure data and event analysis

information has also been useful for informing

Box 25: Maintaining support

Hospital 1: Reporting of improving MR-1 rates maintains support and enthusiasm for the initiative.


c) In-country awareness and support for the

initiative

Inclusion of medication reconciliation within

Standard 4 (Medication Safety) of the National

Safety and Quality Health Service Standards

has raised the awareness of the process as

an important patient safety activity. The need

to demonstrate compliance with the Standards

for accreditation will drive the widespread uptake

of the process in Australian hospitals. In addition,

one of five medication safety priority activities in

the Australian Safety and Quality Goals for Health

Care requires everyone aged 65 and over to have

their medicines reconciled within 48 hours of

admission to hospital. The Commission continues

to support the initiative through working with

the hospitals implementing the High 5s Project

and leveraging off this work to advocate the

process with key stakeholders and develop

resources to assist hospitals with implementation.

France

Impact of the Correct Site Surgery SOP

This SOP has had significant impact on health-care

facilities in terms of improving communication

between health-care professionals and improving

the safety culture.

a) Patient care/clinical impact

All health-care professionals interviewed by the

LTA highlighted the impact of this SOP on patient

safety and improved communication between team

and patient.

b) Hospital processes

The High 5s Project has allowed health care

organizations (HCOs ) to optimize their surgical

care pathway and to better distribute tasks and

roles to each professional involved thus linking

all of the professionals involved in the patient’s

care ‘from scheduling to the intervention’.

c) Culture of safety

ñ Nurses feel more empowered, reflecting a real

cultural shift to be able to speak up in instances

of discordance which in turn can foster effective

communication among members of the

perioperative team.

ñ Standardized site-marking allows for shared

responsibilities among surgeons and nurses

when site-marking is delegated.

ñ The High 5s Project has had a positive impact

on teamwork for the hospitals at full

implementation.

d) Information

An ongoing process of information gathering

and verification along the patient’s preoperative

pathway has added value to the quality of patient

records. Most of the Project teams highlighted

that the use of the High 5s Checklist has led

to structured information and improved

communication among nurses, anesthetists,

physicians, secretaries and others involved in

the care pathway (Box 26).

A common and unique tool to collect information

during the surgical pathway, including procedures

already implemented before the High 5s Project

was in place has been developed in several

hospitals. It is known as the “Carnet de bloc”

or “livret de parcours interventionnel”.

Box 26: How best to implement the CSS SOP

The four HCOs at full implementation wish to continue using the CSS SOP. They believe that

implementing this SOP is sustainable in their hospitals provided that the pre-operative verification

component is less complex and time consuming and that an electronic checklist is linked to

the patient’s medical record.

Among the four HCOs at partial implementation, three are facing major barriers but are continuing

to carry out the SOP despite the fact that they have not succeeded in reaching full implementation.


Impact of the Medication Reconciliation SOP

This SOP has also had significant impact in

health-care facilities in the re-engineering of care,

improving communication between health

professionals, and improving the safety culture.

a) Process re-engineering of care to improve

efficiency

To improve the efficiency of medication

reconciliation and increase the number of patients

who received medication reconciliation within

24 hours, one Med Rec hospital has reorganized

its reconciliation process. To maintain the benefits

of medication reconciliation, pharmacists involved

in the process have designed a training programme

for physicians and residents. This programme

includes specific time to explain the flowchart

and makes physicians aware of how to stop

medication errors.

Physicians dedicated the same amount of time

to medication reconciliation before and after

the process re-engineering. However, there is no

shared information system between the hospital

and the community. The next challenge for this

hospital is full implementation of medication

reconciliation from patient admission to discharge.

b) Improvement of communication between

healthcare professionals for better patient care

In another hospital, it was noticed that after

18 months of implementation, this SOP had

actually contributed to increased cooperation

between physicians and pharmacists. Beyond

the improvement experienced in medical care,

the benefits of better communication and

coordination of health-care professionals can

contribute to saving physician time and

improvements to patient safety (Box 27).

Box 27: Improvements in the safety culture and quality of care

One Med Rec hospital used the experience of participating professionals to identify and highlight

potential significance of failures of the process, to assess the risks if these failures occur and to propose

preventive measures. The experience also helped to prioritize failures in terms of the potential severity

of their effects, frequency of occurrence, and the risks associated with their non-detection.

This study was conducted using FMEA (Failure Mode and Effects Analysis) to evaluate potential risks

emerging in the process of medication reconciliation.

The researchers used the Joint Commission worksheet adapted from the model used by Good

Samaritan Hospital (Dayton, Ohio) and a predefined scale index published by Williams. This work led

to a better understanding of the constraints of the process and sensitization of all stakeholders involved

in the process.

To improve practice, better performance must be defined as:

- setting a fixed time every day for reconciliation and immediate correction or documentation in

the computerized file;

- training for and assessing the quality of the patient interview; and

- recording the data collected by the pharmacist in the computerized medical file.

c) Measuring clinical impact of unintentional

discrepancies

One university hospital took the lead in studying

the clinical impact of unintentional discrepancies.

Based on their experiences in monitoring

practices, they developed an algorithm to assess

the potential clinical impact of unintentional

discrepancies. To evaluate the robustness of

the algorithm, the national Med Rec group initiated

a study in volunteer hospital participating in

the High 5s Project in order to see if there are

differences in how professional groups approach

unintentional discrepancies in different hospitals.

Germany

a) Hospital culture

In some hospitals the existing culture was

positively stimulated as a result of implementation

of the CSS SOP. Project coordinators reported

that there was heightened perception among

staff that there is an active commitment to patient

safety in the organization. Other hospitals

did not recognise a cultural change as a direct

result of the project because of its small

magnitude in relation to other initiatives.

b) Hospital processes

Feedback of some hospitals: “It’s not just

a checklist, it has helped us to fundamentally

change the way we look at our process.” Thus

the systematic implementation of the SOP radiates

to other not yet standardised processes, and other

projects were affected, for example in the

introduction of wristbands with identification tags

for patients. Hospitals experienced in the use

of surgical checklists prior to the High 5s Project

reported that the project provided a new impetus

in their implementation efforts and that the existing

processes were improved as a result.

c) Patient care/clinical impact

At this time, the overall change in patient safety

cannot be described with certainty. However,

the procedural steps can help to identify safety

gaps in the care of individual patients through

the documentation of the checklist. As a result,

the risk potential for patients at certain steps of

the process has been reduced. Project coordinators

reported that they feel that the awareness of

patient safety as a whole has improved in their

organizations.

d) Communication

By way of the CSS SOP, the pre-operative processes

have been defined more clearly, and a simplification

has subsequently been introduced. Because

of the implementation and individualisation of

the checklists, there is more transparency about

responsibilities (who does what and when) in

the already existing communication process.

It is difficult to draw conclusions about the impact

of the evaluation approach at this time.

Because of the fact that the checklist data

collected since mid-2012 is being fed back much

more rapidly to the hospitals, it will be interesting

to see whether an improvement in the process

measure results is noticeable after 2012. In

addition, national bi-annual benchmark reports

have been provided to the hospitals since

mid-2012, and their impact will be analysed later

through an updated implementation questionnaire.

The Netherlands

Impact was seen in several areas:

a) Leadership

The SOP implementation makes Med Rec

a multi-disciplinary process with clear steps for

each professional group, including the engagement

of patients. Hospital pharmacists contributed

more to the treatment of the patient, patients

were more involved in the process, doctors and

nurses were further aware of the hospital pharmacy

contribution during the Med Rec process, and

nurses experienced easier access to the hospital

pharmacy (Annex 19A and B).

b) Hospital culture

Implementing this SOP contributed to greater

awareness regarding the:

1. patient safety culture in hospitals; and

2. the existence of discrepancies in medication lists.

Some hospital staff have stated that, “because

of SOP implementation there’s more awareness

about the risks of medication errors. In combination

with the national patient safety programme there

are lots of positive changes made. It has led to

more awareness.” The SOP is part of patient safety

in our organization”.

c) Hospital processes

Through implementing the SOP, it has become

clear which professional group is responsible

for creating the BPMH. The steps are set to identify

and resolve discrepancies and assure interview

of patients about their medications within 24 hours

after admission in the Emergency Department.

The hospital pharmacy and through it the

medication reconciliation process, now becomes

part of the treatment of the patient.

d) Patient care/clinical impact

Hospitals have found it difficult to demonstrate

the impact of the SOP on patient safety. However,

with the help of the four performance measures,

it is possible to show the improvement in SOP

implementation (Annex 19A and B). One hospital

stated that SOP implementation had a huge impact

on the reduction of discrepancies in medication

accuracy during transfers, as well as an impact

on medication safety (Annex 19C). Although

the measured reduction in medication errors is

important, the clinical relevance of the reduction

measured is open to question. Broader studies

are needed, such as assessment of the duration

of hospitalization in order to demonstrate

its clinical impact on patient safety.

e) Communication

High 5s Project hospital teams were fully aware

that communication strategies were needed to

implement and sustain this SOP. They used internal

meetings, training sessions, newsletters, posters

and lots of discussion to introduce the SOP


interventions and to spread the new ways of

working together. The commitment to identifying

and resolving discrepancies within a multidisciplinary

processes has become a structured way of working

which, in turn, has improved Med Rec

communication among pharmacists, physicians,

nurses and other health-care professionals.

f) Resolving discrepancies

A greater awareness of medication discrepancies

was created in the Emergency Department

because of implementing this SOP. The SOP also

contributed to awareness raising regarding

this issue in other departments (Annex 19C).

In the Netherlands, a reduction of 75%-90%

in discrepancies within five months after SOP

implementation has been seen. A challenge faced

by most hospitals has been to increase the average

percentage of patients reconciled within 24 hours

because of insufficient pharmacy staff available

seven days a week. More resources are needed

to reconcile 100% of the patients within the

24 hour limit.

Singapore

a) Process for change

To enable effective implementation, the Ministry

of Health teamed up with patient safety champions

in each hospital to form a local High 5s Network.

In addition, the Ministry of Health funded an

additional Healthcare Performance Office (HPO)

High 5s executive in each hospital to assist in data

collection and gap closure. The Project was

a joint collaboration between the Ministry of

Health and all public hospitals, involving surgeons,

anaesthetists and nurses.

Gap closures initiated by institutions between July

2010 and March 2011 included:

ñ revising surgical checklists;

ñ educating staff on the SOP;

ñ developing data collection methods;

ñ developing SOP scripts and videos: KK Women’s

& Children’s Hospital, Jurong Health Services

and Tan Tock Seng Hospital (TTSH) wrote

a Time-Out script. These hospitals subsequently

experienced marked improvement in Time Out

compliance, as staff could follow the step-by-

step requirements in the Time Out drill. TTSH

prepared instructional video footage of actual

time-outs being performed. This received positive

feedback as staff learned the SOP steps better

by seeing them played out in real scenarios.

Gap closures initiatives by the Ministry

of Health included:

1. Regular platforms for sharing and learning:

The Ministry of Health hosted bi-annual

sharing platforms for the High 5s Network to

reach mutual consensus on changes to be

effected. In addition, between January and

March 2011 when data collection was first

launched, fortnightly informal meetings were

held to assist executives in overcoming

problems encountered.

2. Standardizing data collection: To standardize

data collection, an excel spreadsheet was

established for executives to enter data and

derive compliance rates. The Ministry of

Health also adapted the WHO Surgical Safety

Checklist to the local context, including

deriving a calculation sheet for institutions’

reference. This adaptation was a core catalyst

for institutions to adopt the High 5s checklist

and begin data collection.

3. Ensuring data validity: The institution’s

executive performed monthly observational

audits of documentation submissions to ensure

rigour of the data collected. In addition, SQI

co-ordinated additional quarterly Cross Review

Validation Exercises (CRVEs) that engaged

executives from participating institutions

to minimise bias. Discrepancies noted during

the observations enabled the institution’s

executives to identify differences in actual

practice versus paper audits and suggest

measures to address them.

b) Strategy for change

The High 5s champions are leaders in their professions

such as head surgeons and anaesthetists. As such,

initiatives were better disseminated to health-care

professional participants. The institution’s

executives were funded by the Ministry of Health

but reported directly to the hospital management.

From January 2011 onwards, monthly compliance

data were reported back to OR staff and senior

management. Based on the data, staff noted areas

that needed improvement and gradually showed

increased compliance with the SOP. Overall,

it took two years to implement the SOP and reach

a steady state in data collection.

107 Introduction

c) Impact of SOP implementation

The High 5s Network met to review and discuss

the data since the Ministry’s last update in May

2012, and observed that in addition to improved

hospital processes, there were other benefits of

the Time Out drill that were not measured through

the performance measures, such as improvement

in communication between surgical team members

and better coordination arising from the process

of identifying themselves to each other before

the procedure. These were noted to be important

intangible impacts of the SOP on the

communication culture in Singapore Hospitals.

The impact of the SOP on both leadership and

culture of the hospitals was positive. Many hospital

leaders gave positive feedback on how the

collaboration had led to:

ñ improved communication between the Ministry

and the hospitals;

ñ embedded a continuous learning culture in

the hospitals, showing that staff were willing

to change conventional practice given the right

resource support (i.e., with Ministry funding

support of the High 5s Executive); and

ñ demonstration that hospital processes can be

re-designed to provide better patient care.

The clinical impact was that there had been

zero wrong-site surgery cases reported from

major operating theatres since the SOP was

implemented.

Overall, there were four key achievements in the

implementation of the Correct Site Surgery SOP

project:

1) Incorporated permanent improvements in

the OR workflow of all hospitals, in particular:

ñ establishing that site-marking should be done

before the patient enters the OR;

ñ standardizing the site mark as an arrow, and

not a cross, circle or tick, or any other symbol;

ñ instituting a Time Out to ensure a last check

for site and side of the operation; and

ñ design of IT solutions to help hospital staff

follow the protocol. Initially, only one hospital

(National University Hospital) had an OR

dashboard. By the end of 2012, two other

hospitals (Khoo Teck Puat Hospital and Tan

Tock Seng Hospital) had similarly created

electronic systems to hard-wire the checklist

into their workflow.

2) Increased awareness of surgical safety issues:

ñ implementing the CSS SOP resulted in increased

awareness of safety components built into OR

workflows. Through monthly audits, OR staffs

in all hospitals were kept abreast of their

compliance data and the areas that needed

improvement. The High 5s Project then became

a buzzword for surgical safety.

3) Established a sharing platform:

ñ the setting up of the local High 5s Network

to implement the project provided an avenue

for staff in different hospitals to learn

from each other and continuously innovate

to support a patient safety culture.

4) Introduced data quality validation measures:

ñ the Ministry of Health successfully initiated

‘Cross Review Validation Exercises’ (CRVEs)

for hospitals to audit each other. While

external audits were not required as part of

the SOP, the executives found that it not only

helped to validate each other’s work, but

it also allowed them to reflect on each

institution’s workflow designs and take the

learning points back to their own institutions.

The journey for Singapore hospitals has been

a positive one, and the hospitals have requested

similar style collaborations with the Ministry

of Health to look at other patient safety matters

such as medication safety protocols, for example,

Medication Reconciliation.


The U.S.A. hospital has found that the High 5s SOP

strengthens processes and procedures. Using the

High 5s CSS SOP as a springboard, they evaluated

their pre-op process and are making improvements.

In particular, they evaluated the patient and

document flow in the pre-op holding area. Key

takeaways from this improvement process were

that nurses played a key role in the patient safety

process, and sufficient care in the pre-op holding

area is currently playing a key role in preventing

significant issues from arising in the OR.


7Developingnetworks andawarenessraising for the High 5sProject

1. Developing global networks andawareness raising

This global collaboration among international

organizations, agencies and WHO Member States

has been very fruitful. From the onset of

the Project, founding members from WHO and

The Joint Commission/Joint Commission

International– serving as the WHO Collaborating

Centre– sought to optimize the prospect of success

by reaching out to all participants and experts

to agree on the following characteristics:

ñ Collaboration and good will: The concepts of

standardization applied throughout the Project

have consistently been built and based on

consensus among the various networks of LTAs,

associates, and experts in the participating

Member States.

ñ Achievement of country ministerial commitments

to the Project through the existing WHO

networks of WHO with Ministries of Health

of the participating Member States.

ñ Shared global vision coupled with local

ownership among all those participating in

the High 5s Project.

ñ Provision of an evidence base for all steps in

the SOPs and the ‘Impact Evaluation Plan’

developed through global networks of experts.

ñ Project design that included engagement and

continuous improvement platforms as well as

the national networks of hospitals – the learning

communities– which permit necessary local

adaptations that do not change the given SOP,

but modify the way it is implemented in ways

acceptable to the hospitals and LTAs in the

Member States.

ñ Emphasis on the role and interrelationship

of the country-level networks of health care

professionals and teams rather than individuals

alone.

ñ Embodiment of the principles of patient-centered

care by actively involving the patients and

families in the SOP processes, particularly as

most participating Member States patients

are becoming more active participants in their

own care.

Building networks nationally and globally was

clearly one of the critical activities of this Project

that underlay its success. Project participants

had been actively building networks, and continue

to do so, since its inception in 2006.

Raising awareness of the Project was supported

by promoting its activities globally, engaging

interest on the importance of standardization as

a means to safety, attracting global experts and

expert institutions to the Project, securing funding,

and influencing public and political opinion about

the importance of patient safety.

International and national approaches to addressing

specific patient safety problems through extended,

multi-country networks can ensure that there

is true capacity-building in patient safety

worldwide. From a global perspective simply seeing

the implementation of standardized patient safety

protocols by a small number of resource-rich

countries is not sufficient. Over time, the networks

that participating Member States, WHO and

The Joint commission have with transitional and

resource-poor countries will help to support the

adaptation and implementation of these protocols

within the developing world. This is one important

way that the participating Member States, can share

their expertise and technical resources worldwide.

Despite the many other successes in building

global and national networks, implementation

of standardized operating protocols across the

boundaries of multiple countries was fraught with

sociopolitical challenges. The inherent differences

among national and international institutional


Developing networks and awareness raisingfor the High 5s Project

structures of governance, as well as the differing

health-care systems and cultures in the Member

States posed expected barriers to the development

of networks and even to raising awareness about

the High 5s Project.

The barriers encountered could be grouped into

the following categories.

a) Bureaucratic. These included processes for

approval and implementation of SOPs which

were significantly different among Member States,

thus preventing smooth ‘on time’ dissemination

across all of the participants. Sharing developments

with the wide base of High 5s Project participants

was slow and demanding because

of legal and bureaucratic restrictions. Ultimately,

a lack of uniformity in rules, regulations, and

timeliness of processes presented itself as one

of the challenges.

b) Norms of work and experiences. Interpreting

and agreeing upon content was sometimes

challenging because of different understandings and

experiences related to the issues being addressed.

c) Technological. Some participating Member

States were more technologically advanced

than others, leaving a cohort not as connected

to the electronic resources that were available

to all. This presented a unique challenge when

deciding on some of the content of the SOPs,

the evaluation process, and the mode

of dissemination of the SOPs.

d) Resources. Lack of resources, monetary and

human, was a significant barrier to spreading

awareness. Sometimes, this barrier prevented the

solution from being disseminated to the population

who could benefit from it the most. Satisfactory

training and reproduction of the High 5s resources

was challenging to complete for this reason.

Raising awareness about the Project on a global

level was achieved through various channels

of communication. The High 5s Project published

papers, presented the Project at international

meetings, issued press releases and linked with

country partners to support national

communication and promotional activities of

the Project. (See Table 10.)

111 Developing networks and awareness raising for the High 5s Project

Table 10: Communicating the High 5s Project globally

Channel Activity

Peer-reviewed High 5s: Addressing Excellence in Patient Safety 2009, at World Hospitals and Health

papers Services, 45, 2

Introduction to High 5s: Action on Patient Safety (submitted for publication)

International 8th Annual International Forum on Quality and Safety in Healthcare, London,

conferences UK, 2013

ISQua 2012: 29th International Conference, Geneva, Switzerland

ISQua 2011: 28th International Conference, Hong Kong, China

ISQua 2010: 27th International Conference, Paris, France

5th Annual International Forum on Quality and Safety in Healthcare, Nice, France, 2010

ISQua 2009: 26th International Conference, Dublin, Ireland

2006 Annual Commonwealth Fund International Symposium on Health Care Policy

2005 Annual Commonwealth Fund International Symposium on Health Care Policy

Press release 2006 The WHO Collaborating Centre on Patient Safety Solutions, the World Health

Organization and the Commonwealth Fund announce Action on Patient Safety (High 5s)

Initiative

Global and See Annex 1

national

High 5s Project

websites

High 5s Distributed to all High 5s Project participants and stakeholders worldwide

newsletter

2. National networks and awarenessraising

Australia

Participation of Australian hospitals in the High

5s Medication Reconciliation initiative has been

used to raise awareness and promote the concept

of multidisciplinary medication reconciliation

at a national level. The Australian Commission

published three articles on medication

reconciliation in professional journals and has

presented on the Australian experience with

the High 5s Project at a number of national and

international conferences over the period 2010

to 2012. These include the International Society

for Quality in Health Care (ISQua), The Society

of Hospital Pharmacists of Australia (SHPA)

Conference, the National Medicines Symposium,

and the Australian Association for Quality in

Health Care (AAQHC) conference. For a full listing

of titles of abstracts and peer-reviewed articles

see Annex 11.

The Commission has developed a medication

reconciliation webpage that houses a number

of resources to assist hospitals to implement

a formal process of medication reconciliation.

http://www.safetyandquality.gov.au/our-

work/medication-safety/medication-reconciliation/

Australian High 5s hospitals contributed to

the development of the resources. The national

Medication Management Plan (MMP) published

by the Commission in October 2010 provides

health service providers with a standardised form

that can be used by nursing, medical, and pharmacy

staff to record the BPMH and reconcile medicines

on admission. Other tools include an online

training presentation on how to use the MMP, and

a range of posters and brochures using the theme

MATCH UP Medicines for use in hospitals to

promote the process of medication reconciliation

(Annex 9).

The Commission also works closely with NPS

MedicineWise to have medication reconciliation

messages included in initiatives to improve the

quality use of medicines in the community. One

of the results of this work is a consumer wallet and

brochure that highlights the importance of being

knowledgeable about your medicines and having

an up-to-date medicines list. High 5s hospitals

provide the wallet to patients on discharge from

hospital with their discharge medication list,

and other medication information. It has been

distributed to a number of hospitals outside of

the High 5s group (see Annex 9 for an illustration

of the wallet insert).

The Commission has also raised awareness among

the ambulance authorities of the importance of

bringing the patient’s own medicines to the hospital

when patients are admitted by ambulance from

home.

France

On the CSS SOP

ñ Awareness was raised through presentations

and posters presented by the High 5s teams and

the LTA at national and international meetings

(e.g., International Conference on Quality

and Safety, French Annual National Patient Safety

Week, HAS Annual Meeting, French Society

for Orthopedic Surgery).

ñ A quarterly newsletter has been widely distributed

to hospitals and institutional partners as well as

within HAS.

ñ HAS has placed particular focus on site-marking

and will soon disseminate the site-marking guide.

ñ A national conference is planned for 2013-2014

will disseminate High 5s Project data and present

the outcomes of the Project to date, as well as

lessons learned.

On the Med Rec SOP

ñ The High 5s teams have realized several

opportunities to present to professional societies

and health care institutions during pharmaceutical

congresses to promote this practice.

ñ Presentations and posters presented by pharmacists

and the LTA, (e.g., French Pharmaceutical

Congress, French Society for Clinical Pharmacy,

French Regional Workshop on Quality and Safety).

ñ Presentations on the High 5s Project and the Med

Rec SOP at the national meetings of OMEDIT

Aquitaine, part of the national OMEDIT network

in collaboration with the Ministry of Health.

This meeting takes place every semester and

brings together stakeholders in medication safety.

This activity helped to provide visibility for all

of the hospitals that have been implementing

Med Rec but are not involved in the High 5s

Project. The LTA’s objective is to gather

information from these teams in order to

understand how they succeeded in setting up

the medication reconciliation process and

to develop a national network for sharing

experiences.


Germany

The participation of Germany in the High 5s Project

has fostered a strong German learning community

of hospitals. The LTA organizes workshops for

the hospitals with the aim of training the trainers

and sharing experiences. In addition to the High 5s

international news bulletins which are circulated

to the German hospitals, the German LTA has set

up a national mailing list. This mailing list is used

to distribute SOP-specific information to

the hospitals on an irregular basis. This information

encompasses recent SOP-specific literature, case

studies from CIRS, television programme tips

or newspaper articles regarding patient safety,

and conference presentations and posters, as well

as other patient safety resources.

The German LTA and the High 5s hospitals

regularly attend national and international patient

safety and quality conferences and give High 5s

presentations, thereby increasing the awareness

of the Project. In addition, SOP-specific

publications in several widely-read professional

journals have achieved much attention for

the Project in Germany. In this context, the LTA

and participating hospitals have received many

inquiries on SOP-specific issues.

The expert advisory committee convened by

the Ministry of Health ensures strong connections

between the High 5s Project and the German

Society of Surgery as well as the German

Association of Hospital Pharmacists. The Ministry

of Health and these experts have established links

between the High 5s Project and related projects

being conducted in Germany. Further, they support

Project implementation by making suggestions

for further Project development and for ensuring

its sustainability.

The Netherlands

The High 5s network started in the Netherlands

in 2009. Within this network, hospitals have

worked together to carry out the SOP. Hospitals

collaborated with CBO to disseminate results

at international/national conferences on patient

and medication safety through workshops/(master)

classes, (poster) presentations and site visits.

Twelve hospitals took part in an extra data-

collection effort on the effect of pharmacy-based

medication reconciliation in the Netherlands.

A list of dissemination activities from 2009 till

present can be found in Annex 20. Because of

a press release in February 2011 highlighting a 75%

reduction in medication discrepancies following

implementation of the Med Rec SOP, and because

the pilot hospitals were successful in their SOP

implementation and results, many more hospitals

became interested in joining the Project. Even

though it was difficult for them to find resources

to participate during the time of financial crisis,

four hospitals joined the network using their own

budgets. Mental health and long-term care were

enthusiastic about the Project, but unfortunately

were not able to allocate budgets and manpower

to become involved in the Project.

Singapore

The High 5s Correct Site Surgery project was

presented at the Ministry of Health Work Plan

Seminar 2012 held in August 2012 which was

attended by the Singapore Permanent Secretary

and Health Minister. The Work Plan Seminar is

an annual event that provides a platform for the

Ministry of Health to share Healthcare 2020 plans

and initiatives with all of the Public Hospitals,

and also serves as a forum for hospital clusters

to share initiatives that they have put in place

to support Healthcare 2020. The two part-time

Network co-chairs were invited to present the

journey and success in implementing the CSS SOP.

The messages highlighted (i) how cross-cluster

co-operation has brought about great success for

everyone and ultimately better care for patients,

and (ii) the innovations and different methods

adopted in the project. Feedback by the hospital

CEOs on the presentation was positive.

Internationally, Singapore efforts to implement

the High 5s Correct Site Surgery SOP have been

showcased via poster display at the 23rd Annual

National Forum on Quality Improvement in Health

Care in Florida in December 2011. An updated

poster was again displayed at the International

Forum on Quality and Safety in Healthcare in Paris

in April 2012, and another updated poster

was presented at ISQUA’s 29th International

Conference in Geneva in October 2012.

Trinidad & Tobago

The piloting of the CSS SOP on the orthopaedic

units has created a renewed awareness of the

importance of staff communication (verbal and

written) in achieving and maintaining patient safety.

Project team members have recognised the need

113 Developing networks and awareness raising for the High 5s Project

for ongoing monitoring, and evaluation of

the Project as well as continuing sensitization

of both executive and clinical staff. National

networks with other public and private hospitals

are yet to be established.


The Joint Commission alongside a number

of partners has made significant strides in raising

awareness of wrong-site surgery. Through

the Center for Transforming Healthcare, The Joint

Commission has engaged with the Lifespan

Hospital System in Rhode Island. A result of that

collaboration was the Targeted Solutions Tool

for Wrong Site Surgery, which allow organizations

to identify, measure, and reduce risks in key

processes that can contribute to a wrong-site

surgery.


8High 5smeetings

1. Steering Group meetings

The High 5s Steering Group includes participants

from all LTA, WHO and Collaborating Centre

leadership, experts from the fields of surgery and

pharmacy, and the US Agency of Healthcare

Research and Quality (AHRQ). Together, the

Steering Group has created and approved the SOP

content and the processes for evaluation and event

analysis. All collaboration takes place either via

electronic modes of communication such as emails,

conference calls, and webinars or through

the two face-to-face meetings every year.

Although the SOPs have been implemented

in different Member States and under different

conditions and cultures, it is both surprising and

comforting that similar problems are found in

all settings. When this happens, the Steering Group

is able to assist each other in identifying solutions

that work. Through collaboration, the High 5s

Project team has found supportive partners in

Ministries of Health and other groups who are

contributing to improving the feasibility and impact

of the standardized protocols.

2. International hospital meeting

A one-day ‘First International High 5s Hospitals

Meeting ‘took place at WHO Headquarters in

Geneva in October 2012. The meeting provided

a platform where High 5s hospital teams and LTAs

met to discuss experiences, share best practices,

compare evaluation data, and provide

recommendations and best ways forward for

the High 5s Project. The hospital meeting served

as a hub for an inter-country exchange of

knowledge and implementation experiences to

strengthen and motivate hospitals’ commitment.

The meeting brought together 80 participants

representing 31 High 5s hospitals from Australia,

France, Germany, the Netherlands, Singapore and

the United States of America. Participants included

hospital technical and management staff, and LTA,

Collaborating Centre and WHO Patient Safety

Programme staff. Three High 5s experts from

Canada and the United States of America joined

the meeting. Each participating Member State

had selected one hospital per SOP to share

implementation experiences and lessons learned.

In addition, High 5s hospitals from Australia,

France, Germany, the Netherlands and Singapore

presented 30 posters. Hospital data showed that

SOP implementation has had positive impacts on

hospital processes and patient care. The poster

jury consisting of the three High 5s experts

together selected one poster per SOP for the

“High 5s Achievement Award”. Singapore National

University Hospital won the “High 5s Achievement

Award” for the CSS SOP, and the “High 5s

Achievement Award” for the Med Rec SOP went

to The Netherlands Medical Center Alkmaar.

In the break-out session, meeting participants

formed five Working Groups to exchange

information on SOP implementation challenges,

qualitative and quantitative evaluation activities,

and the impact of the SOPs across diverse country

cultures and settings. Hospital participants were

able to discuss directly with High 5s Steering

Group members their experiences and challenges,

and to contribute ideas on the best ways forward.

For further information see Annex 22.


The High 5s Steering Group meetingsand international hospital meeting

9Next steps

1. Overview

Implementation of the SOPs is progressing

successfully in participating hospitals and Member

States. A collaborative effort by WHO, the

Collaborating Centre, all LTAs and experts is being

made to keep the SOPs updated and to amend

them as needed. Plans to disseminate the SOPs

and share the evaluation methodologies with

the rest of the world are also being developed.

It is expected that at the last Steering Group

meetings in 2014, discussions will focus on how

to promote and share SOPs, methodologies,

lessons learned and expertise worldwide, and on

how to build networks with the developing world.

The handover of the Information Management

System (IMS) and the primary location of project

data and analysis will be determined by the time

data collection comes to a close in September

2014. The Steering Group intends to make the

transfer smooth such that this utility can be used by

Member States and organizations after the official

close of the High 5s Project anticipated

in early 2015.

The SOPs have been developed with the intent

that hospitals and organizations worldwide can

readily adopt them and improve patient care and

safety. Cultural, bureaucratic and practical lessons

that were learned by the High 5s teams and LTA

staff will be shared with the world community.

Barriers faced by teams were similar despite

the differences in regions and cultural contexts,

for that reason, the documentation of some

solutions to these barriers should be useful.

Paramount to all efforts will be active engagement

with existing partnerships that the LTAs, WHO

and Joint Commission International already

have with transitional and developing countries

to support the promotion, adaptation and

implementation of the protocols within more

countries around the world.

Some of the successes of this international

collaborative have been largely the collaboration

itself. From the inception of the Project, the

leadership was composed of international public

health experts and those interested in patient

safety, along with organizations like WHO

and Joint Commission International. This special

relationship between organizations and individual

experts has led to the successful analysis of

the feasibility and impacts of the High 5s SOPs

at a ‘grassroots’-level.

2. The cross High 5s medicationreconciliation map

An interactive online map was developed by

the Institute for Safe Medication Practices (ISMP)

Canada, the Med Rec SOP lead, to profile High 5s

achievements in the participating Member States

which have implemented the Med Rec SOP.

The map includes Med Rec implementation site

information and highlights High 5’s Med Rec

implementation publications and national supports

such as accreditation standards. Links for all

are supplied where available. Participating Member

States were provided a country-specific map graphic

to use for marketing and communications and to

connect teams and sites. The map is available at

http://www.ismp-canada.org/medrec/map/world/

3. Next steps by LTAs

Australia

SOP implementation

The next step for Australia is to spread the SOP

to patient discharges. Quality improvement

measures are currently being developed and will be

available for hospitals (including High 5s hospitals)

to collect information on discharge medication

reconciliation and how to measure improvement.


Next steps

Adopting new SOPs

The Commission will not be adopting any new

High 5s SOP.

Expansion to additional hospitals

As of January 2013, all hospitals in Australia

are required to meet the NSQHS Standards as part

of the new national accreditation scheme.

This requires hospitals to be assessed against

two criteria in the Medication Safety Standard that

relate to medication reconciliation. The model

of medication reconciliation described in the

Standard 4 - Medication Safety Standard Safety

and Quality Improvement Guide – the resource

developed to assist hospitals implement

the NSQHS Standards - follows that of the High 5s

Medication Reconciliation SOP. The objective

is for the medication reconciliation SOP to be

implemented in all Australian hospitals.

The Commission will be working with the State

and Territories to achieve this objective and is

developing additional resources to assist hospitals

to implement Medication Reconciliation and

monitor their activities. These will include

a MATCH UP Medicines medication reconciliation

toolkit, a BPMH training tool, and performance

measures to measure medication reconciliation

performed on admission and at discharge.

France

Next steps are presented in Table 11.

119 Next steps


Table 11: Next steps for French LTA

CSS Med Rec

Full implementation of the CSS SOP in the four HCOs at partial To support and assist

implementation seven or eight hospitals to

develop full implementation

a)The LTA is reinforcing its support in order for the HCOs to overcome of medication reconciliation.

the barriers identified (visits, conference calls, and team training are Currently, they are not fully

going to be proposed by the CEPPRAL). The goal is to spread SOP operating within the framework

implementation throughout the entire hospital. The standardized of High 5s Med Rec criteria.

evaluation activities would be limited to a few units.

b) Advice is being developed on how to use these SOP results to help

organizations ensure safety when patients are admitted on the day of

the surgical intervention, as this situation will soon become prevalent.

Engage the hospitals CEOs

Targeted communications towards the CEOs of the participating hospitals will be conducted, e.g., for

the CSS SOP, a quarterly newsletter addressing issues of leadership, culture safety, and comparative results

A workshop is planned at the end of the year in order to share experiences and results.

Foster recognition of the contribution of the High 5s teams participating in the Project and reward

their efforts.

Dissemination of lessons learned and results of the High 5s Project

HAS is planning a conference with a large audience of interested stakeholders where outcomes and

lessons learned will be presented. This is essential for the successful integration of the High 5s experience

into the national strategy on patient safety. Publications and communications in French by the hospital

High 5s teams and the LTA will also support the dissemination effort.

Scale up the High 5s Project good practices to the national level

Site-marking and pre-operative verifications Medication Reconciliation

A committee including the Ministry of Health, HAS and

the CEPPRAL has recently been set up to develop a strategy

for spreading the learning about patient safety practices

derived from the High 5s Project to the national level.

Promoting site-marking at the national level

ñ The surgical checklist is a required practice in France but

does not include site markings. Spreading and adoption

of site-marking practices will have a major impact

on surgical safety. As a first step, HAS decided to promote

site-marking in the next HCO accreditation cycle. Feedback

from High 5s hospitals shows that a sense of “ownership”

of this practice has developed within the teams involved

in the Project. A community of “champions” has grown

and will have a major role in the uptake of this good practice

by other teams. Site-marking will also be extended to

outpatient surgery.

ñ Pre-operative verification: Additional analysis of the feedback

and High 5s data will be conducted on the pre-operative

verification process. Data and success stories derived from

this analysis will provide inputs on the HAS surgical safety

programmes.

Develop a strategy on the

spreading of High 5s Med Rec

outcomes and lessons learned

at national level, and also take

into account the experiences

of the health care facilities not

participating in the Med Rec

project which perform

medication reconciliation

ñ Integrate Med Rec as

a patient safety practice

in the medication safety

policy, working in

collaboration

with all the stakeholders

in the field;

ñ Provide Med Rec guideline

and toolkits adapted to the

different types of HCOs.

Adopting new SOPs

The adoption of new SOPs is not being considered

at present because of limited resources and funding

HAS, as well as ongoing policy requirements

concerning budget restrictions in the French public

sector.

To conclude, a large amount of work and sharing

of knowledge and experience has been undertaken

since France became engaged in the High 5s Project.

At present, integrating Project recommendations

and conclusions into the national strategy is one

of the main challenges. To this end, the French

experiences and exchanges within the global

learning community and the High 5s Steering

Group have been invaluable.

Germany

SOP implementation

CSS: The German LTA has encouraged the hospitals

to submit their data on a monthly basis for a period

of at least two years. Several hospitals are eager

to submit data for a longer period and to receive

feedback until the end of 2013. Because the

German data model is very resource-consuming

(i.e., scanning of the checklists, creation of

feedback reports for each hospital), the LTA will

stop accepting checklists at the end of 2013

and will focus on a detailed final analysis of the

checklist data during 2014. A majority of the

hospitals have signaled that they will maintain the

SOP processes (or modified versions of the SOP)

and checklists after the High 5s Project has ended.

Med Rec: Following the translation, adaption

of the implementation tools and the development

of a data collection tool, the hospitals officially

started implementation and evaluation of the SOP

in the spring of 2013. Data collection is to

continue until December 2014, with subsequent

final analysis of the data.

A public workshop will be organized in June 2015

to present the final Project results.

Expansion to additional hospitals

CSS: There is no expansion of the current project

planned. However, following the final analysis of

the multidimensional evaluation data on the

national and international levels, the German

LTA plans to draft a “best implementable surgical

checklist” and a clinically well-assessed SOP.

These High 5s findings will be published as

recommendations for German hospitals.

Med Rec: The German LTA is planning on recruiting

3 to 5 more hospitals for implementation.

The Netherlands

The Dutch Ministry of Health has been supporting

participation in the High 5s Project and the pilot

group since 2009. Follow up activities on the

SOPs are being discussed, and other parties will

be involved in planning about the spreading of

knowledge and outcomes to other hospitals and

facilities. The outcome of the national patient

safety hospital programme (2008-2012) shows

the importance of the theme of medication

reconciliation for elective and discharged patients,

which is still not effective enough. For the target

population (65 years and greater admitted

through the ED) of the High 5s SOP it is clear

that implementation of the SOP is still needed

after comparison of baseline and subsequent

performance measurements among three different

groups of hospitals between 2010 and2012.

Based on the results of the national patient safety

hospital programme and the experiences with

the High 5s SOP for Medication Accuracy

at Transitions in Care, the Dutch Inspectorate

of Health has highly recommended this SOP be

continued in the Netherlands (see Box 28).

121 Next steps

In June 2013, the High 5s SOPs were mentioned

by NIVEL in the evaluation research of the national

patient safety programme in Dutch hospitals

(NIVEL and EMGO). Dutch High 5s Med Rec

hospitals are also interested in adapting the SOP

for Concentrated Injectables SOP. These hospitals

have emphasized that, “the international

contribution is great because of the learning

community. We would also like to implement

another SOP, for example, the SOP for

Concentrated Injectable Medicines. This is also

a theme of the national patient safety programme.

If there’s any external budget available, we would

like to participate in this SOP”.

In the Netherlands the Concentrated Injectables

SOP could also be used to implement the national

guideline on this topic which was also part of

the national patient safety hospital programme

(high risk medication). In 2013, the LTA has started

a pilot on the Concentrated Injectables SOP in

16 hospitals in collaboration with national experts

and the support of the Ministry of Health.

Singapore

The High 5s Network has expressed interest in

implementing the High 5s SOP to other areas

where appropriate and feasible, such as the

Radiology Department and Day Surgery Operating

Theatres, and the Ministry of Health is strongly

supportive of this. They have also suggested

stepping up the frequency of CRVEs so that more

cross institutional observations can be conducted

to validate practices. The Ministry of Health will

continue to maintain a facilitative role for

institutions to do so.

Meanwhile, the Ministry of Health has gone ahead

to adopt the Medication Reconciliation SOP into

the National Standards for Healthcare document

for Public Hospitals in Singapore and to commence

implementation. The target date to begin to

collect data is currently set for April 2013.

The Ministry of Health will continue to support the

High 5s initiative through funding of the headcount

to support the data requirements of the Project

until the end of the Project in December 2015.

Trinidad & Tobago

The Ministry of Health plans for:

ñ full implementation of the CSS SOP at relevant

surgical units of all public hospitals by April 2014;

ñ expansion to the private hospitals within Trinidad

and Tobago during the latter half of 2014;

ñ development and issuance of a National Policy

on Site Marking to all health-care providers

(public and private); and

ñ review and revision of all supportive data

collection tools used by staff in the surgical units

to achieve standardization.


In the U.S.A., there are two potential avenues

for adoption of SOP components via process

of care measures. In general, the benefit of the

process of care measures is that they are designed

to provide prima facie evidence of linking process

to outcome. One avenue of leverage is through

accreditation, while the other avenue is through

re-imbursement.


Box 28: Committed to improve medication reconciliation in Dutch hospitals

The discussion in Dutch hospitals on the capacity of hospital pharmacy technicians to reconcile patients

and to complete the medication reconciliation process is all about costs for extra staff. However,

the Dutch Inspectorate of Health wrote in November 2012:

“WHO committed to improve medication accuracy at transitions in care. WHO integrated this topic

in the High 5s programme to improve patient safety. The implementation of this project which has been

supported by CBO in 15 Dutch hospitals showed us in the Netherlands a reduction of 75% of

inaccuracies in the actual medication overview of patients which can be realized by using the WHO

developed Standard Operating Procedure (SOP). Commitment to this topic is definitely worth it.

Because of the necessary deployment of extra staff, hospitals now appear quite able to actually carry

out medication reconciliation. Investment in this staff pays for itself quickly. Medication wrongly given

or not given while necessary, leads the patient and thereby the hospital to undesirable situations and

to greater costs [12:p.27, 28].

If the accreditation approach were to be pursued,

the obvious outlet would be through The Joint

Commission. The Joint Commission is the

accrediting body for over 20 000 healthcare

organizations and programmes in the U.S.A.

Accredited hospitals incur site visits every third

year and are evaluated by series of criteria.

The Joint Commission currently uses “Accountability

Measures” which are designed to incorporate

four constructs: research, proximity, accuracy, and

adverse events. According to The Joint Commission,

the measure needs to be proven by research that

the process being measured is closely connected

to the outcome it impacts. Further the measure

should accurately assess whether the care process

has actually been provided, and whether the

measure, as designed, minimizes or eliminates

unintended adverse effects. Most SOP measures

would fit within all four of these constructs.

The second avenue would be through hospital

re-imbursement from the US Centers for Medicare

& Medicaid Services (CMS). Starting in FY12,

hospitals are now scored on conducting certain

processes (compared to a benchmark) and

demonstrating their improvement. The scores are

used to calculate incentive payment amounts.

Hospitals risk losing 1% in Medicare inpatient

payments, eventually rising to 2% by 2017. It is

expected that the programme will expand in FY14,

with the addition of 20 more measures.

It should be noted that The Joint Commission

and CMS have worked together to develop

measures in the past. If this course is chosen, an

approach similar to the one taken in 2003 by CMS

and The Joint Commission when they co-wrote

the ‘Specifications Manual for National Hospital

Inpatient Quality Measures’ could be used.

123 Next steps

The High 5s Project

Documents