IS AUTOMATIC DATA COLLECTION FOR QUALITY INDICATORS POSSIBLE?

www.pkssk.fiPohjois-Karjalan sairaanhoito- ja sosiaalipalvelujen kuntayhtymä

IS AUTOMATIC DATA COLLECTION FOR QUALITY

INDICATORS POSSIBLE?

17.3.2011 Matti Reinikainen

North Karelia Central HospitalJoensuu, Finland

www.pkssk.fiPohjois-Karjalan sairaanhoito- ja sosiaalipalvelujen kuntayhtymä

CONFLICTS OF INTEREST STATEMENT:

MATTI REINIKAINEN, MD

- Position: Chief Physician, Dept of Intensive Care, North Karelia Central Hospital, Joensuu, Finland

- Position of responsibility: 1st Secretary, Finnish Society of Intensive Care

- Connection with the Finnish Intensive Care Consortium:

1. Using reports and analyses as department leader

2. Using database for research purposes, PhD thesis under preparation

- Economic interests in this subject: none

TERMS USED IN THIS PRESENTATION

• Clinical information system (CIS) – a computer system that replaces bedside paper

documentation– the system automatically collects data from patient

monitors and ventilators and the lab system and shows the data both numerically and as graphic trends

– bedside screen(s)

• Data collection software– a computer system that is linked to the CIS and

automatically transfers data to the centralised database of the Finnish Intensive Care Consortium

– data accuracy is checked before submission

QUESTIONS TO BE ANSWERED

• If automatic data collection were possible, would there be associated benefits?

• Is it possible?

• Are there drawbacks?

When you are taking care of a patient who is bleeding

…and who is haemodynamically unstable

Do you have time for careful documen-tation of blood pressures etc.?

Or would it be helpful if the data were collected automatically?

Automatic data capture into a clinical information system decreases the time spent by nurses on documentation and increases the time spent on patient care

• Wong DH et al. Changes in intensive care unit nurse task activity after installation of a third-generation intensive care unit information system. Crit Care Med 2003; 31: 2488-94.– Before and after installation of clinical information

system– The percentage of time ICU nurses spent on

documentation decreased by > 30% and the time spent on patient care increased

Automatic data capture into a clinical information system decreases the time spent by nurses on documentation and increases the time spent on patient care

• Bosman RJ et al. Intensive care information system reduces documentation time of the nurses after cardiothoracic surgery. Intensive Care Med 2003; 29: 83-90.

– Randomised controlled trial! – documentation on paper vs. into an information system

– A 30% reduction in documentation time (p < 0.001) was achieved, corresponding to 29 min per 8 h nursing shift

– This time was completely re-allocated to patient care

… or does it?

• Saarinen K, Aho M. Does the implementation of a clinical information system decrease the time intensive care nurses spend on documentation of care? Acta Anaesthesiol Scand 2005; 49: 62-5.

– After the implementation of a CIS, there was a small (statistically non-significant) increase in the time spent on documentation

– However, simultaneously there was a significant increase in the time spent on patient care

– ”…any plans to reduce the ICU staff with the aid of computers were not justified.”

Finland

Pirjo Kontio

North Karelia Central Hospital

Population in the district:

173 000 (+ 200-300 bears)

Finland

FINLAND

• Population 5,3 million

• Area 338 000 km2

- The Finnish Intensive Care Study, 1986-87- 25 ICUs- Niskanen M, Kari A, Halonen P. Five-year survival after

intensive care – comparison of 12 180 patients with the general population. Crit Care Med 1996: 24: 1962-1967.

- The Severity Study- Le Gall J-R et al. A new Simplified Acute Physiology Score (SAPS II) Based on a European / North American Multicenter Study. JAMA 1993: 270: 2957- 13 152 patients (720 from 7 Finnish hospitals)- Aarno Kari as country coordinator

1994

- A quality assurance project started in 1994 - Strong growth since 1998- university hospitals joined in 2000-2002

THE FINNISH INTENSIVE CARE CONSORTIUM

1994 2007

THE FINNISH INTENSIVE CARE CONSORTIUM

• 20 hospital districts on Finnish mainland• The main hospital in each district is called the

Central hospital• 15 non-university hospitals, all adult ICUs

participate in the Consortium• 5 university hospitals

– All ICUs from 3 of these participate– In 2 university hospitals: in addition to participating

units, some specialised units not participating

• Apart from 1 ICU, all units use clinical information systems and automatic data transfer into a centralised database

Data collected by clinical information systems, including laboratory test results

• … are automatically transferred by a data collection software to the centralised database of the Finnish Intensive Care Consortium

• The database is handled by Tieto Healthcare & Welfare (previously by Intensium)

• The results in key performance indicators are calculated and reported, many of them automatically

QUALITY INDICATORS

• DATA COMPLETENESS• ADEQUACY OF PATIENT SELECTION• OUTCOMES• RESOURCE CONSUMPTION

In the following slides:• Blue squares = North Karelia Central Hospital• Red circles = the rest of the Finnish Intensive Care

Consortium (i.e. ”the average ICU”)• Each square / circle is based on data from the

previous 6 months

EXAMPLES OF REPORTS THAT ARE UPDATED (SEMI)AUTOMATICALLY

DATA COMPLETENESS

The data completeness index: the second best performing unit (next to the one with the least missing data) gets the index figure 100. The second worst performer (next to the one with most missing data) gets the index figure 50. Other units get index figures based on how close their performance is to these two.

ADEQUACY OF PATIENT SELECTION

• Basic idea: the indication for intensive care is a temporary danger to life and a possibility to prevent death by intensive care

• ICU admissions may be ”inadequate” when– there is no danger to life and no care of high intensity is

needed – could these patients be managed elsewhere?

– patients are moribund – care is futile

High risk and intensive care

The percentage of patients with a high risk of death (> 0,3) and a high intensity of care (TISS score > 30/d)

Low risk – unnecessary ICU admission?

The percentage of patients who had a low risk of death (< 0,05), received care of low intensity (maximal TISS score < 15/d) and were discharge alive

OUTCOMES

• Hospital mortality rate (crude & standardised mortality ratios)

• Mortality after long ICU stays• Post-icu mortality

• (Also measured, though with a considerable amount of manual work: 6-month mortality & health-related quality of life at 6 months)

VLAD curve, the cumulative amount of ”extra lives saved”; curves for 3 ICUs

SMR

Standardised mortality ratios (O/E-ratio, the number of observed deaths divided by the number of expected deaths, the expected number here being based on the SAPS II model)

Hospital mortality after prolonged ICU care

Hospital mortality of patients who were treated in the ICU for > 6 days

Prolonged care – poor outcome

The percentage of patients who were treated in the ICU for > 6 days and who died in the ICU

Post-ICU hospital mortality

The percentage of patients who died in hospital after discharge from ICU

A problem here?

Readmissions within 48 hrs after discharge

The percentage of patients who were readmitted to the ICU within 48 hrs after ICU discharge

Shortage of beds?

• In relation to care days produced

• (Also measured: ”Cost of lives saved” – the amount of resources consumed per hospital survivor. However, comparisons are difficult because all ICU costs are not easily obtained in Finnish hospitals.)

RESOURCE CONSUMPTION

Care days / nurse / day

The number of patient days (24-h-periods) per each nurse / shift

IN PRACTICE, WE CAN SPEAK ABOUT SEMI-AUTOMATED DATA

COLLECTION

• Many data are entered manually into the clinical information system– Admission data– ICU and hospital discharge data (incl. outcome)– TISS items are documented partly automatically, partly

manually– Some physiological data need to be entered manually

Even automatically collected data are checked and validated,

… in North Karelia by this team

Admission data are checked for accuracy and for missing data

TISS items are checked

• Lab test results are mostly transferred without problems

Some derived parameters can be problematic

The PaO2/FIO2-ratio is correct only if both values are documented correctly

FIO2 is documented automatically but monitoring may not be on e.g. when inhalational drugs are given

(mmHg)

• Values of some physiological parameters have to be checked for possible technical artifacts– e.g. blood pressure (trends of systolic BP in

the next examples)

Unfiltered raw data

Patient 1

Median filtering (10 min) eliminates most technical artifacts

Patient 1

Patient 2

Unfiltered raw data

Patient 2

Median filtering eliminates most technical artifacts – but not all of them

Patient 2

Manual correction needed

HAS DATA COMPLETENESS CHANGED IN ASSOCIATION WITH THE AUTOMATION OF

DATA COLLECTION?

1998 2003 2008 p

Median (quartiles) 1 (1-2) 1 (0-2) 0 (0-0) < 0.001

Mean ± SD 2.0 2.3 1.2 1.6 0.25 0.85 < 0.001

Missing values of SAPS II parameters in the database of the Finnish Intensive Care Consortium:

43210-1-2-3-4-5-6-7-8-9

Years before and after IVT installation. Installation Year =0

100,00

80,00

60,00

40,00

20,00

0,00

Ra

tio

of

10

0%

co

mp

lete

re

co

rds

Mussalo P, Tenhunen J. Abstract presented at the ISICEM Congress, 2007.

- The proportion of 100% complete datasets was determined (incl. admission and discharge data and data on diagnoses)

- This proportion increased after installation of data collection software Installation of data collection software

Association of data completeness with installation of data collection software

ESICM Lisbon, 24.9.2008, Mussalo, Reinikainen, Karlsson, Ruokonen

How often is it possible to measure the outcome?

Mussalo, P. 1,2, Reinikainen, M.3, Karlsson, S.4, Ruokonen, E.5

For the Finnish Intensive Care Quality Consortium

1) Intensium Ltd, Kuopio, for the Finnish Intensive Care Quality Consortium, 2) Department of Computer Science, University Of Kuopio, Kuopio,

3) Department of intensive Care, North Karelia Central Hospital, Joensuu, Finland, 4) Department of Intensive Care, Tampere University Hospital, Tampere, Finland,

5) Department of Intensive Care, Kuopio University Hospital, Kuopio, Finland

ESICM Lisbon, 24.9.2008, Mussalo, Reinikainen, Karlsson, Ruokonen

The mean time from discharge to registration of hospital discharge data is 1 month

Delay to HDD registrationFrom ICU discharge From hospital discharge

STATUS N Mean CI95% Mean CI95%ALL 17146 38,5 (37,93-39,16) 31,2 (30,63-31,78)SURVIVOR 14297 40,8 (40,21-41,46) 32,6 (31,92-33,21)DEAD 2849 27,1 (25,20-28,96) 24,3 (23,14-25,56)

ESICM Lisbon, 24.9.2008, Mussalo, Reinikainen, Karlsson, Ruokonen

Outcome reporting is quick – situation 1st September, 2008

Does this hospital have a summertime problem?

ARE THERE DRAWBACKS?

The sensitivity of sophisticated computerized methods in detecting artifacts in blood pressure trends is inferior to that of experienced human observers

Care days / nurse / day

The number of patient days (24-h-periods) per each nurse / shift

IS THERE A POTENTIAL FOR ERRORS GOING UNNOTICED?

Care days / nurse / day

The number of patient days (24-h-periods) per each nurse / shift

This peak is caused by an error in the data: erroneous data submission after the fusion of two units

IS THERE A POTENTIAL FOR ERRORS GOING UNNOTICED?

• Morrison C et al. Electronic patient record use during ward rounds: a qualitative study of interaction between medical staff. Crit Care 2008; 12: R148 – The installation of an electronic patient record

system had a negative impact on multidisciplinary communication during ward rounds

• ”Pen & paper” is a very flexible system

• A highly sophisticated computer system may not always be that flexible

CONCLUSIONS

• Fully automatic data collection for quality indicators is not possible– Parts of data have to be entered manually

– (At least parts of) data collected automatically have to be checked and validated in order to assure good data quality

• Semi-automatic data collection is possible, it saves time used for documentation and improves data completeness