Introduction to Patient Safety Research Presentation: Developing Solutions: Cluster Randomized Clinical Trial.

Introduction to Patient Safety ResearchIntroduction to Patient Safety Research

Presentation: Developing Solutions: Cluster Randomized Clinical Trial

Presentation: Developing Solutions: Cluster Randomized Clinical Trial

2: Introduction: Study Details Full ReferenceFull Reference

MERIT study investigators. Introduction of the medical MERIT study investigators. Introduction of the medical emergency team (MET) system: a cluster-randomised emergency team (MET) system: a cluster-randomised controlled trial. The Lancet 2005, 365:2091-2097controlled trial. The Lancet 2005, 365:2091-2097

Link to Abstract (HTML) Link to Full Text (PDF)Full text may be Full text may be

ordered online at ordered online at http://www.thelancet.com

3: Introduction: Patient Safety Research Team

Lead researcher - Professor Ken Lead researcher - Professor Ken Hillman, MD Hillman, MD Clinical Academic, Intensive Care Clinical Academic, Intensive Care

UnitUnit University of New South Wales in University of New South Wales in

Sydney, AustraliaSydney, Australia Field of expertise: intensive care Field of expertise: intensive care

medicine, health services researchmedicine, health services research Other team membersOther team members

Dr. Jack ChenDr. Jack Chen Dr. Rinaldo BellomoDr. Rinaldo Bellomo Dr. Simon FinferDr. Simon Finfer Dr. Arthas FlabourisDr. Arthas Flabouris

4: Background: Opening Points

Cardiac arrest patients who die in general wards Cardiac arrest patients who die in general wards have often received delayed or inadequate carehave often received delayed or inadequate care Early intervention in response to physiological Early intervention in response to physiological

instability might prevent further deterioration in many instability might prevent further deterioration in many patients patients

Medical emergency team (MET) is a Medical emergency team (MET) is a hospital-wide hospital-wide approach for early recognition of deterioration and approach for early recognition of deterioration and early resuscitationearly resuscitation Developed in 1989Developed in 1989

MET system includes:MET system includes: Staff educationStaff education MET calling criteriaMET calling criteria Increased awareness of the dangers of physiological Increased awareness of the dangers of physiological

instabilityinstability Immediate availability of a METImmediate availability of a MET

5: Background: Study Rationale

Previous studies showed MET system associated with Previous studies showed MET system associated with a reduction in unplanned ICU admissions, cardiac a reduction in unplanned ICU admissions, cardiac arrests and deathsarrests and deaths Despite the intuitive appeal of early intervention, MET Despite the intuitive appeal of early intervention, MET

system challenges traditional health care delivery by system challenges traditional health care delivery by crossing many boundariescrossing many boundaries

Research team hoped that further scientific proof Research team hoped that further scientific proof could overcome some of these obstaclescould overcome some of these obstacles

6: Background: Setting Up a Research Team

Selecting collaboratorsSelecting collaborators Research team included statisticians, national Research team included statisticians, national

professional bodies, high level managers and on-site professional bodies, high level managers and on-site project officersproject officers

Team members all attracted by the size and originality Team members all attracted by the size and originality of the studyof the study

FundingFunding Obtained bObtained by a peer-reviewed process through the y a peer-reviewed process through the

National Health and Medical Research Council (NHMRC)National Health and Medical Research Council (NHMRC)

7: Methods: Study Design

DesignDesign: cluster randomized clinical trial: cluster randomized clinical trial Primary outcome was the composite of cardiac arrest, Primary outcome was the composite of cardiac arrest,

unexpected death, or planned ICU admission during a 6-unexpected death, or planned ICU admission during a 6-month study period after MET activationmonth study period after MET activation

ObjectiveObjective:: To To investigate whether the medical emergency team investigate whether the medical emergency team

(MET) system reduces the incidence of cardiac arrests, (MET) system reduces the incidence of cardiac arrests, unplanned admissions to intensive care units (ICU), and unplanned admissions to intensive care units (ICU), and deathsdeaths

8: Methods: Study Population and Setting

SettingSetting: : of 46 hospitals assessed for eligibility, 23 of 46 hospitals assessed for eligibility, 23 randomized to control group randomized to control group (n=11)(n=11) or to receive or to receive introduction of MET system introduction of MET system (n=12)(n=12) Potential participating hospitals identified using thePotential participating hospitals identified using the

Australian Hospital and Health Services Yearbook Australian Hospital and Health Services Yearbook Eligibility criteria:Eligibility criteria:

Public hospitals with more than 20,000 estimatedPublic hospitals with more than 20,000 estimated admissions/yearadmissions/year

ICU and emergencyICU and emergency departmentdepartment No current MET system implementedNo current MET system implemented

9: Methods: Outcome Measures

Primary outcome measurePrimary outcome measure Composite of the incidence of cardiac arrests without a Composite of the incidence of cardiac arrests without a

pre-existing not-for-resuscitation order, unplanned ICU pre-existing not-for-resuscitation order, unplanned ICU admissions and unexpected deaths taking place in a admissions and unexpected deaths taking place in a general wardgeneral ward

Secondary outcome measure:Secondary outcome measure: Cardiac arrests without a pre-existing not-for-Cardiac arrests without a pre-existing not-for-

resuscitation orderresuscitation order Unplanned ICU admissionsUnplanned ICU admissions Unexpected deathsUnexpected deaths

10: Methods: Data Collection

Data collectors trained with standardized data Data collectors trained with standardized data collection manualcollection manual

Outcome and process measures obtained in all Outcome and process measures obtained in all hospitals for a baseline period of two-monthshospitals for a baseline period of two-months Halfway through the baseline period, hospitals Halfway through the baseline period, hospitals

randomly assigned to control or intervention grouprandomly assigned to control or intervention group Four-month educational strategy undertaken to Four-month educational strategy undertaken to

prepare hospitals for the introduction of the MET prepare hospitals for the introduction of the MET systemsystem In intervention hospitals (n=12), MET system was In intervention hospitals (n=12), MET system was

activated at the end of baseline period and made activated at the end of baseline period and made available for the next six-months available for the next six-months

Practices in control hospitals (n=11) remained Practices in control hospitals (n=11) remained unchangedunchanged

11: Methods: Data Analysis and Interpretation

CalculatedCalculated Sample size (Kerry and Bland method to account for Sample size (Kerry and Bland method to account for

clustering)clustering) Cluster-level differences in event incidence (weighted t-Cluster-level differences in event incidence (weighted t-

test)test) Individual level differences in event incidence (Rao-Individual level differences in event incidence (Rao-

Scott chi-square test and adjusted t test)Scott chi-square test and adjusted t test) Outcome-specific intraclass correlation coefficient and Outcome-specific intraclass correlation coefficient and

design factordesign factor Multiple linear regressions to adjust for various Multiple linear regressions to adjust for various

factorsfactors Post-hoc exploratory analysis (paired weighted t Post-hoc exploratory analysis (paired weighted t

test) to examine incidence difference between test) to examine incidence difference between baseline and study periodbaseline and study period

12: Results: Key Findings

Overall rate of calls was significantly higher in MET Overall rate of calls was significantly higher in MET hospitals than in control hospitals hospitals than in control hospitals (3.1 vs. 8.7 per (3.1 vs. 8.7 per 1000 admissions)1000 admissions) Similar incidence of the composite primary outcome in Similar incidence of the composite primary outcome in

the control and MET hospitals (5.86 vs 5.31 per 1000 the control and MET hospitals (5.86 vs 5.31 per 1000 admissions)admissions)

Both MET and control hospitals had similar incidence for Both MET and control hospitals had similar incidence for individual secondary outcomes individual secondary outcomes • Cardiac arrests: 1.64 vs 1.31Cardiac arrests: 1.64 vs 1.31• Unplanned ICU admissions: 4.68 vs 4.19Unplanned ICU admissions: 4.68 vs 4.19• Unexpected deaths: 1.18 vs 1.06Unexpected deaths: 1.18 vs 1.06

Incidence of cardiac arrests and unexpected deaths Incidence of cardiac arrests and unexpected deaths fell significantly from the baseline to the study fell significantly from the baseline to the study period in all hospitals combinedperiod in all hospitals combined

There was no significant difference in the change There was no significant difference in the change over time between the MET and control hospitalsover time between the MET and control hospitals

13: Conclusion: Main Points

MET system increased emergency team calling, but MET system increased emergency team calling, but does not affect the incidence of cardiac arrest, does not affect the incidence of cardiac arrest, unplanned ICU admissions, or unexpected deathunplanned ICU admissions, or unexpected death

Study findings suggest the need for: Study findings suggest the need for: Improved intensive monitoring of patients in general Improved intensive monitoring of patients in general

wardswards Rigorous documentation of patients' conditionRigorous documentation of patients' condition Increased attention to education to ensure appropriate Increased attention to education to ensure appropriate

and timely response ratesand timely response rates

14: Conclusion: Discussion

MET system did not significantly reduce the MET system did not significantly reduce the incidence of study outcomes. Possible explanations:incidence of study outcomes. Possible explanations: MET is an ineffective interventionMET is an ineffective intervention MET is potentially effective but was inadequately MET is potentially effective but was inadequately

implementedimplemented Wrong outcomes were studiedWrong outcomes were studied Control hospitals contaminated as a result of being in Control hospitals contaminated as a result of being in

the studythe study Hospitals studied were unrepresentativeHospitals studied were unrepresentative Insufficient statistical power to detect important Insufficient statistical power to detect important

treatment effectstreatment effects Limitations with previous studies have been the use Limitations with previous studies have been the use

of historical controls and the absence of of historical controls and the absence of randomizationrandomization

15: Conclusion: Study Impact

AcademicAcademic One of the first major health services research projects One of the first major health services research projects

to employ a rigorous scientific approach to employ a rigorous scientific approach Potential influence on funding organisations and Potential influence on funding organisations and

researchers to conduct similar large scale scientific researchers to conduct similar large scale scientific studiesstudies

PolicyPolicy Many national, state/province and regional health Many national, state/province and regional health

authorities have mandated MET-type systems despite authorities have mandated MET-type systems despite inconclusive resultsinconclusive results

PracticePractice Despite inconclusive results, there has been widespread Despite inconclusive results, there has been widespread

uptake of MET-type systems around the worlduptake of MET-type systems around the world PatientPatient

Significant population benefits have been demonstrated Significant population benefits have been demonstrated in before/after studies, but not in the MERIT studyin before/after studies, but not in the MERIT study

17: Conclusion: Practical Considerations

Study duration: Study duration: 36 months36 months 1 year preparation1 year preparation 1 year study1 year study 4 years analyzing data4 years analyzing data

CostCost About About $1.0 million USD$1.0 million USD

Required resources:Required resources: Wide range of researchers and clinicians with high level Wide range of researchers and clinicians with high level

data entry and computing infrastructuredata entry and computing infrastructure Support of the national Intensive Care Society to recruit Support of the national Intensive Care Society to recruit

the 23 hospitalsthe 23 hospitals Goodwill and enthusiasm of participating hospitalsGoodwill and enthusiasm of participating hospitals

Conclusion: Practical Considerations (2)

Wide range of competencies were required, Wide range of competencies were required, including:including: Implementation and maintenance of the MET system Implementation and maintenance of the MET system

under studyunder study Communication, including regular site visits to resolve Communication, including regular site visits to resolve

system and research issuessystem and research issues Statisticians with high level skillsStatisticians with high level skills Complex project managementComplex project management

Ethical approval took 7 months to obtainEthical approval took 7 months to obtain Required 23 separate institution ethics submissions as Required 23 separate institution ethics submissions as

well as overarching University ethics approvalwell as overarching University ethics approval

18: Author Reflections: Lessons and Advice

If the research team could change one thing in their If the research team could change one thing in their research, they would include cresearch, they would include clearer upfront learer upfront research questions for data analysis, for example:research questions for data analysis, for example: Dose (MET calls)/response (outcomes) relationshipDose (MET calls)/response (outcomes) relationship Relationship between baseline level of adverse events Relationship between baseline level of adverse events

and improvement and improvement Nature of interventions performed at time of METNature of interventions performed at time of MET

19: Author Reflections: Lessons and Advice (2)

Advice for researchers:Advice for researchers: "The greatest opportunities for improvement in health "The greatest opportunities for improvement in health

care and patient safety are at the interfaces of care, care and patient safety are at the interfaces of care, between the traditional silos."between the traditional silos."

Multidisciplinary research (e.g. statisticians, clinicians, Multidisciplinary research (e.g. statisticians, clinicians, qualitative researchers) is critical for original patient qualitative researchers) is critical for original patient safety researchsafety research

Initial barriers were largely logisticalInitial barriers were largely logistical Getting 23 hospitals involvedGetting 23 hospitals involved Identifying reliable champions in each hospitalIdentifying reliable champions in each hospital Communicating wellCommunicating well

20: Author Reflections: Lessons and Advice (3)

MET concept is simple and easily applicable to the MET concept is simple and easily applicable to the most basic hospitals. However, evaluation and most basic hospitals. However, evaluation and research is more difficultresearch is more difficult

Complexity, expense and relevance may make this Complexity, expense and relevance may make this research largely unfeasible for developing countries research largely unfeasible for developing countries in its current formin its current form Would require significant adaptations based on local Would require significant adaptations based on local

particularitiesparticularities However, could implement simple version of MET However, could implement simple version of MET

system with simplified local evaluation system with simplified local evaluation • E.g.: staff feedback potentially preventable precursors to E.g.: staff feedback potentially preventable precursors to

hospital deaths and cardiac arrestshospital deaths and cardiac arrests

21: Author Reflections: Selecting Design

Randomisation judged to be the most rigorous Randomisation judged to be the most rigorous methodology available but had to be based on methodology available but had to be based on clusters (individual institutions) not patientsclusters (individual institutions) not patients. . Factors Factors in this choice also included:in this choice also included: CostsCosts Infrastructure to conduct the studyInfrastructure to conduct the study Co-operation of many hospitalsCo-operation of many hospitals Research expertiseResearch expertise Level of scientific rigourLevel of scientific rigour

Other potential methodologies included:Other potential methodologies included: Attempting to define, in a prospective fashion, Attempting to define, in a prospective fashion,

potentially preventable antecedents to hospital deaths potentially preventable antecedents to hospital deaths and serious adverse eventsand serious adverse events

Before/after studies in one institutionBefore/after studies in one institution Case control studies in several institutionsCase control studies in several institutions

22: Author Reflections: Ideas for Future Research

Investigation of serious illness where further active Investigation of serious illness where further active treatment would be futiletreatment would be futile I.e. establish systems for “dying safely” and “living I.e. establish systems for “dying safely” and “living

safely”safely” Evaluation of a parallel system to the MET: one for Evaluation of a parallel system to the MET: one for

early recognition and care of dying patientsearly recognition and care of dying patients Hospitalised patients are becoming older with many co-Hospitalised patients are becoming older with many co-

morbidities and are often admitted even when care is morbidities and are often admitted even when care is futilefutile

MET system can become the surrogate “dying” team for MET system can become the surrogate “dying” team for a hospitala hospital

23: Additional References

Cretikos M, Chen J, Hillman K, Bellomo R, Finfer S, Flabouris Cretikos M, Chen J, Hillman K, Bellomo R, Finfer S, Flabouris A, the MERIT Investigators. The objective medical A, the MERIT Investigators. The objective medical emergency team activation criteria: A case-control study. emergency team activation criteria: A case-control study. Resuscitation 2007;73:62-72.Resuscitation 2007;73:62-72.

Creitkos M, Chen J, Hillman K, Bellomo R, Finfer S, Flabouris Creitkos M, Chen J, Hillman K, Bellomo R, Finfer S, Flabouris A, the MERIT Study Investigators. The effectiveness of MET A, the MERIT Study Investigators. The effectiveness of MET system implementation and factors associated with the system implementation and factors associated with the level of MET utilisation during the MERIT study. Crit Care level of MET utilisation during the MERIT study. Crit Care Resus 2007;9:206-212.Resus 2007;9:206-212.

Cretikos M, Bellomo R, Hillman K, Chen J, Finfer S, Flabouris Cretikos M, Bellomo R, Hillman K, Chen J, Finfer S, Flabouris A. Respiratory rate: the neglected vital sign. Med J Aust A. Respiratory rate: the neglected vital sign. Med J Aust 2008;188:657-659.2008;188:657-659.

Chen J, Bellomo R, Flabouris A, Hillman K, Finfer S, the Chen J, Bellomo R, Flabouris A, Hillman K, Finfer S, the MERIT Study Investigators. The relationship between early MERIT Study Investigators. The relationship between early emergency team calls and serious adverse events. Crit emergency team calls and serious adverse events. Crit Care Med (in press).Care Med (in press).

MERIT Study Investigators. The medical emergency team MERIT Study Investigators. The medical emergency team system and not-for-resuscitation orders: Results from the system and not-for-resuscitation orders: Results from the MERIT study. Resuscitation (in press).MERIT study. Resuscitation (in press).

24: Additional References (2)

Chen J, Hillman K, Bellomo R, Flabouris A, Finfer S, the Chen J, Hillman K, Bellomo R, Flabouris A, Finfer S, the MERIT Study Investigators. The impact of introducing MERIT Study Investigators. The impact of introducing medical emergency teams on the documentation of vital medical emergency teams on the documentation of vital signs. Resuscitation (in press).signs. Resuscitation (in press).

Flabouris A, Chen J, Hillman K, Bellomo R, Finfer S, the Flabouris A, Chen J, Hillman K, Bellomo R, Finfer S, the MERIT Study Investigators. Timing and intervention of MERIT Study Investigators. Timing and intervention of emergency teams during the MERIT study. Intensive emergency teams during the MERIT study. Intensive Care Med (submitted).Care Med (submitted).

The MERIT Study Investigators. The reasons for calling The MERIT Study Investigators. The reasons for calling an emergency team between control and MET hospitals an emergency team between control and MET hospitals (in preparation).(in preparation).

Chen J, Flabouris A, Bellomo R, Hillman K, Finfer S, the Chen J, Flabouris A, Bellomo R, Hillman K, Finfer S, the MERIT Study Investigators. The relationship between MERIT Study Investigators. The relationship between baseline incidence of serious adverse events and its baseline incidence of serious adverse events and its change following the introduction of a medical change following the introduction of a medical emergency team (in preparation).emergency team (in preparation).