Open access Research Validity and effectiveness of ...

Page 1

1Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

Validity and effectiveness of paediatric early warning systems and track and trigger tools for identifying and reducing clinical deterioration in hospitalised children: a systematic review

Rob Trubey,  1 Chao Huang,2 Fiona V Lugg-Widger,  1 Kerenza Hood,1 Davina Allen,3 Dawn Edwards,4 David Lacy,5 Amy Lloyd,1 Mala Mann,6 Brendan Mason,7 Alison Oliver,8 Damian Roland,9,10 Gerri Sefton,11 Richard Skone,8 Emma Thomas-Jones,1 Lyvonne N Tume,12 Colin Powell13,14

To cite: Trubey R, Huang C, Lugg-Widger FV, et al. Validity and effectiveness of paediatric early warning systems and track and trigger tools for identifying and reducing clinical deterioration in hospitalised children: a systematic review. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

► Prepublication history and additional material for this paper are available online. To view these files, please visit the journal online (http:// dx. doi. org/ 10. 1136/ bmjopen- 2018- 022105).

Received 6 February 2018Revised 7 March 2019Accepted 8 March 2019

For numbered affiliations see end of article.

Correspondence toDr Rob Trubey; trubeyrj@ cf. ac. uk

Research

© Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY. Published by BMJ.

AbstrACt Objective To assess (1) how well validated existing paediatric track and trigger tools (PTTT) are for predicting adverse outcomes in hospitalised children, and (2) how effective broader paediatric early warning systems are at reducing adverse outcomes in hospitalised children.Design Systematic review.Data sources British Nursing Index, Cumulative Index of Nursing and Allied Health Literature, Cochrane Central Register of Controlled Trials, Database of Abstracts of Reviews of Effectiveness, EMBASE, Health Management Information Centre, Medline, Medline in Process, Scopus and Web of Knowledge searched through May 2018.Eligibility criteria We included (1) papers reporting on the development or validation of a PTTT or (2) the implementation of a broader early warning system in paediatric units (age 0–18 years), where adverse outcome metrics were reported. Several study designs were considered.Data extraction and synthesis Data extraction was conducted by two independent reviewers using template forms. Studies were quality assessed using a modified Downs and Black rating scale.results 36 validation studies and 30 effectiveness studies were included, with 27 unique PTTT identified. Validation studies were largely retrospective case-control studies or chart reviews, while effectiveness studies were predominantly uncontrolled before-after studies. Metrics of adverse outcomes varied considerably. Some PTTT demonstrated good diagnostic accuracy in retrospective case-control studies (primarily for predicting paediatric intensive care unit transfers), but positive predictive value was consistently low, suggesting potential for alarm fatigue. A small number of effectiveness studies reported significant decreases in mortality, arrests or code calls, but were limited by methodological concerns. Overall, there was limited evidence of paediatric early warning system interventions leading to reductions in deterioration.Conclusion There are several fundamental methodological limitations in the PTTT literature, and the predominance of single-site studies carried out in specialist centres greatly limits

generalisability. With limited evidence of effectiveness, calls to make PTTT mandatory across all paediatric units are not supported by the evidence base.PrOsPErO registration number CRD42015015326

bACkgrOunDFailure to recognise and respond to clin-ical deterioration in hospitalised children is a major safety concern in healthcare. The underlying causes of this problem are

strengths and limitations of this study

► Paediatric early warning systems and paediat-ric track and trigger tools (PTTT) are increasing-ly used by paediatric units across Europe, North America, Australia and elsewhere—this study is a timely review of the evidence for their validity and effectiveness.

► A comprehensive search was carried out across multiple databases and included published as well as grey literature.

► The review highlights methodological weaknesses and gaps in the current evidence base and makes suggestions for future research.

► Heterogeneity in study populations, study designs and outcome measures make it difficult to com-pare and synthesise findings across the wide range of early warning systems and PTTT being used in practice.

► The review is limited in scope to quantitative vali-dation and effectiveness studies, so must be con-sidered alongside wider literature reflecting on potential secondary benefits of early warning sys-tems and PTTT for communication, teamwork and empowerment.

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 2

2 Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

clearly multifactorial,1–3 but paediatric ‘early warning systems’ have been strongly advocated as one approach to improving recognition of deterioration in paediatric units.1 2 4

A paediatric ‘early warning system’ can be considered any patient safety initiative or programme which aims to monitor, detect and respond to signs of deterioration in hospitalised children in order to avert adverse outcomes and premature death. Such systems are often multifac-eted and may include the use of rapid response teams (RRT) or medical emergency teams (MET), education or training to improve clinical staff’s ability to identify dete-rioration or strategies aimed at improving staff communi-cation and situational awareness.

An increasingly commonplace paediatric ‘early warning system’ initiative is the use of a ‘track and trigger tool’: these tools, also commonly used in adult care, provide a formal framework for evaluating routine physiological, clinical and observational data for early indicators of patient deterioration. They are typically integrated into routine observation charts or electronic health records and compare patient observations with predefined ‘normal’ thresholds. When one or more observation is considered abnormal, staff are directed to various clinical actions, including but not limited to altered frequency of observations, review by senior staff or more appropriate treatment or management. Tools may be paper based or electronic and monitoring may be automated or manu-ally undertaken by staff.

These tools have been referred to in the literature using a number of different terms: paediatric early warning scores (PEWS); paediatric early warning tools (PEWT), track and trigger tools (TTT) and many others. Here, we refer to the tools themselves using the term ‘paediatric track and trigger tools’ (PTTT). A variety of PTTT have been developed, typically by teams based in specialist paediatric centres and often used as a means of triggering a dedicated response team. Their advocacy has recently led to widespread uptake across a variety of different paediatric units, including many non-specialist centres where patient populations and resources may differ. In the UK, a recent cross-sectional survey found that 85% of paediatric units were using some form of PTTT, most of which were non-specialist centres without a dedicated response team.5 Despite their widespread use, recent reviews have questioned the evidence base for their effec-tiveness in improving patient outcomes.6 7 The current review aimed to build on this work, assessing in depth the evidence base for both the validity of PTTT for predicting in-patient deterioration and the effectiveness of broader ‘early warning systems’ at reducing instances of mortality and morbidity in paediatric settings:

► Question 1: how well validated are existing PTTT and their component parts for predicting inpatient deterioration?

► Question 2: how effective are paediatric early warning systems (with or without a PTTT) at reducing mortality and critical events?

MEthODsThis systematic review is reported in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines.8 Our review protocol is registered with the PROSPERO database CRD42015015326.

search strategyA comprehensive search was conducted across a range of databases to identify relevant studies in the English language. Published and unpublished literature was considered where publicly available, as were studies in press. The following databases were searched through May 2018: British Nursing Index, Cumulative Index of Nursing and Allied Health Literature, Cochrane Central Register of Controlled Trials, Database of Abstracts of Reviews of Effectiveness, EMBASE, Health Management Information Centre, Medline, Medline in Process, Scopus and Web of Knowledge (Science Citation Indexes). To identify additional papers, published, unpublished or research reported in the grey literature, a range of rele-vant websites and trial registers were searched including Clinical Trials. gov. To identify published papers that had not yet been catalogued in the electronic databases, recent editions of key journals were hand-searched. The search terms included ‘early warning scores’, ‘alert criteria’, ‘rapid response’, ‘track and trigger’ and ‘early medical intervention’ (see online supplementary table 1).

Eligibility screening and study selectionPICOS parameters guided inclusion criteria for the vali-dation and effectiveness studies (see online supplemen-tary table 2). Papers reporting development of validation of a PTTT were included for question 1, whereas papers reporting the implementation of any broader ‘paediatric early warning system’ (with or without a PTTT) were eligible for question 2. Both research questions were limited to studies that involved inpatients aged 0–18 years. Outcome measures considered were mortality and crit-ical events, including: unplanned admission to a higher level of care, cardiac arrest, respiratory arrest, medical emergencies requiring immediate assistance, children reviewed by paediatric intensive care unit (PICU) staff on the ward (in specialist centres) or reviewed by external PICU staff (for non-specialist centres), acuity at PICU admission and PICU outcomes. A range of study designs were considered for both questions.

Two of the review authors independently screened the titles and abstracts yielded in the search. Full texts were reviewed independently by six reviewers against the above eligibility criteria and were assigned to the relevant review question if included. Reasons for exclusion were recorded. Separate data extraction forms were developed for validation and effectiveness studies. The forms had common elements (study design, country, setting, study population, description of the PTTT or early warning system, statistical techniques used, outcomes assessed). Additional data items for validation studies included

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 3

3Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

the items in the PTTT, modifications to the PTTT from previous versions, predictive ability of individual items and the overall tool, sensitivity and specificity and inter-rater and intra-rater reliability. Effectiveness studies included an assessment of outcomes in terms of mortality and various morbidity variables. Data extraction was carried out by two reviewers and discrepancies were resolved by discussion. For effectiveness studies, effect sizes and 95% CIs were calculated or reported as risk ratios (RR) or ORs as appropriate, with p values reported to assess statistical significance. Data analysis was conducted using an online medical statistics tool.

Quality appraisalMethodological quality and risk of bias was assessed for each included study using a modified version of the Downs and Black rating scale9 (templates shown in online supplementary table 3).

Patient and public involvementThis review was conducted as part of a larger mixed-methods study (ISRCTN94228292), which used a formal, facilitated parental advisory group. The group comprised parents of children who had experienced an unexpected adverse event in a paediatric unit and provided input which helped to shape the broader research questions and outcome measures. The results of the review will be disseminated to parents through this group.

rEsultsFigure 1 shows the PRISMA flow diagram for both research questions.

study characteristicsTable 1 summarises the study characteristics of validation and effectiveness papers in the review.

types of Ptts and componentsAcross 66 studies, we identified 27 unique PTTT (table 2). Twenty PTTTs were based on one of four different tools: Monaghan’s Brighton PEWS,10 the Bedside PEWS,11 the Bristol PEWT12 and the Melbourne Activation Criteria (MAC).13 Other PTTT described in the literature included the National Health Service Institute for Inno-vation and Improvement (NHS III) PEWS14 (the second most commonly used PTTT in UK paediatric settings5), RRT and MET activation criteria15–18 and one prediction algorithm developed from a large dataset of electronic health data.19

Table 2 illustrates the range of physiological and behavioural parameters underpinning PTTT. Common parameters included heart rate (present in 26 out of 27 PTTT), respiratory rate (24), respiratory effort (24) and level of consciousness or behavioural state (24). All PTTT required at least six different parameters to be collected.

Question 1: how well validated are Pttt and component parts for predicting inpatient deterioration?Nine validation papers meeting inclusion criteria were excluded from analysis: eight did not report any

performance characteristics of the PTTT for predicting deterioration20–27 and one study calculated incorrect sensitivity/specificity outcomes12 (see online supplemen-tary table 4). The remaining 27 validation studies, evalu-ating the performance of 18 unique PTTT, are described in table 3. Four studies evaluated multiple PTTTs3 19 28 29 and one paper described three separate studies of the same PTTT.30

Five cohort studies were included,14 31–34 three based on the same dataset. All other studies were either case-control or chart reviews. Thirteen papers implemented the PTTT in practice,23 30 31 34–43 while the remaining studies ‘bench tested’ the PTTT—researchers retrospectively calculated the score based on data abstracted from medical charts and records. All studies were conducted in specialist centres with only one multicentre study reported.44

Outcome measuresPTTT were evaluated for their ability to predict a wide range of clinical outcomes. Composite measures were used in 8 studies,14 23 29 32 33 37 45 46 cardiac/respiratory arrest or a ‘code call’ was used (singularly or part of a composite outcome) in 6 studies,23 28 29 37 45 47 while 22 studies used transfer to a to PICU or paediatric high-dependency unit as the main outcome.3 11 19 23 28–34 36 37 39 41–44 46 48 49

Predictive ability of individual Pttt componentsThree validation papers reported on the performance characteristics of individual components of the tool for predicting adverse outcomes.11 33 42 Parshuram et al, for instance, reported area under the receiver operating characteristic curve (AUROC) values for individual PTTT items of a pilot version of the Bedside PEWS: ranging from 0.54 (bolus fluid) to 0.81 (heart rate), compared with 0.91 for the overall PTTT.11 All other studies reported outcomes for the PTTT as a whole.

PEWs scoreThe predictive ability of the 16-item PEWS score was assessed by one internal47 (AUROC=0.90) and two external case-control studies28 29 (AUROC range=0.82–0.88) with a range of outcome measures and scoring thresholds. One case-control study used an observed prev-alence rate to calculate a positive predictive value (PPV) of 4.2% for the tool in predicting code calls47 (for every 1000 patients triggering the PTTT, 42 would be expected to deteriorate).

bedside PEWs and derivativesThe Bedside PEWS was evaluated in one internal11 (AUROC=0.91) and five external case-control studies19 28 29 44 46 (AUROC range=0.73–0.90) for a range of different outcome measures and at different scoring thresholds. One case-control study calculated a PPV of 2.1% for identifying children requiring urgent PICU transfer within 24 hours of admission, based on locally observed prevalence rates.19 A modified version of the Bedside PEWS (with temperature added) demonstrated an AUROC of 0.86 in an external case-control study with

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 4

4 Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

a composite outcome of death, arrest or unplanned PICU transfer.29

brighton PEWs and derivativesSix different PTTT based on the original Brighton PEWS were evaluated across 11 studies.19 29 31 37 39–42 45 48 50 The Modified Brighton PEWS (a) was evaluated for its ability to predict PICU transfers in one large prospective cohort study (AUROC=0.92, PPV=5.8%),31 and an external case-control study tested the same score for predicting

urgent PICU transfers within 24 hours of admission (AUROC=0.74, PPV=2.1%).19

An external case-control study used a composite measure of death, arrest or PICU transfer to evaluate the Modified Brighton PEWS (b) (AUROC=0.79) and the Modified Brighton PEWS (d) (AUROC=0.74).29 The latter tool was evaluated in a further internal case-control study for predicting PICU transfer (AUROC=0.82).48

Figure 1 Preferred Reporting Items for Systematic Review and Meta-Analyses flow diagram of study inclusion. PEWS, paediatric early warning scores.

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 5

5Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

The Children’s Hospital Early Warning Score (CHEWS) had a reported AUROC of 0.90 for predicting PICU transfers or arrests in a large internal case-control study.50 A modification for cardiac patients, the Cardiac CHEWS (C-CHEWS) was evaluated by one internal study on a cardiac unit37 (AUROC=0.90) looking at arrests or unplanned PICU transfers, and two external studies of oncology/haematology units41 42 for the same outcome (AUROC=0.95). Finally, the Children’s Hospital

Los Angeles PEWS was evaluated by in a small internal case-control study for prediction of re-admission to PICU after initial PICU discharge40 (AUROC=0.71).

MAC and derivativesThe MAC was assessed by one external case-control study with an outcome of death, arrest or unplanned PICU transfer29 (AUROC=0.71) and a large external cohort study with an outcome of death or unplanned

Table 1 Summary of the study characteristics of the 36 validation (question 1) and 30 effectiveness (question 2) papers included in the review

Validation studies (n=36) N (%) Effectiveness studies (n=30) n (%)

Type Type

Full text 22 (61.1) Full text 21 (70.0)

Abstract 14 (38.9) Abstract 9 (30.0)

Country Country

USA 15 (41.7) USA 18 (60.0)

UK 12 (33.3) UK 3 (10.0)

Canada 2 (5.5) Canada 2 (6.7)

Australia 0 (0.0) Australia 3 (10.0)

Other 5 (13.9) Other 3 (10.0)

Multiple 1 (2.8) Multiple 1 (3.3)

Unclear 1 (2.8) Unclear 0 (0.0)

Year of study Year of study

Pre-2012 10 (27.8) Pre-2012 15 (50.0)

2012 3 (8.3) 2012 1 (3.3)

2013 6 (16.7) 2013 2 (6.7)

2014 5 (13.9) 2014 6 (20.0)

2015 7 (19.4) 2015 0 (0.0)

2016 2 (5.6) 2016 2 (6.7)

2017 3 (8.3) 2017 1 (3.3)

2018 0 (0.0) 2018 3 (10.0)

Setting Setting

Specialist/tertiary 33 (91.7) Specialist/tertiary 29 (96.7)

Non-specialist/community 0 (0.0) Non-specialist/community 1 (3.3)

Unclear 3 (8.3) Unclear 0 (0.0)

Single-centre/multicentre Single-centre/multicentre

Single-centre 35 (97.2) Single-centre 28 (93.3)

Multicentre 1 (2.8) Multicentre 2 (6.7)

Study population Study population

General inpatients 23 (63.9) General inpatients 20 (66.6)

Specialist population 11 (30.6) Specialist population 5 (16.7)

Unclear 2 (5.6) Unclear 5 (16.7)

Study design Study design

Case-control 18 (50.0) Uncontrolled before-after 26 (86.7)

Case/chart review 10 (27.8) Controlled before-after 1 (3.3)

Cohort 7 (19.4) Interrupted time series 2 (6.7)

Pilot study 1 (2.8) Cluster randomised trial 1 (3.3)

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 6

6 Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

Tab

le 2

S

umm

ary

of P

TTTs

PT

TT

nam

e (r

efer

ence

s)D

evel

op

men

t/m

od

ifica

tio

n d

etai

lsS

core

/tr

igg

er

Cho

ice

of

thre

sho

lds/

par

amet

ers

Ag

e-d

epen

den

t th

resh

old

s

No

. of

item

s in

th

e to

ol*

PT

TT

par

amet

ers

Res

pir

ato

ry

rate

Hea

rt

rate

Res

pir

ato

ry

effo

rt/

dis

tres

sLO

C/

beh

avio

urO

xyg

en

satu

rati

on

Cap

illar

y re

fill

tim

eO

xyg

en

ther

apy

Sys

tolic

b

loo

d

pre

ssur

eP

ain

Sta

ff

conc

ern

Ski

n co

lour

Air

way

p

rob

lem

sTe

mp

erat

ure

Pul

ses

Fam

ily

conc

ern

Oth

er it

ems

Pae

dia

tric

ear

ly w

arni

ng s

yste

m (P

EW

S) s

core

and

der

ivat

ives

PE

WS

sco

re28

47

Dev

elop

ed fo

r us

e in

Can

adia

n te

rtia

ry

cent

re.47

Nur

se-g

ener

ated

ca

ndid

ate

item

s re

duc

ed

by

focu

s gr

oup

s/D

elp

hi

and

eva

luat

ion

with

clin

ical

d

atas

et (c

ode

blu

e ca

lls,

n=87

; con

trol

s, n

=12

8).

Dev

elop

men

t an

d v

alid

atio

n d

atas

ets

not

ind

epen

den

t.

Sco

reE

xper

t op

inio

nYe

s16

✓✓

✓✓

✓✓

✓✓

✓✓

Bol

us fl

uid

, m

edic

atio

ns,

hom

e ox

ygen

, any

p

revi

ous

adm

issi

on

to a

n IC

U, c

entr

al

veno

us li

ne in

situ

, tr

ansp

lant

rec

ipie

nt,

seve

re c

ereb

ral

pal

sy, g

astr

osto

my

tub

e, g

reat

er t

han

thre

e m

edic

al

spec

ialti

es in

volv

ed

in c

are.

Bed

sid

e P

EW

S11

19

25 2

6 28

44

46 5

9 64

65

70D

evel

oped

for

use

in U

S

tert

iary

cen

tre.

11 R

outin

ely

colle

cted

item

s as

sess

ed

for

dis

crim

inat

ory

abili

ty

usin

g cl

inic

al d

atas

et (P

ICU

ad

mis

sion

, n=

60; c

ontr

ols,

n=

120)

. Dev

elop

men

t an

d v

alid

atio

n se

t no

t in

dep

end

ent.

Sco

reE

xper

t op

inio

nYe

s7

✓✓

✓✓

✓✓

✓

Mod

ified

Bed

sid

e P

EW

S (a

)30M

odifi

catio

n to

Bed

sid

e P

EW

S fo

r us

e in

Dut

ch

tert

iary

cen

tre.

Ad

ded

te

mp

erat

ure;

mod

ified

w

ord

ing

of r

esp

irato

ry e

ffort

an

d o

xyge

n th

erap

y ite

ms.

Sco

reE

xper

t op

inio

nYe

s8

✓✓

✓✓

✓✓

✓✓

Mod

ified

Bed

sid

e P

EW

S (b

)49M

odifi

catio

n to

Bed

sid

e P

EW

S fo

r us

e in

US

ter

tiary

ce

ntre

. Cha

nged

nor

mal

th

resh

old

s fo

r H

R a

nd R

R

bas

ed o

n an

alys

is o

f loc

al

clin

ical

dat

a.

Sco

reH

R/R

R d

ata

driv

enYe

s7

✓✓

✓✓

✓✓

✓

Bri

ght

on

PE

WS

and

der

ivat

ives

Brig

hton

PE

WS

10 5

4In

itial

dev

elop

men

t fo

r us

e in

UK

ter

tiary

cen

tre.

A

dap

ted

from

exi

stin

g ad

ult

scor

es, b

ut a

men

ded

bas

ed

on lo

cal c

linic

al c

onse

nsus

. S

mal

l aud

it of

pat

ient

s (n

=30

) des

crib

ed b

ut n

o fo

rmal

val

idat

ion.

Sco

reE

xper

t op

inio

nN

o5

✓✓

✓✓

✓✓

✓Q

uart

er h

ourly

ne

bul

iser

s,

per

sist

ent

vom

iting

p

osts

urge

ry.

Mod

ified

Brig

hton

P

EW

S (a

)19 3

1 39

Mod

ifica

tion

of B

right

on

PE

WS

for

use

in g

ener

al

med

ical

war

d o

f a U

S

tert

iary

cen

tre.

Alte

red

th

resh

old

s fo

r ox

ygen

th

erap

y; c

hang

ed w

ord

ing

for

resp

irato

ry e

ffort

; m

odifi

ed e

scal

atio

n al

gorit

hm.

Sco

reE

xper

t op

inio

nN

o5

✓✓

✓✓

✓✓

✓Q

uart

er h

ourly

ne

bul

iser

s,

per

sist

ent

vom

iting

p

osts

urge

ry.

Mod

ified

Brig

hton

P

EW

S (b

)45 7

2M

odifi

catio

n of

Brig

hton

P

EW

S fo

r us

e in

US

ter

tiary

ce

ntre

. Ad

ded

age

-d

epen

den

t th

resh

old

s fo

r H

R a

nd R

R.

Sco

reE

xper

t op

inio

nYe

s5

✓✓

✓✓

✓✓

✓Q

uart

er h

ourly

ne

bul

iser

s,

per

sist

ent

vom

iting

p

osts

urge

ry.

Con

tinue

d

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 7

7Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

PT

TT

nam

e (r

efer

ence

s)D

evel

op

men

t/m

od

ifica

tio

n d

etai

lsS

core

/tr

igg

er

Cho

ice

of

thre

sho

lds/

par

amet

ers

Ag

e-d

epen

den

t th

resh

old

s

No

. of

item

s in

th

e to

ol*

PT

TT

par

amet

ers

Res

pir

ato

ry

rate

Hea

rt

rate

Res

pir

ato

ry

effo

rt/

dis

tres

sLO

C/

beh

avio

urO

xyg

en

satu

rati

on

Cap

illar

y re

fill

tim

eO

xyg

en

ther

apy

Sys

tolic

b

loo

d

pre

ssur

eP

ain

Sta

ff

conc

ern

Ski

n co

lour

Air

way

p

rob

lem

sTe

mp

erat

ure

Pul

ses

Fam

ily

conc

ern

Oth

er it

ems

Mod

ified

Brig

hton

P

EW

S (c

)52M

odifi

catio

n of

Brig

hton

P

EW

S fo

r us

e in

a U

S

haem

atol

ogy/

onco

logy

uni

t.

Alte

red

thr

esho

lds;

cha

nged

re

spira

tory

effo

rt w

ord

ing;

m

odifi

ed e

scal

atio

n al

gorit

hm; a

dd

ed a

nd

rem

oved

item

s. N

o fo

rmal

va

lidat

ion

stud

y re

por

ted

.

Sco

reE

xper

t op

inio

nN

o3

✓✓

✓✓

✓✓

✓✓

Mod

ified

Brig

hton

P

EW

S (d

)48M

odifi

catio

n of

Brig

hton

P

EW

S fo

r us

e in

a U

S

tert

iary

cen

tre.

Mod

ified

wor

din

g of

b

ehav

iour

com

pon

ent,

ad

ded

age

-dep

end

ent

thre

shol

ds

for

HR

and

RR

; re

mov

ed n

ebul

iser

s an

d

per

sist

ent

vom

iting

.

Sco

reE

xper

t op

inio

nYe

s3

✓✓

✓✓

✓✓

✓

Mod

ified

Brig

hton

P

EW

S (e

)71M

odifi

catio

n of

Brig

hton

P

EW

S fo

r us

e in

a U

S

tert

iary

cen

tre.

Mod

ified

wor

din

g of

b

ehav

iour

and

res

pira

tory

ef

fort

item

s; a

ltere

d

thre

shol

ds

for

O2

ther

apy;

re

mov

ed n

ebul

iser

s an

d

per

sist

ent

vom

iting

item

s.

No

form

al v

alid

atio

n st

udy

rep

orte

d.

Sco

reE

xper

t op

inio

nN

o3

✓✓

✓✓

✓✓

✓

Texa

s C

hild

ren’

s H

osp

ital P

EW

S22

Mod

ifica

tion

of B

right

on

PE

WS

for

use

in a

US

te

rtia

ry c

entr

e.M

odifi

ed w

ord

ing

of

beh

avio

ur c

ateg

ory;

ad

ded

sc

orin

g ite

ms

to r

esp

irato

ry

and

car

dio

vasc

ular

ca

tego

ries;

cha

nged

O

2 th

erap

y th

resh

old

s;

mod

ified

esc

alat

ion

algo

rithm

.

Sco

reE

xper

t op

inio

nN

o5

✓✓

✓✓

✓✓

✓✓

Hou

rly r

esp

irato

ry

trea

tmen

ts;

per

sist

ent

vom

iting

p

osts

urge

ry.

Chi

ldre

n’s

Hos

pita

l E

arly

War

ning

S

core

50

Mod

ifica

tion

of B

right

on

PE

WS

for

use

in a

US

te

rtia

ry c

entr

e. A

ltere

d

thre

shol

ds

for

O2

ther

apy;

ch

ange

d w

ord

ing

for

beh

avio

ur a

nd r

esp

irato

ry

cate

gorie

s; a

dd

ed s

taff

and

fa

mily

con

cern

; rem

oved

ne

bul

iser

s an

d v

omiti

ng;

mod

ified

esc

alat

ion

algo

rithm

.

Sco

reE

xper

t op

inio

nN

o5

✓✓

✓✓

✓✓

✓✓

✓

Chi

ldre

n’s

Hos

pita

l Car

dia

c E

arly

War

ning

S

core

23 4

1 42

67

Mod

ifica

tion

of B

right

on

PE

WS

for

card

iac

war

d

of a

US

ter

tiary

cen

tre.

A

ltere

d O

2 th

erap

y th

resh

old

s; a

dd

ed it

ems

to

beh

avio

ur, r

esp

irato

ry a

nd

card

iova

scul

ar c

ateg

orie

s;

add

ed fa

mily

and

sta

ff co

ncer

n; a

dd

ed a

ge-r

elat

ed

thre

shol

ds;

rem

oved

ne

bul

iser

s an

d v

omiti

ng

item

s; m

odifi

ed e

scal

atio

n al

gorit

hm.

Sco

reE

xper

t op

inio

nYe

s5

✓✓

✓✓

✓✓

✓✓

✓✓

✓✓

Tab

le 2

C

ontin

ued

Con

tinue

d

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 8

8 Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

PT

TT

nam

e (r

efer

ence

s)D

evel

op

men

t/m

od

ifica

tio

n d

etai

lsS

core

/tr

igg

er

Cho

ice

of

thre

sho

lds/

par

amet

ers

Ag

e-d

epen

den

t th

resh

old

s

No

. of

item

s in

th

e to

ol*

PT

TT

par

amet

ers

Res

pir

ato

ry

rate

Hea

rt

rate

Res

pir

ato

ry

effo

rt/

dis

tres

sLO

C/

beh

avio

urO

xyg

en

satu

rati

on

Cap

illar

y re

fill

tim

eO

xyg

en

ther

apy

Sys

tolic

b

loo

d

pre

ssur

eP

ain

Sta

ff

conc

ern

Ski

n co

lour

Air

way

p

rob

lem

sTe

mp

erat

ure

Pul

ses

Fam

ily

conc

ern

Oth

er it

ems

Bur

n-sp

ecifi

c P

EW

S24

Mod

ifica

tion

of B

right

on

PE

WS

, for

use

in a

sp

ecia

list

Bur

n C

entr

e of

a

US

ter

tiary

cen

tre.

Ad

ded

te

mp

erat

ure;

ad

ded

inta

ke

and

out

put

sco

ring

item

s;

add

ed s

kin

com

pon

ent.

Sco

reE

xper

t op

inio

nN

o6

✓✓

✓✓

✓✓

✓✓

✓In

take

; out

put

s;

skin

.

Chi

ldre

n’s

Hos

pita

l Lo

s A

ngel

es

PE

WS

40

Mod

ifica

tion

of B

right

on

PE

WS

for

use

in a

US

te

rtia

ry c

entr

e. A

dd

ed

med

ical

his

tory

sco

ring

item

; ad

ded

sin

gle

vent

ricle

p

hysi

olog

y sc

orin

g ite

m;

chan

ged

O2

ther

apy

thre

shol

ds;

ad

ded

item

s to

re

spira

tory

cat

egor

y.

Sco

reE

xper

t op

inio

n4

✓✓

✓✓

✓✓

✓R

RT,

cod

e b

lue

or t

rans

fer

from

/to

PIC

U in

p

ast

2 w

eeks

; si

ngle

ven

tric

le

phy

siol

ogy;

any

as

sist

ed v

entil

atio

n.

Mel

bo

urne

Act

ivat

ion

Cri

teri

a (M

AC

) and

der

ivat

ives

MA

C3

13 3

3 62

Initi

al d

evel

opm

ent

for

use

in a

n A

ustr

alia

n te

rtia

ry

cent

re t

o ac

tivat

e M

ET.

A

dap

ted

from

ad

ult

ME

T ca

lling

crit

eria

, usi

ng a

ge-

app

rop

riate

thr

esho

lds.

N

o fo

rmal

val

idat

ion

stud

y re

por

ted

.

Trig

ger

Exp

ert

opin

ion

Yes

9✓

✓✓

✓✓

✓✓

✓C

ard

iac

or

resp

irato

ry a

rres

t.

Mod

ified

MA

C63

Mod

ifica

tion

of M

AC

for

use

in a

Can

adia

n te

rtia

ry

cent

re, t

o ac

tivat

e an

R

RT.

Rem

oved

car

dia

c/re

spira

tory

arr

est

outc

ome.

N

o fo

rmal

val

idat

ion

stud

y re

por

ted

.

Trig

ger

Exp

ert

opin

ion

Yes

8✓

✓✓

✓✓

✓✓

✓

Car

diff

and

Val

e P

EW

S32

33

Mod

ifica

tion

of M

AC

for

eval

uatio

n in

a U

K t

ertia

ry

cent

re. R

emov

ed c

ard

iac/

resp

irato

ry a

rres

t ou

tcom

e;

alte

red

thr

esho

lds

of

som

e ite

ms;

eva

luat

ed a

s ag

greg

ate

scor

e ra

ther

tha

n si

ngle

-ite

m t

rigge

r.

Sco

reE

xper

t op

inio

nYe

s8

✓✓

✓✓

✓✓

✓✓

Bri

sto

l pae

dia

tric

ear

ly w

arni

ng t

oo

l (P

EW

T) a

nd d

eriv

ativ

es

Bris

tol

PE

WT3

12 2

8 34

35

Initi

al d

evel

opm

ent

for

use

in a

UK

ter

tiary

cen

tre.

Initi

al

cand

idat

e ite

ms

dra

wn

from

un

valid

ated

Ply

mou

th to

ol—

retr

osp

ectiv

ely

eval

uate

d

for

abili

ty t

o p

red

ict

adve

rse

even

ts a

mon

g ca

ses

(n=

360,

HD

U o

r P

ICU

tr

ansf

ers)

. Dev

elop

men

t an

d v

alid

atio

n d

atas

et n

ot

ind

epen

den

t.

Trig

ger

AP

LS v

alue

sYe

s14

✓✓

✓✓

✓✓

✓✓

✓✓

Req

uire

d n

ebul

ised

ep

inep

hrin

e;

hyp

erka

laem

ia;

susp

ecte

d

men

ingo

cocc

us;

dia

bet

ic

keto

acid

osis

; p

ersi

sten

t co

nvul

sion

.

Mod

ified

Bris

tol

PE

WT

(a)68

Mod

ifica

tion

of B

risto

l P

EW

T fo

r a

UK

ter

tiary

ce

ntre

. Ad

just

ed w

ord

ing

of

Airw

ay p

aram

eter

s; a

dd

ed

resp

irato

ry it

ems;

ad

ded

A

VP

U e

valu

atio

n; r

emov

ed

susp

ecte

d m

eing

ococ

cus

and

dia

bet

ic k

etoa

cid

osis

; ad

ded

pH

<7.

2 an

d

unre

solv

ed p

ain.

No

form

al

valid

atio

n st

udy

rep

orte

d.

Trig

ger

AP

LS v

alue

sYe

s14

✓✓

✓✓

✓✓

✓✓

✓✓

✓R

equi

red

neb

ulis

ed

epin

ephr

ine

or n

o im

pro

vem

ent

afte

r ne

bul

iser

s; p

H

<7.

2; u

nres

olve

d

pai

n or

cur

rent

an

alge

sic

ther

apy;

fit

ting.

Tab

le 2

C

ontin

ued

Con

tinue

d

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 9

9Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

PT

TT

nam

e (r

efer

ence

s)D

evel

op

men

t/m

od

ifica

tio

n d

etai

lsS

core

/tr

igg

er

Cho

ice

of

thre

sho

lds/

par

amet

ers

Ag

e-d

epen

den

t th

resh

old

s

No

. of

item

s in

th

e to

ol*

PT

TT

par

amet

ers

Res

pir

ato

ry

rate

Hea

rt

rate

Res

pir

ato

ry

effo

rt/

dis

tres

sLO

C/

beh

avio

urO

xyg

en

satu

rati

on

Cap

illar

y re

fill

tim

eO

xyg

en

ther

apy

Sys

tolic

b

loo

d

pre

ssur

eP

ain

Sta

ff

conc

ern

Ski

n co

lour

Air

way

p

rob

lem

sTe

mp

erat

ure

Pul

ses

Fam

ily

conc

ern

Oth

er it

ems

Mod

ified

Bris

tol

PE

WT

(b)38

Mod

ifica

tion

of B

risto

l P

EW

T fo

r ca

rdia

c w

ard

of

a U

K t

ertia

ry c

entr

e.

Am

end

ed H

R a

nd R

R

thre

shol

ds.

Ad

just

ed

wor

din

g of

airw

ay

par

amet

ers;

ad

ded

re

spira

tory

item

s; a

dd

ed

AV

PU

eva

luat

ion;

rem

oved

su

spec

ted

mei

ngoc

occu

s an

d d

iab

etic

ket

oaci

dos

is;

add

ed p

H <

7.2

and

un

reso

lved

pai

n.

Trig

ger

HR

/RR

dat

a d

riven

Yes

14✓

✓✓

✓✓

✓✓

✓✓

✓✓

Req

uire

d n

ebul

ised

ep

inep

hrin

e or

no

imp

rove

men

t af

ter

neb

ulis

ers;

pH

<

7.2;

unr

esol

ved

p

ain

or c

urre

nt

anal

gesi

c th

erap

y;

fittin

g.

Oth

er P

TT

T

NH

S In

stitu

te fo

r In

nova

tion

and

Im

pro

vem

ent

PE

WS

14

Des

igne

d a

s p

art

of a

NH

S

Inst

itute

fello

wsh

ip p

roje

ct.

Ad

apte

d fr

om a

dul

t sc

ores

an

d B

right

on P

EW

S.

No

form

al d

evel

opm

ent

or

inte

rnal

val

idat

ion

stud

y p

ublis

hed

.

Sco

reA

PLS

val

ues

Yes

6✓

✓✓

✓✓

✓

Pae

dia

tric

med

ical

em

erge

ncy

team

(P

ME

T) t

rigge

ring

crite

ria (a

)15

Initi

al d

evel

opm

ent

for

use

in a

US

ter

tiary

cen

tre

to

activ

ate

a M

ET.

Ret

rosp

ectiv

e ch

art

revi

ew

of c

ase

pat

ient

s (n

-44,

cod

e ca

lls) u

sed

to

gene

rate

ca

ndid

ate

item

s. C

linic

al

jud

gem

ent

used

to

sele

ct

final

item

s. N

o fo

rmal

va

lidat

ion

of fi

nal t

ool

rep

orte

d.

Trig

ger

Exp

ert

opin

ion

No

4✓

✓✓

✓✓

✓W

orse

ning

re

trac

tions

; cy

anos

is.

PM

ET

trig

gerin

g cr

iteria

(b)16

Initi

al d

evel

opm

ent

for

use

in a

US

ter

tiary

cen

tre

to

activ

ate

a M

ET.

Min

imal

des

crip

tion

of t

ool

dev

elop

men

t—au

thor

s d

elib

erat

ely

chos

e b

road

cr

iteria

and

cat

egor

ies

of

illne

ss r

athe

r th

an s

pec

ific

vita

l sig

ns.

No

form

al v

alid

atio

n st

udy

rep

orte

d.

Trig

ger

Exp

ert

opin

ion

Unc

lear

12✓

✓✓

✓✓

✓✓

Car

dia

c or

re

spira

tory

ar

rest

; sei

zure

s w

ith a

pno

ea;

pro

gres

sive

le

thar

gy; c

ircul

ator

y co

mp

rom

ise/

acut

e sh

ock

synd

rom

e.

Pae

dia

tric

rap

id

resp

onse

tea

m

(PR

RT)

trig

gerin

g cr

iteria

(a)17

Initi

al d

evel

opm

ent

for

use

in a

US

ter

tiary

cen

tre,

to

activ

ate

an R

RT.

Trig

gerin

g ite

ms

elec

ted

th

roug

h ex

per

t co

nsen

sus

loca

lly—

refe

renc

e to

si

mila

rity

to M

AC

and

PM

ET

trig

gerin

g cr

iteria

(a).

No

form

al v

alid

atio

n st

udy

rep

orte

d.

Trig

ger

Exp

ert

opin

ion

No

6✓

✓✓

✓✓

✓

PR

RT

trig

gerin

g cr

iteria

(b)18

Initi

al d

evel

opm

ent

for

use

in c

allin

g R

RT

team

in a

te

rtia

ry c

entr

e in

Pak

ista

n.

Min

imal

exp

lana

tion

for

sele

ctio

n of

cal

ling

crite

ria.

No

form

al v

alid

atio

n st

udy

rep

orte

d in

the

lite

ratu

re.

Trig

ger

Unc

lear

Yes

8✓

✓✓

✓✓

✓✓

Con

vuls

ion.

Tab

le 2

C

ontin

ued

Con

tinue

d

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 10

10 Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

PICU or HDU transfer33 (AUROC=0.79, PPV=3.6%). A derivative of the MAC using an aggregate score, the Cardiff and Vale PEWS (C&VPEWS), was tested using the same cohort and outcome measures in an earlier external study (AUROC=0.86, PPV=5.9%)32 and was the best performing PTTT in an external case-control study evaluating multiple PTTT29 (AUROC=0.89).

bristol PEWtThe Bristol PEWT was evaluated by five external valida-tion studies: two chart review studies3 35 (no AUROC), one small cohort study of PICU transfers34 (AUROC=0.91, PPV=11%), and two case-control studies looking at code calls28 (AUROC=0.75) and a composite of death, arrests and PICU transfers29 (AUROC=0.62).

Other PtttThe NHS III PEWS was tested by one external cohort study looking at a composite of death or unplanned transfers to PICU or HDU14 (AUROC=0.88, PPV=4.3%) and one external case-control study looking at a composite of death, arrests and PICU transfers29 (AUROC=0.82). Zhai et al developed and retrospectively evaluated a logistic regression algo-rithm in an internal case-control study looking at urgent PICU transfers in the first 24 hours of admis-sion19 (AUROC=0.91, PPV=4.8%).

Across PTTT, studies reporting performance charac-teristics of a tool at a range of different scoring thresh-olds demonstrate the expected interaction and trade-off between sensitivity and specificity—at lower triggering thresholds, sensitivity is high but specificity is low; at higher thresholds, the opposite is true.

Inter-rater reliability and completeness of dataAccurate assessment of the ability of a PTTT to predict clinical deterioration is contingent on accuracy and reliability of tool scoring (whether by bedside nurses in practice or by researchers abstracting data) and the availability of underpinning observations. Only five papers made reference to accuracy or reliability of scoring,28 31 37 42 45 with mixed results: for example, two nurses separately scoring a subset of patients on the Modified Brighton PEWS (a) achieved an intra-class coefficient of 0.92,31 but a study nurse and bedside nurse achieved only 67% agreement in scoring the C-CHEWS tool.37 Completeness of data was reported in 11 studies.11 14 19 29 30 32 33 42 44 45 47 An evaluation of the Modified Bedside PEWS (a) reported that ‘the PEWS was correctly performed and could be used for inclusion in the study’ in 59% of cases,30 a prospective study bench-testing the C&VPEWS found an average completeness rate of 44% for the seven different parameters in daily practice,32 while a multicentre study of the Bedside PEWS reported that ‘only 5.1% (of observation sets) had measurements on all seven items'.44P

TT

T n

ame

(ref

eren

ces)

Dev

elo

pm

ent/

mo

difi

cati

on

det

ails

Sco

re/

trig

ger

Cho

ice

of

thre

sho

lds/

par

amet

ers

Ag

e-d

epen

den

t th

resh

old

s

No

. of

item

s in

th

e to

ol*

PT

TT

par

amet

ers

Res

pir

ato

ry

rate

Hea

rt

rate

Res

pir

ato

ry

effo

rt/

dis

tres

sLO

C/

beh

avio

urO

xyg

en

satu

rati

on

Cap

illar

y re

fill

tim

eO

xyg

en

ther

apy

Sys

tolic

b

loo

d

pre

ssur

eP

ain

Sta

ff

conc

ern

Ski

n co

lour

Air

way

p

rob

lem

sTe

mp

erat

ure

Pul

ses

Fam

ily

conc

ern

Oth

er it

ems

Logi

stic

reg

ress

ion

algo

rithm

19In

itial

dev

elop

men

t b

ased

on

dat

a m

inin

g of

ele

ctro

nic

heal

th r

ecor

ds

in U

S t

ertia

ry

cent

re. E

xtra

cted

24

hour

s of

clin

ical

dat

a fr

om

inp

atie

nts

(n=

6722

con

trol

s,

526

PIC

U t

rans

fers

) and

us

ed lo

gist

ic r

egre

ssio

n m

odel

to

sele

ct 2

9 ite

m

tool

. Val

idat

ion

per

form

ed

on s

ubse

t of

dev

elop

men

t d

atas

et.

Sco

reE

xper

t op

inio

nYe

s29

✓✓

✓✓

✓✓

✓✓

✓A

cuity

leve

l (lo

cal

mea

sure

); tis

sue

per

fusi

on a

nd

oxyg

enat

ion.

*Mul

tiple

par

amet

ers

are

ofte

n re

qui

red

to

be

colle

cted

for

each

sco

ring

item

/cat

egor

y, e

g, s

corin

g th

e ‘c

ard

iova

scul

ar’ c

ateg

ory

in t

he B

right

on P

EW

S r

equi

res

colle

ctio

n/ev

alua

tion

of H

R, s

kin

colo

ur a

nd c

apill

ary

refil

l tim

e.†D

enot

es a

stu

dy

incl

uded

in t

he e

ffect

iven

ess

revi

ew.

AP

LS, a

dva

nced

pae

dia

tric

life

sup

por

t; A

VP

U, a

lert

, voi

ce, p

ain,

unr

esp

onsi

ve; H

DU

, hig

h-d

epen

den

cy u

nit;

HR

, hea

rt r

ate;

LO

C, l

evel

of c

onsc

ious

ness

; NH

S, N

atio

nal H

ealth

Ser

vice

; PIC

U, p

aed

iatr

ic in

tens

ive

care

uni

t; P

TTS

, pae

dia

tric

tra

ck a

nd t

rigge

r to

ol; R

R, r

esp

irato

ry r

ate;

RR

T, r

apid

res

pon

se

team

.

Tab

le 2

C

ontin

ued

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 11

11Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

Tab

le 3

S

umm

ary

of P

TTT

valid

atio

n st

udy

outc

omes

PT

TT

Firs

t au

tho

r, ye

arC

oun

try

Stu

dy

po

pul

atio

nS

tud

y d

esig

nN

umb

er o

f ce

ntre

sP

TT

T u

sed

in

pra

ctic

e?

Inte

rnal

/ex

tern

al v

alid

atio

n st

udy?

Out

com

e m

easu

res

Sam

ple

siz

eS

core

or

trig

ger

?

Sco

re

test

ed/

max

imum

sc

ore

Whi

ch s

core

use

d

(fre

que

ncy

of

sco

ring

)?*

AU

RO

CS

ensi

tivi

tyS

pec

ifici

tyP

PV

NP

V

No

tes

on

accu

racy

/re

liab

ility

of

sco

ring

and

m

issi

ng d

ata

Qua

lity

sco

re

(max

=24

)

Pae

dia

tric

ear

ly

war

ning

sys

tem

(P

EW

S) s

core

Dun

can

2006

47C

anad

aA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oIn

tC

ode

blu

e ca

ll fo

r ac

tual

or

imp

end

ing

card

iop

ulm

onar

y ar

rest

215

(87

case

s)S

5/26

Max

24

hour

s b

efor

e ev

ent

(hou

rly)

0.90

78.0

95.0

4.2†

No

det

ails

on

dat

a ab

stra

ctio

n.13

% o

f elig

ible

cas

es a

nd

84%

of e

ligib

le c

ontr

ols

excl

uded

due

to

inco

mp

lete

cl

inic

al d

ata.

14

Rob

son

2013

28U

SA

All

inp

atie

nts

Cas

e-co

ntro

l st

udy

(retr

osp

ectiv

e)

1N

oE

xtC

ode

blu

e ca

ll19

2 (9

6 ca

ses)

S5/

32M

ax 2

4 ho

urs

bef

ore

even

t (s

ix

hour

ly)

0.85

86.6

72.2

Four

res

earc

hers

sco

red

P

TTT

from

20

char

ts,

inte

r-ra

ter

relia

bili

ty o

f 0.9

5.

No

det

ails

on

exte

nt o

f m

issi

ng d

ata.

8

Cha

pm

an

2017

29U

KA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oE

xtD

eath

, arr

est

or

unp

lann

ed P

ICU

tr

ansf

er

608

(297

ca

ses)

S7/

32M

ax 4

8 ho

urs

bef

ore

even

t (p

er

usua

l pra

ctic

e)

0.82

70.0

75.0

72.6

72.0

Dat

a ab

stra

ctio

n b

y si

ngle

res

earc

her.

36%

of

ob

serv

atio

n se

ts

cont

aine

d H

R, R

R, O

2 S

ats,

sy

stol

ic B

P, t

emp

erat

ure

and

ass

essm

ent

of

cons

ciou

snes

s.

17

Bed

sid

e P

EW

SP

arsh

uram

20

0911

Can

ada

All

inp

atie

nts

Cas

e-co

ntro

l st

udy

(retr

osp

ectiv

e)

1N

oIn

tU

rgen

t P

ICU

tr

ansf

er (w

ithou

t co

de

blu

e ca

ll)

180

(60

case

s)S

8/26

Max

24

hour

s b

efor

e ev

ent

(hou

rly)

0.91

82.0

93.0

Ava

ilab

ility

of s

corin

g ite

ms

in m

edic

al r

ecor

ds

varie

d

from

27%

(cap

refi

ll tim

e) t

o 93

% (o

xyge

n th

erap

y).

21

Par

shur

am

2011

44C

anad

a an

d U

KA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (p

rosp

ectiv

e)4

No

Ext

Urg

ent

PIC

U

tran

sfer

or

imm

edia

te c

all t

o re

susc

itatio

n te

am

2074

(686

ca

ses)

S7/

26M

ax 2

4 ho

urs

bef

ore

even

t (h

ourly

)

0.87

64.0

91.0

PTT

T sc

ores

cal

cula

ted

el

ectr

onic

ally

aft

er

abst

ract

ion

by

rese

arch

nu

rse.

5.1

% o

f rec

ord

s ha

d

all s

even

item

s re

cord

ed,

31%

had

at

leas

t fiv

e ite

ms.

22

Rob

son

2013

28U

SA

All

inp

atie

nts

Cas

e-co

ntro

l st

udy

(retr

osp

ectiv

e)

1N

oE

xtC

ode

blu

e ca

ll19

2 (9

6 ca

ses)

S7/

26M

ax 2

4 ho

urs

bef

ore

even

t (s

ix

hour

ly)

0.73

56.3

78.1

See

ab

ove.

8

Zha

i 201

419U

SA

All

inp

atie

nts

Cas

e-co

ntro

l st

udy

(retr

osp

ectiv

e)

1N

oE

xtU

rgen

t P

CU

tr

ansf

er w

ithin

24

hou

rs o

f ad

mis

sion

6352

(53

case

s)S

7/26

Max

24

hour

s b

efor

e ev

ent

(hou

rly)

0.82

73.6

71.7

2.1†

Dat

a ex

trac

ted

from

el

ectr

onic

hea

lth r

ecor

ds.

E

xclu

ded

tw

o ite

ms

of

Bed

sid

e P

EW

S (o

xyge

n th

erap

y an

d r

esp

irato

ry

effo

rt) d

ue t

o d

ifficu

lty

abst

ract

ing.

17

Gaw

rons

ki

2016

46Ita

lyS

tem

Cel

l Tr

ansp

lant

Uni

tC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oE

xtU

nexp

ecte

d d

eath

, ur

gent

con

sult

with

RR

T or

urg

ent

PIC

U t

rans

fer

99 (1

9 ca

ses)

S6/

26S

core

4 h

ours

b

efor

e ev

ent

0.90

79.0

97.5

Dat

a ab

stra

cted

by

rese

arch

nu

rses

. No

det

ails

on

exte

nt

of m

issi

ng d

ata.

Con

flict

ing/

mis

sing

ob

serv

atio

ns

reso

lved

by

inte

rvie

ws

with

cl

inic

al s

taff.

15

Cha

pm

an

2017

29U

KA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oE

xtD

eath

, arr

est

or

PIC

U t

rans

fer

608

(297

ca

ses)

S6/

26M

ax 4

8 ho

urs

bef

ore

even

t (p

er

usua

l pra

ctic

e)

0.88

72.0

89.0

86.0

77.0

See

ab

ove.

17

Mod

ified

B

edsi

de

PE

WS

(a)

Fuijk

scho

t 20

1530

(s

tud

y 1)

The

Net

herla

nds

Onc

olog

y w

ard

Cas

e-co

hort

stu

dy

(retr

osp

ectiv

e)1

Yes

Int

Em

erge

ncy

med

ical

in

terv

entio

n or

re

view

ed b

y P

ICU

sta

ff or

sta

ff co

ncer

n

118

(15

case

s)S

8/28

Unc

lear

(min

imum

ei

ght

hour

ly)

73.0

41%

of a

dm

issi

ons

excl

uded

from

stu

dy

due

to

inco

mp

lete

PTT

T sc

ores

.

10

Fuijk

scho

t 20

1530

(s

tud

y 2)

The

Net

herla

nds

All

inp

atie

nts

Cas

e-co

hort

stu

dy

(retr

osp

ectiv

e)1

Yes

Int

PIC

U t

rans

fer

Unc

lear

(24

case

s)S

8/28

Sco

re 2

–6 h

ours

b

efor

e ev

ent

(min

imum

eig

ht

hour

ly)

66.6

Hig

h ra

te o

f exc

lusi

ons

rep

orte

d d

ue t

o m

issi

ng

dat

a.

10

Fuijk

scho

t 20

1530

(s

tud

y 3)

The

Net

herla

nds

All

inp

atie

nts

Cas

e-co

hort

stu

dy

(pro

spec

tive)

1Ye

sIn

tE

mer

genc

y m

edic

al

inte

rven

tion

Unc

lear

(14

case

s)S

8/28

Unc

lear

(min

imum

ei

ght

hour

ly)

100

No

det

ails

on

mis

sing

dat

a.10

Cha

pm

an

2017

29U

KA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oE

xtD

eath

, arr

est

or

PIC

U t

rans

fer

608

(297

ca

ses)

S7/

28M

ax 4

8 ho

urs

bef

ore

even

t (p

er

usua

l pra

ctic

e)

0.87

69.0

91.0

87.9

79.0

See

ab

ove.

17

Mod

ified

B

edsi

de

PE

WS

(b)

Ros

s 20

1549

US

AA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oIn

tU

rgen

t P

ICU

tr

ansf

er46

28 (8

48

case

s)S

8/26

Max

dur

ing

adm

issi

on70

.084

.0N

o d

etai

ls o

n d

ata

abst

ract

ion.

Res

pira

tory

ef

fort

cat

egor

y ex

clud

ed

due

to

diffi

culty

ab

stra

ctin

g.

No

det

ails

on

mis

sing

dat

a.

9

Con

tinue

d

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 12

12 Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

PT

TT

Firs

t au

tho

r, ye

arC

oun

try

Stu

dy

po

pul

atio

nS

tud

y d

esig

nN

umb

er o

f ce

ntre

sP

TT

T u

sed

in

pra

ctic

e?

Inte

rnal

/ex

tern

al v

alid

atio

n st

udy?

Out

com

e m

easu

res

Sam

ple

siz

eS

core

or

trig

ger

?

Sco

re

test

ed/

max

imum

sc

ore

Whi

ch s

core

use

d

(fre

que

ncy

of

sco

ring

)?*

AU

RO

CS

ensi

tivi

tyS

pec

ifici

tyP

PV

NP

V

No

tes

on

accu

racy

/re

liab

ility

of

sco

ring

and

m

issi

ng d

ata

Qua

lity

sco

re

(max

=24

)

Mod

ified

B

right

on

PE

WS

(a)

Tuck

er

2009

31U

SA

Gen

eral

med

ical

un

itC

ohor

t st

udy

(pro

spec

tive)

1Ye

sIn

tP

ICU

tra

nsfe

r29

79 (5

1 ca

ses)

S3/

11M

ax d

urin

g ad

mis

sion

(fou

r ho

urly

)

0.89

90.2

74.4

5.8

99.8

Intr

acla

ss c

oeffi

cien

t of

0.9

2 re

por

ted

for

two

bed

sid

e nu

rses

sco

ring

55 p

atie

nts.

N

o d

etai

ls o

n m

issi

ng d

ata.

14

Zha

i 201

419U

SA

All

inp

atie

nts

Cas

e-co

ntro

l st

udy

(retr

osp

ectiv

e)

1N

oE

xtU

rgen

t P

CU

tr

ansf

er w

ithin

24

hou

rs o

f ad

mis

sion

6352

(53

case

s)S

2/11

Max

24

hour

s b

efor

e ev

ent

(hou

rly)

0.74

68.4

81.6

2.3

Dat

a ex

trac

ted

from

el

ectr

onic

hea

lth r

ecor

ds.

O

nly

incl

uded

rec

ord

s w

ith

com

ple

te P

EW

S s

core

: 64

% o

f elig

ible

cas

es a

nd

51%

of e

ligib

le c

ontr

ols

excl

uded

.

17

Feni

x 20

1539

US

AP

ICU

tra

nsfe

rs

amon

g al

l in

pat

ient

s (e

xclu

din

g ha

emat

olog

y on

colo

gy,

surg

ical

and

ca

rdia

c w

ard

s)

Cas

e-co

ntro

l st

udy

(retr

osp

ectiv

e)

1Ye

sE

xtN

on-e

lect

ive

PIC

U

tran

sfer

follo

wed

b

y d

eter

iora

tion

even

t

97 P

ICU

tr

ansf

ers

(51

case

s of

P

ICU

tra

nsfe

r fo

llow

ed b

y ‘d

eter

iora

tion

even

t’)

S3/

11M

ax d

urin

g ad

mis

sion

80.0

43.0

61.0

67.0

No

det

ails

on

mis

sing

dat

a.15

Mod

ified

B

right

on

PE

WS

(b)

Akr

e 20

1045

US

AA

ll in

pat

ient

sC

hart

rev

iew

stu

dy

(retr

osp

ectiv

e)1

No

Int

Rap

id r

esp

onse

te

am c

all o

r co

de

blu

e ca

ll

186

case

s(1

70 R

RT

calls

, 16

cod

e ca

lls)

S4/

13M

ax 2

4 ho

urs

bef

ore

even

t (m

inim

um fo

ur

hour

ly)

85.5

Sco

res

abst

ract

ed

from

cha

rts

by

sing

le

nurs

e, h

avin

g ca

libra

ted

w

ith a

dva

nced

nur

se

pra

ctiti

oner

.C

ateg

orie

s sc

ored

mis

sing

if

any

item

s m

issi

ng. 2

5% o

f ch

arts

mis

sing

beh

avio

ural

st

ate,

26%

car

dio

vasc

ular

co

lour

.

14

Cha

pm

an

2017

29U

KA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oE

xtD

eath

, arr

est

or

PIC

U t

rans

fer

608

(297

ca

ses)

S4/

13M

ax 4

8 ho

urs

bef

ore

even

t (p

er

usua

l pra

ctic

e)

0.79

61.0

84.0

78.4

69.0

See

ab

ove.

17

Mod

ified

B

right

on

PE

WS

(d)

Ska

letz

ky

2012

48U

SA

Med

ical

sur

gica

l w

ard

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oIn

tP

ICU

tra

nsfe

r35

0 (1

00

case

s)S

2.5/

9M

ax 4

8 ho

urs

bef

ore

even

t (fo

ur

hour

ly)

0.81

62.0

89.0

Dat

a ab

stra

cted

from

m

edia

l cha

rts

and

not

es.

Beh

avio

ur c

ateg

ory

abst

ract

ed fr

om L

OC

. No

det

ails

on

mis

sing

dat

a.

15

Cha

pm

an

2017

29U

KA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oE

xtD

eath

, arr

est

or

PIC

U t

rans

fer

608

(297

ca

ses)

S4/

9M

ax 4

8 ho

urs

bef

ore

even

t (p

er

usua

l pra

ctic

e)

0.74

46.0

90.0

81.3

63.0

See

ab

ove.

17

Chi

ldre

n’s

Hos

pita

l Ear

ly

War

ning

Sco

re

McL

ella

n 20

1450

US

AA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1Ye

sIn

tA

rres

t or

un

pla

nned

PIC

U

tran

sfer

1136

(360

ca

ses)

S4/

12M

ax in

ad

mis

sion

(fo

ur h

ourly

)0.

9084

.280

.9N

o d

etai

ls o

n m

issi

ng d

ata.

10

Chi

ldre

n’s

Hos

pita

l Car

dia

c E

arly

War

ning

S

core

McL

ella

n 20

1323

US

AC

ard

iova

scul

ar

unit

Cas

e-co

ntro

l st

udy

(retr

osp

ectiv

e)

1Ye

sIn

tA

rres

t or

un

pla

nned

PIC

U

tran

sfer

312

(64

case

s)S

3/12

Max

18

hour

s b

efor

e ev

ent

(four

ho

urly

)

0.86

95.3

76.2

50.8

98.4

Stu

dy

nurs

e an

d b

edsi

de

nurs

es a

sses

sed

sco

res

for

37 p

atie

nts,

67%

ag

reem

ent.

No

det

ails

on

mis

sing

dat

a.

9

Agu

lnik

20

1641

US

AO

ncol

ogy

unit

Cas

e-co

ntro

l st

udy

(retr

osp

ectiv

e)

1Ye

sE

xtU

npla

nned

PIC

U

tran

sfer

330

(110

ca

ses)

S4/

12M

ax 2

4 ho

urs

bef

ore

even

t (fo

ur

hour

ly)

0.96

86.0

95.0

PTT

T sc

ores

ab

stra

cted

b

y re

sear

cher

. Did

not

ab

stra

ct if

vita

l sig

ns w

ere

pre

sent

but

no

PTT

T sc

ore

calc

ulat

ed b

y nu

rse.

No

det

ails

on

mis

sing

dat

a.

14

Agu

lnik

20

1742

Gua

tem

ala

Onc

olog

y un

itC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1Ye

sE

xtU

npla

nned

PIC

U

tran

sfer

258

(129

ca

ses)

S4/

12M

ax 2

4 ho

urs

bef

ore

even

t (th

ree

hour

ly)

91.0

88.0

Res

earc

her

eval

uate

d

char

ts a

nd c

alcu

late

d

scor

es, r

epor

ting

14%

er

ror

rate

(PTT

T sc

ore

calc

ulat

ed in

corr

ectly

) and

3%

om

issi

on r

ate

(vita

l si

gns

reco

rded

but

no

PTT

T sc

ore

calc

ulat

ed).

One

out

of

130

cas

es e

xclu

ded

d

ue t

o m

issi

ng P

TTT

doc

umen

tatio

n.

16

Chi

ldre

n’s

Hos

pita

l Los

A

ngel

es P

EW

S

Man

del

l 20

1540

US

AIn

pat

ient

s d

isch

arge

d fr

om

PIC

U t

o w

ard

Cas

e-co

ntro

l st

udy

(retr

osp

ectiv

e)

1Ye

sIn

tE

arly

unp

lann

ed re

-ad

mis

sion

to

PIC

U

(with

in 4

8 ho

urs

of d

isch

arge

from

P

ICU

to

war

d)

189

(38

case

s)S

2/10

Firs

t sc

ore

assi

gned

on

war

d, p

ost-

PIC

U

dis

char

ge

0.71

76.0

56.0

No

det

ails

on

mis

sing

dat

a.12

Tab

le 3

C

ontin

ued

Con

tinue

d

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 13

13Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

PT

TT

Firs

t au

tho

r, ye

arC

oun

try

Stu

dy

po

pul

atio

nS

tud

y d

esig

nN

umb

er o

f ce

ntre

sP

TT

T u

sed

in

pra

ctic

e?

Inte

rnal

/ex

tern

al v

alid

atio

n st

udy?

Out

com

e m

easu

res

Sam

ple

siz

eS

core

or

trig

ger

?

Sco

re

test

ed/

max

imum

sc

ore

Whi

ch s

core

use

d

(fre

que

ncy

of

sco

ring

)?*

AU

RO

CS

ensi

tivi

tyS

pec

ifici

tyP

PV

NP

V

No

tes

on

accu

racy

/re

liab

ility

of

sco

ring

and

m

issi

ng d

ata

Qua

lity

sco

re

(max

=24

)

Mel

bou

rne

Act

ivat

ion

Crit

eria

Tum

e 20

073

UK

Inp

atie

nts

with

an

unp

lann

ed

PIC

U t

rans

fer

Cha

rt r

evie

w s

tud

y (re

tros

pec

tive)

1N

oE

xtU

npla

nned

PIC

U

tran

sfer

33 c

ases

TN

/AU

ncle

ar87

.8D

ata

abst

ract

ed b

y tw

o re

view

ers.

Ref

eren

ce t

o ‘la

rge

num

ber

of m

issi

ng

reco

rds

and

ob

serv

atio

n ch

arts

’.

11

Tum

e 20

073

UK

Inp

atie

nts

with

an

unp

lann

ed

PH

DU

tra

nsfe

r

Cha

rt r

evie

w s

tud

y (re

tros

pec

tive)

1N

oE

xtU

npla

nned

PH

DU

tr

ansf

er32

cas

esT

N/A

Unc

lear

87.5

See

ab

ove.

11

Ed

war

ds

2011

33U

KA

ll in

pat

ient

sC

ohor

t st

udy

(retr

osp

ectiv

e)1

No

Ext

Dea

th o

r un

pla

nned

PIC

U o

r H

DU

tra

nsfe

r

1000

(16

case

s)T

N/A

Any

trig

ger

over

ad

mis

sion

(per

us

ual p

ract

ice)

0.79

68.3

83.2

3.6

99.7

Ob

serv

atio

n ch

arts

al

tere

d t

o in

clud

e al

l P

TTT

par

amet

ers.

56%

of

reco

rds

mis

sing

at

leas

t on

e co

mp

onen

t. M

issi

ng d

ata

assu

med

to

be

norm

al.

17

Cha

pm

an

2017

29U

KA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oE

xtD

eath

, arr

est

or

PIC

U t

rans

fer

608

(297

ca

ses)

TN

/AM

ax 4

8 ho

urs

bef

ore

even

t (p

er

usua

l pra

ctic

e)

0.71

93.0

49.0

64.0

88.0

See

ab

ove.

17

Car

diff

and

Val

e P

EW

SE

dw

ard

s 20

0932

UK

All

inp

atie

nts

Coh

ort

stud

y (p

rosp

ectiv

e)1

No

Int

Dea

th o

r un

pla

nned

PIC

U o

r H

DU

tra

nsfe

r

1000

(16

case

s)S

2/8

Max

sco

re d

urin

g ad

mis

sion

(per

us

ual p

ract

ice)

0.86

69.5

89.9

5.9

99.7

Ob

serv

atio

n ch

arts

al

tere

d t

o in

clud

e al

l P

TTT

par

amet

ers.

56%

of

reco

rds

mis

sing

at

leas

t on

e co

mp

onen

t. M

issi

ng d

ata

assu

med

to

be

norm

al.

18

Cha

pm

an

2017

29U

KA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oE

xtD

eath

, arr

est

or

PIC

U t

rans

fer

608

(297

ca

ses)

S3/

8M

ax 4

8 ho

urs

bef

ore

even

t (p

er

usua

l pra

ctic

e)

0.89

80.0

86.0

84.0

82.0

See

ab

ove.

17

Bris

tol p

aed

iatr

ic

early

war

ning

to

ol (P

EW

T)

Tum

e 20

073

UK

Inp

atie

nts

with

an

unp

lann

ed

PIC

U t

rans

fer

Cha

rt r

evie

w

(retr

osp

ectiv

e)1

No

Ext

Unp

lann

ed P

ICU

tr

ansf

er33

cas

esT

N/A

Unc

lear

87.8

See

ab

ove.

11

Tum

e 20

073

UK

Inp

atie

nts

with

an

unp

lann

ed

PH

DU

tra

nsfe

r

Cha

rt r

evie

w

(retr

osp

ectiv

e)1

No

Ext

Unp

lann

ed P

HD

U

tran

sfer

32 c

ases

TN

/AU

ncle

ar84

.4S

ee a

bov

e.11

Wrig

ht

2011

35U

KA

ll in

pat

ient

sC

hart

rev

iew

(re

tros

pec

tive)

1Ye

sE

xtC

ard

iac

arre

st55

cas

esT

N/A

If tr

igge

red

24

hou

rs b

efor

e ev

ent

49.1

One

cas

e ex

clud

ed d

ue t

o m

issi

ng n

otes

. No

det

ails

on

mis

sing

dat

a.

11

O’L

ough

lin

2012

34U

KA

ll in

pat

ient

sC

ohor

t st

udy

(pro

spec

tive)

1Ye

sE

xtP

ICU

tra

nsfe

r33

1 (7

cas

es)

TN

/ATr

igge

red

dur

ing

adm

issi

on (1

2 ho

urly

)

0.91

100

81.0

11.0

No

det

ails

on

mis

sing

dat

a.6

Rob

son

2013

28U

SA

All

inp

atie

nts

Cas

e-co

ntro

l st

udy

(retr

osp

ectiv

e)

1N

oE

xtC

ode

blu

e ca

ll19

2 (9

6 ca

ses)

TN

/ATr

igge

red

24

hour

s b

efor

e ev

ent

(6

hour

ly)

0.75

76.3

61.5

See

ab

ove.

8

Cha

pm

an

2017

29U

KA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oE

xtD

eath

, arr

est

or

PIC

U t

rans

fer

608

(297

ca

ses)

TN

/AIf

trig

gere

d

48 h

ours

bef

ore

even

t (p

er u

sual

p

ract

ice)

0.62

96.0

28.0

56.0

88.0

See

ab

ove.

17

Mod

ified

Bris

tol

PE

WT

(b)

Cla

yson

20

1438

UK

Car

dia

c w

ard

Coh

ort

stud

y (p

rosp

ectiv

e)1

Yes

Int

‘A d

eter

iora

ting

pat

ient

’12

6 (u

ncle

ar

num

ber

of

case

s)

TN

/AU

ncle

ar12

.597

.0N

o d

etai

ls o

n m

issi

ng d

ata.

5

NH

S In

stitu

te fo

r In

nova

tion

and

Im

pro

vem

ent

PE

WS

Mas

on

2016

14U

KA

ll in

pat

ient

sC

ohor

t st

udy

(retr

osp

ectiv

e)1

No

Ext

Dea

th o

r un

pla

nned

PIC

U o

r H

DU

tra

nsfe

r

1000

(16

case

s)S

2/7

Max

sco

re o

ver

adm

issi

on (p

er

usua

l pra

ctic

e)

0.88

80.0

81.0

4.3

99.7

Ob

serv

atio

n ch

arts

al

tere

d t

o in

clud

e al

l P

TTT

par

amet

ers.

56%

of

reco

rds

mis

sing

at

leas

t on

e co

mp

onen

t. M

issi

ng d

ata

assu

med

to

be

norm

al.

15

Cha

pm

an

2017

29U

KA

ll in

pat

ient

sC

ase-

cont

rol

stud

y (re

tros

pec

tive)

1N

oE

xtD

eath

, arr

est

or

PIC

U t

rans

fer

608

(297

ca

ses)

S2/

7M

ax 4

8 ho

urs

bef

ore

even

t (p

er

usua

l pra

ctic

e)

0.82

83.0

65.0

69.6

80.0

See

ab

ove.

17

Logi

stic

re

gres

sion

al

gorit

hm

Zha

i 201

419U

SA

All

inp

atie

nts

Cas

e-co

ntro

l st

udy

(retr

osp

ectiv

e)

1N

oE

xtU

rgen

t P

ICU

tr

ansf

er w

ithin

24

hou

rs o

f ad

mis

sion

6352

(53

case

s)S

>0.

5M

ax 2

4 ho

urs

bef

ore

even

t (h

ourly

)

0.91

84.9

85.9

4.8

Dat

a ex

trac

ted

from

el

ectr

onic

hea

lth r

ecor

ds.

N

o d

etai

ls o

n ex

tent

of

mis

sing

dat

a b

ut a

utho

rs

rep

ort

that

‘mis

sing

dat

a w

ere

a m

ajor

cau

se o

f in

corr

ect

pre

dic

tion’

.

17

Tab

le 3

C

ontin

ued

Con

tinue

d

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from

Page 14

14 Trubey R, et al. BMJ Open 2019;9:e022105. doi:10.1136/bmjopen-2018-022105

Open access

Question 2: how effective are early warning systems at reducing mortality and critical events in hospitalised children?Eleven papers meeting inclusion criteria were excluded from analysis for providing insufficient statistical informa-tion (eg, denominator data, absolute numbers of events) to calculate effect sizes.39 51–59 Further details on papers excluded from analysis are provided in online supple-mentary table 5. Findings from the 19 studies included in the analysis are summarised in table 4.

type of early warning system interventionsSeventeen interventions involved the introduction of a new PTTT,13 15–18 60–72 one intervention introduced a mandatory triggering element to an existing PTTT71 and one study reported a large, multicentre analysis of MET introduction with no details on PTTT use.73 Twelve interventions included the introduction of a new MET or RRT,13 15–18 60–65 69 while four further interventions intro-duced a new PTTT in a hospital with an existing MET or RRT. Only three studies therefore evaluated a PTTT in the absence of a dedicated response team.67 68 70 A staff education programme was explicitly described in 10 interventions.13 15 17 61 62 64 67 68 70 72

Of the 18 studies that used a PTTT, only 7 used a tool that had been formally evaluated for validity: 3 used the Bedside PEWS,64 65 70 2 used the MAC,13 62 1 used the Modi-fied Brighton PEWS (b)72 and 1 used the C-CHEWS.67 One study did not report the PTTT used,61 while 10 studies used a variety of calling criteria and local modifi-cations to validated tools that had not been evaluated for validity.15–18 60 63 66 68 69 71

Mortality (ward or hospital wide)Two uncontrolled before-after studies (both with MET/RRT) reported significant mortality rate reductions postintervention: one in hospital wide deaths per 100 discharges17 (RR=0.82, 95% CI 0.70 to 0.95) and one in total hospital deaths per 1000 admissions (RR=0.65, 95% CI 0.57 to 0.75) and deaths on the ward (‘unex-pected deaths’) per 1000 admissions62 (RR=0.35, 95% CI 0.13 to 0.92). Seven studies found no reductions in mortality, including two high-quality multicentre studies.13 15 60 63–65 73 Parshuram et al conducted a cluster randomised trial and found no difference in all-cause hospital mortality rates between 10 hospitals randomly selected to receive an intervention centred around use of the Bedside PEWS and 11 usual care hospitals, 1-year postintervention (OR=1.01, 95% CI 0.61 to 1.69).64 Kutty et al73 assessed the impact of MET implementation in 38 US paediatric hospitals with an interrupted time series study, and reported no difference in the slope of hospital mortality rates 5 years postintervention and the expected slope based on preimplementation trends (OR=0.94, 95% CI 0.93 to 0.95).

PICu mortalityTwo uncontrolled before-after studies (both with MET/RRT) reported a significant postintervention reduction in P

TT

TFi

rst

auth

or,

year

Co

untr

yS

tud

y p

op

ulat

ion

Stu

dy

des

ign

Num

ber

of

cent

res

PT

TT

use

d

in p

ract

ice?

Inte

rnal

/ex

tern

al v

alid

atio

n st

udy?

Out

com

e m

easu

res

Sam

ple

siz

eS

core

or

trig

ger

?

Sco

re

test

ed/

max

imum

sc

ore

Whi

ch s

core

use

d

(fre

que

ncy

of

sco

ring

)?*

AU

RO

CS

ensi

tivi

tyS

pec

ifici

tyP

PV

NP

V

No

tes

on

accu

racy

/re

liab

ility

of

sco

ring

and

m

issi

ng d

ata

Qua

lity

sco

re

(max

=24

)

Bur

ton

Pae

dia

tric

Ear

ly

War

ning

Sco

re

Ahm

ed

2012

36U

KP

ICU

ad

mis

sion

s on

lyC

hart

rev

iew

(re

tros

pec

tive)

1Ye

sIn

tP

ICU

ad

mis

sion

23S

4/19

Max

24

hour

s b

efor

e ev

ent

(unc

lear

)

93.0

Dat

a ex

trac

ted

from

cas

e no

tes

by

two

revi

ewer

s. N

o d

etai

ls o

n m

issi

ng d

ata.

4

‘Bet

wee

n th

e Fl

ags’

PE

WS

Bla

ckst

one

2017

43U

KU

rgen

t P

ICU

ad

mis

sion

s on

lyC

hart

rev

iew

(re

tros

pec

tive)

1Ye

sE

xtU

rgen

t P

ICU

ad

mis

sion

100

TN

/AU

ncle

ar91

.0D

ata

extr

acte

d fr

om h

ealth

re

cord

s. N

o d

etai

ls o

n m

issi

ng d

ata.

8

All

stud

ies

cond

ucte

d in

a s

pec

ialis

t/te

rtia

ry c

entr

e.P

PV

and

NP

V v

alue

s in

ital

ics

rep

rese

nt r

esul

ts fr

om c

ase-

cont

rol s

tud

ies—

thes

e va

lues

are

mis

lead

ing

in is

olat

ion

bec

ause

the

y as

sum

e th

at t

he w

ider

pre

vale

nce

rate

of t

he a

dve

rse

even

t is

eq

ual t

o th

e ca

se t

o co

ntro

l rat

io u

sed

in t

he r

esea

rch

stud

y (e

g, if

the

res

earc

hers

stu

die

d 3

00 c

ases

and

300

con

trol

s, t

he p

reva

lenc

e ra

te o

f ad

vers

e ev

ents

for

the

calc

ulat

ion

of P

PV

is 5

0%).

As

per

the

coh

ort

stud

ies,

pre

vale

nce

rate

s of

crit

ical

eve

nts

are

typ

ical

ly fa

r lo

wer

am

ong

hosp

italis

ed p

aed

iatr

ic p

opul

atio

ns t

han

the

case

-con

trol

rat

ios

used

in s

tud

ies,

and

so

PP

V v

alue

s w

ould

be

cons

ider

ably

low

er in

clin

ical

pra

ctic

e.S

tud

ies

clas

sifie

d a

s in

tern

al v

alid

atio

n if

the

sett

ing

for

the

stud

y w

as t

he s

ame

hosp

ital a

nd s

ame

rese

arch

tea

m a

s th

ose

who

dev

elop

ed t

he s

core

. Stu

die

s cl

assi

fied

as

exte

rnal

val

idat

ion

if th

e sc

ore

was

tes

ted

in a

diff

eren

t ce

ntre

and

by

a d

iffer

ent

rese

arch

tea

m t

o th

ose

who

dev

elop

ed it

.*T

ypic

ally

, stu

dy

rese

arch

ers

colle

cted

or

abst

ract

ed m

ultip

le P

TTT

scor

es fo

r ea

ch p

atie

nt a

t d

iffer

ent

time

poi

nts,

but

can

onl

y us

e on

e sc

ore

per

pat

ient

for

the

anal

ysis

of t

he t

ool’s

pre

dic

tive

abili

ty. T

his

colu

mn

spec

ifies

whi

ch s

core

the

res

earc

hers

use

d. I

n m

ost

case

s, t

he s

tud

y te

am u

sed

the

max

imum

PTT

T sc

ore

reco

rded

for

each

p

atie

nt in

a g

iven

stu

dy

win

dow

, eg,

24

hour

s p

rior

to a

crit

ical

eve

nt fo

r ca

se p

atie

nts.

The

tex

t in

par

enth

eses

des

crib

es t

he fr

eque

ncy

with

whi

ch s

core

s w

ere

asse

ssed

or

abst

ract

ed fo

r ea

ch p

atie

nt, i

f thi

s in

form

atio

n w

as d

escr

ibed

in t

he p

aper

.†C

ase-

cont

rol s

tud

y, b

ut P

PV

val

ue c

alcu

late

d b

ased

on

clin

ical

pre

vale

nce

of e

vent

as

mea

sure

d a

t lo

cal c

entr

e d

urin

g th

e st

udy.

AU

RO

C, a

rea

und

er t

he r

ecei

ver

oper

atin

g ch

arac

teris

tic c

urve

; Ext

, ext

erna

l val

idat

ion;

HFN

C, h

igh

flow

nas

al c

annu

la; I

nt, i

nter

nal v

alid

atio

n; M

ax, m

axim

um; N

/A, n

ot a

pp

licab

le; N

PV,

neg

ativ

e p

red

ictiv

e va

lue;

PH

DU

, pae

dia

tric

hig

h-d

epen

den

cy u

nit;

PIC

U, p

aed

iatr

ic in

tens

ive

care

uni

t; P

PV,

pos

itive

pre

dic

tive

valu

e; P

TTT,

pae

dia

tric

tr

ack

and

trig

ger

tool

; RR

T, r

apid

res

pon

se t

eam

; S, s

core

; T, t

rigge

r.

Tab

le 3

C

ontin

ued

on October 3, 2021 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-022105 on 5 M

ay 2019. Dow

nloaded from