Evaluating the Effectiveness of a Training Intervention in ...€¦ · Evaluating the Effectiveness of a Training Intervention in Neonatal Resuscitation: Establishing Grounds for

Evaluating the Effectiveness of a Training Intervention in

Neonatal Resuscitation:

Establishing Grounds for Further Rollout

Faith Kayembe (MA, BSc, PGCert.Ed, RM, Krankenschwester Germany)

Senior Midwifery Lecturer/Project Lead

Canterbury Christ Church university, Kent, UK

Aim

To share findings from a training project evaluation

OBJECTIVES

Background

Methods

Findings

Implications for practice

What this presentation will not do…

Background

Zambia

4

Neonatal mortality rate: 24:1000 (Central Statistics 0ffice (CSO) 2015)

Scoping (2015)

Intervention (2016)

Immediate evaluation (2016)

7 month evaluation (2017)

Scoping: Benefactors, Sponsors & Partners

Mr & Mrs Chowen – Project Benefactors Prof. Newport – Chair BLHL Prof. Hatzidimitriadou - CCCU

Mr Wisdom Chelu – Zambia

MoH Ms Jean Musonda -

UTH

Scoping Key suggestions: • Staff knowledge/skills in

neonatal care including assessment, initial management at birth and resuscitation

• Standardisation of approach

to neonatal resuscitation • Leadership

• Clarify communication systems • Equipment including fetal &

neonatal monitoring • Transportation between wards • Thermoregulation • Discharge examination of the

newborn • Health care support role

Intervention…

Multidisciplinary Team

Teaching material Neonatologist

Senior Neonatal

Nurse

Senior

Resuscitation

Officer

Midwife/

Lecturer

Intervention: Preparation and set up

Teaching: theory, skills, simulation, TTT

Immediate evaluation

0

5

10

15

20

25

30

35

Pre-testscore

Post-testscore

Candidates

Scores

Defining effectiveness…

Author Measure of effectiveness

Dempsey et al (2015) Opiyo et al (2008) Disu et al (2015)

Neonatal mortality rate

Trevisanuto et al (2015) Dempsey et al (2015) Opiyo et al (2008)

Clinical practice/ Change in team work and resuscitation behaviour

Hoban et al (2013) Dempsey et al (2015) Fuchs et al (2015) Bookman et al (2010) Carlo et al (2009) Enweronu-Loryea et al (2009)

Knowledge (immediate and long-term)

Defining effectiveness for the Zambia project:

Further training

More staff

Quality and integrity of training

Positive impact on trainee knowledge

Goals

Clinical practice

Methods

Methodology Single Case study approach (Yin

2009; Taylor & Thomas-Gregory 2015)

Selection/recruitment of participants

o Process/arrangements

o different levels

o informed consent

Methods - Triangulation

1-1/focus groups/telephone;

Range of settings:

Health/shopping centres

Interviews Documents Observation

Training records;

Pre/post-test results;

Training

programme

Lectures;

Team interaction/ Meetings;

Feedback;

Skills and

Simulation Stations

Review of goals

Questionnaire

Findings

Thematic map

Clinical Impact

Moving

forward

Further Training

Activity

Positive

accounts from

practice

Challenges

Successes

Challenges

Efforts & work

in progress

Suggestions/Support

required

Implications for

future practice

Quality

of

training

Review

of goals

Clinical impact

Individual accounts from practice (I)

‘’… from August up to February this year, we have not recorded any mortality … the referrals can even be counted … it has greatly improved our skills…’’ (Midwife)

‘’Since I received that neonatal resuscitation…we refer them at the point the babies are stable now … we know how to manage…’’(Nurse)

Individual accounts from practice (II)

‘…we managed to resuscitate the baby until the baby was out of danger. And then we referred …for further management…’ (Midwife from lower level hospital)

‘… the training we have got has really impacted on our services in a positive way, in that our babies and mothers are being saved. We no longer lose lives…’ (Midwife)

Clinical guidelines

Interviewee by a resucitaire in a local neonatal unit – notice algorithm on the wall

Clearer view of algorithm on the wall

Internalising Clinical knowledge (I)

‘…you don’t even go through those complicated stages, where you need to intubate and the rest…’’’ (Paediatrician)

‘’… I’ve seen it as a good thing, this course. It has really changed the way we used to look at some of the things, as in where we resuscitate now, where the priority is not centred in oxygen...’’ (Midwife)

‘…I really did apply, you know, stage by stage – the … inflation, yes. … inflation breaths was not something that was emphasised, yes, we would go into the ventilation breath immediately …’ (Paediatrician)

Internalising Clinical knowledge(II)

‘

‘…apart from the resuscitation itself there are other things … We are now delivering the babies onto the mother’s abdomen because we have learnt about the skin to skin contact. We are delaying cord clamping. All those majors are helpful in making the baby alive. Yes, so there is change like that…’ (Midwife)

Important insights from Clinical log of experience

…delivered as an emergency caesarean section due to cord presentation and fetal distress

…resuscitaire warmed…baby…wiped, stimulated and wrapped in dry clean clothing…assessed…not breathing…floppy…blue…low heart rate

…five(5) inflation breath given chest expansion observed…

…baby reassessed breathing normal, heart rate fast, tone good and colour pink …cried…wrapped in warm clothing and given to mother…

…I did not start the clock…

Challenges to clinical application

Equipment

Poor Staffing levels

Clinical and professional guidelines – limitations

Competing guidelines

Lack of Leadership/mentorship

Further training

Successes Support

Candidate receiving certificate of attendance from Medical superintendent LGH

Formal activity

Summary of training

Date Venue Numbers Staff groups trained

6-8.4.17 Livingstone 50 Doctors, nurses, midwives, anaesthetists (From one hospital)

9-10.3.17 LUSON 40 Paediatric nurses (Qualified nurses) 20-21.2.17 Lusaka 50 Midwives, nurses, doctors, anaesthetists

2016 LUSON 39 Paediatric nurses (Qualified nurses)

Informal activity ‘’…Our unit, we always get new doctors,

new nurses, … as part of, probably you could call it an orientation, when they come to the unit we do take them through resuscitation of the babies …’’

(Senior Neonatal Nurse)

Quality of training: evolving content Initial launch content

Immediate care at birth

Newborn physiology

Resuscitation at birth

Prematurity /Meconium

Teamwork/communication/documentation

Post resuscitation care

Post launch content

Immediate care at birth

Newborn physiology

Resuscitation at birth

Prematurity /Meconium

Teamwork/communication/documentation

Post resuscitation care

Neonatal Intubation

Umbilical Vein Cannulation

Day 2 – KMC/Nutrition & growth

monitoring/infection

prevention/hand hygiene

One day Two days

Teamwork and constitution

Team member consulting with team leader during a session – note manual in use for reference

Team members discussing approach prior to a session

Trainers working in pairs to support each other facilitating a skills station

Observation of training… Set

(includes environment) Please tick √ below or mark as N/A if not applicable Needs further

development

Effective and

competent

Comments

Checks and adjusts

layout and equipment

√√√√√ √

Introductions √√√√√ √

Establishes usefulness

and clearly states

learning outcomes

√ √√√ √ Could be

emphasised

further

Uses visual aids

appropriately

√√ √√√ √

Uses voice appropriately √ √√√√ √

Uses eye contact

appropriately

√√√√√ √

Demonstrated

enthusiasm

√ √√√√ √

resus.org.uk

Lectures

Candidate being taught how to administer ventilation breaths using a BVM

Learning to perform cardiac compressions

Teaching UVC insertion

Team leader offering support to refine the rate of ventilation breaths a little more accurately

More Observations …

Test Type Lowest Score Highest

Score

Mean Score

Pre test 33% 80%. 61.9%

For Post test 57% 97%. 84.7%

Test Type Lowest Score Highest

Score

Mean Score

Pre test 53% 90% 84.7%

For Post test 67% 100% 88.12%

PRE TEST AND POST TEST - FIRST GROUP

PRE TEST AND POST TEST - FIRST GROUP

Challenges to further training

Funding for rollout

Recognition

Equipment

Previous learning/approaches

Review of goals

Goal Achieved Not achieved No response

Target Lusaka Province health centres/staff

8 1

Create list of potential participants

6 3

Quarterly training 5 3 1

Identify and approach funders

3 5 1

Formulate action plan

7 2

Trainers split into two groups

9

Support from MoH 9

Create minimum requirements for Health Centres – equipment…

8 1

Database for trained personnel

5 4

First training Sept./Oct. 2016

9

Train in-service nursing students

Suggestions moving forward

Recognition

Train more trainers

Evaluation in the clinical area

Mentorship

----------------

o CPD strategies

oNeonatal mortality outcomes

Moving forward…

Further forward…

Government and professional body recognition, funding and support are vital to ensure successful rollout of neonatal resuscitation training

Moving Forward…

Standardisation of neonatal resuscitation guidelines to facilitate application of knowledge and skills to practice

Implications for future practice ‘…we have come up with a profile ... all the members

having input … I think actually it will give us a good direction because we will be guided with what we have documented ourselves…’

Focused

Pilot site/

Clinical area

Inform further training

activity - quantity &

quality

Measure impact

Neonatal morbidity & mortality /clinical

practice

outcome measures

Adjust post evaluation/

audit of relevant issues

Sufficient grounds for further rollout?

the evidence…

Further training activity undertaken

More staff strained

Knowledge improvement

Goals achieved/progress

Clinical impact

Lives are Being Saved

Questions?

Reference list

Bookman L.; Engmann, C.; Srofenyoh, E.; Enweronu-Laryea, C.; Owen, M. ; Randolph, G.; Price, W.; Barker, P. (2010) Educational impact of a hospital-based neonatal resuscitation program in Ghana Resuscitation, Vol.81(9), pp.1180-1182

Carlo WA, Wright LL, Chomba E, Mcclure EM Carlo ME, Bann CM, Collins Harris, H. (2009)Educational Impact of the Neonatal Resuscitation Program in Low-Risk Delivery Centres in a Developing Country. The Journal of Paediatrics, Vol.154 (4), pp.504-508

Central Statistical Office (CSO) (2015) Zambia Demographic and Health Survey 2013-14. Rockville, Maryland, USA: Central Statistical Office, Ministry of Health, and ICF International. 2015

Dempsey E., Pammi M., Ryan A.C., Barrington K.J. (2015) Standardised formal resuscitation training programmes for reducing mortality and morbidity in newborn infants (Review) Cochrane Database of Systematic Reviews

Disu E.A., Fergusson I.C., Njokanma O.F., Anga L.A., Solarin A.U., Olutekunbi A.O., Ekure E.N., Ezeaka V.C., Esangbedo D.O., Ogunlesi T.A. (2015) National neonatal resuscitation training program in Nigeria (2008-2012): A preliminary report Nigerian Journal of Clinical Practice Jan-Feb Vol. 18 Issue 1 p.102-109

Enweronu-Laryea, C.; Engmann, C.; Osafo, A.; Bose, C. (2009) Evaluating the effectiveness of a strategy for teaching neonatal resuscitation in West Africa Resuscitation, Vol.80 (11), pp.1308-1311

Fuchs, T. ; Nibbe, Y. ; Mahmoud, E. (2015) Impact of on-site training of neonatal resuscitation techniques in Shirati district hospitals: Does on-site training improve knowledge, attitude, and practices? Annals of Global Health, January-February Vol.81(1), pp.26-27

Hoban R, Bucher S, Neuman I, Chen M, Tesfaye N, Spector JM (2013) ‘Helping babies breathe’ training in sub-Saharan Africa: Educational Impact and learner impressions Journal of tropical Paediatrics Jun;59(3):180-6

Opiyo N, Were F, Govedi F, Fegan G , Wasunna A, English M, Belizan JM. (2008) Effect of Newborn Resuscitation Training on Health Worker Practices in Pumwani Hospital, Kenya (Newborn Resuscitation Training) PLoS ONE, Vol.3(2), pp.1599

Resus.org.uk - Neonatal resuscitation guidelines (2015)

Taylor R & Thomas-Gregory A (2015) Case study research Nursing standard Vol.29(41), pp.36-40

Trevisanuto, D ; Bertuola, F ; Lanzoni, P ; Cavallin, F ; Matediana, E ; Manzungu, Ow ; Gomez, E ; Da Dalt, L ; Putoto, G (2015) Effect of a Neonatal Resuscitation Course on Healthcare Providers' Performances Assessed by Video Recording in a Low-Resource Setting Plos One, Dec 11, Vol.10(12)

Yin RK (2009) Case Study Research: Design and Methods. 4th ed. Sage Publications, London.