ECHO report 2015-6 max - European Commission · TYC Transforming Your Care USA United States of America . 4" " Executive Summary ... the response rates were too low for statistical

1    

Evaluation of Project ECHO (Extension for Community Healthcare Outcomes) Northern

Ireland programme 2015-6

Report for Health and Social Care Board

May 2016

2    

Contents

TITLE PAGE

NUMBER

Contents 2

Abbreviations 3

Executive summary 4

Acknowledgements and Contributors 7

Chapter 1 – Introduction 9

Chapter 2 - Methods for Evaluation of ECHO with Healthcare Professionals 12

Chapter 3 - Diabetes ECHO 15

Chapter 4 - Optometry ECHO 33

Chapter 5 - Nursing Homes ECHO 55

Chapter 6 - Dermatology for GP trainees ECHO 72

Chapter 7 - Carers support ECHO 92

Chapter 8 - Hub feedback 112

Chapter 9 – Overall Discussion 118

Chapter 10 – Recommendation Summary 124

References 125

3    

Abbreviations

AMD Age-related Macular Degeneration

DHSSPS Department of Health, Social Services and Public Safety

ECHO Extension for Community Healthcare Outcomes

GP General Practitioner

GPST GP Speciality Trainee

GPwSI GP with a special interest

HCP Healthcare Professional

HSC Health and Social Care

ICP Integrated Care Partnership

NH Nursing Home

NI Northern Ireland

NIH Northern Ireland Hospice

NIMDTA

OCT

Northern Ireland Medical and Dental Training Agency

Optical Coherence Tomography

PIL Participant Information Leaflet

TYC Transforming Your Care

USA United States of America

4    

Executive Summary

Introduction

Project ECHO (Extension for Community Healthcare Outcomes) uses tele-conferencing

technology to improve access to specialised care through supporting and training primary

health care professionals (HCPs) remotely (at ‘spokes’) from a centralised ‘hub’ of experts.

ECHO has been proven to improve care across the United States, and a pilot study in

community hospice nurses in Northern Ireland (NI) showed an improvement in knowledge

and self-efficacy of HCPs.(1) In order to determine if ECHO would be effective in other

contexts in NI, the HSC funded five ECHOs knowledge networks: Diabetes, Optometry,

Nursing Homes, Dermatology for GP trainees, and one for supporting carers of palliative

care patients. This report includes the results of the evaluation of the project and

recommendations for the future use of ECHO in NI.

Methods

The four ECHOs with Healthcare Professionals (HCPs) were evaluated using similar

methods. A pre ECHO knowledge and self-efficacy assessment was undertaken, along with

collection of demographic data for spoke participants. At the end of each ECHO program, a

post ECHO knowledge and self-efficacy assessment was undertaken by spoke participants,

to determine if there was a change over the period of the project. They were also asked to

complete a retrospective-pretest evaluation of self-efficacy (i.e. how competent and

confident do they feel they were before the ECHO project with the benefit of hindsight). A

survey of all participants’ (hub and spokes) views on ECHO in general was performed, and

focus groups were held with spoke members of each ECHO network.

The carers support ECHO was evaluated using a questionnaire of all participants, and a

focus group of hub members.

Results

Two networks (optometry and dermatology for GP trainees) demonstrated a statistically

significant improvement in knowledge and self-efficacy. The other two clinical networks

(nursing homes and diabetes), while they demonstrated an improvement in knowledge and

self-efficacy, the response rates were too low for statistical analysis. All four networks with

HCPs demonstrated very positive views towards ECHO and the education and support that

it provided, both through a questionnaire and through focus groups of spoke members. Both

hub and spoke members valued being involved in ECHO networks, and all would participate

again in ECHO if given the opportunity. All said it had improved the care they provided for

patients.

5    

Due to difficulties with recruitment and retention, only four carers participated in the carers

program. While feedback from the carers was positive, and all said they would participate

again and recommend the program to others, half stated that they would prefer a face-to-

face carers service. Hub members of the carers ECHO identified that it was a good forum

for information sharing, but that it lacked the face-to-face support that carers can particularly

benefit from. They identified issues with recruitment and some strategies that may be helpful

for future carers ECHOs.

Conclusion

The quantitative and qualitative findings from this study support the use of ECHO in Northern

Ireland for healthcare professionals by demonstrating statistically significant improvements in

knowledge and self-efficacy in two networks, and positive feedback across all the networks

replicating findings in other ECHO knowledge networks in the US setting,(2-4) and one pilot

study in Northern Ireland.(1) Due to low numbers secondary to issues with recruitment and

retention of carers, it is not possible to draw firm conclusions on the usefulness of ECHO for

carers and further research is needed.

As a low-cost high-impact model, ECHO can be adapted to meet the needs and resources of

different communities and populations. At a time when health care providers are under

mounting pressure to do more and spend less, this model provides an affordable solution to

addressing growing need in the UK in training and supporting healthcare professionals.

Further research is needed to look at the impact on patient care and service delivery.

Suggestions from evaluation

• ECHO should continue to be developed and implemented to help educate healthcare

professionals across Northern Ireland in a cost effective manner

• Adequate funding is necessary to allow protected time for all to participate, both at

the hub and the spokes

• The importance of the skills of the facilitator was a recurring theme, and emphasises

the need for good quality facilitation training and supervision to ensure networks

exhibit fidelity to the ECHO model

• Future ECHO networks should continue to be evaluated to ensure they meet the

needs of the population they are trying to educate and support

• To improve the response rates in future evaluations, funding for individual practices

or participants could be dependent on participating in the evaluation process. This

should therefore reduce the bias of a partial response, and improve the

generalisability and hence usefulness of the results

6    

• Further studies are required to determine if the ECHO model improves patient care

and reduces costs e.g. through reduced referral rates, improved quality of life, and

better patient and staff outcomes

• Minor technological issues should be addressed to make connectivity easy and not

detract from the learning environment. Issues of not being able to connect from

some sites due to HSC security policies also need addressed

• Ensuring that preparatory work is available in good time for the spoke members, and

that additional online resources are easily accessible were highlighted as contributing

to the overall benefits associated with being part of an ECHO network

• More research is needed to determine the usefulness of a carers ECHO network

The suggestions made in this report could contribute to designing a future Carers

ECHO and through further evaluation determine more conclusively if the ECHO

format is a useful methodology in providing information and support to carers in a

cost effective manner.

7    

Acknowledgements

This project could not have been completed without the enthusiastic support and

commitment of all the teams who were involved in the setting up and running of the five

ECHO clinics in a short period of time. They, along with the Education Department at the

Northern Ireland Hospice, established and maintained the ECHO clinics throughout the

project. Their commitment to providing excellent education to others and hence improving

patient care was outstanding, and the evaluation of the clinics would not have been possible

without their expert involvement in the evaluation assessments.

We are particularly indebted to all the participants who so willingly gave of their time and

effort to participate both in Project ECHO and in its evaluation, and who throughout the

project demonstrated a commitment to patient care which was truly inspiring.

Administration support for the ECHO evaluations through Patricia Marshall, Áine McMullan,

Rebecca Donnelly, Ciara McClements, Tracey McTernaghan and Claire Armstrong was

invaluable.

Our sincere thanks also go to the Health & Social Care Board for funding this study, and in

particular to the HSC ECHO project Board, to Dr Sanjeev Arora and the ECHO team at the

University of New Mexico for ongoing support and encouragement and to the Senior

Management Team at the Hospice for having the vision to invest in the ECHO way of

working.

Contributors to the report

Dr Clare White (Principle Investigator), Consultant Palliative Medicine, N. Ireland Hospice – all aspects of the study Dr Clare McVeigh, Nurse Research Lead, N. Ireland Hospice – focus groups, carers ECHO evaluation and write up Prof Max Watson, Medical Director, N. Ireland Hospice – set up and running of Project ECHO NIH Dr Lynn Dunwoody, School of Psychology, Ulster University, N. Ireland – statistical analysis Administration Support Patricia Marshall, Information Systems Administrator, N. Ireland Hospice - data entry Áine McMullan, Project manager, Project ECHO NI - Coordination of ECHO information

8    

Tracey McTernaghan, N. Ireland Hospice – coordination of ECHO data collection and marking Claire Armstrong, N. Ireland Hospice – coordination of ECHO data collection and marking Rebecca Donnelly, Regional ICP Administrative Support, Health and Social Care Board – coordination of ECHO data collection Ciara McClements - Project Manager Transforming Your Palliative and End of Life Care (TYPEOLC) Delivering Choice Programme, Northern Ireland - coordination of ECHO data collection and marking

Dermatology ECHO Dr Nigel Hart, Senior lecturer in General Practice, Queens University Belfast and Associate Director in General Practice, NIMDTA Dr Siobhan McEntee, GP in Glengormley Practice, GP Trainer, GP Program Director with NIMDTA Claire Loughrey, Director of General Practice, NIMDTA Optometry ECHO Mr Raymond Curran, Head of Ophthalmic Services, Health and Social Care Board, Belfast Mrs Margaret McMullan, Clinical Ophthalmic Adviser, Health and Social Care Board, Belfast Mr Michael Williams, Senior Lecturer, Centre for Medical Education, Queen's University Belfast | Honorary Consultant, Medical Ophthalmology, Belfast Health and Social Care Trust Prof Augusto Azuara-Blanco, Clinical Professor Centre for Vision and Vascular Science, Queen’s University Belfast | Consultant Ophthalmologist, Belfast Health and Social Care Trust Diabetes ECHO Dr Roy Harper, Consultant Endocrinologist, South Eastern Trust/ HSCB Dr Neil Black, Consultant Endocrinologist, Western Trust Nursing Home ECHO Sue Foster, Head of Education, Northern Ireland Hospice Corrina Grimes, Allied  Health  Professinal  Consultant,  Public Health Agency

9    

Chapter 1 - Introduction

Accessible education for healthcare professionals (HCPs) is essential if they are to be

enabled to provide high quality care in a rapidly evolving health care environment. This is

particularly challenging within a community context as practitioners may work in isolated

settings, with varied access to educational opportunities and peer review of practice. With

the constraints of limited resources and the isolated setting in which many HCPs work, new

innovative or creative approaches to education and mentorship that facilitate individual and

collective learning and changes in practice are required.

One such approach is to use the structure of ECHO (Extension for Community Healthcare

Outcomes), which was developed by University of New Mexico and uses point to point

video-conferencing technology involving a central specialist ‘hub’ linking with primary care

providers remotely at ‘spokes’. ECHO uses a collaborative model of medical education and

clinical support, and aims to empower and equip HCPs to provide better care to more

people, right where they live.(5) Participants in the primary care setting (at the spokes)

receive evidence-based or best practice guidance from specialists at the hub, case-based

learning from peer-presentations and have opportunity for live questions and answers. The

ECHO model does not provide direct patient care, but through training using real life cases

provides front-line HCPs with the knowledge and support to manage similar patients with

complex conditions. It does this by engaging HCPs in a sustained learning system and

partnering them with specialist mentors to form a community of practice.(5) It is therefore

different from telemedicine (picture 1), which aims to treat patients directly. Instead it aims

to ‘multiply’ knowledge through educating others to treat more patients than could ever be

directly cared for by one individual HCP. Through ECHO there is opportunity to quickly

translate new knowledge into practice, and thus improve outcomes for patients in more

remote settings.

ECHO is now used across the United States of America (USA) for 45 different disease and

health conditions including hepatitis C, diabetes, asthma, pain management and

rheumatology, and has been shown to improve patient outcomes.(2-4) The impact on such

outcomes, and the reduction in waiting lists at central specialist clinics have been key in

promoting its widespread uptake in health systems which are struggling to meet the needs of

patients living far from central services.(2-4) Indeed the Senate in the USA has just passed a

bill supporting the widespread use of ECHO across the country.

10    

Picture 1 ECHO vs. Telemedicine

ECHO uses cloud based ‘Zoom’ software which is compliant with encryption standards and

which allows staff to connect with ECHO sessions using standard computers, laptops,

tablets and hand held devices. Zoom allows for up to 100 spokes to join a single ECHO

session and does not require expensive dedicated telemedicine equipment though is

dependent on a reliable broadband connection.

In 2014 the Northern Ireland Hospice (NIH) undertook a six month pilot project of ECHO with

its community hospice nurses, and the evaluation showed statistically significant

improvements in knowledge and self-efficacy(1); 96% recorded gains in learning, and 90%

felt that ECHO had improved the care they provided for patients; 83% would recommend

ECHO to other healthcare professionals (HCPs); 70% stated the technology used in ECHO

had given them access to education that would have been hard to access due to

geography.(1) This was the first evaluation of ECHO in the UK and Europe.

In 2015 the Health and Social Care (HSC) Board received funding of £403k from the

Executive Change Fund in 2015-16 and worked in partnership with NIH to pilot the use of

the ECHO model across the HSC to determine if the successes of ECHO in the USA could

be replicated in Northern Ireland. The project pilot period was from October 2015 until 31st

March 2016.

11    

There were approximately 40 ECHO sessions set up (November 2015 – March 2016)

across the following Knowledge Networks using the hub and spoke model:

• Diabetes for GPs

• Optometry for Optometrists

• Palliative Care and Quality Improvement for Nursing Homes (26 Nursing Homes

involving over 90 healthcare professionals)

• Dermatology for GP Trainees

• Support for Carers of Patients with Palliative Care needs

Each of these ECHO knowledge networks was evaluated to help inform decisions around

the future of ECHO in Northern Ireland. The ECHO knowledge networks are described in

turn along with the results of their evaluation.

12    

Chapter 2 - Methods for Evaluation of ECHO with Healthcare Professionals

These methods relate to the evaluation for each of the four clinical ECHO networks with

HCPs- Diabetes, Optometry, Nursing Homes and Dermatology for GP trainees. The carers

support evaluation is described separately as it required a different evaluation strategy.

Aims and Objectives

The aim of the study was to evaluate the use of ECHO Knowledge Networks for HCPs

working in Northern Ireland.

In order to do this, the study objectives were:

• To determine if the use of ECHO improves HCPs knowledge and self-rated skills and

confidence (self-efficacy) in the management of patients

• To explore the HCPs experience and perceived usefulness of ECHO in meeting

knowledge and support needs

• To determine if hub participants benefited from participating in ECHO

Study Design

A prospective longitudinal cohort study was undertaken for each ECHO, using a combination

of qualitative and quantitative methods. The study comprised of two stages:

Stage 1: Baseline assessment – prior to ECHO commencement for spoke participants

Stage 2: End of ECHO assessment for hub and spoke participants.

All ECHO evaluations, with the exception of the Dermatology for GP trainees, were run by

an independent evaluation team from the NIH research department. Assessment tools were

specifically written for each particular ECHO, but adapted from a master format. The creation

of the assessment tools appropriate to each network was undertaken by the clinical lead

running that network.

The Dermatology ECHO for GP trainees evaluation was run by the hub members (Dr Hart

and Dr McEntee) and the evaluation team from NIH was only involved in obtaining general

feedback on ECHO from participants and through running the focus group.

STAGE 1 –Baseline data collection prior to ECHO commencement:

Spoke participants: Each HCP participating in the ‘spokes’ (or community) was asked to

complete the following tools, based on the research aims for the study:

1. A confidential self-efficacy tool focusing on participant confidence in managing different

situations that they face in delivering the relevant service in their particular clinical area.

13    

2. A written knowledge assessment. This was developed by each individual ECHO clinical

leads team with input from the research team as required.

3. Demographic data for all participating HCPs included gender, age, profession, area

worked in (rural / urban), and length of time working in their particular area.

The tools took a maximum of 60 minutes to complete and were undertaken prior to

commencement of the ECHO network, either at a training day or they were sent out to

participants and returned once completed. Participants’ names were not recorded on the

tool; instead each participant was allocated a unique code that was only identifiable to the

administration team. Each participant was provided with information about the evaluation

using a participant information leaflet (PIL). Participation in the evaluation was considered

an essential requirement of being involved in the ECHO, and hence if participants did not

wish to participate in the evaluation the intention was that they would be unable to

participate in the ECHO network. Consent to take part in the evaluation was presumed on

completion and return of the questionnaires.

STAGE 2- After ECHO Knowledge Network completed:

Spoke participants - at the end of the ECHO project all HCPs at the spokes were asked to

complete the following survey data:

• Questionnaire of self-efficacy to determine if there was a change over the period of

the project. They were also asked to complete a retrospective-pretest evaluation of

self-efficacy (i.e. how competent and confident do they feel they were before the

ECHO project with the benefit of hindsight). This was used to try to reduce the bias

of self-evaluation and facilitate HCPs to be more objective in assessing the impact of

ECHO on self-efficacy.(6, 7) The pre-test, post project and retro-pretest

questionnaires were all compared using a unique identifier code

• Knowledge assessment (the same as pre-ECHO) with results being compared with

their pre-ECHO assessment

• Questions relating to participants overall views on ECHO and the network they were

involved in.

Hub participants - All hub participants were asked to complete a questionnaire via Survey

Monkey®(8) of their experiences of participating in an ECHO network.

Focus Groups

To address the second objective of exploring the HCPs experience and perceived

usefulness of ECHO in meeting knowledge and support needs, focus groups were

14    

undertaken with the HCPs who had attended the ECHO programme at the spokes and who

indicated a willingness to participate in a focus group discussion. All spoke participants who

participated in ECHO received an invitation letter via email requesting if they would be willing

to participate as well as a PIL. The intention of this stage of the study was to provide greater

insight into the perceptions and experiences of participating in the ECHO knowledge

networks, problems encountered and benefits, and whether or not it would be useful to

continue with the programme. A focus group schedule, based on the aims and objectives of

the project was used to elicit information. There was a cooling off period of a minimum seven

days between receiving the letter inviting participation, the PIL and participating.

With the participant’s permission, the discussion was audio-taped and supplemented by field

notes. Participants’ names were not recorded on the tape; instead each participant was

allocated a unique code that was only be identifiable to the researcher. The groups were

facilitated by a researcher who was not involved in the running of ECHO using a structured

framework. Focus groups were conducted using the ECHO technology following the final

ECHO session or at another pre-arranged time.

Inclusion Criteria

All HCPs participating in the ECHO programme were invited to participate in the relevant

aspects of the study.

Governance and Consent

Approval was granted from N. Ireland Hospice / Ulster University research governance

committee. Consent to take part in the study was presumed on completion and return of the

questionnaires. Formal consent was taken for the focus groups. No patients were involved

in this aspect of the evaluation, only HCPs who were assured that this evaluation was about

evaluating ECHO, not about evaluating HCPs and their individual knowledge and practice.

Data analysis - Due to the small sample, descriptive statistics and where possible, non-

parametric tests (Friedman and Wilcoxon) were used to summarise the participant

characteristics and survey data. The data from the focus group was analysed by adopting a

thematic analysis framework described by King and Horrocks (2010). (9) This thematic

analysis approach allowed the investigator to be flexible in their interpretation and

development of the themes, and not to be confined by a rigid analytical framework.

15    

Chapter 3- Diabetes ECHO

Background

There are over 83,000 people living with diabetes in Northern Ireland with rates of

approximately 5% of the general population. Approximately 90% of people living with

diabetes have type two diabetes, the prevalence of which is rising rapidly. Diabetes care is

delivered in both primary and secondary settings, and in NI approximately two-thirds of

people are looked after solely by primary care through Practice Nurses and GPs.

Historically a larger proportion of people in Northern Ireland are looked after in secondary

care when compared to other areas in the UK. GPs and Practice Nurses are experienced

professionals, but their confidence and knowledge base in diabetes care varies from practice

to practice. It is estimated that 10% of the healthcare budget is spent on delivering diabetes

care including the management of complications from diabetes. Diabetes is therefore a key

target for the HSC and is one of the priority conditions on which Transforming your Care

(TYC) and Integrated Care Partnership (ICP) initiatives focus.

Diabetic patients need multidisciplinary input to deliver all the care that is needed. This is

appreciated to varying degrees in different care settings. It is also a complex therapeutic

area which continues to change rapidly, so it is hard to keep up to date particularly if it is not

the clinician’s main clinical area. These factors can decrease confidence in using new

therapies to manage patients. This can contribute towards therapeutic inertia or encourage

premature referral to speciality care, which results in overloading of such services when the

problem could be dealt with without attending a secondary care clinic. The converse is that

delay in an appropriate referral can lead to an increased complication risk. Primary care

professionals therefore need supported and educated to manage patients appropriately in

the primary care setting, and to know when to refer to secondary care.

The Intervention

The Diabetes ECHO trained and supported primary care providers, GPs and practice

nurses, to improve their knowledge and skills in the management of patients with diabetes.

The ECHOs were held weekly on a Wednesday afternoon from 1.30-3.00pm and covered a

curriculum devised by spoke participants including diagnosis, appropriate use of Type 2

agents, use of insulin, and management of diabetes at the end of life. For detail see table 1.

Hub - The hub included diabetologist’s from the regional diabetes strategy group who

expressed an interest in being involved in the ECHO network. There was also an ICP

Clinical Lead with an interest in diabetes, a Trust dietician, a Trust Special Diabetic

Pharmacist and a National Care Advisor from Diabetes UK.

16    

Spokes - Involvement in ECHO was opened to all GP’s in the region. Thirteen GP’s, who

were joined by some of their practice nurses and practice based pharmacists took part.

Table 1 Diabetes ECHO Network

Diabetes ECHO Network

Clinical

Champion/Lead

Facilitator

Admin Support

Dr Roy Harper – Diabetologist South Eastern Trust

Dr Neil Black - Diabetologist Western Trust

Dr. Glynis Magee - Diabetologist Southern Trust

Florence Findlay White (Diabetes UK)

Tracey McTernaghan

Frequency of clinics

Weekly and then bi-weekly. 9 sessions in total.

Dates: Dec – 9th,16th/Jan – 13th,20th,27th/Feb – 3rd,17th/Mar –

2nd,16th

Start date Wednesday 9th December

Training Date 25th November 2015 at Jennymount

Hub members

• Barney McCoy (ICP Clinical Lead – South)

• Lesley Hamilton (Western Trust)

• Neil Black (Western Trust)

• Roy Harper (SE Trust/HSCB)

• Brid Farrell (PHA)

• Magee, Glynis (diabetologist – Southern Trust)

• Lynne Thomas (dietician) – SE Trust

• Rosemary Donnelly - Specialist diabetic Pharmacists (SE

Trust)

Hub costs

As per costing schedule

Based on 12 weeks of clinic and 4 hours clinic time (inc prep) and

one session of training

£11,481 paid to the 5 Trusts on 1st December 2015

Spoke members 13 GP’s confirmed (Representing all ICP areas across the region

except West)

Spoke costs £170 for GP practices per session, “backfill”.

Equipment needed

for spokes

Only 4 webcams required as most using their own equipment,

laptop/ipad

17    

Curriculum

Curriculum developed at training day.

• Currently, what is the best way to diagnose diabetes in

both adults and children?

• Communicating the diagnosis news to patients and

inspiring change.

• When & how do I start insulin in Type 2 diabetics?

• What Type 2 agents should I be using and when? (Part 1)

• What Type 2 agents should I be using and when? (Part 2)

• The practicalities of Insulin including the ‘sick day rule’.

• What has my diabetic patient been told in their structured

education programme?

• What do I do when my diabetic patient is dying? How do I

recognise this and manage the last year of life?

• Rules – Diabetes & driving

Evaluation Methods

Evaluation was undertaken as described in Chapter two. Participants completed their pre-

test evaluations at the training day, and were emailed their post-test evaluations which were

returned by post or email.

Results – Diabetes ECHO

There were 13 GP sites who participated in ECHO, and while practice nurses joined

intermittently, only the GPs were invited to take part in the evaluation as they were the

primary target of the ECHO and attended consistently. Eight spoke participants completed

the pre ECHO assessments (response rate 61.5%), with six completing the post ECHO

assessments (response rate 46.2%). Demographic data for all participants in the pretest

evaluation are shown in table 2. For the rest of the results, only the participants who

completed the pre and post evaluations are included.

18    

Table 2 Diabetes Demographic data Pre ECHO Evaluation

Range Totals

Age 20-30 0

31-40 1

41-50 4

51-60 3

60+ 0

Gender Male 6

Female 2

Work area Urban 3

Rural 3

Mixed 2

Years in practice <5 1

6-10 0

11-20 3

21+ 4

Knowledge and Self-efficacy Assessments

The mean scores of knowledge improved slightly between the pre ECHO and post ECHO

assessments for the six participants. Average knowledge scores improved from 21.63 to

22.86 (out of a possible 40 marks); from 54% to 57%. Two participants’ score dis-improved,

one was the same and three improved. Means and standard deviations are shown in table

3. Due to the low response rate no further statistical analysis was possible. Error bars are

shown in figure 1.

Table 3 Diabetes ECHO Knowledge and total self-efficacy results

Pre- ECHO Post-ECHO Retro-Pre ECHO

Outcome Mean SD Mean SD Mean SD

Knowledge 21.63 3.25 22.86 2.79 - -

Total Self

efficacy

3.55 0.70 4.50 0.33 3.60 0.48

19    

Figure 1 Error bars representing the mean and two standard deviations either side of the mean for the diabetes knowledge assessment

Self efficacy results improved (table 3), and a higher average score for the post-test

evaluation in all areas compared with the pretest and retro-pretest evaluation. The different

domains are shown in table 4. Due to the low response rate no further statistical analysis

was possible.

Table 4 – Diabetes ECHO Self efficacy assessment - Participants self-rated confidence in

each area (1 - not confident at all, 5 - very confident)

Question

Pre ECHO average (Range)

Retropre ECHO Average (Range)

Post ECHO average (Range)

Diagnose and classify patients with diabetes 4.38 (3-5)

4.00 (3-5)

4.83 (3-5)

Treat patients with diabetes using optimised treatment regimens

4.00 (2-5)

3.67 (2-5)

4.50 (2-5)

Understand the possible side-effects and limitations of most pharmacological treatments used in diabetes

3.88 (2-5)

3.33 (2-5)

4.33 (2-5)

Initiate and support patients as they use injection therapies

2.63 (1-5)

3.17 (1-5)

4.17 (1-5)

Assess and expertly manage diabetes complications and co-morbidities

3.50 (2-5)

3.33 (2-5)

4.50 (2-5)

Educate and motivate patients with diabetes 3.75 (3-5)

4.00 (3-5)

4.83 (3-5)

Serve as a local expert within my practice and area for diabetes questions and issues

3.38 (1-5)

3.67 (1-5)

4.50 (1-5)

20    

General Feedback of Diabetes ECHO

Box 1 demonstrates participant’s views on ECHO in general. The six respondents were very

positive, with 100% having learnt ‘a lot’ through ECHO participation and enjoyed it, 100% felt

it helped translate knowledge into practice more than other teaching sessions they had been

involved in and had improved the care they provided for patients, and 100% would

participate again.

Box 1 General ECHO Diabetes Results

1. Rating on a scale of 1-5 the quality of learning / usefulness from each area (1- poor, 5- excellent)

1 2 3 4 5

Review of previous session 0 0 1 2 3 Presentations 0 0 0 1 5 Case based discussions 0 0 0 1 5

2. Overall do you feel you have learnt through participating in ECHO?

A lot 6

A little 0 NO 0

3. Did you find participating in ECHO enjoyable?

A lot 6

A little 0 No 0

4. Do you think that participating in ECHO has improved the care you provide for patients?

A lot 6

A little 0 No 0

21    

5. Do you think the format of ECHO helps translate knowledge from teaching into practice more than other teaching sessions you have been involved in?

Yes 6

No 0

6. Would you recommend ECHO to other healthcare professionals in your area?

Yes 5

No 0

7. Have you used any of the online resources via Moodle, and if yes have you found these useful?

Used and found useful

Used and found NOT useful

Power point presentations 4 0 Video of the teaching

sessions 4 0 Video of case presentations 4 0 Other supporting materials 2 0

8. Regarding ECHO technology…

Agree Disagree Unsure It has given me access to education that would have been hard to access due to geography 4 2 0 It was a good medium to access teaching / education at a different location from where I work 6 0 0 Any technical difficulties were acceptable and did not put me off participating in ECHO 5 0

No issues

Any technical difficulties did not significantly reduce my learning 5 0

No issues

9. How do you rate your overall ECHO experience? (1- poor, 5- excellent)

1 2 3 4 5 0 0 0 0 6

22    

10. Would you participate in ECHO sessions in the future if the opportunity arose?

Yes 6

NO 0

Focus Group Results

The focus group was conducted with five of the spoke GPs. Analysis of the focus group data

uncovered two overarching themes, each with their own descriptive and interpretative

themes that are outlined in thematic diagrams (figure 2 and 3). Overarching theme 1, ECHO

Enhanced Clinical Knowledge and Skills, explored the reasons why participants perceived

that ECHO optimised their clinical practice. Interpretative themes 1A and 1B highlighted the

contributing factors that resulted in participant’s perceived enhancement of clinical practice.

Overarching theme 2, Consideration for the Future of ECHO, displayed the key conceptions

that participants perceived should be considered for future ECHOs.

Overarching Theme 1: ECHO Enhanced Clinical Knowledge and Skills

This theme encompassed how participants perceived ECHO to be an education platform

that enhanced their clinical knowledge and skills. Interpretative theme 1A (figure 2) depicted

how participants perceived that ECHO optimised the care they delivered their patients

through the knowledge they had gained, and also through their increased confidence in

caring for patients with diabetes. Interpretative theme 1B (figure 2) illuminated how ECHO

also created a safe learning environment for participants. Findings highlighted that this was

perceived to have been achieved by the encouragement of the hub and the sharing of

knowledge. The learning environment was also enhanced as ECHO was perceived as

providing a platform which met the differing learning styles of participants.

23    

Figure 2 Theme 1 ‘ECHO Enhanced Clinical Knowledge and Skills’ for Diabetes ECHO

Descriptive Themes Interpretative Themes Overarching Themes

1 ECHO Enhanced

Clinical Knowledge and Skills

1A Optimised Patient

Care

Increased GPs confidence in their skills when caring for

patients with diabetes

Increased knowledge and skills regarding diagnosing

diabetes

Increased knowledge regarding new treatments

1B Created an effective learning environment

Importance of the Hub

Encouraged sharing of knowledge

Highlighted areas for further learning

Adapted to participants learning styles

24    

Figure 3 Theme 2, ‘Consideration for the Future of ECHO’ for Diabetes ECHO

Descriptive Themes Interpretative Themes Overarching Themes

Interpretative Theme 1A: Optimised Patient Care

The majority of participants described how they perceived that the ECHO program had a

positive impact on patient care. Many felt this was due to the impact of ECHO on increasing

their confidence when providing care to patients with diabetes in the primary care setting:

“I think my confidence in dealing with some patients with diabetes has risen significantly.”

(DIA/12)

For some, this was achieved through collaborative working at the ECHO clinics:

2 Consideration for

the Future of ECHO

2A There needs to be

funding

Funding allows protected time

Funding ensures quality teaching

2B Furture ECHO

participants

Team approach at the spokes

Future hub members

2C Technological considerations

"Technical hitches"

What worked well

25    

“It really raised my confidence in dealing with diabetic patients. I really enjoyed sharing

cases with other GPs and realising that I wasn't on my own when I got stuck as to where to

go next, and it just reassured me and made me feel more confident in taking the next step in

treating my diabetic patients.” (DIA/14)

“It's a confidence thing. It's really improving everybody's confidence in how they manage

these patients and bringing our expertise up a little bit more and helping to understand how

other people deal with their problem cases that we have, which is extremely important for all

of us.” (DIA/15)

One participant felt that ECHO would also be beneficial to his colleagues in helping build

their confidence in managing patients with diabetes:

“Some of my other partners tend to be very reluctant (regards diabetes)…. anybody who has

a blood sugar above 7 tends to get landed at my door, but I would encourage my partners to

take part in something like this, which would help.” (DIA/15)

One participant perceived that the ECHO program had increased their knowledge and skills

in relation to diagnosing the type of diabetes a patient has:

“There were quite a few things I learned from this. I thought it was useful to go through the

diagnosis of diabetes, which at first glance seems relatively straightforward but in a small

number of cases can be difficult and different, and looking at things outside of the normal

Type 1 Type 2 diabetes was useful and made me think again about categorising people with

diabetes.” (DIA/05)

Some participants also illuminated that the ECHO clinics had increased their awareness and

confidence in relation to considering new treatment methods:

“I thought all the presentations were useful but I thought it was particularly useful, the

discussions around some of the newer drugs and how they can be used effectively.”

(DIA/05)

“I think the thing for me was where to go after you’d started a couple of different types of

drugs and you’re thinking about referring the patient for consideration of insulin and

becoming more comfortable with some of the newer drugs, the sodium-glucose co-

transporter 2 (SGLT2) inhibitors, and things like that, and certainly it has helped with that

significantly.” (DIA/12)

26    

Overall, these findings have highlighted how the learning gained through the ECHO platform

enhanced the confidence and clinical skills of the participants. This was perceived to

enhance patient care due to raised awareness amongst participants on how to optimally

diagnose and treat patients with type 1 and type 2 diabetes.

Interpretative Theme 1B: Created an effective learning environment

Many participants expressed that the enthusiasm and encouragement at the hub added to

the positive learning environment provided through the ECHO platform. Many felt that this

positively attributed to the success of the program:

“The enthusiasm at the hub is really, really important and I think on this occasion was

extremely successful.” (DIA/15)

“There was great enthusiasm from the hub, which sort of encouraged and motivated

everybody, and I think that was extremely useful and helpful.” (DIA/15)

Findings additionally illuminated that participants found the multi-professional presence at

the hub beneficial:

“The other point I would add, which I thought was particularly good, was the multi

professional aspect to it, the fact that we had not just GPs and consultants but there were

diabetic specialist nurses, dieticians and others there. So there was a good breadth of

experience and complementary knowledge sets and experience sets there.” (DIA/05)

“I think just to emphasise what was said earlier about the multidisciplinary approach, I think

having groups of people who are coming at it from a different angle really means that you're

sharing a lot more information. Having the diabetic specialist nurses and the dieticians and

the consultants and GPs and the practice nurses, all in the one group, really enriched the

experience and really helped us all to appreciate where others are coming from.” (DIA/05)

Findings also highlighted how participants perceived ECHO as providing an effective

learning environment through encouraging the sharing of knowledge and experiences:

“One of the big benefits for me has been hearing what my colleagues [the spokes] are doing

and realising that I can be a little bit more adventurous with some things and that I can be a

little less adventurous with other things, and actually realising that we’re pretty much doing

much the same stuff, maybe approaching it in slightly different ways but very definitely doing

very similar things.” (DIA/12)

27    

One participant highlighted that the ECHO experience had highlighted areas for further

learning:

“For me it’s highlighted a lot I don't know, really it’s highlighted areas within diabetes that I

realise actually I need to do a lot more reading in that area, and then it’s reinforced other

areas where I feel more comfortable. But it certainly highlighted a lot of things that I didn’t

know, that I didn’t know.” (DIA/05)

The majority of participants conveyed that ECHO provided and effective learning

environment due to the formats ability to adapt to their learning styles. This perception

resulted from the mixture of didactic and experiential learning provided at the clinics:

“In terms of the content, I thought the content was very good. It was well presented and

there were lots of new information and lots of experiential learning, as well as factual

learning.” (DIA/05)

“The presentations, I thought, were very good, very thought provoking and encouraged me

to read around them. But I got the most out of our discussions. Our case discussions, I think,

were extremely well thought through.” (DIA/15)

One participant also illuminated that they benefited from the interaction with other clinicians:

“The way I like to learn is interaction with people, so this is very easily accessible interaction

with other colleagues and with consultants and other specialists. So I was very grateful for

the experience.” (DIA/14)

Accounts demonstrated how the ECHO platform created an effective learning environment

for GPs with a special interest in diabetes. This was achieved by the presence of an

encouraging hub and an environment that nurtured the sharing of knowledge amongst

clinicians. The learning environment was also optimised as the platform lended itself to

meeting the varying learning styles of participants and providing them with the opportunity to

further consider their clinical practice.

28    

Overarching Theme 2: Consideration for the Future of ECHO

Findings were indicative of the need for further considerations when providing future

ECHOs. This was conveyed though participant’s views on the need for appropriate funding

to participate in ECHO and who should participate in ECHO, both at the hub and the spokes.

Findings also illuminated the technological considerations that may be needed to enhance

the ECHO experience. (Figure 3)

Interpretative Theme 2A: There needs to be funding

Participants conveyed that for ECHO to successful in the future, there needs to be adequate

funding. The majority of participants highlighted the need for protected time to be funded for

future ECHOs:

“For this particular set of 10 [current ECHO clinics] we all got payment for it so we were able

to protect the time, we were able to set aside the time to do it. But if this was an ongoing

thing and we were looking at doing it once a fortnight, I suspect that the enthusiasm might

wane as people realise that life is very busy and if I'm not being funded to provide backfill for

this how am I possibly going to keep it going?”(DIA/12)

“One of my concerns for the future of it, is how do we manage to get protected time in what

is a very busy day, a very busy week, in order to get a group of people together? Time is a

big factor here for all of us in general practice and as it is obviously in hospital as well, and

the funding does give us protected time.” (DIA/12)

One participant also highlighted the need for appropriate funding to ensure the teaching

remains of a good quality:

“What it necessitates though [funding] is that the teaching is of high quality and it is really

worthwhile.” (DIA/15)

In summary, the presence of appropriate funding is needed to facilitate future ECHOs. This

is perceived to be needed not just to protect the time of clinicians to participate, but also to

ensure the high standard of education provided is being maintained.

Interpretative Theme 2B: Future ECHO Participants

Findings also conveyed perceptions of who should participate in future ECHOs to optimise

the learning gained. One participant highlighted that having more than one team member at

each spoke would be of benefit to clinical practice:

29    

“There was one session where my practice nurse joined me, and that was useful because

we were able to follow and complete the discussion outside of the ECHO session and it

meant that the learning was much more widespread within the practice. And I think that is

something I would encourage for the future, is we try and have teams of people meeting

together, not just one individual from the team, and I think that if you can create enthusiasm

in our practice then the standard of care will rise much higher.” (DIA/12)

Participants also illuminated their thoughts on who should be present at the hub for future

ECHOs. One participant conveyed how the presence of other specialist clinicians who

advise on conditions related to diabetes, would benefit the hub:

“Maybe also to consider other specialties with regards, for example... there's so much, foot

care, renal disease etc. obviously as part of the programme for a future date would be

helpful as well.” (DIA/15)

Accounts displayed how membership at the hub and spokes must be carefully considered to

ensure optimal multi-professional collaboration. It is also perceived that increased

membership at each spoke will increase the impact of learning and thus optimise patient

care.

Interpretative Theme 2C: Technological considerations

Findings were indicative of the need to consider the potential technological needs of

participants for future ECHOs. Some participants conveyed that technical issues from the

hub were often encountered:

“I thought the volume sometimes at the hub, or hearing what they were saying clearly at the

hub was difficult at my end, that may have been our problem at our computer end, and every

time the speaker [facilitator or teacher] moved or turned their head I seemed to lose the

sound quite a bit. (DIA/15)

“The sound in the hub, occasionally it would drop, again, as people were turning their head,

or it wasn’t the person directly in front of the camera, if it was someone else who was

contributing.” (DIA/12)

One participant expressed that they found it difficult to connect to ECHO through Wi-Fi in

their workplace:

30    

“One day I had to work on Wednesday and I tried to link in from work, but we don't have Wi-

Fi and the work computer wasn't set up, I didn't have the speaker and the headphones. It

was my own fault. I tried to bring my laptop in but couldn't connect up because the laptop

can't connect to work. So issues with NHS or GP connection that was all.” (DIA/14)

However, the technology was also considered as promoting optimal participation in ECHO

due to the flexibility it provided:

“The other thing just to say is I set myself a target of trying to be in a different location for

every session, and the car was parked in different car parks around Northern Ireland, so 3G,

4G signal was perfectly adequate, which I didn't expect it to be. I actually expected it to drop

out a lot more than it did. So I think the technology worked really well.” (DIA/14)

One participant perceived however that the success of the technology can often be

attributed to initial installation of the equipment and the ongoing administrative support:

“I think just the initial setting up of the various places where you're going to do ECHO is very,

very important and have shadow runs prior to the actual meetings to make sure everybody is

up and running would be helpful and less stressful. The ECHO administrator did a great job

[in coordinating these], I have to say.” (DIA/15)

Discussion

While the knowledge assessment for the diabetes ECHO did not demonstrate a substantial

improvement in knowledge scores and no statistical analysis was possible due to low

numbers, self-efficacy scores improved following the completion of the ECHO knowledge

network, and the feedback received from participants was extremely positive. When asked

about their overall views of participating in the network, 100% stated they had learnt ‘a lot’

through participation and enjoyed it, 100% felt it helped translate knowledge into practice

more than other teaching sessions they had been involved in and had improved the care

they provided for patients, and 100% would participate again. A potential reason for the lack

of improvement in knowledge score when compared with the perception of learning was the

assessment itself- it was difficult (average scores were 54% and 57%) and may not have

fully assessed the topics that were being taught in the hub. This was likely due to the tight

timeframe of setting up the evaluation before the ECHO started when the curriculum had not

been finalised, and could easily be addressed in future evaluations, with knowledge

assessments being written based on the curriculum that will be covered in the upcoming

31    

ECHOs sessions. It also highlighted the disconnect that existed before the ECHO between

what consultants in Diabetes perceived as the diabetic knowledge level among GPs.

The focus group findings highlighted the safe and effective learning environment that the

ECHO platform can provide for GPs working with patients with diabetes in the primary care

setting. ECHO optimised the delivery of care to patients through effectively enhancing the

knowledge and skills of GPs in relation to a variety of areas such as: diagnosing diabetes

and utilising new and innovative treatment options. Participants also illuminated the

importance of the hub and how effective it had been in motivating and encouraging the

spokes. For future ECHOs it has been suggested that specialists in other areas should be

invited to the hub, and spokes should consist of multiple members of the care delivery team.

ECHO was conveyed as an effective way to deliver clinically relevant education that

promotes patient care and it provided a key network of support for all involved.

The response rate to the evaluation was low and future evaluations should have

mechanisms in place to encourage participation, for example funding being dependant on

participation in the evaluation process or protected time being made available through

additional funding. This could help prevent bias and improve the generalisability of results.

While spoke members commented that they felt that participating in ECHO had improved the

care they provided for patients, it was not within the scope of this evaluation to look at the

impact on service delivery or patient care, and future studies should look to address this to

consider the cost effectiveness of ECHO through potential reductions in referral rates to

secondary care, improvements in diabetic control and reduction in long term complications.

Suggestions from evaluation

- The diabetic ECHO network should continue and be made available to more GPs

and other primary care professionals

- Technological issues need addressed including sound quality from the hub and

connectivity from NHS / GP sites

- Funding needs to be continued to allow protected time for participation and ensure

high quality teaching from specialists at the hub is maintained

- More than one participant at each site should be encouraged to enhance the

‘community of practice’ locally and to allow spoke participants to continue the

discussions and learning after the ECHO session is finished

- Future knowledge assessments should be based on the curriculum being taught

32    

- Participation in the ECHO network and receipt of funding could be dependent on

participation in the evaluation process, or funding should be made available to allow

participants to take part in the evaluation. This could improve the generalisability of

results and prevent bias

- Future studies should look at impact on service delivery and patient care, for

example referral rates to secondary care and diabetic control of patients managed in

the primary care setting, to determine if participation in the ECHO knowledge network

has a direct impact on these areas.

33    

Chapter 4 - Optometry ECHO

Background

Ophthalmology is a high demand specialty, typically accounting for 7-8% of all outpatient

appointments, both regionally and nationally, each year. In the Northern Ireland context, this

results in an excess of 100,000 out-patient appointments annually. Many ophthalmic

conditions are age-related, and many are long-term conditions (LTC’s) where ‘cure’ may not

be possible, and management to maintain useful vision is the goal. Glaucoma and macular

degeneration are two such conditions.

The Optometry ECHO had the objective of, providing primary care optometrists a safe space

to improve their knowledge base, and in turn helping them to better manage patients with

glaucoma or macular eye disease (age-related macular degeneration - AMD). By tele-

mentoring and case-sharing, ECHO aimed to enhance the available knowledge and clinical

diagnostic skills in primary care, thus helping to improve case-handling and referral patterns.

The Intervention

The Optometry ECHO knowledge networks were on Friday mornings from 9.30 – 11.00 am

and covered topics relating to AMD and Glaucoma. Twenty-one Optometrists from across

the region took part in 12 ECHO sessions to gain additional knowledge, skill and confidence

in the diagnosis and management of patients with these long term ophthalmic conditions.

The ECHO curriculum supported these primary care providers in elements of care, such as

diagnosing macular disease ‘at the margins’ and how and when to refer. For more details

see table 5.

Hub - The hub members included academic clinicians who were specialists in the identified

sub-specialties of AMD and Glaucoma. These included the HSCB Ophthalmic Service lead,

a Clinical Senior Lecturer and Consultant Ophthalmologist, a Professor of Ophthalmology

and Consultant Ophthalmologist, a Biomedical Scientist, a Primary Care Optometrist, a

university research Optometrist and a Clinical Optometric Adviser from the HSCB.

Spokes - The spokes were selected because they are all holders of the College of

Optometrists’ Professional Certificate in either (i) glaucoma or (ii) medical retina. The ECHO

programme was alternately around these sub-specialities.

34    

Table 5 Optometry / Ophthalmology ECHO

Optometry/Ophthalmology ECHO

Clinical

Champion

Facilitator

Educators

Admin support

Raymond Curran

Margaret McMullan

Prof Agusto Azuaura-Blanco (Glaucoma)

Mr Michael Williams (AMD)

Dr Ruth Hogg: Lecturer QUB

Dr Linda Knox

Dr Gerry Mahon: Biomedical Scientist QUB/BHSCT

Claire Armstrong

Frequency of

clinics

Weekly (12)

Fridays at 9.30-11am

Training day Friday 13th November 2015 9.30-2.00 at Jennymount

Sessions

20, 27 November

11, 18 December

8, 15, 22, 29 January 2016

5, 12, 19, 26 February 2016

Hub members

Mr Michael Williams:Clinical Senior Lecturer, Honorary Consultant

Ophthalmologist QUB/BHSCT

Prof Agusto Azuara-Blanco: Clinical Professor/Consultant

Ophthalmologist QUB/BHSCT

Dr Ruth Hogg: Lecturer QUB

Dr Gerry Mahon: Biomedical Scientist QUB/BHSCT

Mr Alan Rundle: Primary Care Optometrist

Raymond Curran: AD of Integrated Care, Head of Ophthalmic

Services, HSCB

Mrs Margaret McMullan: Clinical Optometric Adviser, HSCB

ECLO’s (Eye care liaison officer) – invited as required pertinent to

curriculum

Hub costs

Based on 12 weeks of clinic and 4 hours clinic time (inc prep) and one

session of training

£11,481 paid to the 5 HSC Trusts on 1st December (for participation in

all ECHO networks as required)

Spoke members 21 Primary care optometrists to allow 1-1.5 hours

Spoke costs £165 per ECHO session

35    

Spoke

Equipment

8 webcam & microphones/headsets

Majority using their own equipment

Equipment costs 8 webcams & microphones/headsets x £50 = £400

Curriculum Developed. Alternating between glaucoma and AMD with a session on

Optical Coherence Tomography

• AMD - Presentations, photos and images to make decisions on

• Glaucoma - The optic disc and glaucoma: Importance of disc

size. Recognising a normal disc. The ISN'T rule. Difficult

discs. Tilted discs. Glaucoma suspects

• AMD - Diagnosing macular disease at the margins. How and

when to refer

• Glaucoma - Fields. Some dots are missing: Is it glaucoma?

What do I do? And: is the patient fit for driving?

• Glaucoma - IOP: Ocular hypertension - OHTS study - central

corneal thickness - refractive surgery

• AMD - What should be done for patient with early AMD? Are

supplements worth it?

• Glaucoma - Systemic medications and glaucoma. Compliance-

adherence: what to do?

• AMD - OCT Machines: “Believers, Non-Believers and Doubters”

why buy one/why not buy one/what to do with one?

• AMD - Neovascular AMD: diagnosis, monitoring and

management. What to tell patients about it?

• Glaucoma - Rare Discs AMD - Dry AMD and the Eye Clinic

Liaison Officer: how patient’s needs are addressed

• Glaucoma – Gonioscopy

Evaluation Methods

Evaluation was undertaken as described in Chapter two. All participants completed their

pre-test evaluations at the training day, and were emailed their post-test evaluations which

were returned by post or email.

Results

Twenty one spoke participants completed the pre-ECHO assessments (response rate

100%), with 11 (response rate 52.4%) completing the post ECHO assessments (only 10

completed the retro-pretest evaluation of self-efficacy). Demographic data for all participants

are shown in table 6. For the rest of the results only the participants who completed the pre

and post evaluations are included.

36    

Table 6 Optometry Demographic data Pre ECHO Evaluation

Range Totals Age 20-30 2 31-40 7 41-50 10 51-60 2 60+ 0 Gender Male 4 Female 17

Qualifications BSC Honours Optometry 18

Other 3 Do you already hold Prof Cert in Yes 20 Glaucoma? No 1 Do you already hold Prof Cert in Yes 3 Medical Retina? No 18 Years in profession <5 0 6-10 4 11-20 8 21+ 9

LCG area of practice Belfast 8 Southern 2

South Eastern 2 Northern 7 Western 1 North & West 2 Current Area of Work Rural 5 Urban 10 Mixed 6 Current area of work

Independent Practice 14 Corporate / Multiple (>3 practices) 6

Do you have access to OCT in your Yes 6 practice? No 15 Do you regularly use OCT in your Yes 5 clinical practice? No 16 Is the Optometry practice you work in Yes 17 connected to FPS BSO web portal? No 4

Knowledge and Self-efficacy Assessments

The mean scores of knowledge improved between the pre-ECHO and post-ECHO

assessments. Average knowledge scores improved from 23.7 to 27.6 (out of a possible 38

marks); from 62% to 73%. Only one participants score dis-improved, one was the same and

the rest improved. Due to the small sample, non-parametric tests were used to analyse the

37    

data. Wilcoxon results indicated a significant improvement in knowledge (p=0.008) pre and

post-ECHO. More details are shown in table 7 and figure 4.

Eleven participants completed the pre and post-test self-efficacy evaluations, with ten the

retro-pretest evaluation. Self-efficacy results improved (table 7), with a lower average retro-

pretest evaluation score than pretest score (in all but one area), and a higher average score

for the post-test evaluation in all areas. The different domains are shown in table 8. Due to

the small sample, non-parametric tests were used to analyse the data. A Friedman test

showed that overall self-efficacy improved significantly across the three (pre, retro-pre and

post) time points (p=0.006); post hoc tests using a Bonferroni adjusted alpha (p=0.017) to

control for Type I errors revealed statistically significant improvement in post ECHO self-

efficacy (p=0.007) and between Retro-Pre and Post ECHO self-efficacy (p=0.008). More

details are shown in table 7 and figure 5.

Table 7 Optometry ECHO Knowledge and total self-efficacy results

Pre-ECHO Retro-pre ECHO Post ECHO

Centiles 25th Median 75th 25th Median 75th 25th Median 75th

Knowledge

19.0 24.0 25.5 - - - 25.0 27.0 32.0

Self-efficacy

3.29 3.57 4.29 3.07 3.43 3.93 4.21 4.23 4.71

Figure 4 Box Plot for Optometry Knowledge Scores

38    

Table 8 – Optometry ECHO Self efficacy assessment - Participants self-rated confidence in

each area (1 - not confident at all, 5 - very confident)

Question Pre test

Average

(Range)

Retro Pretest

Average

(Range)

Post test

Average

(Range)

To examine and assess optic nerve head

appearance

4.00

(3-5)

3.80

(3-4)

4.60

(4-5)

Identify signs of glaucomatous damage to

the optic nerve head

4.10

(3-5)

3.90

(3-5)

4.70

(4-5)

Interpret visual field plots 3.80

(2-5)

3.70

(2-5)

4.40

(4-5)

Identify signs of macular disease (both

types dry and wet)

4.00

(3-5)

3.30

(2-4)

4.50

(4-5)

Differentiate the clinical appearance of wet

AMD and Dry AMD

3.70

(2-5)

3.80

(2-5)

4.50

(4-5)

Interpret OCT scans with respect to

macular changes

2.70

(1-4)

2.30

(1-4)

3.80

(3-5)

Overall I am confident in my ability to do

my job well and provide a safe and quality

service to patients

4.30

(3-5)

3.80

(3-4)

4.60

(4-5)

Figure 5 Box Plot for Optometry Self Efficacy Scores

39    

General Feedback of Optometry ECHO

Box 2 demonstrates participant’s views on ECHO in general. Participants were very positive

about their experience of ECHO, with 100% having learnt through participation, 100% felt it

helped translate knowledge into practice more than other teaching sessions they had been

involved in, and 100% would recommend it to others and participate again.

Box 2 General ECHO Optometry Results

1. Please rate each on a scale of 1-5 the quality of learning / usefulness from each area (1- poor, 5- excellent)

1 2 3 4 5 Review of previous session 0 0 0 3 7 Presentations 0 0 0 0 11 Case based discussions 0 0 0 2 8

2. Overall do you feel you have learnt through participating in ECHO?

A lot 10

A little 1 No 0

3. Did you find participating in ECHO enjoyable?

A lot 10

A little 1 No 0

4. Do you think that participating in ECHO has improved the care you provide for patients?

A lot 7

A little 3 No 0

5. Do you think the format of ECHO helps translate knowledge from teaching into practice more than other teaching sessions you have been involved in?

Yes 11

No 0

40    

6. Would you recommend ECHO to other healthcare professionals in your area?

Yes 11

No 0

7. Have you used any of the online resources via Moodle, and if yes have you found these useful?

Used and found useful

Used and found NOT useful

Power point presentations 4 1 Video of the teaching sessions 6 1 Video of case presentations 4 1 Other supporting materials 8 2

8. Regarding ECHO technology…

Agree Disagree Unsure It has given me access to education that would have been hard to access due to geography 8 1 2 It was a good medium to access teaching / education at a different location from where I work 11 0 0 Any technical difficulties were acceptable and did not put me off participating in ECHO 11 0 0 Any technical difficulties did not significantly reduce my learning 11 0 0

9. How do you rate your overall ECHO experience? (1- poor, 5- excellent)

1 2 3 4 5

0 0 0 1 10

10. Would you participate in ECHO sessions in the future if the opportunity arose?

Yes 11

No 0

41    

Focus Group

Ten optometrists participated in the focus group from the spokes for the Optometry ECHO.

Analysis of the focus group data uncovered three overarching themes, each with their own

descriptive and interpretative themes that are outlined in thematic diagrams. Overarching

theme 1, ECHO Enhanced Clinical Knowledge and Skills, explored the reasons why

participants perceived that ECHO optimised their clinical practice (figure 6). Interpretative

themes 1A, 1B and 1C highlighted the contributing factors that resulted in participant’s

perceived enhancement of clinical practice. Overarching theme 2, ECHO Exceeded

Expectations and Changed Misconceptions, highlighted the preconceptions participants had

before ECHO commenced and how these altered throughout the project (figure 7).

Interpretative themes 2A and 2B illuminated the intricacies of why and how perceptions

changed in a positive way. Overarching theme 3, Consideration for the Future of ECHO,

displayed the key conceptions that participants perceived should be considered for future

ECHOs (figure 8).

Overarching Theme 1: ECHO Enhanced Clinical Knowledge and Skills

This theme encompassed how participants perceived ECHO to be an education platform

that enhanced their clinical knowledge and skills (figure 7). Interpretative theme 1A depicted

how ECHO provided an environment that facilitated the development of interdisciplinary

relationships and communication, within the field of optometry and ophthalmology.

Participants also perceived that ECHO optimised the care they delivered to their patients

through the knowledge they had gained, and also through their increased confidence in

making appropriate referrals. Interpretative theme 1C illuminated how ECHO also created a

safe learning environment for participants. Findings highlighted that this was perceived to

have been achieved by the hub members encouraging learning and treating all queries with

respect. The learning environment was also enhanced by the relationships that were built

between members of the spokes.

42    

Figure 6 Theme 1 ‘ECHO Enhanced Clinical Knowledge and Skills’ for Optometry ECHO

Descriptive Themes Interpretative Themes Overarching Themes

1 ECHO Enhanced

Clinical Knowledge and Skills

1A Nurtured

interdisciplinary relationships and communication

Strengthened the link between primary and

secondary care providers

"Relationship between ophthalmologists and

optometrists has increased because of ECHO."

1B Optimised Patient

Care

Increased optometrist confidence in their skills

Reduced unnecessary referrals

1C Created a safe learning

environment

Expertise at the Hub

Encouraged sharing of knowledge

Built relationships between optometrists

"No questions are silly."

43    

Figure 7 Theme 2 ‘ECHO Exceeded Expectations and Changed Misconceptions’ for

Optometry ECHO

Descriptive Themes Interpretative Themes Overarching Themes

Figure 8 Theme 3 ‘Consideration for the Future of ECHO’ for Optometry ECHO

Descriptive Themes Interpretative Themes Overarching Themes

2 ECHO Exceeded Expectations and

Changed Misconceptions

2A "Didn't know what to

expect."

Would it be too formal?

How would the technology work?

2B ECHO challenged

misconceptions

Addressed concerns about technology

"Afraid that knowledge may not

be up to par."

3 Consideration for

the Future of ECHO

3A Changes to be made

"Cutting it fine to have your homework

done."

Accessibility of resources

3B Furture ECHO

participants

Potential for Enhanced

relationships between optometrists and GPs

Relevant to Other professionals in the

community

44    

Interpretative Theme 1A: Nurtured interdisciplinary relationships and communication

The majority of participants described how they perceived the relationships that were

fostered within the ECHO environment. Many felt that the ECHO platform provided a forum

which strengthened the links between primary and secondary care providers. Participants

perceived that this was optimised through inter-professional learning about the role of

optometrists in the community, and ophthalmologists in the secondary care setting:

“They [ECHO Hub] also learnt a lot from us [Spokes], what it's like in the front line

[Community setting] and the decisions you have to make. So I think ECHO has been

great for understanding, we understand more about how they work [Secondary Care

Setting], what happens with patients but they also understand more about what it's like in

the community.” (OP/15)

Participants perceived that through raising awareness of professional’s roles, ECHO actively

participated in facilitating interdisciplinary relationships:

“Realistically what it's [ECHO’s] doing is building relationships, where beforehand there

was this barrier of a hospital that stops us [Community Optometrists] from building that.”

(OP/1)

Findings also illuminated that ECHO enhanced relationship building between optometrists

and ophthalmologists:

“I think everybody would agree that certainly the relationship between ophthalmologists

and optometrists has increased because of ECHO. I'm sure that the ophthalmologists

would probably think a lot more about our knowledge and certainly understand what

we're looking at in practice, and we also understand more about what they're looking at

and what they would want to see and that relationship has been fantastic.” (OP/1)

Overall, these findings suggested that the unique learning environment that was nurtured by

the ECHO platform facilitated the building of relationships between professionals across the

primary and secondary care setting. This led to direct enhancement of interdisciplinary

awareness of the key role various clinicians have in patient care.

45    

Interpretative Theme 1B: Optimised Patient Care

Participants also envisaged that the ECHO program had a positive impact on patient care.

Many felt this was due to ECHO increasing the perceived confidence they had in relation to

their clinical skills. This resulted from the education provided at ECHO:

“I felt so much more confident on the glaucoma case about what to look out for and

what way to manage it.” (OP/14)

“I think we've done macular and glaucoma in depth and it was fantastic, what we've

learnt out of it and certain learning points, I definitely feel more confident managing

this in practice.” (OP/15)

For many this was a very positive outcome from ECHO as they had hoped taking part would

improve patient care:

“I definitely went into ECHO hoping that it would be able to improve the care that I

provide to patients, and I definitely feel that I have.” (OP/12)

Findings additionally illuminated the perceived impact of ECHO on referrals to secondary

care services. The majority perceived that the learning they had achieved through ECHO

enabled them to make more appropriate referrals in relation to various conditions:

“Our disc assessment, I think, is better, our ability to reason and to look and decide

whether we want to keep the patient in practice or refer, I think it's better. I think it'll

reduce, certainly for me it will reduce some unnecessary referrals but it will also help

me pick up on things that I really need to refer.” (OP/1)

“I think I feel more confident now, when I'm doing things with patients, to look close,

especially looking at discs or looking at the macula more closely and thinking 'what

else could be going on that we can't see?' that it's perfectly okay to send somebody

for further investigation without necessarily knowing exactly what you can see or

what you can't see, but also being a bit more confident about what you can see and

thinking 'yeah, this is definitely something that needs sorted out' rather than being a

bit woolly about what you're sending in.” (OP/13)

The majority of participants also perceived that the learning gained had helped them to feel

more confident in dealing with particular conditions. This resulted in participants reducing

unnecessary referrals:

46    

“I found that knowing the pathway and knowing the treatment that our patients were

going to be receiving in the hospital helped me to recognise when I'm actually more

happy and confident to keep them in practice and when it's safe to do that, and that's

been a really useful tool. So of course, it's lovely to be able to refer them when I'm

sure and I have the confidence to do that, but also the reverse, to keep the patients

here and to have confidence in our own ability and skill set.” (OP/11)

Findings also highlighted perceptions that reducing the number of unnecessary referrals

would result in decreased pressure on secondary care services. One participant expressed

the perception that more education through ECHO on anterior segment conditions would

help decrease the demand for acute eye service referrals:

“If we were more confident about dealing with these [anterior segment conditions] the

pressure on acute eye services would be lessened.” (OP/9)

In summary, these findings have highlighted how the learning gained through the ECHO

platform enhanced the confidence and clinical skills of the participants. This resulted in

improved patient care due to raised awareness amongst participants of how to act on and

manage specific conditions, and appropriately refer to other services. Awareness of

appropriate referral criteria also resulted in the perception that ECHO could reduce referrals

to secondary care services. Thus, enhancing the provision of care in the primary care setting

and ultimately reducing the demand on secondary care services.

Interpretative Theme 1C: Created a Safe Learning Environment

Many participants expressed that the expertise at the hub added to the positive learning

environment provided through the ECHO platform. Findings illuminated how the

ophthalmology presence at the hub was beneficial to the spokes:

“I personally found it really beneficial, especially the two ophthalmologists that sat in

on a lot of the sessions, you were getting feedback from them when they were

receiving our referrals, and local protocols were particularly beneficial. So I certainly

definitely felt I had benefitted from it.” (OP/12)

The majority also expressed that alongside the hub, the spokes provided encouraging peer

support and advice at the sessions:

47    

“Just the fact that you had an ophthalmologist there and you have your peers, that

you can get an immediate answer, it's just brilliant, because quite often you're just left

wondering about things, 'am I doing the right thing?' and so it’s just to have that

immediate response was just great.” (OP/14)

The majority of participants also referred to the important role of the facilitator at each

session. Findings indicated that participants perceived the facilitators to be key in optimising

the learning gained at the ECHO session:

“Both the facilitators had listened to what we were looking for them to talk about at

ECHO and they really stuck to that, they really looked at it, and they listened to that

and then they addressed them, and it was incredibly useful.” (OP/1)

The ECHO hub was also regarded by participants as providing a safe learning environment

through the support provided by the hub when asking questions:

“I think every question that was brought up, it was made to feel significant and you

felt you could bring up any issue, which I think is very important.” (OP/14)

Alongside the perceived benefits of ECHO to those directly involved, participants also

expressed that the learning they gained through ECHO was shared amongst other

professional colleagues. Many expressed that colleagues would come to them for guidance

due to their involvement in ECHO:

“Colleagues that weren't involved in the ECHO session, they would sometimes come

to me with cases that they're not so sure about and I definitely feel more confident in

providing answers to their queries as well.” (OP/12)

This resulted in a perceived benefit to the other colleagues’ patients:

“I found other colleagues asking me things and it was something that I could maybe

bring as a question in ECHO, so the whole team did learn, and because I work in

different practices, that was spread among different practices too. So yeah, it's

helped in the care of my patients and in the patients of my colleagues too.” (OP/15)

The majority of participants highlighted that the ECHO program allowed for relationship

building between the spokes. Many expressed that they felt less isolation within their field

48    

due to building relationships with the various optometrists who were partaking in the

program:

“So I think, from a relational point of view, it's been fantastic as well, as well as

getting to know the spokes. I had never met X [member of the spoke] and yet last

week I walking into a room and chatted to him as if I had known him for months. So,

at the end of the day, I've got to know another 20 optometrists that I would feel very

confident going and talking to if I was to meet in a meeting. So I think that's been

fantastic.” (OP/1)

“I think with meeting all the spoke optometrists too, you know, we know names but

we never see each other really, so it was great to put a face to a name.” (OP/09)

Accounts demonstrated that participants viewed ECHO as a safe learning environment that

was enhanced through the expertise, and skilled facilitation, present at the hub. Key learning

was also nurtured from the peer support provided by the spokes and the ability to build

relationships with other colleagues. Evidence also displayed how ECHO can often reach

wider than the participants themselves. Participants shared the learning gained with other

colleagues and thus perceived the impact of ECHO to go beyond them and their patients to

the patients of others.

Overarching Theme 2: ECHO Exceeded Expectations and Changed Misconceptions

Overarching theme 2 encompassed how participants perceived ECHO as exceeding their

expectation and altering the preconceptions they held before participating (figure 7).

Interpretative theme 2A conveyed how participants were wary of ECHO before it started due

to a lack of awareness of what it specifically entailed. However, interpretative theme 2B

encapsulated how the misconceptions participants held were changed throughout the

program.

Interpretative Theme 2A: “Didn’t know what to expect.”

Many participants conveyed how they were concerned that ECHO would be too formal.

However this perception changed:

“I think you probably did think it might be a little bit more formal than it was, and I

think it was really, like the others have said, because it was quite informal, even with

the ophthalmologists in there and their lectures and things, it never felt so formal that

you couldn't interact and ask questions,” (OP/13)

49    

This participant expressed that the more comfortable environment was directly related to the

expert communication skills of the hub members:

“The ophthalmologists that we had, seemed to be very good communicators. The

level they were pitching things at, the things they were saying, the informality of the

whole thing, I think they were extremely good at communicating.” (OP/13)

Findings also highlighted that many had concerns about the influence of technology on the

ECHO program. Many were unsure if this would negatively impact on the interaction

between the spokes:

“At the beginning I was unsure about the technology and I thought we would be very

detached from one another.” (OP/9)

However, for this particular participant this concern was unfounded as the program went on:

“But as the weeks went on, I think the spokes started to talk more to one another and

ask each other questions.” (OP/9)

Some were concerned about the use of technology due to their own computer skills:

“I was a bit apprehensive, first of all, before using it because I'm not a very computer

person.” (OP/20)

Overall, the concerns regarding the use of technology and the format of the sessions were

reduced as participants partook in the program. This was facilitated by the ECHO

equipment, and also the members of the hub and spokes.

Interpretative Theme 2B: ECHO Challenged Misconceptions

The misconceptions held by participants before ECHO were reduced throughout the

program. The reasons for this are illuminated within interpretative theme 2B. Accounts

highlighted how initial fears regarding technology through the IT support given prior to

commencing the program:

“But they came out and installed the camera for me in my office, and you literally just

go in, type in the code and up it pops, and it's been fantastic, no bother. So very

good.” (OP/20)

50    

“I couldn't find any fault with it at all. It was as easy as anything. I literally pushed the

button and the thing, the camera comes on, the sound works. I never had any bother

with it at all. Once you put your code in it just turns everything on by itself, it was

great.” (OP/13)

Participants also found it useful to use alternative tablet and Smartphone devices to

participate in the ECHO clinics:

“I think it was very easy to use. I just used it on an iPad mini and I'm just amazed

how easy it is to use, and I think people have been even accessing it on a phone, so

it's been fantastic technology.” (OP/15)

Using mobile devices also allowed participants to have flexibility in where they joined the

clinic:

“Because it's on the iPad, and I do locum and I've done it in three other practices and

at home, so it's been amazing, you take your iPad and connect to the Wi-Fi, and you

can really do it anywhere and that's the amazing thing about it.” (OP/15)

It was expressed by some participants that they held the misconception that ECHO would

not be aimed at a high enough level to meet their learning needs. However one participant

highlighted that this was not the case:

“I think my concern going into ECHO was that it was going to be pitched at a very low

level and we would end up listening to an awful lot of stuff that we already knew, and

it turned out to be not like that at all.” (OP/1)

On the other hand, some participants expressed that prior to partaking in ECHO many may

feel they do not have the knowledge required. However it was perceived that ECHO does

provide a safe environment for learners at various levels:

“Others may be afraid that their knowledge may not be up to par, and I think it's

important that people do realise that ECHO is a safe environment and that you're not

going to be wrong, as such, and you can get your questions answered without

anybody judging you, which I think is very important.” (OP/14)

In summary, findings conveyed that misconceptions regarding ECHO were soon altered

after the program commenced. This was facilitated by the ease of use of IT equipment and

the safe learning environment provided by the ECHO platform.

51    

Overarching Theme 3: Consideration for the Future of ECHO

Findings were indicative of the need for further considerations when providing future ECHOs

(figure 8). This was conveyed though participant’s views on changes to be made that were

discussed in interpretative theme 3A. Findings also illuminated the role of ECHO for other

participants and this was demonstrated in interpretative theme 3B.

Interpretative Theme 3A: Changes to be made

Participants conveyed that in order to effectively repeat the ECHO program, more

consideration may be needed regarding the amount of time participants are given to review

the case presentations. Many expressed that they did not feel they were given sufficient time

to prepare for the session:

“For me personally, the information was usually emailed, but it tended to come out on

the Thursday, just the day before ECHO, and I work late on a Thursday and it was

just you were getting home and by the time you were looking at it, it was maybe 9:30,

10 o'clock at night, which was when you were a bit more tired. So if we had it

maybe... it did sometimes comes out on the Wednesday, but if it did come out a

couple of days beforehand it would be useful to have a bit more time to look over the

information.” (OP/20)

“There were some weeks that you were cutting it quite fine to have done your

homework, so it would be useful. It's not always possible but it would have been

useful to have had it a bit earlier.” (OP/11)

“Even 24 hours earlier, if we had the information 24 hours earlier. I, as well, struggled

and I was looking at the case presentations at 9 o'clock before ECHO started and it

would have been better to have it a day earlier.” (OP/9)

Many participants also conveyed that they had difficulty accessing online resources that

were made available to those participating in ECHO:

“I found a little bit of difficulty getting onto the site with the resources.” (OP/09)

“I did have a look on the online resource site, it did take me a while, I did find some

resources but it was a wee bit of ‘hoking’ around to find it.” (OP/20)

“I did find it difficult to get on to the online resource site and some of the links were

difficult to open.” (OP/19)

52    

In summary, findings highlighted the need for future ECHOs to consider the time participants

are giving to prepare for the session. Accessibility of resources that are made available

throughout the program also needs to be addressed to ensure participants receive optimal

information and learning.

Interpretative Theme 3B: Future ECHO Participants

Findings indicated that ECHO is perceived to be of benefit across multiple professions, and

healthcare settings. Participants conveyed that ECHO may be a beneficial platform to use to

enhance interdisciplinary working between optometrists and GPs:

“So communication can only be good, and you could even imagine this [ECHO]

between us [optometrists] and GPs, perhaps, or within different professions, which

would perhaps be beneficial as well.” (OP/15)

Ultimately this could promote partnership working in the primary care setting:

“I think GPs don't necessarily understand what we know, some of the time. The GPs

in the rural area where I work, send me a lot of patients but I'm not sure that GPs in

general understand how much more useful we can be to them than they think we

are.” (OP/13)

Participants also envisaged that the ECHO model would be beneficial across other

disciplines:

“I would imagine this model could be shared out among any profession. In any

healthcare profession there would be cases that could be brought and discussed [at

ECHO] with perhaps somebody, maybe a consultant with more junior doctors, in the

same situation, we would have peers and also those who can provide answers. So I

imagine it would be quite easily shared out.” (OP/14)

“I was saying to a dentist a few months ago, he's not long qualified but he's taken a

different route than most dentists, he's gone in to hospital dentistry and he's seeing a

lot of very interesting cases, and one comment he made was I'm glad I'm not out in

the community setting. So it sounded to me like oh dear, that's quite like our

profession. So I think dentistry, yeah, a general dentist would probably gain the same

way that we have from ECHO.” (OP/9)

53    

Interpretative theme 3B illuminated the benefits of ECHO across disciplines and how it could

promote partnership working, especially within the primary care setting.

Discussion

Optometrists who participated in the ECHO knowledge network demonstrated statistically

significant improvements in knowledge and self-efficacy following the 12 ECHO sessions.

When asked about their views on ECHO in general, 100% stated they had learnt through

participation, 100% felt it helped translate knowledge into practice more than other teaching

sessions they had been involved in, and 100% would recommend it to others and participate

again.

The focus group findings highlighted the safe and effective learning environment that the

ECHO platform provided for optometrists working within the primary care setting. ECHO

optimised the delivery of care to patients through effectively enhancing the interdisciplinary

relationships between optometrists and ophthalmologists. Patients also benefited from the

enhanced knowledge, skills and clinical confidence that participant’s experienced as a result

of ECHO. Although participants often had their doubts about the effectiveness of ECHO due

to technological concerns, or a desire to maintain ‘credibility’ amongst their peers, the ECHO

pilot soon changed these perceptions by the reassuring and encouraging environment

provided by the hub, and by the hub facilitators. ECHO was perceived as an effective way to

deliver clinically relevant education that will promote safe and quality patient care and reduce

unnecessary referrals to secondary care services. It provided a key network of support for all

involved and their colleagues. Irritants such as the tight timeframe of receiving ‘homework’

24 hours before the ECHO session leaving little time to prepare and the accessibility of

resources on-line should be easily addressed in future networks.

The response rate to the evaluation was lower than expected (52.4%) and future evaluations

should have mechanisms in place to encourage participation, for example; funding being

dependant on participation in the evaluation process, protected time being made available

through additional funding and processes in place to encourage non-responders.

While spoke participants felt that participating in ECHO had improved the care they provided

for patients and had improved the appropriateness of referrals to secondary care, it was not

within the scope of this evaluation to look at the impact on service delivery or patient care,

and future studies should look to address this to consider the cost effectiveness of ECHO

through potential reductions in referral rates to secondary care and improvements in

optometry care for patients in the community.

54    

Suggestions from evaluation

- The optometry network should continue and be made available to more optometrists

with consideration for involvement of other primary care professionals where

relevant.

- Case presentations and ‘homework’ should be made available earlier to allow more

time for preparation.

- Training on how to access online resources needs to be improved.

- Funding needs continued to allow protected time for participation.

- Participation in the ECHO network and receipt of funding should be dependent on

participation in the evaluation process, or funding should be made available to allow

participants to take part in the evaluation. This should improve the generalisability of

results and prevent bias.

- Future studies should look at impact on service delivery and patient care, for

example referral rates to secondary care and management of patients in the primary

care setting, to determine if participation in the ECHO knowledge network has a

direct impact on these areas.

55    

Chapter 5 - Nursing Homes ECHO

The Intervention

The Nursing Home ECHO knowledge network supported nursing home staff throughout

Northern Ireland to improve their knowledge and skills in the care and management of

patients with a wide range of healthcare needs. Issues such as renal failure, palliative and

end of life care, drug management, and heart failure were discussed with a team of multi-

disciplinary professionals.

To enable the Nursing Homes to embed changes in their systems, the HSC Safety Forum

also supported the nursing home staff to build knowledge in quality and safety improvement

tools and techniques during the ECHO clinics. This included the Model for Improvement, the

Plan Do Study Act (PDSA) Cycle, and measuring for improvement, along with a range of

other ideas for change.

The sessions were on a Tuesday afternoon from 2-4pm. This network completed 10 ECHO

sessions. More information on the set up of this ECHO is shown in table 9.

Hub - In the ECHO hub there were a number of HCPs including a Palliative Medicine

Consultant, a Lecturer in Palliative Care, the HSC Safety Forum Clinical Director and Patient

Safety Advisor, a Pharmacist, a Dietician and other HCP’s as required including a

Physiotherapist, Occupational Therapist, Speech and Language Therapist, Chaplain and

Social Worker.

Spokes - Recruitment was via three methods

1. Nursing Homes were recruited through the existing HSC Safety Forum Nursing

Home collaborative.

2. Invitations were also send to homes within the Four Seasons and Care Circle

organisations.

3. Expression of interest were sought through members of the Regional Transforming

Your Palliative Care Programme Board.

There were a total of 26 nursing homes involved with up to 70 staff participating at the

spokes in any one session.

56    

Table 9 Palliative Care in Nursing Homes ECHO

Palliative Care – Nursing Homes Clinical Champion/ Facilitator Admin support

Max Watson & Sue Foster Ciara McClements/ Rebecca Donnelly

Frequency of clinics

10 sessions – 2 hours each (Tuesdays 2pm-4pm) Weekly initially (November) and then every other week

Session dates

November – 3rd, 10th, 17th, 24th December – 8th January – 12th, 26th February – 9th, 23rd March – 15th

Training 15th September and 29th October Hub members

Facilitator/Educator – Max Watson / Sue Foster and other NIH staff as required (Consultant / GP, Nurse, Pharmacist, Dietitian, Physiotherapist, Occupational Therapist, Speech and Language Therapist, Chaplain, Social Worker) Corrina Grimes - PHA Janet Haines – Woods - HSC QI Safety Forum Gavin Lavery – HSC QI Safety Forum

Hub costs Hospice staff costs Spoke members Staff from 26 Nursing Homes across Northern Ireland. Spoke costs

Nursing Homes will be paid an ‘Education Grant’ in February of approximately £1500

Equipment needed for spokes

21 webcams required (9 for Four Seasons) 12 installed Extra equipment required for Four Seasons: 2nd hand pc - £120 plus vat KVM switch - £37 plus vat USB wireless adapter (may be required) - approx £12 So approximately £200 x 9 homes = £1800

Total Spoke Equipment costs

Approx £ 5,000

Curriculum development update

1. Communication 2. Symptom Management in Palliative & EOL Care 3. Pain Management 4. Ethical Issues 5. Drug Management 6. Recognising Death & Dying 7. Delirium 8. Behavioural & Psychological Symptoms in

Dementia (BPSD) 9. Heart Failure

57    

Evaluation Methods

Evaluation was undertaken as described in Chapter 2. There were different assessments for

registered and unregistered staff in both knowledge and self-efficacy. All participants

completed their pretest evaluations at the training day or in their individual nursing homes

under the supervision of their manager, and managers were emailed the post-test

evaluations to administer to staff and then return them by post or email.

Results – Nursing Home ECHO

Sixty two registered and 30 unregistered spoke participants completed the pre ECHO

assessments (92 total), with eight registered nurses (response rate 12.9%) and two

unregistered staff (response rate 6.7%) completing the post ECHO knowledge assessments.

Seven registered and three unregistered staff completed the post ECHO self-efficacy

assessments. Demographic data for all participants are shown in table 10 (registered staff)

and 11 (unregistered staff). For the rest of the results only the participants who completed

the pre and post evaluations are included.

Table 10 Nursing Home Demographic data Pre ECHO Evaluation for Registered Staff

Range Totals

Age 20-30 15

31-40 6

41-50 22

51-60 13

60+ 6

Gender Male 2

Female 60

Profession Registered Nurse 36

Team Leader/ Deputy Manager 12

Manager 13

Other 1

Years in profession <5 22

6-10 4

11-20 7

21+ 29

Current Area of Work Rural 8

Urban 17

Mixed 29

Not stated 8

58    

Table 11 NH Demographic data Pre ECHO Evaluation for Unregistered Staff

Range Totals

Age 20-30 13

31-40 6

41-50 7

51-60 2

60+ 0

Not stated 2

Gender Male 3

Female 27

Profession Healthcare assistant 23

Other 7

Years in profession <5 14

6-10 9

11-20 5

21+ 2

Current Area of Work Rural 8

Urban 6

Mixed 9

Not stated 7

Knowledge and Self-efficacy Assessments

The mean scores of knowledge improved between the pre-ECHO and post-ECHO

assessments. For registered staff average knowledge scores improved from 41.8 to 45.6

(out of a possible 50 marks); from 84% to 91%. No participants score dis-improved, two

were the same and the rest improved. For unregistered staff one participants score

improved from 76% to 84%, the other participant dis-improved by 1 mark from 90% to 88%.

Due to the low response rate no statistical analysis was possible.

Seven registered and three unregistered participants completed the three self-efficacy

evaluations. Self-efficacy results improved (table 12 for registered and table 13 for

unregistered staff), with a lower average retro-pretest evaluation score than pretest score in

all areas for registered staff and in all but one area (heart failure patients) in unregistered

staff, and a higher average score for the post-test evaluation in all areas. For registered

staff, overall confidence, happiness and feeling of support improved, however, so did stress

levels. Due to the low response rate no statistical analysis was possible.

59    

Table 12 – NH ECHO Self efficacy assessment for Registered Staff- Participants self-rated confidence in each area (1 - not confident at all, 5 - very confident)

Question Pre ECHO Average (Range)

Retropre Average (Range)

Post Average (Range)

Understand the principles of palliative care and apply them in my daily work

3.93 (1-5)

3.50 (2-5)

4.43 (3-5)

Break bad news to clients/residents and their families 4.04 (2-5)

3.50 (2-5)

4.57 (4-5)

Communicate effectively with clients/residents and their families

4.61 (3-5)

4.0 (3-5)

4.71 (4-5)

Recognise when a client/resident is approaching the dying phase and manage the dying phase appropriately

4.29 (2-5)

3.83 (2-5)

4.71 (3-5)

Know what medications will help a client/resident to control their symptoms and to give these appropriately

4.00 (1-5)

3.67 (3-5)

4.29 (4-5)

Play a role in providing good symptom management to clients/residents

4.05 (1-5)

3.50 (2-5)

4.43 (3-5)

Assessing and managing a client/resident’s pain 4.13 (2-5)

3.50 (2-5)

4.29 (3-5)

Provide appropriate bereavement support to clients /residents /their families

4.16 (2-5)

3.33 (2-5)

4.29 (2-5)

Make decisions about clients/resident’s care within an ethical framework

4.02 (1-5)

3.17 (2-5)

4.14 (2-5)

Support and facilitate advance care planning 3.89 (1-5)

3.67 (1-5)

4.43 (3-5)

Manage clients/residents with dementia 4.27 (2-5)

3.83 (2-5)

4.57 (3-5)

Manage clients/residents with heart failure 3.84 (2-5)

3.50 (2-5)

4.00 (2-5)

Manage clients/residents with renal failure 3.05 (0-5)

2.83 (2-4)

4.14 (3-5)

Manage clients/residents with COPD 3.32 (0-5)

3.17 (2-5)

4.14 (3-5)

Overall my confidence in my ability to do my job well and provide an excellent service to clients/residents is…

3.46 (0-5)

3.83 (2-5)

4.57 (3-5)

Overall my happiness in my job is… 3.57 (0-5)

3.67 (2-5)

4.43 (3-5)

Overall my stress level in relation to my job is… 2.61 (0-5)

3.17 (2-4)

3.43 (3-5)

Overall I feel supported in doing my job. 3.39 (0-5)

4.00 (3-5)

4.43 (3-5)

60    

Table 13 – Nursing Home ECHO Self efficacy assessment for Unregistered Staff- Participants self-rated confidence in each area (1 - not confident at all, 5 - very confident)

Question Pre ECHO Average (Range)

Retropre Average (Range)

Post ECHO Average (Range)

Understand the principles of palliative care and apply them in my daily work

3.60 (2-5)

2.67 (2-3)

4.00 (4)

Comfort clients/residents/families who have received bad news

3.74 (2-5)

2.00 (2-3)

4.67 (4-5)

Communicate effectively with clients/residents and their families

4.17 (1-5)

2.33 (2-3)

4.33 (4-5)

Recognise when a client/resident is approaching the dying phase and play my role in helping with the dying phase appropriately

3.54 (1-5)

2.00 (1-4)

4.67 (4-5)

Know when a client/resident needs medications to control their symptoms and to advocate for them

3.43 (1-5)

2.33 (1-4)

4.67 (4-5)

Play a role in providing good symptom management

3.34 (1-5)

2.00 (1-3)

3.67 (3-4)

Helping to monitor and help with a client’s/resident’s pain

3.60 (1-5)

2.00 (1-3)

4.00 (3-5)

Provide appropriate bereavement support to clients/residents/ their families

3.46 (1-5)

1.67 (1-3)

4.00 (4)

Make decisions about clients/residents care within an ethical framework

3.00 (1-5)

1.67 (1-3)

3.67 (3-4)

Support and facilitate advance care planning 2.60 (1-5)

2.00 (2)

3.67 (3-4)

Care for clients/residents with dementia 4.26 (2-5)

2.67 (2-3)

4.67 (4-5)

Care for clients/residents with heart failure 3.40 (1-5)

1.33 (2)

3.00 (0-5)

Care for clients/residents with renal failure 3.09 (1-5)

2.00 (1-3)

4.00 (3-5)

Care for clients/residents with COPD 3.49 (1-5)

2.00 (1-3)

4.33 (4-5)

Overall my confidence in my ability to do my job well and provide an excellent service to clients/residents is

4.11 (2-5)

3.67 (3-4)

4.67 (4-5)

Overall my happiness in my job is… 4.43 (0-5)

2.67 (2-3)

3.00 (3)

Overall my stress level in relation to my job is… 3.34 (0-5)

2.00 (2)

2.00 (2)

Overall I feel supported in doing my job. 4.11 (0-5)

2.33 (2-3)

2.67 (2-3)

61    

General Feedback of NH ECHO

Box 3 demonstrates participant’s views on ECHO in general. Due to low numbers of

responses the registered and unregistered staff have been combined. Participants were

very positive about their experience of ECHO, with 100% having learnt through participation

and found it enjoyable, 100% felt it helped translate knowledge into practice more than other

teaching sessions they had been involved in and improved the care they provided for

patients, and 100% would recommend it to others and participate again.

Box 3 General feedback from NH ECHO- registered and unregistered staff

1. Please rate each on a scale of 1-5 the quality of learning / usefulness from

each area (1- poor, 5- excellent)

1 2 3 4 5

Review of previous

session 0 0 2 2 2

Presentations 0 0 0 6 5

Case based

discussions 0 0 1 2 7

2. Overall do you feel you have learnt

through participating in ECHO?

A lot 9

A little 1

No 0

3. Did you find participating in

ECHO enjoyable?

A lot 10

A little 0

No 0

62    

4. Do you think that participating in ECHO has improved

the care you provide for patients?

A lot 9

A little 1

No 0

5. Do you think the format of ECHO helps translate

knowledge from teaching into practice more than other

teaching sessions you have been involved in?

Yes 10

No 0

6. Would you recommend ECHO to other

healthcare professionals in your area?

Yes 10

No 0

7. Have you used any of the online resources via Moodle, and if yes

have you found these useful?

Used

and

found

useful

Used

and

found

NOT

useful

Power point

presentations 7 0

Video of the

teaching sessions 6 1

Video of case

presentations 6 1

Other supporting

materials 7 0

63    

8. Regarding ECHO technology

Agree Disagree Unsure

It has given me access to education that would have been hard to

access due to geography 9 0 1

It was a good medium to access teaching / education at a different

location from where I work 9 0 1

Any technical difficulties were acceptable and did not put me off

participating in ECHO 8 0 1

Any technical difficulties did not significantly reduce my learning 8 0 0

9. How do you rate your overall ECHO

experience? (1- poor, 5- excellent)

1 2 3 4 5

0 0 0 3 7

10. Would you participate in ECHO sessions in

the future if the opportunity arose?

Yes 10

NO 0

Focus Group

Five registered nurses and one nursing home manager who participated at the spokes for

the nursing home ECHO participated in the focus group. Analysis of the focus group data

uncovered two overarching themes, each with their own descriptive and interpretative

themes that are outlined in thematic diagrams. Overarching theme 1 (figure 9), ECHO

Enhanced Clinical Knowledge and Skills, explored the reasons why participants perceived

that ECHO optimised their clinical practice. Interpretative themes 1A and 1B highlighted the

contributing factors that resulted in participant’s perceived enhancement of clinical practice.

Overarching theme 2, Consideration for the Future of ECHO, displayed the key conceptions

that participants perceived should be considered for future ECHOs (figure 10).

64    

Figure 9 Theme 1 ‘ECHO Enhanced Clinical Knowledge and Skills’ for Nursing Home ECHO

Descriptive Themes Interpretative Themes Overarching Themes

1 ECHO Enhanced

Clinical Knowledge and

Skills

1A Optimised Patient

Care

Enhanced relationships between the Nursing

Homes and GPs

Increased knowledge, confidence and skills

regarding palliative and end of life care

1B Created an effective learning environment

Importance of the Hub

Healthcare assistants benefited from ECHO

Benefited from interaction with other homes

Resources were excellent

65    

Figure 10 Theme 2 ‘Consideration for the Future of ECHO’ for Nursing Home ECHO

Descriptive Themes Interpretative Themes Overarching Themes

Overarching Theme 1: ECHO Enhanced Clinical Knowledge and Skills

This theme (figure 9) encompassed how participants perceived ECHO to be an education

platform that enhanced their clinical knowledge and skills. Interpretative theme 1A depicted

how participants perceived that ECHO optimised the care they delivered to their patients

through the knowledge and confidence they had gained, and also through enhancing

relationships with GPs. Interpretative theme 1B illuminated how ECHO also created an

effective learning environment for participants. Findings highlighted that this was perceived

to have been achieved by the effectiveness of the hub and the interaction between the

spokes. The learning environment was also enhanced as ECHO was perceived as providing

a platform which benefited healthcare assistants, alongside registered nurses.

2 Consideration for

the Future of ECHO

2A Challenges

Time to participate

Not on a weekly basis

Too much in one session

Staff enthusiasm lacking at times

2B Future ECHOs

Seperate sessions for staff without a medical or nursing background

ECHO for GPs and Nursing Homes

66    

Interpretative Theme 1A: Optimised Patient Care

The majority of participants described how they perceived that the ECHO program had a

positive impact on patient care. Many felt this was achieved by enhancing the relationship

they had with GPs. ECHO aided in increasing participants confidence when discussing

patients with the GP, and helping the GP to value their opinions on patient care:

“I think it’s definitely made me feel more confident dealing with the likes of the doctors. After

our last presentation, regarding our client with the pain in the hands, I was able to go back to

his GP and have quite a lengthy discussion, and they were asking where I had got these

suggestions from and once I mentioned this ECHO project and the consultant facilitator, she

was straightaway on board with giving out the treatment. You’re not just seen as a nurse

trying to tell the doctor what to do; you’ve a bit more evidence to back up what you’re saying,

I think, which has helped.” (NH/3)

“We did our case study on a resident in her pain relief, we discussed what we had discussed

on the case presentation with the GP and her medication has been reviewed as a result of

that. So yes, it’s been very productive.” (NH/86)

Many also felt that ECHO increased their knowledge, confidence and skills regarding

palliative and end of life care:

“I think it’s [ECHO] given a level of confidence and also I think the discussions have given

the nurses something to check against, particularly say, around some drugs and their

interactions and things like that, and I think it’s just how people think a little more, which is

just great.” (NH/50)

“After the heart failure session it promoted us to have a discussion about monitoring and

blood results for our resident that was given high doses of diuretics. So it just helped us to

focus as well, rather than just blindly follow the treatment plan without any questioning or

checking. So it was helpful for that resident as well.” (NH/1)

Overall, ECHO was perceived as optimising patient care through enhancing the relationship

between nursing homes and GPs and increasing the knowledge, skills and confidence of

participants.

67    

Interpretative Theme 1B: Created an effective learning environment

Participants conveyed that ECHO created an effective learning environment for the spokes.

Accounts portrayed that this was contributed to by the effectiveness of the hub and the

facilitator:

“The consultant facilitator was amazing, very respectful of what’s in nursing homes and what

we’re doing and always constantly very positive about the work that we’re already doing. It

was very much supported.” (NH/86)

“I believe they [the hub] were very supportive and it was clear how respectful everybody was

to each other in the hub. The respect when it was you’re turn and not making you feel silly,

any questions or queries that you raised, because sometimes it can be very isolating in a

nursing home, trying to lead and drive forward quality improvement. But nobody was made

to feel stupid about any queries they raised. It was treated that it was supportive for

learning.” (NH/1)

Participants also conveyed that healthcare assistants also benefited from the ECHO

program:

“The carers that attended did feel it was very, very useful, and it was all carers at that

session.” (NH/1)

For some participants ECHO was perceived as being beneficial for healthcare assistants in

the future as well:

“I do feel that it has opened our minds to they [healthcare assistants] need as much

information as we require as nurses, because they’re the ones that are doing the hands on

care every day, they’re the ones that are assessing the residents every day and reporting

back to the nurses. So definitely I think the care assistants will benefit from this programme

greatly.” (NH/86)

The majority of participants perceived that they benefited from the interaction with the other

spokes that they got at ECHO:

“Overall perceptions have been very positive, that’s my personal opinion. I feel I have

benefited from it, plus I've really enjoyed networking with everybody.” (NH/1)

“But I think the interaction between all the care homes was brilliant.” (NH/50)

68    

This interaction also helped to dispel participants concerns regarding ECHO:

“I suppose, going into it, I was apprehensive but that was totally cleared very quickly, and it

was lovely, the whole camaraderie between everybody, because we’re all in the one boat,

we’re all having the same problems and it’s great to share knowledge and information with

each other. So I really thought it was very good.” (NH/86)

“I suppose with us being in a nursing home we’re quite isolated so the beauty of this is that it

is reachable to everybody and we can all share information and we are part of the group,

we’ve been talking about this outside of the session as well, so it’s been a great resource.”

(NH/86)

The resources were also viewed as being extremely beneficial to all participants:

“I think the resources that have been supplied have been fantastic. The information, there's

a hell of a lot of printing to be done, even yet! So it is definitely valuable material and I

thought we were going to get some really good material out of it.” (NH/1)

“Plus, a lot of the stuff on the website, I could print that off and if staff weren’t able to attend

these sessions we were able to disseminate all the information. So they got a lot of

information out of it.” (NH/86)

Overarching Theme 2: Consideration for the Future of ECHO

Findings were indicative of the need for further considerations when providing future ECHOs

(figure 10). This was conveyed though the challenges participant’s perceived regarding

getting the appropriate time to be able to dedicate to ECHO and also the timing of the

sessions themselves. Accounts also illuminated the challenges of getting staff enthusiastic

regarding ECHO. Interpretative theme 2B depicted perceptions of how the future of ECHO

may be considered.

Interpretative Theme 2A: Challenges

Accounts conveyed the challenges that participants perceived in relation to ECHO. Findings

suggested that getting time to participate in ECHO was an issue for participants:

“I think at the start I thought a lot more staff would be involved and I was very keen for that. It

was very difficult with staffing issues and that to get staff off the floor, so that would be the

only thing that I personally was disappointed in.” (NH/86)

69    

Many perceived that having the ECHO session on a weekly basis was challenging:

“I would like things spaced out a little bit more and not on a weekly basis. So maybe

fortnightly.” (NH/50)

“I think weekly for a period was a bit of a challenge because time is always precious to

everybody.” (NH/1)

Some also felt that there was a lot covered in one session which made it difficult to engage

fully in the ECHO and cover all the information:

“I felt sometimes maybe the second presentation wasn't starting until maybe 3:50, 3:45 and

so we’d maybe spent a lot of time on one that didn’t have an awful lot of issues and then

maybe the second case presentation had a lot more that we could have discussed but

because the presentation was done first.” (NH/86)

“I think we probably tried to squeeze too much into a session and I do think that having the

quality framework going on as well was probably overcrowding the session and sometimes

that meant we didn’t get all of the case studies done etc.” (NH/50)

Some felt challenge by staff’s lack of enthusiasm to take part in ECHO:

“We didn’t have the enthusiasm as much from the staff, the three of us seemed to be the

Three Musketeers wanting to do everything but we didn’t get as much enthusiasm from the

staff.” (NH/1)

“Well, as I said, I thought that the staff would have been more participating in it. I was the

lone ranger over here [laughter] I really, really hoped that the staff would have got involved

because I thought they would have got a lot out of it, they would have really enjoyed it, as I

did.” (NH/86)

Interpretative Theme 2B: Future ECHOs

Findings highlighted that for future ECHOs participants perceived that separate sessions for

staff without a medical or nursing background may be required:

“It’s better to have a separate one [ECHO], a simpler session, because they [non-registered

nursing staff] did enjoy it and the amount they learned from it but there are areas which they

couldn’t quite understand. So probably better split the sessions.” (NH/10)

70    

“I think some of the sessions were a little bit kind of medical based, I do think sometimes you

get more benefit by segregating groups of staff but I also think there is a place for mixing

staff as well. So I would go for both.” (NH/50)

“I feel that a lot of the information was very nurse led and I had my activity therapist in on

some of the sessions and it just looked like it was completely over her head.” (NH/86)

Many also felt that future ECHOs would be of benefit between GPs and Nursing Homes:

“ECHO could really be used between nursing homes and GPs as well I think that would be

of great benefit to a nursing home.” (NH/1)

Discussion

The response rates for the post ECHO evaluation were very low despite repeated and

persistent efforts to encourage participation. There was an improvement in knowledge and

self-efficacy, but no statistical analysis was possible due to the low numbers.

General feedback on the ECHO experience was positive, and from the low number of

responses, 100% stated they had learnt through participation and found it enjoyable, 100%

felt it helped translate knowledge into practice more than other teaching sessions they had

been involved in and improved the care they provided for patients, and 100% would

recommend it to others and participate again.

The focus groups confirmed this- the ECHO platform was viewed as a positive learning

experience that promoted the awareness of palliative and end of life care amongst nursing,

healthcare assistants and other staff within the nursing home setting. Of particular benefit

was the role ECHO had in nurturing the relationships between GPs and nursing home staff

and the confidence and skills it gave staff in the delivery of palliative care. Nursing homes

also conveyed that the interaction between spokes, that the ECHO platform lends itself to,

also enhanced the ECHO experience. For future ECHOs with this group of participants it

may be useful to consider the timing of the sessions to allow more staff to join and get the

most out of each session. Funding for protected time may have helped this issue, as staff

reported that with the busyness of the job it was hard to get time to attend due to patient

care needs.

A weakness is the low response rate that is likely to have led to a biased sample of more

enthusiastic HCPs participating. This was reinforced in the focus group where participants

commented on the lack of enthusiasm of some other staff. The ability to generalise these

71    

results is therefore limited. More research is necessary to determine the benefits of ECHO

in this setting, and whether it improves staff knowledge, self-efficacy and ultimately patient

care.

Suggestions from evaluation

- Due to the very low response rate further research is needed in this area to

determine how useful ECHO is in nursing homes for healthcare professionals. While

the responses from participants were very positive, the response rate of

approximately 10% makes the results ungeneralisable as it was likely a biased

sample.

- Consideration should be given to incentives for participating in the evaluation for staff

e.g. funding dependent on this or funding for protected time to participate.

- Funding needs to be considered for protected time for staff to participate.

- The timing and frequency of any future sessions would need to be considered closely

with the involved spokes.

- Research looking at the direct impact on patient care and analysis of evidence of

using the quality improvement learning would be likely to provide very valuable

information as to the cost effectiveness and direct outcomes of ECHO investment.

72    

Chapter 6 - Enhanced Dermatology for GP Trainees ECHO

Background

Skin disease is common and distressing. Around 24% of the population consult their GP

with a skin problem in any 12-month period.(10) It is estimated that approximately 14% of all

GP consultations are in relation to disorders or concerns about the skin.(9) Establishing and

maintaining competence in this area of medicine is therefore essential for any GP. There is

variable (and often limited) training in dermatology at undergraduate level leading to

confidence and competence gaps at the post-graduate stage.(11-13) Most skin disease can

be appropriately and efficiently managed in primary care.

Of the nearly 13 million people presenting to GPs with a skin problem each year in England

and Wales, around 6.1% (0.8 million) are referred for specialist advice.(14) While there are

well over 1000 dermatological diseases, 10 of them (eczema, psoriasis, acne, urticaria,

rosacea, infections/infestations, leg ulcers and stasis eczema, lichen planus and drug

rashes) account for 80% of consultations for skin disease in General Practice and specially-

collected data from four specialist Dermatology departments in England show that

specialists most commonly see people with skin lesions (35-45%), eczema, psoriasis and

acne.

Within one health trust area in Northern Ireland demand for secondary care dermatology

services has been seen to grow year on year with a growth of 4.5% for 13/14 on the

previous year.(15) This growth in demand contributes to greater waiting times, delays in

effective clinical management and an increase in patient distress. Additionally, new initiatives

around models of care are being developed. Prominent among these is ‘Transforming Your

Care’ (TYC) which aims to increase the proportion of care delivered within community

settings.(16) The Kings Fund suggests a number of approaches to demand management

among these being ‘Schemes to manage GP referrals’.(17) It is likely that progress in

respect of confidence and competence among GPs and the adoption of best practice

guidelines/ frameworks is likely to yield the greatest dividends.

GP Trainees follow a well-defined curriculum(18) which is broad in its reach and completed

within a short 3 year time period. “Care of People with Skin Problems” is one of the core

competencies of this curriculum for the ‘Membership’ exam of the Royal College of General

Practitioners (MRCGP).(18) Some GP Trainees will get the opportunity to gain deeper

experience in the management of dermatology problems through secondary-care clinical

attachments but the majority will not. An opportunity to increase knowledge and competence

73    

during training in respect of the management of dermatological conditions has the potential

to have a career-long impact for future cohorts of GPs.

Trainees at the GP ST2 level have 100+ consultations per week. These consultations are

unselected and undifferentiated and will cover the full range of clinical areas for which

anyone might consult their GP. On the basis of average consultation rates it is therefore

expected that each GPST2 will see and manage 14+ patients with dermatological issues

each week.

The Northern Ireland Medical and Dental Training Agency (NIMDTA) is an organisation

which operates on behalf of the Department of Health, Social Services and Public Safety

(DHSSPS) to train medical and dental professionals. NIMDTA commissions, promotes and

oversees postgraduate medical and dental education and training throughout Northern

Ireland. NIMDTA organises and delivers the recruitment, selection and allocation of doctors

and dentists to foundation, core and specialty training programmes and rigorously assesses

the performance of trainees through annual review and appraisal. It works in close

partnership with local education providers to ensure that the training and supervision of

trainees support the delivery of high quality, safe, patient care. NIMDTA now recruit

approximately 85 trainees each year to the GP Training programme. These GP specialist

trainees (GPST) have a 3 year run-through training programme (GPST1, GPST2 and

GPST3) of which 18 months takes place within hospital attachments and 18 months in

General Practice (6 months in GPST2 and 12 months during GPST3).

The Intervention

Twenty-eight GP Trainees (GPST2 level) from across the region took part in this ECHO over

5 sessions to learn about the common dermatological conditions of eczema, psoriasis and

acne. For more details see table 14. The Programme Lead (a GP Training Programme

Director with NIMDTA who is also a GP with enhanced dermatology training developed a

Programme Resource Pack comprising a range of written material, pictures and videos that

were used in the sessions. (These resources were based on her work previously done with

the Northern Area Primary Care Dermatology Pathway and were reviewed by local

consultants).

The ECHO sessions were scheduled to take place on a 2-weekly basis and ran for 1.5 hours

around lunchtime to facilitate maximum participation and minimum disruption to the working

day. The meetings were conducted via video-conference technology that allowed GP

Trainees to join from any location that provided a webcam and internet connection thus

avoiding the need to journey to a central location.

74    

Participants were asked to prospectively identify ‘Cases’ from their GP surgeries and to use

the agreed ‘Case Proforma’ setting out the anonymised details of a non identifiable patient,

the challenges faced, the treatment modalities used and the outcomes achieved. The hub

team provided feedback and guidance on management options and advice on refinements

to the management options available. It was expected that in each meeting up to 3 cases

would be discussed.

Most of the GP Trainees joined the ECHO Clinics via videoconference principally from one

of two locations

1. One of the seven community offices of the NIH for those GPST2s in close proximity

or

2. From the nearest GP Practice which had a webcam facility

Some of the GP Trainees joined using their own tablet or laptop devices from home as

access to the GP Practice Wi-Fi network was forbidden by the local Health and Social Care

security policies. A map of the locations of trainees, Northern Ireland Hospice community

offices and locations of the GP Webcams can be seen in Picture 2.

Hub - The hub team was chaired by the Programme Lead (GP with enhanced Dermatology

Skills) and a variety of invited guest team members (e.g. Consultant or Specialist Nurse

Dermatologist and a Pharmacist with enhanced knowledge of prescribing for dermatological

conditions).

Spokes - The spokes were identified as the half of the GPST2 cohort who are placed in

General Practice during the first 6 months of their ST2 year. There were 28 GP trainees

involved in total. Some based at NIH community sites (16) and some at training practices

(12).

Picture 2 - Locations of trainees, NIH community offices and locations of the GP Webcams

75    

Table 14 GP Trainee/ Dermatology Knowledge Network

GP Trainee/ Dermatology Knowledge Network Clinical Champion/Lead Facilitator Teacher

Dr Nigel Hart Dr Siobhan McEntee

Frequency of clinics 5 clinics Start date

21st October 4th & 25th November 2nd December 13th January

Date for training September 2015 GP Core Day Hub members

Programme Lead GP with enhanced skills Invited others; consultant, Specialist Nurse, Pharmacist,

Hub costs None Spoke members

GP trainees – some based at NIH community sites (16) and some at training practices (12)

Spoke costs No backfill requirement

Equipment needed for spokes

8 webcams/microphones required – installed by NI Hospice Project ECHO Jonathan Pope (BSO) attended installation of a few to ensure no impact on HSC network

Equipment costs 8 webcam/microphones x £145 Curriculum development update

Dermatology curriculum developed • Facial Rashes, eg acne vulgaris • Psoriasis • Eczema in Children • Eczema in Adults • When should I refer?

Evaluation framework Nigel and Siobhan developed a pre & post ECHO

evaluation

Objectives:

The objectives of this pilot study were to determine if the ECHO model can be used to

support the improvement of confidence and clinical knowledge among GP ST2 Trainees in

the management of common dermatological conditions presenting in primary care.

Methods

The knowledge and self-efficacy assessments were run by the network facilitators (Dr Hart

and Dr McEntee). The general ECHO feedback and focus groups were undertaken by the

evaluation team from NIH as described in chapter 2.

76    

The questionnaire for pre- and post-programme self-efficacy and knowledge assessment

were conducted using Survey Monkey®(8) and was carried out with all participants before

and after the Enhanced Dermatology ECHO Programme.

Results

Twenty-six trainees took part in the Enhanced Dermatology ECHO Knowledge Network. All

trainees successfully completed the Pre-ECHO questionnaire. One trainee left the

programme early due to being rotated to a new attachment. One trainee did not fully

complete the post- questionnaire and two trainees did not start the post- questionnaire at all.

Out of 26 original programme participants pre- and post- programme paired data was

collected for 22 trainees (response rate 84.6%).

The Likert scale questions in the self-efficacy part of the questionnaire (self-efficacy and

confidence in respect of managing clinical cases in dermatology) were coded 1 to 5 (1 =

Strongly Disagree Strongly, 2 = Disagree, 3 = No strong views, 4 = Agree, 5 = Strongly

Agree), Question 4 (confidence in relation to strength of steroid usage) was coded 1 to 4 (1

= Mild, 2 = Moderate, 3 = Potent, 4 = Very Potent). The paired results were analysed using

the Wilcoxon signed rank test. All questions showed a significant improvement (P < 0.001) in

assessment of self-efficacy and confidence for managing clinical cases in dermatology from

before the programme to after.

The responses to knowledge assessment were marked, scored and totalled for each of the

trainees. Completion of the knowledge assessment achieved a maximum total score of 50.

The scores were analysed using the t-test. The pre- and post- mean percentage scores

showed a significant improvement (Pre- 35.5% (SD 7.6) and Post- 60.6% (SD 4.3),

P<0.001).

General ECHO Feedback

There were 8 responses (30.8% response rate) to a Survey Monkey for GP trainees about

their general views on ECHO (box 4). All felt they had learnt through participating in ECHO,

found it at least ‘a little’ enjoyable, and 100% felt it had improved the care they provided for

patients. All would participate again in future ECHOs.

77    

Box 4 General ECHO Feedback for GP Trainees Dermatology Network

1. Please rate each on a scale from 1-5 the quality of the learning / usefulness from each

area (1 – Poor, 5 - Excellent)

Answer Options 1 2 3 4 5

Review of previous session 0 0 3 5 0

Presentations 0 0 1 4 3

Case based discussions 0 0 1 6 1

2. Overall do you feel you have learnt through participating in

ECHO?

Answer Options

Response

Percent

Response

Count

A lot 87.5% 7

A little 12.5% 1

No 0.0% 0

Unsure 0.0% 0

If Unsure please state why 0.0% 0

3. Did you find participating in ECHO enjoyable?

Answer Options

Response

Percent

Response

Count

A lot 62.5% 5

A little 37.5% 3

No 0.0% 0

Unsure 0.0% 0

If Unsure please state why 0.0% 0

4. Do you think that participating in ECHO has improved the

care you provide for patients?

Answer Options

Response

Percent

Response

Count

A lot 75.0% 6

A little 25.0% 2

No 0.0% 0

Unsure 0.0% 0

78    

5. Do you think the format of ECHO helps translate

knowledge from teaching into practice more than other

teaching sessions you have been involved in?

Answer Options

Response

Percent

Response

Count

Yes 50.0% 4

No 12.5% 1

Unsure 25.0% 2

6. Would you recommend ECHO to other Healthcare

professionals in your area?

Answer Options

Response

Percent

Response

Count

Yes 87.5% 7

No 0.0% 0

Unsure 12.5% 1

7. Regarding ECHO technology...

Answer Options Agree Disagree Unsure

Response

Count

It has given me access to education

that would have been hard to access

due to geography

5 2 0 7

It was a good medium to access

teaching / education at a different

location from where I work

8 0 0 8

Any technical difficulties were

acceptable and did not put me off

participating in ECHO

7 0 1 8

Any technical difficulties did not

significantly reduce my learning 7 0 1 8

8. How do you rate your overall ECHO experience? (1 - Poor, 5 -

Excellent)

Answer Options 1 2 3 4 5

Experience Rating 0 0 1 5 2

79    

Focus Group

The focus group was conducted with eight GP trainees who participated at the spokes.

Analysis of the focus group data uncovered two overarching themes, each with their own

descriptive and interpretative themes that are outlined in thematic diagrams. Overarching

theme 1, ECHO Enhanced Clinical Knowledge and Skills, explored the reasons why

participants perceived that ECHO optimised their clinical practice (figure 11). Interpretative

themes 1A and 1B highlighted the contributing factors that resulted in participant’s perceived

enhancement of clinical practice. Overarching theme 2, Consideration for the Future of

ECHO, displayed the key conceptions that participants perceived should be considered for

future ECHOs (figure 12).

Overarching Theme 1: ECHO Enhanced Clinical Knowledge and Skills

This theme encompassed how participants perceived ECHO to be an education platform

that enhanced their clinical knowledge and skills (figure 11). Interpretative theme 1A

depicted how participants perceived that ECHO optimised the care they delivered their

patients through the knowledge they had gained, and also through their increased

confidence in caring for patients with dermatological conditions. Interpretative theme 1B

illuminated how ECHO also created an effective learning environment for participants.

Findings highlighted that this was perceived to have been achieved by the expertise of the

hub and the sharing of knowledge from remote places. The learning environment was also

enhanced as ECHO was perceived as providing learning that previously had been limited for

this group of clinicians.

9. Would you participate in ECHO sessions in the future if the

opportunity arose?

Answer Options

Response

Percent

Response

Count

Yes 100.0% 8

No 0.0% 0

Unsure 0.0% 0

80    

Figure 11 Theme 1 ‘ECHO Enhanced Clinical Knowledge and Skills’ for Dermatology for GP

trainees ECHO

Descriptive Themes Interpretative Themes Overarching Themes

1 ECHO Enhanced

Clinical Knowledge and

Skills

1A Optimised Patient

Care

Optimised the approach to patient care

Increased confidence dealing with dermatological

conditions

Enhanced the quality and relevance of referrals

1B Created an effective learning environment

Expertise at the Hub

"It covered all the basics"

Limited training in this area pre- ECHO

Preconceived it would be harder

"We could do it from remote places."

81    

Figure 12 Theme 2 ‘Consideration for the Future of ECHO’ for Dermatology for GP trainees

ECHO

Descriptive Themes Interpretative Themes Overarching Themes

Interpretative Theme 1A: Optimised Patient Care

The majority of participants described how they perceived that the ECHO program had a

positive impact on patient care. Some felt it helped them to consider how they were

managing their patients and feel more prepared to take alternative approaches that were

discussed at ECHO:

“I think even one of the most helpful things was some of the resources in the lecture notes

that we got emailed, things like emollient ladders and the steroid ladders, which are just very

helpful to have on your desk in front of you and probably made you feel more confident in

prescribing those and using them. From my perspective, that was one of the most useful

things in terms of its impact on practice.” (GPT/02)

2 Consideration for

the Future of ECHO

2A Changes to be

made

Cameras in more practices

Have on a dedicated study

day

2B Challenges

Getting spokes to mute and unmute

Too Many spokes

2C Furture ECHO

participants

Consideration of the hub members

Ability to view missed sessions

82    

“I thought it was quite good overall and quite a good idea, and I think I benefitted from it and

it probably changed the way I managed a few patients, and involved some of the stuff that

we went through.” (GPT/06)

Findings also suggested that the ECHO experience had encouraged participants to adopt a

more holistic approach to the care of their patients:

“It made me think, not just about the physical aspect of the condition but more about the

psychological, the emotional impact that the conditions have on the patients as well, and that

was something I learned through ECHO and something that I was able to explore with the

patients as well. I think it's going to help my management of patients in the future, so I think

it's had a good impact already and it will continue to do so.” (GPT/07)

“You take a more holistic view of things, what their work is, has their condition improved

when they’re off work for a while or off on holidays, things that you probably wouldn't have

asked before.” (GPT/08)

Many also felt that patient care was optimised due to the impact of the ECHO program on

increasing their confidence and skills when providing care to patients with dermatological

needs in the primary care setting:

“I think where I notice it most, I wasn't able to get to the psoriasis ECHO and I still, compared

to all the other conditions [covered at ECHO] that we're managing, I would probably feel less

confident about managing psoriasis. So I think the ECHO does probably add a different

component. I do see the extra dimension that ECHO has added to understanding and

treating conditions, because I can compare the acnes and the others, eczemas, and I'm

more comfortable with those, given I've had the ECHO experience of it.” (GPT/03)

Findings also illuminated how participant’s confidence was enhanced due to the case

presentations that were used during the ECHO sessions:

“I think it was really useful having the case discussions, because even though obviously all

the ECHO communication was given and everyone said it was really useful to have that on

your desk to flick through, but it was really good then hearing real life cases and what would

be different... obviously every patient is different and what we could do, what we need to

think about. I thought that was good and increased our confidence in dealing with those

slightly more difficult situations.” (GPT/04)

83    

ECHO also provided a platform to increase GP trainee’s confidence in relation to the use of

topical medications. One GP trainee expressed that the confidence they had gained through

ECHO had been translated into practice as they were more effective in information giving to

patients. This was also perceived as helpful in building relationships and promoting

compliance:

“I think that patients benefit overall if we are able to be more confident in our approach, and

obviously confidence and competence, and I think certainly it is about the practical things,

how much of an emollient to give someone, how much steroid cream, the practical points of

how to explain how to put it on, and I think whenever you are able to explain those things to

patients it gives them more confidence that you know what you're talking about rather than

just saying 'here, have a steroid' but not really... explain things like fingertip, how much to

use and give them the right amount for that week and then tell them to come back. I think

that really helped them to be compliant and come back and see you about it. So it improved

the relationship but also they're kind of ... it helped. So I do think it really did benefit

patients.” (GPT/05)

Accounts also highlighted how the ECHO platform enabled participants to improve the

quality of referrals they made to other services

“There were people who I had seen who needed referred and I felt more confident making

that decision because I had the knowledge that we had gained.” (GPT/04)

Some participants also expressed that the learning they had gained from ECHO gave them

the knowledge and skills to provide further treatment to their patients and avoid unnecessary

referrals to secondary care services:

“One case in particular that I had was a baby with eczema that wasn't responding to

treatment that we'd given, and I didn't really have much experience with that sort of thing, the

woman was quite anxious so I was thinking about maybe referring to paediatric dermatology

just to have a look because I wasn't really sure what was going on. But then the session on

eczema, so it was going into that that I felt a bit more confident and actually rang the mum

back and just had a chat with her and said we can try these different things instead of

referring you on to paediatrics. So I think she was reasonably happy with how that worked

out. So I think that's a good example of changing practice and probably preventing a referral

to secondary care.” (GPT/06)

84    

The presence of an experienced interdisciplinary team at the hub was also perceived to be a

factor in reducing the number of unnecessary referrals due to the clinical knowledge and

skills gained:

“I value the fact that it was an interdisciplinary team at the table [hub], you did get the

perspective of a pharmacist, you got the perspective of a nurse, who's maybe seeing some

of these patients, and all the little things that they teach the patients, we have been able to

glean a bit off, and in a way that can help avoid referral because maybe we have chosen the

right treatment but it maybe wasn't being applied right or wasn't being applied well enough.

So we learn from that.” (GPT/03)

Participants also illuminated that in reducing inappropriate referrals, ECHO may also have

the potential to help alleviate issues regarding waiting lists to specialist services:

“I think now with the fact that waiting times are so long to see specialists, I think something

like ECHO could be used for GP’s to discuss with a specialist to say is there anything I could

do with this in the meantime, does this need referred? That sort of advice would be useful

and would be needed at primary care.” (GPT/06)

In summary, participant’s views highlighted the impact of ECHO on optimising patient care

through increasing the knowledge and skills base of GP trainee’s in relation to

dermatological conditions. Findings also highlighted how the learning gained from the hub

provided information on specific conditions that aided participants to optimise the

appropriateness of their referrals to secondary care services.

Interpretative Theme 1B: Created an effective learning environment

Many participants expressed that the expertise at the hub added to the optimal learning

environment provided through the ECHO platform. Some participants felt that the facilitator

was vital in enhancing the quality of the ECHO sessions:

“Generally, the facilitators do quite a good job. It was maybe a bit tricky initially with the

muting and unmuting function that people had mentioned previously. But I think overall it

was well done, I think everybody got a chance to participate. So I think overall it was pretty

well done.” (GPT/06)

Many also felt that the expertise at the hub was able to provide good practical guidance for

the management of patients with dermatological conditions:

85    

“I think the relative experience at the table [hub] was quite good as well. You had a nurse

and a pharmacist, and it was kept very practical, which is helpful, because that's what we

have to deal with rather than maybe all the exact signs behind everything. The day-to-day

steps of it, I found quite helpful.” (GPT/03)

“The practical advice is excellent, things like apply the emollient half an hour before the

steroid and then also the guidelines, as the guys have said before, are excellent.” (GPT/08)

Accounts also expressed that ECHO “covered the basics” and provided useful information

on the day-to-day care of this client group:

“I thought it was very helpful. I think it covered all the basics. It covered the day-to-day things

quite well and the ladders and pathways for treatment have been quite useful. So yeah, I

think it's benefited me, certainly.”

(GPT/08)

The resources were also highlighted as being a contributing factor to the effective learning

environment within ECHO:

“To me, the resources were very helpful. I suppose the main thing, it was great to have the

lecture before it so you could have it printed out and have a look at it, but the main thing I

found the most helpful were all the guidelines, just having them in a folder and being able to

look at them throughout the day whenever you were in the GP practice dealing with patients.

So I thought they were very good.” (GPT/01)

“I think it was good having all the handouts, that after the first session we had the handouts

sent out before, and I think it was good to have the presentation there and rather than

making lots of notes you could just allocate it as you went along, and I suppose knowing

about what cases were coming up. So being able to use that resource was really useful.”

(GPT/04)

The education provided by the ECHO sessions were particularly welcome as participants

expressed the training they received regarding dermatology had been limited:

“I think our undergraduate dermatology teaching is only a week, so you're coming into a GP

with very little knowledge about how to manage all the common skin conditions, and inside a

couple of weeks of being in a GP practice you realise how common dermatology issues are.

So definitely, it's been really helpful and I don't really know what they would have done,

86    

coped with all the dermatology without it, to be honest, because you're not really taught that

at all.” (GPT/01)

“We have very little, certainly personally, day-to-day experience of dermatology, before

general practice. In hospital it tends to not be really dealt with on a medical ward, it would

just be "see your GP about that." So we were on the receiving end of that and I think it has

been helpful.” (GPT/03)

However, prior to the start of ECHO many held preconceptions that it would be harder to

engage in the sessions but felt the facilitator helped them get involved:

“I suppose it's a very new format to all of us. I've never had any teaching delivered that way

before so I wasn't sure what to make of it. I thought it would be harder. Initially, before we

had done any, I thought it would be quite hard to feel a part of it or feel engaged with it, but I

think, you know, the facilitator did quite a good job of keeping everyone involved and asking

different people things. So I found it easier to engage with what I had thought whenever the

idea was explained at the beginning.” (GPT/03)

Some felt unsure of how they would find using the technology:

“I thought it was going to be a lot harder to be involved with and thought it was going to be

more like watching a lecture up on a screen and it was much more high tech. I thought, apart

from a few initial hiccups, it went very smoothly, it went a lot better than I thought it would.”

(GPT/04)

However perceptions changed as the ECHO progressed and using the technology became

more ‘natural’:

“I initially thought this was going to be a bit strange and a bit odd but eventually it seemed to

be quite natural, was probably my perception, it actually came a lot more naturally than I

thought it was going to.” (GPT/02)

Many also felt it was positive learning environment as it could accessed even in “remote

places” and without having to travel:

“I think having the Zoom in your own practice is a lot more practical than travelling

somewhere.” (GPT/05)

87    

“When I heard about it I was quite interested to see how it would work, because I know

dermatology is really big area and we do need to focus more time on it, so I think it was

interesting to see how it was set up all over Northern Ireland and that we were able to make

it work where we didn't have to keep attending one area to get to each input but we could do

it from kind of remote places. So it was quite an interesting experience to be a part of.”

(GPT/07)

“Certainly I would just echo the fact that the camera was in my practice. I'm in quite a remote

location and it just wouldn't have been feasible to be involved in it. So yeah, that was very

helpful.” (GPT/02)

Overall, the ECHO platform contributed to an effective learning environment that was

facilitated by the effectiveness of the hub at providing clinical expertise and good facilitation.

Participants expressed negative preconceptions that they had before ECHO, and the lack of

training they perceived they had in this area previously. However, these perceptions

changed as the ECHO program commenced and the training provided practical guidance

from an interdisciplinary perspective.

Overarching Theme 2: Consideration for the Future of ECHO

Findings were indicative of the need for further considerations when providing future ECHOs

(figure 12). This was conveyed though participant’s views on the need for appropriate

equipment in various locations, alongside protected study time, to enhance the accessibility

of ECHO. Participants additionally highlighted the specific challenges of participation that

may be addressed for future ECHOs. Finally, interpretative theme 2C explored the accounts

expressed that considered the hub at future ECHOs and the availability of sessions to those

who did not attend.

Interpretative theme 2A: Changes to be made

Accounts provided guidance on potential changes that could be considered for future

ECHOs. Participants expressed that GPs would benefit from ECHO equipment being

installed in more practices:

“The doctor that I work with had to travel very far for teaching, and certainly when I was here

and the camera wasn't I would have had to go a great distance also and that just wouldn't

have been feasible. I think they've worked very well with that and they got the cameras into

most practices. Certainly a way for improvement would be maybe, if possible, to get them in

more. If that's possible, I don't know, but certainly I think they did very well from that point of

view.” (GPT/08)

88    

Many participants also expressed that future ECHOs may benefit from clinicians having

dedicated study time to take part in ECHO:

“The only thing I would maybe suggest would be, so on Thursdays, we have teaching in our

locality groups, and I just wonder, sometimes on a Wednesday afternoon it was really quite

rushed to set that hour aside for ECHO and then come back to your work afterwards and

maybe leave a home visit until later on that evening, whereas on the Thursday, because it is

a teaching day we would be a lot more relaxed, and I think in a sense we don't feel as

pressured. I don't know if anyone else had that experience or felt like that, because certainly

I was rushing to go or to come and sit down and then having to finish when I got back and

that kind of took away from the teaching because you were kind of thinking what do I have to

do later, kind of thing. (GPT/05)

“Even if it's not about kind of work pressure and having to do stuff on the day, I think it would

be more relaxed to have more time. Like, when you have a day put aside for a study day,

because that was the aim of it, to learn more about it.” (GPT/07)

Interpretative Theme 2B: Challenges

Some participants expressed the challenges they faced when participating in ECHO. Some

felt that getting the spokes to mute and unmute was difficult and often took up time:

“I think it was difficult at times trying to get everyone involved in muting and unmuting and

there was a lot of the group time that way, but I suppose that's something that can't be

helped with the technology.” (GPT/01)

“I agree with the muting and unmuting point. Sometimes that's my fault as well so I can't

really blame anyone for that, but it was an issue throughout.” (GPT/08)

Some participants also expressed that there may have been too many spokes present at the

sessions to enhance optimal participation:

“I suppose at times it was a bit difficult trying to get participation because there maybe were

so many web cams.” (GPT/01)

“Probably I would agree with that and I suspect people would be even happier to take part or

to speak up if they're in a smaller, in their locality group. So I think you might find if you were

able to do that you would get even more involvement with people because they don't feel

like they're just out on their own talking to a web cam.” (GPT/02)

89    

Interpretative Theme 2C: Future ECHO Participants

For future ECHOs, participants also expressed that it may be beneficial to consider the

membership at the hub. Some felt that if there were more experienced clinicians taking part

at the spokes, then the hub would need to reflect this:

“I think if you're setting it up for more experienced GPs it would be quite useful to have the

dermatology ECHO where you could speak to dermatologists about problem cases maybe

or cases that are difficult to manage or cases where it wasn't quite sure that you should refer

or you shouldn't.” (GPT/06)

Some felt that if there were too many experienced members at the hub then that may hinder

the participation of the spokes:

“I think if it was a mixed crowd of attendees, it would be different if we had an expert at the

table, if it became a mixed crowd with some very experienced GPs with a special interest,

some of the questions that we would ask among ourselves, you might begin to wonder is it a

silly question and you're not going to ask it. So that would be the only thing, if you were

going to broaden it out to a more than GP trainee core attendee then it might limit what we'd

ask and what we'd get out of it at times.” (GPT/03)

Participants also highlighted the benefits of being able to review missed sessions online:

“There may be the odd session, obviously, that you can't make, and I think that having them

online would be quite nice, to go back and have a look. Often there's important notes and

information in the discussion that you might not get in the printed material from the lecture.

So that would be useful.” (GPT/06)

Discussion

The results of this study demonstrate that the ECHO programme among GPST2 Trainees

has been successful in improving confidence among the participants and also resulted in an

improvement in their knowledge base.

The programme created a collaborative learning environment within which participants could

improve their knowledge and confidence to manage patients with dermatology issues

through the activity of case discussion and refinement of management options. With a

minimal intervention of five sessions, GP Trainees grew in confidence and showed an

improvement in knowledge for the management of eczema, psoriasis and acne. It is

anticipated that this programme will result in better decision-making and management plans

for patients presenting to this cohort of future GPs. Whilst it was not one of the research

90    

questions or a measured outcome of this study, it is a reasonable expectation that the

improved confidence and knowledge among this cohort of GP trainees would lead to a

reduction in the need for further advice from or a referral to secondary care dermatology

services. To answer this question a different study design would be required.

While there were only eight responses (30.8% response rate) to a survey about participants

general views on ECHO, all respondents felt they had learnt through participating in ECHO,

found it at least ‘a little’ enjoyable, and 100% felt it had improved the care they provided for

patients. All would participate again in future ECHOs. While there may be bias in this low

response rate, the results indicate positive views on ECHO that were confirmed by the focus

group.

The focus group findings highlighted the safe and effective learning environment that the

ECHO platform can provide for GP Trainees in the primary care setting. The ECHO network

helped to improve care delivery by enhancing the knowledge and skills of GP trainees in

relation to a variety of areas such as: taking a holistic approach to patient care, awareness of

alternative treatments and how to effectively prescribe and administer topical medications.

Through this increased confidence, participants also voiced that the ECHO program had

increased their ability to make more appropriate referrals. Findings were also indicative that

ECHO may be an effective platform with which to reduce referrals to secondary care

services and promote care delivery by primary care providers. Due to its ability to reach

participants in remote areas, ECHO was seen as an effective learning platform that saved

clinicians time into having to travel for teaching.

In relation to potential future ECHO networks the timing of sessions may need to be

considered to incorporate protected study time. Better knowledge on how to access the

recorded session would be useful for those who have missed a particular network meeting.

This facility was actually available but required participants to share their email address with

the network administrator.

For any future network, consent should be obtained in advance of ECHO starting.

Membership of the hub is also an important factor and it was perceived that these members

need to be carefully adapted to the needs of the spokes. In summary ECHO has been

perceived by GP trainees as an effective way to deliver clinically relevant education that will

promote patient care amongst primary care services.

91    

Suggestions from evaluation

- The dermatology network for GP trainees was found to be useful and should

continue for future trainees with consideration for involvement of other professionals

where relevant.

- Addressing minor technological issues with connectivity and muting microphones

could enhance the experience further

- The number of spokes in a session should be considered so there are not too many

that may detract from learning.

- Access to a recorded session would be useful for those who miss a session.

- Timing of the sessions needs considered as to whether they should be moved to a

non-clinical day.

- Participation in the ECHO network and receipt of an incentive e.g. certificate of

participation for trainees should be dependent on participation in the evaluation

process, or time should be made available to allow participants to take part in the

evaluation. This should improve the generalisability of results and prevent bias.

- Future studies should look at impact on service delivery and patient care, for

example referral rates to secondary care and management of patients in the primary

care setting, to determine if participation in the ECHO knowledge network has a

direct impact on these areas.

92    

Chapter 7 - Carer support ECHO

Background

The carer of a patient can be defined as, “ The informal caregiver who provides a patient

with unpaid support for their physical, financial and psychological needs”.(19) Caring for a

patient with palliative care needs can be stressful for a wide variety of reasons as the

physical, psychological and social needs associated with a life-limiting disease are not only

experienced by the patient, but also by the family carer..(20) Reasons for stress for the carer

may include a perceived lack of knowledge, lack of confidence, fear of the unknown or fear

of the future, and a feeling of isolation to name but some.

HCPs have the potential to increase or decrease the burden felt by the carer by

acknowledging that the illness is experienced by both the patient and the carer.(21)

However, although HCPs have a responsibility to recognise the needs of the carer alongside

the patient to provide holistic palliative care,(22) obtaining support can be challenging for

carers. One challenge carers’ may face is trying to access formal carers’ support. This can

be potentially challenging due to the need to travel, the additional time travel takes on top of

the actual time at the meeting, and often the need to arrange for someone else to be present

with the person they care for whilst they are absent.

In 2015 the NIH carer’s service ceased due to fiscal issues. This carer’s service involved a

group of carers coming together for a weekly meeting over a six week period at a central

location with a facilitator. At this meeting carers were provided with relevant education to

help them in their caring role and it also allowed carers to provide and receive support for

and from one-another. This service had a positive impact on the holistic support provided to

carers, but unfortunately had to stop when charity funding ended as it was not commissioned

The Intervention

In order to continue to provide support to carers, across various geographical locations, NIH

hoped to re-establish the delivery of support to carers through the utilisation of ECHO. This

innovative carer’s service ran in collaboration with three other hospices, in rural and urban

areas, throughout NI (Southern Area Hospice, Macmillan Unit Antrim and Foyle Hospice).

The carers ECHO service involved a group of carers linking in from their own homes via

computer / laptop / hand-held device, using zoom technology and a webcam / inbuilt

computer camera. Carers were able to see each other and the ECHO facilitator at the

central ‘hub’.

93    

Carers were provided education on the topics:

Ø Nutrition and Diet

Ø Medication

Ø What to expect

Ø Relaxation and mindfulness: coping with stress

Carers also had the opportunity to discuss personal experiences and learn from one-

another. In order to determine if this innovative method of providing carer support and

education was useful and acceptable to carers, the service was evaluated. The initial

evaluation was to assess carers’ views of ECHO and to explore the views of the hub

members.

Methods

Carers Perspective

At the end of the ECHO programme all carers who started the ECHO (including those who

did not complete the programme) were sent an invitation letter and PIL explaining the

evaluation. They were asked to complete a survey which included questions about their

demographic data, experience of technology prior to ECHO, and their overall views on the

acceptability and usefulness of the ECHO carers project. The survey could either be

completed on paper and returned or completed via Survey Monkey. Each carer was

assigned a unique confidential identifier code. Participants’ names were not recorded on

any tool. Consent to take part in the evaluation was presumed on completion of the survey.

This was made explicit to the carer in the PIL. The master list was kept in a locked area,

and no-one had access to it except the lead evaluator if required. Participants were

informed that the code would not be de-identified unless in the situation described under

ethical considerations.

Focus Group with Hub Members

Each hub member participating in the ECHO was invited via email to partake in a focus

group following the final ECHO session. Each member was provided with information about

the evaluation using a PIL. Before the focus group was commenced written informed

consent was obtained from each member. A focus group guide was used for the focus

group.

Ethical Considerations

It was recognised that there could have been ethical implications when approaching active

carers to take part in the survey, and the time period when the carer was approached

needed to be carefully considered.(23) Therefore the evaluation team liaised directly with the

94    

carers ECHO program facilitator to ensure that it was a suitable time to ask the carer to

participate in a survey.

Carers and HCPs were provided with PILs to ensure they fully understood the evaluation

and to enable them to give informed consent. Consent was implied by completion of the

written tool (carers) or returned consent forms (hub members). Carers and HCPs were made

aware prior to data collection that every endeavour would be taken to ensure their privacy

and identity was fully protected throughout the duration and after the evaluation was

complete. Each carer and HCP was allocated a unique confidential identifier code in order

for documentation related to that carer to be numbered instead of their details being used;

only the evaluation team had access to the carer’s or HCP’s identification number details.

The access to original data was restricted to the evaluation team.

Analysis

All data was analysed using descriptive statistics in aggregate. Focus groups were

recorded, transcribed and analysed using thematic analysis.

Results - Carer Survey

Fifteen carers were recruited to ECHO but only four carers went on to participate in the

carers ECHO and all responded to the survey. Demographic data of participants are shown

in box 5. Carers were of a wide age range, and the majority were female. One was still

working, one was a carer in the community, one was retired and one had given up work to

care for her husband. Three attended all four sessions. Participants had varied previous

experience with technology prior to participating in ECHO.

Box 5 Carers Demographic Data

What is your age?

Answer Options Response

Percent Response

Count

35 to 44 25.0% 1

45 to 54 50.0% 2

65 to 74 25.0% 1

What is your gender?

Answer Options Response Percent

Response Count

Female 75.0% 3

Male 25.0% 1

95    

With regard to l iv ing arrangements, please t ick al l the answers that apply

Answer Options Response

Percent Response

Count

Live with the person you are caring for long term 50.0% 2

You stay with the person you care for sometimes 25.0% 1

You and the person you care for don't live at the same address 25.0% 1 When taking into account your caring commitments, do you f ind i t easy to attend face-to-face meetings aimed at support ing you as a carer i f you want to?

Answer Options Response Percent

Response Count

Yes 0.0% 0

No 33.3% 1

Unsure 66.7% 2

With regard to transport do you...

Answer Options Response Percent

Response Count

Have your own car 75.0% 3

Get public transport 25.0% 1

What is your relat ionship to the patient?

Answer Options Response Percent

Response Count

Spouse 50.0% 2

Child 25.0% 1

Other 25.0% 1

What age is the person you are caring for?

Answer Options Response Percent

Response Count

41-50 25.0% 1

61-70 25.0% 1

71-80 50.0% 2

What gender is the person you care for?

Answer Options Response

Percent Response

Count

Male 50.0% 2

Female 50.0% 2

96    

Does the person you care for need someone with them at al l t imes?

Answer Options Response Percent

Response Count

Yes 25.0% 1

No 50.0% 2

Unsure 25.0% 1

Have you previously attended a carer support group?

Answer Options Response Percent

Response Count

Yes 75.0% 3

No 25.0% 1 With regard to your experience with technology prior to start ing the ECHO please put one t ick in each row that best applies to you

Answer Options No Experience

A l i t t le Experience

Quite a lot of

Experience

Very Experienced

Rating Average

Using a computer 0 2 1 1 2.75

Using Skype 1 2 1 0 2.00

Emailing 0 1 1 2 3.25 Using social media e.g. Facebook, twitter etc.

2 1 1 0 1.75

Use of the internet to find out information 1 1 2 0 2.25

Online Shopping 1 0 2 1 2.75

When asked their views about the ECHO network, all respondents found the sessions either ‘very’ or ‘quite’ interesting, all found them ‘very’ relevant, and one person found it ‘quite’ stressful. The others did not. See figure 13 for more information.

Figure 13 Carers Views on the ECHO Sessions

0 0.5

1 1.5

2 2.5

3 3.5

4 4.5

Generally interesting

Generally relevant

Helped me in my

caring role

Enjoyable Tiring Stressful

How did you find these sessions?

Not at all

A little

Quite a lot

Very much

Unsure

97    

There were four sessions in the program and all respondents stated the sessions were

‘excellent’. When asked about the technology, 100% stated it worked well but only one

person stated they found it easy to use. Half felt they could communicate with others during

the sessions via the technology well, three people found it easy to relate to others, one

found it difficult to speak out and two would have preferred being in the same room. Figure

14 shows participants views when asked about using the technology compared with

traveling to a meeting. Half would prefer a face to face carers support service, and half

preferred the ECHO technology. All would participate again and would recommend the

service to others. Reasons given for stopping attending are in table 15.

Figure14 Carers views on ECHO compared with traveling to a meeting and feelings of

support

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0%

I found it convenient time

wise

I found it convenient not having to travel

I found it convenient not having to leave

the person I was caring for to attend the meeting

I would have preferred to do

this from somewhere that was not home

I would have preferred to travel to a meeting

How did you find being able to attend a session using the technology rather than having to travel to a meeting? (Please tick all that apply to you)

0

1

2

3

4

5

By other carers

Did you feel supported during the sessions? (Please put a tick in each line)

Not at all A little Quite a lot Very much Unsure

98    

Table 15 Reasons for stopping attending sessions

If you stopped attending sessions, was it because you...(please tick any applicable

answers)

Answer Options Response

Percent Response Count

Would have preferred to meet people face to

face 25.0% 1

Too busy due to other life circumstances 50.0% 2

The person I care for became too unwell 50.0% 2

Other (please specify) 50.0% 2

Results of Focus Group with Hub Members

Five HCPs who participated at the hub for the ECHO programme for Carers participated in

the focus group. Participant’s profiles are shown in table 16.

Table 16 – Hub focus group Participants Profile

Profession Role

Palliative Care Nurse Consultant Hub Member

Social Worker Hub Member

Physiotherapist Hub Member

Project Manager Facilitator and co-coordinator

Occupational Therapist Hub Member

The data was analysed by adopting a thematic analysis framework as described by King and

Horrocks.(24) Analysis of the focus group data uncovered two overarching themes, each

with their own descriptive and interpretative themes that are outlined in thematic diagrams.

Overarching theme one, challenges and suitability of Echo for carers explored the

challenges faced when conducting this particular ECHO program, and how suitable it was for

this group (figure 14). Overarching theme two, consideration for the future of ECHO for

carers, depicted the particular elements of ECHO that participants perceived needed

considered to enable the success of future carer’s programmes via the ECHO platform

(figure 15).

99    

Figure 14 Theme 1 ‘Challenges and Suitability of ECHO for Carers’ for Carers ECHO

Descriptive Themes Interpretative Themes Overarching Themes

1 Challenges and

Suitability of ECHO for Carers

1A Suitability of ECHO

for carers

Built a rapport with carers

Carers gained knowledge and support

Encouraged consideration of end of

life

Support in their own home

1B Challenges of ECHO

for carers

Lacked support of face-to-face

Did it address carers needs?

Too many healthcare professionals

Technology

Recruitment and retention was challenging

100    

Figure 15 Theme 2 ‘Consideration for the future of ECHO for Carers’

Descriptive Themes Interpretative Themes Overarching Themes

Overarching Theme 1: Challenges and Suitability of ECHO for Carers

This theme encompassed how participants perceived ECHO to have some elements of

suitability in relation to providing support and information to carers of people with a life-

limiting condition. Interpretative theme 1A depicted how participants perceived that ECHO

did facilitate building a rapport with carers and also helped carers to gain knowledge and

support for their caring role. Perceptions also highlighted how ECHO provided a platform

which was accessible in carers own homes and also encouraged carers to consider the end

of life for the person they cared for. Interpretative theme 1B however illuminated the

perceived challenges of using this platform for carers to provide support and information.

2 Consideration for

the Future of ECHO for Carers

2A Future Hub Members

Facilitator is key

Rolling programme

2B Future for carers

Do not need a disease specific

ECHO programme

Blended approach needed

Recruitment considerations

Good to know about the person being

cared for

101    

Accounts demonstrated issues regarding the lack of face-to-face support for carers and

concerns from hub members regarding optimally addressing the needs of the spokes.

Findings also conveyed concerns regarding the number of professionals at the hub in

comparison to carers, and issues regarding technology. Recruitment was also perceived as

a challenge and this was discussed further in interpretative theme 1B.

Interpretative Theme 1A: Suitability of ECHO for Carers

Accounts portrayed how the ECHO for carers allowed professionals at the hub to build a

rapport with the carers. Some felt this rapport was similar to that at face-to-face carers

groups:

“I think there was a feeling of a bit of rapport with the carers. I know there was a small

number but you did feel that it was kept quite informal and you did feel that you got a bit of

personality from them. So I don't think it was very cold compared to face to face, I think there

still was that bit of interaction with people on a certain level.” (CE3)

Many also felt that the programme provided carers with knowledge and support in relation to

being a carer:

“I just wanted to say one of the real positive experiences has been the feedback carers have

given me. A carer has been in touch to say he felt supported in it.” (CE5)

Many participants perceived that carers benefited from the information provided at the

sessions:

“I think the information sessions and the teaching element of it was good and I think the

carers who were involved gained a lot of knowledge through that. The presentations that I

saw were very good and very informative and I'm sure those bits of knowledge gained.”

(CE2)

Findings also highlighted that similarly to other carer groups, ECHO encouraged carers to

consider the end of life for the person they cared for:

“I think the other positive thing is one of the carers started to think about her end of life

wishes for her mum. I know that would happen in any carer session really, that if people are

feeling supported they feel confident to make those decisions. But that was one of the

things.” (CE5)

102    

“I think the good thing was that it maybe started to make them [carers] think. Like the lady

who said she was going to think about end of life care for her mum and what her mum's

wishes would be and things. It maybe made them think about things that they wouldn't have

if they hadn't been part of the group.” (CE4)

Findings illuminated perceptions that ECHO was a positive form of delivering a carers

program as the support was provided in their own home:

“It [ECHO] allows the carers to be in their own home.” (CE1)

“I was just thinking about there's times [a carer] would have been sitting with a cup of coffee,

and in some ways that's nice that people can be relaxed in their own home.” (CE3)

One carer expressed that this would have enabled them to still carer for their husband whilst

participating:

“I think one of the positive things, and I know there was a carer, she was caring for her

husband with Motor Neuron Disease. She said she would have liked to join just for that

security of being able to access that training but still being there to be able to care for him in

the home.” (CE5)

Interpretative Theme 1B: Challenges of ECHO for Carers

Findings also conveyed that participants perceived some challenges to delivering an ECHO

program via the ECHO. Some felt that it lacked the support that would be available at a face-

to-face carers group:

“There wasn't much room to actually provide support to people, compared to, say, a group

setting where if someone was in distress you could spend more time with them or if

someone was very quiet and just wanted to listen to the others. I think it was more

information sharing, and that was very good but it lacked, I feel, the support of one to one or

group face to face contact. So I think that was missing in it, just because of the set up of

Echo.” (CE2)

It was also conveyed that carers may have missed the support of each other that is

perceived to be provided at a face-to-face session:

“I suppose I'm just comparing it with what I've known in the past with carers' groups, and I

think the main thing about a carers' group, I see myself as facilitating it but the carers

103    

support each other, although there's a programme but the carers sort of learn from each

other and support each other. And I suppose that's the other side of it. The information

sessions were all very good but I still think that element was missing.” (CE2)

Findings portrayed that hub members also had concerns before ECHO started as to whether

it would optimally address the needs of carers:

“I suppose I did have my concerns how it would work with carers in that particular setting

and I thought would it address all their needs?” (CE5)

“I've been involved in a couple of other ECHO projects and my experience has been with

other professionals. I suppose my concern with the carers was, as has already been said, is

that we're working with service users and protection around them.” (CE2)

Many also expressed concerns regarding if ECHO would provide optimal support and how

this would be managed:

“I was also concerned around, when I had heard it talked about before, I had heard the word

"support" used, as well as information, and I suppose I was concerned about how you could

support a group remotely and how, if there was a huge group of carers, which wasn't the

case in the end, but if it had been, how to facilitate remotely, and if you had someone with an

agenda or who had very strong views, I was concerned how that could be managed via the

ECHO technology.” (CE1)

One participant expressed that prior to ECHO commencing; they had perceived it would

entail more practical support:

“I thought it would have been more kind of practical support that they would be wanting,

more of hands on type learning.” (CE4)

Accounts additionally illuminated that participants perceived some the sessions which carers

requested may not have been fully addressed. Some felt that if sessions didn’t take place

then the carer may have missed out:

“I suppose the weeks that were cancelled for a certain reason then didn't happen, and they

were issues that people had asked at the sessions to be addressed. So it's just if you missed

your week then it was gone. So the full programme wasn't really delivered as such.” (CE4)

104    

Some also portrayed that they were unsure if the sessions fully addressed what the carer

had requested:

“The first week was about getting the agenda and it was supposed to be getting what carers

wanted, but I suppose we were all new to it as well and you were never sure if you really got

the topics or got the things that were actually important to them, and is there a way of doing

that or do you need an introductory session and then for them to come back to do that, or

afterwards. But I don't know if in some ways we sort of set some of the sessions; it wasn't

always what they wanted.” (CE3)

Findings illuminated that participants perceived there may have been too many professionals

at the hub at times, in comparison to carers:

“Sometimes I just felt that there were six or seven of us [professionals at the hub] and one or

two carers and it was just unbalanced a bit.” (CE2)

Accounts also portrayed that having a lot of professionals at the hub also meant using a lot

of professionals at the onetime to deliver a programme to a small group:

“You need a lot of professionals [for the hub] and you're using a lot of professional time for a

small group for [ECHO], if you want it you can have it teaching but then you could use a few

professionals but then you don't have the support element. So it's hard to get a balance

between the two.” (CE3)

Accounts additionally highlighted the challenges faced during the ECHO programme in

relation to the technology used. Prior to the commencement of ECHO some were concerned

how the technology would work:

“I suppose I was a bit nervous about technology and how that would work and whether it

would be complicated or not. I thought the idea sounded good but for me, it was just sort of

wondering how the technology would go.” (CE3)

Some felt this was the reasoning behind some carers not partaking in the programme:

“So I think technology was one of the reasons why some carers didn't participate.” (CE5)

105    

However many of these concerns were eased during the ECHO programme:

“One of my fears had been about the technology, but actually that side of it was very easy.

So it is a good system to link into. It works well on a teaching perspective, being able to get

information across.” (CE3)

Some technical difficulties were experienced during the ECHO sessions though:

“Our only problem was we couldn't hear very well the first couple of weeks, so we ended up

buying speakers and then that helped, but we found it very hard to hear. Anyone hadn't

muted themselves there was a lot of background noise so it was hard to hear who was

talking. So that was the only thing with the technology for us.” (CE4)

“One week [at ECHO] I couldn't see myself, and from then on every time I tried to access

Zoom, IT had worked on it an hour beforehand, yesterday for an hour and a half, I had to

come here today [the Hub], they still couldn't get it to work. But I've had them [IT] working on

it on three occasions recently for different things and we still can't get the network

connection, for some reason.” (CE1)

Findings depicted the challenged the programme faced due to issues regarding the

recruitment and retention of carers. Within the recruitment stage there were 15 carers to

partake but only four participated. Many felt it was hard to recruit due to difficulties in

explaining a new mode of delivery for carer support and information:

“I think recruitment was one of the really challenging things with this pilot. Because it was

new I think it was hard to explain the process to people, because it was just a concept at that

time.” (CE5)

“Although I have some experience of ECHO in other settings, it was trying to explain how it

would work, and I think, even if you're running a carers' group face to face, it's difficult to

recruit people, to get them to come, because you couldn't explain what it is, and sometimes I

felt that I was trying to explain something that I didn't know... I couldn't put it across well, I

think, to some of the carers just how it would work. So I think that put people off as well.”

(CE2)

106    

Some perceived prognostication of the person being cared for challenged recruitment:

“From our point of view it was very hard for us, because the original brief was that they had

to have a prognosis of at least six months and for us that ruled out anybody we had as a

patient at the time. So we had to look towards oncology, the Macmillan nurses, to see if they

could get anybody for us, and the ones they had been dealing with had maybe just got bad

news or things like that. So they found it hard to recruit so it was people who were already in

a support group were our best target audience for the pilot because it was just hard to get

anybody else that fitted what we were looking for.” (CE4)

Retention was also a challenge within the programme:

“We had 15 participants that wanted to participate in the ECHO. Other lines just took over

and they couldn't always participate in the sessions, and I suppose that's one of the things

that's always going to happen in this type of setting,” (CE5)

“it was smaller numbers, very small numbers and not consistent people attending, so that

made it more difficult.” (CE1)

Overarching Theme 2: Consideration for the Future of ECHO for Carers

Overarching theme 2 encompassed the perceptions of the hub members regarding issues

that should be considered for future ECHO programmes for carers. Many felt that for an

optimal hub the facilitator was very important and it was also perceived that other

professionals may benefit from a rolling programme to enable attendance. In relation to the

carers, participants portrayed that the ECHO programme for carers does not need to be

disease specific; however a blended approach may be of benefit. Accounts also illuminated

ways in which to optimise recruitment and the benefits of knowing about the person the carer

is caring for.

Interpretative Theme 2A: Future Hub Members

Findings highlighted the importance of an effective facilitator in coordinating the programme

and providing consistency to the carers:

“I think we were blessed in having a very good facilitator who coordinated the whole thing.

That was a lot of work, a lot of pressure, and she facilitated the meetings extremely well.

That's a big commitment and I think for it to go forward and for it to be successful you need

somebody like that.” (CE1)

107    

“You still need that one person [facilitator] who's a good coordinator, to be consistent for the

carers as well, to make it feel like they're connecting to something rather than just looking at

a programme. “ (CE3)

Some felt it was also important to have a rolling programme to enable professionals to

attend relevant sessions and not every session:

“I do think it would be very hard for us [professionals] to justify our time here [at ECHO] when

there's so many other things that we are supposed to be involved in with our patients here

[in work]. If it's a small group, and a support group, you know justifying our time to our

managers would be very hard whereas if it's a rolling programme that you're there for set

weeks or certain weeks I think that would be more sustainable.” (CE3)

“I wonder if it's to be a rolling programme or an ongoing teaching programme or something

could you rotate it around the trusts, because it just means it would keep it fresh but also it

would open it up to different staff and maybe allow them to maintain it so that you're only

doing it every so often, which would also justify it to managers.” (CE3)

Interpretative Theme 2B: Future for Carers

Participants highlighted their perceptions of carers taking part in future ECHOs. Findings

highlighted perceptions that future ECHO sessions for carers did not need to be disease

specific:

CE2: “Carers are carers and even though they're caring for someone with a different illness,

even if it was all cancer related illness there would be differences in care for someone with a

brain tumour as opposed to something else. So I think there's more common ground, no

matter what the illness is. There's more commonality between just being a carer than trying

to break it up. It's just an opinion, you know?

CE1: I would agree.”

“Carers had the same issues, the same feelings of isolation and the same family issues. The

specific stuff around symptom control, some of that might be similar as well but we didn't go

into that detail so we felt it was only fair to open it up to people caring for any [life-limiting]

condition.” (CE5)

However some felt that a blended approach to the support provided may be needed in the

future:

108    

“It’s one of them things where one size doesn't fit all and maybe if we used the ECHO for the

information and then followed up face to face or had a blended approach then you would

use the ECHO to save professionals time, especially if you were doing the one to one

sessions with the carers, because I can imagine that would be quite time restrictive. But

yeah, maybe look at different ways of incorporating this way to have reached them in the

home and then bring them out for further support, if that's possible.” (CE5)

Participants also talked of how recruitment issues may be addressed in future Carers

ECHOs. Some felt that recruiting through primary care providers may enhance participation:

“I'm just wondering, for us, we're seeing people in an acute setting or in a hospice setting,

but it might be people that you could target, people like the community nurses and district

nurses and the hospice nurses might be the people, because they're going into the homes

and they're seeing how people are managing, they would be better suited to recruit people in

their own home and talk about it there.” (CE3)

Findings additionally highlighted that knowing details of the person being cared for may help

in providing optimal care:

“It may also be useful to know a little bit about the person that the carer is caring for and their

condition. There could be many and varied conditions and if you're giving a talk you need to

be sure that you're actually addressing what their needs are and not going off on a tangent

about a topic that has no relevance to the condition of the person that they're caring for.”

(CE1)

Discussion

This ECHO program for carers found it hard to recruit and retain sufficient numbers of

carers. While all of the carers who participated in the ECHO network participated in the

evaluation, it was not possible within the scope and timeframe of this evaluation due to the

need for formal ethical approval to survey the people who did not participate in the ECHO

network to determine their reasons.

The survey results highlighted that the majority of carers viewed participating in the carers

group via ECHO as a positive experience. Some were unsure if they would find it easy to

attend face–to-face meetings due to caring commitments; although the majority had

previously attended a carer’s support group. Although many felt it was convenient to use the

ECHO model for the carers group, some still would have preferred communicating with

109    

others in the same room. Carers gave excellent feedback regarding the sessions delivered

and felt very much supported by all professionals involved. However, one carer found some

of the sessions stressful but it was not possible to explore the reasoning for this. Overall, it

was perceived that the technology worked well and all participating carers would recommend

the ECHO programme and participate again. While this feedback is positive, it would be

dangerous to draw firm conclusions based on the responses of four participants.

The focus group findings illuminated the intricacies of the experiences of hub members

involved in this particular ECHO programme. Similarly to the survey findings, professionals

highlighted the need for face-to-face support alongside the ECHO sessions. However it was

perceived that the ECHO platform enabled carers to gain knowledge and support whilst also

fostering relationships between the carers and professionals. The ECHO platform provided

support in the carers own home which was seen as beneficial, however there were concerns

as to whether the sessions appropriately addressed the areas which carers wanted covered.

Many acknowledged the benefits of ECHO in encouraging consideration of end of life needs,

however it was also highlighted that this was a feature of many carer support groups.

Recruitment and retention was challenging, and professionals felt that this was due in part to

the difficulties in explaining the new concept to carers, that the technology may have been

off-putting to some, and also due to the prognosis of the people being cared for. It was

highlighted that for future recruitment it may be of benefit to utilise primary care providers to

identify potential carers. For future ECHOs the facilitator needs to play a key role as the pilot

programme was well supported and coordinated by the facilitator. Professionals depicted the

difficulties in justifying time to attend the ECHO clinics and suggested a rolling programme

may be more feasible for future sessions. Findings also conveyed that although disease

specific carers groups are not perceived to be required, it would be beneficial to hub

members to be aware of the needs of the person being cared for.

Due to the very low numbers of participants it is difficult to draw firm conclusions on the

usefulness or not of the ECHO model in providing support to carers. While there were

positive feedback from both carers and hub members, there were also significant issues with

recruitment and retention, and likely cost effectiveness due to the large numbers of HCPS at

the hub and few carers at the spokes. Perhaps most significantly the perceptions of some of

the hub members towards seeing ECHO as a potentially useful means of carer support grew

stronger through the process. An economic analysis was not possible within the scope of

this study. The lack of face to face contact is also a recurring theme for both carers and hub

members. A larger study with sufficient time and funding to get ethical approval to explore

the issues raised here more in depth is necessary to determine the future use of ECHO in

110    

this setting. The feedback from participants and hub members has helped identify some

issues that could potentially be addressed in future programs e.g. using a mixed or blended

approach with some ECHO sessions for information sharing and some face to face sessions

for more supportive care.

It was not within the scope of this evaluation to look at the impact on the carer or the person

they cared for, and future studies should look to address this to consider the outcomes for

carers.

Suggestions from evaluation

ECHO may be a viable method with which to deliver carers support in the future but

further research is needed as this study was of insufficient size to draw firm conclusions.

Future studies with appropriate time frames and funding need to explore the feasibility of

ECHO as a platform to deliver carers support and also examine the outcomes for carers

and those they care for.

- A blended approach may be most helpful to also allow face-to-face sessions within

the programme.

- Carers need to be given time to reflect on the chosen topics to ensure they address

their needs.

- Recruitment needs to be carefully considered and other professionals who are

working with this client group need to be involved i.e. primary healthcare providers.

- A rolling programme needs to be considered to ensure that professionals can attend

sessions that are relevant to their expertise.

- The facilitator role needs to be appropriately assigned and remain consistent for the

carers and professionals.

111    

Chapter 8 - Hub Feedback

Methods

To determine if there was a benefit to hub members in participating in ECHO knowledge

networks and to determine their views on having participated in ECHO, all hub members

were sent out a survey via Survey Monkey as described in Chapter 2.(8)

Results

Eighteen hub members responded to the survey- seven from the optometry ECHO, nine

from the Diabetes ECHO, one from GP dermatology and one from the NH ECHO. Results

are shown in table 16. All participants (100%) who participated in the survey indicated they

had learnt through ECHO, both from hub and spoke members, they found it enjoyable and

would recommend it to others. They felt it helped translate knowledge into practice, and all

responders indicated they would be willing to participate in future ECHOs. They were

positive about the use of ECHO technology.

Table 16 Hub feedback on ECHO

Number %

Profession Physician

GP

Optometrist

Allied Healthcare Professional

Other (please specify)

6

1

4

4

3

33.3

5.6

22.2

22.2

16.7

Whilst participating at the ECHO

Hub did you (all that apply)?

Facilitate a session

Deliver a teaching presentation

Participate in discussions regarding a case

Provide specialist advice / guidance on

topics being discussed

5

11

15

14

27.8

61.1

83.3

77.8

Do you feel ECHO offered you

an effective platform with which

to pass on relevant knowledge

and skills to others?

A lot

A little

No

17

0

1

94.4

0

5.6

Overall do you feel you have

learnt personally through

participating in ECHO?

A lot

A little

No

17

1

0

94.4

5.6

0

112    

Did you learn from other Hub

members?

A lot

A little

No

13

5

0

72.2

27.8

0

Did you learn from participants

at the spokes?

A lot

A little

No

14

4

0

77.8

22.2

0

Did you find participating in

ECHO enjoyable?

A lot

A little

No

18

0

0

100

0

0

Do you think your participation

in ECHO has changed your

practice at all?

A lot

A little

No

Unsure

4

12

0

2

22.2

66.7

0

11.1

Do you think the format of

ECHO helps translate

knowledge from teaching into

practice?

Yes

No

Unsure

18

0

0

100

0

0

Would you recommend ECHO

to other Healthcare

professionals in your area?

Yes

No

Unsure

18

0

0

100

0

0

Would you like to participate in

ECHO sessions if the

opportunity arose in the future?

Yes

No

Unsure

Unanswered

17

0

0

1

100

0

0

Do you have ideas as to how

ECHO could be embedded

within your own working week to

provide a sustainable model of

maintaining a community of

practice which would benefit

patients?

Yes

No

Unsure

Unanswered

14

0

2

2

87.5

0

12.5

113    

Participants had the opportunity to write comments about how ECHO could be embedded

within the working week to provide a sustainable model of maintaining a community of

practice which would benefit patients. Comments were generally positive and included the

need for having participation job-planned with appropriate funding / backfill, that either hub

members should be able to link virtually or there should be other hub sites across the region,

and the need to gather evidence that the ECHO program in NI affects practice. Comments

were:

“ECHO could be adopted to facilitate sharing experience & learning for my local trust

diabetes team as the team spans primary & secondary care & across three hospital

sites, could be intra/inter- professional potentially be use to facilitate training for d-

Nav service reviews across the trust & wider if the service gets funding to rollout

across NI or even UK.”

“A shorter programme to address diabetes education for staff, as a rolling

programme, aiming to provide base line information and support practices. To create

specific " levels" / programmes of support and education. To engage practice nurses

in sessions and identify topics of use to them. Engage other members of specialist

diabetes teams across various trust to input and spread learning regionally. ? spread

of practices in spokes- to enhance learning from all areas. Establish as part of

Diabetes Network ( regional strategy)”

“As detailed in draft submission for 2016-2017: Key themes in the responses to the

first draft I did with ideas about how it could have wider participation were to keep

disciplines together (not separate GP and practice nurse ECHOs) and to recognise

and facilitate spoke participation by finding backfill. The hope is that GP Federations

could help engagement. However, advice from Barney McCoy was that spoke

participants needed funded. I have no idea about this. The devil is in the detail of how

this will run to get wider representation from spokes and even within the hub. I

envisage a core hub, inviting other clinical staff to participate here and there perhaps

via video link rather than physical presence. I also suspect that we should run 2-3

diets of a fixed didactic programme in 1 year, and have other sessions in between

that focus on cases in the classic ECHO model. We would need to discuss the detail

of this based on your SuperHub experience. We can then adapt a specific running

order in agreement with core hub members. I talked to Damien McMullan, palliative

care consultant in Derry who is a regional clinical lead. He is very keen to bring this

to palliative care more and will talk to Max. However, he’d like us to club together as

a few specialities like diabetes, palliative care and perhaps another like respiratory to

114    

create another hub in the northwest, perhaps focused in the Hospice or Hospital. A

second one could run from NI Hospice Belfast. That would allow much wider

inclusion, but would mean funding that possibly exceeds our current capacity. It

would mean that a session would take up 2-3h for people from this area, rather than

5h as at current (travel time included).”

“I personally have flexibility in my working week.”

“Needs properly job planned. Also the funding for ECHO was in my opinion not done

well. There was only 2 weeks notice given to hub members at the beginning to

cancel clinical sessions rather than the mandatory 6. Also, trust were paid in advance

for backfill of these sessions but to date have not passed on this money for the

purpose that it was intended resulting in a significant increase in my hospital waiting

lists because I participated in ECHO under the ?false pretence that it would be

backfilled.”

“ECHO is an ideal methodology to reach and work with a significant number of HSC

and other staff which could be otherwise difficult due to current constraints on staff

time. The Safety Forum's model for working is through themed "breakthrough QI

Collaboratives". This involved bringing HSC and other staff (e.g. nursing homes)

together for a 12-18 month period through learning sets (face to face\) and action

period in local organisations. An area of focus is identified, for example, sepsis, falls,

etc... and front line teams come together to learn how they can improve practice in

the identified area; making changes, measuring and striving for reliability and

sustainability over time. ECHO provides a way to bring more staff together - they can

meet in their own local organisations through the multi-web technology and have

access to colleagues in other Trusts. This helps to reduce time away from their own

organisation promoting more rapid QI and also means that more frequent sessions

can be run rather than waiting for 4 months between each face to face learning set. It

is felt that this would be an excellent way to progress taking account of current

constraints and building that community of practice in a number of areas.”

“An hour or so allocated to this during the working week would seem to me to be

perfectly acceptable. However as with other associated professions, enhanced

remuneration should be taken into account as part of the CET (Continued Education

and Training) grant in recognition of the improvement both to the quality of patient

care and improved efficiencies within the primary and secondary care interface.”

115    

“Could become part of working week if properly job planned. As it stands funding for

backfill went direct to trust finance departments, there was insufficient notice to

reschedule clinics and my clinics and patients have suffered as a result of my

absence to attend ECHO, backfill money has not to date been passed on by trust.”

“other specialist diabetes dieticians GP's within own trust regional dietetic projects

e.g. outcomes”

“No harm to the excellent staff at Hospice Hub, but it takes me 3h round trip to get to

and from the hub from work in Derry - half a day. If we did this at NW Hospice or

WHSCT location, more of us from the West and South West could take part as Hub

more regularly and more often. I'd definitely like to have second hub at West/NW

region from which different specialities could run ECHO at different stages of the

week. There should be two hubs: Belfast and Derry to greatly increase our reach and

capacity. Include and actively invite practice nurses. The pilot had mainly GPs with

expressed interest and some with less and a few PNs. PNs deliver most diabetes

care throughout NI, particularly where the GPs in their practice have less interest in

diabetes. We need GPs from the WEST involved -conspicuous absence that

unnerved me! Maybe they wouldn’t engage, maybe they weren’t asked. Openly and

widely invite and encourage, targeting a geographical spread of GPs across an area.

That may facilitate other participants. Is it feasible that more people could observe

online while live, though not be able to speak/be seen - reducing bandwidth

requirement at hub for extra people to add in? I think a regular session with staff in

the hub rotating in is a good idea from same Trust, missed Trusts or, different Trusts

in rotation. Rotating from one Trust manning hub to another would allow attraction of

GPs and PNs from their area I hope, spreading the love around. An ongoing

programme with advance notice of topics will allow participants to perhaps attend a

series in a row for a cycle of topics, and dip in and out of others later as the series of

topics come round again as they inevitably will. I want more!”

“Since being involved in echo I have now become and enthusiast and can think of a

lot of different scenarios where this would be excellent for educational delivery and

also it lends itself beautifully to multidisciplinary interaction”

“Although I am very enthusiastic about ECHO and I have greatly enjoyed I think it

would be important to gather evidence that ECHO is achieving what is meant to be

achieving Evidence from the USA may not be generalisable to our setting. We may

need to do a trial”

116    

“ECHO not only provides a safe space for tele-mentoring, but it also offers an

excellent vehicle to share experiences, two-way and every way. By facilitating peer

discussion and peer-to-peer learning, it provides a governance framework, building

trust, that will enable care to be shared, and delivered, safely, closer to the patient's

home. ECHO is a model of excellence for demonopilisation of medical knowledge

and building a strong community of practice.”

“ECHO was a fantastic experience. As a hub member, I arrived thinking it was all

about cascading my knowledge and experiences to the spokes, but I learnt a great

deal from the spokes, from their insightful questions on my presentations, directly

from the comments but particularly from conversations between spoke members.

The nature of the work done by the spoke members was brought home to me by

ECHO, and ideas and motivation to carry on work between hub and spokes greatly

increased. I hope can continue, ideally monthly, and be developed, for example to

ensure dissemination of spokes' learning to 'sub spokes'. The phrase 'community of

learning' may have sounded trite at the beginning, but that's truly what ECHO was all

about, and has created: this community now needs nurtured.”

“It may prove difficult to continue on a weekly basis as I would be time limited, but

perhaps some form of monthly platform?”

“Ophthalmic staff within the Health and Social Care Board are progressing work to

reform eyecare service by delivering an integrated approach to the commissioning

and provision of eyecare services. This work involves the development of a pathway

approach for high demand and long term ophthalmic conditions. Better ways of

working must be considered in order to manage demand and afford patients good

outcomes and experience by adopting a multidisciplinary team approach spanning

the primary and secondary care settings. Ophthalmic services are one such service

in which patients have access to qualified, skilled and well equipped professionals in

both settings and hence significant opportunities exist for joined up patient-centred

care. Eyecare pathways incorporate models of care which have governance, intra

and inter professional communications and evidence based care as core

requirements. ECHO facilitates an innovative approach to embedding and supporting

all of these essential components. Implementation and roll out of ECHO within the

work I do in HSCB Ophthalmic services (Optometry/Ophthalmology) would have

several benefits: 1. Regular and consistent opportunities for the two way sharing of

knowledge and experience between ophthalmic professionals 2. Opportunities for

117    

service development in relation to a shared approach to the management and review

of risk stratified patients who have 'stable' long term eye conditions. This would

reduce demand on secondary care review appointments and allow patients to

receive safe and effective care closer to home. 3. Optometrists in primary care within

a community of practice supported by ECHO would, in time, transform into pockets of

excellence which would become a valuable resource adding resilience into the

system. 4. ECHO facilitates the development of cross-professional links to support a

holistic approach to addressing the needs of patients. For eyecare this will

incorporate links with the voluntary sector, low vision support services, medical

services, pharmaceutical services and other groupings.”

Discussion

While many hub members thought that participating in ECHO was going to be of benefit to

the spokes, all members also learnt themselves from both other hub members and those at

the spokes. All found participation enjoyable, and that the format of ECHO helps translate

knowledge from teaching into practice. All would participate again in the future and would

recommend to other HCPs in their area. The need for adequate funding and job-planning

was highlighted in order to allow hub members to continue to participate.

118    

Chapter 9 - Overall Discussion

Over the last six months, four ECHO knowledge networks have been operating, through a

total of 37 sessions, involving over 150 HCPs in Northern Ireland. In addition a small ECHO

with four carers of palliative care patients has been facilitated, with much learning about the

options for future use and research.

Two networks (Dermatology for GP trainees and Optometry) showed statistically significant

improvements in knowledge and self-efficacy, in keeping with the results from the initial NI

ECHO pilot in community hospice nurses.(1) The response rates for the Diabetes and

Nursing Home networks were low and statistical analysis couldn’t be performed.

Feedback from the four networks in HCPs showed an overall very positive view on the

ECHO networks and their impact on patient care by both the participants in the spokes and

the hubs, and this was confirmed by the focus groups. The combined feedback from the

four networks is shown in box 5, and demonstrates that the most useful areas in the

sessions are the presentations followed by the case based discussions. All participants felt

they had learnt through participating in ECHO, all had enjoyed it on some level, and all felt it

had improved the care they provided for patients, perhaps the best marker of the networks.

Only one person across the networks did not think the format of ECHO helped translate

knowledge from teaching into practice more than other teaching sessions they had been

involved in, and no one said they wouldn’t recommend it to other HCPs. The majority found

ECHO gave them access to education that would have been hard to access due to

geography (26 out of 34 respondents – 76.5%), that it was a good medium to access

teaching / education at a different location from where they worked (34 out of 35

respondents – 97%), that any technical difficulties were acceptable and did not put them off

participating in ECHO (31 out of 33 respondents – 93.9%) or reduce their learning (31 out of

32 respondents – 96.9%). On a scale of 1 to 5 (1- poor, 5 – excellent) 34 out of 35

respondents (97.1%) rated it 4 or 5, and 100% would participate in ECHO sessions in the

future.

119    

Box 5 Summary of Spoke Results on General ECHO feedback

1. Rating on a scale of 1-5 the quality of learning /

usefulness from each area (1- poor, 5- excellent)

1 2 3 4 5

Review of previous session 0 0 6 12 12

Presentations 0 0 1 12 24

Case based discussions 0 0 2 11 19

2. Overall do you feel you have learnt through

participating in ECHO?

A lot 32

A little 3

NO 0

3. Did you find participating in ECHO

enjoyable?

A lot 31

A little 4

NO 0

4. Do you think that participating in ECHO has improved the care

you provide for patients?

A lot 28

A little 6

NO 0

5. Do you think the format of ECHO helps translate knowledge from

teaching into practice more than other teaching sessions you have

been involved in?

Yes 31

NO 1

Unsure 1

120    

6. Would you recommend ECHO to other healthcare

professionals in your area?

Yes 33

NO 0

Unsure 1

7. Have you used any of the online

resources via Moodle, and if yes have you

found these useful (dermatology network

did not have access to this)?

Tick if used

and found

useful

Tick if

NOT

useful

Power point presentations 15 1

Video of the teaching

sessions 16 2

Video of case presentations 14 2

Other supporting materials 17 2

8. Regarding ECHO technology…

Agree Disagree Unsure

It has given me access to education that would

have been hard to access due to geography 26 5 3

It was a good medium to access teaching /

education at a different location from where I

work 34 0 1

Any technical difficulties were acceptable and

did not put me off participating in ECHO 31 0 2

Any technical difficulties did not significantly

reduce my learning 31 0 1

121    

9. How do you rate your overall ECHO

experience? (1- poor, 5- excellent)

1 2 3 4 5

0 0 1 9 25

10. Would you participate in ECHO sessions in the

future if the opportunity arose?

Yes 35

NO 0

Unsure 0

The pilot ECHO program for carers found it hard to recruit and retain sufficient numbers of

carers, with only four participating in the network. The survey results highlighted that the

majority of carers viewed participating in the ECHO carers network as a positive experience,

although while many felt it was convenient to use the ECHO model, some still would have

preferred a face-to-face meeting. Carers gave excellent feedback regarding the sessions

delivered and felt very much supported by all professionals involved. Overall, it was

perceived that the technology worked well and all participating carers would recommend the

ECHO programme and participate again. The focus group findings with hub members

highlighted the need for face-to-face support alongside the ECHO sessions. However it was

perceived that the ECHO platform enabled carers to gain knowledge and support whilst also

fostering relationships between the carers and professionals.

Hub members felt that the issues with recruitment and retention were due in part to the

difficulties in explaining the new concept to carers, that the technology may have been off-

putting to some, and also due to the prognosis of the people being cared for. It was

highlighted that for future recruitment it may be of benefit to utilise primary care providers to

identify potential carers. The feedback from participants and hub members has helped

identify some issues that could potentially be addressed in future programs e.g. using a

mixed or blended approach with some ECHO sessions for information sharing and some

face to face sessions for more supportive care.

Weaknesses in this evaluation of the four ECHO networks in HCPs include the low

evaluation response rate. Despite efforts to improve the response rates, with administration

122    

staff contacting participants to encourage on several different occasions, the rates remained

low. Future ECHOs evaluations could improve this response rate by considering several

options, for example recognition (certificate) for participation in ECHO being dependent on

completing the evaluation, funding being dependant on completing the evaluation, or

additional funding being made available to allow protected time to participate in the

evaluation.

Another weakness is that some participants post ECHO were able to complete their

knowledge evaluations unsupervised and return them by email or post. This opens up the

opportunity for this not to be a true reflection of knowledge as they could have referred to

notes or other sources of information. As there was no opportunity to get everyone together

at the end of ECHO, it was unlikely that many would have travelled to come to an evaluation

session without one of the incentives described above. In terms of knowledge evaluations it

may the case that less is more and having a shorter Survey Monkey assessment which was

not too long might ensure higher rates of completion.

For the Nursing Home focus group, the moderator had sat in on some hub sessions,

although had not provided the teaching. It is recognised that this may cause bias for the

nursing home focus group results, but due to the tight timeframe for the report it was not

possible to get another moderator. The moderator did not participate in any other ECHO

network.

Due to the short timeframe between the confirmation of funding and setting up of the

networks and needing to get a pre-ECHO evaluation performed on the training days, some

of the curricula had not been fully set before the pre ECHO evaluation was carried out This

led to difficulties in writing the knowledge papers, which may have led to questions being

asked that were not then covered during the sessions. This could be better address in future

studies with a longer timeframe to set curricula and then write focussed knowledge

assessments.

The low numbers of participants in the carers ECHO makes it impossible to draw firm

conclusions on its future potential and more research in this area is needed.

Due to the short timeframe for this evaluation and the level of funding it was not possible to

consider the direct impact on patient care and service delivery, and future studies should

consider these areas to determine the cost effectiveness of ECHO.

123    

Conclusions

ECHO is being adopted across the United States and the world: by academic medical

centres, large clinical systems, national medical care providers, and even federal

government agencies such as the US Department of Defence. As a low-cost high-impact

model, ECHO has potential to meet the needs and resources of different communities and

populations where there is good access to broadband, and a suitable trained facilitator and

a group of experts willing to share knowledge and support from a central hub. At a time

when health care providers are under mounting pressure to do more and spend less, this

model provides an affordable solution to addressing growing need.

Through the four ECHO knowledge networks that have been evaluated in this report, and

building on the evaluation in community hospice nurses,(1) the results support the on-going

use of ECHO for HCPs education and support locally in Northern Ireland. There was a

statistically significant improvement in knowledge and self-efficacy in two of the networks,

with positive feedback about learning and impact on patient care across all HCP networks.

Due to the very low numbers of participants it is difficult to draw firm conclusions on the

usefulness or not of the ECHO model in providing support to carers. While there were

positive feedback from both carers and hub members, there were also significant issues with

recruitment and retention, and likely cost effectiveness due to the large numbers of HCPs at

the hub and few carers at the spokes. An economic analysis was not possible within the

scope of this study. The lack of face to face contact is also a recurring theme for both carers

and hub members. A larger study with sufficient time and funding to get ethical approval to

explore the issues raised here more in depth is necessary to determine the future use of

ECHO in this setting.

In addition, by demonstrating that the previously documented successes of ECHO from the

United States and in Northern Ireland have been replicated in this prospective study we are

encouraged to believe that the UK could benefit from widespread use of the ECHO model to

help support the delivery of high quality care irrespective of geographical location as in the

United States.

Research is now necessary to determine if the positive outcomes from this study lead to an

impact on service delivery and ultimately improved patient care.

124    

Chapter 10 – Recommendation Summary

• ECHO should continue to be developed, implemented and evaluated to help educate

healthcare professionals across Northern Ireland in a cost effective manner.

• Adequate funding is necessary to allow protected time for all to participate, both in

the hub and at the spokes.

• The importance of the facilitator was a recurring theme, and so this high standard

needs maintained through training and supervision to ensure the networks remain

useful.

• Future ECHO networks should continue to be evaluated to ensure they meet the

needs of the population they are trying to educate and support.

• To improve the response rates in future evaluations, funding for individual practices

or participants should be dependent on participating in the evaluation process, or

funded time should be given. This should therefore reduce the bias of partial

response, and improve the generalisability and hence usefulness of the results.

• Funded studies should be performed looking at the direct impact on patient care to

determine if the ECHO model improves patient care and reduces costs e.g. through

reduced referral rates, improved quality of life, better prescribing etc.

• Minor technological issues should be addressed to make connectivity easy and not

detract from the learning environment. Issues of not being able to connect from

some sites due to HSC security policies also need addressed.

• Further addressing other minor issues such as having preparatory work available in

good time for the spoke members and easily accessible online resources will only

improve on the experience of ECHO

• More research is needed to determine the usefulness of a carers ECHO network.

The suggestions made in this report should be considered if designing a future

Carers ECHO and adequate time and funding is needed to evaluate if the ECHO

format is useful in providing information and support to carers in a cost effective

manner.

125    

References

1.   White  C,  McIlfatrick  S,  Dunwoody  L,  Watson  M.  Supporting  and  improving  community  health  services—a  prospective  evaluation  of  ECHO  technology  in  community  palliative  care  nursing  teams.  BMJ  Supportive  &  Palliative  Care.  2015.  2.   Arora  S,  Thornton  K,  Murata  G,  Deming  P,  Kalishman  S,  Dion  D,  et  al.  Outcomes  of  treatment  for  hepatitis  C  virus  infection  by  primary  care  providers.  New  England  Journal  of  Medicine.364(23):2199-­‐207.  3.   Colleran  K,  Harding  E,  Kipp  BJ,  Zurawski  A,  MacMillan  B,  Jelinkova  L,  et  al.  Building  capacity  to  reduce  disparities  in  diabetes:  training  community  health  workers  using  an  integrated  distance  learning  model.  Diabetes  Educator.38(3):386-­‐96.  4.   Harkins  MS,  Raissy  H,  Moseley  K,  Luttecke  K,  S.  A.  Extension  for  Community  Healthcare  Outcomes  Project  -­‐  Asthma  Specialty  Consultation  Via  Telehealth  to  Improve  Asthma  Care  in  rural  New  Mexico.  US  Respiratory  Diseases.  2011;7(1):7-­‐9.  5.   http://echo.unm.edu/about-­‐echo/  Accessed  26.02.15.  6.   Ammentrop  J,  Sabroe  S,  Kofeod  PE,  J  M.  The  effects  of  training  in  communication  skills  on  medical  doctors'  and  nurses'  self-­‐efficacy:  A  randomised  controlled  trial.  Patient  Education  and  Counseling.  2007;66:270-­‐7.  7.   Nimon  K,  Zigarmi  D,  Allen  J.  Measures  of  Program  Effectiveness  Based  on  Retrospective  Pretest  Data:  Are  All  Created  Equal?  American  Journal  of  Evaluation  2011;32(1):8-­‐28.  8.   https://www.surveymonkey.com/  (accessed  25  Mar  2016).  9.   Kerr  OC,  Benton  EC,  Walker  JJ  et  al  Dermatological  workload:  primary  versus  secondary  care.  British  Journal  of  Dermatology  2007:  157  (suppl.  1).  .  10.   RCGP  Birmingham  Research  Unit.  Weekly  Returns  Service  Annual  Report  2006.  11.   APPGS  reports  (1998,  2004,  2013).  12.   Davies  E,  Burg,S.  Audit  of  dermatological  content  of  U.K.  undergraduate  curricula  British  Journal  of  Dermatology  2009:  160,  999-­‐1005.  13.   Chiange  Y  et  al  (including  C  Griffiths  &  S  Burge)  Undergraduate  dermatology  education:  a  survey  of  UK  medical  students  British  Journal  of  Dermatology  2008:  159  (Suppl.1).  14.   Skin  Conditions  in  the  UK:  A  Health  Care  Needs  Assessment:  Schofield,  Grindlay,  Williams  2009.  15.   Personal  communication  from  Northern  Health  and  Social  Care  Trust.  16.   Transforming  Your  Care:  A  Review  of  Health  and  Social  Care  in  Northern  Ireland  2011  (http://www.dhsspsni.gov.uk/transforming-­‐your-­‐care-­‐review-­‐of-­‐hsc-­‐ni-­‐final-­‐report.pdf).  17.   The  Kings  Fund.  Approaches  to  demand  management:  Commissioning  in  a  cold  climate  2010.  18.   http://www.rcgp.org.uk/training-­‐exams/gp-­‐curriculum-­‐overview/online-­‐curriculum.aspx.  19.   Miles,  A.  Asbridge,  J.  E.  (2014)  ‘Carers–vital  partners  in  increasing  the  person-­‐centeredness  of  health  and  social  care  systems.’  European  Journal  for  Person  Centered  Healthcare,  2  (4),  p.p.  421-­‐424.  20.   Matzo,  M;  Witt  Sherman,  D.  (2010)  Palliative  Care  Nursing  Quality  of  Care  to  the  End  of  Life.  New  York:  Springer  Publishing  Company.  .  21.   Hynes,  G;  Stokes,  A;  McCarron,  M.  (2012)  ‘Informal  care‐giving  in  advanced  chronic  obstructive  pulmonary  disease:  lay  knowledge  and  experience.’  Journal  of  clinical  nursing.  21  (7‐8),  p.p.  1068-­‐1077.  22.   Mc  Veigh,  C.  (2015)  Palliative  Care  for  patients  with  non-­‐malignant  respiratory  disease.  Nursing  Standard.  29  (36),  p.p.  44-­‐51.  23.   Addington-­‐  Hall,  J.  and  Mc  Pherson,  C.  (2001)  ‘After-­‐  Death  Interviews  with  Surrogates/  Bereaved  Family  Members:  Some  Issues  of  Validity.’  Journal  of  Pain  and  Symptom  Management.  22  (3),  p.p.  784-­‐  790.  .  24.   King,  N.  and  Horrocks,  C.  (2010)  Interviews  in  Qualitative  Research.  London:  Sage  Publications  Ltd.  .