Centre for effective services nuala doherty

Designing and Implementing Effective ServicesCrosscare Teen Counselling Conference

Nuala Doherty - Director, Centre for Effective Services

Sarah Rochford - Intern27th September 2013

© The Centre for Effective Services 2013

Overview

1. Designing effective services

What is evidence?

Sourcing and Using evidence: Challenges and Benefits

Putting Evidence in Context

Outcomes

Logic Models

2. Implementing effective services

The Implementation Gap

Implementation Drivers

How CES Supports evidence into practice

3

A small organisation with charity status

Established in 2008

(5 year grant)

A multi-disciplinary staff team

Jointly funded by

Government

• Dept. Environment, Community and Local Government

• Dept. Children and Youth Affairs

Atlantic Philanthropies

All Island organisation

Belfast

Dublin

Introduction to CES

Part

ner

ship

s &

Co

llab

ora

tio

ns

Better Outcomes for Children, Young People, Families and Communities

EffectivePolicy

EfficientSystems

Good Practice

Evidence to Implementation

All-Islan

d

CES Mission

The Centre for Effective Services works with others to connect and support the

implementation of effective policy, efficient systems and good practice, using the best

available evidence, so that children, families and communities thrive.

5

1. What do we mean by Evidence?

Evidence-basedA programme, service or intervention that has consistently

been shown to produce positive results by independent research studies that have been conducted to a particular

degree of scientific quality.

Evidence-informedPractice based on the integration of experience, judgement and

expertise with the best available external evidence from systematic research.

Types of evidence

Consultation

Practice Wisdom

Policy

Research

What Works?

• For whom

• In what circumstances

• At what point in time

• Using which approach

• Under what conditions

Developing a shared understanding of the evidence journey

• Good Practice - A community of judgement of what works

• Promising Approaches - Some positive findings but findings are not

consistent or rigorous enough to be sure

• Research Based - Based on sound theory informed by a growing body

of empirical research

• Evidence Based - rigorously evaluated with consistent results

Evidence must be fit for purpose and take account of the complex nature of service delivery and social needs. It helps us to assess and make decisions on interventions with an ‘evidence lens’

What are the Benefits of Using Evidence?

It can improve

outcomes for children,

families and communities

It is Ethical –do no harm.

It can help us to understand how and why it worked and for whom, so

it we can develop service

delivery for the future

It can increase

chances of future

funding & sustainability

for organisations

Relating costs to

benefits can inform the

debate about policy

priorities and

resource allocation

Establishing what is not

known helps to identify

risks, limitations and grey areas for

new services.

What are the Challenges of Using Evidence?

• Be able to select, weigh and interpret evidence

• Contested definitions of evidence

• Limits to the evidence - Findings from different studies can sometimes be contradictory

• Dearth of research in some areas and can be incomplete

• Take into account the complexity and systemic nature of social problems and their underpinning issues

• The best evidence in the world will not result in better outcomes if poorly implemented

Evidence Needs a Context to be Useful

• Assess how ‘workable’ this approach will be for your area

• Understand the qualitative local experiences, needs and strengths in your

area

• Look carefully at the service delivery mechanism most likely to work for your

client group

• Attempts to distil out the ‘active ingredients’ of effective programmes to

identify effective practice and mainstream.

• Resources are important - think about sustainability from the start

• This will let you make an informed choice

Outcomes

• Outcomes are the changes for service users or other targets of change that happen as a result of an intervention or service being provided.

• There are levels of outcomes, i.e. individuals, families, specific groups, communities or organisations themselves.

• Outcomes can be conceptualised in various ways:• soft outcomes - changes in feelings, thinking and perceptions.

• Hard outcomes - changes in behaviours, attainment or status which can be measured.

• Short-term outcomes, medium or long-term.

• Being outcomes as opposed to outputs focused allows for the development of indicators of success and provides a common end for stakeholders.

• It provides a more direct, focused approach to service provision.

Logic Models

A Logic model is key in the design phase in ensuring your services or intervention is outcomes focused, and helps us to be accountable.

It is a depiction of a programme/intervention showing what it will do and what it is to accomplish.

A series of “if-then” relationships that, if implemented as intended, lead to the desired outcomes.

Research shows logic models can improve communication, clarify programme purpose, improve service delivery, provide coherence across complex tasks and diverse environments, and help us do better evaluation (what variables to measure).

14© Centre for Effective Services 2013

What

we do

Who we

reachWhat results

INPUTS OUTPUTS OUTCOMES

Program commitments

Activities Participation Short Medium Long-term

What we

invest

More specifically….

Evidence

Monitoring and evaluation

15

The “Implementation Gap”

2. Implementing Effective Services

16

Understanding Implementation

Implementation involves

— a specified set of purposeful activities

— at the practice, programme and systems levels

— putting into practice a programme, intervention or service with established and recognisable processes that are necessary to achieve desired outcomes.

At its simplest, implementation can be described as the carrying out of a plan for doing something. It focuses on operationalising

the plan – the How , rather than the What

Science to Service

Best EvidenceService

GapIMPLEMENTATION

Source: Implementation Masterclass, Dublin, May 2011, K. Blase and D. Fixsen

17

Bridging the Gap

What Works?

Fidelity is important.

Fidelity is the degree to which the activities undertaken in a programme are true to the design of the original programme

on which it is based.

Implementing properly matters!

Higher Fidelity is correlated with better outcomes across a wide range of programmes and practices

19

Implementation Drivers

Leadership

Staff capacity to support clients with the selected practices

Organisational capacity to support staff in implementing practices with fidelity

Capacity to provide direction, vision, & the“right” leadership approach

Improved Fidelity and Outcomes

Source: Adapted from Implementation Masterclass, Dublin, May 2011, K. Blase and D. Fixsen

20

Implementation Drivers: Improved Fidelity & Outcomes

Staff capacity to support clients with the

selected practices

Selection

Training

Coaching

21

Implementation Drivers: Organisational support

Organisational capacity to support

staff in implementing practices with fidelity

Improved Fidelity and Outcomes

Systems Intervention

Facilitative Administration

Decision Support Data System

22

Implementation Drivers: Leadership

Leadership

Capacity to provide direction, vision, & the

“right” leadership approach

• Adaptive challenges cannot be solved by existing systems.• Adaptive leadership involves finding solutions that are new and break away from normal approaches. This involves convening staff, developing plans & reaching a consensus.

• Technical challenges are those which can be solved by existing organisational systems.• Technical leadership involves managing these challenges by knowing what needs to be done and doing it.

Improved Fidelity and Outcomes

TechnicalAdaptive

Practitioners and Implementation

Need support

Need to feel their work

and contributions

are valued

Need to feel involved and

that their opinions are

heard

Need to be empowered to use their

skills and wisdom

How CES Promotes Evidence in Practice

Prevention and Early Intervention

Projects

Practice Seminars e.g. Sourcing

Useable Evidence

Strategic Advice and Support

The What Works Process

CES Library Online Resource for Youth Workers

Evidence Route Map for Youth Work

To Conclude…..

• In the Design phase, factors to consider are:

o The Evidence

o Local context and need

o Logic Models to focus outcomes and accompanying indicators

• During Implementation, key ingredients are:

o Building staff competency

o Organisational Readiness

o Leadership to match the challenge and Championing of the new approach

Effective InterventionsThe “WHAT”

Effective Implementation

The “HOW”

Positive Outcomes for

Children

Source: Implementation Masterclass, Dublin, May 2011, K. Blase and D. Fixsen

Evidence + Implementation = Better Outcomes for Children

Designing Implementing Effective Services

If you are interested in ‘evidence based/informed practice’

• www.effectiveservices.org

• www.iriss.org.uk

• www.ceni.org

• www.scie.org

• www.cssp.org

Nuala DohertyDirector, Centre for Effective Services

[email protected]

www.effectiveservices.org