Session 8: Evaluating Partnerships

Evaluating Partnerships

BE0964 Partnership & Collaborative Working

Assignment Part 1

• Carry out a critical analysis of the Coventry City Council case study by evaluating the key strategies adopted by the partners / collaborators and by analysing the development of the alliance over time by applying relevant themes, theories and concepts studied in the module. Draw conclusions from your analysis and develop an evaluation of the success or failure of the alliance by applying an evaluation model of your choice

Introduction

• Why evaluate partnerships?

• What is a ‘successful partnership’ and how can it be measured?

• Introduce some partnership evaluation approaches

Benefits of Evaluation

• Evaluation is often a neglected part of partnership working.

• Evaluation is an important part of partnership and inter-agency working

• – what is happening? • – what is going right, what

is going wrong and why!

Benefits of Evaluation

• Ideally, evaluation needs to be considered at the beginning of a partnership

• In the past this has not been the case and evaluation has often been an after-thought

An effective evaluation strategy:

• Why you want an evaluation?

• Who wants the evaluation to be done?

• When the evaluation will be done?

An effective evaluation strategy:

• What the evaluation will look at - what questions will be asked; what information/data will be collected and what criteria and indicators will be used?

• How the information will be collected?

An effective evaluation strategy:

• Who will be responsible for the evaluation?

• Who will do the evaluation?

• How much the evaluation will cost?

• How the results of the evaluation will be used?

Why it is important to evaluate?

• To learn about how to work in partnership

• To measure progress against objectives

• Improving management processes and procedures

Why it is important to evaluate?

• Seeing what has been achieved - both intentional and unintentional outcomes

• Improving effectiveness of partnerships

Why it is important to evaluate?

• Sharing experiences (often with other partnerships) - identify good practice, in areas such as promoting community involvement for example

• To work out cost effectiveness and value for money

Why it is important to evaluate?

• To identify strengths and weaknesses

• To report to partners, local people, regional and national governments

Why it is important to evaluate?

Formative ‘This is a type of ‘developmental’ evaluation that feeds information and guidance back into an intervention so that improvements can be made’. Process ‘The aim of this type of evaluation is to find out exactly how an intervention works’. Outcome ‘This type of evaluation focuses on the final results of the intervention. A typical question for an outcome evaluation to address would be whether the intervention has made significant improvements in a client’s life’

Effectiveness of Partnerships

Important at 3 levels

Problems with Evaluation

• It is a qualitative problem for which you need quantitative measures

• How do you determine that it is ‘functioning’?

• How do you measure how well it is ‘functioning’?

• How do you assess ‘added value’?

Problems with Evaluation

Need to assess; – Opportunity costs – Impact of partnership activities on the individual bodies – Areas of synergy– What are the ‘extra’ areas of achievement

Evaluation can focus on:

• Quantity - outputs such as the rate that crime figures have gone up or down

• Quality - outcomes such as reducing the fear of crime and improving the quality of life for residents

• Processes - management systems such as project appraisal and development systems or partnership working

Partnerships also need to decide the focus of the evaluation

• ‘bottom-up’ – evaluation can highlight to service users, residents or the public how the partnership has impacted upon them, or

Partnerships also need to decide the focus of the evaluation

• ‘top-down’ – evaluation can convince funders/sponsors (such as local or central government) that targets are being met

What is a ‘successful partnership’ and how can it be measured ?

6 features of successful partnerships: 1. Needs to be recognition

and acceptance of the need for a partnership approach

2. development clarity and realism of purpose

3. ensure there is commitment and ownership

“Partnership , New Labour and the Governance of Welfare” Chapter 4, Hudson and Hardy

What is a ‘successful partnership’ and how can it be measured ?

4. develop and maintain trust5. create clear and robust

partnership working arrangements

6. the ability to monitor, measure and learn

Evaluation Models

Life Cycle Model

One approach to establishing the role of evaluation for partnerships is to focus on the concept of a partnership ‘life cycle’.

http://www.simoons.com/2011/09/how-do-you-create-your-partnerships/

Life Cycle Model

It is helpful to think of partnerships going through a series of stages, during which particular tactics are most appropriate to ensure partnership progress and succeed.

The Partnership Life Cycle

http://www.transformationpartners.co.uk/J15/images/stories/lifecycle.png

5 vital stages approach

1. Forming:

2. Frustration

3. Functioning

4. Flying

5. Failing1. Consensus Building, set vision & objectives

2. Problems, re-focus or go to step 5

3. Up & running

4. Real achievement

5. All partnerships have a shelf life

5 vital stages approach

Partnerships need to work through the stages of the life cycle in order to function with greatest effectiveness (or "fly".)

Even in the best partnerships, there is a tendency for partnerships to falter and perhaps fail

Unless the partners consciously manage their progress through the critical stages of the life cycle.

5 vital stages approach

People often don’t know when a partnership has outgrown it’s usefulness

This can lead to Partnership fatigue & Partnership proliferation

A note of caution.....

Partnerships are context specific and come in all shapes and sizes

Therefore, a model of a successful partnership can only be a guide – no ‘one solution fits all’

Introduction to differing approaches to evaluating partnerships

Web resources

Local Partnerships

Partnershipbrokers

The Scottish Government

OxfamInternational Institute for Sustainable Development

In a good evaluation, we might expect to see …

• Measurement of progress against funders criteria (performance targets)

• Measurement of progress against partnerships own targets

In a good evaluation, we might expect to see …

• Value for Money – efficiency and effectiveness

• Discussion of the health of the partnership – how

• well partners work together

• Process indicators – involvement, accountability and transparency

Evaluation Tools

Generic Tools

Generic Tools

Balanced Scorecard (Kaplan & Norton) - 4 Processes

Balanced Scorecard

Balanced Scorecard

Example of the Balanced Scorecard in UK Public Policy

• The Neighborhood Renewal Unit (NRU) asks Local Strategic Partnerships (LSPs) to self assess (score) performance against core criteria including:

• – Strategy– Action-Focus – Efficiency

• -Inclusiveness -Performance Management - Learning and Development

• Scorecard translated into traffic lights to show levels of delivery and achievement

•

LSP Traffic light assessments

• LSPs partnership receive a grading: • • A Green assessment = successful delivery

• Amber/Green = mostly successful delivery and • strategies in place to improve delivery • • Amber/Red = little evidence of successful

delivery & little progress on improving delivery • • Red = no evidence of successful delivery &

significant issues around partnership working • Traffic Light assessment

Smarter (Ideal) Partnerships Toolkit

The Partnership Wheel

Evaluation tools

• There are many approaches….

– Generic– Bespoke– Top Down– Bottom Up

Research Partnering Evaluation Tools for yourself…

Summary

• Why evaluate partnerships?

• What is a ‘successful partnership’ and how can it be measured?

• Introduce some partnership evaluation approaches

Further Reading

Partnering and the Balanced Scorecard (n.d) http://hbswk.hbs.edu/item/3231.html

Partnership Effectiveness - Process & Methodology - http://whitehorsedc.moderngov.co.uk/mgConvert2PDF.aspx?ID=4620

JISC (2014) PESTLE and SWOT analyses. Available at: http://www.jiscinfonet.ac.uk/tools/pestle-swot/

Balloch, S. & Taylor, M. (eds.) (2001) Partnership Working: Policy and Practice, The Policy Press.