Showing its worth it: Measuring the impact of volunteering @IVRtweets @VolunteeringEng Nissa Ramsay 11 th February 2012.

Showing it’s worth it: Measuring the impact of volunteering

@IVRtweets@VolunteeringEng

Nissa Ramsay11th February 2012

Introduction

How important is impact measurement to your volunteering?

• Very Important

• Not important

• I’m going to sit on the fence

Introduction• Who here thinks they could measure the impact of their

volunteering project?

• I’d be ace at this

• I could blag it

• I don’t know what you’re talking about

Today…

My vision:

To enhance the capacity of the student volunteering sector

Specifically,

To increase how you value impact measurement

To improve your understanding of impact measurement

Lesson 1… Always consider who is asking the question, to who, what

their relationship is and the context

Lesson 2… The best evaluation questions are part of the delivery

process: It should not be hard work

Lesson 3… Measuring impact is not an exact science but a balancing

act between the ideal world information you would want and the reality of working in your role

Why bother?You should do this if:

• You have a genuine desire to improve the volunteer experience

• You have to as a requirement of your funding

• You want to apply for funding

• You want to shout about how amazing your work is

• Your volunteers are always deserting you

• You can’t convince any one to volunteer for you

You should think twice about doing this if:

• You have no time/money/resources you’re willing to devote

• You are only doing this as a requirement of your funding

• You have no use/goal for the results

Lesson 4… Measuring impact is also about improving the quality of

your work and delivery (not just the end statistics)

Lesson 5… Measuring impact can help focus your project/

organisation on its core purpose (BUT can also take you off course)

Lesson 6… Measuring impact can work on a range of levels:

Project, Thematic area, organisation

Where to start1. Theory of change

Articulates how, when and why a change happens as a result of the project/intervention

= measure your impact by testing the assumptions behind this

= show if you create positive outcomes and why YOU have

= Can be on a project/programme or organisation level

Case Study: Student Hubs…www.studenthubs.org/xwiki/bin/View/main

http://www.youtube.com/watch?v=TLXTnc2YNuk

A mission statement/overall aimWhat?

• A mission statement

• Describes the overall impact of the programme

• The bigger picture – you aren’t measuring this directly

Why?• This is your impact: it sets the context as to why you are doing this

Student Hubs empowers students to create positive social change, now and in the future

Specific aimsWhat?

• Precise statements about change

• Break down mission in to components and target groups

• And/or consider your activities – why are you doing all of them and what do they amount to

Why?• This will help identify target groups and the core components of your work/ impact/beneficiaries

Student Hubs will inspire students to take social action

Outcome indicatorsWhat?

• Statements of change that we can measure, link to specific aim

• Achievable and a change measure (perceptions/feelings)

Why?• This will help decide who to involve in the evaluation, why and what questions you will ask them

Output indicatorsWhat?

• Activities and numbers/targets involved

Students feel more motivated to be socially active

500 students involved have taken on new volunteer roles

The connections

Vision

Aim 1Aim 2

Aim 3

Change = Outcomes

Activities = Outputs

If your aim is: To provide high quality volunteering opportunities• Volunteers’ views on support, management and training

• Their volunteer manager (approach / availability etc)• Perceptions of bureaucracy (e.g. CRB, interviews)• Importance of payment of out-of-pocket expenses• Recognition received• Workload and level of responsibility

•Whether they see their involvement as meaningful

• What they see as the impact of their volunteering• Whether they perceive their help as being needed• Whether they perceive any barriers limiting this

Your turn…Group discussion:

Choose either

- A project one of you are running

- Student Hubs

Identify

- Specific aims: To increase awareness/perception…

- Outcome indicators: Volunteers have increased skills

- Output indicators: 25 volunteers have been trained in…

Lesson 7… Measuring impact has to start with understanding what

you are trying to achieve and how

Methodologies and tools• Surveys

• Online (e.g. Surveymonkey)• Access to a sample group (e.g. email addresses for online) • Response rate and representativeness (using incentives)

• Focus groups• Do not have managers in the same room!

• In-depth interviews• Informal approaches (e.g. walking interviews)

• Observation• Volunteer testimonials and case studies• Life history interviewing• Mapping exercises

Involving volunteers in the process• Peer research and volunteer involvement in the process

• Accessing new skills and experiences

• Ability of volunteers to form different relationships with respondents, as

peers

• Quality of data vs. professional development of volunteers

• Project guidance and steering groups

• Volunteer / user perspective (e.g. the development of research tools)

Improving the quality of research• Gain organisational buy-in and support

• Ask only what you need to know

• Avoid leading questions

• Always pilot surveys

• Don’t cherry-pick respondents but pick randomly

• Opportunities for tracking / longitudinal research

Using toolkits• The Volunteer Impact

Assessment Toolkit• Hard copy and website

• Assesses impact on

volunteers, the organisation,

service users and wider

community

• Guidance on tools and

methods

• £29.99 plus postage

Limitations, challenges and pitfalls• Self-reported nature of satisfaction

• Unavoidable to a certain degree

• Can attempt to triangulate the results (e.g. speak to staff, clients)

• Ensuring a good response rate• Can the results speak on behalf of all volunteers if changes are to be

made?

• Working with volunteers with specific resource needs• e.g. disability, low confidence, illiteracy, English as a second language

• Cost and time implications but everyone needs to have the opportunity

to participate

Taking it forwards: options

Undertaking in-house Sub-contractedGreater understanding of specific volunteering context

Draws on expertise in volunteering from a wide variety of fields

Cheaper option (financially but not in staff time)

Requires limited in-house staff time and easy to manage

Allows independent verification and increases confidence in results

Volunteers may be more honest and at ease when not talking to staff

Communication and dissemination• Accessibility of findings

• Full report vs. summary

• Plain English and awareness of audience

• Warts and all (honesty and accuracy)

• Will increase trust in findings

• Feedback to the respondents and the volunteer body• Recommendations / changes need to be communicated as well as

reasons for not changing something

• Review of changes / developments

Conclusion• How important is impact measurement?

• Very Important

• Not important

• I’m going to sit on the fence

• Who here thinks they could measure the impact of volunteering within a project?

• I’d be ace at this

• I could blag it

• I’m not really sure

The results• At the start of my session

• 92% felt impact measurement was ‘very important’

• 0% felt it was ‘not important’

• 8% felt they were ‘going to sit on the fence’

• At the end of my session

• 96% felt impact measurement was ‘very important’

• 0% felt it was ‘not important’

• 4% felt they were ‘going to sit on the fence’

ConclusionAt the start of my session:

• 8% of you thought you’d be ace at measuring impact

• 45% of you thought you could blag measuring impact

• 28% of you ‘don’t know what I’m talking about’

At the end of my session:

• 10% of you thought you’d be ace at measuring impact

• 73% of you thought you could blag measuring impact

• 2% of you ‘don’t know what I’m talking about’

The results

As a result of me running this session today:

28% more people can blag impact measurement

[email protected] 520 8900www.ivr.org.uk

Thank you