ICAM Are We Making a Difference?

Are we making a difference?

A guide to monitoring and evaluating dog population management interventions

Dr Elly Hiby Scientific Coordinator

Background

ICAM Coalition formed in 2006 Discuss, share and learn But what is really making a difference? Indicators project

To develop guidance on monitoring and evaluation of DPM that supports us to track progress, learn and subsequently improve impact through the use of measurable indicators

Sharing our language

Impact: (aka goal or aim) the change we hope to contribute towards through our interventions

Example: an intervention that catches, neuters and returns a number of stray dogs every month in an Asian city. The desired impact of this intervention is to reduce the density of dogs and to improve the welfare of the stray dogs

Sharing our language

Indicators: (aka metrics) are measurable signs of impacts; they are the things we would see or hear if our desired impact was occurring.

Example: In our Asian city intervention, for the impact of reducing dog density a suitable indicator may be the number of dogs seen on a set of standard routes along public roads

Sharing our language

Methods of measurement: this is how data relating to the indicators was collected

Example: The method of measurement for the indicator of dog density may be a street survey observing dogs on public property, conducted once every 6 months, always following the same protocol (e.g. same routes, same time of day)

Sharing our language

Monitoring: systematic and routine data collection Intervention progress and indicators

Evaluation: uses data collected through monitoring to answer questions about what difference did this intervention make? Conducted regularly but not frequently aka Impact assessment

Why should I care?

To inform donors of the impacts created through their funding

To inform the public of any impacts on them and/or their dogs as beneficiaries of the intervention

To provide evidence for lobbying to sustain or replicate an intervention

To compare interventions and their relative impacts To learn and improve our impact

Why should I care?

Develop concept of dog population

situation and intervention

Initial assessment and analysis

Intervention planning

Implementation

Why should I care?

Develop concept of dog population situation

and intervention

Initial assessment and analysis Intervention planning

Set impacts and identify indicators

Develop monitoring and evaluation plan Implementation

Monitor progress of intervention and

indicators

Evaluation and learning

Test assumptions, refine concept and

evolve the intervention

Why should I care?

Success depends on knowing what works

Bill Gates

Centre for Global Development 2006 report When will we ever learn? Improving lives through impact evaluation (Savedoff et al., 2006)

Rules. Gold standard. Best yet. Enjoy. Be inspired. Try it. Innovate. Share.

8 Impacts

1. Improve dog welfare 2. Improve care provided to dogs 3. Reduce dog density/Stabilise turnover 4. Reduce risks to public health 5. Improve public perception 6. Improve rehoming centre performance 7. Reduce negative impacts of dogs on wildlife 8. Reduce negative impacts of dogs on

livestock

Indicators

Each impact has a list of indicators Recommended Suggested

Indicators

Each impact has a list of indicators Recommended Suggested

e.g. Body condition score recommended

Indicators

Each impact has a list of indicators Recommended Suggested

e.g. Dog-dog interactions suggested

Image from www.flickr.com, available under creative commons licence, credited to Nazly Ahmed

Image from www.flickr.com, available under creative commons licence, credited to Godo Godjo

Method of measurement

Each indicator is accompanied by at least one suggested method of measurement

Method of measurement

Each indicator is accompanied by at least one suggested method of measurement

7 Methods of Measurement

1. Questionnaire surveys 2. Participatory research methods 3. Street surveys 4. Secondary sources of information 5. Clinic records 6. Behavioural observation 7. Street surveys and questionnaires for

vaccination coverage

e.g. Street surveys

You choose

Impacts Indicators

Method of measurement

Background

Making your impact assessment robust

You choose

Impacts Indicators

Method of measurement

Accessing the guidance

On USB stick in your conference pack Download the full document at www.icam-coalition.org

Navigating the guidance

Navigating the guidance

Use our online impact assessment tool Sign up as a user Choose your impacts Series of questions creates

guidance.pdf incl indicators and MoM tailored to your intervention

Share

[email protected] Representatives of

coalition members Follow-up survey, and

more

Thanks to

Collaborating partners Boyd Orr Centre, University

of Glasgow JMICAWE, University of

Edinburgh Animal Welfare and

Behaviour group, University of Bristol

Department of Veterinary Tropical Diseases, University of Pretoria

Subject experts and DPM implementers

8 Impacts

1. Improve dog welfare 2. Improve care provided to dogs 3. Reduce dog density/Stabilise turnover 4. Reduce risks to public health 5. Improve public perception 6. Improve rehoming centre performance 7. Reduce negative impacts of dogs on wildlife 8. Reduce negative impacts of dogs on

livestock

Are we making a difference?A guide to monitoring and evaluating dog population management interventionsBackgroundSharing our languageSharing our languageSharing our languageSharing our languageWhy should I care?Why should I care?Why should I care?Why should I care?Slide Number 11Slide Number 128 ImpactsIndicatorsIndicatorse.g. Body condition score recommended Indicatorse.g. Dog-dog interactions suggested Method of measurementMethod of measurement7 Methods of Measuremente.g. Street surveysYou chooseYou chooseAccessing the guidanceNavigating the guidanceNavigating the guidanceSlide Number 28Slide Number 29Slide Number 30Slide Number 31Slide Number 32Slide Number 33Slide Number 34Slide Number 35Slide Number 36ShareThanks to8 Impacts