Campaigning for Change - Learning From the United States

National Council for Voluntary Organisations

Campaigning Effectiveness

Campaigning for change:

Learning from the United States

Campaigning Effectiveness, NCVOFebruary 2011Brian Lamb

What is campaigning and influencing?You might call it influencing, voice, advocacy or campaigning, but all these activities areabout creating change. At NCVO we use the word campaigning and define this as themobilising of forces by organisations or individuals to influence others in order to effectan identified and desired social, economic, environmental or political change.

Whatever you call it and whether you are trying to save a local community centre fromclosing or lobbying government, campaigning is about creating a change. The impact isthe real change created by a campaign – the difference it makes to people’s lives.

What is covered?One of the most fundamental questions campaigners often ask themselves is whether theircampaign is making a difference. Being able to answer this question is crucial for campaignsto develop effectively, and for campaigners to communicate the impact of their campaignsto a wide range of audiences.

While many existing evaluation models focus on the sequencing of activities, this guideoutlines a novel, systematic approach to evaluating campaigning which has emerged inthe United States. The guide goes through a step by step process, firstly in outlining theapproach in different stages with examples, and secondly in drawing out the key implicationsfor UK campaigners.

This approach, known as Theory of Change, challenges campaigners to develop clear aimsand strategies, to question how change occurs, the relationship between campaign stagesand activities, as well as to make explicit the assumptions often implicit within campaigning.

2



Overview

Campaigning for change: Learning from the United States

Contents

3 Introduction

5 Theory of Change Model

22 Implications for campaigners in the UK

23 Issues to consider

24 Conclusion and summary

26 Annotated guide and bibliography

29 Appendix 1: Evaluation chart

31 Appendix 2: Pro forma Theory of Change mapping tools

IntroductionCampaigning does not easily lend itself to standardmanagement approaches to strategy, planning andevaluation. Many of the models for organisationaldevelopment represent a poor fit in reflectingthe constantly changing environment in whichcampaigners work, the judgements they makeabout the activities which bring about change andhow to assess when they have achieved their goals.Planning, evaluation and impact measurementoften become an afterthought rather than anintegral part of the campaigning process. Yetsuccessful campaigning rests on developingappropriate strategies for change as much ason the successful delivery of a campaign.

A number of Charitable and PhilanthropicFoundations in the US have found a solution tothis problem through developing an overarchingapproach to setting the strategy, planning andevaluation of campaigning. What has driven theircollaboration has been the determination todevelop a better way of understanding theassumptions campaigners make about whatdelivers change and more testable hypothesesabout how change will occur. Related to this theFoundations have wanted a framework in whichthey would then be able to assess progress towardschange and support stakeholders and beneficiariesin defining what that success should look like.

They have achieved this aim by taking the coreinsights from basic business models and combinedit with a level of academic rigour about how changeoccurs, and a deeper understanding of the methodsand approaches deployed by campaigners to

bring about change. This approach has majorimplications for how we should think about whatconstitutes best practice in the development ofcampaigns strategy and evaluation, and providesa powerful model that campaigners could benefitfrom in ensuring their campaigns are successful.

As campaigning becomes increasingly scrutinisedin regard to its effectiveness, having the abilityto develop strategies which are based on anunderstanding of how change occurs, which setclear goals to bring about change, and whichmonitor progress towards those goals, will becomefundamental to good and effective campaigning.

Philanthropy and Advocacyin the US1

In the United States, charitable foundationsplay a much larger role as funders of projectsand advocacy work than their counterparts inthe UK. This is due to considerable differences inthe funding of community groups in the US andthe comparatively smaller role of state funding.Foundations based in the US have thereforetaken the initiative in producing models to supportassessing the quality of bids they receive andthe planning and evaluation of activities theyare funding, from campaigning and advocacyorganisations. Foundations have also built selfevaluation, using what they have termed a‘Theory of Change’ (TOC) or ‘Composite LogicModel’2 into the organisations they are funding.

The Aspen Institute, Atlantic Philanthropies,Annie E. Casey Foundation, The CaliforniaEndowment, the Kellogg Foundation, and theJames Irvine Foundation are just some of the

3



Introduction


1. Advocacy is the commonly used term forcampaigning activities in the US: “We defineadvocacy as “a wide range of activities conducted toinfluence decision makers at various levels.” Thismeans not only traditional advocacy work likelitigation, lobbying, and public education, but alsocapacity building, network formation, relationshipbuilding, communication, and leadershipdevelopment.” Centre for Evaluation Innovationand Innovation Network, Inc. (2010) AdvocacyEvaluation Update, #9, May, p.1

2. The literature can be confusing as theseapproaches are often given slightly differentnames to describe the same underlying model,though they emphasise different aspectswithin those models. The most commonusages are the Composite Logic Model or Theoryof Change. For the purposes of this reportI have used Theory of Change throughoutas a means of referring to them collectively.However, it is important to note that theoriginal use of the term primarily referred

to challenging the assumptions behind someof the developments in community activismin the mid 1990s, and only became morejoined to the Logic Model approach later.

leading foundations that have been behind thedrive to develop a shared and consistent approachto funding advocacy work and evaluating itsimpact, often through working in concert. All havean impressive track record in funding advocacyprojects. They represent some of the largestfunders of health and social welfare programmesin the US. Typical of that commitment is thePresident of Atlantic Philanthropies GaraLaMarche, who argues, “funding advocacyand advocates is the most direct route to supportingenduring social change for the poor, the disenfranchisedand the most vulnerable among us, including theyoungest and oldest in our communities.”3

A number of partners from both academia andthe commercial sector, such as the Harvard FamilyCentre, Innovation Network, OrganisationalResearch Services (ORS), the TCC Group (TCC),and Blueprint Research and Design havecollaborated with grantees to build up animpressive body of thinking and practical tools thatconstitutes a consistent approach to campaigningand advocacy methodology. There is suchconsistency and working between organisationsaround this approach that it could be characterisedas a community of practice – in which professionalshave come together to increase the store ofknowledge around this issue.4

Some of the leading exponents of the Theoryof Change or Composite Logic Models are JuliaCoffman (Harvard Family Research Project),Astrid Hendricks and Barbara Masters (TheCalifornia Endowment), Jackie Kaye (The AtlanticPhilanthropies) and Tom Kelly (Annie E. Casey

Foundation), while more than 50 funders,evaluators and advocates also helped developand refine the model. Another leading proponenthas been Sarah Stachowiak and associates atOrganizational Research Services (ORS), a changeconsultancy for the Annie E. Casey Foundation.5

The work of the Foundations represents a powerfulcollaboration from those who have spent the last10 years developing a new approach. Given itsgrowing status, and the fact that it is becomingthe de facto way to conceive, deliver and conductcampaign strategy and planning, it is worth lookingin more detail at how the model developed, whatit asks of campaigners and what its strengths andweaknesses are.

With some notable exceptions6 the same focushas been missing on this side of the Atlantic.While many organisations will follow elementsof a standard business model of planning, theapplication of a developed TOC approach hasbeen missing. UK campaigners have much to gainfrom looking at the TOC model developed bythe Foundations as it provides a unified andcomprehensive means of approaching campaigningwith a clear focus on campaign strategy. TOCachieves this through prioritising setting clear goals,assessing outcomes and emphasising evaluation.This comprehensive approach to the developmentof campaign strategy encourages campaigners tothink harder about what delivers change and themeans they deploy to do so.

4



Introduction


3 The Atlantic Philanthropies (2008) ‘Investingin Change’, Atlantic Reports, May

4 Etienne Wenger defines Communities ofPractice as: “… groups of people who share aconcern, a set of problems, or a passion abouta topic, and who deepen their knowledge andexpertise by interacting on an ongoing basis.”Wenger, E., et al. (Harvard Business School,2002) Cultivating Communities Practice:A guide to managing knowledge (Cambridge,MA: Harvard University). McKinsey & Co.describe a community of practice as “A groupof professionals, informally bound to one

another through exposure to a commonclass of problems, common pursuit ofsolutions, and thereby themselves embodyinga store of knowledge.” McKinsey & Co., citedin Interoperability Clearinghouse (ICH)Interoperability Clearinghouse Glossary of Terms:http://www.ichnet.org/glossary.htm.

5 For how this has taken place, see Coffman,J. (Harvard Family Research Project, 2009)A users guide to Advocacy Planning and Evaluation(Harvard, Harvard Family Research Project).

6 See Coe, J., and Mayne, R. (NCVO, 2008)Is your campaign making a difference? (London,NCVO), which used some elements of TOCand logic models, Lofgren, G. et al. (NewPhilanthropy Capital, 2008) Critical Masses:Social campaigning, a guide for donors and funders(London, New Philanthropy Capital),which firmly identified the TOC modelas key to good evaluation. The OverseasDevelopment Institute (ODI) has madesignificant contributions in the areas of policyand planning but not to the evaluation ofcampaigning nor to the TOC model.

Theory of Change – thedevelopment of a new approachto campaign planning?TOC is an extension of a basic planning toolusually referred to as the Logic Model7. It is worthexamining the basics of the Logic Model as itformed the building blocks for the developmentof the TOC approach.

The Logic ModelThe Logic Model is simply the first stage intrying to capture the interrelationshipsbetween different parts of your activities andimpacts.

A logic model looks at the process from identifyingresources through to activities, outputs, outcomesand impact as a logical chain of events to form aproject plan. Often they are more useful fordescribing the progress of an existing plan andmonitoring service delivery than for capturing thecomplexity of campaigning or advocacy work, andtherefore will be familiar to many who haveundertaken project planning more generally.

Your work plan – what resources you think youneed to implement a work program and what youintend to do.

1 Resources include the human, financial,organisational and community resources aprogram of activity has available to directtowards doing the work. Sometimes thiscomponent is referred to as Inputs

2 Activities are what the program does with theresources. Activities are the processes, tools,events, and actions that are an intentional

part of the program implementation. Theseinterventions are used to bring about theintended program changes or results

Your Intended Results include all of thecampaigns desired results (outputs, outcomesand impact).

3 Outputs are the direct products of programactivities and may include types, levels andtargets of services to be delivered by theprogram

4 Outcomes are the short-term or long-termresults of any given activity. Short-termoutcomes should be attainable within 1–3years, while long-term outcomes should beachievable within a 4–6 year timeframe

5 Impact is the fundamental intended orunintended change resulting from theoutcome that occurs in organisations,communities or systems as a result ofprogram activities withina particular timescale.8

This process is essentially the building block for anytype of strategic planning. The Logic Model restson the idea that you should make clear what thelogical links are between your activities, outcomesand impacts through specifying resources,activities, short and long-term outcomes.

A number of approaches have been developedfrom this basic model, depending on theFoundation – the specific format relative to theirwork. However it is important not to get caught upin terminology but to focus on the underlyingconcepts which form the basis of the Theory ofChange model.

5



Theory of Change Model


7 Both the Aspen Institute and the KelloggFoundation have done extensive work on theLogic Model. For more information on theirapproaches seehttp://www.theoryofchange.org orhttp://www.wkkf.org. Kellogg rests more

heavily on the Logic Model and you need tolook at more recent work referenced here tosee how this has developed, not least in theirmore recent thinking where it is enriched witha comprehensive Theory of Change approach.

8 For a fuller outline see W.K. KelloggFoundation (2004) Logic Model DevelopmentGuide (Michigan, W.K. Kellogg Foundation)

Resources/inputsResources/inputs

Activities Outputs Outcomes Impact

Theory of ChangeModel“A theory of change lays out what specific changes thegroup wants to see in the world, and how and why agroup expects its actions to lead to those changes.”9

The development of the Theory of Change modelrests on the basic approach of a Logic Model, as itlooks at the sequencing of activities. However TOCgoes further: it requires more specificity about theaim and the conditions needed to reach that aim.There is a greater focus on what types of activityare undertaken and the interrelationship betweenthe activities.

TOC also encourages organisations to develop amore sophisticated understanding of what isrequired for change to take place and whatstrategies can be used along the way. It challengescampaigners to think about what the links might bebetween the activities they undertake and the endgoals they seek by using insights from some of thebest thinking in political and social theory abouthow change happens. It therefore results in a moreuseful guide for steering a campaign than otherplanning processes. By introducing more focus onthe rationale that campaigners use, the TOC modelhas the capacity to illuminate strategic choices andassess how these are followed through.

The TOC model challenges campaigners to thinkhard about the assumptions they make whenselecting the campaign activities which they hopewill lead to change.

How does the Theory of Changemodel work?The main elements of the TOCmodel are asfollows:

1 Stating a clear aim

2 Mapping activities to achieve yourcampaign aim

3 Outcomes and how to get there – using‘so that’ chains

4 Understanding how social change happens –outcome mapping

5 Capacity of the organisation to achievechange

6 Evaluation built into the model

Contrary to Marx’s famous critique thatphilosophers have only sought to interpret theworld not to change it – campaigners have toooften sought to change the world without firstinterpreting it.

The basis of the TOC model is to state a final aimor impact and then describe what would need tohappen to arrive at that point. The advantage ofusing TOC is that it immediately points to theinterrelationship between activities and outcomes.TOC involves having both a theory, and theories,about different ways in which change can bebrought about and what methods andinterventions will work best. TOC looks across thewhole campaigning cycle and provides a frameworkfor evaluating progress, which is built into theplanning assumptions and methods. In doing so, itprovides the basis for a strong focus in the plan onhow social change happens.

“A theory of change clearly expresses therelationships between actions and hoped for results,and could also be described as a roadmap of thestrategies and belief systems (e.g., assumptions, ‘bestpractices’, experiences) that make positive change inthe lives of individuals and the community. A theoryof change can be articulated as a visual diagram thatdepicts relationships between initiatives, strategiesand intended outcomes and goals.”10

6





9 Guthrie, K., et al (Blueprint Research &Design, Inc, 2006) The Challenge of AssessingPolicy and Advocacy Activities: Strategies for aProspective Evaluation Approach (Los Angeles,The California Endowment), my emphasis.Prepared for The California Endowment.

10 ORS (2009) Ten Considerations for AdvocacyEvaluation Planning: Lessons learned from KidsCount grantee experiences. Prepared for Annie ECasey Foundation (Seattle, ORS).

TOC models suggest that you start with thedesired end result and work backwards throughwhat would be needed at each stage of the processto achieve the intended result. This is often donethrough mapping each stage of the process onto aframework, listing the barriers, assumptions andsteps that need to be taken to achieve your goal.They have found that it is helpful, whereverpossible, to complete this process in teams, withstakeholders and relevant beneficiaries to create ashared view of the goals or change the organisationis seeking. Also by having a number of inputs to theprocess there is a better chance of ensuring youhave captured all the thinking necessary tocomplete the plan successfully.

Therefore most Theory of Change modelsfollow a similar set of processes, which includethe elements described below.

1 Stating a clear aimThe first stage of the process is to start with theultimate campaign aim – the overall purpose ofyour campaign, the change you wish to see andthe impact you want to make. The aim should bea visionary statement that encapsulates theultimate purpose of the campaign. It should becompelling, inspiring and targeted – identifyingwhat needs to change and articulating what thechange will look like.

Examples of ultimate impacts are:

• Children have equal opportunities tosucceed in school

• Child poverty will be abolished by 2020

• Women are free from violence in the home

You can list the ultimate impact at the bottom of apage and work backwards from this ultimate goal.These stages are often best undertaken in teams,with stakeholders or beneficiaries when starting astrategy or when the method is being used toevaluate the consistency of an existing plan.

2Mapping activities to achieveyour campaign aimThis stage examines the specific activities thatwould be needed to bring about the campaigngoal or impact you want to make. These activitiesmay include campaigns to change policy orpractice, bring about changes in law, behaviour,public opinion or awareness, capacity-buildingefforts, community activity and so on.

Here are some examples:

• Development of alliances

• Public awareness campaigning

• Creating the political will for change

• Legal advocacy

• Building community capacity to campaign

These strategies need to have a causal or logicalrelationship to the end goal that you are tryingto achieve and you need to be aware of theappropriate type of strategy to achieve yourparticular goal. This assumes an overallunderstanding of what types of actions lead todifferent types of changes – ‘see section 4Understanding how social change happens.’

3 Outcomes and how to get there –using ‘so that’ chainsIt is crucial to be clear about outcomes as opposedto the final impact you are trying to achieve, andwhat the interrelationship is between thosedifferent outcomes in achieving the goal of yourcampaign aim. Therefore, it helps to check thelinkages between your different activities.

This takes the first activity and associated strategylisted and creates a ‘so that’ chain based on thefollowing question:

‘What X or Y activity should we do to result in(blank) for individuals, families, organisations orcommunities?’. The answer for the x or y should bethe direct outcome or result of the strategy. Youthen repeat this question until you have linked eachstrategy to your goal. A ‘So That’ analysis is where

7





the process checks out the validity of a particularset of assumptions, by drawing all the logical linksbetween a number of stages in a campaign. Theanalysis starts with the impact desired and worksbackwards to the outcomes and outputs thatwould be necessary to get to that point.

These have then been built into a more fluidprocess to take account of different types of policywork or streams of activity. This will produce atemplate chart that should look like this.

A typical example of a worked up ‘So That’ chainfrom a project on improving school performance:

Sample ‘So That’ Strategy:

Formal establishment of a localcollaboration committed to children’sschool readiness and early learning

So That

A shared collective plan isdeveloped to address young children’shealth and school readiness needs

[Influence]

So That

A pilot program is implementedto provide families with access

to dental and health screening clinicson-site at two neighbourhood schools

[Influence]

and

Support programs for parents ofyoung children are offered on-site ata school in English and Spanish

[Influence]

So That

1 Children get their health needs addressed[Individual Impact]

and

2 Children have improved nutrition[Individual Impact]

and

Parents are more aware of how to supporttheir young child’s brain development

[Individual Impact]

So That

Children enter school healthy[Population Impact]

So That

Children are more likely to do well in school[Population Impact]11

‘So that’ chains can be very simple or complexdepending on the issue, size of the project or levelof analysis. If dealing with multifaceted issues theremay be a number of chains looking at differentaspects of achieving one goal or a number of goalsmay be interrelated. This approach can also beapplied to much smaller, micro processes at thecommunity level, but the crucial point is always toalign the different activities undertaken with theexpected effects or outcomes in working towardsthe desired result.

A key part of this process is also to be aware ofthe factors that might help or hinder the changesought, and what strategies are deployed toaddress these factors. This process is often calledreverse planning or backwards mapping as the ideais start from the goal to be achieved and workbackwards to the conditions needed to achieve it.

The outcomes must be clearly stated. Differentapproaches have characterised the outcomesslightly differently. ORS12 divide these into impact,which is the final goal, outcomes, the means to helpachieve those goals and leverage outcomes, whichhelp you get the capacity to achieve change, suchas changes in political will or public support.In a similar fashion Coffman, one of the leading

8





11 Adapted from ORS (2004) Theory ofChange: A Practical Tool For Action, Results andLearning (Seattle, ORS). Prepared for the AnnieE. Casey Foundation

12 Ibid

proponents of this approach, divides outcomes upbetween advocacy capacity and policy outcomes,leading to the achievement of policy goals and thenfinal impact. The key point is to be clear what therelationship is between the particular outcomesand the final impact.

9





The most complete version of the model to date isreproduced below (in an adapted form), and wasdeveloped by Coffman and colleagues.13

Thinking back to the original typology of the TOCmodel it is possible to see the genesis from thesimple Logic Model to the much more

Improved servicesand systems

Positive social andphysical conditions

Impacts



Impacts



Impacts

Policy development Placement on thepolicy agenda

Policy adoption Policy blocking Policyimplementation

Policy monitoringand evaluation

Policy maintenance

Policy goals

Electronic outreach/Social media

Coalition and network building

Briefings/presentations

Organisationalcapacity

New advocates(including unlikelyor nontraditional)

New donors

Earned media Grassroots organising and mobilization

Public serviceannouncements

Partnerships oralliances

New champions(including policy-makers)

More or diversifiedfunding

Paid media Rallies and marches Polling Collaboration andalignment(including messaging)

Organisationalvisibility or recognition

Media partnerships Voter education Demonstrationprojects or pilots

ImpactsActivities/tactics Interim outcomes

Communications and outreach Advocacy capacity

Issue/policyanalysis andresearch

Policymaker and candidateeducation

Litigation or legaladvocacy

Awareness Public will Media coverage

Salience Political will Issue reframingPolicy proposaldevelopment

Relationshipbuilding with decision makers

Lobbying

Attitudes or beliefs Constituencyor support basegrowth

Politics and policy Policy

Advocacy and policy change Composite Logic Model

13 Adapted from Coffman, J. A user’s guide to Advocacy Planning and Evaluation

sophisticated way this model has been populated,with the different strategies and activities thatcampaigners might undertake – both in terms ofcampaign actions and policy influencing.

In working through how the model is constructed,it is suggested that there are some key questionswhich can help those using the model to informthe process:

1 To what degree is there clarity and consensusamong key stakeholders regarding beliefs andassumptions, audiences, models of change,strategies and key outcome areas?

2 To what degree is the emerging picture ofchange compatible with the organisation’sbeliefs, approaches and overall culture? Forexample, beliefs about how change happens,timeframe for that change, implied roles andrelationships between the different groups tobring that about.

3 To what degree does the Theory of Changeyou are developing have implications for thecapacity of the organisation to carry out theirplan?

The above model is one of the most comprehensiveattempts in TOC modelling to capture the activitiesand tactics, interim outcomes, policy goals andimpact relating to policy change that would be thebuilding blocks of a campaign. The main aim ofsetting out the process in this way is so campaignerscan trace a path through the specific activities andoutcomes, in order to select which activities arebest for the particular outcome being sought.

Further, Coffman argues that “Because the modelidentifies a full range of possible advocacy activitiesand outcomes, it can be used to identify whatcollaborators or opponents are doing and how theycomplement or compete with the strategy. Also, thecomprehensive layout facilitates contingencyplanning; alternative paths to the policy goal can beidentified if the current strategy is not successful.”14

Some organisations have taken this further bydeveloping contingency logic models, which drawon the concept of scenario planning. These modelsthen speculate that an important element of thesocial or political context has changed and use theTOC to identify how the strategy should change toaccount for this.

TOC also provides the opportunity to outline thekey elements of the campaign path to test the keyassumptions that have been deployed in thecampaign logic and the interventions undertaken.A communications campaign from the US on guncontrol was charted out in this example (on thefollowing page), from Coffman’s evaluation of anumber of different communication campaigns,which show the different stages of the campaignin TOC format15. This public communicationcampaign focuses on engaging the media as themain strategy and traces through the key activitiesundertaken to achieve change. These activitiesinclude public service announcements anddeveloping website information, all leading togreater public awareness and behaviour change.The campaign achieved a reduction in injuries dueto firearms. The example is set out as below so thatit is possible to identify the relationships betweenthe different activities.

By following this model, campaigners are promptedto think systematically about the underlyingassumptions to their strategies, and to deploy themto build more secure plans and interventions thatfocus on the outcomes they wish to achieve.

Building up a TOC chart places emphasis onthe conditions for success – the interim changesand outcomes. TOC models suggest that it iscrucial to focus on the links within campaigns asmuch as on the final policy or other changeoutcomes you might be seeking. Both are essentialto campaigning, but policy change has often beenoveremphasised at the expense of some of the

10





14 Coffman, J., A user’s guide to AdvocacyPlanning and Evaluation, p7.

15 Adapted from Coffman, J. (HFRP, 2002)Public Communication Campaign Evaluation: An

Environmental Scan of Challenges, Criticisms,Practice, and Opportunities. Prepared for theCommunications Consortium Media Centre.(Harvard, Harvard Family Research Project)

building blocks needed to achieve change, in theplanning and evaluation of advocacy efforts.Changes in public will, political will, base ofsupport, capacity of advocacy organisations andstrengthened alliances are the crucial structuralchanges that must happen on the way to legislative,policy or other changes.

4 Understanding how socialchange happensCentral to TOC models is an overall understanding

of what strategies bring about what types of socialchange. Understanding how social changehappens, then ensuring that the types of activitiesundertaken match the overall strategy beingpursued is fundamental to the approach. Simplyhaving a set of activities that are linked is notenough if the overall strategic assumptions aboutwhat will deliver change are mistaken. ORS defineda number of different reasons for why changehappens drawn from a number of academic studiesand approaches, which they characterise as:

11





TheoryMajorChange orLarge Leaps

Background

The theory was developed by looking at bigchanges in Government approaches and policy,or how industry undergoes major change.

Argues that major social change comes aboutwhen significant coalitions come together.

Politics: political factors are crucial, including the‘national mood’ (e.g., appetite for ‘biggovernment’), particular interest groups andadvocacy campaigns, and/or changes in electedofficials.

How change happens

Normally through a complete paradigm shift inthinking, often accompanied by a change ofgovernment or a decisive event in an industry orsector. But there can also be a decisive policy shift,for example a complete reframing of the welfarebenefits system.

Core policy beliefs are unlikely to change unless:

• Major external events such as changes insocioeconomic conditions or public opinion areskilfully exploited by proponents of change

• New learning about a policy surfaces acrosscoalitions, which changes views about it.

Policy can be changed through a window ofopportunity when advocates successfully connectwith a number of areas of the policy process at onetime, by influencing the way in which their issue isperceived, or by finding the perfect solution andpromoting this.

Global theories

‘Coalition’Theory orAdvocacyCoalitionFramework

PolicyWindows

Activities Short-term andintermediate outcomes

Ulitmate(Behavourial) outcome

Impact

Public serviceannouncements– Television– Radio– Print– Interactice– Out-of-home

Exposure to thepublic serviceannouncement– Donated media time– Ad recall

Awareness aboutthe potentialhazards of gunsin the home



Attitudes towardssafe gun storage

Target audiencevisits website formore information

Target audiencestores guns moresafely

Reduction in thenumber of injuriesand deaths fromfirearms

Activities ofcampaigncollaborators

Analysing change in this way is very helpful inaligning an overall campaigning approach with theactivities that will bring about the type of changebeing sought. Ensuring synergy between thecampaign goal and the type of strategy beingdeployed can be crucial in securing success –deploying the wrong tactics can equally ensurefailure.

Issue framing as a strategyfor changeStachowiak has illustrated for each theory (detailedin the chart above), the ways in which it combineswith a particular set of activities to deliver the

outcomes anticipated by the theory, and theneventually a particular impact. This approachalso helps to identify the gaps in the logic betweenparticular causes and effects in the campaign.See following diagram for an example of the way in whichusing Issue Framing to redefine an issue would work fora communications campaign.

Putting together a Theoryof Change outcomemapBy putting all of these different activities togetherit is possible to arrive at a fully formed TOC modelfor your campaign, which can be mapped outrelatively simply with the arrows in this next diagram

12





Strategies and tactics(normally part of a broader goal in support of one of the change strategies outlined above)

Focus on whoholds powerand how it isexercised

Community-based orgrassrootscampaigning

TheoryMessaging andFrameworksTheory

Based on the assumption that there are a numberof faces of power, formal, informal and hidden.

This approach argues that power really resides withcommunities or should do. The aim is therefore tohelp communities to mobilise and have a voice. Bydoing so they can claim power back from othermore formal routes.

This approach argues that power really resides withcommunities or should do. The aim is therefore tohelp communities to mobilise and have a voice. Bydoing so they can claim power back from othermore formal routes.

When enough people coalesce locally around anissue, it is possible for change to happen. Thisnormally means mass activism at the communitylevel to bring pressure onto the target as much bythe weight of community opinion as by other means.

Change happens through a process of targetingthose who are in power and reclaiming space inwhich to conduct public dialogue – making visiblethe ways in which hidden power operates. Thisthen leads to changes in legislation or policy andpractice, which need to be sustained.

Background

Based on the theory that how issues are presentedand represented, especially in the media, will setthe parameters for which issues get taken up andwhich are kept off the agenda.

How change happens

People are presented with information in differentways that ‘frame’ a decision or issue, in a way thatgets their attention and support, where it mightnot have done before.

Change happens when you can either disrupt adominant frame of reference for your own ends,or create one that forces key players to act wherethey would not have done before.

16

16 Adapted from Stachowiak, S. (ORS, 2007)Pathways to Change. 6 Theories about HowChange Happens (Seattle, ORS)

13





Mobilize new actors• Public• Legislators• New allies/unexpected allies

Redefine issue/Issue framing

Get media attention to focus on new definitionor aspect of policy

Strengthened alliances

Increased number of allies / partners

Strengthened base of support

Increased visibility of issue

Shift in social norms

Increased awareness of issue

STRATEGIES

OUTCOMES


Increased media attention


• Increased agreement about issue definitionand need for change• Increased salience of and prioritization of issue


Increased political and public will for issue

Improved policies

• “Significant” changes in instituitions• “Significant” changes in policy

Impact

Changes in soicial and/or physical conditions

17 From Adapted from Stachowiak (ORS),Pathways for Change, p.13

17

14





Development of Narrative Model• Developing Smart Chart

Website communications• Fostering Success website• Oregon First YouthConnection sub site• Publish digital stories

Media outreach• Press releases• Editorial board meetings• Blog posts• Editorials, Op Eds, lettersto the editor

Sharing consumer stories• Wesbite• Event• Testimony/legislativebriefings• Incorporate into mediaoutreach

Publications• Report• Data book• Policy briefs

Implementation andcoordination of narrativemessages with partners• Training/presentation• Material distribution• Technical assistance



Increased sense ofurgency amonglawmakers andsystemadministrators

Increased beliefamong public andlawmakers that theissue is curablethrough publicsolutions

Increased perceptionamong lawmakersthat there is publicwill to invest inimprovement inCPS/child welfaresystem

Increased senseof communityresponsibility

Increased awarenessand clarity about keymessages amongthe public

Strengthened alliances

Increased consistency ofkey messages/amongkey partners in statewidecoalition

• Increased echo messaging among media, partners and lawmakers• Increased ownership of the message among lawmakers


• Changes in policy/administrative rules• Increased investments in the child welfare system

Policy change

Oregon has a world class child abuse/neglectprevention and intervention system• Reduced child maltreatment• Improved outcomes for foster youth• Increased number of youth are safe intheir home

All Oregon childrenare safe

All Oregon childrenthrive

So that So that

2010

AC

TIV

ITIE

SIN

TER

MED

IAT

EO

UT

CO

MES

LON

G-T

ERM

OU

TC

OM

ESIM

PAC

T/G

OA

LS

Children First for Oregon: Outcomemap of strategic communicationsstrategy for “Fostering Success”Draft Theory of Change

– representing the ‘So That’ links between thedifferent intervention strategies and the actionstaken to achieve them. On the previous page is anexample from ORS which was part of their workwith Children First in Oregon.18 The black barsrepresent the overall theory of change strategies,the activities are then grouped below these.

It is possible to complete Theory of Changemapping at this level or bring it down to a muchmore local level of analysis. ORS use the analogy ofthe view from different heights, reflecting the factthat different audiences inside and outside yourorganisation require different levels of detail.

30,000 foot vantage point. An outcome mapfrom this high-level vantage point is a ‘zoomed out’view, like looking out of an aeroplane window.It is about achieving a long-term goal, includingthe efforts of other partners that you have beenworking with.

This vantage point may be most relevant forgeneral communication with multiple funders,ensuring partners understand their respective rolesand for geting an overall picture of all the elementsof the campaign.

10,000 foot vantage point. Encompasses thebreadth of the work of one organisation. This isthought to be useful if an organisation is seeking todefine its particular role or contribution, or if anorganisation wishes to express how its own internalstrategies and outcomes are related.

This vantage point may be most relevant for boardmembers, staff teams, close partners and funders.

1,000 foot vantage point. This would be usedto illustrate the activities and intended resultsconnected with a singular strategy or related setof actions. This view would be most useful if anorganisation is involved in evaluation planning, ortrying to get a picture of what is likely to happenand/or change in a distinct near term-time period(e.g. the next 1–2 years).

This vantage point may be most relevant for closepartners, staff teams, or constituents.19

ORS also note that there is a danger that in listingall the activities in sequence it can appear thateither they all happen at once or that the logicsequence ploughs on irrespective of what ishappening in other elements of the plan. This isoften not the case and one of the points of the ‘sothat’ chain is to ensure you sequence the order inwhich things should happen. Campaigns can be aniterative process, with some parts of the campaign‘stuck’ at a particular stage and campaigners oftenhave to retrace steps to come at an issue from adifferent angle.

5 Capacity of the organisationto achieve the changeOne of the advantages claimed for the TOC modelis that it illustrates the various elements thatorganisations should have in place to ensure theyhave the capacity to carry out their strategy. Inpart, this derives from the models’ original focus ingrant giving and therefore the need to establish thecapacity of the grantee to deliver on the project, orat least identify the steps they would need to take.

The recent work by the California Endowment hasexemplified this approach and pulled togetherthinking and research on what organisationalcapacities are needed as the basis for a successfulcampaigning approach. The main factors theyidentify are:

• Leadership – exemplified by commitmentto investing in advocacy, credibility withstakeholders and beneficiary groups,support from the board, a clear visionand commitment to work for long-termchange as well as monitoring progress

• Adaptability – by being able to respond tothe needs and aspirations of the community,the ability to monitor and evaluate thecampaign and make changes and adjustments

15





18 Adapted fromORS, Ten Considerations forAdvocacy Evaluation Planning

19 Summarised from Guinapp et el, (ORS,2009) Getting Started: A self-directed guide to

outcome map development. Prepared for theAnnie E Casey Foundation (Seattle, ORS)

• Goodmanagement – including clearcommunication with staff about goals,clear financial planning and externalcommunication

• Technical expertise – such as policy capacityand campaigning skills.

These categories all form the basis of a clearevaluation of organisational capacity as part ofa TOC model, as they form a vital preconditionrelating to the viability and achievability of thechange model being developed.

The model also suggests that there are a number ofquestions that an advocacy organisation should beasking routinely as part of the process of evaluatingtheir capacity to achieve their campaign aims.

These are:

1 To what extent does the organisationunderstand and articulate advocacy goalswith the support of the board? (Leadership)

2 To what extent is advocacy important tothose goals and integrated with otherstrategies? (Leadership)

3 How effective is monitoring of the externalenvironment for advocacy opportunities andthe internal environment for capacity torespond? (Adaptive)

4 What strategic relationships does theorganisation have and which are needed toimplement the Theory of Change strategy?(Adaptive)

5 How is information on advocacy sharedthroughout the organisation? (Management)

6 How well do teams function in order tocapitalise on advocacy work? (Management)

16





Ability to motivate and persuade• Authentic organizational commitment to advocacy• Ability to relate to constituencies• Strong relationships with community leaders

Board leadership• Engaged and committed to advocacy work• Diligence with respect to monitoring short-term and long-term objectives

Strategic visioning• Comprehensive advocacy approach• Clear and consistent communication of projectgoals and objectives• Long-term goal orientation

Leadership distribution

Leadership

Non-staff resource management• Internal knowledgemanagement• External knowledge sharing

Staff coordination• Deliberate communicationsystems• Internal team building

Staff role clarity and human resourcedevelopment

Financial management

Relationship management

Management

Legal knowledge of policyadvocacy work

External communication skills/information dissemination (including media)

Policy issue and theory knowledge• Policy change process knowledge• Substantive issue expertise• Political knowledge and skills• Specify advocacy strategy skills(eg mobilization, policy, analysis,litigation, etc)

Finance and fundraising skills

Interpersonal skills

Facilities/equipment

Technical

Strategic partnerships

Strategic positioning• Community needs and asset assesment• Assessing feasibility of opportunities•Monitoring and assessment of progresss

Resource flexibility

Monitoring and measuring progress• Short-term metrics• Focus on behaviour change• Flexible objectives• Plan for reflection

Adaptability

Access window of opportunity/threat:• Offence• Defence

20 Raynor (TCC Group)What Makes anEffective Advocacy Organization?, p.13’

20

7 What skills and resources do we have andneed in order to implement the particularTheory of Change plan?21 (Technical)

Evaluating organisational capacity has beenintegrated into the TOC model to make explicitthe inputs part of the process.22 Focusing on whatthe essential inputs and capacities are that anorganisation has to put in place to achieve changein a systematic way, allows for a proper focus onorganisational and community capacity. Evaluatingthis at the start of the process ensures thatorganisations do not over commit themselves andtheir supporters to goals that could never be

achieved, but instead to match resources to activitiesand potential outcomes. It provides some keycategories to focus on before starting, but alsowhen evaluating later down the line. It can alsobe used during the process if the campaign is notprogressing as expected.

6 Evaluation built into the model“In particular, the articulation of a theory of change andclear purposes of evaluation can be immensely usefulfor communication about expected results, as well asidentification of useful approaches to tracking, documentingand evaluating advocacy and policy change work.”23

17





1 Shift in social values,attitudes or behaviour

2 Increase organisationalcapacity

3 Create or improvealliances or coalitions

4 Improved base of support

5 Improved policiesand legislation

6 Impact of campaigningactivity

Values, attitudes and behaviour that would need to be changed tobring about the impact you are seeking, or would create a blockageif not addressed.

The skills, training, and strategic capacity of the organisation to carryout campaigning and influencing work increases and allows moreeffective interventions.

The level of co-ordination or competence of the organisations inyour sector. Also the capacity to work across non-sector boundariesincluding unlikely supporters so as to be able to deliver your agenda,ensure that policy wins are protected and change can be embedded.

The necessary level of public support or public will for any particularchange, especially amongst opinion leaders, activists and thegeneration of positive media coverage or support for your issue.

Different types of policy and legislative improvement as a result ofthe campaign. This could be from policy adoption, changes tolegislation, implementation of recommendations or changes inpractice or funding.

The ultimate changes in individual lives as a result of the otheractivities undertaken by the campaign continues to then change theway the issue is seen or develops – people become more empowered.

Outcome categoriesType of outcomes Components

21 Summarised from Raynor, J., et al. (TCCGroup, 2009)What Makes an Effective AdvocacyOrganization? A Framework for DeterminingAdvocacy Capacity (San Fancisco, TCC).Prepared for California Endowment; Alliancefor Justice (2005) Build Your Advocacy

Grantmaking: Advocacy Evaluation Tool andAdvocacy Capacity Assessment Tool (Washington,D.C., Alliance for Justice).The latter is anothervery useful capacity assessment tool.

22 Ibid.

23 ORS, Ten Considerations for AdvocacyEvaluation Planning

24 Adapted from Guinapp et el, (ORS)Getting Started: A self-directed guide to outcomemap development. Development Exercises.Prepared for the Annie E. Casey Foundation

24

The benefit of the TOC models is that it providesa focus on what each approach is trying to deliver,encourages you to be explicit about theassumptions you are making and provides indicatorsto ascertain if they are being achieved. As such,TOC models address one of the perennial issues incampaigning – ‘how can I tell if what I am doing isactually going to make any difference in practice?’

Most evaluation models break down a numberof activities such as examining how far each activityhas been achieved, how successful thoseparticipating in the activity thought it was, and howeffective that activity was at achieving its targets.The TOC approach also forces the evaluator toquestion much more closely:

• The assumptions behind the interventions

• The selection of one type of activity to seeif the method chosen matches the outcomedesired

• The relationship between the different stages.

There is also a focus on the different types ofactivities and outcomes you need to do to achieveyour final goal or impact. These can form the basisof an evaluation framework depending on whichtype of outcome categories you chose to use.25

“Without a theory of change, use of indicators will leadto activity-driven monitoring.”26

One of the major strengths of the TOC model thatpractitioners have identified is the ability to have aprospective as well as retrospective approach toevaluation. Prospective evaluation sets out goals fora campaign at the outset and measures how wellthe campaign is moving toward those goalsthroughout its life.

The advantages of the approach for evaluationhave been seen as:

• Being able to set a clear and testablehypothesis about how change will occur thatallows accountability for results, but also

increases credibly because the change waspredicted to occur in a certain way

• The ability to articulate a Theory of Changeabout how and why the activities of a givencampaign will lead to the end result thecampaign is trying to achieve

• Using a Theory of Change as a framework todefine measurable benchmarks and indicatorse.g. the number of supporter actionsachieved, changes in legislation or funding –for assessing both progress towards a desiredpolicy change and building organisationalcapacity for campaigning in general

• An agreement among stakeholders about whatdefines success and what it takes to get there

• The ability to test the assumptions aboutwhat political, social and economic factorscould influence the outcome of the changebeing sought, and to find any gaps in thestages of the process, necessary to achievethe change being attempted

• An agreed basis for reports to funders,policymakers and boards that will remainconsistent over time

As well as allowing for a rigorous examinationof activities undertaken, TOC allows for regularfeeding back of information – helping campaignersto reflect, as they progress their strategies. Themodel allows campaigners to assess whether thecampaign is working and where to adjust theiractivities and plans. As Coffman concludes,“by more deeply integrating evaluation withimplementation, prospective evaluation providesadvocates and funders with data on progress longbefore policy change can be achieved, and collectsinsights that advocates can use to continuouslyimprove and refine their strategies.”27

There are a number of processes that could beexamined at each stage of evaluation which followthe stages of the TOC model of planning:

18





25 Appendix 1, shows how this can be brokendown into a number of evaluation areas foreach activity. See also Guthrie, K., et al., TheChallenge of Assessing Policy and advocacy activities:Strategies for a Prospective Evaluation Approach

26 Snowden, A., cited in Guthrie, K., at al.,(2009) The Challenge of Assessing Policy andAdvocacy Activities (Los Angeles, CaliforniaEndowment), p.11

27 Coffman, J. (Centre for Evaluation, 2009)Overview of Current Advocacy Evaluation Practice(Washington D.C., Centre for EvaluationInnovation), p. 9.

MeasuresThis step involves identifying specific measures(indicators or benchmarks) that, when capturedand tracked over time, will signal whether thecampaigning strategy elements have beensuccessfully implemented or achieved. Differentkinds of measures go with different CompositeLogic Model elements – see Appendix 1.

Activity/tactic measuresCommonly known as outputs, these ‘measuresof effort’ count what and how much advocacyactivities or tactics produce or accomplish.Although these measures capture what was done,they do little to explain how well it was done orhow well it worked with target audiences. Becausethey count tangible products, people, or events,activity/tactic measures are the easiest of allevaluation measures to identify and track.

Interim outcomemeasuresLinked to interim outcomes, these measures signalprogress toward the achievement of policy goals.Unlike measures that are associated with activitiesand tactics, they are ‘measures of effect’ anddemonstrate changes that happen — usually withintarget audiences — as a result of advocacy efforts.

Policy goal measuresThese measures signal whether policy goals havebeen achieved.

Impact measuresThese measures demonstrate what will happenafter a policy goal is achieved. They show theeffects of policy goals for the programs, systems,or populations that policies aim to improve.28

Impact measures are often the hardest to evaluatebut the advantage of focused evaluation of impactis the clear framework this gives organisations.An example of this form of evaluation is from the‘Superwoman’ project, which focused on raisingthe employment opportunities for survivors ofdomestic violence. The evaluation of the project to

support victims of domestic violence back into work(shown in the diagram below), illustrates the endgoal and the three key outcomes that would beneeded to achieve this goal. Evaluation would thenbe against whether these outcomes and impact havebeen achieved.29

Organisations need to be clear about the ways inwhich they expect to bring about change, but theycan evaluate against each of the potential strategieschosen. For each of the various change strategiesthere are some obvious indicators that can beapplied. As Coffman argues, different Theories ofChange adopted by the campaign will determinewhat needs to be evaluated. Further evaluations willnot necessarily focus on every stage of the processof change but will pick out key determinants orissues that the organisation needs to focus on.

Example of using a TOC approachwith partners to pilot differentapproaches towards evaluationThe Annie E. Casey foundation used a TOCapproach with its KIDS COUNT initiative. KIDSCOUNT is a network of child advocates in all50 US states, the District of Columbia, PuertoRico, and the Virgin Islands. The Foundation hasinvited several grantees to participate in a pilotproject to develop evaluation strategies fortheir advocacy and policy change work. ORS

19





Long-term employment at livable wagefor domestic violence survivors

Survivors attaincoping skills



Survivors havemarketing skills innon-traditionaljobs

Survivors know andhave appropriateworkplacebehaviour

28 Adapted from Coffman, J. (2009) A user’s guideto Advocacy Planning and Evaluation, whichillustrates this in a logical sequence.

29 ActKnowledge and the Aspen Institute (2003)Guided Project Example: Superwoman (New York,ActKnowledge and the Aspen Institute)

Superwoman project evaluation

SummaryAt the end of this process a typical plan would have:

• Produced a clear analysis identifying long-termgoals and the assumptions behind them

• Backwards or reverse mapping of the issue,which connects all the preconditions orrequirements necessary to achieve the goalusing outcome mapping and ‘so that’ chains

• Identified the campaign activities that will beundertaken to create the end goal for change

• Developed indicators to measure theoutcomes in order to assess the performanceof the campaign

• Involved stakeholders and beneficiaries in theprocess

• Produced a written plan or narrative to explainthe logic of the campaign

worked with these grantees to develop theirevaluation strategies, which included thedevelopment of outcome maps (see ORS’sOrientation to a Theory of Change for an easy-to-follow overview of Theory of Changetechniques, including how Theory of Changedevelopment fits into other types of outcomes-based planning).30Once designed, theexpectation is that advocates will implementtheir own evaluations. While this process is stillunderway, the evaluators, advocates and theFoundation have found that the process ofidentifying outcomes and their linkage tostrategies calls into question a host of strategicquestions, including consensus within theorganisation, transparency, real-time relevance,belief in the value of evaluation, and theinterconnectedness among organisationalstrategies.

Grantees ExperiencesOrganisations that have used a TOC approach havebeen very clear about the potential benefits thisapproach has delivered. This comment from theevaluation of the Georgia Connection Partnershipstressed the clarity it had achieved and theadvantages this brought:

“We worked to develop an organizational theory ofchange and so far, the payoff has been wonderful. Notknowing that we would be facing a major state budgetcrisis this year, it was absolutely the right and mosttimely thing we could have done! We are able to clearlyshow, describe and defend our work with our funders, thelegislature, our partners and our board. People say‘Oh, now I really get it. I see what you do.”Executive Director, Georgia Family ConnectionPartnership31

While another grantee stressed the effect inincreasing credibility with Government, throughbetter engagement and clarity about values andaccountability:

“Our work to measure the impact of our advocacy effortsbeginning with the development of a theory of changehas moved our work forward significantly. The processof defining our strategies, outcomes and goals gave ourteam a framework for discussing the values and directionof our organization in the coming years. By modeling theaccountability we seek from government anddocumenting the outcomes of our work, we are betterpositioned to advocate for a system that effectively serveschildren.”Director of Policy and Research, North CarolinaAction for Children32

Other evaluations have reflected similar commentsfrom the organisations that Foundations haveworked with. This has been reflected across thespectrum from small community projects to verylarge multi-goal campaigns. Though obviouslythere are some issues that small organisations

20





30 ORS (2008) Orientation to theory ofchange (Seattle, ORS)

31 Quoted in Ten Considerations for AdvocacyEvaluation Planning, p.4

32 Ibid, p.5

encounter in this and in any other developedplanning system, relating to capacity.33

For most of the major funders involved, they nowoperate a very clear planning framework withgrantees that follows the TOC modelling andevaluation process – both to assess the capacity oforganisations to receive grants and how they workwith them through the process. Often this willinvolve helping organisations to implement themodel with developed training packages andtechnical support.

21





33 One of the more comprehensiveevaluations is ORS (2009) Ten Considerations forAdvocacy Evaluation Planning, but there aremany other evaluations on different projectsincluding Anderson, A. (The Aspen Institute,2004) Theory of Change as a Tool for StrategicPlanning: A Report on Early Experiences (NewYork, The Aspen Institute)

Implications for Campaignersin the UKWhile the political context in the US is clearlydifferent to the UK, as well as the organisationalsetting and funding strategies, the underlyingprocess of developing campaigns is very similar andtherefore the learning from the US experience isvery relevant to UK campaigners.

TOC as a developing community of practiceprovides the basis for a distinct and integratedapproach spanning the development of strategy,delivery and evaluation of campaigns. In doing so itprovides a common framework for campaigners,evaluators and funders, which has been seen tobring many advanatages to the Foundations whohave used it.

In respect of Strategy, the TOC Models’ strengthsare that it:

• Provides a common language and approachfor planning and evaluation where everyonehas the same framework as the starting point

• Builds the capacity of organisations to thinkmore strategically about their goals

• Makes implicit assumptions explicit andtherefore easier to test and modify in thelight of experience

• Builds from the actual strategies and activitiesthat campaigners undertake but gives a clearframework to integrate thinking about these

• Keeps focus on the final goal to be achievedand clarity about steps along the way

• Provides a framework for developingdifferent scenarios about how change mighthappen and how to plan for contingencies.

In respect of Delivery its strengths are:

• It identifies the resources and milestones fora plan and allows organisations to test therelationship between these different activitiesat the beginning of the process

• Allows for adjustments in the framework

against the experience of testing these outduring the campaign

• Makes clear and then forces organisations totest, the relationship between activities,outputs and outcomes

In respect of Evalution its strengths are:

• It encourages a prospective not justretrospective evaluation approach. Theframework allows you to specify theprerequisites to change and the steps toachieving them, so you can build in themilestones and conditions needed

• It focuses on the contribution towards theachievement of the stages of the process,rather than worrying about overall attributionbetween agencies

• By using the actual activities and objectivesdrawn from policy and campaigns analysisand focusing on the activities thatcampaigners actually do, it allows you tomove away from ‘evaluation speak’ and ontothe actual ground that campaigners occupy.This mirrors the process campaigns actuallytake, without imposing additional frameworksonto the process

• It makes explicit the underlying assumptionsand relationships behind activities, allowsthese to be tested and related to final goalsorganisations are aiming to achieve

22



Implications


Issues to considerThere are a number of issues that need to beconsidered for organisations thinking of using thisapproach.

Complexity

A Theory of Change is not necessarily suitable forevery intervention and may be overly complex ifyou have a very simple issue or problem. Essentially,TOC is trying to look at and test your overallstrategy and planning assumptions across acomplete area of activity or plan. If you need aninstrumental tool to ensure that one element ofthat plan is going to be delivered, a simple LogicModel might be more appropriate, rather thanfocusing on the whole process.

While the basis of the TOC model is simple, it canbecome a very complex exercise depending on thenumber of strands a campaign has. Smallerorganisations or groups have found complexity anissue. The planning stages should be done as partof a team whenever possible as this allows anumber of different insights and the opportunityto build a consensus for the approach as you goalong. However this does require significantorganisational commitment to the process.

Clarity about aims

It can be difficult to identify the right overallaim and state this clearly enough. Good ideas arenot always the same as a good model. However,the process of trying to identify a clear aim isimportant – if this is not easy to clarify,organisations should be wary of embarkingupon a campaign.

Be prepared for the fact that the process mayreveal that there is not agreement within teamsor across the organisation about what thecampaigning aims and strategies to achieve themare. However, while this needs managing, one ofthe strengths of the process is that it brings these

issues to the surface and allows teams to deal withthem, either establishing a consensus for the planor that an approach being considered is not goingto work.

Scale of the task

Identifying the steps and assumptions about whatis going to lead to a particular change can be off-putting, as the analysis may only reveal that much isbeyond the control or capacity of the organisation.This can be demoralising, especially for smallerorganisations with fewer resources. However, thiscan also be helpful in focusing objectives back ontowhat can be achieved, and in allowing organisationsto plan for what is in their control.

As the Alliance for Justice Campaign argue, “it canbe difficult to show cause and effect between onespecific organisation’s advocacy activities and policychange”. The TOCModel should be specific aboutwhat the organisation can do and be sensitive tocommunities and alliances as part of the strategy.The evaluation should look at how well theorganisation is delivering the campaign and whetherthe Theory of Change postulated is being borne outin reality – rather than focusing on how muchorganisation x has contributed over organisation yto the delivery of the campaigns’ aims.

TOC does not have all the answers?

TOC is not an all encompassing advocacy andcampaign evaluation approach in terms of itsapplication, because to analyse specific activitiesyou would also need to use other methods.34

However, it is extremely helpful in providing theoverall framework in which you evaluate campaignsand points towards the key relationships betweendifferent facets of your activities, which might needevaluating. It enables you to know where to look inthe process and what linkages should be tested.

Because the TOC model has predominantly beendeveloped by Foundations to improve their grantmaking programmes, this has been both a major

23



Issues


34 Coffman, J. A user’s guide to AdvocacyPlanning and Evaluation gives some very usefulexamples of types of evaluation methods

specific to campaigning that can be integratedwith a TOC approach.

source of strength and weakness. The strength isthat the Foundations have all been intimatelyinformed in developing the approach by existingprogrammes of work where they have beenexplicitly applying this approach. The drawback isthat this learning has often been on very specificand local or at best State-wide programmes ofactivity, often in the children’s and health sectors.These have not been routinely scaled up beyond astate-wide context, though this is being addressedin more recent work. The model has not thereforebeen tested to the same extent on nationwidecampaigns that might be more typical in the UK.

Change can take many years and TOC is very goodin accounting for the difference between short-term tactical activities and keeping a focus onlong-term change. This does however require thatboth organisations and funders have a long-termcommitment to investing in change and in theirstrategic capacity to bring it about.

ConclusionMany campaigners and advocates spend timereflecting on what they are doing and how theymake a difference. By placing the focus on howchange happens and then putting in place thestrategies and resources to achieve this, TOCmodels work with the grain of campaigners’thinking and practice. This has helped those usingthe model to integrate it into their everyday workwithout it being an extraneous and onerous add-on.

The model meshes a strong focus on a logicalsequence of activities with an understanding of thecontext that surrounds making social and politicalchange happen. Using this model will not suddenlymake campaign planning into a science, but it doesappear to have been very successful as a systematicapproach enabling organisations to focus onimpact. For funders it has brought greater clarity tothe process of funding decisions, assessingoutcomes and the capacity of organisations todeliver what is being asked of them.

In a period where there will be even greaterscrutiny on what makes organisations effective,

TOC could give campaigners a model that providesa common framework, based on a community ofpractice, which would help bring more rigour tocampaigning. Furthermore, it would help provide aclearer way for organisations to demonstrate theireffectiveness and their contribution. Campaigningstill suffers in some areas by being seen as nothaving developed the tools and analytical insight ofother social disciplines. TOC provides a powerfulantidote to that view by encouraging a morerigorous approach to showing how changehappens, and how campaigns have consciouslybrought this about. At the very least, this arguesfor the model to be taken very seriously as partof the developing practice of campaigners overthe coming years.

24



Conclusion


25



Summary


Overall Summary of different approaches

35 Developed and updated from Guthrie, K., et al. (2006) The Challenge of Assessing Policy and Advocacy Activities

35

Approach/organisation

Categories Focus Plus Minus

Kellogg Logic Model Projects focused Basis for a Logic Modelapproach to planning.Extremely comprehensive.

Can appear daunting andtechnical, the early modelwas not very specific tocampaigning.

Harvard FamilyProgramme/CompositeTheory of Changemodel/CaliforniaEndowment, AspenInstitute

Embeds TOC withindifferent campaign andadvocacy strategies.Focuses on the process ofdifferent strategies andselecting the rightapproach to achieve theultimate aim.

Strong focus on the policycycle. There is a growingemphasis on organisationalcapacity as part of themodel.

Campaigning – inparticular in regard to whatother actors might do tofacilitate or hinder.

Detailed understanding ofthe different elements thatgo to make up a campaignstrategy, and the logicalroute to take to achievethe ultimate impact, with agood account of the policycycle. Substantial focus inevaluation on what otheractors are doing in thepolitical sphere. Strongfocus on media andcommunications theory inearlier work. Strong focuson overall organisationalcapacity as part of themodel.

Less focus on thecommunity change orinvolvement necessary toachieve campaign ends inmore recent work.

Annie E. Casey/ORS-TOC-mapping

TOC model with a focuson Impact Outcomes,

Influence Outcomes,and Leverage Outcomes.Linked to this is a 6 pointfocus on Norms,Organisational Capacity,Alliances, Support Base,Policy Windows, andImpact, as the keystrategies for change.

Campaign organisationsand changes in thecommunity.

Detailed understanding ofdifferent strategies forchange and measurementof strategies and thecampaigning process.Realistic about changes tosociety, so there is a realfocus on creating morecapacity at the communitylevel as well as long-termconditions for influencing;this is seen as an outcome.

Only recently has workbeen done focused on theconditions necessary forlarger scale policydevelopment and change.

Alliance for Justice Outcomes-focused andevaluation of progresstowards goals.

Capacity building at thecommunity level.

Campaigning but with avery strong focus on legaladvocacy and politicallobbying for change.

Detailed evaluation ofpolicy; assessment ofrelevant short and long-term gains including howto assess these. Illustratedfrom working practices.Very embedded inWashington D.C., andState lobbying

No overall Theory ofChange or ordering aroundwhat produces outcomes.

Not so easy to generaliseoutside of the US context

Theory of Change.org/ActKnowledge

Standard butcomprehensive TOCmodelling with onlineresources.

Projects- focused Thorough introduction toTOC for all kinds ofprojects.

Not campaign-specific.

Annotated guide and bibliography:further reading on the Theory ofChangeModelNational Council for Voluntary Organisations(NCVO) http://www.ncvo-vol.org.ukCoe, J., and Mayne, R. (NCVO, 2008) Is yourcampaign making a difference? (London, NCVO)

Lamb, B. (NCVO, 2011) The good guide tocampaigning and influencing (London, NCVO)

Alliance for Justice http://www.afj.orgAlliance for Justice (2005) Build Your AdvocacyGrantmaking: Advocacy Evaluation Tool and AdvocacyCapacity Assessment Tool (Washington, D.C., Alliancefor Justice)

Annie E. Casey Foundation and OrganizationalResearch Services (ORS) http://www.aecf.orgOrganizational Research Services (ORS) specialisesin outcome-based evaluation and Outcome-basedplanning, serving non-profit, philanthropic andpublic organisations. The following were preparedfor the Annie E. Casey Foundation:

Guinapp, A., Reisman, J., and Stachowiak, S. (ORS,2009) Getting Started: A self-directed guide to outcomemap development (Seattle, ORS). A summation ofmuch of the earlier work of the ORS group for theAnnie E. Casey Foundation, but in the form ofa practical guide with helpful examples for eachstage of the process.

Reisman, Gienapp, and Stachowiak, (ORS, 2007)A Guide to Measuring Advocacy and Policy (Seattle,ORS)

ORS (2009) Ten Considerations for AdvocacyEvaluation Planning: Lessons learned from Kids Countgrantee experiences. (Seattle, ORS). Pulls togethersome of the key learning of in applying the TOCapproach.

ORS (2004) Theory of Change: A Practical Tool ForAction, Results and Learning (Seattle, ORS)

Other ORS publicationsORS (2008) Orientation to theory of change (Seattle,ORS). This worksheet outlines a TOC model and

examines a ‘Layer Cake’ approach to looking athow change happens:http://www.organizationalresearch.com/publications/orientation_to_theory_of_change.pdf

Stachowiak, S. (ORS, 2007) Pathways to Change. 6Theories about How Change Happens (Seattle, ORS).Anna Stachowiak outlines some of the key theoriesunderlining strategies to bring about change; thepaper is very useful in ordering Theory of Changeactivities.

Aspen Institute http://www.aspeninstitute.orgA key project is Aspen Institute’s GlobalInterdependence Initiative. Resources include the‘Advocacy progress planner’:http://www.planning.continuousprogress.orgThis is a comprehensive online tool to build acomposite logic model. Definitions and tips areoffered throughout the process, and the endproduct is a customised TOC composite plan.

See also Anderson, A. (The Aspen Institute, 2004)Theory of Change as a Tool for Strategic Planning:A Report on Early Experiences (New York, The AspenInstitute). Roundtable on Community Change.Analysis of the application of the TOC approach,in a case study.

Aspen Institute with ActKnowledgehttp://www.actknowledge.org/ActKnowledge and the Aspen Institute (2003)Guided Project Example: Superwoman (New York,ActKnowledge and the Aspen Institute)

The Atlantic Philanthropieshttp://www.atlanticphilanthropies.orgThe Atlantic Philanthropies (2008) ‘Investing inChange’, Atlantic Reports, May

In 2008, The Atlantic Philanthropies issued areport, Investing in Change: Why Supporting AdvocacyMakes Sense for Foundations. It is available fordownload on their website, where there are links toother advocacy resources and a list of Foundationsthat support advocacy.

26



Bibliography


The California Endowmenthttp://www.calendow.orgGuthrie, K., Louie, J., David, T., & Foster, C. C.(Blueprint Research & Design, Inc, 2006) TheChallenge of Assessing Policy and Advocacy Activities:Strategies for a Prospective Evaluation Approach(Los Angeles, The California Endowment). Thereis a second report by the same authors – (2006)The Challenge of Assessing Policy and AdvocacyActivities: PART II—Moving from Theory to Practice.Prepared for The California Endowment.

The California Endowment has an advocacy toolkiton its website. In January 2009, the Foundationhosted Advocacy Evaluation Advances, a nationalconvening on advocacy and policy evaluationattended by funders, advocates and evaluators.Materials from the convening are available on theFoundation’s website:http://www.calendow.org/Article.aspx?id=3774.

Centre for Evaluation Innovationhttp://www.evaluationinnovation.orgThe Centre was established to develop areas ofevaluation in which traditional approaches havesignificant shortcomings. This includes advocacyand policy change efforts, systems change andcommunications. The main focus of the Centre ison advocacy evaluation.

Publications available on the website include:Coffman, J. (Centre for Evaluation, 2009) Overviewof Current Advocacy Evaluation Practice (WashingtonD.C., Centre for Evaluation Innovation)

Foster, C., and Louie, J (Centre for EvaluationInnovation; Blueprint research + design forphilanthropy, 2010) Grassroots Action and Learningfor Social Change: Evaluating Community Organizing(Washington D.C., Centre for EvaluationInnovation)

Campbell, M., & Coffman, J., (The James IrvineFoundation, Centre for Evaluation Innovation,2009) ‘Tools to Support Public PolicyGrantmaking’, The Foundation Review, Vol.1, No.3

Masters, B. (Centre for Evaluation, 2009) EvaluatingPolicy Change and Advocacy: The Funder’s Perspective(Washington D.C., Centre for EvaluationInnovation)

GrantCraft http://www.grantcraft.orgThis project of the Ford Foundation publishedProscio, T. (Grantcraft, 2005) Advocacy Funding: ThePhilanthropy of Changing Minds (New York,Grantcraft), which has a strong emphasis onadopting a Theory of Change. In 2009, Grantcraftreleased McGarvey, C., and Mackinnon, A.(Grantcraft & Centre for Community Change,2009) Funding Community Organizing: Social Changethrough Civic Participation (New York, Grantcraft),produced in partnership with the LinchpinCampaign, a project of the Centre for CommunityChange.

Harvard Family Research Project (HFRP)http://www.hfrp.orgCoffman, J. (HFRP, 2002) Public CommunicationCampaign Evaluation: An Environmental Scan ofChallenges, Criticisms, Practice, and Opportunities.Prepared for the Communications ConsortiumMedia Centre. (Harvard, Harvard Family ResearchProject). An early example of the TOC modelapplied to communication campaigns.

HFRP (2007), ‘Advocacy and Policy Change’, TheEvaluation Exchange, Vol. XIII, # 1, Spring. This issueis devoted to the topic of advocacy and policyevaluation. The Centre has also been involved inproducing a number of publications on advocacyand planning using the Composite Logic Model,the best summation is in:

Coffman, J. (Harvard Family Research Project,2009) A users guide to Advocacy Planning andEvaluation (Harvard, Harvard Family ResearchProject). This publication brings together thinkingon the ‘Composite Logic Model’ for policy changeand evaluation methods suitable for this approach.

Harvard Business School http://www.hbs.eduWenger, E., McDermott, R., and Snyder, W. M.(Harvard Business School, 2002) Cultivating

27



Bibliography


Communities Practice: A guide to managing knowledge(Cambridge, MA: Harvard University)

Innovation Network (Innonet)http://www.innonet.orgCentre for Evaluation Innovation and InnovationNetwork, Inc. (2010) Advocacy Evaluation Update,#9,May

Innonet maintains an online advocacy evaluationresource centre and offers a free e-newsletter co-produced with the Centre for EvaluationInnovation: Advocacy Evaluation Update. Innonethave also produced a comprehensive bibliographyof sources on evaluation and campaign andadvocacy planning, including guides for fundersand advocates.

See also Innovation Network, Inc. Pathfinder: APractical Guide to Advocacy Evaluation. Commissionedby The Atlantic Philanthropies.

New Philanthropy Capital (NPC)http://www.philanthropycapital.orgNPC was one of the first organisations in the UK toadopt the TOC approach. They have producedLofgren, G., Lumley, T., O’Boyle, A. (NewPhilanthropy Capital, 2008) Critical Masses: Socialcampaigning, a guide for donors and funders (London,New Philanthropy Capital), which features the TOCmodel in regard to steps in advocacy work and theplanning process.

RobertWood Johnson Foundationhttp://www.rwjf.orgRobert Wood Johnson Foundation (2008)Evaluating Communication Campaigns: 2007 Researchand Evaluation Conference, September 27–28,2007 (Princeton, Robert Wood JohnsonFoundation). The paper includes the use of TOCmodels in evaluation.

TCC Group http://www.tccgrp.comA consultancy that has championed use of theTOC model.

Raynor, J., et al. (TCC Group, 2009)What Makesan Effective Advocacy Organization? A Framework forDetermining Advocacy Capacity (San Fancisco, TCC).

Prepared for California Endowment. This looks atorganisational capacity through the lens of TCC’s‘Capacity Logic Model’, which is an extension ofthe logic model.

This paper is intended to guide nonprofits seekingto engage in advocacy, Foundations that want toexpand their advocacy grantmaking, and evaluatorswho wish to assess advocacy.http://www.tccgrp.com/pdfs/EffectiveAdvocacy_final.pdf

The James Irvine Foundationhttp://www.irvine.orgCoffman, J. (The James Irvine Foundation, 2008).Foundations and Public Policy Grantmaking (SanFrancisco, The James Irvine Foundation). HowFoundations could use Theory of Change to deliverbetter outcomes around grant making.

Also, as cited above, for a more detailed look atsimilar issues: Campbell, M., & Coffman, J., ‘Toolsto Support Public Policy Grantmaking’

W.K. Kellogg Foundation (WKKF)http://www.wkkf.orgW.K. Kellogg Foundation (2004) Logic ModelDevelopment Guide (Michigan, W.K. KelloggFoundation). This publication remains the standardexposition of how Foundations could use a LogicModel to evaluate social programme development,with clear examples.

The WKKF produced the first fully comprehensiveguide to the Logic Model which was veryimportant in provoking others to follow suit.

Other references:McKinsey & Co., cited in InteroperabilityClearinghouse (ICH) Interoperability ClearinghouseGlossary of Terms:http://www.ichnet.org/glossary.htm

Overseas Development Institute (ODI):http://www.odi.org.ukODI has produced a wealth of research intoplanning and policy, though not specificallyto evaluating campaigning or to the Theory ofChange model.

28



Bibliography


29



Appendix 1


Appendix 1: Evaluation chart

Outcomemeasures for policy and campaigning work

Outcome category Types of analysis and some prospective measures

Social attitudes – awareness-changing and behaviour change

Outcomes • Changes in awareness• Increased agreement of the definition of a problem(e.g., common language)

• Changes in beliefs• Changes in attitudes• Changes in values• Changes in the salience of an issue• Increased alignment of a campaign goal with core societal values• Changes in public behaviour

Examples of strategy • Media campaign - amount of earned media, websitedevelopment and hits

• Message development (e.g., defining the problem, framing,naming solutions and success at getting these adopted)

• Development of trusted messengers and champions• Publicity campaigns - relationships established with mediaoutlets

• Advertising and media developed

Analysis – who or what has to be changed for the effectswe are trying to evaluate?

• Population groups• Individuals• Associations of Individuals

Building organisational capacity and competence

Outcomes • Improved organisational capacity around campaigning andpolicy work (level of skills, resources, ability to sustain activity,credibility of organisation with other partners and decisionmakers.)

• Increased ability of coalitions working toward policy changeto identify policy change processes

Examples of strategy • Leadership development• Organisational capacity building• Communication skills building, strategic planning, qualityand robustness of planning and delivery

Analysis • Campaigning departments of organisations• Organisations• Campaign coalitions• Campaign leaders

Strengthened Alliances

Outcomes • Increased number of partners supporting issue• Increased alignment across coalitions or between coalitionsand other targets

• Increased alignment with more powerful players

Examples of strategies • Increased investment in coalition and alliance building• Supporting the development of other organisations’ capacity• Growth in public campaigning

30



Appendix 1


Social attitudes – awareness-changingand behaviour change

Types of analysis and some prospective measures.

Analysis • Organised campaign groups• Other organisations• Individual views of coalitions

Strengthened Support through organising and campaigning

Outcomes • Increased public support - measured by attitude surveys, etc• Changes in public will• Increased visibility of message• Increased awareness of messages in key opinion-former groups

Examples of strategies • Community campaigning and organising• Media campaigns• Development of alliances• Increased policy debate• National and local public campaigning

Who are you measuring • Individuals• Groups• Organisations• Institutions

Improved polices, legislation, or change to current practice

Outcomes • Policy development• Policy placement on the agenda• Policy adoption• Policy implementation• Policy enforcement• Policy blocking of unacceptable changes

Strategies • Research development• Policy proposals• Service modelling, pilots, demonstration programmes• Public will campaigns - changes to public support for new policypositions or legislation

Unit of analysis • Policy makers, civil servants, corporate officers• Legislators, Ministers, MPs, think tanks• Administrators, national and local government, quangos

Changes in impact

Outcomes • Improved social and physical environment, e.g., environmentalimprovements, health improvements, increased personalcapacity and empowerment, better public control over decisionmaking, more involvement in the political life of a community,more entitlements that are being taken up and having a positiveeffect on people’s lives

Strategies • Combination of effects from other activities leads to change inlife experiences or material changes in the environment

Unit of Analysis • Population or individual depending on the issue.• Environmental change

36 Adapted from Coffman, (ORS, 2009) Getting Started, to show more campaign related as well as policy development outcomes.

36

Appendix 2: Pro forma Theory of Change mapping tools37

31



Appendix 2


STR

AT

EGIE

SO

UT

CO

MES

GO

AL(

S)

(Complete chain from outcomes to goals)

37 Pro Forma adapted from Coffman, (ORS, 2009) Getting Started, p.29

AcknowledgementsThanks to Philip Hadley and Susie Rabin whooversaw the project, and to Davinder Kaur, AmeliaGudgion, Sasha Daly and Nicola Gilbert, who allmade helpful comments on drafts of the report.Further thanks to Jonathan Ellis, Advocacy Directorat the Refugee Council, who pointed me towardssome useful sources on Theory of Change.

This resource has been produced by CampaigningEffectiveness, NCVO, jointly funded by theBig Lottery Fund and Capacitybuilders.

About the authorBrian Lamb, OBE has held Director level postsin campaigning and advocacy in both Scope andRNID. He now runs a consultancy on campaigning,policy and communications for VCO and publicsector organisations. He has worked extensivelyon Government and ministerial working groupsand was chair of a national coalition on, and leda national independent Inquiry into, specialeducational needs. Brian has written widely aboutcampaigning, evaluation and policy issues and ischair of the Campaigning Effectiveness, NCVOadvisory group. He is a founding member of theEvery Disabled Child Matters campaign andlectures on campaigning and voluntary sectoreffectiveness.

About CampaigningEffectiveness, NCVOCampaigning Effectiveness, NCVO supports andempowers people and organisations to changetheir world through campaigning and influencingpolicy. We bring together experience and expertiseand drive excellence in campaigning and policywork across civil society by providing support,knowledge, tools and resources. For furtherinformation about our work go to www.ncvo-vol.org.uk/campaigningeffectiveness

Campaigning for change:Learning from the United StatesThis guide provides campaigners with an answerto the age-old question, ‘is your campaign makinga difference?’ By examining a novel and systematicapproach to campaign evaluation developed in theUS, through a step-by-step process with practicalexamples, the guide draws out key implicationsfor UK campaigners and illustrates how theseimplications can be put into practice.

The approach, known as Theory of Change,draws on key insights from social and politicaltheory, challenging campaigners to questionhow change occurs, the goals they seek to achieveand the impact they wish to make, as well asthe relationship between campaign activities.The guide enables campaigners to betterdemonstrate impact and develop muchmore effective campaigns.

NCVONational Council for Voluntary OrganisationsRegent’s Wharf8 All Saints StreetLondon N1 9RL

T: 020 7713 6161F: 020 7713 6300

E: [email protected]

Textphone: 0800 01 88 111

Free advice and supportwww.askNCVO.org.uk

HelpDesk: 0800 2 798 798 [email protected]

Charity Registration: 225922

32



Acknowledgements


Campaigning for Change - Learning From the United States

Documents