Behavior Change · and behavior change? Why? Impact Evaluation Bene!ts achieved because of the BCC intervention Are changes in outcomes due to the BCC program? • Did communities

Behavior Change Communication

Monitoring the process Does it work?

Webinar 4 | June 26

Assessing the situation What you need to know

Webinar 2

BCC

People eat food not nutrition Integrating BCC into nutrition programs

Webinar 1

BCC strategy and roll out The devil’s in the detail

Webinar 3

Yana Manyuk

Sight and Life Team

Expertise in social marketing & behavioral change communication 8 years experience in applying behavioral change approaches and methods to a range of projects and programs within the government, UN, and NGO organizations at HQ, regional, and field level MA from Middlesex University London in Health and Social Marketing; Distinction BA in Cultural & Media Studies - Maastricht University

Social Media

Use #SALwebinar to join the conversation!

Dr. Eva Monterrosa

Expertise in nutrition and dietetics, cultural anthropology and public health implementation science 15 years of experience in nutrition program design and assessment, formative research, social marketing, behavior change communication with academia, UN agencies, government ministries and NGOs. PhD, Nutrition, Cornell University; MS in Human Nutrition and Metabolism; BS in Nutrition and Food Science – University of Alberta

Sight and Life Team

Monitoring the Process Does it work?

Agenda

Review BCC Process Cycle Step 4, 5, 6 & 7

BCC Process Step 8: Monitoring and Evaluation

Theory of Change

BCC Objectives and Monitoring Framework

2 3 1 4

Monitoring Plan

5 6

Overview of BCC Process

Review BCC Process Cycle Step 4, 5, 6 & 7

1

3 2 4 5 6

BCC Strategy and Roll-Out

11

Checklist

Step 4. Creative Design, Step 5. Pretesting, Step 6. Training & Step 7. Implementation Completed

Strategy based on -  Exchange (reducing cost & increasing short-term benefits

from perspective of the audience) -  Operational mix -  Content strategy

√

BCC creative brief written and shared with stakeholders √

Creative concepts & content pretested √

Training strategy is developed based on key BCC elements √

Implementation plan including costing and phasing √

BCC is a Process…

Step 1. Goals and

Objectives

Step 2. Desk

Review

Step 3. Client

Research

Step 4. Creative Design

Step 5. Pre-testing

creative

Step 6. Training

Step 7. Implement

Step 8. Monitor

and Evaluate

Webinar Objectives

Construct their own theories of change and results framework

Understand the utility of outcome monitoring for tracking results

Program managers should be able to

Learn how to set SMART objectives and what to include in a monitoring and evaluating (M&E) plan

BCC Process: Step 8 Monitoring and Evaluation

1

3 2 4 5 6

15

Continuing function that provides the management and main stakeholders with early indications of progress, or lack thereof, in the achievement of results (milestones, targets, KPIs). Monitoring is a management function – it is led and executed by the management team.

A selective exercise that attempts to systematically and objectively assess progress and achievement of an outcome (or impact). Evaluation is an evidence function – often led by an independent party not involved in the entity being evaluated.

Monitoring Evaluation

Step 8: Monitoring and Evaluation

Does one have a quality program that is functioning as intended?

a. Prerequisite for monitoring function

•  Plausible program theory (tools: PIP, results chain) •  Monitoring will not fix a non-existent or non-effective program •  Monitoring plans are often developed shortly after the design of the

intervention is approved

b. Purpose for monitoring •  Provide information on results achieved •  Actionable information that yields recommendation(s) to improve

or manage better the program

c. What is monitored •  Inputs, activities, outputs, and some short-term outcomes •  Indicators and targets set and agreed upon by management team

d. Who is responsible for monitoring Program manager/operations team

e. How do we monitor Strategies and tools based on program needs for tracking results and ability to make changes

f. Timing •  On-going indicators are reported on a monthly or quarterly basis •  Comparability of indicators and data over time is critical


Key Elements for BCC Monitoring & Evaluation

Element 1 | Theory of Change

Element 3 | Approach and Methods for the Monitoring and Evaluation Plan

Element 2 | BCC Objectives and Monitoring Framework


Monitoring and Evaluation

Theory of Change

1

3 2 4 5 6

Theory of Change

How the intervention is expected to change behavior?

Question Tools to answer the question

What is the change process and is it plausible?

Conceptual model or behavior theory – Steps 2 & 3 Exchange – Step 4

What are the components, key functions of the intervention? Operational mix – Step 4

19

Maternal Food Choice

Physiological factors •  Appetite •  Pregnant or

lactating

Access to Food •  Order of eating •  Food sharing •  Quantity &

Quality

Autonomy •  Household

decision-making

•  Mobility •  Workload

Cognitive Factors •  Knowledge •  Motivations •  Aspirations

Family-Related Factors •  Family hierarchy

and dynamics •  Household economy •  Role of husband •  Role of mother-in-

law

Health and Nutrition Resources Provision and uptake of services provided by frontline workers and doctors

Cultural and Economic Context (Patriarchal society, male preference, poverty)

Maternal Factors

EXCHANGE VALUE FOR HUSBAND

A small investment in food and care today = Lower risk of complication and expensive medical bills

(short-term) and a healthy baby (long-term)

EXCHANGE VALUE FOR WIFE

Small changes in food consumption today = Tangible effects on child's development

EXCHANGE VALUE FOR MOTHER-IN-LAW

Encouraging a son to be a good father and husband = Ensuring a good delivery and strong child born

What is the change process and is it plausible?

Look at exchange value


Operational Mix Component Key Function To Whom

Promotion Media campaign Persuasion and aspiration - to motivate

Husbands, Mother-in-law, Wives

Place/Product First antenatal care receive a welcome package

Persuasion and aspiration - to motivate, to improve opportunity, improve capability

Husbands, Wives

Place/Service/Product Home visits by front line workers

Targeted education on food function Support for snacking behaviors and foods, filling treat bag - to motivate, improve capability

Wives

Support to Mother-in-law to encourage son to buy foods on grocery list - to improve opportunity

Mother-in-Law

What are the components and key functions of the intervention?


Media Campaign

Motivations – Exchange value: 1) husbands, 2) wife, 3) MIL Increasing opportunity (access), and self efficacy (capability)

Counseling on Welcome Kit –

shopping list, SD card, letter from Chief Minister

Treat bag/box

Wife (pregnant/lactating)

Mother-in-law supports son

YOUNG HUSBAND

Champion (baby)

At first antenatal care visit

Food Function

Cards

Front Line Worker

Incr

ease

sna

ckin

g of

nut

rien

t den

se

food

s (t

wo

snac

ks fr

om a

t lea

st tw

o diffe

rent

food

s)

*Items in blue text = key interventions

Home

Theory of Change

Monitoring and Evaluation BCC Objectives and Monitoring Framework

1

3 2 4 5 6

Setting SMART Objectives

25

Our objectives were refined throughout the BCC process

Step 1. Initial Behavior Objective Women increase food intake during pregnancy

Step 3. Refined Behavior Objective Women increase food intake between meals of nutrient dense foods available in home

Step 5. Refined Behavior Objectives Women snack two times per day and choose two different snacks from seven food options

Step 8. SMART Objective At 6 months post pregnancy enrollment women are snacking once per day

SMART Objectives Specific, measurable, achievable, realistic, time-bound

BCC Objective 1 At 3 months post pregnancy enrollment, 50% husbands express intentions for buying at least 2 of the 7 foods on the food list.

BCC Objective 2 At 6 months post pregnancy enrollment, 50% of women are snacking once per day.

BCC Objective 3 At 6 months post pregnancy enrollment, 50% of women report eating at least 5 foods on the list in the previous week

BCC Objective 4 After 12 months, The Champion T.V. campaign evokes attributes (hope, pride) such as 'better future' 'better jobs' achieving more than I did’ among 75% of listeners

Increase snacking of nutrient dense foods among pregnant and lactating women (PLW)

GOAL


Examples of SMART BCC Objectives

27

•  Agree with your brain trust on the BCC objectives

•  Make sure the objectives support the behavioral goals

•  Objectives must focus on outcomes and not outputs

•  Objectives will need indicators and these must be

included in the program monitoring plan

Tips on Setting SMART Objectives


Inputs Activities Outputs Outcomes Impact

Results from activities – deliverables Response to stimulus

Changes observed because of set of activities. outcomes achieved during the lifecycle of the project

Benefits achieved

The action planned Stimulus

Resources available to you

Indicators that measure each domain

Monitoring Framework

Results Chain

Inputs Ac*vi*es Outputs Outcomes Impact

Impr

oved

inta

kes

amon

g PL

W

Pregnancy registration at first antenatal care visit and welcome package Intention about

snacking

Beliefs - expectancy

Fathers buy foods on food list

Welcome package counseling

Wives consume snack

Awareness for snacking

Intention to buy foods on food list

Media Campaign

Television Billboards

Emotion Media Exposure: Eligible households view campaign

Eat More, Eat Better Results Chain


Self-efficacy and capacity HW PC

Media Exposure: Eligible households view campaign Parents

understand benefits of snacking

Home visits are engaging (interest)


Media Campaign

Impr

oved

inta

kes

amon

g PL

W

Front line workers visits home

Pregnancy registration at first antenatal care visit Intention about

snacking

Self esteem and engagement of HW and PC

Beliefs - expectancy

Fathers buy foods on food list

Welcome package counseling

Training of HW

Television

Encouraging/supporting Mother-in-law

Mentoring of PC

Billboards

Food & Function cards

Self efficacy – making a food choice

Emotion

Wives consume snack

Wives snack foods on food list

Treat bag (other material) is being used (interest) Control –

having a choice

Awareness for snacking

Intention to buy foods on food list

Eat More, Eat Better Results Chain

Monitoring and Evaluation Plan Approach and Methods

1

3 2 4 5 6

Results from activities – deliverables Response to stimulus

Changes observed because of a set of activities outcomes achieved during the lifecycle of the project

Benefits achieved

The action planned Stimulus

Resources available to you




MonitoringSpace Evalua*onSpace


33

Strategy Application in BCC Main Questions Answered

Process/Implementation Monitoring

Ongoing/periodic assessment of outputs •  Service delivery:

To quantify what has been done; when, where, and how it has been done

•  Service utilization: Who has been reached; how the audience is reacting to the messages & activities

• Are activities being implemented according to schedule or as planned?

• What problems/bottlenecks have arisen during implementation?

• Which components of the program are or are not working?

Process Evaluation (Formative Evaluation)

Usually one-off activity, in-depth, diagnostic •  Studies the domains of service utilization

and service delivery

•  Completed by an independent evaluator

•  How well is the program functioning?

Inputs Activities Outputs

Outcomes Impact


34

Strategy Application in BCC Main Questions Answered

Outcome Monitoring/ Adequacy evaluation

Measures change in outcomes against BCC objectives (e.g., knowledge, self-efficacy, skills, attitudes, behaviors)

•  Did the desired changes in outcomes take place?

•  How much did knowledge, attitudes, and behavior change? Why?

Impact Evaluation

Benefits achieved because of the BCC intervention

Are changes in outcomes due to the BCC program?

•  Did communities with the program have better results than communities without the program?

•  Did people with greater exposure to the program experience better results than people with little or no exposure?


•  Quality of BCC activity implementation (process monitoring)

•  The extent to which your audience practices the desired behaviors (outcome monitoring)

•  Why people practice or do not practice the desired behaviors (outcome monitoring)

What to include in your BCC monitoring plan

•  Throughout the project (Process Monitoring)

•  At specified time points (Outcome Evaluation, based on BCC objectives)

•  Who measures what, when, and where

•  Budget for monitoring activities

•  Quantitative Methods •  (How many/What %?) •  Qualitative Methods

(Why/Why not?)

1Monitoring Approach

2Timeline and Roles and Responsibilities

3Methods and Indicators

Means, Methods and Tools

1Reports & Analysis

2Participatory

3Validation

•  Progress reports •  Implementation plans or work

plans •  Project documentation – log

frames, results chain, •  Records (low technology)–

logs, inventory, patient forms, meeting minutes etc.

•  Management information system (high technology environments)

•  Focus groups (staff, clients) •  Steering meetings •  Stakeholder meetings

•  Supervisory field visits •  Beneficiary/client surveys

Monitoring Plan

37

Method Example Application in BCC

Inventory Tracking Tracks products/materials in stock & distributed Can help assess demand/popularity Quantity, date, and geographical locations

Event Monitoring

Uses simple questionnaire to the collect number of attendees and their feedback after an event

Bounce-back Cards Simple and short questionnaires on the back of a post-card about the intervention (e.g., How much of the leaflet did you read? Do you think it is too long?)

Audience Satisfaction Surveys

Suggestion or comment boxes, observations, qualitative interviewing/FGD’s

Social Media Monitoring Web tools that record the quantity and quality of audience interactions online provided the BCC intervention has an online component

Real-time Activity Monitoring

High quality quantitative and qualitative data collection on ongoing BCC activities through mobile technologies (e.g. receiving real time client feedback, tracking health-center performance, drop-our rates correlations)

Monitoring Methods

Monitoring and Evaluation Indicators

38

Indicator Type Application in BCC

Unaided Message Recall Measures awareness and recall ability

Understanding of Messages To what does target audience appropriately understands the promoted messages

Operationalized as clarity or confusion

Trustworthiness of BCC ideas/content

Believe in the content of provided messages or ideas.

Operationalized as belief expectancy

Intention to Adopt a Behavior Does the target audience intend to do the behavior in near future

Behavior Extent to which the target audience is practicing the behavior

Indicator Examples for Eat More, Eat Better

Engaging home visits

Awareness of snacking

Emotion

INDICATORS TARGETS MEANS FOR ERIFICATION/ WHO WILL VERIFY

I learned new information about how food affects my child from the front line worker visit today

SMS 1-yes, 2-no 75% wives in catchment households respond ‘yes’

Questions on ‘pregnant women need to eat between meals’

Exit surveys at antenatal care (ANC) Field monitors

85% of clients surveyed correctly state that women need to eat between meals

Likert scale 1 to 5: ‘I feel that my husband is more involved in my food needs’

Exit survey at ANC Field monitors

65% of women respond a 4 or more

Measuring Changes in Behavior

What can affect quality of data?

What it is: Attributing BCC outcomes to BCC intervention How to mitigate risk: Use comparison/control groups or base-line data. Triangulate data, use outcome indicators that can only be linked to your BCC intervention

What it is: Some behaviors are more difficult to adopt during certain periods of time (e.g. easting a diversified diet after harvest) How to mitigate risk: Conduct surveys at the same time of year

What it is: When people overreport positive behaviors and underreport negative behaviors How to mitigate risk: Apply good principles for effective interviewing

1Self-report Bias

2Attribution

3Seasonality

•  Identify activities that are proving especially popular

•  Positive feedback from audiences on specific intervention components

•  New emerging national or international initiatives to capitalize on

Monitoring

41

Utilizing Monitoring results

Opportunities Threats •  Intervention not

performing against expected results

•  Negative media exposure •  Misconception and

misinformation around the BCC intervention in community

Good management and accountability mechanisms should be in place to take action for re-prioritization of activities (opportunities) and course correction (threats) if necessary


42

•  Process monitoring should be part of your on-going management activities

•  BCC SMART objectives should have clear indicators in the larger

monitoring and evaluation plan

•  If you have staff monitoring, make sure they are trained to collect BCC

process monitoring. Establish an efficient system for staff to quickly report

on issues and problems with BCC activities

•  Reserve a portion of overall method and evaluation budget for BCC

monitoring

•  When program evaluations are conducted include BCC indicators where

appropriate

Tips on Integrating BCC Monitoring and Evaluation with Program Monitoring and Evaluation


43

Checklist

Step 8 BCC Monitoring and Evaluation (M&E) Completed

M&E Elements discussed and approved by Brain Trust 1.  Theory of Change 2.  BCC Objectives 3.  Monitoring Framework

√

BCC M&E Plan that includes 1.  M&E Framework and Approach 2.  Indicators and Methods 3.  Roles and Responsibilities 4.  Budget and Timelines 5.  Decision and Action System

√

Overview of the BCC 8-Step Process

1

3 2 4 5 6

Overview BCC 8-Step process

Step 1. Goals and

Objectives

Step 2. Desk

Review

Step 3. Client

Research

Step 4. Creative Design

Step 5. Pre-testing

creative

Step 6. Training

Step 7. Implement

Step 8. Monitor

and Evaluate

BCC Process

46

What have we learned

1. Four Key Principles

(1) Process that culminates in a communication strategy -  Systematic and strategic mix of activities and communication, NOT ONLY promotion -  Only start BCC if barriers are eliminated (2) Audience focused, (3) Evidence Based -  Focus on behaviors (behavior objectives) -  Make no assumptions of what the audience needs and wants – ask them through research -  At the heart of BCC design is the exchange. Look for insights that help you position the

desired behavior from their perspective -  Turn constraints into opportunity - design around constraints -  Do outcome monitoring – we are accountable

(4) Integration with program and services -  Embed BCC in service delivery process – be cognizant of provider time, skills for BCC

BCC Process

47

What have we learned

3. Resource constraints Invest time and money in BCC process – not doing so is a high risk/high gamble situation. Share your experiences with us! A multi-stakeholder engagement secures buy in for the BCC process – Brain Trust Human beings are creative – unleash creative potential of your team, draw on more cost-effective resources: freelance journalist, writers, graphic designers, industrial designers

2. Iterative process Distinctions between steps – gives structure to a complex process. Look to create synergies between the steps where possible! When faced with challenges, go back to your research insights and put the audience at the center of every problem you aim to solve

Sight and Life BCC Expertise

48

Sight and Life offers expert BCC advice & services to support:

•  Integrated Nutrition BCC program design, from research to implementation

•  BCC Nutrition formative research

•  BCC training

•  Development or Co-design of Nutrition BCC related courses

•  Strategic advice on demand generation within nutrition specific and sensitive

interventions

For more information contact us at: [email protected]

Join Sight and Life on the journey… for a world free of malnutrition Twitter | Facebook | LinkedIn | sightandlife.org Contact us at [email protected] Offices in Switzerland and India

Sources: IndiKit.net by People in Need. Accessed through: https://www.indikit.net/sector/1-food-security-and-nutrition Petr Schmied (2017). Behavior Change Toolkit for International Development Practitioners. Published by People in Need (PIN) Menon, S., Karl, J., & Wignaraja, K. (2009). Handbook On Planning, Monitoring And Evaluating For Development Results. UNDP Evaluation Office, New York, NY. Michie, S., Van Stralen, M. M., & West, R. (2011). The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implementation science, 6(1), 42. Rossi PH, Lipsey MW, Freeman HE. Et al. Evaluation: A systematic approach. Sage Publications: 2004. Photo credit: Michael Bloem

Behavior Change · and behavior change? Why? Impact Evaluation Bene!ts achieved because of the BCC intervention Are changes in outcomes due to the BCC program? • Did communities

Documents