1 What are Monitoring and Evaluation? How do we think about M&E in the context of the LAM Project?

1

What are Monitoring and Evaluation?

How do we think about M&E in the context of the LAM

Project?

2

Objectives of Monitoring & Evaluation

Collect, analyze and use accurate and reliable information

Brazil

3

Objectives of Monitoring & Evaluation

• Improve program planning

• Improve program performance

• Improve program management

• Determine outcomes of activities

• Evaluate program impact on beneficiaries

4

Characteristics of Monitoring

• Occurs throughout project on a regular basis

• Provides early indication of progress, or a lack of progress

• Provides periodic oversight of activity implementation

5

Characteristics of Monitoring

• Can be tasked to any staff, not a specific evaluation staff person

• Informs a project about strengths & weaknesses: do we refine our strategy?

6

Characteristics of Monitoring

• Determines whether activities are being performed correctlyTo what extent are planned

intervention activities being realized? What services are provided, to

whom, when, how often, for how long and in what context?

7

Is Monitoring Supervision?

• We monitor an intervention

• We supervise an individual

8

Characteristics of Evaluation

• Time-bound• Systematic and objective• Assess performance and impact• Carried out selectively – costly & time

consuming• Process & Impact

Process: organization & managementImpact: effects on beneficiaries

9

Types of Evaluations1. Baseline Surveys: provides project with

information about the target population at beginning of project intervention

Knowledge about breastfeeding, complementary feeding, maternal nutrition, and LAM (BCM/LAM)

Practices related to BCM/LAM Cultural beliefs related to BCM/LAM Household-based in the community Large, representative sample

10

Types of Evaluations

2. Endline Surveys: provides project with information about the target population at the end of project intervention

The endline survey is compared to the baseline and like the baseline it is:

- Household-based in the community

- Large, representative sample

11

Types of Evaluations

Endline Surveys: The endline asks the following questions:Did we meet our objectives?Did we have an impact: is there a measurable

difference in people’s knowledge & practice?Where were we strong and where were we

weak?What problems remain to be addressed?

12

Types of Evaluations

3. Performance Monitoring: measures knowledge & skill levels of individuals trained in the field

Results are used to refine training strategy & define future training needs

Results are used for better supervisory visits

13

Types of Evaluations

Performance Monitoring: Ideally, performance monitoring

evaluations are not necessary─ An effective supervisory system

would include on-going evaluation of field worker’s knowledge & skills (counseling, methodologies) following training

14

Monitoring & Evaluation Indicators

How do we track & measure progress over time?

Guatemala

15

What is an Indicator?

• A variable

• That measures

• One aspect of a program

16

What is an Indicator?

• A measure of whether you are heading in the right direction

• A measure of what your program has achieved

• A measure of the behavior you’re monitoring

17

An appropriate set of indicators will include at least one for

each significant element of the program

18

Characteristics of Indicators

• Quantitatively or qualitatively measurable

• Relevant to the goals they represent

• Objectively verifiable & reliable

• Meet international professional standards

• Understandable & appreciated by project participants and other stakeholders

• Clarity on the appropriate levels: clinic-specific, country wide

19

Characteristics of Indicators

• Clarity on the appropriate levels

Is the indicator clinic-specific? Is the indicator region-specific? Is the indicator country wide?

20

Developing Indicators

• Establish during initial program planning

• Reflect overall objectives

• Link to behaviors

• Include with baseline

21

How To Measure Indicators?

• Knowledge, practice and coverage surveys (KPC)

• Health Facility Records

• Health Information Systems

• Formative research to identify behaviors & indicators

22

LAM Indicators

1. Exclusive Breastfeeding Rate, Infants 0

- < 6 months

Guatemala

23

LAM Indicators

Exclusive Breastfeeding Rate, Infants 0 - < 6 months

Age range for LAM to be applied Eligible LAM users Optimal infant feeding message Includes ALL infants in the age range Can also be written 0 – 5 months

24

Exclusive Breastfeeding Rate, Infants 0 - < 6 months

• WHO indicator using 24 hour recall• Reflects women eligible to use LAM• 1995: 46.1%• 1998 – 1999: 38.8%

Decrease of 7.3%

Source: Guatemala DHS Reports

25

LAM Indicators2. LAM Rate:

Proportion of eligible women (infants less than 6 months), who choose LAM as a method of birth spacing

Altiplano, Bolivia

26

LAM Rate

• Formula: Current LAM users divided by all women with infants 0 < 6 months

• Accuracy: A true estimate of LAM use by eligible women

• Data Source: Service statistics (MIS) & Household-level survey (DHS)

27

LAM Indicators

3. LAM User Rate:

Proportion of all women of reproductive age who use a modern contraceptive method (few eligible to use LAM) who choose LAM as a method of birth spacingLlano, Bolivia

28

LAM User Rate

• Formula: Current LAM users divided by all women of reproductive age who use a modern method of family planning

• Accuracy: Underestimates actual LAM use by eligible women

• Data Source: Service statistics (MIS) & DHS

29

LAM Indicators

4. Appropriate LAM Rate: Proportion of women who give birth in a given period of time who consciously & deliberately accept LAM as a birth spacing method and meet the 3 LAM criteria Guatemala

30

Appropriate LAM Rate

• Formula: Number of women who use LAM as a birth spacing method and meet the 3 LAM criteria divided by the total number of women with infants 0 < 6 months

31

LAM Indicators

5. Model LAM Rate: Proportion of women who give birth in a given period of time who consciously & deliberately accept LAM as a birth spacing method, meet the 3 LAM criteria, and know the 3 LAM criteria

Jordan

32

Model LAM Rate

• Formula: Number of women who use LAM as a birth spacing method, meet the 3 LAM criteria, and know the 3 LAM criteria divided by the total number of women with infants 0 < 6 months

33

Denominators• Exclusive Breastfeeding Rate:

Infants 0 < 6 months• LAM Rate:

Women with infants 0 < 6 months• LAM User Rate:

Women of reproductive age using a modern contraceptive method

• Appropriate LAM Rate: Women with infants 0 < 6 months

• Model LAM Rate: Women with infants 0 < 6 months

34

LAM Monitoring & Evaluation Strategies

• Integration of LAM as a contraceptive method into the national MIS

• LAM acceptors integrated into national MOH/FP data

35

Lessons Learned

• Increase and sustain LAM UsersOptimal breastfeeding

practices Integrated & supportive

environment

36

Lessons Learned• Data Collection

Monitor clinics for accurate data collection

Absolute LAM numbers do not reflect LAM success – need proportion

Additional information

- Age of infant

- Family Planning Transition information

- Length of time using LAM

37

Lessons Learned

• Integrate the concept of MonitoringUtilize results of

supervisory checklist for program management

38

Next Steps

• Review monthly and quarterly data for changing LAM trends

• Review MIS and supervision data on a regular basis

• Conduct regular review meetings and analysis for program management