Using Case-based Methods for Evaluating Complexity in the Health Sector

Using case-based

methods for

evaluating complexity

in the health sector

Anne LaFond John Snow, Inc. (JSI) American Evaluation Association November 2015 Chicago, Illinois

Overview

Why focus on case

studies?

What are case studies?

Illustrations

Overview

Overview

• Complexity: health systems and health interventions

• The setting matters

• Greater depth of understanding needed

The Context

Going beyond simple

solutions

Replicating success

Going to scale

Overview

• Research method or an empirical inquiry that investigates a contemporary phenomenon within its real-life context

• When the boundaries between phenomenon and context are not clearly evident, and

• In which multiple sources of evidence are used.

The Case

Study

The case as an

example for learning

about interventions in

context

Mixed methods

Comparative cases to

increase potential

learning and relevance

Overview

Case study research comprises an all encompassing method – covering the logic of design, data collection techniques, and specific approaches to data analysis

The Method

More variables than

data points.

Relies on multiple

sources of evidence,

requiring the data to

converge or triangulate.

Theoretical propositions

to guide data collection

and analysis.

6

Not…..

7

Case-based research is

Lessons from the 2013 IOM

Evaluation of PEPFAR:

Use of Case-Based

Approaches

Bridget Kelly

WHAT DRIVES ROUTINE

IMMUNIZATION SYSTEM

PERFORMANCE IN AFRICA? A MIXED METHODS COMPARATIVE CASE STUDY

APPROACH

THE AFRICA ROUTINE IMMUNIZATION SYSTEM

ESSENTIALS PROJECT (ARISE)

Overview

• History of investment in national immunization programs in Africa.

• Why does coverage improve in some settings and not in others?

Context

The Complexity:

Many variables

associated with high

levels of

immunization

coverage but little

understanding of the

dynamics of

coverage

improvement.

Conceptual Framework: Causal Loop

Diagram for Demand for Immunization

Dynamics

Rwashana A, Williams D, Neema S. Systems dynamics approach to immunization healthcare issues in developing countries: a case study of Uganda, Health informatics J 2009.

Penta3 Coverage for All Districts

(2008 and 2010, Ghana)

Overview

• Mixed methods, multiple case studies

• Case selection: Countries with DPT3 coverage improvement

• Nested case: Three districts per country with recent DPT3 coverage improvement and one district with steady or unchanged coverage

Methods

Overview

• Focus on district level and the dynamics of change

• Mixed methods: Coverage trend analysis and verification; RI Situation Analysis (service readiness); Open-ended, semi structured key informant interviews district, facility, community levels; FGD; field observation.

• Iterative field level analysis by district • Grounded theory building • Interdisciplinary team: Routine

immunization expert, researchers, country program

• Cross district case consistency • Close collaboration with technical and

country experts • Country level validation (stakeholder

workshop)

Design &

Data

Collection

Theory building

Cadre of Community

Centered Health

Workers

Health System and Community Partnership

Regular Review of

Program and Health Worker

Performance Immunization

Services Tailored to Community

Needs

Four Direct Performance Drivers

LaFond, et.al, 2014

Take vaccination into heart of

the community

More workers, build trust, local support, vaccine

supply.

Raised

awareness,

improved

access,

increased use

Cadre of Community-centered Health Workers

Effect

Mechanism

Transformational step

Ethiopia

“75% of Health Extension Workers’ (HEW) time is spent on house-to house to do sensitization and deliver services. As a result of this effort the awareness of the community about the importance of immunization has improved. This motivates HEWs to become more dedicated to their work.”

“Community Health Nurses are also very much in touch with the community. For instance the new outreach site being introduced was at the request of the community members and the Community Health Nurses are in the process of discussing with them the most convenient day for the Child Welfare Clinics in their community.”

21

Challenges • Quality and availability of

coverage data to select cases

• Consistent approach across country cases

• Intensive fieldwork stage with iterative analysis and theory building

What did we learn from use of case-based

approaches?

• Appropriate and relevant for learning about and within complexity

• Expands and enriches the tools available beyond indicators and metrics

• Lends credibility through techniques to help understand “how” and “why” questions.

• Triangulation is critical for analysis and interpretation

Guidance

• No need to be intimidated or apologetic

• What does “best practice” mean in this context?

• Fit-for-purpose

• Informed and intentional

• Well-documented

information For more information,

please contact:

[email protected]

thank you ARISE was funded by

the Bill & Melinda Gates Foundation

and implemented by

Anne LaFond, Natasha Kanagat, Jenny Sequeira, Robert Steinglass, Rebecca Fields

(JSI). Judith Justice (Ethiopia), Cheikh Niang, (Cameroon) and Ann Larson (Ghana)

in collaboration with

Sangeeta Mookherji, George Washington

University

For more information about JSI’s

research, monitoring, evaluation &

health information work, contact

[email protected]

Follow @jsihealth