Collecting High-Quality Data. M&E Plan FrameworkIndicators Data Collection Data Quality Data Use and Reporting Evaluation Strategy Budget Part of the.

Collecting High-Quality Data

Part of the M&E Plan

Data Quality:

How well our M&E data “tell the true story.”

Data Quality:

How well our M&E data “tell the true story.”

Actual

Results

Actual

ResultsReported

Results

Reported

Results

What is Data Quality?

?

Adapted from: http://www.cpc.unc.edu/measure presentation by Win Brown, USAID/South Africa, School of Health Systems and Public Health, Monitoring and evaluation of HIV/AIDS Programs, Data Quality; March 2, 2011.

Elements of Data Quality

Validity Data measure what they are supposed to measure.

ReliabilityEveryone defines, measures, and collects data the same way—all the time.

CompletenessData include all of the values needed to calculate indicators. No variables are missing.

Precision Data have sufficient detail. Units of measurement are very clear.

Timeliness Data are up to date. Information is available on time.

IntegrityData are true. The values are safe from deliberate bias and have not been changed for political or personal reasons.

Adapted from: http://www.cpc.unc.edu/measure presentation by Win Brown, USAID/South Africa, School of Health Systems and Public Health, Monitoring and evaluation of HIV/AIDS Programs, Data Quality; March 2, 2011.

Validity and Reliability: Hitting the Target

NOT ValidNOT Reliable

Reliable butNOT valid

X X X X X X X X X X

XXX XXXXXXX

Reliable AND Valid!!!

XXX

XXXXXXX

Adapted from: http://www.cpc.unc.edu/measure presentation by Win Brown, USAID/South Africa, School of Health Systems and Public Health, Monitoring and evaluation of HIV/AIDS Programs, Data Quality; March 2, 2011.

Precision

Indicator Description

Treatment success rate Cure PLUS completed treatment

Indicator Description

Treatment success rate All patients in the cohort:- with smear conversion and - who completed full coursenegative smear at 5 months of treatment but do not PLUS meet cure definition

DIVIDED BY:Total number of smear-positive patients in the treatment cohort

MULTIPLIED BY: 100

Which indicator description will yield the most precise result?

Completeness

Often related to:• ease of collecting and reporting data• data sources• training

NGOpartner

Number of members participating in social

mobilizationComment

Friends of TB Not available Participant log not maintained

TB Matters 10

TB Helpline Not available Unclear how to determine who actively participated

Stop TB NOW! 12

TOTAL ???

Timeliness

1. Are we meeting internal and external deadlines?• Communicate expectations clearly.• Offer support to collect/analyze where needed

(budget?).

2. Are we analyzing results often enough to be useful for program management?• The sooner we know about a problem, the sooner

we can fix it!

Integrity

• Often difficult and sensitive topic.• Routine verification from the start can help

avoid bias of any kind.

• A partner submits perfect reports every month on time and meets or exceeds targets.

• A partner submits reports with a few errors every month, sometimes 1-2 days late; usually meets or comes close to targets.

Which data would you verify and why?

Data Quality Plans

How can we ensure: Strategy Resources

Validity

Reliability

Completeness

Precision

Timeliness

Integrity

Helpful Resources

MEASURE Evaluation Project

Data Quality

http://www.cpc.unc.edu/measure/tools/monitoring-evaluation-systems/data-quality-assurance-tools

Data Use

http://www.cpc.unc.edu/measure/our-work/data-demand-and-use