Understanding causal pathways within health systems policy evaluation through mediation analysis: an application to payment for performance (P4P) in Tanzania Laura Anselmi, Peter Binyaruka, Masuma Mamdani Josephine Borghi Payment for Performance: a health systems perspective A workshop for scientists and practitioners Dar es Salaam, 26 November 2015
14
Embed
Understanding the causal pathways within health systems policy evaluation through mediation analysis: an application to payment for performance (P4P) in Tanzania - Laura Anselmi
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Understanding causal pathways within health systems policy evaluation through mediation
analysis: an application to payment for performance (P4P) in Tanzania
Laura Anselmi, Peter Binyaruka, Masuma Mamdani
Josephine Borghi
Payment for Performance: a health systems perspective
A workshop for scientists and practitioners
Dar es Salaam, 26 November 2015
Rationale
• Programme evaluation has mainly focused on measuring the impact on outcomes, with little attention to the causal pathways
• The relevance of undertaking process evaluation to be integrated with outcome evaluation is increasingly recognised
• Process evaluation is particularly relevant for complex interventions: It increases confidence in the plausibility of outcome effects
It increases the external validity of the evaluation
• Process evaluations have been carried out, but without formal assessment of causal pathways
Causal mediation analysis
• A causal mechanism is a process through which a programme or intervention influences an outcome
• It can be identified by specifying intermediate outcomes or variables(mediators) that are on the causal pathway between intervention and outcome
• Causal mediation analysis has been employed to test change pathways within the evaluation of public health programmes
• Mediators have been limited to individual level indicators, psychological or physical
• Health system mediators which are relevant to the evaluation of health systems or services interventions have not been considered
Sequential ignorability b):Given treatment status and pre-treatment confounders the mediators are ignorable (no confounders affecting both mediators and outcome)
Mean patient satisfaction with interpersonal care (0-1 scale) (1) 6.7***BS
Mean kindness ranks for HW at delivery (0-1 scale) (1) 10.3***BS
* p<0.10, ** p<0.05, *** p<0.01 , BS: Significant at 5% level with Bonferroni adjusted p-value for multiple outcomes: Bonferroni adjusted p-value
Financing 0.0047, Governance 0.0017, Human resources 0.0414, (1) Out of all women delivering in a HF in same catchment area (2) equipment,
vaccines, drugs, medical supply
Results: P4P direct and indirect effect (Step 3)
• Facility based delivery
P4P total effect: +8.2%
P4P indirect effect through mean of all financing indicators: +1 %
P4P indirect effect through reduction in stock-out of oxytocine: +1.8 %
P4P direct effect: +7.2% or +6.4%
• Delivery in public health facility
P4P total effect: +6.5 %
P4P indirect effect through reduction in stock-out of oxytocine: +1.9 %
P4P direct effect: +4.6%
• Uptake of two doses of IPT during pregnancy
P4P total effect: +10.3 %
P4P indirect effect through reduction in last supervision being 90 days ago: +1.5 %
P4P direct effect: +8.8%
Sensitivity analysis
• Semiparametric mediation analysis to quantify the sensitivity of results to the assumption of no confounders affecting mediator and outcome
• Analysis carried out at the HF level
• Estimate a logit model for binary mediators
• Multiple hypothesis testing adjustment of p-values for families of mediators
• DiD with district fixed effects
Summary: Indirect effects of P4P
• P4P significantly affects a number of financing, governance and human resources factors which could potentially mediate its effect on maternal care outcomes
• The effect of P4P on the reduction of oxytocine injection stock-out mediates the effect of P4P on institutional deliveries (22%) and deliveries in a public health facility (30%)
• The effect of P4P on the frequency of supervisions mediates 15% of the effect of P4P on the uptake of at least two doses of IPT during pregnancy
Some reflections on mediation analysis
• Mediation analysis rarely applied using difference in difference
• Quasi–experimental setting + Difference-in-Difference provide confidence that the assumption of no pre-treatment confounders is satisfied
• But how plausible are the assumptions of no confounders between mediator and outcome?
• Data availability limits testing pre-trends for mediators
• Possible differences in results according to the level of the analysis
• Little analysis of the role of individual level factors or other moderating factors
• Possibly simplified description of the causal chain
Conclusions
• Mediation analysis is helpful to quantify causal direct and indirect effects and the relative relevance of change pathways
• It requires assumptions to identify causality and these can not be tested formally
• Quantify the P4P indirect effects helps in thinking about relative cost-effectiveness compared to alternative interventions