Difference-in-Difference Development Workshop
Difference-in-Difference
Development
Workshop
Typical problem in proving causal effects
Using differences to estimate causal effects in experimental data (treatment+control groups)
Wish: ‘treatment’ and ‘control’ group can be assumed to be similar in every way except receipt of treatment
This may be very difficult to do
A Weaker Assumption is..
In absence of treatment, difference between ‘treatment’ and ‘control’ group is constant over time
With this assumption can use observations on treatment and control group pre- and post-treatment to estimate causal effect
Idea– Difference pre-treatment is ‘normal’ difference– Difference post-treatment is ‘normal’ difference + causal effect– Difference-in-difference is causal effect
Graphically…
y
Time
Treatment
Control
Pre- Post-
A
B
C
What is D-in-D estimate?
Standard differences estimator is AB But ‘normal’ difference estimated as CB Hence D-in-D estimate is AC Note: assumes trends in outcome variables the same
for treatment and control groups This is not testable Two periods (before and after) crucial
The Grand Experiment (Snow)
Water supplied to households by competing private companies
Sometimes different companies supplied households in same street
In south London two main companies:– Lambeth Company (water supply from Thames Ditton, 22
miles upstream)– Southwark and Vauxhall Company (water supply from
Thames)
In 1853/54 cholera outbreak
Death Rates per 10000 people by water company– Lambeth 10– Southwark and Vauxhall 150
Might be water but perhaps other factors Snow compared death rates in 1849 epidemic
– Lambeth 150– Southwark and Vauxhall 125
In 1852 Lambeth Company had changed supply from Hungerford Bridge
What would be good estimate of effect of clean water?
1849 1853/54 Difference
Lambeth 150 10 -140
Vauxhall and Southwark
125 150 25
Difference -25 140 -165
Card and Krueger (1994)
Basic microeconomic theory of the firm: factor demand curves slope downwards.
Hence, if minimum wages are binding, we would expect employment to fall if minimum wage is raised.
Natural experiment: New Jersey raising its minimum wage from $4.25 to $5.05 on 1 April 1992 while the minimum wage in neighbouring Pennsylvania remained unchanged.
Data: wages and employment in 65 fast-food restaurants in Pennsylvania and 284 in New Jersey in Feb/March 1992 (i.e. before the rise in the NJ minimum wage) and in Nov/Dec 1992 (i.e. after the rise).
Difference-in-difference design to investigate the impact of minimum wages on employment.
What data we have?
698 observations– Sheet: an identifier for each restaurant (each has
two observations, pre- and post-)– NJ: dummy for whether a NJ restaurant– After: dummy for whether post- observation– Njafter: nj*after– Fte: full-time equivalent employment– Dfte: change in full-time equivalent employment
Tabulate command
Tabulate in STATA:– tabulate var (or tab var) – just a simple table– tab var, g(newvar) – generating a new variable– tab var, su(othervar) – summarising some other
variable
Let’s get our first DinD estimator
tabulate nj after, su(fte) means
Before After Diff
PA 20.3 18.3 -2.0
NJ 17.3 17.5 +0.2
Diff +3.0 +0.8 ??
Going from means to statistics
reg dfte nj
_cons -2.046154 1.062988 -1.92 0.055 -4.136864 .0445564 nj 2.328724 1.178371 1.98 0.049 .0110768 4.646372 dfte Coef. Std. Err. t P>|t| [95% Conf. Interval]
Total 25772.7145 348 74.0595245 Root MSE = 8.5701 Adj R-squared = 0.0083 Residual 25485.8728 347 73.4463192 R-squared = 0.0111 Model 286.841779 1 286.841779 Prob > F = 0.0489 F( 1, 347) = 3.91 Source SS df MS Number of obs = 349
… and with robust standard errors
Coeff SE P-value
OLS 2.329 1.17 0.049
Robust OLS 2.329 1.47 0.114
reg dfte nj reg dfte nj, robust
An alternative specification …
reg fte nj after njafter, robust
_cons 20.3 1.501537 13.52 0.000 17.3519 23.2481 njafter 2.328724 1.930761 1.21 0.228 -1.46211 6.119558 after -2.046154 1.788875 -1.14 0.253 -5.55841 1.466103 nj -2.998944 1.591452 -1.88 0.060 -6.123581 .1256939 fte Coef. Std. Err. t P>|t| [95% Conf. Interval] Robust
Root MSE = 8.9641 R-squared = 0.0089 Prob > F = 0.2682 F( 3, 694) = 1.32Linear regression Number of obs = 698
Alternative specifications…
reg fte nj after njafter, cl(sheet) xtreg fte nj after njafter, fe i(sheet)
Any key differences? Should there be any?
Suppose we’d like to observe many estimations
STATA commands for results-sets Guy named Roger Newson
– estimates store– outreg (works mostly with regressions)– parmest/parmby
Summary
A very useful and widespread approach Validity does depend on assumption that
trends would have been the same in absence of treatment
Can use other periods to see if this assumption is plausible or not
Uses 2 observations on same individual – most rudimentary form of panel data