21.Technical Dif in Dif Premand Holla ENG PP

7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP

1/44

www.worldbank.org/hdchiefeconomist

TheWorldBank

HumanDevelopment

Network

Spanishmpact

!valuation"und

www.worldbank.org/hdchiefeconomist


2/44

DIFFERENCE

S

This material constitutes supporting material for the #mpact !valuation in $ractice# book. This additional material is made freel% but please acknowledge its use as follows& 'ertler( $. ).* +artine,( S.( $remand( $.( -awlings( . B. and hristel +. ). 0ermeersch(1232( mpact !valuation in $ractice& 4ncillar% +aterial( The World Bank( Washington D 5www.worldbank.org/ieinpractice6. The content of this presentation re7ects the views of the authors and not necessaril% those of the World Bank.

Technical Track Session III

IN

&PANELDATA

DIFFERENCE


3/44

Structure of this session

When do we use Di8erences9in9Di8erences: (Dif-in-Dif or DD)

!stimation strateg%& ; wa%s to lookat DD

!


4/44

When do we use DD3

We can=t alwa%s randomi,e!.g. !stimating the impact of a >past? program

4s alwa%s( we need to identif%o which is the group a8ected b% the polic% change

(treatment), ando which is the group that is not a8ected (comparison)

We can tr% to @nd a >natural e


5/44

! wa"s to looks atDD

1

n a Bo< 'raphicall%

n a-egression


6/44

The box#rou$ a%ected "

the $olic" chan'e(treatment)

#rou$ that is not

a%ected " the$olic" chan'e(comparison)

4fter theprogram start

Y1 (ui) | Di=1 Y1 (ui) | Di=0

Before theprogram start

Y0 (ui) | Di=1 Y0 (ui) | Di=0

(Y1|D=1)-(Y0|

D=1)

(Y1|D=0)-(Y0|

D=0)

DD=(Y1|D=1)-(Y0|D=1)! - (Y1|D=0)-(Y0|

D=0)!


7/44

Graphically

Dutcome

0ariable

Y0 | Di=1

Y1| Di=0Y0 | Di=0

TE2 TE3 Time

!nrolled

Not

enrolled

!stimated

4T!

Y1 | Di=1

DD=(Y1|D=1)-(Y

0|D=1)! - (Y

1|D=0)-(Y

0|

=


8/44

Re'ression (for ) ti*e$eriods+

1

0

1

0

1 0 1 0

.1( 1) .1( 1) .1( 1).1( 1)

( | 1) ???

( | 1) ???

( | 0) ???

( | 0) ???

( ( | 1) ( | 1)) ( ( | 0) ( | 0)) ???

it i i it

i i

i i

i i

i i

i i i i i i i i

Y t D t D

E Y D

E Y D

E Y D

E Y D

DD E Y D E Y D E Y D E Y D

= + = + = + = = +

= =

= =

= =

= =

= = = = ==


9/44

Re'ression (for ) ti*e$eriods+

1 1

0 0

1 1

0

.1( 1) .1( 1) .1( 1).1( 1)

( | 1) .1 .1 .1.1 ( | 1)

( | 1) .0 .1 .0.1 ( | 1)

( | 0) .1 .0 .1.0 ( | 0)

( | 0) .0 .1

it i i it

i i i i

i i i i

i i i i

i i

Y t D t D

E Y D E D

E Y D E D

E Y D E D

E Y D

= + = + = + = = +

= = + + + + = = + + +

= = + + + + = = +

= = + + + + = = += = + + 0

1 0 1 0

.0.0 ( | 0)

( ( | 1) ( | 1)) ( ( | 0) ( | 0))

( )

i i

i i i i i i i i

E D

DD E Y D E Y D E Y D E Y D

+ + = =

= = = = =

= +

=


10/44

If we ha,e *ore than) ti*e $eriods-'rou$s.

1 1

where is the intensity of the treatment

in group in p

.1( ) .

eriod

)

.

1( .T I

it it

it

itY t

D D

i

i D

t

= =

= + = + = + +

We use a regression with @


11/44

Identi/cation in DD

The identi@cation of the treatment e8ectis based on the inter9temporal variationbetween the groups.

I"e" hanges in the outcome variable Y

over time( that are speci@c to thetreatment groups.

I"e")umps in trends in the outcome

variable( that happen onl% for thetreatment groups( not for the comparisongroups( e


12/44

Warnin'sDD/ @


13/44

Warnin's

DD attributes an% di8erences in trendsbetween the treatment and controlgroups( that occur at the same time asthe intervention( to that intervention.

f there are other factors that a8ect thedi8erence in trends between the two

groups( then the estimation will bebiasedG


14/44

Violation of Equal TrendAssumption

Dutcome

0ariable

TE2 TE3Time

!nrolled

Notenrolled

Y0 | Di=1

Y1 | Di=0Y0 | Di=0

!stimate

d mpact

Y1 | Di=1

Bias


15/44

Sensiti,it" anal"sis for di%0in0di%$erform a >placebo? DD( i.e. use a >fake?

treatment groupo !


16/44

Fre1uentl" occurrin' issuesin DD$articipation is based in di8erence in outcomes prior

to the intervention. !.g. >As&ene'ter dip selectioninto treatment in7uence b% transitor% shocks on pastoutcomes 54shenfelter( 3JKL* ha% et al.( 122 6.

f program impact is heterogeneous across individual

characteristics( pre9treatment di8erences inobserved characteristics can create non9paralleloutcome d%namics 54badie( 1226.Similarl%( bias would occur when the si,e of theresponse depends in a non9linear wa% on the si,e of

the intervention( and we compare a group with hightreatment intensit%( with a group with low treatmentintensit%When outcomes within the unit of time/group arecorrelated( S standard errors understate the st.

dev. of the DD estimator 5Bertrand et al.(


17/44

E2a*$le 3

Schoolin' and laor *arketconse1uences of school

constructionin Indonesia. e,idence fro* an

unusual $olic" e2$eri*ent

Esther Du4o5 6ITA*erican Econo*ic Re,iew5 Se$t )773


18/44

Research 1uestionsSchool

infrastructure

!ducational

achievement

!ducationalachievement:

Salar% level:

What is the economicreturn on schooling:


19/44

Pro'ra* descri$tion3JK;93JKL&The ndonesian government builtA3(222 schools e.ui/a'ent to one sc&oo' per00 c&i'dren et%een and 12 3ears o'd

The enrollment rate increased from AJ toLbetween 3JK; and 3JKL

The number of schools built in each region

depended on the number of children out ofschool in those regions in 3JK1( before thestart of the program.


20/44

Identi/cation of thetreat*ent e%ect

B% regionThere is variation in the number of schools receivedin each region.

There are 1 sources of variations in theintensit% of the program for a given individual&

B% ageo

hildren who were older than 31 %ears in 3JK1did not bene@t from the program.o The %ounger a child was 3JK1( the more it

bene@ted from the program Obecause she spentmore time in the new schools.


21/44

Sources of data

3JJ population census.ndividual9level data on&o birth dateo 3JJ salar% levelo

3JJ level of educationThe intensit% of the building programin the birth region of each person inthe sample.

Sample& men born between 3J2 and3JK1.


22/44

A /rst esti*ation of thei*$actSte$ 3. Let8s si*$lif" the $role*and esti*ate the i*$act of the$ro'ra*9

We simplif% the intensit% of the

program& highor low

o oungcohort of children whobene@tted

o ldercohort of children who did notbene@t

We simplif% the groups of childrena8ected b% the program


23/44

Let8s look at the a,era'e ofthe

outco*e ,ariale :"ears ofschoolin';Intensit" of the 79!? 793) DD(797@+


24/44

Intensit" of the

3193K5older cohort6

L.21 J.M 039!

793) DD(797@+

Let8s look at the a,era'e ofthe

outco*e ,ariale :"ears ofschoolin';


25/44

Placeo DD(Cf9 $9>@5 Tale !5 $anel


26/44

Ste$ ). Let8s esti*ate this with are'ression

.( . ) .( . )

education level of person i in

region j in cohort k

1 if the person was born in a region with a

high program intensity

1 if the pe

ijk j k j i j i ijk

ijk

j

i

S c P T C T

with

S

P

T

= + + + + +

=

=

= rson belongs to the young cohort

dummy variable for region j

cohort fi!ed"effectdistrict of birth fi!ed"effect

error term for person in region in cohort

j

k

j

ijk

C

i j k

=

==

=


27/44

Ste$ !. Let8s use additionalinfor*ation

#e will use the intensity of the program in each region$

where % the intensity of building activity in region j

% a vector of regional characteristi

.( . ) .( . ) = + + + + +

j

j

j j i j iijk k ijk

P

C

S c P T C T

&' &'

& &

cs

#e estimate the effect of the program for each cohort separately$

where a dummy variable for belonging to cohort i

.( . ) = ==

= + + + + + i

j j i j iijk k l l ijk l l

d

S c P d C T


28/44

Pro'ra* e%ect $ercohort

l

4ge in 3JKM

For y B De$endent ,ariale B


29/44

For y B De$endent ,ariale BSalar"


30/44

Conclusion-esults& "or each school built per 3222

students*o The average educational achievement

increase b% 2.319 2.3J %earso

The average salaries increased b% 1.A O.M

+aking sure the DD estimation isaccurate&o 4 placebo DD gave 2estimated e8ecto Ise various alternative speci@cationso heck that the impact estimates for each

age cohort make sense.


31/44

E2a*$le )

Water for Life.The I*$act of the Pri,atiationof

Water Ser,ices on Child6ortalit"

Seastin #aliani5 ni,ersidad de San AndrsPaul #ertler5 C


32/44

Chan'es in water ser,icesdeli,er"

3703T"$e of $ro,ision*ethods Nu*er of*unici$alities

4lwa%s public 3JA;J.K

4lwa%s anot9for9pro@tcooperative

3M; 1L.J

onvertedfrom public to private

3;L1K.J

4lwa%s private 3 2.1

No information 3A ;.1

Total ==377


33/44

Figure1: Percentage of Municipalities with Privatized

Water Systems

0

5

10

15

20

25

30

1990 1991 1992 1993 1994 1995 1996 1997 1998 1999

Year

%ccum

ulate

id f


34/44

se outside factors todeter*ine who $ri,aties

The political part% that governed themunicipalit%o "ederal( $eronist % $rovincial parties&

allowed privati,ationo -adical part%& did not allow privati,ation

Which part% was in power/whether the

water got privati,ed did not depend on&o ncome( unemplo%ment( ineCualit% at the

municipal levelo -ecent changes in infant mortalit% rates

i


35/44

it it it t i ity( d) * + ,

infant mortality rate in munic. in year

dummy variable that takes value 1 if

municipality has private water provider in year

vector of covariates

= + + + +

=

=

=

it

it

it

where

y i t

dI

i t

x

x

t

i

fi!ed time effect

fi!ed municipality effect

=

=

Re'ression


36/44

DD lt P i ti ti


37/44

DD results. Pri,atiationreduced infant *ortalit"

S iti it l i


38/44

Sensiti,it" anal"sis3

1

heck that the trends in infant mortalit% were

identical in the two t%pes of municipalitiesefore privati,ationo ou can do this b% running the same eCuation(

using onl% the %ears before the intervention O thetreatment e8ect should be ,ero for those %ears

o "ound that we cannot rePect the null h%pothesisof eCual trends between treatment and controls(in the %ears before privati,ation

heck that privati,ation onl% a8ects mortalit%through reasons that are logicall% related towater and sanitation issues."or e


39/44

I*$act of $ri,atiation on death fro*,arious causes

DD on co**on su$$ort

Pri ati ation has a lar'er e%ect


40/44

Pri,atiation has a lar'er e%ectin $oor and ,er" $oor

*unici$alities than in non0$oor*unici$alities6unici$alities

A,era'e*ortalit" $er

3775 1990

Esti*atedi*$act

chan'ein

*ortalit"Non9poor .3 2.32 F

$oor K.3 92.KAKQQQ 932.K

0er% poor J.MA 91.13MQQQ 91;.M

C l i


41/44

Conclusion

$rivati,ation of water services is associatedwith a reduction in infant mortalit% of 9K.

Ising a combination of methods( we found that&

The reduction of mortalit% is&o Due to fewer deaths from infectious and

parasitic diseases.o Not due to changes in death rates from reasons

not related to water and sanitation

The largest decrease in infant mortalit%occurred in low income municipalities.

R f


42/44

References4badie( 4. 51226. >Semiparametric Di8erence9in9Di8erences!stimators?( 4e/ie% o #conomic 5tudies( K1.

4shenfelter( . 53JKL6& >!stimating the !8ect of Training$rograms on !arnings(? *&e 4e/ie% o #conomic and 5tatistics,A2( 3. pp. MK9K"

ha%( Ren( +c!wan( $atrick and +iguel IrCuiola 51226& >The

central role of noise in evaluating interventions that use testscores to rank schools(? American #conomic 4e/ie%( J( pp.31;K9L.

'ertler( $aul 5122M6& >Do onditional ash Transfers mprovehild Health: !vidence from $-'-!S4=s ontrol -andomi,ed

!Schooling and abor +arket onseCuences ofSchool onstruction in ndonesia& !vidence "rom an Inusual$olic% !Water forife& The mpact of the $rivati,ation of Water Services on hild+ortalit%(?6ourna' o 7o'itica' #conom3, 0olume 33;( pp. L;9312"

Bertrand( +.( Du7o( !. and S. +ullainathan 5122M6. >How muchshould we trust di8erences9in9di8erences !stimates:(?8uarter'3 6ourna' o #conomics.


43/44

Thank JouThank Jou


44/44

K & AK & A

21.Technical Dif in Dif Premand Holla ENG PP

Documents