Heterogenous Employment Effects of Job Search Programmes: A Machine Learning Approach Michael Lechner (jointly with Michael Knaus & Anthony Strittmatter) Swiss Institute for Empirical Economic Research (SEW) University of St. Gallen | Switzerland | December 2017
42
Embed
Heterogenous Employment Effects of Job Search Programmes ... London kurz.pdfHeterogenous Employment Effects of Job Search Programmes: A Machine Learning Approach Michael Lechner (jointly
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Michael Lechner (jointly with Michael Knaus & Anthony Strittmatter)Swiss Institute for Empirical Economic Research (SEW)University of St. Gallen | Switzerland | December 2017
1 | Introduction
2 | Institutions & data
3 | Empirical strategy & econometrics
4 | Results & robustness
5 | Conclusions & further research
1 | Introduction
2 | Institutions & data
3 | Empirical strategy & econometrics
4 | Results & robustness
5 | Conclusions & further research
Motivation
Understanding differential effects of policy measures for different types ofindividuals is important for the efficient allocation of public expenditures
• Same for private sector
Common practice is to search for effect heterogeneity by includinginteraction terms or slicing data
• Spurious heterogeneity likely to be discovered: multiple testing problem− Report only factors that are ‘significant’ ex-post data mining
– In medicine, researchers have to pre-specify analysis plan
– Ex.:If based on 50 independent test statistics, probability for false positives is 92% (5% sig. level)
• Some important heterogeneity may be overlooked− Impossible to stratify & estimate (semiparametrically) for all possible strata− Even for regression models there may be more possible interactions than data points
Our research questions
1) Do causal machine learning methods provide useful tools to
uncover effect heterogeneities in active labour market
programmes?
2) Did Swiss job search programmes have differential effects for
different groups of unemployed and case workers?
Literature | Causal ML for heterogeneity | 1
Goal: Finding and estimating CATE’s under CIA
ML methods are effective in prediction• Able to deal with very high dimensions (N & p)
• Computationally efficient
• Semiparametric (sort-of)
Literature | Causal ML for heterogeneity-latest | 4
Least Absolute Shrinkage and Selection Operator (LASSO) (or similar)-type approaches based on transformed covariates oroutcomes
• Tian, Alizadeh, Gentles, Tibshirani (2014, JASA): Experimental (plus) • Chen et al. (2017): General weighting functions
Trees with larger leaves & IPW (or transformed outcomes)• Athey & Imbens (2016, Nat. Acad. Science)
Random Forests with deep trees• Wager & Athey (2017, JASA)
...
Literature | Active Labour Market Programmes
Effects of active labour market programmes• This has now become a very large literature
• Typically based on observational studies informed by rich
administrative data employing a selection-on-observables
identification strategy
• Nice summary, e.g., by meta study of Card, Kluve, Weber (2015)
• Results generally mixed
Literature | Job search programmes
Considerable literature• E.g. Cottier, Lalive (2017), Crepon, van den Berg (2016)
Mixed results• Negative for Germany (Lechner, Wunsch, 2008)• Negative for Switzerland (Gerfin, Lechner, 2002)• More positive Danish studies (Graversen, van Ours, 2008)• …
Heterogeneity• Card et al. (2015) report better results for disadvantaged participants• Lechner & Wunsch (2009) report better results during recessions
Our (intended) contributions | 1
Show how new causal machine learning tools can be fruitfully applied in a causal framework to uncover effect heterogeneities in ALMPs
Check Swiss Job Search programmes for heterogeneities• Swiss data has case worker information
− Advantage for selection-bias correction & heterogeneity analysis
If heterogeneities were discovered, translate them into information useful for policy makers
Our approach
Use informative Swiss administrative data such that CIA plausibly
holds for conditional programme effects
Use (mainly) the LASSO (Least Absolute Shrinkage and Selection
Operator) based methods suggested by Tian et al. (2014) to
investigate the heterogeneity
Reanalyse a typical programme that has already been evaluated• Tested (and thus a bit older) administrative data set with a standard
ALMP programme for a ‘normal’ developed country
The results of the paper in a nutshell
Methods ‘work’ and provide useful information• Main conclusions robust to particular type of method & its
implementation
Swiss job search programmes• Substantial heterogeneity in the beginning (lock-in phase)
− Heterogeneity is related to type of unemployed– Programme works better for UE with bad a-priori labour market chances
– Programme works better for foreigners (probably because of lack of network for informal job search) – this effect has been overlooked so far …
− Case worker heterogeneity seems to play only a very limited role
• Heterogeneity fades out after 1 year
1 | Introduction
2 | Institutions & data
3 | Empirical strategy & econometrics
4 | Results & robustness
5 | Conclusions & further research
Institutional setting | ALMP
Active labour market programmes part of Swiss UI system• Standard set of programmes (subsidized employment & training)• >500 mio CHF (=450 mio EUR) expenditures
Job search programme• Content: Learn how to search and apply for a job• Duration ~ 22 days• Class room training• Private providers• Active job search by participants is supposed to continue during the
programme
Data | Social security data & case worker survey
Data is a (merged) combination of• Social security data
− Data sources and main variables– AVAM: Information from counselling process
– ASAL: Information relevant for paying out benefits
− AHV: Information relevant for paying out pensions− Variables useful for selection and heterogeneity analysis
– Employment histories and individual socio-demographic information
• Regional data− Economic environment
• Case worker survey− Sociodemographics of case worker− Counselling strategies
Definition of treated and control group |2
Job search programmes start early• Treated: First participation during first 6 months of UE spell• Control
− No programme participation in this period− Not employed prior to randomly allocated start date from start date
distribution of treated
12’000 participants, 72’000 controls, 1’300 case workers• (First) Inflow inflow into UE in 2003 and first caseworker• UE is 24-55 old & receives UE benefits …• Case worker replies to questionaire (response rate 84%)
Objects of interests: Conditional average treatment effects (CATE)
Useful to disguish between two types of conditioning variables• X: Variables needed to remove selectivitiy• Z: Variables capturing (‘policy-relevant’) heterogeneity
− Z may be larger, smaller, partially or fully overlapping with X
• Distinction of X & Z is usually absent from many papers on the topic
Sometimes useful to disguish between effects for treated & non-treated (if components of X do not appear in Z)
• Ex.: Precise pre-specified treatment rules that can be used in prediction
CATE | 1
Goal is to estimate CATE(z) under CIA
1 0
1 0
1 0
1 0 1 0
( ) ( | ) ( | );( , ) ( | , ) ( | , );
Main assumptions) , | , , ,) 0 ( 1| , ) ( , ) 1,) , . (a bit too strong)
z d E Y Z z D d E Y Z z D d
a Y Y D X x Z z x zc P D X x Z z p x zd X X Z Z
z E Y Z z E Y Z zθ
χ
γ = = − =
= = = − = =
= = ∀ ∈ ∀ ∈Ξ< = = = = <
= =
Identification in this study | 1
Why is CIA plausible in this particular implementation?• Discussed in many previous papers