Return on Investment and Performance Measurement of Workforce Development Programs Kevin Hollenbeck Vice President, Senior Economist W.E. Upjohn Institute for Employment Research [email protected]February 25, 2015 Presented at OERC Conference: Workforce Outcomes: Informing Policy & Practice through Education & Labor Market Data, Columbus, Ohio
26
Embed
Return on Investment and Performance Measurement of Workforce Development Programs
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Div. of Services to Blind na na na na naaDefined as third full quarter after quarter of exit. na -- not available for this draft.
***,**,*Statistically significant at the 0.01, 0.05, 0.10 levels
NOTE: Estimates are preferred specifications as “boxed” in Hollenbeck and Huang (2006) tables. They are regression-adjusted estimates of impact levels for 2007/2008 cohorts
of WIA Dislocated Workers, WIA Youth, Community College Worker Retraining, and High School Career and Technical Education. They are regression-adjusted difference-in-
difference net impact estimates for all other programs.
Washington Net Impact Results
Table 2 Long-Terma Net Impacts of Washington’s Education and Training System, by Program
Div. of Services to Blind na na na na naaDefined as average over quarters 9 - 12 after quarter of exit. na -- not available for this draft.
***,**,*Statistically significant at the 0.01, 0.05, 0.10 levels
Note: Estimates are preferred specifications as “boxed” in Hollenbeck and Huang (2006) tables. They are regression-adjusted estimates of impact levels for 2005/2006 cohort of
WIA Dislocated Workers, WIA Youth, Community College Worker Retraining, and High School Career and Technical Education. They are regression-adjusted difference-in-
difference net impact estimates for all other programs.
Washington Net Impact Results
Washington Net Impact Results
What Does State Do with Results?
Return on Investment (Theory)
Benefits Participants Taxpayers Society
Higher productivity 0 + +
Higher earnings + 0 +
Fringe benefits + 0 +
Less unemployment/turnover + + +
Lower income maintenance transfers - + 0
Higher taxes - + 0
Costs
Tuition, fees -/0 - -
Forgone wages - 0 -
Total (Net Benefits) + ?? +/??
Return on Investment (Actual
Framework)
Benefits Participants Taxpayers Society
Higher productivity 0 + +
Higher earnings + 0 +
Fringe benefits + 0 +
Less unemployment/turnover + + +
Lower income maintenance transfers - + 0
Higher taxes - + 0
Costs
Tuition, fees -/0 - -
Forgone wages - 0 -
Total (Net Benefits) + ?? +/??
Note: Entries in the table that are highlighted are omitted from empirical estimation.
Return on Investment – Secondary CTE
Table 4 Participant and Taxpayer Benefits and Costs per Completer in Secondary CTE Programs
Benefit/Cost
First 2.5 years Lifetime (until 65)
Participants Taxpayers Participants Taxpayers
Benefit
Earnings
Fringe Benefits
Taxes
2,753
551
−475
0
0
475
34,603
6,920
−5,969
0
0
5,969
Transfers
UI
TANF
FS
Medicaid
16
−91
−73
−27
−16
91
73
27
168
−442
−380
−169
−168
442
380
169
Costs
Forgone earnings
Program costs
−28
0
0
704
−28
0
0
704
NOTE: ’00 $. Discount rate of 3% real.
SOURCE: Hollenbeck and Huang (2006).
Return on Investment – Postsecondary
CTE
Table 3 Participant and Taxpayer Benefits and Costs per Participant in Postsecondary CTE Programs
Benefit/Cost
First 2.5 years Lifetime (until 65)
Participants Taxpayers Participants Taxpayers
Benefit
Earnings
Fringe Benefits
Taxes
10,386
2,077
−1,792
0
0
1,792
79,239
15,848
−13,669
0
0
13,669
Transfers
UI
TANF
FS
Medicaid
−2,137
351
107
45
2,137
−351
−107
−45
−2,629
933
331
161
2,629
−933
−331
−161
Costs
Forgone earnings
Program costs
2,100
3,519
0
6,877
2,100
3,519
0
6,877
NOTE: ’00 $. Discount rate of 3% real.
SOURCE: Hollenbeck and Huang (2006).
Return on Investment
Table 5 Benefits, Costs, and Annual Rates of Return for Postsecondary and Secondary CTE Programs
over the First 2.5 Years and Lifetime for the Average Participant
Benefit/Cost
Secondary CTE Postsecondary CTE
First 2.5 years Lifetime (age 65) First 2.5 years Lifetime (age 65)
Participants
Benefits
Costs
irr
2,654
-28
na
34,731
-28
na
9,037
5,619
20.93%
80,204
5,619
8.81%
Taxpayers
Benefits
Costs
irr
650
704
-3.14%
6,792
704
4.70%
3,431
6,877
-24.28%
14,873
6,877
2.48%
Society
Benefits
Costs
irr
3,304
676
88.64%
41,523
676
25.35%
12,468
12,496
-0.09%
95,077
12,496
6.65%NOTE: Table entries are for average participant. Benefits include earnings, fringe benefits, and income-related transfer
payments. Costs include tuition and fees (if any), foregone earnings, and public program costs per participant. $ figures
are in real 2000$. na means that irr could not be calculated because of 0 or negative costs. Discount rate is 3% real.
Return on Investment
• Secondary and postsecondary CTE have huge
payoffs for participants, especially in short-run
• State/taxpayers eventually get positive ROI, but
payback takes slightly more than 2.5 years for
secondary and considerably longer for
postsecondary
• “Social” ROI quite positive for secondary and for
postsecondary
Results from Other States
Table 5 A Comparison of Findings Across States
Outcome WIA-Adult
WIA-Dislocated
Workers WIA-Youth TAA
Postscondary
Education
Employment in 3rd quarter (%)
Indiana 14.8*** 17.0*** 3.4 3.2 17.9***
Washington 9.7*** 8.7*** 4.2** na 10.3***
Virginiaa, c 3.4*** −3.9** −5.9*** 2.8***
Quarterly earnings in 3rd quarterb ($)
Indiana 549** 410** 24 −122 1,490***
Washington 711*** 784*** 66 na 1,275***
Virginiaa, c 146*** 62 −154*** 1,539***
NOTES: *, **, *** impact estimate is significant at 0.10, 0.05, and 0.01 level. aWIA-Adults and WIA-Dislocated Workers were combined in the study.bIndiana results in 2008$; Washington and Virginia in 2005/2006$.cVirginia results are for the 4th quarter after exit.
Results from Other States
Table 6 Net Impacts, Disaggregated by Training Status
NOTES: *, **, *** impact estimate is significant at 0.10, 0.05, and 0.01 level.
Results from Other States
Table 7 Quarterly ROIs Using Lifetime Earnings Flow Estimates
WIA-Adult
WIA-
Dislocated
Workers WIA-Youth TAA Postsecondary
Individual program participant (%) 16.32 2.64 13.27 −0.93 29.87
Government (%) −0.04 1.50 −1.73 5.01 1.82
Society (takes into consideration
individual and government) (%)7.60 2.13 0.22 −0.40 9.66
Results from Other States
Table 2. Net Impact Indicators for Virginia’s Workforce Programs, FY2005
Program
Short-term
employment/
in school rate
Long-term
employment/
in school rate
Short-term
earnings
level
Long-term
earnings
level
Credential
completion rate
DOE and VCCS programs
AEL (DOE)
Postsecondary CTE (VCCS)
−9.58***
0.49
−9.07***
2.81***
289***
1,213***
-21
1,539***
65.48***
22.68***
DRS and DBVI programs
DRS
DBVI
17.63***
17.37***
16.17***
25.00***
429***
1,948***
241***
1,318**
8.79***
6.78***
DSS programs
FSET
TANF/VIEW
−9.29***
−2.22***
−9.55***
−2.30***
−404***
414***
−529***
175*
−0.41***
0.46***
VEC and Senior Advisor programs
TAA (VEC)
WIA Adultsa (Senior Advisor)
WIA Youth (Senior Advisor)
−6.15***
4.75***
−2.91*
−5.88***
3.39***
−3.88**
−210***
442***
480***
−154***
146**
62
65.03***
53.96***
76.12***
W/P (VEC) post–pre impactsb −0.36 0.09 −250*** 38* na
aIncludes WIA Dislocated Workers.bW/P program impacts were computed using a post-pre methodology as described in text. na means not available because credential completion is not meaningful in a post-pre