Economic Evaluation of Public Policy and Programs

Economic Evaluation of Public Policy and

ProgramsCourse Overview

ECON 4140Fall 2010

Page 2

• This course will apply economic theory and statistics to the evaluation of economic policy and programs.

• Economic policy/programs include– Spending on public infrastructure (roads, bridges, hospitals,

etc.)– Spending on programs (environmental protection, training

unemployed, vaccinations for H1NI, )– Regulations– Social marketing (health warnings on cigarettes,

temperature set-backs on thermostats • The analytical structure comprises what is known as

cost-benefit analysis.

Objectives of the course

Page 3

Course components

1. Cost-benefit/cost-effectiveness analysis

2. Measuring and valuing program impacts

3. The political economy of cost-benefit analysis

Page 4

• Overview of economic programs and policy

• Time value of money• Risk and uncertainty• Option and existence values

Part 1: Theoretical foundations of cost benefit analysis and cost effectiveness analysis

Page 5

– Experimental and quasi experimental design

– Net impact evaluation– Review of net impact evaluations– Key evaluation methods (qualitative and

quantitative)– Performance measurement and monitoring

Part 2: Measuring and valuing impacts

Page 6

Part C: The political economy of cost-benefit analysis

– Distributional issues)

Page 7

Discussion questions

Imagine you had been asked to provide advice to the government on the following policies.

1. First, what is your opinion? Why?2. Now how you would frame the problem and what

evidence is needed to support your conclusion?3. What are arguments for an opposite policy?

What evidence would support that view?4. What issues of fairness exist, if any?

Page 8

Policy proposals• Making university free

– requiring repayment of costs through an income tax surcharge

– Requiring no repayment• Offering to pay everyone in the workforce a $50

subsidy to take a flu shot every year.• Increasing the price of gasoline by 1¢/liter each

month for the next 2 years and:– Placing the proceeds in general revenue or– Devoting the proceeds to energy conservation and

pollution control technologies• Subsidizing those who purchase fuel efficient

vehicles

Page 9

The beginning

Page 10

… divide half a sheet of paper by a line into two columns; writing over one Pro and over the other Con. Then during the three of four days consideration I put down under the different heads, hints of the different motives that at different times occurred to me, for and against the measure. When I have thus got them all together, I endeavour to estimate the respective weights; and where I find two or three equal, I strike them both. If I find a reasons pro equals two reasons con, I strike out the three. If I find two reasons con equal to some three reasons pro, I strike out the five; and thus proceeding I find at length where the balance lies; and if after a day or two or farther consideration, nothing new that is of importance occurs on either side, I come to a determination accordingly. And although the weight or reasons cannot be take with the precision of algebraic quantities, and yet when each is thus considered, separately and comparatively, and the whole lies before me, I think I am able to judge better and am less liable to make a rash step.

Benjamin Franklin, 1772

Page 11

CEA and CBA: a quick tourCEA CBA

Scope Local – single output/outcome

Global – multiple outcomes valued

Unit of measure for outcome

Natural Money

Time frame Better in the Immediate/short-term

Extended over many years is common.

Main Decision Purpose

Retrospective Prospective (ex-ante)Value for money (ex-post)

Application Activities-outputs-outcomes (more on this later)

Outcomes or impacts only

Reference At least one program alternative

No reference needed since when benefits exceed costs, the project can rationally proceed.

Page 12

Types of CBA

• Ex-ante CBA attempts to project costs and benefits forward to support decision-making…it is most useful for planning and ranking alternative spending

• Ex-post CBA provides insights after the project and may determine value for money, servng as a guide for future policy.

Page 13

Example 1 :Vaccination programs for employees

• Three vaccine programs are being evaluated for cost effectiveness.

• Outcomes include:– Sick days – Total number of employees affected

• The program with the lowest number of sick days per dollar cost is the most cost effective in terms of outcomes.

• The key assumption is that the three programs are essentially similar. No program has a markedly different profile in terms of adverse impact; the only difference is efficacy.

Page 14

Example 2: CEA Training interventions

A common goal for many training interventions is the return to work.

CEA compares the costs of reaching the same outcome by alternative methods.

Typical examples of outcomes include:• Return to work for six months• Hours of work after the intervention• Number of trainees who become employed• Wages after training• Post intervention Employment Insurance benefit

Page 15

A fast tour of CBA

• CBA compares investments – especially those in the public domain - using economic concepts of social welfare.

• CBA is rooted in economic theory.

• The welfare of stakeholders emerges from the comparisons of utility for individual stakeholders as revealed by their preferences and valuations.

• Stakeholder choices in the market reveal their preferences and become the guide for how to increase (maximize) their welfare.

Page 16

Key assumptions of CBA

1. Society is composed of individuals who are free to make choices (such as how much they work), and social welfare is the simple sum of individual welfare.

2. People are the best judges of their welfare (consumer/producer sovereignty).

3. Changes in individual welfare are measurable by the willingness to pay for something.

Page 17

The main features of cost-benefit analysis• Cost-benefit analysis includes all social benefits and

costs. • It sums the values of an investment across all users• It must translate change in welfare into a dollar

amount. This can include benefits such as:– Avoided harms (deaths from disease)– Lost wages due to inability to work– Value of time saved.

• This creates an inevitable value (political) perspective for CBA.

Page 18

CBA uses money to measure everything

The main advantage of using money to value outcome is that all outcomes may be included. This is a major advantage over CEA Many non-economists are uncomfortable with the main steps in CBA:

• Translating all benefits and costs into money.• Comparing the changes in welfare among different groups.• Decisions favour majority rule and those with the greatest willingness

to pay, regardless of personal circumstances.• Participants in the economy appear to count more than those who do

not (the benefits for those earning a wage may appear to count more than the benefits for those who are unemployed, such as children, retired persons, students, etc.)

Page 19

Example 1 – Traffic congestion

Overpass: A new suburban subdivision is created beyond a main east/west transcontinental rail line. With 2,000 new households, new retail malls, and a main road linking north and south Winnipeg, traffic delays caused by rail traffic cause substantial delays.

Option 1: Create an overpass at a cost of $30 millionOption 2: Impose restrictions on rail traffic

CBA compares the ratio of benefits to the costs for each option, as well as the “hidden” option of doing nothing.

Page 20

Translating benefits into $

• Because CBA needs to express all benefits and costs as a dollar value, simplifying assumptions must be used.

• These assumptions can dramatically alter a CBA.

• Careful peer review requires a team approach to create a valid CBA.

Page 21

Example 2 – Vaccination programs

Benefit• Reduced short-term cost due to

illness• Reduced long-term cost for caring

for the small number of catastrophic incidents

• Averted loss of incomes for those who are disabled/dying

• Averted costs of lost time at work and play.

Cost• Vaccination program• Economic loss for the small

number who experience adverse reactions to vaccine.

Page 22

Page 23

Steps in a CBA

Enumerate costs and benefitsValue the costs and benefitsComparing costs and benefits

Page 24

Appendix 1: Case study

Page 25

Taking Charge! – A training program for single parents on social assistanceTaking Charge! was a pilot program jointly funded by HRDC and the Manitoba government. It focused on offering a range of supports for single parents on income assistance.Key features included:

• High level of support (daycare, counseling, basic education, volunteer experience, job placement, etc.)

• The program recruited IA clients, performed employability assessments, developed tailored training programs, contracted with service providers to delivery the training, and supported job placement.

http://www.hrsdc.gc.ca/en/cs/sp/hrsdc/edd/reports/1999-000448/sp-ah109-e.pdf

Page 26

Cost-benefit model

Basic for comparison:• “Control” group (Income assistance –

welfare clients who have never taken a training program)

• “Comparison” group (Income assistance participants in four other training interventions)

• “Program” group (Taking Charge!)

Page 27

Nature of the CBAThe outcome of interest is whether TC! participants return to work faster than those on Income Assistance and whether a TC! “grad” has a “better” employment outcome.

Better is defined as– Higher wages– Longer employment stints– Fewer stints on income assistance

We addressed this question using regression analysis and survival analysis.

Page 28

Page 29

Costs summary for Taking Charge! Overall costs and activity for the main interventions (Fiscal 1997/98)

TC! 94/95 - 97/98

Employment Connections

97/98 OFE 96/97

Clients served 3237 937 569 Total program cost $10,053,735 $1,256,200 $156,000 Average cost per client $3,106 $1,341 $274 Special needs payments $6 $12 $33 Average cost per client served $3,112 $1,353 $307 % clients employed** 0.45 0.55 0.07 Clients employed 1456 515 39 Average cost per client employed $6,905 $2,439 $4,000 * For TC!, this is clients served from program inception to March 31, 1998. For Employment Connections, this is clients served in 1997/98 and for OFE clients served in 1996/98. **After nine months of intervention termination, based on the follow-up survey. Sources: TC! annual reports, Employment Connections annual reports, OFE annual reports.

Page 30

Summary of post-intervention experience

Percent remaining on assistance

Group 3 months 6 months 9 months

Average time to come off

assistance (months)

Treatment (TC!) 79% 71% 65% 20 Employment Connections 76% 63% 55% 14 Comparison* 79% 67% 58% 16 Control 87% 82% 74% 23 * Includes EC, OFE, Community Partners and Pathways. The average time to come off assistance just for the three groups net of Employment Connections is 17.5 months.

Page 31

The benefit-cost framework

Costs Benefits A. Income assistance

payments during training E. Increased benefit reductions

because of earned income + B. Training allowances

(books, special needs) + F. Reduced income assistance

due to lower time on welfare + C. Cost of training + G. Increased taxes from

employment earnings = D. Total Costs = H. Total Benefits

Page 32

Average training time (months) and average income assistance payments (Item A in Table 76)

Adjusted average

duration of training

(months)

Average monthly IA

during training

Total IA received during training

(Adjusted duration of

training multiplied by IA during training)

Average monthly IA

after training

Group

a b c a b c d Treatment (TC!)

4.4 3.7 0.7 $904 $3,978 $3,345 $633 $682

Comparison 2.7 2.4 0.3 $891 $2,406 $2,138 $267 $650 Control* 0.0 0.0 0.0 $897 $0 $0 $0 $693 * To support the statistical analysis, we defined the intervention for the Controls as 1 month, but in reality the intervention period for these IA participants is 0.