Overview of Cost-Effectiveness Analysis and Cost-Benefit Analysis and their application to Labour Market and Social Development Policies An overview of.

Overview of Cost-Effectiveness Analysis and Cost-Benefit

Analysis and their application to Labour Market and Social

Development PoliciesAn overview of issues

Conference on Cost-Effectiveness in

EvaluationSponsored by:

Social Development Canada

Human Resources and Skills Development Canada

and the Centre of Excellence for Evaluation,

Treasury Board Secretariat

Presentation by Greg Mason

June 17, 2004

2Page

• Review the foundations and techniques of cost-benefit analysis (CBA) and cost-effectiveness analysis (CEA)

• Illustrate how CEA and CBA may be applied to current Treasury Board practices for evaluation

Overview of this presentation

Overview of objectives

3Page

• Many studies claim to be to be CBAs, but are in fact CEAs

• CEA calculates the direct financial cost of reaching specific outcome/output levels and requires one other alternative for comparison

• CBA compares all benefits to all costs and can “stand alone.” If the benefit/cost ratio exceeds 1, the program is socially valuable

• CBA – typically prospective and used for major capital investments

• CEA – typically retrospective and useful for evaluating discrete interventions

• CBA – a macro (societal) view

• CEA – a micro view of program activities, outputs, or outcomes


Differences between CBA and CEA

4Page

• Both CBA and CEA examine the costs of producing net outcomes

• CBA really only makes sense when applied to assessing the net value of achieving social outcomes. In other words, CBA measures the full (social) costs of the full (social) benefits resulting from an intervention

• CEA measures the financial cost of producing a specific net outcome

Evaluation analysis most usefully focuses on thecost-effectiveness of activities, outputs, and

immediate outcomesOverview of objectives

Core ideas for this session

5Page

Net change to welfareof all stakeholders value in $

Social cost ($)

Activities/outputs/outcomes(actual changes – not $)

CBA

CEADirect program cost ($)


CBA and CEA compared

6Page

CBA takes the long and wide view of program costs and outcomes

CEA takes a short and narrow program view of program outcomes

Time

Ra

ng

e o

f effe

cts


Scope of CEA and CBA

7Page

Cost-benefit analysis


8Page

• CBA offers a framework to compare the welfare of all stakeholders to a program or investment:– Direct beneficiaries– Indirect beneficiaries– Workers involved in program delivery– Funders (taxpayers)

• CBA imposes a range of assumptions to delimit the scope of the problem– How should distant (future) costs and benefits be valued

(social discount rate)?– What is the social cost of underemployed resources (labour)?– What is the cost of a human life?– Who should be excluded from the calculation of impacts?


CBA involves value judgements in an accounting framework

9Page

• The CBA equation divides the total dollar value of benefits by the total dollar value of costs

• Desirable programs have a high benefit/cost ratio (or low cost/benefit ratio)

• Many outcomes are difficult to value (increasing the life span arising from health investments)


CBA must translate all outcomes and impacts into a money equivalent

10Page

• CBA often applied to capital-intensive projects

• Costs– Direct capital and labour– Diversion of resources from next best use (opportunity

costs)– External (environmental impacts)– Costs of funding (tax distortions, tax expenditures)

• Benefits– Increased tourist revenues (tax revenues)– Increased profile (Winnipeg, Manitoba, Canada)– Increased multicultural understanding plus intangibles

CBA places a money value on intangible costs and benefits


CBA example: Canadian Museum for Human Rights

11Page

Enumerate costs and benefits

• CBA uses “with and without” criteria for setting up benefits, not “before and after”

• All social costs and benefits need to be identified

• Double counting is common


Steps in CBA (1)

12Page

Example: Sports and culture events are claimed to produce significant economic impacts for a community. The usual approach is to apply a multiplier to total revenues.

This assumes that audience spending has no alternative use. In fact, the net incremental tourist expenditures associated with an event are typically only incremental spending on such events. Increased spending by community residents represents a diversion of spending, not an increment.


Double counting

13Page

Comparing costs and benefits

• Present value is used to place all future costs/benefits on an equivalent basis

• It seems reasonable to place a higher value on $10,000 received now, than $10,000 to be received in two years

A bird in the hand is worth two in the bush


Steps in CBA (2)

14Page

Steps in CBA (3)

Program/project ranking and selection

• If the program/project has a benefit/cost ratio that is greater than 1, it “pays” to proceed

• Programs/projects that do not have a benefit/cost ratio greater than 1 are eliminated

• Select the program/project with the highest benefit/cost ratio


15Page

• Converting intangibles into money value (Many government programs deal in intangibles)

• Outcomes are long-term

• Avoiding double counting (Hard to do this when programs have broad social outcomes)

• Linking observed changes to program costs (Attribution complicated by the fact that the federal government may not be the sole public funder and many other factors contribute to the outcomes)


Challenges for CBA – programs

16Page

Cost-effectiveness analysis


17Page

• Because CBA compares the welfare among stakeholders, methods to translate welfare into a common denominator (i.e., money) strike many as artificial

• For this reason, CEA is conceptually and operationally simpler. It is also more applicable to evaluation and performance measurement of public programs

• When a cost-benefit study is specified, a cost- effectiveness study is most often what is really desired


Advantages of CEA

18Page

CEA can be aligned with value-for-money concepts and the results chain

– Economy – the unit cost of an activity

– Efficiency – the unit cost of an output

– Effectiveness – the unit cost of an outcome


CEA aligns with value-for-money auditing

19Page

Inputs OutputsActivitiesImmediateoutcomes

Intermediateoutcomes

Finaloutcomes

Ex

tern

al

fac

tors

Ex

tern

al

fac

tors

Ex

tern

al

fac

tors

Area of influenceArea of control

The “results chain”

Aligning CEA to the results chain

20Page

Activities OutputsImmediateoutcomes

(Typically within a year)

Longer-termoutcomes

(Up to 15 years)

CEA (cost of producing a unit...)Focus on the amount of activity, output, or outcome

CBA – value ofoutcomes to society

(Net value to all stakeholders including programparticipants and non-participants, tax payers, etc.)

Less precision/application


CEA / CBA and the results chain

21Page Cost-effectiveness analysis

Activities OutputsImmediateoutcomes

(Typically within a year)

Longer-termoutcomes(Up to 15 years)

Cost EconomyCost to complete activities(E.g., Staff time per client

assessed, delivery ofintervention...)

Cost EfficiencyCost per output

(E.g., Cost per parent trained,cost per trainee)

Cost EffectivenessCost per unit outcome

(E.g., Total program costs divided bynumber that maintain employment for

specified time, cost per arrest avoided...)

Cost economy, efficiency, effectiveness along the results

chain

22Page

The key is to identify and enumerate immediate outcomes:

– Success in athletic competition (medals)– Increase in number of hours worked (active labour

market program)– Reduced incidence of hospital adverse incidents

(“mistakes”)– Reduced waiting times for diagnosis– Reduced use of social assistance (active labour market

program, welfare reform)– Diversion of clients from in-person to electronic service

(e.g., electronic tax filing)


CEA – Moving along the results chain

23Page

• Activities should be discrete and measurable (e.g., staff years):– Processing grant applications– Consulting with applicants/clients– Preparing media

• Costing should be allocated by activity


Measuring efficiency

24Page

• Accounting estimates• Quasi-experimental techniques• “Full” randomization designs• Simulation studies

General framework for CEA outcome measurement – active labour market

programs


25Page

“Hybrid” analyses allow a cost-effectiveness calculation to incorporate many of the features of a cost-benefit model


The transition from CEA to CBA: the GAIN model

26Page

Costs “Benefits”

Income assistance payments during training

Reduced SA payments because of earned income of clients

+ Training allowances (books, special needs)

+ Reduced SA due to shorter time on welfare by clients

+ Cost of training (personnel, contracts)

+ Increased taxes from employment earnings

= Total costs = Total benefits


Application of the GAIN model to Taking Charge!

Overview of Cost-Effectiveness Analysis and Cost-Benefit Analysis and their application to Labour Market and Social Development Policies An overview of.

Documents