My presentation erin da802

Analysis for Public DecisionsCh. 4: A Basic Framework for Policy Analysis

- E.S. Quade

Presented by Erin Mihalik

Quick Summary

This chapter discusses the methodology as if the analysis were being carried out for a single decision-maker who commissioned it.

• The successful application of analysis to policy problems is an art.

• There are principles and procedures which will offer guidance, but these cannot be followed blindly.

• For policy analysis to be successful, it must:

– Discover a course of action that (if not clearly the best available option) is at least satisfactory

– Have its findings accepted and incorporated into a policy or decision that can be implemented WITHOUT being so modified that it no longer brings about the desired result.

The analyst’s basic procedure:

1. To help the decision-maker determine what he wants.

2. To search out the possible ways of getting what he wants

3. To work out the consequences that would follow a decision to adopt each of the alternatives, and

4. To rank the alternatives according to a criterion specified by the decision-maker (or to present them to him for ranking along with the necessary information)

That sounds easy… TOO EASY!

The Problem…

Public decisions don’t always fit into the “basic procedure”

BUT, even so, the “basic procedure” is a good starting point and a reasonable strategy for discovering good solutions.

The Elements of Analysis

1. The Objective

2. The Alternatives

3. The Impacts

4. The Criteria

5. The Model (or Models)

1. The Objectives

The objectives are what a decision-maker seeks to accomplish or attain by means of his decision.

One of the hardest tasks an analyst has is to discover if the stated objectives are

REALLY the objectives desired!

2. The Alternatives

• The alternatives are the options or means available to the decision-maker by which, it is hoped, the objectives can be attained.

Juvenile Delinquency, for example…

Alternatives don’t have to be obvious!

So what are some alternatives for

combatting juvenile delinquency?

Education… Recreation…

Family subsidy….

Police surveillance…

Low-income housing…

And whatever other options we might

discover later!

3. The Impacts

The designation of a particular alternative as the means of accomplishing the objective implies a certain set of consequences. Some of these consequences are benefits and others are costs.

Huh??? In English please, Mr. E.S. Quade!

It just means that no matter which alternative we choose, there will be positive and negative side effects!

The Impacts

There may be other impacts associated with an alternative that, while they have little effect on the attainment of the desired objective, they should still be considered. These are called spillovers or externalities.

By broadening the objectives, the externalities can be internalized or made part of the study.

• In the narrow sense, costs are the resources required to implement an alternative that is no longer available for other uses once it is implemented.

• In the broadest sense, costs are the “opportunities foregone” – all the things we cannot have or do once we have a particular alternative.

Many, but not all, costs can be expressed in dollars or other quantitative terms.

One example would be traffic fatalities.

Traffic Fatalities Example:

This example includes things that can and cannot be expressed quantitatively.

Delay caused to motorists by lowering the speed limit can be partially expressed quantitatively…

But things like irritation to the drivers and speeding elsewhere might lead to MORE accidents! This is very hard to quantify.

4. The Criteria

A criterion is a rule or standard by which to rank the alternatives in order of desirability.

This provides a way to relate objectives, alternatives, and impacts.

5. The Model

Any decision analysis needs a process that can predict or indicate the consequences that flow from an alternative. This is provided by the model.

Abstractly, a model is just a set of generalizations or assumptions about the world. It’s a simplification of the real world.

Policy Models are usually elaborate mathematical structures programmed for a computer.

Other models are simple mental models and used throughout the analytic process.

The Process of Analysis

When searching for the best alternative, we go through 5 stages that must be linked together and repeated several times.

The Five Stages

• Formulation – clarifying and constraining the problem and determining objectives.

• Search – identifying, designing, and screening the alternatives.

• Forecasting – predicting the future environment or operational context.

• Modeling – building and using models to determine the impacts.

• Evaluation – comparing and ranking the alternatives.

The Importance of Being Repetitive

I have a problem I want you to help me analyze.

Sure! But it’s very possible that you don’t fully understand your

problem, so I’m going to ask you a lot of repetitive questions so we can

solve the REAL issue!

Successful analysis depends upon a continuous cycle of:

• Formulating the problem

• Selecting objectives

• Designing alternatives

• Building better models

Repetition

It’s important to cycle and recycle through the questions so the answers become more focused.

Are you sure?

Really sure?

???

The Iterative Nature of Analysis

Clarifying the problem

Evaluating costs and

effectiveness

Opening new alternatives

Questioning assumptions

Interpreting results

Determining objectives and

criteriaSearching out and designing alternatives

Collecting data and information

Examining alternatives for

feasibility

Building and testing models

ITERATION!

Formulation

Formulation encompasses an attempt to isolate the questions or issues involved to fix the context within which these issues are to be resolved, to clarify the objectives, to discover the major factors that are operative, and to get some feel for the relationships among them.

This stage is SUPER important! All the time we spend redefining the problem helps to make the problem much more clear. A clear problem

helps us find a better solution!

An Example…

Tell me about your problem…

There’s way too much turnover in my training program! Everyone seems to quit or get fired!

Help me reduce the turnover!!

Eh-hmmmmm!!!

The same “problem” can have different perspectives from different levels within the company!

My company doesn’t HAVE a turnover problem. We train people

to do dangerous jobs – if the turnover’s high, it’s in the safety

interest of those we terminate and the other workers! I don’t want

anyone getting hurt!

Well, that certainly changes

things!

To solve one problem, you may need to solve other problems, first!

• For example, urban air pollution.

First you need to consider other aspects of city life, like local transportation and controls… and these might affect housing and jobs!

A systematic investigation of these issues might alter your perception of the issue.

Formulation is highly subjective

We consider what facts will be meaningful to the decision-maker. In the transportation example, we should consider the practical things, but we can leave the outlandish hypotheticals at home!

Search

The search phase is concerned with finding the alternatives and the data and relationships on which the analysis is to be based. It is usually more productive to look for additional alternatives than it is to look for more precise schemes for comparison.

It’s important to consider a wide range

of alternatives!

Search is the activity that depends most on talents that the analyst may not have.

Scientific!

Economic!

Medical!

Language!

Engineering!

Legal!

Artistic!

An analyst’s success may often depend on the technical competence of his associates!

Sociology!

Cost Analysis!

Every system belongs to a hierarchy.

There are subsystems for every system and there are wider systems that the system you’re studying forms a subsystem.

Your System

Clear thinking will be necessary to find the

links between all these systems so I can

formulate all the costs and benefits!

Forcasting

• The impacts or consequences that follow from the decision to accept and implement a particular alternative depend upon both:

– The properties of the alternative, AND

– On the situation or environment that exists when and during the time the alternative is to be effective.

The Transportation Example…

• The population distribution in the region may affect the choice of routes and carriers.

– If the population is evenly distributed, carriers who can make many stops will be preferred.

Scenario writing

• In policy studies, predictions of the future environment is most often done by scenario writing.

• A scenario is a description of the essential features of the future context in which the alternatives are to be implanted.

• Scenario writing is the preparation of a logical sequence of hypothetical (but credible events) that could lead from the present to the future.

Does that mean no personal commuter

dragons?

Exactly! Personal commuter dragons aren’t

exactly … credible.

Modeling

• Building a model of the system and its environment is the standard approach to compare the costs associated with different ways of operating a future system.

• A model could be a simple table or graph, or it may be written in a mathematical language.

• If you are lucky, the same model can be used to predict outcomes from a competing system – if not, another model must be built.

Systems Engineering Models

The end objective is to optimize the performance of a system, so the model building must be subservient to this objective. A systems engineering team must:

1. Ensure model building is carried out with a sense of purpose(designed as efficiently as possible)

2. Tie together the various specializations that may be needed for building models of the subsystems.

3. Ensure that work is concentrated where it is most needed. As a general rule, models should be kept as simple as possible.

4. Decide when the model is adequate for the purposes for which it is needed

5. If the model is used for planning, see that an effective dialogue is conducted between the systems team and the managers who will use the model. This dialogue must start when the model is BEING built.

Evaluation

Systems analysis has used two principal conceptual approaches to rank the alternatives:

1. Fix the task or the level of effectiveness and then seek to determine the alternative which is likely to achieve this level of effectiveness or accomplish the task at the lowest cost.

2. Fixed budget. For a specified cost level or budget to be used in attaining the objective the analysis attempts to determine which alternative will produce the highest effectiveness.

The ideal is to do more than to prepare a comparison of the alternatives ; we also want to rank them according

to criteria, so the decision-maker’s choice is easier!

Cost-Effective Approach

• Cost-effectiveness approaches are useful when the relative merit of numerous proposals is under investigation.

• It is not as useful when the question is one of absolute merit (such as in deciding between funding adult education or highway construction).

Fixed Budget Approach

• This can be useful for questions of absolute merit, even if we can’t measure accomplishment in a clear way.

• We can “normalize” the alternatives and develop a table of comparative effectiveness measures (this can be both qualitative and quantitative). Then decision-makers may be able to decide on the basis of their own subjective criteria which type of program they prefer.

• When decision-makers are judging whether something is worth the cost, the most common approach is to express the benefits and costs associated with each alternative in dollars as a function of time, discount the future benefits and costs at some appropriate rate, and compare the alternatives on the basis of the present value of net benefits.

• Alternatives can also be compared on the basis of the internal rate of return (discount rate so NPV=0). This is the “classical” cost-benefit approach – it’s hard to use well when analyzing complex policy issues.

• It may not be possible for the analysts to prepare an unambiguous ranking of the alternatives.

• When this occurs, the best scheme may be to list the characteristics and impacts of the alternatives and let the decision-makers determine the ranking using their own judgment.

What if my method of comparison isn’t able to answer all the questions

the decision-makers want to consider?

Don’t worry!! No method of comparison is going to answer

ALL the questions. Just do your best to anticipate and answer

as much as you can!

No method of comparison is likely to answer all the questions.

Conclusive Summary

• Even if the model and its inputs are excellent, the conclusions proposed may be unacceptable due to reasons based on morale, tradition, politics, and organizational behavior.

• It’s important for the analysis USER to distinguish what the study actually shows from any recommendations made by the analyst based on what the analyst FEELS should be done. – Some say an analyst should not even make his

conclusion known.

• Whether or not the decision-maker made a right decision based on analysis can never be verified. Even long after the decision has been made, we may still have no way of telling whether the best action was chosen.

The Process Illustrated

• There are as many ways to approach a problem as there are analysts.

• We’ll take a look at one example…

Short-Haul Transportation

In this example, we turn to short-haul transportation systems about 10-15 years in the future.

So this study must consider the following:

Additionally, how the costs/benefits are distributed among social groups and localities must be considered.

Item Cost Beneft

Jet aircraft Serious noise and air polution Reduced travel time

Automobiles Pollution, personal injuries, energy shortages, urban sprawl

Increased individual mobility

High-speed railways Noise pollution; land required

Obvious Considerations Less Obvious

Time saved Environmental effects

Monetary Cost Societal effects

Potential procedure for carrying out the analysis:

Design of cases for

comparison

Building models to

predict impacts

Prediction of impacts

Comparison of cases

Prediction of operational

context

Design of alternative

systems

Choice of a criterion:

selection of impacts

The impacts must be worked out on a case-by-case basis

In the transportation hypothetical, we have the following categories:

Impact Impacted

Transportation service impacts Occur to users of the system.

Financial impacts Occur to the operators and society

Economic impacts Involve changes in income and employment

Community impact Changes in the activity patterns, tax base, and environment

Distributional impacts This considers how the various aggregate impacts are distributed among different social groups and locations

Developing the model…

Because in this example, there are so many impacts which are diverse and difficult to value monetarily, the desirable approach would be to present the impacts in their natural units to be ranked by the decision-makers.

Example of a model illustrating impact values…Impacts Base

CTOL case

VTOL case

TACV case

Impacts Base CTOL case

VTOL case

TACV case

Transportation service impacts

Community impacts

Passengers (millions yearly) 7 4 9 Noise (thousand households 10 1 20

Door-to-door trip time (avghr)

2 1.5 2.5 Air pollution (% all emissions) 3 9 1

Door-to-door trip cost (avg. in $)

17 28 20 Petroleum savings (%) 0 -20 +30

Airport congestion (% reduction

0 5 10 Households displaced 0 20 500

Financial Impacts Land taken (acres) 0 25 8000

Investment costs ($ millions)

150 200 2000 Taxes lost ($ millions) 0 0.2 2.0

Net annual subsidy ($millions)

0 0 90 Landmarks destroyed None None Ft. X

Economic impacts (peak yr Distribution impacts

Added jobs (thousands) 20 25 100 % low-income trips taken 7 1 20

Added sales ($ millions) 50 88 500 % of noise-impacted low-income households

2 16 40

• Having developed the required models, measures of the various impacts are obtained and the set presented to the decision-makers for their inspection in the form of a “scorecard”.

• This alone can’t be expected to complete the analytic work, for the decision-makers will have questions and will want additional comparisons made and further systems and mixes of alternatives investigated.

• But after these questions are answered, the decision maker will be in a better position to make a decision than before the analysis.

Analysis for Public DecisionsCh. 4: A Basic Framework for Policy Analysis

- E.S. Quade

Presented by Erin Mihalik

Questions?