Results based accountability 101 active (2009)

Results-BasedAccountability

(AKA: Outcome-Based Accountability or OBA)

The Fiscal Policy Studies InstituteSanta Fe, New Mexico

Websitesraguide.org

resultsaccountability.com

Book - DVD Ordersamazon.com

resultsleadership.org

SIMPLE

COMMON SENSE

PLAIN LANGUAGE

MINIMUM PAPER

USEFUL

Results-Based Accountabilityis made up of two parts:

Performance Accountabilityis about the well-being of

CUSTOMER POPULATIONSfor Programs – Agencies – Service Systems

Population Accountabilityis about the well-being of

WHOLE POPULATIONSfor Communities – Cities – Counties – States - Nations

POPULATIONACCOUNTABILITY

Fiscal Policy Studies Institutewww.resultsaccountability.comwww.raguide.org

For Whole Populationsin a Geographic Area

Community Outcomesfor Christchurch, NZ

1. A Safe City

2. A City of Inclusive and Diverse Communities

3. A City of People who Value and Protect the Natural Environment

4. A Well-Governed City

5. A Prosperous City

6. A Healthy City

7. A City for Recreation, Fun and Creativity

8. City of Lifelong Learning

9. An Attractive and Well-Designed City

UK National OutcomesA working list based on the 2007 National Indicators Proposal

1. Strong Communities

2. Safe Communities

3. Healthy and Successful Children and Young Peoplea. Be Healthyb. Stay Safec. Enjoy and Achieved. Make a Positive Contributione. Economic Well-being

4. Healthy and Successful Adults and Elders

5. Communities that Achieve Inclusion and Equality

6. Communities with a Thriving Local Economy

7. Communities with a Sustainable Environment

A Stable, Secure, Inclusive

INTERNETthat supports the

world economy and the world community

ICANN Population Result(Healthy Internet)

INTERNET REPORT CARD

What measures do you use?

What do the data baselines look like?

What is the story behind the baselines?

STRATEGIC ROLE OF ICANN

GLOBAL STRATEGIC THINKING

HEALTHY INTERNET(per prior page definition)

What Would It Take?

New Zealand Kruidenbuurt Tilburg, Netherlands

Coventry, UK

Country Neighborhood

Local Council

Every timeyou present

your program,

Use atwo-part

approach.

Result: to which you contribute to most directly.

Indicators:

Story:

Partners:

What would it take?:

Your Role: as part of a larger strategy.

Population Accountability

Performance Accountability

Your Role


For Programs, Agencies and Service Systems

Fiscal Policy Studies Institutewww.resultsaccountability.comwww.raguide.org

“All performance measures

that have ever existed

for any program

in the history of the universe

involve answering two sets of

interlocking questions.”

HowMuchdid we do?

( # )

HowWell

did we do it?

( % )

Quantity Quality

Performance Measures

EffortHow hard did we try?

EffectIs anyone better off?


Effort

Effect

HowMuch

HowWell


How much service did we deliver?


How welldid we

deliver it?

How much change / effect

did we produce?

What quality of change / effect

did we produce?

Quantity Quality

Effe

ct

Effo

rt

O

utpu

t

In

put

How much did we do?


How welldid we do it?

Is anyonebetter off?

Quantity Quality

Effe

ct

Effo

rt

# %

How much did we do?

Education

How well did we do it?

Is anyone better off?

Quantity Quality

Effe

ct

E

ffort Number of

studentsStudent-teacher

ratio

Number ofhigh schoolgraduates

Percent ofhigh schoolgraduates

How much did we do?

Education



Quantity Quality

Effe

ct

E

ffort Number of

studentsStudent-teacher

ratio

Percent of 9th graders who

graduate on timeand enter college or

employment after graduation

Number of 9th graders who

graduate on timeand enter college or

employment after graduation

How much did we do?

Drug/Alcohol Treatment Program



Number ofpersonstreated

Percent ofstaff withtraining/

certification

Number of clientsoff of alcohol & drugs - at exit - 12 months after exit

Percent of clientsoff of alcohol & drugs - at exit - 12 months after exit

Quantity Quality

Effe

ct

E

ffort

How much did we do?

Fire Department



Number ofresponses

ResponseTime

Quantity Quality

Effe

ct

E

ffort

# of fireskept to

room of origin

% of fireskept to

room of origin

How much did we do?

General Motors



# of production hrs

# tons of steel

Employees pervehicle

produced

# of cars sold

$ Amount of Profit

$ Car value after 2 years

Quantity Quality

Effe

ct

E

ffort

Source: USA Today 9/28/98

% Market share

Profit per share

% Car value after 2 years

How much did we do?

Personnel Department



Quantity Quality

Effe

ct

E

ffort

Number ofApplicantsProcessed

AverageRecruitment

Period

WorkforceNew Hires

WorkforceTurnover Rate

(non-promotions)

CustomerSatisfaction

How much did we do?

Information Technology (MIS)



Quantity Quality

Effe

ct

E

ffort

Number ofIT projects

AverageResponse timeto Svc requests

Amount ofUnscheduled

Downtime

Rate ofUnscheduled

Downtime

CustomerSatisfaction

How much did we do?

Policy Development



Quantity Quality

Effe

ct

E

ffort

Number ofpolicy projects

Percent of projects on track

Percent of comments fully

addressed

Number of comments fully

addressed

How much did we do?

Volunteering



Quantity Quality

Effe

ct

E

ffort

Number ofvolunteers

Percent involved 1 year or more

Number who say their

participation made a difference

Percent who say their

participation made a difference

How much did we do?

Performance AccountabilityTypes of Measures found in each Quadrant



# Clients/customers served

# Activities (by type of activity)

% Common measurese.g. client staff ratio, workload ratio, staffturnover rate, staff morale, % staff fully trained, % clients seen in their own language,worker safety, unit cost

% Skills / Knowledge (e.g. parenting skills)

#

% Attitude / Opinion (e.g. toward drugs)

#

% Behavior (e.g.school attendance)

#

% Circumstance (e.g. working, in stable housing)

#

% Activity-specific measures

e.g. % timely, % clients completing activity, % correct and complete, % meeting standard

Point in Timevs.

2 Point Comparison

# %

How much did we do?

Choosing Headline Measures and the Data Development Agenda



Quantity Quality

E

ffect

Effo

rt

# Measure 1 ----------------------------

# Measure 2 ----------------------------

# Measure 3 ----------------------------

# Measure 4 ----------------------------

# Measure 5 ----------------------------

# Measure 6 ----------------------------

# Measure 7 ----------------------------

#1 Headline

#2 Headline

#3 Headline

#1 DDA

#2 DDA

#3 DDA% Measure 8 ----------------------------

% Measure 9 -----------------------------

% Measure 10 ---------------------------

% Measure 11 ---------------------------

% Measure 12 ---------------------------

% Measure 13 ---------------------------

% Measure 14 ---------------------------

# Measure 15 ----------------------------

# Measure 16 ----------------------------

# Measure 17 ----------------------------

# Measure 18 ----------------------------

# Measure 19 ----------------------------

# Measure 20 ----------------------------

# Measure 21 ----------------------------

% Measure 15 ----------------------------

% Measure 16 ----------------------------

% Measure 17 ----------------------------

% Measure 18 ----------------------------

% Measure 19 ----------------------------

% Measure 20 ----------------------------

% Measure 21 ----------------------------

LR

UR

Primary v. Secondary

Direct v. Indirect

Internal v. External

Baseline & Story

How Population

&Performance Accountability

FIT TOGETHER


your program,

Use atwo-part

approach.


Indicators:

Story:

Partners:




Program:Performance measures:

Story:

Partners:

Action plan to get better:


Your Role


Indicators:

Story:

Partners:


Your Role: within the larger strategy.



Story:

Partners:



Your Role


your program,

Use atwo-part

approach.


your program,

Use atwo-part

approach.


Indicators:

Story:

Partners:





Story:

Partners:



Your Role

Shortcut

IN CLOSING

Well-Known Alternativesto Results-Based Budgeting

● Line Item Budgeting

● Program Budgeting

● Zero-based Budgeting

● Needs-Based Budgeting

● Program Planning and Budgeting

● Bottom Line Budgeting

● Management by Objectives Budgeting

LESS Well-Known Alternativesto Results-Based Budgeting

● Fear-Based Budgeting

● Fantasy-Based Budgeting

● Power-Based Budgeting

● Guess-Based Budgeting

● Guilt-Based Budgeting

● Caffeine-Based Budgeting

● Carbon-Based Budgeting

● Bouillabaissed Budgeting

“If you do what you always did,

you will get what you always got.”

Kenneth W. JenkinsPresident, Yonkers NY NAACP

“We will succeed together…

or fail separately.”

- Winston Churchill or

- Sonny and Cher

Book - DVD Ordersamazon.com

resultsleadership.org

THANK YOU !

Websites

raguide.orgresultsaccountability.com

“Data is theunblinking eye

of reform.”

Barbary Curley,Area Director, Hyde Park Office, Boston

A Stable, Secure, Inclusive

INTERNETthat supports the

world economy and the world community

ICANN Population ResultVersion 2

INTERNET REPORT CARD

What measures do you use?

What do the data baselines look like?

What is the story behind the baselines?

STRATEGIC ROLE OF ICANN

GLOBAL STRATEGIC THINKING

HEALTHY INTERNET(per prior page definition)

Stable: Rate of IPv6 adoption: 2008 less than 1% in any country

Secure: Rate of significant viruses and security breeches

Inclusive: Percent of world population with internet access

Supports world economy: Percent of world GDP related to internet commerce

Supports world community: Per capita rate of internet traffic

Results and Possible Indicators

Results-Based Accountability

COMMON LANGUAGE

COMMON SENSE

COMMON GROUND

THE LANGUAGE TRAPToo many terms. Too few definitions. Too little discipline

Benchmark

Target

Indicator Goal

Result

Objective

Outcome

Measure

Modifiers Measurable Core Urgent Qualitative Priority Programmatic Targeted Performance Incremental Strategic Systemic

Lewis Carroll Center for Language Disorders

DEFINITIONS

Children born healthy, Children ready for school, Safe communities, Clean Environment, Prosperous Economy

Rate of low-birthweight babies, Percent ready at K entry, crime rate, air quality index, unemployment rate

1. How much did we do? 2. How well did we do it? 3. Is anyone better off?

RESULT or OUTCOME

INDICATOR or BENCHMARK

PERFORMANCE MEASURE

A condition of well-being for children, adults, families or communities.

A measure which helps quantify the achievement of a result.

A measure of how well a program, agency or service system is working. Three types:

Popu

latio

nPe

rfor

man

ce

Children born healthy

Rate of low-birthweight babies Percent ready at K entry

Children ready for school

crime rate

Safe communities

air quality index

Clean Environment

unemployment rate

Prosperous Economy

INDICATOR

RESULT

PERFORMANCE MEASURE

Popu

latio

nPe

rfor

man

ce

= Customer Results

From Ends to Means

ENDS

MEANS

From Talk to ActionPo

pula

tion

Perf

orm

ance

RESULT

INDICATOR

PERFORMANCEMEASURE

Customer result = EndsService delivery = Means

From Talk to Action

Results – Indicators – Performance Measures in

Amharic, Cambodian, Laotian, Somali, Spanish, Tigrigna, Vietnamese

The Three L’s of Successin any serious change effort

1. Leadership

2. Language

Searching for the 3rd L

LeverageLatitudeLevityLithiumLuggage

Criteria for

Choosing Indicatorsas Primary vs. Secondary Measures

Communication Power

Proxy Power

Data Power

Does the indicator communicate to a broad range of audiences?

Does the indicator say something of central importance about the result?

Does the indicator bring along the data HERD?

Quality data available on a timely basis.

Choosing IndicatorsWorksheet

Outcome or Result_______________________

Candidate IndicatorsCommunication

PowerProxyPower

DataPower

H M L

H

Measure 1

Measure 2

Measure 3

Measure 4

Measure 5

Measure 6

Measure 7

Measure 8

HData

Development

Agenda

Safe Community

H M L H M L

H H

H L

Three Part Indicator List for each Result

Part 1: Primary Indicators

Part 2: Secondary Indicators

Part 3: Data Development Agenda

● 3 to 5 “Headline” Indicators● What this result “means” to the community● Meets the Public Square Test

● Everything else that’s any good (Nothing is wasted.)● Used later in the Story behind the Curve

● New data● Data in need of repair (quality,timeliness etc.)

Leaking Roof(Results thinking in everyday life)

Experience

Measure

Story behind the baseline (causes)

Partners

What Works

Action Plan

Inches of WaterBASELINE

? Fixed

Not OK

Turning the Curve

Action Plan # 2

The Matter of Baselines

Baselines have two parts: history and forecast

H

M

L

History Forecast

Turning the CurvePoint to Point

OK?

19

75

19

80

19

82

19

90

20

00

20

05

0

5

10

15

20

25

30

Th

ou

san

ds

Source 1982 to 2005: Actual data from the NHTSA Fatality Analysis Reporting System (FARS)Source 1975 to 1981: Estimate based on NHTSA data on % of fatality drivers with BAC of .10 or greater.

Alcohol-Related Traffic FatalitiesUnited States 1975 to 2005

MADD

Rebound

6.00%

8.00%

10.00%

12.00%

14.00%

16.00%

18.00%

20.00%

Ncle 14.5 14.5 16.8 14.5 17 15 11.9 10.6 9.5 9.3

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

Source: Connexions Tyne and Wear, UK

Newcastle, UK

Revised 9 Nov 2007

Nov 08 – Jan 09

8.5

- Rosell

“If I include you,you will be my partner.

If I exclude you,you will be my judge.”

RBA Categories Account for All Performance Measures(in the history of the universe)

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Product Output Impact

Process Input

Effe

ctEf

fort

Cost

Total Quality Mgmt (TQM)

Effectiveness

Efficiency Admin overhead, Unit cost

Quantity Quality






Benefit value

Process Input

Effe

ctEf

fort

Cost




Quantity Quality






Benefit value

Process Input

Effe

ctEf

fort

Cost

1. Did we treat you well?

2. Did we help you with your problems?

*



* World’s simplest completecustomer satisfaction survey


Quantity Quality






Benefit value

Process Input

Effe

ctEf

fort

Cost


OBA Categories Account for All Performance Measures(in the history of the universe)


How much did we do?

The Matter of Control



Quantity Quality

Effe

ct

E

ffort

LeastControl

PARTNERSHIPS

MostControl

The Matter of Use

1. The first purpose of performance measurement is to IMPROVE PERFORMANCE.

2. Avoid the performance measurement equals punishment trap.

● Create a healthy organizational environment.

● Start small.

● Build bottom-up and top-down simultaneously.

1. To Ourselves Can we do better than our own history?

2. To Others When it is a fair apples/apples comparison.

3. To Standards When we know what good performance is.

Comparing Performance




1. To Ourselves First

Can we do better than our own history?

Using a BaselineCHART ON THE

WALL

CHARTS on the WALL

CHARTS ON THE WALL


1. To Ourselves First Can we do better than our own history?


Reward?

Punish?


1. To Ourselves First Can we do better than our own history?




The Matter of Standards

Quantity

Effe

ctEf

fort

1. Quality of Effort Standards aresometimes WELL ESTABLISHED

● Child care staffing ratios● Application processing time● Handicap accessibility● Child abuse response time

2. Quality of Effect Standards arealmost always EXPERIMENTAL

● Hospital recovery rates ● Employment placement and retention rates

● Recidivism rates ● School reading scores

3. Both require aLEVEL PLAYING FIELD and an ESTABLISHED RECORDof what good performance is.

BUT

AND

Advanced Baseline Display

Your Baseline

Comparison Baseline

Goal (line)Target or Standard

Instead:Count anything better than baseline as progress.

Avoid publicly declaringtargets by year if possible.

●

Create targets only when they are:

FAIR & USEFUL

x

How much did we do?

Not All Performance Measures Are Created Equal



LeastImportant

Quantity Quality

Effe

ct

E

ffort

MostImportant

Least

Most

AlsoVery Important

Quality

© The Farside 1989

" Bummer of a Birthmark Hal "

Next Generation ContractingQuantity

Effe

ctEf

fort

3. What we purchase in the lower quadrants are not deliverables, but rather a RELATIONSHIP where funder and grantees work together to maximize customer results.

WHICH MEANS

BUT2. They break down inthe Is anyone better off?quadrants (because of case mix

differences and perverse incentives).

1. Traditional purchasing methods work fine in the upper quadrants.

Quality

How much did we do?

Program Performance Measures

How welldid we do it?

Is anyonebetter off?

Quantity Quality

Effe

ct

Effo

rt

# %

Lay

Definition

All Data have two Incarnations

Technical

Definition

HS Graduation Rate % enrolled June 1 who graduate June 15

% enrolled Sept 30 who graduate June 15

% enrolled 9th grade who graduate in 12th grade

Select 3 to 5 Performance Measuresat each level of the organization

3 - 5 3 - 5 3 - 5

3 - 5 3 - 5 3 - 5

3 - 5

3 – 5 most important

or composites

“Get over it!”Be disciplined about what’s most important. Don’t get distracted.

?

. .

Contributionrelationship

Alignmentof measures

Appropriateresponsibility

THE LINKAGE Between POPULATION and PERFORMANCE

POPULATION ACCOUNTABILITY

Healthy Births Rate of low birth-weight babiesChildren Ready for School Percent fully ready per K-entry assessmentWorld-class workforce Percent of adults with higher education degrees

CUSTOMERRESULTS

# students Student / teacher

ratio

# with5 A-C GCSE’s

% with5 A-C GCSE’s

PERFORMANCE ACCOUNTABILITY

POPULATIONRESULTS

Local Education Agency

Performancemeasure

PopulationIndicator

Division #1

Program #1

Different Kinds of Progress

1. Data

a. Population indicators Actual turned curves: movement for the better away from the baseline.

b. Program performance measures: customer progress and better service: How much did we do?

How well did we do it? Is anyone better off?

2. Accomplishments: Positive activities, not included above.

3. Stories behind the statistics that show how individuals are better off.

Board of Directors MeetingAGENDA

1. New data

2. New story behind the curves

3. New partners

4. New information on what works.

5. New information on financing

6. Changes to action plan and budget

7. Adjourn

1. New data

2. New story behind the curves

3. New partners

4. New information on what works.

5. New information on financing

6. Changes to action plan and budget

7. Adjourn

3 - kinds of performance measures. How much did we do? How well did we do it? Is anyone better off?

RBA in a Nutshell2 – 3 - 7

2 - kinds of accountability Population accountability Performance accountability

7 - questions from ends to means in less than an hour.

plus language discipline Results & Indicators Performance measures

Results based accountability 101 active (2009)

Business

healthy city

safe city

quality of change effect

graduation number

city of inclusive

city of people

number of persons

safe communities