Top Banner
Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders amazon.com resultsleadership.org
110

Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Mar 26, 2015

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Outcome BasedAccountability

The Fiscal Policy Studies InstituteSanta Fe, New Mexico

Websitesraguide.org

resultsaccountability.com

Book - DVD Ordersamazon.com

resultsleadership.org

Page 2: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

SIMPLE

COMMON SENSE

PLAIN LANGUAGE

MINIMUM PAPER

USEFUL

Page 3: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Outcome Based Accountabilityis made up of two parts:

Performance Accountabilityabout the well-being of

CUSTOMER POPULATIONSFor Programs – Agencies – and Service Systems

Population Accountabilityabout the well-being of

WHOLE POPULATIONSFor Communities – Cities – Counties – States - Nations

Page 4: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Results and Performance Accountability

COMMON LANGUAGE

COMMON SENSE

COMMON GROUND

Page 5: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

THE LANGUAGE TRAPToo many terms. Too few definitions. Too little discipline

Benchmark

Target

Indicator Goal

Result

Objective

Outcome

Measure

Modifiers Measurable Core Urgent Qualitative Priority Programmatic Targeted Performance Incremental Strategic Systemic

Lewis Carroll Center for Language DisordersMeasurable urgent systemic indicatorsCore qualitative strategic objectivesYour made up jargon here

Page 6: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

DEFINITIONS

Being Healthy - Staying Safe - Enjoying and achievingMaking a Positive Contribution - Economic Well-being

Rate of low-birthweight babies - Rate of confirmed child abuse Percent 16 & 19 yr. olds with 5 A-C GCSE’s - Rate of volunteering Rate of child poverty

1. How much did we do? 2. How well did we do it? 3. Is anyone better off?

OUTCOME

INDICATOR

PERFORMANCE MEASURE

A condition of well-being for children, adults, families or communities.

A measure which helps quantify the achievement of an outcome.

A measure of how well a program, agency or service system is working. Three types:

= Customer Outcome

Popu

latio

nPe

rfor

man

ce

Safe Communities - Clean Environment

Crime rate - Air quality index

Page 7: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

From Ends to Means

ENDS

MEANS

From Talk to ActionPo

pula

tion

Perf

orm

ance

OUTCOME

INDICATOR

PERFORMANCEMEASURE

Customer outcome = EndsService delivery = Means

From Talk to Action

Page 8: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

1. Safe Community

2. Crime Rate

3. Average Police Dept response time

4. An educated workforce

5. Adult literacy rate

6. People have living wage jobs and income

7. % of people with living wage jobs and income

8. % of participants in job training who get living wage jobs

IS IT A OUTCOME, INDICATOR OR PERFORMANCE MEASURE?

OUTCOME

INDICATOR

PERF. MEASURE

OUTCOME

INDICATOR

OUTCOME

INDICATOR

PERF. MEASURE

Page 9: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Results – Indicators – Performance Measures in

Amharic, Cambodian, Laotian, Somali, Spanish, Tigrigna, Vietnamese

Page 10: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.
Page 11: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Translation Guide/Rosetta StoneNot the Language Police

Ideas

1. A condition of well-being for children, adults, families & communities

2.

3.

etc.

Group 1 Group 2 Group 3 etc.

OUTCOME RESULT GOAL

TRANSLATION

Back to the Idea

Page 12: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

POPULATIONACCOUNTABILITY

Fiscal Policy Studies InstituteSanta Fe, New Mexicowww.resultsaccountability.comwww.raguide.org

For Whole Populationsin a Geographic Area

Page 13: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Community Outcomesfor Christchurch, NZ

1. A Safe City

2. A City of Inclusive and Diverse Communities

3. A City of People who Value and Protect the Natural Environment

4. A Well-Governed City

5. A Prosperous City

6. A Healthy City

7. A City for Recreation, Fun and Creativity

8. City of Lifelong Learning

9. An Attractive and Well-Designed City

Page 14: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Outcomes forChildren, Families and Communities

A Working List● Healthy Births

● Healthy Children and Adults

● Children Ready for School

● Children Succeeding in School

● Young People Staying Out of Trouble

● Stable Families

● Families with Adequate Income

● Safe and Supportive Communities

Page 15: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Georgia Policy Councilfor Children and Families

RESULTS

● Healthy Children

● Children Ready for School

● Children Succeeding in School

● Strong Families

● Self Sufficient Families

Page 16: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Every Child Matters – Children Act

Outcomes for Children and Young People

Being Healthy: enjoying good physical and mental health and living a healthy lifestyle.

Staying Safe: being protected from harm and neglect and growing up able to look after themselves.

Enjoying and Achieving: getting the most out of life and developing broad skills for adulthood.

Making a Positive Contribution: to the community and to society and not engaging in anti-social or offending behaviour.

Economic Well-being: overcoming socio-economic disadvantages to achieve their full potential in life.

Page 17: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

1. Health & Emotional Well-being

2. Quality of life

3. Making a positive contribution

4. Exercising choice & control

5. Freedom from discrimination & harassment

6. Economic well being

7. Personal dignity & respect

Our Health, Our Care, Our Say – White Paper

Outcomes for Adults

Source: A New Outcomes Framework for Performance Assessment of Adult Social Care 2006 - 07

Note that 8. Effective leadership and (9. Effective commissioning are means and not ends

Page 18: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Georgia

Lehigh Valley, PADayton, OH

Santa Cruz, CA

REPO

RT C

ARDS

Page 19: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

New Zealand Kruidenbuurt Tilburg, Netherlands

Portsmouth, UK

Country Neighborhood

City

Page 20: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.
Page 21: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Leaking Roof(Outcomes thinking in everyday life)

Experience

Measure

Story behind the baseline (causes)

Partners

What Works

Action Plan

Inches of WaterBASELINE

? Fixed

Not OK

Turning the Curve

Action Plan #2

Page 22: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Outcomes

Page 23: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.
Page 24: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Criteria for

Choosing Indicatorsas Primary vs. Secondary Measures

Communication Power

Proxy Power

Data Power

Does the indicator communicate to a broad range of audiences?

Does the indicator say something of central importance about the outcome?

Does the indicator bring along the data HERD?

Quality data available on a timely basis.

Page 25: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Choosing IndicatorsWorksheet

Outcome _______________________________

Candidate IndicatorsCommunication

PowerProxyPower

DataPower

H M L

H

Measure 1

Measure 2

Measure 3

Measure 4

Measure 5

Measure 6

Measure 7

Measure 8

HData

Development

Agenda

Safe Community

H M L H M L

H H

H L

Page 26: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Three Part Indicator List for each Outcome

Part 1: Primary Indicators

Part 2: Secondary Indicators

Part 3: Data Development Agenda

● 2 or 3 or 4 “Headline” Indicators● What this outcome “means” to the community● Meets the Public Square Test

● Everything else that’s any good (Nothing is wasted.)● Used later in the Story behind the Curve

● New data● Data in need of repair (quality,timeliness etc.)

Page 27: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

The Matter of Baselines

Baselines have two parts: history and forecast

H

M

L

History Forecast

Turning the CurvePoint to Point

OK?

Page 28: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

The Cost of Bad Outcomes

Invest in prevention to reduce or avoid out-year costs.

The costs of remediating problems after they occur

InvestmentTrackCost

$300 billion

Revenue

Convergenceof Cost & Revenue

Page 29: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

2005

COST OF BAD OUTCOMESUnited States 1970 to 2010

Page 30: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

6.00%

8.00%

10.00%

12.00%

14.00%

16.00%

18.00%

20.00%

Ncle 14.5 14.5 16.8 14.5 17 15 11.9 10.6 9.5 9.3

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

Source: Connexions Tyne and Wear, UK

Newcastle, UK

Revised 9 Nov 2007

Page 31: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Figure 5: Pre and Post Turning the Curve ExerciseBreakdown of offences committed for the data periods 01/01/04 -

31/03/04 and 01/01/05 - 31/03/05

0255075

100125150175200225250

01/01/04 - 31/03/04 01/01/05 - 31/03/05

Criminal Damage Public Order Theft Overall Decrease

Paulsgrove, Portsmouth, UK

Youth Nuisance

Page 32: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

MADD

Page 33: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Rebound

Page 34: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.
Page 35: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.
Page 36: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Alternative to Traditional Evaluation Methods:DEMONSTRATING a CONTRIBUTION

to complex change efforts… requires 3 elements:

Given a Curve to Turn

1 We tried a bunch of stuff that had a credible

chance of making a difference …

2 …and it had a timely relationship to….

3 …. a turn in the curve.

c FPSI

Page 37: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

- Rosell

“If I include you,you will be my partner.

If I exclude you,you will be my judge.”

Page 38: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Performance Accountability

For Services, Agencies and Service Systems

Fiscal Policy Studies InstituteSanta Fe, New Mexicowww.resultsaccountability.comwww.raguide.org

Page 39: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

“All Performance Measures

that have ever existed

for any service

in the history of the universe

involve answering two sets of

interlocking questions.”

Page 40: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

HowMuchdid we do?

( # )

HowWell

did we do it?

( % )

Quantity Quality

Program Performance Measures

Page 41: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

EffortHow hard did we try?

EffectIs anyone better off?

Performance Measures

Page 42: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Effort

Effect

HowMuch

HowWell

Program Performance Measures

Page 43: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much service did we deliver?

Program Performance Measures

How welldid we

deliver it?

How much change / effect

did we produce?

What quality of change / effect

did we produce?

Quantity Quality

Effe

ct

Effo

rt

O

utpu

t

In

put

Page 44: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

Program Performance Measures

How welldid we do it?

Is anyonebetter off?

Quantity Quality

Effe

ct

Effo

rt

# %

Page 45: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

Education

How well did we do it?

Is anyone better off?

Quantity Quality

Effe

ct

E

ffort Number of

studentsStudent-teacher

ratio

Number of 16 olds with 5 A to C

GCSE’s

Number of fixedterm exclusions

Percent of 16 yr olds with 5 A to C

GCSE’s

Rate of fixedterm exclusions

Page 46: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

Education

How well did we do it?

Is anyone better off?

Quantity Quality

Effe

ct

E

ffort Number of

studentsStudent-teacher

ratio

Number of 16 olds with 5 A to C

GCSE’s

Number with goodschool attendance

Percent of 16 yr olds with 5 A to C

GCSE’s

Percent with goodschool attendance

Page 47: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

Pediatric Practice

How well did we do it?

Is anyone better off?

Number ofpatientstreated

Percent ofpatients treated

in less than1 hour

Quantity Quality

Effe

ct

E

ffort

#children

fullyimmunized

(in the practice)

%children

fullyimmunized

(in the practice)

Page 48: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

Drug/Alcohol Treatment Program

How well did we do it?

Is anyone better off?

Number ofpersonstreated

Percent ofstaff withtraining/

certification

Number of clientsoff of alcohol & drugs - at exit - 12 months after exit

Percent of clientsoff of alcohol & drugs - at exit - 12 months after exit

Quantity Quality

Effe

ct

E

ffort

Page 49: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

Home Care Services

How well did we do it?

Is anyone better off?

# of persons served

# visits

# visits per week

Ave length of visit

% service users who say the service is reliable, flexible, respectful, friendly (SPRU))

Rate entry into hospital, residential or nursing care

% carers report the service helped the service user remain at home

# entry into hospital, residential or nursing care

# carers report the service helped the service user remain at home

Page 50: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

Fire Department

How well did we do it?

Is anyone better off?

Number ofresponses

ResponseTime

Quantity Quality

Effe

ct

E

ffort

# of fireskept to

room of origin

% of fireskept to

room of origin

Page 51: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

General Motors

How well did we do it?

Is anyone better off?

# of production hrs

# tons of steel

Employees pervehicle

produced

# of cars sold

$ Amount of Profit

$ Car value after 2 years

Quantity Quality

Effe

ct

E

ffort

Source: USA Today 9/28/98

% Market share

Profit per share

% Car value after 2 years

Page 52: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

Not All Performance Measures Are Created Equal

How well did we do it?

Is anyone better off?

LeastImportant

Quantity Quality

Effe

ct

E

ffort

MostImportant

Least

Most

AlsoVery Important

Quality

Page 53: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

RBA Categories Account for All Performance Measures(in the history of the universe)

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Product Output Impact

Process Input

Effe

ctEf

fort

Cost

TQM

Effectiveness

Efficiency

Page 54: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Process Input

Effe

ctEf

fort

Cost

TQM

Product Output Impact

RBA Categories Account for All Performance Measures(in the history of the universe)

Page 55: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investmentClient results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Process Input

Effe

ctEf

fort

Cost

TQM

1. Did we treat you well?

2. Did we help you with your problems?

*

Product Output Impact

RBA Categories Account for All Performance Measures(in the history of the universe)

* World’s simplest completecustomer satisfaction survey

Page 56: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Process Input

Effe

ctEf

fort

Cost

TQM

Product Output Impact

RBA Categories Account for All Performance Measures(in the history of the universe)

Page 57: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

The Matter of Control

How well did we do it?

Is anyone better off?

Quantity Quality

Effe

ct

E

ffort

LeastControl

PARTNERSHIPS

MostControl

Page 58: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

The Matter of Use

1. First Purpose is to Improve Performance as a contribution to improving results

2. Avoid the Performance Measurement Equals Punishment Trap

● Acknowledge the experience as real.

● Work to create a healthy organizational environment

● Start small.

● Build bottom-up and top-down simultaneously.

Page 59: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

1. To Ourselves First Can we do better than our own history?

2. To Others When it is a fair apples/apples comparison.

3. To Standards When we know what good performance is.

Comparing Performance

Page 60: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

2. To Others When it is a fair apples/apples comparison.

3. To Standards When we know what good performance is.

Comparing Performance

1. To Ourselves First

Can we do better than our own history?

Using a BaselineCHART ON THE

WALL

Page 61: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Comparing Performance

1. To Ourselves First Can we do better than our own history?

2. To Others When it is a fair apples/apples comparison.

Reward?

Punish?

3. To Standards When we know what good performance is.

Page 62: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

1. To Ourselves First Can we do better than our own history?

2. To Others When it is a fair apples/apples comparison.

Comparing Performance

3. To Standards When we know what good performance is.

Page 63: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

The Matter of Standards

Quantity

Effe

ctEf

fort

1. Quality of Effort Standards aresometimes WELL ESTABLISHED

● Child care staffing ratios● Application processing time● Handicap accessibility● Child abuse response time

2. Quality of Effect Standards arealmost always EXPERIMENTAL

● Hospital recovery rates ● Employment placement and retention rates● Recidivism rates

3. Both require aLEVEL PLAYING FIELD and an ESTABLISHED RECORDof what good performance is.

BUT

AND

Page 64: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Advanced Baseline Display

Your Baseline

Comparison Baseline

Goal (line)Target or Standard

Instead:Count anything better than baseline as progress.

Avoid publicly declaringtargets by year if possible.

Create targets only when they are:

FAIR & USEFUL

Page 65: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

Program Performance Measures

How welldid we do it?

Is anyonebetter off?

Quantity Quality

Effe

ct

Effo

rt

# %

Page 66: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Lay

Definition

All Data have two Incarnations

Technical

Definition

Graduation Rate % enrolled June 1 who graduate June 15

% enrolled Sept 30 who graduate June 15

% enrolled 9th grade who graduate in 12th grade

Page 67: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

Separating the Wheat from the ChaffTypes of Measures Found in Each Quadrant

How well did we do it?

Is anyone better off?

# Clients/customers served

# Activities (by type of activity)

% Common measurese.g. client staff ratio, workload ratio, staffturnover rate, staff morale, % staff fully trained, % clients seen in their own language,worker safety, unit cost

% Skills / Knowledge (e.g. parenting skills)

#

% Attitude / Opinion (e.g. toward drugs)

#

% Behavior (e.g.school attendance)

#

% Circumstance (e.g. working, in stable housing)

#

% Activity-specific measures

e.g. % timely, % clients completing activity, % correct and complete, % meeting standard

Point in Time vs. Point to Point

Improvement

Page 68: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How much did we do?

Choosing Headline Measures and the Data Development Agenda

How well did we do it?

Is anyone better off?

Quantity Quality

E

ffect

Effo

rt

# Measure 1 ----------------------------

# Measure 2 ----------------------------

# Measure 3 ----------------------------

# Measure 4 ----------------------------

# Measure 5 ----------------------------

# Measure 6 ----------------------------

# Measure 7 ----------------------------

#1 Headline

#2 Headline

#3 Headline

#1 DDA

#2 DDA

#3 DDA% Measure 8 ----------------------------

% Measure 9 -----------------------------

% Measure 10 ---------------------------

% Measure 11 ---------------------------

% Measure 12 ---------------------------

% Measure 13 ---------------------------

% Measure 14 ---------------------------

# Measure 15 ----------------------------

# Measure 16 ----------------------------

# Measure 17 ----------------------------

# Measure 18 ----------------------------

# Measure 19 ----------------------------

# Measure 20 ----------------------------

# Measure 21 ----------------------------

% Measure 15 ----------------------------

% Measure 16 ----------------------------

% Measure 17 ----------------------------

% Measure 18 ----------------------------

% Measure 19 ----------------------------

% Measure 20 ----------------------------

% Measure 21 ----------------------------

Page 69: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Select 3 to 5 Performance Measuresat each level of the organization

3 - 5 3 - 5 3 - 5

3 - 5 3 - 5 3 - 5

3 - 5

Pick the 3 – 5 most important of the 9 – 15 measures or create composites.

“Get over it!”Be disciplined about what’s most important. Don’t get distracted.

?

. .

Page 70: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Customer Outcomes

Page 71: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

LR

UR

Page 72: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.
Page 73: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.
Page 74: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

How Population

&Performance Accountability

FIT TOGETHER

Page 75: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Contributionrelationship

Alignmentof measures

Appropriateresponsibility

THE LINKAGE Between POPULATION and PERFORMANCE

POPULATION ACCOUNTABILITY

Being Healthy Rate of low birth-weight babiesStaying Safe Rate of child abuse and neglect Enjoying and Achieving Percent with 5 A-C GCSE’s

CUSTOMEROUTCOMES

# ofinvestigations

completed

% initiatedwithin 24 hrs

of report

# repeatAbuse/Neglect

% repeatAbuse/Neglect

PERFORMANCE ACCOUNTABILITY

POPULATIONOUTCOMES

Child Protection Service

Page 76: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Contributionrelationship

Alignmentof measures

Appropriateresponsibility

THE LINKAGE Between POPULATION and PERFORMANCE

POPULATION ACCOUNTABILITY

Healthy Births Rate of low birth-weight babiesChildren Ready for School Percent fully ready per K-entry assessmentSelf-sufficient Families Percent of parents earning a living wage

CUSTOMEROUTCOMES

# personsreceivingtraining

Unit costper person

trained

# who getliving wage jobs

% who getliving wage jobs

PERFORMANCE ACCOUNTABILITY

POPULATIONOUTCOMES

Job Training Program

Page 77: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.
Page 78: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Every timeyou make a

presentation,

use atwo-part

approach

Outcome: to which you contribute to most directly.

Indicators:

Story:

Partners:

What would it take?:

Your Role: as part of a larger strategy.

Population Accountability

Service:Performance measures:

Story:

Partners:

Action plan to get better:

Performance Accountability

Your Role

Page 79: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Every timeyou make a

presentation,

use atwo-partformat

Result: to which you contribute to most directly.

Indicators:

Story:

Partners:

What would it take?:

Your Role: within the larger strategy.

Population Accountability

Service:Performance measures:

Story:

Partners:

Action plan to get better:

Performance Accountability

Your Role

Page 80: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Different Kinds of Progress

1. Data

a. Population indicators Actual turned curves: movement for the better away from the baseline.

b. Service performance measures: customer progress and better service: How much did we do?

How well did we do it? Is anyone better off?

2. Accomplishments: Positive activities, not included above.

3. Anecdotes: Stories behind the statistics that show how individuals are better off.

Page 81: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.
Page 82: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.
Page 83: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.
Page 84: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Population Results

1. Population2. Results (Outcomes, Goals)3. Indicators (Benchmarks) Data Development Agenda Report Card4. Baseline5. Story behind the baseline Cost of Bad Results Research Agenda Part 1

6. Partners 7. What works Research Agenda Part 2

8. Action Plan (strategy)9. Funding Plan (budget)

Program Performance

1. Customers (Clients)2. Performance measures Customer results Quality of Effort Quantity of Effort Data Development Agenda3. Baseline4. Story behind the baseline Research Agenda Part 1

5. Partners6. What works Agency/program actions Partner's actions Research Agenda Part 2

7. Action Plan (strategy)8. Funding Plan

Framework:

__________

Framework Crosswalk Analysis

(For Population Well-being, Across Communities, Across Systems)

(For Programs, Agencies and Service Systems)

Example

Input

Activity

Output

Outcome

Goal

Logic Model

Page 85: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Board of Directors MeetingAGENDA

1. New data

2. New story behind the curves

3. New partners

4. New information on what works.

5. New information on financing

6. Changes to action plan and budget

7. Adjourn

1. New data

2. New story behind the curves

3. New partners

4. New information on what works.

5. New information on financing

6. Changes to action plan and budget

7. Adjourn

Page 86: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

What’s Next?A Basic Action Plan for Outcome Based Accountability

TRACK 1: POPULATION ACCOUNTABILITY

● Establish outcomes● Establish indicators, baselines and charts on the wall● Create an indicators report card● Set tables (action groups) to turn curves

TRACK 2: PERFORMANCE ACCOUNTABILITY

● Performance measures, and charts on the wall for programs, agencies and service systems● Use 7 Questions program by program in management, budgeting and strategic planning

Page 87: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

IN CLOSING

Page 88: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

“If you do what you always did,

you will get what you always got.”

Kenneth W. JenkinsPresident, Yonkers NY NAACP

Page 89: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

------Book - DVD Orders------amazon.com

resultsleadership.org

THANK YOU !Websites

raguide.orgresultsaccountability.com

Page 90: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

EXERCISES

Fiscal Policy Studies InstituteSanta Fe, New Mexicowww.resultsaccountability.comwww.raguide.org

Page 91: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

30

35

40

45

50

55

60

65

70

2001 2002 2003 2004 2005 2006 2007 2008

North West

England

North West incl Eng & Maths

England incl Eng & Maths

2008 National Target

GCSE 5+ A*-C

All Children and Young People in the NW Succeed in Literacy and Numeracyand Can Lead Productive Lives

PSA 10

This is our baseline

How do we maintain

momentum?

Page 92: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

20

25

30

35

40

45

50

55

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Und

er 1

8 co

ncep

tion

rate

s pe

r 10

00, 1

5-17

yea

r ol

d fe

mal

es

North West

England

2010 National Target

Under 18 Conception Rates

All Children and Young People in the NW Have Healthy Lifestylesand Productive Lives

PSA 3

This is our baseline

How do we turn the curve?

Page 93: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Manchester

North West

England

2

4

6

8

10

12

14

16

2004 2005 2006 2007 2008 2009 2010

Manchester

North West

England

Percentage NEET

All Children and Young People in the NW Can Lead Productive Lives

PSA 12Projected reduction required to meet 2010 target Actual Data

Page 94: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

60

65

70

75

80

2004 2005 2006 2007 2008

North West

England

2006 National Target

Percentage of 19 year olds with at least Level 2 qualification

All Children and Young People in the NW Can Lead Productive Lives

PSA 11

This is our baseline

How do we maintain momentum?

Page 95: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

5

6

7

8

9

10

2002-03 2003-04 2004-05 2005-06 2006-07 2007-08 2008-09 2009-10

North West

England

Percentage care leavers with 5+ GCSEs A*-C

All children and young people in the NW live in stable and safe environments and Can Lead Productive Lives

PSA

This is our baseline

Page 96: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Turn the Curve Exercise: Population Well-being

5 min: Starting Points - timekeeper and reporter - geographic area - two hats (yours plus partner’s)

10 min: Baseline - pick an outcome, and an indicator curve to turn - forecast – OK or not OK?

15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes

15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works

10 min: Report convert notes to one page

Two pointers to action

Page 97: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

ONE PAGE Turn the Curve Report: Population

Outcome: _______________Indicator

(Lay Definition)IndicatorBaseline

Story behind the baseline --------------------------- --------------------------- (List as many as needed)

Partners --------------------------- --------------------------- (List as many as needed)

Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost

SharpEdges

4. --------- Off the Wall

4. --------- Off the Wall

Page 98: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

The first step in performance accountability is to

DRAW A FENCEAround something that has

ORGANIZATIONAL OR FUNCTIONAL IDENTITY

The Whole Organization

Division ADivision B

Unit

Division C

FunctionUnit

1

Page 99: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Creating a Working Baselinefrom Group Knowledge

Now

Graduation Rate

75% Not OK?

BackcastingForecasting

Page 100: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Turn the Curve Exercise: Program Performance

5 min: Starting Points - timekeeper and reporter - identify a program to work on - two hats (yours plus partner’s)

10 min: Performance measure baseline - choose 1 measure to work on – from the lower right quadrant - forecast – OK or not OK?

15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes

15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works

10 min: Report Convert notes to one page

Two pointers to action

Page 101: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Service: _______________Performance Measure

(Lay definition)PerformanceMeasureBaseline

Story behind the baseline --------------------------- --------------------------- (List as many as needed)

Partners --------------------------- --------------------------- (List as many as needed)

Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost

ONE PAGE Turn the Curve Report: Performance

SharpEdges

4. --------- Off the Wall

4. --------- Off the Wall

Page 102: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

THANK YOU !

WEBSITES:

www.raguide.orgwww.resultsaccountability.com

BOOK ORDERS: www.trafford.comwww.amazon.com

Page 103: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

2 customers

1 measure

1 measure

Discuss how we’re doing

2 partners

2 what works ideas

(1 nc/lc)

Discuss how to implement

20 MinuteExercise

40

Page 104: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

1 population

1 result

1 indicator

Discuss how we’re doing

2 partners

2 what works ideas

(1 nc/lc)

Discuss how to implement

20 MinuteExercise

1. What population are we concerned about? (e.g. U.S. population)

2. What condition(s) do we want for this population? (Americans are free of cancer.)

3. How could we measure these conditions? ( (e.g. cancer rates)

4. How are we doing on the most important

of these measures? (baseline history)5. Who are the partners with a role to play in doing better?6. What works – what would it take – to do better? (What is our role?)7. What do we propose to do?

POPULATION ACCOUNTABILITY

Page 105: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Some datafor the

Turn the CurveExercises

Page 106: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

30

35

40

45

50

55

60

65

70

2001 2002 2003 2004 2005 2006 2007 2008

North West

England

North West incl Eng & Maths

England incl Eng & Maths

2008 National Target

GCSE 5+ A*-C

All Children and Young People in the NW Succeed in Literacy and Numeracyand Can Lead Productive Lives

PSA 10

This is our baseline

How do we maintain

momentum?

Page 107: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

20

25

30

35

40

45

50

55

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Und

er 1

8 co

ncep

tion

rate

s pe

r 10

00, 1

5-17

yea

r ol

d fe

mal

es

North West

England

2010 National Target

Under 18 Conception Rates

All Children and Young People in the NW Have Healthy Lifestylesand Productive Lives

PSA 3

This is our baseline

How do we turn the curve?

Page 108: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

Manchester

North West

England

2

4

6

8

10

12

14

16

2004 2005 2006 2007 2008 2009 2010

Manchester

North West

England

Percentage NEET

All Children and Young People in the NW Can Lead Productive Lives

PSA 12Projected reduction required to meet 2010 target Actual Data

Page 109: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

60

65

70

75

80

2004 2005 2006 2007 2008

North West

England

2006 National Target

Percentage of 19 year olds with at least Level 2 qualification

All Children and Young People in the NW Can Lead Productive Lives

PSA 11

This is our baseline

How do we maintain momentum?

Page 110: Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders.

5

6

7

8

9

10

2002-03 2003-04 2004-05 2005-06 2006-07 2007-08 2008-09 2009-10

North West

England

Percentage care leavers with 5+ GCSEs A*-C

All children and young people in the NW live in stable and safe environments and Can Lead Productive Lives

PSA

This is our baseline