State Health Access Reform Evaluation Lynn A. Blewett, Ph.D. State Health Access Data Assistance Center State Coverage Initiatives (SCI) National Meeting.

State Health AccessState Health Access

Reform EvaluationReform Evaluation

Lynn A. Blewett, Ph.D. State Health Access Data Assistance Center

State Coverage Initiatives (SCI) National Meeting

Nashville, Tennessee February 8, 2007

Funded by a grant from the Robert Wood Johnson Foundation

2

Presentation Objectives

• To get you to think about measurable outcomes during your discussions of health reform

• To be specific about your expectations for reform initiatives – How you will know if the program achieved its

objectives? What data will you need?

• To talk to each other and reach consensus on expectations…..

• Think about state-specific data needs and build those into your legislation

3

States Are in Different Phases of Reform Process

In

Discussion

Legislation Introduced

Legislation Passed/

Implementation

Post –Implementation

Colorado

Minnesota

Illinois (all)

New Jersey

New Mexico

PA (all)

WI (all)

California

Oregon

WI

Massachusetts

PA (kids)

Vermont

WI (kids)

Illinois (kids)

Maine

4

Framework for Reform

Public Sector

Initiatives

Employer-Based

Initiatives

Insurance

Reform

Tax credits

HIFA expansion waivers

SCHIP premium assistance

Small employer

purchasing pools

5

POLICY FORMULATION-Agenda Setting-Assessment of Options-Political Setting

POLICY IMPLEMENTATIONPOLICYAdoption

POLICY MODIFICATION

Program Evaluation, Assess impact of policiesInfluence future policy formulation

INPUTSPreferences of individuals, organizations, interest groups, along with cultural

Demographics, ethical, legal, social and technological inputs

Rulemaking Operation

Source: Longest, 1998Longest, 1988.Policy Making Process in the US

6

Performance Assessment or Program Evaluation?

• Performance Measurement– ongoing assessment

– use of performance standards

– early warning system to management

– vehicle for improving accountability to the public

• Program Evaluation– typically more in-depth examination of program

– broader context of environment and alternatives

– overall assessment of whether the program works

– identification of adjustments that may improve its results

Source: U.S. Government Accountability Office, 1998.

7

Levels of Measurement

• Population-based– Data collected from sample of target

population– Survey data, focus groups

• Program-based– Data collected from program recipients,

clients, or participants– Administrative data

8

Program Evaluation Data

• Quantitative Data– Surveys, secondary data (uninsured rate)– Measures activities carried out– Administrative data: program utilization,

behavior change, or cost per unit of change

• Qualitative Data– Interviews, focus groups, observations,

document review– Use to understand attitudes, beliefs, reactions,

to assess quality, to explain perceptions

9

Performance Assessment

• How do you know if your health reform was successful?

• Key Areas to Assess:– Coverage– Cost and Efficiency– Fairness and Equity– Choice and Autonomy

• Develop goals that are measurable, assessable and realistic.

• Consider short-term and long-term outcomes.

10

On Average Uninsurance Estimates from State Surveys are 22% Lower than CPS

11.2%

15.2%

12.1%

7.4%

14.2%

16.1% 16.4%

11.7%

0%

5%

10%

15%

20%

AL 2003 AR 2001 GA 2003 MA 2004

State Estimate CPS from State Survey Year

Uninsurance Estimates for Select States

11

Increased Sample Size in State SurveysHousehold Sample Sizes for Selected States: 2001 Expanded CPS Annual Demographic Supplement and State Initiated Surveys

States

Households Interviewed in Expanded Sample 2001 CPS

Households Interviewed in Recent State-Initiated Household Survey

Alabama 1,254 7,299 Arkansas 903 2,572 Colorado 1,522 10,000 Connecticut 1,339 3,985 Georgia 1,094 10,088 Hawaii 1,028 6,000 Indiana 1,389 10,126 Kansas 1,268 8,004 Maine 1,268 3,500 Maryland 1,273 5,000 Massachusetts 1,432 2,362 Montana 866 4,000 New Hampshire 1,281 5,177 South Carolina 1,014 1,600 Utah 932 7,250 Vermont 1,173 8,622 Washington 1,350 6,726 Wisconsin 1,481 2,436 * Source: Current Population Survey Demographic Supplement 2001

12

State Health AccessData Assistance Center

• Funded by the Robert Wood Johnson Foundation– University of Minnesota, School of Public Health

• Goals:– Help states monitor rates of health insurance coverage

using state and federal data – Research factors associated with access and coverage – Provide targeted policy analysis and technical

assistance to states

• Bridge between state and federal agencies; between survey data and state health policy

• Technical assistance on use of federal survey data and best practices on state surveys

13

State Health Access Reform Evaluation

• National Program Office of the RWJF – Co-located with SHADAC at University of MN, SPH

• Goals:– Fund and coordinate evaluations of state health reform

– Identify and fill gaps on research to identify what works and why

– Organize and disseminate findings in a manner that is meaningful and user-friendly

– Inform state and national policy on health care access and coverage

• Technical assistance on program evaluation and assessment

14

State Health Reform

If States are the Laboratories…..

Where are the lab

reports?

15

Concluding Comments

• Staff - Ask key legislative leaders what they will want to know next year?

• Discuss long- and short-term program objectives and identify the monitoring process up front

• Make sure staff understand legislative/policy objectives

• Make sure policy makers understand limitations of data/measurement

• Build performance assessment into legislative initiatives

16

More Concluding Comments

Don’t forget to think about…. • Performance assessment and

– Monitoring along the way– Likely within state government

• Program evaluation– How are we going to know in 2-3 years if the program

achieved its objectives?– Possible outside vendor/contractor

• Think now about what you will want to know later – It’s more difficult to go back and evaluate a program

without good baseline data and information

17

Federal DataFederal Data(e.g., CPS)(e.g., CPS)

Better Understanding of the Better Understanding of the Characteristics of the UninsuredCharacteristics of the Uninsured

State DataState Data(e.g., State Surveys)(e.g., State Surveys)

The SHADAC Vision

Increase Coverage and AccessIncrease Coverage and Access

THANK YOU!

18

Contact Information

www.shadac.org

www.statereformevaluation.org

University of MinnesotaSchool of Public Health

Division of Health Policy and Management

2221 University Avenue, Suite 345 Minneapolis Minnesota 55414

(612) 624-4802

Principal Investigator: Lynn A. Blewett, Ph.D. ([email protected])