Program Evaluation in Public Health California’s Efforts to Reduce Tobacco Use 1989-2005 David Hopkins Terry Pechacek.

Program Evaluation in Public Health

California’s Efforts toReduce Tobacco Use 1989-2005

David Hopkins

Terry Pechacek

California in 1988

Population: 28,400,000

Adult smoking prevalence: 22.8%

4.8 million adult smokers

A Funded Mandate

Voters approved ballot initiative in November, 1988 Excise tax increase of 25 cents/pack Earmarked funding (20%) for a statewide

program

What to do, how to do it, and how to evaluate it?

The Status of Tobacco Control, 1988

Limited experience with effective population-based interventions Clinical interventions (low success

rates, relapses were common)

Price (published econometric studies)

Mass media (Fairness Doctrine Campaign 1967-1970)

California Had Program Options

A top-down program Interventions selected and implemented

by the Tobacco Program statewide

A program built on the results of smaller-scale demonstration projects Trials would help to determine the

independent impact of intervention options

A Comprehensive Approach was Advocated by NCI* (and others)

Funding of community coalitions Local emphasis and control

Multiple channels of intervention Multiple targets of intervention

A field test was needed

* “Standards for Comprehensive Smoking Prevention and Control”

National Cancer Institute

Multiple Channels, Multiple Targets

Target

Goals

Media

Campaigns

Smoke-free

Policies

Community

Activities

School-based

Programs

Increase

Cessation X (X) X (X)

Reduce Initiation X (X) X X

Reduce

Exposure to ETS

X X X (X)

Interventions

The California Tobacco Prevention and Control Efforts, 1990

An excise tax (price increase of 23%)

Paid mass media campaign Funding for community organization and

interventions (67 Local Lead Agencies)

Funding for school-based programs Funding for intervention and treatment research

Challenges in Evaluation

A Public Health Example

Evaluation was Built into the Mandate

Some surveillance systems were in place BRFSS; State cigarette tax receipts

California added some more California Tobacco Surveys

Programs (components) were evaluated through contracts (independent evaluators)

A research program was funded within the University of California

Local Evaluation was Included

Funding for local intervention and research projects came with strings… 10% of budget to be spent on evaluation

…and with support Directory of experts for consultation or to

conduct evaluations Database of instruments and information Annual conferences

Oversight was Established

Appointed committee Annual review of surveillance and research

results Advice and recommendations Periodic publications summarizing program

progress and direction

Guardians

Evaluations of the California Program: 1990-2004

Outcomes, Study Designs, and Program Results

Smoking Prevalence among Adults in California Decreased 32.5% between 1988-2004

0

5

10

15

20

25

30

88 89 90 91 92 93 94 95 96 97 98 99 0 1 2 3 4

22.8%

15.4%Definition Changed

Source: California DHS 2005 Year

Percent

Consumption Decreased 55.6% in California (compared to 32% in the rest of the US) 1988-2003

0

50

100

150

200

85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 0 1 2 3

141

45.8

California

Source: California DHS 2003 Year

Packs / Person

Rest of US

Comparison of Age-Adjusted Rates of Death From Heart Disease: California 1979-1998

100

110

120

130

140

150

160

170

180

190

79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98

California

Source: Fichtenberg and Glantz; NEJM 2000 Year

Age-Adjusted Heart Disease Mortality per 100,000

Predicted US rates

Some Interventions Have Been Evaluated: California’s Telephone Quitline

Design: RCT Analysis:

Intention to treat

Receipt of counseling Inter arm: 72.1% Comp arm: 32.6%

Study Arm N

12m

QuitComp 1309 6.9%

Inter 1973 9.1%

Source: Zhu et al. NEJM 2002

Differences: P<0.001 by log-rank test

Prolonged Abstinence by Study Arm

Evaluation: What Has Gone Well

Provided dozens of publications influencing tobacco prevention and control efforts

Documented the overall impact of a comprehensive tobacco control effort

Documented the independent contributions of some components Helpline Smoke-free policies

Contributed to Program survival

Evaluation: What Has Not Gone Well

Local program impact is still unclear Comparisons have been difficult Most evaluations have not been published

The effectiveness of some interventions remains unclear School-based programs

Evaluation: Adjustments

Adopted more uniform surveillance tools Combined BRFSS and CTS survey results since

1993

New questions provide new directions Smoke-free policies (work, home)

California’s Program Became the Model

Other States adapted the comprehensive State-level approach to tobacco control Massachusetts Florida Arizona Oregon

California’s experience contributed to the contents of CDC’s Best Practices Guideline in 1998

Discussion

Additional Slides

Targeted Outcomes and Measurements

Increasing

Cessation

Reducing Initiation

Reducing

Exposure

Targeted Goals Outcome Measurement Options

Population

ConsumptionPrevalence

Cessation

(Smokers)

PrevalenceYouth

(students)

Policies Exposures

Home Work Home Work

California Tobacco Control Section Funding, 2001-2002 ( $106.5 m )

Admin 1.7m (1.6%)

Evaluation 6.3m (6%)

Media 45.2 m (42%)

Competitive Grants

35.7m (33.5%)

Local Lead Agencies

17.4 m (16 %)

Tobacco Control is Always Outspent by the Industry

0

10

20

30

40

50

60

90 91 92 93 94 95 96 97 98 99 0 1 2

Year

California Tobacco IndustryPer Capita Expenditures ($)

Source: California DHS 2004

Creating an Environment to Reduce Tobacco Use

Smoker Quit Success

Contemplation

Relapse

Modified from: Population-based Smoking Cessation. NCI Monograph 12; 2000

Creating an Environment to Promote Cessation

Advice Treatments

Smoker Quit Success

Counseling

Modified from: Population-based Smoking Cessation. NCI Monograph 12; 2000

Creating an Environment to Promote Cessation

Community

Events

Smoke-free Policies

Costs

Media Messages

Worksite

Events

Advice Treatments

Quitline

Smoker Quit Success

Counseling

Modified from: Population-based Smoking Cessation. NCI Monograph 12; 2000