Using the Community Guide to Move the Research Agenda Forward Peter A. Briss, MD, MPH February 03, 2005
Jan 01, 2016
Using the Community Guide to Move the Research Agenda
Forward
Peter A. Briss, MD, MPH
February 03, 2005
Why Evidence-Based Public Health?
Resources are tight … and getting tighter
Public health is more visible—therefore our decisions are more carefully examined
Increasing pressure to be accountable
Gaps between scientists and decision-makers—priorities, language, and approaches
Increasing pressure to embrace “evidence” methods
Evidence and Public Health Decision Making
Good news: Major efforts underway to assess the body
of evidence for a wide range of public health interventions
More and more high quality reviews available
But capacity not what it might be
Strong evidence on the effect of many policies/programs aimed to improve public health
But…Awareness and Use Are Not What They Might Be
Bad news: Many public health professionals are unaware
of this evidence
Some who are aware of it don’t use it
Many existing disease control programs use interventions with insufficient evidence, while better-documented alternatives are available
Failing to use an effective intervention is a missed opportunity that can adversely affect fulfilling mission and getting public support
The Community Guide Seeks to Answer Many Important Questions:
Under what circumstances is an intervention appropriate?
Does it work?
How well?
For whom?
What does it cost?
Does it provide value?
Are there other barriers I need to know about?
So What Does One Do with This Kind of Information?
Know what to expect
Know which programs are more likely to be successful
Support decisions about research What programs need additional research to
support decisions? What research is needed, (e.g., formative,
effectiveness, replication, or dissemination)?
Advise program planners and evaluators
Essential Information, But Only One Piece Of The Puzzle
Community assessment
Priority setting
Objective setting
Intervention selection
Implementation
Evaluation
Repeating the cycle
What’s Been Accomplished So Far? 171 findings to date
More in the pipeline…
Book publication in Jan 2005: Oxford University Press
People are beginning to use the Community Guide as a starting place to access evidence-based prevention advice
Beginning to see effects on practice, policy, research
What’s Been Published Relevant to Cancer Prevention?
Primary Prevention Tobacco Use (2000, 2001, in preparation)
Physical Activity (2001, 2002, in preparation)
Skin Cancer Prevention (2003, 2004)
Improving Vaccination Coverage (1999, 2000, in press)
Improving processes of health care Promoting Informed Decision Making
(2004)
Culturally-competent health care (2003, in preparation)
Population-based interventions for the detection of oral cancer (2001, 2002)
What’s Been Published Relevant to Cancer Prevention? (cont’d)
More on the Way Early phase
Alcohol Worksite Health Promotion
Midcourse HIV Sexual Behavior Nutrition
Late course Obesity Promoting Cancer Screening
We Know Less About Moving Research Forward Than Practice
Collaboration between the Community Guide and the network is an evolving work in progress
Need for ongoing dialogue: What does the network need?
formats? detail? additional information?
We’d also like to get feedback from you that might influence our more general reviews or communications
Still Building the Airplane . . .While We’re Flying
We have only about 4 years of experience in trying to use the Community Guide to move research forward in a variety of areas, but I’ll talk generally about some potential uses I recommend you also read chapter 12
in the book
Research Phases: Health Promotion Programs (After NCI And NHLBI)
1. Basic research —
2. Hypothesis development —
3. Pilot applied studies Very small scale
4. Prototype studies Experimental or Q-ExpSmall scale
5. Efficacy trials ExperimentalNumbers sufficient for behavioral evaluation
6. Treatment effectiveness trials Exp or Q-ExpWith outcomesSTD deliveryLarge scale, real world
7. Implementation effectiveness As above (#6)trials Several types of delivery
8. Demonstration studies As above (#6)Unrestricted population(s)
This Research-to-Practice View Is Useful But Incomplete
The world is not linear-sequential
No place to put synthesis steps
More consistent with “programs that work” models than with synthesis can’t say much about characteristics that
contribute to success or failure
Based primarily on science push and little on user pull
No place to put research that might follow demonstration of effectiveness
Perform Research Appropriate to the Stage of Progress of the Field
Define the problem Identify targets of intervention Develop theory-based interventions
and taxonomy Evaluate effectiveness
Perform Research Appropriate to the Stage of Progress of the Field (cont’d)
Consider: Targeted replication research that answers important
new questions Whether applicability can be broadened and, if so,
what is required
Targeted dissemination research
Other “post-effectiveness” research questions Research and support for improving fit
Cost and cost effectiveness
Identification and reduction of implementation barriers
What else?
Testing/production/dissemination of “how to”
materials
How Can Reviews Help to Inform Additional Research
Identify what is already known and where are the remaining gaps:
Object is to move a field downstream
Hope is to help identify “low hanging fruit”
better complement work that has already been done
Identify opportunities to kill multiple birds with one stone
For example, replication research might be paired with work on economics or identification and reduction of barriers
A Case Story
There are now many examples of implementation of Community Guide and follow-up evaluative or research efforts
Designing new studies to add to what’s already known is harder than it appears
A Case Story
In 2000, the TF recommended client reminders as one of several client-oriented interventions to improve coverage with vaccines that are recommended for everyone in a particular age group (i.e., universally-recommended vaccines)
What Was The Evidence?
31 intervention arms of reminders used alone produced a median improvement in coverage of 8 pct pts (range –7 to 31 pct pts)
Intervention characteristics, populations, settings were diverse
What Else Did the Task Force Say?
Should be applicable to most adults and children in the US for whom universally recommended vaccines are applicable and in whom improvements in coverage are needed
Suggested a 4-step process for implementing recommended interventions Assess current intervention activities and needs
Assess barriers to vaccination
Select interventions that address local barriers “Using additional interventions when coverage is already
high or using additional interventions that are poorly matched to local problems are unlikely to result in important benefits”
Monitor progress and effects Adequate implementation?
Periodically reassess and adjust
What Else Did the Task Force Say?
Client Reminders for Adult Flu Shots: Methods
Site
Sample and design
Data collection
3 Health Plans in CT
~9500 high-risk adults, 18–64 yrs 55% response rate
Mail survey
Challenges: Implementation
Little formal or informal a priori assessment of locally-important barriers to vaccination due to time and other constraints
The fit of this intervention to locally-important problems was largely unknown
Client Reminders for Adult Flu Shots:Additional Information
Most (55%) of the people who did not get vaccinated this time had never been vaccinated Might require additional strategies
Previously vaccinated people who were not vaccinated most commonly reported access barriers for which a reminder might not be expected to provide substantial help
If This Was An Effectiveness Study
Change in coverage below the median but well within the reported range
If This Was A Replication Study
Were these results importantly different from what was expected?
If so, why? Population (barriers, coverage) Setting (IPA) Intervention (type, implementation,
something else?) What we learn from this addition is harder
to interpret than I might have expected
If This Was A Dissemination Study
Identification of several important implementation barriers Ensuring fit Implementing a reminder in the way it
was defined in the guide We learned less about how to
address the barriers
Opportunities for Improvement
Improved communications between guideline developers, scientists, implementers, and decision makers
Better positioning of recommendations as part of a portfolio of resources to support decision making
Better positioning of intervention selection as part but not all of comprehensive program planning
Probably broaden the range of questions that are addressed by “replication research”
This Network Will Have A Balanced Portfolio of 4 Main
Areas Of Study “Nearly sufficient” Replication Dissemination Evaluation
“Nearly Sufficient Evidence”
One or two well done studies could provide sufficient evidence for a recommendation
Evidence of Effectiveness
Quality of Execution
Design Suitability
Number of Studies
Consistent Effect Size
1. Strong
Good Greatest > 2 Yes Sufficient
Good Greatest or Moderate
> 5 Yes Sufficient
Good or Fair Greatest > 5 Yes Sufficient
Meet criteria for sufficient evidence Large
2. Sufficient
Good Greatest 1 -- Sufficient
Good or Fair Greatest or Moderate
> 3 Yes Sufficient
Good or Fair Greatest, Moderate or Least
> 5 Yes Sufficient
3. Insufficient Insufficient design or execution
Too few No Small
Translating to Recommendations
Examples “Nearly Sufficient”
Small numbers of studies trending positive
Few existing studies Coded yellow
Likely Have More “Nearly Sufficient” Examples Than Can
Be Immediately Funded Likely to need additional priority
setting criteria, e.g., Commonly done by programs (DCPC
survey) Already in the PLANET “Hot topics” “High stakes” Controversial
Replication Research
Replicate recommended interventions in populations or community settings in which they have not been previously evaluated, Underserved populations Health departments and other cancer control
partners.
Consider whether you also want to evaluate particular intervention subtypes
Examples (Replication Research)
Some fundamental questions have been addressed rarely B+C
Effectiveness among never-screened CRC
Effectiveness in promoting screening other than FOBT
We Could Use Some Feedback
What applicability information would be most useful to you? Types of information
Population, Setting, Intervention Level of detail
We’re willing to pull more info if needed
Other Ways To Set Priorities
Commonly done by programs (DCPC survey)
Not yet in the PLANET Data set missing a characteristic of
setting or population that is essential from the perspective of the B+C program
How To Effectively Disseminate
Research on how to effectively disseminate or implement within health departments or with community groups or other cancer control partners Guide-recommended community interventions
Examples Relevant To Dissemination
Research that identifies and addresses barriers to implementation
Identification and sharing (e.g., on the PLANET) of useful “tools”
Other related research, e.g., on cost or cost effectiveness Very little economics thus far except for
reminders
Evaluations Of Recommended Interventions Already
Implemented Evaluate fidelity to recommended
interventions Determine, as much as possible, if
they are as effective as might be expected
Examples (already implemented)
Surveys of programs about what they say they’re currently doing (or not doing)
Audits of what they’re actually doing (or not doing)
Checks of whether programs match what was recommended
Identification and sharing (e.g., on the PLANET) of useful “tools” (i.e., “how to” advice)
Potential Priorities for Evaluation of Already-Implemented Interventions
Recommended interventions that are commonly practiced (e.g., based on the DCPC survey)
Interventions that are not commonly practiced for which identification of sharable tools might help
Identification of barriers and ways to overcome them
Gaps between research and program and program and research keep us from making the most of investments in each of these
This project has great potential to help narrow the gaps
What sets of recommendations are
most relevant to the CPCRN? For
example, is anything with cancer
relevance potential grist for the
Network?
Do we want to focus on certain
categories of evidence, e.g. sufficient,
strong or insufficient?
Should we focus on particular kinds
of insufficient evidence, e.g. where
there are
unresolved issues, e.g., minorities?
Should the Network make national
selection in topic areas and apply
nationally/locally?
How do we go from the national
level recommendations to
regional/local implementation?
For example, if we are to
disseminate or replicate programs,
do we give first priority to CPCRN
member programs?
How do we go from the national level
recommendations to regional or local
implementation?
For example, if we are to disseminate or
replicate programs, do we give first
priority to CPCRN member programs? Should
they have appeared in PLANET? Do we want
to recommend strongly that members register
for PLANET when requested to do so?