Emory P r even tio n Research C enter Emory P r even tio n Research C enter Michelle Carvalho, MPH, CHES; Cam Escoffery, PhD, MPH, CHES; Louise Wrensford, PhD; Michelle Kegler, DrPH, MPH Georgia Public Health Association Annual Meeting April 12, 2011
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Frameworks for Translation of EvidencePassive diffusion is not enough to encourage the
adoption of evidence-based interventions.*
Frameworks are needed to guide active dissemination strategies to translate evidence into community practice
*Pentz, Jasuja, G. K., Rohrbach, L. A., Sussman, S., & Bardo, M. T. (2006). Translation in tobacco and drug abuse prevention research. Evaluation & the Health Professions, 29(2), 246-271.
This product is in the public domain. Please cite this work in this manner:
Wilson KM, Brady TJ, Lesesne C. An organizing framework for translation in public health: the Knowledge to Action Framework. Prev Chronic Dis2011 Mar;8(2):A46. The National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) Knowledge to Action Framework, Centers for Disease Control and Prevention, NCCDPHP Work Group on Translation, May 2009.
Interactive Systems Framework
Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., et al. (2008). Bridging the Gap Between Prevention Research and Practice: The Interactive Systems Framework for Dissemination and Implementation. American Journal of Community Psychology, 41(3), 171-181.
Mini-grants Programto Disseminate EBPs
A “push-pull method” (i.e. funds + TA) increases
demand while building capacity*
2 cohorts: 2007 & 2008 (12-18 month period)
12 SW GA community organizations awarded
Received up to $4000 & technical assistance (TA)
Implemented 5 RTIPs programs (nutrition or PA)
* Green LW, Orleans T, Ottoson JM, Cameron R, Pierce JP, Bettinghaus EP. Inferring strategies for disseminating physical activity policies, programs, and practices from the successes of tobacco control. Am J Prev Med. 2006;31(4)(suppl):S66–S81.
http://rtips.cancer.gov/rtips
16 Mini-grants sites funded2007-2012
Bainbridge
Albany
Blakely
Pelham
Sylvester
Nashville
Adel
Valdosta
Cordele
5
Tifton
Thomasville
2007-200912 Awarded Sites
& 5 Programs
Funded Organizations Evidence-Based Program
4 Churches Body and Soul
4 Worksites Treatwell 5-A-Day
2 Community Coalitions Parents as Teachers (PAT) High 5 Low Fat Program
Senior Center Little By Little Nutrition Program
Hospital Diabetes Management Center
Patient-Centered Assessment & Counseling for Exercise (PACE)
Engaging Community ExpertiseEmory PRC Community Advisory Board (CAB) roles:
Adaptation: How and why did sites adapt core elements of the intervention?
Project Report Forms* Monthly calls Interviews (coordinators)* Committee focus group
Context What contextual factors may have affected intervention adoption and implementation?
Interviews (coordinators)* Committee focus group Monthly calls Mini-grant applications* Census data
Maintenance What plans has the site made to continue promoting health after the end of the project?
Interviews (coordinators)* Committee focus group
Resources What resources did EPRC provide to support this project?
EPRC financial records* TA log
To what extent did grantees perceive that EPRC technical assistance helped them to implement the programs with fidelity?
Interviews (coordinators)*
* Collected in both cohorts (Other tools in 1st cohort only)
Program FidelityFidelity: “faithfulness” to the implementation of
program elements in the way they were intended to be delivered in the original intervention
Core elements*: required components that represent the theory and internal logic of the intervention and most likely produce the intervention’s effectiveness
Key process steps: required implementation or program delivery steps that are conducted to contribute to the intervention’s effectiveness
*Eke, Neumann, Wilkes, Jones. Preparing effective behavioral interventions to be used by prevention providers: the role of researchers during HIV Prevention Research Trials. AIDS Education & Prevention 2006, 18(4 Suppl A):44-58.
Program Core ElementsCore elements for each program were identified based on:underlying theory & process evaluation
findingspublished articles describing the programavailable program materialsprogram description on NCI’s Research
to an evidence-based program in order to make it more suitable for a particular population and/or an organization’s capacity.
Fidelity Findings95% of core elements conducted across all sites9 of 12 (75%) sites conducted all core elements
3 (of 7) sites in 1st cohort did not conduct all core elements
All 5 sites in 2nd cohort conducted all core elements
Decided not to conduct due to context/climate:
“They thought that if we had some type of event like that [family picnic/party], that would be saying now you’re asking me to take unemployment weeks but...you’re having an event...” - Site coordinator
Contextual Factors(related to implementation)
BARRIERS FACILITATORS Schedule/time conflicts* Difficulty with recruitment or
retention* Lack of resources/funds* Difficulty with changing
Leadership support* Staff/volunteers* Print materials/resources* In-kind resources/facilities* Partnerships* Donated Resources* Fit with mission Fit with
Infrastructure/Activities* Mentioned in both cohortsBlue text = barrier that prevented completion of core element(s) - 1st cohort
Fidelity-Adaptation Continuum
Added/customized materials
Added activities
Shifted primary audience
Held concurrent physical activity
& weight loss events
Changed delivery format/process steps
Expanded audience (to community)
Shifted focus to other behaviors
Did not complete all core elements
HIGHFIDELITY
MAJOR ADAPTATION
MINOR ADAPTATION
LOWFIDELITY
ADAPTATION EXAMPLES
NE
ED
S E
VA
LU
AT
ION
Adaptation QuotesExpanded the program from worksite/coalition to the community
“This project seems to have opened the door for a brand new [obesity] issue that our county had not talked about…all of a sudden the light went on …[the collaborative] said we need to add this to our benchmarks as a group and start working on this.” - Site coordinator
Added physical activity & weight loss events (to nutrition program)
“Because of the nutrition part of it, people began to feel better and they had more energy. So they was able to do more physical activities and wanted to do more as far as looking at weight loss…” - Site coordinator
Reasons for AdaptationsExpand program reach (broader community)
Generate/maintain engagement
Strengthen/reinforce program message
Fit program to organization’s infrastructure/activities
Reach specific audiences (esp. underserved)
Added content to reach specific audiences (teen parents)
“You got to think about being also sensitive to the age of the parent. If you have [a parent] that’s maybe 14…give them something that can be kinda fun…”
- Site coordinator
A Tale of 4 Sites…1. Body & Soul/Church: Minimal Adaptation
Minor additions (incentives & activities)
2. Body & Soul/Church: Major Adaptation Shifted focus to physical activity/weight loss
3. Little by Little/Senior Center:
Intermediate-Major Adaptation Assisted delivery of CD-ROM & added activities
3. Treatwell 5-a-Day/CBO: Major Adaptation Shifted audience to Advisory Board, then community Newsletters monthly local newspaper stories
LimitationsSmall number of sites (n=12) in rural SW GALimited measurement of fidelity & implementation qualityTime span 12-18 months – more time needed to learn
about maintenanceSelf report/social desirabilityData reflects information from only
5 intervention programsData may not be generalizable to other
settings, populations, regions & programs
2010-12 Mini-grants CohortMini-grants period will span 2 years4 sites funded at $8000 eachStructured and proactive TA and trainingRTIPs programs:
CATCH: Coordinated Approach to Child HealthFamily MattersBody & Soul
Process evaluation focus on TA & training
(Adapted from McK leroy et al., 2006)
Implementation plan Successful pilot of adapted
intervention
Prepare agency Pre-test materials
Target population Interventions Goodness of fit Stakeholders Organizational capacity
Assess
Prepare
Pilot
Implement Implement adapted EBI
Decide to adopt, adapt, or select another intervention
Make necessary changes to EBI
Select
Su
pe
rvisio
n a
nd
Qu
ality A
ss
ura
an
ce
Pro
ce
ss
Mo
nito
ring
an
d O
utc
om
e M
on
itorin
g
Fo
rma
tive, P
roc
es
s, a
nd
Ou
tco
me
Eva
lua
tionF
ee
db
ac
k L
oo
ps
, Ch
ec
kp
oin
ts
Map of the Adaptation ProcessDeveloped a structured TA model derived from the
Map of the Adaptation Process (Mckleroy et al., 2006)
Focus on objectives of each key step:
EBP Training Topics (pre-award)Session Title
What Do We Mean By Evidence-Based?
Needs Assessment and Program Planning
Finding an Evidence-Based Program
Selecting a Program That Fits Your Community
Adapting the Evidence-Based Program with Fidelity
Implementing an Evidence-Based Program
Evaluating Your Program
TEACH model: Translating Evidence into Action through Collaboratives for Health TA Contact Structured TA Topics
(examples)Stage in Map of Adaptation Process
Pre-award Training See prior training slide Assess, Select, Prepare
Kick-Off Training for awarded sites
EBIs, Needs assessment, Organizational readiness, Core elements
Assess, Select, Prepare
Site Visit Fit, Adaptation, Evaluation planning
Assess, Select, Prepare, Pilot
Conference Call Implementation Work Plan, Partnerships
Tools Adapted from:Lesesne, C. A., Lewis, K. M., Moore, C., Fisher, D., Green, D., & Wandersman, A. (2007). Promoting Science-based Approaches to Teen Pregnancy Prevention using Getting To Outcomes: Draft June 2007. Unpublished manual.
TEACH Evaluation Questions
Kept the original evaluation questions and added capacity questions related to the impact of TEACH:
Do attitudes toward EBAs become more positive as a result of the TEACH process?
Does self-efficacy for EBA behaviors increase as a result of the TEACH process?
Does organizational capacity for EBAs increase as a result of the TEACH process?
Process Evaluation PlanBaseline survey (n=20)- 80 close-ended items
Follow-up at 3 (n=12)-76 closed + 4 open-ended items
Additional follow up at 24 months TA tracking databaseProject Report FormsQualitative interviews w/ coordinators at 24
months
Participant descriptions Completed baseline surveys (n=20) Included directors, coordinators, educators9 (45%) held supervisory or managerial roles6 (30%) were “front line staff”15 (75%) had a bachelors degree or higherAveraged 9 years at current organization 6 (33.3%) reported prior experience with EBPsAlmost all (18) reported someone from their organization advocated for the use of an EBP for the currently funded mini-grant
Survey topic areas Example Measures – Survey Questions
Attitudes about EBPs 14 items
(Hannon et al, 2009)
Likert Scale: Strongly Disagree Strongly Agree
• EBPs are easy to understand.• EBPs are easy for us to adapt for use in our community.
Skills related to EBPs 18 items
(Chinman et al., 2008)
Likert Scale: Very hard Very Easy
• Assess organizational readiness to implement an evidence-based program.
• Determine what needs to be changed in an EBP to increase fit to your community.
Organizational functioning*38 items
Likert Scale: Strongly Disagree Strongly Agree
• We have appropriate staff skills to achieve our mission.• Staff use data/information to inform their decision-
making.• The leadership of the organization fosters respect, trust,
inclusiveness, and openness in the organization.
*Levinger and Bloom, 2000; Weiss et al., 2002; Preskill and Tores, 1998; Caplan, 1971; Kenny and Sofaer, 2000; Schminke et al, 2002)
Preliminary Results: Skills Related to EBPs
Tasks (1= very hard; 5= very easy) Mean SD
Tasks with higher reported ability
Define goals and objectives for your program. 3.95 .83
Discuss the benefits of using evidence-based programs. 3.90 .72
Develop an implementation work plan. 3.80 .70
Tasks with lower reported abilityPlan for maintenance of program. (e.g. leverage of resources)
2.75 .97
Develop solutions to identified implementation barriers. 3.00 1.03
Describe the steps of the program adaptation process. 3.20 .77
Prepare for the implementation of your program. (e.g. training of staff, hiring of staff, piloting, partnerships)
3.20 1.15
Attitudes about EBPsAll scores of negative statements were reversed. The higher the mean score, the more positive their attitude about EBPs.
*Reverse Coded
Implications for PracticeUsing evidence-based strategies and programs can
save time and can benefit communities
Mini-grants, training and technical assistance are promising strategies to translate evidence into community practice
Evidence-based programs can be strategically adapted to meet the needs of a community
More evaluation is needed to determine how best to adapt and implement EBPs with fidelity
Acknowledgements Mini-grant sites Sally Honeycutt Kirsten Rodgers Karen Glanz Johanna Hinman Jenifer Brents Molly Russ Yao Shi
The CPCRN is part of the Prevention Research Centers Program. It is supported by the Centers for Disease Control and Prevention and the National Cancer Institute (Cooperative agreement # 1U48DP0010909-01-1)
JK Veluswamy Margaret Clawson Megan Brock Nidia Banuelos Alma Nakasone Amanda Wyatt Deltavier Frye Ana Iturbides