Emory P r even tio n Research C enter Emory P r even tio n Research C enter Michelle Kegler, DrPH, MPH Michelle Carvalho, MPH, CHES
Dec 26, 2015
Emor yP r event ionResear chCent er
Emor yP r event ionResear chCent er
Michelle Kegler, DrPH, MPH
Michelle Carvalho, MPH, CHES
Mini-Grants as a Strategy for Dissemination
Mini-grants are common in health promotion initiatives & have potential for creating demand for evidence-based interventions
Mini-grants can be combined with dissemination strategies shown to workTraining workshops (Rohrbach 2006; Elliot 2004)
Increases adoption, capacity, fidelity, maintenanceTechnical Assistance (Pentz 2006; Shepherd 2008; Rohrbach 2006)
Ongoing support, feedback, coachingIncentives (Basen-Engquist ,1994; Glanz, 2002)
stipends, equipment, materials
EFFECTIVENESS AND
IMPLEMENTATIONSTUDIES
DECISION to ADOPT
DIFFUSION
NCCDPHP Knowledge To Action FrameworkMay 2009
INSTITUTION-ALIZATION
RESEARCH PHASE TRANSLATION PHASE
EFFICACY STUDIES
PRACTICE
DISCOVERYSTUDIES
DISSEMINATION
ENGAGEMENT
Practice-based Discovery
KNOWLEDGE INTO PRODUCTS
INSTITUTIONALIZATION PHASE
Practice-based Evidence
Translation Supporting StructuresResearch Supporting Structures
DECISION to TRANSLATE
EVALUATION
Insitutional-ization
Supporting Structures
This product is in the public domain. Please cite this work in this manner:
The National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) Knowledge to Action Framework, Centers for Disease Control and Prevention, NCCDPHP Work Group on Translation, May 2009. Adapted from Wilson K & Fridinger F. Focusing on Public Health: A Different Look at Translating Research to Practice. Journal of Women’s Health; 2008;17(2):173-179.
Interactive Systems Framework
Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., et al. (2008). Bridging the Gap Between Prevention Research and Practice: The Interactive Systems Framework for Dissemination and Implementation. American Journal of Community Psychology, 41(3), 171-181.
Mini-grants Programto Disseminate EBPs
A “push-pull method” (i.e. funds + TA) increases
demand while building capacity*
2 cohorts: 2007 & 2008 (12-18 month period)
12 SW GA community organizations awarded
Received up to $4000 & technical assistance (TA)
Implemented 5 RTIPs programs (nutrition or PA)
*Orleans, C., Gruman, J., & Anderson, N. (2002). Designing for Dissemination: The Larger Challenge of Translation: An Extraordinary Opportunity for Cancer Control. Designing for Dissemination Collaborative Meeting, Washington, D.C.
12 Awarded Sites & 5 Programs
Funded Organizations Evidence-Based Program
4 Churches Body and Soul
4 Worksites Treatwell 5-A-Day
2 Community Coalitions Parents as Teachers (PAT) High 5 Low Fat Program
Senior Center Little By Little Nutrition Program
Hospital Diabetes Management Center
Patient-Centered Assessment & Counseling for Exercise (PACE)
Engaging Community ExpertiseEmory PRC Community Advisory Board (CAB) roles:
Prioritized behavioral risk factors: nutrition, physical activity, tobacco prevention/cessation
Helped to develop mini-grants and TA process
Facilitated promotion of program to community
Joint EPRC/CAB review committee
selected grantees
Currently co-authoring presentations
and publications
Program Core ElementsCore elements for each program were identified based on:underlying theory & process evaluation
findingspublished articles describing the programavailable program materialsprogram description on NCI’s Research
Tested Intervention Programs (RTIPs) website
Construct Evaluation Question(s) Data Collection MethodsReach What proportion of the intended audience
participated in each activity? Project Report Forms* Demographics form
Implementation Fidelity: To what extent were core elements of the program implemented as described in program materials?
Project Report Forms* Monthly calls Interviews (coordinators)*
Adaptation: How and why did sites adapt core elements of the intervention?
Project Report Forms* Monthly calls Interviews (coordinators)* Committee focus group
Context What contextual factors may have affected intervention adoption and implementation?
Interviews (coordinators)* Committee focus group Monthly calls Mini-grant applications* Census data
Maintenance What plans has the site made to continue promoting health after the end of the project?
Interviews (coordinators)* Committee focus group
Resources What resources did EPRC provide to support this project?
EPRC financial records* TA log
To what extent did grantees perceive that EPRC technical assistance helped them to implement the programs with fidelity?
Interviews (coordinators)*
* Collected in both cohorts (Other tools in 1st cohort only)
Project Report FormDocumented progress on core elements
Treatwell - Core Element Documentation Examples
Employee Advisory Board Meeting topic, date, # participants
EatWell Discussion Series Session topic, date, # participants
Exposure to “5-A-Day” campaign
Activity, date, # participants
Promotion of national hotlines
Please describe how you have promoted the national hotlines (or More Matters website)
Environmental Change Please describe environmental and/or policy changes made at your site
Family/home component Newsletter topic, Date mailed, # mailed
Annual family event Description, Date, # participants
Interview/Focus Group GuidesSection Topic Example Questions
Staff/Volunteers •What motivated you personally to be involved in this program?•How would you describe the level of support of your site’s leadership ?
Implementation •What factors within your organization helped to implement the program? •What barriers did you face in implementing the program?
Participants •In general, how did people respond to the program?
Adaptation •Did you change anything in the original program or its program materials to better reach participants?
Technical Assistance
• If you feel you needed technical assistance, what kind of help was needed?
Maintenance •Do you think your organization will continue similar program activities after the program is over?
Fidelity Findings95% of core elements conducted across all sites
9 of 12 (75%) sites conducted all core elements
3 (of 7) sites in 1st cohort did not conduct all core elements
All 5 sites in 2nd cohort conducted all core elements
Core Element of Body & Soul Reach within ChurchesA B C D
Project Committee Low Low Low LowKick-off event High High High HighChurch-wide nutrition event with Pastor High Med High MedOther church-wide event #1 Low Low Low MedOther church-wide event #2 Med Low Low MedOther church-wide event #3 Low Med N/A N/AAt least one motivational interviewing call Low Low Low LowCore Elements of Treatwell Reach within Worksites
E F GEmployee Advisory Board Low Low LowEatWell Discussion Series Low High LowAt least one other activity to change individual behavior
Low Low --
Family/home component of learning Low N/A LowAnnual family/holiday event Low N/A High
Participation and Reach of Core Elements Implemented by 2007 Mini-Grant Recipients
Reach = proportion of the site population that participated in a given event:Categories: Low (<1/3), Medium (1/3 to 2/3), High (>2/3).
Contextual Factors(related to implementation)
BARRIERS FACILITATORS Schedule/time conflicts* Difficulty with recruitment or
retention* Lack of resources/funds* Difficulty with changing
behavior Staff/leadership transitions Slow economy/worksite
financial difficulties
Leadership support* Staff/volunteers* Print materials/resources* In-kind resources/facilities* Partnerships* Donated Resources* Fit with mission Fit with
Infrastructure/Activities* Mentioned in both cohortsBlue text = barrier that prevented completion of core element(s) - 1st cohort
Fidelity-Adaptation Continuum
Added/customized materials
Added activities
Shifted primary audience
Held concurrent physical activity
& weight loss events
Changed delivery format/process steps
Expanded audience (to community)
Shifted focus to other behaviors
Did not complete all core elements
HIGHFIDELITY
MAJOR ADAPTATION
MINOR ADAPTATION
LOWFIDELITY
ADAPTATION EXAMPLES
NE
ED
S E
VA
LU
AT
ION
Reasons for AdaptationsExpand program reach (broader community)
Generate/maintain engagement
Strengthen/reinforce program message
Fit program to organization’s infrastructure/activities
Reach specific audiences (esp. underserved)
Added content to reach specific audiences (teen parents)
“You got to think about being also sensitive to the age of the parent. If you have [a parent] that’s maybe 14…give them something that can be kinda fun…”
- Site coordinator
LimitationsSmall number of sites (n=12) in rural SW GALimited measurement of fidelity & implementation qualityTime span 12-18 months – more time needed to learn
about maintenanceSelf report/social desirabilityData reflects information from only 5 intervention
programsData may not be generalizable to other settings,
populations, regions and programs
2010-12 Mini-grants CohortMini-grants period will span 2 years4 sites funded at $8000 eachStructured and proactive TA and trainingRTIPs programs:
CATCH: Coordinated Approach to Child HealthFamily MattersBody & Soul
Process evaluation focused on TA and training
(Adapted from McK leroy et al., 2006)
Implementation plan Successful pilot of adapted
intervention
Prepare agency Pre-test materials
Target population Interventions Goodness of fit Stakeholders Organizational capacity
Assess
Prepare
Pilot
Implement Implement adapted EBI
Decide to adopt, adapt, or select another intervention
Make necessary changes to EBI
Select
Su
pe
rvisio
n a
nd
Qu
ality A
ss
ura
an
ce
Pro
ce
ss
Mo
nito
ring
an
d O
utc
om
e M
on
itorin
g
Fo
rma
tive, P
roc
es
s, a
nd
Ou
tco
me
Eva
lua
tionF
ee
db
ac
k L
oo
ps
, Ch
ec
kp
oin
ts
Map of the Adaptation ProcessDeveloped a structured TA model derived from the
Map of the Adaptation Process (Mckleroy et al., 2006)
Focus on objectives of each key step:
EBI Training Topics (pre-award)Session Title
What Do We Mean By Evidence-Based?
Needs Assessment and Program Planning
Finding an Evidence-Based Program
Selecting a Program That Fits Your Community
Adapting the Evidence-Based Program with Fidelity
Implementing an Evidence-Based Program
Evaluating Your Program
TEACH model: Translating Evidence into Action through Collaboratives for Health TA Contact Structured TA Topics
(examples)Stage in Map of Adaptation Process
Pre-award Training See training slide Assess, Select, Prepare
Kick-Off Training for awarded sites
EBIs, Needs assessment, Organizational readiness, Core elements
Assess, Select, Prepare
Site Visit Fit, Adaptation, Evaluation planning
Assess, Select, Prepare, Pilot
Conference Call Implementation Work Plan, Partnerships
Assess, Select, Prepare, Pilot
Ongoing Contact Overcoming barriers, implementation fidelity, maintenance
Assess, Pilot, Implement, Maintenance
TEACH Evaluation Questions
Kept the original evaluation questions and added capacity questions related to the impact of TEACH:
Do attitudes toward EBAs become more positive as a result of the TEACH process?
Does self-efficacy for EBA behaviors increase as a result of the TEACH process?
Does organizational capacity for EBAs increase as a result of the TEACH process?
Process Evaluation PlanBaseline survey (n=17) – 80 itemsFollow-up at 3, 6, and 24 months TA tracking Access databaseProject Report FormsQualitative interviews w/ coordinators
at 24 months
Survey topic areas Example Measures – Survey Questions
Attitudes about EBPs (Hannon et al, 2009)
Likert Scale: Strongly Disagree Strongly Agree• EBPs lack real world experience.• EBPs are easy to understand.• EBPs are easy for us to adapt for use in our community.
Skills related to EBPs (Chinman et al., 2008)
Likert Scale: Very hard Very Easy• Discuss the benefits of using evidence-based programs.• Assess the fit of a potential program or strategy to your
organization or community.• Determine what needs to be changed in an EBP to
increase fit to your community.
Organizational functioning*
Likert Scale: Strongly Disagree Strongly Agree• We have appropriate staff skills to achieve our mission.• The leadership of the organization fosters respect, trust,
inclusiveness, and openness in the organization.• Staff are encouraged to take the lead in initiating change
or in trying to do something different.
*Levinger and Bloom, 2000; Weiss et al., 2002; Preskill and Tores, 1998; Caplan, 1971; Kenny and Sofaer, 2000; Schminke et al, 2002)
Next Steps2 manuscripts in progress:
Process evaluation of a mini-grants program to disseminate evidence-based nutrition programs to rural churches and work sites
Balancing fidelity and adaptation: Case Studies in implementing evidence-based chronic disease prevention programs
Conduct process evaluation of current mini-grants program (TA, training, fidelity, adaptations)
Dissemination research grant proposals
Acknowledgements Sally Honeycutt Cam Escoffery Kirsten Rodgers Karen Glanz Johanna Hinman Jenifer Brents
The CPCRN is part of the Prevention Research Centers Program. It is supported by the Centers for Disease Control and Prevention and the National Cancer Institute (Cooperative agreement # 1U48DP0010909-01-1)
JK Veluswamy Margaret Clawson Megan Brock Nidia Banuelos Alma Nakasone Amanda Wyatt Ana Iturbides
Emor yP r event ionResear chCent er
Emor yP r event ionResear chCent er
QUESTIONS?