Testing the NIRN Implementation Model and the CFIR Implementation Constructs: Lessons Learned from a Multi-Site Implementation of Motivational Interviewing Melanie Barwick, Ph.D., C.Psych Associate Scientist, Learning Institute Scientific Director Knowledge Translation, Research Institute The Hospital for Sick Children Associate Professor, Psychiatry & Dalla Lana School of Public Health University of Toronto, Canada Raluca Barac, PhD Melissa Kimber, PhD(c) Sabine Johnson, MA Clinical Research Project Managers The Hospital for Sick Children and The CIHR Emerging Team in Knowledge Translation for Child and Youth Mental Health 26th Annual Children’s Mental Health Research and Policy Conference Tampa FL March 4 th , 2013
40
Embed
Testing the NIRN Implementation Model and the CFIR ...cmhconference.com/files/2013/cmh2013-8b.pdfTesting the NIRN Implementation Model and the CFIR Implementation Constructs: ... Melanie
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Testing the NIRN Implementation Model and the
CFIR Implementation Constructs: Lessons Learned from a Multi-Site Implementation of
Motivational Interviewing
Melanie Barwick, Ph.D., C.Psych Associate Scientist, Learning Institute
Scientific Director Knowledge Translation, Research Institute The Hospital for Sick Children
Associate Professor, Psychiatry & Dalla Lana School of Public Health University of Toronto, Canada
Raluca Barac, PhD
Melissa Kimber, PhD(c)
Sabine Johnson, MA Clinical Research Project Managers
The Hospital for Sick Children and
The CIHR Emerging Team in Knowledge Translation for Child and Youth Mental Health
26th Annual Children’s Mental Health Research and Policy Conference Tampa FL March 4th , 2013
Research Team Melanie Barwick, Sickkids / U Toronto Charles E. Cunningham, McMaster Rosemary Tannock, Sickkids / OISE Rhonda Martinussen, OISE Peter Chaban, Sickkids Kathryn Bennett, McMaster Don Buchanan, Hamilton Wentworth District School Board Bruce Ferguson, Sickkids Dean Fergusson, Ottawa Health Research Institute Institutional Partners Children’s Mental Health Ontario Ontario Ministry of Children and Youth Services Ontario Ministry of Education Ontario Centre of Excellence for Child and Youth Mental Health Community Based Research Partners Associated Youth Services of Peel Lynwood Hall Child Development Institute Craigwood Youth Services
1) To test the National Implementation Research Network (NIRN) model and the factors identified in the Consolidated Framework for Implementation Research (CFIR) in the Canadian mental health context in order to inform the knowledge base on successful implementation of evidence-based practices.
2) To examine the experience of clinicians and supervisors involved in the process of implementing Motivational Interviewing in four child and youth mental health provider organizations
3) To inform change at the practice and system levels and refine existing theoretical and meta-theoretical approaches to implementation.
Implementation of evidence-based practices in mental health care is essential for improving health outcomes.
Together, the NIRN1 model and CFIR2 constructs provide a comprehensive approach to guide the implementation process, but both frameworks require further empirical investigation.
To this end, we implemented Motivational Interviewing in four child mental health organizations in Canada, using NIRN as a guide and measuring key CFIR constructs.
These findings have significant implications for implementation, theory, research and practice.
1 Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. and Wallace, F (2005). Implementation
research: a synthesis of the literature. NIRN Monograph 2 Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. (2009) Implement Sci.
Implementation Science – Learning how to bring evidence to practice
'Implementation Research is the scientific study of methods to promote the systematic uptake of [clinical] research findings and other evidence-based practices into routine practice, and hence to improve the quality (effectiveness, reliability, safety, appropriateness, equity, efficiency) of health care or well-being. It includes the study of influences on healthcare professional and organizational behaviour.'
Source: Adapted from An implementation research agenda, Implementation Science 2009, 4:18 Martin P Eccles1 , et al.
Source: Fixsen, Blase, Timbers & Wolf (2007). In search of program implementation – 792 replications of the teaching-family model. The Behavior Analyst Today, 8(1), 96-110. Balas EA, Boren SA. In: Yearbook of Medical Informatics 2000: Patient-Centered Systems. Stuttgart: Schattauer; 2000:65-70. Dean Fixsen [Institute of Medicine 2000,2001.,2009; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education, 1983; Dept of Health and Human Services, 1999]
Intervention Implementation Team No Implementation Team
Effectiveness
80% 3 years
14% 17 years
Application of implementation science and practice
(% of participants who DEMONSTRATE KNOWLEDGE; DEMONSTRATE NEW SKILLS in a practice setting; and USE
NEW SKILLS in the Classroom
Training Components
Knowledge Skill Demonstration
Use in the Classroom
Theory & Discussion
10% 0% 0%
+ Demonstration & Training
30% 20% 0%
+ Practice & Feedback in Training
60% 60% 5%
+ Coaching in the Classroom
95% 95% 95%
Joyce, B., & Showers, B. (2002). Student achievement through staff development. Alexandria, VA: Association for Supervision and Curriculum Development.
Barwick MA., Bennett LM, Johnson SN, McGowan J & Moore JE (2012). Training health and mental health professionals in motivational interviewing: A systematic review. Children and Youth Services Review, 34 (2012), pp. 1786-1795,
Barwick MA, Bennett LM, Johnson S, Chaban P, Barac R & Hawke L. Bringing evidence to the classroom: exploring educator preferences for practice change. Barwick MA , Johnson S, Bennett-Akrong L. Hawke L, Barac R, Kimber M. Bringing evidence to children’s mental health care: exploring practitioner preferences for practice change.
Cunningham CE, Barwick MA, Short K, Chen Y, Ratcliffe J, Rimas H & Mielko S. Modeling the Mental Health Practice Change Preferences of Educators: A Discrete–Choice Conjoint Experiment. Submitted to Journal of School Psychology, January 2nd 2013.
• Motivational interviewing (MI) is a counseling approach that focuses on helping clients explore and resolve ambivalence and centers on motivational processes within the individual that facilitate change.
• Most recently, MI has been defined as a collaborative, person-centered form of guiding to elicit and strengthen motivation for change (Miller & Rollnick, 2009).
• A large body of work studying the effectiveness of MI has led to several systematic reviews published in support of MI effectiveness for a range of client outcomes (Rubak et al., 2005; Heckman, Egleston, & Hofmann, 2010).
We developed an e-learning curriculum for the Ontario Centre of Excellence for Child and Youth Mental Health to support capacity building among child and youth mental health providers in the implementation of evidence based practices.
Available for public access as of March 30, 2013
Contact: Mark MacAulay Manager, Implementation Support Program l Ontario Centre of Excellence for Child and Youth Mental Health T: (613) 737- 2297 x2895 l F: (613) 738-4894 l E: [email protected] www.excellenceforchildandyouth.ca
b) monthly checks of fidelity to Motivational Interviewing based on coding of audiotaped therapy sessions =
Measure of Implementation outcome (fidelity)
Clinicians taped monthly sessions three months before the training (n = 3 tapes), throughout the period when they received training and coaching (n = 9 tapes) and three months post-coaching (n = 3 tapes).
All therapy sessions were scored for clinicians’ fidelity to Motivational Interviewing using the Behaviour Change Counselling Index (BECCI; Lane et al., 2005). 20% also scored on MITI (Moyers et al.).
a) focus groups with clinicians, supervisors, and members of the provider organizations’ implementation teams, conducted following the completion of coaching sessions in Motivation Interviewing
b) process notes from the monthly coaching calls with supervisors and clinicians as well as the monthly calls with implementation teams from the four organizations.
Qualitative analyses were coded both inductively (interpretive description based on transcripts) and deductively (based on the CFIR and NIRN frameworks).
Quantitative Results 1) Repeated measures analyses showed a shift in clinicians’
practice and attitudes towards EBP
2) Results showed increased clinician Motivational Interviewing adherence and competence over time = evidence of successful implementation outcome (fidelity). Of the 12 clinicians with pre- and post-implementation data 10 clinicians (84%) showed increased fidelity.
These findings suggest that the present implementation approach (training + coaching + fidelity monitoring + feedback) is successful in producing practice change.
In addition, qualitative analyses revealed important aspects of the implementation process such as:
the role and format of the implementation team
the nature of the research facilitation
the necessity of the pre-implementation phase
the characteristics of the selected EBP (i.e., whether it is a manualized approach or a stand-alone therapeutic element)
the existence & feasibility of practice fidelity measures for use in the field
impact on the clinicians’ practice and organizational culture
These findings bring important refinements to the NIRN and CFIR frameworks and contribute to the knowledge base informing successful implementation of EBPs in practice.
1) Prediction (CFIR constructs IO) requires a decent sample size. These can be difficult to achieve in IS case studies because of the nature of case study, attrition due to real world events, cost
2) Fidelity measures can pose problems if they don’t lend themselves to the real world (not practical for independent use in practice)
3) Measuring fidelity to EBP practice is difficult due to (a) ‘barriers’ in audio and visual recording, and (b) feasibility – not all therapy happens in a quiet office
4) Pre-implementation likely takes 4-6 months (longer than we anticipated); but this has implications for research funding time frames
IO = implementation outcome
Melanie Barwick, PhD,CPsych Associate Scientist
Scientific Director Knowledge Translation The Hospital for Sick Children
Associate Professor, Department of Psychiatry University of Toronto