Top Banner

Click here to load reader

of 45

Circe Stumbo, West Wind Education Policy Inc. and CCSSO CCSSO/SCEE National Summit on Educator Effectiveness April 10, 2013 Implementation Science: Closing

Jan 19, 2016

ReportDownload

Documents

Slide 1

Circe Stumbo, West Wind Education Policy Inc. and CCSSO

CCSSO/SCEE National Summit on Educator EffectivenessApril 10, 2013Implementation Science:

Closing Gaps Between Policy and PracticeAcknowledgementsSpecial thanks to Dean Fixsen, Karen Blase, and the National Implementation Research Network for their generosity in sharing their findings and their passion for all things implementation. This PowerPoint slide deck is built off their original work, with permission. 2

Sixth Year of Working with SEAsState Implementation & Scaling-up of Evidence-based Practices Centerscalingup.org3

Implementation Science`A policy is one thingImplementation of a policy is a very different thingStudents cannot benefit from a policy they do not experience

An Implementation Failure labeled as an Intervention FailureExample: $500 million for Homebuilders (Family Support Services) 1993-1998Funding only for interventionsNo fidelity criteria insisted upon by the developersNational evaluation = not effectiveChapin Hall Center for Children, 2002An Implementation Failure labeled as an Intervention FailureExample: $500 million for Homebuilders (Family Support Services) 1993-1998Funding only for interventionsNo fidelity criteria insisted upon by the developersNational evaluation = not effectiveBut, over 25% was spent on in-office interventions with parents or children(< 0 fidelity)Chapin Hall Center for Children, 2002Longitudinal Studies of CSR ProgramsSource: Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006Evidence-Base For EffectivenessEvery Teacher TrainedEvery Teacher Continually Supported

Actual SupportsYears 1-3Fewer than 50% of the teachers received some trainingFewer than 25% of those teachers received supportFewer than 10% of the schools used the CSR as intendedVast majority of students did not benefitOutcomesYears 4-57Formula for Success8Socially Significant OutcomesEffective Policies/ InterventionsEffective Implementation Methods Enabling Contexts Formula for Success9WHY: Socially Significant OutcomesEffective Policies/ InterventionsEffective Implementation Methods Enabling Contexts (Vision!)Formula for Success10WHY: Socially Significant OutcomesWHAT: Effective Policies/ InterventionsEffective Implementation Methods Enabling Contexts Formula for Success11WHY: Socially Significant OutcomesWHAT: Effective Policies/ InterventionsEffective Implementation Methods WHERE: Enabling Contexts Stakeholders, competencies11Formula for Success12WHY: Socially Significant OutcomesWHAT: Effective Policies/ InterventionsHOW & WHO: Effective Implementation Methods WHERE: Enabling Contexts A lot of what you are doing is pushing up against what the state should/must dohow to change orientation. Remember, if put zero in the equation, wont get to the outcomes. 12Table DiscussionsShare a situation you have experienced/ observed where one of the variables in the formula was zero

Implementation ScienceExcellent evidence for what does not workImplementation without changing supporting roles and functionsImplementation by edictDiffusion/dissemination of information by itselfImplementation by following the moneyTraining alone, no matter how well done

Paul Nutt (2002). Why Decisions Fail(c) Dean Fixsen and Karen Blase, 200815Nutt, P. C. (1986). Tactics of Implementation. Academy of Management Journal, 29(2), 230-261.5%-15% realize intended outcomesWhat does work? An Implementation FrameworkImplementation Teams Implementation DriversImprovement CyclesImplementation Stageshttp://sisep.fpg.unc.edu/learning-zone/science-of-implementation/implementation-frameworks An Implementation FrameworkImplementation Teams Implementation DriversImprovement CyclesImplementation Stageshttp://sisep.fpg.unc.edu/learning-zone/science-of-implementation/implementation-frameworks IMPLEMENTATIONImplementation ScienceEffectiveNOT EffectiveEffective NOTEffectiveINTERVENTIONStudent Benefits(Institute of Medicine, 2000; 2001; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999)Poor outcomesInconsistent, not sustainable, poor outcomesPoor outcomes; sometimes harmfulIMPLEMENTATIONImplementation TeamsEffective INTERVENTION80%, 3 Yrs14%, 17 YrsImpl. TeamNO Impl. TeamNOTEffectiveBalas & Boren, 2000 Green, 2008Fixsen, Blase, Timbers, & Wolf, 2001Implementation TeamA group that knows:the intervention/policyimplementationimprovement cycles20Simultaneous, Multi-Level InterventionsSchoolManagement (leadership, policy)Administration (HR, structure)Supervision (nature, content)TeacherRegionDistrict Implementation TeamStatePartnersIowas Collaborating for Kids (C4K)22

Students & FamiliesBuilding Teachers and StaffBuilding Leadership and Implementation TeamDistrict Leadership and Implementation TeamsRegional Implementation TeamsState Transformation TeamStateDepartment of Education LeadershipCascading Logic Model23An Implementation FrameworkImplementation Teams Implementation DriversImprovement CyclesImplementation Stageshttp://sisep.fpg.unc.edu/learning-zone/science-of-implementation/implementation-frameworks Improvement CyclesPractice-Policy Communication LoopsPlan-Do-Study-Act cyclesUsability Testing25Students & FamiliesBuilding Teachers and StaffBuilding Leadership and Implementation TeamDistrict Leadership and Implementation TeamsRegional Implementation TeamsState Transformation TeamStateDepartment of Education LeadershipPolicy Enabled Practice (PEP)Practice Informed Policy (PIP)Practice/Policy Communication Loops26Students & FamiliesBuilding Teachers and StaffBuilding Leadership and Implementation TeamDistrict Leadership and Implementation TeamsRegional Implementation TeamsState Transformation TeamStateDepartment of Education LeadershipPolicy Enabled Practice (PEP)Practice Informed Policy (PIP)Practice/Policy Communication Loops27Students & FamiliesBuilding Teachers and StaffBuilding Leadership and Implementation TeamDistrict Leadership and Implementation TeamsRegional Implementation TeamsState Transformation TeamStateDepartment of Education LeadershipPolicy Enabled Practice (PEP)Practice Informed Policy (PIP)Practice/Policy Communication Loops28Students & FamiliesBuilding Teachers and StaffBuilding Leadership and Implementation TeamDistrict Leadership and Implementation TeamsRegional Implementation TeamsState Transformation TeamStateDepartment of Education LeadershipFeedback LoopsFeedback Loops29North Carolina Example30

Students & FamiliesBuilding Teachers and StaffBuilding Leadership and Implementation TeamDistrict Leadership and Implementation TeamsRegional Implementation TeamsState Transformation TeamStateDepartment of Education LeadershipAuthentic Work Informs thePEP-PIP CyclePlan-Do-Study-ActPlan-Do-Study-Act31Students & FamiliesBuilding Teachers and StaffBuilding Leadership and Implementation TeamDistrict Leadership and Implementation TeamsRegional Implementation TeamsState Transformation TeamStateDepartment of Education LeadershipAuthentic Work Informs thePEP-PIP CycleUsability StudiesUsability Studies32Table DiscussionsHow do teams at each level of your system interact? What work are the teams doing with each other? What feedback loops exist between teams? How might you increase interaction among teams? Have/might you incorporate improvement cycles into pilots and policy innovations?33

Growing Implementation Capacity UsingTransformation ZonesImplementation TeamMinimum of 3 people (4-5 preferred) Tolerate turnover; sustainable35State Transformation TeamFirst School Implementation TeamN = 10 SchoolsGrowing Implementation Capacity (Phase 1)External Support & 2 FTEsFirst Regional Implementation TeamN = 9 StaffInvest up front to build capacity (c) Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 20083636State Transformation TeamFirst School Implementation TeamN = 10 SchoolsGrowing Implementation Capacity (Phase 2)External Support & 2 FTEsFirst Regional Implementation TeamN = 3 StaffSubsequent Regional Implementation Team (3 Staff)Subsequent Regional Implementation Team (3 Staff)(c) Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 20083737State Transformation TeamFirst School Implementation Team, ExpandedN = 20 SchoolsGrowing Implementation Capacity (Phase 2)External Support & 2 FTEsFirst Regional Implementation TeamN = 3 StaffSubsequent Regional Implementation Team (3 Staff)Subsequent School Implementation TeamsN=20 SchoolsSubsequent School Implementation TeamsN=20 SchoolsSubsequent School Implementation TeamsN=20 SchoolsSubsequent Regional Implementation Team (3 Staff)(c) Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 20083838State Transformation TeamFirst School Implementation Team, ExpandedN = 20 SchoolsGrowing Implementation Capacity (Phase 2)External Support & 2 FTEsFirst Regional Implementation TeamN = 3 StaffSubsequent Regional Implementation Team (3 Staff)Subsequent School Implementation TeamsN=20 SchoolsSubsequent School Implementation TeamsN=20 SchoolsSubsequent School Implementation TeamsN=20 SchoolsSubsequent Regional Implementation Team (3 Staff)(c) Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 20083939State Transformation TeamFirst School Implementation Team, ExpandedN = 20 SchoolsGrowing Implementation Capacity (Phase 2)External Support & 2 FTEsFirst Regional Implementation TeamN = 3 StaffSubsequent Regional Implementation Team (3 Staff)Subsequent School Implementation TeamsN=20 SchoolsSubsequent School Implementation TeamsN=20 SchoolsSubsequent School Implementation TeamsN=20 SchoolsSubsequent Regional Implementation Team (3 Staff)(c) Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 20084040State Transformation TeamFirst School Implementation Team, ExpandedN = 20 SchoolsGrowing Implementation Capacity (Phase 2)External Support & 2 FTEsFi