Welcome to SIG Day Jennifer Coffey, Ph.D. SPDG Program Lead
Feb 14, 2016
Welcome to SIG Day
Jennifer Coffey, Ph.D. SPDG Program Lead
SIG DAY AGENDASPDG Program Area Meeting
2011 OSEP Project Directors’ Conference
SIG Day Planning Committee
Kathe Shelby, OH Karen Jones, DE Teresa Farmer, AL Letha Bauter, OK Veronica MacDonald, TN Renee Scott, KY Jennifer Coffey Jeanna Mullins, MSRRC Pat Gonzalez Audrey Desjarlais
Please Welcome The Regional Parent Technical
Assistance Directors
And don’t forget… Early Bird on Professional
Development Systems - Tuesday, July 19 (7:30am-8:30am) Location: Maryland A
SIG Night - Tuesday, July 19 - (6:00-8:00pm)Location: Ireland’s Four Fields, 3412 Connecticut Ave. NW, Metro Station: Cleveland Park
The Year in Review & Looking Forward
Opportunities to Learn and Share Professional Development Series Revising the Program Measures and
Creating Methodology OSEP Policies SPDG Competition Building Partnerships
Opportunities to Learn and Share
Monthly Webinars – “Directors’ Calls” Evaluator Community of Practice Resource Library “Regional Meetings” Project Director’s Conference PLC’s
SPDG REGIONAL MEETING
RESOURCES
To view the SPDG Regional Meeting Materials go to: http://signetwork.org/content_pages/27
PLCs Adolescent Literacy Behavior & School
Climate Coaching Collaboration with
IHEs Family Engagement Grant Management Implementation
Conversations
Low Incidence Affinity Group
RTI & Multi-Tiered models of intervention
Scaling Up & Comprehensive PD Approaches
Secondary Transition
PROFESSIONAL LEARNING COM
MUNITIES - W
IKIS
To view the Professional Learning Community (PLC) Wikis go to : http://signetwork.org/wikis
Evidence-based Professional Development
Models of and Evaluating Professional Development› Date: January 12, 3:00-4 :30pm ET › Speakers: Julie Morrison, Alan Wood,
& Li Walter (SPDG evaluators) SPDG REGIONAL MEETINGS
› Topic: Evidence-based Professional Development
Evidence-based PD Innovation Fluency
› Date: March 24, 3:00-4:30pm ET› Speaker: Karen Blase, SISEP
Professional Development for Administrators› Date: April 19, 3:00-4:30pm ET› Speakers: Elaine Mulligan, NIUSI Leadscape› Rich Barbacane, National Association of Elementary
School Principals Using Technology for Professional
Development› Date:May 18, 2:00-3:30pm ET› Speaker: Chris Dede, Ph.D., Learning Technologies at
Harvard’s Graduate School of Education
2011 PROFESSIONAL DEVELOPM
ENT SERIES Go to the Home Page to link each webinar segment: http://signetwork.org
• Evidence-Based Intervention Practices Insert your SPDG initiative here
• Evidence-Based Implementation Practices Professional Development
Staff Competence: Selection, Training, Coaching, and Performance Assessment Drivers Adult learning methods/principles Evaluation
14
Two Types of Evidence-Based Practices
15
HOW?
CA: ERIA’s Evidence-based Practices
The Program Guide articulates a comprehensive set of practices for all stakeholders.
16
Implementation Practices Intervention Practices Initial Training Team-based Site-level
Practice and Implementation
Implementation Rubric facilitates self-eval
Ongoing Coaching Booster Trainings Implementation Rubric
reflection on next steps
The 5 Steps of ERIA Data-informed
Decision-making Screening and
Assessment Progress Monitoring
Tiered Interventions and Learning Supports
Enhanced Literacy Instruction
CA: Two Integrative Evaluation Tools Serve as Implementation Drivers
Program Guide articulates PD model› introduces and illustrates › contextualizes the training› gets away from “you had to be there”
Implementation Rubric operationalizes PD model› drives ongoing implementation› enables fidelity checks› is possible to evaluate
Everyone is on the same page Sustainability (beyond funding, staff turnover) Scale-up (recruit new sites/districts, beyond SPDG) Diversity of approaches enabled
17
18
HOW?
19
Training must be … › Timely › Theory grounded (adult learning)› Skill-based
Information from Training feeds back to Selection and feeds forward to Coaching
Best Practices in Training
Selection Training Coaching
(Blase, VanDyke, & Fixsen, 2010)
20
Design a Coaching Service Delivery Plan Develop accountability structures for
Coaching – Coach the Coach! Identify on-going professional development
for coaches
Best Practices in Coaching
Coaching Performance Assessment
Training
(Blase, VanDyke, & Fixsen, 2010)
21
Must be a transparent process Use of multiple data sources Fidelity of implementation should be assessed at
the local, regional, and state levels Tied to positive recognition Information from this driver feeds back to
Selection, Training, and Coaching and feeds forward to the Organization Drivers
Best Practices in Performance Assessment (Fidelity)
22
Assess fidelity of implementation at all levels and respond accordingly
Identify outcome measures that are …› Intermediate and longer-term› Socially valid› Technically adequate: reliable and valid› Relevant data that is feasible to gather, useful
for decision making, widely shared and reported frequently
Best Practices in Decision Support Data Systems
Why focus on professional development?
“No intervention practice, no matter what its evidence base, is likely to be learned and adopted if the methods and strategies used to teach or train students, practitioners, parents, or others are not themselves effective.”
"Let's Be Pals: An Evidence-based Approach to Professional Development." Dunst & Trivette, 2009
Using Research Findings to Inform Practical
Approaches to Evidence-Based Practices
Carl J. Dunst, Ph.D. Carol M. Trivette, Ph.D.Orelena Hawks Puckett Institute
Asheville and Morganton, North Carolina
Presentation Prepared for a Webinar with the Knowledge Transfer Group, U.S. Department of Health and Human Services, Children’s Bureau Division of Research and
Innovation, September 22, 2009
“Adult learning refers to a collection of theories, methods, and approaches for describing the characteristics of and conditions under which the process of learning is optimized.”
Six Characteristics Identified in How People Learna Were Used to Code and Evaluate the Adult Learning Methods
PlanningIntroduce Engage the learner in a preview of the material, knowledge or practice that
is the focus of instruction or training
Illustrate Demonstrate or illustrate the use or applicability of the material, knowledge or practice for the learner
Application
Practice Engage the learner in the use of the material, knowledge or practice
Evaluate Engage the learner in a process of evaluating the consequence or outcome of the application of the material, knowledge or practice
Deep Understanding
Reflection Engage the learner in self-assessment of his or her acquisition of knowledge and skills as a basis for identifying “next steps” in the learning process
Mastery Engage the learner in a process of assessing his or her experience in the context of some conceptual or practical model or framework, or some external set of standards or criteria
a Donovan, M. et al. (Eds.) (1999). How people learn. Washington, DC: National Academy Press.
Additional Translational Synthesis Findings
The smaller the number of persons participating in a training (<20), the larger the effect sizes for the study outcomes.
The more hours of training over an extended number of sessions, the better the study outcomes.
The practices are similarly effective when used in different settings with different types of learners.
Practices
Number
Mean Effect Size (d)
95% Confide
nce Interval
Studies Effect Sizes
Pre-class exercises 9 9 1.02 .63-1.41Out of class activities/self-instruction
12 20 .76 .44-1.09
Classroom/workshop lectures 26 108 .68 .47-.89
Dramatic readings 18 40 .35 .13-.57Imagery 7 18 .34 .08-.59Dramatic readings/imagery 4 11 .15 -.33-.62
Effect Sizes for Introducing Information to Learners
Practices
NumberMean Effect Size (d)
95% Confide
nce Interval
Studies Effect Sizes
Using learner input for illustration 6 6 .89 .28-1.51
Role playing/simulations 20 64 .87 .58-1.17
Real life example/real life + role playing 6 10 .67 .27-1.07
Instructional video 5 49 .33 .09-.59
Effect Sizes for Illustrating/Demonstrating Learning Topic
Practices
Number
Mean Effect Size (d)
95% Confide
nce Interval
Studies
Effect Sizes
Real life application + role playing 5 20 1.10 .48-1.72
Problem solving tasks 16 29 .67 .39-.95
Real life application 17 83 .58 .35-.81Learning games/writing exercises
9 11 .55 .11-.99
Role playing (skits, plays) 11 35 .41 .21-.62
Effect Sizes for Learner Application
Practices
Number
Mean Effect Size (d)
95% Confidence
Interval
Studies
Effect Sizes
Assess strengths/weaknesses
14 48 .96 .67-1.26
Review experience/make changes
19 35 .60 .36-.83
Effect Sizes for Learner Evaluation
Practices
Number
Mean Effect Size (d)
95% Confidence
Interval
Studies
Effect Sizes
Performance improvement 9 34 1.07 .69-
1.45Journaling/behavior suggestion
8 17 .75 .49-1.00
Group discussion about feedback 16 29 .67 .39-.95
Effect Sizes for Learner Reflection
Practices
Number
Mean Effect Size (d)
95% Confidence
Interval
Studies
Effect Sizes
Standards-based assessment 13 44 .76 .42-
1.10Self-assessment 16 29 .67 .39-.95
Effect Sizes for Self-Assessment of Learner Mastery
Summary of Training Findings
To be most effective need to actively involve the learners in judging the consequences of their learning experiences (evaluate, reflection, & mastery)› Need learner participation in learning new
knowledge or practice› Need learner engagement in judging his or
her experience in learning and using new material
Innovation Fluency Definition: Innovation Fluency refers to
the degree to which we know the innovation with respect to:› Evidence› Program and Practice Features› Implementation Requirements
35
Implementation Pre-Requisites After you
› Have chosen based on student needs› Looked for “best evidence” to address
the needAn Evidence-Based Practice or ProgramAn Evidence-Informed Initiative or
Framework Systems Change and Its Elements
36
Implementation Pre-Requisites After you
› Have chosen based on student needs› Looked for “best evidence” to address
the needAn Evidence-Based Practice or ProgramAn Evidence-Informed Initiative or
Framework Systems Change and Its Elements
Then it’s time to:› Clearly identify and operationalize
the elements
37
Professional Problem Solving 9 Critical Components
Parent Involvement Problem Statement Systematic Data
Collection Problem Analysis Goal Development
Intervention Plan Development
Intervention Plan Implementation
Progress Monitoring Decision Making
Critical Component Ideal Implementation
Acceptable Variation
Unacceptable Variation
Unacceptable Variation
Critical Component 1: Description
Description of implementer
behavior
Professional Practices in Problem Solving: Benchmarks and Innovation ConfigurationsIowa Area Education Agency Directors of Special Education, 1994
38
Interaction of Leadership and Implementation Support Drivers Regarding Administrators
Project Level Providing Leadership
District Level Providing Leadership
Building Level Providing Leadership
Develop systems for district and building administrators to implement practices with success
Develop systems for building administrators to implement practices with success
Develop systems for building staff to implement practices with success
Purpose: To Develop project Capacity (e.g., data systems, information resources, incentives) and Competency (e.g., selection, training, coaching) so administrators can implement practices with success
Provides guidance, visibility, funding, political support for MiBLSi
Students
Building Staff
Building Leadership Team
LEA District Leadership Team
Across State
Multiple District/Building Teams
All staff
All students
Multiple schools w/in local district
Who is supported?
How is support provided?
Provides guidance, visibility, funding, political support
Provides coaching for District Teams and technical assistance for Building Teams
Provides guidance and manages implementation
Provides effective practices to support students
Improved behavior and reading
ISD Leadership Team
Regional Technical
Assistance
MichiganDepartment of
Education/MiBLSi Leadership
Multiple schools w/in intermediate district
Provides guidance, visibility, funding, political support
MiBLSi Statewide Structure of Support
40
Developing CapacityThrough “Manualization”
Manuals are created to provide information and tools for implementation
Various levels
District Level
Building Level
41
Developing Capacity Through “Practice Profiles” (Implementation Guides)
• Implementation Guides have been Developed for – Positive Behavioral Interventions and Supports at the
Building Level– Reading Supports at the Building Level– Building Leadership Team– District Leadership Team
• Quick Guides have been developed for– Principals– Coaches
42
Practice Profile: Building Leadership Team Example
43
Program Performance Measures
Thank You› To all who joined the large group discussions› To the small working group members:
Patti Noonan Jim Frasier Susan Williamson Nikki Sandve Li Walter Ed Caffarella Jon Dyson Julie Morrison
Performance Measures Performance Measurement 1:
Projects use evidence-based professional development practices to support the attainment of identified competencies.
Performance Measurement 2: Participants in SPDG professional development demonstrate improvement in implementation of SPDG-supported practices over time.
Performance Measurement 3: Projects use SPDG professional development funds to provide follow-up activities designed to sustain the use of SPDG-supported practices. (Efficiency Measure)
Performance Measurement 4: Highly qualified special education teachers that have participated in SPDG supported special education teacher retention activities remain as special education teachers two years after their initial participation in these activities.
Continuation Reporting 2007 grantees Everyone else
Applying for a New SPDG Bidders’ Webinars Planning with potential partners Gathering needs data Increasing your knowledge about
evidence-based professional development
SPDG Program – OSEP Policies
No-cost extensions Personnel changes Continuation Reports Carrying out the activities in your
application
PTI/CPRC Poll Results Must contract or subgrant with a PTI or a CPRC
› Mandatory Recommended that
› Use good contracting practices› Communicate regularly and ensure all understand
expectations› Track progress with implementation and outcome
data – working together toward at least 1 of your SPDG’s objectives
› When applying for a new SPDG, meet with your PTI/CPRC (or both) to discuss the work you might do together
Highlights from the Survey It provides us with an avenue for receiving parent
input; disseminating information, resources and training to parents; and increasing parent involvement in their child's education.
We value the insights our Parent Center provides and use their feedback to adjust practices so that parents are more engaged in their children's education. They provide a perspective we wouldn't otherwise have. They, also, are a tremendous asset to the project through their development of parent materials and professional development. The professional development they provide for parents is the bridge between parents and the work within the project.
One of the greatest strengths is the desire, of both organizations, to provide family members with high quality, meaningful professional development to strengthen their ability to advocate, not only for their own children, but for all children. The collaboration is built on mutual respect and the shared vision of building meaningful family involvement in the schools and districts in our state.
The relationship in the past has been strained, but through our conversations with the new SPDG proposal, we have turned a corner and I am hopeful we can have a true partnership going forward.
Our PTI is well connected with families across the state, they work well with our
office and are willing to participate to help improve achievement for SWD. The PTI has recently been more involved and has brought in a national parent engagement expert. We are working to build from this Epstein work and bring those components of parent engagement into our SDPG work.
Challenges for these partnerships
Didn't fund them at the same level as other partners so PTI is not able to be at the table for all conversations. Overcoming entrenched idea by school districts that family engagement is an add on after they have perfected their process - okay to involve families in development
Regarding collaboration, there are not many challenges as our PTI is very responsive and collaborative. It's getting the work done and the systems in place that sometimes delay other activities to occur or move forward.
Challenges cont. Keeping collaborative efforts aligned with
the multiplicity of educational initiatives in the state; occasional conflict on perspectives of how grant funds and activities are to be carried out and perceptions of how the other PTI activity is aligned with grant objectives.
We need to schedule more time for face to face collaboration, and plan to do so in the future.
Matching grant needs with expertise in the PTI/CPRC
Ideas to Improve the Partnerships
SIGnetwork- perhaps build some networking/PD sessions that the SPDG and PTIs are required to attend/participate in together.
The requirement of the SPDG for the SEA to partner with the PTI is a topic that needs to be addressed. The feeling that I have gotten when it is raised is that regardless of the PTI's position, it is the SEA's responsibility to make the relationship work….
Continue to share ideas about examples of collaborative efforts.
More ideas Continue to share ideas about examples
of collaborative efforts. Clarify role of PTI. Emphasize cooperation,
communication, collaboration with both SEA and LEAs. Deemphasize advocacy/adversarial role.
Perhaps some joint webinars regarding what other SPDG projects are doing around parent and family collaboration would be helpful.
Go Mobile: Data Collection with an iPadCarlos Romero,
SPDG Program Technology Survey FindingsAudrey Desjarlais, Signetwork CoordinatorMay 2011
Response Rate 49 Respondents 76% state grantee response rate (35 of
46 funded state grantees) Included evaluators, directors, and
coordinators Representation: 25 – state
departments, 4 – IHEs, 6 - independent contractors, 5 - unknown
Top 6 Purposes to learn about technology applications:
Develop online training modules Evaluate activities and programs (includes
conduct needs assessment) Setup online habitats for
collaboration/communication Provide TA, consultation, coaching, and
mentoring Manage data (including collection, analysis,
reporting) Knowledge Management - disseminate
news/info
Technologies to learn about: Learning Management Systems Web conferencing Online meetings spaces Podcasts Collaborative Document Editing Learning Modules Multiuser Virtual Learning Environment (Second
Life) Online surveys Project Management Rapid e-learning modules Wikis
Resource/information repositories Video conferencing Media sharing sites Mobile device technologies Blogs Social Networking Vidcast Threaded discussions Vlogs Video Text-based live chat
Technologies to learn about:
Barriers to exploring, adopting and implementing emerging technologies:
Lack of knowledge (individual, staff) Lack of time (to investigate learn, and practice) Lack IT support Lack the infrastructure (at all levels: state,
district, school) Lack funding Accessibility issues: security/firewalls, internet
connection speed Rigorous approval process Things change too quickly
Strategies for how we’ll tackle those barriers for implementing technologies
Follow-up to Regional Meetings Website Updates May 2011
EVENTS CALENDAR New Events Calendar is Live – Details on all our PD events – PLC sessions, Implementation Conversations, Directors’ Webinars, Evaluators’ Events, Reporting Guidance and Bidders’ Webinars!
RESOURCE LIBRARY Resource Library is LIVE! Includes links, materials, resources, and information from SPDG grantees and National OSEP funded TA&D Centers.
EVALUATION Resources focused on Evaluation is now available. Includes information on reporting, instruments, data collection and display, and more.
Concurrent Sessions• Evaluators – Maryland C
• Directors, Coordinators and PTI Directors – Maryland AB
Speed Share: SPDG Partnerships with PTIs and CPRCS• 6 Speed Share Rounds
• Pairs will have 5 minutes to share information with one another.
SPEED SHARE - DISCUSSION ITEM
S
• Introduce yourself (name, state, role).• Identify the top 1-3 activities your state has
been successful with parent engagement.• Explore how your projects make use of
contracts with parent organizations (PTIs/CPRCs)
• Describe the challenges you’ve experienced collaborating with one another.
• Explain how we can build relationships with PTIs and CPRCS. What are the effective practices?
• Describe what kinds of needs assessment you did to develop your collaborative contract and action plan.
• Share product or material links.• Share strategies for Family Engagement
Dissemination.
Communities of Practice• Cynthia Glimpse, Technical Assistance Coordination
Center (TACC)
• Amy Jenks, Director, New Hampshire SPDG
PLACEHOLDER - ADD Presenter’s SLIDES HERE
PLC Discussions – 3, 2, 1 Format1 – Coaching2 – Collaboration with IHEs, Leadership
Development, and Capacity Building and Restructuring
3 - Implementation Conversations4 – Response to Intervention / Multi-tiered Models of
Intervention5 – Scaling Up and Comprehensive Systemic
Professional Development6 - Technology
Opportunities to Learn and Share