SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE? SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE? Charles Smith, Ph.D. Executive Director, David P. Weikart Center for Youth Program Quality Vice President of Research, Forum for Youth Investment May 9, 2013; 9:00-9:30 am
48
Embed
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE? Charles Smith, Ph.D. Executive Director, David P. Weikart Center for Youth Program Quality Vice President.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
SYSTEM-BUILDING AND QUALITY:WHAT’S AT STAKE?
Charles Smith, Ph.D. Executive Director, David P. Weikart Center for Youth Program QualityVice President of Research, Forum for Youth Investment
May 9, 2013; 9:00-9:30 am
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
Agenda
• Quality Improvement Systems1. Building QIS
2. Site Level Process
3. System Accountabilities
• Local Models of Quality Improvement System Accountability
• Why Build Systems for Developmental Settings?
• APPENDIX
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
Building QIS
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
l
2012-2013 Dissemination
Pol
icy
Set
ting
Org
an
iza
tion
Se
ttin
gP
oint
of S
ervi
ce
Set
ting
>21,125 StaffEstimate based on mean of 6.5
staff per site in YPQI Study Sample
85 Networks/ Systems
>276,250 Child & Youth
Estimate based on mean daily attendance of 85 youth per day in YPQI
Study Sample
>3250 Sites
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
Building a QIS: Stages and tasks
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
Site Level Continuous Improvement Process
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
Instructional Practices“Quality” at the Point of Service Level Setting
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
Four Continuous Improvement PracticesOrganization Level Setting
(Plus 10 hours of TA/coaching for site managers to implement the four CI practices)
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
Targeted Staff Trainings for Instructional SkillsCI Practice #4
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
System Accountabilities
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
System Supports for CI Practices
Policy: Eligibility, Targeting, Low/high stakes
Training, TA & Coaching
EvaluationExternal Raters, Program Evaluation
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
System Accountabilities: Higher Stakes
Objective Data Publicity Action Improved
Outcomes
Higher Stakes
Accountabilities
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
Objective Data
Meaningful Information
Action/ Expertise
Improved Outcomes
Lower Stakes Accountabilities
Interpretive Community
•Team Self Assessment •Review external scores
Team Planning and Implementing
•Improvement planning•Performance coaching
Higher Stakes Accountabilities
System Accountabilities: Lower Stakes
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
Higher Stakes: System Needs and Challenges• System Needs1
– Standards beyond licensing regulations– Accountability policies based on assessment and monitoring– Program and practitioner outreach and support– Financing incentives specifically linked to compliance with
quality standards
• Challenges2
– Differences in structure and design (e.g. measures)– Lack of coordination across agencies and data systems– Policies lack clarity about goals, timing and expectations for
improvement1. National Child Care Information and Technical Assistance Center. (2009). Quality Rating Systems: Definition and Statewide Systems. Fairfax, VA: National Child Care Information and Technical Assistance Center.http://www.dataqualitycampaign.org/files/4_NCCIC_QRIS.pdf2. Tout, K., Zaslow, M., Halle, T., and Forry, N. (2009). Issues for the next decade of quality rating and improvementsystems. Washington, DC: Office of Planning, Research and Education.
• Arkansas QRIS– Program or Business Administration Scale (PAS or BAS)– Traveling Arkansas Professional Development Registry (TAPP)– Program Quality Assessment (PQA)– School-Age Care Environmental Rating Scale (SACERS)– Various other criteria– http://www.arbetterbeginnings.com/wp-content/uploads/2011/12/
• Oakland requires participation, scores are tied to funding, and reports go to the city government
• AR’s system is voluntary, but once in it, scores feed into 3 tier system that are used for incentive grants and published ratings for families
Policy Stakes Higher
Accountabilities Oakland AR QRIS
Participation Mandatory x Financial Incentives x xPublished Scores x xCI Practices Oakland AR QRIS
Self Assessment optional xExternal Assessment x xPlanning with Data x xPerf Feedback & Coaching x xTraining: Instruction Methods x x
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
Oakland Unified School District (OUSD) and Oakland Fund for Child and Youth (OCFY)
• http://publicprofit.net/Services/Evaluation/• OUSD and OCFY are funders• Public Profit is the intermediary and evaluator• System is voluntary for self assessment, required for external
assessment and planning• Process:
– All sites get external assessments of 2 program offerings– All sites receive individualized “planning with data” type meetings with the
evaluators, who go over their scores and work with them to create improvement plans
– Reports of external assessments are also sent to city government – All sites have access to Methods trainings– Programs with low scores receive additional coaching
• Incentives:– Programs with extremely low scores (scale scores under 2) that don’t improve
over the course of 2-3 years could lose their funding– No sites have lost funding almost no programs score that low
Arkansas Quality Rating and Improvement System (QRIS)• http://www.arbetterbeginnings.com/• Arkansas State University serves as intermediary for system for school-age care
programs• System is voluntary• Process:
– A three-tiered rating system that the PQA scores feed into, but it is only one of multiple measures.
– Components included in the ratings are:• Administration• Administrator/Staff Qualifications & Professional Development• Learning Environment• Environmental Assessment• Child Health & Development
• Incentives:– Ratings are used to offer both incentive grants and published ratings for families to use to
decide where to send their children– Incentive Grants are available upon meeting certification standards at each of the 3 levels
• At level 1 and level 2, it is renewable for a maximum of 9 years (not to exceed 6 years at either level 1 or level 2).
• At level 3, the Incentive Grant is available annually, as long as the facility continues to meet the standards.
• Incentive grant amounts are based on a combination of licensed capacities, current Level and the number of years spent at that level.
• MI/OK require process to maintain funding, but focus on supports and coaching
• Kansas City has 3 tier incentive system based on completion of YPQI elements to get funding at the different levels
• VT has 5 tier recognition program based on various practices where programs get funding, public awareness, discounts, and funding opportunities
Policy Stakes Middle
Accountabilities MI / OK 21 CCLC
Kansas City UW VT
Participation Mandatory x Financial Incentives x x xPublished Scores CI Practices MI / OK
21 CCLCKansas
City UW VT
Self Assessment x x xExternal Assessment x x xPlanning with Data x x xPerf Feedback & Coaching x xTraining: Instruction Methods x x x
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
Kansas City United Way
• http://www.unitedwaygkc.org/nonprofits/qualitymatters.html• UW is the intermediary, partners with Francis Institute for coaching
and University of Missouri, Kansas City for evaluation/external assessments
• System is voluntary• Half the sites that participate are United Way funded programs, but
United Way funds the YPQI process for all sites• Process:
– Have high fidelity to YPQI assess-plan-improve, with all sites doing assessments, planning and receiving coaching
• Incentives:– Have a 3 tier incentive system that is based on completion of the elements of the YPQI,
and they receive $300, $500 and $750 accordingly– Example: Participation Level 3: $750 – Completion of:
• Conduct a fall team based PQA self-assessment which includes observation and team consensus and enter data into the Online Scores Reporter by [DATE].
• Program Improvement Plan created and entered into Online Scores Reporter by [DATE].• 75% completion rate of goals set in Program Improvement Plan**by [DATE]• Attend 7 different workforce training (1-4 site staff may attend each training and must stay
• http://www.vermontafterschool.org/• http://dcf.vermont.gov/cdd/stars• Are intermediary, they serve both 21st CCLC and AHS/QRIS programs• Both systems encourage use of the YPQI, but is voluntary• Process:
– Have a 5 tier star system for recognition of programs, based on practices in these areas:• Compliance with state regulations• Staff qualifications and training;• Interaction with and overall support of children, families, and communities;• How thoroughly providers assess what they do and plan for improvements; and• The strength of the program’s operating policies and business practices.
• Incentives: – The benefits for the star system, tiered based on number of stars the program has earned:
• The Child Care Financial Assistance Program (CCFAP) pays a higher rate on behalf of families.
• Bonus payments for EACH level achieved:• Public awareness of STARS participation if requested. Options include: listing on the
STARS website, supply of STARS brochures, and a customized press release.• The opportunity to apply for grants open only to programs that are in STARS or are
nationally accredited• Discount on purchases from a list of corporate sponsors.
Self Assessment x xExternal Assessment x xPlanning with Data x xPerf Feedback & Coaching xTraining: Instruction Methods x x
SYSTEM-BUILDING AND QUALITY: WHAT’S AT STAKE?
Indianapolis Marion County Commission on Youth (MCCOY)
• http://www.mccoyouth.org/• MCCOY is intermediary• Very, very low stakes--Participation is entirely voluntary• Process
– Sites do the full Assess-Plan-Improve sequence– Sites can do self and external assessment, usually only once a year, but could do
more– No coaching or training in instructional coaching– Methods workshops are offered for all sites– Recruitment can be hard, but they focus on partnerships and Methods to get sites
engaged– Programs choose to join cohorts (2-3 cohorts per year)– Programs can choose to participate multiple times but there is no emphasis on
tracking year to year improvement.
• Incentives– Programs have access to supports and learning community, and a lot of flexibility in
Seattle & Washington State- Raikes Foundation and School’s Out Washington• http://raikesfoundation.org/Secondary.aspx?file%3daboutmission• http://www.schoolsoutwashington.org/index.htm• Raikes Foundation is the funder• Schools Out Washington is the intermediary, with other local supports• Has funded 1-3 cohorts across the state since 2008 • System is voluntary, programs apply• Programs span funding and accountability streams• Process
– There is an intensive application process that comes with funding to programs to subsidize their participation and all of the supports are free
– High level of fidelity to YPQI– Sites get less and less supports over a 3 year period…beyond that they can apply
for funding to get a al carte services
• Incentives:– Participation comes with funding to programs to subsidize their participation and