EEC Board Policy and Research Committee Meeting May 5, 2014

1

EEC Board Policy and Research Committee Meeting

May 5, 2014

Quality Rating and Improvement System (QRIS)Status Update

QRIS Accomplishments

2

2009-2011• Developed standards• Piloted standards and made revisions• Launched QRIS

2012• Hired QRIS Staff - 6 Program Quality Specialists and 4 QRIS Health Advisors• Began verifying QRIS applications and granting Level 2 ratings

2013• Continued application verification and granting QRIS Level 1 and 2 ratings • increased collaborations with external partners and stakeholders• Started comprehensive evaluation of QRIS• Began development of QRIS policies and inter-rater reliability protocols

2014• Adopted new policy for Environment Rating Scales (ERS)• Completed QRIS Field Survey• Completed QRIS Validation Study Revisions• Completed 406 classroom and provider ERS Reliable Rater assessments

Next StepsSpring 2014• Hire and train 4 PQS to fill the PQS that transitioned to licensing roles• Begin granting Level 3 programs• Complete QRIS evaluation and revision process- EEC Board vote in June• Work with public schools developing a plan to create 4th QRIS program type• 1st QRIS Conference for QRIS Improvement Grantees

Summer 2014• Develop QRIS Policy Manual• Begin granting Level 4 programs

Fall 2014• Develop on-line training modules for each level of QRIS • Develop standardized training modules for QRIS coaches, mentors, family systems support

and other QRIS support staff• Launch at-scale QRIS Validation Study• Launch revised on-line application system

Spring 2015• QRIS/NAEYC Alignment Study and QRIS/Head Start Alignment Study• QRIS Working Group and QRIS Professional Development Review Teams- Annual QRIS

Guidance Updated

December 2015• QRIS Validation Study Findings Released

3

4

Space

and Fu

rnishi

ngs

Person

al Care

Routi

nes

Langua

ge and

Reaso

ning

Activit

ies

Intera

ction

Progra

m Struc

ture

Parent

s and

Staff

Overal

l0

20406080

100120

012.4

0 0 1 1 0 031

61.7

12.448.5

13.4 19.70

21.5

69

25.9

87.651.5

85.6 79.2100

78.5

Percent of programs at quality benchmarks on ECCERS-R

% in good range% in Minimal Range% Inadequate

Although the overall average of the ECCERS-R scores was over a 5, the table indicates that a significant portion of the programs (21.5%) fell in the MINIMAL range overall. Review of the frequencies of the subscales indicates significant portions of programs falling in the minimal range for Space and Furnishings (about 31%), Personal are Routines (approximately 62%), Activities (about 49%) and Program Structure (approximately 20%). Additionally, approximately 12% of programs scored in the INADEQUATE range for Personal Care Routines, further underscoring the need for additional supports and technical assistance in this area.

5

Space

and Fu

rnishi

ngs

Person

al Care

Routi

nes

Langua

ge & Rea

soning

Activi

ties

Intera

ction

Progra

m Struc

ture

Paren

ts and

Staff

Overal

l0

20406080

100120

9.546

4.8 7.9 8 3 0 1.6

34.9

36.5

14.236.5 25.3 33.5

039.7

55.617.5

8155.6 66.7 63.5

10058.7

Percent of programs at quality benchmarks on ITERS-R

% in good range% in minimal range% in inadequate range

In contrast to the preschool programs, the data indicates a greater portion of the programs falling in the minimal and inadequate ranges, both overall and among the subscales. Of particular note, around 40% of programs did not meet the GOOD benchmark overall, twice as many as with the preschool classrooms. Among the subscales, substantial portions of programs did not meet the GOOD benchmark for Space and Furnishings, Personal Care Routines, Activities and Program Structures. This suggests additional supports are needed for the Infant and Toddler programs in the promotion of quality as measured by the ITERS-R.

6

Space

& Furni

shing

s

Person

al Care

Routi

nes

Langua

ge & Rea

soning

Activi

ties

Intera

ction

Progra

m Struc

ture

Paren

ts & St

affOve

rall

0

20

40

60

80

100

120

1545

0 10 5 10 0 5

60

35

40

65

25 25

5

55

25 20

60

25

70 6595

40

Percent of programs at quality benchmark on the FCCERS-R

% in good range% in minimal range% in inadequate range

Unlike the center-based programs, a majority of the observed family child care homes did not meet the GOOD bench mark OVERALL; instead a majority fell in the minimal range. A majority of programs also fell below the GOOD range for Space and Furnishings, Personal Care Routines and Activities. Importantly, significant portion of programs fell in the INADEQUATE range for Space and Furnishings and Personal Care Routines. It is important to note that despite the large number of programs in the minimal range OVERALL, a majority of family child programs fell in the GOOD range for Language and Reasoning, Interactions, Program Structure and Parents and Staff. Despite this, a significant number of programs fell in the minimal range for Language and Reasoning (40%) and Program Structure (35%).

7

Space

and Fu

rnishi

ngs

Persoa

nl Care

Routi

nes

Langu

age an

d Rea

soning

Activi

ties

Intera

ction

Progra

m Struc

ture

Paren

ts and

Staff

Overal

l01234567

Average Scores by Program Type

ECCERS-RITERS-RFCCERS-R

As can be seen, the preschool scores are typically higher, followed by the infant toddler programs and then the family child care homes. This pattern is found across all subscales and the OVERALL, except for the Interactions subscales in which the family child care homes score slightly higher than the infant and toddler classrooms. This suggests that more support is needed for family child care homes followed by infant and toddler classrooms to raise quality.

8

QRIS Participation Data

Since the launch of the on-line, QRIS Program Manager (QPM) in January 2011, 5,329 programs (unique count) have created a total of 7,429 QRIS applications (final status) using the QPM system. April 2014- Major accomplishments:

• 709 of the programs participating are nationally accredited: 35 COA, 117 NAFCC and 557 NAEYC.

• 6,334 of those applications have been granted a QRIS Level. • Level 1: 4,834• Level 2: 1,467• Level 3 pending reliable rater: 3• Level 3: 30

QRIS Program Type

Central MA

Metro Boston

Northeast Southeast and Cape

Western MA

(blank) Grand Total

After School/Out of School Time

120 163 221 221 126 3 854

Center/School Based

328 369 374 500 346 5 1922

Family Child Care

903 926 1549 563 710 1 4652

Grand Total 1351 1458 2144 1284 1182 9 7428

Note: In FY13, Metro (Region 4) was dissolved for Grantee purposes. Towns were reassigned to neighboring regions to align with licensing offices. 9 programs were not assigned to a region in the QPM. These are license-exempt programs that were not recognized automatically by EEC’s system.

9

1,3511,458

2,144

1,284

1,1829

7,428

Programs by RegionCentral MAMetro BostonNortheastSoutheast and CapeWestern MA(blank)Grand Total

854

1,922

4,652

Programs by Type

After School/Out of School TimeCenter/School BasedFamily Child Care

Field Experiences with and Impressions of QRIS

10

• The purpose of the survey was to gather input and feedback about how the QRIS is working and the ways in which the system can be improved.

• Random selection of programs and providers in QRIS, and high response rates, suggest the views expressed are reflective of those in QRIS more generally.

UMass Donahue Institute Applied Research and Program Evaluation

Promising Findings

11

• Despite some ongoing skepticism and frustration extending from early implementation, most acknowledge the inherent value in a quality improvement system and appreciate its focus on quality.

• Majority of QRIS participants believe the system helps programs improve, is an important priority for EEC, and will help elevate the public perception of early education and care in the state.

• Data show an emerging understanding of the QRIS, particularly around the system’s goals, standards, and benefits of participation.


12

• Most QRIS participants believe the system has helped improve their quality.

- Attributed to clearer goals and improvement plans, increased self-reflection through use of tools, and self-assessment.

- FCC providers were most optimistic about improvement.

• Most programs and providers plan to work toward the next QRIS level, many of whom expect to do so within the next year.

Promising Findings Continued


13

• Mixed views in the field about whether QRIS ratings accurately reflect a program’s quality.

• Common barriers to moving up to the next level include education and training requirements, time, cost, and documentation.

- Concern that progress might be hindered by just one or two standards

- Education requirements were a particular concern for centers and family care providers

• Several respondents from public schools and family care providers felt that some aspects of the system were designed without their settings in mind.

Challenges and Barriers


14

Recommendations and Conclusions

Develop a robust infrastructure to help programs and providers improve and advance

• Ensure adequate logistical and technical infrastructure is in place to accept, verify, and communicate about applications.

• Expand on-site support and technical assistance for programs and providers.

• Build programs’ and providers’ capacities to effectively use measurement tools included in the system through further training.

• Increase access to EEC-funded or reduced-cost college courses and trainings.


15

Simplify and align QRIS with other quality measures and requirements

• Reduce complexity by decreasing paperwork, reducing measurement tools, and removing standards not viewed as integral to quality.

• Consider QRIS, NEAYC, and licensing in relation to one another. Align and streamline systems so that they can function in complementary ways.

• Deliberate the role of public schools in the QRIS and how the system can be aligned to best support quality improvement in school-based settings.

Recommendations and Conclusions (continued)


16

Remove or reduce common barriers to maximize success

• Address the issue of higher education requirements for centers and family care providers.

• Provide greater flexibility in relation to some of the standards or how the standards are met.

Enhance communication and messaging to build on initial success of QRIS

• Develop focused messaging plans to ensure that programs and providers of all types have access to clear and consistent information about the system, its requirements, and its timelines.

• Build momentum and enthusiasm by capturing early success stories.

Recommendations and Conclusions (continued)


17

EEC Plan for Improved Information Sharing and Technical Assistance

• Bi-weekly QRIS announcements via email subscription

• 2nd Thursday /month- Technical assistance webinars for coaches and mentors

• 2nd Thursday /month – Technical Assistance webinars for family child care systems support

• 4th Thursday /month – Topic/content based learning for programs and providers

• Monthly Orientation Sessions in all EEC regional offices

• QRIS On-line Learning Community

18

QRIS Standards Evaluation and Revision Process

EEC began a comprehensive evaluation and revision process in May 2013. Steps in this process include :

1. Review QRIS standards and measurements with our UMass Donahue partners.

2. Collaborated with RTT Tech Assistance staff to identify research that supports best practices.

3. Program Quality Unit began development of guidance documentation to inform inter-rater reliability protocols, as well made initial recommendations for standards revisions.

4. Collaborated with EEC Policy, Legal, Fiscal, I.T. and Field Operations Units.

5. Established QRIS Working Group and Professional Development Review Team.

6. Vet standards revisions with EEC staff, the field, EEC Board and other stakeholders.

Key Findings from QRIS Evaluation Process

Some standards are presenting significant barriers for programs to move forward in QRIS:

• Programs/providers are not prepared for a health consultant visit at level 2

• Lack of qualified health consultants in the field

• Bachelor degree requirement is unattainable for many educators and providers

• Ongoing Professional development does not always connect the learning experiences to certifications

• Inter-rater reliability protocols need to be established and maintained

• Programs and providers need QRIS guidance documents

• Advancement to the next level in QRIS after one year is not possible for some programs and providers

• QRIS levels need further definition19

20

Where do we want QRIS be on January 1, 2016?

21

Statutory Requirements for a Quality Rating Improvement System (QRIS)

In 2008, EEC was charged with establishing a method of measuring quality in early education and care programs.

Massachusetts General Laws Part I, Title II, Chapter 15D, sec. 12(a) The department shall establish a comprehensive system for measuring the performance and effectiveness of programs providing early education and care and services. This system shall include, but not be limited to, outcomes of the kindergarten readiness assessment system and additional educationally sound, evaluative tools or developmental screenings that are adopted by the department to assess developmental status, age-appropriate progress and school readiness of each child; outcomes of evidence-based intervention and prevention practices to reduce expulsion rates; and evaluations of overall program performance and compliance with applicable laws, standards and requirements;

(e) The comprehensive system for measuring the performance and effectiveness of programs shall be designed to measure the extent to which every preschool-aged child receiving early education and care in the commonwealth through the Massachusetts universal pre-kindergarten program has a fair and full opportunity to reach such child's full developmental potential and shall maximize every child's capacity and opportunity to enter kindergarten ready to learn.

22

National QRIS Implementation44 states have a launched a statewide QRIS system

23

Race to the Top Funding State Funding

QRIS Support Staff

• 6 Program Quality Specialists• 1 Manager of Program Quality &

Improvement • 4 QRIS Health Advisors• 1 Child Health Care Advisor• 1 Manger of QRIS Improvement Grant • QRIS Improvement Grant readers and

administration support• Environment Rating Scales (ERS) Reliable

Raters• Information Technology Support

• 1/3 time EEC Professional Development support staff for QRIS

• Family Child Care Systems support staff

Professional Development

• Educator Scholarships• QRIS Measurement Tools Training• Training for EEC staff and ongoing support• Training and ongoing support for EPS Coach

and Mentor• Program and Provider Business Planning

• College coursework and degree attainment• QRIS content related trainings• Educator & Provider Support (EPS) Staff, Coaches, Mentors and Trainers

Fiscal Incentives

• QRIS Improvement Grant – program planning and durable goods

• Infant/Toddler rate increase

Program Quality Supports

• Research – QRIS Validation Study, QRIS Field Survey, support with QRIS evaluation and revisions

Family Engagement

• QRIS Family and Community Awareness Campaign

• Coordinated Family and Community Engagement (CFCE) Grant

Program Quality Supports

24

QRIS Staffing• High QRIS caseloads- average of 833 programs:1 Program Quality Specialist

Fiscal Incentives for Programs and Providers• Increased salaries to compensate for advancement on the Career Ladder (QRIS

requirement)• Increased reimbursement to compensate FCC Providers as they further their education• Staff benefits (QRIS requirement)• Tiered system of reimbursement (supports the foundation of a Quality Rating and

Improvement System)

Information Technology• Updates for the on-line application (difficult to use, slow, outdated and does not reflect all

current QRIS policy) • Data reporting limited and inconsistent (duplication, not well aligned with other systems)• No ability to renew applications

Needed Program Quality Supports

25

How do we sustain QRIS?

• Create a strong vision for QRIS

• Develop a strategic plan, including; concrete goals, objectives, timelines and key policy questions

• Identify strengths and challenges with the current supports

• Make a fiscal commitment

• Build an infrastructure that will sustain QRIS

• Strong communication with the field, families and other stakeholders

• Continually evaluate and revise the system to reflect needs, research and best practices

Questions, comments, concerns…

EEC Board Policy and Research Committee Meeting May 5, 2014

Documents

qris level

level of qris

qris conference

qris accomplishments

qris coaches

revisionslaunched qris

qris applications

qris program type