SB-191 - An Overview to Implementing Effective Evaluation

• History and context

• Main provisions

• Reforms in other states

• Framework to evaluate principals

• Affecting student achievement

• Timeline

HIGHLIGHTS

SB-191ISSUES BRIEF VOLUME 1, ISSUE 1

AN OVERVIEW TO IMPLEMENTING EFFECTIVE EVALUATION

SEPTEMBER

2013

DISCOVER LESSONS LEARNED ON SB-191 WITH THIS PANORAMIC BRIEF

AUTHOR: Jeffrey A. Roberts

EDITORS: Greg Benchwick and Bruce Caughey

PEER REVIEW:CASE and CDE

GRAPHIC DESIGN: FiG Advertising

PROOFREADER: Kris Kitto

ABOUT THE AUTHORJeffrey A. Roberts has worked in journalism and public policy for more than 30 years at The Denver Post, the Fort Myers (Fla.) News-Press and the Center for Colorado’s Economic Future at the University of Denver. He recently was named Executive Director of the Colorado Freedom of Information Coalition.

The opinions expressed in this publication are those of the authors and do not necessarily represent those of the Colorado Association of School Executives (CASE), its donors, members or departments. The contents of this publication, including all figures, tables, and drawings, are the intellectual property of CASE, unless otherwise noted. All rights reserved. Removal or alteration of copyright notices or trademarks is not permitted. Forwarding or reproduction of this publication or parts thereof for commercial use is not permitted without the explicit written authorization of CASE. All program names or services of CASE used in this publication as well as corresponding logos are trademarks or registered trademarks of CASE in the United States. CASE does not guarantee the accuracy or completeness of information, text, graphics, links, or other elements contained in this publication. This publication is provided without any warranty, whether explicit or implicit. This applies in part but not exclusively to a warranty of marketability and suitability for a particular purpose as well as a warranty of non-violation of applicable law.

Copyright © 2013 CASE/Jeffrey A. Roberts

THIS ISSUES BRIEF WAS PREPARED BY THE COLORADO ASSOCIATION OF SCHOOL EXECUTIVES (CASE) WITH FUNDING FROM THE BILL AND MELINDA GATES FOUNDATION.

TABLE OF CONTENTSLetter from CASE Executive Director 1

Introduction 4

Overview 5

Defining Expectations Under the Seven Quality Standards 6

Background 7

Main provisions of SB-191 10

SB-191 Timeline 11

Reforms in Other States 13

National Trends in Effectiveness Implementation 15

Principal Effectiveness in Colorado 17

Framework for System to Evaluate Principals 18

Steps in the Year-Long Principal Evaluation Process (State Model): 19

Understanding the Rubrics for Success 21

Measures of Student Learning 23

As this is the first in a series, we have not discussed the time and money that will be required to effectively implement these school improvement initiatives. But we know it’s important.

We know that implementing SB-191 will certainly require a lot of both: more time for principals to properly evaluate teachers, more time for administrators to properly evaluate principals, and more money – or a reallocation of existing resources – for start-up costs and on-going annual expenses.

In its final report, the State Council for Educator Effectiveness acknowledged that “these costs will be a burden to districts at a time when districts are already under severe financial pressure.“We understand that and will continue to advocate for more comprehensive financing for these initiatives.

Welcome to the new world. Colorado’s education system is changing, and one of the primary drivers of this change will be the successful understanding and implementation of SB-191. This issues brief provides us with a broad framework to understand the new law, looking at the background, historic context, lessons learned from other states and more.

DIRECTOREXECUTIVE

FROM CASE

Colorado Association of School Executives

BRUCE CAUGHEY

LETTER

1

The state is helping districts with some non-cash assistance, as well. Using a Round 3 Race to the Top federal grant, CDE has developed the Colorado State Model Evaluation Systems for teachers and principals, the CDE Resource Bank of assessments and assessment review tools, training opportunities and professional development materials. The five integration pilot districts and one integration BOCES are receiving additional help from the Colorado Legacy Foundation to implement both the new evaluation systems and the updated Colorado Academic Standards.

But that’s not enough. One-time start-up costs for districts were estimated at $53 per student in a 2011 study commissioned by the Council on Educator Effectiveness. That figure, which was not adjusted for district

size, includes training on the new system for educators and evaluators, setting up data systems, selecting evaluation measures and creating an appeals process. Major ongoing costs include the conducting of evaluations, the regular analysis of student data and the identification of ineffective teachers who will need improvement plans and related support.

As a part of this statewide push to successfully implement SB-191, CASE has established the Leadership Academy for Principals, an ongoing seminar series that delivers robust learning opportunities centered on the Principal Quality Standards, and helps principals and other leaders unravel the complexities of the law.

It’s a complicated system, but one that, if properly implemented, will provide for a better education future for Colorado’s students. As we roll up our sleeves and get to work, I think it’s important that we recognize the commitment, sacrifice and amazing work Colorado educators are putting into this process of change.

Bob Dylan was right, the “waters around us have grown... the old road is rapidly rising,” and if we aren’t careful it very well could shake our windows and rattle our walls. Know this, no matter where these changing times take us, CASE will be there, walking the path with you, as we come together to step into the challenging and important task of improving Colorado’s education system, one Quality Standard, one student, one teacher, one principal at a time.

2

development and requires that teachers and principals be rated in one of four categories: highly effective, effective, partially effective or ineffective. To get or keep non-probationary status (tenure), teachers must show that they are effective or better.

This issues brief from the Colorado Association of School Executives (CASE) is the first in a series of briefs that will begin to explain the new rules and how they are being implemented. The paper focuses on how principals are to be evaluated. It also provides an overview of the new law, some history behind the legislation and a short guide to similar reforms in other states. In addition, it offers some perspective and advice from administrators in Colorado school districts that are piloting the revamped system.

School leadership ranks second only to classroom instruction among school-based factors that influence student achievement, according to Wallace Foundation research. Many of the things that happen inside school buildings, when examined separately, affect learning in small ways. But a strong principal can create an environment in which individual factors have a synergistic effect when blended, producing considerably better results.

Recognizing the importance of school leadership, federal and state policymakers a few years ago began to push for reforms aimed at ensuring that each public school has an effective principal. The Colorado General Assembly passed legislation in 2010 that thoroughly overhauled the evaluation process for both principals and teachers.

The bill was designed to increase educator effectiveness by making evaluations more meaningful and the feedback from them more useful, with the end goal of improving student achievement.

SB-191 goes into effect statewide in the 2013-14 school year and requires that at least half of a principal’s evaluation be based on the academic growth of students in that principal’s school and that at least half of a teacher’s evaluation be based on the academic growth of his or her students. The remaining portions measure how well they meet the new standards that define what it means to be an effective principal or teacher. The law also mandates that evaluations be conducted annually, promotes new opportunities for professional

INTRODUCTION

4

district’s superintendent, who is advised by a review panel. Mutual consent hiring also is part of SB-191. District administrators no longer will be allowed to “force place” a teacher into a school without the principal’s approval. Teaching positions will be filled with the consent of both the teacher and principal, with input from other teachers at the school.

Twenty-seven school districts piloted the model system, including 15 selected by CDE and others that are partnering with the state to map their own systems to the expectations in SB-191. Five districts and one BOCES received grant money from the Colorado Legacy Foundation to be “integration” pilots, meaning they are implementing both the evaluation system and updated Colorado Academic Standards. Feedback from the pilots, which began testing the principal rubric and evaluation matrix during the 2011-12 school year, is being used to make improvements as the law takes full effect.

With SB-191 in effect, Colorado school districts and Boards of Cooperative Education Services (BOCES) must assure the Colorado Department of Education (CDE) that they are implementing new evaluation systems that align with the Quality Standards written by the State Council for Educator Effectiveness and rules approved by the State Board of Education. A district may develop its own system, as long as it submits a crosswalk of locally developed standards to the state Quality Standards, or it may use the Colorado State Model Evaluation System developed by CDE. Hybrids are allowed, such as using the state model for principals and a different aligned system for teachers.

The Colorado State Model Evaluation System is designed to be a continuous process of regular conversations, observations and feedback involving the evaluator and the teacher or principal being evaluated. A year-long cycle starts with an orientation to make sure the system is

understood. Educators assess themselves and set goals that are reviewed by their evaluators. Their progress is checked mid-year, giving them opportunities to improve before they are evaluated again at least two weeks prior to the end of the school year. Year-end ratings lead to additional goal setting and performance planning.

For teachers, SB-191 will fundamentally change the concept of tenure in Colorado once it takes full effect. Under the existing system, teachers who have three years of satisfactory performance on the job and are re-employed for a fourth year automatically earn non-probationary (tenured) status, making it difficult for them to be fired later if they perform poorly. Under the new process, tenure is no longer tied to length of service. Non-probationary status is earned after three straight years of effective ratings and it can be revoked after two consecutive years of ineffective or partially effective ratings. Teachers can appeal a second consecutive ineffective rating to their

OVERVIEW

5

Principals and assistant principals under the state model are rated on six Quality Standards pertaining to professional practices, and a final Quality Standard pertaining to measures of student learning. To determine how well principals meet these standards, the administrators evaluating them use their own observations, input from teachers in that principal’s school, those teachers’ ratings and how much those teachers have improved. As with professional-practice standards for teachers, districts have some flexibility in determining the weight of each professional-practice standard for principals, but each standard must account for no less than 7.5 percent of the total.

1. Strategic leadership2. Instructional leadership3. School culture and equality leadership4. Human resources leadership5. Managerial leadership6. External development leadership

7. Leadership as it pertains to the academic growth of students in a principal’s school. At least one other measure of student academic growth also is required. To make this determination, the state model calls for using CDE’s School Performance Framework, a snapshot of a school’s academic achievement, academic growth, growth gaps and post-secondary readiness. At least one other measure of student academic growth is also required. According to CDE, this measure should be aligned to the goals and expectations of the teachers in the school. For example if the school has a particular focus on third-grade reading (maybe identified in their School Improvement Plan), the principal could have the school’s third grade reading TCAP scores count as a portion of their evaluation. This measure should highlight the focus of school-wide collaborative efforts. The academic growth standard accounts for at least half of a principal’s total rating.

6

DEFINING

UNDER THE

QUALITY STANDARDS

EXPECTATIONS

7

critical, cannot fully ensure effectiveness.

There also was a perception that, in many school districts, teacher and principal evaluations were pro forma exercises focused mostly on complying with minimum requirements. Reform advocates argued that evaluations should have consequences and provide meaningful feedback. What was needed, they said, was an evaluation system tied to research-driven definitions of educator quality and determined in part by an educator’ s impact on their students’ progress toward gaining the necessary knowledge and skills to succeed beyond high school. “If you take a student who is already two or

Measures of quality for public school educators historically have emphasized indicators of professionalism such as certification, years of service and postgraduate credentials. To be considered a “highly qualified teacher” under the federal No Child Left Behind Act of 2001, a core-content teacher must hold a degree, be fully licensed (unless granted a waiver) and demonstrate subject-matter competency, typically by passing at least one exam required by the state. In the 2000s, with research showing that student learning is closely tied to teacher performance in the classroom and the leadership of school principals, many began to reconsider what makes an effective educator. It was now thought that a depth of knowledge, while

three grade levels behind and you put them in the classroom of an ineffective teacher or an ineffective principal, we know that … you make that gap so wide it’s almost un-closable,” says State Senator Mike Johnston, the primary sponsor of SB-191, in a video posted on his website.

The genesis of Colorado’s educator effectiveness law can be traced to state legislative efforts in the 1990s to foster greater accountability in primary and secondary education. Four years before the passage of the No Child Left Behind law, which mandated model content standards and assessments for all states, the Colorado Student Assessment Program (CSAP) began phasing in a series of tests designed to measure

BACKGROUND

7

students’ grasp of reading, writing, math and other subjects as they moved toward graduation. The CSAP was intended to provide information on the effectiveness of teachers, schools and districts as well as student performance, but for the first several years it showed only an aggregated percentage of proficient or non-proficient students at certain grade levels.

In 2006, the creation of a statewide data system that assigned a unique identification number to each student allowed for tracking the performance of public school children from year to year, even if they moved to a different school in another Colorado district. This longitudinal approach to collecting and analyzing standardized test

to the success, or lack thereof, of individual students.

Legislation in 2009 added an identifier system for teachers, initially as a way to collect information on the “teacher gap” problem, where inexperienced teachers are disproportionately placed in low-performing schools. Once fully implemented, matching educators with the longitudinal growth of their students also will provide valuable data for improving instruction and allow for teachers to be evaluated in part on the progress of their students, as specified in SB-191.

Reforms approved at the local level in Colorado also helped set the stage for changes eventually mandated by the state. The Eagle County School

data led to the adoption in 2008 of the Colorado Growth Model, a statistical method of comparing the long-term academic achievement of individual students and groups of students. Instead of point-in-time information, a calculated growth percentile could now describe how much a student had progressed during the previous year. The data also are used to determine whether a student is on track to reach or maintain proficiency.

Other legislation sought to improve workforce readiness in Colorado by aligning the state’s preschool-to-postsecondary education systems and by expanding and refining model content standards and assessments. A critical technical step involved linking educators

8

and former principal of a Mapleton high school noted for sending all 44 members of a senior class to four-year colleges. Johnston is a Democrat, and many members of his party joined with the CEA in opposing his legislation, while most Republicans favored it. Democratic Governor Ritter and the state’s three previous governors, Democrats Richard Lamm and Roy Romer and Republican Bill Owens, all pushed for SB-191’s approval, writing in a joint letter that “no proposal has greater promise for transforming education in Colorado.” Eight Democratic representatives voted for the bill as midnight approached on the second-to-last day of the 2010 legislative session, ensuring its passage in the House. In the Senate, 13 of 21 Democrats voted yes.

Ritter signed SB-191 on May 20, 2010. The bill codified the 15-member State Council for Educator Effectiveness, which then worked to develop detailed definitions of teacher and principal effectiveness and the Colorado State Model Evaluation System. The State Board of Education adopted the Council’s recommendations in November 2011, and the General Assembly approved the new rules during its 2012 session. Rules governing some aspects of the new law are still being developed, such as rules regarding the evaluation of school professionals other than principals and teachers.

District in 2001 became one of the first school systems in the nation to pay teachers based on improved instruction and student achievement rather than years of experience and advanced degrees. Denver Public Schools and the Denver teachers’ union agreed in 1999 on a pay-for-performance pilot that became ProComp, a performance-based teacher compensation system funded by a voter-approved mill levy. Goals set forth in the school district’s “Denver Plan” led to LEAP (Leading Effective Academic Practice), a teacher evaluation and professional development program now being piloted. The Brighton, Jefferson County, Harrison and Douglas County school districts also pioneered new systems for evaluating and providing incentives for teachers.

The Race to the Top initiative, a $4.35 billion competitive federal grant program announced by the White House in 2009, sparked a fevered push by Colorado and other states to impose state-level reforms. Among the criteria for winning millions of dollars in education grant money, states were to “design and implement rigorous, transparent, and fair evaluation systems for teachers and principals that … differentiate effectiveness using multiple rating categories that take into account data on student growth as a significant factor.” Colorado’s bid was thought to be strong, based on reforms already underway in the state. But when first-round winners were announced in

March 2010, Johnston had not yet introduced SB-191 and Colorado finished 14th out of 16 finalists, losing points for lacking enough union support for its application, failing to narrow achievement gaps and an inability to create systems that ensure “great teachers and leaders.”

Despite efforts at consensus building, the many education leaders in the state were divided on SB-191. The primary opposition to Johnston’s bill came from the state’s largest teacher’s association. The 40,000-member Colorado Education Association (CEA) launched a radio ad campaign that criticized the legislation as “too much, too fast and too costly.” The CEA argued that SB-191 circumvented work already being done by the Council on Educator Effectiveness appointed by Governor Bill Ritter, saddled cash-strapped school districts with unfunded mandates and ignored the responsibilities of students and parents. Union concerns about due process prompted the amendment requiring an appeals procedure for teachers facing a second consecutive ineffective rating. Teachers also complained that academic growth models used in the evaluation process did not capture some of the subjects they taught.

The debate over SB-191 blurred traditional party politics in the Colorado General Assembly. The bill was written and shepherded by Johnston, a freshman lawmaker at the time

9

MAIN

• Teacher effectiveness is factored in before seniority if district-level layoffs are being considered.

• Teachers and principals are evaluated annually.

• Student learning over time accounts for at least half of an evaluation.

• Professional practice Quality Standards, which define what it means to be an effective teacher or principal, account for the remainder of an evaluation.

• Non-probationary status (tenure) for teachers is earned after three consecutive years of demonstrated effectiveness rather than after a certain length of employment.

• Non-probationary status for teachers may be lost after two consecutive years of ineffective or partially effective ratings.

• Forced placement of teachers is prohibited.

• Non-probationary teachers can move to another district and retain their status.

• Teachers and principals rated ineffective must be provided with a remediation plan that includes professional development opportunities.

• Teachers may appeal a second consecutive rating of ineffectiveness to the district superintendent.

10

PROVISIONSOF SB-191

SB-191 TIMELINE

2014-15 school year• Evaluations based on the Quality Standards continue to be implemented.• Final ratings of partially effective or ineffective begin to be considered in the loss of non-probationary status for teachers (after two consecutive years of similar ratings).• CDE continues to collect feedback and adjust the state model.

2012-13 school year• The Colorado State Model Evaluation System for principals and teachers is piloted.• CDE collects data and feedback from pilot districts, shares lessons learned and makes adjustments to the system as needed.• Criteria are determined for specialized service professionals (other licensed school personnel) such as nurses, audiologists and psychologists.

2013-14 school year• On July 1, 2013, every school district is required to submit assurances to CDE that they are implementing the Colorado State Model Evaluation System or a locally developed system that meets statutory and regulatory requirements.• New evaluation requirements, based on Quality Standards, are implemented statewide. In the first year, a final rating of partially effective or ineffective does not count toward the loss of non-probationary status for teachers.• Rubrics for evaluating specialized service professionals are tested.• CDE continues to collect feedback and adjust the state model.

2011-12 school year• The Colorado State Model Evaluation System for principals is piloted.

2011

2012

2013

2014

2015

11

13

REFORMS IN OTHER STATESWhile passage of SB-191 cemented Colorado’s status as a leader in the movement to reform teacher tenure, other states also have changed their educator evaluation policies to varying degrees. From 2009 to September 2012, according to the National Council on Teacher Quality, 36 states and the District of Columbia enacted reforms.

Colorado is now one of 25 states requiring that teacher ratings include multiple categories, not just “effective” or “not effective.” It is also one of 39 states requiring annual observations of classroom instruction as part of the teacher evaluation process. Twenty-two of those states require teachers to be observed several times each year.

As in Colorado, the new wave of legislation in many states also addressed the accountability and performance of school principals. Before Race to the Top incentivized states to redesign principal evaluation systems, most states required school districts to have some process for evaluating principals.

14

NEW PRINCIPAL EVALUATION SYSTEMSAt least 34 states since 2009 have passed legislation requiring school districts to adopt new principal evaluation systems. A study of states that won Race to the Top grants found that four impose state systems on school districts, four mandate state systems with local control over some components and 10 set a minimum framework for districts to base their own models. Some examples:

DELAWARE

FLORIDA

Uses a state-created appraisal system for administrators with five components: vision and goals, culture of learning, management, professional responsibilities, and student improvement. Student growth is a critical part of principal evaluations.

Made districts responsible for developing their own principal evaluation systems, although 50 percent of an evaluation must be based on student performance, with instructional leadership and professional responsibilities making up the other half.

MINNESOTAThe evaluation of principals must meet eight criteria that include on-the-job observations, the use of longitudinal student-growth data as 35 percent of the evaluation, surveys to identify strengths and weaknesses in exercising leadership, and improvement plans for principals who do not meet standards of professional practice.

• The number of states factoring any sort of evidence of student learning into the evaluation of teachers increased from 15 to 30. In 2009, just four states considered student achievement an important criterion in rating teacher performance. By 2012, student achievement/academic growth had been made the preponderant factor in 11 states and a significant factor in nine other states.

• The number of states requiring annual teacher evaluations rose from 14 to 23, with 43 states by 2012 requiring annual evaluations for all new teachers.

• The number of states making teacher effectiveness central to the awarding of tenure went from zero to nine (including Colorado). The vast majority of states, however, still grant tenure with little consideration given to teacher performance.

NATIONAL TRENDS IN

BETWEEN 2009 & 2012

EFFECTIVENESSI M P L E M E N T A T I O N

15

QUALITY STANDARDS

1. Strategy 2. Instruction 3. Culture 4. Human Resources 5. Management 6. External Development

Professional Practice Standards

• Number and Percentage of Teachers • Other Measures Aligned with CDE Guidelines

Student GrowthMeasures

PERFORMANCE STANDARDS

Ine�ective Partially E�ective E�ective Highly E�ective

Framework for System to Evaluate Principals

7. Student Growth

STATE COUNCIL FOR

50% 50%

DEFINITION OF PRINCIPAL

EFFECTIVENESS

• Number and Percentage of Teachers • Other Measures Aligned with CDE Guidelines

WEIGHTING

WEIGHTING

How Much Does Each Standard Count Towards Overall Performance?

Scoring Framework: How Do Measuresof Quality Standards Result in a Determination of Individual Performance?

EDUCATOR EFFECTIVENESS

As defined by the State Council for Educator Effectiveness: “Effective principals in the state of Colorado are responsible for the collective success of their schools, including the learning, growth and achievement of both students and staff. As the school’s primary instructional leader, effective principals enable critical discourse and data-driven reflection about curriculum, assessment, instruction and student progress, and create structures to facilitate improvement. Effective principals are adept at creating systems that maximize the utilization of resources and human capital, foster collaboration and facilitate constructive change. By creating a common vision and articulating shared values, effective principals lead and manage their schools in a manner that supports the school’s ability to promote

equity and to continually improve its positive impact on students and families.”

Principals are held accountable for the successes or failures of the schools they lead. Under the state model, principals and assistant principals are measured against the state definition of effectiveness using a rubric that guides evaluators through each standard. The rubric is available as an Excel spreadsheet while the state works with RANDA to develop an online performance management system for the model system. Pilot districts have used the new rating system for principals since the 2011-12 school year, and CDE is working with them to identity and fix problems before the system is fully in place statewide.

CDE asked principals and principal evaluators (mostly superintendents) involved in the pilots to compare their districts’ previous evaluation systems to the state model after using it for a year. They said the state model was significantly better at guiding professional growth, serving as a basis for improved teaching and learning, providing actionable feedback to the person being evaluated and setting high standards for the person being evaluated. They were somewhat less positive – but positive nonetheless – that the state model, compared with their previous evaluation systems, would do a better job of improving student growth, providing an accurate assessment of their performance and identifying areas needing improvement.

PRINCIPAL EFFECTIVENESS IN COLORADO

17

QUALITY STANDARDS

1. Strategy 2. Instruction 3. Culture 4. Human Resources 5. Management 6. External Development

Professional Practice Standards

• Number and Percentage of Teachers • Other Measures Aligned with CDE Guidelines

Student GrowthMeasures

PERFORMANCE STANDARDS

Ine�ective Partially E�ective E�ective Highly E�ective

Framework for System to Evaluate Principals

7. Student Growth

STATE COUNCIL FOR

50% 50%

DEFINITION OF PRINCIPAL

EFFECTIVENESS

• Number and Percentage of Teachers • Other Measures Aligned with CDE Guidelines

WEIGHTING

WEIGHTING

How Much Does Each Standard Count Towards Overall Performance?

Scoring Framework: How Do Measuresof Quality Standards Result in a Determination of Individual Performance?

EDUCATOR EFFECTIVENESS

18

1. Training – Everyone involved in using the Colorado State Model Evaluation System must be trained by personnel approved by CDE.

2. Orientation – At the start of each school year, each district provides an orientation to make sure new principals/assistant principals understand the system and to review any changes made to the system since the previous year.

3. Self-Assessment – Each principal/assistant principal uses the state-provided rubric to complete a self-assessment at the beginning of the school year.

4. Review of Goals and Performance Plan – Once the self-assessment is completed, the evaluator and principal/assistant principal review the school’s annual goals and the

7. End-of-Year Review – The evaluator and principal/assistant principal discuss the performance ratings, self-assessment ratings and supporting documentation.

8. Final Ratings – If the evaluator and principal/assistant principal do not agree on the final rating, a suggested two-week period is provided to collect and summarize additional evidence needed to arrive at the correct rating.

9. Goal-Setting and Performance Planning – The principal/assistant principal develops a professional performance plan to address identified growth areas and the training/resources required to make improvements.

19

STEPS IN THE YEAR-LONG PRINCIPAL EVALUATION PROCESS (STATE MODEL)

principal’s performance plan, taking into account unique circumstances regarding the school’s culture, student body and community as well as changes in district initiatives.

5. Mid-Year Review – The evaluator and principal/assistant principal review progress made toward achieving school and personal goals. The principal is given a clear understanding of his or her potential effectiveness rating based on evidence available to date.

6. Evaluator Assessment – Evaluators review the performance of principals/assistant principals throughout the year and record their ratings on the rubric as they collect information.

The state model rubric describes each of the six professional practices standards used to determine principal effectiveness, and each standard is subdivided into elements. For example, the Strategic Leadership standard includes the element School Vision, Mission and Strategic Goals, defined as: “Principals collaboratively develop the vision, mission, values, expectations and goals of the school, collaboratively determine the processes used to establish these foundations, and facilitate their integration into the life of the school community.” Principals are rated on this and other elements on the following levels:

20

Ratings for each element are tallied, using a point system, to get overall ratings for each standard. Those ratings determine the overall rating on professional practices. There is space on the rubric for evidence used to support the ratings, evaluator comments and responses from the principal being evaluated.

Although the technical nature of the rubric might give the impression that evaluations are a scientific process relying solely on data, CDE guidelines stress the importance of human judgment in the process. What matters most in this regard, the guidelines say, is that evaluators are properly trained to exercise their judgment in ways that minimize error and bias.

BASIC

PARTIALLY PROFICIENT

PROFICIENT

ACCOMPLISHED

EXEMPLARY

Educator’s performance on professional practices is significantly below the state Quality Standard.

Educator’s performance on professional practices is below the state Quality Standard.

Educator’s performance on professional practices exceeds state Quality Standard.

Educator’s performance on professional practices significantly exceeds state Quality Standard.

Educator’s performance on professional practices meets state Quality Standard.

STEPS IN THE YEARLong Principal Evaluation Process (State Model)

For the process to work as intended, evaluators must fully understand each element of the rubric, says Margaret Crespo, executive director of secondary education for the Thompson School District in Loveland and a member of the State Council for Educator Effectiveness. Crespo is responsible for evaluating 20 principals and assistants in her district, which is an integration pilot.

Because the evaluation process is continuous – not based merely on one or two conversations – it is important for evaluators to keep the standards in mind whenever they interact with or observe principals in their purview, she says. Her job regularly requires meetings inside school

buildings, and she takes advantage of these opportunities to think about how her principals stand up to elements of the rubric. Doing so has made Crespo a more effective coach, she says, and conversations with principals have become “much more pointed, more directly based on the quality standards and specific to strategies within the rubric.”

The rubric itself makes it easier to have what Crespo calls “courageous conversations” with principals, “open and honest conversations about growth, and I don’t just mean the academic growth of kids.” The standards provide a guideline for moving a principal from ineffective to the next

category without making it seem personal.

George Welsh, superintendent of the small Center school district in the San Luis Valley, says that his district’s previous form for evaluating principals was “virtually useless. I had a principal say to me that a blank sheet of paper would have been more effective.” Under the state model system, which Center is piloting, the evaluator can clearly see what a principal must do to meet a standard.

“It’s evidence-based,” he notes. “You can’t check the box because you think (a principal) does something. You can’t check the box until you see evidence that they do it.”

21

UNDERSTANDING THE RUBRICS FOR SUCCESS

community support for a school. “A lot of principals don’t do that naturally.”

During the pilot, Eagle County has used teams of three administrators from its central office to evaluate each principal. Each administrator concentrates on two standards in his or her area of expertise. Dividing up the process makes it less overwhelming for the evaluator while still providing meaningful feedback for those being evaluated, Smyser says.

State Board rules require that principals also be evaluated on the first six Quality Standards using input from teachers at a principal’s school and the number and percentage of teachers in the school who are rated effective, highly effective, partially effective and ineffective, as well as the number and percentage of

It may not be necessary, however, to thoroughly document evidence for every element of the rubric, says Sandra Smyser, a State Council member who was superintendent of the Eagle County School District, an integration pilot.

Smyser was appointed in March as Superintendent of the Poudre School District. She worries that in the quest to meet the requirements of the new law, “people will do meaningless work putting together enormous portfolios of data representing everything” on the matrix. “If you’re looking for ineffectiveness, you can look on the rubric and find something missing. But if you’re looking for evidence of effectiveness, it doesn’t take long to find it as long as you’ve got good principals … You might be missing a few blocks, but it doesn’t matter, you’re still a good principal.”

As a pioneer in teacher pay for performance, Eagle County has had a comprehensive system for evaluating teachers for several years. But a separate method of evaluating other district employees, including principals and assistant principals, was “very un-comprehensive,” according to Smyser. She says her principals seem to appreciate the breadth of attention afforded to them under the state model process, “that it’s not just problem-based supervision.” The standards are “forcing conversations on topics that we didn’t really talk about before,” such as building

teachers who are improving their performance. Optional evidence used to measure a principal’s professional practices might include the perceptions of students, parents and other administrators, teacher retention data, budgets or school website pages.

The Colorado Legacy Foundation is developing a survey to let teachers anonymously give feedback about a principal’s performance and how it impacts student learning. Pilot districts are testing the survey, which is designed to help evaluators address portions of the state model rubric that are observable by teachers, such as whether a principal has put systems in place to keep parents informed. The survey is not intended to measure how well a principal is liked by teachers at his or her school.

22

The seventh standard that determines principal effectiveness is directly tied to data on measures of student learning, with the goal to ensure that students make at least a year’s progress in a year’s time, regardless of the level at which they begin. This factor, as previously mentioned, makes up at least half of the overall rating given to a principal or assistant principal.

Districts have some flexibility in determining the calculation of student progress but it must reflect data from the School Performance Framework, with student longitudinal growth carrying the greatest weight of any framework component. That includes results from the

Transitional Colorado Assessment Program (TCAP) in subjects such as math that generate year-to-year comparisons for individual students (from fourth through 10th grade). Districts must use at least one other measure of student academic growth, according to State Board rules, that reflects “the growth of students in all subject areas and grades, not only those in subjects and grades that are tested using Statewide Summative Assessments.” This other measure should also reflect “the broader responsibility a principal has for ensuring the overall outcomes of students in the building.”

CDE has developed an Excel spreadsheet to help districts determine whether a particular assessment is a proper measure of student growth that aligns to Colorado Academic Standards, and whether it is fair and unbiased. Working with the Colorado Content Collaboratives, a group of P-12 educators from across the state, CDE also is building a Resource Bank of “high-quality” assessments for all grades and content areas. Districts are advised to select assessments “that will have the greatest impact on student learning and are the most appropriate for measuring student learning impacted by an educator.” Chosen assessments may already be given in a district.

23

MEASURES OF STUDENT LEARNING

Acuity tests and the EXPLORE exam for middle schools; and the ACT for high schools.

For its principal evaluations, Center is using building-level growth data from the School Performance Framework and the results of nationally normed MAP (Measures of Academic Progress) tests from the Northwest Evaluation Association. The district already was administering MAP tests, which are designed to measure growth during a school year and can be linked to proficiency levels from state assessments. Other data may be factored into the equation eventually, Welsh says, but Center will “keep it simple for now.”

“We’re not in a position financially to go out shopping for more assessments, so it would be convenient to use assessments that are already validated and reliable,” says Patrick Mount, a former high school math teacher who is working on the Thompson District’s integration pilot as a strategic data fellow. To help determine which additional growth measures to use for both teacher and principal evaluations, Thompson has asked its teachers to inventory the assessments they currently use, review CDE information on other possible assessments, fill out a survey with their thoughts and recommendations, and take part in focus groups.

Picking the right assessments for evaluating teachers is trickier than picking assessments for evaluating principals because teacher evaluations must include measures of student learning attributed directly to their teaching. “Principals are responsible for everything that occurs in the building,” notes Mount, so their evaluations are based on collectively attributed growth, how all students in a school have fared over time. Additional measures of collectively attributed growth under consideration in Thompson include: the Developmental Reading Assessment (DRA2) and Acuity math and reading assessments for elementary schools; the

24

Colorado Association of School Executives4101 South Bannock StreetEnglewood, CO 80110

www.co-case.org