Leveraging Teacher Evaluation for Improving Teaching and Learning Laura Goe, Ph.D. Research Scientist, ETS Sr. Research and Technical Assistance Expert,

Leveraging Teacher Evaluation for Improving Teaching and Learning

Laura Goe, Ph.D.Research Scientist, ETSSr. Research and Technical Assistance Expert, Center for Great Teachers and Leaders

Georgia Title IIa Conference March 2013 Savannah, Georgia

Former teacher in rural & urban schools• Special education (7th & 8th grade, Tunica, MS)

• Language arts (7th grade, Memphis, TN) Graduate of UC Berkeley’s Policy, Organizations,

Measurement & Evaluation doctoral program Research Scientist in the Understanding Teaching Quality

Research Group at ETS Senior Research & Technical Assistance Expert for the

federally-funded Center for Great Teachers and Leaders

Laura Goe, Ph.D.

2

The ultimate goal of all teacher evaluation should be…

TO IMPROVE TEACHING AND

LEARNING

The goal of teacher evaluation

3

1. Using teacher evaluation results to inform professional growth opportunities

2. Characteristics of high-quality professional development

3. Assessing impact of professional growth

4. Recruitment, retention, and distribution of effective teachers

To be discussed…

4

Effective evaluation for accountability relies on:• Clearly defined and communicated standards for

performance• Quality tools for measuring and differentiating

performance• Quality training on standards and tools• Evidence to make decisions

Accountability vs. professional growth

5

Effective evaluation for professional growth relies on:• Clearly defined and communicated standards for

performance• Quality tools for measuring and differentiating

performance• Quality training on standards and tools• Evidence to make decisions

Accountability vs. Professional Growth

6

Consider whether human resources and capacity are sufficient to ensure fidelity of implementation• Poor implementation threatens validity of results

Establish a plan to evaluate measures to determine if they can effectively differentiate among teacher performance • Need to identify potential “widget effects” in measures

• If measure is not differentiating among teachers, may be faulty training or poor implementation, not the measure itself

Examine correlations among results from measures Evaluate processes and data each year and make needed

adjustments

Validity of measures key to PD aligned with evaluation results

7

Different approach; not looking at “absolute gains” Requires ability to determine and/or link student outcomes

to what likely happened instructionally Requires ability to “diagnose” instruction and

recommend/and or provide appropriate professional growth opportunities• Individual coaching/feedback on instruction

• Observing “master teachers”

• Group professional development (when several teachers have similar needs)

Interpreting results for alignment with teacher professional learning options

8

Five components in an aligned teacher evaluation-professional growth system

High quality

standards for

instruction

Multiple standards-

based measures of

teacher effectiveness

Quality training

on standards and tools

Trained individuals to

interpret results and make

professional development

recommendations

High-quality professional

growth opportunities for individuals and

groups of teachers with similar growth

plans

9

Characteristics of High Quality Professional Development (Archibald, et al, 2011)• Alignment with school goals, state and district standards and

assessments, and other professional learning activities including formative teacher evaluation

• Focus on core content and modeling of teaching strategies for the content

• Inclusion of opportunities for active learning of new teaching strategies

• Provision of opportunities for collaboration among teachers

• Inclusion of embedded follow-up and continuous feedback

High quality professional growth opportunities

10

A set of practices teachers should aspire to A teaching tool in teacher preparation programs A guiding document with which to align:

• Measurement tools and processes for teacher evaluation, such as classroom observations, surveys, portfolios/evidence binders, student outcomes, etc.

• Teacher professional growth opportunities, based on evaluation of performance on standards

A tool for coaching and mentoring teachers:• Teachers analyze and reflect on their strengths and challenges and

discuss with consulting teachers

Teaching standards

11

Increases inter-rater reliability Increases the validity of system

• Without valid results, professional development based on results will be unlikely to improve practice

Ensures mutual understanding Is a form of professional development for those

trained Training, certification, and calibration on instruments

may be more important than who the evaluator is

Quality training for teachers and leaders on teaching standards and evaluation tools

12

Teaching and Learning Academy began April ‘96 • Nationally commended program intended to

• “…provide a collegial place for teachers, teacher leaders and administrators to meet, study, and discuss application and implementation of learning…to impact student growth and development”

• Practitioners propose and develop courses• Responsive to school/district evaluation results

• Offerings must be aligned with NSDC standards

• ~300+ On-line and in-person courses, many topics

Memphis professional development system: An aligned system

13

“The plan includes specific objectives for improvement, activities and a time line for meeting these objectives, criteria for measurement of progress toward meeting the objectives, a record of participation in recommended activities, and a record of performance on specified criteria.” http://www.pelham-city.k12.ga.us/docs/GTEP%20manual.pdf

Professional Growth Plan (From GTEP manual)

14

“The increased practice of more professional learning in individual schools connected to student data and improvement plans has increased the opportunities for supporting the improved performance of students and staff. This practice has lead to more results-driven professional learning, which is standards-based, job-embedded, and collaborative.” http://www.doe.k12.ga.us/School-Improvement/Documents/FY%202011%20Professional%20Learning%20Annual%20Report%204-3-12.pdf

Georgia School Keys Professional Development

15

Tap in-district expertise to provide professional development • Could be teachers, administrators, instructional leaders, etc.

(Memphis is a good example)

Find outside supports to fill gaps in internal capacity• For example, partner with a local college

Create mechanism for connecting teachers to PD based on evaluation results (such as Memphis model)

Prepare for structural changes in the school week/calendar

Professional development planning

16

Strong, effective teachers and leaders are the key to improving student outcomes

Effective teachers may be in limited supply Ways to get effective teachers and leaders:

• Hire only highly effective teachers• Remove less effective teachers and leaders and replace

them with more effective ones• Provide guidance and support to help less effective

teachers become highly effective teachers

Effective teachers

17

Each year, close to 200,000 teachers leave the profession, with another 200,000 transferring schools.

This teacher turnover costs our public school system about $5 billion annually (Alliance for Excellent Education, 2005).

Teacher turnover

18

Any teacher can have a year where they struggle The key is to identify whether this is a “difficult”

year or a “pattern” of poor performance• Response to a “difficult” year should be mostly supportive

with targeted assistance• Response to a pattern should be more intensive with

diagnosis of problem areas, improvement plan, time limit for improvement, etc.

Teachers want to be successful!

A single “bad” year vs. a trend

19

Analyzes evaluation data to connect with PD opportunities at the teacher, school and district level• Ideally provides guidance at the time of feedback

Can be a trained administrator, evaluator, coach, mentor, etc. (for individual teachers)

Could be a committee (for determining grade, subject, school or district-wide needs)

Interpreting results and making professional development recommendations

20

“Specific criteria for measurement of progress on the PDP should be described. Measurement of progress involves keeping a record of successful completion of activities and determining whether or not the teacher's performance has improved in the targeted areas. ‘Improvement in targeted areas’ means previously marked areas have been judged to show improvement. The evaluator makes this judgment. For teachers who have received a required plan for specific needs development, progress toward completion of the PDP shall be evaluated during the next evaluation cycle if the individual is employed in the same school system. http://www.pelham-city.k12.ga.us/docs/GTEP%20manual.pdf

Criteria for Measurement of Progress (from GTEP manual)

21

Research suggests that those who leave (whether the school, the district, or the profession) tend to be less effective than those who stay in schools

The exception is first year teachers who leave—they tend to be more effective than those who stay (Hanushek and Rivkin, 2010)

Are those who leave more effective?

22

Investigated the contribution of “match between teachers and schools” to student achievement (Jackson, 2010)

Showed that teacher effectiveness is higher after a move to a different school

25% of what is typically considered to be a teacher effect may actually be a teacher-school (match) effect

Match between teachers & schools

23

“…characteristics that predict individual effectiveness may not predict organization effectiveness, even if the assumptions about individual predictors are valid” (Harris & Rutledge, 2010, pg 923)

Person-Organization fit matters more when the organization has a “distinctive culture, the career ladder is lengthy, the interaction among organization members is great, and work flexibility is high.” (ibid, pg 924)

“Fit” between person and organization

24

Evaluators found that the probability of teacher turnover fell as the magnitude of the TEEG bonus award increased, while the probability of teacher turnover increased sharply among teachers receiving no bonus award, or a relatively small award (Springer et al., 2009)• “…a $3,000 award reduced turnover for beginning teachers by

roughly 23 percentage points”

• “…awards of $3,000 reduced turnover rates to less than a quarter of what the turnover rate was prior to implementation of a TEEG performance pay plan.”

Impact of incentives on teacher turnover

25

Using longitudinal elementary school teacher and student data, we document that students have larger test score gains when their teachers experience improvements in the observable characteristics of their colleagues. Using within-school and within-teacher variation, we further show that a teacher’s students have larger achievement gains in math and reading when she has more effective colleagues (based on estimated value-added from an out-of-sample pre-period). Spillovers are strongest for less-experienced teachers and persist over time, and historical peer quality explains away about twenty percent of the own-teacher effect, results that suggest peer learning. (Jackson & Bruegmann, 2009)

Teacher peer effects

26

• Consider a cohort model of teacher recruitment• Retain cohorts through ensuring collegial interactions such as

professional learning communities are a priority

• May include a pay incentive for retention at the cohort level (i.e., percentage of cohort that stays in school)

• Partner with institutions of higher education to better prepare teachers for teaching at-risk students• Create a feedback loop with IHEs

• Field experiences in high-needs schools should be extensive and high quality

Cohort model to improve recruitment and retention of effective teachers

27

Many states have the capability to follow teachers’ movements but don’t track them

Develop a database that allows you to track teachers from school to school as well as exiting and reentering the profession

Documenting distribution patterns of effective & less effective teachers

28

With teacher effectiveness data from evaluations, districts may be better able to document the current distribution of effective teachers by the percentage of poor and minority students in the school

The challenge is to document change in the distribution of effectiveness over time and relate changes to specific policies or incentives, such as teacher professional growth

To do this, you need complete and accurate data about teachers’ participation in professional growth activities, mentoring, coaching, participation in professional learning communities, content-specific coursework, etc.

Documenting school & district changes in teacher effectiveness

29

Tracking change: An Illustration

020406080

100

School 101

School 102

School 103

School 104

School 201

School 202

School 203

District 1 District 2

Per

cen

tag

e

Schools Within Districts

Percentage of Classes Taught by Effective Teachers by Year

30

Within your current or proposed teacher evaluation system, do you have a mechanism to ensure that evaluation results are aligned with and being used to guide professional development plans?

Are teachers given the opportunity to reflect on and discuss evaluation results and develop goals for further growth with supervisors, coaches, mentors, colleagues?

Can teachers collaborate with instructional managers to develop a plan for improvement and/or professional growth?

Questions to ask about your district’s professional development plan

31

What additional supports and requirements are in place for teachers when trends in evaluation scores indicate that they are struggling?

What opportunities are available for teachers with high evaluation scores to pursue additional knowledge, growth, and leadership?

What follow-up mechanisms are in place to ensure that teachers are taking maximum advantage of professional growth opportunities?

How will change in teacher practice be documented?

Questions to ask about your district’s professional development plan (cont’d.)

32

Ensure that evaluation systems are valid for identifying effective and less effective teachers

Focus on improving effectiveness of teachers you already have with high-quality, sustained professional development

Develop strategies for supporting and retaining effective and potentially effective teachers

(YOUR GREAT IDEAS HERE)

Next steps

33

Coggshall, J. G., Rasmussen, C., Colton, A., Milton, J., & Jacques, C. (2012). Generating teaching effectiveness: The role of job-embedded professional learning in teacher evaluation. Washington, DC: National Comprehensive Center for Teacher Quality. http://www.tqsource.org/publications/GeneratingTeachingEffectiveness.pdf

Goe, L., Biggers, K., & Croft, A. (2012). Linking teacher evaluation to professional development: Focusing on improving teaching and learning. Washington, DC: National Comprehensive Center for Teacher Quality, http://www.tqsource.org/publications/LinkingTeacherEval.pdf

Imazeki, J., & Goe, L. (2009). The distribution of highly-qualified, experienced teachers: Challenges and opportunities. Washington, DC: National Comprehensive Center for Teacher Quality. http://www.tqsource.org/publications/August2009Brief.pdf

Memphis Teacher Development and Support Individualized Growth Resource Book http://tntp.org/assets/tools/MCS%20TEM-Aligned%20PD%20Resource%20Guide_TSLT%203.12.pdf

Resources/further reading

34

Alliance for Excellent Education. (2005). Teacher attrition: A costly loss to the nation and to the states. Washington, DC: Author. http://www.all4ed.org/files/archive/publications/TeacherAttrition.pdf

Archibald, S., Coggshall, J., Croft, A., & Goe, L. (2011). High-quality professional development for all teachers: Effectively allocating resources. Washington, DC: National Comprehensive Center for Teacher Quality. http://www.tqsource.org/publications/HighQualityProfessionalDevelopment.pdf

Harris, D., & Rutledge, S. A. (2010). Models and predictors of teacher effectiveness: A comparison of research about teaching and other occupations. Teachers College Record, 112(3), 914-960. http://eps.education.wisc.edu/Faculty%20papers/Harris/Manuscripts/IES%20Harris%20Rutledge%20Models%20and%20Predictors%20TCR%20Nonblind.doc

Hanushek, E. A., & Rivkin, S. G. (2010). Constrained job matching: Does teacher job search harm disadvantaged urban schools? Cambridge, MA: National Bureau of Economic Research. http://www.urban.org/uploadedpdf/1001395-constrained-job-matching.pdf

References

35

Jackson, C. K. (2010). Match quality, worker productivity, and worker mobility: Direct evidence from teachers. Cambridge, MA: National Bureau of Economic Research. http://www.ssc.wisc.edu/~scholz/Seminar/Jackson.pdf

Jackson, C. K., & Bruegmann, E. (2009). Teaching students and teaching each other: The importance of peer learning for teachers. Cambridge, MA: National Bureau of Economic Research. http://digitalcommons.ilr.cornell.edu/cgi/viewcontent.cgi?article=1079&context=workingpapers

Springer, M., Lewis, J. L., Podgursky, M. J., Ehlert, M. W., Taylor, L. L., Lopez, O. S., & Peng, A. X. (2009). Governor’s Educator Excellence Grant (GEEG) Program: Year three evaluation report. Nashville, TN: National Center on Performance Incentives. http://ritter.tea.state.tx.us/opge/progeval/TeacherIncentive/GEEG_Y3_0809.pdf

References (cont’d.)

36

37

Questions?

Laura Goe, [email protected]://twitter.com/GoeLaura

Center on Great Teachers and Leaders at AIR1000 Thomas Jefferson Street NWWashington, DC 20007-3835General Information: 877-322-8700www.gtlcenter.org

38