Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and Accountability West Virginia Department of Education
Jan 05, 2016
Educator Effectiveness:State Frameworks and
Local Practice(??)
CCSSO Annual Conference, June 2012
Juan M. D’BrotExecutive Director of Assessment and Accountability
West Virginia Department of Education
Assessment Continuum in WV
• Promoting a balanced assessment system against WV State Standards
Formative DiagnosticsFormative Diagnostics
Interim DiagnosticsInterim Diagnostics
Interim BenchmarksInterim Benchmarks
Summative AssessmentsSummative Assessments
Formative, Interim, and Summative Assessments
• Summative – WESTEST 2– Online Writing Assessment (PT)
• Interim– Benchmarks in Math, ELA, Science, Social
Studies• Formative– Acuity Item Banks in Math, ELA, Science, Social
Studies– West Virginia Writes: Practice Writing Program– Other District Purchased Supports– Teacher Developed Assessments
Major Question: Purpose? • Purpose of Summative
– Evaluative against a set of standards– Least flexible: long-cycle– Broad sense of the educational state of the state or
district• Purpose of Interim
– Instructional,– Evaluative, and/or– Predictive– Medium-cycle
• Purpose of Formative – Process to adjust teaching and learning – Instructional diagnostics– Typically classroom specific– Very short-cycle
Examining the Use of Formative and Interim
• High Use vs. Effective Use– Emerging case study of a district and school
• Context– Targeted professional development– Dedicated school leadership and staff– State-level support around the use of
formative and interim assessments
• Early Results:
Grade 9 School A and B
Grade 10 School A and B
Grade 11 School A and B
WESTEST 2 Writing: At or Above Proficiency
Examining the Use of Formative and Interim
• Strong support for use of writing short-cycle assessments
• Currently examining use of other short-cycle (formative) and medium-cycle (interim) assessments on summative assessment results
• Implications on teacher evaluation systems looking for triangulated evidence of student learning
Multiple Use Concerns
• Tension between Compliance (accountability) and Continuous Improvement (short-cycle assessments)
Adequate Yearly Progress Improvement and Growth
Teacher Evaluation Systems Student Learning
Conflicting?
Multiple Use Concerns• Tension between Compliance (accountability)
and Continuous Improvement (short-cycle assessments)– Summative Assessment
• Gaming• Cheating• “Teaching to the Test”
– Interim/Formative Assessment• Fabricated student learning gains? • Student sorting to highlight teacher “gains”? • What does the research say? Not impact/effect, but
valid link to teacher input/output?• Cognitive vs. Non-Cognitive measures? Scratching
the surface
The West Virginia Case
• A transition from nothing to something– Old: 5 years experience evaluation “only
when the principal determines it to be necessary…”
– New: Yearly evaluation for all educators, principals, and evaluators using research-based professional standards and evidence of student learning
Developing the System• Goal: To develop a supportive system
around valid data to drive improvement
• Access: A variety of data to support self-reflective practice:– Summative Growth Data– Interim Benchmarks– Formative Diagnostic
Developing the System• Context: – Politics: National vs. State Pressure– Policy: Accountability– Pragmatics: Using actionable and valid data
• Development: Stakeholder driven– Board of Education – SEA Staff– LEA staff– Teacher Unions– Legislators– Community Members
Category Input Output
5 Professional Teaching Standards 80%
14 Critical Standard Elements – 4 pt. rubric
5.7%/CSE
Student Learning Outcomes 20%
Summative Growth Data•Math & ELA Only
5%
Other Student Learning Evidence•Any Content 15%
Professionalism (Mandatory) N/A N/A
The System• System Components
The System• 20% Student Learning Outcomes– Minimum 5% Student Assessment Data• School-wide Growth – Student Growth
Percentiles
– A host of evidence– Agreed upon with principal input– Multiple measures• Acuity (Math, ELA, Science, Social Studies)• WV Writes (Writing)• Other evidence of student learning• Teacher Developed Assessments
The System• Overall Ratings
Category 100 Point Scale
Distinguished 88-100
Accomplished 54-88
Emerging 33-54
Unsatisfactory <33
Implementation
• 2011-2012 – Concluded 1st pilot year in a small number of schools – Self-ratings– Accomplished across the board…
• 2012-2013 – 2nd pilot year with schools in every district represented
• 2013-2014 – Full statewide implementation for all schools in the state
Implications for a “Young System”
• Embedded Program Evaluation – WVDE Office of Research small scale
implementation study and program evaluation
– Formative data has resulted in changes to the system and program
– Exploring third-party consulting for enhancements to guidance, system, and process (post-revision)
Lessons Learned• Politics vs. Pragmatism– Representative inclusion vs. Over/under-
representation of certain views– Political support to push a bill to law
• Capacity and Foresight– In-house development and reinventing the
wheel? (…more politics)– Building a system for the present vs.
building a system for the future– Revising the re-invention
Thank You
Juan M. D’Brot([email protected])
Executive Director
Office of Assessment and Accountability
West Virginia Department of Education