Value-Added: Then, Now & in the Future Ohio RttT Webinar Presented by Battelle for Kids June 7, 2011 Race to the Top.

Value-Added: Then, Now & in the Future

Ohio RttT WebinarPresented by Battelle for Kids

June 7, 2011

Race to the Top

THEN…in the beginningRace to the Top

ODE: UsesSAS®

EVAAS® for Value-Added

BFK: UsesSAS®

EVAAS® for Value-Added

School Improvement Collaborative (SOAR)

•Began in 2002 with 42 districts (LEAs)•Currently 100+ districts (LEAs) get expanded value-added reporting

Uses multiple test data (OAA & norm-referenced tests) from grades 2-8 for value-added reports in grades 3-8

Uses all students with three prior data points

Employs URM (univariate response model) statistical approach to determine student growth

Local Report Card Component(LRC)

Statewide accountability metric

Uses OAA test data from grades 3-8 for value-added reports in grades 4-8

Originally, only the Where Kids Count students were used in the analysis

Employs MRM (multivariate response model) statistical approach to determine student gains to a standard

Why Two Systems in Ohio?

Race to the Top

The MRM approach: Is the preferred model when there is consistent,

consecutive testing from year to year Measures growth to a predetermined standard Allows gains to be averaged over time Uses new data to make prior estimates more precise

The URM approach: Accommodates multiple kinds of tests and differing testing

regimens; therefore, it is re-normed each year Provides a comparative measure of growth based on the

pool

The System Evolved...

Race to the Top

BFK SOAR Project (100+ districts/LEAs)

• Currently use TerraNova™ to augment state

tests

2007: BFKT-CAP project

launch, 50+ SOAR districts (LEAs)

receivedteacher-level value-added

reporting

ODE:SAS®

EVAAS® reporting

BFK Ohio Value-Added High

Schools project launch, 30+

schools• Use ACT End-

of-Course Assessments

NOW…a unified system and expansion of teacher-level reports statewide

Race to the Top

One Value-Added (VA) System for Ohio: Ohio VA EVAAS ® reporting

Transparency for LEA accountability on report cards School improvement

Expanded reporting LEAs in BFK projects receive additional reports (no

duplication of grades/subjects that ODE provides) Teacher-level reporting statewide

Our Unified Approach

Race to the Top

ODE:Teacher-Level Value-

Added Reporting

• Grades 4-8 reading and math

ODE: SAS® EVAAS® Value-

AddedReporting

• Grades 4-8 reading & math

• Grades 5 & 8 science

BFK SOAR & High School:

SAS® EVAAS® Value-Added Reporting

• Grade 3, reading & math

•Grades 3,4,6,7 science

•Grades 3-8, social studies

•High school: 9 core subjects

BFKExtended

Teacher-Level Reporting

• Grade 3, reading & math

• Grades 3,4,6,7 science• Grades 3-8, social

studies• High school: 9 core

subjects

Ohio VA Reporting

Ohio’s Value-Added System

Accountability Measures and

Reports

Technical Assistance

and Support

SAS® Data Processing

Maintained by SAS® Single Limited Access Password Protected Data:

District/LEA and school Student information Analytic tools Teacher-level reports

Limited Use Public Access Includes BFK SAS® EVAAS® reporting Enhanced reporting features

Developed & supported by BFK

Regional System Trained VAL’s support

districts/LEAs through DVALT training

Support to teacher-teams

Focus on school improvement

Toolkits Online courses

Maintained by ODELRC

Valued-Added AYP Growth Measure

ODE Reports-School and District (LEA) Measures

Diagnostic Tools

EVAAS®

ODE-BFK

Partnership

Report Enhancements for 2011

Race to the Top

Inclusion of students' historical data across LEAs Previously not included in analysis due to migration across

LEAs Will increase school improvement utility for LEAs with high

mobility Tested & attributed students in ODE reports 

Previously, only Where Kids Count (WKC) students that were used for accountability purposes were available in EVAAS®

Report views of both the WKC students and all tested students with sufficient data will now be available through EVAAS®.

Report Enhancements for 2011 (cont’d)

Race to the Top

Continuation of student names for EVAAS® reports In July, your district and ITC will be able to submit student

names for the SSID crosswalk file An additional benefit of getting this done each year is that

teachers are able to view reports with re-rostered student information.

Watch for instructions next month.

Report Enhancements for 2011 (cont’d)

Race to the Top

Science reports in grades 5 & 8 Will now include school-level science results for grades 5 & 8

statewide Merged EVAAS® site – SOAR & ODE

All users will have only one EVAAS® account All LEAs will receive the state's value-added results for grades

4-8, math and reading and grades 5 & 8 science SOAR districts/LEAs will receive expanded value-added

results through the same EVAAS® site Includes high school level if involved in the Ohio Value-

Added High Schools project All teacher-level reports will be interactive and available

through the EVAAS® site

Recap of Fall 2010 Enhancements

Race to the Top

In 2010-2011: Names added to diagnostic reports to make these

reports more useful for school improvement Technical Advances

Reset the baseline to 2010 results Stabilization of the state system

Projections for 1 & 2 years and pie charts Integration into Ohio Improvement Process (OIP)

Data access for regional support

Accountability Rule Changes

Race to the Top

Increase the standard for determining “a year's growth in a year's time" from one standard error of measure to two; and

Reduce the time period for value-added impact on the report card designation from the current “2 years plus (green) to go up, and 3 years minus (red) to go down” to “1 year plus to go up, and 2 years minus to go down.”

Teacher-Level Value-Added Reporting

Race to the Top Rollout Schedule

30% of LEAs Link in Year 1 RttT (reports received fall 2011) 60% of all RttT LEAs in Year 2 100% of all LEAs in Ohio in Years 3 & 4

Requirements—Accuracy of Reporting Must conduct linkage Minimum number of students and time enrolled

Access to Reporting Online via EVAAS® accounts Password protected

Grades/Subjects Available ODE: grades 4-8, math & reading BFK: grade 3, math & reading; grades 3-8, science & social

studies; high school—algebra I & II, geometry, pre-calculus, biology, chemistry, English 9, 10 & 11

Value-Added & Teacher Evaluation

Race to the Top

Race to the Top

House Bill 153

Teacher Incentive Fund

Measuring Effective Teaching (MET) Bill & Melinda Gates Foundation Study

Value-Added & Race to the Top

Race to the Top

Teacher-Level Reporting Mini Grants for Extended Value-Added Reporting (fall 2011) Support Resources

Value-Added Toolkits—will be updated (fall 2011) Problem-based approach Key reports organized by audience Includes URM examples

Refresh of Value-Added On-line Courses (occurred June 1) Greater interactivity Condensed and practical

Face-to Face Regional Support Network support: regional Value-Added Leaders & District Value-

Added Leadership teams Focus on both teacher-level reporting & value-added uses in school

improvement Webinar Series

2011 Budget Bill Discussions

Race to the Top

Requires school districts/LEAs to establish new policies relative to teacher evaluation that include these requirements:

At least 50% of each evaluation is to be based on measures of student academic growth

Evidence-based and use multiple measures of a teacher's use of knowledge and skills and of students' academic progress

Alignment with the standards for teachers Statements of expectations for professional performance Observation of the teacher on at least two occasions for not

less than thirty minutes on each occasion Each teacher be provided with a written report of the results of

the teacher's evaluation and recommendations

Race to the Top

When applicable to a teacher, those measures shall include student performance on the assessments and the value-added progress metric

For teachers of grade levels and subjects for which those measures are not applicable, student assessments that measure mastery of the course content for the appropriate grade level, which may include nationally normed standardized assessments, industry certification examinations, end-of-course examinations developed or selected by the board, or assessments may be used

Shall include student growth data for students assigned to the teacher during the three most recent school years

If less than three years of growth data are available, the board shall use the growth data for all of the school years that are available and the student academic growth portion of the teacher evaluation may be reduced to 40%

2011 Budget Bill Discussions (cont’d)

Teacher Incentive Fund (TIF)

Race to the Top

Value-added use in 24 Ohio TIF districts/LEAs as one part of multiple measures within award designs (in grades/subjects available)

Participating districts/LEAs will: Define what it means to be an effective educator Utilize quality, reliable data to inform instructional practices Enhance the evaluation systems for principals and teachers to

improve student achievement Design award programs that recognize educators for their

contributions to student achievement and leadership in the district/LEA

Make data-driven decisions about professional development opportunities

Multiple Measures

Race to the Top

MET resources (www.metproject.org) White paper: Working with Teachers to Develop Fair and Reliable

Measures of Effective Teaching Teachers Talk About Multiple Measures Video MET Project Preliminary Findings Research Report

Home Runs, RBIs and Batting Averages: How Today's Educators Measure Uphttp://www.battelleforkids.org/About_Us/Publications

In conclusion, where is it all going?

Race to the Top

Expanded use of value-added analysis for: School Improvement Program Effectiveness Teacher Reflection Educator Evaluation

Additional Webinars

Race to the Top

June 7 @ 3 p.m.Value-Added: Then, Now and in the Future

June 9 @ 3 p.m.Link Before You Leap

June 21 @ 2:30 p.m.Building Capacity Using Value-Added in School Improvement

June 23 @ 2:30 p.m.Implications of Teacher-Level Value-Added Data

Additional Resources

Race to the Top

Contacts:

Ohio Department of Education Matt Cohen, [email protected]

Battelle for Kids

Mary Peters, [email protected]

Help Desk, [email protected] or (866) 543-7555