September 2015 Queens North BFSC Rajeev Bector, Advance Policy Field Manager Dr. Amy Reynolds, Advance Teacher Development Specialist Anna Bulatewicz,

September 2015

Institute

Queens North BFSC

Rajeev Bector, Advance Policy Field Manager Dr. Amy Reynolds, Advance Teacher Development Specialist

Anna Bulatewicz, Advance Teacher Development Specialist

Agenda:

Advance Overall Ratings Report with MOSL Detail Data

• Principal-Based Procedural Appeals

• Data Correction Request

• MOSL Detail Data Report

MOSL Selections Process and Periodic Assessments

Advance District Contacts

Objectives

3

After today’s session participants will be able to:

• Understand teachers’ Advance Overall Ratings report and use MOSL Detail Data Analysis Protocol to inform and improve instruction

• Reflect and consider assessment selections for MoSL and Periodic Assessments as part of a strategic assessment plan.

• Understand the support structure of the new Advance roles at the Superintendent and BFSC offices.

Session Approach and Norms

•Inquiry/Insight Model –Focus on making connections and asking questions. When questions arise during the presentation, please jot them on an index card, along with your email address.

•High Engagement, Low Tech –Respect the learning space by using technology to engage with the work.

•Contribute to the Learning –Learn from our collective experiences and deepen what we may already know.

•Maximize Our Time Together –Use this day to collaborate, troubleshoot, and brainstorm with colleagues.

4

Advance Overall Rating Report withMOSL Detail Data

5

Queens North BFSC

6

*Beginning 9/1, principals can access Overall Rating Reports in the Advance Web Application to print, signand place in the teachers’ files.

Advance Overall Rating (Delivery, DCR, PPA) Timeline

As of September 1st Teachers Should Have Received Their 2014-15 Advance Overall Rating Report Via Email

7

• This year the Advance Overall Rating Report contains information about the students who are included in the teacher's State and Local Measures of Student learning.

• The email also includes a HEDI point value and HEDI rating for each subcomponent of the teacher’s Overall Rating.

Overall Rating

0 - 100 points: 78 

EffectiveHEDI Rating

Local Measures

0 -20 points: 9 

IneffectiveHEDI Rating

State Measures

0 -20 points: 14 

DevelopingHEDI Rating

Measures of Teacher Practice

0 -60 points: 55 

Highly EffectiveHEDI Rating

Teachers Can Now Access and Download Their Overall Rating Report From the Advance Web Application

8

9

Teachers Can Use The Advance Overall Ratings Guide to Better Understand How Their Advance Ratings Were Calculated

Explains when and how procedural

appeals were applied to Overall

Ratings

Explains how MOTP and MOSL

points and subcomponent

ratings were calculated

Explains how MOTP and MOSL

points were combined to create an Overall Rating

Contains information about the Data Corrections

Request (DCR) process

Teachers’ Overall RatingsProcedural Appeals

10

Some Teachers’ Overall Ratings Will be Recalculated According to Procedural Appeals Policies

11

Teachers that meet ALL of the requirements listed below may have their Overall Rating recalculated according to Procedural Appeals policies:

Measures of Teacher Practice (MOTP) Rating is Highly Effective or Effective

Local and State Measures of Student Learning (MOSL) Ratings are Ineffective

50% or more of State and/or Local Measures is based on Group and/or Linked Measures

An asterisk next to a teacher’s HEDI Rating indicates that a procedural appeal has already been applied to their Overall Rating.

Overall Rating

0 - 100 points: 78 

Effective*HEDI Rating

There are Four Types of Procedural Appeals that may Have Been Applied to an Overall Rating

12

If their Measures of Teacher Practice (MOTP) rating is Highly Effective and BOTH State and Local Measures are

based on group and/or linked measures, then their Overall Rating was changed to

Effective

Teachers who receive an Ineffective rating for both State and Local Measures that are based on group and/or linked measures* may have been eligible for a procedural appeal in the following situations:

Requires principal

input

If their Measures of Teacher Practice (MOTP) rating is Highly Effective and EITHER BUT NOT BOTH State and Local Measures are based on group and/or linked measures, then their

Overall Rating was changed to Developing

If their Measures of Teacher Practice (MOTP) rating is Effective and BOTH

State and Local Measures are based on group and/or linked measures, then their

Overall Rating was changed to Developing

If their Measures of Teacher Practice (MOTP) rating is Effective and EITHER

BUT NOT BOTH State and Local Measures are based on group and/or

linked measures, then the principal can choose to maintain or increase the Overall Rating. The Rating will be

changed to Developing if the principal chooses not to respond.

*50% or more of the State and/or Local Measure is based on group and/or linked measures.

TYPE 1

TYPE 2

TYPE 3

TYPE 4

Teachers’ Overall RatingsData Corrections Requests

13

Advance Data Corrections Requests (DCRs)

14

• Teacher-student linkages were determined through the roster maintenance and verification process, which should have been completed by all teachers and principals in the spring of 2015.

• Beginning September 1, teachers and principals will have access to teachers’ Overall Rating Reports. This report will include information about the students who were included in the teacher’s State and Local Measures of Student Learning (MOSL).

• Teachers are encouraged to submit Data Corrections Request (DCR) to their principals September 25. If the principal approves the request, then the principal will submit the request via an online survey October 2.

• The NYCDOE will analyze requests, and if actionable, recalculate Overall Ratings based on corrected student-level data. Teachers will receive an updated Advance Overall Rating via email by mid-October, only if the rating changed.

Purpose of DCR: What It Is and Isn’t

DCR-Eligible Requests:

• If teachers find that inaccurate student roster data was used to calculate their MOSL ratings, they may ask their principal to submit a “Data Corrections Request” (DCR) to have their ratings corrected using updated student rosters.

Is the request being made related to student roster or enrollment data?

NOT DCR-Eligible:

• DCR process is not currently intended to address non-roster/enrollment issues.

15

Teachers’ Overall Ratings Using MOSL Detail Data

16

MOSL Detail Data Can be Used to Reflect on Teaching and Learning

17

This information can be used to:

Reflect on instruction and how it influences student learning

Plan to teach the standards and content more effectively

Plan to strengthen practice by leveraging supports and resources

MOSL Detail Data provide information about:

How a teacher influenced students’ growth overall

How a teacher influenced the growth of individual students

How a teacher influenced the growth of groups of students

The Report will Include Summary Reports for Measures of Teacher Practice as well as State and Local Measures of Student Learning

18

MOSL Detail Data Analysis Protocol

For each Measure of Student Learning:

19

STEP 1• Analyze and reflect on overall

student growth

STEP 2• Analyze and reflect on individual

student growth

STEP 3• Analyze and reflect on growth of

student subgroups

STEP 4• Determine implications for instruction

and next steps

School Leaders Can Support Teachers’ Ongoing Professional Learning by Connecting them with Targeted Opportunities or Designing and Delivering their Own

20

Resources and Supports for Fall 2015

Advance Intranet

• Vision for School Improvement• Collaborative Learning Through Peer Inter-visitation

WeTeachNYC

• Professional Learning Workshops and Coaching

The Handbook for Professional Learning

• Implementing Differentiated Professional Learning

The Office of Periodic Assessment

• Assessment Literacy Series (offered throughout school year)

MOSL Selections Process and Periodic Assessments

21

Queens North BFSC

22

In SY 2015-16, Teachers’ Overall Advance Ratings will Continue to Include a State and Local Measure of Student Learning

(SY 2015-16),

23

24

In small schools with fewer than 30 classrooms, the Principal and UFT Chapter Chaircan jointly agree to form a Committee of only 6 members (3 chosen by each).

25

UPDATED!

Inherited

SY 2015-16

26

• Assessments selected for MOSL with an individual target population must align to the standards of the course in which the assessment will be administered.

• Due to State regulations, some assessments are only available as a Local Measures selection.

• Goal-setting is the required growth measurement for some assessments, due to small sample sizes or a lack of applicable historical achievement data.

Assessment Options and Processes

27

• The individual target population can only be assigned to an assessment if the assessment will be administered in the grade/subject for which it was selected.In some cases, the individual target population is required for the State Measures.

• The grade, school and linked target populations can only be selected for an assessment if that assessment is also selected with an individual target population for another grade/subject.

• Principals and School Local Measures Committee may either choose to assign a linked target population to a single assessment or a bundle of assessments.

Target Population Options and Processes

28

• In order to calculate a rating for teachers with an assessment using Growth model, a minimum of 16 student scores are needed for NYSED growth model (4-8 Math/ELA state tests) across all grade/subjects and a minimum of 6 student scores are needed for NYCDOE growth model within one grade/subject.

• If there are insufficient student scores, then goal-setting must be chosen as the growth measurement.

• For the following assessments, schools must use goal-setting as the growth measurement:

• Language other than English (LOTE) Exams

• Second Language Proficiency (SLP) Exams

• Career and Technical Education (CTE) Exams (both NOCTI & Certiport)

• Arts Commencement Exams

• NEW: Alternate Assessments

Growth Measurement Options and Processes

SCHOOL-LEVEL MOSL SELECTIONS: The Guide Contains Step-By-Step Guidance to Lead Schools Through the Decision-Making Process

29

STEP 1

  REVIEW ASSESSMENT OPTIONSSTEP 2

  REVIEW TARGET POPULATION OPTIONSSTEP 3

REVIEW GROWTH MEASUREMENT OPTIONSSTEP 4

MAKE LOCAL MEASURES SELECTIONS BY GRADE/SUBJECTSTEP 5

PRESENT RECOMMENDED LOCAL MEASURE SELECTIONS TO PRINCIPAL

STEP 6

  FINALIZE STATE AND LOCAL MEASURESSTEP 7

  PREPARE FOR DECISION-MAKINGSTEP 1

ENTER FINAL MOSL SELECTIONS FOR EACH GRADE/SUBJECT IN THE WEB APPLICATIONSTEP 8

The MOSL Selections Guide and Supplements Will Be The Primary Tools Used to Support Schools to Make Selections

30

Provides an eight-step framework for decision-making Explains notable changes to the options and process for

SY 2015-16 Includes considerations at each decision point Provides guiding questions to facilitate decision-making

Outlines the decisions to be made for each school type (Elementary K-5; Middle School 6-8; and High School 9-12)

Identifies where decisions are pre-determined by the State

Provides a list of assessments available for each grade/subject

Provides a space to record selection decisions for each grade/subject

1

2

School-Level MOSL Selections Review

31

For each grade/subject offered at a school, the school must select a State

Measure and a Local Measure.

For each state measure, the principal selects a(n):

Assessment

Target population

Growth Measurement

For each Local Measure the School Local Measures

Committee recommends a(n).

Assessment

Target population

Growth Measurement

STATE MEASURE LOCAL MEASURE*

*Note: SLMC may recommend multiple Local Measures for each grade/subject.

h

STATE ASSESSMENTS

Assessments developed by NYSED to measure student learning (e.g., Regents Exams, State ELA, Math or Science tests, etc.).

f

NYC PERFORMANCE ASSESSMENTS

Performance assessments that require students to construct a response, create a product, or perform a task to demonstrate knowledge or skills (e.g., NYC Performance Tasks, Running Records, etc.).

3RD PARTY ASSESSMENTS

Other approved assessments purchased from an assessment vendor (e.g., Performance Series, Advanced Placement exams, etc.).

When Making MOSL Selections, Schools Consider Three Different Types of Assessments

32

NYC

When Making MOSL Selections, Schools Consider Four Different Target Population Options

33

h

INDIVIDUAL

Includes all the teacher’s students who take the assessment in the teacher’s course(e.g., a 9th grade ELA teacher is evaluated based on the growth of his/her students on the 9th grade NYC Performance Task - ELA).

f

GRADEIncludes all the students in a given grade who take the assessment (e.g., a 7th grade Science teacher is evaluated based on the growth of all 7th grade students on the 7th grade State Math Assessment).

SCHOOLIncludes all students in the school who take the assessment(e.g., a K-5 Music teacher is evaluated based on the growth of all students in the school on all State Math Assessments).

LINKED

Includes only the teacher’s students who take an assessment administered in another teacher’s course(e.g., a HS Theater teacher is evaluated based on the growth of only his/her students on the 9th and 10th grade NYC Performance Tasks in ELA).

34

Measures Of Student Learning (MOSL):Baseline Assessment Administration

35

By March:Schools revise teacher-level MOSL

selections if roster changes necessitate. Ongoing:

Schools use ongoing assessments to track student progress and inform

instruction

2015-2016 MOSL Implementation Stages: Baseline Assessment Administration

36

By Sept. 28:Schools make meaningful

school-level MOSL selections that are aligned

with student needs, instructional priorities, and

strategic assessment plans.

By Late May: Teachers in Grades 4-8 ELA and Math complete the

RMV Process. By June 10

Schools administer all end-of-year MOSL-eligible assessments. Schools must score and submit all

assessment results by June 24.

By end of June: Schools use end-of-year measures of student learning

data to reflect on practice and plan for SY 2016-17..

By Nov 6: Schools make teacher-level MOSL selections

By Oct. 30:

(If baseline administration is selected). Schools administer all baseline assessments to determine

student starting points. By Nov. 13

Schools must score and submit all baseline assessment results for MOSL purposes.

By Nov. 16: Schools set goals for students or groups of students (if

assessments require goal-setting)..

STAGE 1: Preparing for SY

2015-16

STAGE 2: Starting smart

STAGE 3: Tracking progress

STAGE 4: Finishing strong

Which Assessments Include Matching MOSL-eligible Baseline Assessments as an Option?

37

• SANDI/FAST

• WebABLLS

• Performance Series

• ELA & Math

• NYC Performance Tasks • All subjects

• Running Records• F&P

• DRA2

• TCRWP

Most Baseline Assessments are Optional for MOSL Purposes*

38

However, schools may elect to administer MOSL-eligible baseline assessments even if they choose not use the scores for Measures of Student Learning (MOSL) purposes.

What are some reasons why schools may choose to administer baselines for MOSL and non-MOSL purposes?

*Baselines are required for:• SANDI/Fast• Kindergarten grade/subjects when an individual target population and

growth model are selected

39

2014 -2015

2015-2016

• The administration of fall baseline assessments is required in Kindergarten if an individual target population AND growth model are selected,

and strongly encouraged for Grades 1 and 2. • Schools must select Goal-Setting for the

Alternate Assessment grade/subjects.

The administration of fall baseline assessments was optional for all

grade/subjects.

TURN AND TALK

• What questions do you still have about assessments?

• While not required, what is the value of administering baseline assessments in Grade 1 & 2?

New Policy: Baseline Assessment Administration is Required in Some Cases

Baseline Assessment Updates are available on the Advance Intranet

Timeline for Implementing MOSL & Assessment Administration

40

Aug Sep Oct Nov Dec Jan Feb March April May June

Grade/subject MOSL

selectionsby September

28

Baseline assessment

administration &

submissionSeptember 9 – November 13

Teacher-level MOSL

selectionsNovember 6

Goal-Setting*by November

15

End-of-year assessment

administration May 2 – June 10

(most assessments)**

Use assessment results to

inform instruction

(ongoing)

*Only for schools that selected goal-setting as a growth measurement.**see MOSL Assessment Administration Handbook for more information.

41

MOSL Selections: Roles and Responsibilities

PRINCIPAL

Select State Measures for grade/subjects that are not pre-determined by the state (assessment, target population, and/or growth measurement).

Accept ALL of the School Local Measures Committee recommendations or NONE of the recommendations and opt for Local Measures “Default”

For State Measures, determine if your school should administer some, all, or no Fall baseline assessments for MOSL purposes. Also determine if your school should administer some, all, or no Fall baseline assessments for instructional purposes.

Work with the School Local Measures Committee to communicate State and Local Measures selections to teachers by October 13th.

42

MOSL Selections: Roles and Responsibilities cont.

SCHOOL LOCAL MEASURES COMMITTEE

Recommend Local Measures for all relevant grade/subjects (assessment, target population, growth measurement)

For Local Measures, determine if your school should administer some, all, or no fall baseline assessments for MOSL purposes

Recommend a preferred subgroup option (if necessary). Optional: Customize subgroup selection for some or all grade/subjects

If goal-setting is selected, recommend that goals for grade or school-level goal-setting be set by the Committee or by the principal

Work with the principal to communicate State and Local Measures Selections to teachers

Periodic Assessments

43

Queens North BFSC

Overview of Periodic Assessment

Teach

AssessReflect

Analyze the results and reflect on

practice to decide what action is needed next to

increase student learning

Assess student mastery of the

skills and knowledge

identified in the standards

Teach the skills and knowledge identified in the

standards

The Common Core Learning Standards (CCLS) set a high bar for students and require teachers to provide a more rigorous academic experience for students. As we increase rigor in the classroom, we need a way to identify specific areas of strength and need to in order to provide targeted instruction for all students. Understanding how our students are learning and growing on a regular basis is fundamental to achieving our goal of preparing all New York City students to graduate ready for college and careers.

44

45

MOSL and Periodic Assessment Selections Should Be Viewed As One Comprehensive Decision Making Process

46

A Strategic Assessment Plan Can Be Used to Provide Feedback and Data to Support CCLS Implementation

Periodic Assessments & MOSL-eligible Assessments

The Periodic Assessment Portfolio contains most of theassessments that have previously been considered eligible for MoSL purposes:

• NYC Performance Tasks (K-12/ELA, Math, Science, Social Studies)

• Running Records of Literacy (DRA2, F&P, TCRWP)

• Performance Series (3-12)

• Sandi/Fast and WebABLLS

• CTE: Certiport / NOCTI

• Arts Commencement

47

Periodic Assessment Selection Tool

Periodic Assessment Selections Reopen:

The PA Selection Tool will remain open through September 30 in order to allow sufficient time to adjust your PA selections and ensure their alignment with your school’s upcoming Measures of Student Learning decisions for Advance.

If your school made a selection by September 11, PA will send student-facing materials and books.

48

Periodic Assessment Guide and Worksheets

PA Guide contains:

Program summary

Detailed descriptions of assessment choices

Step-by-Step guides to making selections

Selections worksheets

http://intranet.nycboe.net/Accountability/Assessments/PeriodicAssessments/

49

50

Accessing the Advance Web Application (AWA)

NOTE: Principal or Principal MoSL Designee(via Manage User Roles & Access) canenter School-Level MoSL Selectionsin the AWA.

51

NEW FEATURE!Copy last year’s School-Level MoSL Selections

Data Systems and Deadlines

Assessment Data System Deadline Fix Errors by

NYC Performance Tasks Schoolnet

October 30

November 13

Running Records: F&P and DRA2

STARS Classroom

Running Records: TCRWP AssessmentPro

Performance Series Performance Series Website

SANDI/FAST SANDI/FAST Online Portal November 6

WebABLLS WebABLLS

52

For assessments scored at school-sites, schools can access results as soon as scores are submitted

53

ASSESSMENT DATA SYSTEM RELEVANT REPORTS

NYC Performance Tasks Schoolnet Assessment Details, Item Analysis, Standards

Mastery

DRA2 and F&P STARS Classroom All Formal Assessment Elements, Critical Elements of Reading, Student Strengths/Weaknesses

TCRWP AssessmentPro Export School Data

mCLASS Reading 3D

mCLASS: Home Portal Class Summary Report, Student Summary Report

SANDI/FAST SANDI/FAST Online Portal FAST Class View, FAST Results, Standards Report

WebABLLS WebABLLS Total Grid View, Category View, Progress and Benchmark Reports

SLP/LOTE ATS Status Report, Item Report, Item Distribution Report

Advance Support Roles

54

Queens North BFSC

55

Teacher Development and Evaluation Coaches (TDECs) by District: your first point of contact.

TDEC DISTRICT

COMMUNITY DISTRICTS: Teresa Caccavale 24 Debbie White-Winkler 25 Brian Contarsy 26 Erica Urena-Thus 30

HIGH SCHOOLS: Lisa Grevenberg Districts 24, 25, 26 Tracy Atkins Renewal HS Angelica Beissel New Visions

56 56

Thank you!

Please take a moment to complete the feedback form before leaving

For a copy of this presentation please email your TDEC