September 2015 Institu te Queens North BFSC Rajeev Bector, Advance Policy Field Manager Dr. Amy Reynolds, Advance Teacher Development Specialist Anna Bulatewicz, Advance Teacher Development Specialist
Jan 18, 2016
September 2015
Institute
Queens North BFSC
Rajeev Bector, Advance Policy Field Manager Dr. Amy Reynolds, Advance Teacher Development Specialist
Anna Bulatewicz, Advance Teacher Development Specialist
Agenda:
Advance Overall Ratings Report with MOSL Detail Data
• Principal-Based Procedural Appeals
• Data Correction Request
• MOSL Detail Data Report
MOSL Selections Process and Periodic Assessments
Advance District Contacts
Objectives
3
After today’s session participants will be able to:
• Understand teachers’ Advance Overall Ratings report and use MOSL Detail Data Analysis Protocol to inform and improve instruction
• Reflect and consider assessment selections for MoSL and Periodic Assessments as part of a strategic assessment plan.
• Understand the support structure of the new Advance roles at the Superintendent and BFSC offices.
Session Approach and Norms
•Inquiry/Insight Model –Focus on making connections and asking questions. When questions arise during the presentation, please jot them on an index card, along with your email address.
•High Engagement, Low Tech –Respect the learning space by using technology to engage with the work.
•Contribute to the Learning –Learn from our collective experiences and deepen what we may already know.
•Maximize Our Time Together –Use this day to collaborate, troubleshoot, and brainstorm with colleagues.
4
Advance Overall Rating Report withMOSL Detail Data
5
Queens North BFSC
6
*Beginning 9/1, principals can access Overall Rating Reports in the Advance Web Application to print, signand place in the teachers’ files.
Advance Overall Rating (Delivery, DCR, PPA) Timeline
As of September 1st Teachers Should Have Received Their 2014-15 Advance Overall Rating Report Via Email
7
• This year the Advance Overall Rating Report contains information about the students who are included in the teacher's State and Local Measures of Student learning.
• The email also includes a HEDI point value and HEDI rating for each subcomponent of the teacher’s Overall Rating.
Overall Rating
0 - 100 points: 78
EffectiveHEDI Rating
Local Measures
0 -20 points: 9
IneffectiveHEDI Rating
State Measures
0 -20 points: 14
DevelopingHEDI Rating
Measures of Teacher Practice
0 -60 points: 55
Highly EffectiveHEDI Rating
Teachers Can Now Access and Download Their Overall Rating Report From the Advance Web Application
8
9
Teachers Can Use The Advance Overall Ratings Guide to Better Understand How Their Advance Ratings Were Calculated
Explains when and how procedural
appeals were applied to Overall
Ratings
Explains how MOTP and MOSL
points and subcomponent
ratings were calculated
Explains how MOTP and MOSL
points were combined to create an Overall Rating
Contains information about the Data Corrections
Request (DCR) process
Teachers’ Overall RatingsProcedural Appeals
10
Some Teachers’ Overall Ratings Will be Recalculated According to Procedural Appeals Policies
11
Teachers that meet ALL of the requirements listed below may have their Overall Rating recalculated according to Procedural Appeals policies:
Measures of Teacher Practice (MOTP) Rating is Highly Effective or Effective
Local and State Measures of Student Learning (MOSL) Ratings are Ineffective
50% or more of State and/or Local Measures is based on Group and/or Linked Measures
An asterisk next to a teacher’s HEDI Rating indicates that a procedural appeal has already been applied to their Overall Rating.
Overall Rating
0 - 100 points: 78
Effective*HEDI Rating
There are Four Types of Procedural Appeals that may Have Been Applied to an Overall Rating
12
If their Measures of Teacher Practice (MOTP) rating is Highly Effective and BOTH State and Local Measures are
based on group and/or linked measures, then their Overall Rating was changed to
Effective
Teachers who receive an Ineffective rating for both State and Local Measures that are based on group and/or linked measures* may have been eligible for a procedural appeal in the following situations:
Requires principal
input
If their Measures of Teacher Practice (MOTP) rating is Highly Effective and EITHER BUT NOT BOTH State and Local Measures are based on group and/or linked measures, then their
Overall Rating was changed to Developing
If their Measures of Teacher Practice (MOTP) rating is Effective and BOTH
State and Local Measures are based on group and/or linked measures, then their
Overall Rating was changed to Developing
If their Measures of Teacher Practice (MOTP) rating is Effective and EITHER
BUT NOT BOTH State and Local Measures are based on group and/or
linked measures, then the principal can choose to maintain or increase the Overall Rating. The Rating will be
changed to Developing if the principal chooses not to respond.
*50% or more of the State and/or Local Measure is based on group and/or linked measures.
TYPE 1
TYPE 2
TYPE 3
TYPE 4
Teachers’ Overall RatingsData Corrections Requests
13
Advance Data Corrections Requests (DCRs)
14
• Teacher-student linkages were determined through the roster maintenance and verification process, which should have been completed by all teachers and principals in the spring of 2015.
• Beginning September 1, teachers and principals will have access to teachers’ Overall Rating Reports. This report will include information about the students who were included in the teacher’s State and Local Measures of Student Learning (MOSL).
• Teachers are encouraged to submit Data Corrections Request (DCR) to their principals September 25. If the principal approves the request, then the principal will submit the request via an online survey October 2.
• The NYCDOE will analyze requests, and if actionable, recalculate Overall Ratings based on corrected student-level data. Teachers will receive an updated Advance Overall Rating via email by mid-October, only if the rating changed.
Purpose of DCR: What It Is and Isn’t
DCR-Eligible Requests:
• If teachers find that inaccurate student roster data was used to calculate their MOSL ratings, they may ask their principal to submit a “Data Corrections Request” (DCR) to have their ratings corrected using updated student rosters.
Is the request being made related to student roster or enrollment data?
NOT DCR-Eligible:
• DCR process is not currently intended to address non-roster/enrollment issues.
15
Teachers’ Overall Ratings Using MOSL Detail Data
16
MOSL Detail Data Can be Used to Reflect on Teaching and Learning
17
This information can be used to:
Reflect on instruction and how it influences student learning
Plan to teach the standards and content more effectively
Plan to strengthen practice by leveraging supports and resources
MOSL Detail Data provide information about:
How a teacher influenced students’ growth overall
How a teacher influenced the growth of individual students
How a teacher influenced the growth of groups of students
The Report will Include Summary Reports for Measures of Teacher Practice as well as State and Local Measures of Student Learning
18
MOSL Detail Data Analysis Protocol
For each Measure of Student Learning:
19
STEP 1• Analyze and reflect on overall
student growth
STEP 2• Analyze and reflect on individual
student growth
STEP 3• Analyze and reflect on growth of
student subgroups
STEP 4• Determine implications for instruction
and next steps
School Leaders Can Support Teachers’ Ongoing Professional Learning by Connecting them with Targeted Opportunities or Designing and Delivering their Own
20
Resources and Supports for Fall 2015
Advance Intranet
• Vision for School Improvement• Collaborative Learning Through Peer Inter-visitation
WeTeachNYC
• Professional Learning Workshops and Coaching
The Handbook for Professional Learning
• Implementing Differentiated Professional Learning
The Office of Periodic Assessment
• Assessment Literacy Series (offered throughout school year)
MOSL Selections Process and Periodic Assessments
21
Queens North BFSC
22
In SY 2015-16, Teachers’ Overall Advance Ratings will Continue to Include a State and Local Measure of Student Learning
(SY 2015-16),
23
24
In small schools with fewer than 30 classrooms, the Principal and UFT Chapter Chaircan jointly agree to form a Committee of only 6 members (3 chosen by each).
25
UPDATED!
Inherited
SY 2015-16
26
• Assessments selected for MOSL with an individual target population must align to the standards of the course in which the assessment will be administered.
• Due to State regulations, some assessments are only available as a Local Measures selection.
• Goal-setting is the required growth measurement for some assessments, due to small sample sizes or a lack of applicable historical achievement data.
Assessment Options and Processes
27
• The individual target population can only be assigned to an assessment if the assessment will be administered in the grade/subject for which it was selected.In some cases, the individual target population is required for the State Measures.
• The grade, school and linked target populations can only be selected for an assessment if that assessment is also selected with an individual target population for another grade/subject.
• Principals and School Local Measures Committee may either choose to assign a linked target population to a single assessment or a bundle of assessments.
Target Population Options and Processes
28
• In order to calculate a rating for teachers with an assessment using Growth model, a minimum of 16 student scores are needed for NYSED growth model (4-8 Math/ELA state tests) across all grade/subjects and a minimum of 6 student scores are needed for NYCDOE growth model within one grade/subject.
• If there are insufficient student scores, then goal-setting must be chosen as the growth measurement.
• For the following assessments, schools must use goal-setting as the growth measurement:
• Language other than English (LOTE) Exams
• Second Language Proficiency (SLP) Exams
• Career and Technical Education (CTE) Exams (both NOCTI & Certiport)
• Arts Commencement Exams
• NEW: Alternate Assessments
Growth Measurement Options and Processes
SCHOOL-LEVEL MOSL SELECTIONS: The Guide Contains Step-By-Step Guidance to Lead Schools Through the Decision-Making Process
29
STEP 1
REVIEW ASSESSMENT OPTIONSSTEP 2
REVIEW TARGET POPULATION OPTIONSSTEP 3
REVIEW GROWTH MEASUREMENT OPTIONSSTEP 4
MAKE LOCAL MEASURES SELECTIONS BY GRADE/SUBJECTSTEP 5
PRESENT RECOMMENDED LOCAL MEASURE SELECTIONS TO PRINCIPAL
STEP 6
FINALIZE STATE AND LOCAL MEASURESSTEP 7
PREPARE FOR DECISION-MAKINGSTEP 1
ENTER FINAL MOSL SELECTIONS FOR EACH GRADE/SUBJECT IN THE WEB APPLICATIONSTEP 8
The MOSL Selections Guide and Supplements Will Be The Primary Tools Used to Support Schools to Make Selections
30
Provides an eight-step framework for decision-making Explains notable changes to the options and process for
SY 2015-16 Includes considerations at each decision point Provides guiding questions to facilitate decision-making
Outlines the decisions to be made for each school type (Elementary K-5; Middle School 6-8; and High School 9-12)
Identifies where decisions are pre-determined by the State
Provides a list of assessments available for each grade/subject
Provides a space to record selection decisions for each grade/subject
1
2
School-Level MOSL Selections Review
31
For each grade/subject offered at a school, the school must select a State
Measure and a Local Measure.
For each state measure, the principal selects a(n):
Assessment
Target population
Growth Measurement
For each Local Measure the School Local Measures
Committee recommends a(n).
Assessment
Target population
Growth Measurement
STATE MEASURE LOCAL MEASURE*
*Note: SLMC may recommend multiple Local Measures for each grade/subject.
h
STATE ASSESSMENTS
Assessments developed by NYSED to measure student learning (e.g., Regents Exams, State ELA, Math or Science tests, etc.).
f
NYC PERFORMANCE ASSESSMENTS
Performance assessments that require students to construct a response, create a product, or perform a task to demonstrate knowledge or skills (e.g., NYC Performance Tasks, Running Records, etc.).
3RD PARTY ASSESSMENTS
Other approved assessments purchased from an assessment vendor (e.g., Performance Series, Advanced Placement exams, etc.).
When Making MOSL Selections, Schools Consider Three Different Types of Assessments
32
NYC
When Making MOSL Selections, Schools Consider Four Different Target Population Options
33
h
INDIVIDUAL
Includes all the teacher’s students who take the assessment in the teacher’s course(e.g., a 9th grade ELA teacher is evaluated based on the growth of his/her students on the 9th grade NYC Performance Task - ELA).
f
GRADEIncludes all the students in a given grade who take the assessment (e.g., a 7th grade Science teacher is evaluated based on the growth of all 7th grade students on the 7th grade State Math Assessment).
SCHOOLIncludes all students in the school who take the assessment(e.g., a K-5 Music teacher is evaluated based on the growth of all students in the school on all State Math Assessments).
LINKED
Includes only the teacher’s students who take an assessment administered in another teacher’s course(e.g., a HS Theater teacher is evaluated based on the growth of only his/her students on the 9th and 10th grade NYC Performance Tasks in ELA).
34
Measures Of Student Learning (MOSL):Baseline Assessment Administration
35
By March:Schools revise teacher-level MOSL
selections if roster changes necessitate. Ongoing:
Schools use ongoing assessments to track student progress and inform
instruction
2015-2016 MOSL Implementation Stages: Baseline Assessment Administration
36
By Sept. 28:Schools make meaningful
school-level MOSL selections that are aligned
with student needs, instructional priorities, and
strategic assessment plans.
By Late May: Teachers in Grades 4-8 ELA and Math complete the
RMV Process. By June 10
Schools administer all end-of-year MOSL-eligible assessments. Schools must score and submit all
assessment results by June 24.
By end of June: Schools use end-of-year measures of student learning
data to reflect on practice and plan for SY 2016-17..
By Nov 6: Schools make teacher-level MOSL selections
By Oct. 30:
(If baseline administration is selected). Schools administer all baseline assessments to determine
student starting points. By Nov. 13
Schools must score and submit all baseline assessment results for MOSL purposes.
By Nov. 16: Schools set goals for students or groups of students (if
assessments require goal-setting)..
STAGE 1: Preparing for SY
2015-16
STAGE 2: Starting smart
STAGE 3: Tracking progress
STAGE 4: Finishing strong
Which Assessments Include Matching MOSL-eligible Baseline Assessments as an Option?
37
• SANDI/FAST
• WebABLLS
• Performance Series
• ELA & Math
• NYC Performance Tasks • All subjects
• Running Records• F&P
• DRA2
• TCRWP
Most Baseline Assessments are Optional for MOSL Purposes*
38
However, schools may elect to administer MOSL-eligible baseline assessments even if they choose not use the scores for Measures of Student Learning (MOSL) purposes.
What are some reasons why schools may choose to administer baselines for MOSL and non-MOSL purposes?
*Baselines are required for:• SANDI/Fast• Kindergarten grade/subjects when an individual target population and
growth model are selected
39
2014 -2015
2015-2016
• The administration of fall baseline assessments is required in Kindergarten if an individual target population AND growth model are selected,
and strongly encouraged for Grades 1 and 2. • Schools must select Goal-Setting for the
Alternate Assessment grade/subjects.
The administration of fall baseline assessments was optional for all
grade/subjects.
TURN AND TALK
• What questions do you still have about assessments?
• While not required, what is the value of administering baseline assessments in Grade 1 & 2?
New Policy: Baseline Assessment Administration is Required in Some Cases
Baseline Assessment Updates are available on the Advance Intranet
Timeline for Implementing MOSL & Assessment Administration
40
Aug Sep Oct Nov Dec Jan Feb March April May June
Grade/subject MOSL
selectionsby September
28
Baseline assessment
administration &
submissionSeptember 9 – November 13
Teacher-level MOSL
selectionsNovember 6
Goal-Setting*by November
15
End-of-year assessment
administration May 2 – June 10
(most assessments)**
Use assessment results to
inform instruction
(ongoing)
*Only for schools that selected goal-setting as a growth measurement.**see MOSL Assessment Administration Handbook for more information.
41
MOSL Selections: Roles and Responsibilities
PRINCIPAL
Select State Measures for grade/subjects that are not pre-determined by the state (assessment, target population, and/or growth measurement).
Accept ALL of the School Local Measures Committee recommendations or NONE of the recommendations and opt for Local Measures “Default”
For State Measures, determine if your school should administer some, all, or no Fall baseline assessments for MOSL purposes. Also determine if your school should administer some, all, or no Fall baseline assessments for instructional purposes.
Work with the School Local Measures Committee to communicate State and Local Measures selections to teachers by October 13th.
42
MOSL Selections: Roles and Responsibilities cont.
SCHOOL LOCAL MEASURES COMMITTEE
Recommend Local Measures for all relevant grade/subjects (assessment, target population, growth measurement)
For Local Measures, determine if your school should administer some, all, or no fall baseline assessments for MOSL purposes
Recommend a preferred subgroup option (if necessary). Optional: Customize subgroup selection for some or all grade/subjects
If goal-setting is selected, recommend that goals for grade or school-level goal-setting be set by the Committee or by the principal
Work with the principal to communicate State and Local Measures Selections to teachers
Periodic Assessments
43
Queens North BFSC
Overview of Periodic Assessment
Teach
AssessReflect
Analyze the results and reflect on
practice to decide what action is needed next to
increase student learning
Assess student mastery of the
skills and knowledge
identified in the standards
Teach the skills and knowledge identified in the
standards
The Common Core Learning Standards (CCLS) set a high bar for students and require teachers to provide a more rigorous academic experience for students. As we increase rigor in the classroom, we need a way to identify specific areas of strength and need to in order to provide targeted instruction for all students. Understanding how our students are learning and growing on a regular basis is fundamental to achieving our goal of preparing all New York City students to graduate ready for college and careers.
44
45
MOSL and Periodic Assessment Selections Should Be Viewed As One Comprehensive Decision Making Process
46
A Strategic Assessment Plan Can Be Used to Provide Feedback and Data to Support CCLS Implementation
Periodic Assessments & MOSL-eligible Assessments
The Periodic Assessment Portfolio contains most of theassessments that have previously been considered eligible for MoSL purposes:
• NYC Performance Tasks (K-12/ELA, Math, Science, Social Studies)
• Running Records of Literacy (DRA2, F&P, TCRWP)
• Performance Series (3-12)
• Sandi/Fast and WebABLLS
• CTE: Certiport / NOCTI
• Arts Commencement
47
Periodic Assessment Selection Tool
Periodic Assessment Selections Reopen:
The PA Selection Tool will remain open through September 30 in order to allow sufficient time to adjust your PA selections and ensure their alignment with your school’s upcoming Measures of Student Learning decisions for Advance.
If your school made a selection by September 11, PA will send student-facing materials and books.
48
Periodic Assessment Guide and Worksheets
PA Guide contains:
Program summary
Detailed descriptions of assessment choices
Step-by-Step guides to making selections
Selections worksheets
http://intranet.nycboe.net/Accountability/Assessments/PeriodicAssessments/
49
50
Accessing the Advance Web Application (AWA)
NOTE: Principal or Principal MoSL Designee(via Manage User Roles & Access) canenter School-Level MoSL Selectionsin the AWA.
51
NEW FEATURE!Copy last year’s School-Level MoSL Selections
Data Systems and Deadlines
Assessment Data System Deadline Fix Errors by
NYC Performance Tasks Schoolnet
October 30
November 13
Running Records: F&P and DRA2
STARS Classroom
Running Records: TCRWP AssessmentPro
Performance Series Performance Series Website
SANDI/FAST SANDI/FAST Online Portal November 6
WebABLLS WebABLLS
52
For assessments scored at school-sites, schools can access results as soon as scores are submitted
53
ASSESSMENT DATA SYSTEM RELEVANT REPORTS
NYC Performance Tasks Schoolnet Assessment Details, Item Analysis, Standards
Mastery
DRA2 and F&P STARS Classroom All Formal Assessment Elements, Critical Elements of Reading, Student Strengths/Weaknesses
TCRWP AssessmentPro Export School Data
mCLASS Reading 3D
mCLASS: Home Portal Class Summary Report, Student Summary Report
SANDI/FAST SANDI/FAST Online Portal FAST Class View, FAST Results, Standards Report
WebABLLS WebABLLS Total Grid View, Category View, Progress and Benchmark Reports
SLP/LOTE ATS Status Report, Item Report, Item Distribution Report
Advance Support Roles
54
Queens North BFSC
55
Teacher Development and Evaluation Coaches (TDECs) by District: your first point of contact.
TDEC DISTRICT
COMMUNITY DISTRICTS: Teresa Caccavale 24 Debbie White-Winkler 25 Brian Contarsy 26 Erica Urena-Thus 30
HIGH SCHOOLS: Lisa Grevenberg Districts 24, 25, 26 Tracy Atkins Renewal HS Angelica Beissel New Visions
56 56
Thank you!
Please take a moment to complete the feedback form before leaving
For a copy of this presentation please email your TDEC