2018 CAST Academy—Facilitator's Guide - CAASPP · 2018-06-05 · Page 1 Facilitator Guide for California Science Test (CAST) Academy Workshop—Spring 2018 Introduction This workshop

2017–18 California Science Test (CAST) Academy

Facilitator's Guide

April 2018

Page 1

Facilitator Guide for California Science Test (CAST) Academy Workshop—Spring 2018

Introduction This workshop was developed to build the capacity of California practitioners to implement the California Next Generation Science Standards (CA NGSS) by providing a deeper understanding of the standards and helping them to understand how the new science item types can inform teaching and learning. It is recommended that prior to delivering this professional development, administration preview the slides and make adjustments that will best serve their audience. Refer to the Presenter Notes to see if slides may need adjustment for delivering this training in a local setting. This Facilitator Guide is organized into five sections: 1. Welcome and Introductions (26 minutes) 2. Session 1: Remembering the Roles of Assessments (28 minutes) 3. Session 2: Review the CA NGSS, Role of Phenomena, and Concept of Multidimensional

Items (78 minutes) 4. Session 3: Experiencing and Analyzing the CAST Training Test Items (108 minutes) 5. Session 4: Implications of the CAST Item Types and Test Design for Teaching and

Learning (60 minutes) Each section of this guide includes participant objectives, estimated time to deliver the content, and bulleted Presenter Notes that correspond to slides in the associated PowerPoint presentation.

Materials and Facilitation To engage with the activities included in this presentation, participants should be seated in grade-specific table groups. Each table group should be set up to have all the materials:

• Pens • Highlighters • Markers • Poster paper • Sticky notes • Printed copies of the booklets and loose handouts • Printed copies of PowerPoint presentation with space for note-taking

Facilitators should have the following materials: • PowerPoint presentation • Facilitator Guide

Thank you for using the Spring 2018 CAST Academy Workshop to build local knowledge and expertise in the CA NGSS and the new CAST items for improved teaching and learning!

Key to Facilitator Guide Facilitator Guide for CAST Academy—Spring 2018

Page 2

Key to Facilitator Guide

Workshop Objectives Facilitator Guide for CAST Academy—Spring 2018

Page 3

Workshop Objectives • Review the types and roles of assessments in a comprehensive science assessment

program. • Review the California Next Generation Science Standards (CA NGSS), the role of

phenomena, and multidimensional items. • Experience the California Science Test (CAST) training test items. • Discuss implications of the CA NGSS and the CAST training test items for teaching and

learning. • Reflect on next steps.

Welcome and Introductions Facilitator Guide for the CAST Academy—Spring 2018

Page 4

Welcome and Introductions Slides 1–10 26 min.

Objectives • Review agenda, materials, and expectations for participation. • Provide an opportunity for participants to share thoughts about the California Next

Generation Science Standards (CA NGSS) and the California Science Test (CAST).

Detailed Slide Notes Slides Notes

California Science Test (CAST)Academy

Training of Trainers

California Next Generation Science Standards and the California Science Test:

Implications for Teaching and LearningSpring 2018

1

Slide 1—30 seconds Slide Intent: Introduce topic. Presenter Notes:

• Welcome to the California Science Test Academy. • Direct participants to sit at a grade-band table most

appropriate to the grade level of their work since a large portion of the training will involve group discussion.

California Science TestAcademy Agenda

Session 1: Review the types and roles of assessments in a comprehensive science assessment program.

2

Slide 2—1 minute Animated Slide Slide Intent: Articulate intended outcomes of the workshop. Presenter Notes:

• Provide participants with information about the content that will be covered and activities participants will be asked to participate in: − Session 1: Review of the various types of

assessment and the roles of each in a comprehensive assessment system.

− CLICK


Page 5

Slides Notes


Session 1: Review the types and roles of assessments in a comprehensive science assessment program. Session 2: Review the California Next Generation

Science Standards (CA NGSS), the role of phenomena, and multidimensional items.

2

− Session 2: Quickly review the structure of the California Next Generation Science Standards (CA NGSS); deconstruct a single performance expectation; learn about the role of phenomena in teaching, learning, and assessing; and analyze a multidimensional item.

− CLICK



Science Standards (CA NGSS), the role of phenomena, and multidimensional items. Session 3: Experience the California Science Test

(CAST) training test items.

2

− Session 3: Experience the California Science Test (CAST) training test items.

− CLICK



Science Standards (CA NGSS), the role of phenomena, and multidimensional items. Session 3: Experience the California Science Test

(CAST) training test items. Session 4: Reflect on the instructional demands for

students and teachers illustrated by the CAST training test items.

2

− Session 4: Discuss the implications of the new item types for teaching and learning and reflect on next steps for implementation in the classroom.


Page 6

Slides Notes

Introductions

WestEd staffEducational Testing Service (ETS) staffCalifornia Department of Education

(CDE) staffParticipants—Who is here?

• Grade level • Role

3

Slide 3—5 minutes Slide Intent: Establish the identities, roles, and backgrounds of the people in attendance. Presenter Notes:

• Introduce facilitators and other staff around the room.

• Lead participants to indicate by a show of hands their grade level (e.g., elementary or grades three through five, middle school or grades six through eight, high school or grades nine through twelve) and their role (e.g., classroom teacher, instructional coach, district level specialist).

• Provide an ice breaker prompt such as “What was the first job you ever had that required you to pay taxes?” Allow one to two minutes for table groups to introduce themselves to each other and answer the prompt.

• When delivering this professional development on-site, consider editing this slide.

Overview of Materials Booklets

• Participant Booklet• Training Test Items Booklet • Training Test Scoring Guide Booklet• Appendix 1 (from the 2016 Science Framework for

California Public Schools) Loose handouts

• Dimensions of the CA NGSS• Aligning the Training Test Items• Performance Expectations• Item Specifications

Slide deck Facilitator’s Guide (Draft)

4

Slide 4—1 minute Slide Intent: Orient participants to the materials. Presenter Notes:

• Refer participants to the materials to be utilized throughout workshop and the organization of those materials at the table.

• When delivering this professional development on-site, consider hiding or editing this slide.


Page 7

Slides Notes

Reflect . . . (1)

What word comes to mind when you think of the CA NGSS?

5

Slide 5—1 minute Slide Intent: Icebreaker to allow participants to share thoughts and feelings about the CA NGSS. Presenter Notes:

• Present the question and allow a short pause for reflection.

• This slide is set up for the use of Poll Everywhere. Alternatively, ask for volunteers to write their word on a sticky note or in their notes.

6

Slide 6—7 minutes Slide Intent: Icebreaker to allow participants to share their thoughts and feelings about the CA NGSS. Presenter Notes:

• This slide is set up for the use of Poll Everywhere. Alternatively, ask for volunteers to share their word or phrase. Have other participants raise their hand if they had the same word or phrase.

• When delivering professional development on-site, edit this slide to align with how participants will share their responses.

Reflect . . . (2)

What word comes to mind when you think of the CAST?

7

Slide 7—1 minute Slide Intent: Icebreaker to allow participants to share thoughts and feelings about assessment of the CA NGSS. Presenter Notes:

• Present the question and allow a short pause for reflection.

• This slide is set up for the use of Poll Everywhere. Alternatively, ask for volunteers to write their word on a sticky note or in their notes.


Page 8

Slides

8

Notes

Slide 8—7 minutes Slide Intent: Icebreaker to allow participants to share their thoughts and feelings about the CAST. Presenter Notes:

• This slide is set up for the use of Poll Everywhere. Alternatively, ask for volunteers to share their word or phrase. Have other participants raise their hand if they had the same word or phrase.

• When delivering professional development on-site, edit this slide to align with how participants will share their responses.

What the CAST Academy Is and Is NotThe CAST Academy is:

• An introduction to the CAST item types• An opportunity to reflect on the instructional

implications of the new item types, for the teacher and student

9

Slide 9—30 seconds Animated Slide Slide Intent: Level setting about the purposes of the workshop and purposes the workshop is not designed to serve. Presenter Notes:

• The focus of the CAST Academy is to build an understanding of how the CA NGSS are operationalized in the CAST and to reflect on the instructional implications that follow.

• CLICK

What the CAST Academy Is and Is NotThe CAST Academy is:

• An introduction to the CAST item types• An opportunity to reflect on the instructional

implications of the new item types, for the teacher and student

The CAST Academy is not: • An introduction to the basics of the CA NGSS• A training focused on increasing science

content knowledge

9

− Remind participants that over these past few years, there have been many opportunities to deepen understanding and support implementation of the CA NGSS. That is not the purpose of this workshop. Nor is this workshop designed to increase science content knowledge in any particular domain of science—life science, physical science, or Earth and space science.


Page 9

Slides Notes

Appoint a Table Spokesperson

Ensure that all voices are heard. Be prepared to share with the larger

group.

10

Slide 10—2 minutes Slide Intent: Establish group norms. Presenter Notes:

• Ask participants to identify a lead or spokesperson for their table.

• This person is responsible for two things: − Keep the group moving from one

person/response to the next. − If called upon, share with larger group.

Session 1: Remembering the Roles of Assessments Facilitator Guide for the CAST Academy—Spring 2018

Page 10

Session 1: Remembering the Roles of Assessments Slides 11–28 28 min.

Objectives • Develop a shared understanding of the intended purposes and uses of formative, interim/

benchmark, and summative assessments. • Understand the design of the CAST. • Understand the role of performance tasks in providing robust evidence of student

learning.


Session 1: Remembering the Roles of

Assessments

Slide 11—30 seconds Slide Intent: Shift participant focus to Session 1. Presenter Notes:

• Remind participants that we will start with a review of the roles of different types of assessments in a system of science assessment.

Objectives

Develop a shared understanding of the intended purposes and uses of formative, interim/benchmark, and summative assessments.

12

Slide 12—30 seconds Animated Slide Slide Intent: Convey the learning objectives for Session 1. Presenter Notes:

• A system of assessment includes formative, interim or benchmark, and summative assessments.

• CLICK


Page 11

Slides Notes

Objectives

Develop a shared understanding of the intended purposes and uses of formative, interim/benchmark, and summative assessments. Understand the design of the CAST.

12

• Understand the design of the CAST in terms of the number and types of items, the standards assessed, and the role of the CAST in a system of science assessment.

• CLICK

Objectives

Develop a shared understanding of the intended purposes and uses of formative, interim/benchmark, and summative assessments. Understand the design of the CAST. Understand the role of performance tasks in

providing robust evidence of student learning.

12

• Performance tasks can provide evidence of student learning that is more robust than other types of assessment items.

Why a New Science Test?

New standards require new assessments of all types– formative, interim/benchmark, summative.

The new standards represent significant shifts.

• Previous standards included separate content and investigation/ experimentation standards.

• CA NGSS is three-dimensional, integrating science and engineering practices, disciplinary core ideas, and crosscutting concepts.

13

Slide 13—1 minute Slide Intent: Establish the rationale for the new science test, the CAST. Presenter Notes:

• Although science is always changing, most of the big ideas in science have not. Instead it is the expectations that are changing and pushing us to rethink approaches to curriculum, instruction, and to assessment.

• These shifts are significantly different from previous science standards and represent significant shifts in teaching, learning, and assessing science toward the vision described in the California Science Framework.


Page 12

Slides Notes

14

SeptemberCalifornia

State Board of Education

(SBE) adopts CA NGSS

Board Action

Planning Professional Learning

TestingLegend:

2013 2014 2015 2016 20182017 2019

How We Got Here: Key Events

Slide 14—2 minutes Animated Slide Slide Intent: Front-load thinking about the long road from standards adoption to operational CAST. Presenter Notes:

• The long road to the new California Science Test, or CAST, began in September 2013 when the California State Board of Education adopted the California Next Generation Science Standards (CA NGSS).

• Encourage participants to identify where they see their engagement in the timeline.

• CLICK

14

January–FebruaryCA NGSS Framework

Focus Groups & Public Comment

March–MayScience Leadership

Team develops State Implementation Plan

for CA NGSSSeptemberCalifornia



JulyBegin Science Assessment Stakeholder

Meetings

Board Action


TestingLegend:

2013 2014 2015 2016 20182017 2019


• In 2014 the CA NGSS Framework was presented to focus groups and the public for comment. Science leadership teams developed the State Implementation Plan. Science Assessment Stakeholder meetings were convened.

• CLICK

14









Meetings

Board Action


TestingLegend:

April–December CA NGSS Rollout Phase 1

2013 2014 2015 2016 20182017 2019


• CA NGSS Rollout Phase 1 sessions, designed to bring teachers and administrators up to speed on the CA NGSS, were held from April to December 2014.

• CLICK


Page 13

Slides Notes


14









Meetings

Board Action


TestingLegend:


2013 2014 2015 2016 20182017 2019


• CA NGSS Rollout sessions continued in 2015 with Phase 2.

• CLICK


14









Meetings

NovemberSBE adopts the Science Curriculum Framework

MarchSBE adopts

the California

Science Test

Board Action


TestingLegend:


2013 2014 2015 2016 20182017 2019


• In March 2016, the State Board of Education adopted the plan for the California Science Test and in November adopted the Science Curriculum Framework.

• CLICK



14









Meetings


MarchSBE adopts

the California

Science Test

Board Action


TestingLegend:


2013 2014 2015 2016 20182017 2019


• CA NGSS Rollout sessions continued in 2016 with Phase 3.

• CLICK


Page 14

Slides Notes

8








March–JulyCAST Pilot Test


Meetings


MarchSBE adopts

the California

Science Test

March 2017–November 2018K–8 Science Instructional

Materials Review

Board Action


TestingLegend:

April–December CA NGSS

Rollout Phase 3


Rollout Phase 2


Rollout Phase 1

2013 2014 2015 2016 20182017 2019


• A pilot of the CAST occurred during the first half of 2017. Also in 2017, the Kindergarten through Grade Eight Science Instructional Materials Review process began.

• CLICK

May–December CA NGSS Rollout Phase 4



14










Meetings


MarchSBE adopts

the California

Science Test

March 2017–November 2018K–8 Science Instructional Materials

Review

Board Action


TestingLegend:


2013 2014 2015 2016 20182017 2019


• CA NGSS Rollout sessions continued with Phase 4. • CLICK

15





SeptemberCalifornia







for CA NGSS


Meetings


MarchSBE adopts

the California

Science Test



Review

Board Action


TestingLegend:


2013 2014 2015 2016 20182017 2019

October–December CA NGSS Rollout Phase 5

April–JulyCAST Field Test

May–June

Accelerating into CA NGSS

• The CAST field test is during the spring of 2018, and CA NGSS Rollout sessions continued with Phase 5 and Accelerating into NGSS, a workshop for districts not involved in Rollouts 1–4.

• CLICK


Page 15

Slides Notes

15





SeptemberCalifornia



January–July CAST

Operational Test





for CA NGSS


Meetings


MarchSBE adopts

the California

Science Test



Review

Board Action


TestingLegend:


2013 2014 2015 2016 20182017 2019

October–December CA NGSS Rollout Phase 5

April–JulyCAST Field Test

May–June

Accelerating into CA NGSS

• The first operational administration of the CAST is during the 2018–2019 school year.

Examples of Science Assessments

15

Slide 15—30 seconds Animated Slide Slide Intent: Convey the many ways assessment happens in a science classroom. Presenter Notes:

• Ask participants to think about some common forms assessment takes in the typical science classroom.

• CLICK

California Science Test

(CAST) Controlled Experiments

Classroom Observations

Classroom Discussions

End-of-Course Exams

AP Exams

Teacher-Created Tests/Quizzes

Field Study Investigations

Student Portfolios

Publisher Tests

Student Group Work

End-of-Unit Tests

Interim/Benchmark Assessments

Student-Developed Models

Examples of Science Assessments

15

• The slide shows just some of the many examples of science assessment that may have come to mind.


Page 16

Slides Notes

16

Types of Assessments

One way assessments can be categorized is by how often they occur:

Short cycle (formative)• Day-by-day, ongoing

Slide 16—1 minute Animated Slide Slide Intent: Convey that different types of assessments can be described by the timelines on which they occur. Presenter Notes:

• Short cycle assessments are usually used for formative purposes.

• These can be minute-by-minute; they allow the teacher to collect ongoing information to adjust instructional strategies.

• CLICK

16




Medium cycle (interim/benchmark)• Approximately quarterly or every six weeks

• Note the purpose of medium cycle or interim/benchmark assessments: − Monitor students’ academic progress toward the

learning goals reflected in the CA NGSS. − May predict a student’s end-of-year

performance—however, to use interim assessments solely as a predictive measure does not take full advantage of the information they provide about student learning.

− CLICK

16




Medium cycle (interim/benchmark)• Approximately quarterly or every six weeks

Long cycle (summative)• At the end of a course, a semester, or a year

• Long cycle assessments, or culminating assessments, are usually designed for summative purposes. − Evaluate student achievement at the end of

instruction


Page 17

Slides Notes

Description:• “Formative Assessment is a deliberate process used by

teachers and students during instruction that provides actionable feedback used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.”²

Purposes:• Provide immediate or very rapid feedback to teachers

and students • Provide evidence that can be used to adjust teaching

and learning

17

²Smarter Balanced: The Formative Assessment Process. 2015. Retrieved from http://www.smarterbalanced.org/wp-content/uploads/2015/09/Formative-Assessment-Process.pdf

Short Feedback Cycle—Formative Assessment Process

Slide 17—30 seconds Slide Intent: Convey specific information about the formative assessment process and its purposes. Presenter Notes:

• The description of the formative assessment process is from Smarter Balanced.

• Key message is that formative assessment is a process that provides rapid, actionable feedback to teachers and students.





End-of-Course Exams

AP Exams



Student Portfolios

Publisher Tests

Student Group Work

End-of-Unit Tests



18

A Few Examples of Formative Science Assessments

Slide 18—1 minute Animated Slide Slide Intent: Provide examples of formative assessments. Presenter Notes:

• Participants should be asked to consider which of the examples could be used as formative assessments.

• CLICK





End-of-Course Exams

AP Exams



Student Portfolios

Publisher Tests

Student Group Work

End-of-Unit Tests



18

A Few Examples of Formative Science Assessments

• Key message is that many of the assessment examples, not only the highlighted ones, could be used as formative assessments to provide actionable feedback.


Page 18

Slides Notes

Description:• “Interim/benchmark assessments . . . address intermediate

goals on the way to meeting end-of-year, or end-of-course goals. Typically administered quarterly or every six weeks, they cover a shorter period of instruction than long-cycle assessments and, consequently, provide more detail about student learning, although not enough to guide day-to-day teaching and learning.”³

Purposes:• Monitor students’ academic progress toward longer-term

goals• Inform school improvement planning

19

³2016 Science Framework for California Public Schools, Chapter 9 Assessment

Medium Feedback Cycle—Interim/Benchmark Assessment

Slide 19—30 seconds Slide Intent: Convey specific information about interim/benchmark assessments and some typical purposes. Presenter Notes:

• The description of interim/benchmark assessment comes from the 2016 Science Framework for California Public Schools.

• Emphasize that interim/benchmark assessment is used for progress monitoring.





End-of-Course Exams

AP Exams



Student Portfolios

Publisher Tests

Student Group Work

End-of-Unit Tests



20

A Few Examples of Interim/Benchmark Science Assessments

Slide 20—1 minute Animated Slide Slide Intent: Provide examples of interim/benchmark assessment. Presenter Notes:

• Participants should be asked to consider which of the examples is typically used as an interim/benchmark assessment.

• CLICK





End-of-Course Exams

AP Exams



Student Portfolios

Publisher Tests

Student Group Work

End-of-Unit Tests



20

A Few Examples of Interim/Benchmark Science Assessments

• Note that some of the examples of science assessment from the earlier slide were also highlighted as a formative assessment. These are examples of how an assessment could be either, depending on how the assessment is used.


Page 19

Slides Notes

Description:• May be referred to as a “culminating assessment” or an

“end-of-course” assessment• Provides information on students’ knowledge and skills

relative to learning standards • May be used for “high-stakes” purposesPurposes:• Provide an overall description of students’ learning status• Monitor and evaluate student achievement at the group

level• Inform program-level and school-improvement planning

21

Long Feedback Cycle—Summative Assessment

Slide 21—30 seconds Slide Intent: Convey specific information about summative assessments and typical purposes. Presenter Notes:

• Note that summative assessments come at the end of instruction and can include large-scale assessments such as the CAST and school or classroom assessments given at the end of a unit, course, or calendar period.

• Summative assessments can inform teaching and learning at the program, school, and district levels.





End-of-Course Exams

AP Exams



Student Portfolios

Publisher Tests

Student Group Work

End-of-Unit Tests



22

A Few Examples of Summative Science Assessments

Slide 22—1 minute Animated Slide Slide Intent: Provide examples of summative assessment. Presenter Notes:

• Participants should be asked to consider which of the examples is typically used as a summative assessment.

• CLICK





End-of-Course Exams

AP Exams



Student Portfolios

Publisher Tests

Student Group Work

End-of-Unit Tests



22

A Few Examples of Summative Science Assessments

• Note that the highlighted examples are only some of the assessments that could be used as summative assessments.

• There is overlap between examples from the earlier types and those highlighted here. Some of the highlighted examples are exclusively summative, like AP exams or the CAST. Others may be used as interim/benchmark or even formative assessments.


Page 20

Slides Notes

From Assessment Types to Item TypesDifferent item types elicit different information about student learning and can be used for different purposes. Some example item types include:

• Selected response (e.g., multiple choice single select, multiple select)

• Fill-in-blank• Technology-enhanced items (e.g., drag and drop,

inline choice, simulation) • Constructed response• Performance task (i.e., a set of items with a

common stimulus)

23

Slide 23—1 minute Slide Intent: Convey some of the many possible item types on the CAST and other assessments. Presenter Notes:

• The list includes only some of the many item types that are used in assessments, including the CAST.

• Item types are chosen to elicit the evidence most appropriate for the learning we want to assess. For example, the evidence from a drag-and-drop item is very different than the evidence of student learning from a constructed response.

• Performance tasks may include multiple item types. Note that performance tasks in the CAST are delivered solely in an online format and do not require additional materials or hands-on components. This differs from the expected use of performance tasks in a classroom setting.

CAST Operational Test Design (1)

24

Assessment Segment A

Assessment Segment B

Assessment Segment C

Scope and Depth of Measurement

Measures a broad sample of CA NGSS performance expectations (PEs)

Deep measurement of a targeted sample of a few PEs provided in item sets

Broad and deep—full range of measurement of PEs for each grade span.

Number of Items

32 to 34 discrete (stand-alone) items

2 performance tasks with 4 to 6 items each

1 performance task with 4 to 6 itemsOR6 to 7 discrete (standalone) items

See also page 6 of the Participant Booklet.

Slide 24—2 minutes Slide Intent: Provide a high-level overview of the CAST design. Handout: Participant Booklet, p. 6 Presenter Notes:

• The CAST consists of three segments. Segment A is designed to provide broad coverage of the PEs. The performance tasks in Segment B provide deeper measurement of a few PEs.

• Describe discrete or stand-alone items as individual items that each assess one performance expectation (PE), allowing for broad content coverage.


Page 21

Slides Notes

Slide 25—3 minutes Slide Intent: Provide a high-level overview of the CAST design. Handout: Participant Booklet, p. 6 Presenter Notes:

• Individual student scores are based on segments A and B while group scores are determined by segments B and C.

• There are a variety of item types in each of the segments. The item type is chosen to gather the appropriate evidence of student understanding.

• Note that the PEs measured by the grade five test include the PEs from grades three through five and foundational concepts introduced in K–grade two (although specific PEs from K–grade two are not assessed, skills and concepts introduced in K–grade two serve as a foundation for meeting PEs in grades three to five). The grade eight test includes PEs from grades six through eight and the high school test includes PEs from grades nine through twelve.

Why Performance Tasks?

READ: Performance Tasks from 2016 Science Framework for California Public Schools, page 7 in the Participant Booklet.

DO: Mark 2–3 of the most important words and/or phrases (highlight, underline, or circle).

26

Slide 26—4 minutes Slide Intent: Build shared understanding of what performance tasks require of students. Handout: Participant Booklet, p. 7 Presenter Notes:

• To build a shared understanding of the value of performance tasks, ask participants to read “Performance Tasks” and highlight/underline two or three of the most important words that jump out.


Page 22

Slides Notes

Key Phrases

27

2016 Science Framework for California Public Schools, Chapter 9, pp.13-14.

Slide 27—3 minutes Animated Slide Slide Intent: Elicit information from participants about what performance tasks require of students. Handout: Participant Booklet, p. 7 Presenter Notes:

• Invite participants to share some of the words or phrases that resonated with them.

• Encourage other audience members to raise their hands if they selected the same word/phrase (this will give the room a sense of what words or phrases were commonly identified as important).

• CLICK

Key Phrases

. . . assess a single PE or a bundle of related PEs . . .

27


• Briefly go over the importance of each phrase highlighted on the slide that was not shared by participants.

• A performance task is not limited to assessing a single PE. Multiple related PEs can be assessed by the items that make up a performance task.

• CLICK

Key Phrases

. . . assess a single PE or a bundle of related PEs . . . . . . centered around phenomena . . .

27


• The items in a performance task are centered around a phenomenon or an observable event that makes you wonder “what,” “why,” or “how.”

• Let participants know that the idea of phenomena will be addressed in more detail in the next session.

• CLICK


Page 23

Slides Notes

Key Phrases

. . . assess a single PE or a bundle of related PEs . . . . . . centered around phenomena . . . . . . use and apply what they know . . .

27


• Performance tasks require students to use and apply what they know to a new context.

• CLICK

Key Phrases

. . . assess a single PE or a bundle of related PEs . . . . . . centered around phenomena . . . . . . use and apply what they know . . . . . . conduct science and engineering

within the classroom . . .

27


• Performance tasks provide the opportunity for students to engage in doing science and engineering in the classroom.

Reflect and Share: Performance Tasks

28

What is the value to both teachers and students of using performance tasks in science assessments?

• How might evidence gathered from a performance task differ from evidence provided by discrete (standalone) items?

• What is the value to both teachers and students of using performance tasks as formative, interim/benchmark, and as summative assessments?

Capture your thinking in the Performance Tasks graphic organizer (p. 8 in the Participant Booklet). Share your thoughts at your tables.

Reflect

Talk Listen

Slide 28—5 minutes Slide Intent: Participants process the information on the value and uses of performance tasks. Handout: Participant Booklet, p. 8

• Provide approximately three minutes for participants to capture their thoughts about performance tasks in the graphic organizer in the booklet.

• Encourage participants to share their thoughts with their table group.

Session 2: Review the CA NGSS, Role of Phenomena, and Concept of Multidimensional Items

Facilitator Guide for the CAST Academy—Spring 2018

Page 24

Session 2: Review the CA NGSS, Role of Phenomena, and Concept of Multidimensional Items Slides 29–47 78 min.

Objectives • Understand the organization of the CA NGSS. • Examine the structure of the CA NGSS performance expectations and deconstruct each

dimension. • Analyze a multidimensional item. • Recognize the role of phenomena in instruction and assessment of the CA NGSS. • Experience the development of a two- to three-dimensional item based on a

phenomenon. • Examine a CAST item specification and understand the role of item specifications in the

development of the CAST.


Session 2:Review the CA NGSS, Role of

Phenomena, and the Concept of Multidimensional Items


• In this session, participants will review the structure of a performance expectation (PE) from the CA NGSS, deconstruct a grade-level PE, learn about the role of phenomena in teaching and learning, and analyze a multidimensional item.



Page 25

Slides Notes

Objectives

Understand the organization of the CA NGSS.

30

Slide 30—1 minute Animated Slide Slide Intent: Orient participants to the objectives for Session 2. Presenter Notes:

• Provide participants with information about the content that will be covered and activities they will be asked to participate in: − Review the organization of the CA NGSS; − CLICK

Objectives

Understand the organization of the CA NGSS. Examine the structure of a CA NGSS performance

expectation (PE) and deconstruct each dimension.

30

− Examine and deconstruct a PE; − CLICK

Objectives


expectation (PE) and deconstruct each dimension. Analyze a multidimensional item.

30

− Analyze a multidimensional item; − CLICK



Page 26

Slides Notes

Objectives


expectation (PE) and deconstruct each dimension. Analyze a multidimensional item. Recognize the role of phenomena in instruction and

assessment of the CA NGSS.

30

− Consider the role of phenomena and brainstorm phenomena for a PE; and

− CLICK

Objectives


expectation (PE) and deconstruct each dimension. Analyze a multidimensional item. Recognize the role of phenomena in instruction and

assessment of the CA NGSS. Examine a CAST item specification and understand the

role of item specifications in the development of the CAST.

30

− Examine an item specification and its role in the development of CAST items.

Three Dimensions of the CA NGSS The

are what scientists and engineers DO

The are what scientists and engineers KNOW

The are how scientists and engineers THINK

*The CA NGSS logo is a registered trademark of Achieve.

*

31

Slide 31—1 minute Slide Intent: Review the dimensions of the science standards. Presenter Notes:

• With very few exceptions, each performance expectation in the CA NGSS includes each of the three dimensions—a practice, a disciplinary core idea, and a crosscutting concept. The PEs are a three-dimensional statement of what students should be able to do at the end of instruction.

• Note that the CA NGSS logo is a registered trademark of Achieve, the organization that coordinated the development of the CA NGSS. The logo itself is a three-dimensional Mobius strip. Explain that Mobius strip is a flat piece of paper, given a single twist and rejoined into a 3-D shape. If you place your finger on at any point, you can trace both sides of the figure and return to your starting



Page 27

Slides Notes point without ever lifting your finger. Mobius strip is a great model for NGSS because, much like the entire strip of paper is incorporated into a single figure, so are all three dimensions in the CA NGSS.

• Dependent upon the group’s level of mastery/expertise, presenters may wish to consider the content of this slide prior to delivering the workshop. Edit or hide slide as appropriate to your delivery context.

Science and Engineering Practices (SEPs)The CA NGSS contain eight SEPs that develop across the grade bands:1. Asking Questions and Defining Problems2. Developing and Using Models3. Planning and Carrying Out Investigations4. Analyzing and Interpreting Data5. Using Mathematics and Computational Thinking6. Constructing Explanations and Designing Solutions7. Engaging in Argument from Evidence8. Obtaining, Evaluating, and Communicating Information

32

Slide 32—1 minute Slide Intent: Review the Science and Engineering Practices (SEPs). Handout: Loose Handout #1, Dimensions of the CA NGSS Presenter Notes:

• Scientific practices are the behaviors that scientists and engineers engage in (do) as they investigate and build models and theories about the natural world.

• Students in kindergarten through grade twelve should engage in all eight practices in each grade band.

• Each practice progresses in complexity and sophistication across the grade bands.

• The eight practices are not separate; they intentionally overlap and interconnect. − Example: Students engaging with SEP 6 and

constructing an Explanation may need to use a model to explain, or analyze data by way of using mathematical thinking, or potentially obtain, evaluate, and communicate information from multiple sources.

• A performance expectation focuses on some, but not all, capabilities associated with a practice.

• If participants wish to access additional detailed information regarding the progression of the Science and Engineering Practices, refer to appendix 1 of



Page 28

Slides Notes the 2016 Science Framework (https://www.cde.ca.gov/ci/sc/cf/scifwprepubversion.asp) and appendix F of the CA NGSS (https://www.nextgenscience.org/get-to-know).

• Appendix 1 shows the breadth of each practice within a grade band and shows the progression across the grade bands.


Domains of the Disciplinary Core Ideas

Physical SciencesLife SciencesEarth and Space SciencesEngineering, Technology, and Application of

Science

33

Slide 33—30 seconds Slide Intent: Review the domains. Presenter Notes:

• The Disciplinary Core Ideas (DCIs) are divided into four domains or disciplines—Physical Sciences; Life Sciences; Earth and Space Sciences; and Engineering, Technology, and Application of Science.


https://www.cde.ca.gov/ci/sc/cf/scifwprepubversion.asp


https://www.nextgenscience.org/get-to-know




Page 29

Slides Notes

Disciplinary Core Ideas (DCIs) PS1: Matter and Its Interactions PS2: Motion and Stability: Forces and Interactions PS3: Energy PS4: Waves and Their Applications

LS1: From Molecules to Organisms: Structures and Processes LS2: Ecosystems: Interactions, Energy, and Dynamics LS3: Heredity: Inheritance and Variation of Traits LS4: Biological Evolution: Unity and Diversity

ESS1: Earth’s Place in the Universe ESS2: Earth’s Systems ESS3: Earth and Human Activity

ETS1: Engineering Design ETS2: Links Among Engineering, Technology, Science, and Society

34

Slide 34—1 minute Slide Intent: Review the Disciplinary Core Ideas. Handout: Loose Handout #1, Dimensions of the CA NGSS Presenter Notes:

• Each domain (or discipline) is further divided into two to four core ideas.

• Each core idea progresses in complexity and sophistication across the grade bands.

• If participants wish to access additional detailed information regarding the progression of the Disciplinary Core Ideas, refer to appendix 1 of the 2016 Science Framework (https://www.cde.ca.gov/ci/sc/cf/scifwprepubversion.asp) and appendix E and appendix F of the CA NGSS (https://www.nextgenscience.org/get-to-know).


Crosscutting Concepts (CCCs)

The CA NGSS contain seven CCCs that develop across the grade bands:1. Patterns2. Cause and Effect3. Scale, Proportion, and Quantity4. Systems and System Models5. Energy and Matter6. Structure and Function7. Stability and Change

35

Slide 35—1 minute Slide Intent: Review the Crosscutting Concepts (CCCs). Handout: Loose Handout #1, Dimensions of the CA NGSS Presenter Notes:

• The crosscutting concepts help provide students with an organizational framework for connecting knowledge from the various disciplines into a coherent and scientifically based view of the world.

• Each crosscutting concept progresses in complexity and sophistication across the grade bands.







Page 30

Slides Notes • Like the practices, the seven crosscutting concepts

are not separate; they intentionally overlap and interconnect. − Example: A student exploring ecosystems will

need to understand the system and its components, may require thinking about scale (is the system inside of a water droplet or in a region or the entire planet?), may require thinking about the flow of energy and cycling of matter.

• A performance expectation focuses on some, but not all, capabilities associated with a crosscutting concept.

• If participants wish to access additional detailed information regarding the progression of the crosscutting concepts, refer to appendix 1 of the 2016 Science Framework (https://www.cde.ca.gov/ci/sc/cf/scifwprepubversion.asp) and appendix G of the CA NGSS (https://www.nextgenscience.org/get-to-know).


36

Analyze a Multidimensional Item

Slide 36—7 minutes Slide Intent: Introduce and analyze an example multidimensional item. Handout: Participant Booklet, p. 10; Loose Handout #1, Dimensions of the CA NGSS Presenter Notes:

• The example item is from the grade five CAST training test.

• Provide approximately six minutes for participants to individually read through the item and consider for which SEP(s), DCI(s), and CCC(s) the item is gathering evidence of student learning. Remind them to consider the interconnectedness and







Page 31

Slides Notes overlap among dimensions and whether the student engages with more than one of each dimension.

• Participants should refer to the loose handout for the lists of SEPs, DCIs, and CCCs.

37

As a Table Group:Analyze a Multidimensional Item

In your table group: • Share your thoughts on the SEP(s), DCI(s), and CCC(s) assessed by the item. • Discuss how the item gathers evidence of student understanding of each

dimension.

A student filled a plastic cup with ice cubes and placed a lid on the cup. The empty cup with the lid weighed 20 grams. The cup with the lid and the ice cubes weighed 180 grams. The student used a hair dryer to gently heat the ice cubes. After all the ice melted, the student weighed the cup with the lid and the liquid water. The diagram shows the changes that took place.

Complete the bar graph to show the weight of the ice cubes and the weight of the liquid water.

Slide 37—8 minutes Slide Intent: Discuss the alignments of the example multidimensional item Handout: Participant Booklet, p. 10; Loose Handout #1, Dimensions of the CA NGSS Presenter Notes:

• After four to five minutes of table discussion, call upon table spokespersons to share their thoughts on the SEPs, DCIs, and CCCs assessed by the item and how the evidence for each is gathered by the item.

• Emphasize that differences of opinion may occur due to the intentionally integrated and overlapping nature of the dimensions. Students may engage with different dimensions due to different approaches as they respond to the item. There is an intended alignment—this item gathers evidence for specific dimensions.

• The intended alignment of the item is as follows: − SEP: Using Mathematical and Computational

Thinking − DCI: PS1 Matter and Its Interactions − CCC: Scale, Proportion, and Quantity



Page 32

Slides Notes

38

Structure of a PE (1)

PE statement

Also see Participant Booklet, page 11.

Slide 38—30 seconds Animated Slide Slide Intent: Review structure of a performance expectation. Handout: Participant Booklet, p. 11 Presenter Notes:

• Just under the gray bar that indicates the grade level and core idea (e.g., grade five and PS1 Matter and Its Interactions in the example) is the performance expectation itself.

• CLICK

38

Clarification Statement


PE statement


• Following the performance expectation is the Clarification Statement. There are PEs that do not include a Clarification Statement.

• The Clarification Boundary provides some examples.

• CLICK

38

Clarification StatementAssessment Boundary


PE statement


• Next is the Assessment Boundary. Assessment Boundaries define the boundary for the CAST. Teachers may use their discretion about whether they choose to assess past the boundary in their classes. There are PEs that do not include an Assessment Boundary.

• CLICK



Page 33

Slides Notes

38

Clarification StatementAssessment Boundary

Foundation Boxes


PE statement


• The area with the three columns is the Foundations Boxes. The blue first column describes the SEP, the orange second column describes the DCI, and the green third column describes the CCC.


39


SEP Grade-Band Description


Slide 39—30 seconds Animated Slide Slide Intent: Review structure of the Foundation Boxes. Handout: Participant Booklet, p. 11 Presenter Notes:

• The blue first box contains the Science and Engineering Practice associated with the PE.

• The paragraph just under the name of the SEP is the grade-band description of the practice.

• CLICK

39

SEP Grade-Band-Specific Bullet



• The bulleted statement is the SEP grade-band specific bullet associated with this PE.

• CLICK



Page 34

Slides Notes

39

DCI Grade-Band-Specific Bullet



• The bulleted statement in the orange second box is the DCI grade-band specific bullet associated with this PE.

• CLICK

39


CCC Grade-Band-Specific Bullet


• The bulleted statement in the green third box is the CCC grade-band specific bullet associated with this PE.


As a Table Group: Deconstructing a PE

Discuss the following questions for the example PE, 5-PS1-2.

The PE statement: What does the PE statement reveal about how students are expected to combine practices, knowledge, and connections to crosscutting concepts?

The Clarification Statement and the Assessment Boundary:What are the implications of the Clarification Statement and the Assessment Boundary for instruction and for assessment?

The dimensions: How do the foundation boxes support the interpretation of the PE?

You may use the graphic organizer on page 12 of the Participant Booklet.

40

Slide 40—9 minutes Slide Intent: Think about and discuss the components of the example PE. Handout: Participant Booklet, p. 12; Loose Handout—Grade Five, Grade Eight, or High School Performance Expectations Presenter Notes:

• Ask participants to work as a table group for approximately five minutes to answer the question on the slide and included on the graphic organizer. Suggest that they use the graphic organizer to record the discussion.

• You may wish to offer middle school and/or high school PEs for this activity, depending on the grade levels relevant to the participants.



Page 35

Slides Notes

As a Table Group: Deconstructing the Dimensions of a PE

On poster paper or the large paper on your table, answer these questions: What are the discrete skills

and knowledge in the PE? What would be evidence of

student learning for each dimension of the PE?

The progressions in the Appendix 1 handout may be helpful.

41

PE: ___________SEP DCI

CCC

Slide 41—12 minutes Slide Intent: Think about and discuss the components of the SEP for the example PE. Handout: Participant Booklet, p, 13; Appendix 1 from the 2016 Science Framework; Loose Handout—Grade Five, Grade Eight, or High School Performance Expectations Presenter Notes:

• Each table group should have a chart paper pad, an easel, large sheets of paper, and markers.

• Ask table groups to focus on the SEP for the example PE and take approximately five minutes to begin to create a chart that includes the subpractices and/or components of the SEP, how this SEP might intersect with other SEPs, what it looks like when a student uses the SEP, and possible evidence that a student is mastering the SEP.

• Appendix 1 from the 2016 Science Framework is a useful source of information for this task. Appendix1 is included as a separate booklet.

• Invite tables to share their completed deconstructions with the whole group.


Role of Phenomena in the CAST Design Scientific Phenomena

• Phenomena are observable events that occur in the universe that make you wonder “what,” “how,” or “why.”

• Must be engaging but don’t have to be phenomenal.• Provide opportunities for sense-making using an

integration of the three dimensions to explain or make sense of the phenomenon.

Table groups: Generate a list of 2–3 phenomena that could be used in the classroom to facilitate student learning of the example PE, 5-PS1-2.


42

Slide 42—9 minutes Slide Intent: Define phenomena in the context of the CAST. Handout: Participant Booklet, p. 14 Presenter Notes:

• Phenomena are observable events that occur in the universe that make you wonder “what,” “how,” or “why.”

• Provide some example phenomena, such as:



Page 36

Slides Notes − The moon looked different last week than it does

now. − Flower nectar tastes sweet. − On a warm day, water droplets form on the

outside of a cold glass. • Note that students can model a phenomena, collect

data on a phenomena, and use other skills, crosscutting concepts, and core ideas to make sense of a phenomena.


• Example phenomena for 5-PS1-2 could include: − The weight of a cookie is the same whether the

cookie is whole or broken into pieces. − Salt can no longer be seen after being stirred

into a container of water, but the salt is still there. − Mixing water, glue, and a salt solution turns the

liquid mixture into a squishy solid, or slime, but the weight of the mixture does not change.

The CAST Item Characteristics

Items are set in a context suggested by a phenomenon.

Each item is multidimensional. A single item is unlikely to assess all

components of each dimension of a PE.

43

Slide 43—1 minute Slide Intent: Introduce characteristics of CAST items. Presenter Notes:

• The context is the setting in which the phenomenon is presented to students. For the phenomenon of water droplets on the outside of a cold glass, an example context might be a puddle of water forming under a glass of ice water on the lunch table.

• A multidimensional item for the condensation example might ask a student to make a model to show what happens to make the water droplets form on the outside of the glass.

• Remind participants of the results of their deconstruction of a PE. It would be difficult for a single item to encompass all of the components of that PE.



Page 37

Slides Notes

CAST Item Specifications Tools for item writers

• Ensure items align with the claim made by the PE• Strategies for developing multidimensional items

Source: CAST Item Specifications and ECD Documentation 44

Slide 44—1 minute Animated Slide Slide Intent: Define purposes and uses of the CAST item specifications. Presenter Notes:

• Item specifications were developed through a thoughtful, multistep process. − Each SEP was deconstructed into subpractices

and subpractice assessment targets (focal knowledge, skills, and abilities).

− Then the DCI was deconstructed into DCI targets.

− This information, along with the CCC, was integrated into a description of the task and evidence that the task might elicit.

• Item specifications may also include possible contexts, common misconceptions, and any additional boundaries.

• Item specifications are primarily intended to be used by item writers.

• CLICK



Guides for reviewers• Alignment of items• Valid assessment of the PE


• Item reviewers are the other primary audience for item specifications.

• CLICK



Page 38

Slides Notes



Guides for reviewers• Alignment of items• Valid assessment of the PE

Tools for educators• Understanding the three-dimensional nature of the

assessment• Strategies for developing items for formative,

interim/benchmark, and summative use in the classroom


• Item specifications can also provide useful information to educators and to the public who might be more interested in a high level of detail about the CAST items.

Structure of the CAST Item Specifications

45

Slide 45—2 minutes Slide Intent: Familiarize participants with the template of a CAST item specification. Presenter Notes:

• An item specification includes the full text of the PE and foundation boxes.

• Below the foundation boxes, the subpractice and subpractice assessment targets describe the subpractice item developers determined to be most appropriate for the PE, along with the assessment targets for the DCI and CCC.

• The lists of phenomena and common misconceptions are not intended to be exhaustive.

• Examples of Integration of Assessment Targets and Evidence provide descriptions of appropriate tasks and student evidence for the PE.

• The evidence statements that were developed for the NGSS are also included.



Page 39

Slides Notes

As a Table Group: Using the CAST Item Specification

Use the CAST item specification for 5-PS1-2 in your loose handouts to do the following: Compare the Assessment Targets for each dimension

in the item specification to the discrete skills, knowledge, and evidence on your poster. Compare the Examples of Integration of Assessment

Targets in the item specification to the example item. Consider any revisions to your deconstruction of the

PE in light of the information in the item specification.

46

Slide 46—12 minutes Slide Intent: Compare the item specification to the deconstructed PE. Handout: Participant Booklet, p. 15; Loose Handout—Grade Five, Grade Eight, or High School Performance Expectations Presenter Notes:

• Give participants eight to nine minutes to compare their deconstruction of the PE to the item specification for that PE.

• Note that all of the subpractices for an SEP are not included in an item specification nor are all components of a DCI necessarily included.

• Remind participants to revise their deconstruction of the PE, as needed, based on the item specification.

How do you see yourself using the item specifications as a resource? What would be best uses of the item specifications? How might you use the item specifications to inform

teaching and learning? Capture your thinking on page 16 in the Participant Booklet. Share your thoughts at your table.

47

Reflect and Share Reflect

Talk Listen

Slide 47—10 minutes Slide Intent: Participants capture their thoughts on the implications of the CAST item specifications. Handout: Participant Booklet, p. 16 Presenter Notes:

• Item specifications may be unfamiliar to participants. Allow four to five minutes to reflect on and note takeaways on the implications and uses of item specifications for their contexts.

• Allow four to five minutes for sharing in table groups. • Invite table representatives to share main themes

from table discussion with the larger group.

Session 3: Experiencing and Analyzing the CAST Training Test Items Facilitator Guide for the CAST Academy—Spring 2018

Page 40

Session 3: Experiencing and Analyzing the CAST Training Test Items Slides 48–61 108 min.

Objectives • Experience CAST discrete items and performance tasks from the perspective of a

student. • Understand the design of CAST items and scoring rubrics. • Understand the alignment of the items as an integration of multiple dimensions of a

performance expectation (PE). • Reflect on the instructional demands for students and teachers illustrated by the CAST

training test items.


Session 3: Experiencing and Analyzing the CAST Training Test Items


• In this session, participants will take a deep dive into the CAST training test items.

• Remind participants of the work they have completed thus far in understanding the three dimensions, brainstorming phenomena, deconstructing PEs, and examining item specifications. Note that participants should draw upon their thinking about the CA NGSS as they examine the CAST items. Remind participants that not every item can assess a whole PE, and to keep in mind the deconstructions and item specifications when looking for the parts of the PE that are being assessed in each item.


Page 41

Slides Notes

Objectives

• Experience California Science Test (CAST) discrete items and performance tasks from the perspective of a student.

49

Slide 49—30 seconds Animated Slide Slide Intent: Articulate the objectives for Session 3. Presenter Notes:

• This session will provide participants with time to interact with the CAST training test discrete items and performance tasks as students would.

• CLICK

Objectives


• Understand the design of CAST items and scoring rubrics.

49

• Develop a common understanding of the design of the CAST items and the associated scoring rubrics.

• CLICK

Objectives



• Understand the alignment of the items as an integration of multiple dimensions of a performance expectation.

49

• Understand the alignment of the items to the dimensions of a performance expectation.

• CLICK


Page 42

Slides Notes

Objectives



• Understand the alignment of the items as an integration of multiple dimensions of a performance expectation.

• Reflect on the instructional demands for students and teachers illustrated by the CAST Training Test items.

49

• Set up the expectation that participants will be asked to reflect on the demands these items make on students and to consider instructional implications of these types of items.

The Test Development Process

50CAST Item Review Copyright © 2018 by Educational Testing Service. All rights reserved

Assessment development

experts receive new items/PTs from item

writers

Slide 50—3 minutes Animated Slide Slide Intent: Develop an understanding of the rigorous multistep process for the development of the CAST items. Presenter Notes:

• At the beginning of the process, the CDE consults with assessment experts to develop the framework for the assessment.

• Item writers, including California educators, who have been trained on the CA NGSS, the CAST Item Specifications, and best practices for CAST item writing develop and submit discrete items and performance tasks.

• CLICK





writersAssessment

development experts and accessibility experts

review and revise items/PTs

• As in earlier stages of the NGSS rollout, CAST development involved hundreds of individuals, including CDE staff and California educators. Coordinating the CAST development with the CDE were assessment development experts from Educational Testing Service (ETS).

• ETS assessment development experts and accessibility experts review the submitted items and performance tasks to ensure the items meet established criteria such as correct scientific content, grade-level appropriate vocabulary, and alignment to the CA NGSS.

• CLICK


Page 43

Slides Notes





writersAssessment



Educator participants

review items/PTs

• California educators then review and may recommend edits to the items and performance tasks.

• CLICK





writersAssessment




review items/PTs


experts reconcile recommended edits

• The edits recommended by the educator participants are reconciled by assessment development experts at ETS.

• CLICK





writersAssessment




review items/PTs



California Dept. of Education

review/approve items/PTs

• The items and performance tasks are then submitted to the California Department of Education for review and approval.

• CLICK


Page 44

Slides Notes





writersAssessment




review items/PTs





Census Pilot/Field test

items/PTs

• The items and performance tasks that meet with the California Department of Education approval are then field tested.

• For an item to get from item writer to field test typically takes about one year due to the extensive review process that ensures their quality.

• All students in a tested grade participate in the field test.

• CLICK





writersAssessment




review items/PTs





Census Pilot/Field test

items/PTs

Educator participants review items with

field test data

• After the field test is complete, California educators review the items along with the field test data for each item. At this point, the California educators may recommend to CDE that the item be approved for use on the CAST or that the item be removed from the pool of usable items.

Slide 51—16 minutes Slide Intent: Demonstrate the functionality of the online CAST training test items. Presenter Notes:

• Go to the Online Practice and Training Tests Portal (http://www.caaspp.org/practice-and-training/index.html).

• Demonstrate logging on, selecting a grade, and accessing the CAST training test. Emphasize that the training test can also be an instructional tool.

• Point out features of the online training test, including both discrete items and performance tasks. Demonstrate the functionality of the technology-enhanced items.

http://www.caaspp.org/practice-and-training/index.html

http://www.caaspp.org/practice-and-training/index.html


Page 45

Slides Notes

Individually and as a student, complete the grade five, grade eight, or high school training test items in the CAST Training Test Items booklet.

Consider these questions as you interact with the items:• How would your students react to items like these?

oWhat would they be good at? oWhat would they struggle with?

• How are phenomena used in the items and performance task?

52

Takin

Slide 52—16 minutes Slide Intent: Set the expectation for individual work on the grade-level training test items. Front-load thinking about the adaptations necessary to put the items in a paper-pencil format. Handout: Training Test Items Booklet; p.5 for grade five, p. 19 for grade eight, or p. 35 for high school. Presenter Notes:

• The booklet represents the online items translated to a paper-pencil version to allow their use in this setting. Therefore, the functionality of the technology-enhanced items is not available.

• Note that participants should respond to the items as a “typical” student in the selected grade level.

• Remind the participants to consider the framing questions on the slide as they interact with the training test items. Also emphasize that the test should be taken on paper rather than online in order to preserve and analyze responses to the items.

Reflect: CAST Training Test Initial Reactions

Reflect on your experience with the items.

• What would students need to know and/or experience to successfully respond to these items that is different than for pre-NGSS items?

• How are the demands different for discrete items as compared to items grouped in a performance task?

• What potential barriers might special populations and other students, in general, encounter in the items?

Capture your thoughts on page 18 in the Participant Booklet.

53

Slide 53—5 minutes Slide Intent: Provide guiding questions and time for participants to record their initial reactions to the training test items. Handout: Participant Booklet, p. 18 Presenter Notes:

• Direct participants to record their initial thoughts and reactions to the training test items in the graphic organizer in their booklets.


Page 46

Slides Notes

As a Table Group:Debriefing the CAST Items

With your table group, begin to brainstorm a shared list.• What would students need to know and/or

experience to successfully respond to these items that is different than for pre-NGSS items?

• How are the demands different for discrete items as compared to items grouped in a performance task?

• What potential barriers might special populations and other students, in general, encounter in the items?

You may use the graphic organizer on page 19 in the Participant Booklet.

54

Slide 54—15 minutes Slide Intent: Think about and discuss the demands items like the CAST training test items place on students and teachers. Handout: Participant Booklet, p. 19 Presenter Notes:

• Give table groups three to four minutes to share their individual reflections and begin to discuss the focus questions.

• Give participants eight to ten minutes to begin to collect the group’s ideas on the focus questions. Remind them that there will be opportunities to revisit their list.

• Invite group spokespersons to share responses with the larger group.

Scoring Guide

Slide 55—30 seconds Slide Intent: Transition to the Scoring Guide Booklet. Handout: Training Test Scoring Guide Booklet; p.5 for grade five, p. 7 for grade eight, or p. 10 for high school items. Presenter Notes:

• Orient the participants to the Training Test Scoring Guide Booklet and the page where each grade level begins.

• Remind participants that they may have discussed evidence of student learning in each dimension as they deconstructed a PE. Note that scoring is a process of examining evidence to draw conclusions about students’ level of mastery of a PE.


Page 47

Slides Notes

Scoring Terms

56

Stimulus

Slide 56—1 minute Animated Slide Slide Intent: Review Scoring Guide terms. Handout: Participant Booklet Key Terms, p. 28 Presenter Notes:

• Review a short list of terms that are relevant to the Scoring Guide. Note that a more complete list of key terms and acronyms can be found on page 29 of the Participant Booklet. The sample item pictured is from the grade five training test.

• The stimulus is a short scenario associated with an item or set of items in a performance task.

• CLICK

Scoring Terms

56

Stimulus

Item

• The item is the question or prompt to which the student is responding.

• CLICK

Scoring Terms

56

Stimulus

Item

Exemplar and rubric

• The Scoring Guide Booklet includes answer keys for selected response and technology-enhanced items along with rubrics and exemplar responses for constructed response items. This example is on page 5 of the Training Test Scoring Guide Booklet.

• Exemplars are good examples of a specific score point.

• A rubric provides a standardized approach to scoring student responses.

• CLICK


Page 48

Slides Notes

Scoring Terms

All terms are in Key Terms, page 28 in the Participant Booklet.

56

Stimulus

Item

Exemplar and rubric

• These terms and others can be found in the Key Terms and Acronyms on page 29 of the Participant Booklet.

Scoring Your Responses

Use the Training Test Scoring Guide booklet to score your responses to the

items.

57

Slide 57—8 minutes Slide Intent: Shift focus to using the Scoring Guide to self-score. Handout: Training Test Scoring Guide Booklet Presenter Notes:

• Orient participants to the Scoring Guide in terms of the different formats for the different item types.

• Give participants six to seven minutes to score their own responses.

As a Table Group:Debriefing the Scoring Experience

• Discuss your scores for the constructed response item (grades 5 and 8) or items (high school).

• Based on the scoring guide, what do students need to know and be able to do to successfully complete items like the training test items?

• Edit or annotate your shared list as needed based on the table discussion.


58

Slide 58—12 minutes Slide Intent: Table groups participate in a discussion of the scoring experience. Handout: Participant Booklet, p. 20; Training Test Scoring Guide Booklet Presenter Notes:

• Give table groups four to five minutes to discuss their scores for the constructed response items (one each in grades five and eight, two in high school).

• Give participants six to seven minutes to revisit and edit their lists as needed. Remind them that there will be additional opportunities for editing the lists.

• Ask spokespersons to share any key discussion points or concerns that came up for the group.


Page 49

Slides Notes

Aligning the CAST Training Test Items

For each item in the grade-level Training Test, fill in the Aligning the Task (#2) handout to indicate: • Which of the SEPs are students likely to use?• Which DCIs will students need to use? • Which of the CCCs are students likely use?

Refer to the handout, Dimensions of the CA NGSS (#1), for the list of SEPs, DCIs, and CCCs.

59

Slide 59—10 minutes Slide Intent: Orient participants to the process for judging alignment of the CAST training test items. Handout: Training Test Items Booklet; Loose Handouts—Aligning the Training Test Items and Dimensions of the CA NGSS Presenter Notes:

• Orient participants to the materials for this activity. Note that the sheet for recording their alignment judgments is separate from the booklets to reduce the amount of page flipping needed. Remind them of the loose handout, Dimensions of the CA NGSS, as a reference for the lists of SEPs, DCIs, and CCCs.

• Each item should be an integration of two to three dimensions of a single PE. Remind participants that there are intentional overlaps and interconnections among the SEPs, CCCs, and DCIs, as they discovered during the earlier deconstruction activity. Note that a single item is unlikely to assess all components of any given dimension.

• Establish an understanding of alignment as a judgmental process with differences of opinions likely and expected.

As a Table Group:Alignment Discussion

Discuss your alignments for each item and compare to the alignments in the Training Test Scoring Guide.

• Remember—the overlapping and integrated nature of the dimensions means it’s sometimes possible to see more than one alignment.

Edit or annotate your shared list as needed based on the table discussion.

60

Slide 60—12 minutes Slide Intent: Shift from individual work to a table group discussion of alignment. Handout: Training Test Scoring Guide Booklet; p.14 for grade five, p. 15 for grade eight, or p. 16 for high school items. Presenter Notes:

• Orient participants to the pages in the Scoring Guide Booklet with the alignment information.

• Ask participants to compare their alignment judgments with others in the table group and to the alignments noted in the Scoring Guide.


Page 50

Slides Notes • Invite the table groups to revisit and edit their

shared list, as needed. • Ask spokespersons to share any key discussion

points or concerns that came up for the group.

Reflect: Instructional Implications of the Scoring Process

Think about the training test items for your grade level and look back at your table’s shared list. What kinds of learning experiences and assessment

opportunities would support students’ success on items like these? Make some notes on the graphic organizer on page 21

in the Participant Booklet and be ready to share with your table in Session 4.

61

Slide 61—8 minutes Slide Intent: Shift participant focus to individual reflection on the implications of the items and scoring process. Handout: Participant Booklet, p. 21 Presenter Notes:

• Remind participants of the expectation set at the beginning of this session to start thinking about the instruction implications of items like the CAST training test items.

• Note that time for sharing these reflections will happen in Session 4.

• Allow approximately eight minutes for participants to record their thoughts on the graphic organizer.

Session 4: Implications of the CAST Item Types and Test Design for Teaching and Learning


Page 51

Session 4: Implications of the CAST Item Types and Test Design for Teaching and Learning Slides 62–70 60 min.

Objectives • Reflect on the instructional implications and demands for teaching and learning to

demonstrate success on items like those illustrated in the CAST training test. • Consider ideas for developing CA NGSS-aligned lessons, formative assessment

opportunities, and other types of assessments that will support all students’ learning and performance on the CAST.



Slide 62—30 seconds Slide Intent: Shift participant focus to the final session. Presenter Notes:

• This session will be focused on considering implications of all of today’s materials.

• Provide a brief recap of all of activities in which participants have thus far engaged: deconstructing PEs; considering the role of phenomena; experiencing, scoring, and aligning CAST items. All along, participants have been reflecting on implications for teaching and learning. During this session, participants will apply all of their learning and reflections to developing specific ideas for developing and implementing learning experiences that will help their students achieve the goals of the CA NGSS and succeed on the CAST.



Page 52

Slides Notes

Objectives

• Reflect on the instructional implications and demands for teaching and learning to demonstrate success on items like those illustrated by the CAST Training Test.

63

Slide 63—30 seconds Animated Slide Slide Intent: Orient participants to the objectives for Session 4. Presenter Notes:

• Table groups will share and discuss the implications and demands for teaching and learning illustrated by the CAST training test item types and the CAST design, specifically the inclusion of performance tasks.

• CLICK

Objectives

• Reflect on the instructional implications and demands for teaching and learning to demonstrate success on items like those illustrated by the CAST Training Test.

• Consider ideas for developing CA NGSS–aligned lessons, formative assessment opportunities, and other types of assessments that will support all students’ learning and performance on the CAST.

63

• Participants will work together, in both table groups and as a larger group, to share and consider ideas for CA NGSS-aligned lessons, formative assessment opportunities, and other assessments that will support the learning of all students.

As a Table Group:Share Instructional Implications

64

Share your reflections:• What kinds of learning experiences and assessment

opportunities would support students’ success on items like these?

Review the loose handout #3, Instructional Shifts Required by the CA NGSS.

• Discuss how these shifts align with your thinking about the learning experiences and assessment opportunities that would support your students.


Slide 64—10 minutes Slide Intent: Share and discuss individual reflections from the end of Session 3. Handout: Participant Booklet, p. 23; Loose Handout—Instructional Shifts #3 Presenter Notes:

• Table leaders should initiate the discussion and ensure all table members have an opportunity to share.

• Consider asking table groups to share a reflection or an “aha” with the larger group, if time allows.

• Note that they are going to be asked to make posters with ideas for learning experiences, so be sure to keep track of their ideas.

• If participants wish to access additional detailed information regarding instructional strategies for the



Page 53

Slides Notes NGSS, refer to chapter 11 of the 2016 Science Framework (https://www.cde.ca.gov/ci/sc/cf/scifwprepubversion.asp).

As a Table Group:Learning Experiences and Assessment Opportunities Poster

65

Make a poster of ideas for learning experiences and assessment opportunities that support student progress toward the learning goals of the CA NGSS Briefly describe ideas for 1–2 new learning experiences

you can implement for students in your setting. Describe how each new learning experience includes the

use of a phenomenon and the integration of the three dimensions of the CA NGSS. Describe ideas for using assessments that can scaffold

your students’ learning in the context of the new experiences.

Slide 65—15 minutes Slide Intent: Set expectations for table group creation of posters with ideas for learning and assessment experiences. Handout: Participant Booklet, p. 24 Presenter Notes:

• Ask each table group to capture their ideas for learning experiences and assessment opportunities on a poster.

• Remind them to include adequate detail in their poster for their colleagues at other tables to understand what the ideas entail.

Visit each poster• Identify ideas that seem most relevant to your students

and/or instructional context. Note these ideas in the graphic organizer on page 25 in the Participant Booklet.

• Use a sticky note to provide constructive feedback.

Return to your own poster• Review the feedback provided to your team.• Share ideas identified during the gallery walk that seem

most relevant. • Discuss any additions and/or revisions to your poster.

Gallery Walk

66

Slide 66—17 minutes Slide Intent: Set the expectations for reviewing ideas from other table groups and for providing and receiving feedback. Handout: Participant Booklet, p. 25 Presenter Notes:

• Participants should prepare for the gallery walk by carrying sticky notes and a pen for leaving comments on other tables’ posters. They may also want to carry the Participant Booklet for note-taking.

• If time allows and there are multiple grade levels represented by participants, they may wish to visit posters for grade levels other than the one they have been working on to look for key similarities and differences across the grade levels.

• Inform the participants that they will have approximately 15 minutes to complete their walk and be back at their tables.





Page 54

Slides Notes • Table group discussions should begin once all

table group members have returned. • Dependent upon the group’s size and grade-level

makeup, presenters may wish to consider the content of this slide prior to delivering the workshop. As an alternative or an addition to the gallery walk, table groups could be called upon to present their posters to the larger group. Edit the slide as appropriate to your delivery context.

Closing Thoughts and Takeaways

Reflect on today’s session and discuss with your table group:• What strategies did you learn that are most relevant to

your classroom instruction or context? • What ideas will you take back to the classroom, school,

and/or district? • What actions can you take to enhance your capacity to

support effective teaching and learning of the CA NGSS? Share ideas with larger group.You may use the graphic organizer on page 26 of the Participant Booklet.

67

Reflect

Talk Listen

Slide 67—15 minutes Slide Intent: Transition to wrapping up the day’s learning. Handout: Participant Booklet, p. 26 Presenter Notes:

• Direct table groups to discuss the ideas and strategies that they plan to take back to their classroom, school, and/or local educational agency. Be prepared to share at least one of the ideas with the large group.

• After five minutes, ask table spokespersons to share out to larger group.

• Presenters may wish to consider the content of this slide prior to delivering this workshop. Dependent upon the group’s level of mastery/expertise, consider changing questions to more deeply probe reactions to the activities.

Additional CAST Resources

68

California Science Standardshttps://www.cde.ca.gov/pd/ca/sc/ngssintrod.asp

2016 California Science Frameworkhttps://www.cde.ca.gov/ci/sc/cf/scifwprepubversion.asp

California Science Testhttps://www.cde.ca.gov/ta/tg/ca/caasppscience.asp

California Science Test Informational Videohttps://www.cde.ca.gov/ta/tg/ca/castpilotvideo.asp

California Science Test Administrationhttp://www.caaspp.org/administration/about/science/

CAST Training Testhttp://www.caaspp.org/practice-and-training/

CAST Training Items Scoring Guideshttp://www.caaspp.org/ta-resources/practice-training.html

Slide 68—1 minute Slide Intent: Provide participants with links to CAST resources. Handout: Participant Booklet, p. 27 Presenter Notes:

• Refer participants to resources found on the California Department of Education’s Web site.



Page 55

Slides Notes

Copyright Restrictions

Permission is granted in advance for reproduction of these resources, in part or in their entirety, for the explicit use by educators to better understand the California Science Test. For any other purpose (including commercial enterprises), this resource, or portions thereof, may not be reproduced without permission from the California Department of Education. To obtain permission to reproduce the information (text or graphics) contained in this document, submit the specifics of your request in writing to the Copyright Program Office, Communications Division, California Department of Education, 1430 N Street, Suite 5602, Sacramento, CA 95814. Fax: 916-319-0100.

69

Slide 69—30 seconds Slide Intent: Provide information on the copyright restrictions on the use of this presentation.

Thank you!

69

Slide 70—15 seconds Slide Intent: Adjourn the workshop.

2018 CAST Academy—Facilitator's Guide - CAASPP · 2018-06-05 · Page 1 Facilitator Guide for California Science Test (CAST) Academy Workshop—Spring 2018 Introduction This workshop

Documents