CTB CADDS Sally Valenzuela Director, Publishing Strategic Initiatives CTB/McGraw-Hill

CTB CADDSSally ValenzuelaDirector, Publishing Strategic InitiativesCTB/McGraw-Hill

2

Abstract

Evidence-based item development approaches require item authors to make explicit how items provide validity evidence to support claims.

Advances in technology require increased consideration of accessibility and interoperability during item authoring.

Refined item development processes are needed to meet these requirements.

3

CADDS

Serves as a bridge between standards and assessment

Provides opportunities for systematic unpacking of standards for item development

Documents steps in reaching assessment goals Maintains focus at all steps on evidence to support

interpretations and uses of test information

4

CADDS

Details information about the content to be assessed

Describes item types that will be used Uses detailed specifications for each item/task Increases direction for item writers

5

CADDS

1. Define the intended inferences and decisions to be based on test scores.

2. Define the achievement construct.3. Draft performance level descriptors (expectations of

students).4. Define the evidence to be elicited by the item pool.5. Complete item writer assignments to meet the item

pool specification requirements.6. Complete item creation (authoring and editing).7. Field test assessment items and tasks.8. Implement the operational test.

6

INFERENCE about students

(Step 1)

Achievement Construct (Step 2)

Assessment item/task

creation (Step 6)

Item Writer Assignments

(Step 5)

Performance Level

Descriptors (Step 3)

Item Pool Evidence(Step 4)

Pilot, field, operational

testing (Step 7/8)

DESIGNDEVELOPMENT

7

CADDS

1.Define intended inferences and uses of the assessment data.

• Adoption of CCSS has created need for transition assessments.

• Test designs are incorporating new item types.• Reporting requirements are changing.• “Alignment” issues abound.

8

CADDS

2.Define the test construct(s) that will become assessment targets.

• What cognitive tasks are required by the standard?• How do we consider students progression of learning?• What is the instructional context for a given standard?• How do we incorporate performance levels?

9

CADDS

3.Develop initial proficiency level descriptors to guide development and interpretation of test scores.

• Identify source(s) of PLDS.• Define the role of PLDs in item specifications and authoring.

10

CADDS

4.Define the evidence.1. Item pool or test blueprint2. Specifications3. Item/task templates4. Instructions to item writers

5.Develop items and performance tasks based on specifications.1. specifications templates2. cognitive task frameworks3. tagging for accessibility and interoperability

11

CADDS

6.Refine items and tasks through collaborative review by stakeholders.

7.Field test items and tasks in appropriate small- or large-scale settings.

8.Implement the operational test and continue the design and specification validation process.

12

Implementation

Expansion of item specifications template

Performance description Rules for source materialsPerformance level descriptors Rules for item/task problem

Grade level placement Rules for response requirements

Assessment targets/standards Administration requirements21st century skills Accessibility requirementsCognitive rigor Administrator directionsProblem/processing type Technology requirementsPerformance description Rules for source materials

13

Implementation Articulation of cognitive tasks during item

development Cognitive Task Frameworks

• Traditional (DOK, Bloom’s)• Within CCSS (Conley)

– Mathematical Practice Standards– Selected ELA standards

• Cognitive Rigor Matrix (Hess)

14

Item Authoring

Clearly defined elements for authors• construct evidence requirements• parameters for student responses• options for item type, cognitive demand, other variable

factors Articulated focus on evidence, validity and

accessibility

15

CCSS StandardModeling with Geometry G-MGApply Geometric Concepts in Modeling Situations.

Model with Mathematics

Make sense of problems.

Attend to precision.

Item 1

Item 2

Item 3

Above Proficient

Proficient

Basic

Item 1

Item 2

Item 3

Above Proficient

Proficient

Basic

Item 1

Item 2

Item 3

Above Proficient

Proficient

Basic

16

Item Development Implications

Explicit articulation of evidence to support claims Rigorous specifications development More direction for item writers Clearer distinction among item attributes Refinement of item development plans