Formative Assessment Institute Barb Rowenhorst Jackie Jessop Rising Janet Hensley Jennifer Nehl Pam Lange.

Formative Assessment Institute

Barb Rowenhorst

Jackie Jessop Rising

Janet Hensley

Jennifer Nehl

Pam Lange

Outcomes• Integrate the Seven Strategies of Assessment FOR Learning

into effective assessment practices.

• Align benchmarks to classroom assessments.

• Design and evaluate multiple-choice items for classroom use.

• Understand how to analyze student multiple choice responses to guide instruction and learning.

• Use knowledge of quality questions to help guide analysis of student responses.

• Develop a plan for “next steps” for the student and teacher.

AgendaTuesday, September 23, 2008

7:00 to 8:00 am Continental Breakfast8:00 to Noon Welcome

Year Two VisionAssessment IQAssessment Dialogue/Development

Noon-1:00 Lunch1:00-4:00 pm Assessment Development/Work time

Wednesday, September 247:00 to 8:00 a.m. Continental Breakfast8:00 to 10:15 Using Assessment Results10:15 to 10:30 Break10:30 to Noon Cassie Erkins Noon-12:45 Lunch12:45 to 3:00 Cassie Erkins Continued3:00 – 3:15 Homework Assignment/Closure

Credit Options

Remember to sign in each session

• PTSB Credit

• Graduate Credit – University of Wyoming

• See Deanna

http://fai.tie.wikispaces.net/

Dates: 2008-2009 Trainings

Formative Assessment Institute

September 23, 24, 2008

Selected Response (MC/TF/Short

Answer)

November 10-11, 2008

Rubrics/Strong and Weak Student Work

February 11-12, 2009

Extended Written Response

March 30-31, 2009 Performance Assessment

OF and FOR Assessment - Purpose

Assessment OF Learning (Summative Assessment):How much have students learned as of a at a particular point in time?

Assessment FOR Learning (Formative Assessment):How can we use assessments to help us with instruction so students learn more?

Divide into groups

Which is Which?

It isn’t the method that determines

whether the assessment is summative or formative…

it is how the results

are used.

Seven Strategies for Assessment FOR Learning

Where am I going?1.Provide a clear and understandable vision of the learning target.2.Use examples and models of strong and weak work.

Where am I now?3. Offer regular descriptive feedback.4. Teach students to self-assess and set goals.

How can I close the gap?5. Design lessons to focus on one aspect of quality at a time.6. Teach students focused revision.7. Engage students in self-reflection and let them keep track of and share their learning.

Kinds of Learning Targets

Knowledge targets– the facts and concepts we want students to

know.

Reasoning targets– students use what they know to reason and

solve problems.

Skill targets– students use their knowledge and reasoning to

act skillfully.

Product targets– students use their knowledge, reasoning, and

skills to create a concrete product.

Dispositional targets– students attitudes about school and learning.

Benchmark

Learning Target

Learning Target

Learning Target

Learning Target

Learning Target

Learning Target

StudentsStudents

Move Move

UpUp

The The

LadderLadder

ThroughThrough

AA

SequenceSequence

OfOf

LearningLearning

StepsSteps

Assessment OFOF Learning

Assessments FORFOR

Learning

Barb

Assessment IQ

Assessment IQ

How many did you get

right?

Assessment IQ

• “New Assessment Beliefs for a New School Mission” (Stiggins)

• Posted on FAI Wiki/DVDwww.assessmentinst.com/forms/NewBeliefs.

pdf

Janet

Essential Question

As educators, what information are we really getting from the assessments we give our students?

What information are our students receiving?

A Journey into the Past!Share with your group

a BAD assessment experience you had

when you were a student.

Why do you remember it?

Now identify a GOOD assessment

from your past. What made it good?

Discussion: Test you took in the past.Bad Assessment-Common Traits Good Assessment-Common

Traits

As you look at each trait from the assessments, analyze them for evidence of

the seven strategies. Ex: “Was there a clear and understandable vision of the

learning target to the student?”

Where am I going?Strategy 1

Provide a clear and understandable vision of the learning target.• Share the learning target(s) in advance

of the lesson.• Use student-friendly language.• Check for understanding.

Where am I going?Strategy 2

Use and post student examples/models of strong and weak work (keep anonymous).• Share models of student work.• Allow students to score samples.• Demonstrate the process of creating a

performance, project, etc.

Where am I now?Strategy 3

Offer students frequent, descriptive feedback.

• Descriptive feedback directly reflects the learning target(s).

• Address the targets they are doing well and their next steps toward achieving their learning goal.

Where am I now?Strategy 4

Teach students to self-assess and set goals.

• Self-assessment is a necessary part of the learning process.

• Students can identify their strengths and the areas in which they need to improve based upon examples/models of proficient work.

How can I close the gap?Strategy 5

Design lessons that focus on one learning target at a time.

• Differentiate student instruction.• Develop confidence before moving on.• Be careful not to reach beyond the

next sequential learning target.

How can I close the gap?Strategy 6

Teach students focused self-revision.

• Teachers should model revision practice using strategies and examples.

• Students should qualify their responses in order to isolate learning targets.

How can I close the gap?Strategy 7

Engage students in self-reflection, and allow for student record keeping.

• Students need to share their progress with all assessment users.

• The teaching, learning, assessment cycle becomes a partnership.

Discussion: Test you took in the past.Bad Assessment-Common Traits Good Assessment-Common

Traits

As you look at each trait from the assessments, analyze them for evidence of

the seven strategies. Ex: “Was there a clear and understandable vision of the

learning target to the student?”

Target Assessment Match

Page 100

Assessment Method

Target to be

Assessed

Selected Response

Extended Written

Response

Performance Assessment

Personal Communicatio

n

Knowledge Mastery

Good match Good match Not so good Works, but not the best option; inefficient

Reasoning Proficiency

Good match Very good match Can work Works best as follow up question

to reasoning

Skills (Performance)

Not a good match

Not a good match Good match Very strong match

Ability to Create Products

Not a good match

Very strong match

Good match Not so good

Links Among Achievement Targets and Assessment Methods

Adapted from Table 4.1 p. 100

Cognitive Levels of Questioning

When designing questions, we need to look at the different

cognitive levels and assure that we are assessing all levels.

– Bloom’s 6 levels of questioning

– Webb (Norman Webb’s 4 levels)

Self-Assessment of Skill Level

Pick a unit from the one(s) you brought:

• Estimate the levels being assessed. • Do this by calculating a rough percentage of

Bloom’s and/or Webb. – Ex. 20% Knowledge, 30% Comprehension, 40%

Application– Ex. Webb Level 1 = 30%, Webb Level 2 = 40%

Side Note: 80% of questions in the classroom are level 1

80% of benchmarks are level 2 and 3

Reminder:

•It is necessary to have deconstructed your benchmarks.

•Students need to have the benchmarks in ‘student friendly language.’

Selected Response

•Multiple Choice

•True/False

•Short Response

Quality Multiple Choice Items

A multiple choice item consists of three parts:

• the stem (the question)• the key (correct answer)• the distracters (plausible choices)

Multiple Choice Item Development

Considerations:

• Correct response provided

• Meaningful distracters

• Not limited to “lower order skills” (Webb level or Bloom’s

level)

Multiple Choice Example

Which measurement unit below would be best to use if you wanted to measure the mass of a bumblebee?

A. gram

B. kilogram

C. millimeter

D. centimeter

Multiple Choice Item Development

Multiple Choice Item Development

DemonstrationStep 1

• Identify a standard to be assessed.

Step 2

• Analyze the standard for:

• Webb level

• Bloom’s level

Step 3

• Identify the context for the item.

• Identify any stimuli (text or graphics).

• Real-world contexts are more engaging for students (such as real problems that relate to Wyoming, sports, entertainers etc.)

• Stimuli such as graphics should clarify the stem for students (not be just visually appealing).

Multiple Choice Item Development

Demonstration

Step 4

Write a draft stem and write the correct answer.

• Stems should pose a single problem, although the solution may require multiple steps.

• Avoid the use of negatives in the stem.

• Stems should be clearly written and as concise as possible.

Multiple Choice Item Development

Demonstration

Step 5

Identify three distracters that contain common errors students would make.

• Be sure none of the distracters is a possible answer.

• Avoid clues in the distracters.

• Make sure all of the options are comparable in length, complexity, and grammatical form. (!!)

• All options should be in logical order.

Multiple Choice Item Development

Demonstration

Step 6

Review the item.

• Does it align with the benchmark?

• Does it align with Webb levels or Bloom’s levels?

• Is the correct response accurate?

• Are the distracters appropriate?

Multiple Choice Item Development

Demonstration

Things to Watch . . .

• Don’t throw in pictures.

• Only use bold when necessary.

• Don’t use italics – hard to read.

• Grades K-3

• Stems should be a complete sentence (not unfinished sentences).

You Try It . . .In groups of 2 or 3, use the Multiple-Choice Item

Checklist to work with your test items.

Multiple-Choice Item Development

You Try ItUsing practice handout, design a multiple-choice item with a partner.

•Use checklist to analyze a question.

•Choose one question per table to record on chart paper. Record benchmark and the question.

•Post on wall.

Multiple-Choice Item Development

• Do a gallery walk and provide descriptive feedback on post-it notes for three posted questions.

• Review the observations of others and generate “ah-ha’s” to share with whole group.

• Dialogue at table

• Share out

Multiple-Choice Item Development

You Try It

Multiple Choice Item Development

Independent Practice Template

Individual or in teams, work with unit assessment items.

Work Time

30 Minutes

30 minutes

Stop and have at least three people look at your

questions.

Additional Handouts:

True/False Questions

Short Answer Questions

Work Time

Formative Assessment Institute

Welcome Back

Item AnalysisMultiple Choice

Assessment

Barb Rowenhorst

Item Analysis

• As educators, what information are we really getting from the assessments we give our students?

Questions a Teacher May Want

Answered by Assessments• Are my students succeeding?

• Are my students improving?

• Is student success or non-success because of my teaching?

• What does this student need?

• What do these students need?

• How should I group the students?

• How can I improve?

Adapted from Stiggins (2001)

Problem Learning Target Right? Wrong?Simple

mistake?More

study?

1Place Value: Write numerals in expanded form to 10 thousands place.

x

2Place Value: Write numerals in expanded form to 10 thousands place.

x

3Place Value: Write numerals in expanded form to 10 thousands place.

x

4Place Value: Identify place value to the thousands place.

x

5Place Value: Put numbers in order through the thousands.

x

6Place Value: Put numbers in order through the thousands.

x

7Place Value: Put numbers in order through the thousands.

x x

Problem Learning Target Right? Wrong?Simple

mistake?More

study?

8 Write fractions to match models.

x

9Write fractions to match models. x x

10Write fractions to match models. x

11Write fractions to match models. x x

12Subtract 3-digit numbers with borrowing. x

13Subtract 3-digit numbers with borrowing. x x

14Subtract 3-digit numbers with borrowing. x

15Subtract 3-digit numbers with borrowing. x x

Item Analysis

• Michael Fullan has been saying for years that successful schools are places where teams of teachers meet regularly to focus on student work through assessment and change their instructional practice accordingly to get better results (Fullan, 2000).

Item Analysis: Scenario

Item Analysis: Example

Item Analysis: Example

Item Analysis: Practice

Using the scored assessment you brought…

• Determine whether you want to analyze the test, a portion of the test, or certain questions.

• Use the handout to practice analyzing your students’ responses.

Item Analysis: Practice

Item Analysis Sharing• Find someone that is not in your district.

• Share the “aha’s” you had as you analyzed your multiple choice assessment.

• What are some ways you might group students when re-teaching or clarifying content?

• How did this process help analyze student responses?

Questions a Student May Want Answered by

Assessments• Am I succeeding?

• Am I improving?

• What do I need to do to succeed here?

• What should I do next to succeed?

• Am I capable of success?

• Is learning worth the effort?

• Where do I want all this schooling to take me?

Adapted from Stiggins (2001)

I am good at these!

I am pretty good at these, but need to do a little review.

I need to keep learning these.

You Be George

Student Involvement

Crafting & Analyzing Assessments

• Improves monitoring of students’ progress so teaching can be better planned

• Communicates to students what the teacher values

• Clarifies what the teacher’s learning targets are

• Allows improvement of assessments already given in the books

• Frees teacher to design lessons that are not chained to set assessments

• Improves the validity of teacher’s assessments

Adapted from Nitko (2001)

Break

Please be back at 10:30 sharp!

Cassie Erkins

FAI Homework

Need snapshot when Becky finishes

Dates: 2008-2009 Trainings

Formative Assessment Institute

September 23, 24, 2008

Selected Response (MC/TF/Short

Answer)

November 10-11, 2008

Homework/Rubrics/ Strong and Weak

Student Work

February 11-12, 2009

Homework/Extended Written Response

March 30-31, 2009 Performance Assessment

http://fai.tie.wikispaces.net/

Safe Travels