Formative Assessment Institute
Barb Rowenhorst
Jackie Jessop Rising
Janet Hensley
Jennifer Nehl
Pam Lange
Outcomes• Integrate the Seven Strategies of Assessment FOR Learning
into effective assessment practices.
• Align benchmarks to classroom assessments.
• Design and evaluate multiple-choice items for classroom use.
• Understand how to analyze student multiple choice responses to guide instruction and learning.
• Use knowledge of quality questions to help guide analysis of student responses.
• Develop a plan for “next steps” for the student and teacher.
AgendaTuesday, September 23, 2008
7:00 to 8:00 am Continental Breakfast8:00 to Noon Welcome
Year Two VisionAssessment IQAssessment Dialogue/Development
Noon-1:00 Lunch1:00-4:00 pm Assessment Development/Work time
Wednesday, September 247:00 to 8:00 a.m. Continental Breakfast8:00 to 10:15 Using Assessment Results10:15 to 10:30 Break10:30 to Noon Cassie Erkins Noon-12:45 Lunch12:45 to 3:00 Cassie Erkins Continued3:00 – 3:15 Homework Assignment/Closure
Credit Options
Remember to sign in each session
• PTSB Credit
• Graduate Credit – University of Wyoming
• See Deanna
http://fai.tie.wikispaces.net/
Dates: 2008-2009 Trainings
Formative Assessment Institute
September 23, 24, 2008
Selected Response (MC/TF/Short
Answer)
November 10-11, 2008
Rubrics/Strong and Weak Student Work
February 11-12, 2009
Extended Written Response
March 30-31, 2009 Performance Assessment
OF and FOR Assessment - Purpose
Assessment OF Learning (Summative Assessment):How much have students learned as of a at a particular point in time?
Assessment FOR Learning (Formative Assessment):How can we use assessments to help us with instruction so students learn more?
Divide into groups
Which is Which?
It isn’t the method that determines
whether the assessment is summative or formative…
it is how the results
are used.
Seven Strategies for Assessment FOR Learning
Where am I going?1.Provide a clear and understandable vision of the learning target.2.Use examples and models of strong and weak work.
Where am I now?3. Offer regular descriptive feedback.4. Teach students to self-assess and set goals.
How can I close the gap?5. Design lessons to focus on one aspect of quality at a time.6. Teach students focused revision.7. Engage students in self-reflection and let them keep track of and share their learning.
Kinds of Learning Targets
Knowledge targets– the facts and concepts we want students to
know.
Reasoning targets– students use what they know to reason and
solve problems.
Skill targets– students use their knowledge and reasoning to
act skillfully.
Product targets– students use their knowledge, reasoning, and
skills to create a concrete product.
Dispositional targets– students attitudes about school and learning.
Benchmark
Learning Target
Learning Target
Learning Target
Learning Target
Learning Target
Learning Target
StudentsStudents
Move Move
UpUp
The The
LadderLadder
ThroughThrough
AA
SequenceSequence
OfOf
LearningLearning
StepsSteps
Assessment OFOF Learning
Assessments FORFOR
Learning
Barb
Assessment IQ
Assessment IQ
How many did you get
right?
Assessment IQ
• “New Assessment Beliefs for a New School Mission” (Stiggins)
• Posted on FAI Wiki/DVDwww.assessmentinst.com/forms/NewBeliefs.
Janet
Essential Question
As educators, what information are we really getting from the assessments we give our students?
What information are our students receiving?
A Journey into the Past!Share with your group
a BAD assessment experience you had
when you were a student.
Why do you remember it?
Now identify a GOOD assessment
from your past. What made it good?
Discussion: Test you took in the past.Bad Assessment-Common Traits Good Assessment-Common
Traits
As you look at each trait from the assessments, analyze them for evidence of
the seven strategies. Ex: “Was there a clear and understandable vision of the
learning target to the student?”
Where am I going?Strategy 1
Provide a clear and understandable vision of the learning target.• Share the learning target(s) in advance
of the lesson.• Use student-friendly language.• Check for understanding.
Where am I going?Strategy 2
Use and post student examples/models of strong and weak work (keep anonymous).• Share models of student work.• Allow students to score samples.• Demonstrate the process of creating a
performance, project, etc.
Where am I now?Strategy 3
Offer students frequent, descriptive feedback.
• Descriptive feedback directly reflects the learning target(s).
• Address the targets they are doing well and their next steps toward achieving their learning goal.
Where am I now?Strategy 4
Teach students to self-assess and set goals.
• Self-assessment is a necessary part of the learning process.
• Students can identify their strengths and the areas in which they need to improve based upon examples/models of proficient work.
How can I close the gap?Strategy 5
Design lessons that focus on one learning target at a time.
• Differentiate student instruction.• Develop confidence before moving on.• Be careful not to reach beyond the
next sequential learning target.
How can I close the gap?Strategy 6
Teach students focused self-revision.
• Teachers should model revision practice using strategies and examples.
• Students should qualify their responses in order to isolate learning targets.
How can I close the gap?Strategy 7
Engage students in self-reflection, and allow for student record keeping.
• Students need to share their progress with all assessment users.
• The teaching, learning, assessment cycle becomes a partnership.
Discussion: Test you took in the past.Bad Assessment-Common Traits Good Assessment-Common
Traits
As you look at each trait from the assessments, analyze them for evidence of
the seven strategies. Ex: “Was there a clear and understandable vision of the
learning target to the student?”
Target Assessment Match
Page 100
Assessment Method
Target to be
Assessed
Selected Response
Extended Written
Response
Performance Assessment
Personal Communicatio
n
Knowledge Mastery
Good match Good match Not so good Works, but not the best option; inefficient
Reasoning Proficiency
Good match Very good match Can work Works best as follow up question
to reasoning
Skills (Performance)
Not a good match
Not a good match Good match Very strong match
Ability to Create Products
Not a good match
Very strong match
Good match Not so good
Links Among Achievement Targets and Assessment Methods
Adapted from Table 4.1 p. 100
Cognitive Levels of Questioning
When designing questions, we need to look at the different
cognitive levels and assure that we are assessing all levels.
– Bloom’s 6 levels of questioning
– Webb (Norman Webb’s 4 levels)
Self-Assessment of Skill Level
Pick a unit from the one(s) you brought:
• Estimate the levels being assessed. • Do this by calculating a rough percentage of
Bloom’s and/or Webb. – Ex. 20% Knowledge, 30% Comprehension, 40%
Application– Ex. Webb Level 1 = 30%, Webb Level 2 = 40%
Side Note: 80% of questions in the classroom are level 1
80% of benchmarks are level 2 and 3
Reminder:
•It is necessary to have deconstructed your benchmarks.
•Students need to have the benchmarks in ‘student friendly language.’
Selected Response
•Multiple Choice
•True/False
•Short Response
Quality Multiple Choice Items
A multiple choice item consists of three parts:
• the stem (the question)• the key (correct answer)• the distracters (plausible choices)
Multiple Choice Item Development
Considerations:
• Correct response provided
• Meaningful distracters
• Not limited to “lower order skills” (Webb level or Bloom’s
level)
Multiple Choice Example
Which measurement unit below would be best to use if you wanted to measure the mass of a bumblebee?
A. gram
B. kilogram
C. millimeter
D. centimeter
Multiple Choice Item Development
Multiple Choice Item Development
DemonstrationStep 1
• Identify a standard to be assessed.
Step 2
• Analyze the standard for:
• Webb level
• Bloom’s level
Step 3
• Identify the context for the item.
• Identify any stimuli (text or graphics).
• Real-world contexts are more engaging for students (such as real problems that relate to Wyoming, sports, entertainers etc.)
• Stimuli such as graphics should clarify the stem for students (not be just visually appealing).
Multiple Choice Item Development
Demonstration
Step 4
Write a draft stem and write the correct answer.
• Stems should pose a single problem, although the solution may require multiple steps.
• Avoid the use of negatives in the stem.
• Stems should be clearly written and as concise as possible.
Multiple Choice Item Development
Demonstration
Step 5
Identify three distracters that contain common errors students would make.
• Be sure none of the distracters is a possible answer.
• Avoid clues in the distracters.
• Make sure all of the options are comparable in length, complexity, and grammatical form. (!!)
• All options should be in logical order.
Multiple Choice Item Development
Demonstration
Step 6
Review the item.
• Does it align with the benchmark?
• Does it align with Webb levels or Bloom’s levels?
• Is the correct response accurate?
• Are the distracters appropriate?
Multiple Choice Item Development
Demonstration
Things to Watch . . .
• Don’t throw in pictures.
• Only use bold when necessary.
• Don’t use italics – hard to read.
• Grades K-3
• Stems should be a complete sentence (not unfinished sentences).
You Try It . . .In groups of 2 or 3, use the Multiple-Choice Item
Checklist to work with your test items.
Multiple-Choice Item Development
You Try ItUsing practice handout, design a multiple-choice item with a partner.
•Use checklist to analyze a question.
•Choose one question per table to record on chart paper. Record benchmark and the question.
•Post on wall.
Multiple-Choice Item Development
• Do a gallery walk and provide descriptive feedback on post-it notes for three posted questions.
• Review the observations of others and generate “ah-ha’s” to share with whole group.
• Dialogue at table
• Share out
Multiple-Choice Item Development
You Try It
Multiple Choice Item Development
Independent Practice Template
Individual or in teams, work with unit assessment items.
Work Time
30 Minutes
30 minutes
Stop and have at least three people look at your
questions.
Additional Handouts:
True/False Questions
Short Answer Questions
Work Time
Formative Assessment Institute
Welcome Back
Item AnalysisMultiple Choice
Assessment
Barb Rowenhorst
Item Analysis
• As educators, what information are we really getting from the assessments we give our students?
Questions a Teacher May Want
Answered by Assessments• Are my students succeeding?
• Are my students improving?
• Is student success or non-success because of my teaching?
• What does this student need?
• What do these students need?
• How should I group the students?
• How can I improve?
Adapted from Stiggins (2001)
Problem Learning Target Right? Wrong?Simple
mistake?More
study?
1Place Value: Write numerals in expanded form to 10 thousands place.
x
2Place Value: Write numerals in expanded form to 10 thousands place.
x
3Place Value: Write numerals in expanded form to 10 thousands place.
x
4Place Value: Identify place value to the thousands place.
x
5Place Value: Put numbers in order through the thousands.
x
6Place Value: Put numbers in order through the thousands.
x
7Place Value: Put numbers in order through the thousands.
x x
Problem Learning Target Right? Wrong?Simple
mistake?More
study?
8 Write fractions to match models.
x
9Write fractions to match models. x x
10Write fractions to match models. x
11Write fractions to match models. x x
12Subtract 3-digit numbers with borrowing. x
13Subtract 3-digit numbers with borrowing. x x
14Subtract 3-digit numbers with borrowing. x
15Subtract 3-digit numbers with borrowing. x x
Item Analysis
• Michael Fullan has been saying for years that successful schools are places where teams of teachers meet regularly to focus on student work through assessment and change their instructional practice accordingly to get better results (Fullan, 2000).
Item Analysis: Scenario
Item Analysis: Example
Item Analysis: Example
Item Analysis: Practice
Using the scored assessment you brought…
• Determine whether you want to analyze the test, a portion of the test, or certain questions.
• Use the handout to practice analyzing your students’ responses.
Item Analysis: Practice
Item Analysis Sharing• Find someone that is not in your district.
• Share the “aha’s” you had as you analyzed your multiple choice assessment.
• What are some ways you might group students when re-teaching or clarifying content?
• How did this process help analyze student responses?
Questions a Student May Want Answered by
Assessments• Am I succeeding?
• Am I improving?
• What do I need to do to succeed here?
• What should I do next to succeed?
• Am I capable of success?
• Is learning worth the effort?
• Where do I want all this schooling to take me?
Adapted from Stiggins (2001)
I am good at these!
I am pretty good at these, but need to do a little review.
I need to keep learning these.
You Be George
Student Involvement
Crafting & Analyzing Assessments
• Improves monitoring of students’ progress so teaching can be better planned
• Communicates to students what the teacher values
• Clarifies what the teacher’s learning targets are
• Allows improvement of assessments already given in the books
• Frees teacher to design lessons that are not chained to set assessments
• Improves the validity of teacher’s assessments
Adapted from Nitko (2001)
Break
Please be back at 10:30 sharp!
Cassie Erkins
FAI Homework
Need snapshot when Becky finishes
Dates: 2008-2009 Trainings
Formative Assessment Institute
September 23, 24, 2008
Selected Response (MC/TF/Short
Answer)
November 10-11, 2008
Homework/Rubrics/ Strong and Weak
Student Work
February 11-12, 2009
Homework/Extended Written Response
March 30-31, 2009 Performance Assessment
http://fai.tie.wikispaces.net/
Safe Travels