Assessing your assessment misd

How Do You Know?

or “How do I know you

know what I want you to

know, you know?”

Our Goals Today•To collect strategies and examples to help you better assess your students’ learning.•To develop a deeper understanding of assessment to help your students perform better on many types of assessments.

What do you rate, rank or score?

•Interest Rates•Sports Teams•Consumer Reports•Gas Mileage•Best & Worst Dressed

What is ranked or rated in education?

•States•Districts•Schools•Teachers•Students

Does assessment matter?

•Who values assessment? •What is its purpose?•Can we make it matter?

We value what We value what we assess and we assess and we assess what we assess what

we value.we value.

What is Assessment?

“Any standardized procedure for eliciting

the kind of behavior we want to observe or

measure.” (Frederiksen, 1984)

Assessment is a double-edged

sword.Reported scores get

attention – whether it’s good attention or

bad attention.

Have you heard kids say…

Will this be on the test?

Hurry and give me the test before I forget what I studied!

Why do I need to know this?

What’s Our Purpose?TAKS Test

Unit Test

Benchmark Test

Informal Assessment

Engaging Test Items Include

•Some kind of hook that captures students’ interest

•Some real world connection

•Information that the student may learn while taking the test

Thought Provoking Test Items

•Require the use of both knowledge and skills in order to arrive at the answer.

•Go beyond reading and recalling.

Well Constructed Test Items

•Do not give away the answer.

•Do not favor the test-wise student.

•Do not favor the good reader over the good student.

•Match the objective.

Well-Constructed Test Items

• Avoid the use of negative words in the stem and/or options.

• Have responses in logical order.

• Have one clearly correct answer.

• Are factually correct.

Well-Constructed Options

•Are free of excess repetition

•Are parallel in degree of specificity

•Are parallel in length

•Are parallel in grammatical structure

Well-Constructed Options

•Do not subsume other options

•Are not the opposite of one another

Well Constructed Distractors

•Are plausible to the naive student, i.e., there is a rationale for why a student might choose the distractors

Take a 6 Minute Break

Sample Questions

Sample Science Question

Electrical fires are NOT extinguished with—A waterB foamC dry chemicalsD carbon dioxide

Improved Science Question

If the electric pencil sharpener in class began to burn, water would be a poor choice to extinguish the fire because water—A is a conductorB may not be nearbyC will ruin classroom cabinetsD is the universal solvent

Use “NOT” sparingly

The discovery of what natural resource in California in 1848 and 1849 caused rapid population growth in the region?

A. Gold B. Silver C. Copper D. Iron

Sample S. S. Question

TEKS requires analysis of effect, not simple identification.

What effect did the discovery of gold in California in 1848 and 1849 have on people in the U.S.? It resulted in—

A. massive population growth in that regionB. decreases in silver and iron productionC. increases of copper and tin productionD. record high employment numbers

Improved S. S. Question

Higher level question rather than straight recall.

These people served as public officials during the Civil War. Which of the following matches is incorrect?

Jefferson Davis — secretary of state for the Confederacy Ulysses S. Grant — commander of the Union army Robert E. Lee — commander of the Confederate army Abraham Lincoln — president of the United States

Sample S. S. Question

Low level question asking for which is incorrect.

Robert E. Lee was a significant individual during the Civil War because he was—

A. president of the United StatesB. commander of the union armyC. commander of the confederate armyD. secretary of state for the confederacy

Improved S. S. Question

Higher level question asking for a generalization.

Sample Literary Question

Which answer means “applying human characteristics to nonhuman objects?”A hyperboleB personificationC onomatopoeiaD simile

Improved Literary Question

Which sentence is an example of personification?A He has a brain the size of a pea.B Playing the piano is like a bird soaring high.C The playful dog greeted us with his bow-wow.D The flowers were suffering from the immense heat.

Application

A Christmas Carol-Scrooge is able to change because— A. Marley puts a spell on himB. he learns to feel for those around himC. he is afraid of dyingD. it is the Christmas season

Sample L. A. Question

Does not have one clearly correct answer.

Scrooge’s change in attitude at the end of the story is caused by—

A. his fear of dyingB. his new found compassion for othersC. his love of moneyD. his experience visiting “Christmas Past”

Improved L. A. Question

When Mrs. Avery discovers that her son T.J. went to the forbidden Wallace store, T.J. lies, saying he went only to retrieve his brother Claude. As a result, Mrs. Avery whips Claude. That Claude allows his mother to punish him for his brother’s lie shows that—

A. Claude is not very brightB. Claude is more afraid of T.J. than of his own motherC. T.J. is a responsible son and brotherD. Mrs. Avery is not very tough when she whips her children

Sample L. A. Question

Favors the good reader.

Claude allows his mother to punish him for his brother’s lie. This shows that Claude is—

A. not very brightB. more afraid of T.J. than of his own motherC. a responsible son and brotherD. a silly child

Improved L. A. Question

From Older Run—From her actions on page 160 we can infer that one of Cookie’s characteristics is that she is—

A. a followerB. loyal C. meanD. scared

Sample L. A. Question

Choices are not grammatically parallel.

From Older Run—From her actions on page 160 we can infer that one of Cookie’s characteristics is that she is—

A. weakB. loyal C. meanD. scared

Improved L. A. Question

What is the predominant literary device used in “Oh Captain, My Captain”?

A. Extended metaphorB. DictionC. RepetitionD. Allusion

Sample L. A. Question

Favors the test-wise student.

The title and contents of the poem, “Oh Captain, My Captain,” is an example of—

A. Extended metaphorB. DictionC. RepetitionD. Allusion

Improved L. A. Question

The newspaper articles in Lincoln’s wallet suggest that he—

A. never realized that he was dislikedB. never realized he was admiredC. cared about what others thoughtD. knew he would go down in history as a great president

Sample L. A. Question

Answer choices A & B subsume one another

Assessment

We must assess the TEKS (Student Expectations) to the Depth and Complexity of the TAKS.

The verbs in the student expectations tell us the level to which the student should achieve.

How to Write an Aligned Question

Identify TEKS/SE for Development of Questions:

Use sample and released questions as one guide for development.

Look to grade levels above for other examples.

Use open-ended examples.Read the verbs of the SEs and interpret the

meaning.

How to Write an Aligned Question

Identify Reading Selections

• Narrative

• Informative

• Functional

Sometimes use those with a theme connection.

Steps in Thinking

Identify steps required for correct answer.Identify level of question:• Find it – answer is stated• Look closer – Stated but difficult to locate• Prove it – Inferred or implied based on

clues• Take it apart – viewing reading from a

writing perspective through literary analysis and author’s perspective.

Write Questions

Use good question stems that mirror TAKS.

Write the correct answer and the text evidence required for each answer.

Write the corresponding TEKS/SE.

TAKS Open-Ended Response

Most of them deal with character analysis:

• Actions

• Personal conflict

• Point of view

• Trait

• Attitude about relationships

TAKS Types of Passages

In Junior High, passages are about 50/50 Fiction vs. Nonfiction (Informative)

Paired passages may be same of both or one of each.

Most Significant Errors (All Tests)

#1 Students use prior knowledge not connected to information in passage.

#2 Students “match” information on an inference question.

#3 Students use personal experiences.

#4 Students substitute answer choices for word meaning rather than using context.

Question Starters• Predict “What will happen if…?”• Evaluate “Which of these best

explains…?”• Apply “Which of these is required to …?”• Analyze “What were the effects of …

on…?• Compare “How is … like (different)…?• Draw conclusions or hypothesize

“Which of these is supported by …?

Develop an Engaging Item• Use a theme/ unit /topic

• Use a sentence or two to engage the student

• Ask for a prediction or inference (you may use a question starter)

• Think of three wrong answers and one correct one

• Explain why a student might choose each of the wrong answers

•Use your knowledge of item writing and your checklist to critique the items others have written.

•Ask others to critique your test items.•Look over past tests with fresh eyes.

Work Smarter

Using the feedback of others

improves your questions.

Take a 6 Minute Break

Write an Item

•Improve a “rotten” item•Write a new item

Share an Item

•What made the original item “rotten?”•What did you do to improve it?•Was this task difficult? Why or why not?

Surviving Test Development

• Use question starters that lead to higher order questions.

• Clone good items.• Use open-ended questions.

Convert student wrong answers into options for multiple choice items.

Surviving Test Development

•Review existing item resources for possible items to bank

•Use a checklist to evaluate items•Only keep higher order thinking

skill items•File the items by TEKS

Bump up the LevelApply the Skill

A student is putting together an insect collection for a scout project. Today she found a moth, a dragonfly and a spider. The spider should not be included in her collection because the spider—A does not have wingsB is an arachnidC lays eggsD is poisonous

Bump up the LevelAsk students to predict

A toaster catches on fire in your kitchen and your mother begins to fill up a pitcher with water. If water is poured on an electrical fire it will—A extinguish the fireB cause the fire to spreadC protect the toasterD damage other appliances

Bump up the LevelAsk students to make a

generalizationThe MASH doctor wouldn’t go outside because—A he was afraid of the dark.B he sunburned easily.C he could hear the bombs nearby.D he was tired.

Bump up the LevelSynthesize – Main Idea

The main idea of MLK’s speech is --A all men can achieve their dream.B all people should be able to live together and have equal rights.C men will no longer go to war against each other.D people dream too much

Bump up the LevelUse a graphic organizer

Raising the Bar

If you can’t say it, you don’t know it.

In what ways can students “say it” in your

classroom?

Raising the BarAsk for an open-ended

response

Free response on tests“What do you know that

wasn’t asked on this test?”“Tell your neighbor.”

Raising the BarAssess Often

•Ticket out the door

•Lab write-ups, journal entries, topic summaries•Ask “why” as a follow-up

The most powerful tool a teacher possesses to

assess student knowledge.

“Tell me why.”

Every test question can be improved.Improving tests

improves instruction.

Test writing isn’t for wimps!