The ABC’s of Instruction Using assessment to improve performance The ABC’s of Instruction Using assessment to improve performance.

The ABC’s of The ABC’s of InstructionInstructionUsing assessment to Using assessment to improve performanceimprove performance

Ross DanisRoss Danis

International Clinical ProfessorInternational Clinical Professor Seton Hall UniversitySeton Hall University S. Orange, NJ, USAS. Orange, NJ, USA

Associate Dean of EducationAssociate Dean of Education Drew UniversityDrew University Madison, NJ, USAMadison, NJ, USA On the web at rossdanis.comOn the web at rossdanis.com E mail: at [email protected] mail: at [email protected]

THE NOTES WERE SOURTHE NOTES WERE SOUR

  BECAUSEBECAUSE

THE SEAMS WERE SPLITTHE SEAMS WERE SPLIT

What are our “bagpipes” for What are our “bagpipes” for assessment?assessment?

What is our experience with the What is our experience with the word?word?

How does assessment work with How does assessment work with students, teachers, and schools?students, teachers, and schools?

The Bagpipes of the PastThe Bagpipes of the Past

CoercionCoercion IntimidationIntimidation Rewards or punishments linked to Rewards or punishments linked to

evaluative judgmentsevaluative judgments Infrequent or vague feedbackInfrequent or vague feedback Limited personal contact and Limited personal contact and

involvementinvolvement

The The newnew Bagpipes of Bagpipes of AssessmentAssessment

The primary purpose of The primary purpose of assessment is assessment is to to improve performance, improve performance, not merely audit it.not merely audit it.

EmilyEmily

Read the writing sampleRead the writing sample Watch the InterviewWatch the Interview DiscussDiscuss Report outReport out

Key DistinctionsKey Distinctions

AssessmentAssessment EvaluationEvaluation

And….And….

Assessment Assessment forfor Learning Learning

Assessment Assessment ofof Learning Learning

Our GoalsOur Goals

To assess student achievement by To assess student achievement by knowing how to select and develop knowing how to select and develop classroom assessments that fit each classroom assessments that fit each unique context.unique context.

To frame “Learning Targets” to underpin To frame “Learning Targets” to underpin sound classroom assessments.sound classroom assessments.

To manage and communicate assessment To manage and communicate assessment results involving students when results involving students when appropriate and in ways that promote appropriate and in ways that promote learning.learning.

FramingFraming

We will frame standards of effective We will frame standards of effective practice and provide a cognitive practice and provide a cognitive structure that will assist you in structure that will assist you in extending your repertoire of sound extending your repertoire of sound practices.practices.

Indicators of Sound Classroom Indicators of Sound Classroom Assessment (1.4)Assessment (1.4)

Read the Indicators of Sound Classroom Read the Indicators of Sound Classroom Assessment Practice Assessment Practice

Engage in a small group discussion that is Engage in a small group discussion that is focused on the five components of sound focused on the five components of sound classroom assessment.classroom assessment.

Decide as a group, which areas are likely Decide as a group, which areas are likely to pose the greatest challenge and why. to pose the greatest challenge and why.

Report out to the whole group in order to Report out to the whole group in order to determine patterns or themes.determine patterns or themes.

If we want to use assessment as a If we want to use assessment as a tool for learning, students need totool for learning, students need to

Know where they’re goingKnow where they’re going

Know where they are nowKnow where they are now

Know how to close the gapKnow how to close the gap

Where am I going?Where am I going?

1.1. Provide a clear and understandable Provide a clear and understandable vision of the learning target.vision of the learning target.

2.2. Use examples and models of strong Use examples and models of strong and weak work.and weak work.

Where am I now?Where am I now?

3. Offer regular descriptive 3. Offer regular descriptive feedback.feedback.

4. Teach students to self-assess and 4. Teach students to self-assess and set goals.set goals.

How can I close the gap?How can I close the gap?

5. Design lessons to focus on one 5. Design lessons to focus on one aspect of quality at a time.aspect of quality at a time.

6. Teach students focused revision.6. Teach students focused revision. 7. Engage students in self-reflection, 7. Engage students in self-reflection,

and let them keep track of and share and let them keep track of and share their learning.their learning.

These Strategies are a ProgressionThese Strategies are a Progression

The seven strategies just presented The seven strategies just presented unfold in a classroom over time. unfold in a classroom over time. Students will have trouble with later Students will have trouble with later steps ( such as self assessment) if steps ( such as self assessment) if they have not had experience with they have not had experience with earlier steps.earlier steps.

Assess What?Assess What?

““If you don’t know where you are If you don’t know where you are going, any road will get you there.”going, any road will get you there.”

One can't “know where they are One can't “know where they are now” and “know how to close the now” and “know how to close the gap” unless they “know where they gap” unless they “know where they are going.”are going.”

What’s in a name!What’s in a name!

Content StandardsContent Standards BenchmarksBenchmarks GoalsGoals OutcomesOutcomes Enduring UnderstandingsEnduring Understandings Essential LearningsEssential Learnings Essential questionsEssential questions

Curriculum is not the textCurriculum is not the text

Curriculum is not the textCurriculum is not the text Curriculum is not the textCurriculum is not the text Curriculum is not the textCurriculum is not the text Curriculum is not the textCurriculum is not the text Curriculum is not the textCurriculum is not the text

Learning TargetsLearning Targets

Statements of intended learning in Statements of intended learning in each subject at each grade (or each subject at each grade (or course) level.course) level.

Learning TargetsLearning Targets

In small groups, identify theIn small groups, identify the Benefits to parentsBenefits to parents Benefits to teachersBenefits to teachers Benefits to studentsBenefits to students

How are Learning Targets How are Learning Targets Organized?Organized?

Varies by country, by district, and by Varies by country, by district, and by school.school.

You must understand your own You must understand your own curriculum framework.curriculum framework.

Good curriculum guides are critical to Good curriculum guides are critical to good assessmentgood assessment

ArticulationArticulation

Horizontal ( across grade levels) andHorizontal ( across grade levels) and

vertical ( through grade levels) vertical ( through grade levels) articulation is essential to use articulation is essential to use assessment as a tool for learning.assessment as a tool for learning.

Knowing how to “posthole” is critical.Knowing how to “posthole” is critical.

What is “Postholing”What is “Postholing”

Essential learningsEssential learnings Key conceptsKey concepts Critical attributesCritical attributes Foundational principlesFoundational principles

Let’s take the first steps togetherLet’s take the first steps together

Individually, identify between four or Individually, identify between four or five key learnings in a specific five key learnings in a specific subject at a specific grade level. subject at a specific grade level.

In other words, what In other words, what knowledge/understanding should knowledge/understanding should students have after having taken a students have after having taken a particular subject at a particular particular subject at a particular grade level.grade level.

Next step…Next step…

Put the “Learning Targets” in student Put the “Learning Targets” in student friendly language. These “Learning friendly language. These “Learning Targets” are statements that begin Targets” are statements that begin with “I can.” with “I can.”

““I can multiply two digit by two digit I can multiply two digit by two digit numbers.”numbers.”

Why is this important?Why is this important?

Minimizes curriculum drift.Minimizes curriculum drift. Reduces redundancy.Reduces redundancy. Allows families to learn together and Allows families to learn together and

reinforce what is taught in school.reinforce what is taught in school. Creates an opportunity for inter and Creates an opportunity for inter and

multi disciplinary teaching and multi disciplinary teaching and learning across grades and subjects.learning across grades and subjects.

And one more stepAnd one more step

Gather in small groups, organized by Gather in small groups, organized by grade level and share with one another grade level and share with one another your understanding of the key Learning your understanding of the key Learning Targets at that grade level.Targets at that grade level.

Gather in small groups, organized by Gather in small groups, organized by single subject across grade levels ( if single subject across grade levels ( if possible) and share Learning Targets in possible) and share Learning Targets in your discipline across grade levels or your discipline across grade levels or courses.courses.

Develop a planDevelop a plan

Only you know your circumstances Only you know your circumstances well enough to understand what has well enough to understand what has to take place in order to develop a to take place in order to develop a plan to design, develop, distribute, plan to design, develop, distribute, and use, Learning Targets.and use, Learning Targets.

This could take a year to complete This could take a year to complete but it will have an enormous impact but it will have an enormous impact on your capacity to effectively use on your capacity to effectively use assessment as a tool for learning.assessment as a tool for learning.

The Nuts and Bolts of Standardized The Nuts and Bolts of Standardized TestsTests

More than the required tests from More than the required tests from the state or the country. Classroom the state or the country. Classroom tests can be considered standardized tests can be considered standardized if they are administered to all if they are administered to all students in the same grade or students in the same grade or course.course.

Standardized Tests 2Standardized Tests 2

Norm- Referenced: This means that Norm- Referenced: This means that student scores are referenced to a norm student scores are referenced to a norm group. Includes percentiles and grade group. Includes percentiles and grade equivalents. They do not reflect what each equivalents. They do not reflect what each student has learned, but how much, student has learned, but how much, compared to all other students.compared to all other students.

Criterion-Referenced Tests: Shows how Criterion-Referenced Tests: Shows how learning compares to performance on learning compares to performance on specified learning targets. Sometimes specified learning targets. Sometimes called “Standards-based.”called “Standards-based.”

Part 3: Assessment MethodsPart 3: Assessment Methods

Four Basic Categories:Four Basic Categories: 1. Selected Response and short answer1. Selected Response and short answer 2. Extended Written Response2. Extended Written Response 3. Performance Assessment3. Performance Assessment 4. Personal Communication.4. Personal Communication.

All are legitimate when use correlates highly All are legitimate when use correlates highly with the learning target.with the learning target.

Extended Written ResponseExtended Written Response

Comparing two pieces of literature Comparing two pieces of literature for example.for example.

We score in one of two ways: A We score in one of two ways: A Rubric or pointes provided for Rubric or pointes provided for specific pieces of information specific pieces of information provided.provided.

Selected ResponseSelected Response

Formats include multiple choices, Formats include multiple choices, True/False, Matching, Fill in True/False, Matching, Fill in questions.questions.

PerformancePerformance

Based on observation and judgment. Based on observation and judgment. We look at a performance or product We look at a performance or product and make a judgment about its and make a judgment about its quality.quality.

Personal CommunicationPersonal Communication

Gathering information about Gathering information about students through personal students through personal communication is just what it sounds communication is just what it sounds like- we find out what students have like- we find out what students have learned through interaction. This learned through interaction. This includes conference, class includes conference, class participation asking questions during participation asking questions during instruction. Checking for instruction. Checking for understanding in real time and understanding in real time and making adjustments to teaching.making adjustments to teaching.

Which Method?Which Method?

Which assessment method would Which assessment method would you choose for each of the following:you choose for each of the following:

1. Ability to write clearly and 1. Ability to write clearly and coherentlycoherently

Group discussion proficiencyGroup discussion proficiency Reading ComprehensionReading Comprehension Using Specified proceduresUsing Specified procedures Proficiency conducting investigations Proficiency conducting investigations

in science.in science.

PortfoliosPortfolios

A collection of artifacts put together A collection of artifacts put together to get at the full story. To help to get at the full story. To help students, teachers, and others students, teachers, and others understand in depth one or more understand in depth one or more aspects of student learning. aspects of student learning.

Portfolios are collecting and Portfolios are collecting and communicating devices, not a communicating devices, not a method of assessment such as method of assessment such as performance assessment.performance assessment.

Portfolios as learning experiencesPortfolios as learning experiences

Teachers or students ( or both) can Teachers or students ( or both) can author a portfolio. When the subject author a portfolio. When the subject is the author, and when they is the author, and when they assemble and share portfolios, they assemble and share portfolios, they perform acts of metacognition that perform acts of metacognition that deepen ability to learn, desire to deepen ability to learn, desire to learn, and the learning itselflearn, and the learning itself

Portfolio as StoryPortfolio as Story

Every story needs a theme- a Every story needs a theme- a struggle, achievement, competence, struggle, achievement, competence, celebration- to guide the selection of celebration- to guide the selection of components, artifacts, and data.components, artifacts, and data.

Growth PortfoliosGrowth Portfolios

Students select evidence related Students select evidence related given target at two or more points in given target at two or more points in time.time.

Annotations explain the level of Annotations explain the level of achievement each artifact achievement each artifact represents.represents.

The student writes a self-reflection to The student writes a self-reflection to summarize growth.summarize growth.

Achievement PortfoliosAchievement Portfolios

Document achievement at a point in Document achievement at a point in time.time.

They include the best and most They include the best and most recent data organized by the recent data organized by the learning targets each represents. learning targets each represents. Certain targets call for multiple Certain targets call for multiple samples to demonstrate level of samples to demonstrate level of achievement, while others need only achievement, while others need only one.one.

Competence PortfoliosCompetence Portfolios

Offer evidence in support of a claim Offer evidence in support of a claim to have attained as acceptable or to have attained as acceptable or exemplary level of achievement.exemplary level of achievement.

Sometime referred to as an Sometime referred to as an “Exhibition of Mastery.”“Exhibition of Mastery.”

Sampling is an issue. Want to show Sampling is an issue. Want to show a high level of achievement is a high level of achievement is sustained, not sustained, not

Portfolio Contents: Focus on Portfolio Contents: Focus on TargetsTargets

All forms of assessment (tests, All forms of assessment (tests, performance, personal performance, personal communication) can be included.communication) can be included.

All kinds of learning– knowledge, All kinds of learning– knowledge, dispositions, skills- can be the focus dispositions, skills- can be the focus of evidence.of evidence.

Portfolios can reflect a single Portfolios can reflect a single learning target, a series of learning learning target, a series of learning targets, or all learning targets.targets, or all learning targets.

ArtifactsArtifacts Clearly identify the learning targets about Clearly identify the learning targets about

which the portfolio is intended to which the portfolio is intended to communicate.communicate.

Sampling: Need enough artifacts to ensure Sampling: Need enough artifacts to ensure that accurate information is being that accurate information is being provided. The size of the sample depends provided. The size of the sample depends on the learning targets at the focus of on the learning targets at the focus of achievement.achievement.

The selection process is strengthened by The selection process is strengthened by the dialogue between and among the the dialogue between and among the students and the teacher.students and the teacher.

Self ReflectionSelf Reflection

Do not pass over this stepDo not pass over this step Once students have pointed out what Once students have pointed out what

they have learned; have justified the they have learned; have justified the claim with evidence; and set goals claim with evidence; and set goals for the future, what is left?for the future, what is left?

Open ended self-reflection that goes Open ended self-reflection that goes beyond the goalsbeyond the goals

(Refer to 11.2 and 3)(Refer to 11.2 and 3)

ConferencesConferences

1. Offering Feedback1. Offering Feedback 2. Setting Goals2. Setting Goals 3. Planning and Intervention3. Planning and Intervention 4. Demonstrating Growth4. Demonstrating Growth 5. Communicating achievement5. Communicating achievement

(Conduct mock conference)(Conduct mock conference) Provide Sample forms 12.2. 3. and 4. Provide Sample forms 12.2. 3. and 4.

Concept:Concept:

Planning Planning

BackwardsBackwards  “What would it look like “What would it look like if we succeeded?”if we succeeded?”

Planning Backwards:Planning Backwards:

“How do we identify “How do we identify what we value most, then what we value most, then measure what mattersmeasure what matters?”?”

Tool:Tool: RubricRubric“By what criteria would we “By what criteria would we

measure success?”measure success?”

“What would success look“What would success looklike?”like?”

RubricRubric

““If you can describe it, If you can describe it, you can measure it.”you can measure it.”

  Trait: e.g knowledgeable; friendly

4 

Descriptors: e.g The waiter knows the menu well; what is fresh today; what the chef’s specialties are; …

Indicators: e.g. what does it look and sound like? What are the stories and scenes that help me understand the descriptors? 

  Knowledgeable

4 

The waiter knows the menu well; what is fresh today; what the chef’s specialties are; what wines will complement various dishes; basic knowledge about what’s going on in the City regarding theater, etc.    

  Knowledgeable

4 

The waiter knows the menu well; what is fresh today; what the chef’s specialties are; what wines will complement various dishes; basic knowledge about what’s going on in the City regarding theater, etc. 

   “Sir, I particularly recommend the duck. Chef Henri grew up in the Loire Valley, where they eat duck three meals a day.”    “Yes, Madam, I do have a recommendation for you. Do you enjoy musicals? We hear that the revival of Man of La Mancha is getting standing ovations every night.”

  Knowledgeable

4 

The waiter knows the menu well; what is fresh today; what the chef’s specialties are; what wines will complement various dishes; basic knowledge about what’s going on in the City regarding theater, etc. 

       “Sir, I particularly recommend the duck. Chef Henri grew up in the Loire Valley, where they eat duck three meals a day.”        “Yes, Madam, I do have a recommendation for you. Do you enjoy musicals? We hear that the revival of Man of La Mancha is getting standing ovations every night.”        “Good evening, Mr. and Mrs. Danis. Welcome back.”

  Knowledgeable

4 

The waiter knows the menu well; what is fresh today; what the chef’s specialties are; what wines will complement various dishes; basic knowledge about what’s going on in the City regarding theater, etc.; the names of regular customers.

Your RubricYour Rubric

11 22 33

??

??

Why Rubrics?Why Rubrics?

For feedback on performance rather For feedback on performance rather than audit of performancethan audit of performance

• Compares current performance Compares current performance against an exemplary standardagainst an exemplary standard

• Demystifies exemplary standard; Demystifies exemplary standard; takes it out of one person’s headtakes it out of one person’s head

• Gives specific description of higher Gives specific description of higher levels of performance, tells us levels of performance, tells us “what to do next”“what to do next”

Why Rubrics?Why Rubrics? Is a vehicle for ongoing discussion Is a vehicle for ongoing discussion

of what good work looks likeof what good work looks like

• Facilitates self-assessment in the Facilitates self-assessment in the context of what the whole context of what the whole organization is trying to accomplishorganization is trying to accomplish

Forces us to ask what is most worth Forces us to ask what is most worth measuring rather than what is measuring rather than what is easiest to measureeasiest to measure

Reality Check: Reality Check:

“What gets in the way “What gets in the way of this work?”of this work?”

OnOn TIME TIME

….. He had discovered Time, and Death, and God.

from Brave New World by Aldous

Huxley

Effective thinking Effective thinking about assessment about assessment

makes change makes change

more likelymore likely

Checking for Understanding…Checking for Understanding…

1.1. Elicit ____responseElicit ____response

2.2. Check for understandingCheck for understanding

3.3. Interpret the behaviorInterpret the behavior

4.4. Act on the InterpretationAct on the Interpretation

a.a. Move onMove on

b.b. PracticePractice

c.c. Re-teachRe-teach

d.d. QuitQuit

3 Ways 3 Ways to Check to Check

for Understandingfor Understanding

1.1. SamplingSampling

2.2. Choral ResponseChoral Response

3.3. SignalingSignaling

LEAN LINELEAN LINE

1.1. CostlyCostly

2.2. SincereSincere

3.3. ScaleScale

4.4. MirrorsMirrors

5.5. Variety of foodsVariety of foods

MotivationMotivationMotivation is the act of maintaining students' focus Motivation is the act of maintaining students' focus

on a given task in order to satisfy a need or on a given task in order to satisfy a need or desire.desire.

Task Reward

Task Reward

Developing a New Report CardDeveloping a New Report Card

Identify the purpose of the report Identify the purpose of the report card.card.

Identify the skills and dispositions Identify the skills and dispositions that will be assessed and reported.that will be assessed and reported.

Identify the criteria that will be used Identify the criteria that will be used to create the scores and or narrative.to create the scores and or narrative.

Decide on how frequently these Decide on how frequently these reports should be distributed.reports should be distributed.

Using data to inform curriculum and Using data to inform curriculum and instructioninstruction

How can your school or grade level How can your school or grade level team come together to discuss gaps team come together to discuss gaps in instruction uncovered by in instruction uncovered by assessment?assessment?

What mechanism or support system What mechanism or support system is in place or could be put into place is in place or could be put into place to modify instruction based on to modify instruction based on student performance data?student performance data?

Action PlanAction Plan

Goal setting:Goal setting: Learning Targets: Develop, Learning Targets: Develop,

disseminate, review, revise.disseminate, review, revise. Report cards: Review, revise, Report cards: Review, revise,

implementimplement School and team-based modifications School and team-based modifications

of curriculum and instruction based of curriculum and instruction based on data.on data.

Action Plan IIAction Plan II

Portfolios: Appropriate use as a Portfolios: Appropriate use as a vehicle to improve instructionvehicle to improve instruction

Rubrics: Designed and used by Rubrics: Designed and used by students and teachers in a variety of students and teachers in a variety of subjects to identify exemplars and subjects to identify exemplars and demystify excellence.demystify excellence.

Matching the appropriate Matching the appropriate assessment to the content assessment to the content