Do You Know What Your Students Know? Examining Student Work to Inform Instruction Lani Seikaly, partner Hillcrest and Main, Inc. and project director of.

Do You Know What Your Students Know?

Examining Student Work to Inform Instruction

Lani Seikaly, partnerHillcrest and Main, Inc.

and project director of MDK12http://mdk12.org [email protected]

http://mdk12.org/

Lani Seikaly, Hillcrest & Main, Inc.

What do principals need to put in place to improve their student achievement?

How do teachers and schools identify what students know and still need to learn in relation to state standards?

State assessments only inform schools where their students are performing at the time of testing.

Teachers must know where their students are at any point in the

school year.


AYP provides us a clear target.

However, to meet AYP, we need to monitor student level data at the classroom level on an ongoing basis.


How do state standards and NCLB change expectations for what happens in our schools?


Standards-based education changes everything.

• Defines what we expect all students to know and be able to do.

• Expects educators to take all students to proficiency on those standards.


Before Standards

Teachers taught what they thought was important.

After Standards

Teachers are expected to teach the content standards.


Before Standards

Teachers had different expectations for different groups of students.

After Standards

Teachers are expected to take all students to proficiency.


Before Standards

The focus was on how well teachers taught.

After Standards

The focus is on how well students learn.


Before Standards

Only selected students had access to higher level instruction.

After Standards

All students are expected to have equity of opportunity for higher level instruction.


What activities do teachers

need to engage in today that they might not

have needed to do 15 years ago?


Align instruction and assessment with state content standard indicators

Know where their students are performing on the indicators

Work toward a common understanding of content standards and proficient work

Diagnose what students know and still need to learn in relation to those indicators and the criteria for proficient work

Engage in grade level team examinations of student work


How do we get answers to these questions…

• Where are each of our students in relation to the content standards they must attain?

• What do they know and are able to do?

• What do they still need to learn?


We have a good deal of research to draw on in identifying what effective schools and leaders do to improve student achieve-ment.


Black and Wiliam in their 1998 Phi Delta Kappan article,

“In the Black Box: Raising Standards through Classroom Assessment,”

assert,“There is a body of firm evidence that formative assessment is an essential component of classroom work and that its development can raise standards of achievement. We know of no other way of raising standards for which such a strong prima facie case can be made.”

http://www.pdkintl.org/kappan/kbla9810.htm

http://www.pdkintl.org/kappan/kbla9810.htm


If formative assessments are so critical, then how do we create

good assessments?


We examined the National Research Council book, Knowing What Students Know, which identified the key concepts on which good assessments are built.From Knowing What Students Know by James W. Pellegrino, Naomi Chudowsky, and Robert Glasser/


Every assessment, regardless of purpose, rests on three pillars:

• Cognition, a model of how students represent knowledge and develop competence in the subject domain

• Observation, tasks or situations that allow one to observe students’ performance

• Interpretation, an interpretation method for drawing inferences from the performance evidence thus obtained.

James W. Pellegrino, Naomi Chudowsky, and Robert Glasser


To understand what students know, staff need to understand

the knowledge and cognitive domains of the content standard indicators

how students learn

how to provide students opportunity to demonstrate what they know

how to interpret student responses


Knowing What Students Know also suggests that

“Student work should focus adult-student and

adult-adult conversations to discuss standards.”


We have also read about the importance of collaborative

examinations of student work.


Joan Richardson, editor of the National Staff Development Council newsletter, believes that

“The practice of having teachers work together to study student work is one of the most promising professional development strategies in recent years. Examining student work helps teachers intimately understand how state and local standards apply to their teaching practice and to student work.


“As they see what students produce in response to their assignments, they can see the successes as well as the situations where there are gaps. In exploring those gaps, they can improve their practice in order to reach all students.”


The Aspen Workshop on High Schools recommended in its summary report for the Transforming High Schools Task Force that the continuous and collaborative examination of student work along with the personalization of schooling are the two critical strategies for transforming high schools at the local level.


“At the risk of sounding overly simplistic, the use of student work as the unrelenting focus of adult conversations can be the catalyst of fundamental changes in the educational experience of adolescents, and the transformation of teaching and learning at the high school level.”


Kate Nolan, Director of Re-Thinking Accountability for the Annenberg Institute of School Reform, believes

“Rich, complex work samples show us how students are thinking, the fullness of their factual knowledge, the connections they are making. Talking about them together in an accountable way helps us to learn how to adjust instruction to meet the needs of our students.”


So we understand

that to know where each of our students is in relation to content standards, we need to regularly examine student performance to inform our instruction and to monitor student progress. What would that look like? How would we operationalize that?


What does a team examination of student work look like?

• How do teachers define proficiency?• How do teachers diagnose strengths

and needs?• How do they record the diagnostic

information?• Who leads the discussion?• What questions need to be asked?


Examining Student Work

Though teachers have always examined student work as part of their grading process, the new focus on accountability and standards has driven a more structured and collaborative examination of student work.


Examining Student Work

The focus of the examination has shifted from a summative evaluation of student performance to a diagnostic evaluation of student performance, teacher assignment, and implications for instruction.


The Examining Student Work Protocol asks teachers to

• Reach consensus about what makes a student response proficient on an assignment or assessment

• Diagnose student strengths and needs on the performance

• Determine next instructional steps based on the diagnosis


In the first part of the protocol,

a team of teachers work through the process of reaching consensus on what the team believes constitutes a proficient response on a selected text and question.


Stiggins argues that we really can’t assess accurately if we don’t understand the target:

“To assess student achievement accurately, teachers and administrators must understand the achievement targets their students are to master. They cannot assess (let alone teach) achievement that has not been defined”.

Stiggins, Richard J. 2001. “The Principal’s Leadership Role in Assessment.” NASSP Bulletin (January 2001): 13–26.


Part 1: Reaching Consensus about Proficiency

• What did you ask the students to do? • Which content standard indicator were you

assessing? • What did you consider proficient

performance on this assignment? • Exactly what did students need to say or

write for you to consider their work proficient?


It is not enough that an individual teacher defines proficiency.

It is critical that at least a grade level team has reached consensus on the definition of proficiency to ensure that all students are held to the same performance expectations.


Teachers were surprised at how “off

the page” their teammates were.

Think of all the mixed messages our students are receiving when we haven’t defined proficiency on a standard / indicator in the same way.


Let’s take a look at a team of 3rd grade teachers trying to get on the same page about what a proficient answer would need to include for a reading comprehension assessment.


Only after the team has agreed on what constitutes a proficient response are they able to diagnose student strengths and needs.


In the second part of the protocol,

the team examines three student papers to determine if the response is proficient and to identify strengths, needs and instructional next steps.


Each teacher is asked to bring three samples of student work from the same assignment or assessment: a response at the top of the class, a response at the bottom of the class and a response in the middle of the class.


Part 2: Diagnosing Student Strengths and Needs

• What did the student demonstrate that he/she knew?

• What misconceptions or wrong information did the student have?

• What did the student not demonstrate?

• How would you find out if he/she knew it?


Teachers must shift their mindset from scoring (a summative examination) to diagnosing (a formative examination) student performance.


In many cases teachers have spent a great deal of time sorting student responses (either by letter grades or by rubric scores) and virtually no time diagnosing what students know and still need to learn.


Let’s take a look at our team of 3rd grade teachers diagnosing their student responses.


Part 3: Identifying Instructional Next Steps

• Based on the team's diagnosis of the student performance, what would you do next with that student?

• What questions might you ask the student?

• What feedback would you give?• Do you need to re-teach anything to

the class or a subset of the class?


How have teams captured the data?



What can you learn from examining student work?

• What do teachers learn about students?

• What do teachers learn about their instruction?

• What do school teams learn about their team’s understanding of content standards?


Teachers came out of this examining student work process with

• Grade level consensus of what constitutes proficient work on the assignment

• Formative assessment data• Specific information to inform their instruction• Strategies for re-teaching• Deeper understanding of the intent of the

standard / indicator they were assessing.• Probing questions to ask students to better

understand where they were


Andrew Nelson, teacher at Harmony Hills E.S.

“I wish I had caught onto this earlier but at one of the meetings, it became apparent that kids weren’t reading the question. That was a big awakening to me because I was so focused on how to write the answer, we hadn’t spent time unpacking the question.”


Glenn Messier, teacher at Harmony Hills E.S.

“My teammates proficient responses were a little bit more advanced than what I was expecting and looking for. They were looking for a lot more in-depth answers. To get on the same page, I needed to raise the bar for my students.”


Ilise Wolf, teacher at Harmony Hills E.S.

“When you’re working on a team, I really find I get a lot of ideas from my teammates and support and feedback. What I might consider a good assignment for a child, another 2nd grade teacher might have a few extra words to add that would really make a difference to my students.”


Let’s hear what our 3rd grade team learned from their experience in participating in this examining student work protocol.


Principals and facilitators learned When teachers collaborate with other

teachers on examining their own student’s performance on classroom assignments and assessments, teachers are more engaged in the process and take more ownership for making changes to their practice.


Principals and facilitators learned that Teachers began to have the

discussion and reach consensus on what they were looking for on an assignment or assessment before they gave it to the students. Therefore, their teaching was also more aligned with the proficient criteria they had identified.


Principals and facilitators learned that Principals needed to set clear

expectations for their teams, monitor whether they were met, identify useful end products, and engage in an ongoing discussion of the data and what students knew and still needed to learn.


How do we build teacher capacity to do this?

Shouldn’t we do that before we start the process?


This is on the job training.

The focus on interpreting student performance and determining what teachers need to do to support student performance allows teachers to examine their own practice through the lens of student needs rather than the lens of good versus mediocre teaching.


The fact that the process is ongoing allows teachers to

build capacity over time rather than try to absorb everything in an upfront

training.


Bottom line …..

It is only when teachers and schools start to collect the data and diagnose the performance that provides them the necessary information about where a student is in relationship to the indicators they must master that effective data-driven decision making to improve student achievement can happen.


You can find more information on the School Improvement in Maryland Web:

• Examining Student Workhttp://mdk12.org/data/examining/index.html

• Monitoring Student Progresshttp://mdk12.org/data/progress/index.html

• An online course, Using Data to Improve Student Achievement http://mdk12.org/process/course


Lani Hall Seikaly Contact Information

• Email: [email protected]

• Phone: 703 867-3921

• Web site: http://hillcrestandmain.com

• Web site: http://mdk12.org

mailto:[email protected]

http://hillcrestandmain.com/

http://mdk12.org/

http://hillcrestandmain.com/index.html

http://hillcrestandmain.com/index.html

Do You Know What Your Students Know? Examining Student Work to Inform Instruction Lani Seikaly, partner Hillcrest and Main, Inc. and project director of.

Documents

hillcrest main

instruction lani seikaly

standards teachers

students opportunity

state standards

selected students

partner hillcrest

standards of achievement