The Big Picture

RE-THINKING HOW SCHOOLS IMPROVE:

A Team Dialogue Model for Data-Based Instructional Decision Making

2

The Big PictureIn today’s session, we are going to:

1. Re-think our understanding of how schools improve -- moving from the dysfunction of the “old model” to the requirements for what a “new model” might look like.

2. Focus on a “new model” for improving performance that enables content, vertical, or departmental teams to use data more effectively for classroom instructional improvement and increased student learning.

3

“Every organization is perfectly designed to get the results it achieves.”

--W. Edwards Deming

4

What are data?

Data are observations, facts, or numbers which, when collected, organized

and analyzed, become information and, when used productively in

context, become knowledge.

5

The DRIP Syndrome

6

Being Data Rich

Your school may suffer from

You may need ways to organize and use all the data you have.

7

Sources of Student Achievement Data

• External assessment data

• Benchmark or course-wide assessment data

• Individual teacher assessment data--Supovitz and Klein (2003)

8

Data-driven schools and school districts use data for two major purposes:

• Accountability (to prove)• Instructional decision making (to improve)

9

The Hierarchy of Data for Accountability Purposes

External (State & National) Assessments

System Benchmark Assessments

Common School or Course Assessments

Classroom Assessments of Student Work

10

The Hierarchy of Data for Instructional Decision Making

Classroom Assessments of Student Work

Common School or Course Assessments

System Benchmark Assessments

External (State & National Assessments)

11

Think about how long you have been engaged in the school improvement process.

Has the school gotten better each year?

Has the performance of each student improved as a result of each year he/she spends in the school?

If your answer to one or both questions is no, what will it take to change it to yes?

12

Think about your School Improvement Plan and the data on which it is primarily based. Is it . . .

•State assessment data? (data to prove)OR

•Classroom assessments of student work? (data to improve)

13

Then ask yourself this question:

Do you have a school improvement plan?

Or a school accountability plan?

A SIP ? Or a SAP?

14

WHY IS THE OLD MODEL OF SCHOOL IMPROVEMENT NOT

WORKING ANY MORE?

15

WHY IS THE OLD MODEL OF SCHOOL IMPROVEMENT NOT

WORKING ANY MORE?

You may wish to stop the presentation at this time to

discuss your faculty’s views on this question.

16

Why? Wrong Data

We have been using the wrong data. State test data are: Way too general Instructionally insensitive -- not designed for instructional improvement

17

Why? Wrong Time

The data come at the wrong time. State test data are: Out of date when they arrive For students we may no longer have

The results of the changes that are implemented will not be known for a year.

18

Why? Wrong Team

The SIT, a full department, or a Data Committee are the wrong groups to do the analysis.Membership is too diverse (often including parents)Meets too infrequentlyNot connected to immediate classroom needs

19

Why? Wrong Plan

The initiatives that are put in place are: Too global to address

the diversity of students Aimed at performance increases

of groups on average Looking for the “silver bullet” that

will have a schoolwide impact

20

We need a new model.

Uses “real time” data Sessions build on each other Addresses individual students’ needs Results in instructional improvements that will actually

occur at a high level of quality Can be re-directed frequently Has meaning for teachers and is seen by teachers as a

worthwhile use of their time

21

We need a new model.

Uses “real time” data Sessions build on each other Addresses individual students’ needs Results in instructional improvements that will actually

occur at a high level of quality Can be re-directed frequently Has meaning for teachers and is seen by teachers as a

worthwhile use of their time

You may wish to stop the presentation at this time and discuss the extent to which these positive characteristics are present in the data

analysis process your school is currently using.

22

What should that new model look like?

“School improvement is most surely and thoroughly achieved when teachers engage in frequent, continuous, and increasingly concrete and precise talk about teaching practice . . . adequate to the complexities of teaching, [and] capable of distinguishing one practice and its virtue from another.”

--Judith Warren Little

23

Fundamental Concepts of Collaborative Learning Communities

• Teachers establish a common, concise set of essential curricular standards and teach to them on a roughly common schedule.

• Teachers meet regularly as a team for purposes of talking in “. . . concrete and precise terms” about instruction with a concentration on “thoughtful, explicit examination of practices and their consequences.”

• Teachers make frequent use of common assessments.

Continued on next slide

24

“These elements, so rarely emphasized in school . . . improvement plans, deserve our attention more than anything else we do in the name of school improvement.”

--Mike Schmoker (2006)

25

THE CLASSROOM-FOCUSED IMPROVEMENT PROCESS (CFIP):

A Team Data Dialogue Protocol

Components of

THE NEW MODEL

26

The new process needs to be built on:

1. Dialogue

2. Protocols

3. Triangulation of

Data (use of multiple

data sources)

27

Our Goal in the Data Dialogues:

Frequent, continuous, and increasingly concrete and precise dialogue by school teams, informed by data

28

What are the right teams to conduct data dialogues?

Teams that share common standards and assessments

• Grade-level teams• Content teams• Vertical teams

29

When is the right time to conduct data dialogues?

•At a minimum, devote at least one hour to data dialogues every two weeks.

•According to several studies, schools that realized the greatest results from a shift to a data culture scheduled data dialogues at least once a week.

30

Frequency of Data Dialogues

Source: Stanford University, Stanford Research Institute, Education Week, January 24, 2004

20

45

35

4548

7

0

10

20

30

40

50

60

A Few Timesa Year

A Few TimesA month

A Few TimesA Week

Gap Closers

Non-gap Closers

31

What are the right data to use in the data dialogues?

Triangulate three types of data:• External Assessment Data• Course-wide Benchmark Assessment Data• Classroom Assessment Data

--Supovitz & Klein (2003)

32

Conclusions are specific to students in the class.

Conclusions are used to plan upcoming daily instruction.

The plans are implemented.

What is the right plan where the results of the data dialogues should be used?

33

What is the right way to use the results of the data dialogues?

Conclusions are used to identify enrichments and interventions for the students in the class. Conclusions are used to

plan upcoming daily

instruction.

34

What Is a Data Protocol?

A protocol consists of guidelines for dialogue – which everyone understands and has agreed to – that permit a certain kind of conversation to occur, often a kind of conversation which people are not in the habit of having.

Protocols build the skills and culture necessary for collaborative work. Protocols often allow groups to build trust by doing substantive work together.

35

Using a Data Protocol

Protocols can help us to navigate difficult and uncomfortable conversations by:

Making it safe to ask challenging

questions Making the most of scarce time Providing an opportunity for all to be

involved Resulting in an analysis that will lead to

positive action

36

Using a Data Protocol

The point is not to do the protocol well, but to have team dialogue that is:

In-depth Insightful Concrete Precise

37

Six Easy CFIP Steps

1. Understand the data source.2. Begin with a question.3. Look for class-wide patterns.4. Act on the class patterns.5. Address individual students’ needs.6. Improve instruction in the next lesson.

38

CFIP Step 1: Understand the data source.

Build ASSESSMENT LITERACY with questions like these:What assessment is being described in this data report? What were the characteristics of the assessment?

Who participated in the assessment? Who did not? Why?

Why was the assessment given? When?

What do the terms in the data report mean?

Be sure you have clear and complete answers to these questions before you proceed any further.

39

CFIP Step 2: Identify the questions that can be answered by the data.

All data analyses should be designed to answer a question.

Unless there is an important question to answer, there is no need for a data analysis.

40

CFIP Step 3A: Look for class-wide patterns in a single data source.

What do you see over and over again in the data?

What are the strengths of the class? What knowledge and skills do the students have?

What are the weaknesses of the class? What knowledge and skills do the students lack?

41

CFIP Step 3B: Identify patterns of class

strengths and weaknesses from multiple data sources.

TRIANGULATION

•In what ways are the results similar among data sources? For example, how do benchmark test results compare with ongoing classroom assessment data?•In what ways do the results among data sources differ?•Why might these differences occur?

42

Power When Multiple Types

of Data Are Used Reduces the anxiety and the mistakes

of relying on a single measure as the only definition of student successProvides more frequent evidence on which to actDevelops and sustains a culture of inquiry in the school based on data

CFIP Step 4: Act on the class-wide patterns.

• What instructional factors might have contributed to the class-wide patterns?

• What will we do to address patterns of class needs?

• How and when will we reassess to determine progress?

43

44

CFIP Step 5: Drill down to individual

students. Identify and implement needed enrichments and interventions.

What are the implications for enrichments and interventions from

what you learn from the data? Which students need enrichments and interventions?What should enrichments and interventions focus on?

45

CFIP Step 6: Reflect on the reasons for student performance -- What in our teaching might be

preventing all students from being successful?

To what extent did we implement research-based instructional practices as we:

Planned instruction? Introduced instruction? Taught the unit? Brought closure to instruction? Assessed formatively?

46

CFIP Step 6: Reflect on the reasons for

student performance. Identify and implement instructional changes

in the next unit.

How will we change instruction in our next unit?Content focusPacingTeaching methodsAssignments

47

CFIP Step 6: Reflect on the reasons for

student performance. Identify and implement instructional changes

in the next unit. When will we review the data again to determine the success of the enrichments, interventions, and instructional changes?What do the data not tell us? What questions about student achievement do we still need to answer? How will we attempt to answer these questions?

48

The Next Steps

1. Unless teams emerge from the data analysis process with a clear plan of action for their classroom, they have wasted their time.

2. Implement the plan of interventions, enrichments, and changes in instruction.

3. Collect the next set of data.

49

Six Easy CFIP Steps

1. Understand the data source.2. Begin with a question.3. Look for class-wide patterns.4. Act on the class patterns.5. Address individual students’ needs.6. Improve instruction in the next lesson.You may wish to stop the presentation at this time and review the six steps again, keeping

in mind that teams will study and practice using the steps prior to their implementation

of CFIP.

50

Caveats about CFIP

• It is a paradigm shift from the traditional lesson planning format.

• It is not easy, especially at first.• Follow the steps faithfully until they

become second nature.• Expect mistakes and imprecision in the

data.• The results are worth the effort.

51

FOR FURTHER INFORMATION ABOUT CFIP, CONTACT:

Dr. Michael [email protected]

Dr. Ronald [email protected]