Top Banner
1 Prepared for Irving M.S. on May 10, 2006
99

1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

Jan 03, 2016

Download

Documents

Oliver Martin
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

1

Prepared for Irving M.S. on May 10, 2006

Page 2: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

USING DATA PROJECT OVERVIEW• Collaboration between TERC and WestEd

• Funded by the National Science Foundation

• Based on Using Data/Getting Results

• Working with mathematics and science projects nationally to improve teaching and learning

• Provides professional development and materials

2

Page 3: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

TODAY’S PLAN

• 100% of participants will understand the Data Dialogue Process as measured by growth on a

consensogram

3

Page 4: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

TO DO LIST

Participants will:• Understand the data dialogue process

• Identify the benefits and limitations of aggregated, disaggregated, and strand data

• Apply the data dialogue process• Discuss ideas for implementation

4

Page 5: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

THE USING DATA PROCESS/PDSA

5

Achieve

0

5

10

15

20

25

30

35

40

45

50

Crenshaw Hartford Lehman Souers

2002-20032003-2004

StudentLearning

Goal

Generate Solutions

Strategies Outcomes

Strategies Outcomes

Causes

Verify

StudentLearningProblem

Identify

Foundation

Build

Page 6: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA-DRIVEN DIALOGUE

6

PHASE 1Predict

PHASE 3Infer/Question

Adapted from Organizing Data-Driven Dialogue by Laura Lipton Bruce Wellman, MiraVia LLC, 2001

PHASE 2Observe

GoVisual

Page 7: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

THE DATA DIVIDE

7

Page 8: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

8

What do you think bridges the gap between data and results?

What are the features of a culture that supports data-driven

dialogue?

Partner Talk

Page 9: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

COLLABORATIVE CULTURE IS FIRST ORDER OF BUSINESS….

• Creating a collaborative culture is the single most important factor for successful school improvement initiatives and the first order of business for those seeking to enhance the effectiveness of their schools.

Eastwood and Louis (1992). Restructuring That Lasts: Managing the Performance Dip. Journal for School Leadership 2 (2), 213-224.

9

Page 10: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

NEED FOR A COLLABORATIVE CULTURE

• Throughout our ten-year study, whenever we found an effective school or an effective department within a school, without exception that school or department has been a part of a collaborative professional learning community. Milbury McLaughlin, 1995. Creating Professional Learning Communities. Kenote address presented at NSDC Conference, Chicago, IL

10

Page 11: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

USING DATA PROJECT: BUILDING THE BRIDGE BETWEEN DATA AND RESULTS

11

Using DataProfessional

Development

Instructional ImprovementData UseCollaborationLeadership &

Capacity

Culture/Equity

Page 12: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

USING DATA PROJECT ASSUMPTION

12

Collaborative inquiry – school teams constructing meaning of student learning

problems and testing out solutions together through rigorous use of data and reflective dialogue – unleashes the resourcefulness of educators to solve the biggest problems

schools’ face.

Page 13: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

USING DATA PROJECT ASSUMPTION

13

Significant improvement in mathematics learning and closing achievement gaps is a moral responsibility and a real possibility in

a relatively short amount of time - one to three years.

Page 14: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

POSSIBILITY

14

Poverty, Ethnicity, and Achievement

Poverty and Ethnic Minority Enrollment

% o

f S

tud

en

ts P

rofi

cie

nt

or

Ab

ov

e

Source: Accountability in Action by Douglas Reeves, Center for Performance Assessment, Denver, Colorado www.makingstandardswork.com/ResourceCtr/books

XSome high-poverty,

high-minority schools are achieving at this level

Page 15: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

UDP SCHOOLS OUTPERFORM CONTROLS

15

% Proficient 6th Grade Science Students in Tennessee County, State 2004-2005 TCAP

50

55

60

65

70

75

80

85

90

95

Interactions Living /Environment

Food Production /Energy for Life

Diversity / LivingThings

Biological Change Earth in theUniverse

Energy

%

Treatment School 1Treatment School 2Control School 1Control School 2

Source: Personal Communication, 2005

Page 16: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

MATHEMATICS GAINS IN CANTON CITY:OHIO PROFICIENCY TEST

16

0

5

10

15

20

25

30

35

40

45

50

Crenshaw Hartford Lehman Souers

2002-20032003-2004

Per

cent

age

Pas

sing

33%

42%

20%

35%

24%

47%

20%

36%

Middle School

Sixth Grade Proficiency Results

Source: Ohio Department of Education

Page 17: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

MATHEMATICS GAINS IN CANTON CITY: SIXTH GRADE QUARTERLIES

17

50.2

57.3

43.0

49.7

40.7

52.2

0

10

20

30

40

50

60

Pe

rce

nt

Co

rre

ct

Decimals Fractions Total

2003-20042004-2005

District Sixth Grade

ChunkSource: Canton City School District

Page 18: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

MATHEMATICS GAINS IN CANTON CITY: SEVENTH GRADE QUARTERLIES

18

35.0

43.8

31.7

40.1

35.0

40.2

0

10

20

30

40

50

Pe

rce

nt

Co

rre

ct

Computation Algebra Total

2003-20042004-2005

District Seventh Grade

ChunkSource: Canton City School District

Page 19: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

MATHEMATICS GAINS IN CANTON CITY: EIGHTH GRADE QUARTERLIES

19

43.146.8

44.3 45.3

29.131.7

35.238.9

0

10

20

30

40

50

Per

cent

Cor

rect

Computation ScientificNotation

Proportions Total

2003-20042004-2005

District Eighth Grade

ChunkSource: Canton City School District

Page 20: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

JOHNSON COUNTY IMPROVES MATHEMATICS - GRADES 3, 5, 8

20

7788

72

86

36

73

0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

90.00

100.00

Perc

en

t P

rofi

cie

nt

All Low SES SWD

CRT Proficiency: Mathematics Grades 3, 5, 8

20042005

390

420

238

297

28

56

Source: TN DOE

Number on column = number of students

Page 21: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

JOHNSON COUNTY IMPROVES MATHEMATICS - GRADES 9 -12

21

84 8680 83

30

58

0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

90.00

100.00

Perc

en

t P

rofi

cie

nt

All Low SES SWD

CRT Proficiency: Mathematics 9 - 12

200420051

3156

63

83 1

112

Source: TN DOE

Number on column = number of students

Page 22: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

22

What do you think are the changes in school culture that are taking

place in the schools that are improving results?

Table Talk

Page 23: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

INSTRUCTIONAL IMPROVEMENT SHIFT

23

Data to sort, opportunities for

some

Less Emphasis More Emphasis

Data to serve, opportunities for all

Page 24: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

SORTING STUDENTS

24

Pe

rce

nt o

f Stu

den

ts

Percentile

VERY DUMB

SORTA SMART

VERY SMART

SORTA DUMB

Page 25: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DOING THINGS DIFFERENTLY

26

”“The data are causing us to do things differently.

We set a goal for improvement. Now we teach to achieve that goal.

— Mia Merrick, Teacher

Desert Eagle Secondary School

Page 26: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA USE SHIFT

27

Carrot and stick, avoidance

Feedback for continuous improvement, frequent and in depth use

Less Emphasis More Emphasis

Page 27: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

SHIFTS THAT ARE MOVING SCHOOLS FROM RESIGNATION TO

EMPOWERMENT

28

Culture

Instructional Improvement

Collaboration

Data Use

Leadership & Capacity

Less Emphasis More Emphasis

External accountability Internal and collective responsibility, equity

Data to sort, opportunities for some

Data to serve, opportunities for all

Top-down, premature data-driven decision

making

Ongoing data-driven dialogue and collaborative inquiry

Carrot and stick, avoidance

Feedback for continuous improvement, frequent and in depth use

Individual charismatic leaders as change agents

Learning communities with many change agents

Page 28: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

AFFINITY CHART ACTIVITY

• Reflect on the culture of your school• What shifts need to occur to move towards a

more collaborative, data-driven culture?• Use post-its to record your ideas• One idea per post-it• Place them all on the flip chart when your done

29

Page 29: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

LESSONS: COLLABORATIVE INQUIRY WORKS IF…

• High-quality professional development• A clear structure and process is provided for data-

driven dialogue and sense-making• Skilled data teams - data literacy, content and

pedagogy, equity - and data facilitators• Collective response-ability for student learning• Time for data teams to work - weekly• Timely access to robust data - benchmark

assessments, item level data, student work• District and school administrative support

30

Page 30: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

TYPES OF STANDARDIZED ASSESSMENTS: NORM-REFERENCED TESTS (NRT)

• Compare performance of individuals and groups• Rank and sort students by comparing them to a

national “norm” group• Examples: Stanford 9, Iowa Test of Basic Skills,

Terra Nova

31

Page 31: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

SORTING STUDENTS

32

Pe

rce

nt o

f Stu

den

ts

Percentile

VERY DUMB

SORTA SMART

VERY SMART

SORTA DUMB

Page 32: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

TYPES OF STANDARDIZED ASSESSMENTS: CRITERION-REFERENCED TESTS (CRT)

• Interpret performance of individuals and groups in relation to a set of criteria, standards, or benchmarks

• Determine student mastery of the criteria• Used to make instructional and programmatic

decisions (e.g., curriculum)• Examples: most state assessments,

New Standards Reference Exam

33

Page 33: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

FUNDAMENTAL BELIEF

“All kids can learn so we establish high standards that we expect all students to achieve.”

34

Page 34: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

SERVING STUDENTS

35

A - Advanced

P - Proficient

NI - Needs Improvement

W - Warning

Page 35: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

36

“Stop asking me if we’re almost there!We’re nomads, for crying out loud!”

Page 36: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA-DRIVEN DIALOGUE

37

PHASE 1Predict

PHASE 3Infer/Question

PHASE 2Observe

GoVisual

• With what assumptions are we entering?

• What are some predictions we are making?

• What are some questions we are asking?

• What are some possibilities for learning that this experience presents us with?

Surfacing experiences, possibilities, expectations

• What important points seem to “pop out”?

• What are some patterns or trends that are emerging?

• What seems to be surprising or unexpected?

• What are some things we have not explored?

Analyzing the data

• What inferences and explanations can we draw?

• What questions are we asking?

• What additional data might we explore to verify our explanations?

• What tentative conclusions might we draw?

Generating possible explanations

Adapted from Organizing Data-Driven Dialogue by Laura Lipton Bruce Wellman, MiraVia LLC, 2001

Page 37: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

38

Page 38: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

NEVAZOH

• Take a minute and read H4.3• Highlight key points about Nevazoh

39

Page 39: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

IDENTIFY STUDENT LEARNING PROBLEM: DRILL DOWN DEEP

40

StudentLearningProblem

Aggregate / Summary Reports

Disaggregated Results

Cluster / Strand / Subscale

Item Analysis

Student Work

Triangulate Student Learning Data

1 2 3

Page 40: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

BACKGROUND ON NEVAZOH

• Grade 6-8 urban school in Midwestern state• 724 students

• 62% economically disadvantaged• 35% African American, 57% white, 6% multi-racial

• At Risk School – 42% of 6th graders proficient in science in 2004-2005, 33% in mathematics, 51% in reading

• The school did not meet adequately yearly progress and has been placed on “Academic Watch.”

41

Page 41: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

THREE-YEAR AGGREGATE DATA TRENDS:SIXTH-GRADE MATHEMATICS FROM H4.5

42

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

61 61

53

44 47

3336

3125

0

10

20

30

40

50

60

70

80

90

100

2001 - 2002 2002 - 2003 2003 - 2004

State

School

District

Percentage of students at and above proficiency

Page 42: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

AGGREGATE DATA

• What are the benefits of aggregate data?• What limitations do these data have?

43

Page 43: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

IDENTIFY STUDENT LEARNING PROBLEM: DRILL DOWN DEEP

44

StudentLearningProblem

Aggregate / Summary Reports

Disaggregated Results

Cluster / Strand / Subscale

Item Analysis

Student Work

Triangulate Student Learning Data

1 2 3

Page 44: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA-DRIVEN DIALOGUE

45

PHASE 1Predict

PHASE 3Infer/Question

Adapted from Organizing Data-Driven Dialogue by Laura Lipton Bruce Wellman, MiraVia LLC, 2001

PHASE 2Observe

GoVisual

Page 45: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

PHASE 1: PREDICT STARTERS

46

I predict…

I assume…

I wonder…

I’m expecting to see…

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 46: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

THREE-YEAR DISAGGREGATED DATA: PHASE 1, PREDICT

• What do you think the disaggregated data will represent?

• What, if any, achievement gaps do you expect to see reflected in the data?

• Document your predictions on chart paper.

47

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 47: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

GO VISUAL

• Create a graph or graphs to display the disaggregated data from H4.6

48

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 48: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

49

6169

74 77

22 2128

3232 3439 41

0

10

20

30

40

50

60

70

80

90

'90 '92 '96 '00

%White

Black

Hispanic

ExampleExamplePercentage At or Above Basic Proficiency in Mathematics by Percentage At or Above Basic Proficiency in Mathematics by

Race/Ethnicity- NAEP 1990-2000Race/Ethnicity- NAEP 1990-2000

Page 49: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA-DRIVEN DIALOGUE

50

PHASE 1Predict

PHASE 3Infer/Question

Adapted from Organizing Data-Driven Dialogue by Laura Lipton Bruce Wellman, MiraVia LLC, 2001

PHASE 2Observe

GoVisual

Page 50: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA-DRIVEN DIALOGUEPHASE 2: OBSERVE

• What important points seem to pop out?• What is surprising, unexpected? • What gaps do you observe? • Always document your observations on chart

paper.

51

Adapted from Lipton and Wellman, 1999

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 51: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

OBSERVATION REMINDERS

• Made by the five senses• Quantitative and qualitative• Contain no explanations

52

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 52: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

53

BECAUSE

Page 53: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

CONCEPT ATTAINMENT TESTERS

• Our teachers aren’t emphasizing basic skills enough

• 45% of our eighth graders are not meeting the standard in computation

• Teachers aren’t teaching inquiry-based science because they feel too much pressure to cover the curriculum

• On a recent survey, a majority of elementary teachers reported that they needed more professional development in science content

54

Page 54: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

NOW TAKE 10 MINUTES TO DOCUMENT YOUR OBSERVATIONS

55

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 55: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

REFINING OBSERVATIONS• Does each statement communicate a

single idea about student performance?• Are the statements short and clear?• Do the statements use everyday language that

is easy to understand?• Do the statements incorporate numbers or

phrases that quantify data? • Are the statements consistent with the way in

which the data are reported?• Now, go back and put a star next to the

predictions that were confirmed

56

Source: The ToolBelt: A Collection of Data-Driven Decision-making Tools for Educators. Copyright© 2004 Learning Point Associates. All rights reserved. For a more complete account of this process, see http://www.ncrel.org/toolbelt/tools.htm

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 56: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

57

What’s the final step in the data dialogue process?

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 57: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA-DRIVEN DIALOGUE

58

PHASE 1Predict

PHASE 3Infer/Question

Adapted from Organizing Data-Driven Dialogue by Laura Lipton Bruce Wellman, MiraVia LLC, 2001

PHASE 2Observe

GoVisual

Page 58: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DISAGGREGATED DATA:PHASE 3, INFER

• What questions do you have about the data?• What school factors could be contributing to

lowered student learning?• Why do the gaps exist?• Document your inferences on chart paper.

59

Page 59: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DISAGGREGATED DATA: CAUTIONS

• Look at trends over time• Consider sample size. Small numbers within

subgroups can lead to wide variation in results• Consider more than one statistic – not just

percentage proficient and above• Improve program for all students

60

Page 60: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STUDENT LEARNING: UDP CORE VALUE

61

We are committed to acting consistently with our belief that all students can learn and to

being active anti-racists in our schools and our own work. We are committed to working with schools to close achievement gaps within one

to five years.

Page 61: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STUDENT LEARNING: UDP CORE VALUE (CONTINUED)

62

We believe that closing achievement gaps is not only a moral responsibility, it is a

very real possibility — within one to five years of implementing a powerful professional

development program based on collaborative inquiry.

Page 62: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

THE DRIVING QUESTION:

•What does race, class and gender have to do with it?

63

Page 63: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

64

“. . .very little will change for these disenfranchised students unless we directly confront racial, class, cultural and gender biases and the inequitable practices they spawn. Reform that does not put equity center stage has not and will not bring about high levels of mathematics and science achievement all.”

Nancy Love

Powerful Words

Page 64: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

IDENTIFY STUDENT LEARNING PROBLEM: DRILL DOWN DEEP

65

StudentLearningProblem

Aggregate / Summary Reports

Disaggregated Results

Cluster / Strand / Subscale

Item Analysis

Student Work

Triangulate Student Learning Data

1 2 3

Page 65: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STRAND DATA ANALYSIS:QUESTIONS TO CONSIDER

• What content strands are the test items actually measuring? What standards or learning outcomes are being tested?

• What are areas of relative strength and weakness in our students’ performance on content strands?

• Ideally, look at subgroups of student performance in strand areas: How do subgroups of students perform in the strands? Are there achievement gaps among student subgroups?

66

Page 66: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

67

What’s the first step in the data dialogue process?

Page 67: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA-DRIVEN DIALOGUE

68

PHASE 1Predict

PHASE 3Infer/Question

Adapted from Organizing Data-Driven Dialogue by Laura Lipton Bruce Wellman, MiraVia LLC, 2001

PHASE 2Observe

GoVisual

Page 68: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STRAND LEVEL DATA: PHASE 1, PREDICT

• Use Handout H5.3 to help you predict for each strand.

• In which strand areas do you expect to have the lowest percentage of proficient students? The highest?

• In which levels of understanding do you expect to have the lowest percentage of proficient students? The highest?

• Document your predictions on chart paper.

69

Page 69: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

70

What’s the next step in the data dialogue process?

To go visual, use handout H5.7 for Nevazoh’s strand data and create and vertical plot.

Page 70: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STRAND DATA GO VISUAL EXAMPLE:VERTICAL PLOT

71

CRT Content (insert your terminology, e.g., strand, cluster, subscale) ______________ AnalysisSchool____________Assessment________Year____Content Area______________

Patterns, relations, functions63%

60% Problem-solving strategies

38% Number and number relations

100

90

80

70

60

50

40

30

20

10

0

Percentage of students at and above proficiency

Page 71: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STOPLIGHT HIGHLIGHT: MULTIPLE CHOICE ITEM ANALYSIS DATA

72

Adapted from NCREL, Data Retreat Facilitator Guide, 2001

Highlight Color

Green

Yellow

Pink

Meaning

Go! Meets expectations

Caution! Below expectations

Urgent! In immediate need of improvement

% Correct (our cutoffs)

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

70-100

50-69

Below50

Page 72: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

73

After we go visual what do we do?

Page 73: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA-DRIVEN DIALOGUE

74

PHASE 1Predict

PHASE 3Infer/Question

Adapted from Organizing Data-Driven Dialogue by Laura Lipton Bruce Wellman, MiraVia LLC, 2001

PHASE 2Observe

GoVisual

Page 74: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STRAND LEVEL DATA:PHASE 2, OBSERVE

• Which strand areas have more students at proficiency than above proficiency?

• Which strand areas are in the “red zone”?• Which strand areas are in the “green zone”? • Document your observations on chart paper.

75

Page 75: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

76

What’s the final step in the data dialogue process?

Page 76: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA-DRIVEN DIALOGUE

77

PHASE 1Predict

PHASE 3Infer/Question

Adapted from Organizing Data-Driven Dialogue by Laura Lipton Bruce Wellman, MiraVia LLC, 2001

PHASE 2Observe

GoVisual

Page 77: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STRAND LEVEL DATA:PHASE 3, INFER

• Looking at all of the data (proficiency percentages, learning objectives, and number of items on the test), what do you think students do and do not understand and know about mathematics?

• What school factors could be contributing to lowered student learning?

• Document your inferences on chart paper.

78

Page 78: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

WHAT CAN WE LEARN FROM STRAND DATA?

• Specific information about learning objectives or content areas that students do and do not know or understand

• Provide information for further investigation — if only three or four test items within a strand, we need to look at more information

• Disaggregate! If you have access to disaggregated strand data, you have access to a wealth of information!

79

Page 79: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STRAND DATA: CAUTIONS

• Different tests define strands differently. Don’t assume that strands that are named similarly measure the same skills or understanding

• A small number of items within any given strand can skew results

• Strand analysis is most useful when combined with item and student work analysis

80

Page 80: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

FOCUS ON: NEVAZOH STRAND DATA

• Review observations and inferences on the strand (cluster) data

• Looking at all of the data (proficiency percentages, learning objectives, and number of items on the test), what do you think students do and do not understand and know about mathematics?

81

Page 81: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STAND AND DELIVER REFLECTION

82

I understand…

I wonder…

I am struck by…

Page 82: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

IDENTIFY STUDENT LEARNING PROBLEM: DRILL DOWN DEEP

83

StudentLearningProblem

Aggregate / Summary Reports

Disaggregated Results

Cluster / Strand / Subscale

Item Analysis

Student Work

Triangulate Student Learning Data

1 2 3

Page 83: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

ITEM ANALYSIS FOR NU - PART 1

84

TestPart

Strand Outcome#

Item#

CorrectAnswer

A--0

B--1

C--2

D--3 4

Blank Bldg%

Dist%

State%

N =Bldg

M NU 10 1 C 15 23 49 12 0 0 49 48 62 282M NU 10 7 S 47 32 12 0 0 10 12 9 20 282M NU 06 16 D 4 7 4 85 0 0 85 81 87 282M NU 07 19 S 47 15 28 0 0 9 28 23 45 282M NU 08 20 D 6 27 11 56 0 0 56 59 73 282M NU 06 22 B 16 38 26 20 0 1 38 37 53 282

M NU 09 35 D 50 0 33 17 0 0 17 25 40 282M NU 08 41 D 9 12 39 37 0 3 37 27 44 282M NU 06 45 B 33 50 8 4 0 6 50 48 58 282

Nevazoh 6th-Grade Mathematics 2004-2005 CRT AssessmentItem Data - Percent Correct

Answers reported in percent

Correct Answer Column: S = short answer E = extended response

Data is for illustration only. Source: Ohio Department of Education, www.ode.state.oh.us (permission pending)

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 84: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STOPLIGHT HIGHLIGHT: MULTIPLE CHOICE ITEM

ANALYSIS DATA

85

Adapted from NCREL, Data Retreat Facilitator Guide, 2001

Highlight Color

Green

Yellow

Pink

Meaning

Go! Meets expectations

Caution! Below expectations

Urgent! In immediate need of improvement

% Correct (our cutoffs)

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

70-100

50-69

Below50

Page 85: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

ITEM ANALYSIS FOR NU - PART 1

86

Test Part

Strand Outcome #

Item #

Correct Answer

A -- 0

B -- 1

C -- 2

D -- 3

4

Blank Bldg %

Dist %

State %

N = Bldg

M NU 10 1 C 15 23 49 12 0 0 49 48 62 282 M NU 10 7 S 47 32 12 0 0 10 12 9 20 282 M NU 06 16 D 4 7 4 85 0 0 85 81 87 282 M NU 07 19 S 47 15 28 0 0 9 28 23 45 282 M NU 08 20 D 6 27 11 56 0 0 56 59 73 282 M NU 06 22 B 16 38 26 20 0 1 38 37 53 282

M NU 09 35 D 50 0 33 17 0 0 17 25 40 282 M NU 08 41 D 9 12 39 37 0 3 37 27 44 282 M NU 06 45 B 33 50 8 4 0 6 50 48 58 282

Nevazoh 6th-Grade Mathematics 2004-2005 CRT AssessmentItem Data - Percent Correct

Answers reported in percent

Correct Answer Column: S = short answer E = extended response

Data is for illustration only. Source: Ohio Department of Education, www.ode.state.oh.us (permission pending)

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 86: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

ITEM ANALYSIS - PART 2

87

Test Part

Strand Outcome #

Item #

Correct Answer

A -- 0

B -- 1

C -- 2

D -- 3

4

Blank Bldg %

Dist %

State %

N = Bldg

M NU 10 1 C 15 23 49 12 0 0 49 48 62 282 M NU 10 7 S 47 32 12 0 0 10 12 9 20 282 M NU 06 16 D 4 7 4 85 0 0 85 81 87 282 M NU 07 19 S 47 15 28 0 0 9 28 23 45 282 M NU 08 20 D 6 27 11 56 0 0 56 59 73 282 M NU 06 22 B 16 38 26 20 0 1 38 37 53 282

M NU 09 35 D 50 0 33 17 0 0 17 25 40 282 M NU 08 41 D 9 12 39 37 0 3 37 27 44 282 M NU 06 45 B 33 50 8 4 0 6 50 48 58 282

Nevazoh 6th-Grade Mathematics 2004-2005 CRT AssessmentItem Data

Answers reported in percent

Correct Answer Column: S = short answer E = extended response

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Data is for illustration only. Source: Ohio Department of Education. www.ode.state.oh.us (permission pending)

Page 87: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STOPLIGHT HIGHLIGHT: MULTIPLE CHOICE ITEM DATA - DISTRACTOR

PATTERNS

88

Highlight Color Meaning % Correct

(our cutoffs)

Pink Urgent! In immediate need of improvement

Adapted from NCREL, Data Retreat Facilitator Guide, 2001

Highlight high-frequency INCORRECT selections.

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 88: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

ITEM ANALYSIS - PART 2

89

Test Part

Strand Outcome #

Item #

Correct Answer

A -- 0

B -- 1

C -- 2

D -- 3

4

Blank Bldg %

Dist %

State %

N = Bldg

M NU 10 1 C 15 23 49 12 0 0 49 48 62 282 M NU 10 7 S 47 32 12 0 0 10 12 9 20 282 M NU 06 16 D 4 7 4 85 0 0 85 81 87 282 M NU 07 19 S 47 15 28 0 0 9 28 23 45 282 M NU 08 20 D 6 27 11 56 0 0 56 59 73 282 M NU 06 22 B 16 38 26 20 0 1 38 37 53 282

M NU 09 35 D 50 0 33 17 0 0 17 25 40 282 M NU 08 41 D 9 12 39 37 0 3 37 27 44 282 M NU 06 45 B 33 50 8 4 0 6 50 48 58 282

Nevazoh 6th-Grade Mathematics 2004-2005 CRT AssessmentItem Data - Distractor Patterns

Answers reported in percent

Correct Answer Column: S = short answer E = extended response

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Data is for illustration only. Source: Ohio Department of Education. www.ode.state.oh.us (permission pending)

17

Page 89: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA-DRIVEN DIALOGUE

90

PHASE 1Predict

PHASE 3Infer/Question

Adapted from Organizing Data-Driven Dialogue by Laura Lipton Bruce Wellman, MiraVia LLC, 2001

PHASE 2Observe

GoVisual

Page 90: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

91

BECAUSE

Page 91: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA-DRIVEN DIALOGUEPHASE 2: OBSERVE

• What important points seem to pop out?• What patterns or trends are emerging?• What is surprising, unexpected?• What questions do we have now?• How can we find out?• Document your observations

92

Adapted from Lipton and Wellman, 1999

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 92: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

DATA-DRIVEN DIALOGUE

93

PHASE 1Predict

PHASE 3Infer/Question

Adapted from Organizing Data-Driven Dialogue by Laura Lipton Bruce Wellman, MiraVia LLC, 2001

PHASE 2Observe

GoVisual

Page 93: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

ITEM-LEVEL DATA: PHASE 3, INFER

• Examine frequently missed (pink) items and those with high-frequency incorrect (pink) responses

• What inferences and explanations might we draw as to why students are missing these items? Choosing the distractors?

• Record inferences on data wall• What additional data do we want to collect?• What questions do we have now?

94

PHASE 1 Predict

PHASE 3 Infer/Question

PHASE 2 Observe

GoVisual

Page 94: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

IDENTIFY STUDENT LEARNING PROBLEM: DRILL DOWN DEEP

95

StudentLearningProblem

Aggregate / Summary Reports

Disaggregated Results

Cluster / Strand / Subscale

Item Analysis

Student Work

Triangulate Student Learning Data

1 2 3

Page 95: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

USES AND PURPOSES OF STUDENT WORK

• Discuss with a partner:• For what purposes have you looked at

student work?• What protocols have you used?

96

Page 96: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

STUDENT WORK IN THE UDP PROCESS

• To triangulate and verify the student learning problem identified through CRT drill-down

• To better understand student thinking in relation to the identified problem

97

Page 97: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

PUTTING IT ALL TOGETHER

99

Aggregate Data All content areas, over time

Disaggregated DataAll content areas, all subgroups of students, over time

Strand DataStrands for all content areas, all subgroups of students, over time

ItemAnalysis of numerous items, disaggregated, all content areas, over time

Student WorkNumerous samples, disaggregated, over time

Page 98: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

HOW WILL THE UDP WORK AT MY SCHOOL?

• What kinds of things would you have to have in place to do this type of work effectively?• Put 5 things or ideas on post-its and post

• How can Irving implement this next year?• How can you help lead this type of dialogue at

your school?• How will you begin to use the data dialogue

process?• What are the ramifications of waiting another

year?

100

Page 99: 1 Prepared for Irving M.S. on May 10, 2006 USING DATA PROJECT OVERVIEW Collaboration between TERC and WestEd Funded by the National Science Foundation.

EVALUATIONS

101

Good luck!

Proceed with passion

and persistence!