USING OBSERVATION TO ANALYZE EFFECTIVENESS Marybeth Flachbart, Ed.D [email protected].

USING OBSERVATION TO

ANALYZE EFFECTIVENESS

Marybeth Flachbart, [email protected]

WWW.NEUHAUS.ORG

OBJECTIVES

3

1. Review research

2. Clarify types of walk-throughs

3. Identify areas of need through observational data

RESOURCES USED:LEADING FOR INSTRUCTIONAL IMPROVEMENT, FINK, MARKHOLT, 2011VISIBLE LEARNING, HATTIE, 2009 VISIBLE LEARNING FOR TEACHERS, HATTIE, 2012

4

LEADING FOR INSTRUCTIONAL IMPROVEMENTSTEPHEN FINK, ANNEKE MARKHOLT

5

6

VISBILE LEARNING: A SYNTHESIS OF OVER 800 META-ANALYSES RELATING TO ACHIEVEMENT

John Hattie, Professor of Education and Directorof the Visible Learning Labs, University of Auckland

EFFECT SIZE MOVES US BEYOND DID IT WORK

7

• Effect size is a way of quantifying the size of the difference between two groups.

• How well does it work? • Generally used in meta-analysis for

combining and comparing estimates from different studies.

BASIS OF HATTIE’S CONCLUSION

800+ META-ANALYSIS 50,000 STUDIES, AND 200+ MILLION STUDENTS

HATTIE’S RANGE OF EFFECT SIZE

9

.07

Effect Size

Benchmark .40

Enhanced – closer to 1.0

10

VISBILE LEARNING: A SYNTHESIS OF OVER 800 META-ANALYSES RELATING TO ACHIEVEMENT

John Hattie, Professor of Education and Directorof the Visible Learning Labs, University of Auckland

11

Raise your hand if you’ve read any or all of these books

12

If you have read any of the books:

Which one(s)?

What did you think of it?

Please provide answers in the comment box

DISPELLING THE WIDGET MYTH

13

THE WIDGET EFFECT, NEW TEACHER PROJECT, 2009

• Surveyed 14 large American School Districts• 98% of teachers were evaluated as “satisfactory”

“Based on such findings, many have characterized classroom observation as a hopelessly flawed approach to assessing teacher effectiveness. “

14

http://widgeteffect.org/

CAN CLASSROOM OBSERVATIONS IDENTIFY PRACTICES THAT RAISE ACHIEVEMENT?

15

http://educationnext.org/evaluating-teacher-effectiveness/

YES!

17

Defining Classroom Management• Room setup• Access to materials• Transition time• Procedures • Grouping• Climate

EFFECT SIZE:CLASSROOM BEHAVIORAL .80

CLASSROOM MANAGEMENT .52

18

19

Defining Instructional Practice• Link to standards• Use of questioning• Teacher led vs. student led• Cognitive demand of selected activities• Differentiation • Assessment practices• BICS vs. CALP

EFFECT SIZE:

FORMATIVE ASSESSMENT .90MICRO TEACHING .88TEACHER CLARITY .75RECIPROCAL TEACHING .74META-COGNITIVE STRATEGIES .69COOPERATIVE LEARNING .59QUESTIONING .64

20

21

Which is easier to observe, classroom management or instructional practices?

Respond in comment box

GATHERING DATA (FINK, MARKHOLT)

22

Types of walk-throughs• Learning walk• Goal setting• Implementation• Supervisory• Other

YOUR PRACTICE

23

Which types of walk-through do you do most frequently? Why?

1. Learning walk

2. Goal setting

3. Implementation

4. Supervisory

5. Other

OBSERVATIONAL DATA IS NOT EVALUATIVE!

“Stick to the facts m’am, just the facts.”

A Profession in Search of a Practice City and Elmore 2009

26

Developing the discipline of seeing• Seeing is a discipline• It is like a muscle – it gets stronger with

repetition• Foundation of Instructional Rounds

Creating Literacy Leaders Observation Protocol Classroom Observation Date: _____________ Grade Level: _____________________ Subject: ________________________________

Teacher Name: ___________________________________ Begin: _________________ End: _____________ Indicator Estimate % Teacher Directed Observer: _______________________________________________________ Student Directed Independent Work Other Total Y = Yes – Observed or Present N = No – Not observed or no occasion to be observed Classroom Management/Instruction Est.

Time Teacher Directed Whole Class/ Teacher Directed Small Group Est. Time

When waiting for assistance from the teacher, students are occupied with literacy related activities provided by the teacher.

Y N

Teacher clearly states the lesson’s topic, theme, and objectives. (Objective is linked to standards)

Y N

Teacher uses modeling, demonstration, and graphics. Y N

Teacher uses a variety of instructional modes. Y N Teacher explains directly and thoroughly. Y N Transitions between whole group and small group are brief and orderly.

Y N

Teacher uses prompting/cueing. Y N

Teacher has a Reading block schedule posted in the room. Y N Teacher reviews with questioning. Y N Students are engaged and on task. Y N Teacher Student Interaction Teacher re-teaches following questioning. Y N Student- Directed Groups/ Independent Work Teacher uses open-ended questions and encourages

elaboration. Y N

Teacher interacts instructionally with students (explaining, checking, giving feedback). Y N

Teacher encourages peer interaction. Y N

Teacher interacts managerially with students (reinforcing rules, procedures).

Y N Teacher encourages students to paraphrase, summarize, and

relate. Y N

Teacher encourages students to help each other with their work.

Y N

Teacher encourages students to check their own comprehension.

Y N

Observation Statistics (mark all that apply) Teacher encourages students to speak in complete sentences. Y N Observer present when lesson began? Y N Observer present when lesson ended? Y N Other Activity: Observer present during middle of lesson? Y N # of Students _____________ Aide present? Y or N How many?_____________ (If present, please make note of aide activity)

Using the Observation Protocol collect data that is:

• Descriptive not evaluative- just the facts!

• Specific

• About instructional core

• Related to problem of practice (or data needed to be collected)

OBSERVING CLASSROOM PRACTICE

“Teacher introduced the concept of fractions and students applied it in a hands-on activity.”

vs.

“Prompt for student essays: “What role did symbolism play in foreshadowing the main character’s dilemma?”

SPECIFICITY OF EVIDENCE

“Student 1 asked student 2: ‘what are we supposed to write down?’ Student 2 said, ‘I don’t know.’”

“The teacher read from a book that was not at the appropriate level for the class.”

“Teacher introduced the concept of fractions and students applied it in a hands-on activity.”

DESCRIPTION VS. JUDGMENTWITH

WITHOUT

“Students followed directions in the text to make circuit boards.”

“There was too much time on discussion, not enough time on individual work.”

“The students conducted a sophisticated lab experiment.”

DESCRIPTION VS. JUDGMENT

32

• What is the teacher doing and saying?

• What are the students doing and saying?

• What is the task?

WHEN OBSERVING, WHAT DO YOU SEE?

33

Possible data points:• Percentage of teacher vs. student talk• Formative assessment practices• Standard clear to students• Meta-cognitive strategies• Number of questions (types of questions)• Who is doing the questioning?

LEARNING WALK

LEARNING WALK CONTINUED

34

• Amount of time spent reading/writing/researching

• Cognitive demand of student work• Academic Vocabulary use• Level of engagement

What else have you learned by walking around?

TEACHER CHANNEL VIDEOS

HTTPS://WWW.TEACHINGCHANNEL.ORG.

35

36

PRACTICE:

Observing:

What do you see?• What is the teacher doing?• What are the students doing?

WHAT DO YOU WONDER?WHAT QUESTIONS WOULD YOU HAVE

FOR THE TEACHER?WHAT QUESTIONS WOULD YOU HAVE

FOR THE STUDENTS?

37

GOAL SETTING WALK THROUGH

38

GATHER DATA BEFORE YOU SET THE GOAL!

GOAL: IMPROVE WRITING ACHIEVEMENT BY 10%

40

Define the problem of practice:

Is it a lack of instruction in writing?

Is the lack of instruction in mechanics, process, or both?

Is it a lack of practice opportunities to write across the content areas?

Is it a lack of vocabulary?

Are the vocabulary deficits in

oral or written language?

SUPPOSE YOUR STUDENTS WRITING SKILLS WERE WEAK, AND YOU THEN OBSERVED INSTRUCTION…

41

“Based on my observation of 15 classrooms (visited once a week for three weeks), we need to improve the quality of interaction between faculty and students. We need to model and require academic language in both our oral and written communication.”

42

“If we model and then scaffold our students ability to use academic language, we will see an improvement in our students reading, writing, and speaking proficiency.”

THEORY OF ACTION

43

IMPLEMENTATION WALK-THROUGH

IMPLEMENTATION WALK

45

• When do you want to see this change in practice?

• How frequently will you gather implementation data?

• Who will gather the data?• How will that refine your theory of action?• Grade level, whole school, individual

teacher?

WHAT IF YOU SEE VERY LITTLE IMPLEMENTATION?

THOU SHALT…

47

• Only schedule professional development for faculty that you can measure in terms of a change in instructional practice

• Prepare a standard protocol for faculty and share it before observation

• Give sufficient time to implement and resources for job embedded coaching

BE SPECIFIC!

CLARIFY:

49

Message from observer:

Last Friday we had an in-service day where faculty shared their strategies for teaching writing across content areas. I am anxious to see how you’re incorporating these practices in your classroom. I will be doing walk-throughs beginning next week to see how the students are responding to their new learning. Attached is a form I’ll be use to gather data so that we can discuss student work at our PLCs in November. I look forward to seeing you in action.

INSPECT WHAT YOU EXPECT

50

Ask professional development providers (including staff) for an observation protocol

“If you can’t see it in the classroom, it doesn’t exist.”

Elmore

Inquiry Cycle: Habits of Thinking for Instructional Leadership

• Noticing – Describe specific facts related to the Observation Protocol that you recorded.

• Wondering – Based on what you saw (or didn’t see), what are you curious or would like to know more about?

• Analysis – Using what (collective) knowledge is known about what learning should look like in the ideal in relation to this context observed, think about this information as it compare with my noticing and wonderings.

• Theories – Based on multiple observations across all classrooms and preponderance of data, this is one theory (or problem of practice) about student learning is …..

• Leadership Implications – Using Theory as a guide this is one implication for professional development and my decisions as a leader. ( Gathering additional information and data to measure effectiveness)

TICKET OUT

As a result of this presentation what if anything will you add to your classroom observations?

Email response to:[email protected]