Data-Driven Instruction Mid-Year Follow-up Workshop

Data-Driven Instruction Mid-Year Follow-up Workshop

Paul Bambrick-Santoyo

P2

• In the face of great adversity, how did the Houston team respond?

• What key statements and actions helped the team save Apollo 13?

APOLLO 13

P3

GROUND RULES

• EVERY CHALLENGE IS REAL & UNIQUE

• THE EXPERTS ARE IN THE ROOM

• WE ABSOLUTELY CAN FIND SOLUTIONS TO ANY PROBLEM

• NO ISLANDS ALLOWED

• BOTTOM LINE: IMPROVE ACHIEVEMENT

• ACT NOW TO BE READY FOR TOMORROW

P4

P5

P6

OFF-RAMPS ON THE ACHIEVEMENT HIGHWAY

• Unaligned interim assessments

• No structured time in school day

• Infrequent interim assessments

• Externally analyzed

• No analysis done by teachers

• Re-teaching is stabbing in the dark

• No follow-up: no action

P7

DATA-DRIVEN INSTRUCTION AT ITS ESSENCE:

ASSESSMENTSANALYSIS

ACTIONin a Data-driven CULTURE

THE FOUR KEYS:

P8

PRINCIPLES FOR EFFECTIVE ASSESSMENTS:

COMMON INTERIM:• At least quarterly• Common across all teachers of the same grade level

DEFINE THE STANDARDS—ALIGNED TO:• To state test (format, content, & length)• To instructional sequence (curriculum)• To college-ready expectations

ASSESSMENTS:

P9

PRINICIPLES FOR EFFECTIVE ASSESSMENTS:REASSESSES:

• Standards that appear on the first interim assessment appear again on subsequent interim assessments

WRONG ANSWERS: • Illuminate misunderstanding

TRANSPARENT:• Teachers see the assessments in advance

ASSESSMENTS:

P10

DATA-DRIVEN INSTRUCTION AT ITS ESSENCE:

ASSESSMENTS (Interim, Aligned, Reassess, Transparent)

ANALYSIS

ACTION

in a Data-driven CULTURE

THE FOUR KEYS:

P11

• IMMEDIATE: Ideal 48 hrs, max 1 wk turnaround

• BOTTOM LINE: Includes analysis at question level, standards level and overall—how well did the students do as a whole

• TEST-IN-HAND analysis: Teacher & instructional leader together

• TEACHER-OWNED analysis

• DEEP: Moves beyond “what” to “why”

ANALYSIS:

P12

ASSESSMENTS (Aligned, Interim, Reassess, Transparent)

ANALYSIS(Quick, Bottom line, Teacher-owned, Test-in-hand, Deep)

ACTION


THE FOUR KEYS:

P13

• PLAN new lessons based on data analysis

• ACTION PLAN: Implement what you plan (dates, times, standards & specific strategies)

• LESSON PLANS: Observe changes in lesson plans

• ACCOUNTABILITY: Observe changes classroom observations, in-class assessments

• ENGAGED STUDENTS: Know end goal, how they did, and what actions they’re taking to improve

ACTION:

P14



ACTION(Action Plan, Accountability, Engaged)


THE FOUR KEYS:

P15

• VISION: Established by leaders and repeated relentless

• “REAL” LEADERSHIP TEAM: Trained and highly active

• CALENDAR: Calendar in advance with built-in time for assessments, analysis & action

• PROFESSIONAL DEVELOPMENT: Aligned

DATA-DRIVEN CULTURE:

P16




in a Data-driven CULTURE (Vision, Leadership, Calendar, PD)

THE FOUR KEYS:

Phases of Data-Driven Instruction & Interim Assessments

Adapted, Research by Camden County, GA Public School District

P18

IGNORANCE, CONFUSION, OVERLOAD:• “I don’t understand what we’re doing.”

• “This is too much! How am I really supposed to use all this?”

• “All this analysis! What’s wrong with just grading the old-fashioned way?”

• “Uh? Interim assessments? What are those?”

PHASE 1

P19

FEELING INADEQUATE & DISTRUSTFUL:• “How can two questions on a test possible establish

mastery? These tests can never measure what I know about my students’ learning.”

• “This idea of an assessment is terrible! We don’t teach like that format! We teach it this way.”

PHASE 2

P20

CHALLENGING THE TEST:• “Question #26 is a poor question. Answer “b” is a trick

answer.”

• “Question #11 is too hard. We need to make it easier.”

• “The kids made silly mistakes because of the pressure of this pointless test. They know this stuff.”

• Undertone: “I’ve never looked at a test item before, but I’m going to now if you’re going to hold me accountable.”

PHASE 3

P21

ANALYTICAL but SUPERFICIAL:• “They just don’t do well on word problems. I just need to

do more word problems.”

• “They just don’t read enough. I’ll get them to read more.”

PHASE 4

P22

LOOKING FOR CAUSES, BUT NO ACTION:• “These wrong answers tell me that they don’t know the

difference between a summary and a theme.”

• “I always taught grammar in isolation, and this test asked for it in a more authentic form.”

• The problem with solving algebraic equations for them was actually the inability to subtract negative integers.”

PHASE 5

P23

CHANGING TEACHING PRACTICES:• Teachers follow through on analysis

• Lesson plans reflect spiraling, re-teaching, etc.

• Teachers look for best practices outside of their own classroom

PHASE 6

P24

• Phase 1: Ignorance, confusion and overload—“This is too much!”

• Phase 2: Feeling inadequate and distrustful—“This test is terrible!”

• Phase 3: Challenging the test—“Question 26 was too hard—a trick question.”

• Phase 4: Analytical but superficial

• Phase 5: Looking for causes, no action

• Phase 6: Changing teaching practices

Phases of Interim Assessment Implementation

Self-Evaluation:Implementation of Data-Driven

Instruction

The Big Balloons:Challenges We Face

P27

1. NEVER LEFT THE STATION: I’m overwhelmed and/or frustrated; where do I begin?

2. NO TEACHER BUY-IN: How do I get teachers/leaders invested?

3. SURFACE TEACHER BUY-IN, LATER RESISTANCE: I thought they got it, but now they’re resisting. What do I do?

4. UNSUPPORTIVE MENTOR PRINCIPAL:What real results can I accomplish in this context?

5. STUDENT/PARENT BUY-IN: How do you get them invested in this process?

6. NO LEADERSHIP TEAM BUY-IN: How do I deal with a leadership team that doesn’t believe in the power of data-driven instruction?

CHALLENGES:

P28

7. RUNNING TEAM MEETINGS: How do I limit the impact of a negative teacher who’s trying to undermine the process? How do I bring experienced and novice teachers together effectively?

8. CITY/SCHOOL ASSESSMENTS ARE POOR: How do I adapt to mandatory weak assessments that aren’t aligned to state test?

9. CREATING ASSESSMENTS:How do I access quality interim assessment material? How do I build assessments for other subjects or that align to college readiness?

10. ANALYZING RESULTS: What effective templates are working? How do I alleviate time drain? Where do I begin when most standards are deficient?

CHALLENGES:

P29

11. TARGETED RE-TEACH—HOW?: What are effective strategies to reach mastery? How do I do differentiated instruction with teachers who can’t manage their class or haven’t done small groups before?

12. TIME:How do I creatively manage our school schedule to implement all aspects of data-driven instruction? How do I gain time for analysis/action if I cannot extend their day?

13. COMPETING INTERESTS: What are ways to implement data-driven instruction when there are so many other initiatives in the school?

14. “GOOD SCHOOLS/TEACHERS” PROBLEM: What are some strategies for getting staff on board who already have “good scores”? How do I create urgency?

15. ADJUSTING CALENDAR: How can I adjust the original NLNS data-driven calendar when I’m really just starting now? How do I work around a poor school calendar that isn’t what my school needs?

CHALLENGES:

Results Meeting ProtocolSummer Foundations Review

P31

• IDENTIFY ROLES: Timer, facilitator, recorder (2 min) • IDENTIFY OBJECTIVE to focus on (2 min or given)• WHAT WORKED SO FAR (5 min)

• [Or: What teaching strategies did you try so far]• CHIEF CHALLENGES (5 min)• BRAINSTORM proposed solutions (10 min)

• [See protocol on next page]• REFLECTION: Feasibility of each idea (5 min)

• CONSENSUS around best actions (15 min)• [See protocol on next page]

• PUT IN CALENDAR: When will the tasks happen? When will the teaching happen? (10 min)

ACTION: RESULTS MEETING 50 MIN TOTAL

P32

RESULTS MEETING STRUCTURE:PROTOCOLS FOR BRAINSTORMING/CONSENSUSPROTOCOL FOR BRAINSTORMING:• Go in order around the circle: each person has 30 seconds to

share a proposal.• If you don’t have an idea, say “Pass.” • No judgments should be made; if you like the idea, when it’s

your turn simply say, “I would like to add to that idea by…”• Even if 4-5 people pass in a row, keep going for the full

brainstorming time.

PROTOCOL FOR REFLECTION:• 1 minute—silent personal/individual reflection on the list: what

is doable and what isn’t for each person.• Go in order around the circle once: Depending on size of group

each person has 30-60 seconds to share their reflections.• If a person doesn’t have a thought to share, say “Pass” and

come back to that person later.• No judgments should be made.

P33

RESULTS MEETING STRUCTURE:PROTOCOLS FOR BRAINSTORMING/CONSENSUSPROTOCOL FOR CONSENSUS/ACTION PLAN:• ID key actions from brainstorming that everyone will agree to

implement• Make actions as specific as possible within the limited time• ID key student/teacher guides or tasks needed to be done to

be ready to teach—ID who will do each task • Spend remaining time developing concrete elements of lesson

plan:• Do Now’s• Teacher guides (e.g., what questions to ask the students or

how to structure the activity)• Student guides• HW, etc.

NOTE: At least one person (if not two) should be recordingeverything electronically to send to the whole group

Adjusted Results Meeting: Best Practice Sharing

P35

• Divide into subgroups (if large enough group): ID specific challenges each group will address (2 min)

SUBGROUPS:• Identify roles: Timer, facilitator, recorder, electronic copier

(2 min) • Share best practices Ask follow-up questions to

understand success (3-5 min each; 20 min)

• Imagine challenges others are having: Contemplate if your experience addresses those challenges (5 min)

• Consensus around best actions to “publish”: Make plans for what you can do in 30 min (10 min)

• Publish: Produce dazzling display of best practices and put on the wall around your balloon (20 min)

• Select a representative: Pick one expert to stay as guide

1 HR TOTALBEST PRACTICES SHARING:

Adjusted Results Meeting: Finding Solutions to Our Challenges

P37

FACING OUR CHALLENGES: FINDING SOLUTIONS

GROUP:• Identify roles (2 min)• Read through artifacts on the wall (10 min)• Share the challenges you are having (10 min)• Identify the best practices that would be useful for your context

(5 min)• Brainstorm how to make those practices 100% applicable to

your school building (5 min)

INDIVIDUAL:• Write concrete plan for your school upon return from workshop

(18 min)

50 MIN TOTAL

Popping the Balloons:SOLVING All Challenges

Analysis & Action:The Role of Leadership

P40




in a Data-driven CULTURE (Vision, Leadership, Calendar, PD)

THE FOUR KEYS:

Effective Analysis MeetingsVideo Case Studies

P42

IDENTIFY TEACHER LANGUAGE/ANALYSIS/ACTION:• What does the teacher say?• What preparation did they do for the meeting?• What was the quality of her analysis and what do you infer

about the quality of the follow-up action taken with her students?

OBSERVER #1: ANALYZE TEACHER’S ROLE

P43

GUIDING QUESTIONS—PRINCIPAL ANALYSIS GROUP:• What does the principal say? • How does he respond to teacher comments (and to what

does he choose NOT to respond)?• What are the tips you could take about leading analysis

meetings?

OBSERVER #2: ANALYZE PRINCIPAL’S ROLE

P44

GUIDING QUESTIONS FOR SYSTEMS ANALYSIS GROUP:• What data-driven systems are in place at the school that are

apparent in this meeting?• What preparation did the principal do to make this meeting

effective?• Are there additional systems or preparation that could make

the meeting more effective?

OBSERVER #3: ANALYZE SYSTEMS & PREPARATION

Addressing Resistance in Analysis Meetings

Role Plays

P46

• 6 WEEKS PRIOR TO INTERIM ASSESSMENTS: Teachers review assessment and plan toward the rigor of those assessments(TRANSPARENCY)

• A FEW WEEKS PRIOR: Teachers predict performance on each assessment question: a) confident they’ll get it right; b) not sure; c) no way they’ll get it right(TEST-IN-HAND, TEACHER-OWNED)

• PD (timing flexible): Teachers receive model of how to do assessment analysis and complete action plan, and they see model of effective & ineffective analysis meetings(PROF DEVT, DEEP)

PRECURSORS TO EFFECTIVE ANALYSIS MTGS: BEFORE GIVING INTERIM ASSESSMENT

P47

• TEACHER ANALYSIS: Teachers do analysis of results prior to meeting, answering the fundamental question: WHY did the students not learn it? (TEACHER-OWNED, DEEP)

• TEACHER ACTION PLAN: Teachers complete an action plan (ACTION PLAN, ACCOUNTABILITY)

• LEADERSHIP ANALYSIS: Leader analyzes teacher results in preparation for the meeting (LEADERSHIP)

• REVIEW OF TEACHER PLAN: Leader collects teacher action plan/analysis ahead of time to see if it looks acceptable (LEADERSHIP, ACCOUNTABILITY)

• CONTENT EXPERTISE: Leader has a plan ready to access content experts if the problems are beyond own expertise (PROF DEVT)

PRECURSORS TO EFFECTIVE ANALYSIS MTGS: AFTER INTERIM ASSESSMENT GIVEN

P48

• Let the data do the talking

• Let the teacher do the talking (or get them to!)

• Always go back to the test and back to specific questions

• Don’t fight the battles on ideological lines (you’re going to lose)

• There’s a difference between the first assessment and the third

• You’ve got to know the data yourself to have an effective meeting

• Make sure it’s connected to a concrete plan that you can verify

TIPS FOR EFFECTIVE ANALYSIS MEETINGS:

P49

HELPFUL STARTERS FOR ANALYSIS MEETINGS:• “So…what’s the data telling you?”• “Congratulations on the improvement from last time in x

area! You must be really proud of their growth here.”• “So the _____ [paraphrase their frustration: the test was

hard, the students were difficult, etc.]? I’m sorry to hear that. So where should we begin with our action plan moving forward?”

ANALYSIS MEETING HELPFUL PHRASES:

P50

DATA-FOCUSING FOR ANALYSIS MEETINGS:• “So let’s look at question 18…..Why do you think they got it

wrong?”• “You know, I thought it might be a silly mistake, but what

surprised me is that they did really well on questions x & y. Why do you think they did so well on these questions and yet not on your original question?"

• “Let’s look at question 11. What did the students need to be able to do to answer that question effectively? Is this more than they are able to do with you in your class?”

• [When new ideas occur or deeper analysis is done at the meeting than what teacher did previously] “So let’s re-visit the action plan you created and see how we can incorporate these additional ideas.”

ANALYSIS MEETING HELPFUL PHRASES:

P51

ASSESSMENTS—QUESTIONS ON QUADRATIC EQUATIONS:

1. Solve the following quadratic equation:

2. Given the following rectangle with the lengths shown below, find the value of x:

3. You need to build a box where one side of the bottom of the box is one foot less than two times the length of the other and the total area of the bottom of the box is 6 ft2. What should the length & width dimensions of your box be?

062 2 xx

12 x

x

Area = 6

If You Had to Choose:Time Management Exercise

Kim Marshall’s Case Study

P53

• FULL OBSERVATION: Observe & evaluate teachers with pre-conference, full-class observation, detailed write-up, and post-conference

• TARGETED WALKTHROUGHS: Systematic walk-throughs focused on target areas (student work, engagement, objectives, binders, worksheets)

• MINI-OBSERVATIONS: Mini-observations of 3-5 classrooms each day (5-15 minutes per visit) with face-to-face follow-up conversation

• GLOBAL WALK-THROUGHS: Walk through all classrooms each day (1-5 minutes each class)

IF YOU HAD TO CHOOSE THREE:

P54


• LESSON PLAN REVIEW: Collecting & checking teachers’ lesson plans each week, with dialogue sheet for teacher-to-principal conversation

• UNIT PLAN CREATION: Supervising teachers & teams to submit common curriculum unit plans in advance

• INTERIM ASSESSMENT LAUNCH: Managing teachers & teams to use interim assessments to plan whole-group and individual instruction

P55


• TARGETED WALK-THROUGHS: Systematic walk-throughs focused on target areas (student work, engagement, objectives, binders, worksheets)




P56





P57


• TARGETED WALK-THROUGHS: Systematic walk-throughs focused on target areas (student work, engagement, objectives, binders, worksheets)




P58





Applications:Best Use of 1 hr/wk with Teachers

P60

PRE-WORK:• INTERIM ASSESSMENT & LESSON PLAN REVIEW: Checking

teachers’ lesson plans alongside key re-teach standards (action plan) or standards on next assessment identifying:• What are the key moments to observe to check for rigor

or alignment?• Are the standards well embedded in an engaging lesson?

• MINI-OBSERVATIONS: Mini-observations of class each day (5-15 minutes per visit) with brief feedback in hallways

IF YOU HAVE 1 HR/WK OF FACE TIME: PRE-WORK

P61

INDIVIDUAL FACE TIME—1st PRIORITY:• DATA-DRIVEN PROCESS IS FIRST DRIVER OF AGENDA:

• Help teacher analyze interim assessment data & make action plan (2-6 days after asst; important first time through)

• Results analysis meeting (once teacher has done analysis/action plan before meeting with you)

• Review upcoming assessment (6 weeks before)• Anticipate student performance by marking interim

assessment questions “confident,” “not sure,” “no way” (1-2 weeks before asst)

DURING REMAINING TIME/WEEKS:• FEEDBACK ON CLASSES OBSERVED: Dialogue—affirmation of

strengths, suggestions for improvement• 1-2 goals maximum• Make concrete follow-up plan if needed

• PLANNING FOR THE FUTURE: Review action plan/lesson plans, anticipate/plan key moments for upcoming week

IF YOU HAVE 1 HR/WK OF FACE TIME: INDIVIDUAL MEETINGS

P62

TEAM MEETINGS:• RESULTS MEETING PROTOCOL:

• Target just 1 standard to visit for the meeting• If important, redo results meeting around same standard

the following week to evaluate the effectiveness of their implementation

• REVIEW STUDENT WRITING:• Make action plan (2-6 days after asst)• Review upcoming assessment (6 weeks before)• Anticipate student performance by marking interim

assesment questions “confident,” “not sure,” “no way” (1-2 weeks before asst)

• MAKE ACTION PLANS: Mini-observations of class each day (5-15 minutes per visit) with brief feedback in hallways

IF YOU HAVE 1 HR/WK OF FACE TIME:TEAM MEETINGS

Action with Accountability: Concrete Strategies for the Classroom

Uncommon Guide: Adding Rigor Throughout the Lesson

Putting It into Action:3rd Grade Guided Reading Block

A Video Case Study

P65

• CLASS SCHEDULE: Take 10-15 minutes from math block every day to re-teach; find places in the schedule where you could gain learning time for students; utilize lunch/breakfast time

• SCHOOL SCHEDULE: Build in pre-school/after-school tutoring, use lunch/breakfast time to target students in need of improvement

• PULLOUT GROUPS: Other staff members (including you!) pull out small groups of students to work on toughest standards (highest or lowest achieving subgroups) so main teacher can focus on smaller class

SCHOOLWIDE SUPPORT

P66

• ADDITIONAL STAFF: Solicit support of non-core academic staff to teach small groups of students (paraprofessionals, art/PE teachers, SPED, ELL, tutors, parents, etc.)

• EVERY LEADER ON BOARD: All leaders (deans, VPs, dept. chairs, coaches) support the process—target teachers who need most help, follow results meeting protocol, etc.

• PULL: Rotate students into groups with similar skill deficiencies and teach them in those groups for select time periods; rotate groups when in new subjects

SCHOOLWIDE SUPPORT (con’t.)

Final Steps:Building Your Own Action Plan

Plan Implementation in Your School

P68

• ID MOST IMPORTANT ACTIONS TO TAKE FROM THIS WORKSHOP

• Address Challenges• Prepare more effective analysis meetings• Link teacher face-time with data-driven student

achievement• Etc.

• PUT IT ON THE CALENDAR:• Use your implementation calendar and adjust as

radically as you need to get it done!

• LEADERSHIP APPROVAL: • Who are the key players you need around the table to

make this happen? • What is the most effective persuasion/conversation?• If they say no, how can you creatively comply and still

implement your project?

MAKING YOUR OWN ACTION PLAN:

Conclusions

Data-Driven Instruction & Assessment

Paul Bambrick-Santoyo

P70

E.L. HAYNES PUBLIC SCHOOL, WASHINGTON, DC

Jennie Niles, Eric Westendorf & Michelle Molitor,Cohorts 2,6 & 7

DCCAS RESULTS Language Arts Mathematics

Year Tested% Proficient / Adv

Proficient% Proficient / Adv

Proficient

2005-06 40% 28%2006-07 43% 48%2007-08 60% 67%2-year gains: +20 +39

WASHINGTON, DC IN ACTION:

Winner, $100,000, Top Public School in DC

Data-Driven Instruction Mid-Year Follow-up Workshop

Documents

assessments interim

data analysis action

transparent analysis

hand analysis

datadriven culture vision

transparent analysis

deep action action plan

teacherowned analysis