Getting Serious about School Reform: Three Critical Commitments Districts and Schools Should Make
Dec 29, 2015
Getting Serious about School Reform:
Three Critical Commitments Districts and Schools Should Make
40 Years
Same Problems
The problem isn’t at the classroom teacher level.
The problem is at the systems level.
Teacher School District Student
P50 P50 P50 P50
Tentative Findings Reading
Teacher School District Student
P50 P50 P50 P50
P84 P50 P50 P60
Tentative Findings Reading
Teacher School District Student
P50 P50 P50 P50
P84 P50 P50 P60
P98 P50 P50 P70
Tentative Findings Reading
Teacher School District Student
P50 P50 P50 P50
P84 P50 P50 P60
P98 P50 P50 P70
P50 P84 P84 P57
Tentative Findings Reading
Teacher School District Student
P50 P50 P50 P50
P84 P50 P50 P60
P98 P50 P50 P70
P50 P84 P84 P57
P50 P98 P98 P63
Tentative Findings Reading
Teacher School District Student
P50 P50 P50 P50
P84 P50 P50 P64
P98 P50 P50 P76
P50 P84 P84 P59
P50 P98 P98 P67
Tentative Findings Mathematics
Districts and schools must find ways to ensure effective practices occur in every
classroom, but still provide flexibility to the classroom
teacher.
For decades we have operated from the belief that this is
impossible in U.S. education.
Weick (1976,1982)Tightly Coupled Organizations…
• Clearly defined unit of accountability
• Clearly defined criterion for success
• Clear understanding of what produces success
• Entire system mobilizes when criterion for success is not met.
Weick’s Conclusions
Districts and schools are not tightly coupled regarding individual student achievement as the unit of analysis for success.
Weick’s Conclusions
Districts and schools are not tightly coupled regarding the
criterion of student achievement,
BUT THEY ARE TIGHTLY COUPLED REGARDING
THE BUS SCHEDULE AND PAYROLL.
Based on the notion that schools are inherently loosely-coupled we began operating
from a “hands offs” pespective at the classroom level.
Everybody is equal.
The classroom teacher is an autonomous, independent
contractor.
Within a loosely coupled district or school, there will be
individual points of light but no possibility of a strong, unified
focus that can eradicate the 40 year old problems inherent in
the system.
Bellamy and Colleagues (2005)Highly Reliable Organizations (HROs)
Bellamy and Colleagues (2005)Highly Reliable Organizations (HROs)
…• Clearly defined unit of accountability
• Clearly defined criterion for success
• Clear understanding of what produces success
• Entire system mobilizes when criterion for success is not met.
Bellamy’s Conclusions
Districts and schools are not tightly coupled (HROs) regarding individual student achievement as the unit of analysis for success.
Bellamy’s ConclusionsDistricts and schools are not tightly coupled regarding the
criterion of student achievement,
BUT THEY CAN BE IF
THEY ARE WILLING TO COMMIT TO SPECIFIC ACTIONS THAT ARE DISTRICTWIDE (OR
SCHOOLWIDE).
Three Critical Interventions (COMMITMENTS)
• A system of individual student feedback on learning goals at the classroom, school, and district levels (standards-based grading using formative assessments)
• Ensuring effective teaching in every classroom
• Building background knowledge for all students
All three can be approached at the school level but are more powerful at the district level.
Three Critical Interventions(COMMITMENTS)
• A system of individual student feedback on learning goals at the classroom, school, and district levels (standards-based grading using formative assessments)
• Ensuring effective teaching in every classroom• Building background knowledge for all students
Phase I: Track student progress using a formatively-based system.
This draws on the research from formative assessments.
Feedback from classroom assessments should provide students with a clear picture of their progress on learning goals and how they might improve.
# of studies Characteristic of Feedback from Classroom Assessment
Percentile Gain/Loss
Bangert-Drowns, Kulik, Kulik, & Morgan,1991
6 Right/wrong -3
39 Provide correct answers 8.5
30 Criteria understood by student vs. not
understood
16
9 Explain 20
4 Student reassessed until correct
20
Feedback from classroom assessments should provide students with a clear picture of their progress on learning goals and how they might improve.
# of studies Characteristic of Feedback from Classroom Assessment
Percentile Gain/Loss
Fuchs & Fuchs 1988
89 Displaying results graphically
26
49 Evaluation by rule
[uniform way of interpreting results of
classroom assessments using a tight logic)
32
49 Evaluation by rule
[uniform way of interpreting results of classroom assessments using a tight logic)
32
You can’t rely on the 100-point scale.
C. Item 15–16
Two items that asks for application in novel situations that go beyond what was explicitly taught
Total for section =
Total for section =
Total for section =
A. Items 1–10
Ten items that require recall of important but simpler content that was explicitly taught
B. Items 11–14
Four items that ask for application of complex content that was explicitly taught AND in situations similar to what was taught.
Total /100
Total /100
Total for section=
Total for section=
Total for section=
/40
/20
/40
A. Items 1–10
Ten items that require recall of important but simpler content that was explicitly taught
B. Items 11–14
Four items that ask for application of complex content that was explicitly taught AND in situations similar to what was taught.
C. Item 15–16
Two items that asks for application in novel situations that go beyond what was explicitly taught
+
+
Total for section =
Total for section =
Total for section =
All correct
Two correct
None correct
A. Items 1–10
Ten items that require recall of important but simpler content that was explicitly taught
B. Items 11–14
Four items that ask for application of complex content that was explicitly taught AND in situations similar to what was taught.
C. Item 15–16
Two items that asks for application in novel situations that go beyond what was explicitly taught
Total /100
/40
/20
/40
Total /100
+
+
Total for section =
Total for section =
Total for section =
40/40
20/40
0/20
All correct
Two correct
None correct
A. Items 1–10
Ten items that require recall of important but simpler content that was explicitly taught
B. Items 11–14
Four items that ask for application of complex content that was explicitly taught AND in situations similar to what was taught.
C. Item 15–16
Two items that asks for application in novel situations that go beyond what was explicitly taught 60
4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go BEYOND what was taught in class
3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught
2 No major errors or omissions regarding the SIMPLER details and processes BUT major errors or omissions regarding the more complex ideas and processes
1 With HELP, a partial knowledge of some of the simpler and complex details and processes
0 Even with help, no understanding or skill demonstrated
Scale
4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go BEYOND what was taught in class
3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught
2 No major errors or omissions regarding the SIMPLER details and processes BUT major errors or omissions regarding the more complex ideas and processes
1 With HELP, a partial knowledge of some of the simpler and complex details and processes
0 Even with help, no understanding or skill demonstrated
Scale
4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go BEYOND what was taught in class
3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught
2 No major errors or omissions regarding the SIMPLER details and processes BUT major errors or omissions regarding the more complex ideas and processes
1 With HELP, a partial knowledge of some of the simpler and complex details and processes
0 Even with help, no understanding or skill demonstrated
Scale
4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go BEYOND what was taught in class
3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught
2 No major errors or omissions regarding the SIMPLER details and processes BUT major errors or omissions regarding the more complex ideas and processes
1 With HELP, a partial knowledge of some of the simpler and complex details and processes
0 Even with help, no understanding or skill demonstrated
Scale
4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go BEYOND what was taught in class
3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught
2 No major errors or omissions regarding the SIMPLER details and processes BUT major errors or omissions regarding the more complex ideas and processes
1 With HELP, a partial knowledge of some of the simpler and complex details and processes
0 Even with help, no understanding or skill demonstrated
Scale
4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go BEYOND what was taught in class
3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught
2 No major errors or omissions regarding the SIMPLER details and processes BUT major errors or omissions regarding the more complex ideas and processes
1 With HELP, a partial knowledge of some of the simpler and complex details and processes
0 Even with help, no understanding or skill demonstrated
Scale
4 In addition to exhibiting level-3 performance, in-depth inferences and applications that go beyond what was taught in class
3.5 In addition to exhibiting level-3 performance, partial success at in-depth inferences and applications that go beyond what was taught in class
3 No major errors or omissions regarding any of the information and/or processes (SIMPLE OR COMPLEX) that were explicitly taught
2.5 No major errors or omissions regarding any of the simpler information and/or processes and partial knowledge of the more complex information and processes
2 No major errors or omissions regarding the simpler details and processes BUT major errors or omissions regarding the more complex ideas and processes
1.5 Partial knowledge of the simpler details and processes, but major errors or omissions regarding the more complex ideas and processes
1 With help, a partial knowledge of some of the simpler and complex details and processes
.5 With help, a partial knowledge of some of the simpler details and processes but not of the more complex ideas and processes
0 Even with help, no understanding or skill demonstrated
Scale
C. Item 15–16 (SCORE 4.0)
Two items that asks for application in novel situations that go beyond what was explicitly taught
ALL CORRECT
2 OF 4 CORRECT
NONE CORRECT
A. Items 1–10 (SCORE 2.0)
Ten items that require recall of important but simpler content that was explicitly taught
B. Items 11–14 (SCORE 3.0)
Four items that ask for application of complex content that was explicitly taught AND in situations similar to what was taught.
RUBRIC SCORE= 2.5
This represents a shift from an emphasis on isolated classroom
activities for which points are assigned to an emphasis on student progression
toward specific learning goals.
This approach is to be used as another measurement tool in a system of
inherently imprecise measurement tools.
Cizek (2007)State test for a large Midwestern state: rel = .87
Subscales: Estimation and mental computation, geometry, measurement, number and number relations, patterns, algebra, problem solving
Subscale reliabilities = .33 to .57
Cizek (2007)State test for a large Midwestern state: rel = .87
Subscales: Estimation and mental computation, geometry, measurement, number and number relations, patterns, algebra, problem solving
Subscale reliabilities = .33 to .57
Reliability of differences scores = .015
…It …might be that the dependability of conclusions about differences in sub-area performance is nearly zero.
In many cases, a teacher who flipped a coin to decide whether to provide the pupil with focused intervention in algebra (heads) or measurement (tails) would be making the decision about as accurately as the teacher who relied on an examination of sub-score differences for the two areas.
Phase II. Design learning goals in all subject areas and redesign reporting system (i.e. report cards).
Reporting Topics for Language Arts• Word recognition and vocabulary• Reading comprehension• Literary analysis• Spelling• Language mechanics and conventions• Research and technology• Evaluation and revision• Listening comprehension• Analysis of oral media• Speaking applications
Many districts like to organize reporting topics into strands.
Language Arts Strands and Topics
READING– Word recognition and
vocabulary– Comprehension– Literary analysis
WRITING– Spelling– Language mechanics
and conventions
– Research and technology
– Evaluation and revisions
LISTENING & SPEAKING– Listening
comprehension– Speaking applications
3.00 – 4.00 = A2.50 – 2.99 = B2.00 – 2.49 = C1.50 – 1.99 = DBelow 1.50 = F
3.50 – 4.00 = Advanced2.50 – 3.49 = Proficient1.50 – 2.49 = BasicBelow 1.50 = Below Basic
3.00 - 4.00 = A = 95%2.50 - 2.99 = B = 85%2.00 - 2.49 = C = 75%1.50 - 1.99 = D = 65%Below 1.50 = F = 60%
This represents a shift from an approach in which grades are
interpretable only in the context of individual teachers’
classrooms to an approach in which grades can be interpreted
uniformly from teacher to teacher.
Phase III. Implement in a staged fashion.
At this point you’re pretty serious.
If you really want to get serious then….
If you really want to get serious then….replace the time-based
system with a performance-based system.
The time based system is over 100 years old.
The Requirements
of a Time-Based System
1. Must be in-class the entire year.
2. Must behave appropriately.
3. Must complete the work assigned them.
4. Must figure out what each individual. teacher expects of them.
5. Can be passed on without adequate knowledge if they are successful at 1–4.
Students:
Some Unintended Consequences of a Time-Based System
1. Students succeed if they can figure out and are willing to follow the rules.
2. Those students who do not figure out the rules or are unwilling to follow them get so far behind relatively quickly that they have a high probability of dropping out.
The Requirements of a Performance-Based System
1. Must demonstrate competence in important content knowledge.
2. Must behave appropriately if they wish to be involved in classes.
3. Must take some responsibility for their own learning.
4. Cannot be passed on until they demonstrate competence in important knowledge.
Students:
Some Intended Consequences of a Performance-Based System
1. Students do not have to figure out the rules from teacher to teacher.
2. To catch up or move ahead at an accelerated pace, students do not have to spend a specific amount of time in class. Rather, they must demonstrate competence in important content.
3. There are fewer dropouts and more students completing graduation.
Level 1
Level 2
Level 9
Level 10
NO GRADE LEVELS
K-2
3-5
LOWER DIVISION (9-10)
UPPER DIVISION (11-12)
GRADE LEVEL BANDS(Keep Course Structure)
6-8
Three Critical Interventions(COMMITMENTS)
• A system of individual student feedback on learning goals at the classroom, school, and district levels (standards-based grading using formative assessments)
• Ensuring effective teaching in every classroom
• Building background knowledge for all students
Phases of Ensuring Effective Teaching in Every Classroom
• Readiness: Teachers read books, attend professional development activities, and try strategies on their own.
• Phase I: The school/district develops a common “language of instruction” or model.
• Phase II: Teachers systematically interact about effective teaching using the model.
• Phase III: Teachers systematically observe master teachers and each other using the model.
• Phase IV: The school or district monitors the effectiveness of individual teacher’s instructional styles as a form of teacher feedback.
Phases of Ensuring Effective Teaching in Every Classroom
• Readiness: Teachers read books, attend professional development activities, and try strategies on their own.
Phases of Ensuring Effective Teaching in Every Classroom
• Readiness: Teachers read books, attend professional development activities, and try strategies on their own.
• Phase I: The school or district develops a common “language of instruction” or model.
Start with someone else’s model and adapt it to your own needs.
The Art and Science of Teaching as a beginning place.
Q1: What will I do to establish and communicate learninggoals, track student progress, and celebrate success?Q2: What will I do to help students effectively interact withnew knowledge?Q3: What will I do to help students practice and deepentheir understanding of new knowledge?Q4: What will I do to help students generate and testhypotheses about new knowledge?Q5: What will I do to engage students?
Q6: What will I do to establish or maintain classroom rules and procedures? Q7: What will I do to recognize and acknowledge adherence to and lack of adherence to classroom rules and procedures?Q8: What will I do to establish and maintain effectiverelationships with students?Q9: What will I do to communicate high expectations for allstudents?Q10: What will I do to develop effective lessons organizedinto a cohesive unit?
Observing a lesson looks very different from the perspective of
The Art and Science of Teaching.
Fundamental Segments of a Classroom Instruction
• Segments that are routine components of every lesson
• Content specific lesson segments
• Segments that must be enacted on the spot
What do you look for as routine components of every lesson?
• Rules and procedures (Q 6)
• Communicating learning goals (Q1)
• Tracking student progress (Q1)
• Celebrating success (Q1)
What do you look for as routine components of every lesson?
• Reviewing important rules and procedures• Reviewing learning goals• Reviewing student progress• Celebrating success
Fundamental Segments of Classroom Instruction
• Segments that are routine components of every lesson
•Content specific lesson segments
• Segments that must be enacted on the spot
• Critical input experiences (Q2)
• Knowledge practice and deepening activities (Q3)
• Hypothesis generation and testing tasks (Q4)
Identify the type of content segment that is the focus of the
lesson.• Does this segment involve critical input
experiences (Q2) ?
• Does this segment involve knowledge practice and deepening activities (Q3) ?
• Does this segment involve hypothesis generation and testing tasks (Q4) ?
If the segment involves a critical input experience what do you
expect to see?
If the segment involves a critical input experience what do you
expect to see?• Previewing activities
• Info presented in small chunks
• Students processing each chunk in small groups
• Students summarizing and taking notes after content has been introduced
• Students reflecting on their learning
If the segment involves knowledge practice and deepening activities
what do you expect to see?
If the segment involves knowledge practice and deepening activities
what do you expect to see?• Brief review of content• Activities involving similarities and differences• Activities involving identifies errors in thinking• Activities involving massed and distributed
practice• Homework possibly used as an extension of
these activities
If the segment involves hypothesis generating and testing tasks what
do you expect to see?
If the segment involves hypothesis generating and testing tasks what
do you expect to see?• Brief review of content• Students working individually or in groups on
long term tasks• Teacher acting as facilitator and resource
provider
Fundamental Segments of Classroom Instruction
• Segments that are routine components of every lesson
• Content specific lesson segments
•Segments that must be enacted on the spot
• Engagement activities (Q5)
• Consequences regarding rules and procedures (Q7)
• Relationships (Q8)
• Expectations (Q9)
What do you look for regarding segments that must be enacted on
the spot?
What do you look for regarding segments that must be enacted on
the spot?• Engagement activities when student lose focus• Acknowledge of rules and procedures being
followed or not being followed• Behaviors that forge positive relationships with
students• Attention to behaviors that communicate high
expectations for all students
1. Learning Goals and Feedback2. Interacting with New Knowledge3. Practicing and Deepening4. Generating and Testing Hypotheses5. Student Engagement6. Establishing Rules and Procedures7. Adherence to Rules and Procedures8. Teacher-Student Relationships9. High Expectations
SupervisingThe Art and Science of Teaching
Learning Goals and Learning Goals and FeedbackFeedback
Rules and ProceduresRules and Procedures
INVOLVES ROUTINES
ENACTED ON THE SPOTStudent EngagementStudent Engagement
High ExpectationsHigh Expectations
T
eac
her
/Stu
de
nt R
ela
tion
ship
sT
eac
her
/Stu
de
nt R
ela
tion
ship
s Ad
here
nce to
Ru
les a
nd P
roce
dures
Ad
here
nce to
Ru
les a
nd P
roce
dures
Generating/ Testing
Hypotheses
Practicing and
Deepening
Interacting with New
Knowledge
Supervising
The Art and Science of Teaching
ADDRESSES CONTENT IN SPECIFIC WAYS
Phases of Ensuring Effective Teaching in Every Classroom
• Readiness: Teachers read books, attend professional development activities, and try strategies on their own.
• Phase I: The school or district develops a common “language of instruction” or model.
• Phase II: Teachers systematically interact about effective teaching using the model.
Phases of Ensuring Effective Teaching in Every Classroom
• Readiness: Teachers read books, attend professional development activities, and try strategies on their own.
• Phase I: The school or district develops a common “language of instruction” or model.
• Phase II: Teachers systematically interact about effective teaching using the model.
• Phase III: Teachers systematically observe master teachers and each other using the model.
At this point you are pretty serious.
If you really want to get serious then….systematically monitor how much students learn in
teachers’ classrooms.
Phases of Ensuring Effective Teaching in Every Classroom
• Readiness: Teachers read books, attend professional development activities, and try strategies on their own.
• Phase I: The school/district develops a common “language of instruction” or model.
• Phase II: Teachers systematically interact about effective teaching using the model.
• Phase III: Teachers systematically observe master teachers and each other each using the model.
• Phase IV: The school/district monitors the effectiveness of individual teacher’s instructional styles as a form of teacher feedback.
Teacher ID
Q1 …… Q10 Ave pre/postgain
Ave studenteffort
Avestudent learning
39946 2.5 3.0 .76 2.11 3.11
bill01 2.0 2.0 1.33 3.23 3.06
S0S3 1.5 1.5 .43 1.98 2.11
SchoolAVE
2.45 2.36 .53 2.33 2.56
Three Critical Interventions (COMMITMENTS)
• A system of individual student feedback on learning goals at the classroom, school, and district levels (standards-based grading using formative assessments)
• Ensuring effective teaching in every classroom
• Building background knowledge for all students (particularly those who do not come from advantaged backgrounds).
Phase I: Establish a districtwide or schoolwide list of critical terms and accompanying model of vocabulary instruction.
angle
area
average
bar graph
congruent
difference
estimation
hexagon
length
width
octagon
parallel
perimeter
product
rhombus
square
symmetry
triangle
vertical
horizontal
3rd-Grade Mathematics
predict
compare
observe
investigate
analyze
amplify
frequency
vibration
Newton
properties
pitch
food web
life cycle
living organism
structure
inherit
reproduction
environment
characteristics
hypothesis
3rd-Grade Science
actor
animation
audience
cause & effect
commercial
conclusion
cue
detail
directions
drama
ending
facial expression
humor
minor character
main character
plot development
role playing
central idea
mood
myth
3rd-Grade Language Arts
prairie
Oregon Trail
lariat
oxen
expedition
territory
homestead
settlement
pioneer
farm/ranch
Native American
legacy
stampede
fertile
wagon trails
cabin
cattle
seasonal dwelling
plateau
frontier
3rd-Grade Social Studies
Step 1: Provide a description, explanation, or example of the new term.
Step 2: Ask students to restate the description, explanation, or example in their own words.
Step 3: Ask students to construct a picture, symbol, or graphic representing the term or phrase.
Step 4: Engage students periodically in activities that help them add to their knowledge of the terms in their notebooks.
Step 5: Periodically ask students to discuss the terms with one another.
Step 6: Involve students periodically in games that allow them to play with terms.
A Six-Step Process for Teaching New Terms
Mutualism
The interaction of organisms within an ecosystem in a manner that significantly benefits both, although the resulting relationship is not critical to the continued existence of either.
Term, phrase
Category: (standard, unit, alphabetical…)
Picture/graphicDescription
Related terms, phrases
Phase II: Implement the vocabulary program district-
wide or school-wide.
At this point you are pretty serious.
If you really want to get serious then….have students keep track
of their progress using vocabulary notebooks.
Three Critical Interventions
• A system of individual student feedback on learning goals at the classroom, school, and district levels
• Ensuring effective teaching in every classroom
• Building background knowledge for all students
Leadership for Incremental Change
• Emphasize relationships.• Establish strong lines of communication.• Be an advocate for the school.• Provide resources.• Maintain visibility.• Protect teachers from distractions.• Create culture of collaboration.• Look for and celebrate successes.
High above the hushed crowd, Rex tried to remain focused. Still, he couldn’t shake one nagging thought: He was an old dog and this was a new trick.
Leadership for Second-Order Change
• Shake up the status quo.• Expect some things to seem worse.• Propose new ideas.• Operate from strong beliefs.• Tolerate ambiguity and dissent.• Talk research and theory.• Create explicit goals for change.• Define success in terms of goals.
“Hey! They’re lighting their arrows! . . . Can they do that?”
“This ain’t gonna look good on our report, Leroy.”
Thank You!
Robert J. Marzano