Building Better Teachers in the Caribbean Barbara Bruns, Lead Education Economist World Bank Regional Learning Event Improving Teaching and Learning Outcomes in the English-speaking Caribbean Countries with ICT Bridgetown, BARBADOS 13-16 April 2011
Building Better Teachers
in the Caribbean
Barbara Bruns, Lead Education Economist
World Bank
Regional Learning EventImproving Teaching and Learning Outcomes in the English-speaking Caribbean Countries with ICT
Bridgetown, BARBADOS13-16 April 2011
Overview
1. Context: Raising teacher quality is the central education challenge
– Student learning is key to economic gains from education
– Teacher quality is key to student learning
– Teacher effectiveness is highly variable
2. Key challenges for teacher policy
3. Understanding what makes teachers effective in the classroom– New methods for looking “inside the black box”
– Evidence from Brazil: Six conclusions
– Reactions from education policymakers
4. Learning from innovations in the Caribbean and Latin America
1. New evidence on the payoffs to teacher quality
Student learning Is the prime determinant of individual returns and economy-wide gains from education (Hanushek and Woessman, 2007)
Figure 3: Test Scores and Growth in Selected Developing Countries
Source: Hanushek and Woessmann: Education Quality and Economic Growth (2007)
BRAZIL
Chile
China
Colombia
Cyprus
Egypt
Ghana
Hong Kong
IndonesiaIndia Iran
Jordan
Korea, Rep. of
Morocco
Malaysia
Peru Philippines
Portugal
Romania
Singapore
ThailandTunisia
Turkey
Taiwan
Zimbabwe
-4
-3
-2
-1
0
1
2
3
4
-1.5 -1.0 -0.5 0.0 0.5 1.0
Con
diti
onal
Gro
wth
Conditional Test Score
Teacher quality is the prime determinant
of student learning
Best teachers can raise student learning by 2-3 grade levels in a single yearWorst teachers raise learning 0.5 grade (Hanushek and Rivkin, 2005)
Cumulative effects of these variations are huge, esp. given evidence that lower quality teachers are clustered in low-income schools
Highly variable teacher effectiveness found across all countries and contexts to date
Teacher effectiveness varies tremendously (measured as ability to generate year on year learning gains)
…and raising learning is a challenge for the Caribbean and LA
PISA Math 2009
Lower middleincome
Upper middle income High income:NonOECD High income: OECD
300
350
400
450
500
550
Indon
esia
Tunis
iaJo
rdan
Thail
and
Avera
ge: L
ower
midd
le inc
ome
Pana
ma Peru
Alban
iaCo
lombia
Braz
ilAr
genti
naMo
ntene
groKa
zakh
stan
(*) M
exico
(*) C
hile
Roma
niaUr
ugua
yBu
lgaria
Azerb
aijan
Serbi
a(*)
Turke
yRu
ssian
Fede
ration
Lithu
ania
Avera
ge: U
pper
midd
le inc
ome
Qatar
Trinid
ad an
d Tob
ago
Israe
lDu
bai (U
AE)
Croa
tiaLa
tvia
Eston
iaMa
cao-C
hina
Liech
tenste
inHo
ng Ko
ng-C
hina
Singa
pore
Avera
ge: H
igh in
come
: non
OECD
Gree
ce Italy
Spain
Portu
gal
Irelan
dUn
ited S
tates
Luxe
mbou
rgHu
ngary
Unite
d King
dom
Czec
h Rep
ublic
Swed
enPo
land
Austr
iaSlo
vak R
epub
licFra
nce
Norw
aySlo
venia
Denm
arkIce
land
Germ
any
Austr
alia
Belgi
umNe
w Ze
aland
Nethe
rland
sCa
nada
Japa
nSw
itzerl
and
Finlan
dKo
reaAv
erage
: High
inco
me: O
ECD
2. Elements of teacher qualityRecruiting, Grooming and Motivating Talent
Recruiting Failure to recruit talent Recruiting talent
Teachers recruited from bottom third of students, with low cognitive skills
Higher standards for content mastery. Brazil, Chile, Colombia, Jamaica, Guyana, Guatemala
Shortages of math and science teachers: 85% of OECS science teachers lack science background
Special incentives for scarce skills
Low entry requirements to teacher training programs: private schools w/o accreditation
Competitive entry into teacher training:only top 10-30% of students hired as teachers in Finland, Singapore and Korea
Few graduates from top universities pursue teaching as a career
Alternative certification and “public service” appeal (Teach For All model)Chile, Peru, Argentina, Brazil, Colombia, Mexico
GroomingFailure to groom Grooming Talent
Little supervision and support for new teachers despite documented low effectiveness in first year
Induction programs (like medical residency): Effective support, encouragement and performance evaluation in first years
No “de-selection” of low-performing teachers once hired…but also high attrition in some countries
Reforms that allow de-selection. Chile – 2011 reform: school directors can fire 5% of worst performing teachers each yearSingapore – bottom 10% of directors fired each year
Theoretical and ineffective in-service training
In-service training based on observed classroom practice
Teachers not convinced that “every child can learn”
Screen and train teachers for values: perseverance and belief in kids
MotivatingFailure to motivate Motivating high performance
Flat salary scalesAverage salary after 15 years only 30% above salary at entry
More attractive career laddersChile AVDI, Sao Paulo Prova de Promocão,Mexico CM, Peru CPM offer large pay increases with promotions based on competence
Salaries and promotions de-linked from performance
Pay for performance (school bonus pay)
No strategic staffing rewards Incentives to get best teachers to most challenging schools
Parents typically lack ability to pressure teachers to perform
Parents empowered to demand quality services and informed about school performance (Brazil example)
Alibis for failure: child poverty, lack of parent support, lack of school resources
“If the child is not learning, it’s our fault”Sao Paulo state Secretary of Education
Evidence on Pay for Performance
Motivating
Rigorous global evidence limited and conflicting
Positive impact Rural India, Rural Kenya, Israeli high schools and Chile’s SNED (but effect sizes vary a lot)
No impact US (Nashville), NYC (first year)
Emerging evidence from Brazil
Design features matterSize of bonusCriteria for bonus Coverage or predictability Possible perverse impacts over time (teaching to test, cheating)
What changes in the classroom when incentives are introduced?
Ongoing evaluations in 3 different systems in Brazil, possible scope to launch more
3. Understanding what makes teachers effective in the classroom
having materials in the classroom is not
enough
having the teacher in the classroom is not
enough
education is the transformation of inputsinto learning outcomes
ACTIVITIES
construction curriculum
design trainingtesting
supervision
INPUTS
Funding salaries
buildings books
materials
OUTPUTS
Teacher practice in
the classroom
Inputs
Activities
Results
OUTCOMES
Student learning
Completion rates
…and it happens in the classroom
Tool for analyzing“the black box” of the Classroom
• “Stallings method” classroom observations
• Evidence on teacher practice and classroom dynamics from LAC
• First large-scale, systematic classroom observations in LAC
Ongoing in: Brazil (Pernambuco, MG, Rio municipality)
Starting in: Colombia, Mexico, Jamaica, Guatemala
Planned in: Guyana, El Salvador, Chile
What does it measure?
• Teacher’s use of instructional time• Use of materials, including ICT• Core pedagogical practices• Ability to keep students engaged
Stallings “classroom snapshot” instrument:
How does it work?• 10 observations of each class taken at regular
intervals and coded using a standardized grid
Stallings “classroom snapshot” instrument:
What are some key advantages?
• High reliability - across observers/countries• Benchmarking - Results can be compared to
US/OECD good practice indicators• Simple - Relatively short training needed to produce
qualified observers• Analytical power - Stallings variables are:
• statistically representative of the education system•highly correlated with learning results
Stallings “classroom snapshot” instrument:
OBSERVATION CODING GRID
MATERIAL
ACTIVITY
NOMATERIAL
TEXTBOOK NOTEBOOKBLACKBOARD
LEARNING AIDES ICT COOPERATIVE
1. READING ALOUDT 1 S L
E1 S L E
1 S L E
1 S L E 1 S L E 1 S L
E S L E
I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L
CHECK IF CHORAL READING 2.DEMONSTRATION/ LECTURE
T 1 S L E
1 S L E
1 S L E
1 S L E 1 S L E 1 S L
E S L E
I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L
3. DISCUSSION/QUESTIONS AND ANSWERS
T 1 S L E
1 S L E
1 S L E
1 S L E 1 S L E 1 S L
E S L E
I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L4. PRACTICE& DRILL
T 1 S L E
1 S L E
1 S L E
1 S L E 1 S L E 1 S L
E S L E
I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L5. ASSIGNMENT/ CLASS WORK
T 1 S L E
1 S L E
1 S L E
1 S L E 1 S L E 1 S L
E S L E
I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L
6. COPYINGT 1 S L
E1 S L E
1 S L E
1 S L E 1 S L E 1 S L
E S L E
I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L
7. VERBALINSTRUCTION
T 1 S L E
1 S L E
1 S L E 1 S L E 1 S L
E1 S L E S L E
I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L8. SOCIALINTERACTION
T 1 S L E
I 1 S L9. STUDENT(S)UNINVOLVED
I 1 S L
10. DISCIPLINET 1 S L
E11. CLASSROOMMANAGEMENT
T 1 S L E
I 1 S L
12. CLASSROOM MANAGEMENT ALONE T
13. TEACHER SOCIAL INTERACTION OR TEACHER UNINVOLVED T
14. TEACHER OUT OF THE ROOM T
School No.:Classroom No. Exact time of observation:
Coding grid
FIGURE 1: EXCERPT FROM CLASSROOM SNAPSHOT (APPENDIX 2)
CLASSROOM OBSERVATION SNAPSHOT
MATERIAL
ACTIVITY
NO MATERIAL TEXTBOOK NOTEBOOK BLACK
BOARD LEARNING AIDES ICT COOPERATIVE
1. READING ALOUD T 1 S L E 1 S L E 1 S L E
1 S L E 1 S L E 1 S L E S L E
I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L
CHECK IF CHORAL READING
T line: Indicates activities that involve the teacher
I line: Indicates activities that involve the student & not the teacher
1, S, L, E: Indicate one individual, a small, large group and entire class respectively
School and Classroom
Demographic information
OECD Good Practice Benchmarks
Results: Evidence from Brazilian classroomsSix conclusions
#1 Brazil is far from OECD benchmarks
Teachers’ Use of Instructional Time
Rio deJaneiro
City (100 schools)
PernambucoState (300 schools)
Minas Gerais State
OECD Good Practice
Benchmarks
Schools that improved 2005-2007
Schools that declined
2005-2007
Teaching activities 62% 61% 66% 63% 85%
Classroom organization 31% 27% 27% 25% 15%
Non-academicactivities
7% 12% 8% 12% 0%
Teacher out of the classroom
3% 8% 3% 6% -
#2 Improving and Declining Schools Look Different
Teachers’ Use of Time All schools (average) Improving/ Declining Schools
Teacher out of the classroom
4% 3%
6%
Teacher not engagedwith students
3% 2%
3%
Teacher using Q and A 12% 15%
10%
Small or large group of students not engaged
28% 19%
37%
#3 More time on learning is highly correlated with learning outcomes
5th grade
Teacher Use of Time
1. Learning Activities 2. Classroom management 3. Teacher off-task 3.1 Teacherout of room
Total Sample – Rio 58% 37% 6% 1%
10% highest IDEB schools 70% 27% 3% 0%
10% lowest IDEB schools 54% 39% 7% 3%
Difference 0.16 -0.13 -0.03 -0.03
[0,09]* [0,09]* [0,02] [0,01]**
- Erros robustos em parênteses: *Estatisticamente significativo a 10%, **Estatisticamente significativo a 5%, ***Estatisticamente significativo a 1%
- Dados no nível do ano/escola
#4 More time on learning predicted schools’ learning gains and attainment of bonus (PE)
Bônus 2009
Teacher use of Time
1. Learning activities2. Classroommanagement
3. Teacher off-task 3.1 Teacher out of the classroom
Pernambuco State sample 61% 28% 11% 8%
Achieved bonus 2009 62% 27% 10% 8%
Didn’t achieve bonus 2009 53% 30% 17% 12%
Difference 0.09 -0.03 -0.06 -0.04
[0,04]** [0,03] [0,02]*** [0,02]***
Sample of 300 State schools, with 1,800 classrooms observed – math and portuguese classes in 5th, 9th and 12th grades
Standard errors in parenthesis. * Statistically significant at the 10% level, **Statistically significant at the 5% level, ***Statistically significant at the 1% level
# 5 Tool for identifying schools with highly positive (and negative) performance
Extreme situations
% of classrooms observed
Rio de Janeiro
PE
MG pilot
Improvingschools
Decliningschools
Teacher absent from the classroom more than 50% of total class time 0% 2% 0% 0%
Teacher absent from classroom more than 25% of total class time 1% 10% 3% 3%
Teacher spent over 50% of class time using no materials 4% 10% 17% 31%
Teacher spent more than 50% of class time writing on black board 24% 32% 5% 9%
Large group of students not engaged during more than 50% of class time 9% 7% 9% 19%
#6 Teacher practice varies a lot…with huge implications for learning
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0 0.5 1 1.5 2 2.5 3 3.5
70 % total schools
TIME USE IN THE CLASSROOMRio de Janeiro schools, June 2010
Learning activities
Classroom management
Teacher off task
On average, only 61% of class time is used for instruction
But some schools average 81%
Others average 37%
88 more days of instruction per school year
TIME USE IN THE CLASSROOMRio de Janeiro schools, June 2010
Learning activities
Classroom management
Teacher off task
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0 0.5 1 1.5 2 2.5 3 3.5
Across different classrooms in the same school
•70% of time on learning in best class• 52% in worst class
36 more days of instruction per year in the most efficient classrooms
#6 Teacher practice varies a lot…even within schools
Reactions in Brazil From Education secretaries, district supervisors and school pedagogical coordinators…
It allowed me to observe better the dynamics of class, because in my work environment, observations do happen, but not too closely or with such details
This type of classroom observation is directly relevant to my daily work...(PE)
We know this intuitively, but this is the first time we have hard data we can take back to our regional offices and schools to discuss and work on. Secretary of Education (MG)
I was surpised to see...very traditional
teacher practice
Even in different schools and municipalities the teacher practices are the same
Teachers work in a very traditional way: exercises and corrections
Despite the intense technological progress, we still give priority to old ways of teaching-learning
The textbooks that come via MEC are rarely used in the classroom; the teachers prefer to write on the blackboard
the opportunities for Caribbean countries and Latin America
Conclusions• Virtually all Caribbean and LA countries trying to
raise teacher quality• Innovative approaches, universal issues • Little hard evidence on impact or cost-
effectiveness – in OECS, OECD or LAC • Great opportunity to generate knowledge that
will benefit OECS and others• Combining test data with classroom observations
creates a platform for solid evidence on how to “Build Better Teachers”
Thank you! Barbara Bruns