1 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Richard Thompson Associates 16 Conway Road London N14 7BA
BRITISH COUNCIL NEPAL ENGLISH FOR TEACHING: TEACHING FOR ENGLISH PLUS
ET:TE+ TRAINING PROJECT IN CHITWAN AND LAMJUNG DISTRICTS
MONITORING & EVALUATION REPORT
January 2014
Contents
Executive summary
1. Introduction: the English for Teaching:Teaching for English Plus (ET:TE+) Project 2013
2. The Monitoring & Evaluation programme
3. Findings: Classroom observation of teachers
4. Findings: APTIS testing of teachers’ levels of competence in English
5. Findings: Interviewing teachers of English
6. Findings: Other views expressed in schools
7. Conclusions
Appendix: Sample for Monitoring & Evaluation Programme
Executive Summary
The principal purpose of the ET:TE+ training project was to change teacher behaviour in the
classroom: more personal competence and confidence for the teacher in their English and
changes to their pedagogy to facilitate interactive language learning.
Large-sample, standardised monitoring was put in place to evaluate the project at baseline,
mid project and end of project stages. Extensive classroom observation was the key element
in recording the ET:TE+ project’s impact and success: the key indicator was whether in the
classroom teacher behaviour had changed: transfer of training to the classroom in terms of
the three main areas of classroom observation: use of English, interaction and effectiveness
of teaching. The evidence is that there had been very significant change.
There have been dramatic increases in the use of English by teachers in the classroom: a
strong shift from the baseline position through to project end, as this graphic shows:
2 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Baseline monitoring showed 83% of lessons with no more than 10% use of English and just
1% in the top two levels. By end of programme, the 83% has shrunk to 6% and the 1% has
grown to 81%.
There was a growth in student use of English in talking to the teacher and student to student
talk in English, which would most naturally occur in group or paired work situations. Such
interactive situations were almost non-existent at the start of the project; by its end close to
80% of lessons included group work. Students had gained the opportunity to use and practice
English themselves in the classroom, and when asked in focus groups confirmed that these
were daily opportunities.
There had also been very noticeable changes in measures of teacher effectiveness to produce
a more positive classroom atmosphere and learning experience: changes to pedagogy, much
more use of supplementary classroom materials, a focus to lessons with a much greater
variety of student activities.
These classroom findings constitute a powerful vindication of the ET:TE+ training project. The
trajectory strongly suggests that these will be permanent changes, a view reinforced by
qualitative data from interviews with teachers, parents and school managers.
Disaggregating the data by district shows some differences, which may come from the way
the training was delivered as well as from other district-specific factors: there was more
extensive use of classroom English by the teachers in Chitwan where lessons scored higher
for the use of pair or group work. By contrast, Lamjung teachers scored high on giving
encouragement and feedback to students.
Disaggregating by gender, from similar starting scores female teachers made greater
improvement in a range of classroom activities and pedagogical methods, including student
use of English. There was little gender difference in terms of teacher use of English.
All teachers were tested at the beginning and end of the ET:TE+ training with the
internationally-developed and standardized British Council APTIS test, an element which was
widely welcomed. Analysis showed very many teachers with low levels of English with only a
handful reaching Independent User standard. The re-test showed some gains but most
notably a great deal of churn in individual results.
Teachers in interview described their growing confidence with English and the new teaching
methods, with an urgent need to increase classroom resources in order to move to more
interactive teaching. There was great enthusiasm for the ET:TE+ training from teachers, SMC
members and district office staff.
28
55
16
1 0 4 2 13
29
52
0% 1-10% 11-25% 26-50% 51%+
Teacher Used English during the Lesson
Baseline END P
3 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
1. Introduction: the English for Teaching: Teaching for English Plus (ET:TE+)
Project 2013
English for Teaching: Teaching for English Plus (ET:TE+) 2013 was a British Council teacher
training project to support the development of communicative English language teaching in
government Primary schools in Nepal in the two districts of Chitwan and Lamjung, in
association with Nepal English Language Teachers Association (NELTA) in Chitwan and Global
Action Nepal (GAN) in Lamjung as implementing partners.
NELTA and GAN were encouraged to develop their own chosen routes to deliver the common
ET:TE+ training project. Both chose a methodology based on their past experience described
as a “holistic approach”, meaning the selection of the English teacher plus a number of
teachers from each school to better create a critical mass which could change practice across
a school more easily. These teachers may teach a range of subjects from the National
Curriculum. NELTA chose to recruit teachers from across the district and all Resource Centre
[RC] areas while GAN in Lamjung chose to run in a group of three RC areas and clusters of
schools.
Potential teacher recruits to the ET:TE+ training project were nominated by the implementing
partners and underwent an initial screening and selection process in March 2013 using the
British Council’s APTIS test. The test was conducted at central locations in each district,
supervised and administered by British Council Nepal and managed by this team. The APTIS
test has been designed and standardized to be consistent with the Common European
Framework of Reference for Languages [CEFR]. CEFR describes foreign language proficiency
at six levels: A1 and A2 (Basic User), B1 and B2 (Independent User), C1 and C2 (Proficient
User). The international ET:TE course was designed for a minimum of A2 competence levels;
for this project the course was redesigned locally to also be accessible for A1 language users.
The 300 teachers who would participate, 150 in each district, were selected by the end of
March 2013 and the project began. British Council Nepal authored a Training Manual and led
training for the trainers. In each district the teachers then received initial 6 day training in
April 2013 before returning to their schools for the start of the new school year.
143 teachers in Lamjung and 149 teachers in Chitwan actually attended the initial training
and began the ET:TE+ project. In Chitwan this initial training was on a residential basis. GAN
and NELTA then planned follow up sessions. For example in Chitwan NELTA organized two
sessions at 15 day intervals, then trainers visited as many trainees as they could in their
classrooms to provide feedback on classroom practice. Other follow up activities included
teacher’s clubs. GAN made use of local Resource Centres for their sessions. A second 6 day
training course in August 2013 used a second Training Manual and training in its use provided
by British Council Nepal. The second week was non-residential in both districts. Follow up
activities included school and inter-school competitions.
The APTIS testing process was repeated at the end of the project in November/December
2013.
4 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
2. The Monitoring & Evaluation programme
Preparation
Initial preparation and planning by Technical Advisor and Team Leader was shared as
appropriate with British Council Nepal and the ET:TE+ implementing partners. Qualified and
experienced Research Associates (RAs) took part in a five day intensive training session in
Kathmandu in January 2013. This included training by British Council Nepal to standardize
classroom observation recording. The data collection tools were piloted in two schools in
Kavre district and subsequently revised. Training sessions were repeated at the Mid Project
and End of Project stages.
Research Associates were allocated to two district-based teams to conduct school visits. Steps
were taken to ensure consistent, coherent and reliable data were collected from the field by
and across the two district-based teams.
Baseline assessment
A report on the baseline survey carried out by this team was completed for British Council
Nepal in March 2013. This Report draws on the baseline data for comparative purposes.
Monitoring & Evaluation - method
A representative sample of teachers and schools was compiled in each district, using data
collected by:
the baseline survey: teachers and schools from whom a full data set of teacher interview,
classroom observations and focus group outcomes had been collected;
British Council Nepal: APTIS test scores; and
the implementing partners: teachers actually attending training in April 2013 [a small
number were absent, and therefore deleted from the sample]
Details of the sampling are given as an appendix. Analysis of the sample against the full
ET:TE+ cohort demonstrates that the chosen sample was representative.
In Chitwan District the sample consisted of 30 schools each with an identified teacher of
English and 56 others. In Lamjung District the sample covered 24 schools with 24 identified
teachers of English [one per school] to be interviewed, with a higher number – 93 – of other
ET:TE+ teachers available for classroom observation.
Mid Programme and End of Programme monitoring
Central to the Monitoring and Evaluation Programme was unannounced school visits. The
Programme was structured around two series of monitoring visits. The first round of mid
programme monitoring visits was conducted in each district in June/July 2013. The second
round of end of programme monitoring visits to the same sample of schools and teachers
took place in November/December 2013.
At the core of each school monitoring visit was extensive classroom observation. In each
school the identified teacher of English was subject to extended classroom observation [4 or 5
lessons]. All other ET:TE+ teachers in the sample schools were observed in at least one
lesson. The classroom observations were recorded on a proforma devised by British Council
Nepal which concerned 19 measures related to three aspects of the ET:TE+ training project:
Use of English
5 measures of use of English in the classroom, by the teacher including giving instructions
and feedback, and by students speaking to the teacher and to each other in English. These
were each assessed at one of five levels from ‘0%’, completely absent, to ‘51%+’, use of
English on more than half the available opportunities.
5 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Interaction
6 measures of the use of pair or group work, teacher encouragement of and feedback to
students, student involvement in the lesson and the balance between teacher-centred and
student-centred activities. These were each assessed at one of five levels again, this time on
a judgement scale from ’Not met’ to ‘Outstanding’
Effectiveness of teaching
There were 8 measures in the final aspect centred on teachers’ pedagogical skills, techniques
and behavior such as use of materials to supplement the lesson, correction of students and
checking students understood, again recorded on a judgment scale from ’Not met’ to
‘Outstanding’
The proforma included space for the fieldworker to record any additional comments or notes.
The monitoring team underwent standardization training from the British Council. The Team
Leader and Technical Advisor also set up a standardization process of random visits with
Research Associates for shared classroom observation to ensure reliable data recording. This
process was carried out rigorously and led to one indicator being dropped as there was still
some variety of practice within the team that was generating unreliable results. This indicator
was ‘there was an effective balance between teacher-centred and student-centred activities’.
During fieldwork, views were also sought at each school through:
Individual self-evaluation interviews with each identified teacher of English
A focus group of pupils, discussing the English lesson just observed and their
experience of English
A focus group of adult stakeholders, normally including English teacher(s),
headteacher, SMC chair, parents and other local stakeholders
Research Associates also provided short written reports on their work.
Competence in English: APTIS testing
This team managed the delivery of the APTIS testing, in March/April and in
November/December 2013. Findings are discussed below.
Data handling
The data collected by the field researchers was computerized using Microsoft Excel. Before
starting data entry, inconsistencies and errors were corrected and data were checked and
ensured in consultation with the Research Associates. The data were entered by two
experienced data entry personnel. The entered data were cleaned using the Statistical
Package for Social Sciences (SPSS 18). The data entered in MS Excel was converted into
SPSS data files. There were four separate data files in SPSS namely Classroom Observation,
Student Focus Group Discussion, Teacher Interview, and Focus Group Discussion.
Ethics
This study used the ethical guidelines of the British Educational Research Association (BERA).
Unambiguous and clearly written information in the Nepali language was provided to ensure
informed consent from all participants, with the right at any time to withdraw that consent.
All participants were provided with an explanatory sheet and their verbal consent was
recorded in field notes. Participants were assured that data would be anonymised and
confidentiality ensured. Any paper-data was kept safely and after entering data on computers
it was kept securely to prevent unauthorised access. No incentive was offered to participants
and no offers were made to meet any of their expenses.
6 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
3. Findings: Classroom observation of teachers
The findings from classroom observation are the key element in considering the impact and
success of the ET:TE+ training programme. The principal purpose of the training was to
change teacher behaviour in the classroom: more personal competence and confidence for
the teacher in their English and changes to their pedagogy to facilitate interactive language
learning.
Baseline
All the teachers who were initially considered for ET:TE+ training and who took the APTIS test
of English competence were visited at their schools and observed teaching in the classroom.
The total number of Classroom Observations recorded as part of the baseline assessment was
916. Tables in this report include the data from all these Classroom Observations whether or
not teachers went on to take part in the ET:TE+ training.
Mid Project and End of Project monitoring
There were two rounds of monitoring. All the teachers in the sample were visited at their
schools in a first round of monitoring [MID P] in June/July 2013 and observed teaching in the
classroom, a total of 570 complete lesson observations. 314 observations were completed in
Chitwan and 256 in Lamjung. Each teacher of English was observed on several occasions; the
total number of observations in this subset was 348 which was 61% of the total classroom
observations. Other teachers were observed [222 lessons] teaching other subjects through
the medium of English with some English lessons. 94% of observations were Grade 1-5
lessons.
Sample schools and teachers were visited again in the second and final round of monitoring
[END P] in November/December 2013 and observed teaching in the classroom, a total of 552
complete lesson observations. 299 observations were completed in Chitwan and 253 in
Lamjung with a similar balance between teachers. Tables in this section compare Baseline,
Mid Project [MID P] and End of Project [END P] data across the 19 measures and five levels
agreed and standardized in advance.
Classroom observations: all teachers
Table 3.1 Classroom Observations: Use of English: all teachers
0% 1-10% 11-25% 26-50% 51%+
Use of English: all teachers, as %
The teacher used English during the lesson
Baseline 28 55 16 1 0
MID P 9 30 55 6 0
END P 4 2 13 29 52
The teacher gave instructions in English
Baseline 58 40 3 0 0
MID P 12 58 27 2 0
END P 4 4 14 32 45
The teacher gave feedback in English
Baseline 68 31 1 0 0
MID P 18 65 16 1 0
END P 5 5 21 34 35
The students spoke to the teacher in English
Baseline 84 16 0 0 0
MID P 53 45 2 0 0
END P 12 36 39 10 3
The students spoke to each other
in English
Baseline 99 1 0 0 0
MID P 91 9 0 0 0
END P 54 31 9 4 2
7 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
This first table shows in the first three measures dramatic increases in the use of English by
teachers in the classroom. At baseline 83% of lessons were at the lowest two levels; at end of
project 81% where in the top two levels. There is a strong shift by mid project and this
change continues through to project end, by which point just a small minority of lessons show
little use of English.
The greatest change is with teacher use of English. However there is a growth too in student
use of English in talking to the teacher and in student to student talk in English from a very
low base. By the end of the project there has been a positive and significant change.
Table 3.2 Classroom Observations: Interaction: all teachers
Not
met Partially met
Met More than met
Outstanding
Interaction: all teachers as %
The teacher used pair work effectively
Baseline 100 0 0 0 0
MID P 77 23 1 0 0
END P 40 16 22 19 3
The teacher used group work effectively
Baseline 99 1 0 0 0
MID P 63 30 7 0 0
END P 22 17 33 25 3
The teacher gave encouragement to pupils
Baseline 32 66 2 0 0
MID P 1 63 36 0 0
END P 0 5 54 34 7
The students were involved in the lesson
Baseline 43 56 1 0 0
MID P 0 65 35 0 0
END P 0 6 60 27 7
The teacher gave effective feedback as
necessary
Baseline 49 45 5 0 0
MID P 1 74 25 0 0
END P 0 8 54 25 13
There has been an increase in the use of pair and group work, breaking the mold of 100%
and 99% of baseline lessons with no sign of these. At end of project almost 80% of lessons
included some group work and 60% some element of pairs work. We should note here that
the primary-age students who formed our pupil focus groups confirmed that the English
lesson just observed by our researcher was typical of the lessons they now received.
There have been very noticeable changes to produce a more positive classroom atmosphere
and learning experience, with very large gains in the other three measures to reach 90%
‘met’ or better in these measures of classroom interaction.
Table 3.3 Classroom Observations: Effectiveness of teaching: all teachers
Not
met Partially met
Met More than met
Outstanding
Effectiveness of teaching: all
teachers as %
The teacher used a variety of activities
Baseline 58 42 0 0 0
MID P 4 79 17 0 0
END P 2 24 39 28 6
The teacher gave students time to respond
Baseline 28 69 3 0 0
MID P 1 64 35 1 0
END P 0 7 51 33 9
8 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
The teacher used gestures to support
meaning
Baseline 42 57 1 0 0
MID P 16 64 19 0 0
END P 1 12 43 42 2
The teacher used materials to supplement the lesson
Baseline 66 33 1 0 0
MID P 25 48 26 0 0
END P 1 11 44 41 3
The teacher corrected students
Baseline 38 56 6 0 0
MID P 1 72 26 1 0
END P 0 8 52 30 10
The teacher checked students understood
Baseline 52 42 6 0 0
MID P 1 73 25 1 0
END P 0 5 52 33 10
The lesson effectively developed a skill Baseline 70 28 1 0 0
MID P 7 85 8 0 0
END P 0 13 62 23 2
The lesson effectively covered a language area
Baseline 81 18 0 0 0
MID P 8 87 5 0 0
END P 1 11 64 22 3
These are measures of pedagogy. Practice has improved across the board: it is striking how
improvement continues from mid project to end of project monitoring, for example from
‘Partially met’ to ‘met’ or better. No. of observations: Baseline n = 916 Mid project n = 570 End of project n = 552
Classroom observations: comparing teachers of English with other teachers
This subsection compares designated teachers of English [all English teachers at baseline; at
MID P and END P, the 54 teachers one in each of the sample schools] with all other teachers
on selected measures that show noticeable differences between the two groups. Other
measures show less difference: the full series of tables are available in the Technical Report.
Table 3.4 Classroom Observations: Use of English: teachers of English
0% 1-10% 11-25% 26-50% 51%+
Use of English: teachers of English, as %
The teacher used English during the lesson
Baseline 13 64 23 0 0
MID P 0 21 70 9 0
END P 0 0 11 33 56
The teacher gave instructions in English
Baseline 47 50 3 0 0
MID P 1 59 36 3 0
END P 0 1 13 40 46
The teacher gave feedback in English
Baseline 56 42 1 0 0
MID P 7 72 20 1 0
END P 0 2 23 40 34
9 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Table 3.5 Classroom Observations: Use of English: other teachers
0% 1-10% 11-25% 26-50% 51%+
Use of English: other teachers as %
The teacher used English during the lesson
Baseline 42 47 10 0 0
MID P 22 44 32 1 0
END P 8 4 14 25 48
The teacher gave instructions in English
Baseline 68 30 2 0 0
MID P 30 56 13 0 0
END P 9 6 15 25 45
The teacher gave feedback in English
Baseline 79 20 1 0 0
MID P 34 54 11 1 0
END P 9 8 18 27 37
Within a general picture of very much greater use of English the teachers of English are using
the language more in the classroom than their colleagues in the same schools, having begun
from a stronger baseline position. For example on the first indicator here ‘The teacher used
English during the lesson’, at the end of the programme teachers of English recorded 89% in
the top two levels, with none in the bottom two levels, 0 – 10%. ‘Other teachers’ scored
lower at 73% in the top two levels with 12% still using little or no English.
Table 3.6 Classroom Observations: Interaction: teachers of English
Not
met Partially met
Met More than met
Outstanding
Interaction: teachers of English as %
The teacher used pair work effectively
Baseline 99 1 0 0 0
MID P 73 26 1 0 0
END P 34 16 26 23 1
The teacher used group work effectively
Baseline 99 1 0 0 0
MID P 55 36 9 0 0
END P 18 14 40 27 1
The teacher gave effective feedback as necessary
Baseline 48 47 4 0 0
MID P 1 80 19 0 0
END P 0 9 58 23 9
Table 3.7 Classroom Observations: Interaction: other teachers
Not
met Partially met
Met More than met
Outstanding
Interaction: other teachers as %
The teacher used pair work effectively
Baseline 100 0 0 0 0
MID P 82 17 0 0 0
END P 46 15 18 16 5
The teacher used group work effectively
Baseline 99 1 0 0 0
MID P 76 19 5 0 0
END P 27 20 25 24 4
The teacher gave effective feedback as necessary
Baseline 51 43 5 0 0
MID P 0 66 33 0 0
END P 0 6 49 27 18
10 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Teachers of English are using more pair and group work but teachers in other lessons were
given higher gradings more frequently on other measures of improved classroom interaction
such as giving effective feedback. All have benefited from the project training.
Table 3.8 Classroom Observations: Effectiveness of teaching: teachers of English
Not
met Partially met
Met More than met
Outstanding
Effectiveness of teaching: teachers of English as %
The teacher gave students time to respond
Baseline 23 73 4 0 0
MID P 1 59 39 1 0
END P 0 8 57 29 6
The teacher used materials to supplement the lesson
Baseline 61 38 1 0 0
MID P 22 44 34 0 0
END P 0 14 49 34 2
The teacher corrected students
Baseline 36 57 6 0 0
MID P 1 75 24 0 0
END P 0 8 58 27 6
The teacher checked students understood
Baseline 55 40 5 0 0
MID P 1 76 22 0 0
END P 0 5 60 28 7
Table 3.9 Classroom Observations: Effectiveness of teaching: other teachers
Not
met Partially met
Met More than met
Outstanding
Effectiveness of teaching: other teachers as %
The teacher gave students time to respond
Baseline 33 65 2 0 0
MID P 1 71 28 1 0
END P 0 5 44 38 12
The teacher used materials to supplement the lesson
Baseline 70 29 1 0 0
MID P 31 55 14 0 0
END P 1 8 38 47 5
The teacher corrected students
Baseline 40 55 6 0 0
MID P 3 67 29 0 0
END P 0 8 46 32 13
The teacher checked students understood
Baseline 50 45 6 0 0
MID P 1 68 30 0 0
END P 0 6 43 38 13
By the end of the project the English teachers do less well than the other teachers on each of
these selected aspects of pedagogy at the higher levels of more than met/outstanding. The
‘gap’ is always more than 10%, and some outstanding practice is recorded. Perhaps teachers
were able to refer to subject content in for example Science or Social Studies to add to
materials and to generate student questioning to check understanding, and less confident to
do so in an English lesson. They are applying learning from the project in different ways.
Teachers of English: no. of observations Baseline n = 447 MID P n = 348 END P n = 279
Other teachers: no. of observations Baseline n = 469 MID P n = 222 END P n = 273
11 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Classroom observations: comparing all teachers by District
Disaggregating the data by district enables us to see if there were differences in the degree
or nature of changes in the classroom between the two districts, and speculate about the
possible reasons. These might be attributable to differences in the way the common training
package was delivered by the implementing partners: there were different patterns of follow
up sessions with GAN in Lamjung in a smaller geographical area making much use of the
Resource Centres, while the greater spread across Chitwan District brought people together
from a wider spread of schools. The smaller number of schools in the project in Lamjung
meant larger numbers of trainees from some schools – two schools each supplied nine
teachers - than was the case in Chitwan. Finally, the trainees in Chitwan included 48 who
recorded a low APTIS level of A0 in the initial testing, with a smaller number, 38, in Lamjung
scoring A0.
Table 3.10 Classroom Observations: Use of English: CHITWAN DISTRICT
0% 1-10% 11-25% 26-50% 51%+
Use of English: all teachers, as
percentage: Chitwan
The teacher used English during the lesson
Baseline 27 55 17 0 0
MID P 5 30 59 6 0
END P 2 0 8 34 56
The teacher gave instructions in English
Baseline 63 35 2 0 0
MID P 8 71 20 1 0
END P 2 0 13 42 42
Table 3.11 Classroom Observations: Use of English: LAMJUNG DISTRICT
0% 1-10% 11-25% 26-50% 51%+
Use of English: all teachers, as percentage: Lamjung
The teacher used English during the lesson
Baseline 28 55 15 1 0
MID P 13 30 51 5 0
END P 6 5 19 23 47
The teacher gave instructions in English
Baseline 52 45 3 0 0
MID P 18 43 36 4 0
END P 7 8 16 21 49
Much more extensive use of classroom English by the teacher is recorded in Chitwan with
figures across the top two levels of 90% and 84% while in Lamjung figures are 70% for each
measure. Certainly in Chitwan at the time of the project there was very high interest in
schools improving the quality of their English teaching or going English Medium, following
pressure from parents who in our focus groups were very strongly supportive of their children
acquiring a high standard of English.
12 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Table 3.12 Classroom Observations: Interaction: CHITWAN DISTRICT
Not
met Partially met
Met More than met
Outstanding
Interaction: all teachers as %: Chitwan
The teacher used pair work effectively
Baseline 100 0 0 0 0
MID P 68 32 0 0 0
END P 6 27 37 29 0
The teacher used group work effectively
Baseline 100 0 0 0 0
MID P 55 42 3 0 0
END P 0 22 39 39 1
The teacher gave encouragement to pupils
Baseline 38 62 0 0 0
MID P 0 68 32 0 0
END P 0 5 68 26 1
The teacher gave effective feedback as necessary
Baseline 58 37 4 0 0
MID P 0 93 7 0 0
END P 0 11 70 18 0
Table 3.13 Classroom Observations: Interaction: LAMJUNG DISTRICT
Not
met Partially met
Met More than met
Outstanding
Interaction: all teachers as %: Lamjung
The teacher used pair work effectively
Baseline 99 0 0 0 0
MID P 88 12 1 0 0
END P 78 2 4 7 6
The teacher used group work effectively
Baseline 98 1 0 0 0
MID P 72 14 13 0 0
END P 48 11 25 9 4
The teacher gave encouragement to pupils
Baseline 26 71 3 0 0
MID P 2 57 41 0 0
END P 0 4 38 44 14
The teacher gave effective feedback as necessary
Baseline 39 54 5 0 0
MID P 1 52 47 0 0
END P 0 4 34 33 29
Lessons in Chitwan score higher for the use of pair or group work, which suggests strong
teacher planning of interactions to involve students in use of English. These are two areas
where Lamjung does very poorly with 78% and 48% of classroom observations recording no
pair or group work respectively. By contrast, Lamjung teachers score significantly higher on
giving encouragement and feedback to students at the top two levels of more than
‘met/outstanding’. The figures overall serve to again demonstrate the degree of improvement
made across the board: for example in Chitwan the measure around effective feedback
scored 95% below the satisfactory level of ‘met’ at baseline, and by project end this had
dropped to just 11% partially met.
13 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Table 3.14 Classroom Observations: Effectiveness of teaching: CHITWAN DISTRICT
Not
met Partially met
Met More than met
Outstanding
Effectiveness of teaching: all teachers as %: Chitwan
The teacher gave students time to
respond
Baseline 26 71 3 0 0
MID P 0 66 34 0 0
END P 0 11 66 24 0
The teacher used materials to supplement the lesson
Baseline 61 39 1 0 0
MID P 18 43 39 0 0
END P 0 10 51 38 1
The teacher checked students understood
Baseline 65 29 5 0 0
MID P 1 86 13 0 0
END P 0 9 72 19 0
Table 3.15 Classroom Observations: Effectiveness of teaching: LAMJUNG
Not
met Partially met
Met More than met
Outstanding
Effectiveness of teaching: all teachers as %: Lamjung
The teacher gave students time to
respond
Baseline 30 66 4 0 0
MID P 2 61 36 2 0
END P 0 2 32 45 20
The teacher used materials to supplement the lesson
Baseline 71 27 1 0 0
MID P 34 55 11 0 0
END P 2 12 35 44 7
The teacher checked students understood
Baseline 37 57 6 0 0
MID P 1 57 40 1 0
END P 0 2 27 50 21
These areas of pedagogy show a lot more very good practice in Lamjung than in Chitwan,
where the great majority are ‘met’ and thus satisfactory.
Across all the indicators we can tentatively identify differential success in the two districts of
the transfer of training into the classroom. These differences – for example more English in
the classroom in Chitwan, more improvement in some aspects of pedagogical practice in
Lamjung – should be reflected upon in defining and structuring future projects.
Chitwan: Baseline n = 476 MID P n = 314 END P n = 299 Lamjung: Baseline n = 425 MID P n = 256 END P n = 253
14 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Classroom observations: comparing all teachers by Gender
Classroom observation data were generated from a good gender balance of teachers – the
sample group consisted of 98 female and 105 male teachers [see Appendix].
Table 3.16 Classroom Observations: Use of English: all FEMALE teachers
0% 1-10% 11-25% 26-50% 51%+
Use of English: all female teachers, as percentage
The teacher used English during the lesson
Baseline 30 51 19 1 0
MID P 12 30 52 7 0
END P 4 2 10 31 54
The students spoke to the teacher in English
Baseline 83 17 0 0 0
MID P 56 41 2 0 0
END P 12 30 43 11 3
Table 3.17 Classroom Observations: Use of English: all MALE teachers
0% 1-10% 11-25% 26-50% 51%+
Use of English: all male teachers, as percentage
The teacher used English during
the lesson
Baseline 26 60 14 0 0
MID P 6 30 59 5 0
END P 4 2 16 28 49
The students spoke to the teacher in English
Baseline 85 15 0 0 0
MID P 50 49 1 0 0
END P 13 42 34 8 3
Use of English by the teacher during the lesson shows no significant difference; there is more
student use of English to the teacher in lessons with a female teacher.
Table 3.18 Classroom Observations: Interaction: all FEMALE teachers
Not
met Partially met
Met More than met
Outstanding
Interaction: all female teachers as %
The teacher used pair work effectively
Baseline 100 0 0 0 0
MID P 73 26 1 0 0
END P 33 17 22 24 3
The teacher used group work effectively
Baseline 99 1 0 0 0
MID P 58 32 9 0 0
END P 17 16 33 31 3
15 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Table 3.19 Classroom Observations: Interaction: all MALE teachers
Not
met Partially met
Met More than met
Outstanding
Interaction: all male teachers as %
The teacher used pair work effectively
Baseline 99 1 0 0 0
MID P 81 19 0 0 0
END P 46 14 22 14 3
The teacher used group work effectively
Baseline 99 1 0 0 0
MID P 67 27 5 0 0
END P 28 17 33 19 2
Female teachers were observed using more group and pair work.
Table 3.20 Classroom Observations: Effectiveness of teaching: all FEMALE teachers
Not
met Partially met
Met More than met
Outstanding
Effectiveness of teaching: all female teachers as %
The teacher used a variety of activities
Baseline 56 43 1 0 0
MID P 2 78 20 0 0
END P 2 22 38 33 7
The teacher used gestures to support
meaning
Baseline 43 55 2 0 0
MID P 13 66 21 0 0
END P 2 7 43 47 3
The teacher used materials to supplement the lesson
Baseline 61 37 2 0 0
MID P 23 45 32 0 0
END P 1 6 43 47 3
The teacher corrected students
Baseline 31 61 9 0 0
MID P 2 71 26 0 0
END P 0 6 55 30 9
Table 3.21 Classroom Observations: Effectiveness of teaching: all MALE teachers
Not
met Partially met
Met More than met
Outstanding
Effectiveness of teaching: all male teachers as %
The teacher used a variety of activities
Baseline 59 41 0 0 0
MID P 3 81 14 0 0
END P 2 27 41 23 6
The teacher used gestures to support meaning
Baseline 41 58 0 0 0
MID P 20 63 17 0 0
END P 1 18 43 36 2
The teacher used materials to supplement the lesson
Baseline 70 29 0 0 0
MID P 28 51 21 0 0
END P 1 16 45 34 3
The teacher corrected students
Baseline 45 51 3 0 0
MID P 1 72 25 1 0
END P 0 11 49 29 11
16 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Female teachers achieve higher classroom observation ratings than males on the first three of
these measures, with typically 10% fewer ‘partially met’ and 10% more ‘more than met’. The
last measure here, correction of students, is the only measure where more men than women
score ‘more than met/outstanding’.
The overall picture is that from similar starting scores female teachers have made greater
improvement in a range of classroom activities and pedagogical methods, including student
use of English, with little gender difference in terms of teacher use of English.
Male teachers Baseline n = 455 MID P n = 283 END P n = 263 Female teachers Baseline n = 454 MID P n = 287 END P n = 289
Classroom observations: the A0 teachers
As noted above the trainees in Chitwan included 48 who recorded a low APTIS level of A0 in
the initial testing, with a smaller number, 38, in Lamjung scoring A0. These teachers were
thought to be outside the level of personal English needed to benefit from the ET:TE+
training, though their scores were at the top end of A0.
A comparison was made to measure their progress against all teachers.
Table 3.22 Classroom Observations: Use of English: all A0
0% 1-10% 11-25% 26-50% 51%+
Use of English: A0 teachers
against all teachers, as %
The teacher used English during the lesson
Baseline A0 35 50 13 1 0
END P: all 4 2 13 29 52
END P A0 5 3 20 27 45
The teacher gave instructions in English
Baseline A0 62 34 3 0 0
END P all 4 4 14 32 45
END P A0 5 3 20 33 39
The teacher gave feedback in English
Baseline A0 69 28 2 0 0
END P all 5 5 21 34 35
END P A0 5 3 23 38 30
The students spoke to the teacher in English
Baseline A0 82 18 0 0 0
END P all 12 36 39 10 3
END P A0 16 34 38 9 3
The students spoke to each other in English
Baseline A0 100 0 0 0 0
END P all 54 31 9 4 2
END P A0 70 15 5 5 3
17 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Table 3.23 Classroom Observations: Interaction: all A0
Not
met Partially met
Met More than met
Outstanding
Interaction: A0 teachers against all teachers as %
The teacher used group work effectively
Baseline A0 100
END P all 22 17 33 25 3
END P A0 27 21 32 14 4
The teacher gave encouragement to pupils
Baseline A0 35 64 2
END P all 0 5 54 34 7
END P A0 0 3 54 31 11
It is striking that the pattern for the A0 cohort mirrors closely that of ‘all teachers’ [taken
from Table 3.1] in these typical exemplar measures. So their own English may be weak but
they could and did use it as much in the classroom as A1 or A2 teachers. We were not
attempting to evaluate the quality of the English being used which would have involved a
different evaluation process. The A0 teachers used group work too with positive classroom
behaviour.
Table 3.24 Classroom Observations: Effectiveness of teaching: all A0
Not
met Partially met
Met More than met
Outstanding
Effectiveness of teaching: all teachers as %
The teacher used a variety of activities
Baseline 63 38
End P: All 2 24 39 28 6
R2 2 32 33 23 10
The teacher gave students time to
respond
Baseline 35 61 4
End P: All 0 7 51 33 9
R2 0 8 44 31 17
The teacher used gestures to support meaning
Baseline 47 52 2
End P: All 1 12 43 42 2
R2 2 16 41 38 3
The teacher used materials to
supplement the lesson
Baseline 67 32 2
End P: All 1 11 44 41 3
R2 15 40 40 5
The teacher corrected students
Baseline 39 53 9
End P: All 0 8 52 30 10
R2 0 7 46 31 15
The teacher checked students understood
Baseline 51 40 9
End P: All 0 5 52 33 10
R2 0 3 47 35 15
The lesson effectively developed a skill Baseline 68 28 3
End P: All 0 13 62 23 2
R2 0 18 53 26 3
The lesson effectively covered a language area
Baseline 80 18
End P: All 1 11 64 22 3
R2 0 12 59 24 5
18 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
In fact on all the measures of effectiveness of teaching in Table 3.24 the A0 teachers actually
did slightly better than all teachers on all but one measure at higher ‘more than
met/outstanding’ levels.
On some measures there is also a slightly larger percentage of A0 teachers than all teachers
left behind at the ‘not met’ level. This suggests that many but not all A0 teachers were
competent and enthusiastic teachers [with weaker personal levels of English] who made good
progress through the project, especially in general classroom effectiveness and in using
English as the medium of instruction in the classroom. The ET:TE+ project’s training manual,
follow up support, etc, can work effectively for teachers with a low Basic User competence in
English, if care is taken over the selection of participants.
All teachers Baseline n = 916 END P n = 552 A0 teachers Baseline n = 200 END P n = 128
A full set of all the above tables is available as part of the Technical Report.
19 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
4. Findings: APTIS testing of teachers’ levels of competence in English
Nominated teachers were tested on Saturdays in February and March 2013 by British Council
Nepal supported by this team using the British Council’s APTIS test to assess adults’ ability in
English, and again on two successive Saturdays in late November and early December 2013
at the end of the project.
Initial baseline testing
There were two rounds of initial testing in each district at central locations. A total of 443
teachers were tested. Outcomes were:
Table 4.1 All teachers’ overall APTIS scores, initial testing
A0 A1 A2 B1 and better
Total (n = 443) 230 173 31 9
52% 39% 7% 2%
English teachers (n = 84) 27 48 7 2
32% 57% 8% 2%
Non-English teachers (n = 359) 203 125 24 7
57% 35% 7% 2%
All teachers, Chitwan (n = 230) 130 90 10 0
57% 39% 4%
All teachers, Lamjung (n = 213) 100 83 21 9
47% 39% 10% 4%
Female teachers (n = 193) 99 79 11 4
51% 41% 6% 2%
Male teachers (n = 250) 131 94 20 5
52% 38% 8% 2%
There was no statistically significant difference between the performance of female and male
teachers; at a second level of analysis these gender scores remained very similar across the
two districts. Variations between scores at English medium and Nepali medium schools as a
whole or in each district separately did not reach a level of statistical significance.
End of project final testing
All those teachers who participated in the ET:TE+ training project were expected to attend to
be re-tested in November/December 2013. Every effort was made to inform and encourage
teachers to attend the test centre, through implementing partners, DEO staff, etc. 81% did
so; of the remainder perhaps half were known to be ill or had personal reasons leaving them
unable to attend, had changed schools or had dropped out of the project at a much earlier
stage. Attendance was higher in Chitwan than in Lamjung.
Table 4.2 All teachers’ overall APTIS scores, final testing
A0 A1 A2 B1 and better
Total (n = 234) 67 137 30 0
29% 59% 13%
English teachers (n = 47) 10 31 6 0
21% 66% 13%
Non-English teachers (n = 187) 57 106 24 0
30% 57% 13%
20 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
All teachers, Chitwan (n = 128) 21 81 26 0
16% 63% 20%
All teachers, Lamjung (n = 106) 46 56 4 0
43% 53% 4%
Female teachers (n = 110) 30 64 16 0
27% 58% 15%
Male teachers (n = 124) 37 73 14 0
30% 59% 11%
In the final test no one scored at the Independent User level of B1 or better. Comparisons
drawn from Tables 4.1 and 4.2 should be treated with caution in that the baseline in Table
4.1 includes many teachers who did very poorly and did not go on to join the ET:TE+
training. However in Table 4.3 below the scores of all individuals who were tested on both
occasions are brought together and compared.
Table 4.3 Overall APTIS scores, same teachers, initial and final testing
A0 A1 A2 B1 and better
Total, initial test (n = 234) 83 123 23 5
35% 53% 10% 2%
Total, final test (n = 234) 67 137 30 0
29% 59% 13%
These figures show a slight improvement in APTIS test scores in the cohort as a whole with
6% moving up from A0. However this masks a more significant feature which is the high level
of fluidity in the actual scoring. Only 103 teachers [44%] kept the same score. In Chitwan 51
teachers went up at least one level, 11 went down at least one level and 66 recorded the
same level. In Lamjung 23 teachers went up at least one level, 46 dropped at least one level
and 37 recorded the same level. In total 7 individuals went up two levels and 7 down two
levels. Of the 57 who dropped a level 27 were female and 30 male, and of those going up a
level 38 were female and 36 male.
There is no convincing evidence that teachers were more likely to score highly or to improve
if in an English Medium school
Commentary
The CEFR descriptor for A1 indicates a low level of basic language competence:
Can understand and use familiar everyday expressions and very basic phrases aimed at
the satisfaction of needs of a concrete type. Can introduce him/herself and others and can
ask and answer questions about personal details such as where he/she lives, people
he/she knows and things he/she has. Can interact in a simple way provided the other
person talks slowly and clearly and is prepared to help.
Initial APTIS test results showed that 39% of those tested performed at this A1 level with
52% scoring below A1 and therefore graded as A0. Only 9% scored above this basic level at
A2 or better. The teachers took two APTIS tests, the first testing Grammar and Vocabulary
and the second testing Reading. Analysis of the results suggested that it was the Reading test
that became the discriminator at the A0/A1 borderline. Eg. in Lamjung Sample 1, Candidate
53 scored 16 + 44 = B2, while Candidate 8 scored 22 + 4 = A0.
The ET:TE+ training project was developed for students performing on entry at language
levels A2/B1 but was seen in Nepal as accessible to A1 students given the project’s twin
21 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
emphases on development of personal language ability as well as development of pedagogical
skills. It was felt that some of the A0 teachers with respectable scores on the Grammar and
Vocabulary test could also benefit from the project, having been alerted to their low
performance level as a key area for development.
It is disappointing that the APTIS testing did not show much of an improvement in individual
teachers’ competence in English on these measures, however this was not expected in the
project design. There was a great deal of fluidity in scoring – only 44% of those tested
received the same grade in both test rounds. This team’s experience of the APTIS process
would be
- the test was invaluable as a filter to remove those with such low levels of English as to
be inappropriate for the ET:TE+ training
- the testing was taken very seriously by all concerned and added status, importance
and visibility to the ET:TE+ project. One district DEO told us he was very pleased than
an international standard test was being used
- written tests of Grammar and Vocabulary and Reading are not a good fit with the aim
of the training to develop and encourage speaking and listening skills and the use of
those skills in the classroom. Perhaps it would be unreasonable to expect the positive
impact of the training to show up strongly in the repeat APTIS tests
22 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
5. Findings: Interviewing teachers of English
Teachers of English involved in the ET:TE+ project were identified as a subgroup of the
training cohort. At Baseline each of those who identified as teachers of English [n = 84] was
interviewed at their school. Data was 90+% complete. In the monitoring phase, all teachers
in the sample were interviewed at mid programme and again at end of programme [n = 54].
Gender and age data on the group of English teachers showed a gender balance and a spread
of age and therefore experience. The age profile was younger in Chitwan where 45% of the
sample was aged 35 or younger [Lamjung 23%].
Table 5.1 Gender and Age data on English teachers
Chitwan [n = 47] Lamjung [n =37] Total
English
medium
Nepali
medium
English
medium
Nepali
medium
Female 18 7 16 1 42
Male 17 4 16 4 41
Age below 25 3 2 2 0 7
25-35 13 3 5 0 21
36-45 9 5 15 3 32
Over 45 7 5 4 1 17
Teachers and their own English
Baseline data
33% said that part of their own education had been in English but 67% said none of their
education had been in the medium of English. When asked how frequently they used English
outside school, 41% responded ‘daily’, 18% said ‘weekly’ and 41% said ‘less than once a
week’. But an overwhelming 88% felt it was ‘essential’ for their pupils to learn English to a
good standard.
Interviewees were asked about their own English, and their level of confidence in teaching
English.
Table 5.2 How would you describe the standard of your Speaking and Verbal Understanding
in English at the moment?
Baseline Chitwan [n = 47] Lamjung [n =37] Total
English medium
Nepali medium
English medium
Nepali medium
a good standard 7 0 0 0 9%
quite a good standard
21 6 13 3 52%
not a very good standard
7 5 16 2 37%
very weak 0 0 2 0 2%
Interviewees were asked how confident they were at this start point in their teaching of
English. 13% said they were ‘very confident’, while 47% said ‘quite confident’. 38% rated
themselves as ‘uncertain’ with just 2% choosing ‘very uncertain’.
23 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Mid programme responses
Interviewees were asked again about their own speaking and verbal understanding of English.
80% chose ‘quite a good standard’, up from 52% at baseline, and 18% chose ‘not a very
good standard’, down from 37% at baseline. 41% recorded a large or very large difference in
their skill levels in English with 48% saying their confidence to use English was now much or
very much higher.
95% felt ‘quite confident’ or ‘very confident’ to use spoken English all the time in the
classroom. 96% reported they felt ‘more confident’ or ‘very much more confident’ in teaching
English as a result of the ET:TE+ training.
End of project responses
Asked the same question again, numbers choosing ‘not a very good standard’ fell again, down
to 11% from 18% at mid project and from 37% at baseline.
Table 5.3 How would you describe the standard of your Speaking and Verbal Understanding
in English at the moment?
Baseline Mid programme End of programme
a good standard 9% 2% 15%
quite a good standard 52% 80% 74%
not a very good standard 37% 18% 11%
very weak 2% 0 0
74% also reported a large or very large difference in their own skill levels in English, up from
41% at the mid project stage, indicating the progress they felt they had made during the
period of the project.
Confidence levels to use English in the classroom remained high and in interview many
teachers spoke enthusiastically about the difference they felt in their own confidence to teach
English through active use of the language in the classroom and interactive methods.
Teachers and good practice in the classroom
The interviews with teachers of English also probed interviewees’ knowledge and
understanding of good classroom practice in two aspects: student-centred learning and
communicative language teaching. Both of these areas were to be highlighted through the
ET:TE+ training project.
Baseline responses
The expression ‘student centred teaching’ was recognized by 90% of English teachers at
interview. They selected two key characteristics but with significantly lower scores: ‘teaching
to the children’s level’ 74% and ‘interaction between teacher and students’ 71%.
74% recognized the expression ‘communicative language teaching’ but the most-reported
example in this area was ‘pupils talking together in pairs or groups’ mentioned by only 56%.
Research associates conducting the interviews reported much greater hesitancy and
uncertainty in responses. In both these items, responses indicate awareness of the principal
term but a lack of understanding of what it would mean in concrete classroom practice.
Teachers were asked about the availability to them of mentoring and/or peer support. 75%
reported mentoring from headteacher or senior teacher and 66% access to peer support.
24 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Mid project responses
In Mid Project interviews key student-centred aspects were reinforced: ‘interaction between
teacher and students’ is now at 85% and ‘student/student’ interaction up to 81%. ‘teaching
to the children’s level’ has fallen away to 59%. With communicative language teaching 68%
selected ‘pupils talking together’ but only 31% ‘pupils given roles like reporting back’, a more
specific lesson planning technique to promote oral work. All those responding wanted to use
more resources in the classroom.
At Mid Project the great majority of these teachers reported the availability of opportunities to
access mentoring and peer support.
There was a 95% or higher approval rating for the timetable and content of the training and
100% positive feedback on the ET:TE+ trainers and the help received with resources.
Teachers were very proud of having the opportunity to participate in the ET:TE+ programme,
often describing this as the best training they’d ever had.
More than 80% said they had been able to add extra resources in English. These were
overwhelmingly ‘maps, charts and wall displays’ [38 out of 54 teachers], using some
community funding [14/54] or ‘I have bought them myself’ [22/54]. Research Associates
reported the use of homemade instructional materials and resources – flashcards, wall
pictures with English words beneath, etc. Some of these had been made jointly by teacher
and children. A lack of resources was the most quoted barrier to putting ET:TE+ learning into
practice [28/54].
Teachers remained firmly of the view that learning good English is ‘important’ [30%] or
‘essential’ [70%] for their pupils.
End of project responses
When re-interviewed at the end of the project, a very similar picture emerged of a group of
professionals who had benefited greatly from their training weeks and follow up sessions.
There was more comment about the detail of pedagogical techniques and Research
Associates recording these interviews fed back that by this end point six months later
teachers appeared to be discussing established practice rather than practice which they were
trying for the first time. This would be consistent with findings from classroom observations.
Finally the English teachers were asked what they felt they had gained from the project:
Table 5.4 What do you feel you have gained from the ET:TE+ training?
Aspect END P No.
(n = 53)
to improve my speaking and communication in English 49
to improve my method of teaching 41
to gain confidence in my teaching 39
to learn how to use resources 39
to improve my reading and writing English 36
to meet and talk with other English teachers 32
to gain promotion in a school 13
The English teachers’ choice of statements is positive in that they choose statements which
actually correspond well with the aims of the project, which has delivered in its main target
areas.
25 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
6. Findings: Other views expressed in schools
During fieldwork, views were also sought at each school through:
A focus group of pupils, discussing the English lesson just observed and their
experience of English
A focus group of adult stakeholders, normally including English teacher(s),
headteacher, SMC chair, parents and other local stakeholders
Analysis of the qualitative data from focus groups indicates:
The young primary-age children who formed our pupil focus groups confirmed that the
English lesson just observed by our researcher was typical of the lessons they now
received. They are able to use spoken English very infrequently in class; 85% said they
rarely or never talk to a classmate in English and 89% that they rarely or never talk to a
group of classmates in English, but 68% said they sometimes and 11% that they nearly
always talk to their teacher in English. Well over 90% of groups said they had enjoyed
their English lesson.
Adults in each of the 54 focus group meetings included the English teacher, the
headteacher, an SMC member and at least one parent – often groups were quite large,
even the whole SMC with each member keen to add his or her words of support. They saw
the world changing especially for the next generation and the importance of equipping
students with a good standard of English. Schools had a more selfish interest in retaining
the support of parents against private school competition and loss of pupil numbers.
100% agreed or strongly agreed that it is important for children to learn good English and
96% that the school/community will do all it can to support more effective English
teaching: 98% felt that the ET:TE+ training was having a positive effect at the school.
The views of parents were gathered through the focus group meetings and through informal
contacts reported by Research Associates. Parents continue to strongly support their pupils
being taught to a good standard in English. They talked of wanting continuing monitoring and
support for ET:TE+ teachers so they continue to improve. Parents noted the wider range of
activities taking place and reported other teachers also using English more. One comment
from a rural primary in Lamjung was that better English teaching would help retain pupils in
their village school and hence keep it open, rather than parents sending their children into
town, to government or private schooling.
Research Associates reported specific interest from some SMC chairpersons many of whom
have political connections, particularly where the school was considering converting to English
Medium. Headteachers expressed confidence that the project was impacting on teacher and
school, picking out the clear, well-delivered training, the high quality manual and the fact that
training events had taken place outside school session days which helped them
organisationally. These points were made particularly by teachers from rural areas.
26 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
7. Conclusions
1. Standards of competence in English among teachers wishing to join the project were
found to be very low even in a generally more developed district like Chitwan. The initial
APTIS tests found 90% at A0 or A1. What is locally considered ‘good English’ is in
absolute language terms not very good at all. This may reflect previous and to a large
extent current curriculum and teaching methods in Nepali schools and the teacher’s own
learning of English, as well as their limited daily access to and use of English. 41% of
English teachers said they used English themselves ‘less than once a week’, which may be
under-reported given the high status nature of English. Many people including those
active on the ground in the districts were surprised by the low scores. There are large
training needs at a basic level, ie below the ET:TE + threshold level.
2. The low levels of competence in English – well below that required to teach English to
others and to teach other subjects through the medium of English - cast doubt on the
wisdom or the ability of government schools to go ‘English Medium’ unless and until many
of their teachers have improved their own competence in English or what being an
‘English Medium school’ means in language terms. If the standard of English is inadequate
trying to teach subject content through that medium will fail, while English itself is a
much-failed subject at SLC in government schools.
3. The baseline classroom observations also showed the extremely limited extent of good
pedagogical practice actually happening before the course began. Most teachers were
familiar with expressions such as ‘child-centred learning’ but fewer could confidently give
examples of what this meant in the classroom. There was strong support from their
schools to join the project.
4. District officials, teachers and schools when first contacted were enthusiastic to improve
the quality of English learned by their pupils: a good standard of English is rated
“essential” for the next generation, and was often described as the key driver for the
spread of English Medium private schools. This enthusiasm was repeated frequently at the
end of the project, from District Education Officer to minority ethnic Chepang students in
a rural school. The local print and FM radio media treated the initial APTIS testing and the
arrival of ET:TE+ training as major local news events with live interviews in March 2013
by Kalika FM, the mid-Nepal commercial radio station based in Bharatpur, Chitwan. There
were more reporters at the final APTIS testing in Chitwan.
5. Into this very mixed context, of low standards but great positive enthusiasm, the ET:TE+
project has been extremely successful as the data in this Report demonstrate. The
enthusiasm has been maintained, new practices have been tried out and have become
mainstream for many teachers, as their students testify. Teachers have had the
confidence, and a realistic confidence, to make English the language of their classroom
and of their students. Recorded progress shows that classroom practice can be changed
and that change sustained, which is a very positive finding.
6. Key successes evidenced from classroom observation and interviews include
Transformation in the use of English by teachers in the classroom
Positive and significant progress in pupils using spoken English in the classroom, from
a very low base
Introduction and sustaining of interactive group work and pair work
Greatly improved pupil involvement in lessons, and feedback and encouragement from
teachers
The successful introduction and use of more materials in the classroom
Extending the reach of ET:TE+ training to A1 and A0 beginner-standard teachers
Stimulated, motivated teachers strongly supported by schools to improve practice
27 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Additionally, the use of an external test, here APTIS, was valuable and valued. Its use
as a gatekeeper should be reviewed in view of the degree of ‘churn’ in scores and the
progress made by A0 teachers
7. Areas of least change within the aims of the project include
Making interactive, more student-centred classrooms the norm, eg pair and group
work and students’ talking to the teacher in English
Meeting the needs of all teachers – many have such poor levels of English as to be
outside the target group, although those scoring A0 but entering the training made
good progress
The intended focus on rural and deprived locations was not taken forward by
implementing partners, though some remote areas such as Mardi in southern Chitwan
were represented
We would like to thank all those concerned for their invaluable help and support in compiling
this report, in particular district education officers and their staffs, the two implementing
partners, PANT films and of course British Council Nepal. Most thanks must go to the teachers
and their schools for their commitment to and support of the ET:TE+ project.
On behalf of Richard Thompson Associates
Monitoring & Evaluation Team
Technical Advisor Richard Thompson
Team Leader Min Bahadur Ranabhat
Research Associates Ram Prasad Paudel
Kiran Sedhain
Subodha Babu Chiluwal
Gajendra Laudahari
Data preparation and analysis Dr Shankar Khanal
Mr Raju Manandhar
28 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Appendix: Sample for Monitoring & Evaluation Programme
It was agreed in discussing this Monitoring & Evaluation Programme for ET:TE+ that a
sampling process would be used to identify a representative subset of teachers and schools in
each district to be monitored. This is a longitudinal study using data collected:
- as part of the baseline survey, March 2013
- in a first round of monitoring [R1], June/July 2013
- in a final round of monitoring [R2] scheduled for late 2013/early 2014
In each of the two districts a complete list of those who actually participated in the ET:TE+
training by NELTA [Chitwan] and GAN [Lamjung] in April 2013 was kindly supplied by those
organisations. Names were checked against our baseline records to ensure that baseline data
[classroom observations, teacher interviews, APTIS test scores] was in place. A sample was
then developed separately for each district that would contain a full data set and enable the
agreed collection of data under this Programme.
In Chitwan District the sample consisted of 30 schools, with 30 identified teachers of English
[one per school] for extended classroom observation and interview. NELTA is training
teachers from a total of 64 schools. The schools consisted of 8 Higher Secondary Schools, 7
Secondary Schools, 8 Lower Secondary Schools and 7 Primary Schools, with a mix of urban
and rural locations across the district. The schools contained a further 56 teachers training on
the project, teaching a variety of subjects including in some cases some English, giving a
total of 86 teachers of whom 44 were female and 42 male.
In Lamjung District the training project was concentrated by GAN in one part of the District
and in fewer schools [31 in total]. The sample therefore covered only 24 schools with 24
identified teachers of English [one per school] to be interviewed, The schools consisted of 2
Higher Secondary Schools, 6 Secondary Schools, 3 Lower Secondary Schools and 13 Primary
Schools. There were in many schools a higher number of teachers engaged in ET:TE training,
giving a full sample size of 117 teachers, of whom 54 were female and 63 male.
Baseline data for each of the schools and individual teachers in these samples was extracted
from the full baseline data set and used for later comparative purposes. The data set of the
sample was back-checked to test its representativeness. Taking the key question of use of
English in the classroom, as observed and recorded by this team, we found:
Baseline data: Classroom Observations for entire ET:TE+ cohort
0% 1-10% 11-25% 26-50% 51%+
Use of English: all teachers, as percentage (n = 916)
The teacher used English during the lesson
28 55 16 1 0
The teacher gave instructions in English
58 40 3 0 0
The teacher gave feedback in English
68 31 1 0 0
The students spoke to the teacher in English
84 16 0 0 0
The students spoke to each other in English
99 1 0 0 0
29 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14
Baseline data: Classroom Observations for the subset M & E ET:TE+ sample
0% 1-10% 11-25% 26-50% 51%+
Use of English: all teachers in sample schools, as percentage (n = 548)
The teacher used English during the lesson
28 54 17 1 0
The teacher gave instructions in English
57 39 4 0 0
The teacher gave feedback in English
69 29 2 0 0
The students spoke to the teacher in English
82 18 0 0 0
The students spoke to each other in English
98 2 0 0 0
All scores are rounded to nearest %.
These tables demonstrated that the pattern of scores across the entire baseline cohort was
very closely followed by the subset of teachers and schools selected as the M & E sample.
This confirmed the representative nature of the sample in terms of baseline data and enabled
us to proceed with confidence.