Transcript
7/29/2019 Petrie 2013 Exam Teaching Support
1/131
EXAM TEACHING SUPPORT:
AN
INVESTIGATION INTO THE
FORMS OF
PROFESSIONAL
SUPPORT
AVAILABLE TO TEACHERS OF ESOLEXAM PREPARATION CLASSES.
DAVID MARTIN ROBERT PETRIE
MAAPPLIED LINGUISTICS AND TESOL
SCHOOL OF LANGUAGES AND AREA STUDIES
PORTSMOUTH UNIVERSITY
JANUARY 2013
7/29/2019 Petrie 2013 Exam Teaching Support
2/131
i | P a g e
ABSTRACT:
The professional context of helping teachers prepare learners effectively for
English language exams led to the beliefs that exam class teachers do not have
all the information they need and are not provided with sufficient support. This
study investigates these beliefs by examining the research on exam class
teaching, the characteristics of and approaches to exam class teaching and
forms of support for exam class teaching. Five research questions are
formulated asking whether teachers need support, whether teachers feel as
though they need support, what forms of support teachers use, how teachers
feel about these forms of support and how they would like to see such support
improved in the future.
A descriptive survey was therefore designed with content arising from the
review of exam class teaching literature. The questionnaire was distributed to a
random volunteer sample via the internet and seventy-eight responses were
collated. The survey reveals that teachers do feel as though they need a
degree of support with their exam class teaching but did not uncover any clear
evidence showing teachers did need support with their teaching; though as
some teaching practices identified contradict the evidence from washback
studies, it is likely. It also found that teachers rely predominantly on support
resources that may be found in their staffrooms; namely course books,
supplementary books and the ideas of their colleagues; and that they also rate
these forms of support highly. Finally the survey found that teachers want more
support in the form of practical teaching ideas and materials that they can use
with their exam preparation classes.
This study concludes that the initial beliefs on the nature of exam preparation
class teachers are partially correct and that while there is no evidence to
suggest teachers lack information, there are clear indications that teachers lack
the support they want and that this is something which would be best rectified in
the future.
7/29/2019 Petrie 2013 Exam Teaching Support
3/131
ii | P a g e
TABLEOFCONTENTS:
ABSTRACT: ........................................................................................................ I
TABLE OF CONTENTS: .................................................................................... II
LIST OF TABLES AND FIGURES ................................................................... VI
PREFACE AND ACKNOWLEDGEMENTS:.................................................... VII
GLOSSARY OF ACRONYMS: ....................................................................... VIII
CHAPTER ONE: THE RATIONALE AND CONTEXT FOR THE STUDY ......... 1
1.1THE RESEARCHERS PERSONAL CONTEXT ....................................................... 1
1.2A BELIEF ABOUT TEACHERS KNOWLEDGE OF EXAMS ........................................ 2
1.3THE LITERATURE ON TEACHING EXAM PREPARATION CLASSES .......................... 3
1.4A BELIEF ABOUT SUPPORT FOR TEACHERS OF EXAM PREPARATION CLASSES...... 5
1.5CONSTRAINTS OF THE DISSERTATION ............................................................. 6
1.6SUMMARY OF CHAPTER ONE......................................................................... 7
CHAPTER TWO: AN OVERVIEW OF EXAM CLASS TEACHING ................... 8
2.1A DEFINITION OF EXAM CLASS TEACHING......................................................... 8
2.2THE CHARACTERISTICS OF EXAM CLASS TEACHING.......................................... 9
2.3APPROACHES TO EXAM CLASS TEACHING. ..................................................... 11
2.3.1 Constituents of a Methodological Approach ....................................... 11
2.3.2 Overall approaches to exam preparation classes .............................. 13
2.4EFFECTIVENESS OF EXAM PREPARATION CLASSES......................................... 16
2.5FORMS OF SUPPORT FOR EXAM CLASS TEACHING ......................................... 20
2.6SUMMARY OF CHAPTER TWO ...................................................................... 22
CHAPTER THREE: RESEARCH METHODOLOGY ....................................... 24
3.1RESEARCH MATERIALS AND PROCEDURES ................................................... 24
3.2RESEARCHAIMS ........................................................................................ 25
3.3RESEARCH METHODOLOGY......................................................................... 25
3.3.1 Define objectives ................................................................................ 26
7/29/2019 Petrie 2013 Exam Teaching Support
4/131
iii | P a g e
3.3.2 Identify target population .................................................................... 26
3.3.3 Literature review ................................................................................ 26
3.3.4 Determine sample .............................................................................. 26
3.3.5 Identify survey instruments ................................................................ 27
3.3.6 Design survey procedures ................................................................. 27
3.3.7 Identify analytical procedures ............................................................. 28
3.3.8 Determine reporting procedures ........................................................ 28
3.4CONDUCTING A PILOT STUDY ...................................................................... 29
3.5RESEARCH DESIGN .................................................................................... 31
3.5.1 Question Types and Technological Affordances ................................ 31
3.5.2 Design of Section 1: Professional Biography .................................... 33
3.5.3 Design of Section 2: Exam Class Teaching ....................................... 33
3.5.3 Design of Section 3: Professional Support ......................................... 34
3.6RESEARCH PARTICIPANTS........................................................................... 36
3.7SUMMARY OF CHAPTER THREE ................................................................... 38
CHAPTER FOUR: PRESENTATION OF RESULTS ....................................... 40
4.1 SURVEY RESULTS PART TWOEXAM CLASS TEACHING .............................. 40
4.1.1 Survey Question 7: Which exams have you prepared learners for? . 40
4.1.2 Survey Question 8: How confident are you in your ability to preparelearners for these exams?........................................................................... 41
4.1.3 Survey Question 9: Which of the following aspects do you include inyour exam preparation classes? ................................................................. 43
4.1.4 Survey Question 10: What proportion of class time do you spend onthose aspects identified in the last question? .............................................. 44
4.1.5 Survey Question 11: Do your current exam preparation courses use abook? .......................................................................................................... 45
4.1.6 Survey Question 12: Do you like teaching exam preparation classes?.................................................................................................................... 45
4.1.7 Survey Question 13: Why? / Why not? ........................................... 46
4.2SURVEY RESULTS PART THREE: PROFESSIONAL SUPPORT........................... 47
7/29/2019 Petrie 2013 Exam Teaching Support
5/131
iv | P a g e
4.2.1 Survey Question 14: Which of the following forms of support do youuse now or have you used in the past? ....................................................... 48
4.2.2 Survey Question 15: Given the opportunity, which of the followingforms of support do you prefer to use? ....................................................... 49
4.2.3 Survey Question 16: How frequently do you refer to or use theseforms of support? ........................................................................................ 50
4.2.4 Survey Question 17: How would you rate the forms of support thatyou use or have used in the past? .............................................................. 51
4.2.5 Survey Question 18: Where you have indicated a resource as beingsatisfactory or poor, please could you indicate how you would like to see itimproved? ................................................................................................... 53
4.2.6 Survey Question 19: Is there any additional support you feel you needto improve your exam class teaching? ........................................................ 54
4.2.7 Survey Question 20: If there was one thing that you could changeabout your exam class teaching what would it be? ..................................... 55
4.3CHAPTER FOUR SUMMARY .......................................................................... 56
CHAPTER FIVE: DISCUSSION OF FINDINGS .............................................. 59
5.1RESEARCH QUESTIONA:DO TEACHERS FEEL AS THOUGH THEY NEED SUPPORTIN THEIR EXAM CLASS TEACHING? ...................................................................... 59
5.2RESEARCH QUESTION B: DO TEACHERS NEED SUPPORT IN THEIR EXAM CLASSTEACHING? ...................................................................................................... 61
5.3RESEARCH QUESTION C: WHAT FORMS OF SUPPORT ARE USED BY TEACHERSOF EXAM CLASSES? .......................................................................................... 63
5.4RESEARCH QUESTION D: WHAT ARE THEIR FEELINGS TOWARDS THESE FORMSOF SUPPORT? .................................................................................................. 64
5.5 RESEARCH QUESTION E: WHAT CHANGES TO THIS SUPPORT OR WHATADDITIONAL SUPPORT WOULD TEACHERS OF EXAM PREPARATION CLASSES LIKE TO
SEE IN THE FUTURE?......................................................................................... 66
5.6SUMMARY OF CHAPTER FIVE....................................................................... 68
CHAPTER SIX: CONCLUDING REMARKS ................................................... 70
WORD COUNT ................................................................................................. 72
BIBLIOGRAPHY .............................................................................................. 74
DECLARATION ................................................................................................ 78
APPENDIX 1 COLLATED SCREENSHOTS OF THE ONLINEQUESTIONNAIRE: ........................................................................................... 79
7/29/2019 Petrie 2013 Exam Teaching Support
6/131
v | P a g e
APPENDIX 2A RAW DATA FROM THE QUESTIONNAIRE RESPONSES(CLOSED CLASS QUESTIONS) ..................................................................... 86
APPENDIX 2B RAW DATA FROM THE QUESTIONNAIRE RESPONSESTO SURVEY QUESTION 10............................................................................. 96
APPENDIX 2C
RAW DATA FROM THE
QUESTIONNAIRE RESPONSESTO SURVEY QUESTIONS 12 & 13. ............................................................... 104
APPENDIX 2D RAW DATA FROM THE QUESTIONNAIRE RESPONSESTO SURVEY QUESTION 18........................................................................... 112
APPENDIX 2E RAW DATA FROM THE QUESTIONNAIRE RESPONSESTO SURVEY QUESTION 19........................................................................... 115
APPENDIX 2F RAW DATA FROM THE QUESTIONNAIRE RESPONSESTO SURVEY QUESTION 20........................................................................... 118
7/29/2019 Petrie 2013 Exam Teaching Support
7/131
vi | P a g e
LISTOFTABLESANDFIGURES
FIGURE 1:THE INTER-RELATIONSHIP OFASPECTS OF EXAM CLASS TEACHING (AFTERSPRATT 2005) _______________________________________________ 18
TABLE 1:A SUMMARY OF WAYS OF CONTINUING PROFESSIONAL DEVELOPMENT,
AFTER JOHNSON (2009) AND WILDEN (2012) _________________________ 20
TABLE 2:FORMS OF EXAM TEACHING SUPPORT __________________________ 22
TABLE 3: COMPARISON OF QUESTION TYPES AVAILABLE IN GOOGLE DOCS TOQUESTION TYPES GIVEN IN NUNAN (1992, P.144) ______________________ 32
TABLE 4:SURVEY PARTICIPANTS EXPERIENCE AND HIGHEST LEVEL OF QUALIFICATION__________________________________________________________ 37
TABLE 5: ELT QUALIFICATIONS IN THE QUALIFICATIONS AND CREDIT FRAMEWORK /NATIONAL QUALIFICATIONS FRAMEWORK FOR ENGLAND AND NORTHERN IRELAND,ADAPTED FROM OFQUAL (2011). _________________________________ 37
FIGURE 2:COUNT OF RESPONSES FOR EXAMS TAUGHT BY RESPONDENTS________ 40
TABLE 6:OTHER EXAM TYPES INCLUDED BY RESPONDENTS TO QUESTION 7 ______ 41
TABLE 7:PERCENTAGE OF RESPONSES INDICATING THE LEVEL OF SUPPORT NEEDED BYEXAM PREPARATION CLASS TEACHERS FOR A RANGE OF ESOL EXAMS._______ 42
FIGURE 3:DEGREE OF SUPPORT REQUIRED BY RESPONDENTS FOR DIFFERENT EXAMS__________________________________________________________ 43
FIGURE 4:NUMBER OF RESPONDENTS INCORPORATING STATED ASPECTS INTO THEIREXAM CLASS TEACHING _________________________________________ 44
FIGURE 5:NUMBER OF RESPONDENTS WHO USE A COURSE BOOK WITH THEIR EXAMPREPARATION CLASSES ________________________________________ 45
FIGURE 6:RESPONSES TO THE QUESTION "DO YOU LIKE TEACHING EXAM PREPARATIONCLASSES?" AND THE PERCENTAGE OF RESPONDENTS WHO GAVE THAT ANSWER 46
FIGURE 7:FORMS OF SUPPORT USED BY SURVEY RESPONDENTS ______________ 48
FIGURE 8:PREFERENCES IN FORMS OF SUPPORT EXPRESSED BY RESPONDENTS___ 49
FIGURE 9:A COMPARISON OF FREQUENCY OF USE FOR FORMS OF SUPPORT ______ 50
FIGURE 10:OVERALL RESPONDENT SATISFACTION WITH FORMS OF SUPPORT _____ 52
FIGURE 11: LEVELS OF SUPPORT REQUIRED FOR ESOL EXAMS CONTRASTED WITHLEVEL OF EXPERIENCE OF ESOL EXAMS_____________________________ 59
FIGURE 12:COMPARISON OF FREQUENCY OF USE WITH COUNT OF TEACHERS WHO USEFORMS OF SUPPORT ___________________________________________ 64
7/29/2019 Petrie 2013 Exam Teaching Support
8/131
vii | P a g e
PREFACEANDACKNOWLEDGEMENTS:
I would like to thank John Naysmith and Peter Watkins for their help, their
support and their insightful comments on various versions of this manuscript,
especially for pointing out the obvious when the obvious wasnt so clear to me.
Thanks are also due to the anonymous participants who took part in the survey
and to my friends and colleagues at International House Coimbra and
International House Santa Clara who piloted the study.
Thanks must also go to my wife, Alexis, who not only helped provide a modicum
of sanity in stressful times, but balanced this with writing her own dissertation
and juggling childcare duties. Without your help, none of this would have
happened.
For Rosie and Sam, who would doubtless prefer it if it had dragons in it.
7/29/2019 Petrie 2013 Exam Teaching Support
9/131
viii | P a g e
GLOSSARY OF ACRONYMS:
This study uses a number of terms and acronyms, explanations of which aregiven here:
ALTE Association of Language Testers in Europe.
CAE
Certificate in Advanced English. CEFR C1 levelexamination awarded by University of Cambridge ESOLExaminations and also referred to as Cambridge English:Advanced.
CambridgeESOL
The providers and awarding body of a range of ESOL
examinations, including FCE, CAE and CPE, and part of theconsortium that provides and awards the IELTS exam.Cambridge ESOL is the corporate identity adopted by theUniversity of Cambridge ESOL Examinations, which wasalso previously known as UCLES (see below).
CEFR Common European Framework of Reference.
CPD Continuing Professional Development
CPECertificate of Proficiency in English. CEFR C2 levelexamination awarded by University of Cambridge ESOLExaminations and also referred to as Cambridge English:Proficiency
EAP English for Academic Purposes.
ELT English Language Teaching
ESOL English for Speakers of Other Languages
ESP English for Specific Purposes
ETPEnglish Teaching Professional. A professional pedagogicalmagazine for language teachers.
ETSEducational Testing Services, the providers and awardingbody for the TOEFL iBT (see below)
FCEFirst Certificate in English. CEFR B2 level examinationawarded by University of Cambridge ESOL Examinations
and also referred to as Cambridge English: First.
HLTM Humanising Language Teaching Magazine. A professional
7/29/2019 Petrie 2013 Exam Teaching Support
10/131
ix | P a g e
pedagogical magazine for language teachers.
IELTS
International English Language Testing System, the namegiven to both the exam and the organisation that providesand awards the exam, the IELTS organisation is apartnership between British Council, IDP: IELTS Australia
and Cambridge ESOL.
LCCIEB
London Chamber of Commerce & Industry ExaminationsBoard. An examinations provider and awarding body. In2002, LCCIEB merged with GOAL plc to create EDI(Education Development International), which waspurchased by Pearson Education Ltd in 2011.
METModern English Teacher. A professional pedagogicalmagazine for language teachers.
TOEFL iBT Test of English as a Foreign Language internet Based Test.
UCLESUniversity of Cambridge Local Examinations Syndicate. SeeCambridge ESOL.
UODLEUniversity of Oxford Delegacy of Local Examinations. Anexaminations provider and awarding body. No longercurrent, it merged with UCLES (see above) in 1995.
7/29/2019 Petrie 2013 Exam Teaching Support
11/131
1 | P a g e
CHAPTER ONE: THE RATIONALE AND CONTEXT FOR THE STUDY
This chapter details the professional context and personal experiences that
gave rise to this investigation and describes the rationale that underpins it,
articulating beliefs that guide the exploration of the literature in this area.
1.1THE RESEARCHERS PERSONAL CONTEXT
ESOL Exam preparation classes form the core responsibilities of my current
professional context: helping learners prepare, helping teachers of exam
preparation classes and acting as a school source of ESOL exam related
information and advice.
As a result, I have become interested in the way that teachers prepare their
learners for ESOL exams; in the content and delivery of exam preparation
lessons, the beliefs and attitudes that learners and teachers have towards their
classes, and whether or not what goes on in the classroom can be improved
upon in order to maximize learner exam performance.
Having worked in Asia and Europe, I found that learners in different ESOL
markets often request differing exam types. For example, in China learners
often request IELTS or TOEFL, whereas in Poland and Portugal learners
typically request FCE, CAE and CPE. Higher education institutions usually
require overseas students to demonstrate their ability with English by requesting
specific IELTS or TOEFL scores, for example University of Leicester (n.d.) or
University of Southampton (n.d.), and these requirements influence the exam
choices of the learners. Cambridge ESOL state that their exams are officially
7/29/2019 Petrie 2013 Exam Teaching Support
12/131
2 | P a g e
accepted by thousands of universities, employers and governments
(Cambridge ESOL n.d.) and so other stakeholders may also influence these
choices.
Teachers who work in these differing markets often become specialists in
particular exams and lack knowledge of others. This may mean that teachers
are not in a position to offer learners a full overview of all the exam possibilities.
There is also a question of whether language schools drive demand for specific
language exams or whether they react to it, but the tendency to focus on
specific exams means a cyclical process then arises, where lack of knowledge
or information limits choices, which in turn limits the commercial pressures to
provide alternatives, resulting in the specialization of education professionals in
a particular area, reducing the incentive to seek knowledge of alternatives,
resulting in lack of knowledge or information and so on.
1.2A BELIEF ABOUT TEACHERS KNOWLEDGE OF EXAMS
May (1996, p. 159-163) identifies sixteen specific examinations and awarding
bodies, while Burgess & Head (2005, p. 151-153) list fourteen awarding bodies
and a sample range of thirty-five representative exams from those bodies.
europa pages give the four main UK-based awarding bodies as being the
London Chamber of Commerce (LCCIEB), University of Cambridge (UCLES),
Pitman Qualifications, and the University of Oxford (UODLE), providing 47
different exams between them (europa pages, 2011), though as UODLE
merged with UCLES in 1995 (Hawkey, 2009, p. 133) this information may now
be out of date.
7/29/2019 Petrie 2013 Exam Teaching Support
13/131
3 | P a g e
English language exams are available for a wide range of ages, abilities and
purposes. With such a wide range, it seems likely that while teachers may have
experience of one exam they will also lack experience of others. This
specialization may not only affect teachers ability to advise on exam choices
effectively, but it may also affect their teaching of exam preparation classes as
they may attempt to transfer techniques and exam preparation strategies from
one exam to another without full knowledge of differences between the target
exams. In his study on the impact of FCE on the EFL classroom, Perrone
(2010), while not setting out to explore this effect directly, does show a degree
of transfer between an FCE preparation class and a general English class. If it
is possible for transfer to exist across categories of teaching, it seems likely that
transfer takes place across exams.
This has then led me to the belief that: Teachers of exam preparation classes
do not always have all the information they need to know in order to be effective
exam class teachers.
1.3THE LITERATURE ON TEACHING EXAM PREPARATION CLASSES
The specialisation of teachers is reflected in the academic literature, with
research that focuses either on specific exams (e.g. Alderson 2009, Hall 2010,
Hawkey 2009) or provides an in-depth focus on aspects of these exams (e.g.
Carrell 2007, Macqueen & Harding 2009, Uysal 2010). Researchers also
approach the area of ESOL exams from different perspectives, for example
investigating validity issues with an exam or considering washback or impact of
an exam on teaching practice (see also section 2.4), or from the perspective of
a specific skill. This specialisation may reflect the sponsorship of researchers
7/29/2019 Petrie 2013 Exam Teaching Support
14/131
4 | P a g e
by the exam awarding bodies (Hawkey 2004, Carrell 2007) and the
requirements of the research and development programs of the sponsors. One
outcome of the scope of research relating to language testing and exams is that
information is published in a wide range of sources, some of which are
subscription based academic journals, making it difficult for teachers to access.
More easily accessible to teachers might be the pedagogical journals and
magazines, to which some language schools subscribe and some of which host
their content online. However here there is remarkably little content that deals
specifically with exam classes. Contributors to these journals tend to paint with
a broader brush, for example in The Secret of Reading Prowse (1999)
examines the evidence supporting a reading for pleasure approach.
Gradman and Hanania report that extensive reading was a strongpredictor of TOEFL scores. This is something that teachers preparingstudents for FCE and CPE have always known intuitively, but it is nice to
see it proved through research.
(ibid, p10)
Whether Prowses faith in teacher intuition is justified is debatable (see section
2.4) and although the focus of the article is the value of extensive reading
programs and is aimed at a more general readership, it is a concern to see
something that could have wide implications for exam class teaching relegated
to a single sentence. While some articles do focus on teaching exam classes,
for example Dowlings IELTS Reading without tears (2011), they remain a
minority. There seems to be an assumption that teachers will automatically
apply what they have seen in one aspect of their teaching to another this is
not unreasonable to expect but some research findings (e.g. Wall & Hork
2006) do not bear this out.
7/29/2019 Petrie 2013 Exam Teaching Support
15/131
5 | P a g e
1.4A BELIEF ABOUT SUPPORT FOR TEACHERS OF EXAM PREPARATION
CLASSES
The situation of teacher training and development in relation to exam class
teaching is also poor. A review of the online archive of the Teacher Trainer
Journal (n.d.) found nothing that mentioned teaching exam classes. There are
however, currently two books which attempt to influence exam preparation
teaching methodology: May (1996) Exam Classes and Burgess & Head
(2005) how to Teach for Exams. Both provide useful perspectives on
preparing learners for aspects of exams, though the breadth and depth of their
coverage varies.
The counterpoint to this is the wealth of publications available to teachers and
learners from ESOL publishers. These typically focus on one exam in particular
and fall broadly into two categories: the course book and the practice test book.
The former often offers teachers and learners material designed to aid linguistic
development and skills development as part of a broader course, while the latter
generally offers only sample exam tasks and answer feedback, possibly with
some additional task analysis. The inference is that in learning about the
sample tasks given and why the specific answers are correct, the learners will
be able to draw wider conclusions about the target exam in general. There is
evidence to suggest otherwise, as examined in Chapter 2. It could be argued
that if the course book or the practice test book forms the locus of an exam
preparation class, it may also become the primary reference document in terms
of exam information and strategies for approaching exam tasks. This would
mean that any beliefs on the nature of language and on the learning process
held by course book stakeholders form the basis of the exam preparation
7/29/2019 Petrie 2013 Exam Teaching Support
16/131
6 | P a g e
course. Depending on what those beliefs are, this is not necessarily a problem.
Nevertheless, it would require the teacher to be able to make an effective
assessment of the strengths and weaknesses of the course book for their
teaching context and concurrently to have the pedagogical tools available to
do so.
The current depiction of support for exam class teaching has therefore led me
to a second guiding belief: The support available to teachers of exam
preparation classes is not sufficient to their needs.
1.5CONSTRAINTS OF THE DISSERTATION
It is not possible within the constraints of this dissertation to investigate these
beliefs with reference to all of the many English language exams available and
the analysis will therefore be confined to discussing three English language
exams: FCE, TOEFL iBT and IELTS.
These have been selected as they are widely considered to be the most popular
with language learners. It is difficult to determine the truth of this as the relevant
awarding bodies seem reluctant to publish candidate numbers. IELTS states
over 1.7 million tests were administered in 2011 (IELTS, n.d.); Cambridge
ESOL state that over 3.25 million people take their tests annually though this
does not state which tests were taken (University of Cambridge ESOL
Examinations, 2011). The Educational Testing Service website states over 27
million people have taken the TOEFL test (2012), though again it does not state
which versions of the test were taken nor does it state over which time period.
7/29/2019 Petrie 2013 Exam Teaching Support
17/131
7 | P a g e
1.6SUMMARY OF CHAPTER ONE
This chapter has set out the professional context and experiences that underpin
this investigation and has articulated two key beliefs that will guide the further
exploration of the subject area:
Teachers of exam preparation classes do not always have all the information
they need to know in order to be effective exam class teachers.
The support available to teachers of exam preparation classes is not sufficient
to their needs.
To investigate these beliefs more fully, chapter two of this dissertation will
provide an overview of exam class teaching, the key characteristics and
approaches identified to teaching exam classes. It will further examine the
research on the effectiveness of exam class teaching and discuss forms of
support available to exam class teachers identified from the literature.
Chapter three then re-articulates the beliefs stated above as research questions
that can be investigated with the design of an online questionnaire. It further
describes the research methodology adopted, after Nunans eight step process
(1992, p. 141), describes the conduct of a pilot study and discusses the biases
apparent in the research participants who were obtained as a random volunteer
sample.
Chapter four presents the results of the survey, while chapter five discusses
these results in the context of the five research questions outlined in section 3.2
and considers the implications of these findings and draws conclusions as to
the sufficiency of information and support for teachers of exam preparation
classes and discusses how that support might best be rendered in the future.
7/29/2019 Petrie 2013 Exam Teaching Support
18/131
8 | P a g e
CHAPTER TWO: AN OVERVIEW OF EXAM CLASS TEACHING
This chapter provides a definition of exam class teaching, describes its key
characteristics and approaches to exam class teaching identified in the
literature. It further examines the effectiveness of exam class teaching from the
perspective of washback and impact studies that contrast learner performance
in exam classes with alternative instruction modes. Finally it provides a list of
forms of support for exam class teachers derived from an examination of forms
of continuing professional development found in the literature.
2.1A DEFINITION OF EXAM CLASS TEACHING
It is difficult to characterise exam class teaching in an ELT context except in
opposition to the more common term General English, though this term is also
problematic as while it is a term in common currency, it is not one for which a
definition is readily available. Definitions can be found for Business English
(BE) (Ellis and Johnson, 1994), English for Academic Purposes (EAP)
(Hamp-Lyons, 2001), and English for Specific Purposes (ESP) (Dudley-Evans,
2001). There are also many labels and acronyms relating to the nature of
English, for example English as an International Language (EIL) or English as a
Lingua Franca (ELF), and a useful overview of these is provided by Erling
(2005), who also mentionsAhulus (1997) use of General English to describe
a global variety of English as opposed to British and American standard models,
though general English is not used in this way within this study.
Whether it has been clearly defined as so or not, general English has taken on
a meaning of English for everything else. The general English classroom
7/29/2019 Petrie 2013 Exam Teaching Support
19/131
9 | P a g e
recognises that learners enter with individual goals and experiences and
abilities with the language (Richards 2001) and tries to take all these into
account, but aims more broadly, to improve the language knowledge and
language skills generally.
Exam class teaching borrows from this but works towards a different purpose.
The activities and teaching techniques that work in general teaching will help
make a successful exams teacher (Burgess and Head, 2005, p. 1) but it is also
necessary to drive students towards the goal of passing their exams (ibid).
Exam class teaching is constrained by the target exam; the syllabus is defined
by it, as are the materials (ibid). The exam class teacher therefore needs to
know as much as possible about the format and content of the exam (ibid, p.
2) and to believe that exam preparation is a worthwhile exercise (ibid, p. 3).
2.2THE CHARACTERISTICS OF EXAM CLASS TEACHINGBurgess and Head (2005) describe a successful exams teacher as having
effective time management skills, a positive view of exams, extensive
knowledge of the target exam, giving pertinent and effective feedback, listening
and responding to learner worries and motivating learners (2005, p. 3). They
also discuss approaches to teaching reading, writing, grammar and vocabulary,
listening and speaking for exams. May (1996) structures his book Exam
Classes in a similar fashion though provides a range of practical activities
rather than discuss the components or characteristics of exam class teaching.
He does state that Exam techniques have to be built on a solid foundation of
language skills. Students cannot handle a difficult exam course without a
7/29/2019 Petrie 2013 Exam Teaching Support
20/131
10 | P a g e
thorough grounding in reading, writing, listening and speaking. (1996, p. 5),
which is a useful, but limited viewpoint.
There are similarities between these features of exam classes and aspects
identified by Zabawa (2001) and Wilson (2010) as desirable components of
exam preparation class course books. Though these are not based on
evidence from empirical research and are not designed with the specific aim of
suggesting desirable components of exam classes, it could be argued that if
these criteria are desirable in course books, they are desirable in courses and
consequently in teaching.
The justification and rationale that Zabawa uses for the inclusion of his criteria is
that the FCE textbook must be as similar as possible to the FCE examination
format (2001, p. 162). He extends this to discuss teaching language skills and
states that where sub-skills can be identified as being tested in the exam, these
should be focused on in the text book and by extension the course (2001, p.
163-165).
Wilson describes the exam, candidate needs and problems and general
implications for teachers before arriving at her set of criteria (2010, p. 221),
though she does not describe her information sources or how it was developed.
Wilson draws two main conclusions; firstly that the balance between underlying
skills or language work and exam strategies and practice varies significantly in
exam class teaching and teachers therefore need to consider student needs
during the delivery of a course. Secondly, that many course books assume a
lockstep approach is being taken to teaching, which may not be desirable
(2010, p. 231-232).
7/29/2019 Petrie 2013 Exam Teaching Support
21/131
11 | P a g e
From May (1996), Zabawa (2001), Burgess & Head (2005) and Wilson (2010),
the following key characteristics of exam class teaching can be determined:
exam task training / test taking techniques language development (grammar and vocabulary)
listening development
reading development
speaking development
test / exam practice
writing development
As cited from Burgess & Head (2005) previously, many of these aspects bear a
strong resemblance to the content of a general English course, though with an
exam class the focus of the development must be the likely content of the
exam. There is little point in developing the exam class students ability to write
a short story if the primary test task takes the form of a transactional letter.
2.3APPROACHES TO EXAM CLASS TEACHING.
2.3.1CONSTITUENTS OF AMETHODOLOGICALAPPROACH
It is not until the goals, objectives, and content of a language programhave been determined that decisions about methodology can be takenup in detail.
(Richards, 1990, p. 11)
The test is the curriculum.
(unnamed teacher quoted in Wall & Horak, 2006, p. 119)
It is perhaps unfair to suggest that all exam focused classes echo the sentiment
of the teacher quoted above, nevertheless this study has suggested that when
the goals and objectives of an exam course are discussed, those that feature
7/29/2019 Petrie 2013 Exam Teaching Support
22/131
12 | P a g e
most prominently are likely to be those featured in the target exam and that as
such, the exam is likely to determine the content also. The relationship
between an exam and the pedagogical events surrounding it is often described
as washback, which is more fully investigated in in section 2.4.
Richards describes methodology as the activities, tasks and learning
experiences selected by the teacher in order to achieve learning, and how these
are used within the teaching /learning process. (1990, p. 11). A methodology
reflects, he suggests, beliefs and attitudes towards language, learning,
participant roles and materials.
To an extent, the beliefs about language are also determined by the exam.
Richards gives, as an example of how to help determine these beliefs, the
analytical questions is reading viewed as a primarily top -down or bottom-
up process ? What does listening comprehension involve and what skills
does it depend on? (1990, p. 12). In the exam class it is the exam designers
whose beliefs and attitudes to language are paramount as these are
incorporated into the test. In this aspect of washback therefore, the belief about
what listening comprehension involves forms the basis of what aspects of
listening comprehension are tested. There is however, the potential for conflict
between the beliefs embedded into the target exams and the beliefs held by
teachers, learners and organisations.
If exam designers hold beliefs about learning, these are not apparent from the
exams. Rather the exams set the end goal or destination and teachers and
learners are free to decide how best to travel. Consequently startlingly different
methods might describe very similar levels of effectiveness. The Callan School,
7/29/2019 Petrie 2013 Exam Teaching Support
23/131
13 | P a g e
whose method is based on question, answer, repetition (Callan Method
Organisation, n.d.) claims that by the end of their learning process the student
will have mastered the 5621 most common words of the English language. The
student will now be thoroughly prepared for the Cambridge First Certificate
(F.C.E.) exam (ibid). Chia Suan Chong describes adopting a Dogme approach
with her IELTS class So my one-month IELTS class went the Dogme way and
feedback from my students was overwhelmingly positive. A couple said they
had never learnt so much in a month before. (Chong, 2011).
While it is likely that methodologies or approaches that share beliefs with those
held by exam designers and producers are liable to be more effective in exam
preparation classes than those which do not, it is worth returning to Richards for
a moment to remember that other variables may be at play: Studies of the
effectiveness of specific methods have had a hard time demonstrating that the
method itself, rather than other factors, such as the teachers enthusiasm or the
novelty of the new method, was the crucial variable (1990, p. 36).
The problems in identifying successful teaching practice, with the large number
of variables involved, may account for the relative paucity of articles related to
successful exam class teaching. It is easy to describe teaching practice and
reactions to teaching practice, as with Chong (2011), but it is not so easy to
claim that teaching practice has directly led to successful exam performance.
2.3.2OVERALL APPROACHES TO EXAM PREPARATION CLASSES
Senior provides a very general overview of the constituent parts of exam-driven
teaching (2010): confidence building, skills development, exam strategies and
test practice and provides examples of each. She also outlines some of the
problematic areas, that attempting to teach English largely through tests (ibid)
7/29/2019 Petrie 2013 Exam Teaching Support
24/131
14 | P a g e
seldom works well, that teaching to the test in terms of selecting course content
can be limiting and possibly impeding, and that it denies teachers the
opportunity to teach in dynamic and engaging ways (ibid).
Teaching for the test is also an issue that Thompson examines, though his
analysis is kinder as he suggests that when the test and (course) objectives
are in harmony, the effect can be very positive (2001, p. 11). His view, similar
to Zabawa (2001), is that the proper incorporation of the test into the course
curriculum is of prime importance and that tests might lead learners to focus on
areas they might otherwise neglect, for example essay structures. The pitfalls
are those of negative washback, he points out, as including a test either as an
end of course assessment technique or as a course goal can lead learners to
neglect or ignore aspects of their language development that are not included in
the test.
De Hinojosa (2011) takes a more practical and more detailed approach in
Humanising Language Teaching magazine (HLTM) that appears to be based on
her own experience working in a specific context and aimed more at novice
teachers working without the support of colleagues or their institutions. De
Hinojosas suggestions involve making sure classes are relatively homogenous
in their language ability, involving and communicating with parents, giving
homework daily, providing students with exam tips (a broad sample of which are
listed) and testing or assessing the students. De Hinojosas article suffers
slightly from not describing her context, which might provide some
understanding as to why she has made these choices or recommends this
approach. It also ignores alternative contexts in which other stakeholders may
have input on some of the processes she describes, for example the degree of
7/29/2019 Petrie 2013 Exam Teaching Support
25/131
15 | P a g e
contact with parents. The overall strategy would appear to be that of managing
expectations of stakeholders and in maximising test performance potential.
Development of learner ability is not discussed and the inference is that by
preparing learners for the exam development of ability will occur as a by-
product, or as positive washback. There is some debate on this point as will be
discussed in section 2.4.
An alternative approach is offered by Case, also in HLTM, who offers
suggestions on how to humanise your exam classes (2002). His key points
seem sensible; personalisation of content to the learners, build group rapport
and encourage peer collaboration, keeping content and activities relevant to the
exam. Case argues that by asking learners to purchase a self-study book and
work individually at home, language work conducted in class is more reactive to
learner needs. This might suggest a belief that prescribing the language
content of a course is inappropriate and that individualised tuition based on
learner needs is more important. Or it could reflect a belief as given in Burgess
and Head (2005, p. 64) that an exam preparation course is not a suitable
vehicle for language development, but should instead consolidate existing
knowledge.
Case, De Hinojosa, Thompson and Senior provide the only attempts at
describing an overall approach to successful exam class practice apparent in
the pedagogical literature, comprising as it does the small number of journals
aimed at helping teachers improve teaching practice rather than the more
detailed investigations of teaching, learning and language that can be found in
the academic literature. In brief, the Thompson and Senior articles provide not
so much an approach to teaching exam classes but more a list of pros and cons
7/29/2019 Petrie 2013 Exam Teaching Support
26/131
16 | P a g e
for courses where the test provides the main impetus. These are perhaps
considerations rather than prescriptions. Case similarly makes limited
suggestions for adapting existing procedure rather than a process in itself.
Where De Hinojosa attempts a more detailed description she succeeds in
describing aspects of exam class preparation that seems to lack the
consideration that Thompson and Senior urge.
2.4EFFECTIVENESS OF EXAM PREPARATION CLASSES
Contributions to academic research and understanding of the effects of
language tests and exams upon their wider environments, processes and
participants often do so by examining either impact or washback. Taylor (2005)
defines washback as the way a test affects classroom events in the time period
prior to the test being taken. Washback can be viewed either as positive
(having a beneficial effect on course content and conduct) or negative
(constraining teaching and learning) (ibid). Where washback relates to events
leading up to the test, impact describes the events that come afterwards, or the
consequences of tests (ibid, 154). Impact tends to be applied to a larger
context than the classroom; Taylor (ibid) sees it as the effect of the test on
individual life prospects, schools, educational systems and society as a whole.
While she points out there is some disparity of use of these terms amongst
researchers, this study follows the definitions given above.
Various studies have attempted to examine whether test preparation classes
lead to improvements in test scores: Hayes finds no significant gain (2003: i)
when contrasting IELTS preparation with an English for Academic Purposes
(EAP) course, a result repeated with Green (2007), who found a difference
7/29/2019 Petrie 2013 Exam Teaching Support
27/131
17 | P a g e
occurred for course length but not for course type, as might be expected given
the quasi-academic nature of the IELTS exam. Spratt (2005) cites additional
studies by Read and Hayes (2003), Andrews et al (2002), and Cheng (1998) as
reaching similar conclusions. Perrone (2010), in comparing a general English
(GE) course with a FCE course, also found no advantage in test scores for
those students taking the exam preparation class, though as Perrone does not
give pre-course data for either group, it is unclear whether the groups started
from the same intial position or whether one group started from a more
advanced position in terms of test scores, and consequently made less
progress in order to reach the same finishing point as the other group.
Perrones study also examined a total of 35 participants distributed between
four groups, highlighting a common problem in washback and impact research,
whether findings from such small-scale studies can be extrapolated into a larger
context.
Aside from the scope of such studies, other limitations lie in the specific nature
of the principal investigation focus that each study selects. Tsagari (2009)
identifies variables that affect the nature of washback (form, intensity,
specificity, length, intentionality, value) and that are affected by washback
(teachers, students, the subject, resources, materials, classroom conditions,
school management practices, test communication practices, the wider social
context) (ibid, p. 5). Her model of washback as it applies to stakeholders in the
FCE in Greek language schools emphasises that the nature of exam washback
is circuitous and interactive (ibid, p. 8). Studies that investigate single or
particular aspects of washback therefore run the risk of missing the influence
that other aspects may have on the process they seek to examine.
7/29/2019 Petrie 2013 Exam Teaching Support
28/131
18 | P a g e
Spratt examines 11 washback studies and categorises the investigation areas
of the studies as follows: curriculum, materials, teaching methods, feelings and
attitudes, learning. (2005, p. 8) and reviews the findings of the s tudies as
broken down into these areas. As mentioned above however, there are
limitations in this approach as choices made in regard to one aspect may well
affect the others. This inter-relationship is diagrammed in Figure 1 below (after
Spratt 2005).
FIGURE 1: THE INTER-RELATIONSHIP OF ASPECTS OF EXAM CLASS TEACHING (AFTER SPRATT 2005)
In Figure 1 exam class teaching is shown as a matrix in which a choice made in
one aspect will also affect all of the other aspects. Here, a choice made to alter
the curriculum may require an adjustment to teaching methods, the use of
different materials, may alter teacher and learner feelings and attitudes towards
the course, and may affect the learning process. Tsagari, for example, noted
that students in her study did not enjoy their classes and felt tired, disinterested
and bored (2009, p. 7); an attempt to alter such a situation does not only
7/29/2019 Petrie 2013 Exam Teaching Support
29/131
19 | P a g e
require changes to be made to learners feelings and attitudes. It may require
changes to be made in some or all of the other aspects to achieve the desired
result.
Given this interplay between aspects of exam class teaching, the limitations of
some washback and impact studies become more evident. A contrast between
an EAP class and an IELTS class, (e.g. Read and Hayes 2003), or a FCE class
and a GE class (e.g. Perrone 2010), is useful only if we can be sure that the
teaching methodologies and materials used are discernibly different. Having
measured the different proportions of classroom time spent on developing the
listening, speaking, writing and reading skills, Perrone concludes instruction in
the two courses appeared to differ (2010, p. 227), though he does not seek
to account for how or why this might be and as, at his third measurement point,
the GE class contained 29.1% of exam preparation (2010, p. 198) this might be
useful information to uncover.
This section has shown that there is little evidence to suggest that exam
preparation courses give learners a significant advantage when it comes to
exam performance. The reasons why this might be are unclear, as while
different studies have shed light on how exams influence different aspects of
exam classes, little research has been done on how the different aspects of
exam classes affect exam performance and the inter-relationship between
aspects of exam class teaching tends to obscure the usefulness of such
findings for the exam class teacher. This is an area that would benefit from
further research, though sadly it falls outside the purview of this dissertation.
7/29/2019 Petrie 2013 Exam Teaching Support
30/131
20 | P a g e
2.5FORMS OF SUPPORT FOR EXAM CLASS TEACHING
A review of the literature couched in the terms of teacher support reveals that
such support is best viewed through the prism of Continuing Professional
Development or CPD, a term which has become increasingly prevalent over
the course of 2012 (Wilden, 2012) and relates to the process of taking
responsibility for your own development as a teacher (ibid). This highlights the
differing natures of CPD and teacher support as such a view places the onus on
the individual who requires support to find it, rather than on other stakeholders,
e.g. employers, to make it available.
Writers on CPD have identified a number of sources of CPD relevant to ELT
which may also act as sources of support for exam preparation class teachers.
Johnson (2009) and Wilden (2012) suggest ways teachers can professionally
develop and Table 1 below shows a summary of their ideas.
Broadening teaching range (teaching new levels / trying new things)
Conferences (attending / presenting at)
Conversation (informally with colleagues / informally or formally with experts)
Courses (teaching courses / language courses)
Materials Creation
Membership of Professional Associations
Mentoring (being mentored / mentoring another)
Observation (being observed / peer observing)
Online Communities (social media groups and communities) Reading (teaching magazines / books / blogs, individual or group reading)
Research Projects (contributing to / conducting own)
Specialisation of teaching interests
Workshops (attending / giving)
Writing (magazine articles / blogs)
TABLE 1: A SUMMARY OF WAYS OF CONTINUING PROFESSIONAL DEVELOPMENT, AFTER JOHNSON (2009)AND WILDEN (2012)
7/29/2019 Petrie 2013 Exam Teaching Support
31/131
21 | P a g e
Not all of these are appropriate in a support context as they rely on the
individual to provide, rather than receive, support; for example mentoring
another or presenting at conferences.
Reading is a common form of both CPD and teacher support and there are a
number of different sources teachers can draw on. This dissertation has
already drawn on teaching blogs (Chong, 2011), professional magazines
(Senior, 2010) and academic journals (Wilson, 2010), as well as professional
development and methodology books (Burgess & Head, 2005; May, 1996). For
the exam class teacher there are also handbooks from exam providers that give
detailed descriptions of the exam and exam components, e.g. UCLES 2012.
There are also a number of online communities and forums that can provide
support, either via social media for the sharing of ideas and resources or via
communities created by publishers and exam providers, though these latter
may have an additional purpose beyond teacher support.
Teacher training and development is another source of support, for example
teaching courses, workshops and conferences as per Table 1 above. Some
organisations also offer in-service teacher training (INSETT) seminars (Waters
& Vilches 2012), and it is also possible to take part in online seminars or
webinars (Webinars, 2013).
The individuals in the teachers context can also provide support and
conversation with colleagues (as per Table 1) and support from senior staff, of
which mentoring and observation can form a part, are both common forms of
individual support.
7/29/2019 Petrie 2013 Exam Teaching Support
32/131
22 | P a g e
Finally, it is worth also mentioning course books and supplementary books as
probably nothing influences the content and nature of teaching and learning
more than the books and other teaching material used (Cunningsworth, 1995,
p. v), though whether the course book is viewed as a help or a hindrance is
something that will be investigated during the course of the research component
of this study.
Thus forms of support applicable to exam class teaching can be given as per
Table 2 below:
Reading
Teacher blogs and websitesProfessional magazines and journalsProfessional development and methodology booksExam provider handbooks and materials
OnlineSocial media groups and forumsPublisher websites and forumsExam provider websites and forums
TrainingTeacher development coursesExternally provided seminars, webinars and conferencesIn-house teacher development programme(s) (INSETT)
IndividualsSharing ideas with colleaguesSupport from senior staff
MaterialsCourse booksSupplementary resource books
TABLE 2: FORMS OF EXAM TEACHING SUPPORT
2.6SUMMARY OF CHAPTER TWO
Exam class teaching shares many characteristics with general English teaching
but aims towards the goal of successful exam performance, which influences
7/29/2019 Petrie 2013 Exam Teaching Support
33/131
23 | P a g e
choices made in regard to curriculum, materials, methodology and attitudes to
the course.
Methodologies and approaches to exam class teaching may vary considerably,
but successful exam teaching practice remains difficult to identify and there is
little evidence to suggest that exam preparation classes significantly improve
learner exam performance, which suggests there is a need to support exam
class teachers and a number of potential support modes were identified.
The aspects of exam class teaching and forms of support for exam class
teaching that were identified in this chapter will inform the design and content of
the research instrument as described in the next chapter.
7/29/2019 Petrie 2013 Exam Teaching Support
34/131
24 | P a g e
CHAPTER THREE: RESEARCH METHODOLOGY
This chapter details the design and administration of a research instrument
created to investigate the degree of support available to teachers of exam
preparation classes. It sets out a rationale for and clarifies the aims of the
research, provides a methodology for the design and administration of the
research instrument, describes the implementation of a pilot study to trial the
research implement, details the design of the research instrument and provides
an overview of the research participants.
3.1RESEARCH MATERIALS AND PROCEDURES
The research materials comprised an online questionnaire in three parts and
containing a total of twenty questions. Each part focused on a separate aspect
and the content of each section is described in section 3.5. In brief however,
part one contained six questions and focused on participants background and
professional experience, part two contained seven questions and focused on
participants experience with and attitude towards teaching exam classes, part
three contained seven questions and focused on participants experience with
and attitude towards the support they have received or would like to receive in
connection with their exam class teaching.
The questionnaire was designed using the form and spread sheet tools in
Google Docs. It was published online and publicised through various websites
and social media networks along with a request for volunteer participants. The
questionnaires were completed by participants in their own time and without
influence or interference from the researcher. The data were automatically
7/29/2019 Petrie 2013 Exam Teaching Support
35/131
25 | P a g e
inserted into a spread sheet by the Google Docs programme for later retrieval
and analysis.
3.2RESEARCH AIMS
In Chapter One, the following beliefs about exam preparation classes were
articulated:
Teachers of exam preparation classes do not always have all the information
they need to know in order to be effective exam class teachers.
The support available to teachers of exam preparation classes is not sufficientto their needs.
To more fully investigate these beliefs, it is necessary to rephrase them in light
of the findings of the previous chapters as questions that can be answered with
the design of a suitable research instrument. The aims of the research are
therefore to answer the following questions:
A. Do teachers feel as though they need support in their exam class teaching?
B. Do teachers need support in their exam class teaching?
C. What forms of support are used by teachers of exam classes?
D. What are their feelings towards these forms of support?
E. What changes to this support or what additional support would teachers of exam
preparation classes like to see in the future?
To answer these research questions it is necessary to employ a suitable
research methodology.
3.3RESEARCH METHODOLOGY
This section details the methodology used in carrying out the research following
the eight step procedure described by Nunan (1992, p. 141).
7/29/2019 Petrie 2013 Exam Teaching Support
36/131
26 | P a g e
3.3.1DEFINE OBJECTIVES
The objectives of this research have already been stated in the five research
questions given above.
3.3.2IDENTIFY TARGET POPULATION
Oppenheim defines population as those who fall into the category of concern
(1992, p. 38) and as such the target population of this research can be said to
consist of all those who prepare classes for English language exams such as,
but not limited to FCE, IELTS and TOEFL iBT.
3.3.3LITERATURE REVIEW
As more thoroughly detailed in chapter 2, successful exam teaching practice is
difficult to identify and there is some doubt as to the efficacy of exam
preparation classes. Nevertheless, common aspects of exam preparation
classes were identified, as were forms of support commonly available to exam
class teachers. These are incorporated into the questionnaire design as
described below in section 3.5.
3.3.4DETERMINE SAMPLE
Oppenheim defines sample as a smaller group, usually but not always a
representative one, within a population (ibid) and states that in principle every
member of the population should have a statistically equal chance of being
selected. Since this survey was published and distributed online, the sample
may not be completely representative of the population as it precludes those
without internet access. Equally, while the sample might be best described as a
random sample (Nunan, 1992, p. 142), the sample was not selected by the
researcher rather volunteers were asked for and therefore the participants
made the determination as to whether the survey applied to them. The sample
7/29/2019 Petrie 2013 Exam Teaching Support
37/131
27 | P a g e
may therefore not reflect the target population. To determine biases in the
sample a number of questions relating to professional qualifications and
experience were included.
3.3.5IDENTIFY SURVEY INSTRUMENTS
In order to answer the research questions, data was collected through the
distribution of a short survey. The terms survey and questionnaire are here
used interchangeably and with Nunans description of purpose attaching to
both, that they seek to obtain a snapshot of conditions, attitudes, and/or events
at a single point in time (1992, p. 140). Oppenheim (1992) differentiates
between descriptive and analytic surveys. An analytic survey attempts to
describe causality and to investigate the relationships between variables,
whereas descriptive surveys tell us how many members of a population have a
certain opinion or characteristic the job of such surveys is essentially fact-
finding and descriptive (ibid, p. 12). The research instrument employed in this
dissertation can therefore be described as a descriptive survey.
3.3.6DESIGN SURVEY PROCEDURES
The questionnaire was created using Google Docs. Google Docs is a suite of
productivity software made available online to subscribers by Google and
includes word processing, presentation, spread sheet and form design tools.
The ability to quickly and easily create a form for the collection of data via a
simple web page with a variety of question types and an unlimited number of
questions led to the selection of Google Docs for the design and dissemination
of the questionnaire.
Once created, the form was published online in two locations; firstly via Google
docs itself and secondly via the researchers professional blog. A hyperlink to
7/29/2019 Petrie 2013 Exam Teaching Support
38/131
28 | P a g e
the form was then publicised in and amongst various groups and forums on the
social networking sites Facebook, Twitter and Linked In. An email request
containing a hyperlink to the form was also sent to the researchers professional
colleagues and to schools within the International House World Organisation
network, with the additional request that recipients themselves distribute the
email to colleagues and professional acquaintances.
3.3.7IDENTIFY ANALYTICAL PROCEDURES
One of the features of the Google Docs form and spread sheet tool is that
response data is automatically collated into a spread sheet. Google Docs also
includes a Show summary of responses tool that gives a visual representation
of the survey data, though this is more useful with closed class questions,
where possible responses are selected by the researcher, than with open class
questions, where the respondent is free to make an answer of their own
devising (Nunan, 1992, p. 143).
As mentioned previously, the survey is descriptive, it counts a representative
sample and then makes inferences about the population as a whole
(Oppenheim, 1992, p. 12). As the survey makes use of both closed questions
and open questions, the presentation of the data varies somewhat. The data
from the closed questions is presented with reference to the numerical values
achieved. Nunan suggests that data from open questions is much more difficult
to quantify but can be synthesised to reveal possible patterns (1992, p. 146)
and this approach is adopted for the qualitative data produced by the survey.
3.3.8DETERMINE REPORTING PROCEDURES
The findings of the survey are reported and discussed in the context of the five
research questions listed above. Each question is discussed in the light of the
7/29/2019 Petrie 2013 Exam Teaching Support
39/131
29 | P a g e
data generated by the survey and a conclusion is reached. These discussions
are presented in chapter five.
3.4CONDUCTING A PILOT STUDY
The need to conduct a pilot study is stressed by a number of authors. Nunan
states quite plainly that it is imperative to pilot any questionnaire (1992, p.
145), while Oppenheim also emphasises the need to make sure a questionnaire
is fit for purpose.
A pilot study of the questionnaire was therefore conducted. Ten teachers at two
language schools in Coimbra, Portugal were approached and asked to
complete the questionnaire and a further set of feedback questions on the
questionnaire. The teachers were selected as they were known to all have
experience of teaching exam classes and to have a range of professional
experience one participant had two years teaching experience, another had
twenty five. Primarily however, they were selected as a convenience sample as
they are all colleagues of the researcher, could be contacted quickly and easily
persuaded to take part. The downside to using them as a convenience sample
is that their responses were fairly homogenous and represented a fairly similar
attitude to exam class teaching and experience of professional support.
Pilot study participants were emailed a hyperlink to the online questionnaire,
asked to complete it, answer the following questions and email the responses
back to the researcher:
1) How long did it take you to complete?2) Was the wording of the questions clear and easy to follow? If not, which
questions were unclear?
3) Was the supplementary information clear, easy to follow and useful? If not,which questions or sections did this apply to?
7/29/2019 Petrie 2013 Exam Teaching Support
40/131
30 | P a g e
4) Were there any options you would have liked included in questions but whichwere not? If so, which question and what was the missing option?
5) Were there any questions you felt should be excluded from the survey? Why?6) Were there any questions you felt should have been asked but weren't? If so,
what are they?
7) Is there anything else you think I should know about the questionnaire?
Feedback to the pilot study was positive as participants did not report problems
with the questions or rubrics, or feel that questions should be added to or
excluded from the questionnaire, though two of the questions were identified as
being overly similar to each other and these were combined into one question
for the final version.
Of more importance was that in considering the responses to the pilot study it
became clear that not all the questions served the purpose of providing
information to answer the five research questions articulated in section 3.2. The
questionnaire was therefore revised and resent to the pilot group for further
consideration.
It may be that in asking the same participants to report on the questionnaire a
second time, the degree of imposition was increased and participants gave the
survey less consideration than on the first occasion, not all respondents
completed the survey the second time and there were four abstentions. The
feedback was also more perfunctory, though similar in content to the first set of
results. In hindsight a secondary trialling of the survey should have enlisted a
separate set of respondents; unfortunately recruitment of a new set of
participants was not possible in the time frame for this stage of the research.
The responses from the second set of feedback provided more relevant data
7/29/2019 Petrie 2013 Exam Teaching Support
41/131
31 | P a g e
than previously and so the decision was taken to publish the survey and start
accepting responses.
3.5RESEARCH DESIGN
3.5.1QUESTION TYPES AND TECHNOLOGICALAFFORDANCES
The questions comprised a mixture of closed class questions allowing for
numerical data to identify trends and open class questions to allow the
respondents a chance to offer a personal response from which insights into the
topic may be gathered (Oppenheim, 1992, p. 112-115).
Nunan identifies issues with question types and question wording, in particular
the need to avoid revealing the researchers own attitude to the question by
virtue of using leading language, and specifies that questions should be simple
and unique; that the language used in the question should not be confusing and
no question should ask about more than one thing at a time (1992, p. 143).
Nunan also lists a range of closed question types (ibid, p. 144), which seem
designed for use with a paper-based questionnaire, he does not describe the
form of an open class question perhaps thinking it self-evident, though
Oppenheim states they are usually followed by an amount of space (1992, p.
112). In this study the questionnaire is to be designed using the Google Docs
form tool and the question types available using this tool, the class of question
and the applicability of the options to question types as per Nunan (1992, p.
144) are given in Table 3 below.
7/29/2019 Petrie 2013 Exam Teaching Support
42/131
32 | P a g e
Google DocsQuestion Type
DescriptionQuestion
Class
Question Typeapplicabilityafter Nunan
TextShort Answer
(Sentence length)Open not applicable
Paragraph Text Longer Answer(Paragraph Length)
Open not applicable
Multiple ChoiceSelection of one
answer from manyoptions
Closedcategory, quantity
/ frequency
CheckboxesSelection of one ormore answers from
many optionsClosed list
Choose from a listSelection of one
answer from manyoptions
Closedcategory, quantity
/ frequency
Scale
Assign a value inresponse to a singlequestion(values canbe labelled by the
researcher, between3 and 10 values
available)
Closedranking, scale,
quantity /frequency
Grid
Assign values tomultiple options (amaximum of five
values can bedetermined by theresearcher as
column headings.An unlimited number
of options can beadded as rows)
Closed Grid
TABLE 3: COMPARISON OF QUESTION TYPES AVAILABLE IN GOOGLE DOCS TO QUESTION TYPES GIVEN INNUNAN (1992, P. 144)
The questionnaire was divided into three sections. The first section, for reasons
described in section 3.3.4 above, asked about professional biographical data.
The second section asked about exam class teaching and participants attitudes
towards it. The third section asked about experience of support for exam class
teaching and any future support they think important.
7/29/2019 Petrie 2013 Exam Teaching Support
43/131
33 | P a g e
3.5.2DESIGN OF SECTION 1: PROFESSIONAL BIOGRAPHY
As mentioned in section 3.3.4 one of the difficulties in using an online survey
tool is in achieving a balanced sample. The questions in section 1 attempt to
spot any biases in the sample. For example question 2 and 3 asked about
professional qualifications and length of teaching experience. If, for example,
all respondents report possessing Masters degrees and over fifteen years
experience, the results of the survey would not adequately reflect the target
population as one might expect more highly qualified and highly experienced
teachers to need or want less support in their exam class teaching.
The biographical questions were given as follows:
1. What is your current teaching environment?
2. What is the highest rated teaching qualification you possess?
3. How many years teaching experience do you have?
4. How many years experience teaching exam preparation classes do you have?
5. How much time do you spend teaching per week?
6. What type of classes do you currently teach?
3.5.3DESIGN OF SECTION2:EXAM CLASS TEACHING
This section of the questionnaire was designed to provide data to answer the
following two research questions:
A. Do teachers feel as though they need support in their exam class teaching?
B. Do teachers need support in their exam class teaching?
The questions that were asked in this section were:
7. Which exams have you prepared classes for?
8. How confident are you in your ability to prepare learners for these exams?
7/29/2019 Petrie 2013 Exam Teaching Support
44/131
34 | P a g e
9. Which of the following aspects do you include in your exam preparation
classes?
10.What proportion of class time do you spend on those aspects identified in the
last question?
11.Do your current exam preparation courses use a course book?
12.Do you like teaching exam preparation classes?
13.Why? / Why not?
Questions 10, 12, and 13 were rendered as open class questions with space
provided for personalised answers. Questions 7 and 9 were rendered in the
checklist format, question 11 as a two option choice and question 8 as a grid
where participants assigned a value relating to the level of support they felt they
would need for each exam listed.
Questions 8, 12 and 13 were asked in support of research question A.
Question 8 directly asks respondents to specify the degree of support they
would need in preparing classes for specific exams, while questions 12 and 13
allow respondents to record their attitude to teaching exam classes.
Questions 7, 9, 10, and 11 were asked in support of research question B. They
attempt to check the effectiveness of respondents exam preparation class
teaching by asking respondents to describe habits and procedures in their
teaching.
3.5.3DESIGN OF SECTION 3:PROFESSIONAL SUPPORT
This section of the questionnaire was designed to provide data to answer the
following research questions:
C. What forms of support are used by teachers of exam classes?
D. What are their feelings towards these forms of support?
7/29/2019 Petrie 2013 Exam Teaching Support
45/131
35 | P a g e
E. What changes to this support or what additional support would teachers of exam
preparation classes like to see in the future?
The questions that were asked in this section were:
14.Which of the following forms of support for exam class teaching do you use
now or have you used in the past?
15.Given the opportunity, which of the following forms of support for exam class
teaching do you prefer to use?
16.How frequently do you refer to or use these forms of support?
17.How would you rate the forms of support that you use or have used in the past?
18.Where you have indicated a resource mentioned above as being satisfactory orpoor, please could you indicate how you would like to see it improved?
19.Is there any additional support you feel you need to improve your exam class
teaching?
20.If there was one thing you could change about your exam class teaching, what
would it be?
Questions 14 and 15 were rendered as checklist style questions, allowing
respondents to select as many of the options as applicable. Questions 16 and
17 were rendered as grid style questions requiring respondents to assign a
value to the range of exams listed, being frequency of use for question 16 and a
rating from poor to excellent for question 17. Questions 18, 19 and 20 were
rendered as open class questions requiring the respondents to provide their
own answers.
Questions 14 and 16 provide data in answer to research question C as the
responses describe past and present habits of teachers in relation to the
support they use or have access to.
7/29/2019 Petrie 2013 Exam Teaching Support
46/131
36 | P a g e
Questions 15 and 17 provide data in answer to research question D as
respondents are asked in question 17 to allocate a rating to the forms of support
described and are also asked in question 15 which forms of support they would
like to use, given the opportunity. This latter question acknowledges that not all
teachers have access to all the forms of support mentioned and may also give
an indication as to the future disposition of exam preparation class support,
providing data in answer to research question E.
However, questions 18, 19, and 20 are designed to provide more significant
data in answer to research question E. These questions were rendered as
open class type questions to attempt to remove researcher bias in pre-selecting
the responses and to allow participants to reflect on their own situation, sharing
these insights with the researcher.
3.6RESEARCH PARTICIPANTSThe survey attracted a total of 78 responses and a brief profile of the
participants follows in this section. No restrictions were placed on who was
eligible to take part, though the initial rubric included the phrase If you teach
exam classes or have taught them in the past, then your input would be
gratefully received. This may have meant that some potential participants
decided they were ineligible to take part.
Additionally, none of the questions in the survey were made mandatory, which
meant that participants could choose to skip some questions and that
consequently, not all of the figures given below add up to 100%.
The majority of participants, 83%, worked in a non-native speaker environment
while 14% stated they worked in an English-speaking country.
7/29/2019 Petrie 2013 Exam Teaching Support
47/131
37 | P a g e
A breakdown of participants experience and highest level of qualification is
given below in Table 4.
NotStated
1 - 2years
3 - 5years
6 - 10years
10 - 15years
more than15 years
GrandTotal
Not Stated 2 1 3
Bachelor's degree 1 2 1 7 11
Certificate in ELT 1 6 7 2 3 19
Diploma in ELT 1 5 6 13 25
Doctorate 1 1
Master's degree 2 4 2 9 17
Other 1 2 3
Grand Total 2 1 10 19 14 33 79
TABLE 4: SURVEY PARTICIPANTS EXPERIENCE AND HIGHEST LEVEL OF QUALIFICATION
In hindsight, the question as to what constitutes the highest level of qualification
is open to interpretation. The UK Office of the Qualifications and Examinations
Regulator (Ofqual) publish a comparison chart, within which, the hierarchy of
qualifications rendered in the survey would fall as given in Table 5 below:
Level Type of Qualification
8 Doctoral degrees
7
Masters DegreesPGCE (M) - Post Graduate Certificate in Education (Masterslevel)Diploma in ELT (DELTA Cambridge ESOL)
6Bachelors DegreesPGCE (H) Post Graduate Certificate in Education (Honourslevel)
5 Certificate in ELT (CELTA Cambridge ESOL)
TABLE 5: ELT QUALIFICATIONS IN THE QUALIFICATIONS AND CREDIT FRAMEWORK / NATIONALQUALIFICATIONS FRAMEWORK FOR ENGLAND AND NORTHERN IRELAND, ADAPTED FROM OFQUAL (2011).
As Diplomas in ELT, Masters Degrees and some PGCE certificates all qualify
at level 7, it is difficult to say which of these might be considered highest rated.
Nevertheless, many participants are highly qualified and highly experienced
7/29/2019 Petrie 2013 Exam Teaching Support
48/131
38 | P a g e
with 54% possessing either a Diploma in ELT or a Masters degree and with
60% of participants having over ten years of teaching experience.
Furthermore, 56% of respondents reported having five years or more
experience of teaching exam classes but only 64% reported currently teaching
exam classes.
The degree of experience that participants report may well affect their
responses to the survey and as discussed in section 3.3.4 of this chapter, may
not be a representative sample of the target population, though without full
knowledge of the characteristics of the target population this is also difficult to
determine. It may be that exam preparation classes are typically taught by
teachers with more experience and qualifications and that the respondents to
the survey are a representative sample. This data however, obviously has
implications for the discussion of the results that will occur in chapter five.
3.7SUMMARY OF CHAPTER THREE
This chapter has described the design of an online questionnaire to determine
the answer to answer five research questions, as outlined in section 3.2. It has
followed an eight step research methodology procedure as outlined by Nunan
(1992, p. 141).
The objectives of the research have been defined, a target population identified,
and the difficulties associated in determining a sample for the survey, which
might best be described as a random sample of volunteers, discussed.
This chapter has identified the research instrument as being a descriptive
survey (Oppenheim, 1992, p. 12) created in and distributed via the Google Docs
7/29/2019 Petrie 2013 Exam Teaching Support
49/131
39 | P a g e
platform. It detailed the types of questions that were available within the Google
Docs platform and how these relate to survey question types identified by
Nunan (1992, p. 144), presenting a brief rationale for the structure of the survey
and the inclusion of specific survey questions, relating these to the stated
research questions. It described the initial piloting of the questionnaire and
subsequent revisions that were made to it. It has outlined how the data is to be
presented and discussed.
Finally, as the sample was identified as being a random volunteer sample, this
chapter presented an overview of the 78 respondents to the survey, in particular
looking at their length of service and degree of professional qualification. It
found that respondents were generally highly experienced and highly qualified,
and discussed whether this might be a representative sample or not before
concluding that the implications of this would need further examination in the
context of discussing the results in chapter five.
7/29/2019 Petrie 2013 Exam Teaching Support
50/131
40 | P a g e
CHAPTER FOUR: PRESENTATION OF RESULTS
This chapter presents the results from the second and third parts of the online
questionnaire described in chapter three, which asks about participants
experience of exam class teaching (section 4.1) and experience of support for
exam class teaching (section 4.2)
4.1 SURVEY RESULTS PART TWO EXAM CLASS TEACHING
4.1.1SURVEYQUESTION 7: WHICH EXAMS HAVE YOU PREPARED LEARNERS FOR?
The highest rated answers on this question were FCE with 88% and CAE with
73%. Exams produced by Cambridge ESOL scored much higher than other
exam providers though this may indicate a geographical bias in the survey
sample. Figure 2 below shows the count of responses for each option.
FIGURE 2: COUNT OF RESPONSES FOR EXAMS TAUGHT BY RESPONDENTS
0
10
20
30
40
50
60
70
7/29/2019 Petrie 2013 Exam Teaching Support
51/131
41 | P a g e
Respondents
top related