Chapter 8.ASSESSMENT.speaking

7/29/2019 Chapter 8.ASSESSMENT.speaking

http://slidepdf.com/reader/full/chapter-8assessmentspeaking 1/15

Chapter 8: Assessment of Speaking Skills

1

Chapter 8: ASSESSMENT OF SPEAKING SKILLS

8.1 Preamble

8.2 Evaluating speaking

8.3 What to assess?

8.4 How to assess speaking?

8.5 Using rubrics to assess presentations

8.6 Assessing speaking in an interview

8.7 Structured speaking tasks

Summary

References

Upon completion of this chapter, you should be able to:

Explain what to assess

Discuss how to assess speaking skills

Explain how to use rubrics to assess speaking skills Describe to use interviews to assess speaking

Chapter 1: Introduction to Listening

Chapter 2: Teaching Listening

Chapter 3: Listening Activities

Chapter 4: Assessing Listening Skills

Chapter 5: Introduction to Speaking

Chapter 6: Teaching Speaking

Chapter 7: Speaking Activities

Chapter 8: Assessing Speaking Skills




2

In Chapter 6 and 7, we discussed several methods and activities for teaching speaking in the

ESL classroom. In this chapter, we will examine how to assess speaking in the classroom.

Among the issues that will be addressed are as follows:

Is accuracy of pronunciation and grammar important?

Is each student expected to speak for about the same amount of time?

Is it acceptable if a speaker uses many long pauses and repetitions?

If a speaker’s contribution to a discussion is off topic, does it matter?

As the above questions illustrate, the types of criteria we use to assess a speaker’s oral

performance during a classroom activity will depend on which kind of talk we are talking

about and the kind of classroom activity we are using. For example, a speaking activity that

requires talk as performance (e.g., a mini-lecture) would require very different assessment

criteria. These might include: Clarity of presentation: i.e., the extent to which the speaker

organises information in an easily comprehensible order; use of discourse markers, repetition,

and stress to emphasize important points and to make the lecture structure more salient to thelisteners.

Assessment of speaking skills often lags far behind. Most teachers would accept that ‘if you

want to encourage oral ability, then test oral ability’ (Hughes, 1989:44). But the problems of

testing oral ability make teachers either reluctant to take it on or lacking in any confidence in

the validity of their assessments. Such problems include: the practical problem of finding the

time, the facilities and the personnel for testing oral ability; the problem of designing

productive and relevant speaking tasks; and the problem of being consistent (on different

occasions, with different examiners and between different assessors). Another problem, is

deciding which criteria to use in making an assessment. Different speaking activities such as

conversations, group discussions, and speeches make different types of demands on learners.

They require different kinds and levels of preparation and support, and different criteria must

be used to assess how well students carry them out. Some teachers argue that during the

speaking assessments, the focus should be on fluency and not accuracy. “ We just observe,

watch and listen to students and write down mistakes they made” .

8.1 PREAMBLE

8.2 Evaluating Speaking

Students who are actively involved in all speaking tasks and only use English in

class will get the highest score of three points for each class. Students who are

absent, inactive in speaking activities and use their native language will not get

the highest score for the day.




3

Read this comment by Fernando Diez, teaching English to Spanish students:

If a student can speak in English, up to some point you can say that person manages

communicating in that language. I have worked with a lot of students helping them speak

English and have realised the following:

It is important to take into account when evaluating speaking: the message the

learner conveys in English, in other words, the communication which is held among

two people. Don't get me wrong with what follows: "forget" about accuracy of

grammar. That person is trying to tell you something, and you are trying to say

something, he or she is trying to understand your ideas, your messages.

Honestly most of my class-time is devoted to speaking, to connect people through

communication. The classes are entirely in English: the classroom is a special place,

say, an environment where English is the only language among us.

I "don't understand Spanish": it's a funny situation in which I act out as though I not

understand Spanish. And it's their business to solve the problem of communicating

with me. I make faces when "I don't understand".

A few times my new students have asked me what country I'm from. Whether I'm

Spanish like most of them. They are already in the classroom when I come in, waiting

for me.

I don't care if the students have no idea of English. It's good for them to listen in

English. I use miming, eye-contact, the blackboard, drawings, examples, humour and

fun. Little by little, their grammar improves and their vocabulary expands and begin

to utter complete sentences, even to maintain conversations.

[source: Fernando M Díez, English Teacher in Granada, Spain]

http://fernandoexperiences.blogspot.com

Pronunciation, vocabulary, accuracy, communication, interaction and fluency are all markers

of a student’s overall speaking a bilities.

Pronunciation is a basic quality of language learning. Though most second

language learners will never have the pronunciation of a native speaker, poor

8.3 What to Assess?

http://fernandoexperiences.blogspot.com/






4

pronunciation can obscure communication and prevent an ESL student from making

his meaning known. When evaluating the pronunciation of your students, listen for

clearly articulated words, appropriate pronunciations of unusual spellings, and

assimilation and contractions in suitable places. Also listen for intonation. Are

students using the correct inflection for the types of sentences they are saying? Do

they know that the inflection of a question is different from that of a statement? Listen

for these pronunciation skills and determine into which level your student falls.

Vocabulary: After noting your students’ pronunciation levels, move to vocabulary.

Vocabulary comprehension and vocabulary production are always two separate banks

of words in the mind of a speaker, native as well as second language. You should

encourage your students to have a large production vocabulary and an even

larger recognition vocabulary. For this reason it is helpful to evaluate your students

on the level of vocabulary they are able to produce. Are they using the specific

vocabulary you have instructed them in this semester? Are they using vocabularyappropriate to the contexts in which they are speaking? Listen for the level of

vocabulary your students are able to produce without prompting and then decide how

well they are performing in this area.

Accuracy: Grammar has always been and forever will be an important issue in

foreign language study. Writing sentences correctly on a test, though, is not the same

as accurate spoken grammar. As your students speak, listen for the grammatical

structures and tools you have taught them. Are they able to use multiple tenses? Do

they have agreement? Is word order correct in the sentence? All these and more are

important grammatical issues, and an effective speaker will successfully include them

in his or her language.

Communication: A student may struggle with grammar and pronunciation, but howcreative is she when communicating with the language she knows? Assessing

communication in your students means looking at their creative use of the language

they do know to make their points understood. A student with a low level of

vocabulary and grammar may have excellent communication skills if she is able to

make you understand her, whereas an advanced student who is tied to manufactured

dialogues may not be able to be expressive with language and would therefore have

low communication skills. Don’t let a lack of language skill keep your students from

expressing themselves. The more creative they can be with language and the more

unique ways they can express themselves, the better their overall communicationskills will be.

Interaction: Being able to say what you mean with a foreign language is one thing,

being able to interact with others is another. Ask your students questions. Observe

how they speak to one another. Are they able to understand and answer questions?

Can they answer you when you ask them questions? Do they give appropriate

responses in a conversation? All these are elements of interaction and are necessary

for clear and effective communication in English. A student with effective interaction

skills will be able to answer questions and follow along with a conversation

happening around him. Great oratory skills will not get anyone very far if he or she




5

cannot listen to other people and respond appropriately. Encourage your students to

listen as they speak and have appropriate responses to others in the

conversation.

Fluency: Fluency may be the easiest quality to judge in your students’ speaking.

How comfortable are they when they speak? How easily do the words come out? Are

there great pauses and gaps in the student’s speaking? If there are then your student is

struggling with fluency. Fluency does not improve at the same rate as other language

skills. You can have excellent grammar and still fail to be fluent. You want your

students to be at ease when they speak to you or other English speakers. Fluency is a

judgment of this ease of communication and is an important criterion when evaluating

speaking.

Students may excel in one and struggle in another, and not necessarily the ones you might

think. Help your student understand these qualities of effective speakers. Let your studentsknow that you will be listening for these qualities when you evaluate their progress and

encourage them to improve their English in these areas. Also, listen to them both when they

talk to you and when they talk with other students. They should be able to speak well with

one another. After all, most of the English they will use in the future will be with other non-

native speakers. Finally, remember that a true evaluation will take into consideration more

than just the oral interview on the final exam. Listen to your students throughout the

semester. Note how they improve in these areas. Encourage them as speakers and learners,

and you are sure to reap the benefits, too.

Teachers are often asked to evaluate learner progress during courses, maybe by preparing

progress tests. It can seem straightforward enough to test grammar or vocabulary with pen

and paper tests – but if our students’ work includes speaking – then it also seems necessary to

assess their speaking skills. Teachers often feel unsure as to how they could do this. Here are

some ideas.

8.4 How to Assess?

a) Do you agree with the statement ‘forget about accuracy of grammar when

assessing speaking skills’? Explain

b) What do you focus on when assessing ‘pronunciation’ and ‘vocabulary’?

c) Explain what is meant by ‘interaction’ and ‘fluency’ when assessing speaking

skills of students.




6

Too many students!

A frequent problem for teachers is when there are so many learners in one class that it seems

to make it unrealistic to assess speaking. With a list of criteria (such as those above) it now

becomes considerably more straightforward to assess even a large group. Explain to your

class what you will be doing, then, the next three or four times you set speaking tasks (i.e.

where learners work in pairs or groups), walk around class with a list of names, listening in to

various groups and noting successes, keeping track of individual 'can do’s'. Extend your

assessment over a few lessons; keep listening and adjusting your evaluation over a variety of

tasks.

Speaking tasksWhat are possible speaking tasks for assessment? Well, almost anything you do in normal

class work – e.g. narrating a picture story; role-plays; pair work information gap exchanges;

discussions etc. If you have a smaller class and enough time then a “three learners with one

teacher” activity is a very good way to assess, i.e. setting a task that gets the three learners tointeract together while you watch and evaluate.

Self AssessmentAlthough fear of bad marks can sometimes be motivating, it’s surprising to find the amount

of power that students feel when assessing themselves. It can be a real awareness-raising

activity. Distribute a list of criteria and ask students to first write a short line comparing

themselves against each criterion (in English or in their own language) – a reflective view

rather than just a 'yes' or 'no'. Encourage 'guilt-free' honest reflection. After the writing stage,

learners can meet up in small groups and talk through their thoughts, explaining why they

wrote what they did.

Structured interview

Unstructured interview

Structured role-play

Structured role-play (information-gap)

Oral presentations

Retelling a story

Picture-cued story telling

Translation of an extended prose

Most teachers will be familiar with the concept of grading with a rubric, a table with

different criteria and a grading scale. If you have never created a rubric before, it’s really

quite easy. Simply choose the criteria on which you will grade students and list them along

the left side of the page. Then create an even number of columns along the top of the page.

Four is the easiest to start with if this is your first rubric. These columns will represent

potential skill levels of your students. For each criterion, define what level of the ability a

8.5 Assessing Speaking Using a Scale




7

student at each of the four levels would exhibit. For example, the most straightforward way to

label the boxes on the rubric would be, “ Meets expectations high, meets expectations low,

slightly underperforms, does not meet expectations.” The more rubrics you make, the more

detailed you can be in your descriptions. Then, as you evaluate each student, determine at

which level he or she is performing. Take the average level among the criteria and you have

an objective grade with suggestions for areas in which your student can improve.

Some scales to assess ‘speaking’ may use a more detailed ‘scale’. While students make their

presentations, the teacher can use a scale to assess their speaking skills. The scale could be in

the form a ‘rubric’ with ‘descriptors. Refer to the example of a scale given below which has a

5-point scale – 1 to 5. The scale measures the following components of speaking:

Interaction

Pronunciation

Fluency & coherence Vocabulary & grammar.

Each number is given a descriptor which explain what elements of speaking should the

teacher or assessor be looking for.

Interaction

5.

Can initiate &logically develop

simple

conversation onfamiliar topics

Can take turnsappropriately

4.

Interaction isadequately initiated

and developed

Can take turnbut needs little

prompting

3.

Developsinteractions

adequately makes

however minimaleffort to initiate

conversationNeeds constantprompting to take

turns

2.

Contributionsare often unrelated

to those of the

other speaker

Generally passive in the

development of conversation

1.

Contributionsare mainly

unrelated to those

of other speaker

Shows hardly any initiative in the

development of conversation

Very limitedinteraction

Pronunciation

5

Can pronounce

correctly &articulate clearly

Is alwayscomprehensible;uses appropriate

intonation

4

Mostly correct

pronunciation &clear articulation

Can be clearly understood mostof the time; very few phonological

errors

3

Largely correct

pronunciation &clear articulationexcept occasional

errors

Someexpressions cause

stress withoutcompromising

with understanding of spokendiscourse.

2

Frequently

unintelligiblearticulations

Frequentphonological

errors

Majorcommunication

problems

1

Insufficient

accuracy inpronunciation;

many grammatically

errors

Communication is severely

affected




8

The table below is a summary of the score obtained by each student on the FOUR criteria.

Fluency &

Coherence

5

Speaks fluently

almost with norepetition &

minimal hesitation

Develops topicfully & coherently

4

Speaks without

noticeable effort, with a little

repetition

Demonstrateshesitation to find

words or usecorrect

grammaticalstructures and/or

self correction

Topics not

fully developed tomerit

3

Is willing to

speak at length,however repetition

is noticeable

Hesitatesand/or self

corrects;occasionally loses

coherence

Topics mainly developed, but

usually not

logically concluded

2

Usually fluent;

produces simplespeech fluently,

but losescoherence in

complexcommunication

Often hesitatesand/or resorts to

slow speech Topics partly developed; not

always concluded

logically

1

Noticeably/lon

g pauses; rate of speech is slow

Frequentrepetition and/or

self correction

Links only basic sentences;breakdown of

coherence evident

Vocabulary &

Grammar

Can express with some

flexibility andappropriacy on a variety of topics

such as family,hobbies, work,

travel and currentevents

frequently usescomplex formsand sentence

structures; hasenough vocabulary

to expresshimself/herself

Can express with some

flexibility andappropriacy on

most of the topics

Demonstratesability to use

complex forms

and sentencestructures most of the time; expresses

with adequate vocabulary to

express

Communicates with limitedflexibility and

appropriacy onmost of the topics

Sometimesuses complex

forms and

sentencestructures; has

limited vocabulary to/describe/expres

s new points

Communicates with limitedflexibility and

appropriacy onsome of the topics

Complexforms and

sentence structures

are rare; exhibitslimited vocabulary

to express new ideas

Demonstratesalmost no

flexibility, andmostly strugglesfor appropriate

words

Uses very basic vocabulary to

express viewpoints

Name of Student Interaction Pronunciation Fluency &

Coherence

Vocabulary &

Grammar

Total

[max = 20]

Ali Seman 4 3 5 3

Wee Kong Lam 3 5 4 4

M. Sundram 4 4 3 5




9

ANOTHER EXAMPLE OF A RUBRIC TO ASSESS SPEAKING:

Poor

0 pts

Fair

1 pts

Good

3 pts

Excellent

5 pts

Clarity

Poor

All questions andanswers wereawkward and

incomprehensible.

Fair

Questions andanswers wereawkward and

incomprehensible

to understand attimes.

Good

Questions or answers were

awkward at timesbut always

understandable.

Excellent

Questions andanswers were clear

andcomprehensible.

Pronunciation

Poor

Student'spronunciation wasincomprehensible.

Fair

Student'spronunciation

madeunderstanding

difficult.

Good

Student'spronunciation was

understandablewith some error.

Excellent

Student'spronunciation was

like a nativespeaker.

Fluency

Poor

Student wasunable to ask or

respond toquestions.

Fair

Student took a longtime to ask and

respond toquestions.

Good

Students were ableto ask and answer the questions with

little difficulty.

Excellent

Students were ableto communicateclearly with no

difficulty.

Comprehension

Poor

Student wasunable to

comprehendquestions.

Questions had tobe repeated.

Fair

The studentshowed little

comprehension of questions.

Questions had tobe repeated.

Good

The studentunderstood most of what was asked of

him/her.

Excellent

The student fullyunderstood thequestions askedand answered

correctly.

Content

Poor

Did not askappropriatequestion for

information, noresponse to

question.

Fair

Ask someinappropriatequestions for information or

answered questionwith very limited

answers.

Good

Gave appropriatequestions for

survey informationbut responseswere limited in

content.

Excellent

Gave appropriatequestions and

good content inresponses to

questions.




10

Instructions:

a) The speaking test will be conducted for two students at a time.

b) Students should not have pen, paper, mobile phone etc.

c) The teacher will be an interlocutor and an assessor both.

I. GENERAL INTRODUCTION - (1 min)

(The Interlocutor makes the students feel comfortable)

Interlocutor: Good morning, I am ----------------------. Hope you are looking forward

to this brief interaction.

Student A: Yes ----------------/ I am -------------------

Interlocutor: And what about your? (Looking at student B)

Student B: ---------------------------------

Interlocutor: What do you do in your spare time? (To Student B)

Student B: ------------------------------------

Interlocutor: And how about you? (To student A)

Student A: -----------

Interlocutor: What makes you special? (To student A)

Student A: -----------

Interlocutor: What about you? (To Student B)

Student B: -----------

Interlocutor: What do admire in other people? (To student A)

8.6 Assessing Speaking in an Interview

a) Compare the criteria used in the two rubrics. Are they similar? Explain.

b) Compare the descriptors used in the two rubrics. To what extent are theydifferent?

c) Which rubric would you use in your class? Why?

d) Compare these rubrics with the rubrics suggested in the KSSR>

e) Design your own rubric for assessing speaking.




11

Student A: -----------

Interlocutor: How about you? (To Student B)

Student B: ------------------------

Interlocutor: Thank you, this is the end of the first part of the test.

In assessing oral skills, we can use different test items according the relevant purposes or

target information to be tested. Luoma (2004) provides three frameworks about general

purposes of testing oral skills as “linguistically oriented, communication-oriented and

situation- based.”

A) L inguisticall y Oriented: vocabulary, grammar and pronunciation.

This purpose can be carried out by structured speaking tasks. They are called ‘structured’

because they control quite closely what the examinees are going to say. In reading aloud

and sentence repetition, the testers know exactly what the examinee will say, and, in

short-answer questions and reacting to phrases tasks, a short list of acceptable answers

can usually be specified. Structured speaking tasks are typically used to evaluate

linguistic features, particularly pronunciation and grammar. Short-answer questions

and reactions to phrases can also be used to evaluate overall understanding and

comprehensibility. Short-answer questions test comprehension of the questions andability to give relevant information in response. The difference between this and more

extended simulations is that the questions and answers are limited and all the information

needed for answering the questions is usually provided in the task materials. Reacting to

phrases is another structured task that is often used in tape-based tests. The task usually

tests the examinees’ knowledge of conventional politeness exchanges such as greetings,

thanks, apologies, expressions of agreement and polite disagreement, and so on.

B) Communication-Or iented:

The overall communication activity in the task such as telling a narrative or expressing

and defending an opinion, discussing factors that support the chosen opinion and argueagainst others, comparing and contracting things through which advanced oral skills for

description is needed.

C) Situation-Based Task Design:

This belongs to the task-based approach to defining the test construct. This approach is

typically used in specific-purpose testing and in vocational and professional education.

8.7 Structured Speaking Tasks




12

Types of test items and their Functions:

Description tasks

Example 1 — A one-to-one interview: Describe to me the room or area where you work.

Example 2 — A pair task in an interview test (two pictures are provided):

Describe your pictures to each other and then talk about what is similar in your pictures

and what is different.

Narrative tasks

Example 3: A tape-based test (Six pictures should be provided)

Please look at the six pictures below. Tell the story based on these pictures starting from

picture number 1 and going through picture number 6. Take one minute to look at the

pictures.

The narrative is a monologue, and as the test is tape-based the examinees have to tell it in

one long stretch without any feedback from a listener.

Example 4: A face-to-face paired interaction test

You each have a set of pictures. Together they make a story. Each of you tells one part of

he story.

Narrative tasks are also frequently used in speaking tests. They show how well the

examinees can recount a sequence of events, usually in one time frame, either present or

past. Most often, the tasks are based on picture sequences, where the content of the

pictures guides what will be said. It is likely to create some interaction even though one

of the pupils is always the main speaker while the narrative is being constructed.

Information gap between the two speakers is therefore likely to occur. Also, the choice of

good sequences is a difficult matter.

Personal stories often reveal embarrassing details that speakers would be shy to discuss

in a test or, if not, they may be so uneventful that the speakers would consider them

unworthy to tell.




13

Instruction tasks

Example 5: A one-to-one interview

Imagine that we are standing in front of your house. Tell me how to get to the shop from

there.

Example 6: A face-to-face paired interaction test — Feeding the puppy

You cannot go home and your puppy needs to be fed. Your friend says he will do it. Tell

your partner exactly what to do, what he’ll need and where to find things. Follow the

instructions below. Fine what you need in the picture. Tasks — what you have to do; what

you need; where to find things.

The main purpose in giving directions and instructions is getting the message across themaking sure that it has been understood. This tends to mean short exchanges between the

speaker and the listener.

Comparing and contrasting tasks

Example 7: Interaction outline for a pair task in a paired interview

Candidate A compare and contrast two or three of these photographs, saying what kind of

clothing the people are wearing and why the protection might be necessary.

Explaining and predicting tasks

Example 8: A taped-based test

Explaining the contents of a graph or explaining a process is a fairly common task in

many professional and study settings. To do well on the task, the speakers need to set the

scene and identify parts of the information or stages in the process that they are

explaining and present them in coherent order. They also need to explain the significance

of the important parts or stages, so that the listeners understand what the explanation is

about and why it is the being given. Predictions go together well with explanation tasks,

and they can also be fairly monologic. As predictions involve speculation, they may

become more interactive in a face-to-face setting.

Decision tasks

Example 9: in a paired interview

Discuss and decide together: 1) what the advantages and disadvantages are of attending

trade fairs, for instance, and 2) which members of staff would most usefully represent a

company at a trade fair.




14

In speaking tests, the issues that need to be decided are usually not clear-cut, so that

arguments for and against different solutions are needed. The speakers express their

opinions about the concerns and justify them in order to air different viewpoints before

negotiating the conclusion.

Role-plays and simulations

Role-plays simulate different kinds of communication situations that the target group of

the test could plausibly meet outside the test. They can be completed between two

examinees or between an examinee and a tester.

Example 10: A job interview

The employer —inquiring information about candidate’s abilities, qualifications, and

Character

The candidate — providing information about himself and inquiring information about the

Company

Role-play tasks are a way of making communication in a test more versatile because,

rather than talking to a tester, the examinees take on a new role and a new, simulated role

relationship to their communication partner. Their performance shows their ability to

adapt to the requirements of the new role and situation. As long as the situation is relevant

for the target audience and the purpose of the test, this gives useful information for the

tester. The information may simply be a new perspective into the examinees’ linguistic

resources, or the use of different functions from other tasks in the test.




15

SUMMARY

The types of criteria we use to assess a speaker’s oral performance during a classroom

activity will depend on which kind of talk we are talking about and the kind of

classroom activity we are using.

Assessment of speaking skills often lags far behind.

Different speaking activities such as conversations, group discussions, and speeches

make different types of demands on learners.

A frequent problem for teachers is when there are so many learners in one class that it

seems to make it unrealistic to assess speaking.

Pronunciation, vocabulary, accuracy, communication, interaction and fluency are all

markers of a student’s overall speaking a bilities.

Most teachers will be familiar with the concept of grading with a rubric, a table withdifferent criteria and a grading scale.

Vocabulary comprehension and vocabulary production are always two separate banksof words in the mind of a speaker, native as well as second language.

Fluency may be the easiest quality to judge in your students’ speaking. Howcomfortable are they when they speak? How easily do the words come out?

Three frameworks about general purposes of testing oral skills as “linguistically

oriented, communication-oriented and situation- based.”

REFERENCES

Luoma, S. (2004). Developing speaking tasks. In Assessing speaking (pp. 139-

169). Cambridge: CUP

Chapter 8.ASSESSMENT.speaking

Documents