By Dongxue Qin and Moriah Kent SPEAKING ASSESSMENT.
Post on 21-Jan-2016
221 Views
Preview:
Transcript
By Dongxue Qin and Moriah Kent
SPEAKING ASSESSMENT
Trading Questions!•Dongxue and Moriah model the task•Students choose questions and practice individually•Play the game!
Language Area: Speaking• What kind of skill is speaking?
- Production skill
- Differs from writing in what ways?
Writing Speaking
Full, complex sentences, well-organized
Incomplete, simple and loose organization of sentences and thoughts
Information densely packed Simple discourse with less information
Specific vocabulary General vocabulary
Use of discourse markers Frequent use of fillers to facilitate speech
Text written for an unseen audience
Face-to-face communication
A relatively solitary process Negotiation between 2 or more people
Alterations and crossings out kept to minimum
Alternations, corrections, miscues are very common
Reference can easily be made to what was previously written
Memory limitations are important as speech is transitory
Source: Jones, W. (2005). Assessing students’ oral proficiency. In D. Lloyd, P. Davidson, & C. Coombe (Eds.), The Fundamentals of language assessment: A practical guide for teachers in the Gulf (pp.75-86) Dubai: TESOL Arabia Publications, p. 77.
Language Area
Language Area: Why? •Why do we assess speaking?•How do we assess speaking? •What standards should we hold second-language speakers to?
Language Area: Overarching Constructs
•Fluency- Include smoothness, rate of speech, absences of excessive pausing/hesitation, length of utterance and connection, and repetition-Self-corrections can be evaluated by both machines and humans. •Accuracy-Linguistic correctness, grammatical knowledge and ability
Language area: Speaking Competencies
•Grammatical competence•Discourse competence•Sociolinguistic competence•Strategic competence
Language area: Sub-constructs within Speaking•Pronunciation•Pitch, volume, speed, pausing, stress, intonation, rhythm, etc•Comprehensibility/Communicative competence/Fluency •Spoken grammar (Accuracy)•Formality/informality •Vocabulary
Research- Interlanguage (Selinker, 1972 in Folse, p.32)-Before the 1980s, classroom goals were focused on language knowledge, rather than communicative use of a language. (Folse, p.34)
-Krashen’s i+1 in terms of speaking-Swain’s (1985, 1995, 2005) “pushed output” for speakers, “When they are unable to communicate effectively, they must rethink their utterances and modify the relevant parts” (Folse, p. 42).
-Both comprehensible input and pushed output are important for speaking tasks.
Task Creation: Types• Speaking tests can be live or recorded
- TOEFL iBT = recorded- IELTS = live and recorded
•Open-ended or structured/restricted - Open: Give a lecture, presentation, have reciprocal conversation/debate or discussion - Structure: Answer direct questions, expected right/wrong answer• Individual, pair, group tasks - Individual = interview - Pair = role play - Group = group discussions or debates• Construct-based or task-based assessments - Construct: Language ability - Task: Can you use the language in a given domain? •Majority of tasks for assessment are integrated
Task Creation: ProcedurePre-creation: • What’s the focus? Fluency? Accuracy? Or both? • What are your criteria? • How will you score? • Formal or informal language? • Individual, group or pair work? • How much time? • What channel with the input come in? Listening? Visual?
Written? Or a combination?
During creation: • Choose a topic/task • Outline subtopics that learners can discuss as well as
vocabulary they might need to use• Is it a one-way or two-way task? • Is it timed? • Is it open-ended (divergent) or restricted (convergent)?
Task Creation: Considerations • Things to consider when creating a speaking assessment task
- Practicality
- What is your student’s personality like? • Possible considerations with administering speaking assessments
- One-on-one?
- More than one rate? • Inter-rater reliability
- Decide on the criteria/constructs
- Two raters: one to interact with the S and one to evaluate
- Use a score sheet
- What kind of speaking sample are you trying to elicit? (A speech with a purpose? Chunks of speech? Controlled speech? Specific points, etc?
Scoring: Issues•Issues with developing speaking rubrics:
1. Analytic or holistic or combo?2. What constructs are you assessing in
speaking? And what criteria corresponds to a score?
3. What should you say? Scales need to be concrete and clear, practical and avoid vagueness.
4. Norm-referenced or criterion-referenced?
Scoring: How to develop? •Methods of developing scales/rubrics:
1. Intuitive 2. Qualitative3. Quantitative•Ensuring Reliability and validityReliability-Validity-Scoring example scales…
ACTFL speaking scale
- 10 levels, with four sublevels (superior, advanced, intermediate and novice), the last three are further divided into 3 sublevels (high, mid, low)
- Holistic scale focused on what learners can do, not what they know
- Focuses on beginning and intermediate
- Novice Low: “Speakers at the novice-low level have no real functional ability and because of their pronunciation, they may be unintelligible. Given adequate time and familiar cues, they may be able to exchange greetings, give their identity and name a number of familiar objects from their immediate environment. They are unable to perform functions or handle topics pertaining to the intermediate level and cannot, therefore, participate in a true conversational exchange”. (Luoma, p.67)
TOEFL iBT Integrated Speaking Rubric (part)
TOEFL Activity
• Read the passage from a psychology textbook below and listen to the lecture that follows it. Then answer the question. You will have 45-50 seconds to read the text.
• FLOW• In psychology, the feeling of complete and energized focus in an activity is called flow. People who enter a state of flow lose their sense of time and have a feeling of great satisfaction. They become completely involved in an activity for its own sake rather than for what may result from the activity, such as money or prestige. Contrary to expectation, flow usually happens not during relaxing moments of leisure and entertainment, but when we are actively involved in a difficult enterprise, in a task that stretches our mental or physical abilities.
TOEFL Activity continued…• Now, listen to the following lecture, which is about this topic. You can take notes. You will hear it one time, only.
• Question: Explain flow and how the example used by the professor illustrates the concept.
• You will have 30 seconds to prepare. • STOP! • You will now have 60 seconds to respond. • STOP!
TOEFL iBT Integrated Speaking Rubric (part)
INTO CSU AEIN100 Speaking Rubric
5 - Advanced 4 - Satisfactory 3 – Needs Improvement2 and Below – Does Not
Meet Expectations
Content andVocabulary
(Specific ideas and vocabulary from
the text)
Specific vocabulary from the
curriculum is used accurately. All ideas are clear and appropriate for
the context.
General vocabulary is used
correctly. All meaning is clear.
General vocabulary is used, though
not always correctly. Some meaning is unclear.
Little vocabulary is used correctly.
Most meaning is unclear.
Language
(e.g. verb tense (present and past),
subject-verb agreement, word order)
All verbs are used correctly. Word
order is consistently correct.
Most verbs are used correctly.
Word order is generally correct.
Verbs are not used correctly. Word order is inconsistent and incorrect.
Verbs are missing or incorrect.
Word order follows no structure.
Pronunciation
(e.g. individual sounds, stress,
intonation)
Pronunciation of individual sounds
is clear and accurate. Very little interference from the first language is present, and the student can be
easily understood.
Almost all sounds are pronounced
clearly and accurately. Little interference from the first language may be present, and the student can
be understood.
Some sounds are not pronounced
clearly.The student can generally be
understood, but interference from the first language is present and
causes confusion.
Many sounds are pronounced
incorrectly. Heavy interference from the first language disrupts
understanding.
Fluency and Communicative
Ability
(e.g. pace, flow, on topic, ease of speaking)
Speaking is conversational, flowing smoothly and logically. Meaning is complete and clear; there is no need
for follow up questions from the instructor.
Speaking generally flows smoothly, and there is very little hesitation. Meaning is communicated, but the
instructor may ask for clarification.
Speaking is slow and halting.
Meaning is unclear, so the instructor must ask for clarification.
Speaking is very choppy. Little or no meaning is communicated. The instructor guides the conversation
with questions.
Total: /20
INTO CSU Speaking Rubric
- 4 levels- Holistic scale (giving one score)-Used to rate low-intermediate learners:Communicate clearly about concrete topics related to living in an American university community using American English pronunciation
Thank you for your time!
References:
Brown, J.D. (Eds.). (2013) New ways of classroom assessment. Alexandria, Virginia: TESOL International Association.
Coombe, C., Folse, K. & Hubley, N. (2010). A practical guide to assessing English language learners. Ann Arbor, Michigan: The
University of Michigan Press.
Folse, K.S. (2009). The art of teaching speaking: Research and pedagogy for
the ESL/EFL classroom. Ann Arbor, Michigan: The University of
Michigan Press.
Jones, W. (2005). Assessing students’ oral proficiency. In Lloyd, D.,
Davidson, P., & Coombe, C. (Eds.), The fundamentals of language
assessment: a practical guide for teachers in the Gulf (pp. 75-86).
Dubai: TESOL Arabic Publication.
Luoma, S. (2004). Assessing Speaking. Cambridge, UK: Cambridge
University Press.
ETS. (2014). TOEFL iBT speaking rubric.
Retrieved from: www.ets.org/toefl
top related