TESTING WRITING Carolina Salazar Testing and Evaluation
TESTING WRITING Carolina Salazar Testing and Evaluation
THE TESTING PROBLEM
REPRESENTATIVE
VALIDLY SAMPLED
VALIDLY AND RELIABLY SCORED
REPRESENTATIVE
REPRESENTATIVE TASKS
SPECIFICATION OF ALL THE POSSIBLE CONTENT
Operations
Types of text
Adressees
Length of texts
Topics Dialect Style
Expressing,
directing, describing
…
Form, letter, fax,
note…
Boss, friends, family…
1 page, 150
words…Not
specialist,
relevant…
Standard
Formal, informal
…
REPRESENTATIVE SAMPLE OF THE SPECIFIED CONTENT
Try to include a representative sample of tasks
MORE TASKS
MORE VALIDI
TY
EXAMPLES
CCSE (CAMBRIDGE CERTIFICATES IN COMMUNICATIVE SKILLS IN ENGLISH)
FIRST STAGE: READING
TOPIC: Working in a Summer Camp for Children in America
TASK 1
Write a letter to American Summer Camps.
Find out about Dates
Hours of work Type of accomodation
Ask for the application form
TASK 2
Fill in the application form.
TASK 3
You are now working in the ASCFC. You write a postcard to
an English-speaking friend
On your postcard tell your friend:
Where you are Why you are there
Two things you like about the summer camp
TASK 4
You have arranged to go out tonight with two other Helpers
at the ASCFC in Florida. You have to change your plans suddenly, and cannot meet them. Leave them a note. Apologize and explain why you
cannot meet them Suggest a different day to go out
ANALYSIS
Representative sample
of tasks?
Content validity?
Practical?
THE TESTING PROBLEM
REPRESENTATIVE
VALIDLY SAMPLED
VALIDLY AND RELIABLY SCORED
VALIDLY SAMPLED
VALID SAMPLE OF WRITING ABILITY
SET AS MANY SEPARATE TASKS AS FEASIBLE People’s performance Consistent
Offer them more opportunities to start
Desirable
Practical
TEST ONLY WRITING, NOTHING ELSE
NO YES
Creativity
Intelligence
Imagination
Writing
TEST ONLY WRITING, NOTHING ELSE
READING
ILLUSTRATIONS
SERIES OF PICTURES
GRAPHICS – CHARTS
NOTES
THE TESTING PROBLEM
REPRESENTATIVE
VALIDLY SAMPLED
VALIDLY AND RELIABLY SCORED
VALIDLY AND RELIABLY SCORED
VALID AND RELIABLE SCORING
ENSURE VALID AND RELIABLE SCORING Set tasks which can be reliably scored
Set as many tasks as posible
Restrict candidates
Give no choice of tasks
Ensure long enough samples
Create appropriate scales for scoring
HOLISTIC SCORING
“Impressionistic”
Overall impression
Purpose of the test
PRO
Very rapid
CON
Scorer reliability?
http://www.ets.org/toefl/pbt/scores/writing_score_guide/
ANALYTIC SCORING
http://www.ode.state.or.us/wma/teachlearn/testing/scoring/guides/2011-12/wriscorguide_eng_no-dates.pdf
Separate score – separate aspects
PROS
Uneven development of subskills
Scorers consider details of performance
More reliable
CONS
Time consuming
Too specific (The whole is greater than the sum of its parts)
HOLISTIC OR ANALYTIC?
PURPOSECIRCUMSTANCE
S
ACCURACY RESOURCES
DECISIONMULTIPLE SCORING
MORE CONSIDERATIONS ON
SCORING
SOME CONSIDERATIONS Calibrate the scale Collect samples of performance Cover the range of scales
Select and train scorers Native speakers Sensitive to language Experienced Training (3 stages)
Follow acceptable scoring procedures Score each task independently Several scorers Identify discrepancies
Give feedback Useful – positive Decide the content during callibration
CONCLUSIONS
CONCLUSIONS
Pay attention to validity and reliability
Give importance to writing as an independent skill
Always be meaningful and enhance real communication
SOURCES
http://www.ode.state.or.us/wma/teachlearn/testing/scoring/guides/2011-12/wriscorguide_eng_no-dates.pdf
http://www.ets.org/toefl/pbt/scores/writing_score_guide/