Second Language Writers and The Machine Scoring of Essays Deborah Crusan Wright State University
Jan 15, 2016
Second Language Writers and The Machine Scoring of Essays
Deborah CrusanWright State University
Agenda
• Define • Discuss• Consider
But first, we must remember
• "For there is nothing either good or bad, but thinking makes it so."
~ William Shakespeare (1564-1616), Hamlet, II.ii
So . . . What is machine scoring
• Computer technology that evaluates and scores written prose (Shermis & Burstein, 2003)
• web-based • scores and responds in seconds.
Some Machine Scoring Software
Company Scoring Engine Technique Instructional Program
Scoring
ETS e-rater ® Natural Language Processing
Criterion Single holistic score
Pearson Intelligent Essay Assessor (IEA)
Latent Semantic Analysis
WriteToLearn Holistic and component scoring
Vantage Learning
Table adapted from Dikli (2006) Warschauer & Ware (2006)
IntelliMetric ™ Artificial Intelligence
MY Access! (and others)
Holistic and component scoring
How does it work?
Artificial Intelligence (AI)
Latent Semantic Analysis (LSA)
Natural Language Processing (NLP)
Recognized Machine Scoring Platforms
https://www.ets.org/criterion
Recognized Machine Scoring Platforms
Recognized Machine Scoring Platforms
Positive features
• Objective
• Consistent
• Broad in scope• High stakes testing environment
The trouble with humans
DESKILLING OF TEACHERS
AUDIENCE
INADEQUATE DESCRIPTION OF ELL ERRORS
BORROWING
SOCIAL ASPECTS OF WRITING
ONE DRAFT WONDERS
For a complete reference list, contact [email protected]