This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
supporting a case for assessment use." Language Assessment Quarterly, 2(1). 2005.
• Bachman, Lyle F and Adrian Palmer. Language Testing In Practice. Oxford University Press. 1996. http://www.oup.co.uk/
• Bachman, Lyle F and Adrian Palmer. Language Assessment In Action. Oxford University Press. Forthcoming.
• Toulmin, S. E. The Uses of Argument. Cambridge: Cambridge University Press. 2003.
• Watson, Jenny Peterson & Sindhvananda, Kanchana. "Notes on the Thammasat University English Program". Bangkok: Thammasat University Faculty of Liberal Arts. 1972.
• Palmer, Adrian. "Procedures for student classification and grading in courses I-IV". Bangkok: Thammasat University Faculty of Liberal Arts. 1972.
2. Construct validity: the meaningfulness of the interpretations that we make on the basis of assessment scores
3. Authenticity: the degree of correspondence between the characteristics of a given assessment task and the characteristics of a relevant non-assessment language use task
4. Intended Impact: the intended effects that taking a assessment, administering and taking a assessment, and using assessment results have on students, teachers, educational systems, and society
Summary of Reasoning in Example Assessment Use Argument
4. USE/DECISIONSAssign grades at end of
grammar unit.
3. INTERPRETATIONNumbers are interpreted
as students' knowledge of grammar
2. RESULTS/SCORESNumbers are assigned to
performance
1. PERFORMANCE ON ASSESSMENT TASK
Students select answers on M-C Grammar Test Tasks
ReliabilityFor the following reasons…we can consistently associate grammar scores with students' performance on M-C tasks
Construct ValidityFor the following reasons…scores can be interpreted in terms of "knowledge of grammar
AuthenticityFor the following reasons…the M-C task is appropriate for measuring the students' knowledge of grammar in this situation.
Intended ImpactFor the following reasons…using the interpretations of the students' knowledge of grammar to assign grades will have the intended impact on test takers and test users.
2. Place remaining students in appropriate ESL classes
3. Assign grades of A and B in ESL courses (lower grades to be assigned using other measures)
Intended Impact Warrants
1. Individuals a. Eliminating unnecessary
instruction frees students to take other courses.
b. Instruction at appropriate level is more effective.
c. Regularized grading al lows for systematic interpreta tion of grades and reduces complaints of unfairness.
2. Systems a. Relevance of construct to
decisions: University courses focus on grammar, vocabulary, and reading comprehensi on, so measures of these constructs are needed to place students appropriately (co mmon practice).
a. Regularized in struction at different levels over time and across classes maximizes use of resources.
1. Individuals a. Eliminating unnecessary instruction frees students to
take other courses. b. Instruction at appropriate level is more effective. c. Regularized grading allows for systematic interpretation
of grades and reduces complaints of unfairness. 2. Systems a. Relevance of construct to decisions: University courses
focus on grammar, vocabulary, and reading comprehension, so measures of these constructs are needed to place students appropriately.
b. Regularized instruction at different levels over time and across classes maximizes use of resources.
Backing 1. Individuals a. Documented communication from advanced students (see ษ ) b. Standard practice c. Documented communication from teachers and students on
fairness of grades (see ษ ) 2. Systems a. Standard practice. b. Documented teacher feedback on time spent in class preparation
2. Place remaining students in appropriate ESL classes
3. Assign grades of A and B in ESL courses (lower grades to be assigned using other measures)
Authenticity Warrants
1. Relevant instructional task selection: instructional materials consist to a large extent of reading passages and specific selections from passages illustrating grammar, vocabulary, and reading comprehension teaching points.
2. Correspondence of instructional task and test task characteristics: Reading passages are similar in difficulty and content to instructional passages. Many instructional tasks involve selected responses and limited constructed responses.
Backing 1. Examples of instructional reading passages and
instructional tasks can be found in the following course texts (references here).
2. Reading difficulty formulas have been used to calculate difficulty of reading passages in instructional materials and calibrate difficulty of test passages (see TUPT manual). Both instructional and test passages are based upon topics involving general (non technical) background knowledge and selected and limited constructed responses.
instructional materials consist to a large extent of reading passages and specific selections from passages illustrating grammar, vocabulary, and rhetorical organization teaching points.
2. Correspondence of instructional task and test task characteristics: Reading passages are similar in difficulty and content to instructional passages. Many instructional tasks involve selected and limited constructed responses.
1. Knowledge of grammar, vocabulary, and reading comprehension
Construct Validity Warrants
1. The constructs grา ammar, vocabulary,
and reading comprehensionำา have been carefully defined.
2. The selected response grammar, vocabulary, and reading comprehension test tasks allow the test takers to demonstrate their knowledge of grammar, vocabulary, and reading comprehension
2. Results/Scores Total number of correct responses
Backing 1. The construct definitions have been developed by
a committee of teachers with a background in test design. (See definitions of constructs in test design statement.)
2. The test tasks have been designed to focus attention on the testing point in contexts that do not in and of themselves create additional difficultly for test takers. For example, tasks designed to test grammar do not involve difficult vocabulary as well.
Construct Validity Warrants
1. The constructs าgrammar, vocabulary, and reading comprehension have been carefully defined.
2. The selected response grammar, vocabulary, and reading comprehension test tasks allow the test takers to demonstrate their knowledge of grammar, vocabulary, and reading comprehension.
1. Exempt highly proficient students from ESL writing classes
2. Place remaining students in appropriate ESL writing classes
3. Assign grades of A and B in ESL writing courses, (lower grades to be assigned using other measures)
3. Additional Interpretations
of Results 1. Knowledge of
grammar, vocabulary, and rhetorical organization in tasks involving essay writing
Additional Intended Impact
Warrants
1. Individuals a. No additional warrants 2. Systems a. Relevance of construct
to decisions: New university writing courses focus on knowledge of grammar, vocabulary & rhetorical organization in essay writing tasks, so measures of these constructs in essay writing tasks are needed to place students appropriately.
1. Individuals a. No additional warrants 2. Systems a. Relevance of construct to decisions: New
university writing courses focus on knowledge of grammar, vocabulary & rhetorical organization in essay writing tasks, so measures of these constructs in essay writing tasks are needed to place students appropriately.
Additional Backing 1. Individuals 2. Systems a. Documented feedback from instructors
that students who control grammar and vocabulary in reading tasks cannot necessarily perform well on tasks involving essay writing
task selection: instructional materials also involve tasks involving essay writing.
2. Correspondence of assessment task / instructional task characteristics: Assessment essay topics are similar to topics involving general knowledge used in instructional tasks. Length of assessment essay tasks is similar to length of instructional essay tasks.
3. Interpretations of results
1. Knowledge of grammar, vocabulary, and rhetorical organization in tasks involving essay writing
4. Use/decisions 1. Exempt highly proficient
students from new ESL essay writing classes
2. Place remaining students in appropriate new ESL essay writing classes
3. Assign grades of A and B in new essay writing courses, (lower grades to be assigned using other measures)
instructional materials also involve tasks involving essay writing.
2. Correspondence of assessment task / instructional task characteristics: Assessment essay topics are similar to topics involving general knowledge used in instructional tasks. Length of assessment essay tasks is similar to length of instructional essay writing tasks.
1. Knowledge of grammar, vocabulary, and rhetorical organization
Construct Validity Warrants
1. The constructs าknowledge of grammar, vocabulary, and rhetorical organization ำา have been carefully defined.
2. The extended production essay writing test task allows the test takers to demonstrate their knowledge of grammar, vocabulary, and rhetorical organization
Backing 1. The construct definitions have been developed by
a committee of teachers with a background in test design. (See definitions of constructs in test design statement.)
2. The test tasks have been designed to focus attention on the testing point in contexts that do not in and of themselves create additional difficultly for test takers. For example, essay-writing tasks involve topical knowledge common to all test takers.
Construct Validity Warrants
1. The constructs าknowledge of grammar, vocabulary, and rhetorical organization have been carefully defined.
2. The extended production essay writing test task allows the test takers to demonstrate their knowledge of grammar, vocabulary, and rhetorical organization.
• Describe additional decisions and intended impact
– Program directors need to make the following decision: Should they add an essay writing task to the English test given to all students entering Thammasat University?
– Program directors want to increase students' ability to write essays because essay writing is an ability that students currently lack. This ability is needed both in instructional and real-life language use tasks that the students need to perform.
• To make this decision, they need to develop Assessment Use Arguments for two alternatives:
1. Do not add an essay writing task. Continue to use only the M-C tasks to place and grade students in essay writing classes.
2. Add an additional essay writing task and use this to place and grade students in essay writing classes.
• Then decide1. which argument they prefer and can live with…2. on the basis of whether developing the test
according to the preferred argument is worth the cost.