S3 assessment type - which one - ellis

Aligning Assessment Type and Purpose with Use

National Chinese Language Conference 2013

David P. Ellis, PhDNational Foreign Language Center

University of Maryland

Test Type

Achievement vs. Performance vs. Proficiency

Content Response

Achievement

Performance

Proficiency

Achievement vs. Performance vs. Proficiency

Content Response

Achievement Curriculum-Specific Memorized/Rehearsed

Performance

Proficiency

Achievement vs. Performance vs. Proficiency

Content Response

Achievement Curriculum-Specific Memorized/Rehearsed

Performance

Proficiency Non-Specific Novel/Non-Rehearsed

Achievement vs. Performance vs. Proficiency

Content Response

Achievement Curriculum-Specific Memorized/Rehearsed

Performance Domain-Specific Applied/Semi-Rehearsed

Proficiency Non-Specific Novel/Non-Rehearsed

Activity

Either alone or in pairs, classify the following assessments as achievement, performance, or proficiency.

Test Type ActivityTest Type Test Type

Driving test (road) SAT

Driving test (written) Semester final

Vocabulary quiz Writing placement

Role play (known prompt)

Typing test

Role play (unknown prompt)

Translation test

Test Type Activity AnswersTest Type Test Type

Driving test (road) Performance SAT Proficiency

Driving test (written) Achievement Semester final Achievement

Vocabulary quiz Achievement Writing placement Proficiency

Role play (known prompt)

Performance Typing test Performance

Role play (unknown prompt)

Proficiency Translation test Performance

Test Scoring

Test Scoring ConsiderationsCriterion-Referenced Norm-Referenced

Test Reliability Test Validity

Intra-Rater Reliability Inter-Rater Reliability

Test Scoring ConsiderationsCriterion-Referenced Absolute Norm-Referenced Relative

Test Reliability Test Validity

Intra-Rater Reliability Inter-Rater Reliability

Test Scoring ConsiderationsCriterion-Referenced Absolute Norm-Referenced Relative

Test Reliability Precision Test Validity Accuracy

Intra-Rater Reliability Inter-Rater Reliability

Test Scoring ConsiderationsCriterion-Referenced Absolute Norm-Referenced Relative

Test Reliability Precision Test Validity Accuracy

Intra-Rater Reliability Within Inter-Rater Reliability Across

Activity

Either alone or in pairs, classify the scoring method of the following assessments as either criterion-referenced or norm-referenced.

Test Scoring ActivityTest Scoring Type Test Scoring Type

Driving test SAT

Oral Proficiency Interview (OPI)

Semester final

Vocabulary quiz Writing placement

Role play TOEFL

Test Scoring Activity AnswersTest Scoring Type Test Scoring Type

Driving test Criterion-Referenced SAT Norm-Referenced

Oral Proficiency Interview (OPI)

Criterion-Referenced Semester final Depends

Vocabulary quiz Criterion-Referenced Writing placement Depends

Role play Depends TOEFL Norm-Referenced

Test Use

Test Use (Decision Purpose)Reason ExampleDiagnostic

Placement

Progress

Mastery

Selection

Prediction

Test Use (Decision Purpose)Reason ExampleDiagnostic MRI after head injury

Placement Writing test for incoming ESL students

Progress Pop quiz on last night’s reading material

Mastery Driving test

Selection Employment test

Prediction SAT

Test Use (Decision Type)Formative Summative

What

When

Why

Who

Test Use (Decision Type)Formative Summative

What Assessment FOR learning Assessment OF learning

When

Why

Who

Test Use (Decision Type)Formative Summative

What Assessment FOR learning Assessment OF learning

When DURING a course/section At the END of a course/section

Why

Who

Test Use (Decision Type)Formative Summative

What Assessment FOR learning Assessment OF learning

When DURING a course/section At the END of a course/section

Why To INFORM subsequent teaching To ASSIGN a score/grade

Who

Test Use (Decision Type)Formative Summative

What Assessment FOR learning Assessment OF learning

When DURING a course/section At the END of a course/section

Why To INFORM subsequent teaching To ASSIGN a score/grade

Who Teacher, Self, Peers Teacher

Test Design (Item Types)Item Type Advantage DisadvantageConstructed response

Multiple choice

True False

Check all that apply

Cloze

Matching

Categorization

Test Design (Item Types)Item Type Advantage DisadvantageConstructed response Requires a novel response Cannot be machine scored

Multiple choice Increases ability to create reliable tests

Does not require a novel response

True False Easy to create items 50-50 chance w/guessing

Check all that apply Allows for numerous answer combinations

Can be perceived as unfair

Cloze Can test multiple concepts (vocab) within a single context

Unclear what skill is being tested

Matching Allows demonstration of pairwise relationships

Correct answers can be found by process of elimination

Categorization Allows demonstration of semantic relationships

Scoring is complex

Thank YouDavid P. Ellis, PhD

[email protected]