Assessing Student Performance OK. Performance Objective Given a unit of instruction, develop a valid, reliable, criterion referenced student assessment.

Assessing Student Performance

OK

Performance Objective

• Given a unit of instruction, develop a valid,

reliable, criterion referenced student

assessment instrument that scores at least

70 points on the

evaluation

checksheet..

Enabling Objectives

• Distinguish among evaluation, measurement, and testing

• Differentiate between formative and summative assessment.

• Differentiate between criterion-referenced and norm-referenced assessment.

• Explain validity in student assessment.• Explain reliability in student

assessment.• Plan for criterion-referenced

assessment of student performance.

Why assess student performance?

• Assign grades

• Gauge student progressand award credit for task completion

• Improve instruction

• Motivate students to work

• Provide feedback to students

Basics

• Evaluation - the general process of estimating student progress toward achieving performance objectives

• Measurement - the use of a specific tool to estimate an outcome

• Testing - one specific form of evaluation that uses a measurement tool to formally evaluate student performance

Methods of Assessment

• Testing– “Objective”– “Subjective”

• Performance demonstration other than test– Psychomotor Task

• Project• Lab Skill Demonstration

– Higher-Level Cognitive Task• Paper• Project• Portfolio• Etc…

Testing is Either

• Formative

or

• Summative

Formative Testing

• The process of using measurement tools to conduct evaluation for the purpose of IMPROVING student PERFORMANCE

• Student receives feedback of results• Teacher considers results in planning

subsequent instruction• Grades are not recorded!!

Summative Testing

• The process of using measurement tools to conduct evaluation for the purpose of ASSIGNING student GRADES

• Student receives feedback of results• Teacher considers results in planning

subsequent instruction• Grades are recorded

A Test Can Be

• Norm-Referenced

or

• Criterion-Referenced

Norm-Referenced Test

• Measures student performance against other students

• Student scores better or worse than other students

• Competition is between student and peers

• Grade is based on location on “the curve”

• Best students get “A,” poorest students fail

Normal Curve

• On most measures of human behavior, graphing individual results will result in a “bell-shaped,” or normal curve

• Most individual scores will fall toward the middle (mean)

• Fewer scores will fall toward the upper and lower ends

Lowest Scores Highest Scores

Average

Scores

Making Test Norm Referenced

• Make test intentionally difficult

• Average score should be about 50%

• Strong students should tend to score high and weak students should tend to score low

• Award As for highest scores, Fs for lowest scores, Cs for average scores

Criterion-Referenced Test

• Measures student performance against predetermined standards

• Student meets or does not meet the standard• Competition is between the student and the

skill, knowledge, or ability• Grade is based on accomplishment• Everybody can earn a passing grade if they

meet the standard

Making Tests Criterion-Referenced

• Remember that a performance objective has a:– Condition, Task, & Standard

• Criterion = Standard• Write test items using performance objective

standard statements and your test will be criterion-referenced– Every objective 1 or more test items

– Every item an objective

• Validity is assured

Characteristics of a Test

• Validity

• Reliability

• Objectivity

• Discrimination (applies to norm-referenced test only)

• Comprehensiveness

• “Score-Ability”

Validity

• A valid test measures:

– what it is intended to measure

• what the teacher intended for the students to learn

• what the teacher actually taught

• A valid test is FAIR

Questions about Validity

• Does the test actually measure what you intend it to measure?

• Did you teach the content and skills that are being tested?

• Does the test require the student to know or do something other than what you intended and/or taught?

• Does some aspect of the test prevent the student who may know the material from responding correctly?

Example of Problem in Validity

You taught the names and uses of hand tools using lecture with overheads and handouts.

But:

On the test, you ask the students to describe how to maintain the tools in good condition. The problem is you taught one thing (names & uses) but tested

knowledge of another (maintenance).

Another Example

You taught the students to write resumes in the classroom and had them hand write their own

resumes, but provided no computer instruction.

But:

You have them prepare their resumes on a computer and grade heavily on appearance. The problem is you are evaluating their word processing skills at

least as much as their resume writing skills.

A Third Validity Problem

You intended to teach the students how to repair a small engine. You taught the lesson in the classroom using overheads, chalkboard, and a teacher demonstration. The students never touched an engine.

But:On test day, you give them a disassembled engine to

reassemble. The problem is you thought you taught a psychomotor skill, really taught only cognitive content, but are testing the psychomotor skill you never taught

Reliability

• A reliable test provides accurate and consistent results

• Test reliability can be viewed from two perspectives:– Student reliability– Scorer reliability

Student Reliability

• Test items are readable and clear

• Instructions are simple and unambiguous

• Responses test only knowledge of the subject matter and not test wiseness, reading ability, agility, or other unrelated trait

Scorer Reliability

• Items can be scored consistently

• Same scorer would produce similar results on repeated evaluations

• Different scorers would produce similar results if working independently

Objectivity

• Objectively written– items are reliable– items are valid

• Objectively administered

• Objectively Scored

Discrimination

• Important ONLY for norm-referenced testing

• Test separates more knowledgeable students from less knowledgeable students

• Discriminating test is intended to reward best students and punish weakest students

• Ideal for using normal curve to interpret score

Comprehensiveness

• Assessment covers or samples all of the content

• Every performance objective is represented

• Multiple items address each objective

Score-Ability

• Test has scorer reliability

• Scoring is easily completed

• “Objective” items are easiest to score

• “Subjective” items can be scored “objectively”

Review

• Evaluation vs. Measurement vs. Testing• Criterion-Referenced or Norm-Referenced• Formative or Summative• Characteristics of a Test:

– Validity– Reliability– Objectivity– Discrimination (applies to norm-referenced test only)– Comprehensiveness– “Score-Ability”

The Answer

• Rare to find an educator who claims to have the right answer, but…in Career and Technical Education

• Testing should be BOTH normative and summative

AND• Testing should be criterion-referenced

So What?

• Assessment can be positive or threatening

• Do not use assessment

as punishment or as a threat

• Use assessment to improve student performance and

instruction• Assign grades fairly: validly, reliably,

objectively, and comprehensively