Factors that Influence Test Construction

Factors that Influence Test ConstructionReporter : Ms. Reggie “ms_ridge” BatausaBridging Student Sem1 SY 2014-2015

Objectives

At the end of this lesson, the learners are expected to:• Explain the factors that influence test construction

The Function of the Test in the Instructional

ProcessTesting

Frequency

The Use of Open-book and

Closed-book Examinations

The Use of Take-Home

ExaminationMode of Item Presentation

Factors Influencing Test Construction

The Function of the Test in the Instructional Process

PURPOSE TEST CONSTRUCTION

Basic Types of Classroom Tests according to Function and Time of Administration

Before Instruction During Instruction End of Instruction

Function Readiness Placement Formative

Diagnostic

Summative

Focus of measurement

Prerequisite entry skills

Course or unit objectives

Predefined segment of instruction

Most common learning errors

Course or unit objectives

Nature of sample

Limited sample of selected skills

Broad sample of all objectives

Limited sample of selected skills

Limited sample of selected skills

Broad sample of all objectives

Item difficulty Typically has low level of difficulty

Typically has wide range of difficulty

Varies with the segment of instruction

Typically has low level of difficulty

Typically has wide range of difficulty

Time of administration

Beginning of course or unit

Beginning of course or unit

Periodically during instruction

As needed during instruction

End of course or unit

Use of results Remedy entry deficiencies or assignment to learning group

Instructional planning and advanced placement

Improve and direct learning through ongoing feedback

Remedy errors related to persistent learning difficulties

Assign grades, certify accomplishments, evaluate teaching

Airasian & Madaus, 1972, as cited in Linn and Gronland, 2000

Testing Frequency

Single Testing• Selection• Placements• Summative evaluation

Frequent Testing• Diagnosis• Formative evaluation• Motivation

The Use of Open-Book and Closed-Book Examinations

Open-Book Closed-Book

Emphasizes higher objectives in Bloom’s Taxonomy

Emphasizes recall of information

Less time memorizing More time memorizing

Applying specific formula on information

Memorization of information

Skills in utilizing reference materials for a limited time

The Use of Take-Home Examination

Advantages• Students’ own time

and rate of speed• Useful when

references are not found in the classroom or even in school

Disadvantages• Difficulty of scoring • Plagiarism

Mode of Item Presentation

Oral

Written on the board

Mimeographed

Projected on screen

Levels of Item Complexity

Complexity• Consider complexity when developing

test items• What do you want the test taker to

be able to “show” you?

• Three levels• Low• Moderate• High

Low Complexity

Test items require factual knowledge.

Require knowledge of single fact, not multiple facts.

Examples:

• Knowing the name of the piece of equipment

• Knowing what test procedure to use in a simple situation

• Knowing what reference provides instructions

• Knowing which tool to use to repair a particular piece of equipment

Low Complexity Example

Espresso is best brewed ________________.a. From 7 to 10 secondsb. Between 250 ֯ F and 212 ֯ F c. Between 198 ֯ F and 204 ֯ F d. From 30 to 60 seconds

Moderate Complexity

Test items require test taker to know or derive multiple facts.• May be moderately complex in two ways• Require knowledge of more than one fact• Two or more pieces of information are

combined in some fashion to produce a new fact

• Require more complex knowledge about facts because information is combined or compounded

• Correct answer depends on factual knowledge that is complete and definitive

Moderate Complexity Example

You are assigned to perform a safety and health inspection for the Cyber Café. The Barista prepares a cup of espresso and serves it to a customer. You, as the inspector, observed the brewing temperature to be 188֯F and the brew time to be 20 seconds for a single shot. You failed the Barista for preparation of espresso.Select the reason that you failed the Barista for this process.a. Brewing temperature too high.b. Brewing temperature too low.c. Brewing time too long.d. Brewing time too short.

High Complexity

Items require test taker to:

• Analyze & evaluate multiple facts to solve problems

• Diagnose solutions• Make decisions

Correct Answer may not be recognized until it is compared to incorrect responses to identify the best answer under conditions

High Complexity (cont..)

The analysis or evaluation required to recognize the correct response requires:

• Synthesis of multiple facts• Evaluation of each alternative to compare• Occasionally, evaluation of each alternative

to comparison and/or correlation to each other

Typically scenario type test items

• Derived from “real” on-the-job / actual situations

Bloom’s Original Taxonomy (1956)

Evaluation

Synthesis

Analysis

Application

Comprehension

Knowledge

Bloom’s vs. Item Complexity

High

Moderate

Low

• Evaluation• Synthesis

• Analysis• Application

• Comprehension• Knowledge

Knowledge

• Recall from memory• Recall is identical to

original presentation• Recite definitions,

procedures, formulas, etc.

Learning objectives

require learner to:

• State the steps in a procedure

• Define a term• Identify an object

Test items ask learner to:

Comprehension

• Restate in own words• Translate information• Apply designed rules• Recognize examples of

concepts

Learning objectives

require learner to:

• Identify unseen examples• Classify unseen examples

Test items ask learner to:

Application

• Decide rule(s) to apply• Solve problems by selecting

rule/method

Learning objectives

require learner to:

• Solve problems presented without rules/formulas given

• Decide course of action to take without prompting

Test items ask learner to:

Analysis

• Break down complex situations

• Figure out how parts relate to each other

• Figure out how one thing influences another

Learning objectives

require learner to:

• Solve problems given an extensive scenario, description or data• Some information may not

be pertinent

Test items ask learner to:

Synthesis

• Create totally original material, products, designs, equipment, etc.

Learning objectives

require learner to:

• Create totally original material, products, designs, equipment, etc.• Normally cannot be

multiple-choice or other closed-ended test items

Test items ask learner to:

Evaluation

• Judge appropriateness or worth of objects, plans, designs, etc. with some purpose in mind

Learning objectives

require learner to:

• Use all five lower level of Bloom’s Taxonomy to meet objective• Normally cannot be

assessed with closed-ended test items

Test items ask learner to:

The Function of the Test in the Instructional

ProcessTesting

Frequency

The Use of Open and

Closed Book Examinations

The Use of Take-Home

Examination

Mode of Item Presentation

Item Complexity

Factors Influencing Test Construction

Assessment of Learning

Build

Deliver

Score

Support