8/13/2019 Language Testing_developing and Improving Test Item
1/22
DEVELOPING AND
IMPROVING TEST ITEM
8/13/2019 Language Testing_developing and Improving Test Item
2/22
WHAT IS AN ITEM ?
Brown (1996: 49) defines an item is thebasic unit of language testing. The item
is sometimes difficult to define. Some
types of items, like multiple-choice or
true-false items, are relatively easy toidentify as the individual test questions
that anyone can recognize as discrete
units.
8/13/2019 Language Testing_developing and Improving Test Item
3/22
DevelopingNorm-
ReferencedLanguage
Test
Item Format Analysis
DistractorEfficiency Analysis
ItemDiscrimination
Analysis
Item Facility Analysis
8/13/2019 Language Testing_developing and Improving Test Item
4/22
Item Format Analysis
GeneralGuidelines
Receptive
ResponseItems
Productive
ResponseItems
8/13/2019 Language Testing_developing and Improving Test Item
5/22
GeneralGuidelines
Is the itemformat correct lymatched to thepurpose andcontent of the
item? Is there onlyone correctanswer?
Is the itemwrit ten at the
students’ levelof prof ic iency?
Haveambiguousterms andstatements
been avoided?
Have negativesand double
negatives beenavoided? Does the item
avoid giv ingclues that could
be used inanswering
other items?
Are al l parts ofthe item on thesame page?
Is only relevantin format ionpresented?
Have race,gender, and
national ly biasbeen avoided?
Has at least oneother col leaguelooked over the
item?
8/13/2019 Language Testing_developing and Improving Test Item
6/22
ReceptiveResponse
Items
True-FalseItems
Multiple-ChoiceItems
MatchingItems
8/13/2019 Language Testing_developing and Improving Test Item
7/22
True-False Items
Is the statementworded careful ly
enough that it can be judged w ithout
ambigui ty?
Have “absoluteness”clues been avo ided?
8/13/2019 Language Testing_developing and Improving Test Item
8/22
Multiple-ChoiceItems
Have allunintentionalc lues beenavoided?
Are al l of thedistractorsplausible?
Has needlessredundancy beenavoided in the
opt ions
Has the orderingof the opt ionsbeen careful lycons idered? Orare the correct
answers randomlyassigned?
Have distractorssuch as “none ofthe above” and “aand b only” been
avoided?
8/13/2019 Language Testing_developing and Improving Test Item
9/22
MatchingItems
Are theremore
opt ions thanpremises?
Are opt ionssho rter thanpremises to
reducereading?
Are theopt ion and
premise l is tsrelated toone c entral
theme?
8/13/2019 Language Testing_developing and Improving Test Item
10/22
ProductiveResponse
Items
Fill-in Items
Short-response
Items
Task Items
8/13/2019 Language Testing_developing and Improving Test Item
11/22
8/13/2019 Language Testing_developing and Improving Test Item
12/22
Short-responseItems
Is th e i temformatted so thaton ly one relativelyconc ise answer is
poss ib le?
Is the item framedas a clear and
direct quest ion?
8/13/2019 Language Testing_developing and Improving Test Item
13/22
TaskItems
Is the students’task clear lydefined?
Is the tasksuf f ic ient ly narrow(and/or broad) forthe t ime avai lable?
Have scoringprocedures beenworked out in
advance with regardto the approach that
wil l be used?
Have scoringprocedures beenworked out in
advance with regardto the categories oflanguage that wil l be
rated?
Have scoringprocedures beenclear ly def ined interms of what eachscore within eachcategory means?
Is scoring to beas anonymousas possible?
8/13/2019 Language Testing_developing and Improving Test Item
14/22
ITEM F CILITY N LYSIS Brown (1996: 64) states that item facility (IF),
also called i tem di f f icul ty or i tem easiness , is astatistical index used to examine the
percentage of students who correctly answer a
given item.
8/13/2019 Language Testing_developing and Improving Test Item
15/22
ITEM F CILITY N LYSIS
=
Where
Ncorrect = number of students answering correctly
Ntotal = number of students taking the test
8/13/2019 Language Testing_developing and Improving Test Item
16/22
ITEM DISCRIMIN TION N LYSIS Item discrimination (ID) indicates the degree to which an
item separates the students who performed well fromthose who performed poorly. These two groups are
sometimes referred to as the high and low scorers or
upper and lower-proficiency students. The reason for
identifying these two groups is that ID allows teachers to
contrast the performance of the upper group students onthe test with that of the lower-group students (Brown,
1996: 66-67).
8/13/2019 Language Testing_developing and Improving Test Item
17/22
ITEM DISCRIMIN TION N LYSIS
= −
where
ID = item discrimination for an individual item
IFupper = item facility for the upper group on the
whole test
IFlower = item facility for the lower group on the
whole test
8/13/2019 Language Testing_developing and Improving Test Item
18/22
DISTR CTOR EFFICIENCY N LYSIS Brown (1996: 71) states the primary goal of
distractor efficiency analysis is to examine thedegree to which the distractors are attracting
students who do not know the correct answer.
To do this for an item, the percentages of
students who chose each option are analyzed.
8/13/2019 Language Testing_developing and Improving Test Item
19/22
NRT Development and ImprovementProjects
Pilot a relatively large number of test items on a group of studentssimilar to the group that will ultimately be assessed with the test.
Analyze the items using format analysis and statisticaltechniques.
Select the best items to make up a shorter, more effective revisedversion of the test.
8/13/2019 Language Testing_developing and Improving Test Item
20/22
Developing
Criterion-Referenced
Language Test
Item QualityAnalysis
Difference Index
The B-index
CRT ItemSelection
8/13/2019 Language Testing_developing and Improving Test Item
21/22
CRT DEVELOPMENT ND IMPROVEMENTPROJECTS
Brown (1996: 79) states that a central purpose
of a CRT is to assess how much of an objectiveor set of objectives has been learned by each
student, CRT assessment has to occur before
and after instruction in the concepts or skills
being taught in order to determine whetherthere was any gain in scores.
8/13/2019 Language Testing_developing and Improving Test Item
22/22
CONCLUSION
In testing, one way to improve and
develop a test is to examine the
individual items which are called item
analysis. This is usually done for
purposes of selecting the best items thatwill remain on a revised and improved
version of the test.