THE CLOZE TEST AS A MEASURE OF ACHIEVEMENT by …arizona.openrepository.com/arizona/bitstream/10150/554589/1/AZU_TD... · THE CLOZE TEST AS A MEASURE OF ACHIEVEMENT IN EDUCATIONAL
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
The cloze test as a measure ofachievement in educational measurement
In Partia l Fulfillm ent of the Requirements For the D egree of
MASTER OF ARTS
In the G raduate C ollege
THE UNIVERSITY OF ARIZONA
1 9 7 3
STATEMENT BY AUTHOR
This th e s is has been subm itted in pa r t ia l fu lf illm ent of re quirem ents for an advanced degree at The U n ivers ity of Arizona and is d ep o s ited in the U n iv ers i ty Library to be made av a ila b le to borrowers under ru les of the Library.
Brief quo ta tions from this th e s e s are a llow able without sp e c ia l p e rm iss io n , provided th a t accu ra te acknow ledgm ent of source is m a d e „ Requests for pe rm iss io n for ex tended quo ta tion from or reproduction of th is m anuscrip t in whole or in part may be granted by the head of the major departm ent or the D ean of the G raduate C o llege when in his ju d g ment the proposed use of the m ateria l is in the in te re s ts of s ch o la r sh ip . In a ll o ther i n s t a n c e s , how ever, perm ission m ust be obta ined from the au thor.
SIGNED:
APPROVAL BY THESIS DIRECTOR
-This th e s is has been approved on the d a te shown below:
kflA/iety/., O fM A ll i n •/DARRELL L. SABERS 7 D ate
A sso c ia te P ro fesso r of Psychology
ACKNOWLEDGMENTS
The author w ish es to acknow ledge full coo p era tio n and
a s s i s ta n c e of D r. D arre ll S a b e rs , who not only gave much of his t im e,
but a lso he lped in the developm ent of the d es ig n and a n a ly s is of the
d a ta . Also g ra tefu lly acknow ledged are D r. W ilbur S. Ames, who gave
many helpfu l su g g e s t io n s in the d e s ig n and scoring of the c loze te s t
and D r. W illiam L. Logan, who a s s i s t e d in the f ina l w riting of th is
thesis:..
TABLE OF CONTENTS
Page
LIST OF TABLES v
ABSTRACT ....................... v i
I . INTRODUCTION ................... 1
D efin ition and Previous U se of C loze . . . . . . . . . . . . 1Problem . . . . . . . . . . . . . . . . . . . . „ . . . . . . . 3
II . RELATED LITERATURE . . . . . . . . . . . . . . . . . . . . . . 5
Type of C loze T est U sed 9S e lec tio n of the C loze P a s sa g e s . . . . . . . . . . . . . . . 10D ele tio n Procedure and Scoring . . . . . . . . . . . . . . . . 11
1. C orre la tions Among All T e s t s — KR-20 R e liab i l i t ie s onD iag o n a l, and Number of S ub jec ts Below the D iagonal . „ 14
2. C orre la tions Among All T ests C orrected for A ttenuation . . 14
3 „ C orre la tions Among All S u b tes ts of the C loze--K R -20R eliab il i t ie s on D iag o n a l, Number of Subjects = 5 1 . „ 19
v
ABSTRACT
In the p a s t the c loze t e s t has been used only to m easure
read ing com prehension and re a d a b i l i ty . This study in v e s t ig a te d the
c loze as a m easure of c o u rse -o r ie n te d ach iev em en t. The su b je c ts for
th is s tudy were from two. ed u ca t io n a l m easurem ent c l a s s e s and the c loze
p a s s a g e s were s e le c te d at random from a tex t s im ilar to th a t being u sed
by the c l a s s e s . S ign ifican t po s it iv e co rre la tio n s were ob ta ined be tw een
in s truc to r-m ade ach ievem en t t e s t s and the c loze t e s t . The Kuder-
R ichardson re l ia b il i ty of the c loze t e s t was com parable to those
ob ta ined with in s tru c to r-m ad e t e s t s .
v i
CHAPTER I
INTRODUCTION ■
D efin ition and Previous Use of C loze
The word c lo ze comes from the G e s ta l t law of c lo su re w hich
s ta te s . th a t c lo se d a reas more read ily form u n i t s . H ill (197.1, p . 100)
notes tha t n o n g es ta lt p sy ch o lo g is ts u se the term '“c lo su re " to refer to
the " se n s e of com pletion" ob ta ined when a ta sk is f in is h e d .
W ilso n L. Taylor was among the f irs t to in v e s t ig a te the c loze
p rocedure . His d e f in it io n of c loze (Taylor, 1953, p . 416) has been
accep ted by most re se a rc h e rs working w ith th is p rocedure: C lgze is
"a method of in te rcep tin g a m essag e from a 't ra n sm it te r ' (writer or
s p e a k e r ) , m utila ting i ts language pa tte rn s by d e le t in g p a r t s , and so
adm in istering it to ' r e c e iv e r s ' (readers and l is ten e rs ) th a t the ir attem pts
to make the pa tte rns whole again p o te n tia l ly y ie ld a c o n s id e rab le number
of c lo ze u n i t s . " In .i ts most common form, a c loze p a s s a g e c o n s is ts of a
s e t of paragraphs w ith every fifth word d e le te d . The s u b je c t ’s ta sk is to
f il l in the e x a c t d e le ted words by u se of con tex t c u e s .
R esearch on the c loze techn ique has been prim arily concerned
with the m easurem ent of com prehension . R esearchers u s in g the c loze in
the fie ld of reading have d esc rib ed com prehension as th a t w hich is
1
m easured by s tan d a rd ized reading t e s t s . S ta ted le s s o p e ra t io n a l ly ,
Bormuth (1968, p . 50) say s : "C om prehension ab il i ty is thought to be a
se t of g en e ra l ized k n o w le d g e -a c q u is i t io n sk i l l s w hich perm it people to
acquire and exh ib it information ga ined as a co n seq u en ce of reading
prin ted la n g u a g e ." Because a com prehension and ach ievem en t t e s t may
con ta in the same i te m s , the most important d is t in c t io n is score in te rp re
ta t io n . The score on an ach ievem ent t e s t is in te rp re ted as the amount of
knowledge a p a r t ic u la r s tuden t has been able to acquire from reading a
p a s s a g e . A com prehension t e s t sco re re p re se n ts the ab i l i ty of a s tu d en t
to acquire knowledge from w ritten language of the type rep resen ted by
tha t p a s s a g e .
Another way of looking at the re la t io n sh ip of the c loze t e s t to
com prehension and ach ievem ent is to v iew te s ts as forming a con tinuum .
A n as tas i (1972, p . 392) makes the following s ta tem en t concern ing
sp e c if ic i ty of t e s t s : "Existing te s t s of developed a b i l i t ie s may be o rdered
along a continuum in terms of s p e c if ic i ty of e x p e r ien t ia l background they
p r e s u p p o s e . " The following l i s t , ordered from s p e c if ic to genera l
ach iev em en t, exem plif ies th is continuum:
1. C o u rse -o r ie n te d achievem ent t e s t s
2. B road ly-orien ted achievem ent t e s t s
3 . V erbal-type in te l l ig en ce and ap titude te s t s
4 . N on-language and performance t e s t s
5 . "C u ltu re -fa ir" te s t s
D is tin g u ish in g be tw een ap titude and ach ievem ent t e s t s ,
A nas tas i s ta te s th a t ach ievem ent t e s t s m easure e ffec ts of learn ing under
more con tro lled and known conditions th an do ap titude t e s t s . Since th is
is s im ila r to Bormuth's d is t in c t io n b e tw een com prehension and a c h ie v e
m ent, i t is ev iden t tha t com prehension and aptitude are h ighly re la te d .
Therefore , if com prehension is roughly equ iv a len t to v e rb a l- ty p e
in te l l ig en ce t e s t s , then the c loze t e s t , as trad it io n a lly u s e d , would fa l l
a t roughly the same p lace on A n a s ta s i 's continuum .
Problem
Taylor (1957, p . 25), after in v es t ig a t in g the u se of a c loze t e s t
as a m easure of com prehension , concluded : “Although c lo ze and com pre
h en s io n t e s t s were gen era lly s im ilar in the k inds of r e su l ts they y ie ld e d ,
the two kinds of t e s t s were very d ifferen t in the c o s t , e ffo r t , and time
required for c o n s tru c t io n . The ad v an tag es seem to lie w ith c loze pro
cedure in g enera l . . . " Because the method of developm ent of an
achievem ent c loze t e s t is roughly the sam e as th a t for a com prehension
c loze t e s t , the ad v an tag es of the former should be the sam e in terms of
c o s t , effort and time required for co n s tru c t io n .
Although the c loze techn ique has been used to m easure com
p reh en sio n , it has not been used to m easure c o u rse -o r ie n te d a c h ie v e
m ent. Therefore ev id en ce for the v a l id i ty of the c loze as a m easure of
c o u rse -o r ie n te d ach ievem en t is n e e d e d . If the c loze t e s t is indeed a
4valid m easure of c o u rse -o r ie n te d a c h ie v e m e n t , then the fo llow ing
h y p o th ese s should be true:
1. The co rre la tio n s betw een in s tru c to r-m ad e ach ievem ent
t e s t s and the c loze t e s t are s ig n if ican tly g rea te r than zero .
2. The in te rc o rre la tio n s among the c loze and in s tru c to r-m ad e
t e s t s are com parab le .
3. The Kuder-R ichardson re l ia b i l i ty of the c loze t e s t is
s ig n if ic a n t ly g rea te r than ze ro .
4 . The K uder-R ichardson re l ia b il i ty of the c lo ze t e s t is
com parable to those ob ta ined from in s tru c to r-m ad e a c h ie v e
ment t e s t s .
... C orre la tions correc ted for a t ten u a tio n are often preferred for
comparing v a l id i t ie s of t e s t s (N unnally , 1967, p . 204). Therefore , w ith
the information ob ta ined from the in te rco rre la tio n and r e l ia b i l i ty a n a ly s is ,
the co rrec tion for a t ten u a tio n will be ap p l ie d . Assuming th a t the r e l ia b i l -
i t ie s ob ta ined are good e s t im a te s of the true r e l ia b i l i t i e s , the co rrec tion
for a t ten u a tio n is an e s tim a te of the co rre la t io n s th a t would be ob ta ined
if the t e s t s were perfec tly re l ia b le .
CHAPTER II
RELATED LITERATURE
Although T ay lo r 's (1956, 1957) experim ent m ost c lo se ly
approxim ates the u se of a c loze t e s t to m easure ach iev em en t, none of
the s tu d ie s thus far have u sed the c loze to d irec tly m easure c o u rse -
o rien ted ach iev em en t.
C hall (1948) has dem onstra ted th a t reading com prehension can
be in fluenced more by s p e c if ic knowledge (achievem ent) than by read ing
ab il i ty in g enera l (com prehension). After adm in iste ring a com prehension
t e s t in a s p e c if ic s u b je c t a re a , she ranked the s tu d e n ts accord ing to
sco re s on a sp e c if ic knowledge t e s t and accord ing to s c o re s on the
Stanford Reading T e s t . Grouping the upper and low er 27% in e a c h c a s e ,
she found the m eans on the com prehension t e s t d iffered s ig n if ican tly
be tw een the groups in the sp e c if ic knowledge ranking but did not d iffer
in the Stanford Reading T e s t rank ing .
In a study corre la ting c loze w ith m u lt ip le -c h o ic e com prehehsion
sco res before and af te r a w eek-long s tudy period , Taylor (1956, 1957)
found co rre la tio n s of .70 for the p re te s t and .80 for the p o s t - t e s t . The
t e s t - r e t e s t r e l ia b i l i ty of th a t c loze t e s t w a s ... 88. That s tu d y , more than
any o th e r , invo lves the c loze as a m easure of ach ievem ent b e c a u se the
s u b je c ts were a llow ed to s tudy the p a s s a g e s during a o n e -w e e k in te rv a l
and only a sam ple of th e se p a s s a g e s w as u sed for the c lo ze t e s t .
Bormuth (1962, 1963) co n s tru c ted m u lt ip le -ch o ice item s from
p a s s a g e s us ing com prehension sk i l l s including: v o c a b u la ry , f a c t ,
s e q u e n c e , c a u s a l r e la t io n sh ip , main id e a , in fe ren ces from f a c t s , and
a u th o r 's pu rp o se . The re su l ts of th a t s tudy y ie lded Cloze vs m u ltip le -
cho ice t e s t co rre la tio n s ranging from .73 to .8 3 . Bormuth a lso noted
th a t the c lo ze t e s t s were highly re l ia b le as the r e l ia b i l i t ie s ranged from
.84 to .8 8 .
U sing the same com prehension s k i l l s as Bormuth (1962, 1963)
in the m u lt ip le -c h o ic e i te m s , Rankin and C ulhane (1969) found co rre la
tions ranging from .54 to .7 1 .
Repeating h is s tudy with the same com prehension s k i l l s , Bormuth
(1969) ran a fac to r a n a ly s is on the com prehension s k i l l s . The re su l ts
y ie ld ed only one fac to r from the various com prehension s k i l l s . This
would lead ev id en ce to the notion th a t the type of . q u e s tio n included in
the m u lt ip le -ch o ice t e s t has l i t t le e f fec t upon the com prehension s c o re .
Therefore it is co n ce iv ab le tha t the type of ques tio n s a sk e d on an 1
ach ievem ent t e s t would s im ilar ly have l i t t le e ffec t on the ach ievem ent
s c o re . . -
The c loze t e s t , th e n , has been e s ta b l is h e d as an e ffec tive too l
for the m easurem ent of reading com prehension . Since C h a ll has shown
th a t ach ievem ent is co rre la ted w ith co n ven tiona l com prehension t e s t s and
7
o thers have shown th a t conven tional com prehension t e s t s are co rre la ted
w ith c lo ze t e s t s c o r e s , what rem ains is to dem onstra te the d irec t
re la t io n sh ip be tw een ach ievem ent and c lo ze t e s t s c o re s . E s tab lish in g
th is d ire c t r e la t io n sh ip is the su b je c t of the p resen t in v e s t ig a t io n .
CHAPTER III
METHOD
The su b je c ts for th is experim ent were ob ta ined from an
ed u ca tio n a l m easurem ent c la s s taught at The U n ivers ity of Arizona.
These include two c l a s s e s , spring 197.3 (N = 37) and f ir s t summer,
s e s s io n 1973 (N - 3 0 ) , The seq u en ce of t e s t s w as: (1) midterm, (2)
p rac tice w ith a c loze t e s t , (3) c loze t e s t , (4) exam over te x t used in
c la s s (book exam ), and (5) f ina l exam . All t e s t s were adm in iste red w ith in
a th re e -w e e k in te rv a l .
Because the s tu d en ts were only required to take the book and
fina l exam , not a ll the su b je c ts took a ll of the o ther t e s t s . Of the 67
s tu d en ts who took the book and fina l exam , 51 took the c loze t e s t and
60 took the midterm exam .
The midterm and fina l exams e a c h c o n s is te d of a 75-item
m u lt ip le -ch o ice t e s t , and the book exam c o n s is te d o f a 60-item m u lt ip le -
cho ice t e s t , a l l developed by the in s tru c to r . The bulk of the items in
th e se exams were in fe ren tia l and problem so lv ing w ith the remaining being
l i t e r a l . The book exam tended to have more l i te ra l item s than e ither of
the o ther ex am s , i ts co n ten ts inc luded m ateria l covered in the tex t u sed
8
in c l a s s . The midterm and fina l exams were p a ra l le l forms concerning
m ateria l inc luded during the lec tu re as w ell as in the t e x t .
Type of C loze T est U sed
The d e le t io n schem e ch o sen for th is study is the same as tha t
u sed by the m ajority of re se a rc h e rs u s ing the c loze p ro c e d u re .
M acG in itie (1961) has show n tha t d e le t io n schem es of every 6th, 12th,
or 14th word are eq ua lly d iff icu lt to re s to re in a c loze p a s sa g e w hereas
an every 3rd word d e le t io n schem e is more d if f ic u l t . This is important,
b e c a u se more d iff icu lt p a s s a g e s re su l t in t e s t anx ie ty , w hich according
to Brown (1970, p . 59), y ie ld s u n re liab le r e s u l t s . Therefore a good
compromise be tw een d iff icu lty due to high frequency of d e le t io n on one
hand and the advantage of high frequency of da ta on the o ther hand is
found when the ty p ica l d e le t io n schem e of every 5th word is u se d .
The time a llo ted for the c loze t e s t was equa l to th a t a l lo tted for
o ther ex am s. This a l lo t ted time was the lim iting fac to r in determ ining
the c loze t e s t len g th . The experim enter found tha t five p a s s a g e s e ach
lim ited to 20 to 35 items (deletions) can be com pleted w ith in the hour
a l lo t te d .
A random s e le c t io n of m ateria l for c loze used was chosen over
in ten t io n a l s e le c t io n s in ce in ten tiona l s e le c t io n , involv ing scann ing
and ev a lu a tin g large numbers of p o s s ib le p a s s a g e s , is very su b jec tiv e
and time consum ing .
10
The c lo ze t e s t used in th is study was c o n s tru c ted from
p a s s a g e s s e le c te d at random from a tex t s im ilar to th a t be ing used by
the s tu d e n t s . The re a so n for th is was to avoid s p e c if ic r e c a l l on the
part of the s u b j e c t s . The book u sed by the s tu d en ts w as P sycho log ica l
Testing by Anne A n as tas i (1972) „ The book u sed for c lo ze s e le c t io n s
was P rinc ip les of Educationa l and P sycho log ica l T es ting by Frederic G .
Brown (1970) „
S e lec t io n of the C loze P a ssa g e s
Based upon the knowledge of the con ten t of the in s tru c to r-m ad e
t e s t s , the following con ten t a reas were sam pled for c lo ze t e s t d ev e lo p
ment:
1. Score in te rp re ta t io n
2. V alidity
3. R eliab ility
4. Item a n a ly s is
5. T ests in genera l (including s p e c if ic t e s t s such as IQ,
ach ievem en t, p e rso n a l i ty , e t c . ) - '
S e lec tio n of th e se co n ten t a reas was accom plished by the following
ru les :
1. C hap te rs d ea lin g with con ten t a reas were noted from
tab le of c o n te n t s .
2. These chap te rs were then opened a t random.
11
3. The f irs t com plete paragraph(s) on the right hand page was
s e le c te d which does (is) not:
A. have le s s than 100 or more than 175 words be tw een
f irs t and l a s t s e n te n ce (If there was more than one
paragraph needed to meet th is c rite rion then the para
graphs must be in s u c c e s s i o n . ) ,
B. refer to a previous s e c t io n in the te x t ,
C . refer to a figure or p l a t e ,
D . part of an enum eration o r . l i s t in g .
4. If any of the above s i tu a t io n s a p p l ie d , the paragraph
im m ediately af te r th is cho ice w as exam ined . If there was
s t i l l a p rob lem , the paragraph preceding th is cho ice was
exam ined , and a con tinua l s e a rc h was made in th is a l te rn a t
ing manner un ti l a paragraph(s) was found th a t met the
above c r i te r ia . >
D ele tion Procedure and Scoring
Beginning with the second se n te n c e in each p a s s a g e , every
fifth word w as d e le te d un til the l a s t s e n te n c e was re a c h e d . The f irs t
and l a s t s e n te n c e s were kept in tac t and the word s e le c te d for f irs t
d e le t io n was determ ined random ly. Follow ing a method s e t by Taylor
(1957, p . 21), counted words inc luded " . . . any group of language
symbols ( le t te r s , f ig u re s , and re la ted punctuation) s e p a ra te d from
12
neighboring groups by empty s p a c e s , . . Punctuation su ch as
h y p h en s , apos trophes and commas in numbers were a lso su b je c t to
d e le t io n . The d e le ted words were rep laced by underlined b lanks of
equal length which were never s p l i t from the end of one line to the
beginning of ano ther . The t e s t w as th en mimeographed and c o lla ted in
the following order of co n ten t ca te g o r ie s : score in te rp re ta t io n , v a l id i ty ,
t e s t s in g e n e ra l , re l ia b i l i ty , and item a n a ly s is .
In scoring the c loze t e s t , only the e x ac t word w as scored
c o r re c t . Thus if the su b je c t responded " te s t" in s te a d of " te s t s " his
response w as marked wrong. This e lim ina ted the e f fec t of su b jec tiv e
sco r in g .
CHAPTER IV
RESULTS
The re su l ts of the in te rco rre la tio n and K uder-R ichardson
re l ia b il i ty a n a ly se s appear in Table 1. The co rre la tions be tw een the
c loze t e s t and book, midterm and fina l exams were .5 0 , .52 and .6 0 ,
r e sp e c t iv e ly and each is s ig n if ican tly g rea te r than z e ro . The Kuder-
R ichardson r e l ia b i l i t ie s ob ta ined range from .75 to .86 and e a c h is
s ig n if ican tly g rea te r than ze ro .
D ependent F isher Z te s t s showed th a t the m agnitudes of the
in te rco rre la tio n s involving the c loze t e s t are a ll c o m p arab le . T ests of
s ig n if ican ce be tw een re l ia b il i ty co e ff ic ien ts showed th a t e a c h Kuder-
Richardson re l ia b il i ty ob ta ined is com parable to th a t ob ta ined for the
c loze ex cep t in the c a se of the book exam (Feldt, 1969). ̂ The
re l ia b il i ty of the c loze t e s t is s ig n if ican tly (P<(.05) g rea te r than that of
the book exam .
Table 2 show s the in te rco rre la tio n s ob ta ined when the co rrec tion
for a t ten u a tio n is a p p l ie d .. These v a lu e s of r be tw een the c loze and o ther
t e s t s range from .62 to .7 0 . The fac t th a t th e se v a lu e s are not c lo se r
•'•In the p re se n t c a s e , with hypotheses, = p ^ , the s ta t i s t i c reduced to: W = (1 - ^ / ( l - r^ ) , w hich is d is tr ib u ted approxim ately as F w ith (n2 - 1) and (n^ - 1) degrees of freedom (n = number of s u b je c t s ) .
13
Table 1. C orre la tions Among All T e s t s — KR-20 R e liab il i t ie s on D iagona l, and Number of Subjects Below the D iagonal
C loze Book Midterm Final
C loze (.86*) .50* .52* .60*
Book 51 (.75*) .73* .68*
M idterm 49 60 (.83*) .63*
Final 51 67 60 , (.84*)
*P< .01
Table 2. C orre la tions Among All T es ts C orrected for A ttenuation
C loze Book Midterm Final
C loze .62 .62 .70
Book .92 .85
Midterm .76
15
to o n e , in the c a s e s involving the c lo ze t e s t , is an in d ica tio n tha t the
c loze t e s t is m easuring som ething d iffe ren t than e a c h of the o ther t e s t s .
CHAPTER V
DISCUSSION
Validity
The co rre la tio n s be tw een the c loze and o ther t e s t s are som e
what low er in th is s tudy than those reported by Taylor (1956, 1957),
Bormuth (1962, 1963) and Rankin and Culhane (1969) for com prehension
vs c lo se s c o r e s «, This is unders tan d ab le in v iew of the fac t th a t the
s u b je c ts in th is s tudy never read the in tac t c loze p a s s a g e s . However,
the lower co rre la tio n s found in th is s tudy may mean tha t the c loze t e s t
m easu res com prehension but not c o u rse -o r ie n te d ach iev em en t.
An im portant part of e s ta b l is h in g the v a l id i ty of a t e s t is in
noting the v a l id i ty of the c r ite rion m e a s u re s . The in s truc to r-m ade
ach ievem ent t e s t s were co n s tru c ted from a tab le of s p e c if ic a t io n s
sam pling course m ateria l p re sen ted in c la s s and the tex t u sed by the
c l a s s . In th is way the con ten t v a l id i ty of the c r ite rion m easu res has
been e s ta b l i s h e d .
The following is a l i s t of problem areas w hich may have con
tribu ted to the lower co rre la tio n s found in th is s tudy:
1. The tex t u sed to s e le c t the c lo ze p a s sa g e s may have em pha
s iz e d d ifferen t e lem ents of the content a reas s e le c te d than the
tex t u sed in c l a s s .
17
2. The tex t used to s e le c t the c loze p a s sa g e s may have varied
' in s ty le , vocabu la ry and g en era l d iff icu lty le v e l as com
pared to the te x t used in c l a s s .
3. The re la t iv e ly sm all sam ple of c loze p a s s a g e s as compared
with the en tire book lowers the con ten t v a l id i ty of the c loze
t e s t .
4 . The leng th of time betw een the c lo z e , b o o k , midterm , and
f ina l exam s w as su ff ic ien t for some learn ing to take p la c e .
This is p a r t ic u la r ly a problem when s tu d en ts cram for ex am s .
5 . The inheren t assum ption w as made tha t the s tu d e n ts were
equa lly m otivated to perform on each exam .
In re fe rence to point number tw o, it would be in te re s t in g to
know the re la t io n sh ip be tw een d iff icu lty lev e l and in te rco rre la t io n s
betw een c loze and conven tiona l ach ievem ent t e s t s . If the c loze t e s t is
too e a s y or too hard th is would r e su l t in the lowering of the to ta l v a r i
ance w ith a correspond ing lowering of the c o r re la t io n s .
For the pu rposes of future com parison , the m ean , s tandard
d e v ia t io n , and d iff icu lty lev e l of th is c loze t e s t will be no ted h ere . The
mean and s tandard dev ia tio n of the co rrec t re sp o n se s o b ta ined for the
p resen t c lo ze t e s t are 63 .4 (47.3%) and 11 .3 (8.4% ), r e sp e c t iv e ly .
Rankin and C ulhane (1969) have rep ea ted Bormuth's experim ents
e s ta b l is h in g the read ab il i ty le v e ls of c loze p a s s a g e s . Their re su l ts a re ;
Percent C loze PercentItems C orrec t C om prehension
41 75
61 90
The 75% and 90% com prehension sco re s correspond to the in s tru c t io n a l
and independen t reading le v e ls u se d in read ing te s t s . Of the to ta l of 51
s tu d en ts who took the c lo ze t e s t in th is s tu d y , s ix rece iv ed to ta l c lo ze
sco re s l e s s than 41% and only one rece iv ed a score over 61%. There
fore , accord ing to the Rankin and C ulhane c r i te r ia r the d iff icu lty le v e l o f
the c loze p a s s a g e s could be c a l le d in s tru c t io n a l for the m ajority of the
su b je c ts in th is s tu d y . -
R eliab ility
The re l ia b i l i ty of the c loze te s t in th is s tudy is com parable to
those ob ta ined by Taylor (1957) and Bormuth (1962, 1963). Table 3 show s
in te rco rre la tio n s among the s u b te s ts of the c loze and the Kuder-
R ichardson re l ia b il i ty of e a c h . Although the number of items per s u b te s t
is sm all (20-35), a l l ex cep t the con ten t a rea of r e l ia b i l i ty have Kuder-
R ichardson r e l ia b i l i t ie s s ig n if ican tly g rea te r than ze ro . The s iz e of the
in te rco rre la tio n s are roughly tha t ex p ec ted in v iew of the re l ia b i l i t ie s
ob ta ined ex cep t in the c a s e of the re l ia b i l i ty s u b te s t . Although the
re l ia b i l i ty of th is s u b te s t is low , it was included in, the a n a ly s is to
in c re a se the con ten t v a l id i ty of the en tire c loze t e s t .
19
Table 3 . C orre la tions Among All S u b te s ts of the C loze--K R -20 R e liab il i t ie s on D iagona l, Number of S ub jec ts = 5 1
Number of Item s: 35
ScoreIn te rp re ta tio n
25V alidity
28T ests in G eneral
26
R eliab ili ty
20Item
A nalysis
ScoreIn te rp re ta tion (.67*) .42* .47* .31* .58*
V alidity (.68*) .52* .44* .50*
T e s ts in G eneral (.55*) .50* .68*
R eliab ility (.33) .54*
Item A nalysis (.78*)
*P< .01
20
S ugges tions for Further Study
It is th is exper im en te r 's opinion th a t the p re se n t techn ique of
co rre la ting sco re s on com prehensive ach ievem ent t e s t s w ith a c loze
t e s t over the same large con ten t area le a v e s much to be d e s ire d due to
the re la t iv e ly sm all con ten t sam ple inc luded in the c loze s e le c t io n . In
th is w ay , the c loze t e s t is somew hat like a sp e c if ic e s s a y t e s t over a
large con ten t a re a . Therefore a techn ique of co rre la ting more sp e c if ic .
con ten t a reas w ith one ano ther should prove more s a t i s f a c to ry . The
u se fu ln e s s of the c lo ze techn ique as a form ative (d iagnostic) tool in
more sp e c if ic co n ten t a reas of a course could a lso be e s ta b l i s h e d by th is
tec h n iq u e . A more sp e c if ic c r ite rion t e s t could e a s i ly be cons truc ted
by the s e le c t io n of item s of a sp e c if ic con ten t area from a ready-m ade
ach ievem ent t e s t and in c lu s io n of only th e se items on the c r ite rion t e s t .
As for the c loze t e s t , Bormuth (1965) has shown th a t c lo ze t e s t s of le s s
than 50 item s tend to be u n re l ia b le , (The p resen t s tudy has shown tha t
th is is not a lw ays the c a s e b e c a u se a re l ia b i l i ty c o e ff ic ien t of .78 was
ob ta ined for a 20 item s u b te s t . ) Therefore , the number of item s on both
the cr ite rion and c loze t e s t should approach 50 i te m s .
Furtherm ore, there is a confounding var iab le w hich w as uncon
tro lled in th is s tu d y . An in d iv id u a l 's sco re on a c loze t e s t can be thought
of as a com posite of two so u rces of v a r ian ce : com prehension and p rev i
ous knowledge (co u rse -o r ien ted a c h ie v e m e n t) . Of p a r t ic u la r in te re s t for
contro l is tha t portion of com prehension w hich re p re se n ts an in d iv id u a l 's
21
sk i l l in u s in g sy n tax for word id en tif ica t io n s in ce th is is not d ire c t ly
re la ted to th a t w hich is m easured by trad it io n a l c o u rse -o r ie n te d
ach ievem ent t e s t s .
CHAPTER VI
CONCLUSION
Although the re l ia b il i ty of th is c lo ze t e s t is h igh and
com parable to th a t ob ta ined by o ther r e s e a r c h e r s , i t s co rre la tio n with
c r ite rion m easu res of ach ievem ent is not as high as d e s i r e d . Therefore
a rep lic a t io n of th is s tudy u s in g more sp e c i f ic con ten t a reas for
c r i te r ion m easu res to in c re a se the co n ten t v a l id i ty of the c loze t e s t
and c r ite rion m easu res is n e e d e d . Control of the com prehension
variab le is a lso d e s i r a b le .
22
APPENDIX A
CLOZE TEST*
Score In te rp re ta tio n
As with a ll s tanda rd score s c a l e s , any arb itrary mean and
s tandard d ev ia tio n can be u s e d . The recommended procedure is to
use ______ a mean of 50 and s tandard d ev ia tio n of 10.
H owever, norm alized s tandard sc o re s w ith o ther b a se s
can be u sed if the_________ s itu a t io n w arran ts . B ecause norm alized
s tandard sc o re s and l in ea r ly transform ed s tan d a rd •
sc o re s (Z scores) are often________reported on s c a le s with
the_________ sam e c o n s ta n ts (X = ________ 50_______, s = 10), they
c a n _________ e a s i ly be co n fu sed . I f the________ d is tr ib u tio n of
raw sco re s is______norm al, both s e ts of transform ed s c o re s
will have t h e same________ v a lu e s ; the farther the d is tr ib u tio n
of raw sco re s d e v ia te s from _______ norm ality , the g rea te r the
d isc re p a n c y in the two s e t s ______ of__________ derived s c o re s .
W hen norm alized s tandard sco res are reported on
_a______ transform ed s c a le w hich u s e d a mean of
*Perm ission to u se ex ce rp ts from tex t was ob ta ined from c o p y - write owner. F rederick G . Brown, P rinc ip les of E ducationa l and P sycholog ica l T e s t in g , Dryden P ress In c . , 1970 , pp. 59, 120, 173 , 278, 349,
23
24
50 a n d ________Q s tandard d ev ia t io n of 10, they______ are
frequently c a l le d T sco res This usage c a u s e s some
confusion as T sc o re s were o rig inally defined (M cC all,
1922) with reference to a p a r t ic u la r norm group and
not in a generic s e n s e . M oreover, some p e rso n s use
the T -sco re labe l when referring to unnorm alized s tandard score
sys tem s w hich u se the 5 0 - 1 0 b ase ■ However, as
u sed to d ay , the T -sc o re d es ig n a tio n gen era lly ap p lie s to any norm alized
s tandard score system with X = 50 and s = 10.
25
V alidity , '
In our d is c u s s io n we have assum ed tha t the poin t on the t e s t
score sc a le , that d iv ided the a c c e p ta b le (p red ic tio n -su ccessfu l) , and
u n accep tab le (p re d ic t io n -u n su c c e s s fu l) group was know n. For p u rposes
of determ ining the v a l id i ty of a t e s t__________ we could ch o o se .
an arb itrary s e r ie s of sco re s and determ ine the d e c i s io n
making accu racy at ea c h ________ p o in t . For the t e s t u se r . ,
however , a more im p o r ta n t ta sk __ may be to determ ine
the_______ __optim al cu tting po in t , the point_________ on the t e s t
(predictor) sco re tha t w ill s e p a ra te the groups________so as
to p ro d u c e _______the_____ maximum number of co rrec t d e c is io n s
In se t t in g a cu tting score______ _ and determ ining i ts
v a l id i ty , three________ s ta t i s t i c s are of p a r t ic u la r re lev an ce :
the s e le c t io n r a t io , . th e to ta l________ correc t d e c i s io n s , and the
p o s it iv e h its . The s e le c t io n r a t i o ______ i s __________ the propor
tion o f a p p lic a n ts s e le c te d ______ , i . e . , the ra tio of persons
s e le c te d to a p p l ic a n ts . A s the s e le c t io n reguirem ents
become more s tr in g en t ( i . e . , a s a higher cu tting________sco re
is u s e d ) , the s e le c t io n ra tio becom es low er. The other two terms
have been defined p rev io u s ly . .
T ests in G eneral
Although the con ten t Of s c h o la s t ic ap titude t e s t s is b a s ic a l ly
u n l im ite d , in point of fact con ten t is genera lly quite s ta n d a rd iz e d ,
which only a few item types appearing with any re g u la r i ty . Among the
most w idely _____ used item types are: vocabu la ry or a
v a r ia t io n such as an a lo g ie s or sen te n ce com pletion
(se lec tin g the proper word to com plete a sen te n c e ) : reading
com prehension ( where________ the s tuden t reads a paragraph
and then answ ers q u e s tio n s b ased upon the information
c o n ta in e d in ________ the paragraph); num erical a b i l i ty ,
te s te d through s tra ig h t com putation a n d / o r ________word
problems; t e s t s of a b s tra c t reaso n in g (such as number se r ie s
problems_____ or nonverbal an a ly q ies ) ; g enera l information .
item s; and te s t s of______ too ls______ s k i l l s such as use- o f ________
the d ic t io n a ry , map read in g , ________ and the in te rp re ta tio n of
t a b l e s ______ and_____ __g ra p h s . The e x a c t con ten t w il l_________ ,
of c o u r s e , vary w i t h the age leve l of the s tu d en ts
For e x a m p le , in the primary g ra d e s , before ch ild ren have
acguired any fluency in read in g , vocabu lary will be
te s te d by p ic ture vocabulary i te m s , but at o lder
ag es more com plex i t e m ______forms _, such as a n a lo g ie s , can
be in tro d u ced . S im ilarly , items m easuring sk il l in u s ing
27
the d ic t ionary and reading sim ple maps may be inc luded at the e lem entary
le v e l , while at the high schoo l and co lleg e lev e ls s tu d e n ts w ill be\
required to in terp re t com plex ta b le s and g rap h s .
28
R eliab ili ty
A re la ted problem is that of t e s t an x ie ty . Some peoplei
get very t e n s e , n e rv o u s , a n d u p se t a t the
p ro sp ec t of_____ tak ing a t e s t or exam ina tion . Freguentlv
th e se are people who do________ not have an ad eq u a te
unders tand ing of te s t in g procedures a n d / o r a r e _________ unsure
of th e ir own a b i l i t ie s ______. B ecause e x c e s s anx ie ty has a
de tr im en ta l e f fec t on the lev e l of perform ance and often
produces e r ra t ic p erfo rm ance , t e s t a n x i e ty _______ can
contribu te to m easurem ent error. M inor c a s e s of t e s t an x ie ty
can often be a l le v ia te d b y _______ a__________ thorough
ex p lan a tio n of the purposes of the te s t in g a n d ______ the______ _
u s e s of the t e s t s c o re s______ , and by insuring th a t the
su b je c t is fam iliar w ith the________te s t in g p ro c e d u re s . The la t te r
goal can be accom plished by e x te n s iv e p rac tice on
problems s im ila r to th o se to be found on_________
the t e s t . Both approaches d e c re a se anxie ty by reducing unknown
cond itions In more s e v e re 1 c a s e s , s p e c ia l te s t in g arrangem ents
or counse ling .m igh t be n ee d e d .
Item A nalysis
A second fundam ental index is the d isc r im in a tin g power of the
item , the a b i l i ty of the item to d if fe ren tia te betw een subgroups of
s tu d e n t s „ Id e a l ly , we would have an index of item
v a lid i ty w hich •_______showed the re la t io n o f item _______
.re sp o n se s to an e x t r a te s t c r i te r io n m easure; in p ra c t ic e , su ch
e x t e r n a l m easu res are not g en era lly a v a i la b le „ Thus
the d isc r im in a tin g power of the item is com puted
by ■________ comparing ind iv idual item r e s p o n s e s ________to
the to ta l score o n _______ the________ t e s t , b e c u a se the to ta l score_____
provides the b e s t e s t im a te j_ of the s tu d n e t 's overa ll
k n o w le d g e ______ of_________ the con ten t dom ain . N o t e tha t_________
when comparing the re sp o n se on_________an in d iv id u a l item to
the________ (total) t e s t s c o r e , w e _______ are assum ing th a t
a ll the t e s t_________item s are m easuring t h e sam e_________sk il l
or s e t of sk i l l s . Although th is a ssum ption may seem u n ju s t i
fied , the em pirica l d a ta u su a lly support th is g e n e ra l iz a t io n .
REFERENCES CITED
A n a s ta s i , Anne. P sycho log ica l T e s t in g . 3rd ed . M acM illa rd C o. ,1972.
Bormuth, John R. "Com parable C loze and M u lt ip le -C h o ic e Com prehens ion Test S c o re s ," Journal of R ead ing , Vol. X (February,1962), pp . 291-299 ,
'______. "C loze as a M easu re of R ea d ab il i ty , " P roceed ings ofthe In te rn a tio n a l Reading A s s o c ia t io n . Vol. VIII (19 63), pp . 1 3 1 -1 3 4 .. •
. "Optimum Sample Size and C loze Test Length in R e a d a b i l i ty ."Journal of Educational M e a su re m e n t , Vol. II (June 1965), p p . 111-116 .
___________ "An O pera tiona l D efin ition of Com prehension In s tru c t io n , “P sy c h o lin g u is t ic s and the Teaching of R ead in g , Boston: IRA Pre-C onven tion In s t i tu te , April 196.8, pp . 4 8 -6 0 .
_. "Factor Validity of C loze T es ts as M ea su re s of ReadingCom prehension A bility . " Reading R esearch Q u a r te r ly , Vol. IV (Spring, 1969), pp . 358-365).
Brown, F rederick G . P rinc ip les of Educationa l and P sych o lo g ica l T e s t in g , Dryden P re s s , I n c . , 19.70.
C h a l l , Jeanne S . "The In fluence of Previous Knowledge on Reading A b ili ty ," Educational R esearch B u lle t in , Vol. XXVII (January 1948), pp . 11 -20 .
F e ld t , Leonard S. "A T es t of the H ypo thes is that C h ro n b ach 's Alpha or Kuder-R ichardson C oeff ic ien t Twenty is the Same for Two T e s ts , " P sy c h o m etr ik a , Vol. XXXIV (September 19 69). pp . 363-373 .
H ill , W infred F . Learning A Survey of P sycho log ica l In te rp re ta t io n s . Revised e d . London: C hand ler Publish ing C o . , 1971.
30
31
M acG in itie ; W alte r H. "C ontex tual C o n s tra in t in English Prose P a ra g ra p h s ," The Journal of P sy c h o lo g y , Vol. LI (1961), pp. 121-130 .
M cC a ll , W . A. How to M easure in E d u c a tio n , Crowe 11-C o l l ie r and M acm illan , 1922.
N unnally , J u m C . Psycholm etric Theory . M cG raw -H ill Book C o . , 1967.
Rankin, Earl P . , J r . , and Joseph W . C u lh an e . "Com parable C loze and M u lt ip le -C h o ice Com prehension Test S c o re s ," Journal of R ead ing , Vol. XIII (December 1969), pp. 193-198 .
T a y lo r , W ilson L. "C loze Procedure: A New Tool for M easuringR ea d ab il i ty ," Journalism Q u a r te r ly , Vol. XXX (1953), pp. 415- 438.
_________ . "Recent D evelopm ents in the U se of C loze P ro ced u re ,"Journalism Q u a r te r ly , Vol. XXXIII (Winter, 1956), pp. 42 -28 .
__________. "C loze Readability Scores as Ind ices of Ind iv idual D ifferen c e s in Com prehension and A p titu d e ," Journal of Applied P sy c h o lo g y , Vol. XLI (February 1957), pp. 19 -26 .