Innovative Assessment in Large Classes Author(s): Richard W. Buchanan and Martha Rogers Source: College Teaching, Vol. 38, No. 2 (Spring, 1990), pp. 69-73 Published by: Taylor & Francis, Ltd. Stable URL: http://www.jstor.org/stable/27558399 . Accessed: 26/03/2014 11:46 Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp . JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. . Taylor & Francis, Ltd. is collaborating with JSTOR to digitize, preserve and extend access to College Teaching. http://www.jstor.org This content downloaded from 206.87.46.46 on Wed, 26 Mar 2014 11:46:45 AM All use subject to JSTOR Terms and Conditions
6
Embed
Author(s): Richard W. Buchanan and Martha Rogers Source ... Assessment in Large Classes.pdf · Innovative Assessment in Large Classes Richard W. Buchanan and Martha Rogers e would
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Innovative Assessment in Large ClassesAuthor(s): Richard W. Buchanan and Martha RogersSource: College Teaching, Vol. 38, No. 2 (Spring, 1990), pp. 69-73Published by: Taylor & Francis, Ltd.Stable URL: http://www.jstor.org/stable/27558399 .
Accessed: 26/03/2014 11:46
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp
.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].
.
Taylor & Francis, Ltd. is collaborating with JSTOR to digitize, preserve and extend access to CollegeTeaching.
http://www.jstor.org
This content downloaded from 206.87.46.46 on Wed, 26 Mar 2014 11:46:45 AMAll use subject to JSTOR Terms and Conditions
useful suggestions to solve some of the assessment prob
lems frequently encountered with large classes. 'Targe classes" will be defined
here as those with eighty students or
more. Although this definition is some
what arbitrary, it has been our experi ence that eighty students is the break
ing point where traditional teaching
techniques are no longer workable and new ones must be tried. This breaking
point is particularly noticeable in the area of assessment. We've watched
many of our colleagues struggle along with traditional approaches, such as es
say examinations, up to points where
class enrollments exceed eighty. Then
they normally collapse from overwork,
delegate assessment to lower-level as
sistants, or start looking for new ap
proaches.
This paper will show some solutions
to three problems:
1. How to offer students in large classes an opportunity to be assessed in an essay format without straining the resources available for grading
2. How to deal with students who
miss a required examination
Richard W. Buchanan is senior lecturer in marketing at the Massey University in New
Zealand. Martha Rogers is an assistant
professor of marketing at Bowling Green
State University in Bowling Green, Ohio.
w
3. How to generate large numbers
of new, relevant examination questions on a regular basis
It is useful to begin by stressing that
this paper is not, and was never intend
ed to be, an elegant scientific examina
tion of all the factors within its focus.
It is our intention to share techniques that have worked for us in sections
numbering between 50 and 350 stu
dents. One author typically teaches be
tween two and three thousand students
per year.
We have had only one graduate assis tant assigned to each of us for a period of five to ten hours per week, and thus
finding a means of dealing with mass
numbers became a matter of survival.
Virtually all of the solutions suggested
by this article were the result of trial
and-error. As such, this paper cannot
lay claim to having tested all possible solutions. In addition, although we
have kept reasonably accurate records to test the effectiveness of various solu
tions, we have made no attempt to pre sent them as anything other than ap
proximations.
Our three assessment solutions will
be presented and should be used simul
taneously, as a total system. This is in
keeping with our experience that it is
best to treat instructional design as a
system?rather than to treat individual
parts in isolation. To do otherwise of ten causes the solution to one problem to exacerbate another. Therefore, this
paper will not only relate those parts of
the system designed to deal with select
ed problems but will also mention some solutions for problems created by the new system itself.
Objective Tests?Imperfect but Unavoidable
Although people teaching large class
es often try to avoid multiple-choice/ true-false tests, we have found that
such efforts seem to be appreciated by almost no one. Although colleagues may criticize the limitations of anything other than essay tests, they usually are
willing to accept an alternative if more
than fifty students are involved. Admin
istrators may make noises about the de
sirability of essay examinations, but, in our experience, they are rarely willing to
trade the time it takes to grade them for
a lack of participation in either matters
of administration or research/publica tion. Finally, students are not nearly so
fond of them as their comments to the
contrary might suggest. For all these reasons we are assum
ing that the basis for assessment will
primarily be objective questions. This
assumption normally unleashes a storm
of student complaints to the effect that "I just don't do well on objective tests." Although this may be the case
for some, we have found that, general ly, the belief just doesn't hold true.
Through the years we have often
made it a point to offer both essay and
objective final examinations to stu
dents who have been tested up to that
Vol. 38/No. 2 69
This content downloaded from 206.87.46.46 on Wed, 26 Mar 2014 11:46:45 AMAll use subject to JSTOR Terms and Conditions
with multiple-choice than with essay tests (also see Hogan [1981]).
This rule-of-thumb, however, is not
true for all students. And, even if it were true, it will not be useful for
quieting students' objections if they think it is not true for them. For this
reason, we've found it necessary to
provide some way for students to be as
sessed in an essay format?while still
protecting ourselves against the enor
mous time investment required to eval
uate all students in this manner.
Some idea of how great a time in
vestment may be involved can be deter
mined by considering a hypothetical
example. Suppose that a more or less
standard ten-question, short-answer
test intended to be taken in fifty min
utes were to be given. Assuming that it
takes a minimum of two to three min
utes to grade each question means that
assessing each paper in the most mini
mal fashion requires a total of from
twenty to thirty minutes. Multiplying this figure by a not uncommon student
load of six hundred students produces a figure of from two hundred to three
hundred hours. Even if instructors were to spend all of their time grading papers on a forty-hour week basis, each exam would take from five to seven weeks to process.
Some might argue that this situation
could be alleviated by the use of grad ers, but this technique has problems of
its own. Among them are coordination/
management of the graders, variability among graders, and the fact that stu
dents don't normally like to have their
work assessed by someone other than
the instructor.
All of these factors argue for a solu
tion that offers students a chance to be
assessed in an essay format but that
will limit the number of students so as
sessed to reasonable numbers.
Self-Selective Essay Exams
We found that the only system that
would fit into the preceding constraints
had to be based on what many would
term a "cafeteria" approach. The phil
osophical basis of this approach (which is frequently used in structuring em
ployee benefit plans) is to offer "con
sumers" a number of options from
which they can select the combination
of items they prefer. Students are, therefore, offered the
following three options: (1) four objec tive concept tests only, (2) four objec tive concept tests and an optional final, or (3) three objective concept tests and an optional final. In options one and
three, each test is worth 25 percent of
their course grade; in option two, each
test is worth 20 percent. Those students electing to take the
optional final are told
1. their current grade prior to the fi
nal (i.e., Should they quit while they're
ahead?); 2. that the final examination can
hurt them as well as help them (i.e., a
concept test can?under some circum
stances?be dropped, but a final can
not be dropped if attempted); 3. the approximate percentage of
students taking the final examination and the fraction of these improving their grades over the years;
4. that the final examination will
consist of either a fifty-question objec tive test or a ten-question short-answer
essay?both covering the entire course; 5. that students will have to decide
prior to taking the final which version
they will attempt (i.e., they could not
look at both and decide which version was easier); and
6. that most students in the past have preferred the objective version be cause it loads their risk into small (two
points each) components rather than
large (ten-point) "hunks."
When the options are presented to
them in this manner, only 10 to 15 per
cent of the students enrolled in large courses have elected to attempt the fi
nal. Of those taking the final, no more
than 20 percent chose the essay version
?and, typically, only six or seven in a
class of three hundred students.
These numbers, though manageable
enough, have been distilled even fur
ther by a refinement of the system that was produced to meet what proved to
be a product of the authors' teaching
styles. When teaching large classes, we've found it useful to make sure that
the lectures contain enough material
not covered in the supporting text to
make it worthwhile for students to at
tend lectures. We tell the students that
this material will be both presented and
the subject of examination questions
(i.e., at least 30 percent of a test's items
will not be found in the book). Because it is generally impossible to
videotape the lectures, those students
who miss many classes have a very real
problem, although they could miss at
least one concept test without penalty. However, if the final examination cov
ers both the text and the lecture, they are still at risk for those topics covered
during their absences. For this reason
we decided to make the objective ver
sion of the final examination cover the
text only while the essay version is
drawn from both the text and lecture.
Generally, the lectures are more orient
ed to applications of knowledge than to definitions or facts, and we believe
that these applications are better tested in an essay format.
Once this refinement was made, the
percentage of students taking the final
exam remained about the same, but the
number electing the essay version has
dropped to a fraction of 1 percent.
Still, it has always been there if anyone wanted to complain about not doing well on objective tests. To the best of our knowledge, no complaints about
the unavailability of essay tests have ever been made about our large classes.
It may also be useful to know that
the percentage of students attempting the final usually falls over time, pos
sibly because the grapevine eventually
spreads the word that the final is not a
particularly soft option. At any rate, the ceiling on the people attempting it
70 COLLEGE TEACHING
This content downloaded from 206.87.46.46 on Wed, 26 Mar 2014 11:46:45 AMAll use subject to JSTOR Terms and Conditions
there are fewer than four concept tests, because administering three or fewer exams causes the amount of material to
be covered on each one to be unman
ageable. Having more than four seems
impractical because it multiplies the re
sources needed beyond a point of di
minishing returns.
The basis of this system is in direct
contrast to what seems to be an aca
demic tradition of placing relatively
greater emphasis on the final examina
tion than on others such as the concept tests. However, it is not our intention
to load most of a student's evaluation
into his or her performance on only one day of the term.
Abolishing Makeup Exams
Once the "cafeteria" style is adopt
ed, it then becomes possible to use it to
solve other problems such as makeup exams.
Having students absent from a re
quired exam is never a comfortable sit
uation. Professors dread the inconven
ience of constructing a makeup exam
and find distasteful the thought of
serving as judge, jury, and executioner in determining whether excuses are ac
ceptable. At the same time, students
don't like having their integrity ques tioned by an unpredictable, often in
sensitive system that they frequently
suspect of being punitive. These more
or less standard complaints explode in
their intensity when multiplied by the
enrollments of a large class.
Before tackling the problem of
makeup tests, we realized that 15 to 25
percent of students might be absent
from any given examination. When ap
plied to a class enrollment of 80 to 350, and multiplied by several sections, the
total number of students likely to be in
volved is beyond the scope of tradition
al methods for handling them.
The first problem is processing the
flood of individuals who show up at an
instructor's door either prior to or
shortly after an examination with their excuses for being absent. If only five
minutes is spent with each person, the
total time invested would leave little
time for doing anything else. Beyond
this, we have felt totally helpless to de
termine which excuses are truthful, jus
tified, or both.
Even if all the absentees could be ac
commodated, their sheer numbers
make it impossible to arrange a time
and place for a makeup exam that they can all attend. Finally, if a makeup test
is allowed, there is no way to make it
fair for all concerned. If anyone is al
lowed to take the test prior to the regu lar class, then someone is bound to feel
that those taking the makeup will pass
questions on to their friends. And, if a
totally different test is given as a make
up, someone will argue that it is harder
(easier) than the regular test.
At this point it would have been
tempting to surrender the entire matter
and decide to accept absolutely no ex
cuses except those that conform to uni
versity policy and are supported by ap
proved documentation (i.e., student
health center doctor's excuse, etc.).
But, common sense suggests that this
limitation would overlook some per
fectly valid situations, and this would
lead to further conflict. Although such
conflict may be permissible in smaller
class settings, it definitely is not for
large ones.
One thing that large classes teach
their instructors is never to tolerate any situation that strikes a large number of
students as unfair. Reasonable univer
sity administrators are used to discard
ing the opinions of what they may per ceive as a handful of disgruntled stu
dents. They are much more likely to
take action if fifty or a hundred gather outside their door.
After considering all the problems associated with makeup exams, we de
cided to offer the students the option
(previously discussed) of being assessed on the basis of three concept tests and an optional final examination that be comes mandatory if a student misses one of the concept tests. At the time
the students are informed of this op tion they are also told that
1. they do not need to inform the in
structor or get permission to miss a
test;
2. by taking this option, they are
also giving up the ability to drop a low
test (i.e., what really is happening is
that students are given the ability to
drop a low test score?either a bad per formance on a test taken or no per
formance on one they missed); 3. if they miss another test or the
final they will fail the course; and, most important,
4. no makeups will be given for any reason to anyone.
Besides all this they are also told the
specifics of the final examination, which have already been introduced in a preceding section.
This system has had remarkable re
sults. Only a handful of students come
to the office door each year to ask
about the possibility of a makeup. Be
yond this, the percentage of students
electing to miss any given concept test
has averaged around 5 perccent of
those enrolled. And we are relatively certain that any who do miss a test
under these circumstances have reasons
that they think are justifiable. Limitations of this part of the system
should be mentioned. Most important, when a student misses an exam, that
student has not been assessed on a sig nificant percentage of course material.
Although we have not yet tried it, one
solution to this drawback would be to
give more weight on the final exam to
those items assessing the material on
the missed exam. This weighting would
be procedurally simple. Each student
taking the final exam will do so either
voluntarily as a fifth exam or to make
up for a missed exam. The student's
record will reveal which is the case, and, if the latter, which exam was missed. It
is then a relatively simple matter to
weight the items from the missed ex
ams more heavily.
Additionally, in a few rare cases, a
student has tried to test the system either by challenging it or by missing two examinations. In the first cate
gory, an entire hockey team had their
coach call, first, a department chair, and then the dean, trying to get an ex
cused absence. These matters were eas
ily dealt with as soon as both the spe
cifics, the rationale behind the system.
Vol. 38/No. 2 71
This content downloaded from 206.87.46.46 on Wed, 26 Mar 2014 11:46:45 AMAll use subject to JSTOR Terms and Conditions