Brigham Young University Brigham Young University BYU ScholarsArchive BYU ScholarsArchive Theses and Dissertations 2005-12-19 Delayed Versus Immediate Feedback in an Independent Study Delayed Versus Immediate Feedback in an Independent Study High School Setting High School Setting Duane C. Lemley Brigham Young University - Provo Follow this and additional works at: https://scholarsarchive.byu.edu/etd Part of the Educational Psychology Commons BYU ScholarsArchive Citation BYU ScholarsArchive Citation Lemley, Duane C., "Delayed Versus Immediate Feedback in an Independent Study High School Setting" (2005). Theses and Dissertations. 336. https://scholarsarchive.byu.edu/etd/336 This Dissertation is brought to you for free and open access by BYU ScholarsArchive. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of BYU ScholarsArchive. For more information, please contact [email protected], [email protected].
67
Embed
Delayed Versus Immediate Feedback in an Independent Study ...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Brigham Young University Brigham Young University
BYU ScholarsArchive BYU ScholarsArchive
Theses and Dissertations
2005-12-19
Delayed Versus Immediate Feedback in an Independent Study Delayed Versus Immediate Feedback in an Independent Study
High School Setting High School Setting
Duane C. Lemley Brigham Young University - Provo
Follow this and additional works at: https://scholarsarchive.byu.edu/etd
Part of the Educational Psychology Commons
BYU ScholarsArchive Citation BYU ScholarsArchive Citation Lemley, Duane C., "Delayed Versus Immediate Feedback in an Independent Study High School Setting" (2005). Theses and Dissertations. 336. https://scholarsarchive.byu.edu/etd/336
This Dissertation is brought to you for free and open access by BYU ScholarsArchive. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of BYU ScholarsArchive. For more information, please contact [email protected], [email protected].
This dissertation has been read by each member of the following graduate committee and by majority vote has been found to be satisfactory.
_________________________ _________________________________ Date Richard R. Sudweeks, Chair _________________________ _________________________________ Date J. Olin Campbell _________________________ _________________________________ Date Steven C. Yanchar _________________________ _________________________________ Date R. Dwight Laws __________________________ ________________________________ Date Scott L. Howell
BRIGHAM YOUNG UNIVERSITY
As chair of the candidate’s graduate committee, I have read the dissertation of Duane Charles Lemley in its final format and have found that (1) its format, citations, and bibliographical style are consistent and acceptable and fulfill university and department style requirements; (2) its illustrative materials including figures, tables, and charts are in place; and (3) the final manuscript is satisfactory to the graduate committee and is ready for submission to the university library. _________________________ ______________________________________ Date Richard R. Sudweeks Chair, Graduate Committee Accepted for the Department _________________________ ______________________________________ Date Andrew S. Gibbons
Chair, Instructional Psychology and Technology Accepted for the College _________________________ _______________________________ Date K. Richard Young Dean, McKay School of Education
ABSTRACT
DELAYED VERSUS IMMEDIATE FEEDBACK IN AN
INDEPENDENT STUDY HIGH SCHOOL SETTING
Duane Charles Lemley
Department of Instructional Psychology and Technology
Doctor of Philosophy
Although there is general agreement that feedback plays an important role in
student performance, the majority of the studies found in the research literature explore
the impact of different types of feedback in a traditional and university-level setting. In
order to explore the impact of different feedback types in a non-traditional distance
learning setting, 352 high school students enrolled in courses offered through BYU’s
Independent Study (IS) department received either delayed feedback or immediate
feedback generated by Speedback™, BYU’s automated grading and feedback program,
depending on whether they had opted to submit end of unit assignments by mail or
computer.
Results of a comparison of final exam scores indicated that those students
receiving immediate feedback performed significantly better on course final exams, but
surprisingly those who received delayed feedback completed course in significantly less
time.
ACKNOWLEDGMENTS
There are always too many to thank in the space allotted, but I wanted to express my gratitude to the members of my committee, Olin Campbell, Steven Yanchar, Dwight Laws, and Scott Howell, for their time, encouragement, and suggestions over the past months. This has been a learning experience for me in many ways. I need to especially acknowledge Dr. Richard Sudweeks who was not only willing to serve as my chair, but who also buoyed me up and kept me going. I am grateful too for friends and coworkers in BYU Continuing Education who were always encouraging. I especially need to mention Richard Eddy who never missed the opportunity to ask how “it” was going, Ellen Allred who was willing to read through and offer suggestions on this manuscript, and Carol Salmon who encouraged me to begin this process in the first place. A special thank you goes to my best friend and eternal companion, Betty, and our children, Becky, Danny, Kevin, Katy, Matthew, David, Benjamin, and Andrew. They were called upon too often to give up time with Dad so a class could be taken or a project completed. Finally, I need to acknowledge a kind Heavenly Father who always left a window open…
vii
Table of Contents
Page
Abstract ............................................................................................................................... v
Acknowledgments.............................................................................................................. vi
List of Tables ..................................................................................................................... ix
List of Figures ..................................................................................................................... x
Appendix A: Sample of Speedback™ Assignment Questions ......................................... 53
Appendix B: Example of Feedback Provided to Immediate and Delayed Feedback Groups............................................................................................................................... 55
ix
List of Tables
Table Page
1 Group Breakdown by Gender ………………………………………………………21
2 Group Breakdown by Age ………………………………………………………….22
3 Mean Score on Final Examination by Course of Study and Type of Feedback ……28
4 Mean Course Completion Time by Course of Study and Type of Feedback ………30
5 Mean Score on Final Examination by Course of Study and Type of Course and Feedback ……………………………………………………………………………35 6 Mean Completion Time by Course of Study and Type of Course and Feedback ….39
x
List of Figures
Figure Page 1 Representation of a Posttest Only Control Group Design ......................................... 24
1
Chapter 1
Introduction
Educators today are in general agreement with Chickering and Gamson (1991)
who asserted, as one of their seven principles for good practice in undergraduate
education, that one good practice to enhance student learning is to provide prompt
feedback to students. These educators perceived that appropriate feedback received
during a learning experience helps to keep a student on track by offering suggestions for
improvement in areas where he or she is not performing well. A student also finds such
information useful when plotting an individual study strategy. Even at the completion of
a course, feedback can indicate to a student how much has been learned as well as what
areas of knowledge still need to be pursued.
From an independent study perspective, it is unfortunate that the majority of the
research literature in existence today reflects studies of the impact of different types of
feedback in a typical—and usually university-level—classroom setting. While these
studies support the view that feedback plays an important role in a student’s performance,
they ignore the role of feedback to the non-traditional independent learner—especially
those on the high school level.
Students who participate in a traditional classroom course typically work together
in a continuing group or cohort. Each member of the class receives the same instruction
in the same format at the same time from the same instructor. The group moves as a unit
from the beginning of the instructional program to the end. On the other hand, students
who participate in distance learning programs are typically not members of a traditional
educational cohort. Each of these students potentially receives his or her instruction by
2
means of a different format such as by paper, over the internet, by cable television
broadcast, and so forth. These students also typically receive their instruction at a
location that is located away from the physical institutions that provide it. These distance
learners are often referred to as independent study students as that course instruction they
receive is generally delivered to a single student who functions as an independent learner
who typically completes lessons and assignments at his or her own pace within a
specified course completion deadline.
In addition, it is often the case that the feedback received following a graded
assignment or exam is the only consistent communication a distance learning or
independent study student might have as he or she progresses through a course. This
emphasizes the importance that feedback plays in an independent study setting and
suggests that the impacts of different types of feedback need to be examined in light of
that setting.
Independent Study at Brigham Young University
Beginning in the mid-1990s, the Independent Study (IS) department at Brigham
Young University (BYU) began to implement an automated grading and feedback
program known as Speedback™ in their distance learning courses. BYU’s Division of
Continuing Education provided the following description of Speedback™ in their 1994-
95 annual report (Henstrom & Oaks, 1997):
Speedback™ allows Independent Study to take advantage of current
computer technology to provide students a high-quality education at a distance
with the fastest turn-around time possible.
3
In courses that use Speedback™ technology, students provide written
responses to assignments on a Speedback™ form which is submitted to
Independent Study for grading. On the day that it arrives at Independent Study,
the Speedback™ form is scanned into a computer which grades the answers and
generates personalized feedback. The feedback on the lesson is mailed to the
student the same day (p. 196).
As described above, one of the key features of Speedback™ is the quick turn-
around of graded assignments. Traditionally, turn-around time for assignments varied
from a few days to several weeks for students enrolled in a paper-based course. Today,
however, because of the availability of the internet and e-mail capabilities, students
enrolled in either a traditional paper-based or newer web-based course option are able to
receive an immediate response on their submitted assignments. Students who enroll in a
paper-based version of a course continue to have the option to both submit and receive
graded assignments through the mail. By selecting one method of submission (paper vs.
computer) over another, a student creates an immediate versus delayed feedback setting.
Students submitting assignments and unit exams by computer receive immediate
feedback. Those who choose to submit assignments by mail receive delayed feedback if
for no other reason than the required turn-around time in the mail.
In addition to the quick turn-around feature, Speedback™ also includes several
instructional aid features for the student, including an automated feedback program that
responds to each question on Speedback™ graded assignments and exams. The feedback
responses are identical for both the paper and computer-based courses. These responses,
drawn from a pre-prepared database, provide elaborative feedback that indicates whether
4
a response is right or wrong and offers a hint or even further instruction as to why an
incorrect answer selection was incorrect. Feedback responses in this study are distractor
specific and provide a relevant comment for every possible answer as opposed to item
specific feedback that provides a general comment for each question.
Statement of the Problem Three factors define the problem this study addresses: First, independent study
students are not traditional students. Rather than working together in cohorts, they work
independently and at their own pace. Second, as a result, feedback should be especially
valuable to independent study students as it is often the only communication they have
with the course instructor—albeit asynchronous communication. Feedback helps these
students assess their progress through a course and to develop a personal study plan.
Third, since independent study students need feedback in a usable format, it is important
to understand how different types of feedback impact student performance in an
independent study setting. Available research focuses on the impact of feedback in
traditional university-level classroom settings rather than on independent study or
distance learning settings. It is especially lacking for those independent study settings
involving high school students.
The quick turn-around element of Speedback™ has greatly improved the capacity
of student service efforts at BYU Independent Study by allowing for a faster response to
students who have submitted assignments for grading. Use of computerized automated
grading has reduced instructor time by eliminating the need to grade and respond to each
assignment by hand. Corresponding costs for this task have also been reduced by
eliminating the need for teaching assistants and postage required in mailing assignments
5
back and forth between instructors and the Independent Study department. At the same
time, student satisfaction has increased because they receive a timely response to
assignments they have submitted. However, the impact of Speedback™ and, in particular,
of the automated feedback feature, on student performance in an independent study
setting has not been examined.
Statement of Purpose
The purpose of this study was to evaluate the impact of both immediate and
delayed feedback on the performance of students enrolled in high school-level
independent study programs at BYU. This was accomplished in two ways. First, the
relation of both types of feedback to course final exam scores was examined. Second, the
relation of both types of feedback to the amount of time required to complete a course
was compared.
Research Hypotheses
The study focused on the following hypotheses:
1. Students receiving immediate feedback will obtain higher mean scores on the
course final exam than students who receive delayed feedback.
2. The elapsed time between submission of the first course assignment and course
completion will be significantly less for students who receive immediate feedback than
for those who receive delayed feedback.
Organization of the Study
Chapter 1: Introduction. The current chapter contains the rationale and
explanation of the study, Statement of Purpose, and Research Hypotheses. An
explanation of the organization of the study and a summary statement are also included.
6
Chapter 2: Review of Literature. This chapter reviews the current literature
comparing studies that discuss the different types of feedback and how it is used in
educational settings. It is important to note that the majority of these studies are placed in
a traditional university-level classroom. The unique distance learning and independent
study settings are by and large not explored.
Chapter 3: Method. This chapter discusses the method used for the study. A
description of the study sample population, design, instrumentation, analysis used, and
procedure is included.
Chapter 4: Results. This chapter reports the results of the statistical analyses used
in the study.
Chapter 5: Discussion and Conclusion. This chapter discusses the ramifications
of the study’s findings. A discussion of the study’s limitations and resulting questions for
further research are also included.
Summary
In seeking to test the hypotheses outlined above, this study may provide designers
and instructors of high school independent study courses with valuable information as to
what type of feedback might be most effective in an independent study setting. If
providing feedback really does enhance student learning as has been previously
mentioned, it would also seem to be a better practice to make feedback responses as
timely and available to the student as possible. As a quasi-experimental study, however,
this research does not lead directly to causal claims, but does support continuing research
and innovation in the hope of contributing to improved practice.
7
Chapter 2
Review of Literature
This chapter reviews the available literature as it pertains to the use of feedback in
education. It describes what feedback is as well as how it might best be used to enhance
student learning. In addition, feedback will be discussed in terms of what information it
provides the student and when it is provided. Finally, other considerations that impact the
effectiveness of feedback such as applications of its use and student motivation will be
discussed.
Feedback: What It Is
In an instructional setting, the term feedback describes one of the many
procedures that inform a learner whether a response is right or wrong. In addition,
feedback can be used to provide instructional information to the student that explains
why a response was correct or incorrect (Morey, 1992).
Feedback is most effective when it provides a basis for correcting mistakes or
misconceptions a student may have gained during instruction, and does not appear to
have much of an impact as a reinforcing tool (Clariana, 1993; Cohen, 1985). Researchers
argue, for example, that in order for feedback to have an impact as a reinforcing agent,
the likelihood of repeating correct responses on future review exercises or exams would
be higher when feedback is included with initial lesson or unit assignments than it would
in a no-feedback setting. However data demonstrates that the probability of repeating an
initial correct response on a delayed final exam is high whether or not feedback was
given to the student. From this perspective, feedback plays its most important role in
In addition, Dempsey and Driscoll (1996) and Stock, Kulhavey, Pridewater, and
Krug (1992) demonstrated that students spend more time reviewing feedback when their
18
initial response was incorrect than when their answer was correct. Time spent in
reviewing feedback was found to be related to a student’s confidence in his or her
answer. The more confident a student felt that his or her initial answer was correct, the
more time he or she spent in reviewing feedback when the answer was found to be
incorrect. Conversely, if a student was confident in his or her original answer, attention to
feedback was found to decrease when the initial response was, in fact, correct. A
proposed explanation was that learners try to reduce the discrepancy between what they
think they know and what they know. In other words, students typically spend more time
trying to figure out what went wrong when they discover that an answer they felt sure
was correct was actually incorrect.
Another aspect of how a learner uses feedback tends to neutralize the argument
between the immediate feedback and delayed feedback groups. For example, Webb,
Stock, and McCarthy (1994) reported that the results of their studies involving learner
response confidence indicated that delayed feedback was superior to immediate feedback
in allowing a greater probability for more correct responses on a delayed final exam.
However, they also learned that this may not have been due to the likelihood of either
remembering or forgetting an initial response, but rather because of the tendency on the
part of the learners in the delayed feedback group to study the feedback for longer
periods of time. These students simply may have taken the opportunity to study for a
sufficient period of time to learn the material well enough to be able to identify the right
answer on the final exam regardless of their performance on earlier unit tests.
19
Summary
While there appears to be a general agreement that following up graded
assignments and exams with feedback of any type facilitates learning, how best to do so
remains debatable. Proponents of immediate and delayed feedback have both presented
evidence suggesting which type of feedback might produce better student results in the
traditional classroom. The advent of distance education, however, has brought a new
perspective to the old argument. With its necessarily innovative methods of instructional
delivery, proponents of distance learning make the point that feedback implementation
must be tailored to the needs and characteristics of the delivery format as well as the
individual and independent learner (Keenan & Langer, 1993; Morrison, et al., 1995).
As pointed out in Chapter 1, a critical element in any independent study or
distance learning course is that the student typically works alone outside of the traditional
educational cohort setting. Thus individual student motivation in the use of feedback
would seem to be an even more vital factor in its effectiveness in the learning experience
for independent learners. For this reason, I propose that immediate feedback is the most
effective type of feedback in an independent study setting if, for no other reason, than the
fact that the feedback is readily available for student use. The following chapter outlines
the methods and procedures that were used in attempting to substantiate this hypothesis.
20
Chapter 3
Method
This was a retrospective study based on archival data maintained by the
Independent Study department at Brigham Young University. This study used a quasi-
experimental design to examine the impact of immediate and delayed feedback on high
school independent study course final exam grades and the amount of time required by a
student to complete a high school independent study course.
Sample
The sample (n = 352) for this study was taken from the set of students enrolled in
the following high school-level courses offered through the BYU Independent Study
program:
1. Character Education: Exploring Values 1 (XPLR 041)
2. Twelfth-Grade English I (ENGL 055)
3. United States History from 1851 (HIST 043)
4. Basic Health: Choose to be Healthy (HLTH 041)
These courses were selected because they reflected higher enrollment numbers and
therefore offered a larger sample.
Students who completed these courses between January and July, 2005 were
divided into two groups. The first group consisted of all those students enrolled in the
paper versions of the courses who elected to submit and receive graded assignments by
mail. The second group consisted of an equal number of students who were identified
from a larger population of students enrolled in the web versions of the courses using a
simple random selection process. The students in this second group submitted and
21
received graded assignments by computer. Assignments for both groups were identical
and graded using the BYU Speedback™ program. Identical feedback responses were also
generated by Speedback™ and returned to each student via the selected delivery method.
Students making up both groups were drawn from a larger population of students
enrolled in BYU high school independent study courses. The Independent Study program
at BYU utilizes an open enrollment system which allows anyone who desires to enroll in
a course to do so. There are no requirements as to GPA, class standing, or other academic
qualification though students are encouraged to counsel with an academic advisor if they
are planning on using a course to meet graduation requirements.
Registration information that students are asked to provide includes only that
which is necessary to identify a student and provide requested course materials such as
name, address, and birth date. No ethnic or religious preference information is requested.
A breakdown of the students included in each group by gender is shown in
Table 1. Both groups were evenly matched in terms of gender with 47% of the
immediate feedback group and 48% of the delayed feedback group being male, while
53% and 52% of the respective groups were female.
Table 1 Group Breakdown by Gender
Immediate Feedback Group
Delayed Feedback Group
Gender
Number
% of Total Number
% of Total
Male
83
47%
84
48%
Female
93
53% 92
52%
Total
176
100%
176
100%
22
Students were also fairly evenly matched in terms of age. A summary of the
breakdown of the students included in the study by age is included in Table 2. The
majority of the students in both groups indicated they were between 17 and 18 years of
age with 71% of the immediate feedback group and 67% of the delayed feedback group
falling into this category.
Table 2 Group Breakdown by Age
Immediate Feedback Group
Delayed Feedback Group
Age
Number
% of Total Number
% of Total
14
1
1%
1
1%
15
5
3% 6
3%
16
8
4%
9
5%
17
33
19% 26
15%
18
125
71%
118
67%
18+
4
2% 13
7%
Unknown
0
0%
3
2%
Total
176
100% 176
100%
Students making up the study groups come from different parts of the United
States. Those in the immediate feedback group represented 21 states, while students in
the delayed feedback group represented 22 states.
As stated previously, students in the delayed feedback group self-selected the
23
option of submitting assignments by mail. A small sampling of these students (n = 12)
was contacted by telephone in an effort to determine why they selected what seemed to
be an antiquated option. Each of the students contacted indicated they did so because they
did not have personal access to a computer and/or the internet.
Instrumentation Final examinations for the four courses included in the study were designed to use
a machine scorable format and tested different levels of learning as found in Bloom’s
(1968) Taxonomy of Educational Objectives. They included the following:
1. The final exam for the English 055 course was made up of 50 questions
including 49 multiple-choice questions and 1 true/false question. Questions on the exam
tested student learning on the knowledge level of learning and explored knowledge of
terminology, knowledge of specific facts, and knowledge of conventions.
2. The History 043 final exam contained 100 questions including 96 multiple-
choice and 4 true/false questions. All 100 questions could be categorized as knowledge
level questions exploring knowledge of specific facts. An additional 10 questions were
included in the exam that dealt with course maintenance issues.
3. The Health 041 final exam consisted of 100 multiple-choicer questions. These
were designed to explore student learning on the knowledge, comprehension, and
application levels. Knowledge level questions tested knowledge of terminology and
knowledge of specific facts, while comprehension level questions tested the student’s
ability to translate one level of abstraction to another.
4. The final exam for the XPLR 041 course contained 55 questions including 54
multiple-choice questions and 1 true/false question. These tested student learning on the
24
knowledge and comprehension levels. Knowledge level questions explored knowledge of
specific facts, knowledge of generalizations, and knowledge of principles.
Comprehension level questions tested the student’s skills at interpretation.
Design
This study used a quasi-experimental design as the experimental groups were self-
selected making the random assignment of groups impractical (Cook & Campbell, 1979).
A quasi-experimental design typically includes one or more treatment groups who
receive some form of treatment or intervention in the course of the study. A control
group, which does not receive the treatment or intervention, can also be included for
comparison purposes. In this study, the group which received delayed feedback by mail is
classified as the control group. The group which received immediate feedback by
computer is considered to be the treatment group. Further, a pretest and posttest are also
commonly used for comparison purposes. This study utilized a posttest-only control
group design as it intended to test the effects of the treatment (type of feedback) on
students’ scores and time to completion. Figure 1 shows a representative model in which
O represents the observation of the dependent variable and X represents the intervention
or treatment. The designation GT and GC represent the treatment and control groups
respectively. The horizontal line indicates that the groups were not randomly assigned.
GT X O
GC O
Figure 1. Representation of a Posttest Only Control Group Design.
25
The data for this study were gathered from archival records maintained by BYU
Independent Study. This data included course final exam grades as well as end-of-lesson
assignment and final exam submission dates.
Analysis
For data analysis this study used an independent sample t-test (α = .05) to
compare the differences in average final exam scores (Hypothesis 1) and the average
number of days students took to complete a course (Hypothesis 2) between high school
independent study course groups provided with either immediate or delayed feedback.
The t-tests were used to determine whether the observed differences in group means were
statistically significant.
Procedure
Once a student enrolled in either the paper-based or web-based version of one of
the courses identified above, he or she was provided with lesson materials. Both sections
received the same instructional material delivered in two different formats. At the
conclusion of each course lesson, the student submitted an end-of-lesson assignment to
BYU Independent Study for grading either by regular mail or secure e-mail via computer
consistent with the course section in which the student was enrolled. The assignment was
computer scored and feedback, in this case distractor specific elaborative feedback, was
generated for each question on the assignment (See Appendices A & B). The graded
assignment and feedback responses were then returned to the student again either by
regular mail or secure e-mail via computer.
Once the student completed the lesson material and submitted all end-of-unit
assignments for grading, he or she requested a final exam. After the request was
26
processed, exam materials were either mailed directly to a certified proctor by BYU
Independent Study or held for the student to take at the Independent Study Testing Center
on the BYU campus. Approved proctors for independent study exams include full-time
school or public librarians; guidance counselor/counseling staff; full-time teachers,
school superintendents, principals, vice-principals, or other administrators; directors,
principals, or faculty of an LDS seminary or institute of religion; embassy education
officers; military base/station education officers; and local college testing centers. These
individuals apply to BYU Independent Study to serve as a proctor. Upon verifying their
application information, the BYU Independent Study testing personnel certify the
individual as a qualified proctor.
The student met with the proctor and completed the final exam under the
proctor’s direct supervision. Once the student completed the exam, the proctor collected
and returned the exam materials to BYU Independent Study by mail. When received, the
exam was computer graded and the grade was posted and made available to the student.
BYU Independent Study kept the hard copy of the exam on file.
27
Chapter 4
Results
This chapter reports the data gathered and reports of the analyses of the two
research hypotheses introduced earlier.
The high school independent study courses used in this study were (a) English
055, (b) History 043, (c) Health 041, and (d) Exploring Values 041. Students who were
enrolled in the traditional, paper-based sections of these courses and chose to submit
assignments by mail rather than online constituted the delayed feedback group. An equal
number of students were then randomly selected from larger populations enrolled in
computer-based sections of each course. These students received immediate feedback
provided via computer upon submission of their assignments and constituted the
treatment or immediate feedback group in the study.
Hypothesis 1
The first research hypothesis stated that students who receive immediate
feedback would obtain higher mean scores on the course final exam than students who
receive delayed feedback. The mean scores, standard deviations, t-test results, and effect
sizes are summarized in Table 3.
A visual comparison of the pre-treatment and post-treatment means indicate that
the difference is positive for each course. The results of the t-tests indicate that the mean
difference between the immediate and delayed feedback groups was statistically
significant for the English 055 and Exploring Values 041 courses, but not for the History
043 and Health 041 courses. The entries in the far right column of table 3 are effect sizes
and provide a way for describing the practical significance of the mean difference
28
Table 3 Mean Score on Final Examination by Course of Study and Type of Feedback
Immediate Feedback
Delayed Feedback
Course
n
M
SD
M
SD
t
df
p
Effect Size
English 055
29
82.14
11.94
70.62
15.83
3.13
56
.003*
.822
History 043
28
74.96
9.09
70.32
12.58
1.58
54
.120
.423
Health 041
29
86.48
10.80
83.76
10.15
.99
56
.327
.260
Exploring
Values 041
90
83.50
9.97
76.78
12.05
4.08
178
.0001*
.608
between the two types of feedback. The effect sizes indicate that students in English 055
who received immediate feedback scored .82 of a standard deviation higher on the final
exam then those students who received delayed feedback. Similarly, students in the
Exploring Values 041 course who received immediate feedback scored .61 standard
deviations higher on the average than students in the same course who received delayed
feedback.
Cohen (1988) has recommended the following guidelines for interpreting effect
sizes in terms of their relative magnitude:
Observed
Effect Size
Magnitude of Effect
d = .2
Small
d = .5
Medium
d =.8
Large
29
Accordingly, the size of the effect for the English 055 course should be
considered large, while the effect size for the Exploring Values 041 course would be
medium. The effect sizes for the History 043 and Health 041 courses would be
considered small.
Hypothesis 2
The second research hypothesis asserted that the elapsed time between submission
of the first course assignment and course completion would be significantly less for
students who received immediate feedback than for those who received delayed
feedback. Because independent study students at BYU are given up to one year to
complete a course, they could potentially take a substantial amount of time between
registration and actually submitting the first completed lesson. As this is essentially dead
time, for the purposes of this study, completion time was defined as being the number of
days between the submission of the first course assignment and final exam. The mean
number of days required for completion, the standard deviations, t-test results, and effect
sizes for the independent study courses included in this study are displayed in Table 4.
Visual comparison of the pre-treatment and post-treatment means is negative and
shows that students in the delayed feedback group tended to complete their coursework in
less time than those in the immediate feedback group. The t-test results indicate that the
difference in the total means between the immediate feedback and delayed feedback
groups were statistically significant as were the differences in means between the two
feedback groups for the English 055, History 043, and Health 041 courses. The difference
in means between the two feedback groups was not statistically significant for the
30
Table 4 Mean Course Completion Time by Course of Study and Type of Feedback
Immediate Feedback
Delayed Feedback
Course
n
M
SD
M
SD
t
df
p
Effect Size
English
055
29
87.03
68.53
39.24
46.47
3.11
56
.003*
.816
History 043
28
118.39
79.82
50.71
53.25
3.73
54
.001*
.998
Health 041
29
128.41
106.01
67.45
83.56
2.43
56
.018*
.639
Exploring Values
041
90
49.14
37.23
36.51
51.43
1.89
178
.061
.281
Total
176
79.47
73.21
44.32
58.10
4.99
350
.0001*
.532
Exploring Values 041 course. The effect of delayed feedback on course completion times
was .816 for the English 055 course and .998 for the History 043 course. Both of these,
according to Cohen’s (1988) guidelines, should be considered to be large effects. The
effect of delayed feedback on course completion times for the Health 041 course was
.639 and for the Exploring Values 041 course it was .281. The effect size for the Health
041 course would be considered medium and the effect size for the Exploring Values 041
course should be considered small. The effect of delayed feedback on the course totals
was .532 which would be considered a medium effect size.
31
Chapter 5
Conclusions
This chapter includes a summary of the conclusions drawn from the results of this
study reported in Chapter 4, observations, limitations of the study, and recommendations
for future research.
Hypothesis 1
The final grade, commonly understood to be an indicator of a student’s success in
a course, was the dependent variable used to determine whether the use of immediate
feedback was more successful in helping a student achieve in an independent study
course. Do students receiving immediate feedback in a course earn higher scores on a
final exam than those receiving more traditional delayed feedback? The results of the
study indicate that this is the case.
The results of the comparisons between both groups of students enrolled in the
English 055 and Exploring Values 041 courses reflected a statistically significant
difference in mean final examination scores. Students enrolled in these courses who
received immediate feedback scored significantly higher than those students who
received delayed feedback. Immediate feedback students enrolled in English 055 scored
over an entire grade higher than their delayed feedback counterparts. Students enrolled in
Exploring Values 041 who received immediate feedback scored over one half of a grade
higher than those who received delayed feedback.
Similar findings were found in comparing the mean final scores of both groups of
students enrolled in History 043 and Health 041. Although the differences in mean final
exam scores between the immediate and delayed feedback groups enrolled in these
32
courses were not found to be statistically significant, students who received immediate
feedback still scored higher than those who received delayed feedback—an important
consideration for most students. Immediate feedback students enrolled in History 043
scored almost one half of a grade higher than students who received delayed feedback.
Immediate feedback students enrolled in Health 041 scored one forth of a grade higher
than their delayed feedback counterparts. In addition, a review of the effect sizes of all
four courses indicated that the use of immediate feedback had a greater impact on course
final exam grades than the use of delayed feedback.
Hypothesis 2
A continual item of concern for independent study administrators is the need to
keep students involved in a course and progressing toward completion. Hypothesis 2
addresses this concern by comparing the impact of immediate and delayed feedback on
the amount of time needed by a student to complete a high school level independent
study course.
Does the use of computer-delivered immediate feedback encourage a student to
complete a course in less time than use of more traditional delayed feedback delivered by
return mail? Results of the study would indicate that this is not the case as students in the
immediate feedback group took a significantly greater amount of time to complete their
coursework than students enrolled in the delayed feedback group. The average amount of
time for students receiving immediate feedback was 35 days longer than those students
receiving delayed feedback.
An individual examination of the courses included in the study showed that, in
each case, students receiving delayed feedback completed the course in less time than
33
those students receiving immediate feedback. Results of the analyses of 3 of the 4 courses
(English 055, History 043, and Health 041) indicated a significant difference in the
average number of days required for completion between the immediate and delayed
feedback groups. Immediate feedback students enrolled in English 055 took 48 days
longer on average to complete their courses than students who received delayed
feedback. Students enrolled in History 043 who received immediate feedback required 68
more days on average to complete their courses than those students who received delayed
feedback. Immediate feedback students enrolled in Health 041 used 61 more days on
average to complete their courses than students who received delayed feedback. Although
the difference in the completion times between the two groups was not significant for the
course Exploring Values 041, students receiving immediate feedback still took 13 days
longer to complete their courses than those students who received delayed feedback.
Again, a review of the effect sizes indicated that the use of delayed feedback had a
greater impact on average course completion time than did the use of immediate feedback
for all courses.
Observations
A question that immediately comes to mind is concern about the possible impact
that the use of the computer in providing course materials might make in the immediate
feedback group when compared to the use of the paper-version of course materials in the
delayed feedback group. The instructional material presented to both groups is identical
and both groups essentially read black text on a white background. Course delivery via
the internet, however, allows for the inclusion of some extras such as the use of
animation, color, and additional reference material accessed by means of hyperlinks
34
embedded in the course material. In order to explore this possibility a comparison was
made between a group of students enrolled in the paper-versions of each of the courses
included in this study who opted to return their lesson assignments via computer with
both the immediate and delayed feedback groups already discussed. The students in this
new group were also selected randomly from a larger population and the total number
selected was equal to the number in the original groups (n = 176). The mean final exam
scores and their standard deviations received by students in all three independent study
course settings are compared in Table 5.
The average mean score for students enrolled in English 055 using computer-
generated course materials, submitting lesson assignments by computer, and receiving
immediate feedback via computer was highest at 82.14 (SD = 11.94). The average mean
score for students using paper-versions of the course materials, submitting lesson
assignments by computer, and receiving immediate feedback by computer was 78.69 (SD
= 12.93). The average mean score for students enrolled in courses using the paper-version
of course materials, submitting lesson assignments by mail, and receiving delayed
feedback by return mail was 70.62 (SD = 15.83). The global F-test indicated significant
differences between the means, F(2,84) = 5.42, p = .006. Tukey’s HSD test was then used
to ascertain which specific pairs of group means, if any, were significantly different. The
results indicated that the difference between the mean scores of the original immediate
and delayed feedback groups was statistically significant with the immediate feedback
group scoring an average of 11.5 percentage points higher than the delayed feedback
group. This finding is consistent with the findings previously reported in Chapter 4.
However, the difference between the means of the computer-immediate feedback and the
35
Table 5 Mean Score on Final Examination by Course of Study and Type of Course and Feedback
Computer Course with Immediate Feedback
Paper Course with Delayed Feedback
Paper Course with
Immediate Feedback
Course n
M
SD
n
M
SD
n
M
SD
English 055
29
82.14
11.94
29
70.62
15.83
29
78.69
12.93
History 043
28
74.96
9.09
28
70.32
12.58
28
78.57
13.11
Health 041
29
86.48
10.80
29
83.76
10.15
29
88.93
7.72
Exploring
Values 041
90
83.50
9.97
90
76.78
12.05
90
80.08
15.84
36
paper-immediate feedback groups was not statistically significant although the computer
course mean was higher by 3.5 percentage points. Finally the difference in means
between the paper-immediate feedback and the paper-delayed feedback groups was not
significant although the immediate feedback group means was higher by 8 percentage
points.
The average mean score for students enrolled in History 043 using computer-
generated course materials, submitting lesson assignments by computer, and receiving
immediate feedback via computer was 74.96 (SD = 9.09). The average mean score for
students using paper-versions of the course materials, submitting lesson assignments by
computer, and receiving immediate feedback by computer was 78.57 (SD = 13.11). The
average mean score for students enrolled in courses using the paper-version of course
materials, submitting lesson assignments by mail, and receiving delayed feedback by
return mail was 70.32 (SD = 12.58). The global F-test indicated there were no significant
differences in any of the mean comparisons, F(2,84) = 2.09, p = .131.
The average mean score for students enrolled in Health 041 using computer-
generated course materials, submitting lesson assignments by computer, and receiving
immediate feedback via computer was 86.48 (SD = 10.80). The average mean score for
students using paper-versions of the course materials, submitting lesson assignments by
computer, and receiving immediate feedback by computer was 88.93 (SD = 7.72). The
average mean score for students enrolled in courses using the paper-version of course
materials, submitting lesson assignments by mail, and receiving delayed feedback by
return mail was 83.76 (SD = 10.15). The global F-test indicated significant differences in
one or more of the mean comparisons, F(2,81) = 3.48, p = .035. The results of Tukey’s
37
HSD test indicated that the difference between the means of the paper-immediate
feedback and paper-delayed feedback groups was statistically significant with students in
the paper-immediate feedback group scoring an average of 5.2 percentage points higher
on the course final exam than those students in the paper-delayed course. The difference
between the means of the computer-immediate feedback and paper-immediate feedback
was not statistically significant although the paper course mean was higher by 2.5
percentage points. Finally, as reported in Chapter 4, the difference between the original
computer-immediate feedback and the paper-delayed feedback groups was not significant
although the mean score of the immediate feedback group was higher by 2.7 percentage
points.
The average mean score for students enrolled in Exploring Values 041 using
computer-generated course materials, submitting lesson assignments by computer, and
receiving immediate feedback via computer was highest at 83.50 (SD = 9.97). The
average mean score for students using paper-versions of the course materials, submitting
lesson assignments by computer, and receiving immediate feedback by computer was
80.08 (SD = 15.84). The average mean score for students enrolled in courses using the
paper-version of course materials, submitting lesson assignments by mail, and receiving
delayed feedback by return mail was 76.78 (SD = 12.05). The global F-test indicated
significant differences in one or more of the mean comparisons, F(2,267) = 6.16, p =
.002. Tukey’s HSD test indicated that the difference between the mean scores of the
computer-immediate feedback and paper-immediate feedback was not statistically
significant although the computer course mean score was higher by 3.4 percentage points.
The difference between the mean scores of the paper-immediate feedback and the paper-
38
delayed feedback groups was also not significant although the immediate feedback mean
score was higher by 3.3 percentage points. The difference between the mean scores of the
original immediate and delayed feedback groups, however, was found to be statistically
significant with the immediate feedback group scoring an average of 6.7 percentage
points higher than the delayed feedback group as previously reported in Chapter 4.
The mean number of days required for course completion and the standard
deviations for all independent study course settings are summarized in Table 6. Again,
the completion time was defined as being the number of days between the submission of
the first course assignment and final exam. The average completion time for students
enrolled in courses using computer-generated course materials, submitting lesson
assignments by computer, and receiving immediate feedback via computer was 79.47
days (SD = 73.21). The average completion time for students using paper-versions of the
course materials, submitting lesson assignments by computer, and receiving immediate
feedback by computer was 82.42 days (SD = 85.32). The average completion time for
students enrolled in courses using the paper-version of course materials, submitting
lesson assignments by mail, and receiving delayed feedback by return mail was 44.32
days (SD = 58.10). The global F-test indicated significant differences in one or more of
the mean comparisons, F(2,525) = 14.81, p < .0001. Tukey’s HSD test indicated that the
difference between the average completion time of the paper-immediate feedback and the
paper-delayed feedback groups was statistically significant with students in the paper-
delayed feedback group completing their course work in an average of 38 fewer days
than those students in the paper-immediate group. The difference between the average
completion time of the computer-immediate feedback and the paper-delayed groups were
39
Table 6 Mean Completion Time by Course of Study and Type of Course and Feedback
Computer Course with Immediate Feedback
Paper Course with Delayed Feedback
Paper Course with
Immediate Feedback
Course n
M
SD
n
M
SD
n
M
SD
English 055
29
87.03
68.53
29
39.24
46.47
29
92.66
75.70
History 043
28
118.39
79.82
28
50.71
53.25
28
87.71
89.22
Health 041
29
128.41
106.01
29
67.45
83.56
29
143.14
123.20
Exploring
Values 041
90
49.14
37.23
90
36.51
51.43
90
57.91
58.89
Total
176 79.47 73.21
176 44.32 58.10
176 82.42 85.32
40
also statistically significant with students in the delayed feedback group completing their
course work in an average of 35 fewer days than those students in the immediate
feedback group. However, the difference in average completion time between the
computer-immediate feedback and the paper-immediate feedback groups was not
significant although the computer-immediate feedback group completed their course
work in an average of 3 fewer days than those students in the paper-immediate feedback
group.
It would appear then that the use of course materials provided via computer does
not make a significant difference in either the course final exam score or in the amount of
time a student requires to complete an independent study course. These findings focus
our attention on a second concern regarding how or if feedback is used.
An item of interest that came to light in the course of this study was the fact that
nine of the students enrolled in the delayed feedback section of the Exploring Values 041
course submitted all of the course assignments on the same day and arranged to take the
final exam on that day or on the next. These students most likely lived in close proximity
to BYU and had the opportunity to hand-deliver their assignments as well as complete the
final exam at the Independent Study Testing Center on campus. Although the amount of
time the students spent completing coursework could not be determined, it is apparent
that these students did not wait to receive feedback responses for individual end of lesson
assignments before proceeding on to the next lesson or before taking their final exam. It
would seem to be a safe assumption that these students also did not make use of the end
of lesson feedback responses in preparation for their final exam. It should also be pointed
out that this scenario would not be possible for a student in the immediate feedback
41
group. While a student receiving immediate feedback could choose to ignore the
feedback responses he or she receives, the feedback would automatically be provided for
each lesson before the next one could be initiated.
Limitations
During the course of this study it became apparent that there were several factors
that could not be controlled and proved to items of concern. First was the fact that the
students self-selected their course option and whether they consequently would be part of
the delayed feedback or immediate feedback groups. Because random assignment was
therefore not practical, a posttest only control group quasi-experimental design was used
in the study. Cook and Campbell (1979) point out that obvious flaw to this type of
experimental design is the absence of pretest data. This leads to the possibility that any
differences discovered between the study groups could be attributed to the influence of
selection differences rather than the treatment effect being tested. In this case the
possibility of selection differences or the differences between the kinds of people in one
experimental group as opposed to the other could be a real concern because these were
preexisting groups with no explanation as to why students selected the course option they
did. As was indicated previously a number of students in the delayed feedback group
were contacted and they explained that they had chosen that option because they did not
have access to a computer or the internet. Was their course selection motivated then by
simply not having a computer available to them or do they not have a computer because
they belong to a lower socio-economic class and consequently are poorer students
academically than those students who opted for the immediate feedback option?
42
The second factor was the lack of naturalistic and evaluative data about the
student population. This study assumed that students both received and used end-of-unit
feedback. However, without direct input from the students, there is no certain knowledge
as to whether the feedback was used or how it was used. Nor do we know what the
students’ satisfaction level was regarding the feedback.
Another question that is raised here might be the student’s motivation for taking
an independent study course. The fact that a student takes a course only because they
need a few more units and only need to pass a course as opposed to those students who
are (a) trying to accelerate their progress, (b) nontraditional students, (c) making up a
failure, or (d) taking a course not offered by their school can make a potential difference
in how a student might approach the use of feedback responses or a course in general.
A third concern was the restrictions found in the course of gathering grade and
registration data on the students in the experimental groups. Due to limited data
availability, it was only possible to identify those students in the delayed feedback group
that were currently enrolled in or finishing courses during the time utilized for this study.
A final concern was the time limit imposed by the researcher. Independent Study
students at BYU are given up to one year to complete a course however because of a
desire to conclude the study in a timely manner, only those students who completed
coursework in a seven-month period were included.
Recommendations for Future Research
While this study attempted to take a first-time look at an area that has not
previously been examined, the researcher felt as though it broached more questions than
it answered. Some of these include:
43
1. What factors were present that influenced a student’s course option selection?
2. Is there a real difference in student achievement levels between the two
groups?
3. Do students of either or both groups use the feedback responses at all? Why or
why not?
4. Are students of either or both groups careful to review each assignment
completely or do they only review those questions that were missed?
5. Are the feedback responses an integral part of the students’ preparation for the
final exam?
6. Do students in either or both groups feel as though the feedback provided is
advantageous?
Additional research is needed to answer what would seem to be important
considerations in how feedback responses are used by students in independent study
settings. Answers to these questions could also potentially provide insights for instructors
and course designers as to how they might be able to design and present instructional
feedback responses in ways that would encourage students to make better use of them.
Another area of interest for administrators of independent study programs is the
rate of student completion. Does the inclusion of immediate feedback encourage a higher
completion rate than more traditional delayed feedback? Due to the nature of the data
maintenance program encountered by the researcher, this question was unable to be
answered in the course of this study. Additional research would be warranted to examine
this important statistic.
44
Conclusion
The purpose of this study was to compare the impact of different feedback
delivery time frames on student performance in high school independent study course
settings. The results of Hypothesis 1 agree with current research which indicates that
students who receive immediate feedback on lesson assignments tend to perform better
on a delayed final exam instrument than students who receive delayed feedback or no
feedback (Kulik & Kulik, 1988). It would appear that independent study students, like
their in-classroom counterparts, benefit from a rapid review of their initial responses on
lesson assignments. This rapid review, as Dihoff, Brosvic, and Epstein (2003) claimed,
allows for an immediate confirmation of correct responses and, in turn, enhances a
student’s ability to recognize those same responses more confidently in future settings
such as a course final exam. It also serves to identify those concepts that have not been
clearly understood by the student and encourages further efforts in learning the material
in question.
The results of Hypothesis 2 were surprising. It seemed to be a safe assumption
that students receiving delayed feedback would take longer on average to complete an
independent study course if only because of the turn-around time required in submitting
assignments and receiving responses through the mail. Just the opposite proved to be
true, however, and it is apparent that receiving immediate feedback was not a factor in
reducing the time requirement needed to complete an independent study course.
While it is tempting to attribute these findings to the type of feedback alone, that
would not seem fair or feasible in light of the limitations and recommendations outlined
above. Although it is evident that changes occurred between the two experimental
45
groups, other factors must surely come into play. One of the most important of these is
the independent nature of both the learning package and the learner as discussed in
Chapter 1.
It is important to remember that students in a traditional classroom move together
even when reviewing feedback on a past assignment. An independent learner moves at
his or her own pace. In order to review feedback on a past assignment he or she must
make a conscious decision to stop forward progress and return to review a past lesson or
exam. An advantage then of immediate feedback delivered by computer is the fact that it
arrives promptly upon submission of a lesson assignments for grading and before a
student begins to move forward to the next lesson. Even though it is received in a timely
manner, students in this group, as well as those in the delayed feedback group, can
choose to review or ignore the feedback responses and this emphasizes the role of
motivation in how or if a student uses feedback.
Current research demonstrates that student motivation is an important factor in the
use of feedback. Dempsey and Driscoll (1996) and Stock, Kulhavey, Pridewater, and
Krug (1992) found, for example, that students are more inclined to use feedback when
they learned that answers they felt confident were correct were actually incorrect. It
would seem then that the independent learners included in this study would use feedback
in the role of a teacher’s aid helping to clarify concepts that were initially misunderstood.
Feedback responses would conceivably be used even less or not at all in those situations
where students felt confident of their knowledge and understanding of the material in
question.
46
Attempting to prove the two hypotheses included in this study has helped to
create a clearer picture of how different types of feedback might impact student success
and progress in high school independent study courses. The study also generated
questions and concerns about how students use feedback responses. Answers to these
questions through additional research would hopefully help develop administrative and
pedagogical strategies that will foster enhanced student learning.
47
References
Anderson, R. C., Kulhavey, R. W., & Andre, T. (1972). Conditions under which feedback facilitates learning from programmed instruction. Journal of Educational Psychology, 63, 186-188.
Azevedo, R., & Bernard, R. M. (1995). A meta-analysis of the effects of feedback in
computer-based instruction. Journal of Educational Computing Research, 13, 111-127.
Bangert-Drowns, R. L., Kulik, C.-L. C., Kulik, J. A., & Morgan, M. (1991). The
instructional-effect of feedback in test-like events. Review of Educational Research, 61, 213-238.
Bloom, B. B. (1968). Taxonomy and educational objectives: The classification of
educational goals. New York: David McKay Co.
Bugbee, A. C. (1996). The equivalence of paper-and-pencil and computer-based testing.
Journal of Research on Computing in Education, 28, 282-299.
Bunderson, C. V., Inouye, D. K., & Olsen, J. B. (1998). The four generations of
Computerized educational measurement. In Linn, R. L. (Ed.), Educational Measurement (pp. 367-407). New York: Macmillan.
Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning – A
theoretical synthesis. Review of Educational Research, 65, 245-281.
Chickering, A. W., & Gamson, Z. F. (1991). Applying the seven principles for good
practice in undergraduate education. In New Directions for Teaching and Learning, 47. San Francisco: Jossey-Bass Inc.
48
Clariana, R. B. (1993). A review of multiple-try feedback in traditional and computer-
Based instruction. Journal of Computer-Based Instruction, 20, 67-74. Clariana, R. B., Ross, S. M., & Morrison, G. R. (1991). The effects of different feedback-
strategies using computer-administered multiple-choice questions as instruction. Educational Technology Research and Development, 3(2), 5-17.
Clarke, D. E. (2000). Evaluation of a networked self-testing program. Psychological
Reports, 86, 127-128.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.).
Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
Cohen, V. B. (1985, January). A reexamination of feedback in computer-based instruction: Implications for instructional design. Educational Technology, 25(1), 33-37.
Cook, T. D & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis
issues for field settings. Chicago: Rand McNally College Publishing Company. Cyboran, V. (1995). Designing feedback for computer-based training. Performance and
Instruction, 34, 18-23.
Dempsey, J. V., & Driscoll, M. P. (1996). Error and feedback: Relation between content
analysis and confidence of response. Psychological Reports, 78, 1079-1089.
Dihoff, R. E., Brosvic, G. M., Epstein, M. L., & Cook, M. J. (2004) Provision of
Feedback during preparation for academic testing: Learning is enhanced by immediate but not delayed feedback. Psychological Record, 54, 207-231.
49
Dihoff, R. E., Brosvic, G. M., & Epstein, M. L. (2003). The role of feedback during academic testing: The delay retention effect revisited. Psychological Record, 53, 533-548.
Epstein, M. L., Epstein, B. B., & Brosvic, G. M. (2001). Immediate feedback during
Epstein, M. L., Lazarus, A. D., Calvano, T. B., Matthews, K. A., Hendel, R. A., Epstein,
B. B., & Brosvic, G. M. (2002). Immediate feedback assessment technique promotes learning and corrects inaccurate first responses. Psychological Record, 52, 187-201.
Flannelly, L. T. (2001). Using feedback to reduce students' judgment bias on test
questions. Journal of Nursing Education, 40, 10-16.
Henstrom, R. H., & Oakes, K. R. (1997). Independent Study. In The world is our
campus: The history of the division of continuing education at Brigham Young University Provo, Utah (chap. 13). Retrieved February 3, 2005, from http://dce.byu.edu/div_links/chapter.dhtm?chapter=13
Keenan, V. C., & Langer, P. (1993). Instructional feedback in context. Psychological
Reports, 73, 107-112.
Kulhavy, R. W. (1977). Feedback in written instruction. Review of Educational Research,
47, 211-232.
Kulhavy, R. W. & Anderson, R. C. (1972). Delay-retention effect with multiple-choice
tests. Journal of Educational Psychology, 63, 500-512.
50
Kulhavy, R. W., Stock, W. A., Hancock, T. E., Swindell, L. K., & Hammrich, P. L. (1990). Written feedback – Response certitude and durability. Contemporary Educational Psychology, 15, 319-332.
Kulhavy, R. W., Stock, W. A., Thornton, N. E., Winston, K. S., & Behrens, J. T. (1990).
Response feedback, certitude and learning from text. British Journal of Educational Psychology, 60, 161-170.
Kulhavy, R. W., White, M. T., Topp, B. W., Chan, A. L., & Adams, J. (1985). Feedback
complexity and corrective efficiency. Contemporary Educational Psychology, 10, 285-291.
Lee-Sammons, W. H., & Wollen, K. A. (1989). Computerized practice tests and effects
on in-class exams. Behavioral Research Methods Instruments & Computers, 21, 189-194.
Lhyle, K. G. & Kulhavy, R. W. (1987). Feedback processing and error correction.
Journal of Educational Psychology, 79, 320-322.
Mason, B. J. (2001). Relative effectiveness of human and computer-based feedback in a
mastery context. Unpublished doctoral dissertation, University of Nebraska, Lincoln.
Morrison, G. R., Ross, S. M., Gopalakrishnan, M., & Casey, J. (1995). The effects of
feedback and incentives on achievement in computer-based instruction. Contemporary Educational Psychology, 20, 32-50.
Mory, E. H. (1992). The use of informational feedback in instruction - Implications for
future research. Educational Technology Research and Development, 40(3), 5-20.
51
Olina, Z. & Sullivan, H. J. (2002). Effects of classroom evaluation strategies on student achievement and attitudes. Educational Technology Research & Development, 50(3), 61-75.
Peeck, J. & Tillema, H. H. (1979). Delay of feedback and retention of correct and
incorrect responses. Journal of Experimental Education, 47, 171-178.
Peeck, J., Van den Bosch, A. B., & Kreupeling, W. J. (1985). Effects of informative
feedback in relation to retention of initial responses. Contemporary Educational Psychology, 10, 303-313.
Phye, G. D. & Andre, T. (1989). Delayed retention effect - Attention, perseveration, or
Pressley, M. P., Ghatala, E. S., Woloshyn, V., & Pirie, J. (1990). Sometimes adults miss
the main ideas and do not realize it: Confidence in responses to short-answer and multiple-choice comprehension questions. Reading Research Quarterly, 25, 232- 249.
Pridemore, D. R. & Klein, J. D. (1991). Control of feedback in computer-assisted
instruction. Educational Technology Research and Development, 39(4), 27-32.
Schroth, M. L. & Lund, E. (1993). Role of delay of feedback on subsequent pattern
recognition transfer tasks. Contemporary Educational Psychology, 18, 15-22.
Stephens, D. (2001). Use of computer-assisted assessment: Benefits to students and staff.
Education for Information, 19, 265-275.
52
Stock, W. A., Kulhavy, R. A., Pridemore, D. R. & Krug, D. (1992). Responding to
feedback after multiple-choice answers: The influence of response confidence. The Quarterly Journal of Experimental Psychology, 45, 649-667.
Stock, W. A. & McCarthy, M. T. (1994). The effects of feedback timing on learning
facts: The role of response confidence. Contemporary Educational Psychology, 19, 251-265.
Surber, J. R. & Anderson, R. C. (1975). Delay-retention effect in natural classroom
settings. Journal of Educational Psychology, 67, 170-173.
Swindell, L. K. & Walls, W. F. (1993). Response confidence and the delay retention
Webb, J. M., Stock, W. A. & McCarthy, M. T. (1994). The effects of feedback timing on
learning facts - The role of response confidence. Contemporary Educational Psychology, 19, 251-265.
Whyte, M. M., Karolick, D. M., Nielsen, M. C., Elder, G. D., & Hawley, W. T. (1995).
Cognitive styles and feedback in computer-assisted-instruction. Journal of Educational Computing Research, 12, 195-203.
53
Appendix A
Sample of Speedback™ Assignment Questions
54
55
Appendix B
Example of Feedback Provided to Immediate and Delayed Feedback Groups
(Corresponds to Sample in Appendix A)
56
Course: XPLR 041 Question 1 (“a” is correct answer response) All answer responses: He made the point that the minds of young people are very impressionable. Question 2 (“a” is correct answer response) All answer responses: Plato felt that society must teach young people good stories and good examples for their later lives. Question 3 (“c” is correct answer response) Answer response “a”: Youth hear more than casual tales. He insists that what they hear should be purposeful because it stays forever (see paragraph 2). Answer response “b”: It may sometimes be the opposite, but is not necessarily so. Plato states that it becomes indelible and unalterable (see paragraph 2). Answer response “d”: Not necessarily. Youth may hear many unimportant things. Plato's point is that what youth hear becomes indelible and unalterable (see paragraph 2). Question 4 (“d” is correct answer response) Answer response “a”: This is not mentioned. He talks of giving youth virtuous thoughts (see paragraph 2). Answer response “b”: This is mentioned, but the emphasis is on teaching virtuous thoughts (see paragraph 2). Answer response “c”: He spoke against such stories. Plato emphasized teaching youth virtuous thoughts. Question 5 (“b” is correct answer response) Answer response “a”: He was more concerned with the stories the youth would hear even before they could read; his concern was with how they lived. Answer response “c”: This helps, but only if the examples are living virtuous lives. Answer response “d”: This helps, but Plato emphasized that it is essential for all people to live virtuous lives. Some parents do this, but others do not.