Top Banner
J. EDUCATIONAL COMPUTING RESEARCH, Vol. 42(4) 443-458, 2010 TESTING ACCOMMODATIONS FOR UNIVERSITY STUDENTS WITH AD/HD: COMPUTERIZED VS. PAPER-PENCIL/REGULAR VS. EXTENDED TIME KATHRYN S. LEE RANDALL E. OSBORNE DAVID N. CARPENTER Texas State University ABSTRACT Prompted by a previous study investigating the effects of pacing on the academic testing performance of college students with AD/HD, we further explored our preliminary findings, which suggested that a computerized testing environment enhanced the testing performance of college students with AD/HD. We compared the effects of a computerized vs. paper-pencil testing format as well as the effects of regular time vs. extended time on the students’ testing performance. Students taking the computerized version of the test performed significantly higher than those who took the paper and pencil version of the same test. The amount of time the participants had to take the test did not significantly affect their performance. Insights gleaned from the qualitative data furthered our understanding of testing accommodation concerns shared by many college students with AD/HD. According to a recent survey conducted across 10 countries, the World Health Organization estimates the prevalence of adult Attention-Deficit/ Hyperactivity Disorder (AD/HD) is 3.4% (Fayyad, De Graaf, Kessler, Alonso, Angermeyer, Demyttenaere, et al., 2007). Kessler, Adler, Barkley, Biederman, Conners, Demler, et al. (2006) estimate a rate of 4.4% in the United States. AD/HD 443 Ó 2010, Baywood Publishing Co., Inc. doi: 10.2190/EC.42.4.e http://baywood.com
16

Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

Mar 16, 2023

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

J. EDUCATIONAL COMPUTING RESEARCH, Vol. 42(4) 443-458, 2010

TESTING ACCOMMODATIONS FOR UNIVERSITY

STUDENTS WITH AD/HD: COMPUTERIZED VS.

PAPER-PENCIL/REGULAR VS. EXTENDED TIME

KATHRYN S. LEE

RANDALL E. OSBORNE

DAVID N. CARPENTER

Texas State University

ABSTRACT

Prompted by a previous study investigating the effects of pacing on the

academic testing performance of college students with AD/HD, we further

explored our preliminary findings, which suggested that a computerized

testing environment enhanced the testing performance of college students

with AD/HD. We compared the effects of a computerized vs. paper-pencil

testing format as well as the effects of regular time vs. extended time on the

students’ testing performance. Students taking the computerized version of

the test performed significantly higher than those who took the paper and

pencil version of the same test. The amount of time the participants had to take

the test did not significantly affect their performance. Insights gleaned from

the qualitative data furthered our understanding of testing accommodation

concerns shared by many college students with AD/HD.

According to a recent survey conducted across 10 countries, the World

Health Organization estimates the prevalence of adult Attention-Deficit/

Hyperactivity Disorder (AD/HD) is 3.4% (Fayyad, De Graaf, Kessler, Alonso,

Angermeyer, Demyttenaere, et al., 2007). Kessler, Adler, Barkley, Biederman,

Conners, Demler, et al. (2006) estimate a rate of 4.4% in the United States. AD/HD

443

� 2010, Baywood Publishing Co., Inc.

doi: 10.2190/EC.42.4.e

http://baywood.com

Page 2: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

is not outgrown during adolescence as once was thought (DuPaul, Guevermont,

& Barkley, 1991; Resnick, 2005) and is a valid adult disorder (Fayyad et al.,

2007; Kessler et al., 2006). Approximately 2% to 4% of college students are

affected by it (Weyandt & DuPaul, 2006). College students with AD/HD typically

earn lower grade point averages, are on academic probationary status more

often, report more academic problems, and have greater difficulty managing time

and conforming to schedules than their non-AD/HD peers (DuPaul & Weyandt,

2006; Healy, 2006; Heiligenstein, Guenther, Levy, Savino, & Fulwiler, 1999;

Weyandt & DuPaul, 2006). Due to these increased risks (Swartz, Prevatt, &

Proctor, 2005), students with disabilities are encouraged to advocate for basic

accommodations and take advantage of educational services provided by uni-

versity disability offices (Brinckerhoff, McGuire, & Shaw, 2002; Dowrick,

Anderson, Heyer, & Acosta, 2005; Greenbaum, Graham, & Scales, 1995; Meaux,

Green, & Broussard, 2009). Accommodation is defined as “a change in testing

materials or procedures that enables students to participate in assessments in

ways that reflect their skills and abilities rather than their disabilities” (Salvia,

Ysseldyke, & Bolt, 2010, p. 416). The most frequent testing accommodations

provided by disability services in university settings are extended time for tests

and the ability to take exams in minimal disturbance testing rooms away from

peers (Farrell, 2003; Lancaster, Mellard, & Hoffman, 2001). College students

diagnosed with AD/HD who utilize the services offered by the university

reportedly perform better academically than those who do not (Getzel, McManus,

& Briel, 2004; Meaux, et al., 2009; Sarkis, 2008).

Literature addressing the legal and educational contexts of accommodating

students with disabilities is abundant. The reauthorization of the Individuals

with Disabilities Education Act (IDEA) in 1997, the passage of the No Child

Left Behind Act in 2002, and the 2004 reauthorization of IDEA, emphasizing

the inclusion and accountability of students with disabilities in high stakes

testing, have spurred increased emphasis in this area (Elliott, McKevitt, &

Kettler, 2002; Fuchs, Fuchs, & Capizzi, 2005; Niebling & Elliott, 2005).

Numerous studies have examined various types of academic interventions to

assist college students with AD/HD, such as course-specific strategy training

(Allsopp, Minskoff, & Bolt, 2005), training in learning strategies and self-

advocacy skills (Getzel et al., 2004) and coaching (Quinn, Ratey, & Maitland,

2000; Swartz et al., 2005; Zwart & Kallemeyn, 2001).

The most consistent educational service to be offered for persons with

disabilities in post-secondary education is testing accommodations (Tagayuna,

Stodden, Chang, Zeleznik, & Whelley, 2005). Extended time for tests and the

ability to take exams in minimal disturbance testing rooms away from peers are

the most frequent accommodations provided by disability services in university

settings (Farrell, 2003; Lancaster et al., 2001). Few empirical studies investi-

gating accommodations for college students with AD/HD have been conducted

(Trammell, 2003; Weyandt & DuPaul, 2006) even though there “is a glaring

444 / LEE, OSBORNE AND CARPENTER

Page 3: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

need to expand the types of strategies to enhance academic performance among

students with AD/HD” (DuPaul & Eckert, 1998, p. 59).

Synthesizing the literature associated with testing accommodations for students

with disabilities is challenging, because the studies often differ substantially in

terms of variables. For example, there is great variability among the ages and

range of disabilities of the students sampled, the research methodologies used,

as well as the type of technology used to deliver the various accommodations in

which technology is a variable. Additionally, the various accommodations investi-

gated are frequently packaged with other accommodations and are seldom inves-

tigated in isolation (Tindal & Fuchs, 2000). Most of the studies investigating

the effects of accommodations have been with elementary-aged students

(Thompson, Blount, & Thurlow, 2002; Tindal & Fuchs, 2000). Few have investi-

gated the effects of testing accommodations on the performance of secondary

school students (DuPaul & Eckert, 1998) and even fewer have investigated

college students (Lee, Osborne, Hayes, & Simoes, 2008; Tindal & Fuchs, 2000;

Wallace, Winsler, & NeSmith, 1999). To situate this study within the existing

literature, this review will focus primarily on research investigating assess-

ment accommodations of computer-based vs. paper-pencil format and regular

vs. extended time in each of the two presentation formats.

Computer-Based Testing as an Accommodation

Computer-based testing (CBT) “generally refers to using the computer to

administer a conventional (i.e., paper-pencil) test” (Wise & Plake, 1989, p. 5).

Studies investigating computer-based presentation as a testing accommoda-

tion has had mixed findings (Thompson et al., 2002). Brown and Augustine

(2001) found that computer use had no significant effect on the scores of 206

12th grade students. Hollenbeck, Tindal, Harniss, and Almond (1999) found no

differences between stories written with computers and those written without.

In contrast, Burk (1998) found that the performance of students with disabilities

was significantly higher on a computerized administration of a test as com-

pared to a paper and pencil format. Calhoon, Fuchs, and Hamlett (2000) also

reported computer use had a positive effect on performance of 9th through

12th grade students with math and reading learning disabilities, as well as Russell

and Plati (2000) with a group of 8th and 10th grade students writing compositions

via a computer.

Regular Time vs. Extended Time

Studies exploring an extended time accommodation for college students with

learning disabilities have had inconsistent findings. For example, Alster (1997)

found no significant difference in algebra test scores between college students

with learning disabilities in an extended time condition and students without

learning disabilities in both timed and extended-time conditions. Medina (2000)

TESTING ACCOMMODATIONS / 445

Page 4: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

and Zuriff (2000) found that although extended time benefited all participants in

the study, extended time did not benefit college students with learning disabilities

as compared to their non-disabled peers. In contrast, Weaver (2000) found that

postsecondary students with disabilities made significantly higher gains on their

reading test in an extended time condition as compared to students without

learning disabilities.

According to Wallace, Winsler, and NeSmith (1999) extended testing time

may not be sufficient to ”level the playing field” for this population. In fact,

some college students with AD/HD report that extended testing time may

actually hinder their performance. Many reported “that the pressure to finish

the test quickly is what gives them the stimulation they need to focus” (Farrell,

2003, p. 51).

Our Previous Study/Findings

In our previous study comparing the testing performance scores of college

students with AD/HD, we found no significant differences between a computer-

paced and a student-paced testing condition within a computer-based environ-

ment. Interview data, however, revealed that the students perceived that the

CBT did benefit their overall successful performance under both conditions

(Lee et al., 2008). This finding prompted the current investigation, comparing

the testing performance among college students with AD/HD via computerized

vs. paper-pencil and regular vs. extended time.

Local Context of Current Study

Of the 27,000 students enrolled at our large southwestern university, 900

were registered with the Office of Disability Services (ODS). Of these 900

students, 44% had a primary diagnosis of AD/HD. The university’s ODS proc-

tored 3,017 exams, totaling 4,029 hours within the fiscal year 2008 (Schultz,

2008). These numbers strongly support the need for empirical research in the

area of testing accommodations for college students diagnosed with AD/HD.

Statement of Null Hypotheses

1. H0: College students with AD/HD will not perform better on a computerized

version than a paper-pencil version of the same test.

2. H0: College students with AD/HD will perform better in the extended time

condition than in the regular time condition.

Based on our previous research, we expect to reject both null hypotheses.

446 / LEE, OSBORNE AND CARPENTER

Page 5: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

METHOD

This study utilized a mixed-methods design to compare the effects of com-

puterized vs. paper-pencil testing and regular time vs. extended time for college

students diagnosed with AD/HD. According to Creswell (2003), this concurrent

triangulation strategy was appropriate, because we collected quantitative and

qualitative data concurrently, and utilized two different methods “as a means

to offset the weaknesses inherent within one method with the strengths of the

other” (p. 217). We employed a 2 × 2 factorial design so we could simultaneously

investigate the effect of the two independent variables, format and time, on the

dependent variable (testing performance). Participants were randomly assigned to

one of four testing conditions: computerized/regular time; computerized/extended

time; paper-pencil/regular time, or paper-pencil/extended time. Following the

completion of one of the four treatments, each participant completed a written

questionnaire and a follow-up interview.

Participants

Thirty-one students enrolled in a mid-sized public university in the south-

western United States who were registered with ODS with a primary diagnosis of

AD/HD participated in the study (see Table 1). Participants were solicited through

advertisements posted around the university campus and through postings in

newsletters sent by ODS. Students scheduled a time with an examiner to take the

test at their convenience at the university’s psychology department computer lab.

Students were randomly assigned to one of the four conditions. After completing

the assigned testing condition, a written questionnaire, and a follow-up interview,

each participant received twenty dollars as an incentive to participate.

Materials

In order to simulate testing conditions in a university setting as closely as

possible, a passage from a college psychology textbook was selected as the

prompt for the participants. Multiple-choice questions from the college textbook

test bank were used to assess the students’ learning. The text and 11-item multiple--

choice test items were taken from a textbook routinely used in introductory

psychology classes.

Participants assigned to the paper-pencil/regular time or paper-pencil/extended

time treatment were given a paper copy of the 11 multiple-choice questions in one

document. This traditional paper-pencil format was used to simulate real testing

conditions in the college classroom. Participants recorded their responses on a

standardized electronic scoring response form routinely used in the university

setting. Participants assigned to the computerized-regular time or computerized-

extended time treatment answered the multiple-choice questions using a com-

puter. The tests were administered using the university’s web-based open source

TESTING ACCOMMODATIONS / 447

Page 6: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

course groupware which only allowed one multiple-choice question displayed

on the screen at a time. Participants clicked on the correct answer.

After completing the multiple-choice test in the assigned testing condition,

each participant completed a written questionnaire (see Table 2) and a face-to-face

interview with the examiner.

Procedures

Upon arrival at the computer lab, each participant was given a paper form to

record the demographic information as the examiner prepared for administering

the treatment. To ensure consistency, the examiner followed a script each time she

administered one of the four treatment conditions. After providing the written

demographic information, the participant was given a pencil, a highlighter, and

a paper copy of the reading passage and was instructed that 5 minutes were

allowed to “study” the passage in preparation for 11 multiple choice questions.

At that time, the participant was also told how the test would be administered—

either via paper-pencil or computerized. After 5 minutes, the reading passage

was taken from the participant to begin testing.

Participants assigned to the paper-pencil/regular time or paper-pencil/

extended time treatment were given a paper copy of the 11 multiple-choice

questions. They were instructed that they could write on the test itself and

they were to mark their answers on a standardized electronic scoring response

448 / LEE, OSBORNE AND CARPENTER

Table 1. Participant Demographics

Variable N

Gender

Male

Female

Age

18-20

21-23

24-28

29-40

Over 40

Ethnicity

African American

Asian

Caucasian

Hispanic or Latino/a

7

24

9

11

5

5

1

1

1

23

6

Page 7: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

form, routinely used in the university setting. Those in the regular time con-

dition were given 1 minute per question, for a total of 11 minutes to complete the

exam. Those assigned to the extended time condition were given 1.5 minutes

for each question, for a total of 16.5 minutes. Decisions about allowance of

time for “regular” versus “extended” time participants were based on Bolt,

Roach, and Quenenmoan’s (2009) work on accommodations. According to these

researchers, the “standard” accommodation is “time and a half.” An “accepted” if

not written standard in academia for non-accommodated students is 1 minute

per multiple-choice question. Based on these conclusions, 1 minute per question

was considered “regular” time and 90 seconds per question was considered

“extended” time.

Participants assigned to the computerized-regular time or computerized-

extended time treatment took their tests via computer. Those in the regular time

condition had 1 minute per question, for a total of 11 minutes to complete the

exam. Those in the extended time condition were given 1.5 minutes for each

question, for a total of 16.5 minutes.

Upon completion of the assigned testing condition, each participant was

given a written questionnaire to complete. After the participant finished filling

out the questionnaire, the examiner interviewed each participant. The interview

served as respondent validation of the written questionnaire (Creswell, 2007).

The examiner read each participant’s responses aloud and probed for clarifica-

tion and validation of the responses. The examiner recorded the participant’s

remarks verbatim on the written questionnaire and later typed participant

responses to provide consistency and readability. Following the interview, each

participant received $20.

TESTING ACCOMMODATIONS / 449

Table 2. Student Questionnaire

1. What elements of this testing environment were helpful to you?

2. What elements of this testing environment were not helpful to you?

3. In your experience as a student, what have you found to be the most

effective testing accommodations in managing your AD/HD symptoms?

4. If you had the choice to take the same test either by taking a paper-pencil

test or taking it on the computer, which would you prefer? Why?

5. If you were to take a computerized test, would you prefer that the multiple-

choice questions be projected on the screen one at a time, or would you

prefer having a list of several questions on the screen at one time? Please

explain.

6. As a college student, what type(s) of academic assistance and services

would help you to perform more successfully?

Page 8: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

RESULTS

Quantitative Data Analysis

Test score data were analyzed comparing the test format (paper-pencil

versus computerized) and time (regular versus extended time) using analysis

of variance (Isaac & Michael, 1997). We hypothesized that students would

perform better on the computerized version than those given the paper-pencil

version of the same test. We predicted that no differences would be found in the

scores of those students in the regular and extended time conditions. Although

we gathered data from both male and female participants, the already small

sample size, overall, and extremely small number of male participants (7 out of

31) precluded including gender as a variable in the ANOVA.

Data analyses demonstrated that both hypotheses were confirmed. Students

who took the computerized version of the test performed significantly better

(Mean of 10.06) than those who took the paper and pencil version of the same

test (Mean of 9.08), F(1, 29) = 8.937, p = .014. An effect size computation

resulted in a Cohen’s d, for this main effect of .3389. This places the effect size

between Cohen’s description of a small (.20) effect size and a medium (.50)

effect size (Cohen, 1992).The time allocation of regular or extended did not

make a significant difference on the participants’ test scores across both types

of tests (Means of 9.69 and 9.61 for regular and extended time, respectively).

It is important to note, however, that both groups (paper-pencil and computer-

ized) performed better with extended time than with regular time although

the difference (overall) was not significant. It is also important to note that

there was a weak (close to significant) interaction between type of test and

test time. In other words, the extended time showed a trend toward making

more of a difference on test scores for the computerized test takers (Mean of

10.57) than for the paper and pencil test takers (Mean of 9.43), F(1, 29) = 4.102,

p = .066. The means and standard deviations for both sets of analyses are pre-

sented in Tables 3 and 4.

Qualitative Data Analysis

The open-ended questionnaire and interview responses were typed verbatim.

First, the participant responses were organized question-by-question into tables.

Next, we analyzed the sorted data by meaning field analysis to determine emergent

categories. Then we used open coding, axial coding, and selective coding (Gee,

1999) to determine the core categories. Our final step in the qualitative analyses

was interpreting the data and determining the applicability of the findings

(Lincoln & Guba, 1985). The collection of emergent themes provided valuable

insights regarding testing concerns shared among college students diagnosed

with AD/HD.

450 / LEE, OSBORNE AND CARPENTER

Page 9: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

Beneficial Environmental Factors

The most commonly cited beneficial environmental factors were “quiet” (with

many respondents preferring to be alone); an environment of “low distractibility”

(minimal room activity and minimal furnishings); “comfortable environment”

(comfortable chair/desk, workspace, ample room so as not to feel confined);

“a window with natural light” (without fluorescent lighting); and “comfortable

temperature”. Some participants recommended that the room have relaxing colors

of paint and soft music to enhance their performance.

Non-Beneficial Environmental Factors

The most common concern was time. Participants often cited time and the

pressure associated with a time limit as a stressful condition. Several participants

reported being distracted by other environmental conditions such as wind noise

outside, fluorescent lights, and being next to a window. A few participants said

the computer was not helpful. For example, one participant said, “I would prefer

not to take tests on computers.” Another said, “The image on the computer seemed

to vibrate.” Interestingly, several students reported no criticism of the testing

environment. They said that the existing environment was sufficient to meet

their needs.

TESTING ACCOMMODATIONS / 451

Table 4. Mean and Standard Deviations for Test Scores

as a Function Test Time

Regular time Extended time

Test time Mean SD Mean SD

Paper-pencil

Computerized

8.67

9.73b

1.21

1.55

9.43

10.57b

.95

.54

aSignificant difference based on type of test.

Table 3. Mean and Standard Deviations for

Test Scores as a Function of Format

Test conditions Mean SD

Paper-pencil

Computerized

9.08

10.06a

.954

1.305

aSignificant difference based on type of test.

Page 10: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

Most Effective Testing Accommodations in

Managing AD/HD Symptoms

Students most frequently said that extended time and a quiet environment

were their primary concerns for managing their AD/HD symptoms in a testing

environment. Additionally, a few mentioned that they required natural light,

ample space, and a comfortable temperature to perform most successfully.

Choice of Paper-Pencil or Computerized Test

Paper and pencil was preferred over a computerized test by the majority of

participants. Participants who preferred paper-pencil cited their reason as being

their ability to write on the test. Other responses included, “I can quickly recheck

an answer”; “I can take notes on the paper”; “It allows me to mark the question

and go back”; and “I can make marks and cross stuff out.”

Computerized testing provided various advantages according to several

students: “I am able to read faster from the screen”; “There’s no chance of

mis-bubbling the Scantron”; “I can type much faster than I can write”; “I don’t

worry about spelling”; and “It’s easier to go back and change answers”. Several

respondents stated that “it doesn’t matter” if the test is paper-pencil or computer.

Layout Preference of Computerized

Multiple-Choice Questions

The majority of participants said they would prefer one question at a time in a

computerized testing situation. Some remarks were: “My eyes jump around if

there are more than one”; “I can focus better and not be distracted by the other

questions”; “Otherwise, I lose my place a lot”; “Because the other questions

distract me”; and “If I see all the questions it makes me feel like I have to rush

through them.” Two students stated that they would prefer one at a time, but

would prefer the option to view the entire test when they wanted to.

A few participants stated that they would prefer several questions presented

on the screen at a time. Reasons included: “I can do them out of order”; “Seeing

all questions gives you information that is useful”; “If I get stuck I can move on

to the ones I know”; and “Seeing others might help me answer one.”

The Ideal Testing Environment

Most participants stated that their ideal testing environment is one that is quiet,

allows them to be alone, has no time limit, and provides a comfortable chair

and workspace. Several participants reported that they would like the option of

having soft music in the background to serve as a filter to help them mask other

auditory distractions. Several reported a need for relaxing colors of paint, few

distracting visuals, and a break that allows them to walk.

452 / LEE, OSBORNE AND CARPENTER

Page 11: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

DISCUSSION

Since a primary component of AD/HD is difficulty in “attending” to an

important stimulus (American Psychiatric Association, 1994), distractibility

during classroom examinations is a pronounced problem for college students

with AD/HD. Consequently, common accommodations have included a separate

and quiet testing environment. Additionally, assuming that distractibility and

inattention consume more time when compared with an individual who focuses

more easily on a task, extended time for exams has also become commonplace

(Farrell, 2003; Lancaster et al., 2001). Surprisingly, little research has been

conducted to determine the efficacy of such existing testing accommodations

for college students. The research literature is also in need of new and innovative

strategies for accommodating diverse student needs. Not only are educators

encouraged to be responsive to the needs of an increasingly diverse student

population, they are also encouraged to model and facilitate student learning

with contemporary technological tools. We examiners of the current study

believe that “time is of the essence” in education. We must successfully accom-

modate the needs of our changing demographic of students, and we must be

efficient in doing so. The current study was undertaken with the goal of deter-

mining the efficacy of not only the commonly existing AD/HD testing accom-

modation strategies of extended time for testing, but also to offer CBT as an

accommodation strategy. Furthermore, although quantitative research and result-

ing data are invaluable, it was our belief that qualitative information, particularly

student feedback, was equally important in the consideration of what accom-

modations may or may not be viable in practice.

Our study employed commonly existing testing strategies of paper-pencil

testing, and regular or extended time as a condition for some participants, while

other conditions employed the variables of computerized testing with regular

or extended time. All conditions utilized the most common accommodation of

a quiet individualized environment. As reported, the quantitative data analysis

supported the hypothesis that students performed better on a computer-based than

paper-pencil test and is supported in the literature as cited previously in studies

by Burk (1998), Calhoon et al. (2000), and Russell and Plati (2000). It is essential

to note, however, that our findings must be considered within the light of an

admittedly small sample size, especially for conducting a 2 × 2 factorial analysis.

The small sample size of this study also decreased the generalizability of our

findings. The difficulty in solicitation of students diagnosed with AD/HD and

registered with ODS by way of posted announcements contributed to the small

sample size as well as the challenges of many of the participants in following

through with their scheduled appointments to participate in the study. A larger

sample size would allow for identifying differences among participants related to

gender, medication regime, and diagnosis of other possible co-existing learning

disabilities. A larger sample size would also afford a detailed investigation of other

TESTING ACCOMMODATIONS / 453

Page 12: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

important design factors such as the formatting of the paper-pencil and CBT

tests. The CBT test was formatted according to the existing constraints of the

assessment feature in the university’s groupware which only allowed for testing

items to be viewed on the screen one at a time. The paper-pencil version of the

test was administered in the traditional format of the participant receiving all

questions of the test in one document and recording the responses on a stan-

dardized electronic scoring response form.

Although the quantitative data found that students performed better in CBT,

the qualitative data suggested that the majority of students preferred paper-pencil

tests over computerized ones. They cited a preference for being able to write on

and make notes on the test. One reason for the preference of paper-pencil over

computer testing might be one of familiarity (comfort) with paper-pencil, while

few may have much prior experience with a computerized testing environment.

Furthermore, it must be noted that the ability to write on the test may be viewed

as an accommodation, since writing on tests is sometimes not allowed in

the university classroom testing. In the interest of accommodating individual

student needs, perhaps college students with AD/HD could be given a choice of

paper-pencil or computer, with the willingness and approval of the instructor.

Computer testing also could be offered with the ability to make notations in

the margins, skip questions and return to them, and/or view one at a time or as

many as desired.

Quantitative data analysis also showed no significant difference in performance

in the conditions of regular time vs. extended time in either paper-pencil or

computerized testing conditions. Again, analysis of the qualitative data suggested

that extended time is a preferred accommodation as reported by college students

with AD/HD. However, a closer look at the quantitative data did indicate that

overall scores were higher (although not reaching the level of significance) in

the condition of extended time, whether paper-pencil or computer. This finding

is also supported in previously reported studies by Medina (2000), Zuriff

(2000), and Weaver (2000). It should be mentioned, however, that only Zuriff

(2000) found extended time to be a benefit to learning disabled students only,

as compared to all students. Overall, it seems that the statistical trend of higher

scores with extended time, along with extended time as a preference of students

with AD/HD, supports extended time as a viable accommodation.

Qualitative data further reinforced the assumption that reducing distractibility

will assist students with AD/HD in performing better on exams. When asked

about other variables that might impact their testing performance, several partici-

pants reported a variety of preferences such as a quiet environment with com-

fortable yet few furnishings, a comfortable temperature setting, and natural

light. Some participants reported that they would prefer background music while

others said they preferred a quiet environment. The reported variety of individual

preferences demonstrates a need to individualize accommodations in order to

provide students with AD/HD the most effective accommodations.

454 / LEE, OSBORNE AND CARPENTER

Page 13: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

In conclusion, findings indicated that college students with AD/HD scored

significantly higher with computerized testing as compared to paper-pencil. This

supports CBT as a viable and beneficial accommodation for university disability

services to utilize. While several students with AD/HD stated a preference of a

traditional paper-pencil format, they may likely adapt to computerized testing

over time as their skills with computers increase and as computerized testing

becomes more sophisticated with the ability to make notes in the margin, etc.,

as previously mentioned. Although computerized testing may at first represent

more effort on the part of educators, it may prove to be more efficient and cost

effective over time. In fact, computers may provide a distinct advantage to

paper-pencil for all students. Grading and recording grades may be more easily

accomplished with an electronic computer-testing format. Perhaps college

students with disabilities could return to the general classroom in which all

students were afforded choices in testing accommodation, including computerized

versions of tests, that would best accommodate individuals’ diverse academic

needs. As researchers continue to investigate the diverse needs of college student

populations, such as those struggling with AD/HD, it may also contribute to our

overall understanding of how best to educate all students and creatively utilize

technology to improve learning.

REFERENCES

Allsopp, D., Minskoff, E., & Bolt, L. (2005). Individualized course-specific strategy

instruction for college students with learning disabilities and AD/HD: Lessons learned

from a model demonstration project. Learning Disabilities Research and Practice,

20(2), 103.

Alster, E. H. (1997). The effects of extended time on algebra test scores for college

students with and without learning disabilities. Journal of Learning Disabilities, 30(2),

222-227.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental

disorders (4th ed.). Washington, DC: Author.

Bolt, S. E., Roach, A. T., & Quenenmoan, R. F. (2009). Inclusive assessment and account-

ability: A guide to accommodations for students with diverse needs. New York:

Guilford Press.

Brinckerhoff, L. C., McGuire, J. M., & Shaw, S. F. (2002). Postsecondary education and

transition for college students with learning disabilities. Austin, TX: Pro-Ed.

Brown, P. J., & Augustine, A. (2001). Screen reading software as an assessment accom-

modation: Implications for instruction and student performance. (ERIC Document

Reproduction Service No. ED458273) Retrieved February 2, 2008, from ERIC

database.

Burk, M. (1998). Computerized test accommodations: A new approach for inclusion and

success for students with disabilities. Paper presented at Office of Special Education

Program Cross Project Meeting “Technology and the Education of Children with

Disabilities: Steppingstones to the 21st Century.

TESTING ACCOMMODATIONS / 455

Page 14: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

Calhoon, M. B., Fuchs, L. S., & Hamlett, C. L. (2000). Effects of computer-based test

accommodations on mathematics performance assessments for secondary students

with learning disabilities. Learning Disability Quarterly, 23(4), 271-282.

Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.

Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods

approaches. Thousand Oaks, CA: Sage Publications.

Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five

approaches. Thousand Oaks, CA: Sage Publications.

Dowrick, P. W., Anderson, J., Heyer, K., & Acosta, J. (2005). Postsecondary education

across the USA: Experiences of adults with disabilities. Journal of Vocational

Rehabilitation, 22, 41-47.

DuPaul, G. J., & Eckert, T. L. (1998). Academic interventions for students with attention-

deficit/hyperactivity disorder: A review of the literature. Reading & Writing Quarterly,

14(1), 59. Retrieved from Education Research Complete database.

DuPaul, G. J., Guevermont, D. C., & Barkley, R. A. (1991). Attention deficit-hyperactivity

disorder in adolescence: Critical assessment parameters. Clinical Psychological

Review, 11, 233-245.

DuPaul, G. J., & Weyandt, L. L. (2006). School based interventions for children and

adolescents with attention deficit/hyperactivity disorder: Enhancing academic and

behavioral outcomes. Education and Treatment of Children 29(2), 341-358.

Elliott, S. N., McKevitt, B. C., & Kettler, R. J. (2002). Testing accommodations research

and decision making: The case of ‘good’ scores being highly valued but difficult to

achieve for all students. Measurement & Evaluation in Counseling & Development,

35(3), 153-167.

Farrell, E. F. (2003). Paying attention to students who can’t. Chronicle of Higher Educa-

tion, 50(5), 50-52.

Fayyad, J., De Graaf, R., Kessler, R., Alonso, J., Angermeyer, M., Demyttenaere, K.,

et al. (2007). Cross-national prevalence and correlates of adult attention-deficit

hyperactivity disorder. The British Journal of Psychiatry, 190, 402-409.

Fuchs, L. S., Fuchs, D., & Capizzi, A. M. (2005). Identifying appropriate test accom-

modations for students with learning disabilities. Focus on Exceptional Children,

37(6), 1-8.

Gee, J. P. (1999). An introduction to discourse analysis: Theory and method. New York:

Routledge.

Getzel, E. E., McManus, S., & Briel, L. W. (2004). An effective model for college

students with learning disabilities and attention deficit hyperactivity disorders.

LD Online. Retrieved from the Web at http://www.txstate.edu/education/sped/ on

1/20/07.

Greenbaum, B., Graham, S., & Scales, W. (1995). Adults with learning disabilities:

Educational and social experiences during college. Exceptional Children, 61(5),

460-472.

Healy, M. T. (2006). Coping strategies of successful college students with attention

deficit hyperactivity disorder. Dissertation Abstracts International: Section B: The

Sciences and Engineering, 66(11-B), 6273.

Heiligenstein, E., Guenther, G., Levy, A., Savino, F., & Fulwiler, J. (1999). Psychological

and academic functioning in college students with attention deficit hyperactivity

disorder. Journal of American College Health, 47(4), 181-185.

456 / LEE, OSBORNE AND CARPENTER

Page 15: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

Hollenbeck, K., Tindal, G., Harniss, M., & Almond, P. (1999). The effect of using

computers as an accommodation in a statewide writing test. Eugene, OR: University

of Oregon, BRT.

Isaac, S., & Michael, W. B., (1997). Handbook in research and evaluation: A collection

of principles, methods, and strategies useful in the planning, design, and evaluation

of studies in education and the behavioral sciences. San Diego, CA: Educational

and Industrial Testing Services.

Kessler, R. C., Adler, L., Barkley, J., Biederman, J., Conners, C. K., Demler, O., et al.

(2006). The prevalence and correlates of adult AD/HD in the United States: Results

from the National Comorbidity Survey replication. American Journal of Psychiatry,

163(4), 716-723.

Lancaster, S., Mellard, D., & Hoffman, L. (2001). Experiences of students with disabilities

in selected community and technical colleges. The individual accommodations model:

Accommodating students with disabilities in post-secondary settings. Lawrence, KS:

Kansas University, Center for Research on Learning (ED452617).

Lee, K. S., Osborne, R. E., Hayes, K. A., & Simoes, R. A. (2008). The effects of pacing

on the academic testing performance of college students with ADHD: A mixed

methods study. Journal of Educational Computing Research, 39(2), 123-142.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.

Meaux, J. B., Green, A., & Broussard, L. (2009). AD/HD in the college student: A block

in the road. Journal of Psychiatric & Mental Health Nursing, 16(3), 248-256.

Medina, J. G. (2000). Classroom testing accommodations for postsecondary students

with learning disabilities: The empirical gap. Dissertation Abstracts International

Section A: Humanities and Social Sciences, 60(7-A), 23-72.

Niebling, B. C., & Elliott, S. N. (2005). Testing accommodations and inclusive assess-

ment practices. Assessment for Effective Intervention, 31(1), 1-6.

Quinn, P. O., Ratey, N. A., & Maitland, T. L. (2000). Coaching college students with

AD/HD. Silver Spring, MD: Advantage Books.

Resnick, R. J. (2005). Attention deficit hyperactivity disorder in teens and adults: They

don’t all outgrow it. Journal of Clinical Psychology, 61(5), 529-533.

Russell, M., & Plati, T. (2000). Mode of administration effects on MCAS composition

performance for grades four, eight and ten. A report prepared for the Massachusetts

Department of Education.

Salvia, J., Ysseldyke, J. E., & Bolt, S. (2010). Assessment in special and inclusive

education (11th ed.). Belmont, CA: Wadsworth, Cengage Learning.

Sarkis, S. M. (2008). Success for the AD/HD college student. AD/HD Report, 16(5), 1-5.

Schultz, T. (2008). Fiscal Year 2008: Overview of Active Students Served by Disability

Background.

Swartz, S., Prevatt, F., & Proctor, B. (2005). A coaching intervention for college stu-

dents with attention deficit/hyperactivity disorder. Psychology in the Schools, 42(6),

647-656.

Tagayuna, A., Stodden, R. A., Chang, C., Zeleznik, M. E., & Whelley, T. A. (2005).

A two-year comparison of support provision for persons with disabilities in post-

secondary education. Journal of Vocational Rehabilitation, 22, 13-21.

Thompson, S., Blount, A., & Thurlow, M. (2002). A summary of research on the effects

of test accommodations: 1999 through 2001 (Technical Report 34). Minneapolis,

MN: University of Minnesota, National Center on Educational Outcomes.

TESTING ACCOMMODATIONS / 457

Page 16: Testing Accommodations for University Students with AD/HD: Computerized vs. Paper-Pencil/Regular vs. Extended Time

Tindal, G., & Fuchs, L. (2000). A summary of research on test changes: An empirical

basis for defining accommodations. 125 pp. (ED442245) Full Text from ERIC by

Ebsco Host.

Trammell, J. K. (2003). The impact of academic accommodations on final grades in a

postsecondary setting. Journal of College Reading and Learning, 34(1), 76-91.

Wallace, B. A., Winsler, A., & NeSmith, P. (1999). Factors associated with success for

college students with AD/HD: Are standard accommodations helping? Paper pre-

sented at the Annual Meeting of the American Educational Research Association,

Montreal, Quebec, Canada.

Weaver, S. M. (2000). The efficacy of extended time on tests for postsecondary students

with learning disabilities. Learning Disabilities: A Multidisciplinary Journal, 10(2),

47-56.

Weyandt, L. L., & DuPaul, G. (2006) AD/HD in college students. Journal of Attention

Disorders, 10(1), 9-19.

Wise, S. L., & Plake, B. S. (1989). Research on the effects of administering tests via

computers. Educational Measurement: Issues and Practices, 8(3), 5-10.

Zuriff, G. E. (2000). Extra examination time for students with learning disabilities:

An examination of the maximum. Applied Measurement in Education, 13(1), 99-118

(AN 3341986).

Zwart, L. M., & Kallemeyn, L. M. (2001). Peer-based coaching for college students with

AD/HD and learning disabilities. Journal of Postsecondary Education and Disability,

15(1), 1-15.

Direct reprint requests to:

Dr. Kathryn S. Lee

Texas State University

601 University Drive

San Marcos, TX 78666

e-mail: [email protected]

458 / LEE, OSBORNE AND CARPENTER