ICAT: Instructional Computer Adaptive Testing John Fountain Marine Earth and Atmospheric Sci. North Carolina State University.

ICAT: Instructional Computer Adaptive Testing

John Fountain

Marine Earth and Atmospheric Sci.

North Carolina State University

Active Learning• Value documented over 20 years

(Bonwell and Eison, 1991)

• Increasing active learning generally requires reduction in conventional lecture

• Background material often supplied through assignments outside of class time

Is External Material Retained?

• Must compete with distractions: computer games, social media, TV, partying etc.

• Easy for students to avoid necessary effort

• Quizzes: traditional approach to encourage attention to material

• Multimedia: videos etc. – to increase interest and hopefully use

ICAT

• Designed to increase mastery of out-of-class material

• Required Quiz: to maximize student use

• Instructional: Primary design objective, uses videos, images and text.

ICAT Instructional Computer Adaptive Test

• Online Computerized Test (quiz)

• Adaptive: wrong answers lead to additional questions on topic

• Instructional: instructional material presented after wrong answers

• Every question must eventually be answered correctly to complete exam

What Mineral is this?

a.Pyroxene b.Feldspar

c. Quartz d.Calcite

Wrong answers lead to additional questions on subject• The quiz is designed to address

knowledge needed for each answer

• This question requires a knowledge of mineral properties and identification of cleavage

• If they answer “quartz” then they either do not remember the properties of quartz or did not recognize cleavage

What mineral is this?a.Pyroxene b. Feldspar c. Quartz

This wrong answer leads to:Does this sample show cleavage? a. yes b.no

Instructional Material

• Wrong answers lead to instructional material targeted to knowledge area

• In this case on cleavage origin, significance and recognition

DoesThis

Sample Show Cleavage

What isThis

Mineralfeldspar -correct go to nextQ.

Quartz

No

Yes

Video On Cleavage

DoesThis

DoesThis

DoesThis

N

N

N Video

Video

Video

Y

Y

Y

DoesQuartzHaveCleavage?

Yes

Info OnMineral Properties

No

Flow Chart

• Every “main” question has a flow chart that defines the sub-question sequence and instructional material

• Sub-questions designed to cover knowledge areas needed to answer main question

Basic Concepts

• Wrong answers lead to information– Can be text, images, video

• Only information relevant to answer is shown

• Every Question must be answered correctly – eventually.

Reporting

• Automatic report generated for each class1. % first answer correct for each student

2. % answers correct for each question

3. Histogram of answers for each question

• Allows assessment of individual students, of each question and of extent of understanding

ICAT• Initial design for online courses

• Equally applicable to seated courses

• Primary use to increase mastery of material

• Can be used as study aid to prepare for tests

• Or as review to reinforce material

Design

• Web based exam, run from any browser

• Giving exam takes one-line email.

• Questions submitted on form

• Student Identities masked

ICAT Assessment • Comparison of performance of cohorts

taking intro geology lab

• Initial review by Teaching Assistants

• Second review by non-random group

• Modified program in response to each review thus results not yet appropriate for assessment

Scalable and Open Source

• Question bank database (MySQL), can handle any number of questions /courses/subjects

• Server (Apache Tomcat)– (Walmart > 100 million pages/day)

• Written in java (FX2): runs on any platform (except ipad)

Structure

•Client code is very small

•Requires minimal memory and computing power – any processor can handle it

•Secure – no access to database by client program

Future Plans

• Remainder of academic year for assessment and development

• Expand to other courses at NC State and Wake Tech Community Coll.

• Extend to other interested institutions

AcknowledgementsNational Science FoundationOpportunities for Enhancement of Diversity in Geoscience Program

Dr. David McConnellDr. John BedwardKatherine RykerGeoscience Learning Process Laboratory, Dept. of Marine Earthand Atmospheric Sciences, NC State

Doel Gonzalez, Computer Science DepartmentNC State