Lecture 3. Computer-based assessment

1

G E O R G I Y G . G E R K U S H E N K O P H D , A S S O C I A T E P R O F E S S O R

R U S S I A N F E D E R A T I O N , V O L G O G R A D S T A T E T E C H N I C A L U N I V E R S I T Y , D E P A R T M E N T O F " C O M P U T E R - A I D E D D E S I G N A N D S E A R C H E N G I N E

D E S I G N "

2 0 . 0 5 . 2 0 1 1

C H I N A , C H A N G C H U N , N E N U

COMPUTER-BASED ASSESSMENT

2

CONTENT

Defining of computer testing and its typesRequirements to testsAlgorithm of tests creationUnited state exam in Russia

3

DEFINITION

A Computer-Based Assessment (CBA), also known as Computer-Based Testing (CBT), e-assessment, computerized testing and computer-administered testing, is a method of administering tests in which the responses are electronically recorded, assessed, or both. As the name implies, Computer-Based Assessment makes use of a computer or an equivalent electronic device such as a cell phone or PDA. CBA systems enable educators and trainers to author, schedule, deliver, and report on surveys, quizzes, tests and exams. Computer-Based Assessment may be a stand-alone system or a part of a virtual learning environment, possibly accessed via the World Wide Web.

4

Benefits of Computer-Based Testing

• Flexible Test Administration & Test Development

• Enhanced Test Security• Data-Rich Test Results• Advanced Item Types• Consistent & Reliable Scoring• Immediate Score Reporting• Enhanced Logistics

5

Classificationteststests

structurestructure Way of giving exercises

Way of giving exercises

Way of involvement

Way of involvement

Form of implementing

Form of implementing

Content and goalsContent and goals

homogeneoushomogeneous

heterogeneousheterogeneous

InteractiveInteractive

CombinedCombined

Increasing complexity

Increasing complexity

AdaptiveAdaptive

IndividualIndividual

GroupGroup

WritingWriting

MachineMachine

PersonalPersonal

SocialSocial

Progress Progress

Special testsSpecial tests ProgressProgress Tests of overall progress

Tests of overall progress

Sport achievements

Sport achievements

Functional possibilities

Functional possibilities Professional

suitability

Professional suitability DevelopmentDevelopment IntellectIntellect

6

Classification

more…

7

Classification

What kind of tests could you use on www.udutu.com ?

Adobe Captivate 5

8

Classification

Type of evaluation:Objective type tests (on the base of

comparing answers with an etalon)

Recognition level Reproduction level Heuristic (creative) level

Subjective type tests (free form of answering – problem solving, practice e tc.)

9

Classification

InputPreliminary CurrentTopicalTransitional Final

10

CRITICAL PROPERTIES

VALIDITY– equivalence of test exercises to learning content and test results.

RELIABILITY.Test is reliable if we can see the same results for each person after second time of the same testing

11

Rules for test developing

Reasonable choice of scale for evaluation test results

The quantity of exercises should be 30–40 for science subjects and twice more for humanitarian ones

The quantity of exercises in a data base should be 10 times more then in one test (so 300-400 exercises).

12

Student should be informed about the quantity of exercises and time limit

Student should have opportunity to do demonstrative test at least one time before starting the attestation test. It is important for introduction the interface of testing program and ways of answering.

Rules for tests giving

13

During testing there should be only one exercise on the screen

The way of answering should be simple and comfortable

Rules for tests giving

14

Students opportunities during testing:

To submit each answer;

To correct answers before submission the answer;

To skip exercises more then one time;

15

To click the instruction for answering anytime;

To see the result immediately after finishing test.

Students opportunities during testing:

16

The main part of exercise is one sentence no more then 7-8 words

All the answers for one question should be approximately the same size

All the distractors should be the same for student who does not know the correct answer

There should not exist partly correct distractors which can become the correct answer in some conditions

Rules for test designing

Requirements to test exercises17

181111

The content of exercise should not contain repetitions or double negation

Each test exercise should be related with concrete fact, principle or skill

Test exercise should not contain subjective opinion of creator

18

Requirements to test exercises

There should not be unintentional hints or slang words

The time of showing exercise on the screen is 1,5 - 2 minutes.

In each open exercise the quantity of distractors should be 4 or 6, but in exercises for correspondence they should be twice more

19

Requirements to test exercises

Test exercise should be a narration (questions why, how, which etc. are not used).

Do not use words like “sometimes”, “often”, “always”, “big”, “small”, “many” etc.

20

Requirements to test exercises

The key word or definition should be in the first part of an exercise

It is better to make long exercise and short distractors than contrary

Stages of tests creationStages of tests creation

CHOOSING THE RESULTCHOOSING THE RESULT

STRUCTURING THE LEARNING CONTENTSTRUCTURING THE LEARNING CONTENT

SPECIALISING TEST EXCERSISES SPECIALISING TEST EXCERSISES

DESIGNING A TESTDESIGNING A TEST

21

Stages of tests creation(CONTINUED)

Stages of tests creation(CONTINUED)

EXPERTISING EXERSISES AND TESTEXPERTISING EXERSISES AND TEST

CORRECTION A TEST CORRECTION A TEST

ORGANISING A BASE OF TEST EXERSISESORGANISING A BASE OF TEST EXERSISES

CORRECTION OF EXERSISESCORRECTION OF EXERSISES

EVALUATING A TEST VALIDITYEVALUATING A TEST VALIDITY

22

Stages of tests creation(END)

Stages of tests creation(END)

ORGANISING EXPERIMENTAL TESTING ORGANISING EXPERIMENTAL TESTING

CORRECTION OF A TEST DATAS AND SCALE OF EVALUATIONCORRECTION OF A TEST DATAS AND SCALE OF EVALUATION

SERTIFICATION SERTIFICATION

PRELIMINARY SCALINGPRELIMINARY SCALING

23

24

United state exam (Russia)

25

[email protected]

THANK YOU FOR YOUR ATTENTION!

QUESTIONS…