An Introduction to Computer- assisted Assessment Joanna Bull and Ian Hesketh CAA Centre Teaching and Learning Directorate.

An Introduction to Computer-assisted Assessment

Joanna Bull and Ian HeskethCAA Centre

Teaching and Learning Directorate

Overview

What is CAA?

Why use it?

Objective testsIntegration: assessment

methods & strategies

Hands-on session

Computer-assisted Assessment I

Computer-assisted assessment (CAA) is a common term for the use

of computers inthe assessment of student learning.

Computer-assisted assessment II

Deliver, mark and analyse assignments or examinations (computer or web-

based) Collate and analyse data gathered from optical mark readers (OMR) Record, analyse and report on achievement Collate, analyse and transfer assessment

information through networks

Assessment

Formativeprimarily to provide feedback to students

and promote student learning Summative

contributes to final grade for course/module

Diagnosticto determine ability and understanding

Self-assessmentto allow students to evaluate their learning

Why use CAA?

What are the benefits to using CAA?

Why use CAA ?

Increasing the frequency of assessment

Increasing feedback

Widening the range of assessment methods

Provide objectiveand consistent marking

Decrease marking

Administrative efficiency

Increasing the frequency of assessment

Motivating students to learn

Encouraging students to

practice skills

Broadening the knowledge

assessed

Increasing feedback

Students to motivate students needs to be timely, accurate and

constructive

Staff about student performance about test and question performance

Widen range of assessment methods

Different assessment methods for different skills, abilities and knowledge

Balance of assessment methods

Utilisation of multimedia to create innovative assessments

Provide objective and consistent assessment

Assessment can be unreliable, subjective and inconsistent Automated marking enables consistent, objective assessment for large groups Rigorous evaluation of questions ensures overall consistency and standards.

Decrease marking

Potential time savings:

dependent on level of implementation

must be offset against investment in design

may represent a re-distribution of workload

may demand a cultural shift

Administrative efficiency

Automatic transfer of data to MIS, or

SRS

Examination papers and handling

Speed of results

Objective Tests

Examples of question types

Guidelines for writing items

Assessing skills and abilities

Feedback and scoring

Objective Tests

Require students to provide a response to a question with a pre-determined answer:

select a solution from a set of choices identify an object or position supply brief numeric or text responses

Question Types

Multiple choice Multiple response Assertion-reason Graphical hotspot Text/numerical Matching/ranking Sore finger questions Sequencing Assembly Graphing

Designing questions I

Questions single, definitive statement avoid unnecessary or irrelevant

material use clear language use negatives sparingly put as much of the question in the

stem do not use double negatives

Designing questions II

Distractors ensure there is only one correct answer use plausible distractors avoid clues to the correct answer avoid ‘all of the above’ and ‘none of the

above’ base distractors on common student errors avoid ambiguity

Blooms Taxonomy

knowledge, comprehension, application, analysis, synthesis and evaluation

Example questions

Feedback

Objective tests allow provision of :timely, specific formative feedback to

studentsfast and detailed feedback to staff

Enables a consistent level of feedback to large student groups Can focus and direct learning through

anticipated problem areas Enables curriculum design modification during a course

Scoring

One score per correct

answer

Issue of guessing

Negative marking

Discrimination and facility

Discrimination power of an individual question to

differentiate between weaker and better students

Facility measures the difficulty of a question by dividing the number of correct responses by the total number of responses.

Integration: assessment methods & strategies

Appropriateness and balance

Quality of feedback and student learning

Extending the skills and knowledge assessed

Test structure II

Analyse materialcontentdifficultyassessment typequestion style

Identify areas of low coverage and uneven spread

CAA Website

CAA Centrehttp://caacentre.ac.uk

email: [email protected]

CAA in business studies

Introduction to financial accounting Intermediate/advanced financial

accounting Principles of finance Production and operations management Professional certificate in management

Advantages of CAA-business studies

respondents

Speed Remote access Anonymity Neutrality Quick feedback Wide course coverage

Disadvantages of CAA-business studies

respondents

Inflation of students' marks

Difficult to set appropriate questions

Steep learning curve/time investment

Fails to develop students' argumentation skills

CAA Website

CAA Centrehttp://caacentre.ac.uk

email: [email protected]