Migrating your Paper-n-Pencil Test to the Computer with Security in Mind Jamie Mulkey Caveon Test Security Laura Culver-Edgar NCCAOM.

Migrating your Paper-n-Pencil Test to the

Computer with Security in Mind

Jamie Mulkey Caveon Test Security

Laura Culver-EdgarNCCAOM

® Caveon, 2006

Secure Test Administration

RemediationPrevention Detection

® Caveon, 2006

Agenda

Why move to CBT? Prevention (Before)

Policy and process considerations Detection (During)

Monitoring candidates Remediation (After)

Candidate, record, and data management

® Caveon, 2006

Making the decision to move to CBT Better access and distribution of

testing program More efficient testing More flexible testing Ability to develop/deliver

innovative item types Better test security…

® Caveon, 2006

Security Advantages

Ability to turn items on and off as needed. Bad items, compromised items, overexposed

items. Unlikely that people will get the same

items CBT, especially CAT can help in this regard

Ability to investigate item-level data better Response time, unexpected change in ability

during the test, etc. Faster reporting mechanisms

Important in cases of irregular behavior

® Caveon, 2006

Test Administration (Before)

Proper identification procedures Define authorized/unauthorized

devices Process to report any discrepancies Proctors and other test

administration personnel trained Interventions are used when

suspicious activity occurs

® Caveon, 2006

Security Measures - Biometrics

® Caveon, 2006

Testing continuously vs. testing windows Size of testing population and the availability

of testing sites/seats Length of test and availability of testing

sites/seats Monitoring of item exposure through internet

braindumping/forums/chatrooms Monitoring of possible item exposure in

various time zones Results to candidate are released

immediately or after a period of time Size of your item pool Test administration: form-based or CAT

® Caveon, 2006

Test Administration (During)

Managing candidate identification biometrics

Continuous monitoring or examinees Personnel, video/audio monitoring

Physical control over unauthorized material and devices Physical space to hold personal items

Taking advantage of computer technology

Managing an intervention when suspicious activity occurs Halt testing, notification and escalation

® Caveon, 2006

Confirming exam exposure with Data Forensics™Helps validate level of

exposure Has a test been compromised

and to what extent? When and where was it

compromised? Who are the potential

culprits?

® Caveon, 2006

Caveon Case Study: Global Certification Exams - Effect on Test Performance

® Caveon, 2006Incidence of Test Abuse by Exam

Exam 425 has experienced the greatest incidence of test abuse with suspected test abuse occurring in ~180 of every 1000 tests.

High instance of collusion and cheating indicate probable sale and/or sharing of test content on the Internet.

® Caveon, 2006

Test Administration (After)

Relevant data immediately forwarded to the test sponsor

Score validation and reporting Using Data Forensics to Identify

and: Invalidate scores Manage rogue test centers Manage item exposure

® Caveon, 2006

Remediation

Execution of policies Incident response matrix

Sanctions for poor-performing sites Investigation Warning Closure

Cease & Desist / Take down

® Caveon, 2006

Tier 1

® Caveon, 2006

Security Planning

Roles and Responsibilities Security Budget Policies and Procedures Putting an overall plan together

® Caveon, 2006

Questions & Answers

® Caveon, 2006

Thanks for attending!

Jamie Mulkey, [email protected]

916 652-4017 phonewww.caveon.com

Laura Culver-Edgar, MBA, MT(ASCP) 703 299-5891

[email protected]

Please contact us: