Final atp2017 smarter proctoring

#atpconf#atpconf

Frequencies of Academic Integrity Violations Across Multiple Proctoring Modalities

Dr. Mac Adkins – CEO & Founder, SmarterServicesRick Beaudry – CEO, B Virtual

WK  Conference  :  testing2017

#atpconf

ABOUT

the

PRES

ENTE

RAbout the Presenters …

■ Dr. Mac Adkins, CEO & Founder, SmarterServices§ Test Publisher-­ SmarterMeasure Learning Readiness Indicator – Taken by 4 million students.

§ SmarterAssessments – Adaptive Assessment Engine§ SmarterProctoring – Multi-modal Proctoring Management

■ Rick Beaudry, CEO, B Virtual§ UPCEA Corporate Industry Advisory Council

3/6/17 2

#atpconf

Research in Progress

3/6/17 3

■ Kennesaw State University§ Dr. Michael Goodwin, Academic Integrity Coordinator§ Since 2011 over 11,000 students have taken SmarterMeasure§ 372 matches of students with academic integrity violations§ Statistical analysis to identify non-cognitive factors that are

indicators of likelihood to engage in academic integrity violations-­ Time management, -­ Availability of Time, -­ Motivation, -­ Procrastination, -­ Technology Skills, -­ Keyboarding skills, etc.

#atpconf

Abhorrent Behavior Frequencies in Virtual Proctoring

3/6/17 4

#atpconf

Abhorrent Behavior Frequencies - 90 day

3/6/17 5

Unsecure  environment24%

Missing  ID16%

Unapproved  Items13%

Invalid  ID10%

Unable  to  Read  ID8%

Communicating  with  Others

7%

Eye  Movement5%

Bio  Break4%

Blackboard/Whiteboard4%

Unable  to  view  face  during  exam

4%

Reading  out  loud3%

Standing  up2%

#atpconf

Peaks and Valleys of Virtual Proctoring

3/6/17 6

Virtual  Proctoring  Monthly  Variations   in  Volume

#atpconf

Top 4 Reasons for Technical Failures

3/6/17 7

■ 1. Connectivity Issues ■ 2. Audio issues ■ 3. Webcam issues ■ 4. Exam site/LMS issues ■ 5. Aberrant behavior issues

#atpconf

2016 Technical Failure Rates

3/6/17 8

Row  Labels Sum  of  CountAdobe  Acrobat  Failure 3Audio  issues 205Bomgar 1279Customer  exam  site 694Exam  Connection 245Internet/bandwidth  issues 836Java  Failure 7Megameeting 801Password  Issue 146Scheduling  software 5Skytap  issues 14Technical   Issues 50Technology  -­‐ Audio 78Technology  -­‐ Bandwidth 174Technology  -­‐ Browser  (IE  Settings) 25Technology  -­‐ Disconnect  Bomgar 827Technology  -­‐ Disconnect  MM 433Technology  -­‐ Webcam 245Web  Browser 104Webcam 498Grand  Total 6669

#atpconf

% refunds for wait time issues in 2016

3/6/17 9

■ January - 1.40%■ February - 0.84%■March - 1.62%■ April - 3.94%■May - 1.53%■ June - 1.53%■ July - 1.60%■ August - 1.40%■ September - 1.31%■October - 1.00%■ November - 1.30%■ December - 1.15%

#atpconf#atpconf

Test Your Proctoring Prowess

Results of the 2016 Annual Proctoring Survey

WK  Conference  :  testing2017

#atpconf

3/6/1711

What  comes  to  mind  when  I  say  …

Testing  Integrity

On  your  phone/laptop   go  to:

Menti.com    21  04  90

#atpconf

About The Survey

3/6/17 12

■ 35 day period in the Spring of 2016. ■ During this time 365 responses were received.

This is an increase from 284 the prior year.■ Each stakeholder group was presented with

questions specific to their role.

#atpconf

Common Proctoring Modality

■Which proctoring modality was reported by faculty as most commonly used?§ A – Approved Human Proctor§ B – Instructor as Proctor§ C – Local Test Center§ D – Virtual Proctoring

#atpconf

Common Proctoring Modality

■Which proctoring modality was reported as most commonly used?§ A – Approved Human Proctor§ B – Instructor as Proctor§ C – Local Test Center§ D – Virtual Proctoring

#atpconf

Common Learner Authentication

■ The most common form of learner authentication reported was:§ A – Key stroke monitoring§ B – User name / password§ C – Photo ID Scanning§ D – Security Questions

#atpconf

Common Learner Authentication

■ The most common form of learner authentication reported was:§ A – Key stroke monitoring§ B – User name / password§ C – Photo ID Scanning§ D – Security Questions

#atpconf

Satisfaction with Proctoring Modalities

■With which modality of proctoring did facultyreport the lowest level of satisfaction?§ A – Approved human proctor§ B – Virtual proctoring§ C – Automated Virtual Proctoring§ D – Corporate Testing Center

#atpconf

Satisfaction with Proctoring Modalities

■With which modality of proctoring did faculty report the lowest level of satisfaction?§ A – Approved human proctor§ B – Live, Virtual proctoring§ C – Automated Virtual Proctoring§ D – Corporate Testing Center

Note  that  they  were  most   satisfied   with  an  approved  human   proctor.

#atpconf

Psychological Deterrent

■ According to faculty, which proctoring modality was the highest level of psychological deterrent to cheating.§ A – Instructor proctored§ B – Approved human proctor§ C – Virtual proctoring§ D – Local test center

#atpconf

Psychological Deterrent

■ According to faculty, which proctoring modality was the highest level of psychological deterrent to cheating.§ A – Instructor proctored§ B – Approved human proctor§ C – Virtual proctoring§ D – Local test center

#atpconf

Psychological Deterrent

■ According to students, which proctoring modality was the highest level of psychological deterrent to cheating.§ A – Instructor proctored§ B – Approved human proctor§ C – Test Center§ D – Live, virtual proctoring

#atpconf

Psychological Deterrent

■ According to students, which proctoring modality was the highest level of psychological deterrent to cheating.§ A – Instructor proctored§ B – Approved human proctor§ C – Test Center§ D – Live, virtual proctoring

#atpconf

Threat to the Quality of eLearning

■What percentage of faculty considered issues related to proctoring a high level of threat to the reputation of the quality of eLearning. § A – 24%§ B – 32%§ C – 35%§ D – 39%

#atpconf

Threat to the Quality of eLearning

■What percentage of faculty considered issues related to proctoring a high level of threat to the reputation of the quality of eLearning. § A – 24%§ B – 32%§ C – 35%§ D – 39%

Learner  authentication   was  also  considered  a  high   threat.

#atpconf

Cost of Proctoring

■What is the average cost for a one-hour proctored testing session?§ A - $15§ B - $20§ C - $25§ D - $40

#atpconf

Cost of Proctoring

■What is the average cost for a one-hour proctored testing session?§ A - $15§ B - $20§ C - $25§ D - $40

#atpconf

Proctor’s Improvements

■What would proctors like to see improved in the proctoring process?§ A – Assignment scheduling§ B – Communications with learner§ C – Proctoring protocols§ D – Proctoring technology

#atpconf

Proctor’s Improvements

■What would proctors like to see improved in the proctoring process?§ A – Assignment scheduling§ B – Communications with learner§ C – Proctoring protocols§ D – Proctoring technology

#atpconf

Proctor Training

■ How did most proctors rate the training they have received for serving as a proctor?§ A – Very weak§ B – Weak§ C – Strong§ D – Very Strong

#atpconf

Proctor Training

■ How did most proctors rate the training they have received for serving as a proctor?§ A – Very weak§ B – Weak§ C – Strong§ D – Very Strong

#atpconf

Important Issues

■What are the most important issues to School Administrators relative to testing integrity?§ A – Cost to the learner§ B – Cost to the institution§ C – Learner authentication§ D – Proctoring policies

#atpconf

Important Issues

■What are the most important issues to School Administrators relative to testing integrity?§ A – Cost to the learner§ B – Cost to the institution§ C – Learner authentication§ D – Proctoring policies

School   leaders   are  least  concerned  with  proctoring   technologies.

#atpconf

Payment of Proctoring Fees

■When a fee is charged for proctoring services who is most likely to pay?§ A – School pays§ B – Learner pays§ C – Learner and school share the expense§ D – Depends on the situation

#atpconf

Payment of Proctoring Fees

■When a fee is charged for proctoring services who is most likely to pay?§ A – School pays§ B – Learner pays§ C – Learner and school

share the expense§ D – Depends on the situation

#atpconf

Preferred Proctoring Modality

■Which proctoring modality is most preferred by students?§ A – Approved human proctor§ B – Local test center§ C – Virtual proctoring§ D – Instructor proctored

#atpconf

Preferred Proctoring Modality

■Which proctoring modality is most preferred by students?§ A – Approved human proctor§ B – Local test center§ C – Virtual proctoring§ D – Instructor proctored

#atpconf

Difficulty to Cheat

■ According to students, in which proctoring modality would it be most difficult to cheat?§ A – Approved human proctor§ B – Corporate testing center§ C – Location such as a library§ D – Virtual proctoring

#atpconf

Difficulty to Cheat

■ According to students, in which proctoring modality would it be most difficult to cheat?§ A – Approved human proctor§ B – Corporate testing center§ C – Location such as a library§ D – Virtual proctoring

#atpconf

Test Center Volume

■ In the prior year the volume of tests proctored in collegiate testing centers:§ Substantially decreased from prior

years§ Decreased from prior years§ Remained about the same as prior

years§ Increased over prior years§ Substantially increased over prior

years

#atpconf

Test Center Volume

■ In 2015 the volume of tests proctored in collegiate testing centers:§ Substantially decreased from prior

years§ Decreased from prior years§ Remained about the same as prior

years§ Increased over prior years§ Substantially increased over prior

years

#atpconf

Spectrum of Proctoring Modalities

Lecture  Hall  /  Instructor  as  

ProctorCampus

Testing  CentersApproved  Proctoring  

Professionals

Monitoring  and/or  

Restricting  Technologies

Live,  Virtual  Proctoring

Biometric  Learner  

Authentication

Automated  or  Non-­‐live  Virtual  

Proctoring

SmarterProctoring   is  a  Holistic   Solution   that  Manages  All  Of  These  Modalities.

#atpconf

Software that works within your LMS that helps students find different types (Virtual, Testing Center, Human, etc.) of approved proctors, while also allowing the faculty/administrators to see where each student is in the test scheduling process and securely communicating all test details (passwords, allowable resources) to the proctor.

What  is  SmarterProctoring?

Proctoring Process Management System

#atpconf

Comments, Questions

■ Dr. Mac AdkinsCEO & [email protected]

■ Rick BeaudryCEOB [email protected]

#atpconf3/6/17 44

#atpconf3/6/17 45

#atpconf3/6/17 46

#atpconf

■Command Tab

3/6/17 47

#atpconf

3/6/1748

#atpconf