Page 1
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
eAssessment in the Cloud, Sunshine or Thunderstorm?
John Kleeman, Questionmark
www.questionmark.com
Page 2
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Using “clouds” for computing tasks promises a revolution in IT similar to the birth of the web and e-commerce (McKinsey)
What does this mean for eAssessment in HE?
Technology pros and cons Installing eAssessment software locally
“On-premise” Using an eAssessment service “On-
demand”
The promise of the cloud
Slide 2
Page 3
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark offer both, so I can be neutral
Installable assessment management system
You Purchase servers & software
Install software
Install updates
Create your own eAssessment offering based on the software
Questionmark provides service at cloud data center
You Contract for the service
Connect to the service over the Internet
Delegate running the service 24/7 to Questionmark
Use the service for your eAssessment offering
Slide 3
Page 4
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Access to innovation Speed/flexibility of deployment Reliability and uptime Scalability and peaks in demand Security and cheating Governments accessing your data Data protection Ease of customization/integration Connectivity Can you change providers? IT bandwidth Cost, features and other factors
Key factors
Slide 4
Page 5
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Key advantage of On-demand is frequent updates without effort Support of mobile devices
Support of new interoperability standards
Support of uptodate browsers
Feature improvements
Access to innovation
Slide 5
Page 6
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
With an On-demand system, you are usually upgraded automatically
Most providers will Give notice of upgrades
Give opportunity to test
Make sure upgrade is upwardly compatible
Not a problem in practice with a good provider
What happens if I don’t want innovation?
Slide 6
Page 7
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Speed/flexibility of deployment
On-premise On-demand
Hardware needs purchase
System needs provisioning
Software needs installing
Needs planning and can be slow to deploy or increase scale
Turn on within days Increase capacity easily Reduce capacity easily Ramp up for temporary
courses or one-off events
Slide 7
Page 8
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
For high reliability/uptime, redundancy is key Power supplies
Internet connectivity
Firewalls
Servers
Database servers
Application servers
Offsite backup
Everything needs a backup On-demand will usually be superior
as long as participants have Internet connectivity
Reliability and uptime
Slide 8
Page 9
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Typical On-demand redundancy
Internet
Off Site Data Back Ups
Tapes
Page 10
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
What SLA is available? What are support hours? Do they match our hours? Is there a public status page? What is historical uptime? What protection vs power outages? How often are results backed up, e.g. hourly? Are back-ups off-site? Is every server load-balanced? Is there redundancy for everything? Is there monitoring for failures? What happens with planned maintenance? How strong is the provider’s disaster recovery plan?
Some key questions to ask a provider
Slide 10
Page 11
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Typical eAssessment load: Formative: steady during
year Summative peak at end
of term
On-premise Need to scale system to
cope with peaks
On-demand Gives an advantage
Scalability and peaks in demand
Slide 11
Page 12
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Just like a castle Security needs investment &
vigilance
UCISA: “In practice, data is probably more secure in cloud services than can be provided by in house solutions.”
Security and cheating
Slide 12
Picture by Simon Ledingham from geograph project, licensed under Creative Commons
Page 13
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Data center security Is it certified?
Are there 24/7 onsite guards?
Are all visitors escorted and videoed?
Fire suppression, 24/7 monitoring?
Tiered architecture, firewalls between each tier?
All traffic uses HTTPS?
Separate production and staging/development areas?
Can developers access production area?
Security built into application development?
Some key questions to ask a provider?
Slide 13
Page 14
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Be realistic! Governments mostly well-intentioned Does government want your assessment data? Does it matter if they have it?
If you are concerned
Make sure service provider in an acceptable country If you are concerned re US Patriot act, then these should all be
outside USA Service provider
Data center
Data center owner
Can governments access my data?
Slide 14
Page 15
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Data protection
On-premise On-demand
The institution controls and is responsible for data protection
Institution = data controller Provider = data processor
Questions for provider
Employee confidentiality? Background checks? Employee training/testing? Data encrypted in transit? One tenant protected from
other? Hard drives degaussed?
Slide 15
Page 16
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Employees background checked?
Employees sign confidentiality agreement?
How do they monitor third party suppliers?
Training (and testing) in data security?
Are end-of-life disks degaussed?
Is data encrypted if moved?
If US, is provider enrolled in Safe Harbor
Some key questions to ask a provider
Slide 16
Page 17
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
On-demand On-premise
Configure and template
Change core code X
Direct database integration X
Web services
E-learning standards
Connect to Moodle, Blackboard etc
Ease of customization/integration
Slide 17
Page 18
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
In campus universities, local connectivity can favour On-premise
Speed of light also small factor
But … Ray Fleming, Microsoft (2011): I saw some statistics yesterday that says over 85% of the use
of the Deakin University LMS takes place outside of the University. So there is a huge logic for putting things into the Cloud, as your users are out on the Internet – not necessarily in the institution when using your systems
Connectivity
Slide 18
Page 19
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Whatever system you use Questions
Assessments
Results
Are your data
On-premise and On-demand Make sure that the format of data is documented and open
On-demand Make sure that you have the rights to copies of your data
Can you change providers?
Slide 19
Page 20
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
eAssessment and the IT department eAssessment critical so needs professional maintenance
IT departments overloaded, IT skills in short supply
eAssessment is specialist IT skill
Tasks performed once per year difficult to train for
What can happen in On-premise eAssessment projects have to queue for priority in IT
Often a bottleneck
A challenge which On-demand resolves
Getting IT bandwidth
Slide 20
Page 21
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
When considering cost include: Hardware
System software
License costs
IT time
Support costs
Other factors Contract
Trust
Features
Training and SLA
Costs, features and other factors
Slide 21
Page 22
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
• Access to innovation
• Speed/flexibility of deployment
• Reliability and uptime
• Scalability
• Security and cheating
• Getting IT bandwidth
On-demand may be stronger
• Ease of customization/integration
• Connectivity
• Governments accessing your data
On-premise may be stronger
• Data protection
• Can you change providers?
• Costs, features and other factors
Needs review
Summary
Slide 22
Page 23
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
UCISA Cloud Computing Briefing Paper (2011) Download from www.ucisa.ac.uk
How to decide between On-demand and On-premise eAssessment (presented at 2012 CAA conference) http://caaconference.co.uk/pastConferences/2012/caa2012_
submission_23.pdf
Some further resources
Slide 23
Page 24
Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Sunshine or thunderstorm: your questions
Slide 24