The Role of e-Assessment in XXI Century Education: Opportunities and Risks Martin Ripley Managing Director, World Class Arena About WCAL Strategi c objectiv es Three case studies Risks & Issues
The Role of e-Assessment in XXI Century Education: Opportunities and Risks Martin Ripley
Feb 23, 2016
Strategic objectives. About WCAL. Risks & Issues. Three case studies. The Role of e-Assessment in XXI Century Education: Opportunities and Risks Martin Ripley Managing Director, World Class Arena. Strategic objectives. About WCAL. Risks & Issues. Three case studies. website. - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
The Role of e-Assessment in XXI Century Education: Opportunities and Risks
Martin RipleyManaging Director, World Class Arena
About WCAL
Strategic objectives
Three case studies
Risks & Issues
WCAL’s HistoryFormed in 2007
We specialise in the creation of learning resources for the digital age
We create assessment resources – such as school tests
Includes paper-based, on-screen and face-to-face learning events, with technology as a significant theme We operate globally and our particular niche is talent development. “Talent development” is concerned with identifying and supporting future leaders – in social, entrepreneurial, economic, political, scientific and literary fields
www.worldclassarena.net
www.worldclassarena.org.uk
www.worldclassarena.hk
website
contact
About WCAL
Strategic objectives
Three case studies
Risks & Issues
WCAL’s ServicesWe provide the following services:1. Authoring textbooks and related content2. Designing and running teacher training events3. Support and advice for companies considering
introducing technology into their learning and assessment businesses
4. Design and sale of 21st century assessment and testing products
5. Strategic support for schools looking to provide greater challenge and to raise the aspirations of their students
www.worldclassarena.net
www.worldclassarena.org.uk
www.worldclassarena.hk
website
contact
About WCAL
Strategic objectives
Three case studies
Risks & Issues
www.worldclassarena.net
www.worldclassarena.org.uk
www.worldclassarena.hk
website
contact
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
It is estimated that 40 exabytes (4.0 x 10 19) of unique new information will be generated worldwide this year....
About WCAL
Strategic objectives
Three case studies
Risks & Issues
2xThe amount of new technical information is doubling every 2 years
About WCAL
Strategic objectives
Three case studies
Risks & Issues
For students starting a three-year university degree,
this means that...
About WCAL
Strategic objectives
Three case studies
Risks & Issues
...half of what they learn in their first year of study will be outdated
by the end of their course.
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
Researchers have found fairly uniform effects:Assessments signal priorities for curriculum and instruction. School administrators and teachers pay attention to what is tested, analyse test results, and adapt curriculum and teaching accordingly. Teachers tend to model the pedagogical approach reflected in tests. When high visibility assessments are composed of multiple-choice items, teachers tend to rely heavily on multiple-choice worksheets.Curriculum developers, particularly commercial interests, respond to important tests by modifying existing textbooks and other instructional materials.
About WCAL
Strategic objectives
Three case studies
Risks & Issues
Research shows the potential for substantial negative consequences.
Schools and teachers tend to focus on what is tested rather than underlying standards or learning goals, and to ignore what is not tested. Focusing on the test, rather than underlying learning, may encourage a one-time performance orientation and transmission-type teaching. Schools may promote a performance orientation in students, which in turn can work against students’ engagement and persistence in learning, metacognition, and self regulation. Instructional/teaching time is diverted to specific test preparation activities.
About WCAL
Strategic objectives
Three case studies
Risks & Issues
Use of assessment information is the key learning Assessment evidence must be acted upon to inform subsequent instruction. Assessment helps to chart the learning road forward, by identifying and providing information to fill any gaps between the learners’ current status and goals for learning. Assessments can directly support the learning process by incorporating principles of learning and cognition (Herman & Baker Bennett & Gitomer, 2009). For example• by asking students to make public their thinking• by asking students for explanations and providing practice over multiple and
authentic contexts• by making learning goals explicit and involving students in self -assessment
About WCAL
Strategic objectives
Three case studies
Risks & Issues
Quality marks of e-assessment in the 21st century1. Be aligned with the development of significant, 21st century goals:2. Incorporate adaptability and unpredictability: 3. Be largely performance-based. 4. Add value for teaching and learning. 5. Make students’ thinking visible. 6. Be fair. 7. Be technically sound. 8. Valid for purpose. 9. Generate productive and usable feedback for all intended users. 10. Provide productive and usable feedback for all intended users. 11. Build capacity for educators and students. 12. Be part of a comprehensive and well-aligned system of assessments designed to
support the improvement of learning at all levels of the educational hierarchy.
About WCAL
Strategic objectives
Three case studies
Risks & Issues
website
Paper-based assessments
Technology-rich assessments
Innovative assessment
Traditional assessment
Traditional, paper-based, low levels
of innovation
Technology delivers process
improvements – efficient, flexible
Innovative assessment,
designed to change teaching & learning
Migratory strategy
Transformational
strategy
About WCAL
Strategic objectives
Three case studies
Risks & Issues
Top Ten Cited Benefits of Computer-Based Testing 1. Improved access. More administration dates. More administration sites. 2. Efficient collection and scoring of responses. The possibility of instant scoring. 3. High-quality items and test forms that reflect 21st century skills. Greater validity.4. Use ICT tools. 5. Test administration standardised (eg timing).6. Accuracy. 7. Improved motivation.8. The ability to provide teachers and students with detailed information that can be
used to improve future learning.9. Reduced costs - of printing data entry, collection, aggregation, verification, and
analysis.10. Substantial advantages for test security. Student responses should not get lost.
About WCAL
Strategic objectives
Three case studies
Risks & Issues
In 2001 City and Guilds set a target to provide 80% of their objective assessments on-line by 2006. By 2010 they were delivering over 700,000 exams on computer each year. The Graduate Management Aptitude Test (GMAT) is marked twice – once by an expert and once by computer. ACCA has announced that all its global accountancy tests will be on computer by 2015.ExamSoft has completed a trial of 2000 law students in the US completing their bar exams on their own laptop – all sitting in a traditional, large exam hall.The United States Medical Licensing Exam includes a computer-based simulation of a patient with a range of symptoms which, in the examination, must be diagnosed and a treatment plan must be created. In the UK, theory driving tests are booked on-line and completed sitting at a computer. A search engine enables people to locate their nearest test centre.
About WCAL
Strategic objectives
Three case studies
Risks & Issues
1. Wireless GenerationAssessment of early reading
Assesses children’s grasp of reading skills – the alphabet, phonic knowledge, reading, comprehensionIncludes Progress MonitoringCan be performed every 10 days to 2 weeksCan be administered in 2-3 minutesVisual charting of progress makes it easy for teachers to
see if teaching methods are workingProvides a large database of assessments so that
progress monitoring can be done as often as necessary without repetition of content
website
www.wirelessgneration.com
http://www.worldclassarena.net/research3.htm
More information
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
About WCAL
Strategic objectives
Three case studies
Risks & Issues
2. Primum: USA Medical Exam In the 1960s there were concerns that the one-day practical oral assessment was susceptible to inter-rater reliability and there were concerns about resource requirements.Developed a series of Patient Management Problems1999 – computer-based case studies made mandatoryEach examinee has to address nine case studies
http://www.usmle.org/Examinations/step3/step3_content.html
See also: www.worldclassarena.net /research.htm for a review report regarding Primum software
website
contact
About WCAL
Strategic objectives
Three case studies
Risks & Issues
www.worldclassarena.net
www.worldclassarena.org.uk
www.worldclassarena.hk
website
contact
About WCAL
Strategic objectives
Three case studies
Risks & Issues
Software presents the candidate with a scenario and a control screen.From here the candidate can1. Request more comprehensive examinations2. Order medical procedures3. Advance the time 4. Move the patientSeveral thousand tests and procedures are available. Candidates make free text entry.
www.worldclassarena.net
www.worldclassarena.org.uk
www.worldclassarena.hk
website
contact
About WCAL
Strategic objectives
Three case studies
Risks & Issues
www.worldclassarena.net
www.worldclassarena.org.uk
www.worldclassarena.hk
website
contact
About WCAL
Strategic objectives
Three case studies
Risks & Issues
3. World Class Tests
In the two key areas of mathematics and problem solving)
Calibrated against the performance of the best ten percent of pupils aged 9 and13 from a range of countries
When ready model of administration
Internationally calibrated
www.worldclassarena.net
www.worldclassarena.org.uk
www.worldclassarena.hk
website
contact
About WCAL
Strategic objectives
Three case studies
Risks & Issues
www.worldclassarena.net
www.worldclassarena.org.uk
www.worldclassarena.hk
website
contact
CHORTLERS
About WCAL
Strategic objectives
Three case studies
Risks & Issues
A strategic view of riskswebsite
contact
Design Writing Delivery Scoring Results
CostTechnical Reliability
Dumbing Down
Cheating Benefits for learners
About WCAL
Strategic objectives
Three case studies
Risks & Issues
website
contact
Delivery and roll-out
‘Big Bang’ approaches, ie introducing everything at once
Preparing schools – people and systems
Providing support, guidance and training materials
Communication, communication, communication
Dealing with cheating, and concerns about integrity of the assessments
About WCAL
Strategic objectives
Three case studies
Risks & Issues
website
contact
Procurement Finding the expertise needed
Working with a technical partner, possibly a company
Putting in place an effective and reliable procurement approach
Implementing a controlled management approach – innovation in a controlled environment
About WCAL
Strategic objectives
Three case studies
Risks & Issues
website
contact
Item design, reporting
Being too ambitious
Dumbing down
Expecting more than the technology can support
Not focusing on the use for students – what is the pedagogical model of learning
About WCAL
Strategic objectives
Three case studies
Risks & Issues
Risk Mitigation Likelihood ImpactCosts and financial uncertainty
1. Establishing a fixed prices with contractors2. Establishing a change/contingency budget3. Asking technical platform providers to
propose solutions within a specified budget range
High High
Validity 1. Establish validity studies with Subject Matter Experts and teachers
2. Integrate innovative approaches
Medium High
Trends and measurement 1. Implement comparative studies2. Establish new baselines
Medium High
Communication.
Establish communication plans for each of the key stakeholder groups.
Medium High
About WCAL
Strategic objectives
Three case studies
Risks & Issues
www.worldclassarena.net
www.worldclassarena.org.uk
www.worldclassarena.hk
website
contact
About WCAL
Strategic objectives
Three case studies
Risks & Issues
Related Documents
