Evaluating online courses 25 may 2011

Phil Hardcastle

Email: Phil.Hardcastle@@rsc-em.ac.uk

Adam Blackwood

Email: [email protected]

Moving Learning On Line - Evaluating & Reviewing Online courses

April 12, 2023 | slide 1

Plan for the session

Ice Breaking & Introduction Observable Benefits of MLO Measuring Impact and Quality – Tools &

Approaches Review Session Further Resources




Who is here…


B. Williams RSCEM April 12, 2023 | slide 4

Who’s here

A Adult/Community Learning

B Work Based Learning

C FE College

D Specialist College

E 6th Form College


Online learning – a spectrum of use

A VLE is used to make a variety of documents available as a reference point for the learner when he/she is outside the classroom.

Used to deliver learning activities and to encourage co-operation between students on projects. Assignments may be submitted and marked on-line and there will be assessment exercises.

A fully online course requires the tutor to work online in the same way as he/she would have done face to face. The tutor will have to use digital means to promoteengagement and a sense of community.


Where are you on the scale?

Making

documents

availableLearning activities,

assessment

Creating online community







A checklist for assessing courses 1

Is it likely to work?– Has information been provided to enable prospective

users to understand the course being offered whether it will suit them?

– Have students had an introduction or orientation which prepares them to be successful in the course?

– Do students have the necessary hardware & software to complete the course?

– Have students the necessary study skills and IT skills to complete the course – is there opportunity to practise these skills?


A checklist for assessing courses 2

Is it likely to work?– Is the course designed to be interactive?– Are the tutors trained in the techniques that are required

for online tutoring?– Is there a system of support for students (and staff) –

when does it operate 24/7? 7/5? FAQ?– Is data collected about the course so that

administrators, decision makers and course designers can learn from the shared information?

This checklist is adapted from the Appleton Eschool’s online evaluation programme. http://www2.ed.gov/admins/lead/academic/evalonline/evalonline.pdf


Online materials presented to students

Course and module title Tutor name, contact details and

photo Brief welcome/introduction to the

course Structured content (chronological

or topic order) A brief introduction to each

section Lecture notes, presentations,

hand-outs - placed in context A clear indication of where and

how to get help

A reading list A glossary of terms, if

appropriate A course timetable (a list or

calendar) Links to specific

resources/websites for research A link to library services and

search Clearly label all resources

indicating what they relate to Assignments details including

how to submit April 12, 2023 | slide 10

A basic course would contain the following:

Interactivity and “stickiness”

The more engaging the course, the more likely the students are to remain on it and complete assignments.

The course should therefore promote interactivity – with the materials, with the tutor and with other students.

Courses can be assessed for the interactive elements they provide.

Varied media presentation will also help – video and podcasts; short spoken introductions to materials; all these help to give a varied experience to the student.








Outline of the Evaluation Tool How it works. Criteria Download and use Complete survey for


Course Evaluation & ReviewTool

Students

Teaching Practitioners

Data and Quality Assurance

ROI


MLOL Evaluation Tool

The Process


Determine Evaluation CriteriaDetermine Evaluation Criteria

Create a Survey Monkey Answer Grid


Download as Excel FileDownload as Excel File

Add to MLOL Evaluation Tool in the Data Sheet Tab


Provide to Sample Groups…Provide to Sample Groups…

Review PictureReview Picture






Poor Evaluation Picture



Very GoodEvaluation Picture


The Process


Determine Evaluation CriteriaDetermine Evaluation Criteria



Download as Excel FileDownload as Excel File



Provide to Sample Groups…Provide to Sample Groups…

Review PictureReview Picture


Survey Monkey Answer Grid

Option

MLOL Evaluation To

TASK 1 Your Criteria evaluation

http://tiny.cc/mlol7task1


TASKS

TASKS Download the MLOL Evaluation Tool – 10 Minutes

– Evaluation Tool Download at http://www.tiny.cc/mlol7tool

Consider what Criteria you might want to add to the different Sectors

Feedback


Online Lesson Observation

The main concern when observing a lesson is to check whether learning is taking place.

How could you demonstrate that learning was taking place in an online course?– Data from online interactions – forums etc– Reports showing quiz answers, assignments completed

How do learners feel about the quality of learning?– Learner surveys by web or phone or email– Use an online live forum of learners

Feedback to tutors?


Formative and Summative

Online systems and course design are in the very early stages – formative assessment of the course through user surveys, evaluation of online interaction etc can be very useful to help re-focus the course to meet learner needs.

Summative assessment needs to be disseminated around the organisation to make sure that success and failure can be duplicated and avoided.








Staff Development Programme and QA Processes

Staff Development Programme and QA Processes








References and Resources

Google Docs List of references at: http://tiny.cc/mlol7 Can be added to if you have other links and resources you

can suggest. Survey Monkey Example at http://www.tiny.cc/mlol7task1 Evaluation Tool Download at http://www.tiny.cc/mlol7tool


Evaluating online courses 25 may 2011

Education

evaluation criteriacreate

excel evaluation tool

course designers

basic course

criteria evaluation

online materials

online tutoring

brief introduction