Cooperative Online Evaluation

Cooperative Online Evaluation

Morten Flate PaulsenProfessor of Online Education

Director of Innovation and Development at NKI Nettstudier

A 25-minutes keynote presentation at Challenges 201113.05.2011, Universidade do Minho, Portugal

Slides and video available at:www.slideshare.net/MortenFP 1

Something about me• Thought my first online, distance education course in 1987

• Doctor of Education from the Pennsylvania State University, worked for the American Center for the Study of Distance Education

• Has been Adjunct Professor at the Athabasca University in Canada and professor associado at Universidade Aberta in Portugal

• Fallen in love with Portugal

• President of EDEN

2

• EDEN members from 400 institutions in 56 countries – 20th anniversary– 200 institutional members– 2000 potential members of NAP - Network of Academics and Professionals

• Portuguese members:– Antonio Teixeira is Vice President

– Universidade do Minho www.tecminho.uminho.pt

– Universidade Aberta http://www.univ-ab.pt

– University of Porto http://sigarra.up.pt/up/web_page.inicial

– Centro de Filosofia da Universidade de Lisboa www.centrodefilosofia.com

– Centro Naval de Ensino A Distancia – CNED http://www.cned.edu.pt/

– Instituto Politecnico de Santarem www.ipsantarem.pt

– The Portuguese Bank Training Institute – IFB www.ifb.pt

• Conferences– Annual conference, Dublin June 2011

– Open Classroom conference Athens October 2011

– Open Classroom conference Porto 2009

– Annual conference in Lisboa 2008

www.eden-online.org

4

NKI Nettstudier was established as a correspondence school in 1910

• One of Europe’s largest providers of online distance education• More than 130 000 enrolments in online courses since 1987• Broad range of subjects from secondary to master level• About 400 distance education courses online (all in Norwegian)• About 13 000 online students in about 40 countries• About 70 % women• Individual progress plans/individual follow-up• Thousands of potential learning partners• Global student catalogue with 1650 profiles• Always room for more students• Exams at local schools and embassies

October 2010

www.nki.no

The book’s website:www.studymentor.com

The Theory of Cooperative Freedom is described in my book, which you can

download from the Internet.

Online Education and Learning Management

Systems: Global E-learning in a Scandinavian

Perspective

The chapter on Evaluation of Online Students is

available athttp://issuu.com/mfpaulsen/docs/onlineeducation/66

6

One may say that:

Individual learning is conducted alone

Collaborative learning depends on groups

Cooperative learning takes place in networks

Six Dimensions of Freedom

It is difficult to combine individual flexibility and cooperationhttp://ppel4.pbworks.com/

Cooperative learning

• Cooperative learning seeks to develop virtual learning environments that allow students to have optimal individual freedom within online learning communities.

• Cooperative learning is based on voluntary participation in a learning community

• Cooperative learning benefits from transparency

NKI’s Philosophy: We facilitate individual freedom within a learning community in which online students serve as mutual resources without being dependent on each other.

8

Cooperative evaluation

• Cooperative evaluation seeks to develop virtual learning environments that allow students to have optimal individual freedom within online learning communities.

• Cooperative evaluation is based on voluntary participation in a learning community

• Cooperative evaluation benefits from transparency

NKI’s Philosophy: We facilitate individual freedom within a learning community in which online students serve as mutual resources without being dependent on each other.

9

Transparencyor privacy

in evaluation?

10

11

Transparency promotes cooperation

12

Dalsgaard, C. and Paulsen, M.F. (2009) Transparency in Cooperative Online Education. IRRODL, Vol 10, No 3.

Transparency improves quality

13

…in evaluation…

Three degrees of transparencyin evaluation

1. Low: Private e-mail

2. Medium: Class forum

3. High: Global web2.0 services

• Cooperative learning communities must balance transparency and privacy.

• Students should be able to influence their privacy level

• Which transparency level do you choose for evaluation?

• Which transparency level do you choose for evaluation statistics and summaries?

14

15

Assessment could betransparent and cooperative

• Computer-based assessment: If students have access to statistics and information derived from other students taking the same test

• Self-assessment: If students may access statistics or information from other students who have completed the self-assessment

• Peer assessment: If students are encouraged to voluntary assess each other’s work

• Teacher assessment: If students have access to some of the information the tutor provides or derives from assessing other students

Could exams be cooperative?

What if:

• all resources were allowed?

• networking was allowed?

• students could decide where to take exams?

• students could decide when to take exams?

• exam assignments, papers, grades and statistics were more transparent?

16

Cooperative Gating (COG)

Students must meet a condition to access a cooperative resource. Here is an example using In Text Question (ITQ):

After attending Morten Flate Paulsen’s presentation, I perceive that Gating means conditionalaccess to cooperative resources. Paulsen elaborates on this in the article Cooperative Freedomon page 45 in his book Online Education and Learning Management Systems.

Submit your answer

When you have answered the question above and pressed the submit button, you will have access to answers submitted by other students.

How would you explain gating?

18

Rank 1: Kari Iversen:Gate er engelsk og betyr port. Tanken er at porten i utgangspunktet er lukket slik atman ikke får tilgang til en ønsket tjeneste før man har gjort seg fortjent til det.Assess this answer

Rank 2: Ole Olsen:Gating kan stimulere studenter til å bidra med nyttig informasjon fordi de selvønsker å få tilgang til informasjon fra andre.Assess this answer

Rank 3: Ola Nordmann:Etter å ha hørt Morten Flate Paulsens foredrag oppfatter jeg det slik at Gatingbetyr betinget tilgang til kooperative ressurser. Paulsen skriver for øvrig mer omdette i artikkelen Cooperative Freedom på side 45 i boken Online Education andLearning Management Systems.Assess this answer

Readers may like or dislike the answers:

Processos Pedagógicos em ElearningEach Study Unit had 4 Week-long Activities

1. Find, study and share materials related to the theme and organizing it together with ideas and thoughts in an annotated bibliography in your blog

2. Produce a learning object related to the theme, publish it somewhere

3. Write reviews in the forum on one annotated bibliography and one learning object published by colleagues

4. Take part in a structured discussion on issues related to the theme

19http://nettstudier.blogspot.com/2010/04/online-teaching-techniques.html

The unit structure is adopted by the EU-projecte-teacher 2.0

| E-teacher 2.0 | www.e-teacher-2.eu |

Portuguese partner: www.esepf.pt

20

Example of transparency: http://finalreport.ensinoinf.net/

Cooperative evaluation inthe PPEL course

• Peer evaluation• Propose grades for peers

• Reviews of annotated bibliographies and learning objects

• Forum discussions and blog posts

• Some shared their self-evaluation• Propose and justify your own

grade

• Write personal reflection paper

• Teacher evaluation

• Personal feedback

• General feedback

21

Teacher cooperation and evaluation

• Courses thought are automatically listed

• Many teachers provide advice to students in their presentations

• The advice is valuable also for prospective students

22Google translation of

http://www.nki.no/pp/teixeira_dulce

23

24

Continuous tracking of response time

• Introduced in May 2004• Resulted in much discussion in the

teachers’ online forum• A few teachers voiced strong criticism,

doubts and reservations

The average response time for all150 teachers in 2010 was 1.77 days.

In 2008 NKI introduced an automatic e-mail reminder to the tutors who had one ore more assignments waiting for 7 days or more.

Quality goal: XX % of students positive or neutralMy results last quarter/year: 93/84 %

Average teacher results last quarter/year: YY/ZZ %

Frequent, anonymous evaluation by students

Automatic e-mail reminder if quality is under threshold value

Quality Barometer

22

ObrigadoI care, so I share:

http://home.nki.no/morten

https://twitter.com/MFPaulsen

www.slideshare.net/MortenFP

www.facebook.com/mfpaulsen

26

www.eden-online.org/blog