The Current OER Search Dilemma · The Current OER Search Dilemma 6/22/2013 Ishan Abeywardena: Guest Lecture 24th and 27th June 2013 (STOU) 2 Lecture Outline • Review of the current

The Current OER Search Dilemma 6/22/2013

Ishan Abeywardena: Guest Lecture 24th

and 27th June 2013 (STOU) 1

The Current OER Search Dilemma

Guest Lecture24th and 27th June 2013

Sukhothai Thammathirat Open University, Thailand

Ishan Abeywardena MSc, MSc (Brunel), BSc (Bangalore), MIEEE, MBCS, MIET, MTA

Senior Lecturer, School of Science and Technology

Wawasan Open University

Penang, Malaysia1

Acknowledgement

I express my gratitude to:

– Assoc. Prof Dr. Chailerd Pichitpornchai for

extending me this kind invitation;

– Ms. Rattip Phukkeson and all other colleagues at

STOU for having me here.

2




Lecture Outline

• Review of the current OER search dilemma.

• Parametrically measuring the Desirability of

OER using D-index.

• OERScout Technology Framework: A Novel

Approach to OER Search.

3

The Current OER Search Dilemma

4




Main Reference

Abeywardena, I.S., & Chan, C.S. (2013).

Review of the Current OER Search

Dilemma. Proceedings of the 57th World

Assembly of International Council on

Education for Teaching (ICET 2013),

Nonthaburi, Thailand.

Available at:

https://www.researchgate.net/publicatio

n/240310607_Review_of_the_Current_O

ER_Search_Dilemma?ev=prf_pub

5

Talking Points

• Current State of OER

• The Dilemma

• Some Existing Solutions

– Pearson’s Project Blue Sky

– GLOBE

– LRMI

– OERScout

6




Current State of OER

7

The Dilemma

So…how do I find the material I need for my

teaching

8




Literature

• ...The problem is in finding the resources, and more correctly finding the “right” resources. Using a regular search engine like Google to find content is not always a viable option as it will generate too many answers. There is, hence, a need to easily find relevant content...” (Hatakka, 2009)

• “searching this way (using existing search engines such as Google) might be a long and painful process as most of the results are not usable for educational purposes” (Pirkkalainen & Pawlowski, 2010)

• No single search engine is still able to locate resources from all the OER repositories (West & Victor, 2011)

• One of the major barriers to the use and re-use of OER is the difficulty of finding quality OER matching a specific context (Dichev & Dicheva, 2012)

• “…the problem with open content is not the lack of available resources on the Internet but the inability to locate suitable resources for academic use” (Unwin, 2005).

9

Some Existing Solutions

• Google: “…searching this way might be a long and painful process as most of the results are not usable for educational purposes” (Pirkkalainen & Pawlowski, 2010).

• Federated Search: BRENHET2; OpeScout; Global Learning Object Brokered Exchange (GLOBE); and Pearson’s Project Blue Sky.

• Semantic Search: OER-CC ontology; the “Assistant” prototype; the “Folksemantic” project; and “Agrotags”.

Pirkkalainen, H., Pawlowski, J. (2010). Open Educational Resources and Social Software in Global E-Learning Settings. In Yliluoma, P. (Ed.)

Sosiaalinen Verkko-oppiminen. IMDL, Naantali, 23–40.10




The Dilemma

Yeah…but which one do I choose

11

Pearson’s Project Blue Sky

http://www.pearsonlearningsolutions.com/pearson-bluesky/12




GLOBE

http://www.globe-info.org/13

LRMI

http://creativecommons.org/tag/learning-resource-metadata-initiative

14




OERScout

http://www.oerscout.org/15

Questions

16




Desirability of OER using D-index

17

Main Reference

Abeywardena, I.S., Raviraja, R., & Tham, C.Y.

(2012). Conceptual Framework for

Parametrically Measuring the Desirability of

Open Educational Resources using D-index.

International Review of Research in Open and

Distance Learning, 13(2), 104-121 (ISI-

indexed publication).

Available at:

http://www.irrodl.org/index.php/irrodl/articl

e/view/1177/2142

18




Talking Points

• Usefulness of OER

• The Desirability of OER

• Measuring the Desirability using D-index

19

What is the Usefulness of an OER?

• The usefulness of an OER for a

particular teaching or learning need

can only be accurately assessed by

reading through the content.

• This aspect of use and re-use of

OER will remain a human function

regardless of the improvements in

technology.

20




Measuring the Usefulness OER

However, there are other aspects of a resource which are

fundamental to the usefulness of that particular resource and can be

parametrically identified by a software based mechanism:

– Whether a resource is relevant to a user’s needs;

– Whether the resource is open enough for using,

reusing, remixing and redistributing;

– Whether the resource is accessible with respect

to technology.

21

What is Desirability?

• If the user cannot easily use, reuse and remix a resource with available technology, the resource becomes less useful to the user.

• Less useful resources are less

desirable for teaching and

learning needs….

22




Measuring the Desirability of OER

Within the requirement of being able to use and reuse a particular OER, the three parameters of Desirability can be defined as:

– level of openness: the permission to use and reuse the resource;

– level of access: the technical keys required to unlock the resource;

– relevance: the level of match between the resource and the needs of the user.

23

i.e. Desirability

D-index = (level of access x level of openness x relevance) / 25624




Openness

Permission Value

Reuse 1

Redistribute 2

Revise 3

Remix 4

The level of openness based on the four R’s of openness

Mapping the CC licenses to the 4 R’s

Permission Creative Commons (CC) licence Value

Reuse None 1

Redistribute Attribution-NonCommercial-NoDerivs (CC BY-NC-ND)

Attribution-NoDerivs (CC BY-ND)

2

Revise Attribution-NonCommercial-ShareAlike (CC BY-NC-SA)

Attribution-ShareAlike (CC BY-SA)

3

Remix Attribution-NonCommercial (CC BY-NC)

Attribution (CC BY)

4

25

Access

Access to editing

tools

Level of expertise required to

revise or remix

Meaningfully

editable

Source-file access Value

LOW HIGH NO NO 01

LOW HIGH NO YES 02

LOW HIGH YES NO 03

LOW HIGH YES YES 04

LOW LOW NO NO 05

LOW LOW NO YES 06

LOW LOW YES NO 07

LOW LOW YES YES 08

HIGH HIGH NO NO 09

HIGH HIGH NO YES 10

HIGH HIGH YES NO 11

HIGH HIGH YES YES 12

HIGH LOW NO NO 13

HIGH LOW NO YES 14

HIGH LOW YES NO 15

HIGH LOW YES YES 16Th

e l

eve

l o

f a

cce

ss b

ase

d o

n t

he

ALM

S a

na

lysi

s

26




Relevance

Search rank Value

Below the top 30 ranks of the search results 1

Within the top 21-30 ranks of the search results 2

Within the top 11-20 ranks of the search results 3

Within the top 10 ranks of the search results 4

The level of relevance based on search rank (Vaughan, 2004)

• Users will only consider the top ten ranked results for a particular search

as the most relevant;

• Users will ignore the results below the top 30 ranks.

Vaughan, L. (2004). New measurements for search engine evaluation proposed and tested. Information Processing and Management 40, 677–691.27

Calculation

Original search results (OER Commons example)

Calculation of D-index

After application of D-index

28




Sample Search

Search Rank Title CC Lisence File Type

1 18.01 Single Variable Calculus CC BY-NC-SA PDF

2 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text

3 18.01 Single Variable Calculus CC BY-NC-SA PDF

4 18.013A Calculus with Applications CC BY-NC-SA HTML/Text

5 18.02 Multivariable Calculus CC BY-NC-SA PDF

6 Single Variable Calculus CC BY-NC-SA PDF

7 Calculus Online Textbook CC BY-NC-SA PDF

8 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text

9 18.075 Advanced Calculus for Engineers CC BY-NC-SA PDF

10 MATH 140 - Calculus I, Summer 2007 CC BY-NC-SA Protected

Top 10 search results returned by MERLOT for the keyword “calculus”

29

Original Search Results

• The original top ten search results only contain resources which are released under the CC BY-NC-SA license.

• 6/10 resources returned are in PDF format which make them difficult to reuse and remix.

• Resource ranked as number ten is a protected resource which requires a specific username and password to access.

30




Application of D-index

Rank After

Applying D-

index

Original

Search

Rank Title CC Lisence File Type D-index

1 2 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text 0.75

2 4 18.013A Calculus with Applications CC BY-NC-SA HTML/Text 0.75

3 8 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text 0.75

4 14 Multivariable Calculus CC BY HTML/Text 0.75

5 19

MATH 10250 - Elements of Calculus

I, Fall 2008 CC BY-NC-SA HTML/Text 0.56

6 20 18.022 Calculus CC BY-NC-SA PDF 0.56

7 22 Single-Variable Calculus I CC BY HTML/Text 0.50

8 25 Single-Variable Calculus II CC BY HTML/Text 0.50

9 15 Highlights of Calculus CC BY-NC-SA Video 0.42

10 21 Calculus I CC BY HTML/Text 0.38

Top 10 results when D-index is applied to the results returned by MERLOT

31

Results After Applying D-index

• 8/10 resources are in HTML/Text formats

which are the most accessible in terms of

reuse.

• 4/10 resources are available under the CC BY

licence which make them the most open

resources in the list.

32




Benefits of the D-index

The application of the D-indexwould greatly improve the effectiveness of the search with respect to locating the most suitable resources for use and reuse.

33

Let’s discuss more over tea?

34




OERScout Technology Framework

35

Main Reference

Abeywardena, I.S., Chan, C.S., & Tham, C.Y.

(2013). OERScout Technology Framework: A

Novel Approach to Open Educational

Resources Search. International Review of

Research in Open and Distance Learning, In

press (ISI-indexed publication).

Available at:

Currently in press. Will be available at

http://www.irrodl.org

36




Talking Points

• Why not Google?

• Why not native search engines of

repositories?

• What is OERScout?

• Why use OERScout?

37

Go

og

le “

Ad

va

nce

d S

ea

rch

” re

sult

s fo

r O

ER

on

Ch

em

istr

y (

24

th M

ay

20

12

)

38




Why Not Repositories?

Identify which material to look for

(e.g. integration, C++ programming)

Identify the search queries

(e.g. “undergraduate mathematics”)

Locate repository(word of mouth, some link

somewhere, go to the more popular

repositories)

Run multiple queries to find resources

Read each resource to identify the usefulness

(openness, access, relevance)

Identify useful resources

Repeat steps 3-6 on multiple repositories

(hundreds to thousands…..)39

The Declaration

i. Facilitate finding, retrieving and sharing of OER.

Encourage the development of user-friendly

tools to locate and retrieve OER that are

specific and relevant to particular needs.

(UNESCO Paris OER Declaration, 2012)

UNESCO. (2012). Paris OER Declaration, Retrieved September18, 2012 from

http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/CI/CI/pdf/Events/Paris%20OER%20Declaration_01.pdf 40




Specific & Relevant

• Specific: the suitability of an OER for a particular teaching need.

For example, an OER on physics from the final year syllabus of a physics degree would not be suitable for a high school physics class.

• Relevant: the match between the content of the OER and the content needed for a particular teaching need.

For example, physical chemistry is not relevant for a teaching need in organic chemistry.

41

OERScout

A Text Mining Algorithm which

–READS text based OER;

–LEARNS which academic domain(s) and

sub-domain(s) they belonged to;

–RECCOMENDS Desirable material for a

particular academic need.

42




Keyword-Document Matrix (KDM)

43

44




Advantages of OERScout

• Using the KDM, the system generates ranked lists of relevant OER from heterogeneous repositories to suit a given search query;

• Incorporates the Desirability framework to recommend useful resources for academic purposes;

• Uses a faceted search approach to allow users to quickly zero-in on the resources they are after.

45

Benefits to Content Creators

• No need to manually define metadata;

• No need to publicise the availability of material;

• No need to build custom search mechanisms for repositories;

• More visibility and reach of material to a wider audience.

46




Benefits to Users

• Provides a central location for finding

resources scattered across the globe hidden in

high volume repositories;

• Locate only the most relevant resources.

47

Conclusion

The ultimate benefit of OERScout

is that both content creators and

users now only need to

concentrate on the actual

content and not the searching

and location of relevant OER.

48




Questions

49

About…

Ishan AbeywardenaSenior Lecturer, School of Science and Technology, Wawasan Open University, Penang, Malaysia

• MSc in Wireless Enterprise Business Systems, Brunel University, UK.

• MSc in Engineering Management, Brunel University, UK.

• BSc in Computer Science, Bangalore University, India.

• PhD Candidate in Computer Science, University Malaya, Malaysia. Areas of specialisation: text mining,

metadata, faceted search

Professional Member of

– Institute of Electrical and Electronic Engineers (MIEEE)

– British Computer Society (MBCS)

– Institution of Engineering and Technology (MIET)

– Microsoft Technology Associate (MTA)

Official Profile: http://www.wou.edu.my/IshanAbeywardena.html

Professional Profile: http://www.linkedin.com/in/ishansa

Research Profile: http://www.researchgate.net/profile/Ishan_Abeywardena/

Tech Blog: http://www.ishantalks.com

E-mail: [email protected]




References

• Dichev, C., & Dicheva, D. (2012). Open Educational Resources in Computer Science Teaching. SIGCSE’11, February 29–March

3, 2012, Raleigh, NC, USA.

• Caswell, T., Henson, S., Jenson, M., & Wiley, D. (2008). Open Educational Resources: Enabling universal education.

International Review of Research in Open and Distance Learning 9(1), 1-11.

• Hatakka, M. (2009). Build It and They Will Come? – Inhibiting Factors for Reuse of Open Content in Developing Countries,

EJISDC 37(5), 1-16.

• Hilton, J., Wiley, D., Stein, J., & Johnson, A. (2010). The four R‘s of openness and ALMS Analysis: Frameworks for open

educational resources. Open Learning: The Journal of Open and Distance Learning, 25(1), 37-44.

• Pirkkalainen, H., Pawlowski, J. (2010). Open Educational Resources and Social Software in Global E-Learning Settings. In

Yliluoma, P. (Ed.) Sosiaalinen Verkko-oppiminen. IMDL, Naantali, 23–40.

• Unwin, T. (2005). Towards a Framework for the Use of ICT in Teacher Training in Africa. Open Learning 20, 113-130.

• West, P., & Victor, L. (2011). Background and action paper on OER. Report prepared for The William and Flora Hewlett

Foundation.

51

The Current OER Search Dilemma · The Current OER Search Dilemma 6/22/2013 Ishan Abeywardena: Guest Lecture 24th and 27th June 2013 (STOU) 2 Lecture Outline • Review of the current

Documents