Massive Open Online Courses (MOOCs) as a Disruptive Innovation in Higher Education?

Massive Open Online Courses (MOOCs)

as a Disruptive Innovation

in Higher Education? Name: Dr Jürgen Rudolph Student ID: 1616033 Programme: Master of Education Supervisor: Dr Francisco Ben Date: 23 May, 2014

Education Research Project (nine units),

submitted in partial fulfillment of the

requirements of the Master in Education at the

School of Education, University of Adelaide

  2  

Table of contents List of figures and tables 4 Preface and Acknowledgements 6 1. Introduction 9 1.1 Background to the research 9 1.2 Aim of the research, research problem and research questions 10 1.3 Significance and justification of the research 13 1.4 Methodology 15 1.5 Chapter Overview 16 1.6 Definitions of key terms 17 1.7 Delimitations and key assumptions 19 1.8 Conclusion 20 2. Literature Review 21 2.1 Introduction 21 2.2 A brief history of MOOCs 22 2.2.1 cMOOCs 22 2.2.2 The Stanford higher education experiment 25 2.2.3 Stanford University experiment spin-off #1: Coursera 26 2.2.4 Stanford University experiment spin-off #2: Udacity 27 2.2.5 edX 28 2.2.6 Comparison of the three dominant xMOOCs 28 2.3 Do MOOCs offera high quality educational experience? 31 2.3.1 Who are the participants, what is their motivation and what do the 31

completion rates signify? 2.3.2 Required MOOC literacies 34 2.3.3 What courses or content may best be taught via MOOCs, if any? 34 2.3.4 Assessment 35 2.3.5 Accreditation, certification, credit transfer and badges 38 2.4 Are MOOCs sustainable? 40 2.4.1 How much do MOOCs really cost? 41 2.4.2 Revenue models 42 2.4.3 MOOCs as marketing devices 45 2.4.4 Are MOOCs sustainable? 46 2.5 Do MOOCs offer a viable alternative to traditional education? 47 2.6 MOOCs as disruptive innovation and providing access to the 48

bottom of the pyramid? 2.6.1 Are MOOCs a revolution? 48 2.6.2 MOOCs as disruptive innovation? 49 2.6.3 Do MOOCs provide access for the bottom of the pyramid? 52 2.6.4 Unbundling and rebundling 55 2.7 Conclusions and research model 56 3. Methodology and Methods 60 3.1 Introduction and research design 60 3.2 Interpretivist research paradigm and symbolic interactionism 61 3.3 Qualitative research methodology and the circular model 64

  3  

3.3.1 Qualitative research methodology 64 3.3.2 Qualitative versus quantitative research 64 3.3.3 A circular model of the qualitative research process and theory 66 3.4 The six steps of the interview process 67 3.4.1 Links between research issue and interview questions 67 3.4.2 Choice of interview method: convergent expert interview 71 3.4.3 Sample Frame – discussion of selection of experts 74 3.4.4 Introduction of the eight expert interviewees 77 3.4.5 Conducting interviews 82 3.4.6 Transcribing interviews 83 3.4.7 Coding and interpreting interviews 84 3.5 Participant observation in cyberspace as a secondary method 87 3.6 Assessment of the quality of qualitative research 88 3.7 Ethical considerations 91 3.8 Conclusions 92 4. Empirical analysis 93 4.1 Introduction 93 4.2 MOOCs, cMOOCs, xMOOCs and their predecessors 93 4.2.1 The MOOC components 93 4.2.2The first MOOC 97 4.3 The quality of the educational experience in MOOCs 99 4.3.1 Who are the participants, what is their motivation and what 102 do the completion rates signify 4.3.2 Required MOOC literacies 105 4.3.3 Expertise versus wisdom of the crowds 106 4.3.4 What courses or content may be best taught via MOOCs, if any 108 4.3.5 Assessment 109 4.3.6 Accreditation, credit transfer and badges 118 4.4 Are MOOCs sustainable? 120 4.4.1 The cost of MOOCs 120 4.4.2 MOOCs as marketing devices 122 4.5 MOOCs as a viable alternative to higher education? 123 4.6 MOOCs as disruptive innovation and providing access to the 129

bottom of the pyramid? 4.6.1 MOOCs are not much of a revolution 129 4.6.2 MOOCs are not a disruptive innovation 130 4.6.3 Do MOOCs provide access to the Bottom of the pyramid? 133 4.6.4 Unbundling and rebundling 134 4.7 Conclusions 135 5. Synthesis, recommendations and conclusions 136 5.1 Synthesis of literature review and research findings 136 5.1.1 Research question 1: the quality of the educational experience in MOOCs 136 5.1.2 Research question 2: the sustainability of MOOCs 139 5.1.3 Research question 3: MOOCs are not a viable alternative to 141

traditional higher education 5.1.4 Research question 4: MOOCs are not a disruptive innovation and 142

do not provide access to the bottom of the pyramid 5.2 Benefits and recommendations for implementation 145

  4  

5.2.1 Challenges and benefits 145 5.2.2 Six recommendations 147 5.3 The future of MOOCs and opportunities for further research 150 5.3.1 Opportunities for further research 150 5.3.2 The future of MOOCs 151 5.4 Conclusions 151 Bibliographical references 153 Appendices 211 Appendix 1: Comparison of various qualitative methods 211 Appendix 2: The history of distance learning and education and 212

technology in the perspectives of experts Appendix 3: My own experience with MOOC assessment 216 Appendix 4: Author’s biodata 219

List of figures and tables Figure 1.1 The indescribable, indestructible, unstoppable MOOCs 9 Figure 1.2 Research questions in the qualitative research process 12 Figure 1.3 The world population and income pyramid 14 Figure 1.4 The path of disruptive innovation 19 Figure 2.1 MOOC timeline 22 Figure 2.2 Professors Thrun and Norvig 26 Figure 2.3 Thrun demonstrates Google’s self-driving cars 26 Figure 2.4 Georgia Tech’s MSc in Computer Science MOOC 28 Table 2.1 Comparison of MOOCs 30 Figure 2.5 World map of enrolment for all HarvardX offerings as of 10 32

February, 2014 Figure 2.6 Udacity’s free versus subscription model 44 Figure 2.7 College: a packaged bundle 56 Figure 2.8 Research model 59 Figure 3.1 Components of qualitative research design 61 Table 3.1 Research perspectives in qualitative research 63 Figure 3.2 Linear versus circular models of process and theory 66 Table 3.2 Constructive alignment of research question 1 and interview questions 68 Table 3.3 Constructive alignment of research question 2 and interview questions 69 Table 3.4 Constructive alignment of research question 3 and interview questions 70 Table 3.5 Constructive alignment of research question 4 and interview questions 70 Table 3.6 Constructive alignment of sections in chapters 2 and 4 with 71

research questions and interview questions Table 3.7 Convergence in convergent expert interviews 74 Table 3.8 Sampling decisions in the research process 74 Table 3.9 Research sample 76 Figure 3.3 Mayring’s process analysis of general content analysis 86 Figure 4.1 What MOOC means 94 Figure 4.2 The meaning of OER 95 Figure 4.3 Screenshot of Keth Devlin’s “Introduction to Mathematical 102

  5  

Thinking” Coursera MOOC Figure 4.4 Dino 101: Dinosaur Paleobiology 104 Figure 4.5 Dave Cormier’s MOOC #Rhizo 14 116 Figure 4.6 Screenshot of MOOC on “Beauty, Form and Function: An 123

Exploration of Symmetry” Figure 4.7 A future with only 10 universities 125 Figure 4.8 Screenshot from PowerPoint presentation: “Higher Education 129

Doesn’t Do Revolutions” Figure 4.9 Three core elements of higher education 131 Figure 4.10 MOOC on Mobiles for Development by IIT Kanpur & 135

Commonwealth of Learning Figure 5.1 Theses 1-4 on the perception of experts on MOOCs (Research 140

Question 1) Figure 5.2 Theses 5-7 on the perception of experts on MOOCs (Research 142

Question 2) Figure 5.3 Theses 8-9 on the perspectives of experts on MOOCs (Research 143

Question 3) Figure 5.4 Theses 10-13 on the perspectives of experts on MOOCs (Research 146

Question 4) Figure 5.5 Learn with your peers 151

  6  

Preface and Acknowledgements

I started the M.Ed. programme of the University of Adelaide almost three years ago.

My motivation at that time was lifelong learning and specifically, learning more about

(higher) education. I have been in education most of my life, largely on the receiving

end (having completed three postgraduate qualifications thus far), but also as an

Academic Director for two Private Education Institutions (PEIs) in Singapore and as a

lecturer, trainer and facilitator in a wide variety of subjects. Consequently, it appeared

to make sense to pursue an education-specific qualification.

When I started the M.Ed. programme, I was in no hurry, as my focus was on learning,

not the credential. Then, in 2012, I was persuaded to take up Republic Polytechnic’s

Specialist Diploma in Applied Learning and Teaching (SDALT) that I completed in

2013 – this was a practical course, specifically designed for PEI faculty like myself,

and I found it quite meaningful. On the flipside, it derailed me for nine months from

my M.Ed. studies.

My current role as Senior Lecturer for Kaplan Higher Education Singapore also kept

me rather busy, which occasionally made me question the meaningfulness of the

whole undertaking. Did I really need another postgraduate qualification, now that I

had an educational qualification (which was one of my original motivations)? I

occasionally had to remind myself that quitting was not part of my vocabulary.

Writing the thesis has been very rewarding and rather painful at times. The reason

why it has been great fun and intellectually highly stimulating is due to the support of

quite a few wonderful people who deserve to be acknowledged.

My thesis supervisor Dr Francisco Ben has been wonderfully supportive, being

infectiously enthusiastic about the topic and getting me a couple of much-needed

extensions from the university. Rhys Johnson, my awesome boss at Kaplan, created a

highly conducive work environment for my humble research – without his support,

this thesis would have never been completed. Lydia Tan also deserves special

mention for her inestimable assistance with literature search, the compilation of

  7  

bibliographical references, transcription of interviews, drawing of figures and other

technical assistance. Jervin Lim also thankfully did some of the transcribing.

Professor Chad Perry was very kind in sharing some of his most recent and not yet

published writings about convergent expert interviews and thesis writing with me.

Professor Jan Herrington from the Department of Education at Murdoch University

was extremely helpful in sharing with me her incredible knowledge about higher

education and MOOCS. Critically, she suggested to me many MOOC experts as

potential interviewees, and allowed me to drop her name, when contacting them,

which I am certain made my success rate much higher than it would otherwise have

been. Stephen Li, Kaplan Singapore’s Senior Director for University Relations,

provided me with some great references on MOOCs that enriched my review of the

literature.

If this project is worth reading, much of the credit is due to the expert interviewees

who so generously shared with me their time. Dr Bror Saxberg, Chief Learning

Officer of Kaplan Inc., made himself available during a visit to Singapore for a face-

to-face interview, and also previously had been most kind to comment on some of my

articles on MOOCs. Professor Jim Jackson, a former supervisor of mine, entertained

me for 90 minutes via Skype during his expert interview, and also commented on

some of my previous work. His sharp legal mind led me in some less-explored

directions. Dr Samson Tan, a Singaporean learning technologies expert and lecturer

of mine in the SDALT programme, was also very kind in letting me interview him.

I feel very honoured that all three key people who started the whole MOOC

phenomenon granted me interviews. Professor George Siemens, Stephen Downes

and Dave Cormier were all most kind in sharing their precious time for Skype

interviews, very much in the spirit of connectivism. Great thanks are also due to Sir

John Daniel, a famous thought leader on education, who also granted me an

interview.

Professor Tania Aspland taught me two modules during the M.Ed. programme and

was a greater influence on me than she would know, recommending two wonderful

books (by Flick and Biggs & Tang), and being one of my great role models as a

  8  

university teacher. Dr I Gusti Darmawan almost made me consider quantitative

research, which is a very big compliment indeed, and deserves much praise as a

teacher-mentor.

Two years ago, Dr Scott Gardner opened up a whole new world of knowledge

management to me, and it was he, together with Dr Amy Huang and Alfred Tseng

(all from Murdoch University), who were kind enough to permit MOOCs as a

possible additional essay topic for my favourite module as a teacher, BUS378

Knowledge and Organisational Learning. This led to intriguing discussions with some

of my students.

Lastly, and most importantly, huge thanks are due to my wonderful and long-

suffering wife Clara, who has been very understanding during the countless burnt

evenings and weekends that I spent on this project.

Any shortcomings of this work, needless to say, are entirely mine.

  9  

1. Introduction 1.1 Background to the research

The topic of disruptive innovation in higher education is highly current and important.

Disruptive innovation may threaten the traditional approach to higher education (of

studying on physical university campuses) and, simultaneously, it may open up

hitherto largely unavailable opportunities to people at the base of the income pyramid.

In this research project, it will be evaluated whether Massive Open Online Courses

(MOOCs) are a potential disruptive innovation in higher education.

MOOCs have garnered much interest for a variety of reasons. One is the massiveness

(which is already part of the concept) of their unprecedented scale, with some of the

courses having more than 100,000 participants. Another reason is the global demand

for knowledge and information that is manifested by highly diverse learners of all

ages, backgrounds and levels of expertise from all over the world. MOOCs also

“challenge current credit transfer and accreditation systems and practices” (Australian

Trade Commission, 2012, p. 12).

Figure 1.1 The indescribable, indestructible, unstoppable MOOCs

Source: Rudolph, 2013.

  10  

1.2 Aim of the research, research problem and research Questions

The objective of the research project is to shed some light on MOOCs, and to use

Geertz’s (1973) famous term, provide a thick description of the phenomenon. The

research problem is to improve our understanding of MOOCs through the

perspectives of experts. In other words: What are the perspectives of experts on

MOOCs in terms of their potential as a disruptive innovation in higher

education?

With research questions being amongst the most important parts of a research study

plan, Kervin, Vialle, Herrington and Okely (2006) have come up with some rules for

research questions, including:

“Try not to use a research question that can be answered with the words Yes or

No… Starting these questions with the words ‘To what extent…’ (or similar

exploratory words) makes them… much more researchable questions… Be

wary of research questions starting with the word Can or Should…” (Kervin et

al., 2006, p. 51).

Flick (2009, p. 98) emphasises that research questions should be formulated “in

concrete terms with the aim of clarifying what the field contacts are supposed to

reveal”. As a consequence of the above-mentioned considerations and also of my

qualitative methodological approach (see chapter 3), my research questions – or

guiding questions (O’Donoghue, 2007) – are as follows:

1. What, according to experts, constitutes a high quality educational experience

in the context of MOOCs?

2. To what extent are MOOCs sustainable (in the perspectives of experts)?

3. To what extent do MOOCs offer a viable alternative to traditional higher

education, in the perspectives of experts?

4. In the perspectives of experts, to what extent, if at all, are MOOCs

revolutionising higher education, a disruptive innovation and/or are opening

up opportunities for the base of the income pyramid?

  11  

O’Donoghue (2007) cautions that “there is no point in designing a study about

participants’ perspectives on something unless we are convinced before we

commence the study that it is something about which they have fairly well formed

views and that they feel sufficiently free to discuss these views” (pp. 36-37). Through

my earlier conversations with Dr Bror Saxberg, Chief Learning Officer of Kaplan,

USA, who easily qualifies as an expert on disruptive innovation in higher education, I

had one first indication that the above research questions were appropriate.

In qualitative research, the initial understanding of the topic should be regarded as

preliminary and thus be constantly reevaluated (Kleining, 1982). Consequently,

research questions are to be reviewed at various stages in the qualitative research

process (see Figure 1.2). The principle of openness questions the a priori formulation

of hypotheses (Hoffmann-Riehm, 1980). In the context of this project, the theoretical

structuring of the issue was postponed till the structuring by the interviewees had

emerged.

However, the principle of openness, by no stretch of the imagination, means that

attempts to formulate research questions should be abandoned, as Flick (2009, p. 98)

advises: “The less clearly you formulate your research question, the greater is the

danger that you will find yourself in the end confronted with mountains of data

helplessly trying to analyse them”. Research questions should be formulated clearly

and unambiguously “as early as possible in the life of the project” (Flick, 2009, p.

129). It is then that the principle of openness applies, as we should remain open to

surprises, and we can revise our research questions to become more concrete and

more focused (Flick, 2009).

  12  

Figure 1.2 Research questions in the qualitative research process

Source: Flick, 2009, p. 99.

What Flick (2009) and Glaser & Strauss (1967) call sensitising concepts is similar to

the key concepts that are defined and discussed in section 1.6. Other methodological

issues are discussed in Chapter 3.

  13  

1.3 Significance and justification of the research

For higher education to be of a good quality, a deep approach is necessary. Such a

deep approach goes beyond mere memorising and note-taking, students are to be able

to describe, explain, relate, apply and theorise in high-level engagement (Biggs and

Tang, 2011). In Outcomes-based teaching and learning (OBTL), teachers should

address six goal clusters: (1) higher order thinking skills; (2) basic academic success

skills; (3) discipline-specific knowledge and skills; (4) liberal arts and academic

values; (5) work and career development; and (6) personal development (Angelo and

Cross, 1993).

Over the past century, tertiary education has become increasingly massified and it has

become critically important for employability in the global knowledge economy. On

an individual level, earning power appears directly related to the level of education,

and research has shown that tertiary education historically forms a U-curve in terms

of return on investment (ROI). This means that at present, there may be a particularly

high ROI for tertiary education. On a national level, due to globalisation and

increased international competition, the quality and quantity of outcomes of a

country’s education system appear related to a nation’s future trajectory and

international economic position (Goldin and Katz, 2008).

In developed nations, almost everybody is a consumer of education. In the US, for

instance, it was observed as early as 1970 that higher education has been transformed

from a privilege into an assumed right, and for a growing number of adults, into an

expected obligation (Trow, 1970). However, there are widening gaps and increased

inequities in parts of the developed world, not to speak of emerging economies. The

typical disruptive innovation strategy to start with areas of non-consumption may

also be applicable to education, including higher education (Christensen, Horn &

Johnson, 2008; Richardson, 2010).

Prahalad’s base of the pyramid (BoP) theory is perhaps the easiest way to illustrate

that out of a global population of more than seven billion people, the majority

continue to be non-consumers of higher education. Figure 1.3 shows an updated

version of Prahalad and Hart’s original illustration from 2002, as the global

  14  

population has since dramatically grown. Prahalad and Hart’s (2002) estimation of

four to five billion people being non-consumers remains intact.

Figure 1.3 The world population and income pyramid

Source: Arnold & Valentin, 2013, p. 1906. MEP refers to moderate and extreme

poverty, BOP to base of the pyramid.

This project is not the first work to consider the concept of disruptive innovation

within the context of higher education. The concept of disruptive innovation was

originally applied to manufacturing, but later also to K12 (kindergarten to age 18)

education and to higher education (Christensen et al., 2008; Christensen & Eyring,

2011). Key results of disruptive innovations are products or services that exhibit

affordability, accessibility, convenience and simplicity (Christensen et al., 2008;

Richardson, 2010). The innovative products are good-enough, and become accessible

by the masses in the bottom of the pyramid (Prahalad, 2004).

While the concept of disruptive innovation has been applied to tertiary education

before, MOOCs as a potential major disruptive innovation in higher education have

not been systematically researched as yet. This is not surprising, as MOOCs are really

a phenomenon that has only caught global attention in 2012 (despite the fact that the

first MOOC dates back to 2008). Due to the relative novelty of the phenomenon, this

research is bound to cover new ground.

  15  

How exactly may higher education be disrupted? In the traditional university, tertiary

services are bundled together and may appear inseparable. However, there are

currently tendencies of them being unbundled, with the most compelling cases

perhaps being curricula, other educational content and instructional delivery, as they,

in computer-speak, consist of ones and zeros. Other educational services – such as

socialisation, skills, assessment and accreditation – may pose more complex, but

perhaps not insurmountable challenges.

MOOCs are relatively new, hence previously under-researched. In the past two

years, there has been a veritable avalanche of writings, though, due to the usual time

lag of academic, peer-reviewed publications, not yet so much academic literature. It is

not sufficiently explored whether free higher education, offered by some of the

world’s most famous Ivy League universities (Harvard, MIT, Stanford etc.) and new

online portals, will indeed disrupt higher-education-as-we-know-it.

Convergent expert interviews are suited to understand this important phenomenon

better. I am not aware that convergent expert interviews have been used in

educational research, so this may be an innovation in my research.

To sum up, (1) the research topic is novel; (2) the theoretical concept of disruptive

innovation, though not new to higher education, probably only comes to full bloom

when applied to MOOCs; (3) and finally, the method of the convergent expert

interview appears to be novel when applied to the topic.

1.4 Methodology

In chapter 3, my methodological position is detailed. To clearly anticipate it, my

research paradigm is interpretivism, my specific theoretical position is closest to

symbolic interactionism (although I would not totally disavow hermeneutics and

phenomenology, being quite eclectic and normally preferring a Foucaultian toolbox

approach), the specific methodology is qualitative, and the research method is

interviews, more specifically convergent expert interviews.

O’Donoghue (2007, p. 32) makes a very useful distinction between two basic types of

symbolic interactionist studies: on one hand, there is a type of studies that are “aimed

  16  

at generating theory on the perspectives which participants hold with regard to

something”; on the other hand, there are studies “aimed at generating theory on how

participants ‘manage’, ‘deal with’, or ‘cope with’ a phenomenon”.

For the purpose of my research, I will use the first, ‘frozen in time’ approach, as for

my present purpose, the second, more longitudinal approach is not feasible. The term

“perspective” is important to understand, it refers to the conceptual frameworks

through which people make sense of the world (Woods, 1983, Charon, 2001,

O’Donoghue, 2007). Interrelated parts of the framework that make up a perspective

“consist of the participants’ aims or intentions, their strategies, what they see as being

significant for them, the reasons they give for their activity, and what they see as the

expected outcomes of their activity” (O’Donoghue, 2007, p. 39).

1.5 Chapter overview

The structure employed in this research is the usual five-chapter structure. The

Introduction provides an overview of the topic. Initially, the Literature Review was

meant to be based on non-peer-reviewed journal articles by the author (Rudolph,

2012a, b). However, much additional literature was eventually evaluated, given the

exploding academic (and journalistic) interest in the topic. Hence, this section was

written completely anew.

In the third chapter on Research Methodology, I put the method of convergent expert

interviews into a paradigmatic and methodological perspective. The interview process

is discussed and justified. The Empirical Analysis in chapter 4 is predominantly

based on analysing expert interviews. To a very minor extent, some of my personal

experiences as a MOOC student were incorporated. The Conclusions in the final

chapter synthesises the literature review and my research in relation to the research

questions and provides recommendations and an outlook.

  17  

1.6 Definitions of key terms

In this section, the two key terms of this project are discussed: MOOCs (including

related terms such as cMOOCs, xMOOCs, connectivism and the flipped classroom)

and disruptive (versus sustaining) innovation. The term MOOC was created by Dave

Cormier in 2008 (Cormier, 2014b). All four aspects of MOOCs as Massive Open

Online Courses are worthy of careful consideration.

Massive refers to the number of participants, some MOOCs have attracted more than

100,000 participants, and Coursera reached one million participants from all over the

world within five months of its inauguration (Lawton, Ahmed, Angulo, Axel-Berg, &

Burrows, 2013).

Open. To Cormier (2014b), openness is the most important part of the MOOC

acronym. Openness is a “political act” and about “’demolishing established barriers’

of all sorts” (Cormier, 2014b). There is a big question mark whether MOOCs are

open or not, and Siemens (2013) has gone as far to state that MOOCs are

“destroying” open education (see chapter 4). The ‘open’ aspect in MOOC could often

be considered a misnomer, as ownership of course content and platform design is

protected so that the intellectual property can be monetised in some manner (Sandeen,

2013; Siemens, 2013). FutureLearn, for instance, has the following rather non-open

copyright notice posted on its website: “The Online Content and Courses IPR is

protected to the fullest extent possible by copyright laws. All such rights are reserved”

(cited in Siemens, 2013). Different from other xMOOCs, edX provides open

educational and platform resources in the normal sense of ‘open’: not only is

admission to the courses free, the material is available free-of-charge and available for

use and adaptation (Sandeen, 2013).

Online is perhaps the least controversial aspect of the acronym. However, MOOCs

can be combined with offline education, for instance in blended learning or in the

practice (that originated in Harvard in the 1990s) of the flipped classroom where

education technology is used to impart core learning and the classroom is “the locus

for coaching, mentoring and peer interaction” (The Maturing of the MOOC, 2013, p.

13).

  18  

Course. MOOCs are mostly individual courses, and many of them appear to be less

rigorous than for-credit courses at universities. However, Georgia Tech’s Masters

Degree in Computer Science is a course (in the broader sense of the word) that is

available via Udacity.

cMOOCs is the term that is used for the original MOOCs that started in 2008. C

stands for ‘connectivist’, as connectivism is the educational theory by Siemens and

Downes that inspired the cMOOCs. As Cormier, Downes and Siemens are all

Canadians, this is another meaning of the c (The Maturing of the MOOC, 2013).

Connectivism is Siemens’s “theory of learning emphasizing how knowledge and

skills emerge from making connections between different domains of activity such as

experience, learning and knowledge, as well as between individuals in a social

network” (The Maturing of the MOOC, 2013, p. 13).

xMOOCs are “online versions of traditional learning formats (lecture, instruction,

discussion etc.) on proprietary specialist software platforms owned by private

enterprises” that “feature contractual and commercial relationships between

Universities… and technology providers” (The Maturing of the MOOC, 2013, p. 11).

The three US xMOOCs are Coursera, edX and Udacity, with the UK’s FutureLearn

joining this group. xMOOCs are oftentimes just known as MOOCs due to their

dominance in public perception (The Maturing of the MOOC, 2013).

With the MOOC concept clarified, our attention needs to be turned to the other key

term, disruptive innovation. Christensen distinguishes two main types of innovation,

sustaining and disruptive innovation. Sustaining innovation “makes something

bigger or better”, for instance, “universities with more college majors and better

activity centers” (Christensen & Eyring, 2011, p. xxiv). In contrast, disruptive

innovation “disrupts the bigger-and-better cycle by bringing to market a product or

service that is not as good as the best traditional offerings but is more affordable and

easier to use”, for instance, online learning (Christensen & Eyring, 2011, p. xxiv).

Disruptive innovation is initially a boon to non-consumers, and via its own sustaining

innovations, eventually becomes a threat to traditional providers and may drive them

  19  

to their demise (Christensen & Eyring, 2011). The two types of innovation are

illustrated in Figure 1.4.

Figure 1.4 The path of disruptive innovation

Source: adapted from Christensen & Eyring, 2011, p. 16.

1.7 Delimitations and key assumptions

Literature that was published after September 2013, was not systematically included

in chapter 2’s literature review. Hence, only selected literature that was published

after September 2013, is included in it. Some key developments that occurred after

September 2013, are discussed in the concluding chapter. The topic was previously

under-researched, but in and after 2012, the literature has exploded. Thus, many

interesting phenomena related to MOOCs had to be excluded; for instance, the long

tradition of online education and Open Educational Resources (OER); related

phenomena like Khan Academy, Peer-to-Peer University, University of the People,

Western Governors University, and some of the most recent MOOC providers such as

FutureLearn could only be touched upon in passing.

  20  

1.8 Conclusion

In this chapter, the research topic of critically evaluating MOOCs as a potential

disruptive innovation in higher education was introduced. The problem statement

focuses on the perspectives of experts on the research topic, that is further

differentiated into an evaluation of the MOOCs’ quality of the educational experience

and their sustainability, and to what extent MOOCs are a viable alternative to

traditional higher education and disruptively innovate higher education by opening up

opportunities for the base of the income pyramid. The research is significant and has

some innovative elements, due to the novelty of systematically evaluating MOOCs as

a disruptive innovation in the context of convergent expert interviews.

  21  

2. Literature Review  2.1 Introduction

The New York Times coined 2012 the “Year of the MOOC” (Pappano, 2012) and

while there was relatively little research conducted on MOOCs prior to 2012, the

topic literally exploded in the past two years. When I first started researching the

topic, there seemed to be relatively little to write about, whereas now it is all too easy

to drown in a flood of information. Hence, as explained in the introductory chapter’s

delimitations, a focused approach was used that systematically addresses the state of

the literature (especially till September 2013) with regard to my research questions.

Due to the extremely fast developments of the topic, much additional literature till

April 2014 was eventually still included, albeit in a less systematic way.

A literature review is not a compilation of summaries, but rather a critical analysis of

the available literature. Purposes of literature reviews include: identifying of gaps in

the literature, avoiding reinventing the wheel, piggybacking on other researchers,

identifying other researchers in the field for networking purposes (in addition, this is

useful for identifying potential experts for the convergent expert interviews – see

chapter 3), identifying seminal works as well as opposing views, and putting our work

into perspective (Kervin et al., 2006).

Section 2.2 provides a brief history of MOOCs, before sections 2.3 to 2.6 address the

four research questions from the perspectives of the reviewed literature. After the

quality (section 2.3) and the sustainability (section 2.4) of MOOCs has been

evaluated, they are discussed as a potential alternative to higher education (section

2.5) as well as a potential revolution and disruptive innovation, offering access to the

bottom of the income pyramid via unbundling and rebundling of educational services

(section 2.6). Section 2.7 provides conclusions and discusses the research model.

  22  

2.2 A brief history of MOOCs

Figure 2.1 MOOC timeline

Source: The Maturing of the MOOC, 2013, p. 12.

The cMOOCs precede the xMOOCs. After giving due credit to the cMOOCs, I

continue with the ‘Stanford education experiment’ (not to be mixed up with

Zimbardo’s Stanford prison experiment from 1971 – section 2.2.2). This notable

experiment led to two spin-offs, Coursera (section 2.2.3) and Udacity (section 2.2.4).

After introducing edX (section 2.2.5), similarities and differences between these

xMOOCs will be analysed (section 2.2.6).

2.2.1 cMOOCs

cMOOCs are based on Siemens’s philosophical concept and learning theory of

connectivism. In connectivist theory, knowledge is not confined to tacit knowledge

(know-how embodied in people) or explicit knowledge (know-that embedded in text),

it is also distributed throughout networks. Connections between participants and

learning resources are fostered and supported (Downes, 2012). “The principle of

The Maturing of the MOOC

Figure 1: MOOC Timeline

(with acknowledgement to UniversitiesUK)

The consensus points on MOOCs are their importance, popularity and expansion. There is consensus on the reasons that Universities, and learners, have for engaging in MOOCs. These are: brand extension, recruitment, educational innovation and revenue (or cost reduction) opportunity. Learners mostly report satisfaction from studying in MOOCs, and curiosity about the experience. MOOC learners are not (currently) looking for awards.

That the impacts of MOOCs on HE will be profound and enduring is another consensus point. There is disagreement over whether a MOOC-induced transformation of the education landscape would be destructive or creative, and who might be the winners and losers. The possibility exists (for a minority of authors) that a Gartner Hype Cycle3 may be at work in the MOOC phenomenon – in which case the potential impacts may be overplayed.

Controversial points about MOOCs are just as important as the consensus. The field provokes some vocal and emotive polemic. This can directly influence its trajectory, with some Universities changing tack abruptly in the wake of strong opinion. The xMOOC format, which is where recent massive growth has taken place, is the subject of intense comment and speculation in the Academy and Press. Strong commitments from top university brands, stoked by large venture capital investments, have cooked up a powerful

12

͵ The Gartner Hype Cycle is a proprietary methodology from Gartner Research, widely adopted elsewhere, to account for large fluctuations in optimism and consolidation in technology-based industries. A technology trigger leads to inflated expectations, which are then followed by heavy consolidation before the sector reaches a stable plateau. See http://www.gartner.com/technology/research/methodologies/hype-cycles.jsp �

  23  

connectivist learning is that the learning takes place not as a result of absorbing the

course content, but rather in using course content as the basis for conversation and the

creation of additional materials” (Downes 2012, p. 536).

Learning is about belonging to a community, “the network becomes the learning”

(Siemens, quoted in Kolowich, 2013i). Education is "a connection-forming process,"

in which "we augment our capacity to know more" by adding nodes to our personal

networks and learning how to use them properly (Siemens, quoted in Kolowich,

2013i). There appears to be a strong metacognitive component – an important aspect

of learning in a MOOC is “learning how to select content” (Downes, 2012, p. 504).

In addition to Siemens’s connectivist pedagogy, other major influences on cMOOCs

were Couros’s online graduate course (Couros, 2009) and Wiley’s wiki-based course

(Wiley & Hilton, 2009) – which, importantly, opened up university courses to

outsiders and thus transcended institutional boundaries – and the emergence of

massive open online conferences (Downes, 2012). The success of Siemens’s

conference on Connectivism made Siemens and Downes consider MOOCs as “a

longer and more involved enterprise” (Downes 2012, p. 503).

Contrary to the xMOOCs that are “arguably dominated by the ‘drill and grill’

instructional methods with video presentations, short quizzes and testing”, “cMOOCs

emphasise connected, collaborative learning and the courses are built around a group

of like-minded individuals’ platform to explore new pedagogies beyond traditional

classroom settings” (Yuan & Powell, 2013, p. 7).

In 2008, Siemens ran the first MOOC with Stephen Downes, soon to be joined by

Dave Cormier, called “Connectivism and Connective Knowledge” (CCK08). The

course eventually attracted 2,300 non-paying, non-credit participants in addition to

the 25 students who took it for credit for the University of Manitoba’s Certificate in

Adult Education (Downes, 2012; Kolowich, 2013i). The three Canadian academics

(all of whom I interviewed – see chapter 4) designed the course along explicitly

connectivist lines, thus facilitating the course by providing the students with a basic

framework rather than instructing them in a more traditional way (Downes, 2012).

Different from xMOOCs, participants were not confined to a prescribed online

  24  

learning platform, but are “encouraged to use their own platform (blog, photo

account, social network site) to create and/or share resources” (Downes, 2012, p.

536). As Downes recalls about CCK08:

“170 of them created their own blogs, the feeds of which were aggregated

through a tool I created, called gRSShopper…,and the contents delivered by

email to a total of 1870 subscribers... Students also participated in a Moodle

discussion forum, in a Google Groups forum, in three separate Second Life

communities, and in other ways we didn’t know about” (Downes, 2012, p.

505).

Target participants appear to be rather advanced. “What we are trying to do with a

MOOC is to create an environment where people who are more advanced reasoners,

thinkers, motivators, arguers, and educators can practice their skills in a public way

by interacting with each other” (Downes, 2012, p. 509). “The idea behind the first MOOC… was to make online instruction dovetail with the

way people actually learn and solve problems in the modern world” (Kolowich,

2013i). Siemens and his collaborators wanted "to give learners the competence to

interact with messy, ambiguous contexts… and to collaboratively make sense of that

space" (Siemens, quoted in Kolowich, 2013i). CCK08 was the first cMOOC, and

there have been other courses in Canada and in the US, which follow that tradition

(Sandeen, 2013).

  25  

2.2.2 The Stanford higher education experiment

The vision is: change the world by bringing education to places that can’t be reached today. (Thrun, quoted in Markoff, 2011).

In 2011, two Stanford University professors, Thrun and Norvig, both world-leading

artificial intelligence (AI) experts, conducted an experiment in distributed education.

Their course “Introduction to Artificial Intelligence” drew from Stanford’s

homonymous introductory course, offered similar materials, assignments, and

examinations, and was offered free and online to students worldwide in the fourth

quarter of 2011 (AI Class, 2011).

One difference between the Stanford University experiment and earlier MOOCs was

its redefinition of being massive, its enormous outreach. While the AI course was just

one of three experimental online courses offered by the university’s computer science

department, the AI class alone attracted more than 160,000 students around the

globe, with an age range from high school to retirees from more than 190 countries

(Markoff, 2011; Udacity Press Kit 2013). Students themselves took the initiative to

translate the class into 44 languages, and over 400 students outperformed the top

Stanford student (Bennett, 2012).

In terms of credentialing, online students did not get the coveted Stanford credits, but

were ranked in comparison to the work of other online students and received a

statement of accomplishment, indicating their completion and level of achievement

(Markoff, 2011). For grading the 160,000 students, a system running on the Amazon

cloud was used, and in place of office hours, the Google moderator service that

enabled students to vote on the best questions for the professors to respond to

(Markoff, 2011).

While Thrun was amazed by the dizzying popularity of the online course, he was even

more surprised by the students’ attendance of the actual Stanford lecture dwindling

from 200 in-person attendees to just 20 or 30 students, as these students also preferred

to watch the lectures as online videos on their own time.

  26  

“These are students who pay $30,000 a year to Stanford to see the best and

brightest of our professors, and they prefer to see us on video? This was a big

shock to us” (Thrun, quoted in Hsu, 2012).

Figures 2.2 and 2.3: left: Professors Thrun and Norvig; right: Professor Thrun

demonstrates Google’s self-driving cars.

Sources: Markoff, 2011; Hsu, 2012.

2.2.3 Stanford University experiment spin-off #1: Coursera

Coursera was founded by Stanford computer science professors Koller and Ng, and

they have received US$85 million in venture capital (Kolowich, 2013j). From 2013

onwards, Coursera started to offer more than three dozen college courses on a wide

variety of subjects (ranging from Greek mythology to calculus). Professors from top

US universities such as Stanford and Princeton started to teach the courses under their

university’s name and adapted their most popular courses, embedding assignments

and exams into video lectures (Free Online Courses, 2012).

In terms of assessment, multiple-choice tests are computer-scored and peer grading

is used to assess more complex work. Similar to the Stanford University experiment,

students are ineligible for college credits, but can obtain certificates of completion for

a nominal fee (Free Online Courses, 2012).

  27  

2.2.4 Stanford University experiment spin-off #2: Udacity

Like Coursera, Udacity is a spin-off of the Stanford University experiment. It is a

private institution of higher education founded by Thrun and Evans with the original

goal of making free online classes available to everyone. Similar to Coursera, Udacity

has received venture capital, and initially hoped to provide free education, but

generate revenues by placing their top students in top companies. As of November

2013, Udacity offered 30 courses across so-called STEM (science, technology,

engineering and mathematics) subjects (Udacity Press Kit, 2013).

Different from other MOOC providers, “Udacity selects, trains and films the

professors who teach its courses” (Ripley, 2012, p. 40). Co-founder Thrun has gone

on record that he was “personally troubled by the 90 percent dropout rate”,

emphasizing that “MOOCs will only succeed if they make normally motivated

students successful” (quoted in Lewin and Markoff, 2013, p. 3). In later sections of

this thesis, Georgia Institute of Technology’s collaboration with Udacity to offer a

Master’s degree in computer science for US$7,000 via Udacity’s MOOC platform

will be further discussed, with a similar accredited degree pursued on-campus costing

approximately six times as much (Reynolds, 2014; Thrun 2014a).

Figure 2.4 Georgia Tech’s MSc in Computer Science MOOC

Source: Thrun, 2014a.

  28  

2.2.5 edX

edX was founded in May 2012 by Harvard University and the Massachusetts Institute

of Technology (MIT). It followed the success of MITx, an open online learning

project, whose first course had 120,000 students enrolled in March, 2012. Thus,

everybody with the capability to do academically well enough to successfully

complete an edX course, can now get a Harvard and MIT certificate, which, as usual,

carries no credit (Harvard, MIT to offer free Web courses, 2012).

edX is different from Coursera and Udacity, as it is nonprofit and open source. MIT is

famous amongst open education supporters for having started the OpenCourseWare

initiative. edX has set itself three goals, which are (1) “to open up access to quality

education globally”, (2) improve on-campus education and (3) “conduct research to

understand more precisely how students learn” (edX Media Kit, 2013, p. 5). edX has

an explicit social agenda in which MOOCs are viewed “as the great democratizer”

and in which it is believed “that in the future, economics, social status, gender or

geography won’t be a determinate of a student’s access to education or opportunity

for success” (edX Media Kit, 2013, p. 6).

2.2.6 Comparison of the three dominant xMOOCs

The three thus-far dominant MOOCs (Coursera, Udacity and EdX) have generated

astonishing student numbers in record time. As a result, they could be compared with

other big data companies such as Facebook or Twitter, being a platform for hosting

(learning content) and having proprietary algorithms (cf. Davis, 2014). All three

major MOOC platforms have distinct access missions.

One of the most distinctive differences is the way the three providers co-operate with

university partners. At one extreme of the spectrum is Coursera, which uses a

decentralised model, where the universities are mainly responsible for faculty and

content, while “Coursera provides the platform and various instructional and

  29  

assessment tools, format guidelines, course development support, marketing,

enrollment, and customer and technical support” (Sandeen, 2013, p. 6).

On the other extreme of the spectrum of university co-operation is Udacity, which has

the largest degree of vertical integration. Due to their highly detailed production

methods, Udacity has completed fewer courses than the other major MOOC

providers. The third MOOC platform, edX, is somewhere in between on the

continuum. While edX “also directly contributes to course and assessment design”, it

is “perhaps to a lesser degree than Udacity” (Sandeen, 2013, p. 6).

FutureLearn, the British answer to the US xMOOCs, was founded in late 2012, and

only started offering courses in September 2013, and was thus too new to include in

this research project. Table 2.1compares the three dominant xMOOCs according to

some key criteria. The MOOCs are still a very-fast developing phenomenon, and the

below table is largely based on data available on 18 January, 2014.

  30  

Table 2.1 Comparison of MOOCs

MOOC Udacity Coursera edX Launch January, 2012 April, 2012 May, 2012

Number of

enrolments

1.6 million

(January, 2014)

6.1 million

(January, 2014)

2.4 million

(October, 2013)

Fee (if any) Free for basics,

$150 p.m. for

value-added

services

Free for basics,

fees for value-

added services

Free for basics,

fees for value-

added services

Number of courses 30 (January, 2014) 639 (April, 2014) › 80 (October,

2013)

Number of

institutional

partners

Collaboration is

predominantly with

professors

108 (April, 2014) 29 (January, 2014)

Course design

model

vertical integration Decentralization in-between

decentralisation

and vertical

integration

Funding Venture Capital Venture Capital MIT & Harvard

For-profit / non-

for-profit

FP FP NFP

Source: self-developed, sources include: Udacity Press Kit, 2013; edX Media Kit,

2013; Ripley, 2012; Chafkin, 2014; Sandeen, 2013.

Table 2.1 shows that while all MOOC providers started out offering their courses free

of charge, they have since started to offer various kinds of value-adds. This will be

discussed in more detail in section 2.4 on the MOOCs’ sustainability. In the following

sections, the four research questions will be discussed from the perspectives of

academic and popular literature. Many of the reviewed articles are not peer-reviewed,

but often written by academics – peer-reviewed discussions have a substantial time

lag, and after the completion of this project, the publication of many peer-reviewed

articles is anticipated.

  31  

2.3 Do MOOCs offer a high quality educational experience?

The quality of the educational experienced is assessed via five sub-topics: (1)

participants, their motivation and completion rates; (2) literacies that may be required

to be successful in MOOCs; (3) whether there is content that is particularly suitable or

unsuitable to be taught in a MOOC; (4) the validity of assessment in MOOCs; and (5)

accreditation, credit transfer and badges.

2.3.1 Who are the participants, what is their motivation and what do the completion

rates signify?

It would be erroneous to assume that everybody who signs up for a MOOC would

behave like a traditional, fee-paying student who aims to complete the course by

participating in all activities and assessments. In fact, different types of participants

can be discerned. There are (1) auditing learners – who may watch lectures

throughout, but attempt few assessments; (2) completing learners (or active

participants), comparable to traditional students in that they attempt (most of) the

assessments; (3) disengaging learners– who only attempt assessments in the

beginning of the course and then start disengaging; (4) sampling learners– who

briefly explore a course by watching some videos; (5) drop-ins – who are engaged

only with select topics, but do not attempt to complete the course; and (6) a very high

proportion of window-shopping learners or lurkers (The Maturing of the MOOC,

2013).

MOOCs attract a global audience of participants. edX works with open-source

visualisation tools (called Insights) with which a rich trove of data can be analysed

almost real time (Biemiller, 2014b). For instance, Figure 2.5 shows the global

distribution of participants of Harvard MOOCs by 193 nationalities.

  32  

Figure 2.5 World map of enrolment for all HarvardX offerings as of 10

February, 2014

Source: Biemiller, 2014b.

In the BRICS (Brazil, China, India, Russia, South Africa) countries, where MOOCs

are popular, “80 percent of MOOC students come from the wealthiest and most well

educated 6 percent of the population”; and in other emerging economies, some 80

percent of the MOOC participants surveyed already held college degrees, again

showing that mostly the elite and not the bottom of the income pyramid is reached

(Kolowich, 2013l; see section 2.6.3).

However, according to Ho and co-authors’ (2014) data, only three percent of all

registrants were from underdeveloped countries. Perhaps as surprising, only 29

percent of all registrants were female. The overwhelming male participation in

MOOCs has also been observed by other authors (e.g. Kolowich, 2013l; 2014f).

Voss (2013b, p. 4) has observed that most participants are older, non-traditional

students “who use MOOCs for continuing education objectives, they are not students

currently enrolled in an undergraduate or graduate program”. Most MOOC students

already hold traditional degrees (Kolowich, 2013l; 2013o; 2014f; G. Christensen,

  33  

Steinmetz, Alcorn, Bennett, Woods & Emanuel, 2013). For instance, Ho, Reich,

Nesterko, Seaton, Mullaney, Waldo and Chuang (2014), in their quantitative analysis

of 17 edX courses, observed that 66 percent of all registrants already held a bachelor’s

degree or higher; and that 74 percent of those who earned a certificate of completion

held a bachelor’s degree or higher. In an earlier research on Penn State courses

offered through Coursera, more than 80 percent of the participants “had a two- or

four-year degree, and 44 percent had some graduate education” (Kolowich, 2013l).

Non-completion is often not seen as a significant problem, and it has been argued

that lurking and auditing have “validity as a learning activity within MOOCs” (The

Maturing of the MOOC, 2013, p. 5). Downes discusses the motivation of many

MOOC participants: “The majority of active users say they’re taking the course for

fun or a challenge, rather than a credential… The majority engages in sampling

behaviour” (cited in The Maturing of the MOOC, 2013, p. 28).

It has also been argued that, as dropout rates in many US colleges can be 50 percent

or higher, MOOCs “do not do so badly on drop-outs” by that standard, especially as

not much time and no money has been lost (The Maturing of the MOOC, 2013, p. 91).

Some recently published HarvardX and MITx working papers also discuss MOOC

attrition (Rutter, 2014).

“Most MOOC attrition happened after students first registered for a course.

On average, 50 percent of people left within a week or two of enrolling…

While the persistence rates in MOOCs look very different from those of

conventional courses in higher education, they look very similar to how

people interact with other Web-based media” (Rutter, 2014).

A median of 33,000 registrants signed up for MOOCs by 2013, with around ten

percent completing them (Kolowich, 2013j; Agarwala, 2013). We could argue either

that a 10% success rate is appalling or regard 3,300 successful completions as

phenomenal. If students have skin in the game and pay $50 for Coursera’s signature

track programme, the completion rate increases to 70% (Kolowich, 2013j).

  34  

Ho et al. (2014) in their research on edX argue that “course certification rates are

misleading and counterproductive indicators of the impact and potential of open

online courses”. Five percent of all registrants earned a certificate of completion; 35

percent never viewed any of the course materials; 54 percent of those who “explored”

at least half of the course content earned a certificate of completion (Ho et al., 2014).

In future studies, Ho and co-authors hope to the classify registrants according to their

varied reasons for taking a MOOC, so that expectations can be related to outcomes

(Kolowich, 2014f).

2.3.2 Required MOOC literacies

From a purely technical perspective, all that is needed to participate in a MOOC is an

Internet connection and access to some kind of computer (which could be a

smartphone). However, the analysis of MOOC participants in the previous section

showed that they are more often than not already well-educated and thus possess

literacies that not every potential MOOC participant has. Self-motivation also

appears to be important (Ripley, 2012).

There is some evidence that online learning does not work for at-risk students. “The

Columbia study of Washington community-college students found that all students

performed less well in online courses than in face-to-face ones, but the gap was even

wider among those with lower GPAs, men and African-Americans” (Fowler,

2013).San Jose State University’s initial failed experiment with Udacity included at-

risk students (see sections 2.6.1, 2.6.3). A second run with a broader mix of students

had better outcomes, “possibly because more than half of them already held college

degrees” (Kolowich, 2013q).

2.3.3 What courses or content may best be taught via MOOCs, if any?

From the evidence of what is on offer by the three dominant xMOOCs alone, there

does not appear to be much of a limit to what can be taught in a MOOC. Courses in

psychology and philosophy appear to work as well as those in mathematics and

science (Fowler, 2013). “Whenever someone says you can’t teach X online, half the

  35  

time someone is already doing it” (Ng, quoted in Fowler, 2013).

2.3.4 Assessment

It is important that assessment in MOOCs is fair and equitable, as the chances for

universities offering credit transfer would otherwise be remote. Cheating is already a

serious problem in a physical educational environment (Cizek, 1999), but it appears to

be far more serious in MOOCs. Types of cheating in an online environment include

online communication, telecommunication, internet surfing, copying and pasting from

online sources, obtaining answer keys in an illegitimate way, taking the same

assessment more than once, and getting unauthorized help during the assessment

(Underwood & Szabo, 2003; Rogers, 2006; Rowe, 2008; Meyer & Zhu, 2013).

A related issue is learner authentication, as the “MOOC learner is remote,

unverifiable, and identified merely by an email” (The Maturing of the MOOC, 2013,

p. 80). In order to strengthen the validity of MOOC assessments, both participants and

employers have an interest in authentication (Davis, 2014). To address the problem of

learner authentication, Coursera’s Signature Track (for which students need to pay a

fee) uses “keystroke biometrics” in addition to other forms of authentication –

government-issued photo ID, webcam, and credit card (Davis, 2014). Keystroke

biometrics analyses each participant’s typing pattern and rhythm via typing a short

phrase that is retyped with each assignment submission. Nevertheless, this method is

“generally less secure than a fingerprint scan or other biometric methods” (Young,

2013i).

For examination ‘proctoring’, there are two methods, (1) webcam invigilation and (2)

traditional invigilation. In webcam invigilation, after an initial scan of the room, the

students are monitored throughout the exam (Sandeen, 2013). Electronic measures

against cheating during an exam may also include installation of security software and

IP tracking (Rogers, 2006; Rowe, 2008; Meyer & Zhu, 2013).

As “webcam proctoring” may be deemed insufficiently reliable, both Udacity and

edX have formed partnerships with Pearson VUE to conduct externally invigilated

  36  

examinations (Australian Trade Commission, 2012). Externally invigilated

examinations, however, may put students in more remote locations at a disadvantage

(Sandeen, 2013). Apart from MOOC-specific tests, students could also undergo

certain standardised tests to demonstrate their competencies and skills. For instance,

hundreds of universities use the Critical Learning Assessment (CLA), which assesses

critical learning and writing skills (Arum and Roksa, 2011; Vedder, 2011).

 Due to the massiveness of MOOCs, a single instructor is unable to personally

evaluate participants’ work. Rollins (2014) provides an entertaining calculation

through which he aims to show the absurdity of MOOC strategies: assuming that a

professor in a midsize MOOC with 30,000 students spends one hour per student to

review papers or answer questions, this would take him 15 years and cost $1.5 million

in salary. As a consequence, MOOCs tend to “rely heavily on multiple choice

questions, formulaic problems with correct answers, logical proofs, computer code,

and vocabulary activities” (Balfour, 2013, p. 40). MOOC providers have chosen

different methods of scoring, and providing feedback on, student essays, which are

peer grading and automated essay marking (Balfour 2013).

The practice of peer grading is contentious (Cho, Schunn & Wilson, 2006). On

Coursera, after the deadline for an assignment has passed, students have a week to

evaluate five of their classmates’ essays using rubrics, failing which they do not get a

grade for their assignment. Peer grading is critiqued because of possible collusion

(Sadler & Good, 2006). Also, peers are less expert than the professor, hence the grade

awarded and the feedback provided may be less valid. There is the ‘blind-leading-the-

blind’ phenomenon, whereby myth may be elevated to fact, and the ‘charlatan’

phenomenon, where non-expert feedback is mistaken for expert advice (Downes,

2013).

Peer grading may be sounder for formative rather than summative assessment. If peer

grading is used for multiple rounds of revisions with rapid feedback cycles, it may

encourage higher-quality final submissions. Moreover, the act of reviewing itself has

metacognitive benefits because students get to provide feedback on other students’

work, compelling them to reflect and articulate (Peddycord, 2014).

  37  

Coursera uses a peer grading method called Calibrated Peer Review (CPR) which

has largely been positively reviewed in the non-MOOC-specific literature, as it results

in learning the required content as well as improves writing and evaluation skills

(Gunersel, Simpson, Aufderheide, & Wang, 2008; Margerum, Gulsrud, Manlapez,

Rebong, & Love, 2007; Pelaez, 2001; Russell, 2005; Russell, Chapman, and Wegner,

1998; Balfour, 2013). Other research shows that peer grading is surprisingly similar to

expert grading when it comes to the marks (Cho & Schunn, 2003; Paré & Joordens,

2008). There are some important caveats:

“[S]uccessful peer grading arises only from a well-articulated grading

philosophy, training for the would-be reviewers, and high-quality rubrics that

very clearly show what’s right, what’s wrong, and why that is. Without careful

planning and scaffolding, it comes across as a half-hearted attempt to reduce

the tedium of grading. Peer grading already starts at a disadvantage from

having to compete with the internalized expectations of how authority in the

classroom should be distributed” (Peddycord, 2014).

edX uses a machine-based Automated Essay Scoring (AES) application to assess

written work in their MOOCs (Markoff, 2013; Balfour, 2013). An AES system was

first programmed as early as 1973 and reached commercial viability in the 1990s

(Attali, Burstein, Higgins & Zechner, 2007; Shermis, Burstein, Higgins, & Zechner,

2010). Instructors have to first score 100 essays for AES to work. AES applications

usually predict instructors’ marks for a particular writing assignment by using essay

features “such as average word length, number of words in the essay, discourse

element length, proportion of grammar errors, scores assigned to essays with similar

vocabulary, and frequency of least common words” (Balfour, 2013, p. 41). In

addition, AES can report “word usage errors, sentence variety, style, text complexity,

vocabulary, content alignment with existing texts, thesis statements, supporting ideas,

conclusions, and irrelevant segments” (Balfour, 2013, p. 41).

AES appears to be highly reliable, as it is distinguishable from human evaluation in

being more consistent than human evaluation (Balfour, 2013). An added bonus of

AES is that it provides “immediate, consistent feedback to students about important

elements of their writing” (Balfour, 2013, p. 41). However, AES applications do not

  38  

understand texts in the way an expert does. They are unable to assess metaphors,

humour, slang, creative pieces or original research; most AES feedback is vague; and

students may be overly focused on correcting surface features rather than writing an

original piece of work (Graesser & McNamera, 2012; National Council of Teachers

of English, 2013; Balfour, 2013).

Balfour recommends experimentation in MOOCs with a combination of AES

(providing students formative feedback on multiple drafts) and CPR (for summative

assessment). Balfour sees potential synergistic effects by using the two methods in the

proposed sequence:

“[T]he more mechanical writing problems could be corrected earlier in the

writing process, improving the quality of essays feeding into CPR.

Subsequently, students using CPR to review essays may be exposed to higher

quality writing and thinking which may, in turn, benefit them even more than

using CPR with lower quality essays. This combination of AES and CPR may

be very powerful and could produce stronger writers more efficiently than just

human evaluation” (Balfour, 2013).

2.3.5 Accreditation, certification, credit transfer and badges

Academic credit was initially not a major motivation for MOOC learners, which is

small surprise given our observations how well-educated MOOC participants

generally are (see section 2.3.1). However, the MOOC learner demographics may be

bound to change (The Maturing of the MOOC, 2013). Thus far, MOOC certificates

have largely failed to penetrate the traditional system of credits and degrees

(Kolowich, 2013n), and they have even been called close to a “mill racket” (Daniel,

2012). Traditionally, accreditation agencies refuse to accredit courses, as they only

accredit degrees, even though a degree is essentially a collection of courses (Vedder,

2012). There also does not seem much employer recognition for MOOC certificates,

with companies like Yahoo that reimburse employees who have passed Coursera

courses (Belkin & Porter, 2013), being the exception to the rule.

  39  

MOOC accreditation appeared to be accelerating in the US (The Maturing of the

MOOC, 2013; Kolowich, 2013a). By 2014, 14 MOOCs were recommended for credit

transfer by the American Council on Education, the accreditation body for credit

awards, and seven universities agreed to participate in the experiment (Kolowich,

2014h). Surprisingly, no students appeared to have taken up the offer by early 2014, a

fact attributed to the relatively small number of approved courses and a relative lack

of publicity, with first results from this experiment only expected by 2015 (Kolowich,

2014h). This despite the fact that there is a sizeable discount available when one, for

instance, compares the US$89 payable for an invigilated exam versus the $1,050

payable to Colorado State University for a comparable course (Kolowich, 2013o).

Hoxby (2014) provides an economic argument against US elite universities providing

credits for their MOOCs, as this could compromise the more traditional ‘human-

capital investments’ in their portfolios as graduates may then be less inclined to make

generous donations to their alma maters. She argues that MOOCs provide a more

suitable substitute for non-selective college programmes (Hoxby, 2014; Kolowich,

2014e).

Various MOOC providers have taken steps to go beyond the single-course format by

offering sequences of courses which create a new type of alternative credentials.

Coursera markets their new programme, called “Specialisations” (which consist of

three to nine MOOCs each and a capstone project), as a “real-world ‘major’ with

immediate applications to career advancement and life skills” (Coursera

Specializations, 2014) which may be a “better match for the rapidly changing

demands of the job market than traditional degrees are” (Kolowich, 2014g). edX has

started a similarly-structured certificate programme called XSeries (Kolowich, 2013n;

2014g). Although an XSeries certificate in computer science would by no stretch of

the imagination equal a 15-course computer science major at MIT (Kolowich, 2013n),

such alternative credentials could become “legitimate currency in the job market”

(Kolowich, 2014g).

These alternative credentials go back to the concept of digital badges, modeled on

the boy scout badge no less, of which the Mozilla Open Badges concept is the leading

example (Sandeen, 2013). “Badges tend to acknowledge narrow and specific skills

  40  

and competencies and currently are a form of alternative micro-credentialing not

linked to formal academic credit as we know it” (Sandeen, 2013, p. 7).

The Georgia Tech online Master in Computer Science is an innovative experiment

that involves aspects of MOOCs (in co-operation with Udacity) and a major

technology company's support (AT&T) (Young, 2013d). Courses in the programme

will be free through Udacity's site, but course credit and degree are only available for

formally admitted and fee-paying students (who have access to teaching assistants and

sit for invigilated examinations). Admission is selective, with 401 out of 2,300

students having been admitted, and the inaugural batch of students having an average

undergraduate GPA of 3.58 and often industry experience (Kolowich, 2013q).

Consequently, the Master’s programme is “neither massive nor open” and seems “less

like a MOOC than many existing online graduate programs” (Kolowich, 2013q).

In conclusion, MOOC participants are usually very different from traditional fee-

paying students, they form a global audience, are older, better-educated, and often

more interested in learning rather than certification – hence, non-completion of

MOOCs should be viewed in that light. Disappointingly, the research discussed in

section 2.3.1, shows that MOOCs are so far not reaching out much to the bottom of

the pyramid (see section 2.6.3), which, amongst other things, may be related to the

poor not having the requisite literacies (section 2.3.2).

While there do not seem to be too many limits to what can be learned in a MOOC in

terms of subject matter (section 2.3.3), assessment is a critical and unresolved issue,

though there is room for interesting experimentation, like a possible combination of

AES and CPR (section 2.3.4). Due to the unresolved issue of assessment, certification

and credentialing have not made as much progress as advocates would have hoped.

The perceived quality of MOOCs is in direct relationship with other research

questions, including their sustainability, which will be discussed in the following

section.

  41  

2.4 Are MOOCs sustainable?

The sustainability of MOOCS is assessed via three sub-topics: (1) how much do

MOOCs cost, (2) who is paying and revenue models and (3) MOOCs as marketing

devices.

2.4.1 How much do MOOCs really cost?

The cost of MOOCs is to be discussed from the perspective of the various

stakeholders. As it is important not to confuse cost and price, the focus in this

section will be on cost, while the subsequent section (2.4.2) will discuss price in the

context of revenue models.

To students and faculty, perhaps the most meaningful cost is opportunity cost.

However, there is a recent trend to charge MOOC participants for certification and

other fees (see section 2.4.2), and as a consequence, depending on the financial

background of the student, this could be negligible or prohibitive. In the US, the cost

of higher education to students and their families has escalated, with more debt and

higher default rates on student loans (Archibald & Feldman, 2012; Desrochers &

Kirshstein, 2012). If students were able to transfer some credit earned by completing

free or low-priced MOOCs, this would decrease their total cost to obtain a credential

(Sandeen, 2013).

Normally, faculty appears to be a cost item to the university. For instance, Davidson

(2014) was paid US$10,000 for developing and offering her MOOC, which she

generously decided to spend on teaching assistants and other miscellaneous expenses.

It has been estimated “that professors typically spend 100 hours to develop a MOOC,

and then eight to 10 hours each week while the courses were in session” (The

Maturing of the MOOC, 2013, p. 71). However, the amount of time spent appears to

vary significantly, as, for instance, Belanger & Thornton (2013) report on Duke

University’s first MOOC that more than “600 hours of effort were required to build

and deliver the course, including more than 420 hours of effort by the instructor”. In

the Georgia Tech online Master’s co-operation with Udacity, the cost of faculty is a

  42  

consideration, and Udacity was reported to hire course assistants to help Georgia Tech

instructors with both academic and non-academic tasks (Kolowich, 2013m).

To the MOOC providers and the universities, substantial cost is involved. A cost

item is often the fee that the university pays the MOOC provider for hosting the

course. For instance, in a contracting model, edX is paid US$250,000 for producing a

MOOC for a participating University, and if a course is rerun, an additional fee of

US$50,000 per every new edition of the course (Kolowich, 2013e; The Maturing of

the MOOC, 2013). For Coursera, a university estimated its cost to be $50,000 per

MOOC, not including faculty time (Popp, 2013), with MOOCs that incorporate a high

degree of design, assessment, and analytics costing “much more” (Sandeen, 2013, p.

8).

The claim has often been made that due to MOOCs’ ‘one to many’ potential, a

MOOC can reach tens of thousands of students for the cost of reaching ten students at

zero marginal cost per student after the initial investment (Rollins, 2014). However,

Rollins (2014) points to research that shows that online courses can be more costly

than on-campus courses and concludes that by “ignoring history, MOOCs inherit a

tough problem in economics of higher education”.

To complete the overview of the different costs to various stakeholders, universities

and venture capital enabled the birth of xMOOCs. For edX, MIT and Harvard

contributed $60 million in startup capital, and for Coursera and Udacity, venture

capital was used.

2.4.2 Revenue models

With much of the cost being shouldered by universities, there does not appear to be a

very clear revenue model for universities and hence no easily definable return on

investment (ROI). Rollins (2014) has expressed his lack of understanding as to “what

incentive a university has to adopt an expensive course of uncertain effectiveness that

leads to faculty revolt”.

  43  

However, many possible sources of income have been identified for MOOC

providers (e.g. Voss, 2013b, The Maturing of the MOOC, 2013), and some of them

have been put into practice. The following 12 suggestions for revenue generations

have been made, and as the first four are related, their application will be discussed

together.

(1) Certification and credentialling – participants pay for some kind of evidence that

skills and knowledge have been acquired. This could be a badge or certificate, or it

could also be for credit transfer for advanced standing or in the extreme case, a whole

degree.

(2) Participants may pay for human tutoring or assignment marking.

(3) Participants pay tuition fees.

(4) Secure assessments via invigilated examinations are chargeable.

In 2013, for a period totaling less than one year, Coursera announced a revenue of $1

million from ‘verified’ course certificates from its Signature Track, thus far its only

significant revenue source (Kolowich, 2014c; 2013n). If students want to pursue the

Signature Track and obtain a verified certificate, they have to decide early in the

course and pay upfront a fee ranging from $30 to $100 (Young, 2013i). edX’s

XSeries and Coursera’s “Specializations” programmes are expected to create much

additional revenue for the MOOC platforms and their university partners. The MIT

XSeries certificate in computer science is priced at about $425, and Coursera’s

“Specializations” courses will be priced between $200 to $500 (Kolowich, 2014g).

Coursera continues to offer free unverified certificates to participants who pass its

courses. Financial aid may be offered to “students who demonstrated that they could

not afford the fees but could benefit from the verified certificates” (Young, 2013i). In

a dramatic development, Udacity recently started to charge for all its certificates,

discontinuing its previous practice of offering free, ‘non-identity-verified’ certificates

(Kolowich, 2014a). Udacity’s co-founder Thrun (2014b) related fee payment to

credential acceptance:

“We know that many of our hardworking students can’t afford to pay for

classes. At the same time, we cannot hope that our certificates will ever carry

great value if we don’t make this change.”

  44  

Udacity’s new ‘full’ courses cost $150 per month and, in addition to the courseware

(that will continue to be available for free), include contact with coaches, project-

based assignments and job-placement services (Kolowich, 2014a; see Figure 2.6).

Figure 2.6 Udacity’s free versus subscription model

Source: Udacity, 2014.

Georgia Tech’s co-operation with Udacity in the Master of Computer Science

programme was discussed earlier. The low tuition of $7,000 puts“a top-ranked

computer-science program at a price point more comparable to a typical community

college” (Young, 2013d). Course revenue is split 60-40 between Georgia Tech and

Udacity, and the partners hope to bring the economies of scale that MOOCs offer to a

postgraduate degree by aiming to enrol 10,000 students over the next three years

(Young, 2013d; Kolowich, 2013m).

However, attempts to charge for standalone MOOCs have occasionally been

unsuccessful. In 2013, the University of Texas at Austin (UT) conducted a

psychology MOOC, planning for up to 10,000 students and charging $550 per non-

UT enrollee. 125 people were engaged to produce and administer the course, but

fewer than 50 non-UT students enrolled (Rollins, 2014).

(5) Data mining, employee recruitment and applicant screening – companies pay for

access to student performance records.

  45  

(6) Selling the MOOC platform to enterprises to use in their own training courses.

(7) Third-party sponsors pay for selected courses. AT&T has set a precedent by

supporting the Georgia Tech Master’s programme with $2 million, and it aims to

create a pipeline of qualified applicants (Young, 2013d).

(8) Advertising could serve in similar ways like elsewhere on the Internet.

(9) Cross-selling or up-selling more advanced or certified course offerings. The

concept of freemium, where something is given away for free, but premium services

command a premium, which is used in many industries, is also being considered for

application to the marketing and recruitment of students (Australian Trade

Commission, 2012).

(10) Donations from online students and alumni (Free Online Courses, 2012).

(11) Charging universities. There are several models in place. One of edX’s models is

that edX receives US$50,000 of revenue from any course, and a 50% split thereafter

(Kolowich, 2013e).

(12) Content licensing and providing more conventional online courses (Kolowich,

2014c; 2013m). Coursera has begun to work with public universities in the US to use

its platform for “guided” or “adopted” non-MOOC online courses, charging $3,000

for development plus an additional per-student fee (Kolowich, 2013m). These courses

carry credit and students are enrolled with the universities and pay tuition. When

universities license course content from one another, Coursera is also getting a

percentage of the license fee (Kolowich, 2013m).

The provision of a platform for non-MOOC online courses sees MOOCs like

Coursera in potential competition with LMS providers(like Blackboard,

Desire2Learn and Instructure) and perhaps even with textbook publishers (like

Pearson and McGraw-Hill) (Kolowich, 2013m). “In the long term, the fate of

Coursera and Udacity's ambitions may depend on how well their platforms and

content work in a non-MOOC context” (Kolowich, 2013m).

  46  

2.4.3 MOOCs as marketing devices

The idea of MOOCs as a marketing device is linked to our discussion in the previous

section on cross- and up-selling as well as the freemium concept. MOOCs are part of

marketing strategies of universities, signaling, in conjunction with social media

strategies, “futurism and reinvention” (Bogost, 2012).“In cases like Caltech and

University of Pennsylvania, who have together invested some $3.7 million in

Coursera, they are buying a more explicit and long-term version of that advertising.

Likewise, that's how MIT and Harvard see edX” (Bogost, 2012).

2.4.4 Are MOOCs sustainable?

It is difficult to predict whether the current dominant players will be sustainable. edX,

because of the institutional support of Harvard and MIT and these universities’

substantial financial and human resources, may appear most sustainable. Coursera has

the most decentralised model (see section 2.2.6) and thus appears to be most

dependent on the support of its university partners and their professors, as the returns

for them have been “largely intangible” (Kolowich, 2014c). However, as a big data

company and because of economies of scale, it may be sustainable in any event.

Udacity’s sustainability may depend on the success of co-operations with industry and

university partners and its certification programmes.

However, it would be erroneous to equate MOOCs with the three dominant US

xMOOCs. There have been MOOCs before them, and there is absolutely no reason

for them to disappear entirely, even if the three existing major players turned out not

to be sustainable. The sustainability discussion is closely linked with the next research

question, which is whether MOOCs offer a viable alternative to traditional higher

education.

  47  

2.5 Do MOOCs offer a viable alternative to traditional higher education?

“College Is Dead. Long Live College!”

(Time's “Reinventing College” issue)

Udacity’s co-founder Stavens thought “the top 50 schools are probably safe" (quoted

in Carlson & Blumenstyk, 2012). During the dizzying heights of MOOC hype in

2012, Stavens’s colleague Thrun predicted that fifty years into the future, only ten

universities would remain, one of which would be Udacity (Leckart, 2012). Thrun

was much criticized for this and similar statements, but he was certainly not alone,

with the president of Northeastern University, Aoun, saying that with the advent of

MOOCs, “we're witnessing the end of higher education as we know it” (quoted in

Carlson & Blumenstyk, 2012).

However, there is much consensus that “MOOCs do not immediately threaten the

continuation of traditional Higher Education in campus universities offering face-to-

face teaching” (The Maturing of the MOOC, 2013, p. 18). Generally xMOOC

providers are aware of their dependency on traditional education and make

conciliatory statements such as: “MOOCs won’t replace universities, but rather

enhance the quality of education by incorporating blended learning” (edX Media Kit,

2013, p. 6). MOOCs are originally isolated courses that do not amount to a degree.

There have been some potential breakthrough developments like the Georgia Tech

Master’s programme as well as Coursera’s “Specializations” programme and MIT’s

XSeries, amongst others, that obviously aim at addressing this shortcoming.

At present, MOOCs do not seem to provide a viable alternative to traditional higher

education, as universities retain their monopoly on credit-granting privileges

(Kolowich, 2013m). Rather, MOOC providers complement and supplement

universities. The discussion about whether MOOCs are a viable alternative to

traditional higher education is inextricably intertwined with the final research

question, which is also at the heart of the problem statement, which will be discussed

in the following section.

  48  

2.6 MOOCs as disruptive innovation and providing access to the bottom of the

pyramid?

This section discusses whether MOOCs are revolutionary (section 2.6.1), an instance

of disruptive innovation (section 2.6.2), provide access to the base of the income

pyramid (section 2.6.3) and to what extent, unbundling and rebundling of higher

education is occurring.

2.6.1 Are MOOCs a revolution?

I can't teach at Stanford again. You can take the blue pill and go back to your classroom and lecture to your 20 students, but I've taken the red pill and I've seen Wonderland. (Thrun, quoted in Hsu, 2012)

Thrun’s humourous quotation refers to the 1999 movie Matrix, where Neo meets

Morpheus and agrees to follow him by swallowing a red pill (instead of the blue one).

Subsequently, Neo realizes that they live in the year 2199, in which humans are

fighting intelligent machines that control the Earth's surface and harvest human

bioelectrical energy. Humans are kept docile within the Matrix, a simulated reality of

the world as it was in 1999.

It is quite common to ponder whether MOOCs are revolutionising higher education

and whether MOOCs are a “high-tech engine of a transformative revolution that will

remake education as a highly engaging, open and low-cost activity” (The Maturing of

the MOOC, 2013, p. 6). Coursera’s co-founder Koller co-authored a presentation

called “MOOCs: The Coming Revolution” (Voss, 2013a), and Voss (2013b, p. 1) has

proclaimed: “This revolution is not about IT, it is about teaching and learning”. The

revolutionary concept is often linked up with the popular concept of “disruption” that

will be further explored in the next section in the context of disruptive innovation.

There is much skepticism in the literature regarding the revolutionary character of

MOOCs, with Rollins (2014) denying the existence of a “Magister ex machina

  49  

miracle”. Even Coursera’s Koller conceded that MOOCs cannot “really move the

needle on fundamental educational problems” (quoted in Kolowich, 2013o).

Similarly, Udacity’s Thrun self-critically remarked that a “medium where only self-

motivated, Web-savvy people sign up, and the success rate is 10 percent, doesn't

strike me quite yet as a solution to the problems of higher education” (quoted in

Kolowich, 2013m). A disenchanting collaboration between San Jose State University

and Udacity on a statistics MOOC resulted in only half of the students earning a

passing grade (which was a lower passing rate when compared to the face-to-face

version), and other courses also produced underwhelming results (Kolowich, 2013b).

Rollins (2014) critiques MOOCs from a historical perspective and asks rhetorically

why MOOCs should “fare any better than their extinct predecessors?” During the dot-

com era, universities and venture capitalists created online learning ventures that

largely went bust, and going further back to 1885, it was predicted “that mail-

correspondence students would soon outnumber students on campuses”, while later,

“radio lectures would move higher education forward by 25 years” and lavish hopes

were bestowed on “educational television” (Rollins, 2014). Kolowich (2013m) also

concludes that MOOCs may offer universities technology tools and services “that are

more helpful than revolutionary”.

2.6.2 MOOCs as disruptive innovation?

In their book The Innovative University, Christensen & Eyring (2011) view online

learning as a disruptive innovation in higher education (see section 1.6). As the focus

is shifting from professors’ credentials and institutional prestige to what students

actually learn, universities “must… master the disruptive technology of online

learning and make other innovations” (Christensen & Eyring, 2011, p. xxvi). For

universities, the “challenge is to change in ways that decrease its price premium and

increase its contributions to students and society” (Christensen & Eyring, 2011, p.

396).

In 2011, MOOCs had not been highly on the global agenda. However, in 2013,

Christensen has specifically commented on MOOCs as a disruptive innovation.

  50  

Horn and Christensen (2013) argue that the “question is not just whether MOOCs are

going to disrupt traditional education, but how”. They contend that MOOCs “bear the

early hallmarks of a disruptive innovation” as they serve non-consumers, march

upmarket (through improved quality) and redefine quality – in “the current university

system…, most faculty are rewarded for the quality of their research – not for the

quality of their teaching…; in the future, courses might be offered based on employer

demand, not faculty research interests” (Horn & Christensen, 2013).

Horn and Christensen (2013) characterise it as unusual – by disruptive innovation

standards – that it is some of the world’s most famous universities that lead the

MOOC movement. They point to Harvard and MIT being behind edX; Stanford’s

groundbreaking Artificial Intelligence course having morphed into Udacity; and some

Stanford professors having founded Coursera (Horn & Christensen, 2013).

In perhaps self-congratulatory fashion, it is argued that universities invest in MOOCs

“because disruption theory is finally widely enough understood that astute leaders

know how to identify and chase opportunities early” (Horn & Christensen, 2013). The

authors laud the involved universities for setting up “an autonomous business model

with different resources, processes, and priorities” and point to the useful leveraging

on the brands, as being associated with the likes of Harvard, MIT, and Stanford

signals quality (Horn & Christensen, 2013).

Horn and Christensen (2013) see MOOCs evolving into a “scale business” and

“facilitated network model” that optimises and individualises learning opportunities

for millions of students. They appear to seriously consider the possibility of the

traditional degree becoming “irrelevant”. When “learning becomes a continuous, on-

the-job learning process”, then “the need for customization will drive us toward just-

in-time mini-courses” (Horn & Christensen, 2013). This kind of transformation would

be akin to General Motors’ bringing forth choice and variety in replacing Ford’s

Model T (Horn & Christensen, 2013).

When describing the current situation of higher education, Christensen uses the

metaphor of the crevasse, a deep crack in a glacier.

  51  

“I think higher education is just on the edge of the crevasse. Generally,

universities are doing very well financially, so they don’t feel from the data

that their world is going to collapse. But I think even five years from now

these enterprises are going to be in real trouble” (Christensen, quoted in

Howe, 2013).

Christensen anticipates that "[s]ome” universities will “survive”, with most evolving

into hybrid models, “in which universities license some courses from an online

provider like Coursera but then provide more specialized courses in person"

(Christensen, quoted in Nisen, 2013). Even Harvard Business School does not teach

accounting anymore, as they use Brigham Young University’s (BYU)

“extraordinary” online accounting course (Christensen, quoted in Howe, 2013).

Friedman (2013b) comments on the latter aspect: “When outstanding becomes so

easily available, average is over” (Friedman, 2013b).

Christensen’s perspective that MOOCs promote an imminent process of disruptive

innovation and will cause pain to existing Higher Education players is shared by a

significant minority of academics (The Maturing of the MOOC, 2013). The disruptive

innovation has also been viewed in a different way than Christensen. Lawton and

Katsomitros (2012) have regarded it, amongst other things, as the shifting of costs

from students to institutions and future employers. Yet another aspect of MOOCs may

be an innovation in the process of knowledge creation, and them providing a “testing

ground for knowledge growth in a distributed, global, digital world” (The Maturing of

the MOOC, 2013, p. 47).

.

“They are a departure from the traditional brick and mortar Universities and

from the ‘walled gardens’ of conventional learning management systems,

insofar as they share the processes of knowledge work, not just the products.

Facilitators share their sense making habits and their thinking processes with

participants” (The Maturing of the MOOC, 2013, p. 47).

Disruptive innovation by MOOCs has also been described as a potential catalyst for

moving us “from an instruction paradigm to a learning paradigm”:

  52  

“By leveraging the vast resources available via the Internet and by using the

technology available today through the use of multimedia, instructional

design, automated assessment and web-based faculty-student interactive

strategies, the classroom experience is being re-created and high-quality

learning is now available to those individuals who might not otherwise have

access or the financial wherewithal – here and around the world” (DiSalvio,

2012).

The majority of academic observers does, however, appear to not regard MOOCs as a

disruptive innovation. Levin, Coursera’s new chief executive, reiterated that his

company’s MOOCs should be thought of as “additive to what universities are doing,

not disruptive” (quoted in Kolowich, 2014b). MOOCs may be disruptive only in

appearance, but they could end up not disrupting anything and rather being a teaching

aid in traditional, credit-bearing courses (Kolowich, 2013o). “As you go in the belly

of the beast, you will run into this brick wall every single time”, says Horn, a co-

founder of the Clayton Christensen Institute for Disruptive Innovation (quoted in

Kolowich, 2013m). Horn also commented on the MOOC providers’ resemblance with

technology vendors like Blackboard and Pearson that “are not disruptive at all”

(quoted in Kolowich, 2013m).

2.6.3 Do MOOCs provide access for the bottom of the pyramid?

Perhaps the most prominent motivation among professors for teaching MOOCs, is

“altruism – a desire to increase access to higher education worldwide” (Kolowich,

2013h). This motivation can also be found amongst the innovators of the first MOOC

like Downes:

“I want and visualize and aspire toward a system of society and learning

where each person is able to rise to his or her fullest potential without social or

financial encumbrance, where they may express themselves fully and without

reservation through art, writing, athletics, invention, or even through their

avocations or lifestyle… This to me is a society where knowledge and

learning are public goods, freely created and shared, not hoarded or withheld

  53  

in order to extract wealth or influence. This is what I aspire toward, this is

what I work toward” (Downes 2011, p. 3).

In 2014, edX announced a programme called SocialEDU in co-operation with

Facebook, Nokia (as the device manufacturer), service provider Airtel and the

Rwandan government to provide free, localized education to students in Rwanda on

affordable smart phones (Biemiller, 2014a). The mobile teaching app was to be

optimized for a low-bandwidth environment. edX’s president Agarwal explained that

“[i]mproving global access to high-quality education has been a key edX goal from

Day 1” and SocialEDU was part of the learning process how to achieve this goal

(quoted in Biemiller, 2014a).

edX’s good intentions to provide global access to higher education are shared by

Coursera in its mission statement: “We envision a future where everyone has access

to a world-class education that has so far been available to a select few” (quoted in

Kolowich, 2013l).Siemens commented on edX: “Projects like this can impact lives

around the world, for the next billion students from China and India” (quoted in

Harvard, MIT to offer free Web courses, 2012).

However, serious doubt has been cast whether the bottom of the pyramid can be

reached. Our review of MOOC participants has shown that many of them were highly

educated and that participants without access to higher education in developing

countries were highly underrepresented (section 2.3.1; G. Christensen et al., 2013).

This may be partially due to a relative lack of publicity as well as spotty Internet

connections in developing countries (Kolowich, 2013l).

More important for the difficulties of MOOCs in reaching out to the BoP may be the

link to literacies (see section 2.3.2). G. Sharma (2013) calls “the intellectual barrier”

the “elephant in the room [that] is still invisible”. He also warns against “a one-way

transfer of educational materials from the rich north to the poor south” and the

resultant “intellectual neo-colonialism” and asks rhetorically: “What magical powers

do the MOOC platforms… provide… that make all the challenges of facing students

across the world simply disappear?” (G. Sharma, 2013).

  54  

Altbach (2013) critically questions the MOOC phenomenon in a similar vein. There

are cultural biases in academic traditions, methodological orientations and teaching

philosophies of particular academic systems. At present, MOOCs are largely an

American-led effort, with most courses coming from Western countries, thus

potentially bolstering Western academe’s “higher-education hegemony”, and “making

it more difficult for alternative voices to be heard” (Altbach, 2013).

There are serious challenges for MOOCs to be successful even in the US. Udacity’s

co-founder Thrun commented on the failed experiment with San Jose State

University: “These were students from difficult neighborhoods, without good access

to computers, and with all kinds of challenges in their lives. … It's a group for which

this medium is not a good fit” (quoted in Schumann, 2013).

Schuman (2013), in a polemical critique, noted that MOOCs were supposed to bring

higher education to the masses, but they were not ready for the “commodified,

impersonal higher education that MOOCs offer”. She contented that not the students

were to blame for the failed San Jose experiment, but the medium.

“Successful education needs personal interaction and accountability… This

is… the same reason students feel annoyed, alienated, and anonymous in large

lecture halls and thus justified in sexting and playing World of Warcraft

during class – and why the answer is not the MOOC, but the tiny, for-credit,

in-person seminar that has neither a sexy acronym nor a potential for huge

corporate partnerships” (Schuman, 2013).

Schuman (2013) did not doubt that MOOCs were appropriate for highly educated

participants, but concluded that Thrun has involuntarily shown that higher education

cannot be automated. Carlson and Blumenstyk (2012) have predicted the perpetuation

of a two-tier system in which only the less wealthy, “less-prepared students who are

increasingly cut off from the dream of a traditional college education” will study low-

and no-cost MOOC's. The irony is that students at the base of the pyramid need face-

to-face instruction more than other students. To a university president, the “real

disruption” was “the changing demographics” in the US, with the real problem being

  55  

that “higher education has to repeat a whole lot of lower education” (McGuire, quoted

in Carlson & Blumenstyk, 2012).

Similarly, Bates (2013) argues that xMOOCs are likely to “increase inequality, by

undermining publicly funded education, leaving an elite of campus-based universities

for the very rich, resulting in high paid knowledge-worker employment for them, and

massive information transmission delivered to the rest, who will be confined to low-

wage service jobs because of their lack of high-level critical thinking skills”.

2.6.4 Unbundling and rebundling

With curricula increasingly becoming a commodity, MOOCs – and Open Education

Resources (OER) before them – “have opened up access to tried and tested curricula”

especially for more basic undergraduate courses (The Maturing of the MOOC, 2013,

p. 50). In the Australian Trade Commission’s (2012, p. 7) view, not only the creation

and dissemination of courses gets disrupted and unbundled, but MOOCs “may also

disrupt teaching and learning as new pedagogy is created around delivering education

at massive scale” and “how assessment is conducted by incorporating machine

learning and peer-to-peer assessment models”.

As a consequence of this unbundling, the role of the faculty may become

disaggregated. While for centuries, “postsecondary teaching has been vertically

integrated: identifying a subject area, designing a course, sourcing content, organizing

content, determining learning outcomes, designing exercises and assessments,

teaching the course, scoring assessments, and assigning final grades”, the role of

faculty may “become one of curator of information and mentor to students” if other

aspects are unbundled and outsourced to experts (Sandeen, 2013, p. 8). Figure 2.7

shows that university education is a rather complex ‘bundle’.

  56  

Figure 2.7 College: A Packaged Bundle

Source: Staton, 2012, p. 5.

A report by Pearson regarded unbundling as a threat and rebundling as an opportunity

as “those who rebundle well will find they have reinvented higher education for the

21st century” (quoted in The Maturing of the MOOC, 2013, p. 50). The rebundling

may lead to “new network lock-in models” (Siemens, 2013).

2.7 Conclusions and Research Model

In conclusion, MOOCs may not be a revolution or a disruptive innovation, as they are

but a chapter in the history of online learning. Christensen may have been on the right

Draft: Please do not cite without permission from the author.

succeeding within, and adding value to our society. The bundle is outlined here in the below

diagram.

Figure 1: College: A Packaged Bundle

5

  57  

track in his earlier publication (Christensen & Eyring, 2011) when he characterised

online learning (rather than MOOCs) as the disruptive innovation.

In section 2.6.1, Christensen’s characterisation of disruptive innovation as serving

non-consumers, reaching upmarket and redefining quality was referred to. However,

as already shown in section 2.3.1, non-consumers are not normally reached by

MOOCs – at least not yet. It is also suspected that rather than “reaching” upmarket,

the MOOCs’ early adapters were university instructors and other highly-educated and

tech-savvy participants and they may move now ‘downmarket’, which in any event, is

the stated intention in all xMOOCs’ access missions (see section 2.2.6). Christensen’s

third characteristic is the redefinition of quality. As we saw in section 2.3, the quality

of MOOCs, when compared to traditional higher education courses, is unfortunately

questionable, especially in terms of assessment and the value of their credentials.

Christensen’s own definition of disruptive innovation could thus be used against his

characterization of MOOCs as such. xMOOC providers cannot be faulted for their

publicly stated good intentions of providing access to the bottom of the pyramid, and

there are some laudable initiatives. However, they appear to be in the process of

further diluting the openness of their business and educational model. The content

(curricula, video lectures etc.) of the for-profit providers (Coursera and Udacity) was

never open. While admission was originally open as in free, there has been a tendency

of reducing the freely available offerings further.

The MOOC phenomenon is still extremely new and undergoing rapid changes. As a

consequence, the judgment in the literature is still highly ambiguous. The core of the

research model consists of the MOOC concept, with all of its components (Massive,

Open, Online, Courses) being important aspects, and the concept as a whole related to

the four research questions.

As the bidirectional arrows show, there are inter-relations between the MOOC

concept and all four key concepts that are expressed in the research questions: (1)

quality, (2) sustainability, (3) viable alternative to traditional higher education, and (4)

disruptive innovation. All these concepts come with a question mark, as they are all

contested. It was tempting to also attach question marks to all four MOOC

  58  

components, but this was already implicit in the initial definition and discussion of

MOOCs in section 1.6 and thus did not need to be re-emphasised.

There are also bidirectional relationships between all research questions. The question

of high quality is related to all other concepts, as can be briefly described in the

following three considerations. (1) If the MOOC concept does not deliver a quality

learning experience, it may not be sustainable. (2) If it does not deliver a high quality,

it would disqualify itself as a viable alternative to traditional higher education. (3)

There is also a relationship between quality and disruptive innovation, though

disruptive innovation is normally defined as something that just needs to be 'good-

enough'.

Similarly, the relationships between the other concepts could be elaborated on. For

instance, MOOCs cannot be a disruptive innovation if they do not constitute a viable

alternative to traditional higher education. And if MOOCs are not sustainable, then

they would not be much of a disruptive innovation. So, in the final analysis, all five

concepts are interrelated with each other.

The core of the research model (Figure 2.8) is surrounded by the three sources of data,

(1) literature review, (2) expert interviews and (3) participant observation. As is

explained in chapter 3, participant observation refers to my sampling of some

MOOCs and is not a major method in this research.

  59  

Figure 2.8 Research model

Source: self-developed

It will become clearer in chapter 3, that in this qualitative research, a circular model of

process and theory is applied (see Figure 3.1). This circularity further extends to the

structure of the literature review and the analysis of the empirical data. They

continued to influence each other in a circular, rather than in a linear, fashion during

the process of presentation.

  60  

3. Methodology and methods 3.1 Introduction and research design

This chapter designs appropriate methodology and methods to explore the research

questions. My research paradigm is interpretivism, my specific theoretical position is

closest to symbolic interactionism (although I would not totally disavow

hermeneutics and phenomenology, being quite eclectic and normally preferring a

Foucaultian toolbox approach), the specific methodology is qualitative, and the

research method is interviews, more specifically convergent expert interviews.

When constructing a research design, the eight components in Figure 3.1 should be

considered. Briefly, the goal of the study is description (as opposed to testing of

hypotheses or theory development). There is not much of a MOOC-specific

theoretical framework, as MOOCs are a rather new phenomenon and the approach

adopted here would tend to discover a grounded theory, which, however, goes beyond

the scope of this study. In terms of a general educational theoretical framework, I am

strongly influenced by Biggs and Tang’s (2011) constructive alignment approach in

implementing outcomes-based education. The relationship between the research

questions (section 1.2) and the interview guide will be discussed in section 3.4.1.

Empirical material (especially the transcribed expert interviews and my own

observations based on MOOC sampling) was collected voraciously, and not

everything could be used within the humble confines of this project. Methodological

procedures are described in section 3.4. Their degree of standardization and

control is best described as loose, which is appropriate for this study, as MOOCs are

a relatively new field and theoretical analyses of them are relatively underdeveloped

(Miles & Huberman, 1994; Flick, 2009). No goals of generalisation are pursued due

to the exploratory nature of the study.

I discuss the interpretivist research paradigm and symbolic interactionism in section

3.2 and justify the appropriateness of qualitative research methodology within a

circular model in section 3.3. In section 3.4, the six steps of the interview process are

discussed and the eight expert interviewees are introduced. In section 3.5, participant

observation in cyberspace as a secondary method is reflected upon, before quality

  61  

(section 3.6) and ethical considerations (section 3.7) of this qualitative research study

are discussed. The conclusion provides a brief summary of the methodological

approach and links it to the empirical analysis in chapter 4.

Figure 3.1 Components of qualitative research design

Source: adapted from Flick, 2009, p. 133.

3.2 Interpretivist research paradigm and symbolic interactionism

My research is paradigm-guided. In the context of education research, a paradigm is

a very broad term that encompasses assumptions about the social world, elements of

ontology and epistemology, theory and philosophy, methodology as well as methods,

and techniques and topics (Punch, 1998; Denzin & Lincoln, 2011). The research

paradigm provides guidelines that connect theoretical paradigms to strategies of

inquiry and also to methods for collecting empirical materials (Denzin & Lincoln,

2011).

  62  

In the interpretivist paradigm, individual and society are inseparable. All human

action is meaningful and has to be interpreted within the context of social practices.

There are four major assumptions of interpretivism: (1) everyday activity is the

building block of society; (2) there is always some autonomy in everyday activity; (3)

everyday activity nearly always involves social interaction; and (4) it involves

negotiation of meaning, leading to a modification of our views (Blackledge and Hunt,

1985, O’Donoghue, 2007).

Symbolic interactionism, in the words of Blumer (1969, p. 2), has three premises:

“The first premise is that human beings act toward things on the basis of

meanings that the things have for them… The second premise is that the

meaning of such things is derived from, or arises out of, the social interaction

that one has with one’s fellows. The third premise is that these meanings are

handled in, and modified through, an interpretative process used by the person

in dealing with the things he encounters”.

Another central assumption is the Thomas theorem, according to which a situation is

real in its consequences, as long as a person defines a situation as real (Flick, 2009).

There is a relationship between theoretical positions, methods of data collection and

interpretation, and field of application. In Table 3.1, relevant aspects to this research

project are highlighted in bold. This will be elaborated in the following sections.

  63  

Table 3.1 Research perspectives in qualitative research

Approaches to

subjective

viewpoints

Description of the

making of social

situations

Hermeneutic

analysis of

underlying

structures

Theoretical

positions

Symbolic

interactionism

Phenomenology

Ethnomethodology

Constructivism

Psychoanalysis

Genetic

structuralism

Methods of data

collection

Semi-structured

interviews

Narrative

interviews

Focus groups

Ethnography

Participant

observation

Recording

interactions

Collecting

documents

Recording

interactions

Photography

Film

Methods of

interpretation

Grounded theory

coding

Content analysis

Narrative analysis

Hermeneutic

methods

Conversation

analysis

Discourse analysis

Genre analysis

Analysis of

documents

Objective

hermeneutics

Deep hermeneutics

Fields of

application

Biographical

research

Analysis of

everyday

knowledge

Analysis of life

worlds and

organizations

Evaluation

Cultural studies

Family research

Biographical

research

Generation

research

Gender research

Source: adapted from Flick, 2009, p. 457.

  64  

3.3 Qualitative research methodology and the circular model

3.3.1 Qualitative research methodology

In the 1960s, social scientists like Glaser and Strauss (who coined the concept of

grounded theory) concluded that grand theories like Parsons’s system theory, “which

were originally meant to explain more or less everything”, “ended up explaining

almost nothing on the level of everyday phenomena” (Flick, 2009, p. 49). The

grounded theory approach implies that theories “should not be applied to the subject

being studied but are ‘discovered’ and formulated in working with the field and the

empirical data to be found in it” (Flick, 2009, p. 91).

The model of the process in grounded theory includes theoretical sampling,

grounded theory coding, and writing the theory (Flick, 2009). In later sections in

this chapter, it will become clear that I am not adopting the grounded theory process

in any of its more purist forms. The reflexivity of the researcher and the research is a

hallmark of qualitative research.

“Unlike quantitative research, qualitative methods take the researcher’s

communication with the field and its members as an explicit part of

knowledge instead of deeming it an intervening variable. The subjectivity of

the researcher and of those being studied becomes part of the research process.

Researchers’ reflections on their actions and observations in the field, their

impressions, irritations, feelings, and so on become data in their own right,

forming part of the interpretation” (Flick, 2009, p. 16).

3.3.2 Qualitative versus quantitative research

In this research, I am not adopting a rigid approach. Of course, some academics have

occasionally argued that qualitative research is superior to quantitative research

(Oevermann, Allert, Konau & Krambeck, 1979; Kleining, 1982; McKinlay, 1995).

Below, my reasons for choosing qualitative over quantitative research are outlined,

but perceived superiority of qualitative research is not one of them.

  65  

(1) Originally, the focus was on decisions whether to use qualitative or quantitative

research was on philosophical and epistemological standpoints. Positivism as the

underlying epistemology of quantitative research was deemed as incompatible with

interpretivism, which is the foundation of qualitative research.

(2) However, I choose qualitative research largely for practical reasons. All three

major MOOC platforms (Coursera, edX and Udacity) collect a wealth of data on

students’ interactions and outcomes (Sandeen, 2013), and notably, researchers from

Harvard and MIT linked to edX have started to publish their analyses of the data (Ho

et al., 2014). Consequently, the main MOOC providers can be expected to continue

publishing solid quantitative research. Moreover, it may have been challenging to

convince MOOC providers to provide access to their data, as it would be

understandable that their own research departments regard them as sensitive,

proprietary, and the main source for their own publications.

(3) Also, a so-called mixed method approach (referring to a combination of

qualitative and quantitative methods) that has become quite popular in the past

decades, could have been considered. The two different sets of methods “should be

viewed as complementary rather than as rival camps” (Jick, 1983, p. 135). However,

Flick (2009) observes that developing “really integrated qualitative/quantitative

methods of data collection or data analysis remains an unsolved problem” (p. 30). In

any event, for a humble research project like the present one, it did not seem to be

advisable to mix qualitative and quantitative methods, as it would have increased the

complexity to an unmanageable extent.

(4) Flick (2009, pp. 32-33) suggests that a “decision for or against qualitative or for or

against quantitative methods… should be determined by the appropriateness of the

method for the issue under study and the research question”. The problem statement

and the research questions of this research project are about the perspectives of

academic experts with varying exposures to MOOCs. Consequently, qualitative

research is more appropriate than quantitative research for this kind of study.

  66  

3.3.3 A circular model of the qualitative research process and theory

A qualitative model of research process and theory must not be mixed up with the

linear model of quantitative research (theory, hypotheses, operationalisation,

sampling, collecting data, interpreting data, validation). The circular interlinking of

empirical steps (see Figure 3.2) is more appropriate for the discovery process and is

sensitive to the epistemological principle of verstehen (Flick, 2009). The circularity of

the qualitative research process forces the researcher to constantly reflect on the

whole research process. The research process continuously constructs reality. “The

central part reserved for the interpretation of data (compared with their collection or

the a priori construction of elaborated designs) takes into account the fact that text is

the actual empirical material and the ultimate basis for developing the theory” (Flick,

2009, p. 94).

Figure 3.2 Linear versus circular models of process and theory

Source: adapted from Flick, 2009, p. 95.

  67  

3.4 The six steps of the interview process

There are six steps with regard to interviews (Gaskell, 2000) that will be

reconstructed in this section. After (1) linking the research issue and interview

questions, (2) the choice of the convergent expert interview as a method is justified.

(3) The sample frame of the eight expert interviewees is discussed, before (4)

conducting, (5) transcribing, (6) coding and interpreting interviews are explored in

theory and application.

3.4.1 Links between research issue and interview questions

The first step of the preparation of a topic guide, or as briefly mentioned above, a

set of data collection questions (that are closely related to the guiding questions and

the general research question that were spelt out in section 1.2). In the previous

section (3.3), I highlighted the circular approach to the qualitative research process

that also applies to my interview questions. I emailed all questions to my

interviewees, but explained to them that we would not necessarily have to go through

all the questions and could also discuss other things that were of interest to them and

they thought to be of importance.

In Tables 3.2 to 3.5, I disclose the complete list of initial interview questions. In the

rightmost column in each table, the numbers refer to the initial interview questions

and indicate the eventual focus which was determined via the perspectives of the

expert interviewees. Bracketed numbers indicate issues that were not deemed as

important as issues with numbers without brackets, and omitted numbers showed a

relative lack of interest by the experts. The perspectives of the experts in turn had a

great influence on the final structure of the literature review in chapter 2 that is largely

aligned with the structure of the empirical analysis in chapter 4.

  68  

Table 3.2 Constructive alignment of research question 1 and interview questions

Research Question Initial Interview Questions Final

focus

(1) What, according to experts, constitutes a high quality of educational experience in the context of the MOOCs?

(1) Who are MOOC participants / students?

(2) Why do these participants / students enroll in these courses?

(3) What are MOOC participants’ persistence rates (drop out / completion rates)

(4) What skills and knowledge are needed to be successful in a MOOC?

(5) Who are the facilitators?

(6) What kind of training do these facilitators or tutors receive?

(7) What are the characteristics of effective facilitators or tutors?

(8) How do MOOC students perform compared to traditional students enrolled in the same course?

(9) Are MOOCs effective for all types of learners?

(10) How do instructors ensure quality learning experiences in MOOCs with multiple facilitators and thousands of students?

(11) What type of content is best taught in a MOOC, if any?

(12) Is the assessment valid and reliable (issue of cheating)?

(13) Accreditation and Employer Recognition

1-3, 4, (8), (9), (10), 11, 12, 13

Source: self-developed.

  69  

Table 3.3 Constructive alignment of research question 2 and interview questions

Research Questions Initial Interview Questions Final focus

(2) To what extent are MOOCs sustainable (in the perspectives of experts)?

(14) Who is paying for MOOCs (their instructors, platform, facilitators, and tutors)?

(15) How much do MOOCs really cost?

(16) What is the ROI for institutions providing MOOCs?

(17) For MOOCs offering certificates, how will they be received by employers?

(18) What are the major differences and similarities between MOOCs and traditional, credit-bearing online coursers offered in degree and certificate programs?

(19) When is a MOOC too big?

(20) What incentives are there for instructors to teach MOOCs?

(21) Do MOOCs take more time / effort for instructors to teach and students to learn?

14-15, (16), (17), (18), (20), (21)

Source: self-developed.

I considered Research Question 2 right from the start as the least important question,

as this thesis is an Education Research Project rather than a Business School project.

However, as higher education is a big business, I still decided to retain this question.

Interestingly, during the interviewees, the additional theme of MOOCs as a marketing

device emerged and was hence included.

  70  

Table 3.4 Constructive alignment of research question 3 and interview questions

Research Questions Initial Interview Questions Final focus

(3) To what extent do the MOOCs offer a viable alternative to traditional higher education, in the perspectives of experts?

(22) Or are the benefits of the MOOCs illusory, as they in reality harbour undesirable and inappropriate behaviours?

(23) How about the issue of access?

(22), 23

Source: self-developed.

The focus of the experts was on the research question, and the issue of access was

explored together with the issue of the bottom of the pyramid (see Table 3.5).

Table 3.5 Constructive alignment of research question 4 and interview questions

Research Questions Initial Interview Questions Final Focus

(4) In the perspectives of experts, to what extent, if at all, are MOOCs revolutionising higher education, a disruptive innovation and/or are opening up opportunities for the base of the income pyramid?

(24) Are MOOCs revolutionising higher ed? Are they a tipping point for higher ed?

(25) Will MOOCs lead to unbundling and rebundling?

(26) Or will MOOCs force many Higher Education players to radically transform themselves, or die, and a chaotic rout of the sector is in prospect (The Maturing of the MOOC, 2013)?

24, 25, 26

Source: self-developed.

Table 3.6 shows how chapters 2 and 4 are related to the research questions and the

above-mentioned 26 initial interview questions.

  71  

Table 3.6 Constructive alignment of sections in chapters 2 and 4 with research

questions and interview questions

Chapter 2

sections

Chapter 4

sections

Research

Questions

Interview questions

2.3 4.3 1 1-3, 4, (8), (9), (10), 11, 12, 13, (18)

2.4 4.4 2 14-15, (16), (17), (20), (21)

2.5 4.5 3 (22)

2.6 4.6 4 23, 24, 25, 26

Source: self-developed.

3.4.2 Choice of interview method: convergent expert interview

A second step is to choose a method of interviewing. It has long been advised that in

interviews, (1) the interviewer should not take anything for granted, should probe

assiduously for more detail than the interviewee may offer as a first reply to a

question, and (3) it is in the accumulation of insights from a set of interviews that one

comes to understand the life worlds within a group of respondents (Becker and Geer,

1957).

When compared with a survey, in an interview the researcher does not lead the

inquiry with a set of predetermined questions. While the broad content of the

interview is structured by research questions (and a topic guide), the idea is not to ask

a set of standard questions (Gaskell, 2000). However, as I was initially unsure

whether I would have to conduct some of my expert interviews via email, I still

prepared a set of data collection questions (O’Donoghue, 2007). Questions are

invitations to talk at length in the respondents’ own terms, and the researcher can

obtain clarification and amplification of interesting points (Gaskell, 2000).

  72  

Expert interviews are a sub-type of the semi-structured interview (Flick, 2009).

Generally, experts are persons who are particularly competent as authorities on

certain facts (Deeke, 1995). Expert interviews can be a stand-alone or a

complementary method. For the purpose of this research, expert interviews (in

combination with my first-hand experience of some MOOCs) are deemed sufficient

as the almost exclusive, if not a stand-alone, method (see section 3.5).

With MOOCs being a global phenomenon, sampling decisions (see section 3.4.3) also

influence the type of interview. Eventually, interviewees were spread across three

continents. Generally, face-to-face contact and the personal relationship, based on

verbal and non-verbal communications, are considered strengths of interview-based

qualitative research (Flick, 2009).

Online interviews are often discussed in the literature as synchronously occurring in

chat rooms, or in asynchronous form via email exchanges (Flick, 2009). In a physical

space, semi-structured interviews are usually conducted in one meeting with the

interviewee (based on an interview guide). If such a set of questions were to be sent

via email, this would come “closer to the situation of sending out a questionnaire in a

survey than to the situation of a semi-structured interview” (Flick, 2009, p. 267).

Face-to-face interviews also tend to be more spontaneous as compared to email

interviews. However, there are not only disadvantages to online interviews, as

especially asynchronous interviews may allow interviewees more time for reflection,

and the data are already available as texts (rather than requiring transcription – Flick,

2009).

I was initially considering email interviews, as I was worried about the logistical

aspects of recording interviews conducted via Skype. The time difference between my

interviewees and myself was occasionally 12 to 15 hours (due to different time zones

in Canada), and that usually led to slightly awkward hours from my perspective for

the interview, such as midnight. Also, transcribing approximately eight hours of

interviews verbatim is time-consuming (in my experience, the length of the interview

should be multiplied by eight, including writing and checking).

  73  

However, I am convinced that not even the most enthusiastic expert interviewee

would have written 30 pages (which is the length of some of my interviews) via

email, not even if I had broken up the email interview into a series of email

exchanges, only asking one or two questions at any one time (Flick, 2009). Also, the

paralinguistic elements of communication, unlike in emails, do not get lost, as I could

hear the participants clearly – in two out of six cases, the video function in Skype was

turned off to have a better sound quality, but normally I could also see at least the

facial expressions of the interviewees.

In conducting online interviews, the experience of, and attitude towards, using

technology should be considered (Mann & Stewart, 2000). In our context, in the one

interview that I conducted from Skype to a traditional telephone, it was as simple as

picking up a telephone. And in all other cases, the experts were very comfortable

using Skype technology, with some of them even sending me hyperlinks during the

interview.

It is a common question to ask how many interviews are needed to make a scientific

statement. However, this means applying a quantitative logic to qualitative research

(Flick, 2009, p. 31). A key idea of convergent interviews is to aspire to achieve

convergence. Once convergence has been achieved, there is no longer a need for

further interviews. Table 3.7 exemplifies how convergence in the convergent expert

interviews is achieved. Themes of the table are constructed from progressive content

analysis – usually three to five ‘chunks’ of ideas – these are the research issues.

  74  

Table 3.7 Convergence in convergent expert interviews

Interviewee / Issue

1 2 3 4 5 6

A Yes Raise - - - -

B Agree Disagree Raise Raise - -

C Agree Disagree Agree Agree Raise Agree

D Agree Disagree Agree Agree Agree Agree

Sources: adapted from Carson, Gilmore & Perry, 2001; Rao and Perry, 2007.

3.4.3 Sample frame – discussion of selection of experts

A third step is the selection of participants. Sampling decisions are very important in

qualitative research and actually occur at three different stages of the research, as

Table 3.8 shows.

Table 3.8 Sampling decisions in the research process

Stage in research Sampling methods

While collecting data Case sampling

Sampling groups of cases

While interpreting data Material sampling

Sampling within the material

While presenting the findings Presentational sampling

Source: Flick, 2009, p. 115.

In this section, I focus on the data collection stage. Theoretical sampling was used

with an initially estimated sample size of approximately five to ten participants. What

in the convergent expert interview is expressed as convergence is expressed in

grounded theory as theoretical saturation. It occurs when “further coding,

  75  

enrichment of categories, and so on no longer provide or promise new knowledge”

(Flick, 2009, p. 313).

There are different qualitative approaches to sampling. Despite severe time

constraints faced by the researcher, a mere convenience sample was to be avoided.

Theoretical sampling is closely linked to grounded theory and aids in its discovery.

It is defined as “the process of data collection for generating theory whereby the

analyst jointly collects, codes and analyses… data and decides what data to collect

next and where to find them” (Glaser & Strauss, 1967, p. 45). Thus, sampling

decisions (in my case, which expert should I interview next?) are made during, and as

a result of, data collection and analysis (Flick, 2009). Within the constraints of this

project, theory development would have constituted an excessive burden. Hence, a

thick description is the goal here, not grounded theory.

However, many other aspects of theoretical sampling are applicable to this project, as

the sample size is not defined in advance and sampling is finished when theoretical

saturation has been reached (Flick, 2009). In theoretical sampling, cases, amongst

many other things, can refer to individuals, or in the context of my research, experts.

Sampling proceeds according to the relevance of a case, not its representativeness

(Flick, 2009). Sampling decisions fluctuate between the usually mutually exclusive

aims of maximal width or depth (Flick, 2009), my focus was on cases that promised

to be rich in relevant information.

A preliminary list of experts was established and they were contacted via email. Other

pioneers of free high-quality online education as well as general higher education

experts were considered, and to some extent, the snowballing technique, which is

appropriate for small, specialised populations, was used (Rao and Perry, 2007). At the

beginning of my research, I was rather worried that none of the high-powered experts

would wish to talk to me. I had several alternative research strategies in mind,

including participant observation in MOOCs (which I did to a lesser extent as a

secondary method – see section 3.5) and content or discourse analysis of existing

interviews and other artefacts such as YouTube videos. I also thought hard about

experts whom I already knew, and decided, also in order to gain confidence, to

interview them first, before approaching other experts.

  76  

I was positively surprised that most requests for an interview were fulfilled. Only two

experts did not respond at all, one being one of the co-founders of one of the

xMOOCs and the other being a business school professor whose MOOC I had

sampled. Two co-founders of the same xMOOC wrote polite rejection notes, and the

President of another xMOOC recommended his Director of Research (who

regrettably did not respond). I also wrote to one of the world’s top management gurus

whose secretary responded most politely, and to the author of an excellent book on

openness in education who was kind enough to send me various chapters of his new

book (but was too busy for an interview). Table 3.9 summarises the responses.

Table 3.9 Research sample

Research sample Number Countries of origin Occupations Face-to-face interview conducted

2 US, Singapore Chief Learning Officer, Kaplan USA; Senior Lecturer, Republic Polytechnic Singapore

Skype interview conducted

6 Australia (2), Canada (4)

two Australian Professors, three Canadian academics who were all involved in the first MOOC ever; one highly distinguished Canadian open learning expert

Failed to return emails

2 US See above discussion

Expressed regret at being too busy

5 US See above discussion

Source: self-developed.

Generally, when interviewing experts, it needs to be considered that they are very

busy people. As a result, I asked for an hour of their time and kept to the time frame,

with the exception of three cases – the interview with Dr Saxberg was cut short to

approximately 25 minutes because of his demanding meeting schedule, and the

interviews with Professor Jackson and Dr Tan lasted approximately 90 minutes, as

they were in no hurry to end the interview.

  77  

3.4.4 Introduction of the eight expert interviewees

This section introduces the eight experts in chronological order of interviews

conducted.

(1) Dr Bror Saxberg

Source: Saxberg, 2014.

Dr Bror Saxberg is the Chief Learning Officer of Kaplan Inc., USA, where he is

responsible for the research and development of innovative learning strategies,

technologies and products across Kaplan's range of educational offerings as well as

the overall academic standards (Kaplan Leadership, 2014). Dr Saxberg had previously

served in senior management positions for K12, Inc., Knowledge Universe, and DK

Multimedia. His educational qualifications include a B.A. in Mathematics and a B.S.

in Electrical Engineering from the University of Washington; an M.A. in Mathematics

from Oxford University; a Ph.D. in Electrical Engineering and Computer Science

from M.I.T.; and an M.D. from Harvard Medical School (Kaplan Leadership, 2014).

  78  

(2) Professor James Guy Jackson

Source: Jackson, 2014.

Professor James Guy Jackson is Emeritus Professor of Southern Cross University

(SCU). He was the foundation dean of the Law Faculty at SCU, Chair of the

Academic Board and Professor of Law. His immediate past responsibilities were as

VP Academic, Kaplan Asia Pacific. He has also held positions at Darling Downs

Institute of Advanced Education (DDIAE), and at Wollongong and Bond Universities

(Jackson, 2014). Jim Jackson is a trained AUQA/TEQSA auditor and was on the first

TEQSA panel to review an Australian university, and he has written widely on higher

education law and other fields of law. His academic qualifications include BCom

(UNSW); LLB (UNSW); LLM (Hons) (Sydney); Grad Dip in Education (Tertiary);

and PhD (Sydney) (Jackson, 2014).

(3) “Professor A”

Professor A is a Professor of Education at an Australian University and an expert on

educational technologies and authentic learning. Due to the request for anonymity, I

do not disclose more information on Professor A.

  79  

(4) Dr Samson Tan

Source: Republic Polytechnic, 2014.

Dr Samson Tan is an Assistant Director (Academic) at Republic Polytechnic,

Singapore. With more than 16 years of experience in teaching and training both

youths and adults, Dr Tan is presently involved in e-learning and game-based learning

projects and engages in research on education reform and teaching with technology

(Republic Polytechnic, 2014). He holds a Bachelor of Science in Physics from

National University of Singapore; a Postgraduate Diploma in Education from

National Technological University, Singapore; a Master of Business Administration

from the University of Hull, UK; and an Ed.D. from the University of Western

Australia in 2007 (Tan, 2014).  

(5) Mr Stephen Downes

Source: Downes, 2014.

Stephen Downes is Program Manager, Learning and Performance Support Systems,

for the National Research Council Canada. In 2008, George Siemens and he designed

  80  

and taught the first MOOC. Stephen Downes was the winner of the Edublog Award

for Best Individual Blog in 2005 for his blog OLDaily and is Editor at Large of the

International Journal of Instructional Technology and Distance Learning. Associated

with many key developments in learning technology, Mr Downes, in the early 1990s,

developed Canada's first academic MUD (Multi-User Dimension); appears to have

been the world’s first 'edublogger' and is a central figure in the development of

connectivism, a network-based learning theory (Downes, 2014). He is also the author

of 'gRSShopper', an open source application designed to replace Learning

Management Systems. Stephen Downes holds BA (Honours, First Class) and MA

degrees from the University of Calgary (Downes, 2014).

(6) Sir John Daniel

Source: Daniel, 2014.

Sir John Daniel is a well-known thought leader on education. Amongst many other

leadership appointments, he was Vice-Chancellor of The Open University (1990-

2001); Assistant Director-General for Education of UNESCO (2001-04); and

President of the Commonwealth of Learning (2004-12). “His non-executive

appointments have included the presidencies of the International Council for Open

and Distance Education, the Canadian Association for Distance Education and the

Canadian Society for the Study of Higher Education” (Daniel, 2014). He has

published an incredible 350 publications and has been awarded 32 honorary degrees

from universities in 17 countries. “The three countries where he has lived and worked

  81  

have each recognised his contributions with national honours: France – Ordre des

Palmes Académiques (Chevalier – 1986; Officier–1991); United Kingdom – Knight

Bachelor (1994); Canada – Order of Canada (Officer – 2013)” (Daniel, 2014). He

pursued his university studies in Metallurgy at the universities of Oxford and Paris, as

well as a Master’s degree in Educational Technology at Concordia University

(Daniel, 2014).

(7) Mr Dave Cormier

Source: Cormier, 2014a.

Dave Cormier coined the term “MOOC” and was also was involved in conducting the

first MOOC, CCK08. He is a web projects lead at the University of Prince Edward

Island, Canada, co-founder and president of Edtechtalk, and president of Edactive

Technologies, a social software consulting firm (Cormier & Siemens, 2010). He holds

a Masters degree in Educational Literacies from Mount Saint Vincent University,

Nova Scotia, Canada (Cormier, 2014a).

  82  

(8) Professor George Siemens

Source: Penn State, 2014.

Professor George Siemens is the executive director of the Learning Innovation and

Networked Knowledge Research Lab (LINK Lab) at the University of Texas at

Arlington. Amongst other academic positions, he was professor at the Center for

Distance Education. In 2008, Siemens and Downes designed and taught the first

MOOC, and Siemens has since offered additional MOOCs that have gained popular

worldwide attention. Professor Siemens is the originator of Connectivism theory. He

holds a PhD from the University of Aberdeen, UK.

3.4.5 Conducting interviews

Six of the eight interviews were conducted via Skype (one Skype-to-phone and five

pure Skype conversations), and I recorded them with my iPhone 4 which produced a

decent sound quality. The two face-to-face interviews – which were recorded in

Kaplan Singapore’s conference room (in the case of Dr Saxberg’s interview) and in a

meeting room at Republic Polytechnic (in the case of Dr Tan’s interview) – were

recorded on the same device. Using such a machine for recording renders the

documentation independent of both the researcher’s as well as the interviewees’

perspectives and achieves a naturalistic recording (Flick, 2009).

  83  

None of my interviewees was averse to the interview being recorded, and in my

experience, interviewees quickly forgot about the recording, and the conversation

took place naturally. Visual recording was not deemed necessary, as the content of the

interview was the focus, and video recording can also be considered obtrusive,

reminding the interviewee constantly that s/he is being recorded.

3.4.6 Transcribing interviews

After data have been recorded on a technical device, their transcription is a fifth step

on the way to their interpretation. Some transcriptions focus so much on natural

science ideals of precision in measurement that Flick (2009, p. 299) has observed

them entering interpretive social science through the backdoor, and “the formulation

of rules for transcription may tempt one into some kind of fetishism that no longer

bears any reasonable relation to the question and the products of the research”. Overly

precise transcriptions of data absorb much time and energy that could be invested

more meaningfully in their interpretation. For the presentation of the transcripts, I

used Bruce’s (1992) criteria of manageability (for the transcriber), readability and

interpretability (for the analyst). While initially, paralinguistic utterances were

faithfully transcribed, I deleted most of them during coding because they did not add

value and also in the interest of readability of chapter 4. My omissions are

characteristed by “(…)”, emphases by interviewees are in bold, and paralinguistic

aspects such as laughter are put into rectangular brackets.

As stated in the Acknowledgements, my former degree student and her friend

thankfully undertook the arduous task of verbatim transcribing approximately eight

hours of interviews. As is a central feature of the procedure of transcription (Flick,

2009), I did a second check by going through all transcripts carefully, while listening

to the interviews.

  84  

3.4.7 Coding and interpreting interviews

The interpretation of data is at the core of qualitative research and also of grounded

theory research and is the final step of the interviewing process. However, in

grounded theory, both data collection and coding are considered as inextricably

intertwined with the interpretation of data (Flick, 2009). There are different

approaches to coding which I will briefly discuss before I justify the chosen process.

Coding is “the operation by which data are broken down, conceptualized, and put

back together in new ways” (Flick, 2009, p. 307).

Grounded theory coding is the central process by which grounded theories are built

from data through a process of abstraction (Flick, 2009). In Strauss & Corbin’s (2008)

approach, the first step is open coding during which concepts are identified and

developed, resulting in a list of codes and categories that are attached to the interview

transcript. Categories are more abstract than codes, but linked to them. The categories

that were identified in open coding are then further refined and differentiated in axial

codingwhich is the second step. Axial coding, very much like open coding, is

accomplished by making comparisons and asking questions, but axial coding links

subcategories to categories in a complex process that involves both inductive and

deductive thinking (Strauss & Corbin, 2008). The third step, selective coding, leads

to higher abstraction and focuses on potential core concepts. There are various

approaches to grounded theory coding (e.g. Strauss & Corbin, 2008; Glaser, 1992;

Charmaz, 2006), but they have in common that open coding is an important step; that

they base coding and analysis on a constant comparison of the empirical materials;

and that they view theoretical saturation as the end point of coding (Flick, 2009).

Qualitative content analysis is an alternative to grounded theory coding. In content

analysis, “categories are brought to the empirical material and not necessarily

developed from it” (Flick, 2009, 323). However, the categories are repeatedly

assessed against the interviews and modified where necessary. In contrast to grounded

theory coding or other approaches such as thematic coding, the goal of content

analysis is to reduce the material (Flick, 2009). Mayring (1997) proposes a detailed

procedure of qualitative content analysis (the co-called “General Content Analytic

Process Model”), beginning with the following four steps: (1) selection of material

  85  

from the interviews that is relevant for answering the research questions; (2) analysis

of context of data generation; (3) formal characterisation of the material; and (4)

definition of direction of analysis.

The fifth step is the theoretical differentiation of the research questions. The “research

questions of the analysis must be clearly defined in advance, must be linked

theoretically to earlier research on the issue and generally has to be differentiated in

sub-questions” (Mayring, 1997, p. 47). A sixth step, the definition of the analytical

units, is followed by a seventh that involves three types of content analysis: summary,

explication, and structuration

In summarising content analysis, the material is paraphrased. Thus, less relevant

passages as well as paraphrases with the same meanings are omitted (first reduction)

and similar paraphrases are bundled and summarized (second reduction) (Flick,

2009). In addition to summarising content analysis, there is also explicative content

analysis – which “clarifies diffuse, ambiguous, or contradictory passages by

involving context material in the analysis” (Flick, 2009, p. 327) – and structuring

content analysis, which searches for formal structures in the material (Flick, 2009).

The three concluding steps of Mayring’s (1997) approach are (8) the reassessment of

the category system against theory and material; (9) the interpretation of the results

according to the main research questions; and finally, the application of quality

criteria such as validity (see Figure 3.3).

  86  

Figure 3.3 Mayring’s process analysis of general content analysis

Source: adopted from Flick, 2009, and Mayring, 1983.

There are other approaches like Legewie’s (1994) global analysis. This one is less

complex than some of the others, starting with (1) a clarification of the researcher’s

background knowledge and of the research question, (2) coding the text via keywords,

refining the structure by marking central concepts or statements, (3) producing a table

of contents and alphabetically-ordered themes, (4) summarizing the text from the

viewpoints of the participants and (5) considering keywords for the entire text – all

this prior to the actual interpretation of the text.

My own approach is very much in the methodological trend of hybridisation. It is a

mix of some aspects of grounded theory coding, qualitative content analysis and

global analysis. For expert interviews, it is important that the content of the expert

knowledge is analysed and compared, and I found coding from the literature (see

chapter 2) as well as from the expert interviews useful for that purpose.

  87  

3.5 Participant observation in cyberspace as a secondary method

In the history of qualitative research, observation has been a popular and much-

discussed method (Flick, 2009). Denzin defines participant observation as a “field

strategy that simultaneously combines document analysis, interviewing of

respondents and informants, direct participation and observation, and introspection”

(1989, pp. 157-158). I use the term in a narrower sense, as I kept interviewing and

observation apart. While interviews and narratives make the accounts about practices

accessible, the practices themselves are only accessible through observation; and

observation may enable the researcher to find out how something actually works

(Flick, 2009). Observational methods can be classified along five dimensions: (1)

covert versus overt, (2) non-participant versus participant, (3) systematic versus

unsystematic, (4) natural versus artificial situations, and (5) self-observation versus

observing others (Flick, 2009).

(1) Strictly speaking, my research can be considered as covert, as I did not reveal that

I observed the instructors as well as the participants. Ethical implications of this

approach are discussed in section 3.7. (2) I practiced participant observation, as I

certainly became a – very minor (amongst literally thousands of participants) – active

part of the observed field. Applying Gold’s (1958) typology of participant roles to my

research, it can be said that I was “the complete participant”. (3) The observation

undertaken would fall more into the “unsystematic” category, as I remained flexible

and responsive to the processes. (4) In qualitative research, data are generally

collected in natural situations. The MOOCs that I observed certainly also fall into

that category. (5) Finally, while I observed the professors who taught the various

MOOCs and also other participants’ behaviour, the reflections that I share within this

project are largely based on my own reflexive self-observation.

Participant observation was a minor method in my research, and the observations that

I am using for the purpose of this project, were also reflected in half of my interviews.

My approach to observation was thus not highly systematic – it was more of an

afterthought, as it seemed ludicrous to write about a topic that I had not experienced

first-hand.

  88  

Various phases of observation have been discerned. For the purposes in the context of

this project, only three appear relevant: (1) the selection of a setting, (2) initial and

general descriptive observations, and (3) focused observations that concentrate on

aspects that are relevant to the research question (Flick, 2009). Settings selected

included courses on topics where I have prior knowledge such as business strategy

and philosophy, and I selected different providers, namely Coursera and EdX, thus

choosing both for-profit and non-profit providers. I took some notes on things that I

had previously been unaware of or which struck me as interesting. While I was

interviewing experts, they highlighted certain aspects of MOOCs, which also gave me

the opportunity to reflect on some of my observations. In this research, the participant

observation obviously did not occur in a physical environment, but on the Internet

that has become a part of many people’s everyday life (Flick, 2009).

3.6 Assessment of the quality of qualitative research

Qualitative research is unsuitable for the quality standards of quantitative research,

and how to assess quality of qualitative research appears to be a rather vexing

problem (Flick, 2009). The interweaving of illustrative quotations (which will be

made use of plentiful in chapter 4) is labeled as selective plausibilisation and is not a

solution to the problem. There are many suggestions for quality assessment in

qualitative research, but in Flick’s view, “none of them solves the problem of

adequate quality assessment” (Flick, 2009, p. 398). Below, I discuss qualitative

approaches to reliability and validity and some approaches that are specific to

qualitative research.

Reliability appears to be partially applicable in the form of procedural reliability, as

“the quality of recording and documenting data becomes a central basis for assessing

their reliability and that of succeeding interpretations” (Flick, 2009, p. 386). Also, in

case there are several interviewers (which was not the case in this research project),

reliability could possibly increase via interview training. Generally, reliability could

be enhanced if it were possible to check the original statement of the subject (via

  89  

access to the full transcripts) vis-à-vis the researcher’s interpretations. The more

detailed the research process is documented, the higher the reliability (Flick, 2009).

Validity is another key concept against which the quality of quantitative research is

routinely assessed. From a constructivist perspective, validation is a social

construction of knowledge whereby “the production of the data becomes one starting

point for judging their validity; and their presentation of phenomena and of the

inferences drawn from them, becomes another one” (Flick, 2009, p. 388). Validation

has been reformulated as the social discourse through which the trustworthiness of the

interpretations is established (Mishler, 1990).

A particular approach to qualitative research validation is communicative validation.

While in some contexts, ethical problems may arise “in the confrontation of

interviewees with interpretations of their statements” (Flick, 2009, p. 399),

communicative validation may be particularly suitable for expert interviews. I did

this in several stages in my research. After the checking of the transcriptions, I

emailed them to the experts, and made sure that they continued to be fine being

quoted for the purposes of the project. One expert, Downes, edited the interview and

then published it on his website. Also, after I had completed a final draft of the thesis,

I emailed it to the eight experts and asked for feedback, with special reference to

whether I may have misunderstood any of their remarks in chapter 4. Again, this

provided some additional validation.

Generally, there is a shift from validity to validation and towards “increasing the

transparency of the research process as a whole” (Flick, 2009, p. 391). Flick (2009, p.

403) argues that there is “no one right method to use in qualitative research” and that

the quality of qualitative research cannot be assessed by applying criteria. Rather,

quality in qualitative research applies to the whole research process.

Triangulation is a specific approach to quality. Amongst other things, it refers to

methodological triangulation, and more specifically, may refer to using two

qualitative methods (Flick, 2009). I do not wish to overstate the aspect of my

research in which I conducted some participant observation while sampling MOOCs,

but especially if that aspect of my research had been extended, this would have been a

  90  

case of methodological triangulation. Flick (2009) argues that triangulation is

justifiable when its results are “different in focus and level, which means that they

complement each other, or even contradict at first sight” (p. 450). In my research, the

sampling of MOOCS, in addition to the expert interviews, was certainly fruitful as it

produced additional insights.

3.7 Ethical considerations

Although researchers normally face ethical issues in every stage of the research

process (Flick, 2009), ethical considerations did not feature prominently in my

research. For the main method of convergent expert interviews, I dealt with high-

powered experts who were often in the public limelight. Moreover, half of the

interviewees are prominent proponents of openness and were consequently more than

happy to be mentioned by name.

It was not necessary to anonymise most experts’ contributions, and it was also

unnecessary to keep them confidential. I of course offered anonymity and

confidentiality as options in line with the ethics application that was approved by the

University of Adelaide ethics committee (H-2014-064). Only one of the experts

(“Professor A”) requested anonymity, and only in terms of her not to be identified by

name with regard to direct references to the interview.

On the topic of anonymity and confidentiality, I had some refreshing responses during

the interview, for instance from Sir John: “I don’t do confidentially [interviewer

chuckles] and anonymity, (…) I’m an open education resource kind of guy

[interviewer chuckles], so what I say can be quoted wherever you like and I’ll be fine

with whatever, so don’t worry about any of that” (Sir John, interview). Professor

Siemens’s response to being offered anonymity and confidentiality at the beginning of

the interview was comparable to Sir John’s: “Sure, yeah, absolutely, there is no need

for anonymity in this interview as far as I’m concerned” (George Siemens, interview).

When I asked again via email, Professor Siemens’s emailed response was similar:

“Sure – go ahead and mention my name. I don't need anonymity/confidentiality!”

  91  

The usual consideration of informed consent (participants are informed of the

potential risks and benefits before giving their consent) was applied. To use a

Hokkien term, a kiasu (literally: ‘afraid to lose’) approach to informed consent was

employed, thus reconfirming the consent of the experts not only once, but at several

stages of the research. The permission to use the experts’ names was given by them

initially in writing. During the interview, I also asked for permission to use their

names. As an additional safeguard, I provided them with the transcripts of the

interviews. Again, I provided the options of anonymity and confidentiality. Also, the

project, at this stage, is not meant for publication, and the experts will be provided

with the final copies at the time of thesis submission. If publication wholly or in parts

were to be considered at a later stage, I would certainly ask again for permission.

Other applicable ethical principles include non-maleficence (the avoidance of

harming participants), non-invasion of privacy, non-deception about the research’s

aims and beneficence (there should be identifiable benefits of the research – Murphy

& Dingwall, 2001; Flick, 2009). Covert observation is ethically contestable (Flick,

2009). However, the observations that I share about professors teaching MOOCs and

participants’ behaviours within this project are largely based on my own reflexive

self-observation. No raw data were collected.

To some extent, my research may be conceived of as triangulated. This could have

raised additional ethical issues, if it had required participants to collaborate in more

than one way with the research. However, this was not the case, as the experts did not

participate in the MOOCs that I sampled. Thus, there were no relevant ethical issues

with the participant observation aspect of my research.

3.8 Conclusions

The chapter has outlined the qualitative research methodology and design. It

discussed the triangulation of convergent expert interviews and participant

observations and the meaning of quality in a research like this. Finally, the chapter

demonstrated that the ethical guidelines of University of Adelaide have been adhered

  92  

to in order to ensure that none of the participants was harmed and the data are

protected to the required extent.

  93  

4. Empirical analysis 4.1 Introduction

This section shares the results of my research. As detailed in chapter 3, the main

method was expert interviews, though I did some triangulation via participant

observation. The eight hours of expert interviews alone provide for much fascinating

material with delightful ‘digressions’, but due to space constraints, I am very focused

on the research model and the four research questions. Thus, the structure of this

chapter largely mirrors the literature review in chapter 2.

The key terms around the concept of MOOC were defined and discussed in section

1.6. As they are an important aspect of the research model (see section 2.7) and as the

experts (who had been introduced in section 3.4.4) had much to say about them, this

is reflected in the first part of section 4.2. Three of my interviewees (Siemens,

Downes and Cormier) pioneered the first MOOC, so they also contemplate on this

famous pioneering effort in section 4.2.2. Thereafter, sections 4.3 to 4.6 are structured

in a very similar way as sections 2.3 to 2.6 in the literature review, addressing the four

research questions, before the chapter is concluded.

4.2 MOOCs, cMOOCs, xMOOCs 4.2.1 The term MOOC and its components

Sir John referred me to a recurring slide in his publicly available PowerPoint

presentations (see Figure 4.1) stating that “every letter in the acronym is now

negotiable”.

  94  

Figure 4.1 What MOOC means

Source: Uvalić-Trumbić & Daniel, 2014.

As Figure 4.1 shows, the MOOC component terms mean different things to different

people, and massive is no exception. George Siemens defined “massive” as “anything

that exceeds Dunbar’s number of 150 folks in a social setting” (interview). He

referred to British anthropologist Dunbar’s theory that humans can only comfortably

maintain 150 stable relationships. “Once you get to a certain number of people, it

begins to fragment into small learning communities and that is a critical aspect of the

massive part” (George Siemens, interview).

Openness has created a change in the education industry over the last few years

(Stephen Downes, interview). At the same time, ‘open’ is an ambiguous and

controversial term, as it, amongst other things, refers to both open educational

resources (OER) and open admission (interviews with George Siemens and Sir John).

Professor Siemens observed the open education movement as long-term and

philosophical, and contrasted it with the short-term approach of xMOOCs.

“The open education movement, in my eyes, is a philosophical imperative for

society. (…) Through open education resources, open learning opportunities,

M O O C – Every letter is now negotiable!

  95  

we give students and society an opportunity to learn without barriers” (George

Siemens, interview).

The MOOC movement, in contrast, focused on “register[ing] for a course without

fee” as the key aspect of openness:

“So people don’t necessarily care about ‘is this creative commons-licensed or

is this openly licensed’? (…) They are more interested in ‘can I access that?’”

(George Siemens, interview).

Consequently, MOOCs may well destroy the open education movement.

“In a few years time, those opportunities for openness may not be there

anymore. (…) MOOCs may well destroy the open education movement

because people are settling for short-term access, but not considering the

longer term licensing schemes that will (…) allow for next generation

building” (George Siemens, interview).

Figure 4.2 The meaning of OER

Source: Daniel & Uvalić-­‐Trumbić,  2014.

OPEN EDUCATIONAL RESOURCES (OER)

educational materials that may be freely accessed, reused, modified and

shared

  96  

Sir John agreed that “most MOOCs are not, unfortunately, open educational

resources”, but used the example of a mobile technology MOOC which would qualify

as an OER (interview). Nonetheless, various experts highlighted that “unlocking the

academy” and providing open access to previously ‘locked-up’ knowledge was a

strength of MOOCs (Professor A, interview) and something important (Stephen

Downes, interview).

The online aspect was not considered controversial. It was perceived as providing the

opportunity for blended learning (George Siemens, interview).While Sir John had still

“difficulty calling them courses” (interview), Siemens and Downes considered the

coursepart important because there is a start and stop time to it (interviews). The

difficulty of the course concept is reflected in Figure 4.1 and will be further discussed

in the sections on assessment and certification (4.3.5 and 4.3.6).

Looking at the term MOOC as a whole, Professor Siemens concluded that it was a

“messed up”, “bad” and “useless”, but nonetheless “important” concept (interview).

“People like Dave [Cormier] (…) maybe don’t like to use the word. (…)

Dialoguing only with my peers, I agree that MOOCs are a bad word. (…) I’m

interested in dialoguing and influencing change outside of my peer base, (…)

and that’s where MOOC becomes an important word because it’s one of the

few educational ideas that has jumped outside of education (…) and made it to

mainstream media coverage” (George Siemens, interview).

Stephen Downes saw the significance of MOOCs as “open online learning” and

preferred that term to “MOOCs” (interview). Dave Cormier shared with me his

coinage of the term MOOC for which, as he said self-deprecatingly, “I’ve taken

significant grief over the years” (interview). The term originated in a conversation

with George Siemens.

“The funny thing is, none of us paid very much attention to it at the time. So

when people ask us for very close specifics we were like ‘I don’t know, we

were just talking about stuff’” (Dave Cormier, interview).

  97  

He shared with me his regret that the term “MOOC” has taken a life of its own and

that he prefers to use the term “open course” these days (Dave Cormier, interview).

4.2.2 The first MOOC ever

Dave Cormier told me the story of the pioneering MOOC on Connectivism and

Connected Knowledge (CCK08).

“In 2007, George Siemens put a conference on the future of education online.

(…) We (…) decided to do a couple of weeks before the online conference

and a couple of weeks after. We got a pretty good response, we’ve got lots of

people engaged and started talking about it afterwards, ‘well it seems (…) like

there’s something going on here’. And then Stephen [Downes] and George ran

into each other at some conference somewhere, and they decide to co-teach

George’s course at the University of Manitoba. (…) Then, they decided, well,

if they’re gonna do that, they might as well open it up for whoever else wanted

to come along. And I talked to them on and off.(…) We were trying to get our

minds around what it meant that 1,300 people will be doing something online

and producing stuff, creating stuff, and the possibilities (…). So if you think

about a course as something that creates rather than something [that]

transmits, then all of a sudden you got (…) a potential for moving something

(…) which I thought was really interesting” (Dave Cormier, interview).

Continuing with the story of CCK08, Dave Cormier narrated:

“They eventually had 2,400 people in that course. (…) I volunteered to

moderate [the] Friday session where I would speak for the students,

essentially, and moderate between them [Siemens and Downes] in the session

(…). So I ended up doing a facilitating session throughout that first course”

(interview).

  98  

Stephen Downes used the first MOOC ever (CCK08)as an opportunity to reflect on

quality in a MOOC:

“Quality is so much in the eye of the beholder. (…) We had around 2,200

people in CCK08, and we had therefore 2,200 different sets of metrics of

quality. (…) CCK08 really (…) worked for a certain number of them. It was

kind of interesting for a bunch of them, and was ‘whatever’ for probably

hundreds of them [interviewer laughs] (…). The quality of teaching (…) was

horrible [interviewer laughs]. (…) By any measure, we were terrible

instructors. We had no curriculum, we had no course objectives. (…) We did

scaffold practice, but not in any sort of content area” (Stephen Downes,

interview).

In terms of technology, gRSShopper, software developed by Stephen Downes was

used. In my conversation with Dave Cormier, continued experimentation emerged as

a theme in the further development of cMOOCs.

“My favourite one was the PLE [Personal Learning Environment] course. (…)

Then we did the Art Change 11 monstrosity, we did 36 weeks, it was nuts. It

was trying models like (…) ‘what happens if you do this?’ (…) ‘well, that’s

interesting’ (…), you got this abundance, you’ve got all these possibilities,

what happens if you explore?” (Dave Cormier, interview).

Dave Cormier also shared with me a recent course that he conducted during the time

of our interview:

“The course I’m running now is P2PU, and I have a 170 people in a Facebook

group, and I have a 150 in a Google+ group, I’ve got 3,000 tweets, and I’ve

got people all over the place and there’s no central discussion, there’s no

central content. Essentially, I present a challenge every week and then I poke

the communities here and there to keep them running” (Dave Cormier,

interview).

  99  

Stephen Downes used the metaphor of a town square to describe the connectivist

learning in MOOCs.

“[A] model would be (…) a town square, where you have a lot of people in the

same area, and there is an organisation apparent, but (…) there’s a lot of

interaction, and it looks like a group, but if you analyse the behaviour of each

person, each is acting independently and autonomously. (…) They have a

sense of what they are trying to do and what their goals are. There is no (…)

common or shared purpose that is intentionally followed by the individual

actors in the square. (…) A MOOC, (…) it’s a place where individuals

pursuing their own interest and objectives can interact in a way that mutually

supports each other” (Stephen Downes, interview).

Downes observed connectivist aspects in both cMOOCs and xMOOCs:

“There is a phenomenon of self-organising communities springing up around

some of the xMOOCs courses. (…) So to some degree, the connectivist aspect

is beginning to show itself even in the xMOOCs” (Stephen Downes,

interview).

4.3 The quality of the educational experience in MOOCs

What constitutes quality of the educational experience depends on the philosophical

orientation. The experts had traditional, learning science, authentic learning, problem-

based learning and connectivist perspectives – and even that is probably not a

comprehensive list.

For instance, Professor A argued from an authentic learning perspective: “You bring

your own authentic context, which is the reason why you want to learn something,

and you have the motivation to do it and the will to do it and the time to devote to it,

(…) to the MOOC” (interview).

  100  

To exemplify another perspective on quality, Sir John focused on quality assurance

not of the product, but the process:

“How do you assure the quality of open educational resources? Well, it’s

difficult because the whole point of an open educational resource is that you

can modify it and remix it and change it and distribute it and whatever. So the

moment you put an open educational resource out there, it will start to mutate

into other things. So, even if you have some benchmark of quality for the

original, that will soon be changed. So I think you’re forced back to (…)

ensuring the quality of the processes by which that particular artefact, open

educational resources, is produced rather than taking the product itself and

trying to do some kind of an assessment of it” (Sir John, interview).

However, Sir John conceded that to many people, quality equals brand.

“Many people simply equate quality with brand. So they say ‘when it comes

from Harvard, it must be good’. (…) That’s not true! [Interviewer chuckles]

Because Harvard has a high reputation for its research, but there’s nothing that

tells us Harvard is particularly good at online learning. (…) I can well

understand that the man in the street uses brand as a surrogate for quality”

(Sir John, interview).

Dave Cormier highlighted what in his perspective was an outdated educational

concept that was manifest in many xMOOCs and hence had mixed sentiments about

them:

“People’s idea of learning (…) is still very much (…) 19th century

transmission, [it] hasn’t changed a whole lot in 150 years. (…) The idea (…)

of learning is still trapped in the same T-Model that they were taught in, and

so it has been handed down to people. So I think those people have got hold of

a lot of the stuff that we initially worked on, and have managed to turn it into

something very, very boring. Except for one caveat: the idea that people are

  101  

taking the stuff that they know and putting it out there in whatever format for

other people to see, I think that is a positive step” (Dave Cormier, interview).

At the same time, Dave Cormier gave me the example of his favourite MOOC, Keith

Devlin’s “Introduction to mathematical thinking” course on Coursera. The high

quality of that xMOOC also blurred the distinctions between cMOOCs and xMOOCs.

Figure 4.3 Screenshot of Keith Devlin’s “Introduction to Mathematical

Thinking” Coursera MOOC

Source: Devlin, 2014.

“His MOOC may be the best MOOC on Coursera. (…) He has some really

(…) innovative ideas about how to run those things. (…) And this is where the

‘xMOOCs – cMOOCs thing’ doesn’t really make sense anymore. (…) He has

45,000 people taking that course, (…) it’s brilliant, (…) it’s pedagogically

extremely fantastic” (Dave Cormier, interview).

4.3.1 Who are the participants, what is their motivation and what do the completion

rates signify?

Several experts highlighted that MOOC participants thus far have been highly

educated. Stephen Downes commented: “Most people who take MOOCs are

university people. (…) University people like university courses, big surprise

[interviewer laughs]”. University faculty all over the world took MOOCs “to find out

  102  

what the fuss was about” (Sir John, interview), and Professor Jackson viewed

MOOCS as a “great resource”, an “advanced form of Wikipedia” and “a huge

knowledge base”.

“If I was (…) constructing another course in my discipline, (…) I’d be online

looking for every MOOC that I can find because I wanna see how they are

doing it as a teacher (…). I wanna know how they’re assessing things (…),

what clever things they’re doing” (Jim Jackson, interview).

However, demographic change was widely expected. For instance:

“Over time, (…) it will become more widely accessible and more widely

interesting to the population as a whole (…) That’s the big advantage, (…) it’s

still kind of out there” (Stephen Downes, interview).

Dave Cormier experienced his cMOOC participants as rather different from

xMOOC participants, observing that many of them were domain experts in their 40s

and 50s who were passionate about a subject matter.

“If I can’t teach a whole group of people who are passionate about something,

then I’m a moron. [Interviewer laughs.] That’s what cMOOCs are really good

for, bringing those people together, pushing the knowledge forward, that kind

of stuff” (Dave Cormier, interview).

Non-completion was normally not regarded as a big issue. George Siemens half-

jokingly related that he himself does not complete MOOCs he signs up for:

“I love taking MOOCs (…), but I don’t like finishing them [interviewer

laughs], (…) I enjoy getting into the range of ideas and that’s exciting”

(George Siemens, interview).

Sir John observed that “persistence rates are increasing, partly because you’re

getting far less tourists”. In a mobile technology MOOC, success rate was 30%,

  103  

which Sir John deemed “pretty respectable” (interview). Professor Jackson was rather

critical of high dropout rates and reasoned that they were the result “of the lack of

feedback and the lack of any form of personal attention; (…) also a lack of

commitment” (interview). When there was no financial investment on the part of the

participants, it was “easy come, easy go” (Jim Jackson, interview).

Sir John also pointed to the phenomenon of “recreational education”. One person

who “hugely enjoyed” a MOOC (Dino 101: Dinosaur Paleobiology, on Coursera) was

a retired deputy minister in the British Columbia government

“They had some absolutely top experts, and he was blown away by the

technology they use where you could use your mouse to put together a

dinosaur [interviewer laughs] from bones and so on” (Sir John, interview).

Figure 4.4 Dino 101: Dinosaur Paleobiology

Source: Currie & Arbour, 2014.

Sir John argued that “xMOOCs have been (…) provider-driven in the sense, ‘we

have a professor who is a world expert in dinosaurs, he would like to do a MOOC,

we’re sure there are people out there interested in dinosaurs, so let’s go’”

(interview). However, to Sir John, “a truer approach to education says (…) ‘where is

there a group that has a particular educational need, and can we do something about

that?’” (Sir John, interview).

  104  

Professor A made a distinction between formal and informal learning, with MOOCs

being a form of informal learning that could later lead to formal learning (interview).

Stephen Downes also shared with me his current work on distributed personal

learning networks:

“The idea here is to create the technology and the infrastructure that allows

one person to access multiple streams of learning, and to organise and manage

their own learning in their own personal learning environment. (…) MOOCs

is one stream of multiple streams into a personal learning environment”

(Stephen Downes, interview).

4.3.2 Required MOOC literacies

The discussion of MOOC literacies is related to the issue of access. All experts agree

that access to an Internet-enabled computer is insufficient to meaningfully participate

in a MOOC. Downes argued that a set of 21st century literacies is required “to derive

the maximum advantage from open online learning” (interview). For traditional

education, STEM (Science, Technology, Engineering, Mathematics) skills were

required. In contrast, MOOCs required “multiple literacies” (that included literacy,

capability and motivation) provided “a ramping up challenge” (Stephen Downes,

interview).

Cormier highlighted communication skills as a necessary literacy, thus doubting that

“MOOCs can do the transformative work that needs to happen in the Third World to

actually be helpful” (interview). “Classic literacies” (such as decision-making, active

engagement, and openness to enquiry) were the issue, not access to content. “The

mistake that we make is by giving people stuff” (Dave Cormier, interview). Only with

classic literacies would emerging markets develop more of their own MOOCs.

Siemens highlighted the familiarity with social media as a digital literacy that would

serve participants well in decentralised distributive spaces such as MOOCs.

  105  

“Many of us (…) have been involved in digital networks for a period of time

and (…) gone through the cycle of (…) Geocities or Friendster or MySpace

(…) technologies, and more recently Facebook and Twitter. If you are familiar

with that level of knowledge creation, (…) then (…) MOOCs can be very

effective for learning, because as a (…) learner (…), you are comfortable with

knowledge generation and socialisation in social systems” (George Siemens,

interview).

On the other hand, students only familiar with traditional higher education may find

especially “distributed MOOCs” disorienting. Students only familiar “with the (…)

faculty member producing content(…) will find that a difficult transition to learning

in these more chaotic spaces, where there’s all this noise and a lot peripheral

conversations” (George Siemens, interview). In Siemens’s expert opinion, MOOCs

did not serve “at-risk students” or participants from a “low socio-economic

background” well. “They may in the future, but it means we have to build in more

intervention” (George Siemens, interview). Professor Siemens concluded that “a very

motivated, confident, highly-skilled, digital learner will find it very easy to succeed in

MOOCs” (interview).

4.3.3 Expertise versus wisdom of the crowds

Dr Saxberg argued that MOOCs were likely to only be able to create mediocrity, as

the development of expertise required the coaching by experts.

“Whether it’s nursing or law, or business, or programming or accounting etc. –

if you want to become expert, you’re gonna have to have coaching by experts

somewhere along the way (…), except for maybe the .1%. (…) There’s no

way to become an actual, useful, valuable expert, with no feedback from

experts on what you do and how you communicate, and how you solve

complex problems and how you deal with grey areas” (Bror Saxberg,

interview).

  106  

Dr Saxberg also made an implicit argument against the ‘wisdom of the crowds’ and

against peer grading by stating that for complex issues, peers would get it wrong

without the assistance from experts. While “for things that are easy to learn, peers are

great”, “especially when things are hard, peers will reinforce each other’s

misconceptions” (Bror Saxberg, interview).

In Professor A’s perspective, expert performance was one of the elements of authentic

learning and “MOOCs are strong on the expert performance” (interview):

“To have (…) a renowned expert in the world to (…) create those lectures and

to provide (…) information (…) on the basis of their research and their

knowledge and their student’s research (…) is really probably one of the

strengths of a MOOC” (Professor A, interview).

At the same time, Professor A stressed that expert delivery in itself was woefully

insufficient.

“If you (…) had a choice between the world’s best (…) economist teaching

you and thousands of others – and you have no opportunity to interact with

that person, or they don’t take any interest in what you’re doing – (…) and

(…) your average kind of lecturer at a University, where you can call in and

talk to them in their office and (…) after the lecture, or you’re engaged in

activities within the classes that they teach (…), I think I know which one I’d

go for [laughs]” (Professor A, interview).

I suggested to Professor A that there may be an opportunity to have the best of both

worlds by listening to a Nobel prize winner within a MOOC and at the same time

attending the lecture of a mere mortal. Professor A countered: “But then we don’t

need MOOCs for that, do we? We only need YouTube [laughs]” (interview).

Professor A highlighted the opportunity for MOOCs to employ the wisdom of the

crowd.

  107  

“You (…) might be getting students (…) in biology (…) to (…) take note of

the animals that they find in their backyard. (…) You could use a database so

that they log all that information, and then you can get them analyzing (…)

massive amounts of data. (…) A few [of the MOOCs] do that sort of thing

where they (…) draw on the crowd” (Professor A, interview).

One element of authentic learning is providing multiple roles and perspectives. For

instance, many cooking sites on the web have a discussion space at the bottom of the

page:

“You’d get people saying ‘oh, when I cook that recipe, I left out the such and

such ingredient because I didn’t have any, but instead I used this and you

know, it’s much nicer’. So (…) you’re getting different, multiple perspectives

by getting that (…) commentary happening. And (…) MOOCs could be very

strong on that” (Professor A, interview).

At present, however, Professor A thought that many MOOCs underperformed in that

regard.

“A lot of the discussion forums are just so big, and so unwieldy. People just

give up on them. (…) Often people go off task” (Professor A, interview).

4.3.4 What courses or content may best be taught via MOOCs, if any

Several experts argued that fairly basic knowledge could be suitable for MOOCs. Dr

Saxberg envisaged that “especially in the sciences, (…) huge first-year lectures could

be replaced by MOOCs”, and faculty time could then be freed up “to really give very

close personal attention to the students in their second and third years” (interview).

Similarly, Professor A imagined that if you had “a really great lecturer, really

providing fantastic resources and (…) something that can be assessed [via] high-end

multiple choice questions”, this could provide students with a “basic understanding”

for applying that knowledge to problem-solving in a physical classroom environment.

  108  

Professor Jackson argued that online courses, in his experience, can essentially teach

and assess almost everything. He would, however, argue for the need of some offline

component, oral skills for law students being a case in point.

“I reckon you could use a MOOCs to just about teach anything, in terms of

delivery of content. (…) I can’t see why, in theory, MOOCs lend themselves

better to one discipline than another. I think it just turns to the ingenuity (…)

of the teacher” (Jim Jackson, interview).

Professor Siemens concurred that anything can be taught in a MOOC, and reflected

on CCK08 and subsequent developments:

“When we first taught our MOOCs, (…) we had a lot of criticism (…) ‘cause

we were talking about learning theory and knowledge. (…) I remember having

complaints (…): ‘well, sure that can kind of work for such a philosophical

course that’s discussion based, but there’s no way that you can do a MOOC in

computer science or (…) math’. In 2011, with (…) the Stanford MOOC, I

remember (…) journalists writing (…), ‘well, we can do computer science and

math courses in this MOOC format’ [interviewer laughs], ‘but you can’t teach

humanities in this format’” (George Siemens, interview).

Professor Siemens argued that in the end, instructional approaches and formats can be

adopted to any subject that is being offered via a MOOC.

4.3.5 Assessment

Most experts that I interviewed would agree with Dr Saxberg that “most” MOOCs

“don’t have good assessments yet” (interview) As we will see, some experts would

regard that as an understatement, while others would question the appropriateness of

current assessment regimes for 21st century knowledge economies.

  109  

Professor A shared with me her experience of attending a MOOC on cooking that

“encapsulated everything that I think is problematic with MOOCs” (interview).

Initially, she was enthusiastic due to her love for cooking.

“The (…) promise of the MOOC was that we would learn the science behind

cooking and each week, we would need to cook something in our own kitchen

which demonstrated a particular scientific principle. (…) And then we would

upload pictures of the results of what we did and we talk about it. (…) That

sounded really wonderful and a great way to learn in a really authentic kind of

context. (…) I was really happy with the first part of it because they (…)

promised to take us into (…) some of the best kitchens in the (…) restaurants

of the world. (…) We would be able to see how great chefs cook. (…) The

teachers (…) gave excellent little introductory talks” (Professor A, interview).

Professor A then provided the example of how to best cook a steak on a barbecue.

“Everybody thinks that you put the steak on and you cook it really high to start

with, and you turn it over to seal in the juices, and they said that ‘this is a huge

(…) no-no, (…) and in fact, that causes (…) the steak to toughen’. (…) They

(…) made this point many, many times that (…) instead of that, you’d have to

cook slowly, and then you’d have a really nice tender piece of meat. (…) They

(…) never at any point said (…): what do you actually do! – which I thought

would have been a great introduction, (…) Instead of that, we needed to wait

to learn that later” (interview).

  110  

The perceived verbosity may be a teaching deficiency, but I am largely providing

Professor A’s story to illustrate her view that “assessment is the Achilles heel (…) of

MOOCs”.

“They were also talking about pressure cookers and they (…) said that it was

invented in the sixteenth or seventeenth century (…). And they talked about

how (…) amazing an invention it was. (…) How (…) people just (…) could

not believe, (…) you could eat the bones, you could cut them with (…) a fork

or a spoon. (…) And then we had to do a little quiz (…) to work out our

understanding so far. And the very first question was: ‘In what year was the

pressure cooker invented?’ [Both laugh.] And (…) I just could not believe it.

Because (…) there was so much great, (…) really captivating stuff to start

with. And (…) we were expected to recall something, which was of minimal

importance (…) in our learning. (…) Why was it important for us to know that

the pressure cooker was invented in 1671 or whenever it was [both laugh]. So

(…) I think that encapsulated for me, what is wrong with MOOCs. (…)I just

think people haven’t really found really good ways to assess (…) at such a

large scale” (Professor A, interview).

While it does not really matter to most students, whether the pressure cooker was

invented in 1671, 1679 or 1864, MOOCs should be more concerned about assessment

and a student-centric approach in an authentic learning perspective.

“When we talk about authentic learning particularly in relation to technology,

(…) it’s what the students do that matters the most. It’s not so much what the

teacher does. (…) I think when you look at the majority of MOOCs, (…) what

the students are doing is pretty low-level activity. (…) Perhaps reviewing

other people’s work, that’s a bit higher level. But in authentic learning, they

would be creating polished real and useable products that really require a lot

of effort in collaboration with others (…). It’s creating something that gives

them the thinking processes (…) that they will need (…) [to] complete that

activity (…) or perform that role in the real world. (…) So (…) MOOCs are a

bit light on when it comes to what the students do” (Professor A, interview).

  111  

Professor Jackson was at least as critical as Professor A with regard to assessment in

MOOCs, considering assessment in MOOCs a fundamental issue, as “it drives

everything else”:

“You have to be certain that you can in fact assess those things which are

required for the sort of degree or course that you’re running. (…) The

assessment must be linked to the teaching” (Jim Jackson, interview).

Professor Jackson argued that due to inauthentic assessments, MOOCs were not

worthy of university credit equivalents. The identity of the person who did the

assessments remained unverified.

“I have no idea whether you did the multiple choice quizzes (…). You’re

paying your cousin or your uncle or your aunt to do the course for you. Or

(…) more possibly, your secretary [both laugh.]” (Jim Jackson, interview).

Another issue that, in Jackson’s view, made the assessment invalid, was that the

experience and qualifications of the peer assessors was unknown. He asked

rhetorically: “What’s to stop me (…) paying [a person] to (…) tick and tick and put

400 marks (…) wherever they want?”

Generally, Sir John, despite being aware of some encouraging research on the validity

of peer-assessment, suggested to only use peer grading for formative and not for

summative purposes.

“Careful research (…) shows that there is a high correlation between the

results of the peer-grading and the results of grading by experts. (…).

However, what the research says is one thing, what the (…) students accept is

another. (…) There are lots of anecdotal cases of people with PhDs, taking

MOOCs and being absolutely angry to learn that their stuff was being assessed

by some neophyte just out of school. [Interviewer laughs.]” (Sir John,

interview).

  112  

Sir John said that there had been “many aphorisms over the years to (…) criticize

distance learning” such as “‘Would you like to be operated on by a neurosurgeon

who’s done his course by correspondence?’” He commented: “‘Would you like

to be graded only by your fellow students?’ is another similar question”. In order to

award a credible credential, it was “probably important” to have “some kind of

summative evaluation where there are real profs and real experts involved” (Sir John,

interview).

The assessment in the MOOC described in Appendix 3 seemed “fairly lightweight” to

Professor Jackson who saw the danger of MOOC designers “cutting out the tricky

things – that you really need to have the student in your presence, to properly assess”.

The lack of feedback by experts and the lack of qualitative feedback were critiqued:

“Unfortunately your assignment didn’t get marked by experts. (…) You didn’t

get any feedback on whether you’re on the right track or the wrong track, or

were you on a popular track that appealed to a cohort of students or whether

you’re on a completely ignorant track?” (Jim Jackson, interview).

Jackson considered feedback as a critical feature of online learning: “good feedback is

important (…), and timely feedback [is] particularly important” (interview). He

concluded that the assessment for the University of Virginia MOOC on Coursera (see

Appendix 3) was a “disaster”, lamenting that a “grand university established by an

even grander person” (US President Thomas Jefferson in 1819) should have done

better. For MOOCs as for any other course, “proper processes of assessment” need to

be carried out, otherwise “it (…) is a pretence of a valid educational system” (Jim

Jackson, interview).

Professor Jackson argued that keystroke biometrics, that is sometimes discussed as a

possibility to prevent cheating would not be the solution. First, “the uncle can just

whisper into my ear while I’m striking the keyboard”, or secondly, “I pay my uncle to

take the course and he is always the person from day one” (Jim Jackson, interview).

An alternative to keystroke recognition would be eye recognition via webcams in

  113  

order to combat cheating (Dave Cormier, interview). The honour code (that MOOC

participants tick off before taking quizzes) was deemed not robust enough to prevent

cheating (Jim Jackson, interview). Several experts encouraged invigilated

examinations, for instance coordinated by Pearson (Dave Cormier, interview).

Cormier opined that cheating “has long been a part of the educational system” and

argued that current first-year course assessments are “badly evaluated” and may not

be “valid” (interview). Assuming a hybrid model in which MOOCs would replace

physical classroom courses in the first year or two, Cormier argued that if students

“spent two years trying to fake the system, and fake it and fake it and fake it, (…) at

some point, you will have to go and talk to a human, and the human looks at you and

goes ‘you don’t understand any of this?’ – well, then you just wasted two years”

(interview).

Cormier also questioned the quality of current predominant modes of assessment.

“The thing I like most about MOOCs related to assessment is that it forces the

learner to develop their own self-regulating process, and I agree that that

doesn’t work very well (…) because we never give people a chance to actually

develop those things. So we spend 12 (…) or 13 years (…) in the K-12

system. (…) And we do testing on them at 18 or 19 years old, and it turns out

that they can’t think for themselves, well, I wonder why. (…) We trained them

[in a totally behaviouristic way]. (…) And then, all of a sudden, you expect

them to be all like rhizomatic [both laugh], it takes a long transition” (Dave

Cormier, interview).

Cormier is using Deleuze and Guattari’s (1987) term “rhizomatic”, which comes from

botany, and connotes a way of thinking that allows for multiple entry and exit points

in presentation and interpretation, and is opposed to an “arborescent” concept of

knowledge that works with binary choices (Cormier, 2008).

  114  

Figure 4.5 Dave Cormier’s MOOC #Rhizo 14

Source: Cormier, 2014b.

Similarly, Downes commented on assessment:

“There’s something (…) very artificial about course objectives and rubrics and

standardised assessment and all the rest of it. And it’s this artificiality that we

are attempting to escape and to circumvent in CCK08” (interview).

Sir John argued that problems with assessment in online learning had long been

resolved:

“A lot of people, when MOOCs started, and indeed, when online started to

take off, got (…) very (…) worried about the security of assessment and

examination. (…) I think that’s a false problem. (…) Distance learning

universities solved that problem years ago. (…) It’s no bigger a problem than

plagiarism on campus” (interview).

  115  

He also highlighted that sophisticated assessments have been automated since

decades.

“In 1972, at the British Open University, (…) one of the tasks they put me on

was devising (…) computer-marked questions which were much more subtle

and demanding than the usual (…) right-wrong answer thing. (…) I (…)

started using some of these types of questions on my own students. And I had

a riot on my hands because they (…) found them much more difficult. They

sort of had the expectation that a multiple choice question was pretty easy, and

you can guess half the time, and these ones they couldn’t. (…) If you (…) use

your brains, you can come up with assessment that can be automated, and of

course you automate the responses, so that there’s tutorials that come along

with the wrong answers and so on” (Sir John, interview).

However, Sir John argued that “in many subjects, you will still require human beings

in the system”. Thus, there was the need to hire “tutors, mentors, teachers, adjuncts,

(…) that each one can be given a relatively reasonable group of students to deal with,

(…) between 25 and 30 students” (Sir John, interview). While this was less expensive

“than attempting to do the same thing entirely on campus face-to-face”, considerable

costs were still involved (Sir John, interview). On the topic of teaching assistants,

Professor Siemens commented that “if that’s important to a university (…), that’s a

suitable option” (interview).

With reference to Professor A’s earlier comment that assessment may be the MOOCs’

Achilles heel, Professor Siemens begged to differ, preferring to call it an

“underdeveloped” area (interview). Professor Siemens also observed a double

standard, as traditional education made much use of auto-graded systems, but

MOOCs were criticised for the very same practice. He also shared his own experience

with experimentation and innovation regarding assessment which goes back to the

first MOOC ever, CCK08. In that MOOC, summative assessment only occurred for

the 25 properly enrolled, fee-paying students.

  116  

“I (…) individually marked their assignments, I gave them personalized

feedback on papers that they submitted and so on. (…) We did nothing that

says ‘you can’t add an evaluation layer to a MOOC, or integrate a MOOC

within a physical classroom where you’re providing assessment to some of

your students, but not all of the students in the MOOC’. So I think we do have

a lot of opportunity to innovate here” (George Siemens, interview).

Stephen Downes distinguished internal assessment (for progression within a

university programme) versus external assessment (for employment).

“You need to be recognised by a doctor as a (…) doctor. (…) You have to

(…) fly a plane while somebody watches you fly the plane before they’ll let

you fly a plane” (Stephen Downes, interview).

Downes argued that this kind of external assessment “functions (…) in many ways

better than a testing and assessment infrastructure” and emphasised the community’s

ability to recognise experts.

“What we need is the recognition process (…) that allows a person to interact

with the community and become progressively recognised by more and more

(…) people in the field. (…)There’ve been a few crude attempts (…).

LinkedIn is attempting to build the professional reputation thing. Ebay has

buyer and seller reputation. (…) The community as a whole has an excellent

capacity to recognise, (…) is the person an expert?” (Stephen Downes,

interview).

In Downes’s perspective, a MOOC was ideally “a perceptual device for recognising

whether a person is capable and competent in that subject area” (Stephen Downes,

interview).

4.3.6 Accreditation, credit transfer and badges

  117  

When looking at MOOCs as formal learning, it is a valid critique that they largely

offer courses, but not an overall curriculum that could amount to a degree (Jim

Jackson, interview). In Dr Saxberg’s words: “A lot of the MOOCs now are tourist

occasions to a one-shot course” (interview). This critique was addressed in a novel

initiative to offer a Master’s degree by Georgia Tech (which is described in various

sections of chapter 2).

“Georgia Tech (…) is the first major institution saying: ‘We’re gonna take a

shot at saying that you’ll be ready to work at the end of this. That you’ll really

have a Master’s degree from Georgia Tech University and it’ll be valuable’. I

think it’s gonna be terrific to find out how do those people who graduate from

that – presumably with very little expert involvement, ‘cause otherwise it’s not

a MOOC, right? – how do they do?” (Bror Saxberg, interview).

Dr Saxberg commented further that “AT&T, presumably, is expecting that what they

get at the end are actually valuable human beings, out of a MOOC!”

For most MOOCs, their duration is short, often in the range of “five to six weeks”

which raises the question: “what kind of certification do you give for that?” (Sir John,

interview). In this context, Sir John regarded “open badges” as “significant and

important” (interview). Dave Cormier highlighted the “try it on” approach to credits

(which is essentially a freemium model).

“eCornell, (…) they would offer the first course in the programme for free and

then you would pay (…) to actually join the rest of the programme. (…) And

so it’s kind of like a credit transfer but not really. It’s like an informal one.

(…) I think the sort of ‘try it on’ sort of approach to MOOCs is a really good

one” (Dave Cormier, interview)

Dave Cormier also mentioned the Canadian PLAR (Prior Learning Assessment and

Recognition) and opined that Western Governor’s University practice to allow

students to “challenge for credit (…) makes sense” (interview). Similarly, Professor

Siemens argued:

  118  

“[The] university needs to recognize learning at any level and any type of

learning, and that (…) is essentially what this accrediting model could do for

us. But (…) many universities (…) are worried about integrating learning

outside of the classroom, because they are afraid that they will undermine the

university experience” (interview).

Professor Siemens also stated that he was “happy to see (…) OERu [i.e. Open

Education Resources University] (…) starting to create a model where students take

MOOCs, but local universities provide the formal credit or recognition for that

learning” (interview).

4.4 Are MOOCs sustainable?

4.4.1 The cost of MOOCs

Initially, “20 to 50 thousand dollars” was considered sufficient to budget a MOOC

(which “may have been an under-estimate”), whereas at present, “people talk about

200,000 dollars, 250,000 dollars” – which is “a lot to spend on (…) doing a free

course” (Sir John, interview). Sir John observed increasing competition by MOOC

providers and professors, and as a result, heightened sophistication of MOOCs, also

leading to ever-increasing costs.

“Some of the earlier MOOCs had (…) a very old-fashioned and sort of

pedestrian pedagogy of simply talking faces (…) on rather poor video. (…) It

was fine for the first movers, (…) they made a splash. Each additional MOOC

offerer is less likely to get (…) a reward of reputation because they are just a

bit lost in the crowd. So in order to stand out, their MOOCs have to be of

extremely high quality – meaning, usually, entertainment values (…) and

clarity (…) – which of course jacks up the price. (…) This would also

eliminate some of the players who might (…) not be prepared to spend a

quarter of a million dollars a pop on doing a all-singing-and-all-dancing

MOOC” (Sir John, interview).

  119  

Sir John shared that as a result of the exploding costs, “university financial officers

start to ask much more hard questions about why they are doing this for nothing”:

“Where is the business model? And what is the purpose?” At the same time, Sir John

observed other MOOCs, outside the big systems, that were most probably much more

cost-effective.

“Technology marches on, and people are telling me that the large

infrastructures of the Coursera’s (…) are already in many ways old hat, and

that the future is not there. (…) The people who did the mobile technology

MOOC [see section 4.6.3] (…) used (…) something they sort of lashed up

themselves. (…) They said, ‘look, these days you don’t need to give people a

sort of fully integrated learning system like Moodle or the old learning

management systems’. (…) If you tell them, there is a video on YouTube

here, and a piece of text there, and a quiz there, they are perfectly capable of

finding that themselves. They don’t need you to package it in these (…) big

systems” (Sir John, interview).

Professor Siemens emphasized that it was the universities who footed the lion’s share

of the bill for MOOCs: “Universities (…) are still the ones that are paying the people

that are teaching the MOOCs (…), if you look at all the people who are on Coursera

and all the faculty on edX” (interview).

In Dave Cormier’s perspective, xMOOC providers were Learning Management

Systems. Professor Siemens noted similarly:

“Right now, Coursera is a parasite that is feeding off the university system.

(…) Going forward, (…) universities will eventually (…) start to become

aware of some of those value points. (…) By and large, the integrity of

Coursera is founded on the integrity of the university system” (George

Siemens, interview).

  120  

4.4.2 MOOCs as marketing devices

MOOCs were observed explicitly by several experts as marketing devices. Professor

Jackson observed that universities that had little experience with online learning had

jumped onto the MOOC bandwagon:

“The Johnny-come-latelies, who ironically are often the oldest universities,

(…) are putting stuff online of dubious quality, of questionable assessment.

(…) ‘Why are they doing this? What are they giving this away for?’ And the

answer must be that these are purely marketing devices. So the cynic in me

comes out and says that MOOCs are a fairly cynical attempt by some

relatively new players in the scene who put some stuff up either for self-

promotion – that is an academic wanting to get their name out there around the

world (…) – or to promote their University” (Jim Jackson, interview).

Dr Tan (interview) thought that MOOCs provided an “excellent” marketing

opportunity for Singaporean polytechnics or universities. He observed that the two

largest public universities in Singapore, National University of Singapore (NUS) and

Nanyang Technological University (NTU), had already started to offer MOOCs, as it

was “a must-do kind of thing to get into the big league” (Samson Tan, interview). Dr

Tan thought that an NTU MOOC by Professor White on “Beauty, Form, and

Function: An Exploration of Symmetry” was particularly well designed and showed

much marketing intent.

“They went to the botanical gardens in Singapore. (…) He got another

professor, a botanist, to talk about symmetry in flowers, showing symmetry in

orchids, what occurs naturally (…). And then (…) he [Professor White] took

the whole lesson to the CBD [Singapore’s Central Business District] to look at

some of the architectural buildings, (…) using symmetric designs. So he got

an architect to talk about that. And (…) so on. (…) The instructional design

intent is to make the content accessible to students and to create an interest in

the subject itself. (…) A lot of resources, a lot of effort was put into designing

that learning” (Samson Tan, interview).

  121  

Figure 4.6 Screenshot of MOOC on “Beauty, Form, and Function: An

Exploration of Symmetry”

Source: White, 2014.

Dave Cormier considered US government support as important for the sustainability

of MOOCs.

“The only wild card in this is whether or not the American government

decides that it loves MOOCs. (…) I’ve heard some mumbling lately that it

might. (…) Considering how their medical roll out went [both laugh], I’m not

sure if I will trust them with this” (Dave Cormier, interview)

4.5 MOOCs as a viable alternative to traditional higher education?

None of the eight experts regarded MOOCs as a completely viable alternative to

higher education. Although there was some variety within that consensus, most would

agree with Saxberg’s statement:

“I think a lot of people are probably still too overheated about their enthusiasm

for MOOCs, that they think it would literally replace a University”

(interview).

  122  

Jackson noted that MOOCs were not a viable alternative but “a rich tree to be picking

from” (interview). Siemens argued that due to the multiple stakeholders involved in

higher education, MOOCS were not a viable alternative to the university model, and

MOOCs replacing universities was “a misdirected view” (interview):

“If the stakeholders in education were only the teacher and the student, then

possibly you could replace universities with MOOCs. But it’s not. It’s society,

it’s the business sector, it’s the government, (…) there’s a lot of stakeholders.

(…) So then, (…) we can’t repurpose universities based solely on the teaching

relationship between the instructor and the student, because there is research

and other activities that are part of it” (George Siemens, interview).

Reminding me of his perspective on “education as an integrated system”, and that

such integrated complex systems rarely changing due to a single influence, Siemens

did not think “that MOOCs by themselves will change the university” (interview).

There was a range of factors acting as “change pressures”: (1) students being older

and having more industry experience; (2) the global doubling of “the numbers of

students in higher education” “in the next decade” – leading to the need for “more

universities than we currently have in order to teach those students”; and (3) global

knowledge economies requiring“ a different type of education than if you are in a

production or manufacturing economy” (George Siemens, interview).

“If somebody was to say to me ‘will MOOCs change the university?’ I would

say ‘no’. (…) When all of those things are taken together, then I would say

‘yeah, that constellation of change pressures has the prospect of changing

higher education, (…) there could be dramatic changes coming up’ (George

Siemens, interview).

Similarly, Downes argued that the future of universities is the opposite of Thrun’s

prediction that there would be only ten universities be left in fifty years’ time.

Downes predicted a proliferation of universities.

  123  

Figure 4.7 A Future with only 10 universities

Source: Watters, 2013.

In Downes’s perspective, the coming proliferation of universities would be facilitated

by them becoming less costly in the future. Physical libraries that constitute a “huge

part of the overhead of running a university” may be replaced by open digital

libraries, and during our Skype interview, he showed me his 64 GB thumb drive on

his keychain.

“In the developing world, (…) a full library with all of the classics (…), most

people just don’t have access to them. But now you should get that kind of

library (…) I have one on my keychain. [Interviewer laughs; Stephen Downes

shows me his keychain.] (…) 64 gigabytes is enough (…) [for] probably all

the literature up until (…) 1965 (…), all of the literature (…). [Interviewer

  124  

laughs.] (…) I can make you a copy of this on my computer (…), this memory

device costs (…) less than 100 dollars” (Stephen Downes, interview).

Professor A pointed me to some recent statements by Udacity’s co-founder Thrun

where he “reneged on the whole thing”, saying “‘it was a painful moment’” realising

“‘we have a lousy product’” and that he did “not even like the term MOOC”

(Professor A, interview).

Dr Saxberg argued that mediocre training for work may get destroyed by MOOCs

(interview). However, he did also not perceive MOOCs as a viable alternative to

universities because of the need for experts.

“I do not see MOOCs killing off institutions that have to charge tuition that

(…) use experts. Because there is no way to reach expertise without having

coaching form experts, if you’re trying to really get to expertise properly”

(Bror Saxberg, interview).

Just like the university was not doomed, there was no death of the faculty in the

consensual opinion of the interviewed experts. In Saxberg’s view, proclaiming the

death of the faculty was like saying, “once you recorded the string quartet, just shoot

all the people who play” (interview). There was a continued need for musicians

because of their different interpretations over time. Similarly, teaching staff as expert

coaches continued to be necessary to get students ready to work (Bror Saxberg,

interview).

Professor A emphasised the critical importance of the personal connection between

student and teacher, “even in the distance-learning situation”, “in purposeful activity”

(interview). Similarly, Professor Jackson viewed education as a “personal

communication process”.

“If we let the massification of education ruin that, then we’ve lost it (…). ‘I do

feel special when I’m sitting in Jürgen’s lectures, ‘cause occasionally (…), the

great Jürgen [interviewer laughs], looked at me in class or he made some

  125  

really nice comment (…) when I walked into his class, or admired my new

mobile phone or something’ [interviewer laughs]. That personal touch stuff is

absolutely fundamental to the educational experience. (…) There is a terrible

danger [that] (…) MOOCs (…) will fail students in a personal sense, because

the educational experience they get is (…) so wiki-like (…) [and] alienated,

(…) and they drop out of the course” (Jim Jackson, interview).

Cormier jokingly said that MOOCs “are a viable alternative to really bad traditional

higher ed” (interview). He saw introductory courses run as MOOCs as a financial

threat to higher education.

“So much of higher ed is underpinned by people being in giant classes in first

year and a lot of those giant classes are, frankly, not very interactive. So

replacing them with a giant online course probably wouldn’t hurt all that

much?” (Dave Cormier, interview).

Jackson argued that some subjects are potentially less threatened than others, with

pharmacology a case in point:

“I got a bundle of students, (…) [some] of the top .1% in Australia. They are

brilliant students, they’ve got into my university, they need me desperately.

(…) Whatever (…) I do with them (…), they will stay with that course

because it’s so damn hard to get into, and so expensive to take, and they will

go through it and come out as doctors” (interview).

However, IT courses were an altogether different matter. They were “a dime a

dozen”, “you don’t need any particular high-flying ability to get into one (…) and

there’s a million of them”:

“I’m probably going to be much more tempted to take (…) MOOCs, that are

offered perhaps by non-university providers, (…) by IBM or some other

accrediting bodies” (Jim Jackson, interview).

  126  

Professor Jackson concluded that the answer varies “from discipline to discipline”.

Professor A viewed MOOCs metaphorically as a library of course materials and

lectures.

“It would be a very full library. There’s hundreds of people in there and you’re

all moving around in the same space and (…) some, you bump into and talk

to, and others, you just brush past (…). But there’s nothing purposeful (…)

about it” (Professor A, interview).

Professor A cast some doubt on MOOCs being a viable complement to traditional

higher education. The idea of the flipped classroom may work with other web

resources, but not with MOOCs, due to logistical problems in assuring synchronicity

between MOOCs and university courses (Professor A, interview).

In contrast, Professor Jackson was more sanguine about the prospects of a hybrid

model between university courses and MOOCs (interview). Similarly, Professor

Siemens saw MOOCs as complementary to traditional higher education:

“I think, MOOCs will add a good layer to help universities think through and

improve their activites (…) and how they connect with students, transitioning

more to digital network learning models (…). The role of MOOCs will vary

from institution to institution. (…) If you are taking a course at a community

college (…) that may partner with edX as a MOOC, (…) you will take the

instructional part, but get credit through your university. For a top tier

university like Stanford or MIT, MOOCs might give an opportunity for those

students to be a tutor or a mentor in one of those courses (…) so that they can

have practical experience in those kinds of settings (…)” (George Siemens,

interview).

Sir John predicted that “there will be a shakeout in 2014 in the sort of traditional

MOOCs area, really for two reasons, the evaluation results start to come in and some

institutions decide that this didn’t give them the benefits they expected” (interview).

  127  

  128  

4.6 MOOCs as disruptive innovation and providing access to the bottom of the

pyramid?

4.6.1 MOOCs are not much of a revolution

The experts thought of MOOCs largely not as revolution. Saxberg qualified that

“MOOCs are a revolution in the (…) sense that they are attempts to do zero variable

cost delivery of higher education instruction using technology” (interview). Daniel

categorically declined that “higher education does revolutions” amidst “all this

feverish talk about MOOCs and the revolution in higher education” (interview).

Figure 4.8 Screenshot from PowerPoint presentation: “Higher Education

Doesn’t Do Revolutions”

Source: Daniel &Uvalić-­‐Trumbić,  2014.

Experts like Jackson saw the innovation in online learning which long preceded

MOOCs:

“These Universities, (…) they’ve just suddenly discovered online education.

Well, hello, they are about 30 years behind the rest of the world because the

rest of the world has already been there and done that. [Interviewer laughs.]

(…) In fact, most of that innovation has already occurred and the MOOCs

themselves have copied some of that available stuff (…) from existing online

Higher Education doesn’t do Revolutions!

Sir John Daniel & Stamenka Uvalić-Trumbić

  129  

Universities such as Kaplan and Phoenix and (…) my own old place, Southern

Cross University, who were doing innovative stuff in this area 10, 15, 20 years

ago. (…) These issues were already sorted out 20 years ago, (…) definitely

(…) in Australian Universities” (Jim Jackson, interview).

4.6.2 MOOCs are not a disruptive innovation

In an article (Rudolph 2012b), I had considered that, as many MOOCS are led by

Harvard and MIT professors (amongst other world-class universities), participants

may not only get a good-enough, but a high-quality product. Moreover, whereas the

normal disruptive innovation may offer an ultra-low-price product, MOOCs, to the

consumers, were largely free-of-charge. In my conversation with Saxberg, I

suggested, that as a result of the MOOCs being high-quality and free, perhaps the

very concept of disruptive innovation could be disrupted by them. Saxberg,

however, argued that MOOCs are only mediocre, as world-class universities excel in

providing challenging environments for the top students, but are not very good in

teaching average students:

“Oh, you have to be careful, (…) the Harvards, the Yales, MITs (…), they are

filled with experts and amazing thinkers and problem solvers. They are also

extraordinarily good at selecting the very best students that come in. So the

students who come into those institutions are without a doubt, the best learners

in the world. (…) They cannot be stopped from learning, right? (…) It turns

out those institutions, they don’t know much about learning. They are not

actually very good at teaching the 80%. They’re incredibly good at creating

challenging, thought-provoking environments for the .1%. And they’re never

going away. (…) MOOCs (…) are mediocre (…) [in] their ability to lift that

80% up to some level of expertise. (…) It’s good enough, its mediocre, it’s

fine. But (…) it’s not well designed for the purpose” (Bror Saxberg,

interview).

  130  

Sir John argued that MOOCs are not a disruptive innovation and that they are also not

the answer to the problems in the developmental world. Sir John defined higher

education as “teaching and learning plus credentialing” (interview).

Figure 4.9 Three core elements of higher education

Source: Daniel & Uvalić-­‐Trumbić,  2014.

He argued that you cannot “honestly call MOOCs ‘higher education’”, as the

element of credible credentials is missing:

“What people in the developing world want, above all, is credible credentials

that recognize their learning. And getting a certificate of completion by paying

the Pearson company or something for a MOOC doesn’t do that” (Sir John,

interview).

Professor Siemens noted that disruptive innovation as a model was seductive “and a

lot of people” were drawn to it, but strongly disagreed that MOOCs were a disruptive

innovation.

“Christensen’s work on disruptive innovation, (…) even though it’s often used

and cited, (…) is often misunderstood, and (…) when it applies to social

systems, (…) Christensen himself misunderstands it. (…) All of his work was

looking at (…) innovation in mechanical systems that have no

What is higher education?

Credentials

  131  

interconnections with other entities. So he looks at rebar (…) in his early

examples about integrated steel mills, and that’s very different than

knowledge. (…) Knowledge, (…) you can’t package it the same way” (George

Siemens, interview).

Despite Christensen’s publications on education and on MOOCs, Siemens argued that

any attempt to apply the disruptive innovation model to social systems “is a little bit

of a fool’s game (…) because there’s absolutely no research that connects

Christensen’s mechanical manufacturing innovation model to innovation that happens

in knowledge systems” (interview). Siemens argued that, rather than universities

being disrupted or replaced by MOOCs, they should strive in the knowledge era (see

section 4.5).

“We are now moving into what we call a knowledge era. (…) And in a

knowledge era, it seems to me that the dumbest thing to do would be to kill

knowledge institutions [interviewer laughs], namely our universities. If

anything, I feel we should try to build them up. (…) So what’s going to

happen in education in many years, is that there will be a diversification of

education providers, there will be new (…) projects coming along, but by and

large, universities will retain their form, I don’t think they will be disrupted in

Christensen’s model. I think there will be other providers that will come along

and serve emerging segments of the market, but that won’t change the core

market of the universities. So what we need is more and better-integrated

university models (…), not one new model like MOOCs. (…) Christensen’s

model, serving a market place that’s not attended to by existing universities

and then eventually replacing universities, I can’t see that happening” (George

Siemens, interview).

To Siemens, MOOCs were an “augmentation, not a replacement” and Christensen’s

industrial model of disruptive innovation was not applicable to knowledge.

“Non-rivalrous goods can be the norm in the knowledge work setting, and that

is a completely different structure than what he [Christensen] did in his

original work. (…) Because he is a well-known academic, folks let him get

  132  

away with it, (…) even though (…) the logic errors in his transition of rebar to

universities is (…) borderline nonsensical” (George Siemens, interview).

4.6.3 Do MOOCs provide access to the bottom of the pyramid?

Saxberg conceded that “even a mediocre offering (…) in the hills of Kazakhstan [both

laugh] is better than no offering” (interview). Overall, my interviewees offered

several arguments why MOOCs may not provide access to the bottom of the pyramid.

One has to do with the literacies that the poor normally lack (see section 4.3.2).

Another argument is Saxberg’s that expert coaching does not appear to be offered by

free MOOCs. However, in Saxberg’s perspective, MOOCs “may be able to deliver

the same level of learning at lower cost”.

“It lets you move those costs and invest them later in the harder things like the

coaching. (…) Because if they are the same quality and there’s zero variable

cost, that’s a huge benefit, as long as you put the money to work, to actually

finish the job, right?” (interview).

Professor A argued that both on the grounds of lack of literacies and the inappropriate

focus on mere delivery aspects, MOOCs were not suitable for the bottom of the

pyramid.

“You can lead the horse to the water but can’t make it drink. You (…) could

sort of hold it out there and say, here’s your education (…). But (…) you need

to have a context. You need to have collaboration. (…) You need to have

discussion. (…) You need guidance. There’s so many things to learning, (…)

just offering it is not enough in my view” (Professor A, interview).

Other issues that the developing world was facing were connectivity and

electrification (Stephen Downes, interview). Despite all challenges, Downes

emphasized access as a major benefit of MOOCs, as a “game changer” and “the only

  133  

reason I’m in this field”: “people who never got access to anything like this level of

learning, now have access to it” (interview).

Sir John argued that, to him, “the biggest challenge that the world faces at the moment

is the huge rate of youth employment” (interview).

“There were 300 million young people who were not in education, not in

employment and not in training – (…) about a quarter of the [world’s] young

population. Now this is a devastating statistics. I am not pretending that more

higher education will solve that, but MOOCs certainly won’t solve that”

(Sir John, interview).

However, Sir John was also able to provide positive, promising examples, while

generalizing that “the more important developments are actually the MOOCs that are

being done (…) by universities in partnership with other bodies” (interview). Sir John

narrated the story of a MOOC offered jointly by the Indian Institute of Technology,

Kanpur, and the Commonwealth of Learning on the use of mobile technologies in

development (interview).

“It seems to be extremely successful not least in that it attracted a completely

unexpected audience. (…) They had 125 people from the country of Grenada

in the Caribbean. (…) The (…) population of Grenada can’t be more than a

100,000. So suddenly, (…) they picked a subject, which was of huge interest

to (…) people who are working in development. (…) In the developing world,

mobiles are relatively much more important than laptops” (Sir John,

interview).

  134  

Figure 4.10 MOOC on Mobiles for Development by IIT Kanpur &

Commonwealth of Learning

Source: Mobiles for Development, 2013.

4.6.4 Unbundling and rebundling

Sir John observed the unbundling of higher Education by the OERu (Open Education

Resource University), a consortium of international institutions. He perceived that as

a disruption, but reminded his listener that the disruptive innovation was online

education, not MOOCs.

“They are (…) offering essentially to mentor and assess students who elect to

do their own study quasi-independently, using open educational resources or

MOOCs or whatever. (…) This is (…) another trend that (…) is being (…)

accelerated by MOOCs, and that is the unbundling of higher education. (…)

Instead of having an integrated system – where not only one institution, but

often one teacher, designs the course, teaches the course, assesses the student,

  135  

gives them their grades – (…) you’ve now got some students offering a

MOOC; another institution offering assessment and credit for that MOOC;

another one maybe giving credit transfer for the result into a programme and

so on (…). And that (…) is a disruptive sort of thing” (Sir John, interview).

4.7 Conclusions

This concludes the discussion of the research questions through the perspectives of

experts. The experts’ convergence should perhaps not be over-emphasised, as their

occasional divergence has brought out the richness of the social world – and the

opinions about, and interpretations of, it – that should not be reduced to a neat table

demonstrating convergence. To recall two examples, there was much convergence

that MOOCs were not a disruptive innovation, but the current state of assessment was

much more disputed and contested. In the final chapter, a synthesis of the literature

review and the empirical analysis in this chapter will be created, with the formulation

of theses and recommendations.

  136  

5. Synthesis, Recommendations and Conclusions

The research findings from chapter 4 will be compared and contrasted with the

findings of the literature review from chapter 2 in section 5.1. In the previous section

(4.7), advantages of divergence were briefly noted. However, due to the fact that

experts were referenced in the literature review, and experts were also the major

source of the empirical research, a great deal of convergence can be reasonably

expected. After all, both literature review and empirical research analyse the

perspectives of experts.

In section 5.2, recommendations are made for some of the major stakeholders in

MOOCs, in particular learners, instructors and higher education institutions. In

section 5.3, the future of MOOCs is considered, including opportunities for future

research, before the project is concluded.

5.1 Synthesis of literature review and research findings

5.1.1 Research question 1: the quality of the educational experience in MOOCs

In both literature review and empirical analysis, the quality of the educational

experience was assessed by discussing (1) who participates in MOOCs and why; (2)

what literacies MOOC participants may need to have; (3) content suitability for

MOOC purposes; and the critical issues of (4) assessment, (5) certification and

accreditation.

(1) The literature review resulted in a distinction of different types of learners in

MOOCs: auditing, completing, disengaging and sampling learners as well as drop-ins

and lurkers. Despite MOOCs undoubtedly attracting global participation, the bottom

of the pyramid was rarely reached. Many participants were older, non-traditional

students who already held traditional degrees. As mostly ‘completing learners’ would

be interested in certification and there were so many other types of learners, non-

completion was often not seen as a significant problem.

  137  

Much of the literature review on this issue was confirmed in the expert interviews.

Some experts exemplified the high educational level of MOOC participants by

valuing MOOCS as a repository of explicit knowledge and narrating the phenomenon

of recreational education. Typical cMOOC participants were older and more

sophisticated than most xMOOC participants. Occasionally, a demographic change

was forecast which would broaden the participant base. MOOCs were a part of

informal learning (potentially leading to formal learning) and distributed personal

learning networks.

(2) The literature has discussed that technical access alone is insufficient for

successful participation in MOOCs, and certain literacies, including prior education

and self-motivation, may be additionally required. Hence, MOOCs appear to perform

poorly with at-risk students. The expert interviews were largely in agreement with the

literature, but produced a richer discussion. Downes envisaged “multiple literacies”

(that included literacy, capability and motivation) that went beyond STEM skills.

Cormier highlighted communication skills as a necessary literacy, in addition to

“classic literacies” such as decision-making, active engagement, and openness to

enquiry. Siemens stressed the importance of familiarity with social media as a digital

literacy.

(3) There did not appear to be much of a limit to what can be taught in a MOOC in

the review of the literature. However, there was some divergence amongst the experts

on this topic. Several experts argued that fairly basic knowledge could be suitable for

MOOCs. Other experts argued that MOOCs can potentially teach (almost) any kind of

subject matter, provided that appropriate instructional approaches are adopted.

(4) Assessment was perceived as an issue of critical importance both in the literature

and in the interviews. The literature perceives cheating as a serious issue in MOOCs,

which needs to be addressed via learner authentication (that may include “keystroke

biometrics”) and invigilated examinations. As, due to the massive aspect of MOOCs,

it may not be feasible for instructors to personally assess participants’ work, MOOCs,

amongst other things, have employed multiple choice questions, automated essay

scoring (AES) and calibrated peer review (CPR), which all come with certain

limitations.

  138  

Most experts were critical of assessments in MOOCs, but there were many different

interpretations of this complex and important issue. Jackson argued that due to

inadequate learner authentication, MOOCs were not worthy of university credit

equivalents; that assessments in MOOCs oftentimes appeared lightweight; and that

there was a lack of qualitative feedback

The absence of feedback by experts was critiqued. A separate section (4.3.3) had

been created to reflect the importance that the experts had attached to expert

involvement, including assessment by experts. Saxberg argued that MOOCs were

likely to only be able to create mediocrity, as the development of expertise required

the coaching by experts. Mere expert delivery was woefully insufficient for a quality

educational experience.

Assessment was sometimes regarded as the Achilles heel of MOOCs. In contrast,

Daniel, while suggesting limiting peer grading to formative assessments, argued that

problems with assessment in online learning had long been resolved. Cormier,

Downes and Siemens questioned the quality of current predominant modes of

assessment, and Downes distinguished internal assessment (for progression within a

university programme) – while questioning its meaningfulness – versus external

assessment (for employment).

(5) The literature review showed that MOOC certificates have largely failed to

penetrate the traditional system of credits and degrees. While academic credit initially

appeared unimportant to most MOOC learners, this was expected to change. MOOC

accreditation had appeared to be accelerating in the US, but zero students had taken

up any credit transfer offers by early 2014.

The review of the literature also showed that xMOOC providers have taken steps to

go beyond the single-course format by offering sequences of courses that create a new

type of alternative credentials. The Georgia Tech online Master in Computer Science

is marketed as an innovative experiment that involves aspects of MOOCs, but the

programme is neither massive nor open, and could be viewed as more of a traditional

online course. The experts were aware of the discussion in the literature. Some

  139  

experts regarded open badges as an important development towards universities

recognising other types of learning.

Quality is in the eye of the beholder, and at the beginning of chapter 4, some of the

experts’ different philosophical approaches, that influenced their perspectives on

quality, were discussed. Nevertheless, it would appear that the following theses would

be largely agreeable to the experts, and this would also be supported by the literature.

Figure 5.1 Theses 1-4 on the perception of experts on MOOCs (Research

Question 1)

1. MOOCs have not reached out to the poor and less-educated because of several

reasons, requisite literacies being one of them.

2. Non-completion is not a major issue due to many learners not being “completing

learners”.

3. Involvement of human expertise raises the quality of the educational experience,

including assessment.

4. Despite the ongoing development of alternative credentials, the issue of academic

credit for MOOCs is largely unresolved.

Source: self-developed.

5.1.2 Research question 2: the sustainability of MOOCs

In chapters 2 and 4, the sustainability of MOOCS was assessed via three sub-topics:

(1) the cost of MOOCs, (2) their revenue models and (3) their marketing function.

(1) To participants, MOOCs were initially free, but there is a trend to charge them for

certification and other fees that makes them even less open. At present, there is very

limited credit transfer from MOOCs occurring. Thus, MOOCs do not have much

influence in reducing the high cost of higher education to students. Faculty offers

  140  

their time and is normally remunerated for their participation. Universities pay most

of the cost, which is substantial and ranges approximately between US$50,000 and

US$250,000 per MOOC. The concept of reaching massive numbers of students at

zero marginal cost per student after the initial investment is seductive, but hinges on

the avoidance of expert involvement in formative and summative assessment.

In an expert interview, Daniel observed ever-increasing costs due to heightened

competition between universities and professors, which led to a questioning of the

‘business model’ of universities offering MOOCs. However, different from the

dominant xMOOC model (which in essence was an LMS), there were much more

cost-effective models that, rather than using a fully integrated learning management

system, employed a less integrated approach that relied on the participants’ digital

literacy.

(2) Revenue models for MOOCs were discussed in the literature and while it can be

said that the experts were generally aware of that discussion, it was not of great

interest to them. In the literature review, a dozen of often interrelated suggestions how

MOOC providers can create revenue were discussed.

(3) Both the literature review and the expert interviews took note that MOOCs were

part of universities’ marketing strategies. Occasionally, experts noted the irony of

some of the oldest universities jumping onto the MOOC bandwagon, as these had

previously shunned and critiqued online learning.

The sustainability of the xMOOC phenomenon is everybody’s guess, with some of

them having massive financial resources and others positioning themselves as big data

companies. The decision of governments (such as the US government) to make

MOOCs part of their educational policies or not could influence xMOOCs’ longevity.

However, there is no reason to doubt the sustainability of cMOOCs and other low-

cost MOOCs.

In conclusion of the discussion of research question 2, the following theses would

appear to be reasonably supported by the literature and the experts.

  141  

Figure 5.2 Theses 5-7 on the perception of experts on MOOCs (Research

Question 2)

5. MOOCs have not had much of an impact in reducing the high cost of higher

education to students.

6. Universities may start questioning the business model of offering expensive

MOOCs, lest they regard that as part of their marketing strategy.

7. Irrespective of whether xMOOC providers are sustainable or not, low-cost

MOOCs are here to stay.

Source: self-developed.

5.1.3 Research question 3: MOOCs are not a viable alternative to traditional higher

education

During the MOOC hype of 2012, MOOCs were occasionally considered as a viable

alternative to traditional higher education. However, MOOC providers are dependent

on universities, as the latter retain their monopoly on credit-granting privileges.

Rather than being an alternative, the literature generally views MOOCs as having a

role in blended learning, complementing and supplementing universities.

There was convergence amongst the experts – and with the literature – that MOOCs

were not a viable alternative to higher education. Siemens conceptualised education

as an integrated system and pointed to the multiple stakeholders in higher education

which rendered it impossible to repurpose universities based solely on the teaching

relationship between the learner and the instructor. MOOCs by themselves would not

change the university, but they were one of the ‘change pressures’. Siemens and

Downes predicted a proliferation of universities due to ever-increasing global demand

for higher education and more countries becoming knowledge economies.

  142  

However, some experts perceived mediocre higher education as under duress, and

certain subjects as well as introductory courses may be more threatened by MOOCs

than others. Just like universities were not generally doomed, there was no death of

the faculty in the consensual opinion of the interviewed experts. Teaching staff as

expert coaches continued to be necessary to get students ready for work; the personal

connection between student and teacher was of critical importance; and education was

perceived as a personal communication process.

The following theses would appear to be supported by the majority of literature and

the experts.

Figure 5.3 Theses 8-9 on the perspectives of experts on MOOCs (Research

Question 3)

8. MOOCs are not a viable alternative to traditional higher education. They may

complement higher education in blended learning.

9. There will be no ‘death of the faculty’, as human experts play a critical role in

higher education.

Source: self-developed.

5.1.4 Research question 4: MOOCs are not a disruptive innovation and do not

provide access to the bottom of the pyramid

The final research question was discussed in four parts: (1) MOOCs are not

revolutionary, (2) they are not a disruptive innovation, (3) they do not provide access

to the base of the income pyramid, and (4) there may only be limited unbundling and

rebundling of higher education via MOOCs at this point.

(1) It is quite common to ponder whether MOOCs are revolutionising higher

education. However, even co-founders of xMOOCs like Koller and Thrun are

skeptical about revolutionary claims (that they themselves made earlier). If MOOCs

are viewed in their historical context of distance learning, over-the-top claims made

  143  

for them are but the last instance of technophile hype. The experts also thought of

MOOCs largely not as revolution. MOOCs may be unable to offer zero variable cost

for higher education courses (otherwise they might qualify as a revolution). Online

learning was considered as the greater innovation as compared to MOOCs.

(2) Originally, Christensen viewed online learning as a disruptive innovation in higher

education. However, in 2013, he specifically described MOOCs as a disruptive

innovation, claiming that they serve non-consumers, march upmarket (through

improved quality) and redefine quality. To Christensen, it was unusual that some

universities led the MOOC movement, which set out to disrupt universities, as this

meant that incumbents led the disruption that would purportedly disrupt them! Higher

education was on the edge of the crevasse, and most universities would adopt hybrid

strategies, making MOOCs part of their higher education model.

Christensen’s perspective on MOOCs as a disruptive innovation has a minority

following among academics, but the majority of academics beg to differ, amongst

them an xMOOC leader and Christensen’s co-author Horn (who views MOOCs as

resembling technology vendors).

With the possible exception of Saxberg, the experts unanimously converged in

disregarding MOOCs as a disruptive innovation. Persuasively arguing that MOOCs

were not a disruptive innovation and not the answer to the problems in the developing

world, Daniel went further to refuse to call MOOCs ‘higher education’, as the

critical element of credible credentials was missing. Siemens also strongly disagreed

that MOOCs were a disruptive innovation, arguing that Christensen himself

misunderstood his own concept of disruptive innovation, as it was not meaningful to

apply his innovation model that originated from mechanical manufacturing to

innovation in knowledge systems. Rather than universities being disrupted, Siemens

expected them to strive in the knowledge era, and MOOCs were an augmentation, not

a replacement of them.

(3) A common and laudable motivation amongst educationists involved in MOOCs is

the desire to increase access to higher education worldwide. There are some

  144  

promising programmes like edX’s SocialEDU that are too recent to assess. However,

serious doubt has been cast on whether the bottom of the pyramid can be reached.

Access is dependent on literacies, one-way traffic from ‘the west to the rest’ could

amount to intellectual neo-colonialism and a higher-education hegemony, and there

are cultural biases in academic traditions.

Even in the US, MOOCs have met with serious challenges to educate at-risk and low-

income students and were perceived to have involuntarily shown that higher

education cannot be automated. Rather than reaching out to the poor at the bottom of

the pyramid (who lacked requisite literacies), MOOCs might increase inequality and

perpetuate a two-tier system in which low- and no-cost MOOCs were a less attractive

alternative when compared to traditional college education.

The experts offered several arguments why MOOCs may not provide access to the

bottom of the pyramid. The poor normally lack requisite literacies; expert coaching is

not offered in free MOOCs; the focus on delivery aspects of higher education is

insufficient; and there are issues of connectivity and electrification.

(4) With the commodification of many popular curricula, OER and MOOCs have

opened up access to them. AES and CPR were occasionally viewed as potentially

disrupting assessment. The traditional vertically integrated role of the faculty may

become disaggregated, dramatically shifting to a much more specialized role of

curator of information and mentor to students as a result of unbundling. Daniel lauded

the unbundling of higher Education by the OERu (Open Education Resource

University) and perceived that as a part of disruptive innovation, which was online

learning (not MOOCs).

In conclusion of the discussion of the final research question, the following additional

theses, based on the literature and expert interviews, are proposed.

  145  

Figure 5.4 Theses 10-13 on the perspectives of experts on MOOCs (Research

Question 4)

10. MOOCs are not revolutionary and are rather a development within online

learning (which was a greater innovation than MOOCs).

11. MOOCs are not a disruptive innovation.

12. MOOCs do not provide access to the bottom of the pyramid (see thesis 1 in

section 5.1.1).

13. There may only be limited unbundling and rebundling of higher education at this

point.

To recapitulate, the research problem of this project was: What are the perspectives

of experts on MOOCs in terms of their potential as a disruptive innovation in

higher education? The 13 theses that have been developed within the synthesis of

this section would point to a summarised perspective. They could assist in creating a

grounded theory (which is beyond this humble piece of research), but at the same

time, the thick description of the rich diversity of expert perspectives is a valuable

research result in itself.

5.2 Benefits and recommendations for implementation

5.2.1 Challenges and benefits

There are multiple issues with MOOCs, as they suffer “from weaknesses around

access, content, quality of learning, accreditation, pedagogy, poor engagement of

weaker learners, exclusion of learners without specific networking skills” (The

Maturing of the MOOC, 2013, p. 4). MOOCs may also fail to teach creative and

critical thinking skills and may not treat students as individuals (The Maturing of the

MOOC, 2013). In Freedman’s (2013) perspective, “MOOCs are just the latest

  146  

incarnation of bringing watered-down versions of culture, knowledge, and learning to

a mass audience”, and they should not be confused with university courses.

Perhaps precisely because MOOCs were generally not seen as disrupting higher

education, there may be multiple benefits to them for various stakeholders. For

society at large, MOOCs provide an opportunity to gain knowledge on a range of

new and emerging subject areas. These learning opportunities do not need to be tied

to particular outcomes like getting a degree (George Siemens, interview), and they

provide great convenience to working adults who are engaged in lifelong learning.

Access to high quality material is enabled, as MOOCs provide environments for self-

learning, and cMOOCs offer new kinds of collaborative learning experiences (The

Maturing of the MOOC, 2013). In Daniel’s perspective, MOOCs will create indices of

teaching quality that“ may expose the teaching weaknesses in some elite institutions”

and will thus “oblige institutions to do more than pay lip service to the importance of

teaching and put it at the core of their missions”, which is “the real revolution of the

MOOCs” (quoted in The Maturing of the MOOC, 2013, p. 22).

For educators, benefits included the ability to reach a broader learner base; and the

opportunities to interact with competent and motivated learners, and to translate their

research into layperson terms (George Siemens, interview). For universities, MOOCs

provided a relevant “playground”, that is “the opportunity to begin experimenting

what teaching and learning could look like in a digital network structure” (George

Siemens, interview). MOOCs may assist via a flipped classroom to reduce the need

for transmission teaching and increase opportunities for expert coaching and

mentoring (Bror Saxberg, interview). Generally, MOOCs may compel universities to

rethink instruction and improve on their teaching quality.

For online education, MOOCs and their surrounding hype have led to a timely and

highly welcome re-evaluation of online education. Daniel argued that the MOOC

phenomenon’s “lasting importance will be in the spin-offs [and] the by-products (…)

which are not pure MOOCs” (interview). Another benefit, in Siemens’s perspective,

is that MOOCs provide plenty of opportunity for experimentation, including blended

learning (interview). Cormier (interview) emphasised diversity as a benefit of

MOOCs, as the exposure to different cultures made people question their

  147  

westocentrism. He also introduced me to another one of his neologisms, namely

eventedness (Cormier, 2009):

“It’s very hard to join a community. It’s very easy to join an event. (…) The

thing about MOOCs, cMOOCs in particular, is their eventedness, [it] makes

them (…) appealing. (…) Here’s my excitement, looking forward to seeing

more and more small pockets of (…) a 1,000 people or 5,000 people here and

there, coming together, inside of their particular cultural reality or perspective.

(…) If that’s the legacy that MOOCs have, and people feel the permission to

create their own subcultures, I think that’s really exciting. (…) It’s really

healthy for the Internet as well” (Dave Cormier, interview).

5.2.2 Six recommendations

On the basis of our discussion thus far, including the 13 theses developed in section

5.1 and the multiple benefits of MOOCs discussed in the previous section, six

recommendations how to derive maximum benefit from the MOOC phenomenon for

higher education are made.

(1) With asssessment often being considered a problematic aspect of MOOCs,

Balfour’s (2013) suggestion of combining AES for formative assessment and CPR

for summative assessment may well be worth considering. If in addition to that,

human experts can be used to assess MOOC participants, this is likely to be best.

Consequently, a sophisticated mix of assessments that is constructively aligned with

learning outcomes and teaching strategies, is recommended.

(2) The incompletely resolved issue of assessment in MOOCs notwithstanding, a

recommendation for universities is to be more open to testing applicants on what they

have learnt in MOOCs and in other more informal contexts. Admission and advanced

standing are recommended to become more flexible.

(3) The first capital ‘O’ in the acronym MOOC means ‘open’. The interpretation of

‘open’ by edX is truer to the OER movement, and it is recommended for future

  148  

MOOCs to be open in the full sense of the word, i.e. offer open admission, being free

of charge, using open source software and OER – “educational materials that may be

freely accessed, reused, modified and shared” (Daniel & Uvalić-­‐Trumbić,  2014).

(4) OER, and perhaps to a lesser extent, MOOCs (as timetables, ideally, would have

to be synchronised) are useful for the concept of the flipped classroom which is

highly recommended for higher education. The flipped classroom allows shifting

some of the learning to a “personal space, for the students to (…) learn [at their] own

time and at their own convenience or own device” (Samson Tan, interview). There is

less need for lectures, and this frees time up for higher-level learning activities in the

physical environment.

(5) With MOOCs not having been much of a disruptive innovation in providing

access to the bottom of the pyramid, it is recommended to view MOOCs as but a part

of the whole gamut of online learning strategies (including OER and highly

successful attempts to bring education to the masses like the Khan Academy).

(6) Do your own MOOC! My final recommendation to seasoned academics is to

consider doing their own MOOC. Sir John recommended:

“What would be interesting, (…) now that (…) the infrastructure is a bit

easier, it would be to team up with a bunch of like-minded friends (…) on a

subject that you feel the world needs. And keep it simple and straightforward

and see what happens. ‘Cause it seems that everyone who offers any kinda

MOOC is amazed by the (…) response, [it’s] much greater than they thought”

(interview).

Sir John narrated the experience of a Malaysian engineering dean who had offered a

MOOC on emotional intelligence and was surprised that most of the participants were

from western countries. The western participants were interested “looking for (…) an

Asian take on emotional intelligence”, which “was a problem” because most of his

material was “derivative from American textbooks and American thinking” (Sir John,

interview).

  149  

“Being a splendid flexible guy, he really got to work and tried to make it more

indigenous. (…) It blew their [Malaysian engineering students’] minds to be

talking about emotional intelligence with women in Holland [interviewer

laughs] (…). So there are some wonderful spin-offs of these things, there are

some wonderful anecdotes and the full story of this, it will never be told,

because it’s such a mass phenomenon. But I do relish these little stories that I

come across and how people who offered a MOOC, (…) it sort of changed

their whole perception of life” (Sir John, interview).

Dave Cormier thought along similar lines, emphasizing how useful he found it to

conduct MOOCs also for himself:

“More and more people are going to do what I’m doing right now (…): ‘Oh! I

can totally do this to push my research forward to talk about the things that

I’m interested in’. It’s essentially a community building activity” (Dave

Cormier, interview).

  150  

Figure 5.5 Learn with your peers

Source: P2PU, 2012.

5.3 The future of MOOCs and opportunities for further research

5.3.1 Opportunities for further research

Impressive quantitative research has begun to be published (see Ho et al., 2014). It is

intended to relate expectations to outcomes, which is meaningful, as it would give a

clearer picture about the course completions of the ‘completing learners’, as opposed

to those who have different objectives in signing up for a MOOC. Generally, data-

centric learning analytics should be conducted with the massive amounts of MOOC

data continuously and rapidly becoming available.

There is also much room for qualitative research, with sensemaking approaches being

“augmented by data and analytics” (Siemens, 2014). For instance, e-focus groups

could be conducted to find out more about the experience of participants. Also, there

is room for phenomenological studies that take note of the experiences of expert

users.

An important area of research is of course how the mission of the dominant xMOOCs,

to provide access to high quality education for the masses, can be better implemented.

To that intent, developmental experts could be interviewed and consulted, and

  151  

existing efforts like edX’s SocialEDU should be studied carefully. There are also

many MOOCs sprouting outside of the big systems, including many languages other

than English. All this appears to be under-researched at this point which is small

surprise, given the novelty of the phenomena.

5.3.2 The future of MOOCs

Predicting the future of MOOCs and their role in higher education is obviously

extremely hazardous. However, there would appear to be some certainty that the

future will not look like in the following quotation:

“In fifty years, if not much sooner, half of the roughly 4,500 colleges and

universities now operating in the United States will have ceased to exist. The

technology driving this change is already at work, and nothing can stop it. The

future looks like this: Access to college-level education will be free for

everyone: the residential college campus will become largely obsolete; tens of

thousands of professors will lose their jobs; the bachelor’s degree will become

increasingly irrelevant; and ten years from now Harvard will enroll ten million

students” (Harden, 2012).

Professor A’s prediction that MOOCs will have a future, settling down, “maybe in a

couple of years, into a little niche market” (interview) appears to make much more

sense. There could even be some backlash, as students appear to be occasionally

“starved for contact with faculty”, and universities may wish to emphasise the

“human connection in the classroom” (Freedman, 2013).

5.4 Conclusions

Arguing that MOOCs and other online learning should totally replace face-to-face

learning would mean throwing out the baby with the bathwater. Hybrid learning, at

its best, may be pedagogically preferable to both online and classroom approaches.

The idea of the flipped classroom, where students watch lectures on video and then

come for higher-level learning to a physical classroom where they discuss the lectures

with an expert coach and mentor may be considered ideal.

  152  

Higher education should instill a lifelong love for learning and assist in the

development of the abilities to think critically and communicate effectively.

Economic competitiveness and the future of a democratic society depend on these

graduate attributes (Arum and Roksa, 2011). Rosen (2011) imagined the world of

higher education in the year 2036 as more mobile, more disaggregated, more

personalized, more focused on learning outcomes, more accessible, more global, and

last but not least, cooler. Hopefully, it will not take that long!

Having a better understanding of the MOOC phenomenon that created ‘100,000-

student classrooms’ may contribute to educational theory and practice. From a

pedagogical perspective, MOOC teaching should not be a one-way transmission or

what Biggs and Tang (2011) call a Level 1 theory of teaching. For a more

sophisticated teaching and learning experience to occur in MOOCs, one would need

to engage online teaching assistants or e-moderators who are both computer wise and

content experts so that they can appropriately engage the participants.

This analysis of MOOCs, a relatively novel and fast-changing innovation in higher

education, is now concluded. The massive (pun intended) bibliographical references

of this thesis were an attempt to capture the current state of the multi-disciplinary and

multi-faceted literature. It is hoped that the method of convergent expert interview has

been innovatively used to assess MOOCs. The triangulation of qualitative research

methods (that is only hinted at here with some references to the author’s participant

observation in some MOOCs) would appear to be worthy of further consideration for

larger research projects, as is a mixed-method approach.

The over-hyped theoretical concept of disruptive innovation was found not to be

applicable to MOOCs. Universities as knowledge institutions in the knowledge

economy may be rather different from recorded music, movies, newspapers and

libraries, as they may not be disrupted in a similar way. Although MOOCs may not be

a disruptive innovation, it is hoped that innovations such as OER and MOOCs will be

sustaining innovations in the sense that they will make higher education better and

more open.

  153  

  154  

Bibliographical References

A Too-Cosy World? (2012, April 14). The Economist, 69. Retrieved from

http://www.economist.com/node/21552599

Abel, R. (2013, August 26). What is disruptive innovation in education? Learning

Impact Blog. Retrieved November 13, from

http://www.imsglobal.org/blog/?p=284

Abeles, T. (2013). The academy: hiding behind pay walls? On The Horizon – The

Strategic Planning Resource for Education Professionals, 21(2), 75-78.

Retrieved from

http://www.ingentaconnect.com/content/mcb/274/2013/00000021/00000002/a

rt00001

ACE recommends five MOOCs for credit. (2013, March). Women in Higher

Education, 22(3), 5. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1319252369?accounti

d=12629

ACE pilot project to review value of MOOCs. (2012, December). Women in Higher

Education, 21(12), 3. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1260760247?accounti

d=12629

Agarwala, M. (2013). A research summary of MOOC completion rates. EdLab

Teacher’s College, Columbia University. Retrieved from

http://edlab.tc.columbia.edu/ index.php?q=node/8990

Aguaded-Gómez, J. (2013). The MOOC revolution: A new form of education from

the technological paradigm? Comunicar, 21(41), 7-8. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1428056225?accounti

d=12629

AI Class. (2011). Introduction to Artificial Intelligence. Course by Sebastian Thrun &

Peter Norvig. Retrieved from https://www.ai-class.com/

  155  

Allen. C. (2010, July 29). The $100 biology book. The New York Times. Retrieved

from http://www.nytimes.com/roomfordebate/2010/07/25/the-real-cost-of-

college-textbooks/why-that-textbook-costs-100

Al-Sayyed, R., Hudaib, A., Al-Shboul, M., Majdalawi, Y., & Bataineh, M. (2010).

Automated Assessment, Face to Face. International Journal of Emerging

Technologies in Learning (iJET), 5(3), 4-11. Retrieved from

http://www.editlib.org/p/45345/

Altbach, P. (2013, December 4). MOOCs as Neocolonialism: Who Controls

Knowledge? The Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/worldwise/moocs-as-neocolonialism-who-controls-

knowledge/33431

Altbach, P., Reisberg, L., & Rumbley, L. (2009). Trends in Global Higher Education:

Tracking an Academic Revolution. A Report Prepared for the UNESCO 2009

World Conference on Higher Education, United Nations Educational,

Scientific and Cultural Organisation (UNESCO).

Anders, G. (2013, April 3). Sebastian Thrun’s Online Goal. Act where College is not

working. Forbes. Retrieved from

http://www.forbes.com/sites/georgeanders/2013/04/03/sebastian-thruns-

online-goal-act-where-college-isnt-working/

Anders, G. (2012, June 25). How would you like a graduate degree for $100? Forbes.

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1020098775?accounti

d=12629

Anderson, T., & McGreal, R. (2012). Disruptive Pedagogies and Technologies in

Universities. Educational Technology & Society, 15(4), 380-389. Retrieved

from http://0-

go.galegroup.com.prospero.murdoch.edu.au/ps/i.do?action=interpret&id=GA

LE%7CA310150665&v=2.1&u=murdoch&it=r&p=AONE&sw=w&authCou

nt=1

  156  

Angelo, T., & Cross, K. (1993). Classroom Assessment Techniques. San Francisco,

CA: Josey-Bass.

Angst for the Educated, (2011, September 3). The Economist. Retrieved from

http://www.economist.com/node/21528226

Applebaum, R. (2010, July 29). Little incentive to control costs. The New York Times.

Retrieved from http://www.nytimes.com/roomfordebate/2010/07/25/the-real-

cost-of-college-textbooks/there-is-little-incentive-to-control-textbook-costs.

Archibald, R., & Feldman, D. (2012). The anatomy of college tuition. Washington,

DC: American Council on Education. Retrieved from

http://www.acenet.edu/news-room/Documents/Anatomy-of-College-

Tuition.pdf

Arnold, D., & Valentine, A. (2013). Corporate social responsibility at the base of the

pyramid. Journal of Business Research, 66(10), 1904-1914. Retrieved from

http://0-dx.doi.org.prospero.murdoch.edu.au/10.1016/j.jbusres.2013.02.012.

Arum, R., & Roksa, J. (2011). Academically Adrift. Limited Learning on College

Campuses. Chicago: The University of Chicago Press.

Atkins, D., Brown, J., & Hammond, A. (2007). A Review of the Open Educational

Resources (OER) Movement: Achievements, Challenges, and New

Opportunities. Menlo Park, CA: The William and Flora Hwelett. Retrieved

from www.hewlett.org/uploads/files/Hewlett_OER_report.pdf  

Australian Trade Commission (2012). More than MOOCs: Opportunities arising from

disruptive technologies in education. [restricted report]. Australian

government: Australian Trade Commission. Retrieved October 14, 2013.

Attali, Y. (2007). On-the-fly customization of automated essay scoring (RR-07-42).

Princeton, NJ: ETS Research & Development. Retrieved from

http://www.ets.org/Media/Research/pdf/RR-07-42.pdf

Azevedo, A. (2012a). A Dot-Com Entrepreneur's Ambition: Drive Education Costs to

Zero. The Chronicle of Higher Education, 59(12). Retrieved from http://0-

  157  

search.proquest.com.prospero.murdoch.edu.au/docview/1170556070/F3206D

D8C1C3483EPQ/1?accountid=12629

Azevedo, A. (2012b). UC Online Faces Challenges in Era of Free Courses. The

Chronicle of Higher Education, 59(06). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1082420593/93A146

B158B14659PQ/1?accountid=12629

Baer, W. (1998). Will the internet transform higher education? Rand Corporation,

RAND/RP-685. Retrieved from

http://pesona.mmu.edu.my/~neomai/MME2011/RP685.pdf

Bajaj, G. (2013, September 9). A revolution is waiting in the wings. India

Today. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1429154430?accounti

d=12629

Baker III, F., & Surry, D. (2013). Open Education Designs: A Taxonomy for

Differentiating and Classifying Open Learning Environments. In R. McBride

& M. Searson (Eds.), Proceedings of Society for Information Technology &

Teacher Education International Conference 2013 (pp. 189-194). Chesapeake,

VA: AACE. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/48090.

Baker, P., Bujak, K. R., & DeMillo, R. (2012). The evolving university: Disruptive

change and institutional innovation. Procedia Computer Science, 14, 330-335.

Retrieved from http://dx.doi.org/10.1016/j.procs.2012.10.037

Balfour, S. P. (2013). Assessing writing in MOOCs: automated essay scoring and

calibrated peer review. Research & Practice in Assessment, 8(1), 40-48.

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1505317171?accounti

d=12629

  158  

Bates, T. (2013, November 14). MOOCs, Norway, and the ecology of digital learning.

Retrieved August 17, 2013, from

http://www.tonybates.ca/?s=more+provocatively%2C+i+also+argued

Becker, B. . (2013). Connecting MOOCs and Library Services. Behavioral & Social

Sciences Librarian, 32(2), 135-138. Retrieved from

http://www.tandfonline.com/doi/full/10.1080/01639269.2013.787383#.U1tLV

fmSx0E

Becker, H., & Geer, B. (1957). Participant observation and interviewing: a

comparison. Human Organisation, 16(3), 28-32. Retrieved from

http://sfaa.metapress.com/content/k687822132323013/

Belanger, Y., & Thornton, J. (2013, February 5). Bioelectricity: A Quantitative

Approach. Duke University’s First MOOC. Retrieved from

http://dukespace.lib.duke.edu/dspace/bitstream/handle/10161/6216/Duke_Bioe

lectricity_MOOC_Fall2012.pdf.txt?sequence=3

Belkin, D., & Porter, C. (2013, September 26). Job Market embraces Massive Online

Courses. The Wall Street Journal. Udacity Press Kit. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1437051425?accounti

d=12629

Bell, F. (2011). Connectivism: Its Place in Theory-Informed Research and Innovation

in Technology-Enabled Learning. International Review of Research in Open

and Distance Learning, 12(3), 98-118. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/49669.

Bellack, J. (Ed.). (2013). MOOCs: The future is here. Journal of Nursing Education,

52(1), 3-4. doi: 10.3928/01484834-20121227-10

Benderly, B. (2012). BOLD experiment. ASEE Prism, 22(2), 29-33. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1113681224?accounti

d=12629

  159  

Bennett, W. (2012, July 5). Is Sebastian Thrun's Udacity the future of higher

education? CNN. Retrieved from

http://edition.cnn.com/2012/07/05/opinion/bennett-udacity-education/

Benson, R. (2013). How shall we teach our engineers? Mechanical Engineering,

135(8): 16. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1418438088?accounti

d=12629.

Biemiller, L. (2014a, February 24). QuickWire: edX and Facebook Team up to offer

free education in Rwanda. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/quickwire-edx-partners-with-

facebook-to-offer-courses-in-rwanda/50693

Biemiller, L. (2014b, February 20). Harvard and MIT release visualization tools for

trove of MOOC data. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/harvard-and-mit-release-

visualization-tools-for-trove-of-mooc-data/50631

Biemiller, L. (2014c, January 26). QuickWire: For-Profit University will accept

MOOC Credits. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/quickwire-for-profit-university-will-

accept-mooc-credits/49805

Biggs, J., & Tang, C. (2011). Teaching for Quality Learning at University. 4th ed,

Berkshire, England, McGraw Hill, Society for Research into Higher Education

and Open University Press.

Birckbichler, D., & Bias, R. (2013). The Ohio State University Foreign Language

Technology Certificate Program. In R. McBride & M. Searson

(Eds.), Proceedings of Society for Information Technology & Teacher

Education International Conference 2013 (pp. 1251-1255). Chesapeake, VA:

AACE.

Blackledge, D., & Hunt, B. (1985). Sociological Interpretations of Education.

London: Routledge.

  160  

Blignaut, S., & Leendertz, V. (2013). Balancing the Scale: Faculty Comperacy

Perceptions for Teaching Screen People within a Developing ODL Context. In

J. Herrington et al. (Eds.), Proceedings of World Conference on Educational

Multimedia, Hypermedia and Telecommunications 2013 (pp. 18-24).

Chesapeake, VA: AACE.

Blumer, H. (1969). Symbolic Interactionism. Englewood Cliffs, NJ: Prentice Hall.

Bogost, I. (2012, July 18). MOOCs are Marketing. Retrieved from August 17, 2013,

http://www.bogost.com/blog/moocs_are_marketing.shtml

Bowen, W. (2013a). Higher education in the digital age. New Jersey: Princeton

University Press.

Bowen, W. (2013b, March 25). Walk Deliberately, Don't Run, Toward Online

Education. The Chronicle of Higher Education, 59(29). Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1319470142?accounti

d=12629

Boyer, E. (1990). Scholarship Reconsidered: Priorities for the Professoriate.

Princeton, NJ: Carnegie Foundation for the Advancement of Learning.

Braband, C. (2006). Teaching and Understanding. Film, University of Aarhus,

Denmark. Retrieved from http://video.google.com/videoplay?docid=-

5629273206953884671

Brenner, M., Sörensen, P., & Weitz, D. (2014). SPU 27x

Science and Cooking: From Haute Cuisine to Soft Matter Science. edX

MOOC from Harvard University. Retrieved from

https://courses.edx.org/courses/HarvardX/SPU27x/2013_Oct/info

Breslow, L., Pritchard, D., DeBoer, J., Stump, G., Ho, A., & Seaton, D. (2013).

Studying learning in the worldwide classroom: Research into edX’s first

Mooc. Research & Practice in Assessment, 8, 13-25. Retrieved from http://0-

go.galegroup.com.prospero.murdoch.edu.au/ps/retrieve.do?sgHitCountType=

None&sort=RELEVANCE&inPS=true&prodId=AONE&userGroupName=m

  161  

urdoch&tabID=T002&searchId=R1&resultListType=RESULT_LIST&conten

tSegment=&searchType=AdvancedSearchForm&currentPosition=1&contentS

et=GALE%7CA339254329&&docId=GALE|A339254329&docType=GALE

&role=

Brown, A. (2013). Q&A George Siemens. Mechanical Engineering, 135(4), 24.

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1327955407?accounti

d=12629

Brown, B. (2010, July 29). The Faux Value of E-Texts. The New York Times.

Retrieved from http://www.nytimes.com/roomfordebate/2010/07/25/the-real-

cost-of-college-textbooks/the-faux-value-of-e-textbooks

Brown, J. (2013). Free for all. Toronto Life, 47, 37-38. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1324622875?accounti

d=12629

Brown, J. (2013). From the Editor – New combinations for Higher Education.

Research & Practice in Assessment, 8(1). Retrieved from

http://www.rpajournal.com/from-the-editor-new-combinations-for-higher-

education/

Bruce, G. (1992). Comments. In J. Svartvik (ed.), Directions in Corpus Linguistics:

Proceedings of the Nobel Symposium 82, Stockholm, August 4-8, 1991 (pp.

145-147). Berlin: de Gruyter

Buchanan, W. (2013). Too mooc or not? ASEE Prism, 22(9), 61-62. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1368868343?accounti

d=12629

Building Different MOOC's for Different Pedagogical Needs. (2012, June 11). The

Chronicle of Higher Education, 58(39). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1021360017?accounti

d=12629

  162  

Burrow, L. (2013a). What can MOOCs do for you?: Local Implications of a Global

Phenomenon. In R. McBride & M. Searson (Eds.), Proceedings of Society for

Information Technology & Teacher Education International Conference

2013 (pp. 280-282). Chesapeake, VA: AACE. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/48113.

Burrow, L. (2013b). Elements of a Successful MOOC from Participant Perspectives.

In R. McBride & M. Searson (Eds.), Proceedings of Society for Information

Technology & Teacher Education International Conference 2013 (p. 279).

Chesapeake, VA: AACE. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/48112.

Byerly, A. (2012, September 7). Before You Jump on the Bandwagon. The Chronicle

of Higher Education, 59(02). Retrieved from http://0-

go.galegroup.com.prospero.murdoch.edu.au/ps/i.do?id=GALE%7CA3020486

00&v=2.1&u=murdoch&it=r&p=AONE&sw=w

Carey, K. (2013a, January 21). Higher-Education Reform: a Legacy for Obama? The

Chronicle of Higher Education, 59(20). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1275108518?accounti

d=12629

Carey, K. (2012b, September 7). Into the Future With MOOC's. The Chronicle of

Higher Education, 59(02). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA302048595&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=5d6486bb29221888ac55af5288b19cfd

Carey, K. (2013c, March 25). The Brave New World of College Branding. The

Chronicle of Higher Education, 59(29). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1319470141?accounti

d=12629

Carlson, S., & Blumenstyk, G. (2012, December 17). The False Promise of the

Education Revolution; 'Disruptions' have the buzz but may put higher

education out of reach for those students likely to benefit the most. The

Chronicle of Higher Education, 59(17). Retrieved from http://0-

  163  

search.proquest.com.prospero.murdoch.edu.au/docview/1240580361?accounti

d=12629

Carson, D., Gilmore, K., & Perry, C. (2001). Qualitative Research in Marketing.

London: Sage, n/a (extract courtesy of C. Perry).

Casion, J., & Gewolb, J. (2013, June 21). In Online Partnership, Legal Compliance Is

Key. The Chronicle of Higher Education, 59(39). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA334943482&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=999049df16a737b98eefebca1e10a2f6

Canavan, G. (2013). Some preliminary theses on MOOCS. Retrieved September,

2013, from http://gerrycanavan.wordpress.com/2013/02/18/some-preliminary-

theses-on-moocs/http://gerrycanavan.wordpress.com/2013/02/18/some-

preliminary-theses-on-moocs/

Cavanagh, S. (2013). 'MOOC' provider coursera jumps into K-12 and teacher

ed. Education Week's Digital Directions 6, 9. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1370708818?accounti

d=12629http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1370708818?accounti

d=12629

Chachra, D. (2013). A novel approach. ASEE Prism, 22(6), 19. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1315339786?accounti

d=12629

Chafkin, M. (2013, November 14). Udacity's Sebastian Thrun, Godfather Of Free

Online Education, Changes Course. Fast Company. Retrieved from

http://www.fastcompany.com/3021473/udacity-sebastian-thrun-uphill-climb

Charmaz, K. (2006). Constructing Grounded Theory: A practical guide through

qualitative analysis. London: SAGE.

Charon, J. M. (2001). Symbolic Interactionism: An Introduction, An Interpretation,

An Integration. Upper Saddle River, NJ: Prentice Hall.

  164  

Chen, H., & Gilchrist, S. (2013). Online access to higher education on YouTubeEDU.

New Library World, 114(3), 99-109.

doi:http://dx.doi.org/10.1108/03074801311304023

Chng, G. (2012, April 08). ‘A HanZpad for every student’. The Sunday Times,

Singapore, p. 30.

Cho, K. & Schunn, C. (2003). Validity and Reliability of Peer Assessments with a

Missing Data Estimation Technique. In D. Lassner & C. McNaught (Eds.),

Proceedings of World Conference on Educational Multimedia, Hypermedia

and Telecommunications 2003 (pp. 1511-1514). Chesapeake, VA: AACE.

Cho, K., Schunn, C., & Wilson, R. (2006). Validity and reliability of scaffolded peer

assessment of writing from instructor and student perspectives. Journal of

Educational Psychology, 98(4): 891-901. doi: 10.1037/0022-0663.98.4.891

Christensen, C., Aaron, S., & Clark, W., (2005, March). Can Schools Improve?

Harvard Business Review 86(7), 545-550. Retrieved from http://0-

www.jstor.org.prospero.murdoch.edu.au/stable/20441841

Christensen, C., & Armstrong, E. (2001). Disruptive Technologies: A Credible Threat

to Leading Programs in Continuing Medical Education? The Journal of

Continuing Education in the Health Professions, 18(2), 69-80. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/206785083?accountid

=12629

Christensen, C., Baumann, H., Ruggles, R., & Sadtler, T. (2006, December).

Disruptive Innovation for Social Change. Harvard Business Review 84(2), 94-

101. Retrieved from http://0-

web.b.ebscohost.com.prospero.murdoch.edu.au/ehost/detail?vid=2&sid=a7b9

e361-0794-476a-be6d-

7ceecc260fa5%40sessionmgr111&hid=127&bdata=JnNpdGU9ZWhvc3QtbGl

2ZQ%3d%3d#db=bth&AN=23081431

  165  

Christensen, C., & Eyring, H. (2011). The Innovative University: Changing the DNA

of Higher Education from the Inside Out. San Francisco: Jossey-Bass Higher

and Adult Education Series.

Christensen, C., Horn, M., & Johnson, C. (2008). Disrupting Class: How Disruptive

Innovation Will Change the Way the World Learns. New York: McGraw-Hill.

Christensen, C., Horn, M., Caldera, L., & Soares, L. (2011). Disrupting College: How

Disruptive Innovation Can Deliver Quality and Affordability to Postsecondary

Education. Innosight Institute. Retrieved from

http://files.eric.ed.gov/fulltext/ED535182.pdf

Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Woods, D. & Emanuel, E. J.

(2013, November 6). The MOOC Phenomenon: Who Takes Massive Open

Online Courses and Why? Social Science Research Network. Retrieved from

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2350964

Cizek, G. (1999). Cheating on tests: How to do it, detect it, and prevent it. Mahwah,

NJ: Lawrence Erlbaum.

Clarke, T. (2013). The Advance of the MOOCs (Massive Open Online Courses): The

Impending Globalisation of Business Education? Education+ Training,

55(4/5), 6-6. doi: 10.1108/00400911311326036

Clarke, T., & Clarke, E. (2009). Born digital? Pedagogy and computer-assisted

learning. Education + Training, 51(5), 395-407.

doi:http://dx.doi.org/10.1108/00400910910987200

Clark, R., & Mayer, R. (2011). E-Learning and the Science of Instruction: Proven

Guidelines for Consumers and Designers of Multimedia Learning. San

Francisco, CA: Pfeiffer.

Cole, J. (2009). The Great American University. New York: Public Affairs.

College, Reinvented: The Finalists. (2012, November 9). The Chronicle of Higher

Education, 59(13). Retrieved from http://0-

  166  

search.proquest.com.prospero.murdoch.edu.au/docview/1220435136?accounti

d=12629

Cook, L. (1938). Community Backgrounds of Education: A Textbook and Educational

Sociology. New York: Macmillan.  

Cooper, M. (2013, January 7). 2013 - A look forward. The Hispanic Outlook in

Higher Education, 23(7), 54. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1326245102?accounti

d=12629

Cormier, D. (2014a). LinkedIn profile. Retrieved from

http://www.linkedin.com/profile/view?id=4067688&locale=en_US&trk=tyah

&trkInfo=tarId%3A1397126860316%2Ctas%3Adave%20cormier%2Cidx%3

A1-1-1

Cormier, D. (2014b). Explaining Rhizo14 to Oscar. Dave’s educational blog.

Education, Post-Structuralism and the Rise of the Machines. Retrieved March

14, 2014 from http://davecormier.com/edblog/

Cormier, D., & Siemens, G. (2010). The Open Course: Through the Open Door--

Open Courses as Research, Learning, and Engagement. Educause Review,

45(4), 30-32. Retrieved from http://www.editlib.org/p/110311

Cormier, D. (2009, April 15). BJET article – MUVE Eventedness – An experience

like any other. Retrieved from

http://davecormier.com/edblog/2009/04/15/bjet-article-muve-eventedness-an-

experience-like-any-other/

Cormier, D. (2008, June 3). Rhizomatic Education : Community as Curriculum.

Retrieved from http://davecormier.com/edblog/2008/06/03/rhizomatic-

education-community-as-curriculum/

Couros, A. (2009). Open, connected, social: Implications for educational design.

Campus-Wide Information Systems, 26(3), 232-239.

doi:http://dx.doi.org/10.1108/10650740910967393

  167  

Coursera, edX announce more MOOCs. (2013, April 15). Military Times Edge, 18.

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1349148667?accounti

d=12629

Coursera Specialisations: Focused Programs in Popular Fields (2014, January 20).

Coursera blog. Retrieved February 9, 2014, from

http://blog.coursera.org/post/73994272513/coursera-specializations-focused-

programs-in-popular

Crawford, T. (2013, April 29). MOOCs and the Material World. The Chronicle of

Higher Education, 59(34). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1347569587?accounti

d=12629

Cross, J. (2011, January 28). Those who forget the mistakes of history. Internet Time

Blog. Retrieved February 9, from http://www.internettime.com/2011/01/those-

who-forget-the-mistakes-of-history/

Currie, P. & Arbour, V. (2014). Dino 101. Dinosaur Paeobiology. Coursera MOOC

from University of Alberta. Retrieved from

https://www.coursera.org/course/dino101.

Czerkawski, B., & Lieberman, M. (2012). Building Collaborative International

Learning Experiences using Social Networking Services. In T. Bastiaens & G.

Marks (Eds.), Proceedings of World Conference on E-Learning in Corporate,

Government, Healthcare, and Higher Education 2012 (pp. 2022-2025).

Chesapeake, VA: AACE. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/41900.

Daniel, John (2014). Bios. Retrieved from http://sirjohn.ca/wordpress/?page_id=14 Daniel, J. (2012). Making sense of MOOCs: Musings in a maze of myth, paradox and

possibility. Journal of Interactive Media in Education, 3, 1-20. Retrieved from

http://web.a.ebscohost.com/abstract?direct=true&profile=ehost&scope=site&a

uthtype=crawler&jrnl=1365893X&AN=90529542&h=1AxCptn0tSlhYHfLlP

  168  

YGYm8oKLukFh%2bmnDQw%2bfoAovzMyhn6QW2WXSTdrcWR6JKPIw

zmApK5NF61zIcchS0NuQ%3d%3d&crl=c

Daniel, J. & Uvalić-Trumbić, S. (2014, January 15). Higher Education doesn’t do

Revolutions! PowerPoint presentation. Centre for Policy Studies in Higher

Education and Training, University of British Columbia. Retrieved from

http://sirjohn.ca/wordpress/?page_id=29

Darnton, R. (2013, April 25). The National Public Digital Library is Launched! The

New York Review of Books. Retrieved from

http://www.nybooks.com/articles/archives/2013/apr/25/national-digital-

public-library-launched/

Davidson, C. (2014, March 14). Changing Higher Education to change the world. The

Chronicle of Higher Education. Retrieved April, 2014, from

http://chronicle.com/blogs/future/2014/03/14/changing-higher-education-to-

change-the-world/

Davidson, C. (2012, December 14). Size Isn't Everything; For academe's future, think

mash-ups not MOOC's. The Chronicle of Higher Education, 59(16). Retrieved

from http://chronicle.com/section/About-the-Chronicle/83

Davis, J. E. (2014, February 7). MOOCs, Trust, and the Signature Track. The

Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/future/2014/02/07/moocs-trust-and-the-signature-

track/

De Langen, F. (2013a, August 19). Is Higher Education disrupted or not? About

Education, Economics and Policy. Retrieved from

http://frankounl.wordpress.com/2013/08/21/is-higher-education-disrupted-or-

not/

De Langen, F. (2013b, August 30). Educating the bottom of the pyramid or of the

market: the difference between Prahalad and Christensen. About Education,

Economics and Policy. Retrieved from

  169  

http://frankounl.wordpress.com/2013/08/30/educating-the-bottom-of-the-

pyramid-or-of-the-market-the-difference-between-prahalad-and-christensen/

De Meyer, A. (2013). The future of doctoral education in business administration.

Journal of Management Development, 32(5), 477-486.

doi:http://dx.doi.org/10.1108/02621711311328264

De Waard, I., Abajian, S., Gallagher, M., Hogue, R., Keskin, N., Koutropoulos, A., &

Rodriguez, O. (2011). Using mLearning and MOOCs to understand chaos,

emergence, and complexity in education. The International Review of

Research in Open and Distance Learning, 12(7), 94-115. Retrieved from

http://0-www.editlib.org.prospero.murdoch.edu.au/p/49391

Deeke, A. (1995). Experteninterviews – ein methodologisches und

forschungspraktisches Problem. Einleitende Bemerkungen und Fragen zum

Workshop. In: C. Brinkmann, A. Deeke, and B. Voelkel (eds.),

Experteninterviews in der Arbeitsmarktforschung. Diskussionsbeitraege zu

methodischen Fragen und praktischen Erfahrungen. Beitraege zur

Arbeitsmarkt- und Berufsforschung 191. Nuernberg Bundesanstalt fuer Arbeit,

pp. 7-22.

Deleuze, G., & Guattari, F. (1987). A Thousand Plateaus, trans. B. Massumi.

Minneapolis: The University of Minnesota Press.

Dennis, M. (2012). The impact of MOOCs on higher education. College and

University, 88(2), 24-30. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1372330643?accounti

d=12629

Denzin, N. (1989). The Research Act. 3rd ed. Englewood Cliffs, NJ: Prentice Hall.

Denzin, N., & Lincoln, Y. (2011). Introduction. The Discipline and Practice of

Qualitative Research. In: Denzin, N. & Y. Lincoln (eds.), The SAGE

Handbook of Qualitative Research, 4th ed, (pp. 1-21). Thousand Oaks, CA:

Sage.

  170  

DeSilets, L. (2013). A revolutionary journey into Learning/Education. The Journal of

Continuing Education in Nursing, 44(1), 8-9.

doi:http://dx.doi.org/10.3928/00220124-20121227-69

Desrochers, D., & Kirshstein, R. (2012). College spending in a turbulent decade:

Findings from the Delta Cost Project. Washington, DC: American Institutes

for Research. Retrieved from http://www.deltacostproject.org/pdfs/ Delta-

Cost-College-Spending-In-A-Turbulent-Decade.pdf

Devlin, K. (2014). Introduction to mathematical thinking. Coursera MOOC from

Stanford University. Retrieved from

https://www.coursera.org/course/maththink

Dickson, V. (2012, October). Buy-in from top schools helps new online courses tell

story. PRweek, 15(10), 16. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1326420772?accounti

d=12629

DiSalvio, P. (2012, September 4). Pardon the disruption: Innovation changes how we

think about higher education. New England Journal of Higher Education.

Retrieved from http://www.nebhe.org/thejournal/disruptive-innovation-

changing-how-we-think-about-higher-education/

Doll, R. (1964). Curriculum Improvement: Decision-Making and Process. Boston:

Allyn & Bacon.

Donaldson, J., Agra, E., Alshammari, M., Bailey, A., Bowdoin, D., Kendle, M.,

Nixon, L., & Wressell, L. (2013). Massively Open. How Massive Open Online

Courses Changed the World. Creative Commons Attribution. ISBN-13: 978-

1482775334.

Downes, S. (2014). LinkedIn profile. Retrieved from

http://www.linkedin.com/profile/view?id=583526&authType=NAME_SEAR

CH&authToken=dwoM&locale=en_US&srchid=1423753951397120032912

&srchindex=1&srchtotal=116&trk=vsrp_people_res_name&trkInfo=VSRPse

  171  

archId%3A1423753951397120032912%2CVSRPtargetId%3A583526%2CVS

RPcmpt%3Aprimary

Downes, S. (2013). Assessment in MOOCs. Retrieved August 11, 2013, from

http://halfanhour.blogspot.sg/2013/05/assessment-in-moocs.html

Downes, S. (2012). Connectivism and Connective Knowledge: essays on meaning

and learning networks. National Research Council Canada. Retrieved from

http://www.downes.ca/files/books/Connective_Knowledge-19May2012.pdf.

Downes, S. (2011). Free Learning. Essays on Open Educational Resources and

Copyright. National Research Council Canada. Retrieved from

http://www.downes.ca/files/books/FreeLearning.pdf

Duderstadt, J. (2012). The future of the university: A perspective from the Oort cloud.

Social Research: An International Quarterly, 79(3), 579-600. Retrieved from

http://www.socialresearch.newschool.edu/

Dyer, J., Gregersen, H., & Christensen, C. (2009, December). The Innovator’s DNA.

Harvard Business Review 87(12), 61-67. Retrieved from http://0-

web.b.ebscohost.com.prospero.murdoch.edu.au/ehost/results?sid=2d7deae3-

bd84-44b8-9437-

6d17b2082ac3%40sessionmgr111&vid=1&hid=127&bquery=IS+00178012+a

nd+TI+(%22The+Innovator%27s+DNA%22)+and+DT+%222009%22+and+

AU+%22Dyer%22&bdata=JmRiPWJ0aCZ0eXBlPTEmc2l0ZT1laG9zdC1sa

XZl

Education bulletin. (2012, December 01). Military Times Edge, 6. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1288711437?accounti

d=12629

Educause. (2012). What campus leaders need to know about MOOCs. Retrieved from

http://net.educause.edu/ir/library/ pdf/PUB4005.pdf

edX Media Kit (2013). Retrieved from https://www.edx.org/media-kit

  172  

Esposito, A. (2012). Research ethics in emerging forms of online learning: issues

arising from a hypothetical study on a MOOC. Electronic J. e-Learning,

10(3), 315-325. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1036930469?accounti

d=12629

Expert speak. (2013, March 11). India Today. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1314253710?accounti

d=12629

Fabian, N. (2013). How NEHA helps you to keep your job and be employed. Journal

of Environmental Health, 76(1), 74-78. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1411120192?accounti

d=12629

Farmer, J. (2013, August 14). MOOCs: A disruptive innovation or not? e-Literate.

Retrieved from http://mfeldstein.com/moocs-a-disruptive-innovation-or-not/

Farrell, C. (2012, September 24). Our (work) education crisis: Send in the

MOOCs. Business Week, 1. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1112037717?accounti

d=12629

Fasimpaur, K. (2013). Massive and open: massive open online courses (MOOCS),

which are already changing the face of higher education, are starting to create

new opportunities in K-12 professional development. Learning & Leading

with Technology, 40, 12-17. Retrieved from

http://www.iste.org/learn/publications/learning-leading

Feschuk, S. (2013, July 10). A mooch must not ask for fluffier

pillows. Maclean's, 126, 65. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1399171530?accounti

d=12629

Fini, A. (2009). The Technological Dimension of a Massive Open Online Course: The

Case of the CCK08 Course Tools. International Review of Research in Open

  173  

and Distance Learning, 10(5). Retrieved from

http://www.irrodl.org/index.php/irrodl/article/viewArticle/643/1402Continued

Flick, U. (2009). An Introduction to Qualitative Research. 4th ed. London: Sage.

Flynn, J. (2013). MOOCs: disruptive innovation and the future of higher education.

Christian Education Journal, 10(1), 149-162. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1344056030?accounti

d=12629

Free Online Courses By Five Top US Varsities. (2012, April 29). The Straits Times,

p. A25.

Freedman, J. (2013, November 25). MOOCs: Usefully Middlebrow. The Chronicle of

Higher Education. Retrieved from http://chronicle.com/article/MOOCs-Are-

Usefully-Middlebrow/143183/

Frey, T. (2013). Moving from just-in-case to just-in-time living. Journal of

Environmental Health, 75(10), 54-5. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1354972388?accounti

d=12629

Friedman, T. (2013a, January 28). Revolution hits the universities. The New York

Times. Retrieved from

http://www.oakland.edu/upload/docs/Clips/2013/130128%20-

%20universities.pdf

Friedman, T. (2013b, March 5). The Professor’s Big Stage. The New York Times.

Retrieved from http://www.nytimes.com/2013/03/06/opinion/friedman-the-

professors-big-stage.html?_r=1&

Foster, M. (2013). Coursera, sera. The American Scholar, 82(2), 120. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1346751000?accounti

d=12629

  174  

Fox, J. (2011). Disrupting Higher Ed. Harvard Business Review 89(7-8), 162-163.

Retrieved August 28, 2013, from www.elibrarynew.cognizant.com

Gadbury-Amyot, C. (2013, July). MOOCs, web 2.0 and 3.0, twitter, blogs...: What is

a dental hygienist to do? Access: The Newsmagazine of the American Dental

Hygienists' Association, 27, 14-16. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1412208737?accounti

d=12629

Gardner, H. (1993) Educating for understanding. The American School Board

Journal, 20-24.

Gardner, L., & Young, J. R. (2013, March 14). A Bold Move Toward MOOCs Sends

Shock Waves, but Details Are Scarce. The Chronicle of Higher Education,

59(28). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1318429638?accounti

d=12629

Gaskell, G. (2000). “Individual and group interviewing”. In Bauer, M.W. and F.

Gaskell, Qualitative Researching with Text, Image and Sound. A practical

handbook, 39-56. Retrieved from http://0-

dx.doi.org.prospero.murdoch.edu.au/10.4135/9781849209731

Geertz, C. (1973). The Interpretation of Cultures: Selected Essays. New York: Basic

Books.

Giam, S. (2012, April 09). Maths is no problem at this website. The Straits Times, p.

B4.

Gleick, J. (2011). The information: A history, a theory, a flood. London, UK:

HarperCollins.

Glaser, B. (1992). Basics of Grounded Theory Analysis: Emergence vs. Forcing. Mill

Valley, CA: Sociology Press.

Glaser, B. & Strauss, A. (1967). The Discovery of Grounded Theory: Strategies for

Qualitative Research. New York: Aldine.

  175  

Glover, I. (2013). Play As You Learn: Gamification as a Technique for Motivating

Learners. In . Jan Herrington et al. (Eds.), Proceedings of World Conference

on Educational Multimedia, Hypermedia and Telecommunications 2013 (pp.

1999-2008). Chesapeake, VA: AACE. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/112246.

Godwin-Jones, R. (2012). Challenging Hegemonies in Online Learning. Language

Learning & Technology, 16(2), 4-13. Retrieved from

http://www.editlib.org/p/74445

Gold, R. (1958). Roles in Sociological Field Observation. Social Forces, 36, 217-223.

doi:10.2307/2573808

Goldin, C., & Katz, L. (2008). The Race between Education and Technology: The

Evolution of U.S. Higher Educational Differentials, 1890 – 2005. Cambridge,

Mass.: Belknap Press of Harvard University Press

Gose, B. (2012, October 5). 4 MOOC's and How They Work. The Chronicle of

Higher Education, 59(6). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA304835421&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=42af112d17f1a828f551dc14ee72b338

Graesser, A., & McNamara, D. (2012). Automated analysis of essays and open-ended

verbal responses. In H. Cooper, P. Camic, D. Long, A. Panter, D. Rindskopf,

& K. Sher (Eds.), APA handbook of research methods in psychology, Vol 1:

Foundations, planning, measures, and psychometrics (pp. 307-325).

Washington, DC: American Psychological Association.

Graham, G. (2012, October 1). After the Buzz: How the Embrace of MOOC's Could

Hurt Middle America. The Chronicle of Higher Education, 59(06). Retrieved

from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1082420610?accounti

d=12629

Graham, K. (2013, May). Wisdom of the crowd. E.Learning Age, 22-23. Retrieved

from http://0-

  176  

search.proquest.com.prospero.murdoch.edu.au/docview/1370938724?accounti

d=12629

Grose, T. (2012). Clicks and mortar. ASEE Prism, 21(9), 17. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1018566819?accounti

d=12629

Grose, T. (2013). SUN NEVER SETS. ASEE Prism, 22(7), 17. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1331838528?accounti

d=12629

Gunersel, A., Simpson, N., Aufderheide, K., & Wang, L. (2008). Effectiveness of

Calibrated Peer ReviewTM for improving writing and critical thinking skills

in biology undergraduate students. Journal of the Scholar ship of Teaching

and Learning, 8(2), 25-37. Retrieved from http://eric.ed.gov/?id=EJ854838

Guthrie, D. (2012, December 17). Jump Off the Coursera Bandwagon. The Chronicle

of Higher Education, 59(17). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1240640775?accounti

d=12629

Hafner, K. (2010, April 16). Higher Education Reimagined With Online Courseware.

The New York Times. Retrieved from

http://www.nytimes.com/2010/04/18/education/edlife/18open-

t.html?_r=2andpagewanted=all

Harden, N. (2012, December 11). The End of the University as we know it. The

American Interest. Retrieved from http://www.the-american-

interest.com/articles/2012/12/11/the-end-of-the-university-as-we-know-it/

Harvard, MIT to offer free Web courses (2012, May 4). The Straits Times, p. A16.

Haynie, D. (2013, May). Learn for free with MOOCs. U.S.News & World Report, 1.

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1371297333?accounti

d=12629

  177  

Haynie, D. (2013a). 5 reasons international students should consider

MOOCs. U.S.News & World Report, 1. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1412864705?accounti

d=12629

Haynie, D. (2013b). MOOCs expand on college campuses. U.S.News & World

Report, 1. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1412864774?accounti

d=12629

Head, K. (2013). Massive Open Online Adventure; Teaching a MOOC is not for the

faint-hearted (or the untenured). The Chronicle of Higher Education, 59(34).

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1347569590?accounti

d=12629

Healy, P. (2003, June 29). College Rivalry. Boston Globe. Retrieved from

http://pinker.wjh.harvard.edu/about/media/2003_06_29b_bostonglobemagazin

e.html

Heller, N. (2013, May 20). LAPTOP U. The New Yorker, 89, 80-n/a. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1353301760?accounti

d=12629

Herrington, J., Reeves, T., & Oliver, R. (2010). A guide to authentic e-learning. New

York: Routledge.

High, P. (2013, September 12). “Udacity CEO Sebastian Thrun on the Future of

Education”. Forbes. Retrieved from

http://www.forbes.com/sites/peterhigh/2013/12/09/udacity-ceo-sebastian-

thrun-on-the-future-of-education/. -- this one only mentions that the Georgia

Tech and AT&T collaboration is with Udacity.

  178  

Hildebrand, B. (2010, July 29). A small expense, overall. The New York Times.

Retrieved from http://www.nytimes.com/roomfordebate/2010/07/25/the-real-

cost-of-college-textbooks/textbooks-are-a-small-expense-overall

Ho, A., Reich, J., Nesterko, S., Seaton, D., Mullaney, T., Waldo, J., Chuang, I.

(2014). HarvardX and MITx: The First Year of Open Online Courses, Fall

2012-Summer 2013. HarvardX and MITx Working Paper No. 1, January 21.

Retrieved from http://dx.doi.org/10.2139/ssrn.2381263

Ho, A. (2011, October 06). Let 1,000 lecturers bloom. The Straits Times, p. A20.

Howard, J. (2012, September 17). Can MOOC's Help Sell Textbooks? The Chronicle

of Higher Education, 59(04). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1041174052?accounti

d=12629

Hoffmann-Riehm, C. (1980). Die Sozialforschung einer interpretativen Soziologie:

Der Datengewinn. Koelner Zeitschrift fuer Soziologie und Sozialpsychologie,

32: 339-372.

Horn, M. & Christensen, C. (2013, February 20). Beyond the Buzz, where are

MOOCs really going? Wired. Retrieved from

http://www.wired.com/2013/02/beyond-the-mooc-buzz-where-are-they-going-

really/

Howe, J. (2013, February 12). Clayton Christensen wants to transform capitalism.

Wired. Retrieved from http://www.wired.com/2013/02/mf-clayton-

christensen-wants-to-transform-capitalism/all/

Hoxby, C. (2014). The Economics of online postsecondary educaction: MOOCs,

nonselective education, and highly selective education. National Bureau of

Economic Research (NBER) Working Paper Series, working paper 19816.

Retrieved from http://www.nber.org/papers/w19816

Hu, H. (2013, March 28). MOOC migration: massive open online courses are

changing the way we think about higher education. (massive open online

courses). Diverse Issues in Higher Education, 30(4), 10-11. Retrieved from

  179  

http://go.galegroup.com/ps/i.do?id=GALE%7CA325495889&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=688508caf1e543d71c4cd1119b4bf2ed

Hsu, J. (2012, January 25). "Professor leaving Stanford for online education startup".

MSNBC.com. Retrieved from

http://www.msnbc.msn.com/id/46138856/ns/technology_and_science-

innovation/#.T5ipqo4pO9Y

Hylén, J. (2007). Giving Knowledge for Free: The Emergence of Open Educational

Resources. Paris, France: OECD Publishing. doi:10.1787/9789264032125-en.

Ingram, M. (2012, October 29). Airbnb, coursera, and uber: The rise of the disruption

economy. Business Week, 1. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1140012140?accounti

d=12629

Irvine, V., Lee, M., Code, J., & Richards, L. (2013). Multi-access learning: Increasing

flexibility and choice for on-campus and remote students. In Jan Herrington et

al. (Eds.), Proceedings of World Conference on Educational Multimedia,

Hypermedia and Telecommunications 2013 (pp. 840-845). Chesapeake, VA:

AACE. Retrieved September 16, 2013. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/112058.

Jackson, J. (2014). LinkedIn profile.

http://www.linkedin.com/profile/view?id=143436683&authType=NAME_SE

ARCH&authToken=XJBP&locale=en_US&srchid=142375395139089422809

4&srchindex=1&srchtotal=2603&trk=vsrp_people_res_name&trkInfo=VSRP

searchId%3A1423753951390894228094%2CVSRPtargetId%3A143436683%

2CVSRPcmpt%3Aprimary.

Jick, T. (1983). Mixing Qualitative and Quantitative Methods: Triangulation in

Action. In J. v. Maanen (ed.), Qualitative Methodology. London: SAGE, pp.

135-148.

Johnson, V. (2003). Grade Inflation: A Crisis in College Education. New York:

Springer-Verlag.

  180  

Johnstone, D., & Marcucci, P. (2010). Financing Higher Education Worldwide,

Baltimore, MD: Johns Hopkins University Press.

Johnson, L., Adams, S., Cummins, M., Estrada, V., Freeman, A. & Ludgate, H.

(2013). The NMC Horizon Report: 2013 Higher Education Edition. Retrieved

September 15, 2013 from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/46484.

Johnstone, S. (2005) Open Educational Resources Serve the World. Educause

Quarterly, 28(3). Retrieved from

http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMag

azineVolum/OpenEducationalResourcesServet/157357

Kamenetz, A. (2010a). DIY U: Edupunks, Edupreneurs and the Coming

Transformation of Higher Education. White River Junction, VY: Chelsea

Green Publishing Co.

Kamenetz, A. (2010b, July 25). Eliminate Print Textbooks, Go Digital. The New York

Times. Retrieved from

http://www.nytimes.com/roomfordebate/2010/07/25/the-real-cost-of-college-

textbooks/eliminate-print-textbooks-go-digital

Kamenetz, A. (2012, September). The Coursera Effect. Fast Company, 98-103.

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1037694575?accounti

d=12629

Kaplan Leadership, 2014. http://www.kaplan.com/about-kaplan/leadership/.

Karamchandani, A., Kubzansky, M., & Lalwani, N. (2011). Is the Bottom of the

Pyramid Really for You? Harvard Business Review, March, 89(3), 107-111.

Retrieved from http://0-

web.b.ebscohost.com.prospero.murdoch.edu.au/ehost/results?sid=39e4c743-

589f-4876-99d8-

0cdc2cc39c3c%40sessionmgr113&vid=1&hid=127&bquery=IS+00178012+a

nd+TI+(%22Is+the+bottom+of+the+pyramid+really+for+you%22)+and+DT+

  181  

%222011%22+and+AU+%22Karamchandani%22&bdata=JmRiPWJ0aCZ0e

XBlPTEmc2l0ZT1laG9zdC1saXZl

Karlgaard, R. (2013, April 15). Why does america hate late bloomers? Forbes, 1.

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1324017518?accounti

d=12629

Keller, B. (2011, October 4). Tomorrow’s University: Free, Online, Everywhere. The

Straits Times, p. A19.

Kervin, L., Vialle, W., Herrington, J., & Okley, T. (2006). Research for Educators.

Melbourne: Cengage Learning.

Kevin, C. (2013, March 29). The Brave New World of College Branding. The

Chronicle of Higher Education. Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA323653051&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=c9eaf345ef114d819aff3c9430eec6fc

Khalil, H., & Ebner, M. (2013). How satisfied are you with your MOOC? - A

Research Study on Interaction in Huge Online Courses. In Jan Herrington et

al. (Eds.), Proceedings of World Conference on Educational Multimedia,

Hypermedia and Telecommunications 2013 (pp. 830-839). Chesapeake, VA:

AACE. Retrieved September 16, 2013 from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/112057.

Khan, S. (2011, July). YouTubeU Beats YouSnoozeU. The Chronicle of Higher

Education. Retrieved from http://chronicle.com

Kirschner, A. (2012, October 15). A Pioneer in Online Education Tries a MOOC. The

Chronicle of Higher Education, 59(06). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA304835420&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=331d588952411b5e8c2d4c927a9fa2a2

Kleining, G. (1982). Umriss zu einer Methodologie qualitativer Sozialforschung.

Koelner Zeitschrift fuer Soziologie und Sozialpsychologie, 34: 224-253.

  182  

Koch, J. V. (2010, 29 July). Like the Pharmaceutical Market. The New York Times.

Retrieved from http://www.nytimes.com/roomfordebate/2010/07/25/the-real-

cost-of-college-textbooks/how-colleges-profit-from-high-textbook-prices

Koller, D., & Ng, A. (2012). The online revolution: Education at scale. Retrieved

from https://www. aplu.org/document.doc?id=4055

Kolowich, S. (2014a, April 17). Udacity will no longer offer free cerificates. The

Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/udacity-will-no-longer-offer-free-

certificates/51757

Kolowich, S. (2014b, April 14). 2014. The Year the Media stopped caring about

MOOCs? The Chronicle of Higher Education, 14 April. Retrieved from

http://chronicle.com/blogs/wiredcampus/2014-the-year-the-media-stopped-

caring-about-moocs/51737

Kolowich, S. (2014c, March 24). Coursera Hires Former Yale President as Its Chief

Executive. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/article/Coursera-Hires-Former-Yale/145531/

Kolowich, S. (2014d, March 6). edX adds a second tier of members. The Chronicle of

Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/edx-adds-a-second-tier-of-

members/50891

Kolowich, S. (2014e, January 27). Stanford Economist: Elite Colleges should not give

credit for MOOCs. The Chronicle of Higher Education. Retrieve from

http://chronicle.com/blogs/wiredcampus/stanford-economist-elite-colleges-

should-not-give-credit-for-moocs/49811

Kolowich, S. (2014f, January 22). Completion rates aren’t the best way to judge

MOOCs, researchers say. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/completion-rates-arent-the-best-way-

to-judge-moocs-researchers-say/49721

  183  

Kolowich, S. (2014g, January 21). Coursera will offer certificates for sequences for

MOOCs. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/coursera-will-offer-certificates-for-

sequences-of-moocs/49581

Kolowich, S. (2014h, January 17). Credit-for-MOOCs effort hits a snag. The

Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/credit-for-moocs-effort-hits-a-

snag/49573

Kolowich, S. (2014i, January 13). George Siemens gets connected. The Chronicle of

Higher Education. Retrieved from http://chronicle.com/article/George-

Siemens-Gets-Connected/143959/

Kolowich, S. (2013a, February 7). American Council on Education Recommends 5

MOOCs for Credit. The Chronicle of Higher Education, 59(23). Retrieved

from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1287085055?accounti

d=12629

Kolowich, S. (2013b, May 17). As MOOC Debate Simmers at San Jose State,

American U. Calls a Halt. The Chronicle of Higher Education, 59(36).

Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA330654227&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=4cd497998db57d4793785f7b9b98f066

Kolowich, S. (2013c, February 1). 'Bill of Rights' Seeks to Protect Students' Interests

as Online Learning Expands. The Chronicle of Higher Education, 59(21).

Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA317580829&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=131539e38bfbf21089463838f61548cf

Kolowich, S. (2013d, May 10). Faculty Backlash Grows Against Online Partnerships.

The Chronicle of Higher Education, 59(35). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA328939437&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=1490469ade85d074026f94dfa236b405

  184  

Kolowich, S. (2013e, February 21). How EdX Plans to Earn, and Share, Revenue

From Free Online Courses. The Chronicle of Higher Education, 59(25).

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1312506715?accounti

d=12629

Kolowich, S. (2013f, May 3). Listening to Online Education's Ombudsman. The

Chronicle of Higher Education, 59(34). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA328939453&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=d53ad505d2c9769dc8dac90ae2f831c2

Kolowich, S. (2013g, May 3). Making His MOOC an 'Outreach for Poetry'. The

Chronicle of Higher Education, 59(34). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA328939447&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=266dec56ee858a6359689c38dbfab803

Kolowich, S. (2013h, March 22). The Professors Behind the MOOC Hype. The

Chronicle of Higher Education, 59(28). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA323653072&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=dea3206da984fbfd63a198d5f9ffa5d2

Kolowich, S. (2013i, April 29). Why Some Colleges Are Saying No to MOOCs, at

Least for Now. The Chronicle of Higher Education, 59(34). Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1347562186?accounti

d=12629

Kolowich, S. (2013j, April 8). Coursera takes a nuanced view of MOOC dropout

rates. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/coursera-takes-a-nuanced-view-of-

mooc-dropout-rates/43341

Kolowich, S. (2013k, February 4). Georgia Tech and Coursera Try to Recover from

MOOC Stumble. The Chronicle of Higher Education. Retrieved from

  185  

http://chronicle.com/blogs/wiredcampus/georgia-tech-and-coursera-try-to-

recover-from-mooc-stumble/42167

Kolowich, S. (2013l, November 20). MOOCs are largely reaching privileged learners,

survey finds. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/moocs-are-reaching-only-privileged-

learners-survey-finds/48567

Kolowich, S. (2013m, August 8). The MOOC ‘Revolution’ may not be as disruptive

as some had imagined. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/article/MOOCs-May-Not-Be-So-Disruptive/140965/

Kolowich, S. (2013n, September 18). MIT will offer MOOC curricula, not just single

courses, on edX. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/mit-will-offer-mooc-curricula-not-

just-single-courses-on-edx/46715

Kolowich, S. (2013o, July 8). A University’s Offer of Credit for a MOOC gets no

takers. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/article/A-Universitys-Offer-of-Credit/140131/

Kolowich, S. (2013p, August 1). California puts MOOC Bill on Ice. The Chronicle of

Higher Education. Retrieve from

http://chronicle.com/blogs/wiredcampus/california-puts-mooc-bill-on-

ice/45215

Kolowich, S. (2013q, December 13). Georgia Tech Designs its Udacity Pilot to Avoid

Failure. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/georgia-tech-designs-its-udacity-

pilot-to-avoid-failure/48947

Kong, L. (2012, April 18). KL govt in a bind over students’ sit-in protest. The Straits

Times, p. A14.

Kop, R. (2011). The challenges to connectivist learning on open online networks:

Learning experiences during a massive open online course. The International

Review of Research in Open and Distance Learning, Special Issue-

  186  

Connectivism: Design and Delivery of Social Networked Learning, 12(3), 19-

38. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/49670

Kop, R., Fournier, H., & Mak, J. S. F. (2011). A pedagogy of abundance or a

pedagogy to support human beings? Participant support on massive open

online courses. International Review of Research in Open and Distance

Learning, 12(7), 74-93. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/49389

Koutropoulos, A., Gallagher, M., Abajian, S., De Waard, I., Hogue, R., Keskin, N., &

Rodriguez, O. (2012). Emotive vocabulary in MOOCs: Context & participant

retention. European Journal of Open, Distance, and E-Learning. Retrieved

from http://www.eurodl.org/materials/contrib/2012/Koutropoulos_et_al.htm

Kuipers, R. (2012, August). Mental. Variety, 428, 38. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1082005187?accounti

d=12629

Krause, S. (2013). MOOC response about "listening to world music". College

Composition and Communication, 64(4), 689-695. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1369719742?accounti

d=12629

Krug, E. A. (1956). Administering Curriculum Planning. New York: Harper and

Row.

Kurtzleben, D. (2013, June). Three ways MOOCs will change colleges. U.S.News &

World Report, 1. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1412863812?accounti

d=12629

Labaree, D. (1997). How to succeed in school without really learning: the credentials

race in American education. New Haven: Yale University Press.

  187  

Lack, K. A. (2013). Current status of research on online learning in postsecondary

education. Retrieved from http://apo.org.au/sites/default/files/docs/Ithaka-

sr_OnlineLearningPostSecondaryEducation_May2012.pdf

Lamb, B., & Groom, J. (2010). The Open Ed Tech: Never Mind the Edupunks; or,

The Great Web 2.0 Swindle. Educause Review, 45(4), 50-52. Retrieved from

http://eric.ed.gov/?id=EJ951240

Lawton, W., Ahmed, M., Angulo, T., Axel-Berg, A., & Burrows, A. (2013). Horizon

Scanning. What will higher education look like in 2020? The Observatory on

Borderless Higher Education, September, Research series 12. Retrieved from

http://www.voced.edu.au/content/ngv59844

Lawton, W., & Katsomitros, A. (2012). MOOCs and disruptive innovation: the

challenge to HE business models. Retrieved from

http://hdl.voced.edu.au/10707/241343

Lawton, W., & Lunt, K. (2013, January 23). Would you credit that? The trajectory of

the MOOC’s juggernaut. The Observatory on Borderless Higher Education.

Retrieved from http://www.voced.edu.au/content/ngv55971

Last byte. (2013). Learning & Leading with Technology, 40, 48. Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA333842422&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=c641b11691d32b56900a928eee96a4f2

Learning new lessons; free education. (2012, December 22). The Economist, 405,

101-102. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1243347048?accounti

d=12629

Leber, J. (2013, January). The technology of massive open online

courses. Technology Review, 116, 63-64. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1269702250?accounti

d=12629

  188  

Leckart, S. (2012, March 20). The Stanford Education Experiment could change

higher learning forever. Wired. Retrieved from

http://www.wired.com/2012/03/ff_aiclass/3/

Legewie, H. (1994). Globalauswertung. In: A. Boehm, T. Muhr & A. Mengel (eds.),

Texte verstehen: Konzepte, Methoden, Werkzeuge. Konstanz:

Universitaetsverlag, pp. 100-114.

Letters. (2013, June). Toronto Life, 47, 20. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1365236212?accounti

d=12629

Lewin, T. (2012, November 16). College Credit Eyed for Online Courses. The New

York Times. Retrieved from

http://www.case.edu/strategicplan/downloads/CollegeCreditEyedforOnlineCo

urses-NYTimes.pdf

Lewin, T., & Markoff, J. (2013, January 15). “California to Give Web Courses a Big

Trial”. The New York Times. Retrieve from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1269210042?accounti

d=12629

Lim, C. (2012, March 30). eBooks’ gilt edge. The Business Times. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/961310334?accountid

=12629

Little, G. (2013). Massively Open? The Journal of Academic Librarianship, 39(3),

308-309. doi: http://dx.doi.org/10.1016/j.acalib.2013.03.004

Locke, M. (2013). MOOC WILL THESE FOUR LETTERS CHANGE K-

12? Scholastic Administr@tor, 12, 39-40. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1362030801?accounti

d=12629

López-Isa, S. (2013, February 04). Esquina editorial. The Hispanic Outlook in Higher

Education, 23, 4. Retrieved from http://0-

  189  

search.proquest.com.prospero.murdoch.edu.au/docview/1326333542?accounti

d=12629

Lyons, D. (2012, May 14). Cheaper than harvard. Newsweek, 159. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1011571307?accounti

d=12629

Lytle, R. (2012, November 1). Antioch university to offer online coursera courses for

credit. U.S.News & World Report, 1. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1245085356?accounti

d=12629

Lytle, R. (2012, August). EdX announces free online courses for fall

launch. U.S.News & World Report, 1. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1077242508?accounti

d=12629

Lytle, R. (2012, July). More universities join free online education

provider. U.S.News & World Report, 1. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1034717381?accounti

d=12629

Ma, J., Lee, K., & Kuo, G. (2013). A massive open online course on

pharmacogenomics: not just disruptive innovation but a possible solution.

Pharmacogenomics, 14(10), 1125-1127. doi:10.2217/pgs.13.97

Mahraj, K. (2012). Using information expertise to enhance massive open online

courses. Public Services Quarterly, 8(4), 359-368. doi: Using information

expertise to enhance massive open online courses

Mangan, K. (2012). Massive Excitement About Online Courses; It's raising big

questions about the future of higher education. The Chronicle of Higher

Education, 59(06). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1082420607

  190  

Mann, C., & Stewart, F. (2000). Internet Communication and Qualitative Research: A

Handbook for Researching Online. London: SAGE.

Marcus, J. (2012, December 21). Online course start-ups offer virtually free college.

The Washington Post. Retrieved from

http://www.washingtonpost.com/local/education/online-course-startups-offer-

virtually-free-college/2012/01/09/gIQAEJ6VGQ_story.html

Margerum, L., Gulsrud, M., Manlapez, R., Rebong, R., & Love, A. (2007).

Application of Calibrated Peer Review (CPR) writing assignments to enhance

experiments with an environmental chemistry focus. Journal of Chemical

Education, 82(2), 292-295. doi: 10.1021/ed084p292

Markoff, J. (2011, August 15). Virtual and artificial, but 58,000 want course. The New

York Times. Retrieved from

http://www.nytimes.com/2011/08/16/science/16stanford.html?_r=1.

Markoff, J. (2013, April 4). Essay-grading software offers professors a break. The

New York Times. Retrieved from

http://www.nytimes.com/2013/04/05/science/new-test-for-computers-grading-

essays-at-college-level.html

Massis, B. (2013). MOOCs and the library. New Library World, 114(5/6), 7-7. doi:

10.1108/03074801311326894.

Massy, W. & Zemsky, R. (1994). Faculty Discretionary Time: Departments and the

Academic Ratchet. Journal of Higher Education 65, 1-22. doi:

10.2307/2943874

Marovich, B. (2012, September 7). More Than MOOC's. The Chronicle of Higher

Education, 59(02). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA302048574&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=e7cd58be7d6625896af8abf17090fc68

Martin, F. (2012). Will massive open online courses change how we teach?

Communications of the ACM, 55(8), 26-28. Retrieved from http://0-

dl.acm.org.prospero.murdoch.edu.au/citation.cfm?id=2240246

  191  

Mayring, P. (1997). Qualitative Inhaltsanalyse. Grundlagen und Techniken. 7th ed.

Weinheim: Deutscher Studien Verlag.

McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). The MOOC model for

digital practice. Retrieved from

https://oerknowledgecloud.org/sites/oerknowledgecloud.org/files/MOOC_Fin

al_0.pdf

McClusky, F., & Winter, M. (2012). The Idea of the Digital University: Ancient

Traditions, Disruptive Technologies and the Battle for the Soul of Higher

Education. Washington, DC: Westphalia Press.

McGregor, D. (1960). The Human Side of Enterprise. McGraw-Hill, New York.

McKinlay, J. B. (1995). Towards Appropriate Levels: Research Methods and Healthy

Public Policies. In: I. Guggenmoos-Holzmann, K. Bloomfield, H. Brenner &

U. Flick (eds.), Quality of Life and Health: Concepts, Methods, and

Applications. Berlin: Basil Blackwell, pp. 161-182.

Mehaffy, G. (2012). Challenge and Change. Educause Review, 47(5), 25-42.

Retrieved from

http://online.tarleton.edu/fdi/Documents/EDUCAUSE_Mehaffy.pdf

McLoughlin, C. (2013). The pedagogy of personalised learning: exemplars, MOOCS

and related learning theories. In Jan Herrington et al. (Eds.), Proceedings of

World Conference on Educational Multimedia, Hypermedia and

Telecommunications 2013 (pp. 266-270). Chesapeake, VA: AACE. Retrieved

September 16, from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/111968.

Menchaca, M., & Hoffman, E. (2013). Changing with the Times: Designing Online

Programs for Contemporary Students. In Jan Herrington et al.

(Eds.), Proceedings of World Conference on Educational Multimedia,

Hypermedia and Telecommunications 2013 (pp. 767-773). Chesapeake, VA:

AACE. Retrieved September 16, from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/112047.

  192  

Meyer, J., & Zhu, S. (2013). Fair and equitable measurement of student learning in

MOOCs: An introduction to item response theory, scale linking, and score

equating. Research & Practice in Assessment, 8(1), 26-39. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1505317097?accounti

d=12629.

Miles, M., & Huberman, A. (1994). Qualitative Data Analysis: A Sourcebook of New

Methods, 2nd ed. Newbury Park, CA: SAGE.

Milman, N. (2012). MOOCs: what are they? Plus 20 questions we should be asking

about them. Distance Learning, 9(4), 91-93. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1372088500?accounti

d=12629.

Mishler, E. (1990). Validation in Inquiry-related Research: The Role of Exemplars in

Narrative Studies. Harvard Educational Review, 60: 415-442.

Mittell, J. (2013, March 8). The Real Digital Change Agent. The Chronicle of Higher

Education, 59(26). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA322020931&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=e2957418b4ed78ad9caa32d66f54ae2d

Mobiles for Development (2013). A MOOC by Indian Institute of Technology,

Kanpur, and Commonwealth of Learning. Retrieved from

http://m4d.colfinder.org/

MOOC adapts its teaching to individual learning styles. (2013). E.Learning Age, 4.

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1355488165?accounti

d=12629

Monaghan, P. (2012, September 14). What's Next in Online Learning? Stanford

Names Vice Provost to Lead the Way. The Chronicle of Higher Education,

59(03). Retrieved from

  193  

http://go.galegroup.com/ps/i.do?id=GALE%7CA302662522&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=7fca96c36404d3ff16b259fc4fdd563d

Morrow, D., & Soni, G. (2013). A look back as we look ahead: An analysis of a

statewide online professional development initiative. In R. McBride & M.

Searson (Eds.), Proceedings of Society for Information Technology & Teacher

Education International Conference 2013 (pp. 813-818). Chesapeake, VA:

AACE. Retrieved September 16, from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/48214.

Mosher, D. (2012, May 1). “Google’s Sebastian Thrun. Three Visions in the ‘Age of

Disruption’”. Wired. Retrieved from http://www.wired.com/2012/05/google-

sebastian-thrun-future/.

Murphy, E., & Dingwall, R. (2001). “The Ethics of Ethnography”. In P. Atkinson, A.

Coffey, S. Delamont, J. Lofland (eds.), Handbook of Ethnography. London:

SAGE, pp. 339-351.

Murray, A. (2013). Running aMOOC? Massive Open Online Courses. Distant

Learning, 10(2), 11-18. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1399327197?accounti

d=12629.

National Council of Teachers of English. (2013). Machine scoring fails the test.

NCTE Position Statement on Machine Scoring. Retrieved from

http://www.ncte.org/positions/statements/machine_scoring

Nelson, S. R. (2013, March 29). The Soul of Invention. The Chronicle of Higher

Education, 59(29). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA323653046&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=616410d3bc07c44629d500bf0172ef49

New, J. (2013, May 3). Fighting to Reinvent Teaching and Keep Costs Down. The

Chronicle of Higher Education, 59(34). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA328939445&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=33108470bf20708f130a9a34331c7b7d

  194  

Nisen, M. (2013, February 12). Clay Christensen: Higher Education Is 'On The Edge

Of The Crevasse. Retrieved from http://www.businessinsider.com/clay-

christensen-higher-education-on-the-edge-2013-2 - comments12

Noer, M. (2012, November 19). Reeducating education. Forbes, 190(9), 84-100.

Retrieved from https://0-

global.factiva.com.prospero.murdoch.edu.au/ga/default.aspx

Obama, B. (2010). Remarks of President Barack Obama. Address to Joint Session of

Congress. The White House. Retrieved from

http://www.whitehouse.gov/the_press_office/remarks-of-president-barack-

obama-address-to-joint-session-of-congress

Ong, H. (2013, September 9). NUS online modules to give NSmen an early start. The

Straits Times, p. A2.

Open Educational Resources (2012). Wikipedia. Retrieved from

http://en.wikipedia.org/wiki/Open_educational_resources

Opening learning. (2013, August 01). The Economist (Online), Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1417197210?accounti

d=12629

Open Sesame. (2011, January 06). The Economist. Retrieved from

http://www.economist.com/node/17857297

Orchowski, P. (2013, May 13). UNCENSORED. The Hispanic Outlook in Higher

Education, 23, 23. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1355690787?accounti

d=12629

Orchowski, P. (2013, February 04). Mock into democracy at CGS conference. The

Hispanic Outlook in Higher Education, 23, 23. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1326333538?accounti

d=12629

  195  

Osman, S. (2012, April 24). Free university education is costly. The Straits Times, p.

A18.

Ostrander, R. (2012, October 5). Learning to Surf: A Christian College's Provost

Encounters the Digital Revolution. The Chronicle of Higher Education,

59(06). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA304835417&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=bbc74762287eeb968c472bb8abd41589

Ouchi, W. (1981). Theory Z: How American management can meet the Japanese

challenge. New York: Addison-Wesley.

OU MOOC signs up partners across the globe. (2013, July). E.Learning Age, 4.

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1428803969?accounti

d=12629

P2PU. (2012). Announcing Exciting New Courses at P2PU. Retrived from

http://blogs.p2pu.org/blog/2012/04/17/announcing-exciting-new-courses-at-

p2pu/

O’Donnell, J. (2012, September 3). The Future Is Now, and Has Been for Years. The

Chronicle of Higher Education, 59(2). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1038136427?accounti

d=12629.

O’Donoghue, T. (2007). Planning your Qualitative Research Project. An Introduction

to Interpretivist Research in Education. London: Routledge.

Oevermann, U., Allert, T., Konau, E., & Krambeck, J. (1979). Die Methodologie

einer ‘objectiven Hermeuneutik’ und ihre allgemeine forschungslogische

Bedeutung in den Sozialwissenschaften. In H. G. Soeffner (ed.), Interpretative

Verfahren in den Sozial-und Textwissenschaften. Stuttgart: Metzler, pp. 352-

433.

Paldy, L. (2013). MOOCs in your future. Journal of College Science Teaching, 42(4),

6-7. Retrieved from http://0-

  196  

search.proquest.com.prospero.murdoch.edu.au/docview/1323903599?accounti

d=12629

Pantò, E., & Comas-Quinn, A. (2013). The Challenge of Open Education. Journal of

e-Learning and Knowledge Society, 9(1), 11-22. Retrieved September 15,

from http://0-www.editlib.org.prospero.murdoch.edu.au/p/43328.

Pappano, L. (2012, November 4). The year of the MOOC. New York Times. Retrieved

from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1151371049?accounti

d=12629

Paré, D., & Joordens, S. (2008), Peering into large lectures: examining peer and

expert mark agreement using peerScholar, an online peer assessment tool.

Journal of Computer Assisted Learning, 24: 526–540. doi: 10.1111/j.1365-

2729.2008.00290.x

Perry, C. (2012a). “A structured approach to presenting theses: notes for students and

their supervisors”. Circulated draft, courtesy of C. Perry.

Perry, C. (2012b). “Convergent interviewing: a starting methodology for a research

program”. Unpublished draft, courtesy of C. Perry.

Parry, M. (2010, August 29). 'Open Teaching': When the World Is Welcome in the

Online Classroom; Experimenters say diversity means richness. The Chronicle

of Higher Education, 57(02). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA236101855&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=006e213e68a3117ad3ccf7eaf7cde882

Parry, M. (2012, October 5). 5 Ways That edX Could Change Education. The

Chronicle of Higher Education, 59(06). Retrieved

fromhttp://go.galegroup.com/ps/i.do?id=GALE%7CA304835425&v=2.1&u=

murdoch&it=r&p=AONE&sw=w&asid=8ce2ffabea4191f59a6c8ad4ac4a2e17

Peddycord, Barry III (2014, February 12). Why it makes sense for students to grade

another’s papers. The Chronicle of Higher Education, 12 February. Retrieved

  197  

from http://chronicle.com/blogs/future/2014/02/12/why-it-makes-sense-for-

students-to-grade-one-anothers-papers/

Pelaez, N. (2001). Calibrated peer review in general education undergraduate human

physiology. In P. Rubba, J. Rye, W. DiBiase, & B. Crawford (Eds.),

Proceedings of the Annual International Conference of the Association for the

Education of Teachers in Science (pp. 1518-1530). Costa Mesa, CA.

Penn State (2014). Symposium for Teaching and Learning with Technology.

Retrieved from http://symposium.tlt.psu.edu/wp-

content/uploads/sites/1800/2012/11/George-Siemens-Photo.jpg

Petit, F. (2009). Red Alert… Red Alert: A Disruptive Innovation Needed in the

Executive MBA Market. Review of Business Research, 9(3), 148-155.

Retrieved from

http://web.b.ebscohost.com/abstract?direct=true&profile=ehost&scope=site&a

uthtype=crawler&jrnl=15462609&AN=45462912&h=eT0GKA5e8gFF6hFzp

cVwcM%2f3uofXgFEgfYqDHwJ3AjA20I4bBzJPI77yyjpswCskRpLog6fsC

%2fcXvd%2fGVo00ng%3d%3d&crl=c

Popenici, S., & Kerr, S. (2013). What undermines higher education: and how this

impacts employment, economies and our democracies. Macquarie University

ResearchOnline. Retrieved from

http://www.researchonline.mq.edu.au/vital/access/manager/Repository/mq:25

915

Popp, T. (2013, March/April). MOOC U. The Pennsylvania Gazette, 58-63. Retrieved

from http://www.upenn.edu/gazette/0313/PennGaz0313_feature4.pdf

Porterfield, D. (2013, May 3). 2013: Year of the Seminar. The Chronicle of Higher

Education, 59(34). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA328939454&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=33108470bf20708f130a9a34331c7b7d

Prahalad, C. (2004). The Fortune at the Bottom of the Pyramid. Wharton School

Publishing. Retrieved from

  198  

Prahalad, C., & Hart, S. (2002). The Fortune at the Bottom of the Pyramid. Strategy +

Business. Retrieved from www.cs.berkeley.edu/~brewer/ict4b/Fortune-

BoP.pdf

Prahalad, C., & Mashelkar, R. (2010, May). Innovation’s Holy Grail. Harvard

Business Review, 132-141. Retrieved from

http://repository.ias.ac.in/63026/1/25_aut.pdf

Pressler, J. (2012, May 14). The mooch is his own aflac duck. New York, Retrieved

from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1011304388?accounti

d=12629

Pricer, W. (2013). At issue: MOOCs, an annotated webliography. The Community

College Enterprise, 19(1), 55-60. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1416739188?accounti

d=12629

Punch, K. (1998). Introduction to Social Research. Quantitative and Qualitative

Methods. London: Sage.

Rao, S., & Perry, C. (2007). “Convergent interviewing: a starting methodology for an

enterprise research program”. In D. Hine, and D. Carson. Innovative

Methodologies in Enterprise Research. Northampton, Massachusetts: Edward

Elgar, n/a (quoted from draft version, courtesy of C. Perry).

Regalado, A. (2013, January). Online courses put pressure on universities in poorer

nations. Technology Review, 116, 64-65. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1269702258?accounti

d=12629

Ravi. (2012, September 1). Technology can serve millions. Digital

Learning, Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1038812408?accounti

d=12629

  199  

Republic Polytechnic. (2014). Dr. Samson Tan. Retrieved from

http://www.rp.edu.sg/PBLInstitute/PBLSeminar2014/Programme/SamsonTan.

aspx

Reynolds, G. (2014). The New School: How the Information Age will save American

Education from itself. New York: Encounter Books.

Rice, J. (2013). What I learned in MOOC. College Composition and Communication,

64(4), 695-703. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1369719753?accounti

d=12629

Richardson, J. (2010, December 2010 / January 2011). Disrupting How and Where

We Learn. An Interview with Clayton Christensen and Michael Horn.

kappanmagazine.org, 32-38

Ripley, A. (2012, October 29). College is Dead. Long Live College! Time in Udacity

Press Kit (2013).

Rivard, R. (2013a, April 8). Coursera begins to make money. InsideHigherEd.com.

Retrieved from http://www.insidehighered.com/news/2013/04/08/coursera-

begins-make-money#sthash.6Z3hWDG5.dpbs

Rivard, R. (2013b, July 18). Udacity Project on ‘Pause’. Inside Higher Ed. Retrieved

from http://www.insidehighered.com/news/2013/07/18/citing-disappointing-

student-outcomes-san-jose-state-pauses-work-udacity#sthash.TKG7T7uq.dpbs

Rizzardini, R., Chang, V., Gütl, C., & Amado-Salvatierra, H. (2013). An Open Online

Course with Accessibility Features. In Jan Herrington et al.

(Eds.), Proceedings of World Conference on Educational Multimedia,

Hypermedia and Telecommunications 2013 (pp. 635-643). Chesapeake, VA:

AACE. Retrieved September 15, 2013 from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/112023.

Rodriguez, O. (2012). MOOCs and the AI-Stanford like Courses: two successful and

distinct course formats for massive open online courses. European Journal of

  200  

Open, Distance, and E-Learning, 1-13. Retrieved from

http://files.eric.ed.gov/fulltext/EJ982976.pdf

Rogers, C. (2006). Faculty perceptions about e-cheating during online testing. Journal

of Computing Sciences in Colleges, 22(2), 206-212. Retrieved from

http://dl.acm.org/citation.cfm?id=1181936

Rollins, T. (2014, January 20). MOOCs: Been there, done that. The Chronicle of

Higher Education, Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1491100262?accounti

d=12629

Roth, M. (2013, May 3). My Modern MOOC Experience. The Chronicle of Higher

Education, 59(34). Retrieved from http://0-

go.galegroup.com.prospero.murdoch.edu.au/ps/i.do?action=interpret&id=GA

LE%7CA328939446&v=2.1&u=murdoch&it=r&p=AONE&sw=w&authCou

nt=1

Rosen, A. (2011). Change.edu. Rebooting for the new talent economy. New York:

Kaplan Publishing.

Rossi, R., & Mustaro, P. (2013). Perspectives of Quality and Accreditation of MOOC.

In R. McBride & M. Searson (Eds.), Proceedings of Society for Information

Technology & Teacher Education International Conference 2013 (pp. 983-

990). Chesapeake, VA: AACE. Retrieved September 15, 2013 from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/48243.

Rowe, N. (2004). Cheating in online student assessment: Beyond plagiarism. Online

Journal of Distance Learning Administration, 7(2). Retrieved from

http://www.westga.edu/~distance/ojdla/summer72/rowe72.html

Rudolph, J. (2014). LinkedIn profile. Retrieved from

http://www.linkedin.com/profile/view?id=142375395&trk=nav_responsive_ta

b_profile

  201  

Rudolph, J. (2013, November 5). Are MOOCs disruptively innovating Higher

Education? PowerPoint presentation, presented to BUS378 Knowledge and

Organisational Learning (Murdoch University) students at Kaplan Singapore.

Rudolph, J. (2012a). Disruptive Online Innovations in Higher Education. In: The

International Journal of Professional Management, 6.1. Retrieved from

http://ipmajournal.com.

Rudolph, J. (2012b). The “100,000-student classroom”. Disruptive Curriculum

Innovation in Higher Education. In: The International Journal of Professional

Management, 7.3. Retrieved from http://ipmajournal.com.

Russell, A. (2005). Calibrated Peer ReviewTM: A writing and critical-thinking

instructional tool. In Invention and impact: Building excellence in

undergraduate science, technology, engineering and mathematics (STEM)

education. Washington DC: American Association for the Advancement of

Science.

Russell, A., Chapman, O., & Wegner, P. (1998). Molecular science: Network-

deliverable curricula. Journal of Chemical Education, 75(5), 578-579.

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/211933513

Russell, E. (2013). University credits, tuition free. Canadian Medical Association

Journal, 185(9), E395-6. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1413335086?accounti

d=12629

Rutter, M. (2014, January 21). Harvard and MIT release working papers on open

online learning. Retrieved from

http://news.harvard.edu/gazette/story/2014/01/harvard-and-mit-release-

working-papers-on-open-online-learning/

Saadatmand, M., & Kumpulainen, K. (2013). Content Aggregation and Knowledge

Sharing in a Personal Learning Environment: Serendipity in Open Online

Networks. International Journal of Emerging Technologies in Learning

  202  

(iJET), 8(2013), 70-77. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/45231.

Sadler, P., & Good, E. (2006). The Impact of Self-and Peer-Grading on Student

Learning. Educational Assessment, 11(1): 1-31.

doi:10.1207/s15326977ea1101_1

Salas, A. (2013a, May 27). Presenters at E-learning conference talk EdTech

trends. The Hispanic Outlook in Higher Education, 23, 8-11. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1370166690?accounti

d=12629

Salas, A. (2013b, February 4). THE EMERGENCE of FREE ONLiNE

EduCATiON. The Hispanic Outlook in Higher Education, 23, 14-15.

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1326333543?accounti

d=12629

Sandeen, C. (2013). Assessment’s place in the new MOOC world. Research &

Practice in Assessment, 8(1), 5-12. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1505317186

Sangrà, A., & Wheeler, S. (2013). New informal ways of learning: Or are we

formalising the informal? RUSC, 10(1), 286-293. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1348597201?accounti

d=12629

Saxberg, B. (2014). LinkedIn profile. Retrieved from

http://www.slideshare.net/gsiemens/open-education-2013

Saylor Foundation. (2012). Retrieved from http://www.saylor.org/about/

Schrire, S., & Levy, D. (2012). Troubleshooting MOOCs: The Case of a Massive

Open Online Course at a College of Education. In T. Amiel & B. Wilson

(Eds.), Proceedings of World Conference on Educational Multimedia,

Hypermedia and Telecommunications 2012 (pp. 761-766). Chesapeake, VA:

  203  

AACE. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/40832.

Schuman, R. (2013, November 19). The King of MOOCs abdicates the throne.

Sebastian Thrun and Udacity ‘pivot’ toward corporate training. Slat. Retrieved

from

http://www.slate.com/articles/life/education/2013/11/sebastian_thrun_and_uda

city_distance_learning_is_unsuccessful_for_most_students.html

Schwier, R. (2009). Pursuing the Elusive Metaphor of Community in Virtual

Learning Environments. In G. Siemens & C. Fulford (Eds.), Proceedings of

World Conference on Educational Multimedia, Hypermedia and

Telecommunications 2009 (pp. 3072-3082). Chesapeake, VA: AACE.

Retrieved September 16, 2013 from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/31918.

Selingo, J. (2013a). College (un) bound: The Future of Higher Education and what it

Means for Students. New York, Boston: Houghton Mifflin Harcourt

Selingo, J. (2013b, April 8). Colleges Must Prepare for a Buyer's Market. The

Chronicle of Higher Education, 59(31). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1325038908

Sharif, A., Moeini, H., & Gisbert, M. (2013). Quality of Online Learning: Adding

MOOC into the mix?. In . Jan Herrington et al. (Eds.), Proceedings of World

Conference on Educational Multimedia, Hypermedia and Telecommunications

2013 (pp. 922-928). Chesapeake, VA: AACE. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/112071.

Sharing a Passion for Poetry With Students Around the World. (2012, June 22). The

Chronicle of Higher Education, 58(39). Retrieved from

http://chronicle.com/section/About-the-Chronicle/83

Sharma, G. (2013, July 15). A MOOC Delusion: Why Visions to Educate the World

are Absurd. The Chronicle of Higher Education. Retrieved from

  204  

http://chronicle.com/blogs/worldwise/a-mooc-delusion-why-visions-to-

educate-the-world-are-absurd/32599

Sharma, Y. (2013, April 26). Hong Kong MOOC Draws Students from Around the

World. The Chronicle of Higher Education, 59(33). Retrieved from

http://chronicle.com/section/About-the-Chronicle/83

Shermis, M., Burstein, J., Higgins, D., & Zechner, K. (2010). Automated essay

scoring: Writing assessment and instruction. In E. Baker, B. McGaw, & N.S.

Petersen (Eds.), International encyclopedia of education (3rd ed., pp. 75–80).

Oxford, England: Elsevier.

Siemens, G. (2014, April 14). Open Learning Analytics. Retrieved from

http://www.elearnspace.org/blog/

 Siemens, G. (2013). MOOCs: How did we get here? PowerPoint presentation.

Retrieved from http://www.slideshare.net/gsiemens/open-education-2013

Siemens, G., & Gasevic, D. (2012). Guest Editorial-Learning and Knowledge

Analytics. Educational Technology & Society, 15(3), 1-2. Retrieved from

http://0-

go.galegroup.com.prospero.murdoch.edu.au/ps/i.do?id=GALE%7CA3062406

29&v=2.1&u=murdoch&it=r&p=AONE&sw=w

Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and

education. Educause Review, 46(5), 30-32. Retrieved from

http://www.elmhurst.edu/~richs/EC/OnlineMaterials/SPS102/Teaching%20an

d%20Learning/Penetrating%20the%20Fog.pdf

Siemens, G., & Tittenberger, P. (2009). Handbook of emerging technologies for

learning. Manitoba, Canada: University of Manitoba.

Siemens, G., Tittenberger, P., & Anderson, T. (2008). Conference Connections:

Rewiring the Circuit. Educause Review, 43(2), 14-16. Retrieved from

http://www.editlib.org/p/102247

  205  

Simonson, M. (2012). MOOC Madness. Distant Learning, 9(4), 104-105. Retrieved

from http://0-

go.galegroup.com.prospero.murdoch.edu.au/ps/i.do?id=GALE%7CA3278161

71&v=2.1&u=murdoch&it=r&p=AONE&sw=w

Skiba, D. (2012). Nursing Education Perspectives, 33(6), 416-417. Retrieved from

http://0-search.proquest.com.prospero.murdoch.edu.au/docview/1269079895

Skiba, D. (2013a). MOOCs and the future of nursing. Nursing Education

Perspectives, 34(3), 202-4. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1370894459?accounti

d=12629

Skiba, D. (Ed.). (2013b). On the Horizon: The Year of the MOOCs. Nursing

education perspectives, 34(2), 136-137. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1350295198

Snyder, M. (2012). Much ado about MOOCs. Academe, 98(6), 55. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1266785261?accounti

d=12629

Smith, C., & Wright, A. (2012). Ecologies for Open Learning: TOOLS at the

University of Windsor. In T. Bastiaens & G. Marks (Eds.), Proceedings of

World Conference on E-Learning in Corporate, Government, Healthcare, and

Higher Education 2012 (pp. 1734-1739). Chesapeake, VA: AACE. Retrieved

from http://0-www.editlib.org.prospero.murdoch.edu.au/p/41858.

Stannard, R. (2010, April 24). #loveHE: A wide-open web of potential. Times Higher

Education (London). Retrieved from

http://www.timeshighereducation.co.uk/story.asp?storycode=411353

Strauss, A., & Corbin, I. (2008). Basics of Qualitative Research. 3rd ed. London:

SAGE.

Sumell, A. (2013, March 29). I Don't Want to Be Mooc'd. The Chronicle of Higher

Education, 59(29). Retrieved from http://0-

  206  

go.galegroup.com.prospero.murdoch.edu.au/ps/i.do?id=GALE%7CA3236530

44&v=2.1&u=murdoch&it=r&p=AONE&sw=w

Tan, S. (2014). LinkedIn profile. Retrieved from

http://www.linkedin.com/pub/samson-tan/15/3a5/1a1

The attack of the MOOCs. (2013, July 9). The Economist, pp. 55.

The Cape Town Open Education Declaration (2007). Retrieved from

http://www.capetowndeclaration.org/read-the-declaration

The Maturing of the MOOC. (2013). Department for Business, Innovation and Skills,

1-123. Retrieved from

https://www.gov.uk/government/uploads/system/uploads/attachment_

data/file/240193/13-­‐1173-­‐maturing-­‐of-­‐the-­‐mooc.pdf

Theisen, T. (2013). New spaces new realities: Expanding learning any time, any

place. Foreign Language Annals, 46(2), 141-142. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1425423205?accounti

d=12629

Thibault, J. (2011). 241 OER Courses with Assessments in Moodle: How Saylor.org

has created one of the largest Free and Open Course Initiatives on the web.

Moodlenews.com. Retrieved from http://www.moodlenews.com/2012/250-

oer-courses-and-assessments-in-moodle-how-saylor-org-has-created-one-of-

the-largest-open-course-initiatives-on-the-web/

Thirunarayanan, M. (2014, January 23). MOOCs must follow established academic

practices. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/letters/moocs-must-follow-established-academic-

practices/

Thompson, T., & Purdy, J. (2009). When a Good Idea Isn’t Enough: Curricular

Innovation as a Political Process. Academy of Management Learning and

Education, 8(2), 188-207. doi: 10.5465/AMLE.2009.41788842

  207  

Thrun, S. (2014a, January 15). World's First Massive Online Degree Program Starts

Today. Retrieved from http://blog.udacity.com/2014/01/sebastian-thrun-

worlds-first-massive.html

Thrun, S. (2014b, April 16). Phasing out certificates of free courseware completion.

Retrieved from http://blog.udacity.com/

Trow, M. (1970). Reflections on the transformation from mass to universal higher

education. Daedalus 99, 1-42.

Tschofen, C., & Mackness, J. (2012). Connectivism and dimensions of individual

experience. The International Review of Research in Open and Distance

Learning, 13(1), 124-143. Retrieved from

http://www.jstor.org/discover/10.2307/20023931?uid=3738992&uid=2&uid=

4&sid=21104093621643

Tyson, L. (1936). “Ten Years of Educational Broadcasting” School and Society 44,

225-31.

Udacity. (2014). What we offer. [University website]. Retrieved from

https://www.udacity.com/what-we-offer

Udacity Press Kit (2013). Retrieved from https://www.udacity.com/press#

Udacity. (2012). [University website]. Retrieved from

http://www.udacity.com/courses

Unbiased and unloved; CNN's woes. (2012, September 22). The Economist, 404, 72.

Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1069228366?accounti

d=12629

Underwood, J., & Szabo, A. (2003). Academic offences and e-learning: Individual

propensities in cheating. British Journal of Educational Technology, 34(4),

467-477. doi:10.1111/1467-8535.00343

US Sues Apple Over E-book Price Fixing. (2012, April 12). The Straits Times, p.

A30.

  208  

Uvalić-Trumbić, S., & Daniel, J. (2014). MOOCs in the Arts and Humanities: What

are the issues? Symposium:   MOOCs   in   the   Arts   and   Humanities:  

Opportunities, Challenges and Implications Across UK Higher Education,

University of Central Lancashire, Preston, UK. Retrieved from

http://sirjohn.ca/wordpress/?page_id=29

Vázquez-Cano, E. (2013). The videoarticle: New reporting format in scientific

journals and its integration in MOOCs. Comunicar, 21(41), 83-90. Retrieved

from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1428056407?accounti

d=12629

Vedder, R. (2011, September 14). Saylor as Savior? Forbes. Retrieved from

http://www.forbes.com/sites/ccap/2011/09/14/saylor-as-savior/

Vedder, R. (2012, January 23). Beware: Alternative Certification Is Coming. The

Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/innovations/beware-alternative-certification-is-

coming/31369

Veletsianos, G., & Kimmons, R. (2012). Assumptions and challenges of open

scholarship. International Review of Research in Open and Distance

Learning, 13(4), 166-189. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/49315.

Vogel, C. (2012). EdX inches towards 1 billion students. Boston Magazine. Retrieved

from http://www.bostonmagazine.com/news/blog/2012/10/19/edx-inching-1-

billion-students/

Voss, B. (2013a). MOOCs: Get in the Game. Educause Review, January – February,

58-59.

Voss, B. (2013b). Massive Open Online Courses (MOOCs): A Primer for University

and College Board Members. An AGB White Paper. Retrieved from

http://mynkuhelp.nku.edu/content/dam/StrategicPlanning/docs/OnlineEducati

onandInnovation/AGBreport_2013_MOOCs.pdf

  209  

Watters, A. (2013, October 15). “A Future with only 10 Universities (Minding the

Future, #OpenVA)”. Blog post. Retrieve November 12, from

http://hackeducation.com/2013/10/15/minding-the-future-openva/

Webley, K. (2012, September 4). MOOC brigade: Will massive, open online courses

revolutionize higher education? Time. Retrieved from

http://nation.time.com/2012/09/04/mooc-brigade-will-massive-open-online-

courses-revolutionize-higher-education/

Weil, E. (2010, July 29). How students really buy books. The New York Times.

Retrieved from http://www.nytimes.com/roomfordebate/2010/07/25/the-real-

cost-of-college-textbooks/how-students-really-buy-textbooks

White, T. (2014). Beauty, Form, and Function: An Exploration of Symmetry. NTU

MOOC on Coursera. Retrieved from https://www.coursera.org/course/ntusym

Wiley, D., & Hilton III, J. (2009). Openness, Dynamic Specialization, and the

Disaggregated Future of Higher Education. International Review of Research

in Open & Distance Learning, 10(5).

Williams, P. (2013, July). Brave new worlds. E.Learning Age, 1. Retrieved from

http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1428803673?accounti

d=12629

Wilson, R. (2010). For-Profit Colleges Change Higher Education’s Landscape. The

Chronicle of Higher Education, 56(22), A1-A19. Retrieved from

http://www.chronicle.com/article/For-Profit-Colleges-Change/64012

Woods, P. (1983). Sociology and the School. London: Routledge and Kegan Paul.

‘World’s Cheapest Computer’ launched in India (2011, October 06). The Straits

Times, A25

Yin, R. (1994). Case Study Research: Design and Methods. Sage: Beverly Hills.

  210  

Young, J. (2013a). Alternative Networks Challenge Colleges' Role in Alumni's Job

Searches. The Chronicle of Higher Education, 59(18). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA314249428&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=ab1f3e32177ba1f07fb513ff7be4f00d

Young, J. (2013b, January 15). California State U. Will Experiment With Offering

Credit for MOOCs. The Chronicle of Higher Education, 59(20). Retrieved

from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1275122171?accounti

d=12629

Young, J. (2013c, January 15). Don't Call Them Textbooks. The Chronicle of Higher

Education, 59(21). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1275122171?accounti

d=12629

Young, J. (2013d, May 14). Georgia Tech to Offer a MOOC-Like Online Master's

Degree, at Low Cost. The Chronicle of Higher Education, 59(37). Retrieved

from http://chronicle. com/article/Ga-Tech-to-Offer-a-MOOC-Like/139245/

Young, J. (2012e, July 19). Inside the Coursera Contract: how an upstart company

might profit from free courses. The Chronicle of Higher Education. Retrieved

from

http://www.immagic.com/eLibrary/ARCHIVES/GENERAL/CHRON_HE/C1

20719Y.pdfhttp://www.immagic.com/eLibrary/ARCHIVES/GENERAL/CHR

ON_HE/C120719Y.pdf

Young, J. (2013f, April 29). Learning From Big Business. The Chronicle of Higher

Education, 59(34). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1347569592?accounti

d=12629

Young, J. (2013g, March 8). The Bandwidth Divide. The Chronicle of Higher

Education, 59(26). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA322020937&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=e8dc4038fc9fe9aabeca4ea0f0f06679

  211  

Young, J. (2013h, May 20). What Professors Can Learn From 'Hard Core' MOOC

Students. The Chronicle of Higher Education, 59(37). Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1353489367?accounti

d=12629

Young, J. (2013i, January 9). Coursera Announces Details for Selling Certificates and

Verifying Identities. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/blogs/wiredcampus/coursera-announces-details-for-

selling-certificates-and-verifying-identities/41519

Youngberg, D. (2012, August 17). Why Online Education Won't Replace College.

The Chronicle of Higher Education, 58(44). Retrieved from

http://go.galegroup.com/ps/i.do?id=GALE%7CA300340610&v=2.1&u=murd

och&it=r&p=AONE&sw=w&asid=a861473c46d496deb6cf51df83ebdb33

Yuan, L., Powell, S., & Cetis, J. (2013). MOOCs and Open Education: Implications

for Higher Education. Cetis White Paper. Retrieved March 21, 2014, from

http://pdf.thepdfportal.com//?id=101588&nocache.

Zais, R. (1976). Curriculum: principles and foundations. New York: Thomas Y.

Crowell.

Zikmund, W., Babin, B., Carr, J., & Griffin, M. (2010). Business Research Methods,

8th ed. International Student Edition: South-Western, Cengage Learning.

Zhang, Y. (2013). Benefiting from MOOC. In. Jan Herrington et al.

(Eds.), Proceedings of World Conference on Educational Multimedia,

Hypermedia and Telecommunications 2013 (pp. 1372-1377). Chesapeake,

VA: AACE. Retrieved from http://0-

www.editlib.org.prospero.murdoch.edu.au/p/112136.

50 Disruptive Companies 2013. (2013, March). Technology Review, 116, 26-44, 46-

52. Retrieved from http://0-

search.proquest.com.prospero.murdoch.edu.au/docview/1323012359?accounti

d=12629