Faculty of Cognitive Sciences and Human Development design and development 24pgs.pdf · appreciation to my mentor Behrang Parhizkar (Hani) who introduced me to augmented reality,

Faculty of Cognitive Sciences and Human Development

The Design and Development of Android Mobile Augmented Reality

Technology for Kindergarten Science Subject

Mohamed Abdulkarim Mohamedali

Master of Science

2016

The Design and Development of Android Mobile Augmented

Reality Technology for Kindergarten Science Subject

Mohamed Abdulkarim Mohamedali

A thesis submitted

In fulfilment of the requirements for the degree of Master of Science (Cognitive

Science)

Faculty of Cognitive Sciences and Human Development

UNIVERSITI MALAYSIA SARAWAK

2016

The project entitled ‘The Design and Development of Android Mobile Augmented

Reality Technology for Kindergarten Science Subject.’ was prepared by Mohamed

Abdulkarim Mohamedali and submitted to the Faculty of Cognitive Sciences and

Human development in fulfillment of the requirements for a Degree of Master of

Sciences (Cognitive Science).

i

DECLARATION

I declare that the work in this thesis was carried out in accordance with the regulations of

Universiti Malaysia Sarawak. It is original and is the result of my work, unless otherwise

indicated or acknowledged as referenced work. This thesis has not been accepted for any

degree or not concurrently submitted in candidature of any other degree.

Name of Student: Mohamed Abdulkarim Mohamedali

Student ID No: 11021724

Degree: Master of Science

Faculty: Faculty of Cognitive Sciences and Human Development

Thesis Title: The Design and Development of Android mobile Augmented Reality

Technology for kindergarten science subject.

Signature of Student:

Date : 27/05/2016

ii

ACKNOWLEDGEMENTS

My parents deserve special mention for their inseparable support and prayers. They

gave me hope when I needed it the most. Their encouragement to go on is what kept me

going. I would like to express my very great appreciation to my elder brother Moez. I would

not have been able to get here with the ease that I did, for if it were not for him. My fiancé

Maria deservers a special mention as well; her love and persistent confidence in me, has

taken the load off my shoulder during final and the most difficult times of my study.

I would like to express my deep my gratitude to Dr. Ng Giap Weng for his

supervision, advice, and guidance from the very early stage of this research as well as giving

me extraordinary experiences throughout the work. Above all and the most needed, he

provided me unflinching encouragement and support in various ways. I wish to acknowledge

the help provided by Mr. Jubair for his valuable and constructive input during the planning

and development of this research work. His willingness to give his time so generously has

been very much appreciated.

I would also like to express my special thanks, words fail me to express my

appreciation to my mentor Behrang Parhizkar (Hani) who introduced me to augmented

reality, and whose passion for the learning had lasting effect to further my education.

Finally, I would like to thank everybody who was important to the successful

realization of thesis, as well as expressing my apology that I could not mention personally

one by one.

iii

ABSTRACT

This study focuses primary on the development of Augmented Reality (AR) application,

Kindergarten Science Education Based on Android Mobile Augmented Reality Technology,

aka KSAMART, for early childhood education (ECE) in Malaysia. Secondarily, it attempts to

theorize in general, perception in their context of use, and in particular. Broadly, this work

takes a pragmatic viewpoint to understand what works and solution to problems of adoption

and diffusion of technology in early childhood education. An explanatory sequential

quantitative data collection will be used to addresses the intent of this work. The fruits of this

research include novelty of AR application for ECE in Malaysia, and an original contribution

to research literature in the learned Body of Knowledge in AR and Education. In the first

experiment, a quantitative experiment was conducted to determine the usability of AR

application. Given, students are generally still at early stages of physical and mental

development, teachers are recruited as their surrogates. It is a quantitative experiment.

Participants were administered with usability survey. That survey was sufficiently informed

by the work in usability engineering. Second quantitative experiment is conducted to assess

participants learning with KSAMART. In general, direct results of these experiments hinted

at AR application as a usable application in the context of use, particularly in terms of

learning transfer in kindergarten science education. AR application has an important role in

the processes of adoption and diffusion of technology in early children education in Malaysia.

Keywords: Augmented reality, early childhood education

iv

Rekabentuk dan Pembangunan Augmented Realiti Teknologi Mudah Alih Android untuk

Subjek Sains Tadika

ABSTRAK

Kajian ini memberi tumpuan utama pada pembangunan aplikasi Augmented Realiti (AR),

Tadika Pendidikan Sains Berdasarkan Android Mobile Augmented Realiti Technologi, aka

KSAMART, untuk pendidikan awal kanak-kanak (ECE) di Malaysia. Tumpuan kedua ialah ia

cuba untuk membuat teori secara umum, persepsi dalam konteks mereka penggunaan, dan

dalam kebolehgunaan tertentu dan pemindahan pembelajaran. Secara am, kerja-kerja ini

mengambil pandangan pragmatik untuk memahami apa yang bekerja dan penyelesaian

kepada masalah penerimaan dan penyebaran teknologi dalam pendidikan awal kanak-kanak.

An berurutan pengumpulan data kuantitatif penerangan akan digunakan untuk alamat niat

kerja ini. Hasil kajian ini termasuk sesuatu yang baru permohonan AR untuk ECE di

Malaysia, dan sumbangan asal untuk maklumat penyelidikan dalam Ilmu Perbadanan AR

dan Pendidikan. Dalam eksperimen pertama, satu eksperimen kuantitatif telah dijalankan

untuk menentukan kebolehgunaan aplikasi AR. Diberikan, murid-murid secara umumnya

masih di peringkat awal pembangunan fizikal dan mental, guru direkrut sebagai

pembantu/pengganti mereka. Ia adalah satu eksperimen kuantitatif. Para responden telah

ditadbir dengan kajian kebolehgunaan, kajian yang telah cukup dimaklumkan oleh kerja-

kerja dalam bidang kejuruteraan kebolehgunaan. Eksperimen kuantitatif kedua dijalankan

untuk menilai peserta pembelajaran dengan KSAMART. Secara umum, kesan langsung

daripada eksperimen ini membayangkan permohonan AR sebagai aplikasi yang boleh

digunakan dalam konteks penggunaan, terutamanya di tern pemindahan dalam pendidikan

sains tadika pembelajaran. permohonan AR mempunyai peranan yang penting dalam proses

penerimaan dan penyebaran teknologi pada awal pendidikan kanak-kanak di Malaysia.

Kata kunci: Aplikasi Augmented Realiti, awal pendidikan kanak-kanak

v

TABLE OF CONTENTS

DECLARATION i

ACKNOWLEDGEMENTS ii

ABSTRACT iii

ABSTRAK iv

TABLE OF CONTENTS v

LIST OF TABLES xi

LIST OF FIGURES xiii

CHAPTER 1: INTRODUCTION 1

1.0 Introduction 1

1.1 Motivation 3

1.2 Purpose Statement 4

1.3 Research Questions 5

1.4 Research Objectives of this work 6

1.5 Definition of Terms 6

1.6 Summary 8

vi

CHAPTER 2: LITERATURE REVIEW 11

2.1 Augmented Reality: The State of Art 10

2.1.1 Augmented Reality 10

2.1.2 Mobile Augmented Reality 11

2.1.3 Impact of Technology on Learning 12

2.1.4 Mobile Learning is Still in Its Infancy 13

2.1.5 A Child at Play is Exercising Freedom Along Five Distinct Axes 13

2.1.6 Affordance Analysis: Design Methodology for matching Learning Tasks

with learning Technology 14

2.1.7 Three Main Affordance of AR 16

2.1.8 Augmented Reality Learning Experience in Language and Communication:

a Parallel Study to This Work 17

2.1.9 IEEE Project: P1589 - Standard for an Augmented Reality Learning

Experience Model (IEEE SA Standard for an Augmented Reality

Learning Experience Model, 2014) 19

2.2 Human-Computer Interaction 20

2.2.1 Designing User Interface for Children 21

2.2.2 AR Interaction Techniques 22

2.2.3 Designing for Usability 23

2.2.4 Usability Evaluation 24

2.2.5 AR Usability Evaluation 25

2.3 Augmented reality SDKS 26

2.3.1 Comparison of Augmented reality SDKS 27

2.3.1.1 ARToolKit 27

2.3.1.2 D'Fusion 29

vii

2.3.1.3 Metaio 31

2.3.1.4 Vuforia 33

2.3.2 Comparative analysis of the augmented reality SDKs 35

2.3.2.1 Tracking ability 35

2.3.2.2 Operating system (OS) support and graphics 36

2.3.2.3 Storage ability 37

2.3.2.4 Documentation support 37

2.3.2.5 Licensing 38

2.3.2.6 Ease of development 39

2.3.3 Analysis of SDK’s 39

2.4 Summary 40

CHAPTER 3: METHODOLOGY 41

3.1 Design Study 41

3.2 Subjects 42

3.3 Location 42

3.4 Materials 43

3.4.1 Pilot Test 43

3.4.2 Usability Test 44

3.5 Equipment 45

3.6 System Design and Development 46

3.7 Procedures 48

3.7.1 Pilot Test 48

3.7.2 Usability Test 49

viii

3.8 Data Collection 50

3.9 Data Analysis 50

3.10 Summary 50

CHAPTER 4: DESIGN AND DEVELOPMENT 51

4.1 Design 51

4.1.1 Designed Task 51

4.1.2 System Layout 52

4.1.3 System Architecture 55

4.1.4 System Specifications 59

4.1.5 Collection of Resources 60

4.2 Development 61

4.2.1 System Development 61

4.2.1.1 Video Capturing 61

4.2.1.2 Image Tracking 62

4.2.1.3 Display Rendering 68

4.2.2 Construction and Modification of Collected Resources 79

4.2.3 Testing 80

4.2.4 System Modification 80

4.3 Summary 81

CHAPTER 5: FINDINGS AND DISCUSSION 82

5.0 Introduction 82

5.1 Results 82

ix

5.1.1 Experimental Results to Test Case Specification Identifier 83

5.1.2 Experimental Results to Pilot Test Questionnaires 94

5.1.3 Experimental Results to Usability Questionnaires 97

5.1.3.1 User Background Information 97

5.1.3.2 Experimental Results to Usability Questionnaires 99

5.1.3.3 Usability reliability and satisfaction of kindergarten science

application 100

5.1.3.4 Overall system Performance of kindergarten science application 106

5.1.3.5 Satisfaction of kindergarten science application 110

5.1.3.6 Additional feedback of kindergarten science application 113

5.2 Discussion 116

5.2.1 Discussion of Results (Section 5.1) 116

5.2.2 Discussion of Methodology 117

5.2.3 Discussion of Literature 117

5.2.4 Discussion of Related Research 118

5.2.5 Discussion of features integrated into the application 118

5.3 Summary 120

CHAPTER 6: CONCLUSION 121

6.0 Overview 121

6.1 Contribution and Significance of the Study 121

6.2 Recommendations for Future Research and Studies 122

6.3 Summary of the Research 123

x

REFERENCES 125

APPENDICES 134

xi

LIST OF TABLES

Table 2.1 Tracking ability comparisons 36

Table 2.2 Operating system support and graphics comparisons 36

Table 2.3 Storage ability comparisons 37

Table 2.4 Documentation support comparisons 38

Table 2.5 Licensing comparisons 38

Table 2.6 Ease of development comparisons 39

Table 3.1 Study Design 41

Table 5.1 Respondents’ successes or failures in performing the pilot testing with the

KSAMART Unit 1 94

Table 5.2 Respondents’ successes or failures in performing the pilot testing with the

KSAMART Unit 2 96

Table 5.3 Background information for Group 2 98

Table 5.4 Smartphone usage information for Group 2 98

Table 5.5 Cronbach’s Alpha coefficient for the questionnaire 99

Table 5.6 Responses to the questionnaires with regards to the Usability 101

Table 5.7 Descriptive statistics for the “Usability” construct 105

Table 5.8 Responses to the questionnaires with regards to performance 106

Table 5.9 Descriptive statistics for the “Effectiveness and efficiency” construct 110

Table 5.10 Responses to the questionnaires with regards to satisfaction 111

xii

Table 5.11 Descriptive statistics for the “User satisfaction” construct 112

Table 5.12 Responses to Question 1 and 2 for Section C 113

Table 5.13 Responses to Question 3 - 7 for Section C 114

xiii

LIST OF FIGURES

Figure 2.1 Situated Vocabulary Learning. Source 17

Figure 2.2 Sample Interface for Situated Vocabulary Learning 19

Figure 3.1 System Development Life Cycle (SDLC) 47

Figure 4.1 Reading a Book 52

Figure 4.2 System Layout 54

Figure 4.3 KSAMART System Architecture 56

Figure 4.4 Android OS Architecture 58

Figure 4.5 A Screenshot of KASMART 63

Figure 4.6 Algorithm KSAMART 64

Figure 4.7 Pseudo Code for KSAMART 65

Figure 4.8 Algorithm for Color Picking Algorithm 66

Figure 4.9 Snippet of Reading Alpha Value of Object 67

Figure 4.10 Texture Mapping of Virtual Object 69

Figure 4.11 3D Model 72

Figure 4.12 Text Rendering on Image 73

Figure 4.13 3D Model Animation 74

Figure 4.14 GUI in KSAMART 75

Figure 4.15 Video Playback after Detect Book Image 76

xiv

Figure 4.16 Audio Playback after Detecting Book Image 77

Figure 4.17 Touch Interaction on virtual object 79

Figure 5.1 Virtual Object in real world 84

Figure 5.2 2D Virtual Object 85

Figure 5.3 Texturing Mapping 86

Figure 5.4 3D Model 87

Figure 5.5 Text rendering on the image 88

Figure 5.6 Series of 3D model Animations 89

Figure 5.7 GUI in mobile kindergarten AR app 90

Figure 5.8 Audio playback after detect the book image 91

Figure 5.9 Screenshot of video playback after detect the book image 92

Figure 5.10 Touch Interaction on virtual object 93

1

CHAPTER 1

INTRODUCTION

This work focuses primarily on the development of an Augmented Reality (AR) application

on the one hand, and secondarily, perception (more specifically, usability and learning

transfer) in the context of use, on the other. Augmented Reality is defined as ‘the fusion of

any digital information within real world settings, i.e. being able to augment one’s immediate

surroundings with electronic data or information, in a variety of media formats that include

not only visual/graphic media but also text, audio, video and haptic overlays’ (Elizabeth,

Anne, Rebecca, Mark, Yishay, & Rhodri, ; 2012).

AR application of this work, namely Kindergarten Science Education Based on

Android Mobile Augmented Reality Technology, aka KSAMART, is targeting at early

childhood education (ECE) in Malaysia. Moreover, it is a prototyped variation of augmented

reality learning experiences (ARLEs) (Santos, Yamamoto, Taketomi, Miyazaki, &

Kato, ;2013 and Learning Technology Standards Committee (2010) (PDF), “Draft Standard

for Learning Object Metadata.” IEEE Standard, IEEE P1484.12.1–2002/Cor 1/D13.) Besides,

it is derived mainly from merging, assimilating and evolving from multi-disciplinary Body of

Knowledge of Augmented Reality, Human Computer Interaction and Education.

1.0 Introduction

There was a large body of literature that studies the development of Augmented Reality

applications. They included (a) the first Augmented Reality system of Sutherland, (1968), (b)

the first Audio Augmented Reality of Bederson, (1995), and (c) the first Mobile Augmented

Reality of Loomis, Golledge, & Klatzky, (1993). These efforts necessarily were seeing AR

2

application as an information processing tool (Orlikowski, & Iacono; 2001), and KSAMART

is taking the same direction.

On the other hand, Veas, Mendez, Feiner, & Schmalstieg, (2011) attempted to

draw target user’s attention to particular objects in the real world without distracting target

users from performing their task. They applied a saliency modulation technique to video.

These researchers were taking a proxy view or value of AR application. In the same vein,

KSAMART is based on Mobile Augmented Reality Technology. It is best conceptualized as

a augmented reality learning experiences (ARLE), an information processing learning tool

with a repository of information that can be searched and manipulated for learning

Kindergarten Science Education in Malaysia.

KSAMART attempts to build and extend earlier works. They include the

works of (a) Evans, Horowitz, Howe, Pedersini, Reijnen, Pinto, Kuzmova1, & Wolfe; (2011)

who described ‘visual attention’ as a set of mechanisms that limit some processing to a subset

of incoming stimuli. Attentional mechanisms shape what we see and what we can act upon.

Moreover, (b) Levinson and Majid (2014) saw language as ‘digital’ compared for example to

the analogue nature of gesture. In addition, (c) Munley, (2012) provided a literature review

on museum-based learning by young children against traditional home and school settings.

Finally, (d) Chi, (2008) saw learning of complex material, such as concepts encountered in

science classrooms, can occur under at least three different conditions of prior knowledge. (i)

A student may have no prior knowledge of the to-be-learned concepts, although they may

have some related knowledge. (ii) A student may have some correct prior knowledge about

the to-be-learned concepts, but that knowledge is incomplete. In this incomplete knowledge

case, learning can be conceived of as gap filling. Finally, (iii) a student may have acquired

ideas, either in school or from everyday experience, that are “in conflict with” the to-be-

learned concepts (conceptual change). This Section is followed by sections for Motivation

3

(Section 1.1), Purpose Statement (Section 1.2), Research Question (Section 1.3), Research

Objective (Section 1.4), and finally Definition of Terms (Section 1.5).

1.1 Motivation

Hwang, G. J & & Tsai, C. (2011) examined 3,995 papers related to Mobile Learning

published in the Social Science Citation Index (SSCI) database from 2001 to 2010 for the

purpose of analyzing the research trend. That included six major technology-based learning

journals, namely the British Journal of Educational Technology (BJET), Computers and

Education (C&E), Educational Technology and Society (ETS), Educational Technology

Research and Development (ETR&D), Journal of Computer Assisted Learning (JCAL) and

Innovations in Education and Teaching International (IETI). Generally, one of their major

finding was that predominant samples of mobile and ubiquitous learning research were

targeted at students of higher education and elementary schools. It follows that research

similar to this kind of study (ECE) was under represented or overlooked.

On the other hand, Bacca, Silvia Baldiris, Fabregat, Sabine Graf, & Kinshuk (2014)

performed a systematic review study of AR and application with a focus on investigating

factors such as: the uses, advantages, limitations, effectiveness, challenges and features of

augmented reality in educational settings. Their work is based on the recommendations of the

Preferred Reporting Items for Systematic Reviews and Meta-Analyses, PRISMA (Moher,

Liberati, Tetzlaff, & Altman; 2009). They located and analyzed 32 studies published between

2003 and 2013 in 6 indexed journals. They found no evidence of AR applications in the field

“Early childhood education” (0%). A possible explanation of this result is that the technology

could not be ready for being used by children since many aspects of interaction, such as the

tracking and use of markers, need to be solved.

4

Moreover, Elizabeth, Anne, Rebecca, Mark, Yishay, & Rhodri; (2012) discovered

‘The use of AR in education, and particularly mobile learning, is still in its infancy and it

remains to be seen how useful it is for creating effective learning experiences’. Importantly,

there was a significant need to initiate Mobile Learning in Malaysia (Mohamed Amin &

Embi Norazah Nordin; 2013). Mobile Learning was identified as one of the Critical Agenda

Projects (CAPs) and Key Result Area (KRA) of Ministry of Higher Education (MOHE) of

Malaysia (Mohamed Amin Embi, Norazah Mohd Nordin & Ebrahim Panah; 2013).

Deficiencies in the past literature merit further study and the gap in knowledge must be

fulfilled. Consequently, it is a case for the development of an Augmented Reality (AR)

application, aka KSAMART, for early childhood education in Malaysia. KSAMART, AR

application of this work speaks for itself. Their contents, Kids AR Book @ Unit 1 and Kids

AR Book @ Unit 2 (Appendix 7) were targeting at children of ECE going age group

(henceforth, students), taking into consideration providers of education (henceforth, teachers)

and their parents.

1.2 Purpose Statement

Recall the primary focus of this work is on the development of an Augmented Reality (AR)

application and secondary, perception in the context of use. Taking a pragmatic viewpoint to

understand actions and situations, an explanatory sequential quantitative data collection will

be used to addresses the intent of this work. The researcher collects quantitative data (refer

Research Question One and Research Question Two) and analyzes the results.

5

1.3 Research Questions

In order to narrow focus the above mentioned purpose statement, this study asks:

(a) Research Question One:

How useful is KSAMART?

In this experiment, a quantitative test instrument Test Case Specification Identifier (Appendix

1) was administered to determine the usefulness of KSAMART as an independent variable. It

will be performed by the researcher oneself.

(b) Research Question Two:

How easy is to use and how easy is to learn KSAMART?

In this experiment, quantitative instruments such as (i) Pilot Test Questionnaire (Appendix 2)

administered to ten respondents (Group One), and (ii) Usability Test Questionnaire

(Appendix 5) administered to thirty respondents (Group Two). They will be used to evaluate

and measure the usability of KSAMART, for better understanding of what motivates them to

accept this AR application. Note that given students were at their early stages of physical and

mental development, therefore teachers were commissioned as their surrogates. Usability

studies might be found in papers targeting mobile learning for students of higher education

and elementary schools (Hwang, G. J & & Tsai, C.; 2011); however, little or none usability

studies have been conducted at this level or category, that is studies targeting early childhood

education in Malaysia. A need existed to explore and describe that phenomenon. This work is

novel.

6

1.4 Research Objectives of this work

(a) Objective One:

To develop AR application, KSAMART, see also Section 1.3. (a) Research Question One

above.

(b) Objective Two:

To evaluate and determine usability of KSAMART, see also Section 1.3. (b) Research

Question Two above.

1.5 Definition of Terms

• Augmented Reality (AR) – AR is a technology that enhances the users’ senses by

augmenting a virtual layer on the real world (Aaltonen & Lehikoinen, 2006).

• Smartphone – Smartphone is a mobile phone that offers more advanced computing

ability and connectivity than a contemporary feature phone.

• Multimedia Content – Multimedia means audio/video visuals environment where

audio, video, 3D model, animations, text and such will be included as multimedia content.

• Virtual object – The computer generated graphics especially 3D computer graphics

which uses three-dimensional representation of geometric information. Virtual object can be

developed by using computer programming language.

• Tracking – Image (Marker) can be tracked by the system to overlay virtual objects on

image.

• Interactions – Interactions is a technique to control the virtual objects by hand gesture.

7

• Toolkit- A toolkit is an assembly of tools; set of basic building units for graphical user

interfaces.

• Library- a collection of useful material for common use.

• System- is a set of interacting or interdependent components forming an integrated

whole.

• Effectiveness of the Applications – The users evaluated the features of the application

(Kindergarten kids AR learning application) by performing tasks (refer Appendix 2) and

answering the questionnaires (refer Appendix 3). The effectiveness of the systems was

determined based on the users’ success in completing a list of tasks and the feedback from

answering the questionnaires.

• Efficiency of the Applications – The efficiency of the applications were identified

based on users’ feedback towards the two applications by answering the questionnaires (refer

Appendix 4). The perspective of the users in term of appropriateness of time to generate an

action is used to determine the efficiency of the applications.

• Satisfaction with Applications – Satisfaction with applications were identified based

on users’ feedback towards the application. The users evaluated the features of the

applications by answering the questionnaires (refer Appendix 4) after they performed the

tasks.