Virtual Reality and its Impact on Programming Learning Process

Virtual Reality and its Impact on Programming Learning Process

Designing VR-based Programming Learning Practices

Graziella Sundblad

Interaction Design Bachelor 22.5HP Spring/2018 Supervisor: Elisabet Nilsson

2018-01-04 (Dnr. ) 2 (av 61)

Abstract This qualitative thesis work investigates in which way an immersive, embodied and interactive computer-based simulation can lead to an easier understanding of programming concepts. It also presents a concept for VR programming learning tool that could turn learning into a fun and engaging experience for the students. The target group is Interaction Design bachelor students – programming is an important tool to create innovative and interactive artefacts and interfaces, and yet, the students have hard time to understand many of the programming concepts. The current research relies on concepts such as embodiment and tangibility, which reflects on the prototype developed – a highly immersive, embodied VR platform with a strong illusion of tangibility. Additionally, the prototype was inspired by feedback from students and teachers, result of a participatory approach. Finally, the research showed through usability tests that the students experienced that the application made the concepts more graspable and more fun to learn.

Keywords: Virtual Reality, Embodiment, Immersion, Programming, Tangibility, Co-design, Interaction Design, Learning, Cognitive Science.

2018-01-04 (Dnr. ) 3 (av 61)

Acknowledgements I would like to thank all participants in interviews and usability testing for taking time and effort in making this thesis project possible and inspiring me throughout the designing and writing process. I would also like to express my sincere gratitude to my supervisor Elisabet Nilsson for all the guidance, encouragement and awesome ideas, and to my Jedi Grand Master Prof. Daniel Spikol, whose insight, knowledge and support were invaluable in this process.

I would also to extend my thanks to my friend and “partner in crime”, Jakob Håkonsson, who has always been helpful and made this arduous process more enjoyable and less scary.

Finally, I cannot thank enough to my amazing husband and family for all the reinforcement and patience throughout this whole thesis work process and for believing in me.

Thank you!

2018-01-04 (Dnr. ) 4 (av 61)

Abbreviations and Acronyms 2D: two-dimensional

3D: three-dimensional

AR: Augmented Reality

CS: Computer Science

GUI: Graphical User Interface

K-12: From Kindergarten to 12th grade

STEAM: Science, Technology, Engineering, Arts and Mathematics

STEM: Science, Technology, Engineering and Mathematics

TEI: Tangible and Embodied Interaction

TUI: Tangible User Interface

VR: Virtual Reality

XR: Mixed Reality

2018-01-04 (Dnr. ) 5 (av 61)

TableofContents1 INTRODUCTION.......................................................................................6 1.1 Aim and research questions ................................................................................. 7 1.2 Target group......................................................................................................... 7 2 BACKGROUND..........................................................................................9 2.1 Embodied learning ............................................................................................... 9 2.2 Gamification and Student motivation ................................................................ 10

2.2.1 Gamification ................................................................................................... 11 2.2.2 Motivation ...................................................................................................... 11

2.3 Virtual Reality .................................................................................................... 13 2.3.1 Immersion vs Presence ................................................................................... 13 2.3.2 Tangibility within VR ....................................................................................... 14

2.4 Related work ...................................................................................................... 15 2.4.1 Gamified Programming education – From 2D to 3D ......................................... 16 2.4.2 Commercial examples of VR learning tools ...................................................... 20

3 METHODS................................................................................................23 3.1 Design Process Model ........................................................................................ 23 3.2 Literature Review ............................................................................................... 24 3.3 Sketching............................................................................................................ 25 3.4 Co-design workshop ........................................................................................... 25 3.5 Interview ............................................................................................................ 26 3.6 Prototyping ........................................................................................................ 27 3.7 Usability test ...................................................................................................... 27 3.8 Ethical implications ............................................................................................ 28 4 DESIGNPROCESS...................................................................................29 4.1 An overview ....................................................................................................... 29 4.2 Sketching............................................................................................................ 30 4.3 Brainstorming .................................................................................................... 31 4.4 Co-design workshop ........................................................................................... 32 4.5 Interview ............................................................................................................ 34

4.5.1 Interview with teachers .................................................................................. 34 4.5.2 Interviews with students ................................................................................. 36

4.6 Prototyping ........................................................................................................ 38 4.6.1 Technical Specifications .................................................................................. 38 4.6.2 Design prototyping ......................................................................................... 38

4.7 Usability Test ..................................................................................................... 43 4.7.1 Usability test 1 ................................................................................................ 43 4.7.2 Usability test 2 ................................................................................................ 46

5 RESULTS..................................................................................................48 6 DISCUSSION............................................................................................50 6.1 Significance of the Study .................................................................................... 50 6.2 Challenges .......................................................................................................... 51 6.3 Self-critique ........................................................................................................ 51 6.4 Implications for future research ......................................................................... 52 7 REFERENCES...........................................................................................54 8 APPENDIX...............................................................................................60

2018-01-04 (Dnr. ) 6 (av 61)

1 Introduction

Technology and computers are very much at the core of our economy going forward. To be prepared for the

demands of the 21st century—and to take advantage of its opportunities—it is essential that more of our students

today learn basic computer programming skills, no matter what field of work they want to pursue.

(Todd Park – Former U.S. Chief Technology Officer, in CODE, n.d.).

In the 21st century, we experience a society in which digital technologies are deeply-rooted in our everyday lives and people are expected to be producing individuals, rather than mere consumers. That means digital literacy and programming skills become vital in an economy that is technology-driven, no matter what people’s professional goals in life are (Kalelioʇlu, 2015; Schmidt, 2016).

Nevertheless, according to the European Commission, while the demand of programming skills and number of open vacancies within Information and Communication Technology are growing at a steady pace, there are not enough people with programming skills to meet the growing demand (Moreno-León & Robles, 2015).

Aware of these issues, an increasing number of countries have been undertaking school curriculum adjustments that imply digital literacy and programming become compulsory subjects from Elementary school through High school. A forerunner in Europe when it comes to programming education from early years is the United Kingdom. In the UK, there is an initiative to offer to all kids in school year 7 the BBC micro:bit computing platform, which makes easier for students to create computing applications. Micro:bit also provides a solid educational base for learning about programming concepts. This initiative is a partnership between educational authorities, the BBC, and hardware/software companies (Schmidt, 2016, p. 6). In Sweden, similarly to other Europeans countries, there has been an growing emphasis on digital literacy both at primary and secondary school. In March 2017 the Swedish government decided on including programming in the school curriculum, that will be a part of Mathematics and Technology discipline. This measure implies that Math teachers from first grade to High school are supposed to teach programming/computer science associated to what they learn in Math (Heintz & Mannila, 2018). Nevertheless, by examining the directive documents concerning the curriculum changes (Skolverket, 2017), and interviewing K-12 teachers, it can be said that there is no suggestion on how these changes should be put into practice.

2018-01-04 (Dnr. ) 7 (av 61)

What about those who are soon embarking onto the labor market and did not have the chance to learn programming from early school years? That is the case of Interaction Design students, who need programming skills as a tool to create interactive artefacts and interfaces. How are they supposed to learn complex concepts in such a short period of time? Many of them never get to develop programming skills that meet the demands of the labor market. What can be done to facilitate the understanding of programming concepts and motivate the students to learn programming?

In the last few years, many scientific articles have been written tackling programming education in STEM-related university programs (Science, Technology, Engineering and Mathematics), and the use of VR as a learning tool. However, there is a research gap when it comes to the field of Arts, recently integrated into STEM education (Taljaard, 2016). In other words, there is a lack of research on VR impact on the field of Arts, which encompasses Interaction Design.

Even though there is a considerable amount of research on the use of VR in education, the effects of these new forms of interaction on conceptual understanding remain underresearched (Lindgren et al., 2016)

1.1 Aim and research questions This research aims at examining in which way a more immersive, embodied and interactive computer-based simulation can lead to an experienced easier understanding of programming concepts.

Moreover, a second aim is to present a concept for VR-based Programming platform that could turn learning into a fun and engaging experience for the students, and in this case, students in Interaction Design.

In order to achieve these goals, this thesis work will shed light on the following questions:

– In what ways can VR-based learning practices can facilitate the understanding of computer science concepts/programming and;

– In what ways can it be motivational and engaging for the students?

These questions will be answered by designing and presenting a concept involving VR as a learning tool, which will be inspired by, but not solely based upon, co-design sessions.

1.2 Target group The target group of this work consists of Interaction Design bachelor students at Malmö University. Programming skills are considered very important, since it enables us to create interactive, user-friendly, appealing and innovative artefacts and interfaces. In other words, programming is one of Interaction Designers’ main tools, in the same way a hammer is an import tool for a blacksmith.

2018-01-04 (Dnr. ) 8 (av 61)

Yet many students have difficulties in understanding even basic concepts of Programming, something I also observed as Teaching Assistant in Programming II.

In the years I have been studying Interaction Design, there has been a high failure and dropout rate in Programming-related subjects, a problem that needs to be addressed.

Additionally, teachers involved in Programming subjects are also important stakeholders in this research, since their positive attitude toward new technologies is fundamental for a successful implementation.

2018-01-04 (Dnr. ) 9 (av 61)

2 Background In this chapter, relevant theories and concepts within learning and VR will be presented. Furthermore, some theoretical and empirical work concerning the use of VR as a learning tool in STEAM-related fields will be presented.

2.1 Embodied learning A central concept in this research is embodiment, which became relevant in cognitive science since the mid-1980’s (Ziemke, 2003) and can be understood as the representation of knowledge and concepts through body movements (Lindgren & Glenberg, 2013).

Recently, there is a growing literature that supports engaging the body in learning activities. Advocates of embodied learning contends that how we think and reason about the world is greatly coupled to our sensorimotor system and the body’s interaction with the physical world (Lindgren et al., 2016). According to Lakoff and Johnson (1999), even when decoupled from the environment, mechanisms developed for interaction with it play a role in the cognitive process

Goldin-Meadow et al. (2001) also advocate the use of body movements in learning. According to these authors, both gesture and speech form an integrated system in which effort consumed in one modality can lessen the cognitive demands. In other words, gesturing can shift some of the load from verbal working memory to other cognitive systems (Goldin-Meadow et al., 2001).

For instance, full-body movements and gestures can be designed to enact phenomena that involve abstract concepts such as mathematical ones, with the help of physical or digital representation of these concepts.

Moreover, Lindgren et al. (2016) argue that the integration of full-body movements into digitized science lessons increase levels of engagement, motivation and more positive attitudes toward science.

There are several approaches to why embodied interaction can lead to enhanced learner engagement.

According to constructionist learning theories, the physical exploration of one’s environment can lead to attitudes that generate new learning. The student has, in other words, an intensified feeling of agency around material objects that can be manipulated and explored (Papert, 1980). Papert gives an example of the physics Turtle, which is an interactive being that can be manipulated by the student, providing an environment for active learning, not only by interacting with the manipulative but also by being able to invent their own personal sets of assumptions about the artificial world and its laws (ibid).

2018-01-04 (Dnr. ) 10 (av 61)

Another explanation, such as the one given by Winn (2003) is that body-based experiences are considered more immersive and cause an intensified feeling of presence, i.e. the feeling that you are “in” the artificial educational environment (Winn, 2003). Witmer and Singer (1998) also argue that presence can be improved by a high level of enjoyment and by being immersed in the environment (ibid). An example is when a student wears a VR head-mounted display with a wide field of view and when head movements are tracked in real time, allowing the student to look around in the artificial environment in the same manner as in the real world (Winn, 2003).

According to Winn (2003), learning is a result from the reciprocal interaction between external, embodied activity and internal, cerebral activity. In other words, learning cannot be seen as confined to what goes on in the brain. Studies have shown that presence consistently predicts how much students learn, and that reduced presence caused by distraction or discomfort hampers learning (ibid).

Finally, other researchers bring up the possibility to adopt a visual or haptic perspective as a characteristic of embodied interfaces. For them, technology-guided attention can lead to more expert-like thinking and problem solving, and these full-body experiences may engender more expert-like attitudes, such as confidence in one’s ability to tackle problems in a specific domain, which is shown in Lindgren’s research on learning in virtual environment (Lindgren, 2012). His study proves that the use of first-person perspective of simulated events in a virtual environment leads to better memory and more efficient performance than what is achieved with a third-person disembodied perspective of the same events. In Lindgren’s study, those participants who tried first-person perspective in a virtual environment have shown a stronger feeling of competence and identification with the domain, compared to those who tried third-person perspective. Additionally, the former group of participants were less prone to help-seeking behavior than the latter.

2.2 Gamification and Student motivation

Compared with games, reality is hopeless. Games eliminate our fear of failure and improve our chances for

success (McGonigal, 2011, p. 68).

Recently, gamification has been gaining increased attention in several contexts. The interest in gamification can be explained by the idea that it has an impact on our behavior (Buckley & Doyle, 2016). As McGonigal (2011) points out, games lead to strong emotional responses such as curiosity, frustration and pleasure. Besides, games can make people more engaged and productive (ibid). Moreover, gamification can align the interests of the game designer with the motivations of players, and in a business context, such

2018-01-04 (Dnr. ) 11 (av 61)

alignment can make the consumers keeping coming back for more (Zichermann & Cunningham, 2011). Buckley and Doyle (2016) see in gamification a great potential for educators to apply in the classroom to capture the students’ attention and having them engage in a way that nurtures their interest.

According to these authors, gamification impacts students with distinct types of motivation (section 2.2.2) in dissimilar ways. Whereas gamification proves to be effective for learners who are intrinsically driven by a motivation to know or a motivation towards stimulation, the same cannot be supported about those students who are extrinsically motivated, with one exception – students who are extrinsically motivated by identification.

In this subsection, both concepts – gamification and motivation – will be elucidated.

2.2.1 Gamification Kapp (2012, p. 10) defines gamification as “using game-based mechanics, aesthetics and game thinking to engage people, motivate action, promote learning and solve problems”.

As Buckley and Doyle (2016) point out, gamification in education is different from using games as educational tools, rather gamification entails the integration of design elements or activity patterns usually found in games into educational contexts.

One of these design elements are objective and specific rules. In a gamified learning program, rules shape the learning activity and set clear limits on the actions a learner can take (ibid).

A second element is reward systems, which means that the players receive rewards such as badges or prizes for achieving a goal or surpassing an obstacle. The reward is a way of letting the player know that a level of aptitude has been reached, and not necessarily connected to the goal achieved (ibid). In a learning context, students earn points for completing assignments correctly or through getting a high grade, they “level up” and proceed to a more advanced course (Lee & Hammer, 2011).

And finally, paraphrasing Buckley and Doyle (2016, p. 1164), game playing is related to trial, failure and success through practice, experience, reflection and learning. One of the main goals of most games is to develop a positive relationship with failure – failure should be considered a step towards mastery. In a gamified learning context, educators create rapid feedback cycles and keep the stakes for Individual learning low to keep a positive relationship with failure (ibid).

2.2.2 Motivation Motivation in an educational context is considered one of the central factors of learning and employed to explain the attention and effort students dedicate to certain learning activities (Brophy, 2013). One of the teacher's main roles

2018-01-04 (Dnr. ) 12 (av 61)

is, thus, to keep the students' motivation level high, which can be manifested in terms of intensified effort, persistence and enhanced performance (Buckley & Doyle, 2016).

Student motivation is categorized into "intrinsic" and "extrinsic". According to Harlen and Deakin Crick (2003), intrinsic motivation entails students being interested in what they learn and in the learning process, and as Buckley and Doyle (2016) mention, this concept is closely associated with the work of Piaget (1976) who claims that when individuals perceive a mismatch between their experienced knowledge and their internally held knowledge, they feel compelled to put an end to this mismatch.

Intrinsic motivation has been refined conceptually and can be divided into three subcategories , which Vallerand et al. (1992, pp. 1005-1006) describe as follows:

Motivation to know is related to constructs such as exploration, curiosity, learning goals, intrinsic intellectuality, and entails the desire to perform a learning activity for the enjoyment one experiences while learning.

Motivation towards accomplishment involves the desire to engage in an activity for the satisfaction experienced while completing a challenging task.

Motivation to experience stimulation means that an individual engages in an activity to be stimulated, which can manifest itself in several ways, such as sensory enjoyment, aesthetic pleasure and emotional sensations such as fear and excitement.

Extrinsic motivation is, on the other hand, associated with students who engage in learning since it is a means to an end, and somewhat detached from the content and subject of learning (Harlen & Deakin Crick, 2003). The focus is on rewards, used to direct and manage learning behavior, or as Brophy (2013) states, this perspective is best directed through report and conduct cards and award ceremonies.

This theoretical construct has also undergone refinements and can be described as a continuum running from external regulation through introjected regulation to identification (Buckley & Doyle, 2016). Gagné et al. (2015, p. 179) account for this range of extrinsic motivation subcategories as follows:

External regulation is the least autonomous type of extrinsic motivation and emanates from externally pressuring forces. It refers to behaviors performed to obtain rewards or avoid punishments administered by others.

Introjected regulation is, on the other hand, regulation of behavior emanated from internally pressuring forces, such as shame, guilt and ego-involvement, and as Ryan and Connell (1989) put it, this type of internalization is perceived as controlling.

2018-01-04 (Dnr. ) 13 (av 61)

Identified regulation is related to doing an activity because one identifies with its value or meaning and accepts it as one's own. The difference between this construct and intrinsic motivation is that while the latter is driven by inherent satisfaction, the former is driven by the instrumental value a certain activity embodies.

Finally, there is a lot of discussion concerning the impact intrinsic and extrinsic motivation have on learning. For instance, one question is whether learning is better enabled by intrinsic or extrinsic motivation. Furthermore, there are some authors who suggest that the provision of external rewards damages intrinsic motivation (Buckley & Doyle, 2016).

2.3 Virtual Reality Auld and Pantelidis (1994) define VR as a text- and graphics-based environment that is computer-generated. However, the meaning of VR has changed through the years, and nowadays, it can encompass much more, from simple mobile-based simulations to full immersion using specific equipment, such as Oculus Rift or HTC Vive, a VR headset (Taljaard, 2016).

Through this technology, the user has the capability to interact with the virtual world with the help of a hand controller or joystick as well as other types of input, such as voice command and hand motion, which makes VR a remarkable tool to use in education (ibid).

2.3.1 Immersion vs Presence

VR is about psychologically being in a place different than where one is physically located, where that place may be a

replica of the real world or may be an imaginary world that does not exist and never could exist (...)

(Jerald, 2016, p. 45)

Immersion is the unbiased technology that has the potential to engage users in the experience. However, immersion is just one of the facets of the VR experience as it takes a human to perceive and interpret the presented stimuli. Immersion can lead the mind but cannot control it. How the user subjectively experiences the immersion is known as presence (ibid).

Presence, on the other hand, is a rather subjective concept, it is an internal psychological state (ibid), or as Winn (2003) defines it, presence is the belief that you are inside the virtual environment rather than in the laboratory or classroom interacting with a computer. Usually, during a visit to a virtual environment, attention is divided between the virtual one and the real environment outside, which might be noisy and disturbing.

According to Witmer and Singer (1998), presence varies with the extent to which attention is divided between the virtual and the real environment. An elevated level of presence requires exclusive attention to the virtual

2018-01-04 (Dnr. ) 14 (av 61)

environment, which leads to immunity from distraction and allows full engagement with the virtual environment.

2.3.2 Tangibility within VR Piaget (1976) was one of the first people to formalize the idea that manipulating objects has several advantages from early years for the cognitive development. He introduced four states of cognitive development in which “concrete operations” is a preliminary condition for achieving the state of “formal operations”. During the “concrete operations” phase, logical reasoning is achieved through seeing and manipulating objects. Papert (1980) works on this concept by developing programming environments that allow children to build their own knowledge through physical representations of abstract concepts.

Moreover, Fitzmaurice (1995) elaborated an approach to tangible interfaces, which he called “graspable user interfaces”. Tangible interfaces enable direct control of electronic or virtual objects through physical handles. A more recent definition is given by Ishii (2008), who asserts that Tangible User Interfaces (TUIs) weave digital technology into the fabric of a physical environment and making it invisible. In other words, TUI enables the direct manipulation of digital information with our hands and makes it perceptible through our peripheral senses by physically embodying it.

When it comes to VR, how does it work? Can we talk about tangibility? When you manipulate objects in VR, they can feel real as usability tests carried out in this research have shown. But given the previously mentioned definition of tangibility, it does not apply here. However, it is possible to bring tangibility to VR.

VR affords immersive environments and better 3D visualization of digital content, and for this reason used in many contexts related to training and education. A VR-TEI (VR – Tangible Embodied Interactions) system would, thus, have a great potential to add real tangibility and increasing feeling of congruence between the real and virtual worlds. However, there are very few projects that make use of this type of system – usually, they lack an element, namely a tactile input, i.e. a tangible object (Chang et al., 2017).

Usually, commercial VR only tracks your hands, and tangible interaction within virtual environments consists of haptic feedback of tracked controllers, which can lead to a lack of accuracy and consistency between the real and virtual world. However, there are emerging design trends of VR games and applications that attempt to minimize issues of sensory conflict (Harley et al., 2017). Moreover, in the past few years, there have been several projects trying to engage more the body in VR (section 2.4). A way to bring tangibility to VR would be, besides using head- and hand-tracking devices, adding tactile inputs such as tangible blocks to the equation (Chang et al., 2017).

2018-01-04 (Dnr. ) 15 (av 61)

2.4 Related work Interest in and development of VR technology have been around since the 1980’s. However, the technological innovations of the past few years have been laying greater focus on VR and its potential utilities (Hubbard et al., 2017). Moreover, many of these advancements, e.g. improved graphical rendering and room-scale positional tracking, allow for an improved feeling of presence (section 2.3.1) in the virtual environment. This feeling of presence is considered an essential element for educational VR applications, as Hubbard et al. (2017) pinpoint, since students become more focused on the learning material and less distracted by external stimuli in case they are deeply immersed in the virtual environment.

Despite VR’s educational potential, most part of available VR applications are games or for entertainment purpose only. When it comes to the academic arena, the literature on VR within education is still very limited, but there has been some research on VR applications directed toward training individuals and facilitating the learning process (Hubbard et al., 2017).

Most part of these researches examine the benefits of VR to the learning process of STEM-related subjects, i.e. subjects within Science, Technology, Engineering and Mathematics. One example is a research carried out by Seo et al. (2017), who investigate how a VR system can support embodied learning in anatomy education by testing a program called Anatomy Builder VR. Many students, according to these authors, have a challenging time grasping the underlying basic aspects of anatomical form when they start with the study of complete anatomical specimens, due to its visual complexity.

Their study has as approach a constructivist pedagogical model that sees learning as a personal construction resulting from an experiential, dynamic process that leads to the construction of knowledge with personal meaning (Seo et al., 2017). The main conclusion of their research is that participants enjoyed mostly interacting with anatomical contents within the VR program, and specifically, it was pleasurable for the participants of the study to put a skeletal system inside of the anti-gravity field, a setup that is not possible in a real environment (ibid).

Another example is a research conducted by Cecil et al. (2013) that investigated the efficacy of VR technology in aiding the learning process of engineering concepts. In this study, a VR learning environment that allows students to learn at their own pace and can pause whenever they want was tested by Engineering students, and by some K-12 students. The results have shown substantially positive effects of adopting such a VR learning environment. The most remarkable outcome, according to the authors (ibid) was in the improvement in the student performance in understanding concepts such as Genetic Operators, including cross over, mutation and inversion, which reflected in the students' performance in homework, exams and quizzes (ibid).

2018-01-04 (Dnr. ) 16 (av 61)

Finally, it is important to mention that, while some studies have revealed learning benefits of using VR in education, especially when it concerns the learner control and learners' ability to leverage existing knowledge to create new concepts, other studies have demonstrated marginal or even negative effects on learning outcomes. This can be explained by a lack of focus on the way in which those technologies affect the critical factors that facilitate learning, and it can be also due to absence of comprehensive teaching strategies and supporting design practices to direct the learning experience (Goodwin et al., 2015).

2.4.1 Gamified Programming education – From 2D to 3D As Vincur et al. (2017) pinpoint, block-based programming languages, such as Scratch and Blockly, have successfully been implemented in classrooms as an alternative way of teaching basic programming concepts. This success can be explained by a lowered barrier of entry and the gamified visual fostering experimentation and creativity (Vincur et al., 2017).

Scratch is a 2D block-based programming language and online community developed by MIT Media Lab that allows children to program and share interactive media with other users. The target group consists of children from 8 to 16 years old, but it is also used by people of all ages, including younger children with their parents (Resnick et al., 2009).

Figure 1 Scratch interface: You drag the blocks you want and drag them to the gray canvas on the right

2018-01-04 (Dnr. ) 17 (av 61)

Blockly, developed by Google, is a type of 2D visual block language development kit that allows the rapid construction of new block-based programming languages to address a specific content area (Trower and Gray, 2015). Similarly to Scratch, it uses visual blocks that link together to make writing code easier, and can generate JavaScript, Python, PHP or Dart code. It can also be modified to generate code in any textual computer language (Blockly - Google Developers, n.d.).

These block-based programming environments are nowadays very popular among educators and students. Nevertheless, this kind of interaction with virtual objects on computer screen with mouse limits the learner from getting immersed into the learning environment (Vicur et al., 2017), or as Ishii (2008) expresses it,

(…) interactions with pixels on these GUI screens are inconsistent with our interactions with the rest of the

physical environment within which we live (Ishii, 2008: xv).

VR presents a step further in relation to these visual programming language platforms, due to an expressively high degree of immersion VR technology offers, as Vincur et al. (2017) claim.

Some of the projects that attempt to apply VR technology to programming education are described below.

Cubely

Vicur et al. (2017) designed Cubely, an immersive VR programming environment in which programming beginners can solve programming puzzles within a virtual environment. A program in this environment consists of cubes assembled into program structures, while the program execution is traced to individual cubes and can be seen directly in the virtual environment. The team of researchers tested the usability of the VR programming

Figure 2 The Blockly library adds an editor to your app that represents coding concepts as interlocking blocks. It outputs syntactically correct code in the language of your choice.

2018-01-04 (Dnr. ) 18 (av 61)

environment and students' insights in comparison to the 2D block-based programming platform.

With Cubely, the learner manipulates these cubes and put them together to solve programming exercises similar to those found in the Code.org learning platform. To make it more appealing and engaging to the learner, the researchers prototyped the Cubely programming environment to a theme of a popular video game, namely Minecraft. Moreover, the program is assembled in the same environment where the code is executed, which means that the learner does not have to switch between real and virtual environments (Vicur et al., 2017). Cubely presents some advantages against 2D block-based programming platforms, such as the possibility of interacting with cubes using the controllers in both hands simultaneously; the program can be placed anywhere in the virtual environment and; lack of external distractions due to the VR platform's immersiveness (Vicur et al., 2017).

The main finding of their research is that they found easy to work with Cubely and preferred it over the traditional 2D block-based programming, which was assessed through observation and questionnaire (Vicur et al., 2017).

3D-VPL

Ortega et al. (2017) propose a 3D visual programming language as an interactive tool for beginners and intermediate students. Moreover, their research has a goal to help increase the recruitment and retention of female students in Computer Science (CS). As a part of their research, they developed a prototype and inquired students on what symbols work best. However, the research showed that it is difficult for the CS students to present well-defined 3D representations for programming concepts (Ortega et al., 2017).

Figure 3 Cubely VR block-based programming environment (Image from Vicur et al. 2017)

2018-01-04 (Dnr. ) 19 (av 61)

The context for Ortega et al.’s research is a decrease of the number of female students enrolled in Computer Science. According to these authors (Ortega et al., 2017), some of the reasons for this trend are misconceptions and stereotyped view of CS, such as it is boring and does not encourage interpersonal relations (Ortega et al., 2017).

Their idea is to implement a VR programming language inspired significantly by Scratch, and that can be used with VR and AR technologies. The goal is that, through block-based programming in a virtual environment, it will become easier to bridge students into traditional programming. Also, the 3D-VPL is designed with team-based problem solving and pair programming in mind (Ortega et al., 2017).

One of the major problems was to find ideal shapes and symbols for beginners in programming. The researchers' initial shapes were not optimal for the intended target group, so they conducted a survey with 50 CS senior and junior students, asking how they would represent in 3D some of the programming concepts. However, it was not an easy task for the students to come up with a clear graphical representation for the concepts.

Virtual Reality Instructional Modules

Another research that deserves to be mentioned is Stigall and Sharma’s (2017) which presents two gaming modules for teaching object-oriented programming (OOP) and binary search. Regarding theory, their work has

constructivism (section 2.1) and usability as reference point. In other words,

Figure 5 3D Representations of Programming Concepts Provided by Students (Image from Ortega et al, 2017)

Figure 4 Early-stage prototype of 3D-VPL, inspired by Scratch. (Image from Ortega et al., 2017)

2018-01-04 (Dnr. ) 20 (av 61)

one of the assumptions is that the student learns a concept by practicing it and calling on that experience to remember the concept. Also, the teacher has a role that is more important than lecturing, namely helping student understand the lesson in sight (Stigall & Sharma, 2017).

The other aspect, usability, is considered a critical issue when developing gamified learning module and can be defined as the ability to perform the expected functions in a given system to achieve specific goals (Hersh & Leporini, 2012, p.752).

The researchers built as part of the project two VR instructional modules – one for teaching Object-Oriented Programming and another for teaching the binary search algorithm – having constructivism as basis to help students visualize the programming concepts being taught in these modules (Stigall & Sharma, 2017).

The authors tested the OOP module on a total of 20 students, and after the test the students were a survey to fill out. Their findings are that most part of the students considered the interface user-friendly, and the clear majority had the sensation that the modules helped them to understand the programming concepts (Stigall & Sharma, 2017).

2.4.2 Commercial examples of VR learning tools Most part of the VR content available on e.g. Steam and Itch (popular game portals) is for entertainment purpose – game and simulators. However, there are some learning tools, mostly within the realm of Arts or Biology, the latter targeted at K-12 students.

But there are even some programming-related VR applications.

One of them is Rumpus, a beta, open-source live-coding playground for room-scale VR, a project from indie developer Luke Lannini.

Figure 6 The OOP module being used on a desktop computer. (Image from Stigall & Sharma,2017)

2018-01-04 (Dnr. ) 21 (av 61)

Users can play around with pre-made options to design various objects while being able to see the code that goes into making it. The code can also be directly altered with a virtual keyboard that appears either side of the coding window itself (VRFocus, 2016).

Another example is Glitchspace, which allows you to reprogram the game to solve puzzles in an abstract virtual environment, and in order to move forward in the game, the player is supposed to use visual programming to edit environmental geometry by stretching, scaling, rotating, manipulating geometrical objects. This will unlock key programming concepts along the way (Steam Community, 2016).

Finally, there is the VR application Primitive, which is a three-dimensional

Figure 7 Rumpus interface (Image from VRFocus, 2016)

Figure 8 Glitchspace Interface - Unlocking concepts (Image from Steam Community)

2018-01-04 (Dnr. ) 22 (av 61)

representation of open source Java, C#, C++, and JavaScript code in an immersive environment. Primitive turns the output of software analyses into interactive 3D structures that are displayed in the virtual environment. It allows developers to see architectural overviews of the source code, multi-thread runtime animations that allow collaboration in debugging and spatial layout that are clearer (Steam Community, 2018).

Figure 9 Primitive interface – code visualization (Image from Steam Community, 2018)

2018-01-04 (Dnr. ) 23 (av 61)

3 Methods 3.1 Design Process Model

This research is carried out according to the model “Double Diamond Design Process “(Design Council, n.d.).

As illustrated in the model in Figure 10, all design processes involve divergent thinking and convergent thinking, which means that several possible ideas are created (divergent) before they are refined and narrowed down to the best idea (convergent), and this can be represented by a diamond shape.

The double diamond model suggests that the process, i.e. from divergence to convergence, happens twice – firstly to confirm the problem definition and secondly to create the solution.

Furthermore, an import assumption within this model is that the creative process is iterative – ideas are developed, tested and refined several times, and the weaker ideas are dropped in the process.

The model encompasses four distinct phases, namely Discover, Define, Develop and Deliver, which can be described as follows.

Discover is characterized by divergent thinking, which means that designers keep an open mind and try to notice new things and trends, and gather insights, either by observing, interviewing people, going through articles and books in related area.

Figure 10 Double Diamond Design Process (Design Council, n.d.)

2018-01-04 (Dnr. ) 24 (av 61)

Define is a stage in which convergence thinking happens. The designers try to develop and refine the ideas and drop the weaker ideas. In this stage, the designers analyze what should be prioritized and what is feasible. The goal in this phase is to develop a clear brief that frames the fundamental design challenge.

Develop is a phase in which solutions and concepts are created, prototyped, tested and iterated. And through trial and error, designers can improve and refine their ideas.

Delivery is the last stage and encompasses convergence thinking again – the focus is on what can be delivered, and which solutions can solve the defined problem. The resulting project is finalized, produced and launched.

3.2 Literature Review

A literature review is a narrative essay that integrates, synthesizes, and critiques the important thinking and

research on a particular topic (Merriam, 2009, p. 76).

Once you have defined the problem, it is important to find out what has already been written and researched in the field. Literature review has an important role to provide the groundwork for contributing to the knowledge base (Merriam, 2009).

From a design perspective, it is important to know what has already been done, with what success, and what needs to be done. Through literature review the designer can avoid making the same mistakes other researchers/designers have made, and gains insight on how to fill the gaps (ibid).

And finally, having knowledge of what has been written in the field offers a point of reference for discussing the contribution the current study will make to the epistemic community (ibid).

How a literature review is conducted and the scope of the search depends on how well defined the research problem is and on researcher’s familiarity with the topic. In case of a vague sense of the problem, a good idea would be to conduct an overview of the topic, which will help the researcher in identifying central works, theories, issues etc. The next step would be to check bibliographies, indexes and abstracts that address specific aspects of a topic (ibid).

In order to find relevant and reliable texts, Malmö University Library’s search machine – Libsearch – was used, and Google Scholar. When it comes to books, especially methodology ones, books suggested by Interaction Design teachers were used. Furthermore, “backward snowballing” method was used. It means that you start with an initial set of texts, and in these you look for

2018-01-04 (Dnr. ) 25 (av 61)

potential references and citations that could be used to expand your initial set (Wohlin,2014).

And finally, in order to manage the references, Mendeley was used, since it is user-friendly, integrable to Word through a plugin, free to use, and allows users to generate and format bibliographies in several output styles, including APA Harvard, the one used in this thesis.

3.3 Sketching Sketches are sometimes referred to as “thinking drawings” and are called “design drawings” by Lawson (1997). They are usually made by designers and for designers, and central to the process of ideation (Buxton, 2007).

Preece et al. (2015) states that the activity of sketching helps the designer to focus on what is being designed and the details of the interaction. By sketching, the designer is making choices and exploring the requisites more closely and refining your ideas.

According to Buxton (2007), sketches must fulfil some criteria, namely they should be quick, timely, cheap, disposable, abundant, ambiguous possess a clear vocabulary, minimal details, distinct gesture, some degree of refinement, and finally, sketches should be used to suggest and explore rather than confirm.

3.4 Co-design workshop Co-design method has been used in the educational field in the past few years (Barbera et al., 2017).

Penuel et al. (2007) define co-design in the realm of education as:

(…) a highly-facilitated, team-based process in which teachers, researchers, and developers work together in

defined roles to design an educational innovation, realise the design in one or more prototypes, and evaluate each

prototype’s significance for addressing a concrete educational need (p. 606).

Some of the characteristics of the co-design processes are the following:

a) they are based on the design of a concrete and tangible innovation;

b) they collect information on education practice and its context;

c) they have a flexible goal as they involve an iterative implementation process susceptible to being modified;

d) these methods also encompass a shared experience among stakeholders, which requires a collaborative effort to promote a common understanding between actors. This can be achieved through co-design workshops.

2018-01-04 (Dnr. ) 26 (av 61)

(Penuel et al., 2007, pp. 53-55).

According to these authors, many of the practices of participatory design in education are associated with the design and implementation of technological tools designed to support learning processes (Penuel et al., 2007).

As Spikol et al. (2009) point out, the benefits of co-design are the direct involvement of relevant parties – the stakeholders – helping to ensure that the concerns and values of the service/product users are kept in focus.

Co-design needs however a bootstrapping event to put in motion the team’s work such as design workshop, which can help build a common understanding of the need for a product/service and a shared send of what features the product/service must have so that it helps teachers and students in the classroom.

3.5 Interview DeMarrais (2004) defines an interview as “a process in which a researcher and participant engage in a conversation focused on questions related to a research study”. (p. 55)

The main purpose of an interview is to obtain a special kind of information – the researcher wants to understand how someone relevant to the study thinks. Or as Patton states, through interview, the researcher can enter into the other person’s perspective, which cannot be achieved through observation (Patton, 2002).

There are several ways to carry out an interview regarding structure, and the main types are highly structured, semistructured and unstructured.

In this research, only semistructured and unstructured approaches were applied.

Semistructured interview implies an interview guide that includes a mix of more and less structured questions and they are used in a more flexible manner. Additionally, it may contain both questions and list of issues to be explored, without any specific wording or order (Merriam, 2009).

Finally, unstructured interview consists of open-ended questions, of a more exploratory nature. It is more like a conversation than interview, usually when the researcher is unfamiliar with the topic and cannot ask relevant questions (ibid).

The interviews took place in the interviewees’ respective workplaces or school, (except for one) so they would feel comfortable.

Before starting posing the questions, the research topic was presented, and I also made sure that the interviewee was still willing to participate and did not have any objections to having the interview recorded. I also reassured the

2018-01-04 (Dnr. ) 27 (av 61)

interviewee that his/her identity would remain anonymous and the voice recording would only be used for transcription and analysis.

3.6 Prototyping Preece et al (2015, p. 386) define prototype as a manifestation of a design that allows stakeholders to interact with it and to examine its relevance. A prototype is limited in terms of highlighting some features and deemphasizing others.

Additionally, there is a misconception that prototypes are something like scale model of a product. According to Preece et al. (ibid), a prototype can range from a paper-based storyboard to a complex piece of software, or from a cardboard mock-up to a molded piece of metal.

Prototyping is an important stage in the process, since it encourages reflection in design, and prototypes facilitate the communication among stakeholders and team members. They also serve many purposes, such as to test out the technical viability of an idea, to clarify some unclear requirements, to do usability testing and evaluate or to check whether a certain design direction is compatible with the rest of the product development (Preece et al., 2015).

3.7 Usability test Goodman et al. (2012, p. 273) define usability test as “structured interviews focused on specific features in an interface prototype”. It does not matter if it is a lo-fi or hi-fi prototype – usability test can uncover a lot of information about how people use a prototype.

The main goal of usability test is to make sure if a product is usable by the intended target group to execute the tasks for which the product was designed. To achieve this, a combination of methods is used. For instance, as testers perform tasks, they can be filmed so that the researcher can later analyze their interaction with the product. After the experiment, the researcher can also interview the users to elicit their opinions about their experience and the product. Also helpful is observing how users react toward interactive products – it helps the developer to detect usability issues that are otherwise hard to identify (Preece et al., 2015).

After the usability test, the researcher goes through the collected data and can identify issues concerning the prototype’s functionality and presentation. The development team can from this point see if their initial assumptions about how people perceive and use their designs remain valid (Goodman et al., 2012).

The first step of the usability test was to check if everything was working and in place. Then instructions and information were given to the participant, followed by signing down the Letter of Consent (section 3.8; Appendix 1).

2018-01-04 (Dnr. ) 28 (av 61)

After that, the test started. I asked the participants to “think aloud”, a commonly used method in Interaction Design (Goodman et al., 2012, p. 291).

3.8 Ethical implications Whenever you carry out a research involving human test subjects, there are ethical concerns to be taken into consideration. One of the main principles is non-maleficence, which is described as follows:

Harm to research participants must be avoided. Where risks arise as an unavoidable and integral element of the

research, robust risk assessment and management protocols should be developed and complied with.

Normally, the risk of harm must be no greater than that encountered in ordinary life, i.e., participants should not be exposed to risks greater than or additional to those to

which they are exposed in their normal lifestyles (The British Psychological Society, 2014, p. 11).

However, when conducting a research within VR, following the principle blindly would imply that one should not carry out any kind of test if the risk of harm is unknown (Madary & Metzinger, 2016).

Working with VR entails potential risks, especially when it comes to the usability test participants’ own safety. Firstly, there are risks for dizziness, motion-sickness and even epileptic seizure (Winn, 2003). Secondly, the participants are de facto blinded by VR headset – they cannot see the real environment or situate themselves in it. All the movements are made having virtual environment coordinates as reference. It can happen that they bump into a wall or furniture that only exist in the real environment, and as a result hurt themselves. My prototype, however, did not require room-scale setup, which means you did not need to move much to complete the task. Additionally, the prototype consisted of a one-level demo, which implies that the time in immersion is very short. Consequently, there are no serious risks for injuries or discomfort.

Prior to the tests, they were also informed about the possible risks, and they had to sign a letter of consent just to reassure they were aware of that but still willing to participate.

Moreover, there are more general ethical issues concerning the test subject’s integrity, i.e. their right to anonymity should be respected, and gathered material in form of video, images and sketches, after expressed consent, should solely be used for research purposes, in compliance with the ethical standards in research as expressed in “Ethical principles for Humanities and Social Sciences research” (Swedish Research Council, 2007).

In my research, these ethical issues were taken rigorously into consideration, and necessary measures were implemented to ensure the participant’s safety and integrity.

2018-01-04 (Dnr. ) 29 (av 61)

4 Design process 4.1 An overview

As mentioned previously, this thesis work was planned and conducted in line with the Double Diamond Process Model (Design Council, n.d.). In concrete terms, it implies that the first step taken was to explore the field of VR and how it can be used within education, without limiting to a specific subject, such as Mathematics, Physics, Chemistry or Programming, just to name a few. The “Discover” phase of this work consisted in looking for relevant empirical and theoretical texts and conducting market analysis to identify existing VR projects and applications used for educational purposes. After this step, it became clearer what I should focus on, namely Immersive VR for Programming Education. From this point on, the brainstorming process started. During this phase I have also sketched some ideas for the VR interface and logics, both individually and with some classmates interested in VR.

After sketching some concepts, semi-structured and informal interviews were conducted with teachers from Elementary school to University level and some students, to have a better understanding of the problem, their current needs and wishes.

The second phase – “Define” was initiated with a co-design workshop with Interaction Design to discuss ideas for the concept and prototype, and it helped me to focus on the best ideas and delimitate the scope of my thesis work. Additionally, more interviews were carried out with Programming professors and experts with the purpose of obtaining some inspiration and input on how the problem can be tackled in practical terms.

Once it was decided what problem would be addressed and how, the third phase – “Develop” – was about a prototype in VR environment. This stage was characterized by iterative work – my prototype has been through substantial changes as a result from conversations and the initial usability tests.

The final phase – “Deliver” – entailed extra usability test sessions and contact with experts (e.g. software developer from Unity) to refine my prototype and fix eventual issues.

2018-01-04 (Dnr. ) 30 (av 61)

4.2 Sketching The sketching process started early, in June 2017 – before deciding on a specific research topic, I started writing down ideas, possible sub-topics, outlines, etc.

From a very early stage, I knew what issue to tackle, namely learning process involving complex abstract concepts. What was still to be defined is what area of knowledge should be prioritized. After doing some research, the choice of topic was made and the sketches became more issue-specific, and more aspects were added, such as how the interaction could manifest itself, and what the VR application logic would look like. Most part of the sketches were related to mechanism and dynamics rather than design. During brainstorming session with a classmate (section 4.3) we came up with some sketches on the logics and perspective to be used in the virtual environment.

Figure 11 Sketching ideas for the topic

2018-01-04 (Dnr. ) 31 (av 61)

Figure 12 A sketch on how to emphasize the benefits of VR, and how to visualize the abstract concepts such as “iteration/loop”.

Figure 13 This sketch represents the first-person perspective to be used in the virtual environment. In this case, the player possesses agency.

Sketching of design elements came a little bit later in the process, more specifically under the “Develop” phase, in the second divergence wave. My “weapon” of choice was Unity, 3D modelling and graphic design tools rather than pencil and paper. In Unity it was possible to sketch in a VR environment, in a very quick and timely way, and the interactions could also be sketched without using code in this phase. In other words, the sketches done in Unity, with the help of 3D modelling and graphic design tools, fulfilled the criteria mentioned in 3.3.

4.3 Brainstorming From the moment I started writing my thesis draft, I started the brainstorming process with Daniel Spikol, senior lecturer and researcher at the IOTAP lab, at Malmö University. Together we have gone through existing learning platforms for several subjects, such as Mathematics and Chemistry and discussed how we could translate it to a VR environment. After even more brainstorming sessions combined with background research, the choice of topic was made – VR in programming education. From this moment on, Daniel Spikol and I started brainstorming for possible solutions, and concepts on how to tackle programming concepts in a graspable way. The brainstorming sessions were very hassle-free – we just sat in the sofa and surfed on internet looking for existing platforms and products and coming up with our own ideas on which aspects and concepts I could focus on. Moreover, I had a one-hour brainstorming session with a classmate who is also writing thesis on VR. Firstly, I pitched my concept to him and afterwards we started

2018-01-04 (Dnr. ) 32 (av 61)

thinking to ourselves until we came up with ideas, that we just shared and discussed. We also made some sketches together (section 4.3).

In other words, the brainstorming sessions were never an individual activity, I always had an expert or a classmate to discuss all the possible ideas on how to translate the existing 2D learning platforms into VR environment and even on what it could look like in terms of design and mechanics.

After these brainstorming sessions, it became clearer how I would tackle the research problem – I decided to do something like Scratch and Blockly (section 2.4.1).

4.4 Co-design workshop To a one-hour co-design workshop at Orkanen Library, I invited Interaction Design students via several Facebook group pages, for example IDK17, IDK15, and IDK@Malmö. The initial plan was to present them my first hi-fi prototype in Unity, and then they would make suggestions on what interactions to add, what the design, mechanics and dynamics could look like, and so on. However, due to technical issues, I had to change it a little bit – I made a very “old school” prototype with building blocks. The first step was to talk a little bit about my research project and make sure they were there voluntarily, and that they did not have anything against having pictures taken or having the session recorded in camera.

Figure 14 Blocks with text. Some of the words were picked from Scratch

2018-01-04 (Dnr. ) 33 (av 61)

The next step was to give them some instructions on how to interact with the blocks – play freely with them, find your own way to “code” with the blocks, there is no right or wrong.

Besides blocks with text on it, some “blank” building blocks, pen and tape were also distributed, so the participants could add their own text and customize the blocks.

The participants have also received big sheets of paper so that they also could make sketches on how they envisioned the concept or some aspects of it.

Since there were six participants, and the need to ask questions, observe, record and assist them, one of the participants co-hosted the workshop, i.e. we made some labor division.

The conclusions that can be drawn from this session is that each person found a different way to “write” code with the building blocks. Those with more advanced programming skills used traditional programming logics – building code from left to right, whereas those with less or no experience of programming “wrote” code in a more creative and “visual” way, using blocks in blank to complement coded ones to create “animations”. And there were some who were familiar to Scratch and used the same logic.

During the workshop we also had some discussion about my concept – using VR to visualize programming concepts and, thus, facilitate the understanding.

They seemed to appreciate the idea, with some reservations – they were afraid it could become into something too childish or too simplistic for Interaction Design students, and maybe the best target group would be elementary school students. However, they still believed that it could be found a more sophisticated way to use VR in programming education than using, for instance, cute animal graphics and the like. One example of how it could be approached is something like the Human Fall Flat game, with more complicated puzzles and obstacles.

Figure 15 Participants "writing" code in several ways with the blocks

2018-01-04 (Dnr. ) 34 (av 61)

And finally, there were some who were more positive towards AR than VR, due to the social aspect – AR enables collaboration and work in teams in an environment they are familiar with. However, some others pinpointed the advantages of VR over AR, such as immersion and embodiment, elements that facilitate the learning process.

4.5 Interview During the initial steps of my thesis work, interviews with teachers and students were carried out. The purpose was to find out their attitude towards the discipline of Programming and their opinion on VR, and technology in general, as a means for assisting the learning process of complex concepts.

In total, eight people were interviewed in a semi-structured way – two Interaction Design students, two Elementary school teachers (one digitalization pedagogue and one Mathematics teacher), one High School teacher in Mathematics and Physics, one former Interaction Design teacher in Programming, and two active Programming teachers at Malmö University – one within Computer Science and the other within Interaction Design. Moreover, informal interviews were carried out with four Interaction Design first-year students.

4.5.1 Interview with teachers Teachers from elementary school to University level were interviewed about their opinion on importance of programming in our lives, what problems people have in relation to programming and why, how to tackle these problems, and finally on VR as a means of fostering the understanding of programming concepts (Appendix 2).

Regarding the importance of programming, there was consensus that programming is rather a tool than a goal in itself – what it does is to help people to use logic and solve problems. Understanding how programming works, helps High School and Elementary school students to solve Math and Physics problems, and develop logical reasoning. For Interaction Design students, the role of programming is to help us to “talk” with the material and create useful and interactive artefacts and services. Programming allows us to turn a concept into reality.

According to some teachers, students have difficulty in understanding the logic behind programming, and how computers work. Even basic concepts such as array, Boolean and iteration are misunderstood. A reason for this is the abstract nature of the concepts, which makes them be perceived as devoid of any basis in reality. It is also hard for the students to understand that programming has its own logic that differs from our reality – everything in Programming is sequential – one thing happens after the other, in a chain of processes, and not simultaneously even though it feels like otherwise. Another obstacle to understanding programming is, according to some

2018-01-04 (Dnr. ) 35 (av 61)

teachers fear of making mistakes and being perceived as stupid. And finally, motivation to learn programming can also be low, since its threshold is relatively high, i.e. it takes higher efforts to understand and start programming when compared to other disciplines within Interaction Design.

Concerning the question on how to motivate students into learning programming, and to facilitate understanding of complex concepts, there was no unanimity. However, several interviewed teachers have similar strategies to earn the students’ attention and willingness in learning programming and to facilitate the learning process. One is to give problems/exercises that have stronger connection to reality, for instance programming a machine or a robot to perform a certain task. Also important is to enable students’ creativity, give them fun assignments that allows them to create for instance interactive and creative websites. Other teachers say that programming is what it is – it is hard, and there is no point in trying to make it into something it is not. The student should embrace the reality – “programming is difficult, but it is important for an Interaction Designer, so I should make an effort”. But in one aspect there is some consensus – if the teacher notices the students have problem in understanding something, it is essential to take a step back, and offer extra support to those who need it.

Considering the use of technology as a facilitating tool for the learning process, there was some general skepticism in relation to VR. For most part of the interviewed teachers, the high cost of VR gear is considered an issue and therefore it would not be a viable solution. Some teachers were more favorable to AR than VR – concerning accessibility and social aspect. AR can be used with mobile phones, so it is relatively cheap and accessible and easy to use. The social aspect makes sense from a constructivist perspective on learning – according to this approach, knowledge is constructed through the negotiation of meaning within groups (Winn, 2003). Some teachers argued that students learn faster interacting with others, sharing knowledge and collaborating. With AR, it is easier to work in groups and collaborate, while VR makes this harder – you still can work in a cooperative way, but you lose a lot of non-verbal communication such as facial expression. Another argument is that AR adds up to our reality rather than to a fabricated environment we don’t feel any connection to. But for some teachers, VR is superior when it comes to learning due to immersion and presence – inside a VR environment, it is easier to focus on the task. Usually you become completely unaware of what is going on in the world outside, because you are feeling that you are somewhere else, devoid of disturbance and distractions. Moreover, with VR, you can experience with your whole body – embodiment is considered central to learning according to both cognitive and phenomenological approaches (section 2.1). Besides, VR allows a materialization of abstract concepts and visualization of phenomena that are “invisible” to our eyes.

2018-01-04 (Dnr. ) 36 (av 61)

4.5.2 Interviews with students The interviews with students were more of informal nature, without specific questions but instead a list of issues to be discussed in a more open-ended way, like a normal conversation. Both senior and freshmen students in Interaction Design were interviewed (Appendix 2).

The focus varied a little bit from one group to the other. When it comes to senior students, the questions regarded their attitude toward programming, and what the learning process looked like, and suggestion on how to make programming more graspable. The concept of VR in Programming education was also discussed. With the freshmen students, on the other hand, the focus was on a recent situation – a pronounced gap of expectations and learning outcome requirements between Programming I and Programming II. In this case, we discussed what happened, and what difficulties they were facing. Also, a topic of discussion was the pedagogical style of their teachers.

Senior students

The attitude towards programming among senior students varies a lot – among the group, there are those who are very good in programming but do not enjoy it, they prefer creative work, and in their opinion, programming does not leave much room for creativity. However, I could find some students who love programming and the challenge it entails, even though it can be frustrating. What makes them enjoy programming is that it enables them to create fun and interactive artefacts – it feels rewarding to achieve something concrete, after all the struggle with the code.

When it comes to their learning process and difficulties, most of the interviewees did not have any contact to programming prior to their studies in Interaction Design. They had difficulty even with the most basic programming principles, or as some people expressed it “I had no idea how a computer “thinks”, i.e. the logic behind everything the computer executes”. Some of the students seemed to be unaware of the sequential nature of programming at first, as it has been mentioned previously. Even today most part of the interviewees have difficulty with programming. One of the arguments is fear of making mistakes and “crashing the computer or something”, so they do not even try to start programming. In other words, the threshold to learn programming is considered high and difficult to overcome. Another argument is the complexity behind programming and its abstract concepts – “how to relate it to what we are familiar with? “. One of the interviewees had an noteworthy learning process. The person had prior experience and education within Computer Science or similar, and the way this person learned programming was through writing code on paper, no computer allowed. That means, this interviewee and classmates were forced to know how to code by heart.

2018-01-04 (Dnr. ) 37 (av 61)

Additionally, the interviewees gave several ideas on how to tackle the high level of abstractness and complexity of programming. But the idea that was mentioned the most was to visualize abstract concepts, i.e. to make them into concrete entities – either through gamification, or through robotics/electronics with Arduino or Raspberry Pi. Some people have also mentioned the use of block programming, such as Scratch.

And finally, most part of the interviewees showed a positive attitude towards the use of VR as a Programming learning tool, however just to overcome the initial threshold programming entails. There were some participants who showed a bit of skepticism in relation to VR, since there is a loss of the social and collaborative aspect of learning. It was counter argued by others that the social aspect is outweighed by the immersiveness and presence VR experience enables – with the use of VR headset, there is a great potential for focusing on the virtual environment and tasks, without external distraction.

Freshmen students

As mentioned previously, the focus here was on a concrete, recent situation – people dropping out programming course, not handing in assignments, and showing no interest in programming. So, I wanted to know what/how/why happened during the courses Programming I and Programming II.

The answer was unisonous – they experienced a significant gap of expectations and learning outcome requirements between the courses Programming I and Programming II. According to them, in Programming I, they were introduced to basic concepts of JavaScript, HTML/CSS, and the assignments were considered “too easy” – there were no real requirements, and the expectation level was set to a bare minimum. In conclusion, they passed the course without any problem, but they did not feel like they learned anything new, and they felt unchallenged.

Programming II, on the other hand, represented a major shift apropos setup, expectations and demands. The students interviewed felt a prominent discontinuity between these two courses. According to them, the teacher took for granted that they knew what it was being talked about during lectures, i.e. that the students had prior knowledge, something they de facto did not have. It was also their perception that their teacher could not understand the fact that they did not understand the subject. In other words, they felt like they did not receive needed support and could neither keep up with the teacher’s high pace nor live up to expectations. As a result, many would not even try to understand or solve the assignments.

2018-01-04 (Dnr. ) 38 (av 61)

4.6 Prototyping 4.6.1 Technical Specifications

For the prototype, Oculus Rift Developer Kit 2 was used, consisting of a VR head-mounted display and positional tracker. As this version does not work with hand controllers, I chose to combine it with Leap Motion, which is a hand gesture tracker, and makes VR experience even more embodied, with a sensation of tangibility.

In other words, with Leap Motion and its Orion software, you can manipulate objects in virtual environment with your hands.

When it comes to software, Unity game engine was employed to create the virtual environment, the mechanics and the interaction behavior. For the virtual objects, such as the building blocks and furniture, SketchUp Pro for 3D modelling was used.

4.6.2 Design prototyping When the topic choice was made, i.e. VR in programming education, the prototyping process started right away. From the very beginning, Unity felt like the most logical choice since it is a game engine where you can visualize your 3D modelling objects already in an VR environment by simply enabling mixed reality options.

The prototype was inspired by Scratch, a 2D block- programming language (section 2.4.1).and wooden building blocks. The idea is to use assorted shapes that represent different Programming concepts. Besides the shape, the color and texture were also important aspects.

Figure 16 The set-up from left: Oculus Rift DK2 Headset, positional tracker and Leap Motion

2018-01-04 (Dnr. ) 39 (av 61)

4.6.2.1 Prototype 1

The first prototype was based on an already existing project created by Leap Motion and available on their website. It combined VR with hand movement tracker Leap Motion. With this project you could manipulate cubes, cylinders and other objects. So, what I did basically was to substitute the project assets by my own 3D models and apply Leap Motion’s Interaction Behavior script, making the necessary adjustments. At this point, I did not do any coding – the plan was to show this prototype at the first workshop (co-design workshop), but as mentioned previously, I had to change my plans. After the co-design workshop, I started working on my own code in C# (Unity scripts), so that my objects behave in the way I want, i.e. a script that gives you feedback on how you code with the blocks – you are supposed to use them in a specific order, and in case you use wrong blocks or right blocks but in wrong sequence, you will receive a visual feedback. If you use the right blocks in the right sequence, there will also be a feedback for that.

Prototype 1’s system

Figure 17 The first prototype to be tested – overview of the interface

2018-01-04 (Dnr. ) 40 (av 61)

Building blocks: As presented below in Figure 18, there are building blocks of different shapes, and colors. As texture, the name of the programming concept is used. Each type of concept – e.g. variables, operations, data type, numbers – is stored in its corresponding box. This prototype makes use of operators, variables, numbers and coordinates, but only some of the blocks have tags on it, since the purpose is to test how it works.

Interactive hands: In Figure 19, hands are shown interacting with the blocks. The interactive hands can grasp, move and throw objects, and the gestures of the virtual hands are coupled to the real hands’ gestures. Their interaction behavior is controlled by Leap Motion’s own core assets.

Blackboard with instructions: This prototype has a static blackboard with instructions for the level, as illustrated in Figure 20. The text in black background is a texture that was added to a board. This element is probably the prototype’s weakest point, due to its lifelessness.

Docks: The docks are blocks in shape of platforms where you place the corresponding building blocks, presented in Figure 21. When you place the right building block on top of the right dock, it turns green, otherwise it turns red. Once you place all the three building blocks on the corresponding docks, and they turn green, you pass the level.

Scoreboard: The scoreboard in Figure 19 consists of GUI text that is linked to a score managing script, which was coded by me. The score system is an entertaining aspect, since it is possible to have negative results, and you can

Figure 20 Blackboard with instruction

1

Figure 21 Docks where you place the corresponding blocks.

Figure 19 Interactive hands made through Leap Motion hand gesture tracker

Figure 18 Different types of blocks for each type of programming concept

2018-01-04 (Dnr. ) 41 (av 61)

also cheat and get high results by, for instance, placing a building block on its correspondent dock several times. For each time you place a block on the wrong place, you lose one point.

The robot: The robot in Figure 22 was not programmed to show an output that corresponds to what you are supposed to “code” with the blocks. In this

case, it was about moving forward by 100 pixels, but the only thing that happens is that each of the robot’s body parts will turn green for every right move you make.

4.6.2.2 Prototype 2

Prototype 2 is a result of iterations based on feedback from participants in the first round of usability tests. Below follows description of the changes made in the prototype’s system.

Figure 22 The robot that is responsible for the visual feedback

Figure 23 The iterated prototype, used in the last round of usability test

2018-01-04 (Dnr. ) 42 (av 61)

Prototype 2's system changes

Building blocks (Figure 24): The interactive building blocks underwent some changes – a block that represents "loop" (for/while) was added in the mix, which was considered by test participants as a clear metaphor. The blocks' interaction behavior was changed, it became easier to grasp and move the objects once Leap Motion was smudge-free.

Interactive hands: The interactive hands did not suffer any alterations, since they got very positive feedback from usability test participants.

Blackboard with instructions: The blackboard experienced significant changes as seen in Figgure 25 – from static to a dynamic GUI text panel, controlled by a script. This alteration was made as a step further toward a multi-level prototype, so that each level has a different instruction, just by changing the input text. The readability was perceived as improved as well.

Docks: Instead of platforms as illustrated in Figure 26, the new docks are now boxes, a change reflected by received feedback from usability test participants. The reason is that it feel more intuitive, according to them, to move objects from a box to another, than let them rest on a platform.

Scoreboard: The scoreboard illustrated in Figure 25was split into two parts – misses, which keeps track of your mistakes, and score for points earned. As mentioned previously, it is now harder to cheat, due to scripting alterations.

Figure 24 The new building blocks

Figure 25 Dynamic blackboard with instructions

Figure 26 The new docks are transparent, labeled and boxes instead of platforms

2018-01-04 (Dnr. ) 43 (av 61)

It is not possible to get negative points in this prototype, and once you score a point, the building block is destroyed, which implies impossibility of placing the same object in the right dock several times.

The teaching robot (Figure 27: After the initial usability test sessions, I decided to turn the robot from Figure 12 into a teacher with a pointer stick, mainly for entertainment purpose. The robot has one function though – show the instructions and then it leaves the room.

David, the robot (Figure 28): This robot was added in this prototype to show the output of the “written” code.

The previous prototype did not have a relevant visual feedback. Hence David, the robot was placed on the desk where the building blocks are manipulated, and once the code is right, the robot executes the command, right in front of you.

4.7 Usability Test Usability test was realized twice – first after the co-design workshop and the second after prototype iteration. The first one took place at Design workshop at Orkanen building, whereas the other was hosted at the IOTAP (Internet of Things and People) lab, at Malmö University. Recruitment of participants was carried out through Facebook, in Interaction Design group pages, and privately using Facebook messenger.

4.7.1 Usability test 1 In the first usability test, there were five participants – both from Interaction Design and similar programs.

Figure 27 A robot teacher that shows the instructions and leaves the room

Figure 28 David, the robot executes the code commands

2018-01-04 (Dnr. ) 44 (av 61)

The reception of my concept and prototype was mostly positive, and a lot of constructive criticism or suggestions were received, as described below.

Overall, the concept was fun and interesting, and with great potential to make Programming more accessible and graspable. However, this usability test showed that the concept would be more suitable for people without prior or very limited programming knowledge. The reason is that block programming does not have same logic as textual code. Some of the testers had a prior knowledge of Programming, and they tried to “write” the code with the blocks in the same way as people do with textual code. Another reason is that the concepts explored by the prototype were very basic.

When it comes to the prototype itself, the general opinion was that it made Programming concepts more graspable. The metaphors used – building blocks – and the interaction were considered logical and made more sense than textual code. Another interesting observation is that even though the objects only exist in the virtual environment, they felt real for several participants. One of them stated “I know these blocks aren’t real, but I can feel them, as if they were in my hands, and I can play with them!”.

Furthermore, they seemed to have appreciated the “gamified” aspect of the concept – you had a score that could go down to negative numbers, and up to high numbers. And with this prototype, it was easy to “cheat” – you could place a block in a dock as many times as you wished.

Additionally, the VR experience was considered by the majority as immersive – “you feel like you are there, in the virtual environment”. When it comes to presence, I noticed that most of them were in a “bubble”, completely attentive to the virtual environment, and somewhat unaware of the real environment. One of the cues was when some of them would bump into some piece of furniture trying to grab a virtual object that fell on the virtual floor.

Regarding the prototype visual design, there is not so much consensus – for some people, the environment should be something else than a classroom. One of the great advantages of VR is creating unique, unforeseen environments and objects, and a classroom is just too mundane. However, some participants appreciated the fact that it was a simpler classroom, so it becomes easier to focus on the task. For the majority, the classroom could have been a little bit spacier though – it felt claustrophobic since the desk was too close to the one of the walls. Moreover, some people reacted to the floor that was higher than the actual floor, but at same time they understood it could be an advantage, since it becomes easier to pick an abject that falls on the floor. And finally, the interactive objects were somewhat hard to interact with – some were “sticky” and others were too “slippery” and hard to grasp, which was experienced by some as frustrating. One possible explanation is Leap Motion’s tracking accuracy, or lack of. It is very sensitive to light and

2018-01-04 (Dnr. ) 45 (av 61)

smudge. The other explanation is due to the objects’ “collider” settings, which can easily be changed in Unity.

In relation to the prototype’s visual feedback, the general opinion is that it should be clearer when you make a mistake or make a right move. As feedback, a robot standing on the front would have its parts become green or red, and the dock where you put the blocks would also change its color depending on if it was right or wrong block. In other words, you had feedback in two different places, which was experienced as confusing. When you complete the task, you do not have any feedback to let you know that you are done, and nothing happens to the robot either – “wasn’t it supposed to move or something?”.

The fact my prototype lacked visual feedback to signalize completion was premeditated – my objective was to receive their input on what they would like this visual feedback to be like, since I adopted a more participatory/co-design approach. The plan was to use their input, or some of it in the following prototype.

Summarizing, the overall reception was very positive – the concept was considered relevant and fun, the prototype was easy to understand, the mechanics was very logical, and the environment was pleasant, asides aforementioned reservations. The interaction behavior of the virtual objects could though have been better, i.e. the manipulation of the objects should be smoother, but not something that annoyed them to the extent that it ruins the experience. And finally, my main goal was achieved – the VR experience

Figure 29 The screen reflects what the tester is doing

2018-01-04 (Dnr. ) 46 (av 61)

was perceived as immersive, embodied, with a high sense of tangibility and entertaining.

4.7.2 Usability test 2 In the second usability test, there were four participants – a professor/researcher in Interaction Design and Computer Science, two Interaction Design students and one Media Production student. For this test, an iterated prototype/demo was used, based on the inputs from former test session.

In this session, the focus was on the environment that was changed drastically, on the scoring system that underwent substantial modifications, and the visual feedback, which experienced the most significant alterations. When it comes to the interaction behavior of objects, there were minor changes, which were not taken into consideration in my assessment.

The reception of this version was very positive, especially when it comes to the environment. It was a minimalistic, futuristic and spacious room, decorated spartanly with some Programming-related posters. It was perceived as slightly dark though, but nothing that affected the VR experience. It was also suggested to look more like a classroom and less like a sci-fi inspired experiment bunker. However, it was not unanimous, since some of the participants liked this futuristic aspect. Some people also suggested that the Programming posters should be more visible in the room, put close to the GUI text in the front of the room, and maybe even these posters could be interactive, something you can grasp and read it more easily. A smaller detail that also received some criticism was the labels on the boxes – they were in a position that affected negatively the readability. Moreover, some of the participants mentioned that there could be some GUI text with instructions, before you are sent to the room. As the prototype was only one-level demo, adding instructions did not came into mind.

When it comes to immersiveness and presence, all the participants were impressed on how real it feels, and they felt as if they were there, in the futuristic room. Once the VR headset was removed, some of them were wondering “what happened with the desk that was here?”. For one of the participants, VR is generally scary, and this person does not like the feeling that you are being “teleported” to a place that does not exist. VR experience feels like you are escaping from reality, according to this participant. Another aspect was embodiment – for the participants it felt very embodied, since you are using your both hands interact with the virtual objects. Leap Motion, as it was mentioned previously, is not flawless since it can be too sensitive and lack accuracy. However, it is as close as it gets to the real thing, and using hand controllers instead could be more accurate, but then the VR experience would lose a lot of embodiment, a crucial aspect to my concept.

2018-01-04 (Dnr. ) 47 (av 61)

Another mentioned aspect was tangibility – even though there were no real objects to interact it, the interaction with the virtual objects felt real for the participants – they could feel them even though they were only in the virtual environment. And again, there were comments such as “oh it feels so weird that the objects are all gone now”.

And finally, the visual feedback was perceived as much clearer this time – I created two types of “score”, one for misses and the other for right moves. When the person completes successfully the task, a tiny robot on the table moves forward, accordingly to what the person “coded” with the blocks. In other words, the output corresponds to the input.

In conclusion, the iterated prototype received much more positive feedback in all aspects: environment, immersiveness, presence and visual feedback. There is obviously a lot of room for improvement, especially concerning design and smoothness of interaction behavior.

Figure 30 The objects feel real, which can be noticed d in the hand gestures

2018-01-04 (Dnr. ) 48 (av 61)

5 Results My thesis work had as purpose to find an answer to following questions:

– In what ways can VR-based learning practices can facilitate the understanding of computer science concepts/programming and,

– In what ways can it be motivational and engaging for the students?

To answer these questions, an extensive background research was realized, and interviews with teachers, experts and students were carried out, workshops, prototyping and usability tests were also central to my research.

Regarding the first question, there seems to be a consensus on that VR has some features that give it a great advantage in relation to other technologies when it comes to the learning process. Firstly, there is immersion and great potential for presence. The learner becomes a part of the virtual environment, and his/her focus is entirely guided toward the tasks to be solved, without any external disturbance. Empirical and theoretical texts affirm it, and my usability tests corroborated this statement as well. Secondly, VR enables both the materialization of abstract concepts with the help of metaphors as well as visualization of phenomena, which cannot be done as satisfactorily in a two-dimensional desktop-based platform. For instance, in Scratch, which is a two-dimensional block programming platform, if you set a speed of a moving character too high, you cannot see the movement between point A and B. In VR, you can see what is happening – an object moving extremely fast. Thirdly, there is a very important aspect of VR, namely embodiment – as theory and empirical research show in the field of learning and Interaction Design, the use of body parts facilitates the learning process, by removing some of the load from the cognitive system. An example is that counting with your fingers makes basic Math operations easier, and with VR, you can use your hand to manipulate objects and solve puzzles. And finally, there is one aspect that is tightly connected to embodiment, namely tangibility.

When it comes to tangibility within VR it can be debatable whether it is achievable or not. You can feel as if you are touching, manipulating real objects with your VR goggles on. However, these objects only exist in VR, so how can we talk about tangibility? One can argue that the feeling of tangibility is so strong, that VR seems to challenge the traditional definition.

In conclusion, the solution I came up to tentatively answer this question was to create a prototype that highlights all these features VR excels at. The prototype is inspired by Scratch, and translates it to a three-dimensional platform, in a virtual environment. Elements such as variables, operators, constructs and data types are represented by building blocks of different shapes, colors, sizes and textures. People are familiar with wooden building blocks and played with them when they were young. Therefore, I used them

2018-01-04 (Dnr. ) 49 (av 61)

as metaphor for “writing” code. And with the help of Leap Motion attached to VR headset, you can manipulate these virtual objects with your hands. Additionally, a non-distracting environment was achieved by designing it in a minimalistic way – there is a table, a teaching robot, a GUI text panel with instruction and score, a mini robot that executes what the learner coded with the blocks, and some Programming-related posters, and that is all. That means, there is nothing in the room that will steal the attention from the interactive objects and the task to be completed. As it was mentioned previously, people perceived my prototype as logical, clear, incredibly immersive, embodied, and they had a great sensation of tangibility.

Concerning the second question, i.e. “in what ways can VR be motivational and engaging for the students?”, this research and previous work showed that one of the major obstacles for learning is the fear of failing, making mistakes and the high initial learning threshold make people not even try to learn – “it seems so complex, so why even bother?”. On the other hand, students can feel motivated when there is some reward involved, studies have also shown. In other words, a way to motivate students into learning programming would be something that lowers the initial learning threshold and makes learning programming seem conceivable and easier. The learner with a greater likelihood of succeeding since the threshold has been lowered will probably feel more motivated to keep studying. Another aspect within the motivation literature that could be applied here, is gamification of learning. People feel engaged when there is reward and punishment involved – “if I do something right, I get points and go to the next level, and if I lose points, I will have to make an extra effort to gain them back. Additionally, an even stronger motivating factor is that the learner sees the direct result of his/her work, just right in front of him/her.

My prototype made use of these aspects, by having a score for points earned, and misses, and by showing the direct result of successfully “written” code. If you make too many misses, you get some encouragement, and if you accomplish the task with no or very few mistakes, you get congratulated by a robot version of David Cuartielles, lecturer in interactive objects and Arduino co-founder, who moves forward, just like the way the learner coded it to do.

My prototype was perceived as entertaining, and more like a game than a traditional learning platform. During the usability test, my prototype kept them engaged for a long while, since they found interesting how you could interact with the objects to write a code, and that there was a score keeping track of your performance, and even “cooler” was the possibility of cheating.

2018-01-04 (Dnr. ) 50 (av 61)

6 Discussion 6.1 Significance of the Study

Whether you want to uncover the secrets of the universe, or you just want to pursue a career in the 21st century,

basic computer programming is an essential skill to learn.

(Stephen Hawking 1942-2018, CODE)

This thesis tackles a pressing issue – digital literacy and programming skills are considered increasingly vital in our highly technology-driven age, and yet, there is still a shortage of programming skills. According to Moreno-León and Robles (2015), basic programming skills will be required for many jobs in the nearest future. According to the European Commission, the demand for Information and Communications professionals is growing annually by 3%, whereas the number of Computer Science graduates is not keeping up. Therefore, within some years there will be many available job positions for Computer Science specialists that will not be filled, despite the high unemployment rate in Europe (Europa.eu, 2014).

The contributions of the current research are to offer a solution to facilitate the understanding of programming concepts by making them more concrete and by enhancing embodiment and increasing the motivation and engagement of students in Programming courses, by resorting to gamification of Programming learning platform, and lowering the initial learning threshold.

Moreover, even though there are some projects trying to achieve similar goals, and researches that aims at investigating the potential of VR in learning, with focus on STEAM subjects – Science, Technology, Engineering, Arts and Mathematics, there is still a lack of work – both in the industry and research – when it comes to the use of VR as a tool for Programming education and how it affects conceptual understanding, as Lindgren et al. (2016) points out. And that is what this thesis work attempts to do – investigate not only how VR can be used to facilitate understanding of complex and abstract concepts within Programming, but also draw an analysis on in which ways VR can affect conceptual understanding, with focus on 1) embodiment, 2) immersiveness & presence, 3) tangibility and 4) metaphorization and visualization of abstract concepts.

2018-01-04 (Dnr. ) 51 (av 61)

6.2 Challenges I believe that my concept has a great potential. Nevertheless, I am aware that its implementation would be a considerable challenge. As Hannafin and Vermillion (2008) argue, for a new technology to be successfully integrated in the classroom, it is important for the teacher to be on board and show a positive attitude toward new artefacts.

During my research, some teachers and other pedagogical practitioners have shown skepticism toward VR technology. According to them, AR presents more potential than VR, due to its low cost, user-friendliness and collaborative potential. VR, on the other hand, is considered unviable economically, and not particularly collaboration-friendly. There are probably many more teachers who are skeptical in relation to VR, either due to its relatively high cost or since VR is relatively new in the realm of education, and there is a lot of uncertainty around its potential benefits. One can argue that AR is also relatively new, but differently from VR, it is very accessible economically when inbuilt in mobile phones, which means low risk as well.

Let us suppose that teachers, and the department are on board with implementing this concept in Programming. There is one more challenge – how to make this new technology a regular aspect of learning practice. In other words, there is no way to assure that once implemented in the curriculum, the technology and concept will be used on a regular basis in the classroom.

And finally, there is the economical and administrative challenge of implementing a relatively expensive technology. The decision is not up to the teachers to make – for every department, there is a predetermined budget, and the expenses should be approved by the head of department.

6.3 Self-critique This research and results have exceeded my expectations. I was afraid that I would not have time to develop a working prototype but instead rely on “Wizard of Oz”, i.e. a type of sketch that mocks up something that users can experience, and enables us to explore design concepts in action (Buxton, 2007)

The reason for that was that too much time was spent on “Discover” phase – the target group and scope were not defined for a long time. The dilemma was whether I should focus on K-12 students, due to the curriculum reform that makes programming obligatory from first grade to high school senior year, or on Interaction Design students who are expected to know how to program in order to create interactive artefacts and interfaces, but still show lack of skills, with some exceptions. For pragmatic reasons, the latter group was chosen – familiarity with the target group and the issues they go through, the easy access to teachers and students who do not need their parents to sign a Letter

2018-01-04 (Dnr. ) 52 (av 61)

of Consent. Additionally, after interviewing some K-12 teachers, I came up with the conclusion that they are not ready yet to adopt this technology – the teachers are still confused about what these curriculum changes entail in practice, since the directive document (Skolverket, 2017) drafted by the Swedish National Agency for Education is very vague. Nonetheless, soon it would be relevant to develop the concept further to include K-12 students, since there is a need for more innovative and effective tools for teaching programming concepts.

Even though it would have been interesting to focus on K-12 students, Interaction Design students as target group proved to be relevant – they are being prepared for the labor market, and programming skills are vital to “interact” with the material produced – that is, if someone wants to develop something functional, interactive and aesthetically appealing, this person should have some programming skills at least to know whether the concept is achievable or not, and have a good communication with programmers in the team.

If less time was spent with the “Discovery” phase, I would have time to conduct an extra co-design session, including teachers and more usability tests with even further iterated prototype, for validation purposes.

This research relied a lot on students’ and teachers’ insights, but there should have been more participatory, and put these two groups working together in a co-design workshop. Their insights were given separately, so maybe the outcome would have been different if they had worked together during the ideation phases.

Finally, if I were supposed to conduct this research again, I would focus more on the design aspects of the prototype, including the interaction behavior of the virtual objects, and a little bit less on the technical aspects, such as complex coding.

6.4 Implications for future research After achieved success in creating this prototype, there are some iterations and idea implementations that will be carried out. As mentioned previously, my prototype consisted of one level. The next step will be to create more levels with increasing degree of difficulty, i.e. more advanced tasks and a wider variety of Programming operators, functions, data types and variables.

A second measure will be to reduce the gap between block and textual programming, by translating the code “written” with building blocks to textual code, for example Python. It might sound complicated, but I contacted a software developer at Unity who has been helping me to make these implementations.

2018-01-04 (Dnr. ) 53 (av 61)

Furthermore, in the future, a research will be conducted at a larger scale – a significantly more expressive number of students will be recruited, and qualitative methods will be combined with quantitative methods, so as to achieve scientific validity. An example is using survey to ask students about their attitudes and perceptions regarding programming, use of technology, and VR as a learning tool, both prior to usability test and after, in order to observe behavioral changes. Also, the use of quizzes would be a good way to objectively measure how much the VR learning platform helped in understanding programming concepts.

Finally, and foremost, there is something I wanted to implement during the current thesis work, but it was not feasible due to time constraints – tangibility in VR. As Harley et al. (2017) point out, only the hands are tracked in VR, and the rest of the body is left out. Furthermore, tangible Interaction with virtual objects consists mostly of haptic feedback of the controllers. These factors might cause discomfort, since there is a mismatch what you do in the virtual environment and what happens in the real environment. For instance, you manipulate objects in the virtual environment, but they do not exist.

That would contribute both to current VR experience, by adding accuracy and consistency between the real world and the virtual one, as Harley et al. (2017) suggest, as well to research, since tangibility in VR is understudied.

2018-01-04 (Dnr. ) 54 (av 61)

7 ReferencesAuld, L. W. S., Pantelidis, V. S. (1994) Exploring virtual reality for classroom use: The Virtual Reality and Education Lab at East Carolina University, Tech Trends, 39(2), 29-31

Barbera, E., Garcia, I., & Fuertes-Alpiste, M. (2017). A co-design process microanalysis: Stages and facilitators of an inquiry-based and technology-enhanced learning scenario. International Review of Research in Open and Distance Learning, 18(6), 104–126. https://doi.org/10.19173/irrodl.v18i6.2805

Blockly │ Google Developers (n.d). Introduction to Blockly. Available at: https://developers.google.com/blockly/guides/overview. Last accessed: May 18, 2018.

Brophy, J. E. (2013). Motivating students to learn. New York, NY: Routledge.

Buckley, P., & Doyle, E. (2016). Gamification and student motivation. Interactive Learning Environments, 24(6), 1162–1175. https://doi.org/10.1080/10494820.2014.964263

Buxton, B. (2007). Sketching User Experiences: Getting the Design Right and the Right Design. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc.

Cecil, J., Ramanathan, P., & Mwavita, M. (2013). Virtual learning environments in engineering and STEM education. 2013 IEEE Frontiers in Education Conference (FIE), 502-507. doi:10.1109/FIE.2013.6684874

Chang, J. S.-K., Yeboah, G., Doucette, A., Clifton, P., Nitsche, M., Welsh, T., & Mazalek, A. (2017). Evaluating the effect of tangible virtual reality on spatial perspective taking ability. Proceedings of the 5th Symposium on Spatial User Interaction - SUI ’17, 68–77. https://doi.org/10.1145/3131277.3132171

CODE (n.d). Leaders and trend-setters all agree on one thing. Available at: https://code.org/quotes. Last accessed on May 18, 2018

deMarrais, K. (2004). Qualitative interview studies: Learning through experience. In K. deMarrais & S. D. Lapan (Eds.), Foundations for research (pp. 51–68). Mahwah, NJ: Erlbaum.

Design Council (n.d.). The Design Process: What is the Double Diamond? Available at: https://www.designcouncil.org.uk/news-opinion/design-process-what-double-diamond. Last accessed on May 18, 2018

European Commission (2014). Digital Agenda for Europe: Rebooting Europe's Economy. European Commission. DG for Communication

2018-01-04 (Dnr. ) 55 (av 61)

Publications Office of the European Union. Available at: http://eige.europa.eu/resources/digital_agenda_en.pdf. Last accessed on May 18, 2018.

Fitzmaurice, G.W. Bricks: Laying the foundations for graspable user interfaces. In Proceedings of the Conference on Human Factor in Computing Systems (CHI95), Denver, CO, USA,7–11 May 1995.

Gagné, Marylene, Jacques Forest, Maarten Vansteenkiste, Laurence Crevier-Braud, Anja Van den Broeck, Ann Kristin Aspeli, Jenny Bellerose, et al. (2015). The multidimensional work motivation scale: validation evidence in seven languages and nine countries. European Journal of Work and Organizational Psychology, 24(2), 178–196.

Goldin-Meadow, S., Nusbaum, H., Kelly, S. D., and Wagner, S. (2001). Explaining math: gesturing lightens the load. Psychol. Sci.12, 516–522. doi: 10.1111/1467- 9280.00395

Goldschmidt, G. (1991). The dialectics of sketching. Creativity Research Journal, 4(2), 123–143.

Goodman, E., Kuniavsky, M., & Moed, A. (2012). Usability tests. Observing the User Experience, 273-326. doi:10.1016/B978-0-12-384869-7.00011-5

Goodwin, M. S., Wiltshire, T., & Fiore, S. M. (2015). (2015). Applying research in the cognitive sciences to the design and delivery of instruction in virtual reality learning environments. Paper presented at the Virtual, 280-291.

Hannafin, R. & Vermillion, J. (2008). Technology in the classroom. In T. L. Good 21st century education: A reference handbook (Vol. 2, pp. II-209-II-218). Thousand Oaks, CA: SAGE Publications, Inc. doi: 10.4135/9781412964012.n73

Harlen, W., & Deakin Crick, R. (2003). Testing and motivation for learning. Assessment in Education: Principles, Policy & Practice, 10(2), 169–207. doi:10.1080/0969594032000121270

Harley, D., Tarun, A. P., Germinario, D., & Mazalek, A. (2017). Tangible VR. Proceedings of the 2017 Conference on Designing Interactive Systems - DIS ’17,1253–1263. https://doi.org/10.1145/3064663.3064680

Heintz, F., & Mannila, L. (2018). Computational Thinking for All: An Experience Report on Scaling up Teaching Computational Thinking to All Students in a Major City in Sweden. SIGCSE.

Hersh, M. & Leporini, B. (2012). Accessibility and Usability of Educational Gaming Environments for Disabled Students. Proceedings of the 12th IEEE International Conference on Advanced Learning Technologies, ICALT 2012. 752-753. 10.1109/ICALT.2012.181.

2018-01-04 (Dnr. ) 56 (av 61)

Hubbard, R. & Sipolins, A. & Zhou, L. (2017). Enhancing learning through virtual reality and neurofeedback: a first step. 398-403. 10.1145/3027385.3027390.

Ishii, H. Tangible bits: Beyond pixels. In Proceedings of the 2nd International Conference on Tangible and Embedded Interaction, Bonn, Germany,18–21 February 2008.

Jerald, J. (2016). The VR Book: Human-Centered Design for Virtual Reality. ACM Books. https://doi.org/10.1145/2792790

Kalelioğlu, F.(2015). A new way of teaching programming skills to K-12 students: Code.org. Computers in Human Behavior. 52: 200210.

Kapp, K. (2012). The gamification of learning and instruction: Game-based methods and strategies for training and education. San Francisco, CA: Pfeiffe

Kockler, H., Scheer, L., Tepest, R., David, N., Bewernick, B. H., Newen, A, et al. (2010). Visuospatial perspective taking in a dynamic environment: Perceiving moving objects from a first-person-perspective induces a disposition to act. Consciousness and Cognition, 19, 690-701.

Lakoff, G. & Johnson, M. (1999). Philosophy in The Flesh: The Embodied Mind And Its Challenge To Western Thought. Basic Books, New York

Lawson, B. (2006). How Designers Think – The Design Process Demystified. University Press, Cambridge.

Lee, J., & Hammer, J. (2011). Gamification in education: What, how, why bother? Academic Exchange Quarterly, 15(2), 146.

Lindgren, R. (2012). Generating a learning stance through perspective-taking in a virtual environment. Computers in Human Behavior, 28(4), 1130–1139.

Lindgren, R., Tscholl, M., Wang, S & Johnson, E. (2016). Enhancing Learning and Engagement through Embodied Interaction within a Mixed Reality Simulation. Computers & Education. 95. 10.1016/j.compedu.2016.01.001.

Lindgren, R & Johnson-Glenberg, M (2013). Emboldened by Embodiment: Six Precepts for Research on Embodied Learning and Mixed Reality. Educational Researcher. 42. 445-452. 10.3102/0013189X13511661.

Madary, M., & Metzinger,T. (2016). Real virtuality: A code of ethical conduct: Recommendations for good scientific practice and the consumers of VR technology. Frontiers in Robotics and AI. https://doi.org/10.3389/frobt.2016.00003.

McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. New York, NY, US: Penguin Press.

Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco, CA: John Wiley & Sons.

2018-01-04 (Dnr. ) 57 (av 61)

Moreno-León, J. & Robles, G. (2015). The Europe Code Week (CodeEU) initiative: Shaping the skills of future engineers. 10.1109/EDUCON.2015.7096025.

Ortega, F. R., Bolivar, S., Bernal, J., Galvan, A., Tarre, K., Rishe, N., & Barreto, A. (2017). Towards a 3D virtual programming language to increase the number of women in computer science education. 2017 IEEE Virtual Reality Workshop on K-12 Embodied Learning through Virtual & Augmented Reality (KELVAR), , 1-6. doi:10.1109/KELVAR.2017.7961558

Papert, S. (1980). Mindstorms: children, computers and powerful ideas. Brighton: Harvester Press.

Patton. M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage Publications.

Penuel, W.; Roschelle, J.; Shechtman, N. (2007). Designing Formative Assessment Software with Teachers: an Analysis of the Co-Design Process. Research and Practice in Technology Enhanced Learning. 2. 51-74. 10.1142/S1793206807000300.

Piaget, J.(1976) The Grasp of Consciousness; Harvard University Press: Cambridge, MA,USA

Preece, J., Rogers, Y., & Sharp, H. (2015). Interaction design: Beyond human—computer interaction (4th ed.). Chichester, UK: John Wiley & Sons Ltd

Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J.S., Silverman, B., & Kafai, Y.B. (2009). Scratch: programming for all. Commun. ACM, 52, 60-67.

Ryan, R. M., & Connell, J. P. (1989). Perceived locus of causality and internalization: Examining reasons for acting in two domains. Journal of Personality and Social Psychology, 57, 749 761

Schmidt, A. (2016). Increasing Computer Literacy with the BBC micro:bit. IEEE Pervasive Computing. 15. 5-7. 10.1109/MPRV.2016.23.

Seo, J.H., Smith, B.M., Bruner, M., Cook, M.M., Suh, J., Pine, M., Malone, E., Leal, S., Sueda, S., & Bai, Z. (2017). Anatomy builder VR: comparative anatomy lab promoting spatial visualization through constructionist learning. SA '17.

Skolverket, 2017. Tydligare om digital kompetens i läroplaner, kursplaner och ämnesplaner.

Available at: https://www.skolverket.se/skolutveckling/resurser-for-larande/itiskolan/styrdokument . Last accessed on May 18, 2018.

Spikol, D. & Milrad, M. & Maldonado, H.& Pea, R.(2009). Integrating Co-design Practices into the Development of Mobile Science Collaboratories. 393-397. 10.1109/ICALT.2009.175.

2018-01-04 (Dnr. ) 58 (av 61)

Steam Community (2018). Primitive. Available at: https://steamcommunity.com/app/777890. Last accessed on May 18, 2018.

_____________(2016). GlitchSpace. Available at: https://steamcommunity.com/app/290060. Last accessed on May 18, 2018.

Stigall, J., & Sharma, S. (2017). Virtual reality instructional modules for introductory programming courses. 2017 IEEE Integrated STEM Education Conference (ISEC), 34-42.

Swedish Research Council (2017). Good Research Practice. Stockholm: Vetenskapsrådet. https://publikationer.vr.se/produkt/god-forskningssed/ Last accessed on May 18, 2018.

Taljaard, J. (2016). A review of multi-sensory technologies in a Science, Technology, Engineering, Arts and Mathematics (STEAM) classroom. Journal of Learning Design, 9(2), 46–55

The British Psychological Society. (2014). Code of Human Research Ethics. Available at: http://www.bps.org.uk/system/files/Public%20files/code_of_human_research_ethics_dec_2014_inf180_web.pdf . Last accessed: May 15, 2018.

Winn, W. (2003). Learning in Artificial Environments: Embodiment, Embeddedness and Dynamic Adaptation. Technology, Instruction, Cognition and Learning. 1.

Witmer, B.G. and Singer, M.J. (1998). Measuring presence in virtual environments: A presence questionnaire, Presence: Teleoperators and Virtual Environments, 7, 225-240.

Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. ACM International Conference Proceeding Series. 10.1145/2601248.2601268.

Vallerand, R. J., Pelletier, L. G., Blais, M. R., & Briere, N. M. (1992). The academic motivation scale: A measure of intrinsic, extrinsic, and amotivation in education. Educational & Psychological Measurement, 52(4), 1003-1017

Vincur, J., Konopka, M., Tvarozek, J., Hoang, M., & Navrat, P. (2017). (2017). Cubely: Virtual reality block-based programming environment. Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology doi:10.1145/3139131.3141785

VRFocus (2016). Live Code for VR in Experimental Rumpus. Published by Peter Graham, Jul 4, 2016. Available at: https://www.vrfocus.com/2016/07/live-code-for-vr-in-experimental-rumpus/. Last accessed on May 18, 2018.

Zichermann, G. & Cunningham, C. (2011). Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps.

2018-01-04 (Dnr. ) 59 (av 61)

Ziemke, T. (2003). What's that Thing Called Embodiment? Proceedings of the 25th Annual meeting

2018-01-04 (Dnr. ) 60 (av 61)

8 Appendix 1. LETTEROFCONSENTDURINGUSABILITYTESTSESSIONS

2018-01-04 (Dnr. ) 61 (av 61)

2. INTERVIEWGUIDE

Teachers

Do you consider programming important? Why?

Could you tell me about the Programming “crisis” within Interaction Design?

Could you talk a little bit about the course content?

What do you think the students have most difficulty with when it concerns Programming?

What can be said about the students’ engagement and motivation toward Programming?

How do you tackle these issues, i.e. what do you do when you notice the students have difficulty or lack motivation?

What’s your pedagogical approach? Use of metaphors? Connect programming to real life?

Do you believe that using innovative methods can help the understanding of abstract concepts? If so, in which way?

What is your opinion about VR as a learning tool? Pros and Cons?

What are the other alternatives to VR you can think about?

Students

What do you think about Programming? What is hard with programming?

How do you perceive this “Programming crisis” within Interaction Design?

What do you think about the teachers’ pedagogical skills?

How could they improve the course and help people with difficulty?

What is your opinion about VR in general?

What is your opinion about VR as a learning tool? Do you think it could facilitate the understanding of Programming concepts?

What are the pros and cons with VR? Any other alternative to this technology to tackle the issue?

Can ue of metaphors facilitate the understanding? Or visualization?

Do you have any special learning technicques to facilitate the learning process? If so, what?