Using Gamification to Orient and Motivate Students to ...igor.pro.br/publica/papers/CHASE2017.pdf · literature brings approaches to improve existing processes and motivate developers

Using Gamification to Orient and Motivate Students to Contribute to OSS projects

Guilherme C. Diniz*, Marco A. Graciotto Silva* * Computing Department,

Federal University of Technology – Paraná Campo Mourão, PR, Brazil

[email protected], [email protected]

Marco A. Gerosa#, Igor Steinmacher*,#

# School of Informatics, Computing, and Cyber Systems Northern Arizona University

Flagstaff, AZ, USA [email protected], [email protected]

Abstract—Students can benefit from contributing to Open Source Software (OSS), since they can enrich their portfolio and learn with real world projects. However, sometimes students are demotivated to contribute due to entrance barriers. On the other hand, gamification is widely used to engage and motivate people to accomplish tasks and improve their performance. The goal of this work is to analyze the use of gamification to orient and motivate undergraduate students to overcome onboarding barriers and engage to OSS projects. To achieve this goal, we implemented four gaming elements (Quests, Points, Ranking, and Levels) in GitLab and assessed the environment by means of a study conducted with 17 students, within a real OSS project (JabRef). At the end of the study, the students evaluated their experience through a questionnaire. We found that the Quest element helped to guide participants and keep them motivated and points helped by providing feedback on students’ performed tasks. We conclude that the gamified environment oriented the students in an attempt to make a contribution and that gamification can motivate and orient newcomers’ to engage to OSS projects.

Keywords - newcomers; open source software; engagement; gamification; motivation; students

I. INTRODUCTION

The Open Source Software (OSS) model has become an important driving force in today’s software development, resulting in many prominent projects that are used extensively through the entire development stack, from kernels to sophisticated end-user applications. Therefore, it is no surprise that the OSS movement attracts a large, globally distributed community of volunteers. In addition, the number of job vacancies valuing open source knowledge and experience has been rising on a regular basis [1]. This fact motivated students to contribute to OSS projects, and teachers to bring OSS practices and process to the classroom and foster students contributing to OSS projects [1–3].

Exposing students to OSS projects is beneficial both to OSS communities and students. From the perspective of the communities, more professionals will be in touch with OSS, ultimately leading to a higher number of contributions, since students are potential OSS contributors. They usually have the basic theoretical knowledge to contribute to a project, but lack practical skills and knowledge about the underlining technologies. From the perspective of the students, contributing is beneficial since working on OSS projects

enables them to learn real-world skills, attitudes, and experiences [2], [3], which might increase their confidence when applying for industry jobs. One can claim that this is also beneficial since the future workforce of software developers is being prepared practicing in a real scenario.

However, it is already known that newcomers, including the students, face many barriers while attempting to contribute to OSS [4–6]. Some barriers they face include orientation issues that can potentially demotivate newcomers from placing their first contribution. Many recent studies had being conducted aiming to engage and motivate newcomers to OSS projects [4], [7–9].

Gamification, which consists on the application of game elements in non-gaming contexts [10], is calling attention recently since it has been successfully applied to motivate and engage contributors of online and collaborative communities and business [11], [12]. These gaming elements had been successfully applied to different domains, including Software Engineering [12–14] and learning [15], [16]. From the best of our knowledge, there are no initiatives or studies focusing on the use of gamification to engage and support students or newcomers to overcome barriers to join OSS projects.

Therefore, the goal of this work is to propose and analyze the use of gamification elements to motivate and support undergraduate students to overcome the orientation barriers to contribute to OSS projects. To achieve this goal, we have selected four game design elements to help students: Rankings, Quests, Points, and Levels [14], [16], [17]. For each of these elements, we defined a set of rules to describe the operation of the gamified environment and implemented them on GitLab1. To assess the environment, we conducted a study with 17 undergraduate students, who evaluated their experience by means of a questionnaire. This was a preliminary study towards a gamified environment that can be used to foster contributions from students, benefiting both projects and students. The contributions of this study include presenting a gamified version of GitLab; assessing the use of Rankings, Points, Quests, and Levels to motivate and support newcomers to contribute to OSS projects; and proving insights to community building teams on how to use gamification elements to guide newcomers.

1 http://www.gitlab.com

use of game design elements in nonGamifgreat impact in different areasmeansStackOverpoints that can be earned by the users when they perform some tasks.

of intereststudies used newsfeeds and ranking to gamify a version control system to foMelo et alelements to encouragedevelopers. gamrewards,and the quality of their contributions. Snipes et al. mottecscoring systems in the context of software development.

and analyze the use of gamificatEducation. by gamifying aincrease student involvement documentet al.training people on defecthat are agile prpatterns

literature brings approaches to improve existing processes and motivate developers to produce more or better. From academic perspecgamification is mainly used with the purpose of improving teaching and learning. of gamification previously evidenced barriers faced [29]contribution. inin OSS projects, without focusing on teaching and learning.Given the attention being given to OSS onenvironment and the importance of OSS claimpr

elements ove

II. R

Gamification is defined by use of game design elements in nonGamification has being growing in the last years, presenting great impact in different areasmeans of engaging and motivatingStackOverflow and Wikipedia make upoints that can be earned by the users when they perform some tasks.

In software engineering, of interest recentlystudies [14], [16]used newsfeeds and ranking to gamify a version control system to foster more commits from students. Similarly, Melo et al. [21]elements to encouragedevelopers. Lotufogamified bug tracking system, based on reputation and rewards, would and the quality of their contributions. Snipes et al. [23]motivate developers to use more efficient development techniques, and Bacon et al.scoring systems in the context of software development.

In addition, and analyze the use of gamificatEducation. For example, by gamifying aincrease student involvement documentation, bet al. [26] presenttraining people on defecthat have taken are agile processespatterns [15].

From software engineering perspective, we see that the literature brings approaches to improve existing processes and motivate developers to produce more or better. From academic perspecgamification is mainly used with the purpose of improving teaching and learning. of gamification previously evidenced barriers faced [29], avoiding demotivation while contribution. Finitiative to gamify an OSS environment to engage students in OSS projects, without focusing on teaching and learning.Given the attention being given to OSS onenvironment and the importance of OSS claim that this work will presenting a new way of engaging students in OSS.

To achieve our goalelements to engage overcoming barriers to contribute to OSS projects, we

RELATED WORK ON

SOFTWARE

amification is defined by use of game design elements in non

cation has being growing in the last years, presenting great impact in different areas

of engaging and motivatingflow and Wikipedia make u

points that can be earned by the users when they perform

In software engineering, gamification recently [20], and already

[14], [16]. Singer and Schneider [25], for example, used newsfeeds and ranking to gamify a version control

ster more commits from students. Similarly, [21] proposed the use of a newsfeed and ranking

elements to encourage collaboration and commitmentLotufo et al.

fied bug tracking system, based on reputation and would encourage teams to increase th

and the quality of their contributions. [23] investigated

vate developers to use more efficient development niques, and Bacon et al. [24]

scoring systems in the context of software development.In addition, there are some studies that report experiences

and analyze the use of gamificatFor example, Sheth et al.

by gamifying a software engineering course increase student involvement

tion, bug reportingpresented a gamification proposal focused on

training people on defect trackinghave taken advantage of gamification to foster learning

cesses [27], mutant testing

From software engineering perspective, we see that the literature brings approaches to improve existing processes and motivate developers to produce more or better. From academic perspective of software engineering, we see that gamification is mainly used with the purpose of improving teaching and learning. In contrastof gamification as a strategy to previously evidenced barriers faced

, avoiding demotivation while From the best of our knowledge, there is no

itiative to gamify an OSS environment to engage students in OSS projects, without focusing on teaching and learning.Given the attention being given to OSS onenvironment and the importance of OSS

that this work will add to the statesenting a new way of engaging students in OSS.

III.

To achieve our goal of assessing the use of gamification engage and support undergraduate students

coming barriers to contribute to OSS projects, we

ORK ON GAMIFICATION AND

OFTWARE ENGINEERING

amification is defined by Deterding et al. use of game design elements in non-game contexts

cation has being growing in the last years, presenting great impact in different areas [11], [18], [19]

of engaging and motivating people [11]flow and Wikipedia make use of b


gamification has become a topic , and already was target of

Singer and Schneider [25], for example, used newsfeeds and ranking to gamify a version control

ster more commits from students. Similarly, proposed the use of a newsfeed and ranking

collaboration and commitmentet al. [22] investigated

fied bug tracking system, based on reputation and encourage teams to increase th

and the quality of their contributions. Also worth to mention, the use of a gamified system to

vate developers to use more efficient development [24] investigated

scoring systems in the context of software development.there are some studies that report experiences

and analyze the use of gamification on Software Engineering Sheth et al. [12], [25]

software engineering course increase student involvement on

ug reporting, and testing activitiesa gamification proposal focused on

t tracking. Other examples of advantage of gamification to foster learning

, mutant testing [28]

From software engineering perspective, we see that the literature brings approaches to improve existing processes and motivate developers to produce more or better. From

tive of software engineering, we see that gamification is mainly used with the purpose of improving

In contrast, in this paper we make use as a strategy to help students

previously evidenced barriers faced to join OSS , avoiding demotivation while they attempt to

rom the best of our knowledge, there is no itiative to gamify an OSS environment to engage students

in OSS projects, without focusing on teaching and learning.Given the attention being given to OSS onenvironment and the importance of OSS for the society

add to the statesenting a new way of engaging students in OSS.

METHOD

of assessing the use of gamification and support undergraduate students


AMIFICATION AND

NGINEERING

Deterding et al. [10] as “the game contexts

cation has being growing in the last years, presenting [11], [18], [19] , mainly as a

[11]. For example, se of badges and


has become a topic was target of secondary



collaboration and commitment investigated whether

fied bug tracking system, based on reputation and encourage teams to increase the frequency

Also worth to mention, the use of a gamified system to

vate developers to use more efficient development investigated the adoption of

scoring systems in the context of software development. there are some studies that report experiences

Software Engineering [12], [25] showed that

software engineering course it is possible development, activities. Thomas

a gamification proposal focused on examples of areas

advantage of gamification to foster learning [28], and design

From software engineering perspective, we see that the literature brings approaches to improve existing processes and motivate developers to produce more or better. From the

tive of software engineering, we see that gamification is mainly used with the purpose of improving

n this paper we make use students overcoming

to join OSS projects [5], attempt to place their

rom the best of our knowledge, there is no itiative to gamify an OSS environment to engage students

in OSS projects, without focusing on teaching and learning.Given the attention being given to OSS on the academic

for the society, we add to the state-of-the-art by

senting a new way of engaging students in OSS.



as “the game contexts.”

cation has being growing in the last years, presenting , mainly as a

. For example, and


has become a topic secondary



of whether a

fied bug tracking system, based on reputation and e frequency

Also worth to mention, the use of a gamified system to

vate developers to use more efficient development the adoption of

there are some studies that report experiences Software Engineering

showed that it is possible to

development, Thomas

a gamification proposal focused on areas

advantage of gamification to foster learning design

From software engineering perspective, we see that the literature brings approaches to improve existing processes

the tive of software engineering, we see that

gamification is mainly used with the purpose of improving n this paper we make use

coming [5],

place their rom the best of our knowledge, there is no

itiative to gamify an OSS environment to engage students in OSS projects, without focusing on teaching and learning.

academic , we

art by



followed the research method depicted in Figure 1 and steps are

A. Step 1: Defining the gaming elements

The first step be suitable for supporting our goal. Based on the knowledge about the orientation band onengineeselected the following four elements, since they are in line with our goals, points, and ranking.

Questsdefined goal that needs to be accomplishing the tasks may be, for example, points, level change or connected to the be unlocked, enabling the users to acquire new skills. context of motivate andbarriers follow

Levelsto unlock specific and contextualizedCsikszentmihaly keep userthe level is too hardmotivation and stop playing. applied in conjunction on a given step before moving ahead on the contribution process.

Pointswork [13], [14], [16]progress to the users. Acquiring points means that the actions performed by the users are in accordance with what was expected. Point cperformed by the user. For example, in OSS, a user can earn points by having a solving a project member. users motivated and to orientation barriers, more specifically, providing feedback on their actions.

Rankingcompare to other players. different levels (global, regional, per projectare usuallysorted list, with the best players at the top. According to

followed the research method depicted in Figure 1 and steps are explained in the following subsections

Figure 1.

Step 1: Defining the gaming elements

The first step is defining the be suitable for supporting our goal. Based on the knowledge about the orientation b

on studies onengineering [14], [17]selected the following four elements, since they are in line with our goals, and are the points, and ranking.

Quests are assignments or tasks with a clear and welldefined goal that needs to be accomplishing the tasks may be, for example, points, level change or external rewarconnected to the questsbe unlocked, enabling the users to acquire new skills. context of this work

vate and support newcomers obarriers [4]. Quests can be used as a stepfollow to achieve the goal of placing a contribution.

Levels groups different quests and tto unlock specific and contextualized

szentmihaly [30]keep users in a set of challenges aligned the level is too hard motivation and stop playing. applied in conjunction on a given step before moving ahead on the contribution process.

Points is the most [13], [14], [16]

gress to the users. Acquiring points means that the actions performed by the users are in accordance with what was expected. Point can be earned by means of different actions performed by the user. For example, in OSS, a user can earn points by having a pullsolving a quest, or working on a review

ject member. In this work, users motivated and to

tation barriers, more specifically, providing back on their actions.

Ranking enable users to follow their performance and compare to other players. different levels (global, regional, per projectare usually closely related to sorted list, with the best players at the top. According to

followed the research method depicted in Figure 1 and in the following subsections

Figure 1. Overall method followed


defining the gabe suitable for supporting our goal. Based on the knowledge about the orientation barriers from our previous work

on gamification applied to software [14], [17] and to learning context

selected the following four elements, since they are in line and are the widely

are assignments or tasks with a clear and well

defined goal that needs to be performedaccomplishing the tasks may be, for example, points, level

rewards [13]. The quests, since more challenging

be unlocked, enabling the users to acquire new skills. this work, we want to assess how

support newcomers oQuests can be used as a step

the goal of placing a contribution. different quests and t

to unlock specific and contextualized[30] describes the

set of challenges aligned or too easy, the player can easily lose

motivation and stop playing. In our context, applied in conjunction to Quests on a given step before moving ahead on the contribution

is the most applied element [13], [14], [16]. It provide

gress to the users. Acquiring points means that the actions performed by the users are in accordance with what was

an be earned by means of different actions performed by the user. For example, in OSS, a user can earn

pull-request acceptedor working on a review

In this work, Pointsusers motivated and to support them overcoming

tation barriers, more specifically, providing back on their actions.

enable users to follow their performance and compare to other players. It creates a sense of comdifferent levels (global, regional, per project

closely related to Pointssorted list, with the best players at the top. According to

followed the research method depicted in Figure 1 and in the following subsections.

Overall method followed


game elements be suitable for supporting our goal. Based on the knowledge

arriers from our previous work gamification applied to software and to learning contexts

selected the following four elements, since they are in line widely used: quests, levels,

are assignments or tasks with a clear and wellperformed. The rewards of

accomplishing the tasks may be, for example, points, level The level element

, since more challenging be unlocked, enabling the users to acquire new skills.

we want to assess how support newcomers overcoming orientation

Quests can be used as a step-by-step that the goal of placing a contribution. different quests and tasks, enabling users

to unlock specific and contextualized set of actions. describes the flow theory as a way to

set of challenges aligned to their skillset. If , the player can easily lose

In our context, Level to make students focus

on a given step before moving ahead on the contribution

applied element as showed in previous . It provides feedback and show



accepted, helping otheror working on a review suggested by a

Points were chosen to support them overcoming

tation barriers, more specifically, providing

enable users to follow their performance and It creates a sense of com

different levels (global, regional, per project, etc.). Points, and are presented as a

sorted list, with the best players at the top. According to

followed the research method depicted in Figure 1 and its

me elements that would be suitable for supporting our goal. Based on the knowledge

arriers from our previous work [4], [5] gamification applied to software

s [16], we selected the following four elements, since they are in line

quests, levels,

are assignments or tasks with a clear and well-The rewards of

accomplishing the tasks may be, for example, points, level element is very

, since more challenging quests can be unlocked, enabling the users to acquire new skills. In the

we want to assess how Quests vercoming orientation

step that users the goal of placing a contribution.

asks, enabling users set of actions.

as a way to their skillset. If

, the player can easily lose Levels were

to make students focus on a given step before moving ahead on the contribution

in previous and shows



other users, suggested by a chosen to keep

support them overcoming tation barriers, more specifically, providing

enable users to follow their performance and It creates a sense of competition, in

etc.). Rankings presented as a

sorted list, with the best players at the top. According to

Deterding et al. to gamify an environment. Rankingpre

B.

cocan be completed, but are not easy enough to demotivate the players. The rules adjust the complexity level of a task, stidefinition is related to elements chosen. Figure how the rules and elements are interrelated.

the system. Each given amount of rewarded only oncenumber of times (or have its reward reduced). The player can also choose taccording to the user one or more can user. addco

accordance to the steps newcomerplace a contributionthe community.following the flow FLOSScoach portal. in

C.

of GitLab and implemented the necessary features. an open source environment that pmanagement, code reviews, issuand wikis. envdashboard the number of points and the level of a user (1). By clicking on the links, the user can see more details athe points earned and the level, respectively. Also on the dashboard, the user has access to the Ranking and the Quests (2). In the example, we present the Quests the user can see what can be done, what is blocked (and the

Deterding et al. to gamify an environment. Ranking was prestige among the colleagues.

B. Step 2: Defining the rules

Rules, when combined, make up a set of provisions that condition the execution of a game can be completed, but are not easy enough to demotivate the players. The rules adjust the complexity level of a task, stimulating and keeping the players engaged.definition is related to elements chosen. Figure how the rules and elements are interrelated.

Figure 2. How game elements interact according to the rules

The first action the system. Each given amount of rewarded only oncenumber of times (or have its reward reduced). The player can also choose to join a according to the user one or more questscan comprise tasksuser. Points are used to control the addition, when the player achieves an amount of complete required

An initial set of tasksaccordance to the steps newcomerplace a contributionthe community.following the flow FLOSScoach portal. in the following section

C. Step 3: Designing and Implementing the Gamified Environment

To apply the elements and their ruleof GitLab and implemented the necessary features. an open source environment that pmanagement, code reviews, issuand wikis. Figure environment. In the fdashboard the number of points and the level of a user (1). By clicking on the links, the user can see more details athe points earned and the level, respectively. Also on the dashboard, the user has access to the Ranking and the Quests (2). In the example, we present the Quests the user can see what can be done, what is blocked (and the

Deterding et al. [10], ranking to gamify an environment. In our context,

was to motivate tige among the colleagues.

Step 2: Defining the rules

Rules, when combined, make up a set of provisions that ndition the execution of a game

can be completed, but are not easy enough to demotivate the players. The rules adjust the complexity level of a task,

ulating and keeping the players engaged.definition is related to the goal of the game and the game elements chosen. Figure 2 presents a graphical description on how the rules and elements are interrelated.

How game elements interact according to the rules

The first action of a player is the system. Each task, when completed, is rewarded with a given amount of points. To avoid repetition, somerewarded only once and some can only be performed a given number of times (or have its reward reduced). The player can

o join a quest. A given set of according to the user level, and

quests are required to be completed. The tasks, and, when completed, sumare used to control the

tion, when the player achieves an amount of plete required quests, they move to another

An initial set of tasks, questsaccordance to the steps newcomerplace a contribution, learn project practicesthe community. The tasks were created to orient newcomers, following the flow defined FLOSScoach portal. Some examples of tasks

the following section.

Designing and Implementing the Gamified Environment

To apply the elements and their ruleof GitLab and implemented the necessary features. an open source environment that pmanagement, code reviews, issu

Figure 3 brings an example of thronment. In the figure,

dashboard the number of points and the level of a user (1). By clicking on the links, the user can see more details athe points earned and the level, respectively. Also on the dashboard, the user has access to the Ranking and the Quests (2). In the example, we present the Quests the user can see what can be done, what is blocked (and the

ranking is a commonly used technique In our context, the goal

to motivate the students, who can gain tige among the colleagues.

Rules, when combined, make up a set of provisions that ndition the execution of a game promoting challenges that


ulating and keeping the players engaged.the goal of the game and the game

presents a graphical description on how the rules and elements are interrelated.


of a player is to perform , when completed, is rewarded with a . To avoid repetition, some

and some can only be performed a given number of times (or have its reward reduced). The player can

. A given set of quests, and to advance to a given level

are required to be completed. The , and, when completed, sum

are used to control the rankingtion, when the player achieves an amount of

, they move to another , quests, and levels

accordance to the steps newcomers need to take, learn project practices,

The tasks were created to orient newcomers, by Steinmacher et al.

Some examples of tasks

Designing and Implementing the Gamified

To apply the elements and their rules, we of GitLab and implemented the necessary features. an open source environment that provides Git repository management, code reviews, issue tracking, activity feeds

brings an example of th it is possible to see on the

dashboard the number of points and the level of a user (1). By clicking on the links, the user can see more details athe points earned and the level, respectively. Also on the dashboard, the user has access to the Ranking and the Quests (2). In the example, we present the Quests the user can see what can be done, what is blocked (and the

is a commonly used technique the goal of using

students, who can gain

Rules, when combined, make up a set of provisions that promoting challenges that


ulating and keeping the players engaged. The rules the goal of the game and the game

presents a graphical description on


perform a defined task, when completed, is rewarded with a . To avoid repetition, some actions are

and some can only be performed a given number of times (or have its reward reduced). The player can

quests is unlocked to advance to a given level

are required to be completed. The quests, and, when completed, sum points to the

ranking of users. In tion, when the player achieves an amount of points and

, they move to another level. and levels were defined

to take in order and interact with

The tasks were created to orient newcomers, by Steinmacher et al. [5]

Some examples of tasks are presented

Designing and Implementing the Gamified

, we created a fork of GitLab and implemented the necessary features. GitLab is

rovides Git repository e tracking, activity feeds

brings an example of the gamified ssible to see on the

dashboard the number of points and the level of a user (1). By clicking on the links, the user can see more details about the points earned and the level, respectively. Also on the dashboard, the user has access to the Ranking and the Quests (2). In the example, we present the Quests GUI (3), where the user can see what can be done, what is blocked (and the

is a commonly used technique of using

students, who can gain

Rules, when combined, make up a set of provisions that promoting challenges that


The rules the goal of the game and the game

presents a graphical description on

task in , when completed, is rewarded with a

actions are and some can only be performed a given

number of times (or have its reward reduced). The player can is unlocked

to advance to a given level quests to the

of users. In and

were defined in in order to

and interact with The tasks were created to orient newcomers,

at are presented

created a fork GitLab is

rovides Git repository e tracking, activity feeds,

e gamified ssible to see on the

dashboard the number of points and the level of a user (1). bout

the points earned and the level, respectively. Also on the dashboard, the user has access to the Ranking and the Quests

(3), where the user can see what can be done, what is blocked (and the

level requirecan be earned by completinan unblocked quest, a full description of the that need to be accomplished are presented.

For example, for Qthree tasks:

1.

2. 3.

To implement the rules on GitLabalready existing features: issue tracker, thumbs up (similar to the like button on Facebook)task, the player was required to tag the issue identifying that the post was related to a gamified task. For example, when players want to create an issue fulfilling the requirements of the task and choose the tag “quest1_2” (representing the task 2 of quest 1). To approve or disapprove the task, a project owner or a player on a given level neliking the post (using the Thumbs up featurethe usage of tags and thumbs up is presented on 4.Figure 3.

Figure 4.

Examples of tasks and the way they were implemented are presented below: Clarifying a doubt with the community

tracand a project owner needs to approve the entry and

level required to open them) and the number of points that can be earned by completinan unblocked quest, a full description of the that need to be accomplished are presented.

Figure 3.

For example, for Qtasks: Post something on the issue tracker (generally an

introduction or a Fork/Clone the repository Setup the local workspace

To implement the rules on GitLabready existing features: issue tracker, thumbs up (similar to

the like button on Facebook)task, the player was required to tag the issue identifying that the post was related to a gamified task. For example, when players want to accomplish the task 2 of Quest 1, they need to create an issue fulfilling the requirements of the task and choose the tag “quest1_2” (representing the task 2 of quest 1). To approve or disapprove the task, a project owner or a player on a given level ne

ing the post (using the Thumbs up featurethe usage of tags and thumbs up is presented on

Figure 3.

Figure 4. Example of issues usage to implement game rules

xamples of tasks and the way they were implemented are presented below:

Clarifying a doubt with the communitytracker: to earn the points the player places a and a project owner needs to approve the entry and

1

2

3

d to open them) and the number of points that can be earned by completing the task. When a user clicks ian unblocked quest, a full description of the that need to be accomplished are presented.

Quests screen of gamified

For example, for Quest 1, the user

Post something on the issue tracker (generally an troduction or a question)

Fork/Clone the repository Setup the local workspace

To implement the rules on GitLabready existing features: issue tracker, thumbs up (similar to

the like button on Facebook), and tags. When performing a task, the player was required to tag the issue identifying that the post was related to a gamified task. For example, when

to accomplish the task 2 of Quest 1, they need to create an issue fulfilling the requirements of the task and choose the tag “quest1_2” (representing the task 2 of quest 1). To approve or disapprove the task, a project owner or a player on a given level needs to

ing the post (using the Thumbs up featurethe usage of tags and thumbs up is presented on

Example of issues usage to implement game rules

xamples of tasks and the way they were implemented

Clarifying a doubt with the community: to earn the points the player places a

and a project owner needs to approve the entry and

Tags reprquest’s task

d to open them) and the number of points that g the task. When a user clicks i

an unblocked quest, a full description of the questthat need to be accomplished are presented.

Quests screen of gamified GitLab

uest 1, the user should accomplish

Post something on the issue tracker (generally an

To implement the rules on GitLab, we made use of ready existing features: issue tracker, thumbs up (similar to

and tags. When performing a task, the player was required to tag the issue identifying that the post was related to a gamified task. For example, when

to accomplish the task 2 of Quest 1, they need to create an issue fulfilling the requirements of the task and choose the tag “quest1_2” (representing the task 2 of quest 1). To approve or disapprove the task, a project owner or a

to approve it by means of ing the post (using the Thumbs up feature). An example of

the usage of tags and thumbs up is presented on



Clarifying a doubt with the community : to earn the points the player places a


Thumbs up

Tags representing a quest’s task

d to open them) and the number of points that g the task. When a user clicks in

quest and tasks

accomplish

Post something on the issue tracker (generally an

we made use of ready existing features: issue tracker, thumbs up (similar to

and tags. When performing a task, the player was required to tag the issue identifying that the post was related to a gamified task. For example, when

to accomplish the task 2 of Quest 1, they need to create an issue fulfilling the requirements of the task and choose the tag “quest1_2” (representing the task 2 of quest 1). To approve or disapprove the task, a project owner or a

approve it by means of . An example of

the usage of tags and thumbs up is presented on Figure



via issue : to earn the points the player places a question


Thumbs up

D.

study to assess how the element would motivate and support the students steps towards contributing to an OSS project. Firstly, we chose a projectence manager. JabRef OSS project. In addition, one of the project maintainers, which facilitated instrumenting it.

GitLab, including all its versioning histomost recent issues were tagged according to their level of difficulty, until 50 easy tasks were tagged.

Open Source puter Science major atParana. costance of the course, the assignwould contribute some timeronment and consented to be part of the study. varied from 20 to 25 years old, including 5 females and 12 males.

dedicgameceived a document presenting the rules and some guidance about the study. After that, the instructor handedassignment, which basically ctribute that the students succeeded by the end of the 20 days period. However, surprisingly, from the 17 students, 9 tribution good number, if compared to our previous exper

answer it. The way this approval is made in our enviroment is via thumbs up.

Cloning the repositorydence by posting a vidprove the completion, and a project owner (or a user on a higher level

Setting up a local environmentpository.

Finding a bug to work withthe players can select a bug, and asking the community (via comment) whether they can woAn owner or prove the assign

Reproducing a bugevidence of it in validated by an owner or user at a given level.

Submitting a lated to a reported issue,

Having a pulland merged

D. Step 4: Study conduction

After implementing the rules on GitLabstudy to assess how the element would motivate and support the students overcoming orientation barriers steps towards contributing to an OSS project. Firstly, we chose a projectence manager. JabRef OSS project. In addition, one of the project maintainers, which facilitated instrumenting it.

JabRef project was imported to a local private instance of GitLab, including all its versioning histomost recent issues were tagged according to their level of difficulty, until 50 easy tasks were tagged.

The study was conducted with students attending to the Open Source Developmentputer Science major atParana. In this course, students generally are requested to contribute to an OSS project as an stance of the course, the assignwould contribute some time. Out of 18 students, 17 accepted to ronment and consented to be part of the study. varied from 20 to 25 years old, including 5 females and 12 males.

Before starting their assignment, dedicated to explaining the environment, the rules of the game, and presenting JabRef. In addition, the students rceived a document presenting the rules and some guidance about the study. After that, the instructor handedassignment, which basically ctribute with code to that the students succeeded by the end of the 20 days period. However, surprisingly, from the 17 students, 9 tribution by the end of the asgood number, if compared to our previous exper

answer it. The way this approval is made in our enviroment is via thumbs up. Cloning the repository: the players need to provide evdence by posting a video or a set of screenshots that prove the completion, and a project owner (or a user on a

level) needs to approve itSetting up a local environment

Finding a bug to work withplayers can select a bug, and asking the community

(via comment) whether they can woor a user on a higher level

prove the assignment, and often provide support.Reproducing a bug: reproduce a bug

of it in a comment validated by an owner or user at a given level.Submitting a pull-requestlated to a reported issue, independently of its ac

pull-request acceptedand merged, the player earns points.

Study conduction

After implementing the rules on GitLabstudy to assess how the element would motivate and support

overcoming orientation barriers steps towards contributing to an OSS project. Firstly, we chose a project: JabRef2, an open source bibliography ence manager. JabRef is written in Java and is a consolidated OSS project. In addition, one of the project maintainers, which facilitated instrumenting it.

JabRef project was imported to a local private instance of GitLab, including all its versioning histomost recent issues were tagged according to their level of difficulty, until 50 easy tasks were tagged.

The study was conducted with students attending to the Development course, in the last year of Co

puter Science major at Federal University of Technology In this course, students generally are requested to

ntribute to an OSS project as an stance of the course, the assignwould contribute using the gamified environmen

. Out of 18 students, 17 accepted to ronment and consented to be part of the study. varied from 20 to 25 years old, including 5 females and 12

Before starting their assignment, explaining the environment, the rules of the

and presenting JabRef. In addition, the students rceived a document presenting the rules and some guidance about the study. After that, the instructor handedassignment, which basically c

with code to JabRef inthat the students succeeded by the end of the 20 days period. However, surprisingly, from the 17 students, 9

by the end of the asgood number, if compared to our previous exper

2 http://www.jabref.org

answer it. The way this approval is made in our enviro

: the players need to provide eveo or a set of screenshots that

prove the completion, and a project owner (or a user on a ) needs to approve it using the thumbs up

Setting up a local environment: similar to clone the r

Finding a bug to work with: by following somplayers can select a bug, and asking the community

(via comment) whether they can work on the bug or not. a user on a higher level can answer and a

ment, and often provide support.: reproduce a bug

a comment to the bug, which needs to be validated by an owner or user at a given level.

request: submitting a pullindependently of its acaccepted: after being accepted

, the player earns points.

After implementing the rules on GitLabstudy to assess how the element would motivate and support

overcoming orientation barriers steps towards contributing to an OSS project. Firstly, we

an open source bibliography written in Java and is a consolidated

OSS project. In addition, one of the authorsproject maintainers, which facilitated instrumenting it.

JabRef project was imported to a local private instance of GitLab, including all its versioning history and issues. The most recent issues were tagged according to their level of difficulty, until 50 easy tasks were tagged.

The study was conducted with students attending to the course, in the last year of Co

Federal University of Technology In this course, students generally are requested to

ntribute to an OSS project as an assignment. In this istance of the course, the assignment was changed

using the gamified environmen. Out of 18 students, 17 accepted to

ronment and consented to be part of the study. varied from 20 to 25 years old, including 5 females and 12

Before starting their assignment, a twoexplaining the environment, the rules of the

and presenting JabRef. In addition, the students rceived a document presenting the rules and some guidance about the study. After that, the instructor handedassignment, which basically consisted of attempting to co

JabRef in 20 days. It was not expected that the students succeeded by the end of the 20 days period. However, surprisingly, from the 17 students, 9

by the end of the assignment (53%), good number, if compared to our previous exper

http://www.jabref.org

answer it. The way this approval is made in our enviro

: the players need to provide eveo or a set of screenshots that

prove the completion, and a project owner (or a user on a using the thumbs up. : similar to clone the r

following some hints, players can select a bug, and asking the community

rk on the bug or not. can answer and a

ment, and often provide support. : reproduce a bug locally, provide

to the bug, which needs to be validated by an owner or user at a given level.

submitting a pull-request rindependently of its acceptance

: after being accepted

After implementing the rules on GitLab, we designed a study to assess how the element would motivate and support

overcoming orientation barriers on their first steps towards contributing to an OSS project. Firstly, we

an open source bibliography refewritten in Java and is a consolidated

authors is one of the project maintainers, which facilitated instrumenting it.

JabRef project was imported to a local private instance of ry and issues. The

most recent issues were tagged according to their level of

The study was conducted with students attending to the course, in the last year of Co

Federal University of Technology In this course, students generally are requested to

assignment. In this iwas changed so they

using the gamified environment during . Out of 18 students, 17 accepted to use the env

ronment and consented to be part of the study. Students’ ages varied from 20 to 25 years old, including 5 females and 12

a two-hour class was explaining the environment, the rules of the

and presenting JabRef. In addition, the students rceived a document presenting the rules and some guidance about the study. After that, the instructor handed-over the

attempting to coIt was not expected

that the students succeeded by the end of the 20 days period. However, surprisingly, from the 17 students, 9 placed a co

signment (53%), which is a good number, if compared to our previous experiences [5].

answer it. The way this approval is made in our environ-

: the players need to provide evi-eo or a set of screenshots that

prove the completion, and a project owner (or a user on a

: similar to clone the re-

e hints, players can select a bug, and asking the community

rk on the bug or not. can answer and ap-

locally, provide to the bug, which needs to be

request re-ceptance.

: after being accepted

we designed a study to assess how the element would motivate and support

on their first steps towards contributing to an OSS project. Firstly, we

refer-written in Java and is a consolidated

is one of the

JabRef project was imported to a local private instance of ry and issues. The

most recent issues were tagged according to their level of

The study was conducted with students attending to the course, in the last year of Com-

Federal University of Technology – In this course, students generally are requested to

assignment. In this in-so they during

use the envi-Students’ ages

varied from 20 to 25 years old, including 5 females and 12

hour class was explaining the environment, the rules of the

and presenting JabRef. In addition, the students re-ceived a document presenting the rules and some guidance

over the attempting to con-

It was not expected that the students succeeded by the end of the 20 days period.

placed a con-which is a

.

At the end of the period of 20 days, the students were asked to answer a questionnaire composed of a set of 5 points Likertenable students to share their feedback.tionnaire was to assess to what extent gamified overcome session was conducted to help us

In this sectionmostly administered after the assignment period. The questionnaire was organized of Likertmotivated and supported newcomers’ first steps; one section with fourgrade and compare the elements as motivators and means to support newcomers; one last section with four Lquestions to assess theTo facilitate the understanding, wing the questionnaire structure.

A. Quest

The step process to guide newcomers, keeping them motivated and oriented. shares dents answered these three 5pletely

Q1. Q2.

Q3.

Q4.

As it can be seen

were viewed by the studentsstudents evaluated tively.

In addition, orient the contribution. as a guide, 9 people became neutral or disagreed.

At the end of the period of 20 days, the students were asked to answer a questionnaire composed of a set of 5

Likert-scale items, and some openenable students to share their feedback.tionnaire was to assess to what extent gamified environment motivated and helped the students to overcome orientation barriers.session was conducted to help us

In this section, we bring the results mostly on the answers we obtained from the questionnaire administered after the assignment period. The questionnaire was organized in the following way: one section composed of Likert-scale items motivated and supported newcomers’ first steps; one section

four closed-ended questions enabling the students to grade and compare the elements as motivators and means to support newcomers; one last section with four Lquestions to assess the

facilitate the understanding, wing the questionnaire structure.

Quests and Levels

The quest element was chosen to help creating a stepstep process to guide newcomers, keeping them motivated and oriented. The level

the goal of guiding newcomers. dents answered these three 5

Disagree to Completely Q1. The quests stepsQ2. Quests were useful to orient me

contribution;Q3. The quest element kept me motivated to perform the

proposed tasksQ4. The possibility of

me to change levelAs it can be seen

viewed by the studentsstudents evaluated Q3

Figure 5. Results for

In addition, most part of the newcomers found it useful to orient the contribution. as a guide, 9 people became neutral or disagreed.


scale items, and some openenable students to share their feedback.tionnaire was to assess to what extent

environment motivated and helped the students to orientation barriers. A half

session was conducted to help us to

IV. RESULTS

we bring the results on the answers we obtained from the questionnaire

administered after the assignment period. The questionnaire the following way: one section composed

scale items to assess how eachmotivated and supported newcomers’ first steps; one section

ended questions enabling the students to grade and compare the elements as motivators and means to support newcomers; one last section with four Lquestions to assess the future use of a

facilitate the understanding, we present the ring the questionnaire structure.

s and Levels

element was chosen to help creating a stepstep process to guide newcomers, keeping them motivated

level element is linked to the quests, and the goal of guiding newcomers.

dents answered these three 5-points to Completely Agree) items:

steps guided my contribution process;Quests were useful to orient me

; The quest element kept me motivated to perform the proposed tasks and to contributeThe possibility of unblockingme to change levels.

As it can be seen in Figure 5, Quests viewed by the students as good motivators, since

Q3 positively and 14 evalu

Results for Quests-

most part of the newcomers found it useful to orient the contribution. However, regardingas a guide, 9 people became neutral or disagreed.


scale items, and some open-ended question to enable students to share their feedback. The goal of tionnaire was to assess to what extent each element

environment motivated and helped the students to A half-hour, informal debrief

to understand some results.

ESULTS

we bring the results of our study, relying on the answers we obtained from the questionnaire


to assess how each gamesmotivated and supported newcomers’ first steps; one section

ended questions enabling the students to grade and compare the elements as motivators and means to support newcomers; one last section with four L

future use of a gamified environment.e present the results reflec

element was chosen to help creating a stepstep process to guide newcomers, keeping them motivated

element is linked to the quests, and the goal of guiding newcomers. Therefore, the st

points Likert-scale (from CoAgree) items:

guided my contribution process;Quests were useful to orient me to place

The quest element kept me motivated to perform the and to contribute;

ing new quests

, Quests allied with Levels as good motivators, since

and 14 evaluated Q4 pos

-related questions

most part of the newcomers found it useful to regarding the use of quests

as a guide, 9 people became neutral or disagreed.


ended question to The goal of the ques-each element and the

environment motivated and helped the students to hour, informal debrief

understand some results.

of our study, relying on the answers we obtained from the questionnaire


games element motivated and supported newcomers’ first steps; one section

ended questions enabling the students to grade and compare the elements as motivators and means to support newcomers; one last section with four Likert-scale

gamified environment. esults reflect-

element was chosen to help creating a step-by-step process to guide newcomers, keeping them motivated

element is linked to the quests, and Therefore, the stu-

from Com-

guided my contribution process; to place my

The quest element kept me motivated to perform the

new quests motivated

allied with Levels as good motivators, since 12

ated Q4 posi-

most part of the newcomers found it useful to the use of quests

as a guide, 9 people became neutral or disagreed. During an

informal debrief session after the study, we found thapossible explanation for the difference from Q2 answers to Q1 answers is that quests are good to orient the way that needtasks within a quest

are good to motivate ered good newcomers, however, per

B.

studentsfore, the following items were administered:

Q2 are similar. They snegative answers. motivatorthan 30% disagreed with at least one sentence related to motivationthat For example, the profile of the players, the public exposition, or the absence of a reward related to the points.

we see noproviding feedback on the tasks performed by the students.

providing some players, but not others.

C.

tivate the students. assess the element:

informal debrief session after the study, we found thapossible explanation for the difference from Q2 answers to Q1 answers is that quests are good to orient the way that needs to be trodden, but, the obligation to fulfill all specific tasks within a quest

In summary,are good to motivate ered good to show the newcomers, however, perform the steps would be beneficial.

B. Points

The main goal students and give them feedbackfore, the following items were administered:

Q1. Points element motivated me to keep contributingQ2. The more points I got

earn, so I could increase my Level and my Ranking.Q3. Points

tasks I performedAs presented by Figure

Q2 are similar. They snegative answers. motivator for approximately half of the students, but more than 30% disagreed with at least one sentence related to motivation. During the debrief session, it wathat there are other factors that influence For example, the profile of the players, the public exposition, or the absence of a reward related to the points.

On the other hand, when we observe the results for Q3, we see no disagreement, indicating that Points were good on providing feedback on the tasks performed by the students.

Figure 6.

Therefore, oproviding feedback some players, but not others.

C. Ranking

Allied with points, the main goal of Ranking was to mtivate the students. assess the element:

Q1. I wanted could improve my

Q2. The Rankipare my performance with my colleagues

informal debrief session after the study, we found thapossible explanation for the difference from Q2 answers to Q1 answers is that quests are good to orient the way that

to be trodden, but, the obligation to fulfill all specific tasks within a quest bothered some participants

In summary, our results indicate thatare good to motivate the students.

to show the steps that need tnewcomers, however, adding more flexibility on how to

form the steps would be beneficial.

The main goal of choosingand give them feedback

fore, the following items were administered:Points element motivated me to keep contributingThe more points I gotearn, so I could increase my Level and my Ranking.Points were useful to provide me tasks I performed.

As presented by Figure 6Q2 are similar. They show a balance between positive and negative answers. The points themselves were

for approximately half of the students, but more than 30% disagreed with at least one sentence related to

During the debrief session, it waother factors that influence

For example, the profile of the players, the public exposition, or the absence of a reward related to the points.

On the other hand, when we observe the results for Q3, disagreement, indicating that Points were good on

providing feedback on the tasks performed by the students.

Figure 6. Results for Points

Therefore, our results indicate thatfeedback to the newcomers

some players, but not others.

Allied with points, the main goal of Ranking was to mtivate the students. In this case, only assess the element:

anted to perform more tasks and quests so I could improve my RankingThe Ranking motivated me because I could pare my performance with my colleagues

informal debrief session after the study, we found thapossible explanation for the difference from Q2 answers to Q1 answers is that quests are good to orient the way that

to be trodden, but, the obligation to fulfill all specific bothered some participants

our results indicate that queststhe students. Quests were also consi

that need to be followed by the adding more flexibility on how to

form the steps would be beneficial.

choosing points was to motivate the and give them feedback about their actions

fore, the following items were administered:Points element motivated me to keep contributingThe more points I got, the more I was motivated to earn, so I could increase my Level and my Ranking.

were useful to provide me

6, the answers to items Q1 and how a balance between positive and

he points themselves were for approximately half of the students, but more

than 30% disagreed with at least one sentence related to During the debrief session, it wa

other factors that influence students’ answers. For example, the profile of the players, the public exposition, or the absence of a reward related to the points.



Results for Points-related questions

ur results indicate that pointsthe newcomers, and can moti

Allied with points, the main goal of Ranking was to mIn this case, only two items were used to

perform more tasks and quests so I Ranking.

ng motivated me because I could pare my performance with my colleagues

informal debrief session after the study, we found that apossible explanation for the difference from Q2 answers to Q1 answers is that quests are good to orient the way that

to be trodden, but, the obligation to fulfill all specific bothered some participants.

quests and levelswere also consi

o be followed by the adding more flexibility on how to

was to motivate the about their actions. Ther

fore, the following items were administered: Points element motivated me to keep contributing

the more I was motivated to earn, so I could increase my Level and my Ranking.

were useful to provide me feedback on the

, the answers to items Q1 and how a balance between positive and

he points themselves were considered for approximately half of the students, but more

than 30% disagreed with at least one sentence related to During the debrief session, it was possible to feel

students’ answers. For example, the profile of the players, the public exposition, or the absence of a reward related to the points.



related questions

points are good on , and can motivate

Allied with points, the main goal of Ranking was to mitems were used to

perform more tasks and quests so I

ng motivated me because I could copare my performance with my colleagues.

t a possible explanation for the difference from Q2 answers to Q1 answers is that quests are good to orient the way that

to be trodden, but, the obligation to fulfill all specific

levels were also consid-

o be followed by the adding more flexibility on how to

was to motivate the There-

Points element motivated me to keep contributing. the more I was motivated to

earn, so I could increase my Level and my Ranking. feedback on the

, the answers to items Q1 and how a balance between positive and

considered for approximately half of the students, but more

than 30% disagreed with at least one sentence related to to feel

students’ answers. For example, the profile of the players, the public exposition,



are good on vate

Allied with points, the main goal of Ranking was to mo-items were used to

perform more tasks and quests so I

com-

As it is possible to notice in Figure students (9) agreed that the Ranking element was a motivtional factor, while few others disagrQ2). The results aretained for the points, which makes sense, since points and ranking are can be other factors that influenced these results. So, astudy analyzing the profile, goalsdents along with the gamified environment, would provide us explanation on these results.

Our results indicated that motivate

D. Comparing the elements

On the second part of the questionnaire, the students were asked to grade the elements orientation the elements providetunity to choose which elements motivated them and useful on orienting the contribution process.

Figure by the students when we asked related to newcomersmuch havebarriers?”

As it is possible to observe, and in consonance with the results presented before, the on orienting the students towards their contribution.ing at the raw data, we can see that seven people evaluated quests with an 8, three gave a 9

Figure 8.

Pointsstudents

As it is possible to notice in Figure students (9) agreed that the Ranking element was a motivtional factor, while few others disagrQ2). The results are, somehow,tained for the points, which makes sense, since points and ranking are expected to be can be other factors that influenced these results. So, astudy analyzing the profile, goalsdents along with the gamified environment, would provide us explanation on these results.

Figure 7. Results for Ranking

Our results indicated that motivate some players, but not others.

Comparing the elements

On the second part of the questionnaire, the students were asked to grade the elements orientation the elements providetunity to choose which elements motivated them and useful on orienting the contribution process.

Figure 8 presents the distribution of the grades provided by the students when we asked related to newcomers

have each element helpbarriers?”

As it is possible to observe, and in consonance with the results presented before, the on orienting the students towards their contribution.ing at the raw data, we can see that seven people evaluated quests with an 8, three gave a 9

Figure 8. On a 10-points scale hmitigate/reduce these barriers

Points were also well graded. More than a half of the students evaluated with 7 or a better grade (including two

As it is possible to notice in Figure students (9) agreed that the Ranking element was a motivtional factor, while few others disagr

, somehow, aligned with the results otained for the points, which makes sense, since points and

expected to be complementary. Once again, there can be other factors that influenced these results. So, astudy analyzing the profile, goals, dents along with the gamified environment, would provide us explanation on these results.

Results for Ranking

Our results indicated that Rankingsome players, but not others.

Comparing the elements

On the second part of the questionnaire, the students were asked to grade the elements orientation the elements provide. They also had the oppotunity to choose which elements motivated them and useful on orienting the contribution process.

presents the distribution of the grades provided by the students when we asked “Regarding the barriers related to newcomers’ orientation, in a 10

each element helped

As it is possible to observe, and in consonance with the results presented before, the quests on orienting the students towards their contribution.ing at the raw data, we can see that seven people evaluated quests with an 8, three gave a 9, and one gave a 10.

points scale how much have each element helped tomitigate/reduce these barriers

were also well graded. More than a half of the evaluated with 7 or a better grade (including two

As it is possible to notice in Figure 7, just over half of the students (9) agreed that the Ranking element was a motivtional factor, while few others disagreed (4 on Q1 and 3 on

aligned with the results otained for the points, which makes sense, since points and

complementary. Once again, there can be other factors that influenced these results. So, a

and attitudes of the stdents along with the gamified environment, would provide

Results for Ranking-related questions

Ranking, just like Pointssome players, but not others.

On the second part of the questionnaire, the students were asked to grade the elements according to how much

They also had the oppotunity to choose which elements motivated them and useful on orienting the contribution process.

presents the distribution of the grades provided “Regarding the barriers

orientation, in a 10-point scale, how to mitigate/reduce these

As it is possible to observe, and in consonance with the played an important role

on orienting the students towards their contribution.ing at the raw data, we can see that seven people evaluated

and one gave a 10.

ow much have each element helped tomitigate/reduce these barriers


just over half of the students (9) agreed that the Ranking element was a motiva-

eed (4 on Q1 and 3 on aligned with the results ob-

tained for the points, which makes sense, since points and complementary. Once again, there

can be other factors that influenced these results. So, another and attitudes of the stu-

dents along with the gamified environment, would provide

related questions

Points, can

On the second part of the questionnaire, the students according to how much

They also had the oppor-tunity to choose which elements motivated them and were

presents the distribution of the grades provided “Regarding the barriers

point scale, how to mitigate/reduce these

As it is possible to observe, and in consonance with the played an important role

on orienting the students towards their contribution. By look-ing at the raw data, we can see that seven people evaluated

and one gave a 10.

ow much have each element helped to


10). The possible expining feedback to students.

part of the grades equal or below 5. These results for the Rankingelement was not orienting, but motivating the players. Hoever, we expected a better evaluation for sincethe debrief, we understand that athe students evaluated the level out considering it as a part of the flow that guided them from one quest to another.

the students to tell which elements were motivators and useful for orienting them. The results are presented in TaIwith previous results. role on orientation, and, to some extent, motivated the stdents during their contributions. We also see that Ranking elemstudents.

not considervate studentsthink of works in conjunction with

Which element(s) were useful to

orient

Which element(s)

keep contributing?

Which element you

anymore?

E.

evaluate the experience of the students with a gamified OSS environment.provided with the following sentences:

agreed that they would use the gamified environment in the future, and three (13%) disagreed with the sentence. More interestingly, 14 environment supported them during their contribution prcess.

10). The possible expin Section IV.Bing feedback to students.

Levels and part of the grades equal or below 5. These results for the Ranking element are not surprising, since the goal of this element was not orienting, but motivating the players. Hoever, we expected a better evaluation for since it plays an orientation role along with the the debrief, we understand that athe students evaluated the level out considering it as a part of the flow that guided them from one quest to another.

As aforementioned, to crossthe students to tell which elements were motivators and useful for orienting them. The results are presented in TaI. It is possible to notice that the results are, partially, aligned with previous results. role on orientation, and, to some extent, motivated the stdents during their contributions. We also see that Ranking element was considered as a good motivator by 47% of the students.

It is also possible to observe in Table not considered vate students. Once again, this can be related think of levels, as an independent element or something that works in conjunction with

TABLE I.


orient your contribution?

Which element(s) motivated

keep contributing?

Which element you

anymore?

E. Evaluating the environment

The last section of the evaluate the experience of the students with a gamified OSS environment. The students answered two Likertprovided with the following sentences:

Q1. I would use the gamified GitLab in the futureQ2. The gamified

contribution process.

As it is possible to observe on Figure agreed that they would use the gamified environment in the future, and three (13%) disagreed with the sentence. More interestingly, 14 environment supported them during their contribution prcess.

10). The possible explanation relates to the results presented IV.B, which shows that points are good on provi

ing feedback to students. and Ranking had worse results,

part of the grades equal or below 5. These results for the element are not surprising, since the goal of this

element was not orienting, but motivating the players. Hoever, we expected a better evaluation for

it plays an orientation role along with the the debrief, we understand that athe students evaluated the level out considering it as a part of the flow that guided them from one quest to another.

mentioned, to crossthe students to tell which elements were motivators and useful for orienting them. The results are presented in Ta

It is possible to notice that the results are, partially, aligned with previous results. Points and role on orientation, and, to some extent, motivated the stdents during their contributions. We also see that Ranking

ent was considered as a good motivator by 47% of the

It is also possible to observe in Table important to provide

. Once again, this can be related , as an independent element or something that

works in conjunction with quests

IMPORTANCE OF ELEMENT

ORIENTATION AND MOTI

Quests


your contribution?

10

(59%)

motivated you to 10

(59%)

Which element you would not use

(6%)

Evaluating the environment

section of the questionnaire aimed to quickly evaluate the experience of the students with a gamified OSS

The students answered two Likertprovided with the following sentences:

I would use the gamified GitLab in the futureThe gamified GitLab supported me dcontribution process.

As it is possible to observe on Figure agreed that they would use the gamified environment in the future, and three (13%) disagreed with the sentence. More interestingly, 14 students (82%)environment supported them during their contribution pr

lanation relates to the results presented , which shows that points are good on provi

had worse results, part of the grades equal or below 5. These results for the

element are not surprising, since the goal of this element was not orienting, but motivating the players. Hoever, we expected a better evaluation for

it plays an orientation role along with the the debrief, we understand that a possible explanation is that the students evaluated the level of information offered witout considering it as a part of the flow that guided them from

mentioned, to cross-check our results, we asked the students to tell which elements were motivators and useful for orienting them. The results are presented in Ta

It is possible to notice that the results are, partially, aligned and Quests played an

role on orientation, and, to some extent, motivated the stdents during their contributions. We also see that Ranking


It is also possible to observe in Table I important to provide orientation or

. Once again, this can be related , as an independent element or something that

quests.

MPORTANCE OF ELEMENTS WITH REGARD TO

ORIENTATION AND MOTIVATION

Quests Levels Points

10

(59%)

4

(24%) (53%)

10

(59%)

2

(12%)

10

(59%)

1

(6%)

5

(29%) (18%)

Evaluating the environment

questionnaire aimed to quickly evaluate the experience of the students with a gamified OSS

The students answered two Likertprovided with the following sentences:

I would use the gamified GitLab in the futureGitLab supported me d

contribution process.

As it is possible to observe on Figure 9, agreed that they would use the gamified environment in the future, and three (13%) disagreed with the sentence. More

(82%) pointed that the gamified environment supported them during their contribution pr

lanation relates to the results presented , which shows that points are good on provi

presenting most part of the grades equal or below 5. These results for the

element are not surprising, since the goal of this element was not orienting, but motivating the players. Hoever, we expected a better evaluation for Levels element,

it plays an orientation role along with the Quests. From possible explanation is that

information offered witout considering it as a part of the flow that guided them from

check our results, we asked the students to tell which elements were motivators and useful for orienting them. The results are presented in Table

It is possible to notice that the results are, partially, aligned played an important

role on orientation, and, to some extent, motivated the stdents during their contributions. We also see that Ranking


that Levels were orientation or to mot

. Once again, this can be related to how students , as an independent element or something that

S WITH REGARD TO

Points Ranking None

9

(53%)

3

(18%)

10

(59%)

8

(47%)

3

(18%)

7

(41%) (29%)


The students answered two Likert-scale items,

I would use the gamified GitLab in the future. GitLab supported me during the

8 students (53%) agreed that they would use the gamified environment in the future, and three (13%) disagreed with the sentence. More

pointed that the gamified environment supported them during their contribution pr

lanation relates to the results presented , which shows that points are good on provid-

presenting most part of the grades equal or below 5. These results for the

element are not surprising, since the goal of this element was not orienting, but motivating the players. How-

element, From

possible explanation is that information offered with-

out considering it as a part of the flow that guided them from

check our results, we asked the students to tell which elements were motivators and

ble It is possible to notice that the results are, partially, aligned

important role on orientation, and, to some extent, motivated the stu-dents during their contributions. We also see that Ranking


were moti-

how students , as an independent element or something that

None

-

-

5

(29%)


scale items,

uring the

8 students (53%) agreed that they would use the gamified environment in the future, and three (13%) disagreed with the sentence. More

pointed that the gamified environment supported them during their contribution pro-

Figure 9.

In this papergamified environment dents toa subset of gaming elementsmotivate students to place a first contribution to an OSS project

By conducting thisthat using were effective on barriers andof the questtion. The use of provide feedback to the students, and keeping them motivaed to contribute.

In addition, we could observe thatsome extent, Ranking, keep contributing to OSS projeno external reward was provided to the students. A possible way to boost the motivation would be to reward the students using their points and/or ranking.

Unlike the other elementsLevel did not sthem overcompointed that the possibility of ed them to change level

The results presented here, although preliminary, can ptentially benegame design elements to support newcomers and help engaing and motivating their first steps. We highlight the quests as a good way to orient newcomers towards a first contribtion and keep them motivated during thi

Although the results indicate that a gamified environment can support can foster more contributions or keep newcomers engaged. It is important to observe that our goal was to motivatin an academic environment, to put them in touchcodebase and issuesOSS prkeeping a coFurther and longtigate it, since it would be necessary to change internal prlems.

In the future, in a longer study, focusing on analyzing sessing the use of external rewards. It is part of this project use the gamified environment to students and faculties of the university and create a program to reward top contributors and to validate contributicredits for their courses. We plan to use this environment as a test bed for new elements and to evaluate engagement and retention of students.

Figure 9. Results for the evaluation of the gamified GitLab as a and a way to orient the students

V.

In this paper, we presented a preliminary study on how a gamified environment dents to contribute to a subset of gaming elementsmotivate students to

place a first contribution to an OSS projectBy conducting this

that using quests composed of objective and sequential tasks, effective on mitigating/reducing t

barriers and supporting newcomers’ quests provided a step

The use of pointsprovide feedback to the students, and keeping them motivaed to contribute.

In addition, we could observe thatsome extent, Ranking, keep contributing to OSS projeno external reward was provided to the students. A possible way to boost the motivation would be to reward the students using their points and/or ranking.

Unlike the other elementsdid not seem to

them overcoming orientation barriers. However, students pointed that the possibility of ed them to change level

The results presented here, although preliminary, can ptentially benefit OSS communities, who can make use of game design elements to support newcomers and help engaing and motivating their first steps. We highlight the quests as a good way to orient newcomers towards a first contribtion and keep them motivated during thi

Although the results indicate that a gamified environment can support newcomerscan foster more contributions or keep newcomers engaged. It is important to observe that our goal was to motivat

academic environment, to put them in touchodebase and issues, and to contribute for the first time to an

OSS project. This process is different from the process of keeping a community active or of retaining contributors. Further and long-term research wtigate it, since it would be necessary to change internal processes, which can lead to other types of pro

In the future, we plan to further evaluate these elements in a longer study, focusing on analyzing sessing the use of external rewards. It is part of this project use the gamified environment to students and faculties of the university and create a program to reward top contributors and to validate contributicredits for their courses. We plan to use this environment as a test bed for new elements and to evaluate engagement and retention of students.

Results for the evaluation of the gamified GitLab as a and a way to orient the students

V. CONCLUSION

we presented a preliminary study on how a gamified environment can help orienting and

OSS projects. We specifically analyzed a subset of gaming elements and assessed their ability to

to overcome orientation barriers inherent place a first contribution to an OSS projectBy conducting this study with students, we could observe

composed of objective and sequential tasks, mitigating/reducing t

supporting newcomers’ provided a step-by-step towards the contrib

points was also viewedprovide feedback to the students, and keeping them motiva

In addition, we could observe thatsome extent, Ranking, can be used to motivate students to keep contributing to OSS projects. In this preliminary study, no external reward was provided to the students. A possible way to boost the motivation would be to reward the students using their points and/or ranking.

Unlike the other elements, according to the studentseem to keep students

orientation barriers. However, students pointed that the possibility of unlockinged them to change levels.

The results presented here, although preliminary, can pfit OSS communities, who can make use of

game design elements to support newcomers and help engaing and motivating their first steps. We highlight the quests as a good way to orient newcomers towards a first contribtion and keep them motivated during thi

Although the results indicate that a gamified environment newcomers, we are not able to say whether this

can foster more contributions or keep newcomers engaged. It is important to observe that our goal was to motivat

academic environment, to put them in touch, and to contribute for the first time to an

ject. This process is different from the process of munity active or of retaining contributors.

term research would be necessary to invetigate it, since it would be necessary to change

cesses, which can lead to other types of pro

we plan to further evaluate these elements in a longer study, focusing on analyzing sessing the use of external rewards. It is part of this project use the gamified environment to host projects conducted by students and faculties of the university and create a program to reward top contributors and to validate contributicredits for their courses. We plan to use this environment as a test bed for new elements and to evaluate engagement and

Results for the evaluation of the gamified GitLab as a and a way to orient the students

ONCLUSION

we presented a preliminary study on how a orienting and engaging st

OSS projects. We specifically analyzed and assessed their ability to orientation barriers inherent

place a first contribution to an OSS project. study with students, we could observe

composed of objective and sequential tasks, mitigating/reducing the orientation related

supporting newcomers’ first steps. The structure step towards the contrib

viewed as a good strategy to provide feedback to the students, and keeping them motiva

In addition, we could observe that Quests, Pointscan be used to motivate students to

cts. In this preliminary study, no external reward was provided to the students. A possible way to boost the motivation would be to reward the students

, according to the studentskeep students motivated, or

orientation barriers. However, students unlocking new quests

The results presented here, although preliminary, can pfit OSS communities, who can make use of

game design elements to support newcomers and help engaing and motivating their first steps. We highlight the quests as a good way to orient newcomers towards a first contribtion and keep them motivated during this journey.

Although the results indicate that a gamified environment are not able to say whether this

can foster more contributions or keep newcomers engaged. It is important to observe that our goal was to motivat

academic environment, to put them in touch , and to contribute for the first time to an


ould be necessary to invetigate it, since it would be necessary to change communities’

cesses, which can lead to other types of pro

we plan to further evaluate these elements in a longer study, focusing on analyzing behavior and asessing the use of external rewards. It is part of this project

host projects conducted by students and faculties of the university and create a program to reward top contributors and to validate contributicredits for their courses. We plan to use this environment as a test bed for new elements and to evaluate engagement and

Results for the evaluation of the gamified GitLab as a motivator

we presented a preliminary study on how a engaging stu-

OSS projects. We specifically analyzed and assessed their ability to orientation barriers inherent

study with students, we could observe composed of objective and sequential tasks,

he orientation related . The structure

step towards the contribu-as a good strategy to

provide feedback to the students, and keeping them motivat-

Quests, Points, and, to can be used to motivate students to

cts. In this preliminary study, no external reward was provided to the students. A possible way to boost the motivation would be to reward the students

, according to the students, motivated, or to help

orientation barriers. However, students quests motivat-

The results presented here, although preliminary, can po-fit OSS communities, who can make use of

game design elements to support newcomers and help engag-ing and motivating their first steps. We highlight the quests as a good way to orient newcomers towards a first contribu-

Although the results indicate that a gamified environment

are not able to say whether this can foster more contributions or keep newcomers engaged. It is important to observe that our goal was to motivate students

with a real , and to contribute for the first time to an


ould be necessary to inves-communities’

cesses, which can lead to other types of prob-

we plan to further evaluate these elements behavior and as-

sessing the use of external rewards. It is part of this project to host projects conducted by

students and faculties of the university and create a program to reward top contributors and to validate contributions as credits for their courses. We plan to use this environment as a test bed for new elements and to evaluate engagement and

REFERENCES

[1] V. Goduguluri, T. Kilamo, and I. Hammouda, “Kommgame: A reputation environment for teaching open source software,” in IFIP International Conference on Open Source Systems, 2011, pp. 312–315.

[2] B. Morgan and C. Jensen, “Lessons Learned from Teaching Open Source Software Development,” in 10th International Conference on Open Source Systems, OSS 2014, San José, Costa Rica, May 6-9, 2014., 2014, pp. 133–142.

[3] D. M. Nascimento, K. Cox, T. Almeida, W. Sampaio, R. A. Bittencourt, R. Souza, and C. Chavez, “Using Open Source Projects in software engineering education: A systematic mapping study,” in 2013 IEEE Frontiers in Education Conference, 2013, pp. 1837–1843.

[4] I. Steinmacher, T. Conte, M. A. Gerosa, and D. Redmiles, “Social Barriers Faced by Newcomers Placing Their First Contribution in Open Source Software Projects,” in Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, 2015, pp. 1379–1392.

[5] I. Steinmacher, T. Conte, C. Treude, and M. A. Gerosa, “Overcoming Open Source Project Entry Barriers with a Portal for Newcomers,” in 38th International Conference on Software Engineering, 2016 (ICSE 2016), 2016.

[6] M. Zhou and A. Mockus, “Does the initial environment impact the future of developers,” in Proceedings of the 33rd International Conference on Software Engineering, 2011, pp. 271–280.

[7] G. Canfora, M. di Penta, R. Oliveto, and S. Panichella, “Who is Going to Mentor Newcomers in Open Source Projects?,” in Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering, 2012, pp. 44:1–44:11.

[8] A. Schilling, S. Laumer, and T. Weitzel, “Who Will Remain? An Evaluation of Actual Person-Job and Person-Team Fit to Predict Developer Retention in FLOSS Projects,” in Proceedings of the 2012 45th Hawaii International Conference on System Sciences, 2012, pp. 3446–3455.

[9] M. Zhou and A. Mockus, “What make long term contributors: Willingness and opportunity in OSS community,” in Proceedings of the 34th International Conference on Software Engineering, 2012, pp. 518–528.

[10] Deterding, Sebastian, Dixon, Dan, Khaled, Rilla, Nacke, and Lennart, “From Game Design Elements to Gamefulness: Defining "Gamification,” in Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, 2011, pp. 9–15.

[11] A. P. O. Bertholdo and M. A. Gerosa, “Promoting Engagement in Open Collaboration Communities by Means of Gamification,” in HCI International 2016 – Posters’ Extended Abstracts: 18th International Conference, HCI International 2016 Toronto, Canada, July 17–22, 2016 Proceedings, Part II, C. Stephanidis, Ed. Cham: Springer International Publishing, 2016, pp. 15–20.

[12] S. K. Sheth, J. S. Bell, and G. E. Kaiser, “Increasing Student Engagement in Software Engineering with Gamification,” 2012.

[13] R. van der V. Philipp Lombriser, “Improving the Quality of the Software Development Lifecycle with Gamification,” p. 22, 2014.

[14] O. Pedreira, F. García, N. Brisaboa, and M. Piattini, “Gamification in software engineering – A systematic mapping,” Information and Software Technology, vol. 57, pp. 157–168, 2015.

[15] A. Bartel and G. Hagel, “Gamifying the learning of design patterns in software engineering education,” in Global Engineering Education Conference (EDUCON), 2016 IEEE, 2016, pp. 74–79.

[16] D. Dicheva, C. Dichev, G. Agre, and G. Angelova, “Gamification in Education: A Systematic Mapping Study.,” Educational Technology & Society, vol. 18, no. 3, pp. 75–88, 2015.

[17] F. Steffens, S. Marczak, F. F. Filho, C. Treude, L. Singer, D. Redmiles, and B. Al-Ani, “Using Gamification as a Collaboration Motivator for Software Development Teams: A Preliminary Framework,” in SBSC ’15: Brazilian Symp. in Collaborative Systems, 2015, p. 10pp.

[18] A. Marczewski, Gamification: a simple introduction. Andrzej Marczewski, 2013, p. 153.

[19] G. Surendeleg, V. Murwa, H.-K. Yun, and Y. S. Kim, “The role of gamification in education a literature review,” Contemporary Engineering Sciences, vol. 7, no. 2932, pp. 1609–1616, 2014.

[20] D. J. Dubois and G. Tamburrelli, “Understanding Gamification Mechanisms for Software Development,” in Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, 2013, pp. 659–662.

[21] A. de Melo, M. Hinz, G. Scheibel, C. Diacui Medeiros Berkenbrock, I. Gasparini, and F. Baldo, “Version Control System Gamification: A Proposal to Encourage the Engagement of Developers to Collaborate in Software Projects,” in Social Computing and Social Media, Springer, 2014, pp. 550–558.

[22] R. Lotufo, L. Passos, and K. Czarnecki, “Towards improving bug tracking systems with game mechanisms,” in Proceedings of the 9th IEEE Working Conference on Mining Software Repositories, 2012, pp. 2–11.

[23] W. Snipes, A. R. Nair, and E. Murphy-Hill, “Experiences Gamifying Developer Adoption of Practices and Tools,” in Companion Proceedings of the 36th International Conference on Software Engineering, 2014.

[24] D. F. Bacon, Y. Chen, D. Parkes, and M. Rao, “A Market-based Approach to Software Evolution,” in Proceedings of the 24th ACM SIGPLAN Conference Companion on Object Oriented Programming Systems Languages and Applications, 2009, pp. 973–980.

[25] S. Sheth, J. Bell, and G. Kaiser, “Halo (highly addictive, socially optimized) software engineering,” in Proceedings of the 1st International Workshop on Games and Software Engineering, 2011, pp. 29–32.

[26] C. Thomas and K. Berkling, “Redesign of a gamified software engineering course,” in Interactive Collaborative Learning (ICL), 2013 International Conference on, 2013, pp. 778–786.

[27] Jan Nonnen Christian R. Prause and M. Vinkovits, “A Field Experiment on Gamication of Code Quality in Agile Development,” 2012.

[28] J. M. Rojas and G. Fraser, “Teaching Mutation Testing using Gamification,” in European Conference on Software Engineering Education (ECSEE), 2016.

[29] I. Steinmacher, T. Conte, M. A. Gerosa, and D. F. Redmiles, “Social Barriers Faced by Newcomers Placing Their First Contribution in Open Source Software Projects,” in Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, 2015, pp. 1–13.

[30] M. Csikszentmihaly, Beyond Boredom and Anxiety: Experiencing Flow in Work and Play, 25th ed. Hoboken, NJ, EUA: Jossey-Bass, 2000, p. 231.

Using Gamification to Orient and Motivate Students to ...igor.pro.br/publica/papers/CHASE2017.pdf · literature brings approaches to improve existing processes and motivate developers

Documents