INTERIM REPORT ON PROMOTING CITIZEN ......INTERIM REPORT ON PROMOTING CITIZEN PARTICIPATION IN CAMPAIGNS FLAMENCO Project Deliverable 2.1a VUB (SMIT) Responsible SMIT (VUB) Authors

Vrije Universiteit Brussel

D2.1a - Interim Report on Promoting Citizen Participation in CampaingsSeymoens, Tom; Hoelck, Katharina; Bleumers, Lizzy; Lievens, Bram

Publication date:2017

License:Other

Document Version:Final published version

Link to publication

Citation for published version (APA):Seymoens, T., Hoelck, K., Bleumers, L., & Lievens, B. (2017). D2.1a - Interim Report on Promoting CitizenParticipation in Campaings: Flamenco Project Deliverable 2.1a. Unknown.

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portalTake down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.

Download date: 01. Jan. 2021

https://cris.vub.be/en/publications/d21a--interim-report-on-promoting-citizen-participation-in-campaings(44c9c013-25fe-4fa2-b434-35fb7c094afc).html

https://cris.vub.be/en/persons/tom-seymoens(0ea0bff4-54ff-4a6c-a601-42c0ffc75bf8).html

https://cris.vub.be/en/persons/katharina-hoelck(99c89aea-c38b-4561-a4e8-326d2929d1fc).html

https://cris.vub.be/en/persons/lizzy-bleumers(b859ce68-4719-483e-9740-27f556e0e965).html



INTERIM REPORT ON PROMOTING

CITIZEN PARTICIPATION IN

CAMPAIGNS

FLAMENCO Project Deliverable 2.1a

VUB (SMIT)

Responsible SMIT (VUB)

Authors Tom Seymoens (imec-SMIT, VUB), Katharina Hölclk (imec-SMIT, VUB), Lizzy Bleumers

(imec-SMIT, VUB), Bram Lievens (imec-SMIT, VUB).

Promoting participation – interim report 2

TABLE OF CONTENTS

1 Introduction ...................................................................................................................... 4

2 Chapter 1: An Exploration of the Field: Citizen Science.............................................. 5

2.1 A Brief History and Definition of Citizen Science ..............................................................5

2.1.1 Citizen Science vs Related Concepts ...............................................................................6

2.1.2 Participatory Sensing .......................................................................................................8

2.2 Classifications of Citizen Science Projects ..........................................................................9

2.2.1 Classification of Bonney et al. (2009) .............................................................................9

2.2.2 Classification of Haklay (2013) .....................................................................................10

2.2.3 Classification of Wiggins and Crowston (2011) ...........................................................11

2.2.4 The Flamenco Platform as a Supporting Layer Across Typologies ..............................13

2.3 Lessons Learned for Flamenco ...........................................................................................15

3 Chapter 2: Initiating a Citizen Science Project: Reasons and Hurdles ..................... 17

3.1 Motivations for Choosing Citizen Science .........................................................................17

3.1.1 To contribute to science ............................................................................................17

3.1.2 To inform policy .........................................................................................................17

3.1.3 To educate ....................................................................................................................18

3.1.4 To improve buy-in ......................................................................................................18

3.1.5 To raise awareness and engage people ....................................................................18

3.1.6 To build partnerships and improve communication between different

stakeholders .................................................................................................................................18

3.1.7 To gain personal satisfaction ....................................................................................18

3.2 Barriers Inhibiting the Use of Citizen Science ..................................................................20

3.2.1 Data Quality and Biases ............................................................................................20

3.2.2 Requirement of Specialist Equipment / Knowledge .............................................20

3.2.3 Politics ..........................................................................................................................20

3.2.4 Unaware of audience ..................................................................................................20

3.2.5 Research design and implementation issues ..........................................................20

3.2.6 Participant Involvement and Identification ............................................................21

3.2.7 Time consuming and Resourcing issues .................................................................21

3.2.8 Peer review / Mistrust ................................................................................................21

3.2.9 Uncomfortable / unprepared to work with the public ...........................................21

3.3 Overcoming the Barriers for CS Project Initiation & Flamenco ....................................22


4 Chapter 3: Participation in a Citizen Science Project: Motivations and Hurdles.... 24

4.1 Participation: Best Practices ...............................................................................................24

4.2 Factors Affecting Participation ..........................................................................................26

4.2.1 Motivations for Participation in Citizen Science Projects .............................................27

4.2.2 Organizational factors & Participation in Citizen Science Projects ..............................30

4.3 A Segmentation of Participants in Citizen Science Projects ............................................33

4.4 Factors that Benefit Participation & Flamenco ................................................................35

5 Gamification and Behavioral Change........................................................................... 38

5.1 Introduction: Gamification in the context of Behavioral Change ...................................38

5.2 Definition: Gamification and its elements .........................................................................40

5.3 Application: The potential of gamification for citizen science ........................................45

6 Conclusion ....................................................................................................................... 47

7 Bibliography .................................................................................................................... 51

1 INTRODUCTION

The Flamenco-project aspires to develop an open, reusable and reconfigurable citizen observatory

platform for Flanders. This platform should enable a myriad of different stakeholders to launch a

participatory sensing campaign on their topic of choice, each with its specific technological

prerequisites (e.g. the type of data gathered, its source, how it should be stored, etc.) and unique social

circumstances.

Part of the project1 addresses questions surrounding the latter, by taking on two broad objects of study.

Part of the research is reserved to an in-depth analysis of current practices, opportunities and

challenges surrounding environmental and mobility-related citizen science campaigns. This document

targets the second research area and focuses on concepts of motivation and engagement2. It commits

to an investigation, by means of an extensive literature review, into the key mechanisms that motivate

citizens to partake in participatory sensing campaigns.

Chapter one of this interim report starts with defining and differentiating citizen science (CS),

participatory sensing and other related concepts. Afterwards three complementary typologies of CS

projects are presented. In chapter two, we zoom in on the CS organizer’s point of view and question

the reasons for and against the application of the CS methodology. In chapter three we shift focus

toward the participant as we in succession digest the concept of participation, how their motivation

and the organization of the project influences contribution and a segmentation of participants. The

fourth and final chapter of this document is dedicated to behavioral change and gamification. First, an

introduction to general behavior change approaches is given. Gamification is framed as one of several

tools which can be used as part of strategies to steer behavior. Then, the concept is explored in further

depth. Important elements of gamification are outlined. Finally, the application of the concept within

the context of citizen science is discussed.

This report ends with an enumeration of lessons learned from this extensive literature review for

organizers of citizen science projects. As a follow-up we will examine how the distinguished concepts

will be applied in a framework for analyses of CS case studies. The latter will be comprehensively

presented in the next iteration of this report.3

1 Work package 2: from society to technology

2 This relates specifically to task 2.1 of the project (Determinants for motivation and engagement)

3 Integrated report on promoting citizen participation (D2.1b)


2 CHAPTER 1: AN EXPLORATION OF THE FIELD: CITIZEN SCIENCE

2.1 A BRIEF HISTORY AND DEFINITION OF CITIZEN SCIENCE

The first use of the Citizen Science (CS hereafter) concept is attributed to the National Audobon

Society. Back in January 1989, Audobon set up a research program involving 225 citizen volunteers

for the monitoring of acid rain (Gharesifard, Wehn, & Van Der Zaag, 2017). CS gained more traction

in the mid-‘90s under the influence of two authors: Alan Irwin and Rick Bonney (UWE, 2013). Alan

Irwin applied the concept to articulate his sense of urgency to cease the depreciation of lay people’s

expertise and judgement, and rather to endorse more contextual forms of knowledge and

understanding (Irwin, 1995). The ambition is to end up with a more responsive science, by bringing a

greater understanding between the public and scientific sphere (Dickel & Franzen, 2016). Rick

Bonney, on the other hand, approaches CS as a method which enables the general public to contribute

to science through data gathering (Riesch & Potter, 2014; UWE, 2013).

Geoghegan et al. (2016) have established eight principles that surround the concept of CS.

Interestingly, they also allow non-scientific stakeholders from policy and practice communities to

influence or even initiate the different phases of a CS project. The eight principles are the following

(Geoghegan et al., 2016, p. 23):

1. Widening participation in science;

2. Recognizing benefits of participation to citizen;

3. Leading to measurable academic output and/or being used by citizens;

4. Harnessing emotional attachments to particular subjects;

5. Carrying out activities across a range of skills levels

6. Sharing data between experts (paid and voluntary);

7. Prioritizing science over engagement; and

8. Talking about ‘science’ only without separating citizen science and traditional science.

Public participation has been a part of the scientific tradition for a long time, through the extended use

of, amongst other things surveys, interviews and focus groups. CS can be seen as a natural

progression from this practice through its central characteristic of active engagement of the public

(Wiggins & Crowston, 2011).

Based on these previous attempts, we define CS in this report as the intentional and active

engagement, in a non-professional capacity of volunteers in the scientific process (Pocock, Tweddle,

Savage, Robinson, & Roy, 2017, p. 1). More specifically their contribution is possible in the stages of

research initiation and design, data collection, analysis, interpretation and dissemination of the results

(Bonney et al., 2009; Geoghegan et al., 2016; Pocock et al., 2017, p. 1; Rotman et al., 2012). CS,

defined in this manner, acknowledges the value of local and non-expert knowledge to inform public


consultation and engagement in the scientific process, adhering to Irwin’s definition. Section 2.2.3

discloses that public science education is often a (non-)intended consequence of CS through the active

participation of the public. This is expected to have a positive influence on the relationship between

science and society (Martin, 2017), which brings us closer to Irwin’s ambition. The proposed

definition is also allegiant to Bonney’s definition, but expands it by allowing the contributor’s

involvement in the different phases of the scientific process.

CS brought forth the creation of a new type of researcher, the citizen scientist. Whereas the métier of

the scientist used to be restricted to individuals with an elaborate training and the associated

qualifications, citizen scientists are defined by Rotman et al. (2012, p. 1) as “individuals who typically

lack formal credentials and do not hold professional positions in scientific institutions or projects,

who participate in scientific endeavors related to their personal interests”. The segmentation of

Martin (2017), slightly nuances this definition as we will see in section 4.3 as people that work in

academia or have enjoyed tertiary education seem more likely to participate in citizen science projects.

CS has already been proven beneficial in a multitude of domains, such as biology, environmental

studies, astronomy, chemistry and mathematics (Rotman et al., 2012). Several societal and

technological evolutions have contributed to this potential of CS. From a technological point of view,

there has been the emergence of distributed technologies in the last fifteen years that enable the

conduction of “scientific inquiries on a larger scale than ever before by extending beyond

geographically collocated tasks and enabling distributed collaboration” (Beza et al., 2017; Rotman et

al., 2012, p. 1). This technological innovation allows for improved data flows and better feedback to

volunteers (Geoghegan et al., 2016). From a sociological point of view, increasing literacy and

education have supported the citizens’ growing capabilities and self-confidence for contributing in

scientific research (Mccrory, Veeckman, Coppens, & Claeys, 2017). Although further research is

needed, other contributing sociological evolutions might be distinguished. One might wonder, for

instance, how a shift in civic engagement from a dutiful citizen to an actualizing citizen as presented

by Bennett (2008) has an effect on a greater aspiration for participating in the scientific community.

CITIZEN SCIENCE VS RELATED CONCEPTS

A lot of different concepts are closely related to the one of citizen science, they are often used

interchangeably and as a consequence their meaning gets intertwined. In this section we provide some

clarity on the matter, based on the overview created by See et al. (2016). Their inquiry presented the

different terms that relate to the subject of citizen-derived geographical information. We only refer

here to the most important concepts.

Concept Definition


Public Participation in Scientific Research In the analysis of Bonney et al. (2009), that is

discussed in-depth in the next paragraph, extensive use

is made of the concept “public participation in

scientific research” (PPSR). This concept denotes the

level of public involvement in the different research

steps. See et al. (2016) consider this to be equivalent to

the Citizen Science concept.

Science 2.0 Science 2.0 refers to the process where the increasingly

global, collaborative, interdisciplinary nature of the

scientific inquiry is supported by IT, the internet and

mobile devices. See et al. (2016) notice how the greater

potential of inclusion of citizens is only one component

in science 2.0

Crowdsourcing Crowdsourcing entails the act of addressing a

heterogeneous group of individuals through an open

call for voluntarily taking part in a specific task of

varying complexity. The term is more restrictively

applied to purely online activities, that entail a mutual

benefit for the participant (e.g. economic, social

recognition, self-esteem, development of individual

skills) and the crowdsourcer (the resources of the

participant). It differs itself in this sense from citizen

science as their calls for actions are not necessarily

open (e.g. restricted in geographical location, skills

required, time), nor is the mutual benefit implied as the

data collected might be used for other citizen science

purposes later (See et al., 2016).

Citizens’ Observatory Citizens’ observatory can be defined as a process of

joint observation and mutual information and resources

transfer. It thus “[…] promotes communication and the

sharing of technological solutions (e.g., sensors, mobile apps, web portals) and community participatory

governance methods among citizens” (Liu, Kobernus,

Broday, & Bartonova, 2014, p. 4). A citizen

observatory platform makes sure that resources such as

the data, sensors, issues, communication, guidelines

and reports can be shared in a two-way model between

participants and between participants and organizers.

Citizens’ observatory can be seen as an underlying, supporting infrastructure for citizen science projects,

that can in itself make use of participatory sensing data

collection techniques.

Participatory Sensing Participatory sensing has to be seen as a method of data

collection, and can thus be a component of

crowdsourcing or a contribution to an overall citizen

science project (which also takes into account the other

research tracks such as analysis). More specifically,

participatory sensing is data collection and knowledge

sharing by way of smart mobile devices, “[…]

deployed as part of an interactive participatory sensor

network” (See et al., 2016, p.7).

TABLE 1: RELATED CONCEPTS OF CS (BASED ON SEE ET AL. (2016)


Through inaccurate use, the meaning of the different concepts can easily be vulgarized. The common

elements are the connotation they bear of a heightened (potential of) citizen involvement and its

mediation or facilitation by information technology. The degree of both varies in the different

approaches.

Participatory sensing, the last concept in the list, is of paramount importance for the Flamenco-project.

For this reason, we elaborate further on the concept in the next section.

PARTICIPATORY SENSING

Urban sensing can encompass both the use of sensors embedded in a city’s infrastructure, as well as

sensing that relies on the mobile devices that people carry with them as they move around in the city

(Chatzigiannakis, Mylonas, & Vitaletti, 2011). The latter interpretation of urban sensing is often

referred to as people-centric sensing (Lane, Eisenman, Musolesi, Campbell, & Miluzzo, 2008). Lane

and colleagues (2008) distinguish a range of approaches to people-centric sensing based on the extent

to which people, who are using the device, are aware of the sensing process and involvement in the

decision-making process related to the sensing system. Participatory sensing is situated at one end of

the spectrum, and opportunistic sensing at the other end.

Lane and colleagues describe participatory and opportunistic sensing as follows (2008).

Participatory sensing means people knowingly choose to meet requests made by the application on

the device they are carrying. Furthermore, they are involved in choices, for example, on what data is

being shared. Opportunistic sensing, on the other hand, refers to approaches where users of the

mobile device may not be aware of sensing applications, as these are running in the background and

do not require the user to take any action to meet application requests. Here, the focus is on having

sensing take place in a non-intrusive way so that people can use their device for their own purposes,

instead of being actively involved in sensing and associated decision-making. The definition of

participatory sensing highlights that the phenomenon is defined not solely from a technological

perspective – how sensing is implemented – but also from a social perspective – the way people are

engaged and participate in it.

Haklay (2013) considers participatory sensing as a relatively new expression of citizen science. Like

‘volunteered computing’, where people allow projects to use the computing power of their personal

devices (e.g. SETI@home) and ‘volunteered thinking’, where people voluntarily engage in scientific

analysis (e.g. FoldIt), Haklay groups participatory sensing under the category of citizen cyberscience,

a variant of citizen science relying on affordances of technology (such as the Internet, personal

computers and mobile devices, sensors, …) as a scientific tool.


2.2 CLASSIFICATIONS OF CITIZEN SCIENCE PROJECTS

In the previous section, we outlined the concept of CS and how it distinguishes itself from related

concepts, such as participatory sensing. In what follows we aspire to take a closer look at

classifications of citizen science projects. As the Flamenco-platform aspires to host projects with

different scopes, this overview provides a basis for understanding what can be expected. Furthermore,

it allows for a better comparative evaluation of existing use cases, which in itself grants us a greater

comprehension of the different elements underpinning success and/or failure of the CS projects

(Pocock et al., 2017). We successively glance over the classifications of Bonney et al. (2009), Haklay

(2013) and Wiggins and Crowston (2011). In the final part of this section, we attempted to connect

the different classifications.

CLASSIFICATION OF BONNEY ET AL. (2009)

An often referred to classification of CS projects is provided by Bonney et al. (2009). They have

made the distinction based on the degree of public involvement in the different research activities,

which they broke down as follows:

1. Choosing or defining questions for study;

2. Gathering information and resources;

3. Developing explanations (hypotheses) about possible answers to questions;

4. Designing data collection methodologies (both experimental and observational);

5. Collecting data;

6. Analysing data;

7. Interpreting data and drawing conclusions;

8. Disseminating conclusions; and

9. Discussing results and asking new questions

They consequently came up with the following subcategories (Bonney et al., 2009, p.11; Mccrory et

al., 2017; Rotman et al., 2012):

a) Contributory projects: These types of projects are organized by researchers; the public

participates by gathering the required data. The use of the public is advantageous since the

necessary data needs to be collected over “wide geographic areas or over long spans of time”

(Bonney et al., 2009, p. 18). An example given by the authors is the FeederWatch-project, for

which the participants provided data on the birds in their backyard, the researchers then

analyze the data.

b) Collaborative projects: Similar to the contributory projects, the research is designed by

scientists, the public’s involvements however can also include the analysis, interpretation and

presentation of the data to others. The example given by Bonney et al. (2009) is that of water

quality monitoring projects, where the participants not only collect the samples, but also

perform analyses and present their findings towards local government agencies.

c) Co-created projects: Contrary to both collaborative and contributory, in co-created projects

it is the public itself that have questions that need to be answered. Together with scientists


they design the rest of the research steps. As an example, the authors mention public health or

environmental restoration initiatives.

Step in Scientific Process Contributory Projects Collaborative Projects Co-Created Projects

Choose or define question(s) for study X

Gather information and resources X

Develop explanations (hypotheses) X

Design data collection methodologies (X) X

Collect samples and/or record data X X X

Analyze samples X X

Analyze data (X) X X

Interpret data and draw conclusions (X) X

Disseminate conclusions / Translate

results into action (X) (X) X

Discuss results and ask new questions X

TABLE 2: MODELS FOR PUBLIC PARTICIPATION IN SCIENTIFIC RESEARCH (BONNEY ET AL., 2009, P.17)

The most prevailing alliances between volunteers and scientists still never move beyond contribution.

The most common practice entails volunteers supporting the scientists with data collection and partial

data analysis. One can argue that this type of citizen science does not go beyond and is actually

similar to ‘traditional science’ with data stemming from crowdsourcing initiatives. The output is

delivered to scientists who continue the analyses and interpretation to apply it in their research

(Rotman et al., 2012). The barriers mentioned in section 3.2 explain the reasons for this. The

consortium should be aware of this and further investigate when public involvement is advantageous

for project initiators, and how this can be communicated to them. As such, opening up more functions

of the platform when required.

CLASSIFICATION OF HAKLAY (2013)

Analogous to Bonney et al. (2009), Haklay (2013)’s classification is based on the observation that

different degrees of participation and engagement are possible beyond mere data collection (See

Figure 1). At the most basic level, called crowdsourcing, participants deliver resources by gathering

measurements, samples and observations. At level 2, called distributed intelligence, they are

collectively involved in simple interpretations of the data that is gathered. At level 3, participatory


science, the public is involved from the outset. They help define the problem or issue to be tackled

before conducting the scientific activities, and receive help from ‘experts’ in analysing the data.

Finally, at level 4, extreme citizen science, the public is involved throughout the entire scientific

process, possibly but not necessarily collaborating with or facilitated by professional scientists.

FIGURE 1. LEVELS OF PARTICIPATION IN CITIZEN SCIENCE PROJECTS. (ADAPTED FROM FIGURE 2 – LEVELS OF

PARTICIPATION AND ENGAGEMENT IN CITIZEN SCIENCE PROJECTS BY HAKLAY, 2013, FIGURE 2).

CLASSIFICATION OF WIGGINS AND CROWSTON (2011)

Wiggins & Crowston (2011, p.5-8) proposed a different typology of citizen science projects, which is

goal-oriented and identifies five mutually exclusive and exhaustive types. Where the classification of

Bonney et al. (2009) takes into account the influence the public can exert over the different research

steps, the typology of Wiggins & Crowston divides projects based on their intent.

For each type, they identified the inherent scientific, organizational and technology issues.

1. Action-oriented citizen science projects: Often a “bottom-up” and local initiative, set-up by

citizens and aiming to trigger an actual participant intervention in a specific concern.

Examples include wildlife concerns of a local population that then apply scientific methods to

investigate the matter and establish solutions.

a. Scientific issues: Since the projects are often initiated by stakeholders other than

scientists, their role is limited to consultants. As the aim often is to gather evidence

for an intervention, the data is less likely to be picked up by the scientific community.

Lev

els

of

par

tici

pat

ion i

n

citi

zen s

cien

ceLevel 4: Extreme citizen science

Level 3: Participatory science

Level 2: Distributed intelligence

Level 1: Crowdsourcing


Moreover, the aggregation of data from different projects is hurdled by the use of

different scientific methods.

b. Organizational issues: Main challenge for these type of projects, and mainly due to

their grassroots origin, is attaining sustainability. They are oftentimes dependent on

donations and fundraising.

c. Technology issues: Due to the often-humble means of these type of projects,

involving technology is more a burden as they rely on volunteers to develop and

maintain the infrastructure.

2. Conservation-oriented citizen science projects: Often a top-down (researcher-initiated) or

middle-out (management-initiated) and regional initiative, it aims to support stewardship and

natural resource management goals. Public participation often demotes to mere data

collection, but their stewardship is often cultivated through educational material.

a. Scientific issues: When the projects are not under academic oversight, they are

typically still lead by professional researchers in governmental organisations. While

the main goal is to support resource-management decision-making, scientific validity

is valued highly.

b. Organizational Issues: Often dependable upon state funds or grants and part of

complex collaboration partnerships with regional actors and federal agencies.

c. Technology Issues: Most seem to be using web technologies for data entry and results

access.

3. Investigation-oriented citizen science projects: They have a top-down structure and they

are focused on previously well-defined scientific research goals, which require data collection.

Education is often a secondary, though inexplicit purpose of these types of projects, through

the provision of educational materials and tasks.

a. Scientific issues: Since these projects pursuit academic outcomes, valid scientific

results constitute their core concern. Among the methods used to ensure the quality of

the results are the application of uniform equipment, triangulation, etc. Yet, they are

less likely to screen the volunteers, since this subverts the ambition of education.

Uneven spatial distribution of the participants is another threat to a solid scientific

outcome.

b. Organizational issues: Management and sustainability challenges can occur due to the

large number of volunteers necessary.

c. Technology issues: Variates depending on the underlying funding resources and

project lifespan. Most seem to be using web technologies for data entry and results

access.

4. Virtual projects: These can basically be referred to as Virtual Investigation-oriented citizen

science projects, since they also focus on previously well-defined scientific research goals.

The differentiating characteristic lies in the sole use of ICT in all project activities, the

physical location of the volunteers is not meaningful. As an example Wiggins & Crowston

(2011) refer to Galaxy Zoo, where participants had to perform virtual image recognition and

classification tasks.

a. Scientific issues: Similar to investigation-oriented projects, their main interest lies in

producing valid scientific results. Due to their nature, the challenge lies in designing

valuable virtual tasks that also manage to engage the participants. Gamification

elements can be an added value in this respect.

b. Organizational issues: As virtual projects refer to top-down academic research

programs, they are also dependable on funding


c. Technology issues: Complex IT platforms and technological tools are a necessity for

virtual projects, in that sense the help computer scientists are an asset to the project.

5. Education-oriented citizen science projects: a top-down initiative, for which education and

outreach are the primary goals. A further subdivision can be made in this respect by looking

into informal versus formal learning opportunities.

a. Scientific issues: Due to its focus on education, the project’s outcome does not

always translate to larger scientific research efforts. The target is to improve the

scientific inquiry skills of the participants. For this reason, the cost is relatively higher

than for other types of citizen science projects.

b. Organizational issue: A lot of the projects are part of a collaboration between

different partner organizations, and receive their revenue from funding sources. As a

result, it is often unclear for how long the project will be running.

c. Technology issues: The application of technology is variable, the projects’ websites

and tools have design constraints to appeal to their target student audience (be it

young children, families, teachers or students).

THE FLAMENCO PLATFORM AS A SUPPORTING LAYER ACROSS TYPOLOGIES

The three different typologies presented in the previous paragraphs can be bound together based on

the government of the project: top-down, middle-out or bottom-up.

• Action-oriented projects originate from the public and are thus strictly bottom-up. The role

of the scientific community is strictly guiding. The Flamenco platform should be able to

provide guiding mechanisms for the different research steps to support the achievement of

valuable results. In the classification of Bonney et al. (2009) these projects fall under the “co-

created” denominator. Haklay refers to these types of projects as “participatory science”, or

even “extreme citizen science”.

• Conservation-oriented projects originate from non-academic, often governmental

stakeholders. They are often still guided to some extent by researchers, but public stewardship

is cultivated. In the classification of Bonney et al. (2009) these projects could be contributory,

collaborative. It cannot be classified as “co-created” since the initiative is still top-down or

from a management level. They could be called “crowdsourcing” or “distributed intelligence”

according to Haklay.

• Investigation-, Virtual- and Education-oriented projects originate from the academic

community and are consequently top-down. In the classification of Bonney et al. (2009) these

projects can be found under contributory or collaborative. Haklay, in comparison, would opt

for the denominators “crowdsourcing” or “distributed intelligence”

It is interesting to map these classifications to the Flamenco case, where stakeholders who do not stem

from an academic field are encouraged to initiate their own citizen science project. Here, both

stakeholders and the public can be, but are not necessarily the same, and most importantly the

scientific support for the different research steps as distinguished above must be a technological

affordance.


Initiator Process Participant Support Classification

of Bonney et

al.

Classification

of Wiggins &

Crowston

Classification

of Haklay

Public / Ad

hoc

organisation

Bottom-

Up /

middle-

out

Tailored

Public /

themselves

Technological

affordance,

Academic SH

Co-Created Action-

oriented

Participatory

science /

Extreme

citizen science

Non-

Academic

Stakeholder

Top-

Down

Tailored

Public

Technological

affordance,

Academic SH

Contributory,

Collaborative

Conservation-

oriented

Crowdsourcing

/ Distributed

intelligence

Academic

Stakeholder

Top-

Down

Tailored

Public

Technological

affordance,

Academic SH

Contributory,

Collaborative

Investigation-

oriented,

Virtual

projects,

Education-

oriented

Crowdsourcing

/ Distributed

intelligence

TABLE 3: MAPPING THE THREE CLASSIFICATIONS BASED ON THE TYPE OF INITIATOR

Flamenco

2.3 LESSONS LEARNED FOR FLAMENCO

We learned in this first chapter how citizen science can be defined as the intentional and active

engagement, in a non-professional capacity of volunteers in the states of research initiation, research

design, data collection, analysis, interpretation and the eventual dissemination of results. We further

elaborated upon the concept of participatory sensing and defined it as a method of data collection and

knowledge sharing, used for citizen science. It applies smart mobile devices and people knowingly

and intentionally choose to share certain data.

In the second part, we looked at classifications of citizen science projects. Depending on which of the

phases in a citizen science project is taken up by the contributors, they can be categorized by the

classification of Haklay (2013) and Bonney et al. (2009). Alternatively, the classification of Wiggins

and Crowton (2011) is goal-oriented.

As mentioned in the introduction, the central aim of the Flamenco project is to enable a myriad

number of (ICT-agnostic) stakeholders to configure a citizen observatory platform for their specific

application needs and domains. We can imagine that depending on the specific project, more or less

influence is entrusted on or embraced by non-professional scientists. In order to hold its promise as an

open and reusable platform, it should thus provide the means to give support in the different research

tracks. This is even more important, when the project is not initiated by an academic partner, but by a

societal stakeholder. As indicated in Table 3 above, the goals of Flamenco seem to be best aligned

with the first classification. As it there provides support as a citizen observatory for projects that can

be defined as action-oriented, initiated from the bottom-up by a specific segment of the general

segment or ad-hoc organization. The public is thus involved from the outset and have to themselves

express the problem definition, analyze and report the results. The expert that aid the initiator in each

step, should be substituted in the Flamenco platform by expert knowledge imbedded in the system and

guiding the different research tracks. Which results in the following requirements:

1. The platform should thus provide a way to determine who initiates the citizen science project,

and what their corresponding skill-level is.

2. Depending on the skill-level of the initiator, guidelines should be provided on how to conduct

a citizen science project, which steps should be included and how they should go about each

step. This might require the creation of a semi-automatic workflow composition supporting

the initiator when needed, based on the embodiment of expert knowledge in the system.

3. The platform should thus provide a way to determine who participates in the citizen science

project, and what the corresponding skill-level is.


4. Depending on the skill-level and influence of the participant, guidelines should be provided

on how to help out in the citizen science campaign. Instructions could go from how to do a

proper data collection to how to correctly interpret the results.

5. Depending on the goal of the citizen science project, which can be indicated by the organizer,

different support mechanisms need to be set in place.

If the project succeeds in addressing variable needs and application domains it expands further upon

the core promise of citizen science as tool for a more tailored data collection through the distinct

expertise and capability of the targeted citizen (Conrad & Hilchey, 2011).

As mentioned, a core premise of participatory sensing campaigns is the notion of intent and conscious

participation and information sharing. A question that comes to mind is why should contributors do

that? What are the factors that trigger the intent of participation with volunteers? What constitutes

meaningful participation, how can the technology facilitate it, on the one hand, and how do

participants appropriate the sensing technology in accordance to the level of participation they seek?

These questions include but also extend beyond privacy-related issues. These are questions that we

will elaborate on in the third and fourth chapter of this document.

First, we will contemplate on the reasons of the initiators for instituting a participatory sensing

campaigns and what are the obstacles on their way that should be tackled by platforms like the one

provided by Flamenco.


3 CHAPTER 2: INITIATING A CITIZEN SCIENCE PROJECT: REASONS AND

HURDLES

Two valuable question worth taking the time to answer when creating a generic citizen observatory

for an array of stakeholders are: Why would one prefer to undertake a Citizen Science project and

what are potential obstacles hindering its adoption? In a report on behalf of the UK Environmental

Observation Framework (UKEOF), Geoghegan et al. (2016) provide an elaborate overview,

answering these questions. Its main results, supplemented with the results from an additional literature

review can be found below. We will start this paragraph with an overview of the distinguished

motivations for stakeholders to get involved in CS. Subsequently, we take a closer look to the

challenges and barriers.

3.1 MOTIVATIONS FOR CHOOSING CITIZEN SCIENCE Geoghegan et al. (2016) discerned seven types of motivations for Citizen Science Projects. Their

findings are supplemented with insights from a broader literature study. Where possible, we link their

motivations up with the goal-identifying classification of Wiggins & Crowston (2011), as broadly

discussed in section 2.2.3.

TO CONTRIBUTE TO SCIENCE

The most common motivation for initiating a CS project, are the opportunities it brings to advance

scientific knowledge. Herewith we refer to the incorporation of the resources of volunteers, which

supports data collection and monitoring on a temporal, spatial, and a data-intensity scale that would

otherwise be too costly (Pocock et al., 2017). Or as put by Beza et al. (2017, p.2): “The accumulated

time dedicated to the crowdsourced research task, the number of contributions, and, in many cases,

the geographic spread of data entries often exceed the capacities of traditional research”. Moreover,

it enables to gather local knowledge and data that are siloed in other places. This is also

acknowledged by Burgess et al. (2017), who remark that citizen science has the ability to provide data

at a broad scale, while also allowing to zoom in on the local level.

This motivation can be aligned with the ambitions discerned for investigation-oriented citizen science

and the virtual citizen science projects.

TO INFORM POLICY

The volume of data intensity, analysis and interpretation enabled by CS, allows the stakeholders to

inform policy makers more swiftly about matters that preoccupy the local environments (bottom-up),

as well as to monitor activities that already have established a policy priority (top-down).


Although informing policy can be seen as a potential side-effect of all different types of CS projects,

this motivation is very plain in the action-oriented and conservation-oriented citizen science projects.

TO EDUCATE

The mere partaking in academic activities educates their participants with regards to the processes that

underlie the scientific method. Moreover, they gather knowledge about the specific research topic at

hand (Pocock et al., 2017).

We can make the connection with the classification of Wiggins & Crowston (2011) and their

education-oriented projects that refers to top-down initiatives, for which education and outreach are

the primary goals.

TO IMPROVE BUY-IN

Central to the concept of CS is the fact that it enables a larger crowd to act as ‘the eyes and ears on the

ground’ (Geoghegan et al., 2016) for instances when mapping the spread of a specific disease. As a

consequence, this activated mass can also become more engaged and accepting towards the policy

decisions with regards to the specific topic.

This can be aligned with conservation-oriented CS projects, when the definition of conservation is not

limited to natural resource management, but also includes any prevention of deterioration.

TO RAISE AWARENESS AND ENGAGE PEOPLE

This motivation is in line with the previous one, as the authors claim that by raising awareness of for

instance environmental issues, the engaged citizen can get a feeling of shared responsibility, concern

and stewardship (Geoghegan et al., 2016; Williams, Hawthorn-Jackson, Orre-Gordon, & O’Sullivan,

2017). It also helps to hand the participants a bulk of knowledge so they can make better informed

decisions or even change their behaviour. CS can even inspire those participants to not only change

their own behaviour, but influence that of people in their environment as well. Here it can be seen as a

prerequisite for activism.

Similar to the previous motivation, there’s a relation with conservation-oriented citizen science

projects. There is also a definite connection possible with action-oriented projects, when the project is

a bottom-up initiative that triggers participant intervention.

TO BUILD PARTNERSHIPS AND IMPROVE COMMUNICATION BETWEEN DIFFERENT

STAKEHOLDERS

In this respect, one must think of relationships, data and information sharing between scientists,

amateur experts, local interest groups, the public etc. Besides promoting cooperation, it also builds

trust between the different communities. For some stakeholders, it can also be seen as a means of

reaching a targeted audience that might otherwise not have heard about the institution’s existence.

TO GAIN PERSONAL SATISFACTION


A final motivation for conducting Citizen Science mentioned by the respondents of Geoghegan et al.

(2016) was that it gave them personal satisfaction. It made them witness a more direct impact on

society with their work, as did it allow them to increase their accountability towards the people that

often times, though indirectly paid for their work, the public.


3.2 BARRIERS INHIBITING THE USE OF CITIZEN SCIENCE

The use of Citizen Science as a methodology to support the purposes listed above, is faced with a

number of barriers barriers and challenges that potentially inhibit the application of CS for different

projects. Geoghegan et al. (2016) have distinguished eight different elements. As with their research

on motivations, they broadened the applicability of the barriers beyond pure scientific stakeholders,

by interviewing others as well. We complemented their findings with the results from our own

literature study.

DATA QUALITY AND BIASES

The most common barrier refers to a mistrust towards the quality of the data gathered by citizen

scientists as well as their biases (Burgess et al., 2017). Poor data quality might be the result of the

equipment that need to be used by the citizen scientists, e.g. through the use of low-cost sensors. It is

also noted that the participants might not always be as motivated to engage successfully in a project.

Moreover, when very domain-specific data is needed, it is hard to outsource the tasks to laypeople. As

formulated by Rotman et al. (2012, p.2): “Scientists, educated, trained and placed within the

hierarchal academic world, are sometimes wary of letting others, who do not have the same

credentials as they do, into their labs and research”.

REQUIREMENT OF SPECIALIST EQUIPMENT / KNOWLEDGE

Some research projects require skills and capabilities (for example to deal with the specific

equipment) that surpass those of the majority of the public. Even with extensive training to enable

contributions by the public, some participants will not feel comfortable enough to partake (Mccrory et

al., 2017). Tasks on the other hand that are too easy or require regular input might result in the less

motivated and committed participants. At the same time, and due to an increase in citizen science

projects, potential participants might feel overburdened.

POLITICS

With politics, Geoghegan et al. (2016) refer to the question of accountability and liability when

decisions are based on data gathered from a Citizen Science project when no professional

interpretation of the results has been made.

UNAWARE OF AUDIENCE

Stakeholders that wanted to initiate CS projects, feel that they are not aware of the motivations and

interests of their target audience. This complicates a good communication and engagement strategy.

RESEARCH DESIGN AND IMPLEMENTATION ISSUES

For non-academic stakeholders, it might prove to be a challenge to create an appropriate research

design for the questions they have in mind. Included in this should be a good definition of their


hypothesis, a proper collection of data, a good engagement strategy for the right participants and a

thorough analyses of the gathered material.

PARTICIPANT INVOLVEMENT AND IDENTIFICATION

An additional barrier is articulated by Rotman et al. (2012), as they acknowledge the difficulty to let

every participant feel like a full and legitimate member of the research project. They understand this

to be particularly challenging when the different contributors to the project stem from distant

communities with different values, goals and criteria for assessing the quality of the gathered data and

information.

TIME CONSUMING AND RESOURCING ISSUES

To assemble and activate a large base of participants is also very resource-consuming, both in time

and money. When talking about academics initiating the project, some of them fear that if the job can

be outsourced for free to the crowd, this can amount to a devaluation of the professional’s job. In the

research of Geoghegan et al. (2016) this was identified by all respondents as a key barrier.

PEER REVIEW / MISTRUST

The perception and fear exists that peer-reviewers who decide on publication of academic research

papers do not trust results from CS, due to the perceived data quality as mentioned above. This

explains the hesitation to conduct CS, as it is believed to slim the chances of being published. This

can only be solved if the field is becomes more generally accepted as a scientific approach in all ranks

of different institutions. To counter this, projects should take into account a sufficient amount of

participant training, quality assurance and metrics for evaluation comparable to those in traditional

science (Burgess et al., 2017).

UNCOMFORTABLE / UNPREPARED TO WORK WITH THE PUBLIC

Some academics might just feel it to be unconfident or distressing to work together with the public on

their projects.



The entire document provides an overview of which factors can play an important role to achieve

volunteer participation. To that end, an overview was provided in this second chapter of what

motivates and discourages stakeholders to opt for a CS approach. The different motivations of the

initiators are relevant for taking along, especially in light of Flamenco, which tries to provide an

adaptable platform. Additionally, it is commendable to find a good fit between the organizer’s

intentions and the ambitions of the participants, which will be discussed in the next chapter.

As mentioned, the biggest motivator is the (potential) benefit CS brings to the table, as in data

collection on a grand scale: time-, location-, and data-wise. The Flamenco platform should thus be

technologically able to support such intense data collection from different projects at the same time.

In relation to this, it is difficult for different initiators to realize when CS projects relevant to their

interests already exist, and it is not necessary to initiate one themselves. There are three reasons why

project initiators are not aware of the availability of applicable data: when it simply does not exist, if

they have not found about it, or if they are misinformed about the data (Burgess et al., 2017).

Flamenco should thus provide mechanisms so data over the projects could be shared with those

interested, if the participants and initiators have agreed upon opening up the data. The same

mechanisms should be in place for the sharing of relevant documentation on the topic between

initiators. When a project’s main motivation for conducting CS is to inform policy from a bottom-up

outset, a natural requirement is that a report with the results should be easy to generate. Projects that

also want to educate need to have part of the platform dedicated to the provision of material that can

educate the participants about the topic at hand. If the initiators want to improve buy-in or raise

awareness and engage people timely feedback seems of uttermost importance. In this way, they get

the feeling that their actions matter and can see in what direction the results are pointing to. A second

requirement that stems from this is that the platform or the organizers should enable social contact

between the participants, and between the participants and organizers. This improves the engagement

and buy-in as they can be part of a community. Another motivation mentioned above is the

opportunity to build bridges between different stakeholders in the field, possibly with the same

concern. It should thus be made feasible to connect with different and relevant stakeholders through

the platform. This can also support the previous requirement to relieve organizers from redundant

effort.

The main obstacles for applying meaningful CS for societal, non-academic stakeholders demonstrate

the need again for support mechanisms for the organizers and volunteers in the different steps of the

research process. Be it for ensuring a high-level (and quality) data collection, the provision of user-

friendly equipment and interfaces, the research design and implementation and finally the

interpretation and dissemination of results. When citizens initiate all the research steps, they should be


able to get in touch with the necessary knowledge, be it through personal contact or the availability of

timely information. Flamenco has to ensure that the process, the data and results, generated through

its platform, are up to standard and accepted.

Other barriers identified are the fear of unsuccessful activation of a large public, how this can be

resource-consuming. Moreover, they wonder how to make every volunteer feel as an important

contributor. This relates to the perceived difficulty of getting to know one’s target audience

motivations and interests. Next to applying the right means for participant recruitment, it is paramount

to understand why participants want to partake and what can trigger a continued participation, they

should be able to communicate this to the organizers, so proper feedback can be provided.

In this respect, we will take a closer look at the audience in the next chapter. This should assist us to

understand what motivates them and what other factors can influence contribution.

4 CHAPTER 3: PARTICIPATION IN A CITIZEN SCIENCE PROJECT: MOTIVATIONS

AND HURDLES

The previous chapter provided us with an overview of the different opportunities and pitfalls that

citizen science brings to the committed stakeholder. The current one moves away from the organizer’s

point of view and scans the participation side. We consecutively touch the subject of participation, a

segmentation of citizen science participants, their motivation for partaking and how this changes over

time. Our aim of this chapter is to gather the existing knowledge about citizen science participants and

come up with recommendations that stakeholders conducting a citizen science project can take into

account.

4.1 PARTICIPATION: BEST PRACTICES

Reed (2008) provides us an elaborate literature review on participation. He defines the concept as

“[…] a process where individuals, groups and organizations choose to take an active role in making

decisions that affect them” (Reed, 2008, p. 2419). Three elements stick out in this definition: the

voluntary character, active engagement and the influence on their own lives. The latter makes us

expect that when there is a direct reciprocal benefit for contribution, participation has a higher chance

of materializing. It is with regards to this that Mccrory et al., (2017) mention how participation and

citizen engagement can be made more attractive, but not forced, as it always has to remain a voluntary

endeavor. Reed (2008) further refines his scope and focuses on stakeholder participation, with which

he concentrates on those who directly or indirectly have a stake in the matter. With regards to citizen

science, he notes the following interesting benefit of participation: “Stakeholder participation, […],

can empower stakeholders through the co-generation of knowledge with researchers and increasing

participants’ capacity to use this knowledge” (Reed, 2008, p. 2421). Here he thus again mentions how

the development of skill and awareness is a preferable outcome of the process.

Central to the same article lies the formulation of the best practice for stakeholder participation, which

is boiled down to the following eight key features (Reed, 2008):

1. Stakeholder participation needs to be underpinned by a philosophy that emphasizes

empowerment, equity, trust and learning.

a. Participants need to be empowered through their involvement by making sure that

they really have an influence on the process and that they have the capabilities to

actually partake. The latter means handing them the necessary skills for a meaningful

engagement.

b. Inequalities and trust issues between groups need to be tackled, in particular between

the participants and organizers. Only in this manner the actions of volunteers can be


sensed as fair and valid by all groups. This also means that there should exist a two-

way learning between the participants, and between participants and organizers.

2. Where relevant, stakeholder participation should be considered as early as possible and

throughout the process

3. Relevant stakeholders need to be analyzed and represented systematically

4. Clear objectives for the participatory process need to be agreed among stakeholders at the

outset

5. Methods should be selected and tailored to the decision-making context, considering the

objectives, type of participants and appropriate level of engagement

6. Highly skilled facilitation is essential

7. Local and scientific knowledge should be integrated

8. Participation needs to be institutionalized

With regard to Flamenco, where stakeholders are not only to be seen as participants but also potential

initiators of campaigns, this implies the following:

• Hand over the necessary guidelines during each stage of the participation to enhance the

contributor’s sense of self-efficacy.

• Participants should not have the feeling of being merely instrumental to the process, their

motivation for partaking should be understood at the start of each project and supported.

• From the start of the project, the objectives and tasks should be clearly communicated.

With respect to understanding the reasons for participation, in the rest of this chapter we will examine

the different factors that encourage contribution.

4.2 FACTORS AFFECTING PARTICIPATION

One of the main challenges when initiating a CS-project is the attraction of a valuable number of

participants. It is for this reason that we zoom in the concept of participation and its contributing

elements in this section. We present the most common models that explain how participation in CS

can be enhanced (both in number of contributors and meaningfulness of the participation).

Geoghegan et al. (2016) refer to previous work of its contributing authors West and Pateman, who

created an overview of the different factors that affect the decision to partake in citizen science

projects (See Figure 2).

The created framework splits the road to a meaningful participation in three main elements: the

decision to participate, initial participation and sustained participation. Four influencing factors are

then distinguished that influence these steps: the project’s organization, the motivation, the awareness

of the CS opportunity and personal circumstances and demographics.

An online survey was organized by imec-SMIT-VUB, in light of the Smart City Meter initiative and

was completed by 742 respondents that can be classified as innovators or early adopters (see diffusion

of innovations by Rogers). Part of this survey related to respondent engagement in CS projects. Of the

742 respondents 10,2% already participated in a CS project, and only one fifth (20%) knew what it

entails. Keeping in mind that the respondents have a high innovative profile, it is clear that CS is not

yet an established concept nor that the participation to it is a common practice. Creating awareness of

the opportunity is thus a first main barrier to cross. In the rest of this chapter, we consecutively look at

the other factors that are important influencing factors for participation.

FIGURE 2: MODEL OF INFLUENCES ON PARTICIPATION IN CITIZEN SCIENCE

(GEOGHEGAN ET AL., 2016, P.36 – FROM WEST & PATEMAN (2015), BASED ON

PENNER


MOTIVATIONS FOR PARTICIPATION IN CITIZEN SCIENCE PROJECTS

In this section, we provide an inclusive understanding of the different types of motivation that cause

attraction or retention of volunteers, who have to contribute their skills, time and effort to the

scientific cause (Nov, Arazy, & Anderson, 2011).

Nov et al., (2011, p.2) distinguish six different types of motivation to partake in the specific context of

citizen science and operationalized them as follows:

1. Collective motives: the importance attributed to the project’s goals

2. Norm-oriented motives: expectations regarding the reactions of important others, such as

friends, families or colleagues

3. Reward motives: benefits such as gaining reputation or making new friends, which was split

up in the following subcategories:

a. Community reputation benefits

b. Social interaction benefits

4. Collective identification: identification with the group, and following its norms

5. Hedonistic/Intrinsic motivation: the enjoyment associated with participation in the project

They then fit these motivations inside the framework of the theory of reasoned action and the

technology acceptance model. The motivations are the antecedents leading to the intent to continue or

increase participation. (See Fig. 3)

The literature review distinguished the two most important and recurring motives that lead to

participation as intrinsic motives, more specifically the enjoyment of participating, and the collective

motives, thus identification with the project’s goals. Nov et al. (2011) acknowledge how these two

motives seem to return and hence seem to be independent from specific participation contexts. The

FIGURE 3: MOTIVATIONS AFFECTING PARTICIPATION

(NOV ET AL. 2011, P.2)


extensive literature review on motivation to contribute to different research studies by Jennett et al.

(2016) align with these findings. Four of the seven most recurring motives they discovered were

interest in the research topic (-intrinsic), contributing to original research and science (-collective),

sharing the same goals and values as the project (-collective identification), enjoying the research task

(-intrinsic). They thus connect with the collective and intrinsic motives.

From these findings, Nov et al. (2011) derive more concrete design recommendations for the

organizers of citizen science projects: First of all, the project’s goal and achievements should be

timely and regularly communicated towards the potential pool of participants. Whereas, the pure

enjoyment of partaking in a project can be attained to by involving some gamification elements for

the participants. In the next chapter of this deliverable we take a look of what that might entail in

more detail.

Secondary motives that also seemed to have an influence, though in a lesser manner, were the

identification with the group of fellow citizen scientists and organizers. These were also identified by

Jennett et al. (2016) as the felt need to help others and feeling part of a team, as well as receiving

recognition of and appropriate feedback on their contribution. Leading Nov et al. (2011) to advocate

for the establishment of a strong community of volunteers who “[…] share beliefs, interact regularly,

possibly using social media outlets, and work collectively towards a common goal” (Nov et al., 2011,

p.5).

Referring to Curtis (2015), Jennett et al. (2016) mention how the chance on interaction heightens

when projects are more complex. Looking at how co-operation is a powerful motivator to sustain

participation, this can be aligned with another interesting suggestion to create a dynamic contribution

environment, with this they refer to enabling the participants to gradually take on more demanding

tasks and take steps up the “ladder of the responsibilities”(Nov et al., 2011). They note how “being

able to contribute to a project in a number of different ways (e.g. as a team leader, or moderator) can

motivate an individual to sustain participation, particularly if this contribution is felt to be of

importance and is valued” (Curtis, 2015; Jennett et al., 2016, p. 4).

Innate to the expansion of research tasks and responsibilities is the promise of education. A little

underexposed in the framework provided by Nov et al. (2011) is learning and understanding as a

motive, though they can be conceived under intrinsic motivation. Alender (2016) considers both

learning about the topic as learning new skills as catalysts for contribution. She further notes how

especially learning from others and sharing this experience are significant predictors of a gratifying

experience, denoting the creation of an atmosphere of social learning.

Related to this is the MLC model of Jennett et al. (2016) which gives an explanation of how

motivations, learning and creativity are related (See Figure 4). They assert that through an active


participation in both the micro-and macro-levels of a project, the participants gain new knowledge

and identify themselves stronger as volunteers. This new-found identity creates a feeling of

contribution and belonging to the project community as well as a greater sense of self-confidence.

This expands even further as time progresses and their desire to share is nourished the more the

community welcomes their input. A minority of high-involved volunteers will eventually reach a

confidence threshold that enables them to share their own creative ideas, e.g. on a forum. If these are

appreciated and taken up by the community the process maintains itself (Jennett et al., 2016, p. 15).

Questions that need to be answered at the onset of every citizen science project to enable learning are

according to Jennett et al. (2016, p. 18):

• How difficult is it to start contributing?

• Is there progression in the difficulty of the tasks?

• Do the volunteers receive feedback about their contribution?

4.2.1.1 CHANGING MOTIVATIONS

Past exploration into the matter of motivation for contributing to citizen science projects has

delineated how the motives change over time. Research by Rotman et al. (2012) determined how

initially participants are persuaded with intrinsic, egoistic motives, indicating knowledge or

experience expansion and/or the fulfilment of an enjoyable activity. After a while, the participants

enter what Rotman et al. (2012) call a period of reflection and reassessment. In this phase, continued

participation is decided upon by revaluation of their satisfaction with the participation up to now

(intrinsic factors), or they continue for altruistic reasons (where the goal is to increase someone else’s

welfare (Land-Zandstra, van Beusekom, Koppeschaar, & van den Broek, 2016)) or out of collective

interest (where the goal is to increase a group’s welfare (Land-Zandstra et al., 2016)) (Rotman et al.,

2012). In a figure, provided by Rotman et al. (2012) this looks like the following (Figure 5):

FIGURE 4: MLC MODEL OF MOTIVATION, LEARNING AND CREATIVITY IN CITIZEN

CYBERSCIENCE (JENNETT ET AL., 2016)


There is value in these observations for the design of a fitting communication strategy. We will

elaborate on this in the next section, where we are looking into organizational factors and how they

contribute to a greater participation.

ORGANIZATIONAL FACTORS & PARTICIPATION IN CITIZEN SCIENCE PROJECTS

Characteristics of the configuration and coordination of the project have an influence on a (sustained)

contribution and on the volunteers’ motivation. The ones primarily mentioned in literature are the

design of the proper communication and feedback mechanisms. This goes from receiving

acknowledgements of the data collection, notification of reception, affirmation that it is being used

and eventually reviewing the results and its (hopefully positive) impact on the initial cause. These

mechanisms refer also to the reception of clear instructions and guidance from professionals and peers

(Geoghegan et al., 2016).

Geoghegan et al. (2016) and Debacq (2016) provide us with the communication aspects that need to

be set in place in the recruitment phase and throughout the rest of the project. Recruitment has to be

initiated making use of the different media outlets, tailored to the desired audience, research context

and culture. For example, one can try to garner press attention, buy advertisements, make use of direct

mailings or use social media to gain traction. If the organizers envision the consultation of lay people,

the language used must remain simple to create a low threshold for participation. Keeping in mind the

previous section on motivations, the message should include the impact the participation of the citizen

scientist can have on the research at hand.

To allow for a sustained participation, there is reason to belief the development of a sense of

community for everyone involved holds certain benefits. Dickinson et al. (2012) note how for short

participation of large crowds “easy, fun and social” are three key words, for longer projects that

FIGURE 5: A PROCESS MODEL OF VOLUNTEERS AND

SCIENTISTS INVOLVEMENT IN CITIZEN SCIENCE

PROJECTS (ROTMAN ET AL., 2012)


require more effort however, a closer interaction is necessary. This can be done by creating specific

new media tools for the project with requirements stemming from the tailored population. These tools

could range from project-specific applications, towards a platform where all stakeholders can bi-

directionally communicate with each other. Continual feedback about the status of the project, the

quality and the impact of the results, personal and direct contact between groups and the provision of

educational material are three main means that need to be cherished in this respect as well. A side-

effect of communication and handing over educational information is that it can contribute to a higher

quality of the data collected. Participation can also endure if the value of the volunteers’ contribution

is emphasized in all different types of communication (Land-Zandstra et al., 2016).

If we turn this around, the barriers for an effective communication as listed by Debacq (2016) and

adapted to for our purpose are the following:

• Project organizers that are unfriendly / intimidating and use specific jargon

• Lack of attention for the initial interests of the citizen scientists for the project

• The different parts and the timing of the scientific process are not clearly communicated, and

thus not understood by the volunteers

• Absence of regular feedback

• Communication means are not adapted to the targeted audience

The organization of citizen science projects need to consider the perceived control of the volunteers

over the factors that enable data sharing. Herewith, Gharesifard & Wehn (2016) mean that not only

the volunteers need the necessary skills, they moreover need to believe that they are able to contribute.

In their conversation with participants, the organizing stakeholders need to elucidate that it is not

necessary to understand the scientific method of data collection or its supporting technologies, nor

that the volunteers should hold a great understanding of the research topic at hand. Besides the

communication aspect, this also means that the tools used for data collection need to be accurate, of

high quality and user-friendly and that the participants have enough time to complete their tasks. We

have learned from Aoki, Woodruff, & Willett (2017) that another essential aspect of the organization

is a clear view on the project’s goals from the start, as this has an effect on which participants to invite,

from which region and with what type of sensors. As an example, they note how massive data

collection campaigns are not always reaching a critical data density. Volunteers are often also

interested to gain knowledge about how the research can affect their own situation, be it their well-

being or that of their neighborhood. If the data collection is to sparse to get valuable results for their

specific situation, this can leave them disillusioned about their contribution. Aoki et al. conclude as

follows: “[…] resources might be better spent developing low-cost, high-accuracy, fixed sensors, and

building dense sensing networks that can meet the needs of both individuals and scientists” (Aoki et

al., 2017, p. 3147).


To epitomize, Geoghegan et al. (2016) include a table with the most important organisational

(logistical) and dispositional (personal motivations) factors that encourage or discourages

participation.

TABLE 4: SUMMARY OF WHAT ENCOURAGES AND DISCOURAGES PARTICIPATION (GEOGHEGAN ET AL.

4.3 A SEGMENTATION OF PARTICIPANTS IN CITIZEN SCIENCE PROJECTS In the previous paragraphs, we zoomed in on the motivations of stakeholders and participants for

engaging in CS, and what barriers might hurdle their enthusiasm. In this section, we will briefly

glance over the participants of CS projects. A better understanding will help to find out what

motivates them to partake.

In the typology of citizen science projects by Wiggins & Crowston (2011), as presented above, it was

clear that education purposes were often an important goal in each of the categories. Making it into a

priority to get ‘lay-people’ involved and knowledgeable about the scientific processes underpinning

research. According to Martin (2017) this way of thinking holds “[…] the implicit assumption that

“non-scientist” volunteers come to citizen science projects with relatively low levels of scientific

knowledge or understanding, or (sometimes) little support or interest in science” (Martin, 2017, p. 3).

In his study, he rightfully notices, as we have seen in the previous section, how one of the primary

motivations of the participants to engage in citizen science projects is a desire to contribute to science,

which already gives the feeling that a segment of participants are science enthusiasts. Martin, (2017)

made a segmentation of his population, based on their prior Engagement in Science (EiS), a parameter

that was measured using the following the questions, answered on a 7-point Likert scale:

• How interested are you in science, generally? (interest)

• In general, how often do you actively seek out scientific information? (seek)

• In general, how much do you trust scientific research? (trust)

• In general, how easy do you find it to understand scientific information? (understand)

He then went on to look if there was a correlation between EiS and willingness to participate in

scientific research and in what role they would see themselves partaking. He eventually could identify

two main groups: working scientists (respondents that already worked in higher education) and non-

scientists. The latter group was eventually divided into 5 subgroups (From low to very high EiS).

From this segmentation, he then drew the following the conclusions:

• The respondents with a very high prior engagement in science, appeared to be the most

interested and the most likely to participate in scientific research. Their education level was

also the highest of the different groups. As seen in the barriers for citizen science, the

perception exists that involving the public can bring forth bad, unreliable data. This new

insight tempers this belief as the people, who are most likely to participate in CS, seem to be

highly educated.

• Besides this group, involving the high and moderate EIS group seems also attainable. The

main barrier towards their inclusion is their self-perceived lack of knowledge. As mentioned

by Martin (2017, p. 19) they feel “less able to understand science and are less confident in

their abilities to assist” than the very high EIS group. She sees a solution in projects that

market themselves as opportunities to increase volunteers’ knowledge.

• Engaging the Low-EIS group is the most challenging as it might be hard to find a fit between

their interest and the objectives of the CS project. Their lower education levels can be a


barrier for inclusion, even more so since CS projects often don’t engage with the audience on

their terms, which “demonstrates a lack of understanding about the disconnection some

groups have with the “elite” world of science”(Martin, 2017, p. 20) .

It seems that in order to install a great CS with the highest potential of inclusion of groups with a

different socio-demographic background, establishing a relationship of trust between participants and

organizers seems to be vital. We can conclude that different engagement strategies are necessary for

ensuring the inclusion of different types of participants. Providing an easy entry point for people with

no scientific background, avoiding the use of scientific slang and contextualize the process in the

everyday life of its contributors. On the other hand, people who want and are able to learn more about

the science in itself should be provided with that opportunity.


Within Flamenco the different needs of the Advisory Committee of Users (ACU) members, interested

in starting a CS project, have been listed. These go from different data needs, the specific constraints

(e.g. time constraints, population constraints), as well as the needs in the deployment phase (e.g.

communication needs), the execution phase (from privacy concerns to mechanisms that counter the

risks of dropout) as well as the specific output possibilities.4 A subpart of those needs calls out for

mechanisms that supports the involvement of participants from the start until the end of a project. In

this section, we will shortly summarize the literature study of this chapter and elucidate how this can

be useful for Flamenco.

The questions that this chapter tried to answer was a. how can we make participation more attractive

and b. how can we keep the volunteers interested over the run of the project?

A couple of elements were distinguished that influence the intent of participation at the volunteers’

side.: their motivations, awareness of the opportunity, organizational factors, personal circumstances

and demographics.

The two main motives for contributing were the mere enjoyment and identification with the projects’

goals. This fits with the main objective of Flamenco. The project wants to provide the necessary

supporting tools and mechanisms to create bottom-up CS-based campaigns. This allows the projects

to be driven by users’ needs and interests, this would thus potentially make it easier to engage

participants.

Moreover, it seemed that organizational factors can have a grand influence on these and on the

eventual road to participation as well. Which is a positive message for the Flamenco project. The

main lessons learned are the following:

• Awareness of the project should be created using as many communication sources as possible,

tailored to the public and its specific context.

o Awareness towards the final project goal: what will the campaign enable, what

problem does it tackle. Less the scientific purpose, but the societal goal is something

that needs to be addressed.

• Contribution should be reciprocal as in, participants who are interested should also feel that

they get something out of it. In this respect, it is paramount to know what motivates the

specific participation at the start of the project, so they can be supported if possible. As a

consequence, participants feel valid contributors and not just instrumental data collectors.


• Contribution to a project that is interesting to the participants is one of the main motivators.

Thus, the objective of the project should be communicated clearly, and regularly towards the

(potential) pool of participants. It should not promise things that are not part of the project as

this will cause disillusionment.

o What is expected of the participants, as well as the timeline of the project should be

disclosed at the start

• (Potential) Participants have different literacy levels, and not all feel that they are capable of

contribution. If the goal is to include as many people as possible, a big proportion should be

convinced that they do have the right skills, or that they can pick them up along the way

o Hand over the necessary guidelines during each stage of the participation to enhance

the contributor’s sense of self-efficacy. This can vary from just providing guidelines

up till a built-in flow mechanism that guides the user automatically throughout a

process.

o Working together and feeling part of a community is well received by contributors.

Without overburdening them, establishment of a strong community of volunteers who

“[…] share beliefs, interact regularly, possibly using social media outlets, and work

collectively towards a common goal” (Nov et al., 2011, p. 5) is important. Especially

so since learning from each other and being able to share the knowledge is a

gratifying experience for the volunteers.

o Keep the language used in communication simple and friendly

o Enough time should be given to the participants to complete their tasks

o Establish a relationship of trust between participants and organisers

• If tasks are too easy and people do not feel challenged, participation will stop eventually. For

this reason, it has been proposed to create a dynamic contribution environment, with this they

refer to enabling the participants to gradually take on more demanding tasks and take steps up

the “ladder of the responsibilities” (Nov et al., 2011). This educational aspect, comes down to

three things: Don’t make it too difficult at the beginning, make the tasks more challenging

over the course of the project and give enough feedback so participants can actually learn.

• Enough communication from organizers towards participants is important to feel approval:

o Sending acknowledgements and give value of the data collections

o Notifications of reception

o Affirmations that they are being used

o Information about the status of the project

o Giving the participants an opportunity to review the results and witness the impact on

the initial cause

• The tools used need to be accurate, of high-quality and user-friendly for the participants.

Proper and clear instructions on how to use them should be provided as well.

o Sensors should be easy to couple with the created platform

• Volunteers are interested to apply the knowledge and data on their own situation or that of

their neighborhood.

o Hand over the proper information for how they can do that, or if it is not possible (e.g.

too low data-density in their neighborhood) explain this from the beginning.

4 A full list is described in Deliverable 6.3


• The pure enjoyment of partaking in a project can be attained to by involving some

gamification elements for the participant, more on that in the next and final chapter.


5 GAMIFICATION AND BEHAVIORAL CHANGE The following chapter is dedicated to the use of a mechanism to playfully steer behavior referred to as

“gamification”. First, an introduction to general behavior change approaches is given. Gamification is

framed as one of several tools which can be used as part of strategies to steer behavior. Then, the

concept is explored in further depth. Important elements of gamification are outlined. Finally, the

application of the concept within the context of citizen science is discussed.

5.1 INTRODUCTION: GAMIFICATION IN THE CONTEXT OF BEHAVIORAL

CHANGE Gamification approaches can be situated within the strand of behavioral change research. Contrary to

traditional economic approaches, behavioral change approaches as common within behavioral

economics recognize that humans are not rational, thus being closer to psychological approaches (see

e.g. Dawnay and Shah 2005). Indeed, behavioral change approaches originate from the social sciences

or psychology and mainly differentiate themselves through their specific focus (Darnton, 2008).

Nevertheless, they have in common to acknowledge that humans are emotional, influenced by their

context and other individuals. As a result, it is possible to observe a huge gap between most people’s

(usually rational) intentions and their actual actions (see also Blake, 1999). Giving an example, while

around new year a lot of good intentions are formulated, e.g. to smoke less or to do more sports with

presumably positive effects on health or money spent, these intentions are often not followed through.

Behavioral change approaches try to overcome this intention-action gap by designing an environment

that pushes the right choices.

As Soman (2015) outlines, there are four main strategies for behavioral change, whose actual form

might vary depending on whether they are used in the context of a market or policy implementation.

A first option to change behavior is to restrict existing choices. In a policy context that would mean a

regulation of supply, while in a business context this measure would translate into product

unavailability. A second strategy to influence behavior is to give individuals incentives. As part of a

policy, these might be subsidies (positive incentive) or the imposition of taxes (negative incentive). In

a market environment, the promotion or discount of products might be preferable options.

Furthermore, it is possible to use the strategy of information or persuasion to elicit behavioral

change. While in a policy context this would entail the disclosure of specific information, this strategy

translates in the free market into advertising. The final and fourth option is nudging. This is probably

the most misunderstood option of behavioral change, since it is often equated with behavior change

per se. Yet, nudging describes both in a policy and market context the influence of human’s choice


architecture, i.e. it entails a change of the context of the decision to implicitly steer behavior in the

desired direction.

As an example, the goal of a behavioral change intervention might be to increase the consumption of

sustainably caught fish by policy makers to prevent the over-fishing of the oceans. In this case,

strategy one would translate in an abolition of non-sustainably caught fish. The second strategy would

entail a subsidization fish which was caught considering matters of sustainability. Strategy three

would require disclosing information about the ocean as an ecosystem, the fish industry, and

sustainable fishing practices to the consumer. Finally, the fourth strategy could entail giving fish

which was caught sustainably a more prominent shelf place in supermarkets.

While in the given example thus all strategies would theoretically be an option, depending on the

context of the research some strategies might be more interesting, and might be therefore worth

exploring in practice, i.e. in the form of experiment to determine the most effective strategy. Giving

the example, suggested by Kamenica (2012): Monetary rewards might not always be suitable, e.g. in

the case of inherently interesting tasks or prosocial behavior, or it might be difficult to provide the

right amount of reward and options. Also, ethical and practical reasons have to be considered, for

example it might not make sense to forbid sugar altogether to change people’s behavior regarding

sugar consumptions since this would lead to consumer boycott.

When a strategy is chosen, additionally several tools can be combined with the strategy to implement

it most effectively. Such tools could include for example the introduction of decision points (i.e.

constant reminders), categorization (i.e. organizing information or income), and, very popularly,

gamification, which entails the use of game techniques to influence behavior.

Hence, gamification is not a behavioral change strategy per se, but a tool that can be used to execute

certain behavioral change strategies, usually in the context of information or (monetary) incentives.


5.2 DEFINITION: GAMIFICATION AND ITS ELEMENTS Ass mentioned gamification is likely the most frequently used tool as part of behavior change

strategies. A popular definition of gamification is provided by Deterding et al. (2011). Accordingly,

gamification entails the “use of elements of games in non-game contexts” and is concerned with

gamefulness, gameful interaction, and gameful design (p. 10). Specifically, the authors (2011, p.13)

refer to gamification as:

1. the use (rather than the extension) of

2. design (rather than game-based technology or other game- related practices)

3. elements (rather than full-fledged games)

4. characteristic for games (rather than play or playfulness)

5. in non-game contexts (regardless of specific usage intentions, contexts, or media of

implementation).

Gamified applications are thus inherently different from whole games (including serious games) or

more playful design, in the sense of toys.

Yet, also gamifications aims to appeal to the hedonist in users. It is assumed that users seek positive

affective reactions such as joy, mirth and joviality, while avoiding unpleasant ones. Gamification

makes it possible that non-hedonic, i.e. utilitarian potentially non-enjoyable tasks, which however

might be socially or environmentally desirable, can be made more enjoyable. In other words, intrinsic

motivation, i.e. the wish to perform activities for the sake of it are promoted for tasks which would

usually require extrinsic motivations, i.e. motivation from outside to achieve certain valued outcomes

(Ryan & Deci, 2000).

Gamification is therefore more complex than “adding points and badges on top of a system” (Thiel,

Reisinger, Röderer, & Fröhlich, 2016, p. 35). It is rather the process of balancing utilitarian and

hedonic benefits (Hamari and Koivisto, 2015).

In this context, design thinking is highly important when creating gamification in order to get the

participants to participate and to keep engaged and playing. Detering et al. (2011) provide a

comprehensive overview about the levels on which such a game design can take place. The authors

differentiate between design patterns, mechanics, general principles, models, and methods. Giving an

example, design patterns such as a smooth player journey, creating balance between challenge and

skills, and the designing of the set-up as a full experience are essential.5 Game-mechanic such as

5 http://gamifyforthewin.com


competition, mastery, scarcity and discovery can then be applied to make the game attractive (e.g.,

Chou, 2013).

Level Description Example

Game interface

design patterns

Common, successful interaction design components and

design solutions for a known problem in a context,

including prototypical implementations

Badge, leaderboard, level

Game design

patterns and

mechanics

Commonly reoccurring parts of the design of a game that

concern gameplay

Time constraint, limited

resources, turns

Game design

principles and

heuristics

Evaluative guidelines to approach a design problem or

analyze a given design solution

Enduring play, clear goals,

variety of game styles

Game models Conceptual models of the components of games or game

experience

MDA; challenge, fantasy,

curiosity; game design

atoms; CEGE

Game design

methods

Game design-specific practices and processes Playtesting, playcentric

design, value conscious

game design

Table 5: Levels of game design according to abstraction (Detering et al., 2011, p. 12)

However, as the authors note, the description of game elements varies significantly in literature.

While there seems to exist a general agreement of basing frameworks on the concept of intrinsic and

extrinsic motivation, depending on the theoretical focus of the framework, the elements listed might

differ (see also Seaborn & Fels, 2015).

Aparicio, Vela, Sanchez, & Montes (2012) chose Self-Determination Theory (SDT) as introduced by

Ryan & Deci ( 2000) in their framework to refer to elements relevant for gamification. According to

SDT, “human beings can be proactive and engaged or, alternatively, passive and alienated, largely

as a function of the social conditions in which they develop and function” (p. 68). Three factors have

been hereby identified which can foster intrinsic motivation, self-regulation, and well-being:

competence, autonomy, and relatedness. Aparcio et al. (2012) connect these three factors in the

context of gamification with a list of game elements. Accordingly, autonomy is realized through

profiles, avatars, macros, configurable interface, alternative activities, privacy control, and

notification control. The need to feel competent can be addressed via positive feedback, optimal

challenge, progressive information, intuitive controls, points, levels, leaderboards. Finally, relatedness

can be created by considering the creation of groups, messages, blogs, connection to social networks

and the provision of chats.


Blohm and Leimeister (2013) start from the perspective of game design and aim to create so called

“gamified service bundles”. Starting from a specific usage objective, these are translated into game

design elements which are then compiled in the gamified services, which should active certain user

motivates and thus stimulate consumption. The motives include hereby not only self-determination,

but also intellectual curiosity, achievement, social recognition, social exchange and cognitive

stimulation. The authors list hereby two kinds of game design element: game mechanics and game

dynamics. The mechanics are the actual building blocks for the design of the game, while the

dynamics describe their effect (see Table 6

Game-design elements Motives

Mechanics Dynamics

Documentation of behavior Exploration Intellectual curiosity

Scoring systems, badges, trophies Collection Achievement

Rankings Competition Social recognition

Ranks, levels, reputation points Acquisition of status

Group tasks Collaboration Social exchange

Time pressure, tasks, quests Challenge Cognitive stimulation

Avatars, virtual worlds, virtual trade Development/organization Self-determination

TABLE 6: BLOHM AND LEIMEISTER (2013)

Finally, Nicholson (2012) proposes a user-centric framework for “meaningful gamification”. He bases

his framework on a variety of theories, namely Organismic Integration theory (a sub-theory of Deci &

Ryan’s (2000) SDT), Situational Relevance (e.g. Wilson, 1973) Situated Motivational Affordance,

(Deterding, 2011), Universal Design for Learning (Rose and Meyer, 2002), and User-centered Design

(Norman, 1988). Organismic Integration theory suggest that meaningful game elements are

intrinsically motivating, situational relevance describe the users decision about meaningfulness,

situated motivation affordance the need to match user profile and context with the game, universal

design helps to design the most beneficial experience for different user, while user-centric design

brings them all together by putting the user in the center (Seaborn & Fels, 2015). The author

emphasizes the need to put “the needs and goals of the users over the needs of the organization” in

order to create a long-term benefit emphasizing the need to create a personalized gaming experience

for the individual user based on their life instead of external rewards (Nicholson, 2012).

In the end, as Seaborn & Fels (2015) emphasize, most gamification-specific frameworks were

developed in isolation and “there is, as yet, no evidence of their completeness; these frameworks

need to be applied in order to determine their applicability and convergence.”


Hence the overview of Thiel et al. (2016) proves to be especially useful. The authors provide an

comprehensive overview about the most commonly used elements in gamification approaches. The

authors list as common elements for gamification achievement, points, status, expression, feedback,

personalization, challenge, competition, and time constraint (see Table7).

Classifier Elements Description

Achievement e.g., badges A mechanism to show the user his or her

progress and achievements within the system

Points Users can earn virtual points that in some

cases can be used to redeem physical

artefacts.

Status e.g., levels In contrast to points in leaderboards, the

underlying mechanic that aims to motivate is

the strive for recognition by others and

findings one’s place in a community.

Expression e.g., spaces for open creativity

Feedback e.g., notifications Where used, these spaces for open

creativity/creation are usually the main

component of the system.

Personalization e.g., profiles, avatars The system provides the user with additional

information, hints or gives encouraging

statements.

Challenge e.g., missions, quests The system offers a space that contains

information about the specific user or can be

modified by the user.

Competition e.g., leaderboards, highscore lists The system or other users ask the user to

perform a certain activity under predefined

conditions.

Time constraint e.g., due dates, countdowns Competition does not necessarily connect to

rivalry, but can also be neutral comparison.

TABLE 7: OVERVIEW OF GAME ELEMENTS (THIEL ET AL., 2016)

In this context Seaborn & Fels (2015) provide a list of distinct definition of elements that are often

used, namely points, badges, leaderboards, progression, status, level, rewards, and roles.

• Points: Numerical units indicating progress e.g. score.

• Badges: Visual icons signifying achievement e.g. trophies

• Leaderboards: Display of ranks in comparison e.g. rankings

• Progression: Milestones indicating progress e.g. levelling

• Status: Textual monikers indicating progress e.g. title


• Levels: Increasingly difficult environments e.g. stage

• Rewards: Tangible, desirable items e.g. prizes

• Roles: Role-playing elements of a character e.g. class

Summing up, there are several approaches towards game elements within gamification approaches,

depending on the underlying framework used by the prevailing authors. Thiel et al (2016) provide a

good summary of the important elements independent of the theoretical foundation chosen, while

Seaborn & Fels (2015) provide a list of distinct terminology for gamification approaches.


5.3 APPLICATION: THE POTENTIAL OF GAMIFICATION FOR CITIZEN SCIENCE As Thiel et al. (2016) emphasize, gamification can contribute to participation since it can provide a

means for engaging and educating individuals. Indeed, as Geoghegan, et al (2016) outline, with the

increasing use of online platforms for citizen science platforms, gamification is being used by an

increasing amount of projects to stimulate participation.

Gamification is popular in the context of topics as diverse as education (e.g. Foster et al., 2012),

health (e.g. Rose et al., 2013) , sustainability (e.g. Liu et al., 2011), research (e.g. Rap & Marcengo

2012), crowdsourcing ( e.g. Mason et al., 2012) or marketing (e.g. Downes-Le Guin et al, 2012). The

most common cause is hereby data collection, which would be usually resourceful, difficult and/or

highly cost intensive. Giving an example, the Geograph.org.uk engages citizens to map remote areas

in the United Kingdom. Participants competed via the numbers of uploaded images. Likewise, the

Citizens’ Network for the Observation of Marine BiodivERsity (COMBER) project in Greece reached

out to local diving clubs to monitor biodiversity in a marine environment (Hakley, n.d.).

Gamification literature in general has some weakness, for example in terms of coherence of

definitions, the use of theoretical foundations, lacking theoretical or empirical investigation, and

missing long-term studies (Seaborn & Fels, 2015). However, as Murphy (2015) outlines, the

application for gamification for citizen science especially can still be regarded as needing significant

research effort.

Prestopnik and Crowston (2012a) provide an important contribution to this strand of research by

differentiating generally between “task gamification” and “game taskification”. The first concept

refers to adding game elements to a needed task to foster the completion by making it more enjoyable.

As an example, the authors describe the tool Happy Moths which was created by scientists for

taxonomy purposes . Users have to identify moths according to pictures, one of them being “the

happy moth” which is used to evaluate their performance. However, users also achieve scores for

their long-term involvement, leader boards, and high scores to promote competition. This also help to

evaluate the quality of the users’ work (see also Prestopnik &Crowston 2012b).

On the contrary, game taskification refers to building a game with tasks elements. The authors use the

example of Forgotten Island. The game features a whole story-driven gaming world, namely an island

that the user has to build and explore. The classification of insects is hereby only a means to collect

game money for equipment and game items for the game.

As Prestopnik and Crowston (2012a, 2012b) outline, task gamification is easier to realize in the

context of citizen science since it requires fewer resources and shorter development time, yet, game

taskification has immense potential for user involvement (see also Prestopnik and Crowston 2012a).


Crowston & Prestopnik (2013) showed that in both cases the competitive features of the game and the

knowledge that the game was helping scientists highly motivated them to participate, even above the

required benchmark. This finding is also echoed in the work of Cooper et al. (2010) and Curtis (2015),

who show that indeed sometimes participants’ motivation stems from the competitive features, while

for others the contribution to science is on the forefront. Also West et al. (2015) confirmed these

altruistic motives. As Bowser et al. (2014) showed, this might be indeed related to the age of the

participants. While Millenials were attracted in their study to game elements such as earning badges

and peer competition motivating, and elaborations offered in response to open-ended survey questions,

older users wanted to contribute to science and the public good.

However, as Massung et al. (2013) have shown gamification might also demotivate participants to

participate if participants feel they are not successful in competing. As Bowser et al. (2013) add, also

the target group for the citizen project can be decisive. The authors revealed that participants who

identified as amateur scientists did not embraced highly game-like interfaces. Hence, whether

gamification is successful depends highly on the participants’ personal profile.

Once these issues are solved, Eveleigh, Jennett, Lynn, and Cox (2013) propose a range of guidelines

how to successfully valorize gamification for citizen science.

(1) Scoring mechanisms: Scoring mechanism should be detailed enough to show continuous

personal progression next to being a means of comparison with competitors.

(2) Personalized feedback: Feedback should help participants to assess the quality of their

contribution, rather than its quantity, to ensure that a learning process is taking place.

(3) Game elements: Not all participants will like the same features at all times, hence it is

important to stimulate interest by providing, e.g. team challenges on a regular basis.

(4) Narrative: Participants should be able to follow a personalized approach in the project. This

approach might be, e.g. location, time, or interest based and can be stimulate by personalized

narratives.

The main take away for the Flamenco consortium is that although gamification is a useful instrument,

it cannot be simply implemented in every context and expected to be effective. The mechanics that

are used need to be tweaked and tailored towards the project goal, the target audience and their

demographics and self-understanding, as well as the specifics of the activity that needs to be

performed. Within Flamenco, it should further be investigated which elements (e.g. scoring and

feedback mechanisms) can support citizen science projects and should thus be provided as an option

to project initiators to include in their campaigns.


6 CONCLUSION

This deliverable provides some insights on the key mechanisms that motivate citizens to partake in

participatory sensing campaigns. We consecutively looked at the concepts of citizen science and

participatory sensing, the reasons and barriers for choosing a CS approach, the factors that influence

participation and gamification elements that can be used as a strategy to steer behavior and

contribution. At the end of most of the chapters, we already provided an overview of what can be

learned for Flamenco. In Table 8, we summarize the different insights

N° Requirement

Initiators

1 The platform should provide a way to determine who initiates the CS project and what their

corresponding skill level is.

2 Depending on the skill level of the initiator, guidelines need to be provided on how to conduct

a CS project, the steps that need to be included and how they should go about each step. This

might require the creation of a semi-automatic workflow composition supporting the initiator

when needed.

3 Flamenco should clearly communicate its purpose, benefit and value to potential societal

stakeholders.

Participants

4 The platform should provide a way to determine who participates in the CS project, and what

their corresponding skill level is.

5 Depending on the skill-level and influence of the participant, guidelines should be provided

on how to help out in the citizen science campaign. Instructions could go from how to do a

proper data collection to how to correctly interpret the results.

6 Contribution should be reciprocal as in, participants who are interested should also feel that

they get something out of it. In this respect, it is paramount to know what motivates the

specific participation at the start of the project, so these can be supported if possible. As a

consequence, participants feel valid contributors and not just instrumental data collectors.

7 Create a dynamic contribution environment, with this they refer to enabling the participants to

gradually take on more demanding tasks and take steps up the “ladder of the responsibilities”.


Don’t make it too difficult at the beginning, make the tasks more challenging over the course

of the project and give enough feedback so participants can actually learn.

8 Give the participant information on how the project and its results impacts him/herself and his

environment.

9 The project could provide a matchmaking possibility for support between those participants

who appear to have had a scientific training and those who are new to the process.

Communication and recruitment

10 Awareness of the project should be created using as many communication sources as

possible, tailored to the public and its specific context.

11 The objective of the project, and especially the underlying societal goal, should be

communicated clearly, and regularly towards the (potential) pool of participants. It should not

promise things that are not part of the project as this will cause disillusionment.

12 (Potential) Participants have different literacy levels, and not all feel that they are capable of

contribution. If the goal is to include as many people as possible, a big proportion should be

convinced that they do have the right skills, or that they can pick them up along the way.

13 Working together and feeling part of a community is well received by contributors. Without

overburdening them, establishment of a strong community of volunteers, e.g. through the use

of social media or an in-platform communication environment is mandatory.

14 Keep the language used simple and friendly.

15 Send acknowledgements and give value to the data collection, notifications of reception,

affirmations that their contribution is being used, information about the status of the project.

Give the participants an opportunity to review the results and witness the impact on the initial

cause.

Technical factors

16 The Flamenco platform should thus be technologically able to support such intense data

collection from different projects at the same time.

17 Flamenco should thus provide mechanisms so data over the projects could be shared with

those interested, if the participants and initiators have agreed upon opening up the data.


18 Different stakeholders initiating different CS projects should be easily able to contact each

other.

19 Flamenco should provide mechanisms for the sharing of relevant documentation on the topic

between initiators

20 The tools used need to be accurate, of high-quality and user-friendly for the participants.

Proper and clear instructions on how to use them should be provided as well, enhancing the

trustworthiness of the overall project.

21 Sensors and other devices used by the participants should be easy to couple with the platform.

Gamification

22 The application of gamification elements can contribute to a higher and sustainable

participation in CS projects. But be aware that its applicability depends on demographics and

self-understanding of the group. It should thus be personalized and consider participants

wishes and needs.

23 Be aware that gamification can also demotivate participants if they are not successful in

competing.

24 Scoring mechanisms: Scoring mechanism should be detailed enough to show continuous

personal progression next to being a means of comparison with competitors.

25 Personalized feedback: Feedback should help participants to assess the quality of their

contribution, rather than its quantity, to ensure that a learning process is taking place.

26 Game elements: Not all participants will like the same features at all times, hence it is

important to stimulate interest by providing, e.g. team challenges on a regular basis.

27 Narrative: Participants should be able to follow a personalized approach in the project. This

approach might be, e.g. location, time, or interest based and can be stimulate by personalized

narratives.

General / Other

28 Enough time should be given to the participants to complete their tasks.

29 Report with results of a CS project should be easily generated.


30 Part of the platform should give place to documentation about the topic at hand, so

participants can learn more if wanted.

31 Aim for a relationship of trust between participants and organizers.

As a follow-up we will examine how the distinguished concepts will be applied in a framework for

analyses of these and other CS case studies. The latter will be comprehensively presented in the next

iteration of this deliverable (D2.1b: Integrated report on promoting citizen participation).


7 BIBLIOGRAPHY

Alender, B. (2016). Understanding volunteer motivations to participate in citizen science projects: A

Deeper look at water quality monitoring. Journal of Science Communication, 15(3), 1–19.

Aoki, P., Woodruff, A., & Willett, W. (2017). Environmental Protection and Agency : Motivations ,

Capacity , and Goals in Participatory Sensing, 3138–3150.

Aparicio, A. F., Vela, F. L. G., Sánchez, J. L. G., & Montes, J. L. I. (2012, October). Analysis and

application of gamification. In Proceedings of the 13th International Conference on Interacción

Persona-Ordenador (p. 17). ACM.

Bennett, W. L. (2008). Changing Citizenship in the Digital Age. In Civic Life Online: Learning how

Digit (pp. 1–24). http://doi.org/10.1162/dmal.9780262524827.001

Beza, E., Steinke, J., van Etten, J., Reidsma, P., Fadda, C., Mittra, S., … Kooistra, L. (2017). What

are the prospects for citizen science in agriculture? Evidence from three continents on motivation and

mobile telephone use of resource-poor farmers. Plos One, 12(5), e0175700.

http://doi.org/10.1371/journal.pone.0175700

Blake, J. (1999). Overcoming the 'value-action gap' in environmental policy: Tensions between

national policy and local experience. Local Environment: The International Journal of Justice and

Sustainability, 4 (3): 257-278.

Blohm, I., & Leimeister, J. M. (2013). Gamification: Design of IT-based enhancing services for

motivational support and behavioral change. Business & Information Systems Engineering

(BISE), 5(4), 275-278.

Bonney, R., Ballard, H., Jordan, R., McCallie, E., Phillips, T., Shirk, J., & Wilderman, C. C. (2009).

Public Participation in Scientific Research: Defining the Field and Assessing Its Potential for

Informal Science Education. CAISE Public Participation in Scientific Research Inquiry Group

Participants. Washington. Retrieved from http://www.ukeof.org.uk/resources/citizen-science-

resources/MotivationsforCSREPORTFINALMay2016.pdf

Bowser, A., Hansen, D., Preece, J., He, Y., Boston, C., & Hammock, J. (2014). Gamifying citizen

science. Proceedings of the Companion Publication of the 17th ACM Conference on Computer

Supported Cooperative Work & Social Computing - CSCW Companion ’14, (February), 137–140.


Burgess, H. K., DeBey, L. B., Froehlich, H. E., Schmidt, N., Theobald, E. J., Ettinger, A. K., …

Parrish, J. K. (2017). The science of citizen science: Exploring barriers to use as a primary research

tool. Biological Conservation, 208, 113–120. http://doi.org/10.1016/j.biocon.2016.05.014

Chatzigiannakis, I., Mylonas, G., & Vitaletti, A. (2011). Urban pervasive applications : Challenges ,

scenarios and. Computer Science Review, 5(1), 103–118. http://doi.org/10.1016/j.cosrev.2010.09.003

Chou, Y: (2013). Octalysis: Design Breakdown of the Speed Camera Lottery. Gamification.co [Web

blog post]. Retrieved from http://www.gamification.co/2013/04/25/gamification-breakdown-of-the-

speed-camera-lottery/.

Conrad, C. C., & Hilchey, K. G. (2011). A review of citizen science and community-based

environmental monitoring: Issues and opportunities. Environmental Monitoring and Assessment,

176(1–4), 273–291. http://doi.org/10.1007/s10661-010-1582-5

Cooper, C. B., and J. A. Smith. 2010. Gender Patterns in Bird-related Recreation in the USA and UK.

Ecology and Society 15:13.

Crowston, K., & Prestopnik, N. R. (2013). Motivation and data quality in a citizen science game: A

design science evaluation. In System Sciences (HICSS), 2013 46th Hawaii International Conference

on (pp. 450–459). IEEE. Retrieved from

http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6479888

Curtis, V. (2015). Online citizen science projects : an exploration of motivation , contribution and

participation. The Open University.

Darnton, A. (2008). Practical Guide: An overview of behaviour change models and their uses. GSR

Behaviour Change Knowledge Review. London. Government Social Research Unit.

Dawnay, E. & Shah, H. (2005). Behavioural economics: seven principles for policy makers.

London: New Economics Foundation.

Debacq, C. (2016). Motivatie en Communicatie bij Burgerwetenschapsprojecten omtrent

Luchtkwaliteit - Obstakels en Opportuniteiten. Vrije Universiteit Brussel.

Deterding, S. (2011, May). Situated motivational affordances of game elements: A conceptual model.

In Gamification: Using game design elements in non-gaming contexts, a workshop at CHI.

Deterding S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness:

defining "gamification". In Proceedings of the 15th International Academic MindTrek Conference:

Envisioning Future Media Environments (MindTrek '11). ACM, New York, NY, USA.


Dickel, S., & Franzen, M. (2016). The “Problem of Extension” revisited: new modes of digital

participation in science. Journal of Science Communication, 15(1).

Dickinson, J. L., Shirk, J., Bonter, D., Bonney, R., Crain, R. L., Martin, J., … Purcell, K. (2012). The

current state of citizen science as a tool for ecological research and public engagement In a nutshell :

http://doi.org/10.1890/110236

Downes-Le Guin, T., Baker, R., Mechling, J., Ruyle, E., 2012. Myths and realities of respondent

engagement in online surveys. Int. J. Mark. Res. 54, 613–633.

Eveleigh, A., Jennett, C., Lynn, S., & Cox, A. L. (2013, October). “I want to be a Captain! I want to

be a Captain!”: Gamification in the Old Weather Citizen Science Project. In Proceedings of the first

international conference on gameful design, research, and applications (pp. 79-82). ACM.

Foster, J.A., Sheridan, P.K., Irish, R., Frost, G.S., 2012. Gamification as a strategy for promoting

deeper investigation in a reverse engineering activity. In: Proceed- ings of the 2012 American Society

for Engineering Education Conference, pp. AC 2012–AC 5456.

Geoghegan, H., Dyke, A., Pateman, R., West, S., & Everett, G. (2016). Understanding Motivations

for Citizen Science. Final Report on behalf of the UK Environmental Observation Framework

(UKEOF), (May), 124. Retrieved from http://www.ukeof.org.uk/resources/citizen-science-

resources/MotivationsforCSREPORTFINALMay2016.pdf

Gharesifard, M., & Wehn, U. (2016). To share or not to share: Drivers and barriers for sharing data

via online amateur weather networks. Journal of Hydrology, 535, 181–190.

http://doi.org/10.1016/j.jhydrol.2016.01.036

Gharesifard, M., Wehn, U., & Van Der Zaag, P. (2017). Towards benchmarking citizen observatories:

Features and functioning of online amateur weather networks. Journal of Environmental Management,

1–13. http://doi.org/10.1016/j.jenvman.2017.02.003

Haklay, M. (2013). Citizen Science and Volunteered Geographic Information: Overview and

Typology of Participation. In D. Sui, S. Elwood, & M. Goodchild (Eds.), Crowdsourcing Geographic

Knowledge (pp. 105–122). Dordrecht: Springer Netherlands. http://doi.org/10.1007/978-94-007-4587-

2

Haklay, M. (n.d.). Citizen Science and Policy : A European Perspective, 4.

Irwin, A. (1995). Citizen Science: A Study of People, Expertise and Sustainable Development.

Routledge.


Hamari, J., & Koivisto, J. (2015). Why do people use gamification services?. International Journal of

Information Management, 35(4), 419-431.

Jennett, C., Kloetzer, L., Schneider, D., Iacovides, I., Cox, A. L., Gold, M., … Talsi, Y. (2016).

Motivations, learning and creativity in online citizen science. Journal of Science Communication,

15(3), 1–23.

Kamenica, E. (2012). Behavioral Economics and Psychology of Incentives. Annual Review of

Economics, 4, 427-452.

Land-Zandstra, A. M., van Beusekom, M. M., Koppeschaar, C. E., & van den Broek, J. M. (2016).

Motivation and learning impact of Dutch flu-trackers. Journal of Science Communication, 15(1), 1–

26.

Lane, N. D., Eisenman, S. B., Musolesi, M., Campbell, A. T., & Miluzzo, E. (2008). Urban Sensing

Systems : Opportunistic or Participatory? In Ninth Workshop on Mobile Computing Systems and

Applications (HotMobile 2008) (pp. 11–16). Napa Valley.

Liu, Y., Alexandrova, T., Nakajima, T., 2011. Gamifying intelligent environments. In: Proceedings of

the 2011 International ACM Workshop on Ubiquitous Meta User Interfaces. Presented at Ubi-MUI'11.

ACM, pp. 7–12.

Liu, H., Kobernus, M., Broday, D., & Bartonova, A. (2014). A conceptual approach to a citizens ’

observatory – supporting community-based environmental governance. Environmental Healh,

13(107), 1–13.

Mason, A.D., Michalakidis, G., Krause, P.J., 2012. Tiger nation: empowering citizen scientists. In:

Proceedings of the 2012 6th IEEE International Conference on Digital Ecosystems Technologies.

Presented at DEST 2012. IEEE, Campione d’Italia, pp. 1–5. 10.1109/DEST.2012.6227943.

Massung, E., Coyle, D., Cater, K., Jay, M., Preist, C., 2013. Using crowdsourcing to support pro-

environmental community activism. In: Proceedings of the 2013 ACM SIGCHI Conference on

Human Factors in Computing Systems. Presented at CHI'13. ACM, Paris, France, pp. 371–380.

Martin, V. Y. (2017). Citizen Science as a Means for Increasing Public Engagement in Science.

Science Communication, 107554701769616. http://doi.org/10.1177/1075547017696165

Murphy, R. J. (2015). Facilitating Citizen Science Through Gamification(Doctoral dissertation, PhD

thesis, Memorial University of Newfoundland).

Mccrory, G., Veeckman, C., Coppens, P., & Claeys, L. (2017). D6.1 : Engagement strategy for

hackAIR community involvement. Brussel.


Nicholson, S. (2012). A user-centered theoretical framework for meaningful gamification. Games+

Learning+ Society, 8(1), 223-230.

Nov, O., Arazy, O., & Anderson, D. (2011). Dusting for science: motivation and participation of

digital citizen science volunteers. Proceedings of the 2011 iConference: Inspiration, Integrity and

Intrepidity, iConference 2011, 68–74.

Norman, D. (1988). The psychology of everyday things. New York: Doubleday.

Pocock, M. J. O., Tweddle, J. C., Savage, J., Robinson, L. D., & Roy, H. E. (2017). The diversity and

evolution of ecological and environmental citizen science. PLoS ONE, 12(4), 1–17.

http://doi.org/10.1371/journal.pone.0172579

Prestopnik, N., & Crowston, K. (2012a). Exploring collective intelligence games with design science:

A citizen science design case. In ACM Group Conference. Retrieved from

http://crowston.syr.edu/system/files/designing%20citizen%20science%20games.pdf

Prestopnik, N., & Crowston, K. (2012b). Purposeful gaming & socio-computational systems: a citizen

science design case. In Proceedings of the 17th ACM international conference on Supporting group

work (pp. 75–84). ACM. Retrieved from http://dl.acm.org/citation.cfm?id=2389188

Rapp, A., Marcengo, A., Console, L., Simeoni, R., 2012. Playing in the wild: enhancing user

engagement in field evaluation methods. In: Proceedings of the 16th International Academic

MindTrek Conference. Presented at MindTrek 2012. ACM, Tampere, Finland, pp. 227–228.

Reed, M. S. (2008). Stakeholder participation for environmental management : A literature review.

Biological Conservation, 141(10), 2417–2431. http://doi.org/10.1016/j.biocon.2008.07.014

Riesch, H., & Potter, C. (2014). Citizen science as seen by scientists: Methodological, epistemological

and ethical dimensions. Public Understanding of Science, 23(1), 107–120.

http://doi.org/10.1177/0963662513497324

Rotman, D., Preece, J., Hammock, J., Procita, K., Hansen, D., Parr, C., … Jacobs, D. (2012).

Dynamic changes in motivation in collaborative citizen-science projects. Proceedings of the ACM

2012 Conference on Computer Supported Cooperative Work - CSCW ’12, 217.

http://doi.org/10.1145/2145204.2145238

Rose, K.J., Koenig, M., Wiesbauer, F., 2013. Evaluating success for behavioral change in diabetes via

mHealth and gamification: MySugr's keys to retention and patient engagement. Diabetes Technol.

Ther. 15, A114.


Rose, D. H., & Meyer, A. (2002). Teaching every student in the digital age: Universal design for

learning. Association for Supervision and Curriculum Development, 1703 N. Beauregard St.,

Alexandria, VA.

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic

motivation, social development, and well-being. American psychologist, 55(1), 68.

See, L., Mooney, P., Foody, G., Bastin, L., Comber, A., Estima, J., … Rutzinger, M. (2016).

Crowdsourcing, Citizen Science or Volnteered Geographic Information? The Current State of

Crowdsourced Geographic Information. International Journal of Geo-Information, 5(5).

http://doi.org/10.3390/ijgi5050055

Soman, D. (2015). The last mile: Creating social and economic value from behavioural insights.

Toronto: Rotman-utp Publishing.

Seaborn, K., & Fels, D. I. (2015). Gamification in theory and action: A survey. International Journal

of Human-Computer Studies, 74, 14-31.

Thiel, S. K., Reisinger, M., Röderer, K., & Fröhlich, P. (2016). Playing (with) Democracy: A Review

of Gamified Participation Approaches. JeDEM-eJournal of eDemocracy and Open Government, 8(3),

32-60.

UWE. (2013). Science for Environment Policy In- depth Report: Environmental Citizen Science.

Retrieved from http://ec.europa.eu/science-environment-policy%0AAcknowledgements

West, S. E. 2015. Understanding participant and practitioner outcomes of environmental education.

Environmental Education Research 21:45-60.

Wiggins, A., & Crowston, K. (2011). From conservation to crowdsourcing: A typology of citizen

science. Proceedings of the Annual Hawaii International Conference on System Sciences.

http://doi.org/10.1109/HICSS.2011.207

Wilson, P. (1973). Situational relevance. Information storage and retrieval, 9(8), 457-471.

Williams, C. R., Hawthorn-Jackson, D., Orre-Gordon, S., & O’Sullivan, S. (2017). Some cautions in

the use of citizen science: a case study of urban insect collection. Transactions of the Royal Society of

South Australia, 0(0), 1–13. http://doi.org/10.1080/03721426.2016.1268564

INTERIM REPORT ON PROMOTING CITIZEN ......INTERIM REPORT ON PROMOTING CITIZEN PARTICIPATION IN CAMPAIGNS FLAMENCO Project Deliverable 2.1a VUB (SMIT) Responsible SMIT (VUB) Authors

Documents