Playful crowdsourcing of archival metadata through social networks

Playful crowdsourcing of archival metadata through social networks

Dimitris Paraschakis and Marie Gustafsson FribergerDepartment of Computer Science, Malmo University, Sweden

[email protected], [email protected]

Abstract

This paper explores the integration of social networks withcrowdsourcing games for generating archival metadata. Westudied crowdsourcing, gamification and social dynamicsfrom the perspective of cultural heritage and combined theirfeatures in a metadata game prototype on the Facebookplatform. We tested our prototype and evaluated its re-sults by analysing participation, contribution and playerfeedback. The two-week testing phase showed promising re-sults in terms of user engagement and produced metadata.We conclude that deploying metadata games on social net-working platforms is a feasible method for digital archivesto harness human intelligence from large shared spaces.

1 Introduction

Modern practices of archiving cultural heritage embracetechnology for preserving archival content and making it ac-cessible to the general public in a digital format. The avail-ability of open data makes it possible to interact with digitalarchives by means of software applications. This approach,known as “crowdsourcing”, allows citizens not only to con-sume content but also to carry out useful tasks that requirehuman intelligence, such as providing descriptive metadatafor archival objects. Offering these activities as games hasbeen a popular practice to motivate and engage citizens ina playful and fruitful interaction with their historical pastthrough digital collections. Games on social networks havegrown into an industry that attracts more players than anyother class of online games. Facebook demonstrates thephenomenon when a social networking website is becominga meeting place to play games, with more than 1 million ofmonthly active players. However, there seems to be little orno presence of social network games for cultural heritage.

We aim to fill this gap by investigating possible ways ofdeploying “metadata games” for digital archives on socialnetworking platforms. Through related works we identifiedthe requirements and design guidelines to enable such inte-gration, and applied these functionalities in a game proto-type, Art Collector. In this game, players compete for “artpieces” from the game’s photo collection. The gameplayis centered around two main activities: annotating imageswith keywords (“tags”) and guessing tags of other players.As a side effect of gameplay, user-generated metadata aregathered. In addition, validation of metadata takes place

when two or more players agree on a tag for the same im-age. To evaluate the prototype, we used various metrics ofparticipation and contribution, as well as a survey to obtainplayer feedback.

We begin by providing background on crowdsourcing,gamification, social networks and how these are used incultural heritage (Section 2), followed by a description ofthe methodology used in development and evaluation (Sec-tion 3). We then describe the Art Collector game prototype(Section 4) and its evaluation results (Section 5). Finally,conclusions and future work are presented in Section 6.

2 Background

This section provides an overview of the three main conceptson which this work rests: crowdsourcing, gamification, andonline social networks. Both cultural heritage and generalaspects are included.

2.1 Crowdsourcing

The term crowdsourcing appeared in 2006 and is definedas “the process by which the power of the many can beleveraged to accomplish feats that were once the provinceof a specialized few”[1].

In crowdsourcing projects, the larger task is usually bro-ken up into smaller tasks (categorizing an image, tran-scribing a line). For example, in Galaxy Zoo1 participantslook at images of galaxies and classify them according totheir shape. In Trove2, the participants can correct textOCR:ed from old Australian newspapers (the initiative con-tains other types of media and other types of crowdsourcingtasks as well).

Cultural heritage institutions often lack the resources tohandle the amount of material in need of digitization anddigitized material in need of cataloguing.

It is not hard to see how the use of crowdsourcing hasbenefits for the institutions charged with taking care of andmaking available cultural heritage material. However, thepublic can also benefit from such initiatives. While theperspective of the intitutions, where crowdsourcing enableshelp with manual tasks such as tagging and transcription,are most often voiced, the benefits for the public may pro-vide an even stronger argument. Here crowdsourcing gives

1http://www.galaxyzoo.org/2http://trove.nla.gov.au/

an “opportunity to provide meaningful ways for individualsto engage with and contribute to public memory” [2].

Source material for the cultual heritage domain includematerial culture, natural history and historic documents [3].Activities can be correction, transcription, contextualiza-tion, completing collections, classification, co-curation andcrowdfunding [4]. Trove, mentioned above, is an exampleof crowdsourcing in the cultural heritage domain. Anotherexample of a crowdsourcing project in the cultural heritagedomain is the tagging of works of art in the steve.museumproject [5], which includes art museums across the UnitedStates. The analysis of 36,981 user-generated terms con-tributed to steve.museum showed that 86% of these termswere not matched (fully or partially) to the museum docu-mentation. When comparing the collected tags to the Artand Architecture Thesaurus(AAT), 70.2% matched somepart of an AAT term record. When museum staff reviewedtags, 88% of the total collected tags were found to be useful.

2.2 Games for crowdsourcing

Over the past few years, casual games have become widespread. This class of games is aimed at persons who donot consider themselves as gamers [6]. Casual games arelightweight, often “sticky” online games with simplifiedcontrols and straightforward gameplay, which do not re-quire previous video gaming skills or fundamental time in-vestment. According to [7], casual games are played by200 million online users, usually with short game sessions,which however often add up to hours of continuous game-play. Since casual games usually target multiple platformsand varying audiences, they are an attractive platform forcrowdsourcing.

In “Games with a purpose” (GWAP), introduced by [8],gaming practices are applied to crowdsourcing tasks. Theidea is to provide an entertainment incentive for partici-pants to do tasks that are difficult to solve through auto-matic methods; it is thus often referred to as human com-putation.

A classic example of a GWAP is the ESP Game [9], whichpioneered the tagging games genre and served as a proto-type for its many successors later developed by Galleries,Libraries, Archives an d Museums (GLAMs). In this game,two players are shown the same image and asked to describeit with keywords. Players cannot see each other’s input.When two players come up with the same keyword, theyare rewarded with bonus points and the matched keywordis added as a descriptive label for the image. Thus, the gameharnesses collective human intelligence to produce valuableoutput as a side effect of enjoyable gameplay. Features ofGWAP include [8]: score keeping, taboo words, time limit,randomness, player skill levels, and a high score list.

“Tag! You’re it!”3, by the Brooklyn Museum, is one ex-ample of using games to incentivise cultural heritage crowd-sourcing. Another example is “Waisda?”4 by the Nether-lands Institute for Sound and Vision, where players anno-tated TV shows.

3http://www.brooklynmuseum.org/opencollection/tag_game/

start.php4http://woordentikkertje.manbijthond.nl/

Tasks common in crowdsourcing, such as various classi-fication and correction tasks, may not be always enjoyablein themselves. However, when offered as casual games withwell-thought game mechanics, they can become compellingand entertaining experiences. The gamification approachhas been criticised for enticing people into doing work via“gaming tricks” instead of letting them be a part of some-thing bigger, which is a deep interaction with their pastthrough digital collections [2]. To address this problem, webelieve that crowdsourcing games should provide playerswith a clear picture of what they are playing with, whatthey generate and how it will be used.

2.3 Online social networks

Online social networks (OSN) have gained popularityamong GLAMs for their role of a central platform that en-ables closer interaction with their patrons and facilitates thecreation, use, and sharing of information [10]. Small insti-tutions with limited resources can take advantage of socialnetworks to make their digital collections available to wideraudience, whereas large organizations can benefit from theincreased exposure of their collections due to the fact thattheir own user communities often socialize with each otheron social media [11]. For example, the Flickr Commons5

project, launched in 2008, provides a platform for culturalheritage institutions to host their content to achieve a dia-log with large online communities by means of social taggingand commenting on collection objects [12].

Just as many GLAMs move their digital collections fromprivate web portals to shared online spaces, their crowd-sourcing applications can follow the same path thanks to theprovided functionality of many OSNs to build integrated so-cial applications. Games deployed on social networks havedemonstrated a rise in popularity since the genre appearedin 2007, with Facebook remaining as the most popular plat-form for social gaming. These games take advantage ofthe “ready-made” community and encourage deep social en-gagement and interaction by leveraging the functionalitiesof the underlying OSN platform.

The main principle for designing social games is to createcompelling interaction between players by means of com-munication and self-expression, which is the key to sustain-able data crowdsourcing [13, 14, 15]. Design drivers forplayfulness in social gaming, as formulated by [14], includesymbolic physicality, spontaneity, inherent sociability, nar-rativity and asynchronicity. Related to this, [13] studiedthe features of the Facebook platform from the perspectiveof “social change games”, which aim to cultivate awarenessand foster public participation in advancing positive socialchange. This category is particularly applicable to crowd-sourcing games in the cultural heritage sector. The most im-portant Facebook platform features for designing such kindof games include: social graph, virality, micro-transactions,metrics tools, asynchronous gameplay, social infrastructure,head-to-head challenges. In terms of social interaction, [16]grouped the features of Facebook into 3 categories: a) com-munication features, b) collaboration features, and c) com-petition features (refer to Table 1 for a subset of features

5http://www.flickr.com/commons/

Competition Communication CollaborationLeaderboard Friend requests Sharing a trophyAchievements NotificationsChallenges

Table 1: Facebook social features used in Art Collector

used in the game prototype).Although not much research exists in this area, some at-

tempts have been made to crowdsource data from Facebookgames. For example, the Raport Game and the VirtualPet Game were used by [15] to collect data from playersin a question-answering fashion, demonstrating that morethan 80% collected data were found useful. Similarly, theSentiment Quiz game described in [17] was used to collectplayer’s opinions on whether sentences and dictionary termsexpress positive or negative sentiment. In a 3 month period,the application received more than 1000 Facebook users whoevaluated 30,000 quotes on the US presidential candidates.

Using OSNs as a platform also entails risks, such as in-stability and limitations of an underlying platform, identitybreach or misuse of private data, potentially high mainte-nance and marketing costs, and misuse or overuse of notifi-cations, which may lead to “mini-feed spam” [17, 13, 18].

3 Methodology

This section describes the desing, implementation, and eval-uation of the Art Collector Facebook game prototype.

3.1 Design

The prototype design process involved determining gamegenre, content provider, social features, game logic andrules, user interface and technical platform.

Following the example of the ESP Game and its succes-sors, we built a metadata tagging game for the followingreasons: a) tagging is an easy and familiar activity to mostusers, hence a good fit for a casual game; b) tagging allowsfor automatic “validation through agreement” of metadata;c) the adoption of this genre by many GLAMs makes theprototype comparable to similar crowdsourcing games.

The photo collection of the Swedish National HeritageBoard6 was used as game content, since it is of general in-terest and is relatively easy to tag.

Our approach was to take an existing game as a basis(see Section 4-4.2 for details) and improve its algorithmby adding social elements to the gameplay. The subset ofpopular social features of Facebook used in Art Collector,grouped by the classification of [16] is shown in Table 1.Their use is described in Section 4-4.3. The game logic,rules, and interface are given in Sections 4-4.1 and 4-4.2.

3.2 Implementation

Art Collector is a browser game built “on Canvas”, mean-ing that the application is presented from within theapp.facebook.com domain, so that it appears to users as

6http://www.raa.se/

an integral part of the social network. The game was im-plemented using Facebook SDK for PHP and Javascript,and the access to the photo collection was done throughthe SOCH web service7.

3.3 Evaluation

The testing period of the prototype lasted 2 weeks and itsresults were evaluated for 3 factors: participation, contri-bution and feedback.

Game testers were recruited through social networks, bysharing the application fan page on Twitter and thematicFacebook groups related to crowdsourcing, Swedish historyand cultural heritage, gamification, gaming, and so on. Tar-geting diverse groups was needed to evaluate the prototypefrom different perspectives. Before the game launch, wepre-loaded Art Collector’s public gallery with 10 images of4 tags each to ensure that even the first players will haveenough images to “challenge”.

To obtain player feedback, a short 4-page survey wasused, comprised of two open questions, two closed ques-tions and five scale questions. The survey was incorporatedinto the game flow and was shown when a player completedthe full cycle of the game, i.e. its both rounds (see Sec-tion 4-4.1). This time point was chosen to ensure that onlycommitted players provide their feedback, based on actualplaying experience of the entire game. Upon completing thesurvey, a player was rewarded with bonus tokens.

The quantitative analysis of game data was performed af-ter the testing period by querying the database. To monitordaily dynamics of user participation and contribution (e.g.number of users, number of tags, etc.), we created a sepa-rate statistics table in the database. For platform-specificstatistics such as published stories and user demographicswe used the built-in analytical tool called Facebook Insights.

The qualitative data that we analysed were the user-generated tags and players’ responses to the open questionsin the survey. The analysis of user-generated tags was doneby manual checking and classifying each tag in the databaseat the end of the testing phase. The user feedback wasgrouped as positive and negative in relation to the identi-fied categories.

4 Art Collector

4.1 Gameplay

As the name suggests, players of the game become art col-lectors - they compete with each other for art pieces (i.e. im-ages) in order to build the richest private art gallery. Eachart piece is assigned a value corresponding to the numberof tags associated with it (e.g. an image with 6 tags has avalue of 60). Art pieces that have received at least 4 tags areautomatically added to the public gallery, from where theycan be “challenged” by other players. The game consists oftwo roands: “Tag It!” (Figure 2) and “Challenge!” (Fig-ure 3 and 4), which are played iteratively. The main screen

7http://www.ksamsok.se/

Figure 1: Main screen

Figure 2: Round 1

of the game displays a leaderboard, a list of friends, a per-sonal gallery and two buttons for entering each game round(Figure 1). The first round consists of 4 turns (each turn isused for tagging 1 image) and its purpose is to accumulatetokens. A player gets 2 tokens for a matched tag and 4 to-kens for a unique tag. In the second round, a player needs toguess tags added by other players to art pieces of her choice.Art pieces can be chosen either from the public gallery orfrom private galleries of other players. Correctly guessinghalf of the tags8 associated with a challenged image triggersthe winning condition, in which case an art piece goes to theplayers private gallery. Each failed guessing attempt costs aplayer 20 tokens. Three guessing attempts are allowed perimage. When the number of tokens becomes insufficient forchallenge, the player returns to the first round to earn moretokens.

8if the total number of tags is odd, the amount of tags to be guessedis rounded down, e.g. 4 out of 9

Figure 3: Round 2: choosing an image

Figure 4: Round 2: challenge

4.2 Game logic

Although all tagging games serve the purpose of generatinguseful metadata, their game logic can vary. Games such asAlum Tag9 and Dora [19] feature single-player gameplay,where metadata is generated when users annotate an imagewith keywords. Other games, such as the ESP Game, fea-ture more social, multi-player gameplay, where metadata isgenerated as a result of matching inputs from two players.In Art Collector, we took a slightly more complex approachthat combined two methods of metadata generation. Thefirst method is the traditional “tagging for points” logicthat we used in Round 1 and borrowed from the Alum Taggame. Although this strategy has been widely used in sim-ilar games, we found that it did not provide much room forthe integration of Facebook’s social features, which was our

9http://alum.metadatagames.dartmouth.edu/alum/www/index.

php/games/ZenTag/

primary goal.The prototype required a more complex scenario that

could produce metadata through social interaction. Thus,in Round 2 we introduced another type of metadata gener-ation which is based on matching user inputs. Contrary toESP Game, in Art Collector tag matching is used to triggerthe winning condition instead of accumulating points. Infact, points are spent rather than earned. Moreover, tagsgenerated through failed guessing attempts get added tothe challenged image, raising its value. This way, if player’sart piece is challenged, the risk of losing it is compensatedby the likelihood that its value will be increased. This iscalled a “risk reward principle” [14]. We adhere to thelogic that players are motivated to enter accurate tags inorder to guess their opponent’s input correctly. Thus, twometadata generation methods in Art Collector are tied bya single scoring system and a storyline. In their pursuitfor art pieces players are encouraged to play both roundsiteratively, generating metadata as a result of their actions.

In the first round, the priority was given to showing im-ages that already received 1 to 3 tags. This was done withthe purpose to reach the 4-tags minimum that is required forthe inclusion of an image to the public gallery. The side ef-fect of this is automatic validation of matched tags. Whenno more pre-tagged images existed, next series of imageswere loaded randomly from the remote photo collection.

The prototype uses several elements of GWAP and ca-sual gaming, in addition to the social features discussedbelow. In terms of casual behaviour, the game offered rapidprogress through both rounds, varied game difficulty (de-pending on challenged pieces’ value), short step-by-step in-structions and immediate gratification (tokens in the firstround and a “trophy” in the second round). GWAP fea-tures included score keeping, randomness (when loading im-ages from the photo collection), taboo words (not allowingplayers to guess their own tags), player skill levels (achieve-ments), high score list (leaderboard).

4.3 Social features

This section shows how the features listed in the Table 1were implemented in Art Collector to enable social interac-tion between players.

Bragging on the news feed using notifications lets playersshare their winnings with other OSN members, creating ex-tra entry points to the game. Though not yet implemented,another notification could be used for asking friends to guessremaining tags for an image in case of several failed at-tempts, in exchange for a reward.

A friends list was used to recruit more players from in-side the game. Players invite their friends either for a re-ward or out of desire to challenge a friend. Challengingfriends is a powerful competition feature of Facebook thatutilizes emotional ties between players and their friends andallows them to engage in a head to head competition. Theability to challenge a friend in Round 2 of Art Collectorcreates competitive spirit and gives a more personal feel tothe game.

The leaderboard was designed as a ranked list of top 10players, sorted by the value of their private galleries. The

Achievement Triggering conditionPower Tagger adding 50 tagsSuper Tagger adding 100 tagsPower Guesser winning 5 art piecesSuper Guesser winning 10 art pieces

Table 2: Achievements

top 3 art collectors are awarded with gold, silver and bronzemedals, which are indicated on their avatars. It is worthnoting that the intention to beat friends’ scores is oftenstronger than getting any other in-game rewards. So, inArt Collector, a player might use the leaderboard to strate-gically target top players for challenge in order to take theirplace.

The Achievements API of Facebook provides mechanismsto automatically trigger pre-defined conditions met in thegame and post them on the wall as player achievements.Similarly to wall posts, Facebook achievements offer in-creased exposure of game activity on the social network andserve as entry points to the game. Willingness to match afriend’s achievement or to level up to the next achievementare strong motivations for sustainable play. Achievementsused in Art Collector are summarized in Table 2.

5 Evaluation Results

This section presents the results of the game testing, whichlasted two weeks in May 2013. We assessed Art Collec-tor on three metrics: user participation, contribution, andfeedback.

5.1 Participation

Over the two-week period, the application received a to-tal of 103 users. Sharing a fan page on Facebook groupscaused a fairly steady growth in participation (see Figure 6).13.6% of participants were recruited via friend requests sentfrom the game. Player retention was 56.3%, which indicatesthat more than a half of players returned to the game atleast once. However, 35% of visitors were not active in thegame and did not generate tags. 36 stories10 were publishedfrom the application, which were clicked 10 times. Demo-graphic data provided through Facebook Insights indicatesthat male players outnumbered female players, whereas thebiggest age group was between 25 and 34 years (Figure 5).

Figure 5: User demographics: age and gender

10Facebook stories are structured notifications composed of objectsand actions (e.g. ”your friend won a lottery”). Stories are created andshared using the Facebook Graph API.

Figure 6: User participation

5.2 Contribution

The overall playing activity after two weeks yielded 2841tags for 431 images, with an average of 5.33 tags per image.Dynamics of user activity are presented in Figures 7 and 8.

Figure 7: Total number of tags

Figure 8: Total number of tagged images

The number of most productive players identified byearned achievements is presented below:

• 12 Power Taggers with 50+ contributed tags

• 6 Super Taggers with 100+ contributed tags

• 4 Power Guessers with 5+ art pieces in the gallery

• 2 Super Guessers with 10+ art pieces in the gallery

Tag matching in both rounds produced 383 non-uniquetags, which means that 13.5% of the total amount of tagswere validated through agreement.

Manual inspection of contributed metadata revealed thatmore than 90% of tags were English single dictionarywords with correct spelling. We identified some valid non-dictionary user input, such as brands, locations, and dates.The rest of tags were not added in compliance with the gamerules and included multi-word phrases, words in Swedish,and misspelled tags. Notably, only 12 tags (0.4%) were iden-tified as spam (random letter sequences entered by carelessplayers).

5.3 Player feedback

Upon completion of the second round, players were offeredto complete a short survey in reward for bonus tokens. Al-most one third of players participated in the survey: 12students, 6 cultural heritage/open data experts, 3 game ex-perts and 10 casual visitors.

The survey started by asking players to assess the gameon a 5-grade scale in terms of its gameplay, difficulty, in-structions, and images. The results of their ratings are givenin Figure 9.

Figure 9: Game assessment by players

At the next stage, players were asked to select their mainmotivation for playing the game. The analysis of their mo-tivational factors showed that:

• Curiosity: 58.0%

• Entertainment: 16.1%

• Contribute to the photo collection: 12.9%

• Contribute to research: 9.7%

• 1 person left a free text reply: “professional interest ofinteraction designer, curiosity and interest in Swedishculture”

Finally, players were asked to express their overall play-ing experience by emphasizing the strong and the weak sidesof the game and to provide suggestions for its further im-provements. Through qualitative analysis of user feedbackwe were able to categorize their answers into 4 groups: gameidea, gameplay, photo collection, and privacy concerns. Al-most the half of all positive answers (48.4%) were aboutthe concept and the idea of the game. Some of the answers

provided by players were: “Like the idea to give my tags tothe collectn” and “Love the idea of gamification as a meansto contribute to a database of art tags”. 3 players respondednegatively, with comments like: “I am not attracted to an-notate the pictures. It does not give me any fun, more orless like a test”.

Suggestions were also given with regard to game play,such as more rounds, improved navigation and user inter-face, additional information about images in the first round,timed challenge in the second round for more suspense anddifficulty. These improvements will be considered duringthe next game build.

The game photo collection was commented by many play-ers, both positively and negatively. 8 players consideredimages as one of the strong aspects of the game, with com-ments like “nice pictures that make you curious”. 11 playersfound images rather obscure, monotonous, or unfamiliar tonon-Swedish visitors. One player also noted that “the im-ages were photographs – I was expecting art in other forms”.

Two users expressed concerns regarding their privacy:“Can you explain more what data are you gathering andwhat information will be contributed to public memoryplease?” and “I don’t play games that I don’t know welland which request my personal data for unknown reasons”.

5.4 Discussion of results

Taking into account the short period of testing and the ab-sence of Art Collector from the Facebook App Center, gamepromotion on Facebook groups alone attracted visitors withdiverse backgrounds. The existence of various Facebookcommunities grouped by interests makes it possible to tar-get specific audiences related to the topic of a game. Wefound that many members of such groups were enthusiasticand willing to accept new ideas in games. This interest wasalso supported by the players’ feedback. Curiosity to try anew game was the main stimulus for most players.

In terms of generated data volume, the game demon-strated promising dynamics. User participation had a stablegrowth throughout the testing phase, with a good retentionrate. We compared our results to those of Dora - a similartagging game that reported 6039 generated tags over a 3-month testing period [19]. This means that on a bi-weeklyscale, Art Collector generated about 2.8 times more tagsthan Dora did. While the Round 2 of Art Collector ap-pears to be the apparent reason for the increased produc-tivity, we advocate that game’s social components, such asachievements, sharing winnings, and friend requests, indi-rectly contribute to the volume of collected tags by attract-ing more players. Notably, contribution was not evenly dis-tributed between players, with nearly 1/3 of them being un-productive. This is in line with the phenomenon that onlya small portion of users contribute most of user-generatedcontent [4].

When it comes to providing instructions, the balance be-tween “short” and “informative” can be hard to achieve.The volume of instructions grows in proportion to the gamecompexity. While the majority of players stated that theinstructions were pretty clear, we faced some opposing de-mands from players who asked either for shorter instruc-

tions or more detailed explanations. We partially solvedthis problem by integrating some of the instructions intothe game flow (e.g. displaying tooltips for taboo words).We conclude that instructions may directly affect the qual-ity of produced metadata. As noted in [4], “motivating notonly for participation but also supporting quality contribu-tions appears to be a major challenge”.

In terms of data quality, Art Collector performed well.Around 95% of generated metadata were comprised of one-word tags or two-word phrases that were found valid andusable. The relevance of tags to images was not consideredas a factor of validity due to its subjectivity, and shouldbe assessed by the GLAM in charge of the collection. Thevalidity of a tag depends on the context in which it is eval-uated (e.g. “b/w” can be bad for the game but good forthe collection). In Art Collector, instructions played an im-portant role in assuring the quality of user input by makingplayers aware of their contribution to cultural heritage.

Privacy concerns were identified as a possible barrier toparticipation. However, removing access to user data wouldcome at the expense of personalized gameplay. The inclu-sion of the game to the Facebook App Center might alleviatethis problem by giving the game a better reputation.

The quality of the photo collection in an image tagginggame seems to be the concern of most players. In a sense,Art Collector failed to meet the demands of many play-ers in terms of image quality, or better say, interestingness.Skipping too many images comes in conflict with the casualgames rule of fast progress through the game. Combiningcontent from multiple sources and matching it with user’sprofile would help to keep players captivated with interest-ing images and bring variety to the game photo collection.

Although players were communicated the purpose of thegame before entering it, the main attraction for them wasnot contribution to the collection, but curiosity and fun.We explain this by the fact that most people visit OSNsto spend their free time, socialize and have fun - some-thing that any community-based game must be able to of-fer, no matter how serious and noble its purpose is. Asnoted in [14], playfulness in social games must prioritizeemotional engagement over highly sophisticated gameplay,even if there is no contradiction between these two.

Keeping the game well balanced was a challenging butnecessary task. The difficulty of play in the second rounddepends on the quality of user input in the first round. InArt Collector, even short two-word phrases added in thefirst round can make the second round very hard, unlessthe winning condition is triggered by matching one word ina phrase. Nevertheless, other parameters such as numberof guessing attempts or number of tags to be guessed canbe adjusted in order to keep the game well balanced.

6 Conclusion and future work

6.1 Conclusion

To overview and generalize the findings of this research, inthis section we present our conclusions and proposals forfuture work. We also discuss possible improvements of theprototype.

As stated earlier, the main principle of designing OSN-based GWAP games is enabling rich interaction betweenplayers [15]. With this in mind, Art Collector was designedto demonstrate how metadata games for cultural heritagecan be adapted to a complex, multi-player social environ-ment. We conclude that the enrichment of metadata gameswith social features requires more sophisticated logic andstoryline when compared to traditional metadata games,with virality, personalization and social interaction as maindesign drivers. Moreover, from the player feedback we ob-serve that instructions and photo collection are particularlyimportant factors that have to do with the specificity ofthe cultural heritage domain. Properly written instructionsminimize the risks of wrong input and motivate newcomersto participate in tagging “real objects” from digital collec-tions. On the other hand, the interestingness of photo col-lection directly affects player participation and retention. Inthis regard, an OSN platform can provide context awarenessand find suitable content for a player.

The immediate advantage of the migration of crowdsourc-ing games to social networks is the availability of hugeready-made community of potential contributors. The re-cruitment of “art collectors” on Facebook demonstratedthat thematic Facebook groups provide opportunities fortargeting specific audiences grouped by interests and rele-vant background to be engaged in collaborative metadatageneration for GLAMs.

To foster public participation, OSN platforms providemechanisms of viral promotion through social channels, aswell as powerful metrics that help to monitor user engage-ment, retention and many other parameters that are centralto a crowdsourcing application. As a part of viral mecha-nism, game content sharing can be a good way of exposingcollection items from the game to the “outside environment”of social network - the way we did it in Art Collector. Thiscan elicit interest towards the photo collection from non-players, and thus attract more contributors. For existingcrowdsourcing applications in the cultural heritage sector,requirement for registration is considered as one of the ma-jor barriers to participation [19]. For a game deployed onOSN platform, this barrier is reduced since users engage inplay from the state in which they are already authenticated.The only requirement might be to authorize the game uponinstallation, which is something that OSN users are used toand need to go through only once. Still, we observed thatfor some players authorization can create another obstacledue to privacy concerns, although the percentage of suchusers was very low.

To sum up, we believe that social media gaming is thenext step in evolution of crowdsourcing games, allowingthem to be personalized and socialized at greater extentthan before. Since crowdsourcing is about “crowd”, hugeonline audiences of social networks should be a primary tar-get for cultural heritage institutions that wish to leveragethe wisdom of the crowds. Our work demonstrated howthis can be done in theory and practice and pointed to theadvantages and shortcomings of this approach.

6.2 Future work

The fact that social media games “know” more about theirplayers offers new opportunities for personalized gameplay,specifically adjusted to player’s persona. For instance, ac-cess to spoken languages could be used to offer the playera selection of translation tasks. Automatic pre-loading ofimage content matching user’s demographics or personal in-terests stated on the profile page is another possibility.

Micro-transactions as a popular means to monetize so-cial games could be utilized by GLAMs to raise funds forpreservation of cultural heritage. This way, players volun-tarily contribute not only their knowledge and efforts, butalso their financial resources to digital archives - all as aside effect of playing a game. In such cases it is importantto balance ethical implications with the economic needs ofan institution [13].

Interconnecting not only players, but the games them-selves into an eco-system of thematic mini-games wouldhelp to crowdsource multiple tasks on shared data, suchas tagging, validation, correction, linking, translation, andso on [19, 17]. The ability to choose from tasks wouldgive players the freedom of action and ways to define theirown “fun” [20], assuring that committed players never runout of new challenges [17]. We trust that the diversity ofOSN community and the platform’s social infrastructurecould be an ideal environment for such kind of multi-playerasynchronous interaction, where experts and general publicfrom an OSN community collectively enrich digital archivesin a playful manner. Building a prototype of crowdsourcinggames ecosystem based on a cross-OSN-platform GWAPframework like the one proposed in [17] would be an ambi-tious endeavour to allocate human resources from multipleOSNs.

6.3 Towards Art Collector v.1

The next paragraphs present our considerations for the fu-ture implementations of Art Collector.

We found that the Alum Tag scoring system was not well-suited for Art Collector because it did not encourage play-ers to add correct tags. In fact, one of the possible playingstrategies could be to add erroneous tags on purpose tomake the second round hard for the opponents. The prob-lem can be addressed by reversing the logic of the scoringsystem and increasing the score difference between uniqueand matched tags (e.g. awarding 2 points for a unique tagand 6 points for a matched tag). This would motivate aplayer to enter accurate tags in hope that some of them willbe matched and points will be accumulated faster. As anextra measure for securing data quality, tags produced inthe first round could be checked for validity before gettingtied to images. This could be done by using a two-stepsemi-automatic validation based on dictionary. This way,user input is first undergone spell checking through auto-mated tools to filter out non-dictionary words. Tags thatdo not pass the spell checker control are marked for man-ual review that is performed by either a game moderatoror other players. In the second case, validation of metadatamay take place in the context of another game similar to

the Brooklyn Museum’s Freeze Tag!11, where players votefor tags added by other players and decide whether to keepor discard them.

Although Art Collector demonstrated promising dynam-ics when compared to similar non-OSN-based prototype ap-plications such as Dora [19], these prototypes are not di-rectly comparable. To make a more realistic comparison,we plan to launch the game in two experimental setups:stand-alone and OSN-based. The stand-alone version wouldbe a separate web portal similar to GWAP games, using adatabase of registered users as game opponents. The OSN-based version could be a Facebook game like the one pre-sented in this work. The only difference between the twoversions would be the absence of access to the Facebookcommunity and platform infrastructure in the stand-aloneversion. In such a setup, two identical games would belaunched at the same time and monitored for a certain timeperiod. The side by side comparison of user participationand contribution in stand-alone and OSN-based versions af-ter the testing period would allow for more trustworthy andaccurate experimental evaluation of the prototype.

References

[1] J. Howe, “The rise of crowdsourcing,” WiredMagazine, 2006. [Online]. Available: http://www.wired.com/wired/archive/14.06/crowds.html

[2] T. Owens. (2012) Crowdsourcing cultural her-itage: The objectives are upside down. [Online].Available: http://www.trevorowens.org/2012/03/crowdsourcing-cultural-heritage-the-objectives-\\are-upside-down/

[3] M. Ridge, “From tagging to theorizing: deepening en-gagement with cultural heritage through crowdsourc-ing,” Curator: The Museum Journal, vol. 56, no. 4, pp.435–450, 2013.

[4] J. Oomen and L. Aroyo, “Crowdsourcing in the cul-tural heritage domain: opportunities and challenges,”in Proceedings of the 5th International Conference onCommunities and Technologies. ACM, 2011, pp. 138–149.

[5] J. Trant, “Tagging, folksonomy and art muse-ums: results of steve.museums research,” Uni-versity of Arizona, Tech. Rep., Jan 2009. [On-line]. Available: http://arizona.openrepository.com/arizona/handle/10150/105627

[6] J. Juul, A casual revolution: Reinventing video gamesand their players. The MIT Press, 2012.

[7] (2013) Casual games facts. [Online]. Avail-able: http://www.casualgamesassociation.org/news.php#casualgames

11http://www.brooklynmuseum.org/opencollection/freeze_tag/

start.php

[8] L. Von Ahn and L. Dabbish, “Designing games with apurpose,” Communications of the ACM, vol. 51, no. 8,pp. 58–67, 2008.

[9] ——, “Labeling images with a computer game,” in Pro-ceedings of the SIGCHI conference on Human factorsin computing systems. ACM, 2004, pp. 319–326.

[10] J. Murphy and H. Moulaison, “Social networking lit-eracy competencies for librarians: Exploring consid-erations and engaging participation,” in 14th AnnualACRL Conference, 2009.

[11] K. Smith-Yoshimura, “Social metadata for li-braries, archives, and museums,” OCLC Re-search, Tech. Rep., 2012. [Online]. Avail-able: http://www.oclc.org/content/dam/research/publications/library/2012/2012-02.pdf

[12] M. Kalfatovic, E. Kapsalis, K. Spiess, A. Van Camp,and M. Edson, “Smithsonian team flickr: a library,archives, and museums collaboration in web 2.0 space,”Archival Science, vol. 8, no. 4, pp. 267–277, 2008.

[13] J. R. Whitson and C. Dormann, “Social gaming forchange: Facebook unleashed.” First Monday, vol. 16,no. 10, 2011.

[14] A. Jarvinen, “Game design for social networks: inter-action design for playful dispositions,” in Proceedingsof the 2009 ACM SIGGRAPH Symposium on VideoGames. ACM, 2009, pp. 95–102.

[15] Y.-L. Kuo, J.-C. Lee, K. yang Chiang, R. Wang,E. Shen, C. wei Chan, and J. Y. jen Hsu, “Community-based game design: experiments on social games forcommonsense data collection.” in Proceedings of theACM SIGKDD Workshop on Human Computation.ACM, 2009, pp. 15–22.

[16] I. Di Loreto and A. Gouach, “Social casual games suc-cess is not so casual,” LIRMM, University of Montpel-lier - CNRS, Tech. Rep., 2010.

[17] W. Rafelsberger and A. Scharl, “Games with a purposefor social networking platforms,” in Proceedings of the20th ACM conference on Hypertext and hypermedia.ACM, 2009, pp. 193–198.

[18] B. Kirman, S. Lawson, and C. Linehan, “Gaming onand off the social graph: The social structure of face-book games,” in Proceedings of the 2009 InternationalConference on Computational Science and Engineer-ing - Volume 04. IEEE Computer Society, 2009, pp.627–632.

[19] M. Ridge, “Playing with difficult objects: game designsfor crowdsourcing museum metadata,” Master thesis,City University, 2011.

[20] B. Kirman, “Emergence and playfulness in socialgames,” in Proceedings of the 14th International Aca-demic MindTrek Conference: Envisioning Future Me-dia Environments. ACM, 2010, pp. 71–77.