Supporting Content Creation for Games Through Assistive Technologies Dr. Michael Katchabaw Department of Computer Science The University of Western Ontario.

Supporting Content Supporting Content Creation for Games Creation for Games Through Assistive Through Assistive TechnologiesTechnologies

Dr. Michael KatchabawDr. Michael Katchabaw

Department of Computer ScienceDepartment of Computer Science

The University of Western OntarioThe University of Western Ontario

22

OutlineOutline

IntroductionIntroduction The Content Creation GapThe Content Creation Gap Study 1: Storytelling in GamesStudy 1: Storytelling in Games Study 2: Music in GamesStudy 2: Music in Games Concluding RemarksConcluding Remarks

33

IntroductionIntroduction

Modern computer and video games Modern computer and video games would be sophisticated yet rather would be sophisticated yet rather uninteresting and unimpressive pieces of uninteresting and unimpressive pieces of software without all of their luscious software without all of their luscious contentcontent This content defines their worlds, This content defines their worlds,

characters, atmosphere, events, and most characters, atmosphere, events, and most aspects of the games that players directly aspects of the games that players directly experienceexperience

It can come in many forms: artwork, music, It can come in many forms: artwork, music, sound effects, story, dialogue, motion sound effects, story, dialogue, motion capture data, and so on …capture data, and so on …

44

IntroductionIntroduction

Fortunately, modern games tend be Fortunately, modern games tend be relatively flexible and extensiblerelatively flexible and extensible There are many opportunities for people to There are many opportunities for people to

be involved in creating content for gamesbe involved in creating content for games This includes professional developers all This includes professional developers all

the way through to the end user the way through to the end user community comprised of players and community comprised of players and hobbyistshobbyists

Unfortunately, barriers ultimately limit Unfortunately, barriers ultimately limit expression by content creators expression by content creators regardless of where they fit in …regardless of where they fit in …

55

The Content Creation The Content Creation GapGap Content creators often do not have Content creators often do not have

the technical expertise to integrate the technical expertise to integrate their content into games on their owntheir content into games on their own

Technical practitioners often do not Technical practitioners often do not have the creative expertise to create have the creative expertise to create such content in the first placesuch content in the first place

Consequently, there is a gap between Consequently, there is a gap between content creation and its place in content creation and its place in gamesgames

66

The Content Creation The Content Creation GapGap

77

The Content Creation The Content Creation Gap:Gap:The Current SolutionThe Current Solution

Traditionally, programmers are used Traditionally, programmers are used to bridge this content creation gapto bridge this content creation gap

By supporting content creators with By supporting content creators with their technical expertise, content their technical expertise, content works its way into games …works its way into games …

88

The Content Creation The Content Creation Gap:Gap:The Current SolutionThe Current Solution

99

The Content Creation The Content Creation Gap:Gap:The Current SolutionThe Current Solution Problems with this approach ultimatelyProblems with this approach ultimately

limit its effectiveness:limit its effectiveness: Increased costs, both in terms of budget Increased costs, both in terms of budget

and scheduling resourcesand scheduling resources Content creators lose creative control overContent creators lose creative control over

their contenttheir content Communication errors between contentCommunication errors between content

creators and programmers impact thecreators and programmers impact theoverall quality of the experienceoverall quality of the experience

1010

The Content Creation The Content Creation Gap:Gap:Another SolutionAnother Solution Another solution is to provide Another solution is to provide

additional training to content additional training to content creators to increase their technical creators to increase their technical expertiseexpertise

This, in effect, would allow them to This, in effect, would allow them to clear the content creation gap onclear the content creation gap ontheir own …their own …

1111

The Content Creation The Content Creation Gap:Gap:Another SolutionAnother Solution

1212

The Content Creation The Content Creation Gap:Gap:Another SolutionAnother Solution This approach has its problems as This approach has its problems as

well:well: The training processes themselves mightThe training processes themselves might

be expensive and time consumingbe expensive and time consuming Not every content creator will be able to Not every content creator will be able to

successfully acquire this technical successfully acquire this technical expertiseexpertise

In doing this, you might lose some of the In doing this, you might lose some of the benefits of specialization when it comes benefits of specialization when it comes to developmentto development

1313

The Content Creation The Content Creation Gap:Gap:A Better SolutionA Better Solution A better solution can be found through A better solution can be found through

the creation and use of software tools the creation and use of software tools that do not require technical expertisethat do not require technical expertise

These assistive technologies would These assistive technologies would allow content creators to ply their craft allow content creators to ply their craft directly in games, using modes of directly in games, using modes of interaction that are natural and familiarinteraction that are natural and familiar

This would allow content creators to This would allow content creators to overcome the content creation gap too overcome the content creation gap too ……

1414

The Content Creation The Content Creation Gap:Gap:A Better SolutionA Better Solution

1515

The Content Creation The Content Creation Gap:Gap:A Better SolutionA Better Solution Unfortunately, these assistive tools Unfortunately, these assistive tools

and technologies do not generally and technologies do not generally exist at the moment, or are fairly exist at the moment, or are fairly immatureimmature

But, interesting research is currently But, interesting research is currently under way to address these issues under way to address these issues and provide them for useand provide them for use

In doing so, many benefits to content In doing so, many benefits to content creation for games can be realizedcreation for games can be realized

1616

Study 1:Study 1:Storytelling in GamesStorytelling in Games Storytelling is widely recognized by Storytelling is widely recognized by

many as an important element of many as an important element of modern video gamesmodern video games For games that are dependent on their For games that are dependent on their

story elements, the level of quality story elements, the level of quality required for their story at least equals the required for their story at least equals the need for quality in graphics, audio, and need for quality in graphics, audio, and other other aspects of the gamesaspects of the games

Getting story content into games Getting story content into games requires technical assistance, howeverrequires technical assistance, however

1717

Study 1:Study 1:Storytelling in GamesStorytelling in Games Assistive technologies would be Assistive technologies would be

helpfulhelpful Writers could be given a simple and Writers could be given a simple and

natural way of expressing their storiesnatural way of expressing their stories Automation could then prepare this Automation could then prepare this

content for presentation within a game content for presentation within a game with little or no human interventionwith little or no human intervention

This is the intent of our own work This is the intent of our own work towards a Reusable Scripting Enginetowards a Reusable Scripting Engine Our current focus is on story content for Our current focus is on story content for

cinematics or cut-scenes, as opposed to cinematics or cut-scenes, as opposed to in-game story elementsin-game story elements

1818

Storytelling in Games:Storytelling in Games:Story ScriptingStory Scripting To automate the presentation of story To automate the presentation of story

elements from a story, the original story elements from a story, the original story needs to be scripted in such a way that it needs to be scripted in such a way that it can be easilycan be easilyacted out on-screenacted out on-screen This scripting must identify characters, dialogue, This scripting must identify characters, dialogue,

stage directions, setting, and other elements stage directions, setting, and other elements common to traditional dramatic piecescommon to traditional dramatic pieces

This scripting will also need to be precise and This scripting will also need to be precise and formal enough for it to be easily processed and formal enough for it to be easily processed and understood by the software automating the understood by the software automating the presentationpresentation

1919

Storytelling in Games:Storytelling in Games:Story ScriptingStory Scripting Fortunately, the Text Encoding Fortunately, the Text Encoding

Initiative (TEI) has developed an XML-Initiative (TEI) has developed an XML-based specification for marking up all based specification for marking up all different kinds of texts, including different kinds of texts, including dramatic piecesdramatic pieces

Since XML is not necessarily the most Since XML is not necessarily the most natural platform for storytellers natural platform for storytellers writing stories, several tools have writing stories, several tools have been developed according to TEI been developed according to TEI guidelines for importing existing guidelines for importing existing works or writing them from scratchworks or writing them from scratch

2020

Storytelling in Games:Storytelling in Games:Story ScriptingStory Scripting In the end, we could not completely In the end, we could not completely

follow TEI guidelines, and had to derive follow TEI guidelines, and had to derive something with some modifications and something with some modifications and extensionsextensions TEI guidelines are incredibly detailed and TEI guidelines are incredibly detailed and

require information that does not quite make require information that does not quite make sense in oursense in ourtarget applicationtarget application

Several elements in the TEI guidelines are not Several elements in the TEI guidelines are not formal or precise enough yet for our purposesformal or precise enough yet for our purposes

We needed additional elements to link We needed additional elements to link content and assets into story scripts content and assets into story scripts

2121

Storytelling in Games:Storytelling in Games:Story ScriptingStory Scripting

2222

Storytelling in Games: Storytelling in Games: A Reusable Scripting A Reusable Scripting EngineEngine

DirectorDirector StageManager

StageManager

ScriptReader

ScriptReader

AuthoringTools

AuthoringTools

ScriptScript

UserUser DisplayDisplay

WriterWriter

Reusable Scripting Engine

2323

Storytelling in Games: Storytelling in Games: Experiences to DateExperiences to Date Initial experiences involved reenacting Initial experiences involved reenacting

scenes from movies and television scenes from movies and television showsshows Examples include the Princess Bride and Examples include the Princess Bride and

various episodes of The Simpsonsvarious episodes of The Simpsons This demonstrated the general usefulnessThis demonstrated the general usefulness

of our approachof our approach Recent experiences involved Recent experiences involved

replicating cut-scenes and cinematics replicating cut-scenes and cinematics from existing video games …from existing video games …

2424

Storytelling in Games: Storytelling in Games: Experiences to DateExperiences to Date

Scene from Replication of Trauma Center: Second Opinion

2525

Storytelling in Games: Storytelling in Games: ConclusionsConclusions Our Reusable Scripting Engine is an Our Reusable Scripting Engine is an

example of an assistive technology example of an assistive technology to support content creators for to support content creators for gamesgames

Using our software, together with Using our software, together with appropriate authoring tools, we can appropriate authoring tools, we can effectively assist writers in creating effectively assist writers in creating and integrating their story content and integrating their story content into gamesinto games

2626

Study 2:Study 2:Music in GamesMusic in Games There are many recognized There are many recognized

outstanding issues in music for gamesoutstanding issues in music for games Static compositions that are looped or Static compositions that are looped or

repeated throughout a (long) game can repeated throughout a (long) game can become quite tiresomebecome quite tiresome

Music is, at times, disconnected from the Music is, at times, disconnected from the events unfolding in the game, particularly events unfolding in the game, particularly in open ended, non-linear gamesin open ended, non-linear games

Licensing music or hiring talent can be Licensing music or hiring talent can be prohibitively expensive, particularly for prohibitively expensive, particularly for small or medium sized enterprises small or medium sized enterprises

2727

Study 2:Study 2:Music in GamesMusic in Games Adaptive automatic composition Adaptive automatic composition

systems can alleviate or address systems can alleviate or address these issues:these issues: Endless new compositions can be Endless new compositions can be

composed dynamically throughout a composed dynamically throughout a game without any repetitiongame without any repetition

Music can be adapted to the current Music can be adapted to the current emotional context of the gameemotional context of the game

Music composed by a computer is not Music composed by a computer is not rights encumbered and can be more cost rights encumbered and can be more cost effectiveeffective

2828

Music in Games:Music in Games:AMEE IntroductionAMEE Introduction To provide assistive technologies to To provide assistive technologies to

this end, we developed the Algorithmic this end, we developed the Algorithmic Music Evolution Engine (AMEE)Music Evolution Engine (AMEE) Flexible and extensible architecture that Flexible and extensible architecture that

can impose as much or as little structuring can impose as much or as little structuring to music as desiredto music as desired

Can reuse existing musical elements or Can reuse existing musical elements or compose them entirely from scratchcompose them entirely from scratch

Can incorporate emotional characteristicsCan incorporate emotional characteristics Multi-level API to support a range of users Multi-level API to support a range of users

with varying musical and technical with varying musical and technical expertise expertise

2929

Music in Games:Music in Games:AMEE ConceptsAMEE Concepts Pipeline: controls generation processPipeline: controls generation process

Producers: create high-level musical elementsProducers: create high-level musical elements Generators: create low-level musical Generators: create low-level musical

elementselements Pattern libraries: enables reuse of musical Pattern libraries: enables reuse of musical

elements (e.g. Harmonies, Motifs, etc.)elements (e.g. Harmonies, Motifs, etc.) Emotion mapper: enables incorporation of Emotion mapper: enables incorporation of

emotions (e.g. Happy, Sad, etc.)emotions (e.g. Happy, Sad, etc.) High-level classes: model real-world High-level classes: model real-world

entities (e.g. Musician, Instrument, Mood, entities (e.g. Musician, Instrument, Mood, Ability, Style, Ensemble, etc.)Ability, Style, Ensemble, etc.)

3030

SectionProducer

SectionProducer

BlockProducer

BlockProducer

LineProducer

LineProducer

OutputProducer

OutputProducer

Emotion Mapper

Emotion Mapper

Music in Games:Music in Games:AMEE PipeliningAMEE Pipelining

Section containing blocks

Pipeline

I-IV-V-I

3131

Music in Games:Music in Games:Experiences with AMEEExperiences with AMEE We have only just We have only just

begun to scratch begun to scratch the surface with the surface with what we can do what we can do with AMEEwith AMEE

More development More development and user testing and user testing (novice and (novice and expert) is still expert) is still requiredrequired

Original

Sad

Despondent

Happy

Manic

3232

Concluding RemarksConcluding Remarks

Assistive tools and technologies Assistive tools and technologies have the potential to bridge the have the potential to bridge the content creation gap we discussed content creation gap we discussed earlierearlier

This provides content creators This provides content creators supports needed for more direct supports needed for more direct integration of their content into integration of their content into games with many far reaching games with many far reaching benefitsbenefits

3333

Concluding RemarksConcluding Remarks

At Western, we are developing such At Western, we are developing such assistive technologies for various assistive technologies for various types types of content for game integrationof content for game integration To date, efforts have focused on story To date, efforts have focused on story

andandmusic contentmusic content

Experiences so far have been rather Experiences so far have been rather positive, demonstrating great potential positive, demonstrating great potential for on-going research in this areafor on-going research in this area

3434

Questions?Questions?