a35 Innocent

7/30/2019 a35 Innocent

1/15

35

Blending Realities in Game Space

TROY INNOCENT

Center for Electronic Media Art, Monash University

Modes of representation in digital games are explored in terms of game aesthetics, structure, andlogic. A model for world-making is developed that identifies key components of digital medialanguage and their relationship to shifting modes of representation. These ideas are demonstratedthrough three art works created by the Idea-ON>!, Iconica, and Semiomorph. The final work,Semiomorph, combines the theory of semiotic morphism with gameplay to create a digital gamethat generates blended realities. This work is described and analyzed in detail to illustrate theproposed model.

Categories and Subject Descriptors: H.5.1 [Information Interfaces and Presentation]: Multi-media Information SystemsArtificial, augmented, and virtual realities; I.2.11 [Artificial Intel-ligence]: Distributed Artificial IntelligenceMultiagent systems; J.5 [Computer Applications]:

Arts and HumanitiesFine arts

General Terms: Experimentation

Additional Key Words and Phrases: Language of computers, semiotic morphism, digital iconogra-phy, ludology, media arts, digital games, artificial life

ACM Reference Format:

Innocent, T. 2008. Blending realities in game space. ACM Comput. Entertain. 6, 3, Article 35(October 2008), 15 pages. DOI = 10.1145/1394021.1394028 http://doi.acm.org/10.1145/1394021.1394028

1. INTRODUCTION

Electronic spaces can manifest complex relationships between their represen-tation and the underlying system that defines them. Elements such as iconog-raphy, synthetic materials, skins, and emulated media are used to represent the

virtual world built in electronic space. Playing with the relationship betweenthe representation and the system develops its capacity as a dynamic, expres-sive generator of new meanings. This opens up opportunities for the explorationof virtual world not as simulation of the real, but as a medium for the devel-opment of other coherent, alternative worlds whose representation is fluid and

Authors address: Centre for Electronic Media Art, Monash University; email: [email protected] to make digital or hard copies of part or all of this work for personal or classroom use isgranted without fee provided that copies are not made or distributed for profit or direct commercialadvantage and that copies show this notice on the first page or initial screen of a display alongwith the full citation. Copyrights for components of this work owned by others than ACM must behonored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers,to redistribute to lists, or to use any component of this work in other works requires prior specificpermission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 2 PennPlaza, Suite 701, New York, NY 10121-0701 USA, fax +1 (212) 869-0481, or [email protected] 2008 ACM 1544-3574/2008/10-ART35 $5.00 DOI= 10.1145/1394021.1394028 http://doi.acm.org/10.1145/1394021.1394028

ACM Computers in Entertainment, Vol. 6, No. 3, Article 35, Publication date: October 2008.


2/15

35:2 Troy Innocent

mutable. The intersection of the audiovisual language of computers, artificiallife, generative systems, and virtual world design combines this fluid repre-

sentation with the denatured logic of the computer. The dynamic and engagingnature of game space makes it an ideal site for the investigation of these themes.Such ideas have been explored in a series of worlds that are structurally

different, but they all connect fluid representations of electronic spaces withgameplay, interactivity, and system design. Artefact: Semiomorph is a digitalgame that explores semiotic morphism, a systematic translation between signsystems in which signified messages can be mapped onto various signifiers,multiplying and mutating instances of semiosis. Iconica is an artificial worldmade of language, where iconic elements are the basic building blocks of a worldliterally made of language. Six elements from this language relate to unique pic-torial styles and soundscapes used to represent the world, ranging from plasticknowbots and surreal iconography to electronic abstraction and the dirt of thereal world. Finally,Idea-ON>! is a database of ideas and experiences contained

within a collection of virtual worlds that explore the nature of electronic space.This investigation engages with digital games on a number of levels. The

unique aesthetics of games are explored through an exploration of their iconog-raphy, artificiality, and different modes of representation. Game logic as it isexpressed in artificial intelligence and behavioral systems drives the actionsof the entities that populate these worlds. Finally, game spaces are rich in theendemic properties and characteristics of electronic space that inform the lan-guage of computers.

An approach to the creation of new media art based on structures, strategies,and aesthetics of digital games has been developed for exploring the potentialof these areas. This approach works towards discovering opportunities for com-munication and expression that are unique to the medium of electronic space.Three major components of this approach will be articulatedthe art practice

of world-making, the nature of the computer as emulator/shape-shifter, andgameplay/modeling as a generative meaning system.

2. WORLD-MAKING

Making digital games can be seen as an extension of the practice of world-making. Artists may take on the role of world buildercreating coherent al-ternative worlds from patterns of information. Steven Holtzman describes thispractice in terms of how via a system of symbols to represent a view of reality,the artist shares his consciousness of aspects of that reality. [Holtzman 1994].The representations of these worlds are made of meshes, geometric primitives,textures, materials, sound, music, text, graphics, animation, and other mediaelements. However, they are also defined in terms of the relationships between

these elementstheir behavior, spatial location, connection to parameters inthe world, the meaning each representation is intended to signify, and so on.The user/player is factored into this system through their agency and affecton the virtual world. This role may range from passive observer to active en-gagement. This multilevel system offers rich opportunities for new models ofcommunication and expression.



3/15

Blending Realities in Game Space 35:3

In the construction of a virtual world, a number of decisions need to be madeso that it works as a simulation. It is impossible from a computation and design

point of view to model every aspect of reality in a virtual world. So particularaspects of reality need to be selected and simulated in that world. The result isa subset of reality that emphasizes a particular experience or way of being. Adriving game is focused on the experience of driving, a first-person shooter onhunt and kill type behavior, and so on.

Factoring gameplay into the practice of world-making extends this practiceby involving the player/user into the experience. Virtual worlds that set upsituations and allow the player to act out scenarios are naturally quite differentthan static, virtual worlds that enable the navigation of a single point of viewthrough space. The rules of the game set up the potential for many differentoutcomes in the virtual world.

Conventions and rules for the appearance, placement, and behavior of virtualobjects in space are beginning to emerge. Spatial cues suggest pathways for

navigation through worlds. The player is addressed as an agent in the narrativethrough techniques for structuring space and time. Iconography works acrossmediaimage, sound, and action. All of these represent new ways of signifyingmeaning, and comprise part of digital media language.

Formalising and defining electronic space seems to be an impossible task.Every discovery leads to another that opens up new possibilities, every limita-tion that is understood reveals further glitches in the system. In my explorationof electronic space, I have focused on creating personal spaces that are particu-lar to my own intuitions and experience. The collection of icons, sounds, gamecharacters, and spaces represent a personal language in response to the themesthat have motivated the work.

In terms of perception, these worlds can be seen as being real, symbolic,or in-between. They may be seen real, in that they are represented using all

perceptual cues of real experience such as perspectival viewpoint, spatializedsound, immediate feedback, light, and other phenomena. However, they areconstructed from codesymbols and relationships between symbolsthat isrepresented by such realistic simulation. Hence they could be described as thesymbolic made real: the world model/abstraction that forms the underlyingsystem is experienced in a mode of perception usually activated by the realworld. In this way, the idea of experiences and worlds that were only possible intheory, dreams, and the human imagination are actualized in the abstract spaceof the computer. Factoring the computer into the process of generating thesespaces extends this to the potential expression of a nonhuman imagination aswell.

3. REPRESENTATION: SURFACE/STYLE/SKINSLooking deep into the structure of the computer, it is widely recognized that itsessential nature is a network of connected bits that have no inherent meaning.Meanings are assigned to this structure at many levels, but these are not per-manent and can be changed at any time. The structure becomes a series of rela-tionships among languages, images, sounds, texts, and other elements mapped



4/15

35:4 Troy Innocent

over time and space. In places made of language, concepts become worlds thatcan be navigated and restructured, dependent on any possible variables.

These spaces are made of media. They can emulate a multitude of differ-ent media, old and new, and switch between multiple surfaces or skins in aninstant. Representations can shift from one medium to the nextimage maybecome sound and vice versa. The fluid nature of connections and relationshipsin electronic space and the capacity of that space to emulate a wide range ofpossible modes of representation give rise to a unique opportunity. The sameunderlying system or structure may be represented in any number of possibleways. Not only is the system shifting and changing, but the mode of representa-tion becomes an active, dynamic, mutable element opening up new expressivepossibilities. In the case of games, this system is typically the game logic andsimulation that models the world. So this potential is extended to include themutation and shape-shifting of that world in terms of its structure and therepresentation of that structure.

The separation of system and representation is part of a wider cultural logicthat is characteristic of digital media. Generally, we are familiar with the ideaof interface being a representation of data, and that being digital means beingable to reinvent yourself at the click of a mouse: morphing effortlessly fromcalculator to spreadsheet to word processor to video-editing console and backagain. [Johnson 1997]. By way of example, game modifications involve thereplacement of textures, models, sounds, animations, maps, and other elementsin order to create new levels of a game, or entirely new scenarios representedthrough the original gameengine. Typically, therules of play and the underlyingworld remain unchanged, although more sophisticated game mods do alter thefunctionality of the game. Modifications by artists may subvert the meaning ofa game, or transform it into a musical instrument. This has also spawned newareas of activity, most notably the practice of film-making using game engines,

known as machinima [Machinima n.d.].

4. GAME SPACES

A model of game spaces that reflects these ideas and themes could be summa-rized as having the following characteristics:

(1) symbolic world defined in virtual space;

(2) mediated by simulation;

(3) interaction enables control/feedback; and

(4) gameplay defines logic of symbolic world.

These can be seen as the basic elements of game language that most contem-

porary digital games operate with. So we may look at digital games in termsof interaction in terms of the many levels of the interface; simulation in termsof the modeling of certain phenomena; gameplay in the rules of operation andconstraints; and the idea of a symbolic alternative world. The layers of meaningthat are generated by digital games are largely the result of the connection andinterplay between these elements.



5/15


This article looks at the subset of digital games that are situated in virtualworlds; this term itself is quite open to interpretation, so virtual worlds will

be defined as electronic spaces with the following characteristics (described inmore detail in an earlier paper which explored the nature of electronic space[Innocent 2003]):

Feedback loop. The player has direct control of the game and it provides im-mediate feedback. This realtime interaction results in communication betweenthe player and the game as a continuous stream of events.

Mode of perception. Game spaces trigger many of the perceptual cues of realspaces. Spatialized sound, physics simulations, 3D perspective, and the playersability to manipulate the environment create a sense of actually being in thegame world.

Transmutational space. Despite its reliance on the simulation of a real world,the space of games is highly mutable and subject to change. Representationsmay be redefined in a literal sense through changes in parameters of the sim-

ulation, by actions of the players, or direct modification of the software.World as sign system. The nature of the computational medium is a symbolic,

abstract realm that requires everything to be formally defined. These defini-tions can be analyzed in terms of relations between signs, and so the virtualworld literally becomes the embodiment of the sign system.

5. SHIFTING MODES OF REPRESENTATION

Further potential in new forms of communication and expression may be foundthrough exploration of the separation between system and representation thatoccurs in digital media. Outside the realm of the realism typical of computersimulation, alternative worlds based on shifting modes of representation canbe defined in game space. In this model, the system is defined as both the

elements of the virtual world and the relationships between these elements interms of the art practice of world-making. The representation of this system isrecognized as a collection of media that may be mapped and connected to the

various elements defined in the system. The connection between system andrepresentation occurs within the electronic game space through its aesthetics,logic, and structure.

Three works that use different modes of deliverya CD-ROM, an interactiveartificial life simulation, and a 3D gameengage with these themes and usethis particular model of world-making as the strategy for their conception andconstruction. Each work deals with the shifting modes of representation andthe blending of multiple realities that can occur in game spaces.

Idea-ON>! (Figure 1) is an electronic database, a database of ideas and ex-periences, each one contained in a place in the world. A kind of catalogue of

experience or repository for characters, stories, sounds, and images which havebeen collected from my subconscious over time, grouped into four new reali-ties. Visit the Techno Garden to find morphing plants and a fertility goddess,or experience simulated wealth and human mutations in the Cybaroque world;touch an iconbody during the ritual in the prelinguistic world and see a shortdance; or talk with new communication forms in the Communications space.



6/15

35:6 Troy Innocent

Fig. 1. Worlds and modes of representation in Idea-ON>!

Fig. 2. Iconica elements.

Visiting the Idea-ON>! installation can be likened to visiting a sacred sitewhere spirits and myths reside. The information space inside the computerbecomes a dreaming or meditative space, a manifestation of the subconsciouswhere the objective contents of thoughts are stored for others to explore andexperience or add upon if they desire. Similar in the way prelinguistic societieswould have a shared body of myths and legends that made up their perception ofthe universe, a world like Idea-ON>! jumbles together many things, towards aprototype of a dreamlike, surreal, communal cyberspace in which people dream,create, imagine, and play with thought and form.

Iconica (Figures 2, 3 and 4) is an interactive project, typically experienced asa gallery installation, which enables visitors to interact with an artificial worldmade of language. The work has the capacity to evolve, change, and mutate

through human interaction and its own evolutionary process. Visitors to theworld can create, construct, and manipulate objects, influence the evolution ofsocieties, and discover new language elements. Communication with the resid-ing lifeforms occurs via the iconic language on which the world is based. Themultiplicity ofIconica is experienced through this interactionsimultaneouslya cyberspace, a mindspace, an abstract world, and a stylized reality.

Iconica uses a generative system to create unique images and soundtracksfor each of the lifeforms generated by its artificial life model. The audiovisuallanguage it draws upon for these representations relates to different graphicstyles used in digital media that signify various views of reality. Synthetic formsare displayed as clusters of shiny geometric primitives, information forms aredisplayed as numbers and bits, and so on. These shifts in representation are anactive use of semiotic morphism to create a multidimensional space in which

different types of representation coexist and overlap in the same space.Another level of the work is the relationship between the iconic language

used to represent the world of Iconica, the artificial life model, and the waythis language is used to interface with the world. The artificial life model oper-ates using symbols (Figure 3). Energy forms, mutation forms, body structures,spaces, and so on, are all defined as symbolic entities in the artificial life model.



7/15


Fig. 3. Anatomy of an entity in Iconica.

Fig. 4. Selected entities from Iconica, with descriptions.

An iconic equivalent for each of these abstract symbols used in the code of theworld can be found in the iconic language used by the world. A simple gram-matical structure is also used by the iconic language to describe relationshipsbetween elements in the world, such as which element a lifeform is made of,whether it is hungry, and so on. The users communication with lifeforms is



8/15

35:8 Troy Innocent

also achieved through the iconic language of the world. As a result, the userslowly learns this alternative language through interaction with the work. This

engagement results in a different kind of immersion that comes from the in-volvement with the system and language of an artificial world.

6. SEMIOMORPH

This model and the investigation of shifting modes of representation has beenexplored further in the third work, Semiomorph, which looks at the intersec-tion of semiotic morphism with digital games. Semiomorph is a third-person3D digital game that is situated in a virtual world that morphs and shifts itsrepresentation to the player.

Virtual worlds have the innate ability to shift and change, mutate and trans-mutate, and layer multiple modes of representation. Semiotic morphism is amodel for translating the same data between these different modes of rep-resentation through computation. Potentially, hybrid modes of representation

may be computed by the system. It uses algebraic semiotics [Innocent 2003]to adapt semiotic structures into functions and theorems using algebra andset theory. Once defined in this way, semiotic morphism allows the signifiedmessage to be mapped to various signifiers resulting in a system that gener-ates different instances of semiosis. This strategy reflects the nature of thecomputer both as a manipulator of language and computational machine.[Innocent 2003]

Semiomorph [Innocent 2003] explores this idea by connecting gameplay withshifts in representation on the basis of Goguens theory of semiotic morphism[Goguen n.d.]. The gameplay is described by the author as follows:

The goal of the game is to collect enough energy points to create a semiomorphand move onto the next stage. The kind of energy you collect will shift your

mode of representation between word, diagram, icon and simulation. This inturn will change the rules of play and your effect on the world.

You must avoid opposing entities and blast icons. Power-ups make you invin-cible for a short period of time. Muticons switch your mode of representationinstantaneously.

The resulting space is one that is constantly shifting and changing, blendingseveral forms of representation into one experience. Although the underlyingstructure of the space does not change, the different instances of the gameobjects may be simultaneously represented as text, diagram, iconographics, orsimulated reality.

This can be seen in the various representations of game elements (Figure 5).Each mode is personified by a group of game characters competing for screen-time. Both the objects and the space can be represented using any of the four

modes of representation, either at a local or a global level. Semiotic morphismis used to generate the representation of the game space and as the basis of thegameplay.

Semiomorph explores semiotic morphism in an immersive environment, us-ing the visual language of video and computer games as the starting point forits sign system. This environment consists of a 3D space, game objects and



9/15


Fig. 5. Table of game elements in Semiomorph.

characters, and the player. Each of these may be represented using one of foursystems (Figure 6):

(1) Word: text labels appear on spaces and game objects; represented by theMt.Ke,I-t character.

(2) Diagram: spaces and game objects are represented in wireframe mode; rep-resented by the D-Glypha character.

(3) Icon: simple stylized representation of space and game objects; representedby the Specular character.

(4) Simulation: spaces and objects are texture-mapped and made more complexto increase realism; represented by the Realamon character.

In terms of production, the game system was initially specified and designedin terms of its structure. Once this was in place, creating the elements for thegame became a process of filling a database of 3D objects, textures, entities,



10/15

35:10 Troy Innocent

Fig. 6. Semiomorph game space in four modes of representation.

spaces, and sounds. Each media element was determined by the intersectionof the game objects with the four modes of representation. The musician, OllieOlsen, who composed the music and sound effects, worked with tables of gameelements indexed by the different modes of representation. He interpreted theseinto the language of music and sound, creating distinct timbres and composi-tional strategies for each of word, diagram, icon, and simulation.

7. SEMIOMORPH GAME SPACES

The potential of the Semiomorph space is explored further in four separatestages of the game (Figure 7). These stages are played through in the sameorder as they appear here. Each stage uses the same system to define the gameworld, but interprets this system in different ways to express each mode ofrepresentation.

7.1 Icon Mode / SUPER PLASTIC PIXELTM StageThis stage characterizes the iconic mode of representation inSemiomorph. Ref-erencing the plastic reality of game spaces such asSonic the Hedgehog andSu-

per Mario World, it is a stylized 3D world made of colorful surfaces and simple3D forms. Perfectly rounded hills are dotted with flowers, trees, and clustersof energy forms. Everything is what it is. This is a world of icons and it is



11/15


Fig. 7. Four stages of the Semiomorph game.

Fig. 8. Icon: SpecularTM and friends.

represented as such. However, all of these element are subject to shifts in rep-resentation, so this simple world is quickly broken down as hills become words,are rendered as wireframe, or texture-mapped.

Three entities (Figure 8) the iconic mode of representation. Playful, super-deformed figures that reference the hypercute game worlds that this stage is



12/15

35:12 Troy Innocent

Fig. 9. Word: Mt-Ke,I-tTM and friends.

built upon. The music and sound are a collection of bleeps and simple melodies

in a retro video game style. The iconic entity,Specular, may easily defeat wordentities but is vulnerable to attacks from diagram entities.

SUPER PLASTIC PIXELTM is a landscape set up for free exploration. Theplayer begins in the center of the world on top of a hill, from which radiat-ing paths of energy forms may be followed. Around the corners of the world,the power-ups may be found. Collections of blocks break up the space, creatingsmaller zones of activity close to the larger hills and sky that define the bound-aries of this stage. The structure allows long views through the space depictingthe rolling hills populated by icons and entities.

7.2 Word Mode: M3TA+TM

StageAn interpretation of ideas and themes from the realm of hypertext is thestarting point for the word mode of representation. The walls, floor, and otherstructures of this stage are textured with moving surfaces filled with words de-scribing these structures. These collections of descriptions and labels shift andchange in relation to the different modes of representation as word intersectswith diagram, icon, and simulation.

Mt-Ke,I-t (Figure 9) is the character the belongs in this realm. This humanoidfigure has a simple symbol for a head, vectors for hands and feet, and a bodymade of text and hyperlinks. In the game dynamics, word beats simulation buticon beats word. Sound for this stage consists of synthesized voice and abstractelectronic sounds mixed in a fragmented, chaotic compositional style.

The space of this stage is fragmented, consisting of different planes inter-

sected by angled walls. The walls appear solid, but the player may walk throughthem and suddenly finding himself in another space altogether. In many cases,this results in a relocation on the vertical plane as well as a shift from one levelto the next. Although the lack of solid walls means that the player can moveanywhere without restriction, the dense layers of text in the space give a feelingof claustrophobia and intensity.



13/15


Fig. 10. Diagram: D-GlyphaTM and friends.

7.3 Diagram Mode: INFOS CORPTM Stage

Information graphics, data glyphs, and other forms of machine communicationform the basis of the aesthetics and structure of this stage (Figure 10). Grids,lines, and simple geometries are mapped onto surfaces to create layers of infor-mation space. Many game elements are switched to wireframe mode, creatingoverlapping visual patterns of geometric forms. When the modes of represen-tation start shifting, these visual patterns are mixed with solid surfaces ofcolor, realistic textures, and fields of text, resulting in a hybrid informationspace.

A data glyph given life, D-Glypha (Figure 10) roams this space. Simple ma-trices of pixels have been extruded into a three-dimensional form that appearsto be a giant robot that menacingly patrols its territory. Machine rhythmsand bursts of data can be heard throughout this world. As a diagram char-

acter, D-Glypha easily defeats icon characters, but must avoid simulationtypes.This stage is built on a grid with the level based on the maze structure of

Pacman. The player moves through long passages full of energy forms that arebroken up by grids of blocks. Power-ups are in the four corners of the world.

Visibility is limited as the viewpoint of the player is bound by the walls of themaze.

7.4 Simulation Mode: SOFT RAININGTM Stage

The particular kind of realism typical of video games is the basis of this stageof Semiomorph. All surfaces are mapped with photo-realistic textures, and asoft rain falls over the natural, undulating landscape. Pools of water, largetrees, and rocks appear throughout a space that is bounded by mountains.

Particle systems, procedural textures, and reflection maps are used on the gameelements to give them a realistic appearance. This illusionary representationis disrupted as the other modes of representation break it up into wireframes,flat planes of color, and text.

A collection of surreal animals (Figure 11) personify this mode of represen-tation. They crawl about the space in different colors and markings denoting



14/15

35:14 Troy Innocent

Fig. 11. Simulation: RealamonTM and friends.

different variations of the species. These Realamon are related to the Pokemon

virtual creatures. When interacting with the other characters, the realism ofsimulation defeats diagram, but word defeats simulation mode.This stage is a landscape filled with natural forms, butstrangelypopulated by

game icons and blocks. These are scattered about at random to further developthe feel of a natural, organic space. The movement of the player is not restrictedby many obstacles, and large sweeping views of the space are common.

8. CONCLUDING REMARKS

Game spaces are ideally suited to artworks that are based on a model of world-making and on the exploration of coherent, alternative worlds that investigatethe fluid, mutable representation of these spaces. The potential for this kind ofexpression is expanded by connecting the modes of representation with game-play (Semiomorph) or a generative system (Iconica). Such practice could bedescribed in terms of blending realities.

These ideas may also be explored further in terms of the overlap betweenthe virtual and the real. Many of the game objects from Semiomorph have beenreplicated to scale as plastic models in the real world. Thus these sculpturalobjects, called offline artefacts are a literal expression of blending reality.Finally, the increased perception of virtual spaces as being integral to humanexperience extends this blend of the virtual and the real even further.

As these systems of world construction evolve into more complex formsthrough developments in artificial intelligence and artificial life, finding newways to represent and express this complexity will be needed.

REFERENCES

GOGUEN, J. (n.d.) Semiotic morphisms. http://www.cs.ucsd.edu/users/goguen/papers/sm/smm.html. Accessed Feb. 2002.

HOLTZMAN, S. 1994. Digital Mantras: The Languages of Abstract and Virtual Worlds. MIT Press,Cambridge, MA, p. 210.

INNOCENT, T. 2003. Exploring the nature of electronic space through semiotic morphism. In Pro-ceedings of the MelbourneDAC, the 5th International Digital Arts and Culture Conference (May1923).



15/15


JOHNSON, S. 1997. Interface Culture: How New Technology Transforms the Way We Create andCommunicate. HarperEdge, p. 147.

MACHINIMA. (n.d.) http://www.machinima.com/. Accessed Oct. 2003.

Received January 2008; accepted June 2008


a35 Innocent

Documents