SwarmLake Presentation

Φλώρος Ανδρέας Επίκουρος Καθηγητής

Swarm LakeA Game of Swarm Intelligence, Human Interac8on and

Collabora8ve Music Composi8on

1School of Music Studies, Aristotle University of Thessaloniki 2Audiovisual Signal Processing Lab., Dept. of Audiovisual Arts, Ionian University

M. A Kaliakatsos1, A. Floros2, K. Drossos2, K. Koukoudis2, M. Kyzalas2, A. Kalantzis2

Mo8

va8o

n (Con

cept & Nam

e)

Conceptual correlation

(Swarm-Swan)

Auditory interface/display: the use of sound to communicate informaôn from a computer to the user. The primary forum for exploring these techniques is the Internaônal Community

for Auditory Display (ICAD)

Mo8

va8o

n

Sonic Output

User (Interaction) Swarm

(Intelligence)

The Lake

Aesthetics Algorithm

Automated music synthesis

Natural phenomena

Bio-inspired algorithms

Algorithmic means

Objective / aesthetic measures

Music raters / human-like evaluators

Interac^ve evoluôn

produce human–like evaluaôns of the derived music, using perceptual rules

incorporated into the rather unexplored level of human noesis. Adapt algorithms

based on these aesthe^c criteria

produce well-‐ formalized, perceptually–acceptable music composiôns with rich

characteris^cs

The direct involvement of human interac^vity extends the capacity of the overall music

synthesis algorithm that is typically following (and limited by) rules defined in the domain of com-‐

putaônal intelligence and crea^vity

TribeA TribeB TribeC TribeD

Swarm intelligence

Human interaction

Swarm Lake Client

OSC

Synth-perfomers

Conductor App

Panning law

Artificial intelligence

A B C D

The Lake

Archite

cture Overview

The Lake… habitants

Boids-‐based, (semi-‐)autonomous communi8es, driven by swarm AI and exposing social behavior level. User interven8on is also allowed in a decision-‐taking level (affec8ng behavior)

Swarms

A Lake en8ty that integrates a popula8on of autonomous agents popula8on through common parameters and behavioral characteris8cs

Tribes

A cluster (co-‐opera8on) of autonomous agents belonging to the same tribe formed using spa8al posi8oning criteria (i.e. aPrac8on among same–tribe agents)

Herds

Agents

• Energy, depleted as the Agent moves

• Velocity (adjusted by an acceleraôn coefficient) with an upper limit value (fixed per tribe)

• Radius of percepôn: how far an Agent can see/sense

Herds

• Include all characteris^cs of Agents

• Area

• Compactness:

• No intelligence: inherit the collec^ve social behavior that emerges from its cons^tuent agents

AB C D A

BD

(a) single herd (b) two herds

Figure 2. (a) Agents A, B, C and D belong to the same herd even though not all agents perceive the presence of the others(e.g. agent B and D). However, for every agent pair there is a chain of agents that perceive each other (for example D isconnected to B via C). (b) If agent C is removed, then the chain is broken and thus agent D does not belong to the same herdas A and B do.

previously formed its herd, until it perceives none ofthem. As previously, during this movement, onlythe roaming movement acceleration 2 is maintained.The splitting actions may by utilized by the playeras a defense mechanism, since such an action termi-nates potential attacks by enemy predators.

Each move may be terminated before it is executed if thecontrolling player provides a new order. While a herd isperforming a certain move, it may encounter another herdof the same tribe. This will eventually merge the two herdsin a unique one, which carries the players order. If twoherds of the same tribe that have been given different or-ders meet, then the merged herd follows the orders of themost populated herd. If the herds are equally populated,then order to be followed by the merged herd is randomlyselected from the targets of the previous two herds.

The player can also perform two additional tasks whichdo not incorporate herd movement. The first action is theherd healing, where the player heals all the agents in aherd. With this action, all the herd’s agents obtain theirinitial energy level, while the energy “cost” required issubtracted from the player’s repository. The second actionis the agent resurrection, which incorporates the reappear-ance of agents that have previously demised. The energy“cost” for the resurrection of an agent equals to the totalenergy level of the agent that is about to enter the game. Ithas to be noted that the player is not allowed to “create” anagent, thus no resurrection can be performed if none of theplayer’s agents has demised.

3.3 Battle rules

The battle rules apply when a predator herd, following theorder of its controlling player, reaches the target prey herd.The winner herd is the one with the greater fight strength.The fight strength of a herd is determined by the energylevel of the agents in the herd and by their relative posi-tioning in space. The energy level of the herd, denoted byhnrg

, is the energy level sum of all its comprising agents.The positioning of the agents is measured with the herd

compactness and is calculated by

hcmp

=1

h2N

hNX

i=1

fiX

j=1

✓si

� d(i, j)

si

◆k

(1)

where hN

is the number of the herd’s agents, fi

is the num-ber of agents that agent i perceives, s

i

is the sight radius ofagent i and d(i, j) is the distance between agents i and j.The constant k can be characterized as an “overpopulationpenalty”, because large values of k result into lower com-pactness for herds with more agents. For the game setupas presented in this paper, a value k = 2 was utilized.

The energy level difference of the two herds that take partin the battle is stored in the energy repository of the playerwho controls the winning herd. The total energy level dif-ference is divided with the number of agents in the loosingherd and the resulting number is the energy portion thateach agent in the loosing herd has to “give off”. This en-ergy level is subtracted from each agent in the loosing herd,leading to demise of agents whose energy levels are van-ished by this subtraction.

3.4 The Swarm Lake conductor

The Swarm Lake conductor represents a virtual softwareentity that affects the Lake and its enclosed population hor-izontally; that is, a decision made by the conductor poten-tially refers to all competitive tribes and within a largerapplication scope and affects the progress of the perfor-mance. More specifically, the conductor role as it is out-lined by the game scenario is twofold: the application de-cides when and how much food is going to be offered inthe Lake, also defining the specific place where it is goingto be dropped. The final decision is made through a com-plex decision scheme, the analytic description of which isbeyond the scope this work. In summary, it considers themean and latest competitive activity of the tribes, as well asthe energy values appeared in the local tribes repositories.

Clearly, the volume and the time intervals density of theoffered food is a way for controlling the evolution of themusic performance. For example, provided that the con-tinuous absence of food will result into agents starving anda respective lack of energy, the game will soon come to an

Tribe

• Associated with a synth, a set of parameters and a musical scale

• Inherits these values to all member-‐agents

• One for each human player

Movements in the Lake (and human interac8on)

Roaming Mode Forced Mode

• an agent movement is based on its percepôn of the world (i.e. objects and events outside its radius of percepôn do not affect its roaming mode moves)

• 4 roaming movement rules (the first three are borrowed from the boids algorithm) that result into the separaôn of agents to several clusters (herds):

-‐ Shoaling: If a group of agents is perceived that belongs to its tribe, then it moves towards the center of mass of this group

-‐ Collision avoidance: If two same–tribe agents are too close (below a predefined distance threshold) each agent moves away from each other

-‐ Schooling: Each agent adjusts its velocity in accordance to the velociês of the same–tribe agents it perceives (i.e. it aligns its velocity with its herd mean velocity)

-‐ Enemy repulsion: Each agent moves away from the center of mass of the different–tribe agents it perceives (i.e. it is repealed by the enemy herds)

Acceleraôn Coefficient 1




Overall agents acceleraôn

Small random coefficient

Movements in the Lake (and human interac8on)

Roaming Mode Forced Mode

• incorporates player interacôn by providing orders to a herd she/he controls

• the highest velocity limit of the tribe-‐specific agents increases (extensive u^lizaôn of forced mode dispenses the agents’ energy)

• The following herd-‐targeted orders can be applied:

-‐ Move: The herd moves to the specified locaôn. Only roaming movements 2 and 4 are maintained and an acceleraôn coefficient with direcôn towards the target is provided. Completed if one of the agents reaches the specified locaôn.

-‐ Get Food: equivalent to “move” order, except that the target locaôn is a food package. The herd captures the food if one of its agents approaches below a predefined threshold.

-‐ A<ack: become a predator and move towards a selected enemy herd (the prey). Only roaming movements 2 are maintained and an acceleraôn coefficient is applied which is con^nuously poin^ng to the current posiôn of the prey herd. Completed upon a baxle end or split.

-‐ Split: move away from the center of mass of the herd, un^l none agents is perceived. Only roaming movement 2 is maintained. U^lized as a defense mechanism, since it terminates potenâl axacks by enemy predators.

Addiônal acôns

Swarm-‐oriented

Herds merging: While a herd is performing a certain (forced or not) move, it may encounter another herd of the same tribe

This will eventually merge the two herds in a unique one, which carries the players order.

-‐ If two herds that have been given different orders meet, then the merged herd follows the orders of the most populated iniâl one.

-‐ If the herds are equally populated, then order to be followed is randomly selected.

Player-‐oriented Herds healing:

heals all the agents in a herd: all the herd’s agents obtain their iniâl energy level, while the energy “cost” required is subtracted from the player’s repository

Agent resurrecôn: allows the reappearance of agents that have previously demised. The induced energy “cost” equals to the total energy level of the agent that is about to re-‐enter the game

Διαδραστική βίντεο-‐ηχητική εγκατάσταση μετατροπής του ανθρώπινου θυμού σε εικαστικό αποτέλεσμα

BaPle rules

• The winner herd is the one with the greater fight strength determined by

-‐ the energy level of the agents in the herd

-‐ their rela8ve posi8oning in the Lake expressed in terms of the compactness:

-‐ hN: the number of the agents in the herd

-‐ fi: the number of agents that agent i perceives

-‐ si: the sight radius of agent i

-‐ d(i, j): the distance between agents i and j.

-‐ constant k (the “overpopulaôn penalty”): large values of k result into lower compactness for herds with more agents (k = 2 for this work)

AB C D A

BD

(a) single herd (b) two herds

Figure 2. (a) Agents A, B, C and D belong to the same herd even though not all agents perceive the presence of the others(e.g. agent B and D). However, for every agent pair there is a chain of agents that perceive each other (for example D isconnected to B via C). (b) If agent C is removed, then the chain is broken and thus agent D does not belong to the same herdas A and B do.

previously formed its herd, until it perceives none ofthem. As previously, during this movement, onlythe roaming movement acceleration 2 is maintained.The splitting actions may by utilized by the playeras a defense mechanism, since such an action termi-nates potential attacks by enemy predators.

Each move may be terminated before it is executed if thecontrolling player provides a new order. While a herd isperforming a certain move, it may encounter another herdof the same tribe. This will eventually merge the two herdsin a unique one, which carries the players order. If twoherds of the same tribe that have been given different or-ders meet, then the merged herd follows the orders of themost populated herd. If the herds are equally populated,then order to be followed by the merged herd is randomlyselected from the targets of the previous two herds.

The player can also perform two additional tasks whichdo not incorporate herd movement. The first action is theherd healing, where the player heals all the agents in aherd. With this action, all the herd’s agents obtain theirinitial energy level, while the energy “cost” required issubtracted from the player’s repository. The second actionis the agent resurrection, which incorporates the reappear-ance of agents that have previously demised. The energy“cost” for the resurrection of an agent equals to the totalenergy level of the agent that is about to enter the game. Ithas to be noted that the player is not allowed to “create” anagent, thus no resurrection can be performed if none of theplayer’s agents has demised.

3.3 Battle rules

The battle rules apply when a predator herd, following theorder of its controlling player, reaches the target prey herd.The winner herd is the one with the greater fight strength.The fight strength of a herd is determined by the energylevel of the agents in the herd and by their relative posi-tioning in space. The energy level of the herd, denoted byhnrg

, is the energy level sum of all its comprising agents.The positioning of the agents is measured with the herd

compactness and is calculated by

hcmp

=1

h2N

hNX

i=1

fiX

j=1

✓si

� d(i, j)

si

◆k

(1)

where hN

is the number of the herd’s agents, fi

is the num-ber of agents that agent i perceives, s

i

is the sight radius ofagent i and d(i, j) is the distance between agents i and j.The constant k can be characterized as an “overpopulationpenalty”, because large values of k result into lower com-pactness for herds with more agents. For the game setupas presented in this paper, a value k = 2 was utilized.

The energy level difference of the two herds that take partin the battle is stored in the energy repository of the playerwho controls the winning herd. The total energy level dif-ference is divided with the number of agents in the loosingherd and the resulting number is the energy portion thateach agent in the loosing herd has to “give off”. This en-ergy level is subtracted from each agent in the loosing herd,leading to demise of agents whose energy levels are van-ished by this subtraction.

3.4 The Swarm Lake conductor

The Swarm Lake conductor represents a virtual softwareentity that affects the Lake and its enclosed population hor-izontally; that is, a decision made by the conductor poten-tially refers to all competitive tribes and within a largerapplication scope and affects the progress of the perfor-mance. More specifically, the conductor role as it is out-lined by the game scenario is twofold: the application de-cides when and how much food is going to be offered inthe Lake, also defining the specific place where it is goingto be dropped. The final decision is made through a com-plex decision scheme, the analytic description of which isbeyond the scope this work. In summary, it considers themean and latest competitive activity of the tribes, as well asthe energy values appeared in the local tribes repositories.

Clearly, the volume and the time intervals density of theoffered food is a way for controlling the evolution of themusic performance. For example, provided that the con-tinuous absence of food will result into agents starving anda respective lack of energy, the game will soon come to an

• The energy level difference of the herds that take part in a baxle is stored in the energy repository of the player who controls the winning herd.

• The total energy level difference is divided with hN in the loosing herd and the resul^ng energy porôn represents the amount that all loosing herd agents have to “give off”.

• This energy level is subtracted from each agent in the loosing herd, leading to demise of agents whose energy levels are vanished by this subtracôn.

The conductor

• Automated so�ware en^ty

• Affects the Lake horizontally

-‐ a decision made by the conductor potenâlly refers to all compe^^ve tribes a and affects the progress of the overall performance.

• Specific roles:

-‐ determines when, where in the Lake and how much food is going to be offered -‐ a cri^cal parameter for controlling the performance evoluôn through tribe energy starvaôn

-‐ applies a 4-‐channel mixing of the synthesized content by considering the instantaneous spaâl distribuôn of the autonomous agents within the Lake limits

• Decisions can be affected through human interacôn

• Virtual instruments are assigned to all swarm tribes

• Each tribe-‐specific instrument is realized as a Synth in SuperCollider

• An addiônal Synth is defined (i.e. the ghost Synth ) in order to sonify several acôns in The Lake

• General assumpôns:

-‐ There is no reference to real world instruments

-‐ Instruments (and Synths) are designed with completely different parameters

-‐ All Synths are programmed to produce simple but dis^nguishable ^mbres that recall the epoch of the early generaôn of video games

Music Composiôn Algorithm (1)

end and the music synthesis will stop. Thus, the decisionsof the conductor can be regarded as a means for codingmusic score instructions that defines the length of the mu-sic track composed. In a future Swarm Lake version, thisartificial entity can obviously be replaced by a human con-ductor. Moreover, the second conductor responsibility isto apply the 4-channel mixing of the synthesized contentby considering the instantaneous spatial distribution of theautonomous agents within the Lake limits. More details onthis issue are provided next in Section 4.3.

4. MUSIC COMPOSITION ALGORITHM

As it was mentioned at the beginning of the paper, soundsynthesis is performed by assigning virtual instruments toeach tribe. Each tribe-specific instrument is realized asa software SuperCollider 1 synthesizer (Synth), in termsof unit generators (UGens). This pure sound synthesisapproach allows the production of the final sonic output,without the need of buffering prerecorded sound samples.Apart from the four Synths that are mapped to each tribe,Swarm Lake also employs a fifth semi-independent Synthelement in order to represent the Swarm Lake world ac-tion and to fulfill sound synthesis conceptually, spatiallyand spectrally. We hereby refer to this synth as the ghostSynth. The rationale behind the definition of the abovesynthesizers is the realization of tribe-specific instrumentswith no actual reference to any real ones. All Synthsare designed with different parameters, in order to achieveunique sonic characteristics per tribe, thus achieving an in-teresting mix of sounds. This rather arbitrary design pro-cess is necessary for creating meaningful and organizedtribe-specific sonic patterns, which fulfill fundamental hu-man subjective expectations through providing an overall,robust and aesthetic sonic output.

4.1 Primary synthesis modules

Each tribe–specific synthesizer (denoted here as SynthA,SynthB , SynthC and SynthD respectively) is programmedto produce simple but distinguishable timbres that recallthe epoch of the early generations of video games. Theimplementation of each of the Synths included:

1. The definition of a function that analytically describesthe UGens employed within the specific Synth, aswell as their interconnections (such as nesting, addi-tion or multiplication).

2. The realization of multiple signal processing units(i.e. compressors, limiters and expanders) which pro-vide more balanced sound under unpredictable orextreme amplitude changes that may occur duringthe game play.

3. The definition of specific UGen arguments as inter-action parameters, aiming to offer access and controlto the respective generator tones (pitch), gain andADSR envelopes duration, harmonics and phase off-sets. These parameters depend on the particular typeof a specific UGen.

1 http://sourceforge.net/projects/supercollider/?source=directory

Following are the details of each of the Synths realizedfor the purposes of this work.SynthA consists of a series (i.e. an array) of sinus oscil-

lators for creating a plurality of harmonic content. Signalamplitude is modulated using a saw oscillator combinedwith a low–pass filtering unit. A second sinusoidal signalis added to the previous one, which is phase–modulated us-ing a saw oscillator too. Thus, the saw oscillator frequencybecomes the frequency of the phase modulation, and it’samplitude the range of modulation. The above SynthA

functionality can be mathematically modeled as:

SynthA = (Sin1 + Sin2 . . .+ SinV

) ⇤ Saw+Sin(phase : Saw) + Sin (2)

where the last term is added in order to control the plu-rality of the produced sound harmonic content followingthe variations of the compactness defined in Eq.( 1) (seealso Section 4.2 for more details). It should be noted thatthe same control task is performed by the last term of theequations following that describe the rest Synths.SynthB incorporates a band limited impulse oscillator,

whose harmonics are modulated by a sinus modulator. Thederived signal is added with a sinusoidal and a sawtoothone as:

SynthB = bl Imp(harms : Sin) + Sin+ Saw (3)

SynthC is defined by a low–pass filter pulse oscillator,a low–pass filter saw wave oscillator with a sinus oscilla-tor being it’s initial phase offset modulator, a sinusoidaloscillator as a phase modulator, a quadratic noise genera-tor (QNG) and finally an extra generator being a bank ofresonators fed by an impulse generator:

SynthC = (LFPulse+ LFSaw(iphase : Sin)

+Sin(phase : sin) +QNG+Reson(Impulse) (4)

On the other hand, SynthD is modeled as

SynthD = RLPF (pulse) +RHPF (sin)

+Ring(impulse) + impulse(phase : sin) (5)

being the combination of a low–pass filter resonator (withcut–off frequency at tone⇤1) with input a pulse wave gen-erator, a high–pass resonant filter (with cut–off frequencyat tone ⇤ 2) fed by a sinus oscillator, a ring modulator withinput an impulse generator and finally, an impulse oscilla-tor modulated by phase using a sinusoidal oscillator.

Finally, the GhostSynth consists of an 8-input resonantfilter fed by an equal number of sinusoidal oscillators, aring modulator driven by a low–pass frequency saw waveoscillator and a chaos generator (i.e. a feedback sinus os-cillator with chaotic phase indexing). The output of thechaos generator is particularly amplified when the tribesare in war. This Synth is modeled as:

GhostSynth = Reson(Sin1 + Sin2 . . .+ Sin8)

+Ring(Sin(LFSaw)) + Chaos (6)













































































































• Each Synth is controlled by a synth-‐performer so�ware rou^ne

-‐ the API between Swarm Lake events and the sound engine

• Synths addiônally include:

-‐ Realizaôns of mul^ple signal processing units (i.e. compressor, limiters and expanders) which provide more balanced sound under unpredictable or extreme amplitude changes

-‐ The definiôn of specific UGen arguments as interacôn parameters (access and control respec^ve generator tones -‐ pitch, gain, ADSR duraôns, harmonics and phase off-‐sets).

Music Composiôn Algorithm (2)

Tempo & rhythm

Sound dynamics

Melodic content

Music parameters

Tempo is pre-defined and fixed during a performance session

Rhythm is defined in real–time, following the evolution of the game/performance:

During a game session, three different rhythmic changes can occur on the music measure M:

1. Initial rhythmic pattern (applied upon starting a new game): M is divided in four beats ([τ1, τ2, τ3, τ4]) (τ denotes single note durations).

2. If a tribe dies, M is further divided into two more beats (i.e. [τ1, τ2, τ3, τ4, τ5, τ6] for the first dead tribe, [τ1, τ2, τ3, τ4, τ5, τ6, τ7, τ8] for the second one, etc).

3. When the next to last tribe dies, the game ends and music synthesis stops (M ->0).

Every time a new rhythmic section is about to be applied, a new fixed array of durations is sent to each synth–performer.

All arrays have the same sum of durations that follows the applied rhythmic pattern (and the synth-performers are in sync!)

However, each synth–performer plays a different sequence of the globally defined durations:

In order to efficiently follow the performance progress,each Synth is controlled by one synth–performer, a soft-ware routine that triggers the initiation of sound eventswith specific sound parameters, following a pre-definedmap of interactions. Thus, these synth–performers rep-resent the application interface between the Swarm Lakeworld events and the sound engine. A detailed descriptionof the sound parameters and the considered map of inter-actions is provided next.

4.2 Music parameters configuration

Tempo is pre-defined and fixed during a performance ses-sion. On the other hand, rhythm is defined in real–time, fol-lowing the evolution of the game. Each synth–performerhandles a sequence of notes with different durations. Forexample, when a tribe dies, a different global rhythm is ap-plied, aiming to deliver a dramatic character on the gameplay, mainly through dividing the music measure into shorternote durations. During a game session, three different rhyth-mic changes can occur:

1. Initial rhythmic pattern applied upon starting a newgame: the music measure M is divided in four beats([⌧1, ⌧2, ⌧3, ⌧4]), where ⌧ denotes single note dura-tions.

2. As it was mentioned previously, upon death of a tribethe music measure M is further divided into twomore beats (i.e. [⌧1, ⌧2, ⌧3, ⌧4, ⌧5, ⌧6] for the firstdead tribe, [⌧1, ⌧2, ⌧3, ⌧4, ⌧5, ⌧6, ⌧7, ⌧8] for the sec-ond one, etc).

3. When the next to last tribe dies, the game ends andmusic synthesis stops (i.e. M = 0).

Every time a new rhythmic section is about to be ap-plied, a new fixed array of durations is sent to each synth–performer. All arrays have the same sum of durations thatfollows the applied rhythmic pattern; however, each synth–performer plays a different sequence of the globally de-fined durations, for example:

• SynthA ! [⌧1, ⌧2, ⌧3, ⌧4]

• SynthB ! [⌧3, ⌧1, ⌧4, ⌧3]

• SynthC ! [⌧1, ⌧3, ⌧2, ⌧4]

• SynthD ! [⌧4, ⌧2, ⌧1, ⌧3]

while the GhostSynth always plays one note within a sin-gle music measure (i.e. GhostSynth ! M ). Follow-ing the above rhythm description scheme, we ensure thatall synth–performers are synchronized at the end of eachrhythmic pattern.

Focusing on the produced sound dynamics, these are de-fined and controlled in a per tribe basis through appropri-ately shaped ADSR envelopes. Specifically, each synth–performer is assigned a specific ADSR curve, with A, D,S and R durations and gain values being varied in a rangethat is unique for the specific tribe, thus providing a dis-tinct dynamic character of the sonic output produced by

each one. The ADSR gains are analogously mapped to theaverage herd strength value for a specific tribe (as it wasdefined previously in Section 3.3), while the correspond-ing ADSR durations are directly associated with the meanherds energy h

nrg

. By following this mapping approach,high sound dynamics are produced by tribes with excellentphysical condition. It should be also noted that the ADSRenvelope for the GhostSynth remains constant during aspecific game period. Moreover, the overall GhostSynthreproduction level depends on the number of the dead tribes.Initially, the GhostSynth is not audible, since all tribesare alive, but its level is increasing as the competition be-tween the game tribes evolves.

The melodic content of the produced sound is continu-ously determined by the tribe that is dominant in terms ofstrength. Each time a tribe becomes a dominant one, allthe other ones are enforced to follow the melodic charac-ter assigned to the leader tribe, as it is summarized in Ta-ble 1. Technically, this enforcement is applied by sendingspecific arrays of notes to the particular synth–performers.The array of the leader tribe has always double length. Forexample, if tribe D is assumed to be dominant, music syn-thesis will be based on the harmonic minor: tribe D synth–performer will play the complete musical scale, while therest will be randomly playing the first, third, fifth and sev-enth scale note. Hence, the leader tribe is perceived as if itexecutes a music solo session, while the rest bondservanttribes are musically following it. A specific note (e.g. C)is defined per game session as the root of the reproducedscale. Obviously, if there is no leading tribe alternation,then the scale of notes no longer changes and no new ar-rays of notes are generated. This condition results into alack of melodic changes under decreased tribe competitionconditions.

Tribe synth Assigned melody typeSynthA Melodic MajorSynthB Major PentatonicSynthC Melodic MinorSynthD Harmonic Minor

Table 1. Melodic assignments to the Swarm Lake tribes.

Changing note octaves depends on the average of the ab-solute velocity magnitude of each tribe. For example, if aspecific herd is about to attack, it will increase the averagespeed magnitude for the specific tribe. In this case, the re-produced notes for this specific tribe will be selected froma higher octave. Instead, if the tribes are moving leisurelywithin the Lake, then the sonic output will have a low fre-quency character. Moreover, the harmonic content distri-bution of the sound produced by each tribe is controlledby its compactness h

cmp

: the more compact is the popula-tion of one tribe, the richer is the harmonic content of themusic outcome, thus partially affecting the correspondingtimbre characteristics. This extent of harmonic content isprogrammatically controlled through directly mapping themeasured compactness to the amplitude of the last termsof Eqs.( 2)–( 5) presented in Section 4.1.

Sound dynamics

• Tribe-‐specific dynamics control is characterized by

-‐ Uniqueness: each synth–performer is assigned a specific ADSR curve, with A, D, S, R duraôns and gain values being varied in a range that is unique for the specific tribe

-‐ Adapta9on: ADSR gains are analogously mapped to the average herd strength value for a specific tribe, while ADSR duraôns are directly associated with the mean herds energy

• Through this mapping approach, high sound dynamics are produced by tribes with excellent physical condiôn

• Ghost-‐synth dynamics:

-‐ the ADSR envelope remains constant

-‐ the overall GhostSynth reproducôn level increases with the number of the dead tribes (iniâlly, the GhostSynth is not audible).

Melodic content

• It is con^nuously determined by the tribe that is dominant in strength

-‐ all other tribes are enforced to follow the melodic character assigned to the leader one

-‐ the leader tribe is perceived as if it executes a music solo session, while the rest bondservant tribes are musically following it

-‐ Decreased tribe compe^ôn (i.e. no leading tribe alternaôn), introduces a lack of melodic changes.

-‐ a specific note (e.g. C) is defined per game session as the root of the reproduced scale

• Example:

-‐ if tribe D is assumed to be dominant, music synthesis will be based on the harmonic minor: tribe D synth–performer will play the complete musical scale, while the rest will be randomly playing the 1st, 3rd, 5th and 7th scale note.

In order to efficiently follow the performance progress,each Synth is controlled by one synth–performer, a soft-ware routine that triggers the initiation of sound eventswith specific sound parameters, following a pre-definedmap of interactions. Thus, these synth–performers rep-resent the application interface between the Swarm Lakeworld events and the sound engine. A detailed descriptionof the sound parameters and the considered map of inter-actions is provided next.

4.2 Music parameters configuration

Tempo is pre-defined and fixed during a performance ses-sion. On the other hand, rhythm is defined in real–time, fol-lowing the evolution of the game. Each synth–performerhandles a sequence of notes with different durations. Forexample, when a tribe dies, a different global rhythm is ap-plied, aiming to deliver a dramatic character on the gameplay, mainly through dividing the music measure into shorternote durations. During a game session, three different rhyth-mic changes can occur:

1. Initial rhythmic pattern applied upon starting a newgame: the music measure M is divided in four beats([⌧1, ⌧2, ⌧3, ⌧4]), where ⌧ denotes single note dura-tions.

2. As it was mentioned previously, upon death of a tribethe music measure M is further divided into twomore beats (i.e. [⌧1, ⌧2, ⌧3, ⌧4, ⌧5, ⌧6] for the firstdead tribe, [⌧1, ⌧2, ⌧3, ⌧4, ⌧5, ⌧6, ⌧7, ⌧8] for the sec-ond one, etc).

3. When the next to last tribe dies, the game ends andmusic synthesis stops (i.e. M = 0).

Every time a new rhythmic section is about to be ap-plied, a new fixed array of durations is sent to each synth–performer. All arrays have the same sum of durations thatfollows the applied rhythmic pattern; however, each synth–performer plays a different sequence of the globally de-fined durations, for example:

• SynthA ! [⌧1, ⌧2, ⌧3, ⌧4]

• SynthB ! [⌧3, ⌧1, ⌧4, ⌧3]

• SynthC ! [⌧1, ⌧3, ⌧2, ⌧4]

• SynthD ! [⌧4, ⌧2, ⌧1, ⌧3]

while the GhostSynth always plays one note within a sin-gle music measure (i.e. GhostSynth ! M ). Follow-ing the above rhythm description scheme, we ensure thatall synth–performers are synchronized at the end of eachrhythmic pattern.

Focusing on the produced sound dynamics, these are de-fined and controlled in a per tribe basis through appropri-ately shaped ADSR envelopes. Specifically, each synth–performer is assigned a specific ADSR curve, with A, D,S and R durations and gain values being varied in a rangethat is unique for the specific tribe, thus providing a dis-tinct dynamic character of the sonic output produced by

each one. The ADSR gains are analogously mapped to theaverage herd strength value for a specific tribe (as it wasdefined previously in Section 3.3), while the correspond-ing ADSR durations are directly associated with the meanherds energy h

nrg

. By following this mapping approach,high sound dynamics are produced by tribes with excellentphysical condition. It should be also noted that the ADSRenvelope for the GhostSynth remains constant during aspecific game period. Moreover, the overall GhostSynthreproduction level depends on the number of the dead tribes.Initially, the GhostSynth is not audible, since all tribesare alive, but its level is increasing as the competition be-tween the game tribes evolves.

The melodic content of the produced sound is continu-ously determined by the tribe that is dominant in terms ofstrength. Each time a tribe becomes a dominant one, allthe other ones are enforced to follow the melodic charac-ter assigned to the leader tribe, as it is summarized in Ta-ble 1. Technically, this enforcement is applied by sendingspecific arrays of notes to the particular synth–performers.The array of the leader tribe has always double length. Forexample, if tribe D is assumed to be dominant, music syn-thesis will be based on the harmonic minor: tribe D synth–performer will play the complete musical scale, while therest will be randomly playing the first, third, fifth and sev-enth scale note. Hence, the leader tribe is perceived as if itexecutes a music solo session, while the rest bondservanttribes are musically following it. A specific note (e.g. C)is defined per game session as the root of the reproducedscale. Obviously, if there is no leading tribe alternation,then the scale of notes no longer changes and no new ar-rays of notes are generated. This condition results into alack of melodic changes under decreased tribe competitionconditions.

Tribe synth Assigned melody typeSynthA Melodic MajorSynthB Major PentatonicSynthC Melodic MinorSynthD Harmonic Minor

Table 1. Melodic assignments to the Swarm Lake tribes.

Changing note octaves depends on the average of the ab-solute velocity magnitude of each tribe. For example, if aspecific herd is about to attack, it will increase the averagespeed magnitude for the specific tribe. In this case, the re-produced notes for this specific tribe will be selected froma higher octave. Instead, if the tribes are moving leisurelywithin the Lake, then the sonic output will have a low fre-quency character. Moreover, the harmonic content distri-bution of the sound produced by each tribe is controlledby its compactness h

cmp

: the more compact is the popula-tion of one tribe, the richer is the harmonic content of themusic outcome, thus partially affecting the correspondingtimbre characteristics. This extent of harmonic content isprogrammatically controlled through directly mapping themeasured compactness to the amplitude of the last termsof Eqs.( 2)–( 5) presented in Section 4.1.

Tribe-‐specific melodic assignments

Final output rendering

• Mul^channel (4-‐channel) audio stream

• Employment of a four channel equal power panning law realized on the conductor applicaôn side

-‐ the real conductor parallelism: adjusts the overall balance of the various instruments or voices within the music ensemble.

• Technical details:

-‐ the panning law applied follows the tribe posiôn calculated as the average posiôn among the overall amount of a tribe agents

-‐ the GhostSynth posiôn is calculated as the symmetrically opposite point that corresponds to the average posiôn of the four tribes

-‐ in a conceptual level, the GhostSynth acous^cally exists where the evidences of life are limited (while it is strengthen by death)

Swarm Lake -‐ Demo

Swarm Lake Overview

Mul8player game Collabora8ve music making environment

Bio-‐inspired Intelligence Algorithms

Game-‐based scenario User Interac8on

Conclusion

s• A first evaluaôn axempt

-‐ the complexity of mapping between the social, swarm behavior rules and the highly parameterized music synthesis engine results into perceptually consistent and interes^ng forms of sound, in both prac^cal and conceptual levels

-‐ several correspondences between the game group of music–makers and a real-‐work music ensemble are high-‐lighted

-‐ the increased tribe ac^vity and compe^ôn towards the ul^mate survival aim results into an evoluônary music structure free from repe^ôn ar^facts and sta^c characteris^cs

• Some future direcôns

-‐ the involvement of human interacôn in the conductor app

-‐ arbitrary number of tribes support,

-‐ each performer can re-‐define the inherent properês and parameters of its own Synth engine

-‐ emoônally driven behavioral and sound synthesis enhancements in the ar^ficial populaôns

Swarm Lake … to be demonstrated at the 2015 audiovisual

Arts Fes8valMay -‐ June 2015, Corfu, Greece

Any Ques8ons?

Thank you for your kind aPen8on!

Ανδρέας Φλώρος [email protected]

hxp://www.ionio.gr/~floros

Andreas Floros [email protected]

hxp://www.ionio.gr/~floros

SwarmLake Presentation

Technology