Framework for Live Algorithms Tim Blackwell Michael Young Goldsmiths College, London Organised Sound 9(2): 123–136.

Framework for Live Algorithms

Tim Blackwell

Michael Young

Goldsmiths College, London

www.timblackwell.com

Organised Sound 9(2): 123–136

The contextConsider (for now, because it’s easier) improvised performance

i.e. free of compositional directives such as harmony, rhythm, form…

Conventionally such improvisations are explorations of timbre and texture

The contextConsider (for now, because it’s easier) improvised performance

i.e. free of compositional directives such as harmony, rhythm, form…

Conventionally such improvisations are explorations of timbre and texture

What is a Live Algorithm?

A Live Algorithm is

Interactive

Autonomous

Ideas generator

Idiosyncratic

Comprehensible

InteractiveSympathetic, supportive, makes appropriate contributions, tacets

AutonomousNot merely automatic, mechanical and predictable. Must be comprehensibly interactive and capable of novelty

Ideas machineContradictory, individualisticsource of novelty and leadership

Idiosyncratic

Contributions limited by instrument and experience but can be unusual, individualistic

Comprehensible

Suitable for collaboration, at least by humans. Not opaque, but not too transparent either

And, importantly…

Closure

…knows when to stop!

Principles

Expectation of generating form might not be necessary

Local events can be structuringFor example, spatio-temporal self-organisation depends only on local interactions of a certain complexity and on positive and negative feedback.

Here, the space is (interpreted as) a musical/sonic parameter space at some level.

Parameterisations possible at micro (sample, grain), mini (note) and meso (phrase) levels. The problem of emergence might necessitate a need a multi-level system

Knowledge of music rules might not be necessary

LA need not be aware of what it is doing

In the model, LAs and humans interact with meaningless sounds, populating an inert environment.This builds on a former XY model, which expresses the paradox of interaction

(In our nature-inspired systems, The sounds are organised by stigmergy)

Knowledge of music rules might not be necessary

LA need not be aware of what it is doing

Desirability of an Interactive Model and a Conceptual ArchitectureIn the model, LAs and humans interact with meaningless sounds, populating an inert environment.This builds on a former XY model, which expresses the paradox of interaction

(In our nature-inspired systems, The sounds are organised by stigmergy)

The XY model

The model

A B

Conceptual (and Actual?) Architecture

P: Listening, analysingTypically thins degrees of freedom; from real timeIN -> p(t) -> pi

F(p): Ideas enginePatterning in a hidden space H; generative/algorithmic/iterative xi -> xi+1

Q: Interpretation, Playing, synthesizingTypically expands d.o.f.; into real timexi+1 -> q(t) -> OUT

PfQ Architecture

Analysisis typically projection from one level to another, higher level. E.g. samples to event parameters pP: (t) -> p(t) -> pi

We observe that this projection is determined by cultural, personal, genre-specific and even political forces

P might be a map into H and hence hidden state x is a possible parameterisation of the sonic environment.

Interpretationis the process of relating internal states to event parameterisations, and eventually to soundQ: xi+1 -> q(t) -> OUT

A technical complication surrounds the relationship between real and algorithmic time

It is unlikely that both will flow at the same rate: the computational update time xi -> xi+1 might not correspond with the desired time interval between events (t), (t+t...

Internal states might be sampled at a given rate irrespective of iterative time or timing information could even derive from x itself

Information might be derived from averages over a population {x1, x2, x3, ...}

All these complications and possibilities are represented by a single interpretative function Q

Ideas generatorF is parameterised by p and determines state flow of hidden variables xF(p): xi -> xi+1

xi+1 determined F, p(t) and xi

F need not derive from any musical concern, and this may be advantageous

F only concerns patterning and structure

Autonomous 1. x ≠ 0 even if p = 0

2. x = 0 even if p ≠ 0

1. This is usual, expresses state flow in a dynamical system

2. Harder to achieve, but could arrange for Q( x[x-dx, x+dx] ) = q.Alternatively, include dissipation and re-energising so that x 0 between injections of energy (determined by p?)

Idiosyncratic Q and P concern the mappings to and from sound – herein lies the machine's idiosyncrasies and characteristics

Can be quite limited, since interactivity and autonomy are the major prerequisites

Potentially P, Q may cross levels

Interactive autonomy + modification of system state

Analysis parameters p ensure interaction i.e. affects, but does not fully determine, x

For example, p might be an attractor in H. In a dynamic system, x orbits p, but precise trajectory depends on initial conditions

Alternatively, hidden variables x can be regarded as parameters which select mappings Fx: p -> q

The collaborative interface is the asynchronous mapQ Fx P = Mx

OUT(t+t) = Mx IN(t)

Comprehensible:Transparency > Comprehensibilty > Opacity

Very transparent if p H, Q = P-1 and dim(H) is small enough. Easy to establish correlations between human (incoming sound, deposited in the environment by a human) and LA (outgoing sound deposited by the LA). Might become too predictable unless enhanced by some stochastic adjustment (a degenerate solution)

Very opaque if QFP is very complex and dim(H) is big. Very hard to establish correlations or any measure of causal relations between human and LA. Too unpredictable and therefore not collaborative

Closure:Raises issues of knowledge of form

Could argue that LA should not be deprived of knowing the modus operandi of the performance

In humans, closure is influenced by time constraints, visual cues and an increasing capacity to interpret gestures as possible sonic cadences

However, in an LA closure might possibly emerge from lower level dynamics e.g. self-organised criticality

Other desirable properties

MemoryDynamical states do not conventionally posses this

Short term (repetitions, anticipations based on recognition..) >>> attractor persistence, pheromone trailsE.g. Swarm Techtiles, Ant Colony simulations

Long term (draws on previous musical engagements) >>> current states could have memories of previous states… E.g. Particle Swarms (particles have a memory and participate in social networks)…AND the sequence of conditions that caused these states (a contextual memory)E.g. ??

Links with existing research fields:

P: -> pReal-time music analysis and informatics

Q: x -> Real-time audio synthesis

FGenerative Music (Neural, Cellular Autonoma, Genetic Algorithms, Swarms

Links with existing paradigms

Live Electronics:ideas engine F replaced by human volition. State flow is adjustment of “controls” x

Live CodingF = I and Q is adjusted (in software) manually

Generative/algorithmic music:Set p = 0 (turn P off)(t) = F(xi) = FN( x0 )

The system is self-contained - the composer chooses F and the initial condition, x(0)

Reactive SystemsIn a reactive system, IN will necessarily trigger certain transitions xi -> xi+1

Mx = QFP express a causal and necessary chain – a rule-based system

It is conjectured that a reactive system might not be sufficient for a Live Algorithm because it is not sufficiently interactive

This touches on various issues in cognitive science and machine intelligence

Possibly, a large enough rule set might do the job, and might also be indistinguishable from a discrete dynamical system (i.e. an algorithmic model of a continuous DS)

It is hard to see how we might achieve this complexity without drawing on inspiration from dynamical, and other natural, systems

(Our) Experience

Swarm Music:

Very transparent, P and Q operate at the note level

Swarm Granulator:

Less transparent, functions largely at the grain level, although some level-crossing

Swarm Tech-tilesP actually expands into a 2D landscape

F integrates ALife and optimisation Q is a rendering of parts of the landscape visited by x

ArgrophyllaxP: FFT analysisf: stochastic functionH: Fourier spaceQ: coefficients of inverse FFT transform Q

Evaluation Irrelevance of Turing Test? TT is more relevant for the performers than the listeners. A metric: do the performers find the LA to be stucturing?

Applicability: The attributes of a LA should be useful in other musical contextsE.g. intelligent effects pedal, synth plug-ins, accompaniment programs, genre improvisation