-
Evolutionary Music Composition
Martin Dostal
Abstract. Evolutionary music composition refers to machine-based
generation ofmusical pieces by means of evolutionary computation
techniques. In this chap-ter we discuss machine-based music
composition from several viewpoints. First,techniques used to
machine-based music composition are reviewed. Second, mu-sic
composition subtasks, such as generating melodies, harmonization of
musicalphrases and generating rhythm patterns or composition of a
rhythm accompanimentare introduced. Third, two distinct approaches
to the evaluation of generated mu-sic are discussed: interactive
evaluation based on human mentors judgement andautonomous
evaluation of generated musical material by the system itself. The
restof the chapter describes six recognized evolutionary systems
for music composi-tion in detail. In particular, the description is
focused to the design of evolutionaryalgorithms behind these
systems.
1 Introduction
Evolutionary techniques have attained high popularity in machine
learning and opti-mization. These techniques have a wide body of
applications including less conven-tional areas such as
machine-based creating of (art)works. However, the term art-work
may seem as overstatement, depending on your definition on what an
artworkreally is, so we can use rather the term machine creativity
for machine generatedpaintings, music or poetry.
In this book chapter we discuss machine-based generation of
music using evo-lutionary techniques. We assume that the reader is
at least somewhat familiar withmusic, not necessarily as composer
of performer, but at least as a consumer. Alsosome familiarity with
evolutionary techniques is expected.
Martin DostalDept. Computer Science, Palacky University Olomouc,
17. Listopadu 12,77146 OLOMOUC, Czech Republice-mail:
[email protected]
I. Zelinka et al. (Eds.): Handbook of Optimization, ISRL 38, pp.
935964.springerlink.com c Springer-Verlag Berlin Heidelberg
2013
-
936 M. Dostal
Although it may seem unmusical or somewhat provocative, music
compositioncan also be understood as an optimization process. The
human composer searchesin the space of all possible music
compositions that he is able to compose, for onesuch that satisfies
his own artistic or other criteria, such as a style, length or
instru-mentation, for instance. Technically speaking, something
like input constrains canbe more visible in performing applied
arts; the artist works with some requirementsand constraints
specified by the client and the artist is expected to create such
an(art)work that meets the requirements. In particular,
evolutionary techniques areused in music composition in such a way,
that is, as a mechanism for creating orvariating musical pieces
according to some specified criteria.
The chapter is organized as follows. First, music composition
subtasks, such ascomposing melodic material, harmonization, or
generating rhythm accompanimentwill be introduced and a brief
survey of previous work in these fields will be pro-vided. Next
section discusses commonly used techniques in machine-based
musiccomposition. The following section discusses two distinct
approaches used in cre-ative systems, namely interactive and
autonomous systems. Remaining sections willintroduce some
recognized evolutionary systems for music composition in
detail.Used representation of musical knowledge, the design of
evolutionary algorithmand recombination operators will be
discussed. In melodic development, GP-Music,GenDash and GenJam will
be described. The following section will describe Phon-Amnuaisuks
system for harmonization. In generating rhythm accompaniment
twosystems, namely CONGA and GeneticDrummer, will be described.
1.1 Brief HistoryHuman composers use very different approaches
to music composition: from ratherformal, utilizing various musical
knowledge and compositional theories, to intuitive,rehearsal-based
composition of music. Not surprisingly, the techniques
resemblingalgorithmic composition had been used yet prior to the
rise of computers. For in-stance, Mozart proposed Musikalisches
Wurfelspiel (Musical Dice Game), a sys-tem where a dice was used to
random selection of short musical patterns that werecombined
together [1]. Some other composers used choosing of random
numbersin assembling their compositions, such as Johann Philipp
Kirnberger in The EverReady Composer of Polonaises and Minuets
(1757, 1783) or Joseph Haydn in thePhilharmonic Joke (1790).
Reportedly, Mozart, Bach and Bartok used Fibonaccinumbers and
golden section [36] in their compositions. However, it is
somewhatquestionable whether the mathematical background was used
on purpose or ratherintuitively by these composers.
A considerable growth of interest into computer-based
composition (also includ-ing other areas of arts) has begun in late
80-ties of 20. century along with thewidespread popularity of
machine learning and optimization techniques,
especiallyknowledge-based systems, neural networks and genetic
algorithms. One of the firstsuccessful attempts to utilize a
computer in music composition was the Illiac Suiteby Hiller,
Isaacson and Baker [1]. The computer was used to imitate the
composition
-
Evolutionary Music Composition 937
process in a somewhat similar way to the later used evolutionary
techniques. IannisXenakis, a Romanian-born Greek composer, was one
of the pioneers of algorithmiccomposition. He used stochastic
techniques in music composition and developed acomputer program
that helped him in composition [53, 54].
1.2 Music Composition Subtasks
In this section we describe the main application areas of music
composition systemsand provide a brief survey on previous work.
1.2.1 Melody (Motivic) CompositionAutomatic composition of
musical pieces is apparently one of the most challeng-ing problems
in machine art. In past twenty years several systems were
developedand some of them also successfully used in the composition
of true musical pieces(in contrast to experimental, or toy-like
nature of some other systems) or live per-formances. Rodney Waschka
II, an american composer known for usage of algo-rithmic
composition methods, created a composition tool called GenDash [49,
50].GenDash creates variations of musical motives using a genetic
algorithm. Waschkais one of composers that used his system
successfully in publicly performed com-positions including Saint
Ambrose, Sapphos Breath (both chamber operas) [51]or String
Quartet: Ha! Fortune or Eclogues for Woodwind Quintet [50]. We
descibeGenDash in more detail in Section 3. Jacob [31, 32] also
experimented with creatingvariations of melodic motives using
genetic algorithm. Jacobs system also consid-ered an ear module for
listening and evaluating composed melodies in contrastto Waschkas
system which omitted the fitness evaluation at all. GaMusic [39]
wasa system for composing of two-octave melodies on the basis of
interactive geneticalgorithm. Another attempt to evolve melodies
using interactive genetic algorithmhas also been made by Ralley
[40].
Johanson and Poli created the GP-Music system that evolves short
melodic se-quences according to both mentors preferences and
autonomous measures by thesystem itself [33]. GP-Music is a good
example of a toy-like experimental system.The system is based on
genetic programming and we provide detailed descriptionof the
system in Section 2.
Spector and Alpern [42, 43] proposed the GenBebop; a system for
composingshort jazz improvisations, the so called trading fours.
Trading fours refers to four-bar improvisations on a reply to
recently played four bars by another player. How-ever, generating
bebop improvisations is only a particular application of Spector
andAlperns approach. Their aim was more general: they attempted at
a system whichwould enable a construction of artificial artists
with some aesthetic tradition and par-ticular musical preferences.
Similar application area has also been investigated byBiles who
developed GenJam [8, 10, 11, 13]. It is one of the most referred
evolution-ary composition systems so far. GenJam is an interactive
genetic algorithm-basedsystem for trading-fours improvisation. The
system is, however, able to improvisein real time, thus Biles used
GenJam successfully at a lot of live performances with
-
938 M. Dostal
his Virtual Quintet, which is a combo of John A. Biles on
trumpet or flugelhorn andGenJam on tenor sax and other instruments.
GenJam also participated on severalBiles music records with the
Virtual Quintet.
Thywissen [44] introduced GeNotator, a hybrid-algorithmic
composition tool forcomputer assisted composition of music. Gibson
and Byrne [21] designed a sys-tem for composing short musical
pieces using diatonic, four-part Western harmony.The system has
been called NEUROGEN and utilized a genetic algorithm to com-pose
music and neural networks to evaluate the quality of generated
results. Unemi[47, 48] created SBEAT, an interactive system for
breeding short musical phrases(e.g., rhythm patterns) in terms of
musical patterns with various rhythm, pitch andvelocity.
A distinctive part of music composition has been investigated by
Horner andGoldberg who developed a system for thematic bridging
[29]. Thematic bridging isa process that lies in transforming one
musical part into another musical part overan interval of time.
1.2.2 Harmonization
Several researchers aimed at automated harmonization of musical
phrases. Ebicoglu[20] developed CHORAL, a system for the
harmonization of chorales in the styleof J.S.Bach. Phon-Amnuaisuk
et al. developed a system for the four part (soprano,alto, tenor,
bass) harmonization of user-specified melodies [5, 6]. The system
wasbased on genetic algorithm and supported also an autonomous
evaluation of gen-erated harmonies. We describe this approach in
more detail in Section 5. Authorswere also interested in a
comparison of methods that could be used to compose har-monic
accompaniment. They have proposed a different version of the
system; it wasbased on a rule-based inference system instead of
genetic algorithm [6]. Accordingto experimental results, the
authors concluded that the rule-based implementationproduced better
results than the genetic algorithm-based one.
Horner and Ayers [28] have developed a system that composes a
harmonic pro-gression using genetic algorithm using a rule-based
approach. The rules are usedas constraints upon the search space of
possible harmonies. The constraint rules areof two types: first
type defines how individual chords can be voiced and the secondtype
of rules defines how voices are allowed to change from a chord to
next one[15]. The process of harmonization is executed
consecutively for each of the groupof constraint rules. McIntyre
[38] also used genetic algorithm to harmonization. Hissystem
generates chord sequences according to the rules of how chords
should bespelled and it is also used to autonomous evaluation of
generated harmonies.
Hoover et al. [27] have proposed a system for harmonization of
existing musi-cal phrases called scaffolds. The system produces
harmonic variations and rhythmchanges using a combination of
evolutionary computation and neural networks. Theneural network is
responsible for the scaffold transformation in terms of used
pitchesand rhythm.
Issues related to machine harmonization of music had been
critically discussedby Wiggins [52]. Wiggins discusses various
machine learning techniques used to
-
Evolutionary Music Composition 939
harmonization and points-out that the task of harmonization is
not just a matterof filling of the notes according to the rules of
harmonization but also creating amusical structure with a musical
intention behind. The latter point is, however, nottaken into
account in most systems that perform harmonization. In fact, a
local per-forming of algorithmic composition systems (e.g.,
typically on the current measureonly) without a broader context,
such as planning a structure of the composition,is the property of
most current systems. For instance, GeneticDrummer is such alocally
performing system. It generates a phrase with rhythm accompaniment
byindividual measures. During the generating process the system
does not see the pre-ceding nor following measures so that the
composition process is entirely local. Theconsistence of resultant
phrase is, however, ensured by the application of the
sameconstrains (specified by the fitness function) on each measure
present in the phrase.
1.2.3 Rhythm
Horowitz [30] used interactive genetic algorithm for composing
short rhythm pat-terns. The system used also some fitness criteria
(e.g., syncopation, density, down-beat, beat repetition) that were
derived from the mentors evaluation. Tokui and Ibadeveloped a
system called CONGA [48], an evolutionary drum machine that
evolvesmulti-timbre rhythm patterns. CONGA is an interactive system
extended with a neu-ral network which performs as an evaluation
assistant. The system combines geneticalgorithms and genetic
programming: individual measures are generated using ge-netic
algorithm whereas the complete phrases are bred using genetic
programming,see Section 6 for detailed description of the system.
NEAT Drummer [26] evolvesrhythm patterns using a combination of
neural networks and interactive evolutionarycomputation.
GeneticDrummer [1719] is one of a few evolutionary systems that
perform au-tonomous evaluation of generated results using various
fitness operators. The systemevolves a rhythm accompaniment played
on a common drumset. Another differenceto the most of other
existing systems is that the rhythm accompaniment is evolvedto an
existing musical phrase in addition to users specified preferences.
That is, thegenerated accompaniment should match the musical phrase
to which is the rhythmaccompaniment being generated. We describe
GeneticDrummer in detail in Sec-tion 7.
1.3 Methods and Techniques for Algorithmic CompositionIn the
previous section we surveyed briefly application areas of
algorithmic com-position. Another interesting point can be found in
the employment of methodsand techniques behind algorithmic
composition systems. Among widely used evo-lutionary techniques,
such as genetic algorithms (GaMUsic, GenDash, GenJam,GaNotator,
Neurogen, GeneticDrummer . . . ) and genetic programming
(GP-Music,GenBeBop), various other techniques and methods has also
been utilized in algo-rithmic composition. For instance, stochastic
processes (including Markov chains)[2, 16, 53, 54] has been widely
used before the rise of evolutionary techniques.
-
940 M. Dostal
Cybernetic composer [4] is a representative example of systems
based on Markovchains. The system deduces the rhythm of a melody
using Markov chains and com-poses a new melody at a later stage.
Some attempts have also been made withchaotic, non-linear systems
[23, 24].
Knowledge based systems have been successfully used in
harmonization [20, 46].Harmonization as a musical task is supported
with well grounded theory that can beapplied as a set of constrains
and rules in a knowledge-based system. Some attemptshave also been
made with grammar-based systems, see [34, 41] for more detail.
Neural networks have also attained certain popularity in
algorithmic composi-tion. Neural networks have been used in several
hybrid systems (typically with ge-netic algorithms or genetic
programming) to assist the composition process [14, 21].In
evaluation assistance tasks, neural networks learn responses from
the mentor inan interactive system and later helps the mentor with
recommendations or a pre-filtering the results [14, 33, 43, 45].
Another quite common usage of neural net-works is learning from
input data such as a melody, a harmony [25] or a rhythm[26]. Baggi
used also neural networks to composition of swing musical pieces
[7].
Since this book chapter focuses on utilization of evolutionary
computation tech-niques in various subtasks of music composition,
the following text will discusssome representative evolutionary
systems and the evolutionary techniques behindthese systems.
Genetic algorithms and genetic programming are obviously the
twomost popular evolutionary techniques used in algorithmic
composition. Althoughthe basic shape of these algorithms behind
algorithmic composition systems is prac-tically the same as in
other more common or more formal applications of evolu-tionary
computation, some differences can be identified. First,
evolutionary musiccomposition systems typically use small
populations, commonly between 10-50 in-dividuals. Interactive
systems, as discussed above, use small populations owing totime
consuming and demanding evaluation of individuals for the mentor.
However,systems with autonomous evaluation of individuals also use
relatively small popu-lations in comparison to other application
areas of evolutionary computation. Ini-tial populations in music
composition systems, if not generated randomly, oftenuse typical
examples (e.g., phrases, patterns or motives) of a particular
genre, styleor rhythm, for instance. These populations contain
typically tenths of individuals.Broadly speaking, populations can
be construed as an impression of ones musi-cal invention (similarly
to viewing an audio sample of a drum as impression of thesound of
such a musical instrument, for instance) and the evolutionary
algorithmitself as a creative machinery responsible for the
composers hard work. The crucialpart of evolutionary systems is the
fitness evaluation. In musical systems, the fitnessfunction is
either based on some musically meaning criteria or an evaluation by
thementor, as it is discussed in Section 1.4 in more detail. The
mentors evaluation isa quite specific property of many musical
systems in a broad context of evolution-ary systems and their
applications. Also mutation operators are in musical systemsoften
designed to be musically meaning rather than purely random. The
reason isobvious, the mutation should produce at least acceptable
individuals. However theproduction of musically improper changes
that breaks the rules may also be advan-tageous to some extent. The
role of mutation in musical systems is often high in
-
Evolutionary Music Composition 941
comparison to occurrence of mutation in nature. Mutation is,
beside crossover, thecrucial mechanism for creating variations of
musical pieces.
Another important viewpoint for evolutionary techniques is the
appropriatenessof individual techniques for particular musical
subtasks. Seemingly, evolutionarytechniques can be great in musical
tasks less constrained with complex rules. Typi-cally, evolutionary
techniques seem very appropriate for evolving of rhythm
accom-paniment. Why do we argue for that? The creation of appealing
rhythm patterns orgrooves can be based on quite simple rules and
the partial results can be creativelycombined into even better
rhythms and patterns. Also establishing a rhythmicalcoordination
between a musical phrase and a rhythm accompaniment
generatedevolutionary is not that a complex task. On the other
side, usage of evolution-ary techniques to harmonization seems less
appropriate than using more formalapproaches, such as rule-based
systems, for instance. Composing a harmony re-quires a wider and
deeper musical knowledge in order to produce good
sounding,musically correct harmonies. Similar observation holds for
generating melodic mo-tives. In these musical tasks, a musically
meaning structure for a musical piece ora musical intention behind
the composition must be devised evolutionary, whichseems to be a
much more difficult task than creating appealing rhythm patterns.
Forcomposing melodic motives or harmonization, knowledge-based
approaches seemmore appropriate.
1.4 Interactive versus Autonomous Systems
1.4.1 Interactive Systems
Interactive evolutionary systems use evolutionary algorithms as
a technique tomachine-based generating or combining of
compositions. However, one importantpart of the system, namely the
evaluation of generated music, is not performed au-tonomously by
the system itself but rather it is substituted by the decision of
theuser. The user acts as a human mentor that assigns a rating that
specifies the qualityof generated results. Typically, the quality
is expressed by a number in a specifiedrange, or to simplify the
process, it can be reduced to categorization of individu-als into
keepers or losers. Keepers survive to next generation, whereas
losers areremoved. The justification for using a human mentor is
obvious: the mentor canevaluate the aesthetic quality of
individuals that seems almost impossible to be eval-uated
algorithmically. The mentor is also subjective, which could be an
advantageor not, depending on the viewpoint.
One common difficulty in the practical use of interactive
systems is the human fa-tigue. Basically, it has two dominant
causes; first, the amount of evaluations is higheven in small
populations. The mentor must evaluate each individual in each
gener-ation of the composition process which is pretty time
consuming. Second, listeningcarefully and critically to music
fatigues quickly the mentor. Similarly, recordingengineers are
faced daily with this phenomenon. It is not possible to hold long,
con-tinuous recording or mixing sessions for the engineer.
Arguably, the task for thementor is about the same arduousness as
for the engineer.
-
942 M. Dostal
Interactive systems thus typically use small populations in
order to keep theamount of evaluations admissible. Nonetheless, the
task for the mentor is stilltime consuming and fatiguing. Several
interactive systems address this issue witha machine-based mentor
that helps the human in evaluation. These systems [14,33, 43, 45]
typically use an artificial neural network that learns from the
mentorsresponses and provide a preliminary evaluation or
pre-filtering of results (e.g. au-tonomous identification of
losers).
1.4.2 Autonomous Systems
These systems aims at the autonomous evaluation of generated
music in terms ofquality. Though the autonomous evaluation of
aesthetic quality or musical pleasing-ness seems not to be much
viable for the computer, various quantitative or structure-based
measures were devised to evaluate the music on such a basis.
Autonomoussystems use typically heuristic rules, knowledge-based
rules (typically coming fromthe music theory) or some kind of
learning from the mentor as it was mentionedearlier. In evaluation
of melodic material a variety of measures including
horizontalintervals, melodic contours, musical staff density,
rhythm structure or syncopationwere devised [3, 36, 38, 42, 44]. In
evaluation of generated harmonies also a rule-based approach was
successfully used [6, 52]. Rhythm seems to be quite good evalu-able
using heuristic rules since the rhythm is to a large extent
determined by struc-ture of durations. No such a complex problems
arise in rhythm evaluation in com-parison to melodic or harmonic
material. Also, the rhythm accompaniment seemsless subject to
thematic development and thus can be reasonably evaluated at
thelevel of individual measures instead of evaluating long musical
phrases. Horowitz[30] evaluated syncopation, density, downbeat or
beat repetition on the generatedshort rhythm patterns. Dostal [17]
proposed fitness operators that evaluate rhythmconsonance or
dissonance with a particular musical pattern, dynamic consonance
orrhythm density of individual percussion instruments available in
a drumset. Theseoperators are described in detail in Section 7.
Listening
1. Jacobss system for music composition. The webpage contains
several samplesincluding a description.http://www.eng.umd.edu/
blj/algorithmic_composition/
2. GenBebop. The webpage contains a few jazz improvisations
performed by theSpectors and Alperns
system.http://hampshire.edu/
lasCCS/genbebop.html
3. SBEAT. Over twenty examples of various styles and
length.http://www.intlab.soka.ac.jp/
unemi/sbeat/sample.html
-
Evolutionary Music Composition 943
2 GP-Music
Johanson and Poli created the GP-Music system that evolves short
melodic se-quences according to both the mentors preferences and
autonomous measures ofthe system itself [33]. GP-Music is based on
genetic programming. Basically, thesystem is interactive, however
the automated measures can be performed using anextended version of
the system that combines genetic programming with a neuralnetwork.
The network is trained using the mentors responses and performs an
au-tonomous evaluation of generated music afterwards.
Individuals, that is musical sequences, are represented as
program trees as widelyused in genetic programming. Terminals
represent individuals notes (the followingsubset of chromatic scale
is available: C-4 C#4 D-4 D#4 E-4 F-4 F#4 G-4 G#4 A-5A#5 B-5),
chords (C-Chord, D-Chord, E-Chord, F-Chord, G-Chord, A-Chord,
B-Chord) or rest. Notes have a constant duration. The function set
contains operationsto constitute the sequence structure:
Play two (arity 2) concatenates two sequences together. Add
space (arity 1) inserts a rest after each event present in the
input sequence. Play twice (arity 1) repeats the sequence passed as
parameter. Shift up (arity 1) shifts notes in the sequence one note
up. Shift down (arity 1) shifts notes in the sequence one note
down. Mirror (arity 1) reverses the original sequence. Play and
mirror (arity 1) concatenates the original and reversed
sequence
together.
Fig. 1 and Fig. 2 depicts an individual that represents the
following sequence: [F-4rest C-4 rest F#4 C#4 rest C D].
(PLAY-TWO(PLAY-TWO(ADD-SPACE (PLAY-TWO F-4 C-4))F\#4)
(PLAY-TWO(ADD-SPACE C\#4)(PLAY-TWO C D)))
Fig. 1 Example of an individual in GP-Music
GP-Music uses standard genetic programming for evolving melodic
sequences[35]. The mentor evaluates individuals on a 1-100 scale in
each generation. Thusthe same individual may obtain a different
fitness during the repeated evaluation bythe mentor. Automated
measures are performed using a modified back-propagationneural
network, see [33] for more detail about the network
architecture.
-
944 M. Dostal
Fig. 2 Example of an individual in GP-Music
Listening
Authors provide several sound examples including a description
of the parametersetting.http://graphics.stanford.edu/
bjohanso/gp-music/
gp_music-old.html
3 GenDash
GenDash [49, 50] is a compositional tool developed and used by
american composerRodney Waschka II. The system itself is intended
for composing musical pieces ina highly collaborative way with the
human composer. Waschka successfully usedGenDash to compose
complete musical pieces as well as to generate parts of piecesthat
were further processed and arranged into musical pieces by the
author.
GenDash is rather a music composition tool for the human
composer than anautonomous music composition system. The system is
intended primarily to cre-ate variations of measures of music but
without the evaluation of generated musi-cal pieces by the human
mentor or a fitness measure during the evolution process.It means
that GenDash lacks one fundamental part of an evolutionary system
andsuggests that it is a compositional tool which the composer uses
to generate somemusical pieces that he afterwards deliberately
decides to use it in a piece or discard.
The technique behind GenDash is basically a genetic algorithm
without fitnessevaluation. That is, individuals to be reproduced to
the next generation are selectedat random. Individuals represent
measures of music. The initial population of in-dividuals is
predefined by the user. For instance, Waschka used a population
oftwenty-six measures of music drawn from typical Greek and
Medieval songs for
-
Evolutionary Music Composition 945
composing Sapphos Breath, a one-act opera comprised of an
overture and twelvescenes [51].
New individuals are created using standard one-point crossover
between two par-ents. GenDash also allows for mutation. Two types
of mutation are possible. Firsttype is a mutation performed by the
system, such as changing a pitch or a durationof note. Second is
the so called wildcard mutation that is based upon the altering
ofthe individual by the human composer.
Waschka reported successful usage of GenDash to compose ten
musical piecesincluding small- to medium-scale pieces up to 30-40
min duration. The authortweaked the system individually for each
one composition. Some details on Gen-Dash usage on particular
musical pieces can be found in [50, 51].
Listening
1. Waschka II, R. (1996). Empty Frames. Raleigh: Borik Press.2.
Waschka II, R. (2000). Saint Ambrose. Raleigh: Borik Press.3.
Waschka II, R. (2001). Sapphos Breath. Raleigh: Borik Press.
4 GenJamGenJam (a short for Genetic Jammer) is a system for
generating jazz melodic impro-visations that has been developed by
John A. Biles [8, 10, 11, 13]. The system is ableto improvise
full-chorus solos and chase choruses in real time. Chasing choruses
is astandard practice in jazz improvisation where soloists
improvise on four-bar phraseson a tune in a way that a soloist
chases the previously played phrase by anothersoloist by developing
a new four-bar phrase on the basis of the previously playedone. In
GenJam, the chorus chasing is performed between a four-bar phrase
playedlive by the performer and GenJam which chases the performers
improvisation. Thehuman performer plays a melody on an instrument
that is converted in real-time tocorresponding MIDI events using
the Roland GI-10 pitch-to-MIDI-converter. Thesedata are further
processed by GenJam as it will be explained below.
GenJam is based on interactive genetic algorithm that breeds new
melodies. Sincethe system is interactive, the evaluation of
generated phrases is performed by the hu-man mentor. The mentor
listens critically to generated results and provides
on-goingfeedback on the perceived quality by evaluating phrases as
good, bad or neutral. Asit was mentioned, the system interacts with
the performer and the mentor. Addi-tionally, GenJam uses some other
data required for the interactive genetic algorithmoperation:
Measure population that contains chromosomes. Each chromosome
holds onemeasure long melodic phrase and its fitness.
Phrase population. Chromosomes in Phrase population contains the
phrase fit-ness and identifiers to corresponding measure
chromosomes that constitute thephrase.
-
946 M. Dostal
Rhythm sequence. It is a standard MIDI file containing usually
three to fivetracks, typically piano, bass, drums and strings or
guitar. The file is played backduring the live performance. It is
used to evoke an illusion of a jazz quintet withtwo soloists, that
is the human performer and GenJam, and accompanying rhythmtrio.
Head sequence. Similar to the rhythm sequence, the head sequence
is a standardMIDI file that is played back during the live
performance. The sequence containsthe original melody of a
tune.
Chord progression. It contains the harmonic progression of the
tune. The infor-mation about the harmonic structure is used by
GenJam to ensure production ofharmonically correct notes.
Choruses. It provides information to GenJam how each of the
choruses can behandled. In example, a full-chorus solo, chase
chorus or a rest.
GenJam operates in three modes, learning, breeding or demo.
Learning mode isused to evaluate phrases in population. Phrases are
played-back in random orderand the mentor provides a feedback by
responding g for good, b for bad, or a neutralrating is considered
if the mentor does not provide any feedback on the phrase.Good
evaluation from the mentor increases the fitness of the phase by
one, whilebad evaluation decreases it by one. No response on the
phrase does not affect itsfitness. Fitness values are bounded to
interval < 30,30 >. More details can befound in [9].
Demo mode is used to select phrases for a live performance. The
selection isperformed using a tournament selection. No feedback
from the mentor is requiredin this mode.
Breeding mode is intended for creating new phrases using
recombination opera-tions. In each application of the breeding mode
one half of phrases in population isreplaced by newly bred
phrases.
GenJam is designed to be able to operate on a live performance
which is a quiteunique feature among music composition systems.
Human player plays four barsof a melody that GenJam listens to. The
playing is transformed to MIDI eventsand quantized into eight
eighth-note-length intervals, depending on internal repre-sentation
used by the genetic algorithm. In last thirty milliseconds of the
humansfour bar phrase playing, the system breeds a new phrase.
GenJam is aware about thebegin/end of the humans playing since the
complete tune structure including thechord progression and
accompanying tracks is predefined. In other words, impro-visations
by the human and GenJam are performed on the beforehand
determinedparts of the tune.
4.1 Population
GenJam uses two populations; a phrase population and a measure
population.Both populations are of fixed size: phrase population
contains 48 individuals andmeasure population holds 64 individuals.
Individuals are encoded using bit-stringchromosomes.
-
Evolutionary Music Composition 947
A chromosome in phrase population encodes an identifier, fitness
value and iden-tifiers of individuals in the measure population
that constitute the phrase. For exam-ple, consider a phrase
chromosome depicted on Fig. 3. The chromosome has iden-tifier 10,
fitness -5 and the phrase is composed of four measure chromosomes
withthe following identifiers: 20, 34, 11 and 23.
Fig. 3 Example of a phrase chromosome
A chromosome in measure population encodes its identifier (that
is used to ref-erencing measure chromosomes in phrase chromosomes),
a fitness value and one-measure-long melodic phrase in 4/4 time. A
phrase is encoded in a series of eightevents, each representing one
eighth-note-length event. These events are furthertransformed to
MIDI events and played by the synthesizer. An event may
representeither a note, a rest or a hold. Hold event has the effect
of holding the previous event.For example, if a note event is
followed by one hold event, the note will be playedwith
quarter-note duration. Note events encode a relative note (pitch)
of the event(see below). Since the events are internally
represented as four-bit strings, GenJamis limited to 14 note scale
at one time, 2 of 16 possible combinations are reserved tothe rest
and hold events. The rest event is encoded in chromosome as 0, hold
as 15and note events as 1-14. See Fig. 4 which depicts four
chromosomes referenced inthe phrase chromosome from Fig. 3. Fig. 4
contains chromosomes with ids 20, 34,11 and 23 whose fitnesses are:
-3, 12, -22 and 4.
Fig. 4 Examples of measure chromosomes
As it was stated above, note events encode relative notes while
14 different notesare available. These relative notes are mapped
using predefined scale maps and thechord progression of the tune.
Chord progression defines harmonic structure of thetune (limited to
two chords per measure at maximum). A note map is an array
ofpitches representing a scale appropriate to the given chord.
Since the scale maynot be composed roughly of 14 notes, the scale
is extended to 14 notes using scale
-
948 M. Dostal
repetition. For example, a scale consisting of [C D E G A Bb]
would be extendedto [C3 D E G A Bb C4 D E G A Bb C5 D]. Biles
synthesized the scale maps fromsome jazz educational literature.
The resultant scale map is depicted in Table 1 [11].Another, a more
complex map can be found in [10]. Using chord progression andscale
maps, GenJam produces harmonically correct pitches only.
Table 1 Scale map
Chord Scale NotesCmaj7 Major (avoid 4th) C D E G A BC7
Mixolydian (4th) C D E G A BbCm7 Minor (avoid 6th) C D Eb F G
BbCm7b5 Locrian (2nd) C Eb F Gb Ab BbCdim W/H Diminished C D Eb F
F# G# A BC+ Lydian Augmented C D E F# G# A BC7+ Whole Tone C D E F#
G# A#C7#11 Lydian Dominant C D E F# G A BbC7#9 Altered Scale C Db
Eb E F# G# BbC7b9 H/W Diminished C Db Eb E F# G A BbCm7b9 Phrygian
C Db Eb F G A BbCmaj7#11 Lydian C D E F# G A B
Obviously, this kind of representation is limited in several
aspects. First, it can en-code only eighth-note multiplies. Second,
there is not possible to encode a dynamicarticulation of notes.
Third, there are only 14 pitches available at the time. Finally,the
humans playing is quantized to eighth-note-length intervals which
limits therhythmical expressivity of the system.
The populations are initialized at random at the outset of
evolution. Each chro-mosome has assigned a fitness value equal to
zero. In measure chromosomes, thestaff part is initialized with
events. Each position is initialized either with a rest, ahold or a
note event. The probability of a note event is set to 124 and the
probabilityof a rest or and a hold event is set to 524 . Biles set
the probabilities on the basis ofempirical observations.
4.2 Recombination Operators
In GenJam, selection and replacement of individuals is based on
a modified tour-nament selection. A tournament is performed between
four randomly selected indi-viduals (i.e. without regard to their
fitnesses). Two of four individuals with highestfitnesses are
selected as parents that undergo mating using crossover and
mutation.These newly produced offsprings are used as replacement of
the latter two individ-uals (those two with lowest fitnesses) in
the tournament.
Crossover operation is a slightly different version of standard
single-point cross-over. Crossover is performed at one randomly
selected position in the event-part of
-
Evolutionary Music Composition 949
chromosomes. One of the offsprings then undergo mutation, while
the other one iskept unmodified.
Mutation is the most interesting operation of recombination
operations in Gen-Jam. It does not operate on a bit-level as many
conventional, general-purpose mu-tation operations used in genetic
algorithms, but rather at the level of individualevents. The point
is to provide musically meaningful manipulation of measures
en-coded in chromosomes. GenJam performs mutation on both phrase
and measurechromosomes. On measure chromosomes, the following
operations can be applied:
Identity. Returns identical, unmodified chromosome. Reversion.
Produces a chromosome with reverse order of events. For
example,
a reversion of chromosome [20 -3 14 13 14 12 15 9 0 0] would
re-sult in [20 -3 0 0 9 15 12 14 13 14].
Right Rotation. The event-part of a chromosome is rotated by
randomly chosennumber (between 1 and 7) of positions to the right.
For example, a right rotationof chromosome [20 -3 14 13 14 12 15 9
0 0] of 3 positions wouldresult in [20 -3 9 0 0 14 13 14 12
15].
Inversion. Each event is changed to 15-. For example, an
inver-sion of chromosome [20 -3 14 13 14 12 15 9 0 0] would result
in[20 -3 1 2 1 3 0 6 15 15].
Ascending Sort. Events in chromosome are sorted in ascending
order. For ex-ample, ascending sort of chromosome [20 -3 14 13 14
12 15 9 0 0]would result in [20 -3 0 0 9 12 13 14 14 15].
Descending Sort. Events in chromosome are sorted in descending
order. For ex-ample, ascending sort of chromosome [20 -3 14 13 14
12 15 9 0 0]would result in [20 -3 15 14 14 13 12 9 0 0].
Transposition. This operation is performed on a randomly
selected number ofsteps (between 1 and 4) on the direction of
greater minimum distance betweenthe note-event and upper (i.e., 14)
or lower (i.e., 1) bound. Hold and rest eventsare unaffected. For
example, a transposition of chromosome [20 -3 14 1314 12 15 9 0 0]
of 3 positions would result in [20 -3 11 10 11 912 6 0 0].
On phrase chromosomes the following operations can be
applied:
Identity. Returns identical, unmodified chromosome. Reversion.
Produces a chromosome with reverse order of events. For example,
a
reversion of chromosome [10 -5 20 34 11 23] would result in [10
-523 11 34 20].
Right Rotation. The event-part of a chromosome is rotated by
randomly chosennumber (between 1 and 7) of positions to the
right.
Genetic Repair. Replaces the reference to the worst chromosome
in phrase withrandomly selected another one.
Super Phase. Generates completely new phrase. On each position a
tournamentof three measure chromosomes is performed and a reference
to the winning chro-mosome is stored at the corresponding position
in phrase chromosome. Note thatthis operation usually does not
preserve any phrase from the parents.
-
950 M. Dostal
Lick Thinner. This operation replaces a reference to measure in
the phrase chro-mosome that occurs in the phrase population most
frequently. The reference isreplaced by another one selected at
random. The role of the operator is to elimi-nate some occurrences
of highly fit measures from the phrase population.
Orphan Phase. It operates opposite to the Super phase operation,
that is the lessfit chromosomes are considered as winners from
tournaments. This operation isintended to increase the diversity of
population.
Since GenJam is interactive system, the evaluation cannot be
performed in everygeneration of the breeding process. In Demo mode,
only selection is performed. InBreeding mode, all recombination
operators are applied. Fitness values are assignedinteractively by
the mentor in Learning mode. Biles also experimented with a
dif-ferent version of GenJam that eliminates the evaluation of
generated results by thementor, see [12] for more detail. Biles and
Anderson [14] have also made an attemptto enhance the interactive
evaluation by the mentor with a neural network that is ableto
discover worst individuals in population. However, the experiments
did not resultin much success.
Listening
The webpage contains several nice music examples including a
description.http://www.ist.rit.edu/
jab/GenJam.html
5 Phon-Amnuaisuks Harmonization System
Phon-Amnuaisuk et al. developed a system for harmonizing
user-specified melodies[5, 6]. The system produces four-part (while
one part is the input melody) homo-phonic tonal harmony using
genetic algorithm. The genetic algorithm alsoautonomously evaluates
generated harmonies, so that in contrast to interactive sys-tems,
the evaluation does not rely completely on the user.
The system operates with four voices, soprano, alto, tenor and
bass. Sopranovoice is assumed to be a melody voice. Chromosomes are
represented in a matrix-like structure. Each chromosome contains
five rows; the first four rows representsvoices (soprano, alto,
tenor, bass) while the last row describes the duration of thestaff
in individual voices. It follows that all voices have the same
rhythm. Individualchromosomes represent four-voice phrases and the
recombination operations oper-ate on the complete chromosome, not
the individual voices.
Genetic algorithm uses standard single point crossover. Mutation
of chromo-somes is possible using the following musically meaning
operations: Peturb. Transposes alto, tenor and bass voices up or
down by one tone or semi-
tone. The selection of transposition direction is done at
random. Swap. The operation swaps two randomly picked voices (alto,
tenor or bass).
It enables changing chords between open (open voicing is one
that uses eachpart on the closest harmonizing note) and closed
(closed voicing refers to anarrangement of the notes of chords
within a narrow range) positions and changing
-
Evolutionary Music Composition 951
of inversions. Both open and closed harmony are common
techniques for voicingin harmonization.
Rechord. It generates a new chord from the melody stored in the
soprano voice.A chord is constituted by the note in soprano voice
as a root note, and 3rd or 5th.
PhraseStart. Mutates the beginning of each phrase to start with
tonic root posi-tion on a down beat.
PhraseEnd. Mutates the end of each phrase to end with a chord in
root position.Obviously, crossover and mutation can produce
harmonically unpleasant results.However, such individuals should be
eliminated in the reproduction process.
The most interesting part of the system is the fitness
evaluation of generated har-monic structures. Fitness evaluation is
performed autonomously by the system. Fit-ness function utilizes
common harmonization knowledge used in standard four-partharmony
writing. Harmonization knowledge in the system captures various
types ofpossible harmonic structures while some of them are
preferred and some are not.The evaluation of harmonic occurrences
is performed on two levels, within voicesand between voices. For
instance, within voices a stepwise progression is preferredover
large leaps, while dissonant chords and some kinds of leaps are
penalized. Be-tween voices, occurrences of parallel unison,
parallel octaves or progression fromdiminished 5th to perfect 5th,
or crossing voices are penalized. More details on fit-ness
evaluation can be found in [6].
Since the system evaluates only the harmonic movements between
two adjacentchords, it is not possible to compose large-scale and
sophisticated harmonic pro-gressions.
Authors also have developed a different version of the system
which uses a rule-based inference engine instead of genetic
algorithm. A comparison between thesystems has been made and the
rule-based implementation has delivered substan-tially better
results [6]. It is a quite presumable result, since the rule-based
approachseems to be more appropriate for harmonization tasks than
genetic algorithm. Theprocess of writing harmonies is a well
examined part of music composition theory,but genetic algorithms
provide rather an unstructured search in the problem domain.
6 CONGA
CONGA (COmposition iN Genetic Approach) [45] is an evolutionary
drum ma-chine that generates multi-timbre rhythm patterns. To
generating rhythms, a combi-nation of genetic algorithm and genetic
programming is used. Genetic algorithm isused to breed individuals,
usually one measure long rhythm phrases that are furtherarranged
into 416 measures long rhythm patterns using genetic
programming.
Basically, the system is interactive so that the evaluation of
generated phrasesrelies on the mentor. Note also that the system
generates rhythm patterns withoutany regard to other music, so that
its purpose is much closer a drum machine ratherthan a rhythm
accompaniment composition system. However, CONGA also utilizesa
neural network that learns the mentors responses and provides an
assistance inevaluation of individuals.
-
952 M. Dostal
6.1 Generating Phrases
Phrases are generated using genetic algorithm. Individuals in
population representshort phrases, usually one measure or
half-measure long ones. Individuals are repre-sented in
two-dimensional chromosomes with a matrix-like structure. Rows
repre-sent individual timbres (drum instruments) and columns
represent individual eventsin the phrase. In fact, the
representation is close to the way how the classic pattern-oriented
sequencers (e.g., ROLAND TB-303) were programmed by the user.
Eachcolumn corresponds to a predefined note duration according to
the chosen resolu-tion of the system, e.g., one sixteenth or one
eighth note. So that, according to theresolution and meter, the
chromosome has the corresponding length, e.g, for 1/16note
resolution and 3/4 meter, the chromosome length will be equal to
12. Eventsare described by integer values, 0 is used for rest, and
1-127 for note-events. Thevalue describes the strength of the
stroke that corresponds to the velocity parameterused in MIDI
devices. The number of timbers is also user-definable so that the
chro-mosome will contain as many rows as the number of individual
timbres. We depictan example of chromosome and corresponding
notation on Fig. 5. The chromosomecontains three timbres, hi-hat
(HH), snare (SD) and bass drum (BD). The resolutionwas set to 1/16
note. This simple representation, however, does not enable to
cap-ture some other nuances of drum playing, such as different
stroke types (e.g, doublestroke, flam of drag stroke) or possible
instrumentations of particular drum timbers(e.g., head or rim
stroke on drums, or bell or bow stroke on cymbals).
Fig. 5 Example of a phrase chromosome in CONGA
In CONGA, two crossover operations are available. First,
standard one pointcrossover which exchanges selected parts between
parental chromosomes and sec-ond, a crossover operation that
exchanges selected timbers between the parents.Likewise many other
genetic algorithm-based generative systems, GONGA usesseveral
musically meaning mutation operators:
Random. Mutates selected chromosome positions with randomly
selected values(0-127).
-
Evolutionary Music Composition 953
Rotation. Rotates a timbre in the chromosome. Reverse. Reverses
a timbre in the chromosome. Timbre exchange. Exchange selected
timbre parts with the selected chromosome.
6.2 Arranging Phrases to Patterns
Individual phrases generated by the above outlined genetic
algorithm are arrangedinto 416 measures long patterns using genetic
programming [35]. Thus, the pat-terns are represented as programs
with a tree structure. Terminal nodes representreferences to
individual phrases or the Random function, and the non-terminal
nodesrepresent functions from the function set that constitute
phrases. The following func-tions are available:
Sequence. Operates on two nodes. Plays NODE1 and NODE2
consecutively. Repetition. Operates on two nodes. Repeats NODE1
until the length of NODE1
is equal to length of NODE2. Concatenation. Operates on two
nodes. Plays the first half of NODE1 and the
second half of NODE2 consecutively. Reversion. Operates on one
node which is played in reverse order. Random. Plays a randomly
selected phrase individual available in the pattern
individual.
Fig. 6 depicts an example of phrase in tree representation. Two
phrases (i.e., I-4 andI-5) appear in the pattern chromosome that
corresponds to the sequence [firsthalf of I-5] [second-half of I-5
or I-4] [I-4] [I-5].
Fig. 6 Example of a pattern chromosome in CONGA
6.3 Operating with CONGA
The user is allowed to parametrize CONGA in various aspects,
such as populationsizes or length of generated patterns or used
timbers. The system outputs to MIDIand can be synchronized with
another MIDI devices using MTC (MIDI time clock).
-
954 M. Dostal
Using that, the mentor is able to evaluate the evolved patterns
with consideration ofsome other music played by another MIDI
devices.
As it was mentioned above, CONGA utilizes also a neural network
that pro-vides an evaluation assistance to the mentor. A
three-layer back-propagation net-work learns the responses from the
mentor. The assistance is provided in terms ofpreselection of
highly fit individuals to the mentor in order to reduce the
psycholog-ical burden caused by evaluation of each evolved
individual.
7 GeneticDrummer
GeneticDrummer is an evolutionary composer of rhythm
accompaniment. The sys-tem has been developed between 2000 and 2002
by Dostal [17], the author ofthis chapter. The system composes a
matching rhythm accompaniment to the givenmusical piece.
Internally, it uses genetic algorithm that performs, contrary to
mostevolutionary rhythm generators, autonomous quality evaluation
of generated accom-paniment using a fitness function. The user,
however, specifies the fitness criteria inorder to define the
preferred aspects of resulting rhythm accompaniment.
Genetic-Drummer listens also to one selected instrument part of the
musical piece (e.g., abass-guitar or rhythm piano part) to which a
rhythm accompaniment will be com-posed in order to produce an
accompaniment that matches with the music. In Gene-ticDrummer, the
aspect of musically meaning design of the system and the ability
toperform autonomously is strongly emphasized. The system uses
musically meaningrepresentation that captures typical drum playing
techniques and instrumentationpossibilities of individual drums
available in the system. Also, the design of muta-tion and fitness
operations attempts to be musically meaning. These aspects will
beintroduced in more detail later in this section.
7.1 Representation of RhythmIn GeneticDrummer, the
representation enables to express fine nuances of drumplaying
technical possibilities utilized by human drum players and also the
indi-vidual instrumentation possibilities of percussion
instruments. The representationcaptures a conventional drumset
consisting of a snare drum, a bass drum, four tom-toms, a hi-hat,
two crash cymbals, a splash, and a ride cymbal. Each drum
instrumentenables some instrumentation techniques. For instance, on
snare drum a head drumstroke, a rim stroke or a rimshot (that is a
hit to the rim and head simultaneously) ispossible, whereas on bass
drum only a kick pedal stroke is available. Another goodexample of
instrumentation possibilities can be provided with cymbals. Ride
cym-bal is usually played on the bow, edge or bell while the hi-hat
is played on bow oredge. On hi-hat, the level of cymbals openness
is controlled using a foot pedal. Allof these individual
instrumentation differences are captured in GeneticDrummer.However,
drum technique nuances utilized by human players are also
particularlyimportant for musically meaning and flexible
representation. When human drum-mer plays drums with drumsticks, it
is possible (and also commonly used) to play
-
Evolutionary Music Composition 955
more than one note using one drum stroke. These kinds of drum
strokes are great tocoloring the rhythm accompaniment with fine
nuances. Fig. 7 depicts types of drumstrokes available in
GeneticDrummer. That is, a drum stroke is the basic musical
el-ement in GeneticDrummer, not a note as commonly used in other
rhythm generationsystems. Note also, that this approach is commonly
used in drum notation, wheremultiple drum strokes are identified by
a particular symbol instead of writing-downindividual notes
corresponding to the drum stroke type. In a pattern, drum
stroketypes can be easily interchanged (e.g., single stroke to
double stroke or vice versa)and thus used to produce naturally
sounding rhythm variations easily.
Fig. 7 Drum stroke types available in GeneticDrummer
Drum strokes of individual drums or cymbals are encoded with
integer valuescalled rhythm elements which are stored at individual
positions in chromosomes. Arhythm element encodes a drum stroke
type and a drum stroke volume. Drum strokevolume is specified on a
six-point scale from piano to forte fortissimo. Each druminstrument
has assigned an interpretation table that is used to encode and
decoderhythm elements for individual drum instruments. For
instance, Table 2 depicts theinterpretation table for the snare
drum. There is also one special rhythm element:a rest, that is
encoded with the zero value. Other interpretation tables and
furtherdetails on rhythm elements can be found in [19].
-
956 M. Dostal
Table 2 Snare drum interpretation table
playing technique p mp mf f ff fffsingle drumhead stroke 11 12
13 14 15 16double drumhead stroke 21 22 23 24 25 26triple drumhead
stroke 31 32 33 34 35 36drumhead flam stroke 41 42 43 44 45
46drumhead drag stroke 51 52 53 54 55 56drumhead ruff stroke 61 62
63 64 65 66single side-stick stroke 71 72 73 74 75 76
Individuals in a population are represented using
multichromosomes. Multichro-mosomes hold one measure long rhythm
patterns. A multichromosome is a liststructure composed of
chromosomes. Each chromosome corresponds to a particularpercussion
instrument and is composed of rhythm elements, see Fig. 8.
((chromosome representing splash cymbal)(chromosome representing
crash cymbal 2)(chromosome representing crash cymbal 1)(chromosome
representing ride cymbal)(chromosome representing
hi-hat)(chromosome representing snare drum)(chromosome representing
bass drum)(chromosome representing small tom-tom)(chromosome
representing medium tom-tom)(chromosome representing large
tom-tom)(chromosome representing floor-tom))
Fig. 8 Multichromsome structure
Chromosomes are encoded in a pattern-like style, that is each
position corre-sponds to a particular duration depending on the
internal resolution of the systemand the metre of multichromosome.
The most commonly used internal resolution inmultichromosomes is
one sixteenth-note. Using that resolution a chromosome with4/4
metre will contain 16 positions. Fig. 9 depicts a sample
multichromosome in 4/4metre and the corresponding notation. The
(empty) element represent a chromo-some composed of rests, that is
(00 00 00 00 00 00 00 00 00 00 0000 00 00 00 00).
The pattern-oriented representation of rhythm is easy to use in
computers, butit does not enable to encode triplets directly. To
encode triplets we use a specialvalue 300 that is added to the
rhythm element value if a note should be shifted to atriplet
division. For instance, Fig. 10 depicts such an example of triplet
coding. Thismethod of triplet coding was inspired with triplet
coding used in pattern sequencers.
-
Evolutionary Music Composition 957
((empty)(empty)(empty)(empty)(14 00 14 00 14 00 14 00 44 00 74
00 24 00 15 00)(00 00 00 00 14 00 00 00 00 00 00 00 14 00 00 00)(14
00 00 00 00 00 14 00 14 00 00 00 00 00 00
00)(empty)(empty)(empty)(empty))
Fig. 9 Multichromosome and notation example
score:
chromosome:(14 314 314 0 14 314 314 0 14 314 314 0 14 314 314
0)
Fig. 10 Triplet coding example
As it was stated above, individuals in a population are encoded
with multichro-mosomes. GeneticDrummer uses two populations, one
for generating the accompa-niment (the so called groove populaion)
and one for generating rhythm breaks (theso called break
population). Rhythm breaks may appear at the last measure of
agenerated phrase. Needless to say that as a groove population can
also be used a setof individuals representing drum breaks, if the
user has an intention to generate alonger drum solo.
From the musical standpoint, the population contains one measure
long drum pat-terns of particular rhythm and drum-playing style.
The populations can be composedof typical grooves of a particular
human player, for instance. In this case, the Gene-ticDrummer is
able to, to some extent, imitate the playing style of the human
player.Another good approach to build-up populations, is to take a
drum-practice book andrewrite typical patterns of particular rhythm
into multichromosomes. It was actuallythe most frequently used
approach to build-up the populations for experimentingwith
GeneticDrummer. The last, a quite appealing, possibility is to
record drumpatterns played live by the human drummer on drum pads
using a MIDI drumtrigger module and afterwards transform these MIDI
data into the format of
-
958 M. Dostal
multichromosomes. Unfortunately, the transformation must be done
manually atthe present since the conversion from MIDI to
multichromosomes is not yet imple-mented in GeneticDrummer.
7.2 Genetic Algorithm
Basically, GeneticDrummer utilizes the well known Simple Genetic
Algorithm(SGA) [22] that is adjusted to the above introduced
representation, musically mean-ing fitness and mutation. Standard
two parent, one-point crossover is used to createnew individuals.
Crossover performs on multichromosomes, that is each chromo-some
within the multichromosome is crossed at the same position. Two
offspringmultichromosomes are produced as a result of crossover.
Reproduction is performedusing the standard roulette-wheel
proportional selection.
Mutation and fitness is the crucial part of GeneticDrummers
genetic algorithmthat is most responsible for producing acceptable
results. Both mutation and fitnessare functions (written in
CommonLisp as the GeneticDrummer itself) that performmutation of
multichromosomes or fitness evaluation of multichromosomes,
respec-tively. In mutation, several the so called mutation
operators can be used. Mutationoperators are built-in functions
that perform mutation at the level of chromosomes.Similar to
mutation operators, a fitness function can utilize several built-in
fitnessoperators. Fitness operators also operate on chromosomes.
Thus, a fitness functionshould return a number between -100 and
100. The higher value is returned thehigher quality the
multichromosome is. Fitness values above 100 are cut to 100
andvalues below -100, non-numeric values or evaluations that
resulted into an error areset to -100. Since the majority of the
fitness and mutation operators performs onchromosomes with regard
to the selected track of accompanying music, the opera-tors usually
take a chromosome, a measure of accompanying music and some
othervalues specific to the particular operator as parameters. Some
fitness operators, how-ever, does evaluate the chromosome according
to the users specified ideal resultinstead of a measure of
accompanying music. The fitness operators are following(the ai
parameter represents a measure of accompanying music,
chromosomerepresents a chromosome, that is a measure of a
particular percussion instrument,ideal-value parameter represents
the desired level of particular musical effectevaluated by the
fitness operator, such as the rhythm density for instance. Ideal
valueis a number between 0 and 1, where 0 represents no occurrence
of a particular effectand 1 represent maximum occurrence of such
effect): (rhythm-conformity ai chromosome) evaluates a rhythm
consonance
between the chromosome and the accompanying instrument. This
operator isused typically to evaluate a rhythm consonance between
the bass drum and bassguitar, for instance. It is also great in
generating solos.
(rhythm-unconformity ai chromosome) is an inverse operator
torhythm-conformity. Rhythmically dissonant chromosomes are
preferred.
(dynamic-conformity ai chromosome) evaluates the dynamic
con-formity between the chromosome and accompanying music. This
operator is
-
Evolutionary Music Composition 959
used to evaluate dynamic phrasing between drums and accompanying
music orevaluating crescendos.
(instrument-quantity ideal-value chromosome) evaluates
thedensity of drum strokes according the user specified
ideal-value. The operatoralso takes care about multiple drum
strokes, thus double strokes are consideredmore rhythmically dense
than single strokes, for instance. This operator is usefulto let
the user specify how dense drum breaks or bass-drum (e.g., double
bassdrum cadences typical to heavy metal drums) patterns are
preferred.
(instrument-quantity-on-strong-beats ideal-valuestrong-beats
chromosome) operates similar to the previous operator, butthe
evaluation is performed on the strong beats only. This operator is
used forevaluating a syncopation.
Return values of the fitness operators are between 0 and 1. Zero
means least suc-cess and 1 means maximum success. Mutation
operators counterpart fitness oper-ators. The following mutation
operators perform on individual positions (rhythm-elements) in
chromosomes with a given probability. On each position a
probabilistictest is performed in order to decide whether to apply
the operator on the position orlet the position unchanged. The
following mutation operators are available:
(mut-rhythm-element-change p chromosome) alters a drum
stroketype with given probability p. For instance, it can alter
single stroke with the flamstroke.
(mut-accentuate p ai chromosome) accentuates rhythm elements
ac-cording to the rhythm of accompanying music with given
probability p. Thisoperator is useful in generating breaks to
imitate the phrasing of accompanyingmusic.
(mut-bass-teamwork p ai chromosome) improves the rhythm
con-sonance between a chromosome (that is a particular percussion
instrument) andaccompanying music. The operator is applied with a
given probability p.
(mut-syncopize p ai chromosome strong-beats) performs sim-ple
syncopation on the chromosome by means of moving a rhythm element
froma strong beat one position forward or back. This operator is
particularly used inswing tunes to imitate the effect of
syncopation.
Fitness as well as mutation operators are described in much more
detail [17] in-cluding the corresponding formulas. Fig. 11 depicts
a typical example of a muta-tion function in GeneticDrummer. This
mutation function with probability of 6%changes the rhythm elements
of crash cymbal 1 (notes-cc-1) and bass drum(notes-bass) to the
rhythm of accompanying instrument. This is typical for gen-erating
the end of multi-measure breaks, when the break ends with some
ostinatostrokes on cymbals and bass drum.
Fig. 12 depicts an example of a fitness function. This fitness
function prefers alight accompaniment on snare notes-snare) and
tom-toms (notes-tt-10,notes-tt-12, notes-tt-13) that rhythmically
follows the accompanyinginstrument.
-
960 M. Dostal
(lambda (music)(mutation-bass-teamwork notes-cc-1 0.06
music)(mutation-bass-teamwork notes-bass 0.06 music))
Fig. 11 Mutation function example
(lambda (music)(+ (* 10 (instrument-quantity notes-snare
0.2))
(* 10 (instrument-quantity notes-tt-10 0.2))(* 10
(instrument-quantity notes-tt-12 0.2))(* 10 (instrument-quantity
notes-tt-13 0.2))(* 20 (rhythm-conformity notes-snare music))(* 20
(rhythm-conformity notes-tt-10 music))(* 20 (rhythm-conformity
notes-tt-12 music))(* 20 (rhythm-conformity notes-tt-13
music))))
Fig. 12 Fitness function example
One run of the genetic algorithm produces one measure long
rhythm accompani-ment to one measure of accompanying music. The
inputs to genetic algorithm arefollowing: a population, a fitness
function, a mutation function, a measure of accom-panying music and
the parameters applied to genetic algorithm such as the numberof
generations, or starting position for generating a break. After
passing the givennumber of generating the individual with highest
fitness is returned as result.
The rhythm accompaniment to a complete phrase is generated by
the repeatedapplication of the genetic algorithm with the same
parameters except the measurewith accompanying instrument which is
obviously set to the next measure beforegenerating next measure of
rhythm accompaniment. After generating all measuresof the rhythm
accompaniment, the genetic algorithm is executed once again in
or-der to generate a break. A break population is used instead the
groove populationand also the fitness and mutation is defined
specifically for generating a break. Af-terwards the last measure
of generated groove and generated break are combinedtogether
according to the system settings: a user can set at which bar
should thebreak start.
7.3 Interaction with GeneticDrummer
To generate a rhythm accompaniment to a complete tune, the user
is obliged to selectan accompanying instrument to which the drum
accompaniment is to be generated.Then the user cuts the track with
the accompanying instrument into phrases of arbi-trary length.
Typically, the phrases correspond to verses and choruses. Phrases
canhave different length. For each phrase, the user defines
parameters to the geneticalgorithm, as it was described earlier.
The real interaction with GeneticDrummer
-
Evolutionary Music Composition 961
is relatively easy for the user. GeneticDrummer communicates
with the CakewalkProAudio, a well known DAW application.
The author is developing a new, more advanced version of
GeneticDrummer thatis about to be introduced in 2012.
Listening
GeneticDrummer has been used in various genres, such as jazz,
rock or latin. Note,that the below presented results represent a
rough result from the GeneticDrummerand thus they were not edited
afterwards by the user in any way, except for soundprocessing.
1. Chicken, funk classic by Pee Wee
Ellis.http://dostal.inf.upol.cz/data/mp3/chicken-2011.mp3The bass
guitar part was used as music accompanying track to which the
drumaccompaniment was evolved.
2. A Night in Tunisia, a jazz standard written by Dizzy
Gillespie.http://dostal.inf.upol.cz/data/mp3/nit.mp3
3. Blue Bossa, an instrumental jazz tune written by Kenny
Dorham.http://dostal.inf.upol.cz/data/mp3/bluebossa.mp3
4. Summertime, from Porgy and Bess opera written by George
Gershwin.http://dostal.inf.upol.cz/data/mp3/summertime.mp3
5. Sockshop, a typical rock and roll
tune.http://dostal.inf.upol.cz/data/mp3/sockshop.mp3Used groove
population:http://dostal.inf.upol.cz/data/mp3/sockshop-groove-pop.mp3Poulation
used for
breaks:http://dostal.inf.upol.cz/data/mp3/sockshop-break-pop.mp3.
8 Conclusion
Evolutionary music composition has become widely used and
popular techniquein algorithmic composition. Seemingly, the
interest of researchers is growing; thenumber of various
conferences, workshops and books on algorithmic compositionis
growing. Also, there are artists interested in utilization of
algorithmic compositiontechniques in their artworks. Some music
composition systems, such as GenDash orGenJam have appeared in
various compositions or live performances. GenJam hasbeen used on
around 300 various jazz tunes. The system is also used by the
authorin his VirtualQuintet on various recordings, tutorials and
live gigs.
However, there is a field of problems that require further
systematic investigationincluding finding promising representation,
measurement of aesthetic criteria or de-veloping a theoretical
background on generative art. McCormack in his work [37]formulates
and discusses five open problems in evolutionary art in detail.
-
962 M. Dostal
References
1. Alpern, A.: Techniques for algorithmic composition of music
(1995)2. Ames, C.: The markov process as a compositional model: A
survey and tutorial3. Ames, C.: Quantifying musical merit.
Interface 21, 5393 (1992)4. Ames, C., Domino, M.: Cybernetic
composer: an overview, pp. 186205. MIT Press,
Cambridge (1992)5. Amnuaisuk, S.P., Tuson, A., Wiggins, G.:
Evolving musical harmonisation. In: Proceed-
ings of the 4th International Conference on Artificial Neural
Networks and Genetic Al-gorithms, Springer (1999)
6. Amnuaisuk, S.P., Wiggins, G.A.: The four-part harmonisation
problem: A comparisonbetween genetic algorithms and a rule-based
system. In: Proceedings of the AISB 1999Symposium on Musical
Creativity, pp. 2834. AISB (1999)
7. Baggi, D.L.: Neurswing: an intelligent workbench for the
investigation of swing in jazz.Computer 24, 6064 (1991)
8. Biles, J.A.: GenJam: A genetic algorithm for generating jazz
solos (1994)9. Biles, J.A.: Genjam populi: Training an iga via
audience-mediated performance (1995)
10. Biles, J.A.: Interactive genjam: Integrating real-time
performance with a genetic algo-rithm. In: Proceedings of the 1998
International Computer Music Conference, ICMC1998 (1998)
11. Biles, J.A.: Life with genjam: Interacting with a musical
iga. In: Proceedings of the 1999IEEE International Conference on
Systems, Man, and Cybernetics (1999)
12. Biles, J.A.: Autonomous GenJam: Eliminating the fitness
bottleneck by eliminating fit-ness (2001)
13. Biles, J.A.: GenJam: evolution of a jazz improviser, pp.
165187. Morgan KaufmannPublishers Inc., San Francisco (2002)
14. Biles, J.A., Anderson, P.G., Loggi, L.W.: Neural network
fitness functions for a musi-cal ga. In: Proceedings of the
International ICSC Symposium on Intelligent IndustrialAutomation
(IIA 1996) and Soft Computing (SOCO 1996), pp. 3944. ICSC
AcademicPress (1996)
15. Burton, A.R., Vladimirova, T.: Generation of musical
sequences with genetic techniques.Computer Music Journal 23(4),
5973 (1999)
16. Camboropoulos, E.: Markov chains as an aid to computer
assisted composition. MusicalPraxis 1(1), 4152 (1994)
17. Dostal, M.: Genetic algorithms as a model of musical
creativity - on generating of ahuman-like rhythmic accompaniment.
Computers and Artificial Intelligence 24(3), 321340 (2005)
18. Dostal, M.: The genetic drummer plays funk! In: ArtEscapes:
Variations of Life in theMedia Arts, Catalogue of the EvoMUSART
2007 Exhibition (2007)
19. Dostal, M.: Towards representation of rhythm in genetic
algorithm. In: New Trends inArtificial Intelligence, Proceddings of
EPIA 2007, 13th Portuguese Conference on Arti-ficial Intelligence.
Associacao Portuguesa para a Inteligencia Artificial (APPIA)
(2007)
20. Ebcioglu, K.: An expert system for harmonizing four-part
chorales, pp. 385401. MITPress, Cambridge (1992)
21. Gibson, P.M., Byrne, J.A.: NEUROGEN: musical composition
using genetic algorithmsand cooperating neural networks. In: Second
International Conference on Artificial Neu-ral Networks, pp.
309313. IEEE, New York (1991)
22. Goldberg, D.E.: Genetic Algorithms in Search, Optimization
and Machine Learning, 1stedn. Addison-Wesley Longman Publishing
Co., Inc., Boston (1989)
-
Evolutionary Music Composition 963
23. Harley, J.: Algorithms adapted from chaos theory:
Compositional considerations.In: Proc. of the 1994 International
Computer Music Conference, Aarhus, Denmark,pp. 209212 (1994)
24. Herman, M.: Deterministic chaos, iterative models, dynamical
systems and their appli-cation in algorithmic composition. In:
Proceedings of the International Computer MusicConference
(1993)
25. Hild, H., Feulner, J., Menzel, W.: Harmonet: A neural net
for harmonizing chorales in thestyle of j. s. bach. In: Moody,
J.E., Hanson, S.J., Lippmann, R. (eds.) NIPS, pp. 267274.Morgan
Kaufmann (1991)
26. Hoover, A.K., Rosario, M.P., Stanley, K.O.: Scaffolding for
interactively evolving noveldrum tracks for existing songs. In:
Proceedings of the 2008 Conference on Applicationsof Evolutionary
Computing, Evo 2008, pp. 412422. Springer, Heidelberg (2008)
27. Hoover, A.K., Szerlip, P.A., Stanley, K.O.: Interactively
evolving harmonies throughfunctional scaffolding. In: Proceedings
of the 13th Annual Conference on Genetic andEvolutionary
Computation, GECCO 2011, pp. 387394. ACM, New York (2011)
28. Horner, A., Ayers, L.: Harmonisation of musical progressions
with genetic algorithms.In: Proceedings of the 1995 International
Computer Music Conference, pp. 483484.ICMA, San Francisco
(1995)
29. Horner, A., Goldberg, D.E.: Genetic algorithms and
Computer-Assisted music composi-tion. In: International Conference
on Genetic Algorithms, pp. 437441 (1991)
30. Horowitz, D.: Generating rhythms with genetic algorithms.
In: Proceedings of the 1994International Computer Music Conference,
pp. 142143. ICMA, San Francisco (1994)
31. Jacob, B.L.: Composing with genetic algorithms. In:
International Computer Music Con-ference (January 1995)
32. Jacob, B.L.: Algorithmic composition as a model of
creativity. Org. Sound 1, 157165(1996)
33. Johanson, B., Poli, R.: GP-music: an interactive genetic
programming system for mu-sic generation with automated fitness
raters. In: Proceedings of the Third InternationalConference on
Genetic Programming (1998)
34. Keller, R.M., Morrison, D.R.: A grammatical approach to
automatic improvisation.Technical report
35. Koza, J.R.: Genetic programming - on the programming of
computers by means of nat-ural selection. In: Complex Adaptive
Systems. MIT Press (1993)
36. Livio, M.: The Golden Ratio: The Story of Phi, the Worlds
Most Astonishing Number.Broadway Books (2002)
37. McCormack, J.: Open Problems in Evolutionary Music and Art.
In: Rothlauf, F., Branke,J., Cagnoni, S., Corne, D.W., Drechsler,
R., Jin, Y., Machado, P., Marchiori, E., Romero,J., Smith, G.D.,
Squillero, G. (eds.) EvoWorkshops 2005. LNCS, vol. 3449, pp.
428436.Springer, Heidelberg (2005)
38. McIntyre, R.A.: Bach in a box: The evolution of four part
baroque harmony using thegenetic algorithm. In: International
Conference on Evolutionary Computation, pp. 852857 (1994)
39. Moore, J.H.: Gamusic: Genetic algorithm to evolve musical
melodies (1994)40. Ralley, D.: Genetic algorithms as a tool for
melodic development. In: Proceedings of the
International Computer Music Conference, Banff, Canada, pp.
501502 (1995)41. Roads, C.: Grammars as representations for music.
In: Roads, C., Strawn, J. (eds.) Foun-
dations of Computer Music, pp. 403442. MIT Press, Cambridge
(1985)
-
964 M. Dostal
42. Spector, L., Alpern, A.: Criticism, culture, and the
automatic generation of artworks. In:AAAI 1994: Proceedings of the
Twelfth National Conference on Artificial Intelligence,vol. 1, pp.
38. American Association for Artificial Intelligence, Menlo Park
(1994)
43. Spector, L., Alpern, A.: Induction and recapitulation of
deep musical structure. In: Pro-ceedings of the IFCAI 1995 Workshop
on Artificial Intelligence and Music, pp. 4148(1995)
44. Thywissen, K.: Genotator: an environment for investigating
the application of geneticalgorithms in computer assisted
composition. In: Proceedings of the 1996 InternationalComputer
Music Conference, pp. 274277. ICMA, San Francisco (1996)
45. Tokui, N., Iba, H.: Music composition with interactive
evolutionary computation. In:Proceedings of the Generative Art
International Conference, Milan, Italy (2000)
46. Tsang, C.P., Aitken, M.: Harmonizing music as a discipline
of constraint logic program-ming. In: Proceedings of then
International Computer Music Conference (1991)
47. Unemi, T.: A design of genetic encoding for breeding short
musical pieces. In: Workshopon Artificial Life Models for Musical
Applications II: Search for Musical Creativity,pp. 2529 (2002)
48. Unemi, T.: Sbeat3: a tool for multi-part music composition
by simulated breeding. In:Proceedings of the Eighth International
Conference on Artificial Life, pp. 410413. MITPress, Cambridge
(2003)
49. Waschka, R.: Avoiding the fitness bottleneck using genetic
algorithms to composeorchestral music. In: Proceedings of the
International Computer Music Conference,pp. 201203 (1999)
50. Waschka, R.: Composing with Genetic Algorithms: GenDash, ch.
6, pp. 117136.Springer (2007)
51. Waschka, R.: Theories of evolutionary algorithms and a new
simplicity opera: Makingsapphos breath. In: Artificial Life Models
for Musical Applications, pp. 7986 (2011)
52. Wiggins, G.A.: Automated generation of musical harmony:
whats missing? In: Pro-ceedings of the International Joint
Conference in Artifical Intelligence 1999, IJCAI 1999(1999)
53. Xenakis, I.: Formalized music thought and mathematics in
composition. Indiana univer-sity press (1971)
54. Xenakis, I.: Formalized music: thought and mathematics in
composition. Harmonologiaseries. Pendragon Press (1992)
Evolutionary Music CompositionIntroductionBrief HistoryMusic
Composition SubtasksMethods and Techniques for Algorithmic
CompositionInteractive versus Autonomous Systems
GP-MusicGenDashGenJamPopulationRecombination Operators
Phon-Amnuaisuk's Harmonization SystemCONGAGenerating
PhrasesArranging Phrases to PatternsOperating with CONGA
GeneticDrummerRepresentation of RhythmGenetic
AlgorithmInteraction with GeneticDrummer
ConclusionReferences