povel/Publications/MusicRelatedArticles/2007Povel

Testing tonal music theory 1

Artificial Music Construction as a Method to Test Music Theory

Dirk-Jan Povel

Nijmegen Institute for Information and Cognition

Correspondence to: Dirk-Jan Povel Nijmegen Institute for Cognition and Information P.O. Box 9104 6500 HE Nijmegen The Netherlands Email address: [email protected]


Abstract Artificial music construction based on music theoretical insights is advanced as a

method for testing tonal music theory. The method includes the following stages: selection of music-theoretical insights, translation of these insights into a formal model of rules, translation of the rules into an algorithm for music generation, and testing the ‘music’ generated by the algorithm to evaluate the adequacy of the model. An illustration of the method is presented by an algorithm, Melody Generator, which is based on constraints imposed by key and meter, and on a number of basic construction principles. The tone sequences produced by the algorithm do resemble tonal melodies in several respects, but show distinct limitations as well. Shortcomings of the current model are identified and ensuing further developments discussed.


Introduction Music theory has yielded a huge amount of concepts, notions, insights, and

theories regarding the structure, organization and functioning of music. But only since the beginning of the 20th century music became a topic of rigorous scientific research in the sense that aspects of the theory were formally described and became subject of experimental research (e.g. Van Dyke Bingham, 1910). Since the cognitive revolution of the 1960th (Gardner, 1984) a significant increase in the scientific study of music was seen with the emergence of the disciplines of cognitive psychology and artificial intelligence. This gave rise to numerous experimental studies investigating several aspects of music perception and music production (for overviews see Deutsch, 1999; Krumhansl, 2000), and to the development of formal and computational models describing and simulating various aspects of the process of music perception, e.g. meter and key induction, harmony induction, segmentation, coding, etc., and music representation (e.g. Deutsch & Feroe, 1981; Hirata & Aoyagi, 2003; Lerdahl & Jackendoff, 1983; Longuet-Higgins & Steedman, 1971; Marsden, 2005; Povel, 1981, 2000; Temperley, 1997, 2001).

The method In this article I propose an alternative method to test theories of tonal music. The

core of the method consist in the generation of music on the basis of insights accumulated in theoretical and experimental music research. The rationale behind the method is straightforward: if we have a theory about the mechanism underlying some phenomenon, the best way to establish the validity of the theory, is to show that we can reproduce the phenomenon from scratch. Applied to music: if we have a valid theory of music, we should be able to construct music from its basic elements (sounds differing in frequency and duration), at least in some elementary fashion.

Basically the method works as follows: starting from theoretical insights in the phenomenon of music, a set of rules for the generation of music is derived, beginning with global and simple rules that are gradually extended and refined by additional rules. Together, these rules form a model of music theory. This model is then used to develop a computer algorithm for generating music comprising adjustable parameters associated with the various aspects of tonal music: rhythm, harmony, and melody. The validity of the model is estimated by evaluating the output of the algorithm, thereby revealing in what respects the theory is still deficient. The latter may then give rise to adaptation of the model and the algorithm. The method thus follows the well-known empirical cycle.

Compared to the experimental method, the artificial construction method has two advantages: 1) it studies all aspects of music in a comprehensive way (because all aspects have to be taken into account if one wants to generate music) thus enabling to see the ‘big picture’: the working and interaction of all variables; 2) its need for precision, enforced by the fact that the model is implemented as a computer algorithm. Experimental research in music, because of limitations of the method, typically focuses on a restricted domain within the field of music, taking into account just one or two variables. As a result of this, the interpretation of the experimental results and their significance for the understanding of music as a whole, is often open to discussion.


Related work The possibility to create music by means of computer algorithms, so-called

algorithmic composition, has attracted a lot of interest in the last few decades (see for instance: Cope, 1996; Miranda, 2001; Rowe, 2004). Algorithmic composition employs various methods (Papadopoulos & Wiggins, 1999): Markov chains (e.g. Ames, 1989; Cambouropoulos, 1994); Knowledge-based systems (e.g. Ebcioglu, 1988); Grammars (Baroni & Jacoboni, 1978; Cope, 1996, 2005; Steedman, 1984, Sundberg & Lindblom, 1976); Evolutionary methods (genetic algorithms, e.g. Wiggins, Papadopoulos, Phon-Amnuaisuk, & Tuson, 1998; interactive genetic algorithms e.g. Biles (1994); and learning systems (e.g., Todd, 1989; Toiviainen, 1995).

The method presented here also generates music and can thus be seen as an instance of algorithmic composition. However, since its purpose is to serve as a means to test music theory, it does not rely on classical AI methods but capitalizes on music theoretical insights to guide the implementation of a music generating device.

Organization of the paper The article is organized as follows: First, the theoretical foundation of the method

is described: the theoretical starting points, the basic assumptions, the insights regarding the configuration of the time dimension and the ensuing constraints, the configuration of the pitch dimension and the ensuing constraints, and the basic construction rules for generating melodies. Second, the implementation of the algorithm is discussed: the various functions of the program, its underlying structure, and the user interface. Next, the successive steps of constructing a simple and a multi-part melody are described, followed by the description of two dedicated subroutines, one for constructing meter based rhythms, the other for ornamenting a skeleton melody. Finally, the present stage of the model is discussed, and suggestions for future developments of both model and algorithm are presented.

Theoretical foundation Starting point of the project is the conception of music as a psychological

phenomenon, i.e., as the result of a unique perceptual process carried out on sequences of pitches. This process has two distinct aspects:

1. Establishing the tonal contexts of meter, key, and harmony, and representing the input within that context. Through this representation the input, consisting of sounds varying in pitch, is transformed into tones and chords having specific musical functions that are perceived as expectations for future tones and chords1. This part of the process is further detailed below.

2. Discovering the structural regularities in the input (e.g. repetition, alternation, reversal of a pattern) and using those to form a mental representation (Deutsch & Feroe, 1981; Lerdahl & Jackendoff, 1983).

1 “Hearing music does not mean hearing tones, but hearing, in the tones and through them, the

places where they sound in the seven-tone system” (Zuckerkandl, 1956 p. 35).


This basic conception of the process of music perception has guided the choice of assumptions and the development of the model of music construction described below.

Basic assumptions The model is based on the following assumptions

1. Tonal music is conceived within the context of time and pitch of which a. Time is configured by meter (imposing constraints as to when notes can

be played) b. Pitch is configured by key and harmony (imposing constraints as to

what notes can be played) 2. Within that context tone sequences are generated using construction rules that

relate to the (hierarchical) organization of the tones into parts and of parts into larger parts etc., relating to concepts such as motives, phrases, repetition and variation, skeleton, structural and ornamental tones etc.

Next, we successively discuss the configuration of the time dimension and the pitch dimension, the interaction between these dimensions, the basic methods of music construction, and the resulting constraints and rules for the artificial construction of tonal music. It should be noted that we have no a priori estimate of the actual number of rules needed to generate a piece of music (also in view of the fact that different styles have different rules). So the approach we follow is to begin by implementing fundamental constraints and rules, which will be extended and refined in later stages. For this reason the initial stages of the project may seem rather simplistic.

The configuration of the time dimension The time dimension in tonal music is configured by meter. Meter is a temporal

framework in which a rhythm is cast. It divides the continuum of time in discrete periods. Note onsets in a piece of tonal music coincide with the beginning of one of those periods.

Meter divides time in a hierarchical and recurrent way: time is divided into time periods called measures that are repeated in a cyclical fashion. A measure in turn is subdivided in a number of smaller periods of equal length, called beats, which are further hierarchically subdivided (mostly into 2 or 3 equal parts) into smaller and smaller time periods. A meter is defined by means of a fraction, e.g., 4/4, ¾, 6/8, in which the numerator indicates the number of beats per measure and the divisor the duration of the beat in terms of note length (1/4, 1/8 etc.), which is a relative duration.

The various positions defined by a meter (the beginnings of the periods) differ in metrical weight: the higher the position in the hierarchy (i.e. the longer the delimited period), the larger the metrical weight. Figure 1 displays two common meters in a tree-like representation indicating the metrical weight of the different positions (the weights are defined on an ordinal scale). As shown, the first beat in a measure (called the down beat) has the highest weight. Phenomenally, metrical weight is associated with perceptual markedness or accentuation: the higher the weight, the more accented it will be.


Figure 1. Tree-representations of one measure of 3/4 and 6/8 meter with the metrical weights of the different levels. For each meter a typical rhythm is shown.

Although tonal music is commonly notated within a meter, it is important to realize that meter is not a physical characteristic, but a perceptual attribute conceived by a listener while processing a piece of tonal music. It is a mental temporal framework that allows the accurate representation of the temporal structure of a rhythm. Meter is obtained from a rhythm (a sequence of sound events with some temporal structure) in a process called metrical induction (e.g. Longuet-Higgins & Lee, 1982; Povel & Essens, 1985). The main factor in the induction of meter is the distribution of accents in the rhythm: the more this distribution conforms to the pattern of metrical weights of some meter, the stronger that meter will be induced. The degree of accentuation of a note in a rhythm is determined by its position in the temporal structure and by its relative intensity, pitch height, duration, and spectral composition, the temporal structure being the most powerful (Gouyon et al. 2006). The most important determiners of accentuation are (1) Tone length as measured by the inter-onset-interval (IOI or note value); (2) Grouping: the last tone of a group of 2 tones and the first and last tone of groups of 3 or more tones are perceived as accentuated (Povel & Okkerman, 1981; Povel & Essens, 1985).

Metrical stability The term metrical stability denotes how strongly a rhythm evokes a meter. We

have seen that the degree of stability is a function of how well the pattern of accents in a rhythm matches the pattern of weights of a meter. This relation may be quantified by means of the coefficient of correlation. If the metrical stability of a rhythm (for some meter) falls below a critical level, by the occurrence of what could be called ‘anti-metric’ accents, the meter will no longer be induced, leading to a loss of the temporal framework and, as a consequence, of the understanding of the temporal structure.

Basic constraints regarding the generating of tonal rhythm Meter imposes a number of constraints on the use of the time dimension when

generating tonal music: First, it determines the moments in time, the locations at which a note can begin; second the actual placement of notes should result in a


rhythm with a reasonably high metrical stability; third, because meter establishes a hierarchy of ‘importance’ of all its locations, a composer has to take into account these differences because metrical positions with a high weight will be more prominent and are therefore, in principle at least, the preferred locations to position melodic or harmonic important notes (e.g. the last note of a cadence).

However, the placement of an unimportant note on a metrically strong position, or a temporary decrease of the metrical stability of a rhythm (e.g. by means of syncopations), may contribute to the rhythmical interest by creating a certain amount of uncertainty.

Having explained the main functions of meter in tonal music, we can now formulate a guide line for generating the rhythmical aspect of tonal melodies:

In constructing tonal music the metrical stability of the rhythm must be sufficiently high to guarantee the induction of a metrical context.

(1)

The incorporation of this rule in Melody Generator is discussed in the section The

algorithm.

The configuration of the pitch dimension The pitch dimension of tonal music is organized on three levels: key, harmony, and

tones, that maintain intricate relationships represented in what is called the tonal system. This system describes the constituent elements and their function: e.g. how close one key is to another, which are the harmonic and pitch elements of a key, how they are related, how they function musically, etc. On all three levels I and V play central roles: as related keys, as the two primary harmonies, and as the two most important and most frequently occurring tones. For a review of the main empirical facts see Krumhansl (1990). It should be noted that the relations between the elements of the tonal system do not exist in the physical world of sounds (although they are directly associated with it), but refer to knowledge in the listener’s long term memory acquired by listening to music.

Key is the highest level of the system and changes at the slowest rate: in shorter tonal pieces like hymns, folksongs, and popular songs there is often only one key that does not change during the whole piece. A key is comprised of a set of chords and a set of tones, each chord and tone having a specific musical ‘function’, that can be described in terms of stability and attraction. In general unstable chords or tones are attracted by stable tones and chords that are close in pitch (Lerdahl, 1988, 2001; Margulis, 2005).

This musical function of tones and chords only becomes available after a listener has recognized the key of the piece in a process analogous to finding the meter, called key induction (for an overview see Temperley, 2007, Ch. 4). Each key has a limited number of basic harmonies, the triads on the 7 degrees, of which those on the 1st, 4th, and 5th degree are the primary ones.

A basic assumption related to the construction of tonal music, is that it is build on a series of harmonies, or harmonic progression. This leads to the following rule:


Tonal music is build upon a harmonic progression (2) This assumption does not imply that each tonal piece must be accompanied by a

underlying series of chords, but it does mean that each piece is conceived in terms of a virtual harmonic progression (Povel & Jansen, 2002).

The underlying harmonic progression Harmony in tonal music has been the subject of an enormous amount of

publications (e.g., Rameau, 1971/1722; Riemann, 1887; Schoenberg, 1978/1911; Schenker, 1980/1954). We begin by introducing a few basic rules regarding the composition of the harmonic progression. These rules are in part based on the fact that the perceiver must know at all times the interpretational context of the melody, the key. The best strategy to induce (and maintain) the key of a piece is to play the tonic chord. This leads to the following rule: The first harmony of a tonal progression will usually be the tonic (3) As the tonic is the most stable chord, it is understandable that a tonal piece ends on the tonic chord. And since the dominant (seventh) chord is an unstable chord that is most appropriately resolved by a tonic chord, it is very often the penultimate chord of a harmonic progression (Boltz, 1989). Together the two chords thus form a final cadence. This leads to two further rules: The last harmony of a tonal progression will usually be the tonic (4) The penultimate chord of a tonal progression will usually be some form of the dominant chord

(5)

We might mention here that the latter 3 assumptions incorporate Schenker’s notion

that tonal music is constructed by applying transformations on a ‘Background’ consisting of a I-V-I harmonic progression in the Bass (Ursatz) and a descending melodic line (Urlinie), (Brown, 1989, 2005; Jonas, 1934).

Further elaboration of the harmonic progression As for the further definition of the underlying harmonic progression, we mention

three alternatives of increasing sophistication.

Piston’s Table In his ‘Table of usual root progressions’, Piston (Piston & DeVoto, 1989, p. 21 and

44) lists the frequency with which a given chord is followed by an other chord in tonal music. Frequency of occurrence is loosely defined by the terms ‘usually’, ‘sometimes’ and ‘less often’. See Table 1. Since in this Table the choice of a chord is only determined by the previous chord, it only provides a first-order approximation of harmonic progressions in tonal music.


Table 1. The table of usual root progressions for the Major mode from Piston & DeVoto. Given a chord in the first column, the table shows the chords that succeed that chord, usually (1st column), sometimes (2nd column), or less often (3rd column).

Chord Is followed by

Sometimes by

Less often by

I IV or V VI II or III II V IV or VI I or III III VI IV I, II or V IV V I or II III or VI V I VI or IV III or II VI II or V III or IV I VII III I -

The phrase model theory According to this theory phrase construction is based on a harmonic progression of

the type I-PD-V-(I) (e.g. Laitz, 2003, p. 229 ff.), in which I and V respectively denote a tonic harmony (including possible elaborations) and a dominant harmony (plus possible elaborations). PD stands for a predominant harmony (triads on degrees other than I or V, especially the triad on IV). The model can be viewed as an extension of Schenker’s I-V-I Ursatz. This model is at present being implemented in the algorithm described below.

Sutcliffe’s harmonic phrase structure theory This theory assumes that the chord progressions found in tonal music consist of

structural and non-structural chords (such as passing chords), a notion shared with many other theories on harmony. Its central thesis is that after ornamental chords are eliminated from a chord progression, one basic phrase structure remains comparable to those found in language. This phrase has the following structure (see Figure 2):

Figure 2. Harmonic phrase structure according to Sutcliffe. See text for explanation.


A Phrase consists of two parts: an Opening section and a Closing section. The Opening section contains Static Harmony, basically tonic chords alternating with auxiliary chords. The Closing section consists of Dynamic Harmony and a Cadence. Dynamic Harmony is made up mostly of “strong root progressions” of three types: rising 4th, falling 3rd, and rising 2nd. The Cadence, finally, consists of a Dominant section and a Tonic section both of which can be prolonged by auxiliary chords. This basic structure can be extended in various ways by means of auxiliary chords. A detailed description of the theory can be found at http://www.harmony.org.uk/. This model is at present being implemented in the algorithm described below.

Basic constraints regarding the use of tones in tonal melodies A consequence of the assumption that tonal music is build upon a harmonic

progression is that the (majority of) tones associated with a harmony must be elements of that harmony, as other tones would clash with the harmony:

The pitches in a musical fragment associated with some harmony, are predominantly chord tones

(6)

Although there are melodies only existing of chord tones, these are rare; most melodies also comprise non-chord tones such as passing tones, neighbor tones, suspensions, etc. As these tones form dissonants with the harmony, it is actually surprising that they are tolerable at all. It appears however, that non-chord tones are acceptable if they are ‘anchored’ by (resolved to) a succeeding chord tone close in pitch (Bharucha, 1984; Povel & Jansen, 2001). This leads to the following rule: Non-chord tones must be resolved to a following chord tone close in pitch (7) Most often a non-chord tone is anchored immediately, i.e., by the next succeeding tone, but it can also be anchored with some delay, as in the case of F A G, in which G is a chord tone and F and A are non-chord tones. Sometimes, the anchoring of a non-chord tone may be delayed much longer, but in those cases the intermediate tones are often in another voice.

Relations between meter and key So far we have described the major characteristics of the context within which

tonal music is constructed: Meter defines the positions at which notes may begin, as well as the perceptual salience of these positions expressed by their metrical weight. Key defines the basic elements of tonal music: tones and chords, their relation and their musical function. Tones differ in stability. The more stable a tone, the more important its role in the melody.

Now it seems logical that stable pitches in order to enforce their important role, tend to appear at metrical strong positions, as expressed in the following rule.


Stable pitches tend to occur on metrically strong positions (8)

Basic construction rules After having described the global constraints associated with the tonal context:

meter and key, we now discuss the basic construction rules for generating tonal melodies.

A commonly accepted notion regarding tonal melodies is that they consist of parts, phrases, or sections having specific relations: Tonal melodies consist of parts and subparts (9)

The relations between parts is established basically by means of repetition and

varied repetition of earlier parts. Repetition and varied repetition are basic means of constructing parts from previous parts

(10)

Analogous to the assumption made above for the construction of a harmonic

progression, we assume that a tonal melody consists of structural tones (the skeleton melody) and ornamental tones. The notion of a skeleton melody is widely found among authors writing about tonal music (e.g. Zuckerkandl, 1973/1956; Toch, 1977/1948; Salzer, 1982). It is directly related to Lerdahl & Jackendoff’s (1983) reduction hypothesis: “The listener attempts to organize all the pitch events of a piece into a single coherent structure, such that they are heard in a hierarchy of importance” (p. 106), although the skeleton notion is less hierarchical. Tonal melodies consist of structural tones and ornamental tones (11)

Above we have already stated that structural tones tend to fall on strong beats.

Here we add one more rule related to the identity of the structural tones: Structural tones tend to be chord tones (12)

The actual construction of a melody, including the generation of the harmonic

progression, the rhythm, the contour, the definition of a basic melody consisting of structural tones, and the ornamentation of a basic melody, is described in the next section.


The algorithm As an illustration of the method a computer algorithm has been designed for the

generation of tonal melodies. A choice was made for melodies rather than complete pieces of music to keep the problem manageable. Moreover, melodies form the core of tonal music, and most aspects of music construction are involved in their generation. The algorithm, Melody Generator II, is programmed in REALbasic™ (Neuburg, 1999), an object-oriented language that builds applications for Macintosh, Windows, and Linux platforms from a single source code2.

The main functions of the algorithm are: the generation and editing of melodies3, the storage and retrieving of melodies (both temporarily within the program and on/from disk), the displaying of melodies, and the rendering of melodies. Most of these functions are accessed by means of the three main windows on the interface.

The Score Display Window comprises the score of the generated melody, controls for playing the melody, and a few additional controls. See Figure 3.

Figure 3. Score Display Window of Melody Generator II

The Master Control Window contains controls for setting the parameters of the time dimension (e.g. parameters related to meter and rhythm), the pitch dimension (e.g., parameters related to key and harmony), and controls that open windows for alternative ways of constructing melodies. The Window is shown in Figure 4.

2 The program can be downloaded freely at http://www.nici.ru.nl/~povel/ 3For simplicity’s sake I use the word melody for each tone sequence produced by the algorithm,

although this is of course not correct: The very question of the research is to discover whether the algorithm can generate genuine melodies.


Figure 4. Master Control Window of Melody Generator II The Melody Store Window is used for the temporarily storage of melodies. From

this window melodies can be played, send back to the Score Display Window for further processing, or saved to a file on disk. Melody Store Window is shown in Figure 5

Figure 5. Melody Store Window of Melody Generator II.


Apart from these three main windows, there are a dozen additional windows used for different functions like displaying the current KeySpace, displaying properties of the notes in the melody, editing the melody (e.g. change the rhythm, the harmony, or pitches), changing the display parameters, displaying information about the operation of the program, etc. Apart from that, there is a Menu bar with Menu Items for activating various functions.

For the generation of the melodies we needed a structure allowing the flexible construction and modification of a hierarchically organized temporal sequence consisting of parts and subparts, divided into measures, in turn divided into beats and ‘slots’ (locations where notes may be put). For this purpose an object called Melody, was designed comprising one or more instances of the objects Piece, Part, Bar, Beat, Slot, and Note. See Figure 6. Because a part may have a subpart, the hierarchy of a piece can be extended indefinitely.

Figure 6. The object Melody. Also needed is a structure containing all information pertaining to the elements of

the currently selected key. This includes information concerned with the specification of the scale degrees, their stability and their spelling, the various chords on these degrees, etc. This information is accumulated in the object KeySpace. Figure 7 displays one octave of an actual KeySpace.


Figure 7. Part of the object KeySpace used in the construction of melodies in

Melody Generator II.

The generation of a melody in Melody Generator II At present Melody Generator II has 3 different modes for generating a melody.

Mode 1 This mode is used when pressing the button New Melody in the Score Display

Window. A melody will be generated within the tonal context set in the Master Control Window. This context pertains to both the temporal dimension (meter, syncopation, rhythmical density, number of bars, etc.), and the pitch dimension (key, harmonic progression, chordtone-nonchord tone distribution, range, etc.). The melody is then generated by successively constructing or defining an aspect of the melody in the following order: 1) a rhythm (according to the parameters set), 2) a harmonic progression (as set by the harmony selector), 3) a contour (random in this mode), 4) assigning chordtone and non-chordtone locations, and finally 5) assigning pitches to the notes in the rhythm. The stepwise build up of a melody can be followed by checking the by step selector in the Score Display Window.

This mode (which was the first one developed) is useful for evaluating the effects of the various parameters on the resulting tone sequences. It should be noted that in this mode, assignment of pitches is only determined by the current harmony, by whether or not a note is a chordtone or non-chordtone, by the contour and by the range of the melody. Since the chordtone and non-chordtone locations are assigned randomly (though non-chord tones cannot occur at strong beats), as is the contour, the generation of a melody in this mode is mainly determined by the metrical and key context, and not by any construction rules proper. These are introduced in the next modes. A few sequences constructed within this mode are shown in Figure 8.


Figure 8. Examples of tone sequences generated in Mode 1. a) sequence of pitches

randomly chosen from the chromatic scale between C4 and G5. b) sequence of pitches randomly chosen from the diatonic scale (of C major) between C4 and G5. c) sequence based on a I-IV-V-I harmonic progression but with no metrical constraints. d) sequence based on a 4/4 meter but pitches chosen randomly from the chromatic scale. e-g) Sequences witch both metrical and harmonic (either I-IV-V-I or I-II-V-I) constraints increasing in rhythmical density.

Mode 2 In this mode a single melody is constructed based on the rule that a melody

consists of structural tones (the skeleton) and ornamental tones. This mode can be accessed by selecting the Skeleton Window (shown in Figure 6) in the Master Control Window.


Figure 9. The Skeleton Window for the generation of ornamented skeleton

melodies. The main difference with a Mode 1 melody is that after the parameters for rhythm,

harmony and contour are set, a skeleton melody consisting of chord tones is generated (either on the beats or on the downbeats only). For the contour a choice can be made between the following shapes: Ascending, Descending, U-shaped, Inverted U-shaped, Sinusoid, and Random (based on Huron, 1996; Toch, 1977, chapter IV and V). As an example, Figure 10 presents a skeleton melody and a three ornamentations of the skeleton differing in density.

Figure 10. Example of melodies generated in Mode 2. a) A skeleton melody. b-d)

the same skeleton elaborated with increasing rhythmical density i.e. an increasing number of ornamental notes.


There are a few additional parameters to be mentioned briefly: 1) The length of the ‘gap’ at the end of the melody determining the number of empty beats in the last bar. 2) Steps or leaps. For the construction of the skeleton, one may select either by step or by step and leap. In the first case, the intervals between consecutive tones have the size of a step on the triadic level (remember that the skeleton tones are chord tones), in the second case the intervals may have the size of a step or a leap (in some proportion) thus increasing the variation of interval size. 3) Rhythm. It is possible to either change the rhythm (thereby maintaining the setting in the Master Control Window), or to add notes to a rhythm by means of the selector next to the Ornament Slots button. 4) Finally in making the ornamentation it is also possible to use either steps or steps and leaps.

Mode 3 This mode was developed for constructing hierarchically organized melodies, that

is, melodies consisting of a number of parts. The hierarchical organization of tonal music has been described by many authors (e.g., Bamberger, 2000; Lerdahl & Jackendoff, 1983; Schoenberg, 1967). Bamberger describes the hierarchical organization of tunes in terms of trees consisting of motives, phrases, and sections connected by means of 3 ‘organizing principles’: repetition, sequential relations, and antecedent-consequent relations. Lerdahl & Jackendoff (1983) describe the organization of tonal pieces in terms of hierarchical trees, either resulting from a time-span reduction (based on the rhythmical structure) and a prolongational reduction expressing harmonic and melodic patterns of tension and relaxation. Schoenberg (1967) describes the hierarchical organization of classical music on several levels: large forms (e.g. sonata form), small forms (e.g., minuet), themes (e.g., the period and the sentence consisting of phrases and motives). Here only the latter organization is relevant.

Construction of a multi-part melody employs the scheme displayed in Figure 11. Parameter setting is accomplished by means of the controls in the Phrase-based Window, shown in Figure 12.

Figure 11. Steps in the construction of a multi-part melody As shown in Figure 11, first the global parameters pertaining to the context (meter

and key) are set. Next the Melody structure is determined, defined as a sequence of parts, for each of which the duration, the size of the gap at the end of the part, the underlying harmonic progression, the contour (either Ascending, Descending, U-


shaped, Inverted U-shaped, Sinusoid, and Random), the positioning of the skeleton and the mode of ornamentation is specified (see Figure 12).

Figure 12. The Phrase-based Window for the construction of multi-part melodies. As can be seen in Figure 12, are the following Part types (called Phrases) are

distinguished in the construction of a multi-part melody: PhraseA, PhraseA’, PhraseB, Half Cadence, and Full Cadence. PhraseA’ is an elaboration of PhraseA that may consist of a rhythmical modification, an ornamentation, a reduction, or a transposition (either within the current harmony, or to an other harmony). Figure 13 shows a few examples of multi-part melodies.

It can be seen in Figure 15 that the interface needed to specify the parameters of the separate Parts becomes quite complicated even though some parameters can only be set globally (e.g. key, meter, rhythmical density, syncopation). So we have to think of a more flexible interface.


Figure 13. Four multi-part melodies. a) A A’ FC (Final Cadence) in which A’ is an

elaboration of A. b) A A’ FC in which A’ is a transposition of A without harmonic change. c) A A’ FC in which A’ is a transposition of A. d) A B A FC. e and f) A A’ (transposition) A B HC A FC

Special algorithms Next, we describe two important routines used in the construction of melodies: the

generation of a tonal rhythm, and the ornamentation of a skeleton melody.

An algorithm for the generation of a tonal rhythm Above we argued that the metrical stability (the correlation between the accents in

a rhythm and the weights of a meter) of a rhythm must be sufficiently high to induce a meter. The weights of a meter are, of course, fixed (see Figure 1), while the accentuation of the notes in a rhythm are mainly determined by the note value of the notes and by their location in rhythmical groups: in groups of two tones the last tone is accented, and in groups of 3 or more tones both the first and last tones are accented.

Taking into account these specifications of accentuation, we can establish that metrical stability will increase if: 1. longer notes occur at strong metrical positions (positions with a high weight); 2. groups of two notes are positioned with the last note of the group on a strong

metrical position; 3. groups of more than two notes are positioned such that both the first and the last

note of the group coincide with relatively strong metrical positions.


With the help of these rules, we are now in a position to generate rhythms of differing stability for a given meter. To generate a rhythm with a high metrical stability, we use the following recipe (see Figure 14):

1. Put notes on the beats 2. Add notes before some beats to increase the accentuation of the note on those

beats. The higher the metrical weight of a beat the more often that beat will be preceded by a few notes, i. e. the stronger beats will more often be accentuated (see Figure 14 for the default values).

Figure 14. Construction of a rhythm with high metrical stability. The parameters related to rhythm construction can be set in the Master Control

Window after pressing the more triangle in the Time Dimension panel by means of the configuration shown in Figure 15..


Figure 15. Parameters used in the construction of rhythms, showing the default

values. The three upper parameters are for setting the proportion in which notes on beats with metrical weight 0, -1, or -2 are preceded by one or more notes. The number of notes added are set by the parameters in the left bottom corner, while the location of one added tone (either in the middle of the interval or at 3/4 , forming a punctuated rhythm) is set with the UpDownArrows middle right.

Alternatively, the metrical stability of a rhythm can be decreased by adding syncopations, as follows: starting at a random position within a measure (but not the first), the first following slot is found with a weight equal to the lowest weight (-4 for 4/4; -3 for the other meters) + 1, remove that note and add a note to the previous slot. This results in the addition of a note after a beat.

An algorithm for ornamenting a skeleton melody This algorithm, which is used to ornament a skeleton melody, takes account of the

following variables (Sk1 and Sk2 denote the pitches of the Skeleton tones between which one or more tones must be interpolated):

1. The pitch interval between Sk1 and Sk2 2. The number of tones to be interpolated between Sk1 and Sk2 3. Whether the interpolated tone is a nonchord tone, in which case it must

be anchored by the next tone 4. Whether or not repeated pitches are allowed (in the present algorithm

they are not) 5. Whether or not chromatic tones are allowed

Next, for each tone to be interpolated, it is determined whether it should be a chordtone, a nonchord tone, or a chromatic (nonchord) tone. This decision is based on the metrical weight of the slot as follows: if the weight =< -1 then the interpolated tone will always be a chord tone, if the weight = -2 then the tone will be a chordtone in 50% of the cases, if the weight = -3 then it will be a chordtone in 20% of the cases and if the weight = -4, then the tone will never be a chordtone. The decision for the use of chromatic tones is at present not solved in a principled way: the only used chromatic tones are the sharpened 4th and 5th degree, respectively resolving to degree 5 and 6.


Usually, the tone to be interpolated will be at 1/n of the interval between Sk1 and Sk2 (in either chromatic, diatonic, or triadic steps, depending on the choice for either type of tone), where n is the number of tones to be interpolated. A few situations that call for special measures:

1. If the interval between Sk1 and Sk2 (in either chromatic, diatonic, or triadic steps) is zero, special rules are applied so that the interpolated tones hover around Sk2 (see Figure 16).

2. If a large number of tones must be interpolated between Sk1 and Sk2, while the pitch interval between Sk1 and Sk2 is relatively small, the pitches of the first few interpolated tones do not necessarily have to be in between Sk1 and Sk2. At present the rule is that if there are more than two tones to be interpolated, in 50% of the cases the interpolated tone will not be between Sk1 and Sk2. [this may have to be refined]

A flow chart of the main aspects of the algorithm is shown in Figure 16.

Figure 16. Algorithm to interpolate tones between two skeleton tones Sk1, Sk2,

used to ornament a skeleton melody. Sk1: Pitch at start of interpolation interval, Sk2: last pitch of interpolation interval, NrOfSteps: nr of pitches between Sk1 and Sk2 (either triadic, diatonic, or chromatic steps), NrOfInterpols: Nr of tones still to be interpolated (3 in the example), Incr: Nr of chromatic, diatonic, or triadic steps to add to Sk1 to get the pitch of the interpolated tone.


Discussion

In this article an alternative method to test music theory has been proposed. Starting from a few basic assumptions concerning the nature of tonal music, a set of rules for the construction of tonal music is formulated, which is subsequently converted into a melody generating algorithm. The algorithm, Melody Generator, comprises procedures to effectuate the constraints imposed by the contexts of tonal music, i.e. by meter on the construction of rhythm, and by key and harmony on the selection of tones. In addition it provides a set of tools helpful in the construction of melodies such as the possibility to generate rhythms differing in density, to make various harmonic progressions and pitch contours, to add non-chord tones and syncopations. Three different modes of melody generation have been implemented increasing in sophistication and complexity. Melodies in Mode 1 are mainly determined by the constraints imposed by meter and key. The sequences in Mode 2 and Mode 3 are similarly constrained by tonal context but introduce additional rules. Sequences in Mode 2 are based on the notion that tonal melodies consist of structural and ornamental tones. Therefore, in this mode, skeleton melodies can be ornamented in various ways by means of a flexible elaboration algorithm. Mode 3 allows the construction of multi-part melodies motivated by the notion that tonal melodies are hierarchically organized into phrases that are, in part, structurally related.

The validity of the algorithm (and the underlying theoretical notions) must be established by evaluating the ‘melodies’ it generates. In the present stage of development a systematic empirical testing of the algorithm is clearly premature, but an impression of its validity can be gained by listening to the tone sequences the algorithm generates with various parameter settings4. Such informal evaluations indicate that with the proper parameter settings the melodies exhibit many of the typical characteristics of tonal melodies, in particular those that are imposed by the tonal contexts, meter and key: The rhythms are metrically correct, the melodies fit within the key and the harmonic progression, and non-chord tones are properly resolved. Simple melodies that are relatively short and based on a common harmonic progression sound reasonably well. The elaboration algorithm appears a powerful tool that yield a wide variety of musically relevant ornamentations. Also the possibility to generate multi-part melodies adds an important aspect to the melody generation device.

Notwithstanding these positive characteristics, the melodies produced by Melody Generator often seem to lack sufficient coherence and definition.

This shortcoming may well be related to the following features of melody construction in Melody Generator: 1) the three main dimensions, rhythm, harmony and melody are specified sequentially and independently rather then in a parallel and consistent fashion; 2) how and when special features like syncopations, non-chord tones, actually appear in a tone sequence is determined by randomly assigning the

4 The program can be downloaded freely at http://www.nici.ru.nl/~povel/


feature in some proportion preset by the user. Likewise, the user determines the size of the gaps between phrases, the type of variation used in generating a multi-part melody, whether or not chromatic tones will be used, etc.; 3) the implications following from the musical meaning that tones and chords acquire in their specific context, is not sufficiently taken into account. (a preliminary module implementing the expectations for tones and chords as delineated by Lerdahl, (2001) has been added to Melody Generator.)

These three features can be summarized as the missing of a Construction Plan for the generation of a melody. A Construction Plan is a prerequisite for a comprehensive method of melody construction. What is the nature of such a Plan?

In line with the distinction made in the section Theoretical Foundation, two aspects should be discerned in the Plan namely the context within which a tone sequence is interpreted, and the construction rules used to combine tones into larger fragments. This distinction refers to two essentially different aspects of a tonal melody: 1) The interpretation of the tones and underlying chords of a melody, within the currently induced key-space, which makes available the musical meaning of the tones: their stability and their attraction to other tones, experienced as expectations for those tones. 2) On the other hand, one can describe a tonal melody purely in terms of the structural relations between the pitch heights of its constituents. The latter gives rise to descriptions in which the structural relations between tones and groups of tones are specified (e.g. Deutsch & Feroe, 1981; Lerdahl & Jackendoff, 1983). The ‘recipes’ supplied by Schoenberg (1967) also fall in this category.

In line with this distinction the Plan should cover both aspects: one based on the musical meaning of the tones and chords in the constantly changing pitch-space and guided by the expectations for future tones and chords, and the other based on the construction rules that organizes a melody into a hierarchically organized configuration.

It seems likely that a Plan is often not completely specified at the beginning but that it evolves during the compositional process: the composer starts with some kernel idea(s): a small fragment, a motive or phrase, and subsequently expands this kernel employing the compositorial tools common within the style (repetition, variation, ornamentation, etc.) in an incremental fashion, i.e., deciding about the next steps in the compositional process, by considering what has been produced already including the created implications, and taking into account the constraints applying to the remaining part of the melody. This is all quite abstract and has to be concretized considerably before it can be implemented in algorithmic form.

Consequently, the challenge for future development will be to develop this concept of a Plan into a workable algorithm. For this purpose the algorithm should have some memory, as explained above, define the dimensions of rhythm, harmony, and melody in a parallel fashion, and take into account the musical meaning of the tones in the melody. This could perhaps be realized in a model in which a melody is conceived as a path in a multi-dimensional space, in which each of the dimensions imposes constraints on the development of the path. These constraints could be implemented as attractors that determine the path traversed in this space.


References Ames, C. (1989). The Markov process as a compositional model: A survey and

tutorial. Leonardo, 22, 175-187. Bamberger, J. (2000). Developing musical intuitions. New York: Oxford University

Press. Baroni. M., & Jacoboni, C. (1978). Proposal for a grammar of melody. The Bach

chorales. Montréal: Les Presses de l’Université de Montréal. Bharucha, J. J. (1984). Anchoring effects in music: The resolution of dissonance.

Cognitive Psychology, 16, 485-518. Biles, J.A. (1994). Genjam: A genetic algorithm for generating jazz solos. In:

Proceedings of the International Computer Music Conference. See also: http://www.it.rit.edu/~jab/

Boltz, M.G. (l989). Perceiving the end: Effects of tonal relationships on melodic completion. Journal of Experimental Psychology: Human Perception and Performance, 15, 749-761.

Brown, M.G. (1989). A rational reconstruction of Schenkerian Theory. Thesis Cornell University.

Brown, M.G. (2005). Explaining Tonality. Rochester University Press. Cambouropoulos, E. (1994). Markov chains as an aid to computer assisted

composition. Musical Praxis, 1, 41-52. Cope, D. (1996). Experiments in music intelligence. Cambridge, MA: MIT Press. Cope, D. (2005). Computer models of musical creativity. Madison, WI: A-R Editions. Deutsch, D. (Ed.) The Psychology of Music. 2nd Edition. San Diego: Academic Press,

1999. Deutsch, D., & Feroe, J. (1981). The Internal Representation of Pitch Sequences.

Psychological Review, 88, 503-522. Ebcioglu, K. (1988). An expert system for harmonizing four part chorales. Computer

Music Journal, 12, 43-51. Gardner, H. (1984). The mind’s new science: A history of the cognitive revolution. xx:

Basic Books. Gouyon, F., Widmer, G., Serra, X., & Flexer, A. (2006). Acoustic cues to beat

induction: A machine learning approach. Music Perception, 24, 177-188. Hirata, K., & Aoyagi, T. (2003). Computational music representation based on the

generative theory of tonal music and the deductive object-oriented database. Computer Music Journal, 27, 73-89.

Huron, D. (1996). The melodic arch in Western folksongs. Computing in Musicology, 10, 3-23.

Jonas, O. (1934). Das Wesen des musikalischen Kunstwerks : eine Einführung in die Lehre Heinrich Schenkers. Wien: Saturn

Krumhansl, C.L. (1990). Cognitive foundations of musical pitch. New York: Oxford University Press.


Krumhansl, C.L. (2000). Rhythm and pitch in music cognition. Psychological Bulletin, 126, 159-179.

Laitz, S. (2003). The complete musician: An integrated approach to tonal theory, analysis, and listening. Oxford University Press.

Lerdahl, F. (1988). Tonal pitch space. Music Perception, 5, 315-350. Lerdahl, F. (2001). Tonal Pitch Space. Oxford: Oxford University Press Lerdahl, F., & Jackendoff, R. (1983). A generative theory of tonal music. Cambridge,

MA: MIT Press. Longuet-Higgins, H. C., & Lee, C. S. (1982). The perception of musical rhythms.

Perception, 11, 115-128. Longuet-Higgins, H. C., & Steedman, M. J. (1971). On interpreting Bach. In B.

Meltzer, & D. Michie (Eds.), Machine Intelligence. Edinburgh: University Press. Margulis, E. H. (2005). A model of melodic expectation. Music Perception, 22, 663-

714. Marsden, A. (2005). Generative structural representation of tonal music. Journal of

New Music Research, 34, 409-428. Melody Generator II version 1.9.2. http://www.nici.ru.nl/~povel/ Miranda, E. (2001). Composing music with computers. Oxford: Focal Press Neuburg, M. (1999). REALbasic. The Definitive Guide. Sebastopol, CA: O’Reilly.

See also: http://www.realbasic.com/ Papadopoulos, G. & Wiggins, G. (1999). AI methods for algorithmic composition: A

survey, a critical view and future prospects. Proceedings of the AISB'99 Symposium on Musical Creativity.

Piston, W., & Devoto, M. (1989). Harmony. London: Victor Gollancz; (originally published 1941).

Povel, D. J. (1981). Internal representation of simple temporal patterns. Journal of Experimental Psychology Human Perception and Performance, 7, 3-18.

Povel, D. J. (2002). A model for the perception of tonal music. In: C. Anagnostopoulou, M. Ferrand, and A. Smaill (Eds.). Music and Artificial Intelligence. Heidelberg: Springer Verlag. (p. 144-154).

Povel, D. J., & Essens, P. (1985). Perception of temporal patterns. Music Perception, 2, 411-440.

Povel, D. J., & Jansen, E. (2001). Perceptual mechanisms in music processing. Music Perception, 19, 169-198.

Povel, D. J., & Jansen, E. (2002). Harmonic factors in the perception of tonal melodies. Music Perception, 20, 51 - 85.

Povel, D. J. & Okkerman, H. (1981). Accents in equitone sequences. Perception and Psychophysics, 30, 565-572.

Rameau, J.P. (1971, originally published 1722). Treatise on Harmony. New York, Dover Publications, Inc.

Riemann, H. (1887). Handbuch der Harmonielehre. Leipzig: Rowe, R. (2004). Machine musicianship. Cambridge, MA: MIT Press.


Salzer, F. (1982, originally published 1952). Structural hearing. Tonal coherence in music. New York: Dover Publications Inc.

Schenker, H. (1980, originally published 1954). Harmony. Chicago: Chicago University Press.

Schoenberg, A. (1967). Fundamentals of musical composition. London: Faber and Faber.

Schoenberg, A. (1978, originally published, 1911). Theory of Harmony. London: Faber and Faber.

Steedman, M. (1984). A generative grammar for jazz chord sequences. Music Perception, 2, 52-77.

Sundberg, J., & Lindblom, B. (1976). Generative theories in language and music descriptions. Cognition, 4, 99-122.

Sutcliffe, T. (2007). Syntactic structures in music. http://www.harmony.org.uk/ Temperley D. (1997). An algorithm for harmonic analysis. Music Perception, 15, 31-

68. Temperley, D. (2007). Music and probability. Cambridge, MA: MIT Press Toch, E. (1977, originally published 1948). The shaping forces in music: An inquiry

into the nature of harmony, melody, counterpoint, form. New York: Dover Publications.

Todd, P. (1989). A connectionist approach to algorithmic composition. Computer Music Journal, 13, 27-43.

Toiviainen, P. (1995). Modeling the target-note technique of bebop-style jazz improvisation: an artificial neural network approach. Music Perception, 12, 399-413.

Van Dyke Bingham, W. (1910). Studies in melody. Psychological Review, Monograph Supplements. Vol. XII, Whole No. 50.

Wiggins, G., Papadopoulos, G., Phon-Amnuaisuk, S, & Tuson, A. (1999). Evolutionary methods for musical composition. International journal of computing anticipatory systems.

Zuckerkandl, V. (1973/1956). Sound and Symbol. Princeton NJ: Princeton University Press.


Author Notes My former Ph.D. students Peter Essens, René van Egmond, and Erik Jansen, and

my students Jacqui Scharroo, Reinoud Roding, Jan Willem de Graaf, Albert van Drongelen, Thomas Koelewijn, and Daphne Albeda, all played an indispensable role in the development of the ideas underlying the project described in this paper.

I am very grateful to David Temperley for his support over the years, Hubert Voogd for his help in tracing and crushing numerous tenacious bugs, and Herman Kolk for his suggestions to improve the text.