-
HAL Id: hal-01262353https://hal.inria.fr/hal-01262353
Submitted on 26 Jan 2016
HAL is a multi-disciplinary open accessarchive for the deposit
and dissemination of sci-entific research documents, whether they
are pub-lished or not. The documents may come fromteaching and
research institutions in France orabroad, or from public or private
research centers.
L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt
et à la diffusion de documentsscientifiques de niveau recherche,
publiés ou non,émanant des établissements d’enseignement et
derecherche français ou étrangers, des laboratoirespublics ou
privés.
Motivic Pattern Extraction in Music, and Application tothe Study
of Tunisian Modal Music
Olivier Lartillo, Mondher Ayari
To cite this version:Olivier Lartillo, Mondher Ayari. Motivic
Pattern Extraction in Music, and Application to the Studyof
Tunisian Modal Music. Revue Africaine de la Recherche en
Informatique et Mathématiques Ap-pliquées, INRIA, 2007, 6,
pp.16-28. �hal-01262353�
https://hal.inria.fr/hal-01262353https://hal.archives-ouvertes.fr
-
16 Reviewed Article — ARIMA/SACJ, No. 36., 2006
Motivic Pattern Extraction in Music, And Application to the
Study of Tunisian Modal Music
O Lartillot∗, M Ayari†
∗Department of Music, PL 35(A), 40014 University of Jyväskylä,
Finland†Ircam, Place Igor-Stravinsky, 75004 Paris, France
ABSTRACT
A new methodology for automated extraction of repeated patterns
in time-series data is presented, aimed in particular
at the analysis of musical sequences. The basic principles
consists in a search for closed patterns in a multi-dimensional
parametric space. It is shown that this basic mechanism needs to
be articulated with a periodic pattern discovery system,
implying therefore a strict chronological scanning of the
time-series data. Thanks to this modelling global pattern
filtering
may be avoided and rich and highly pertinent results can be
obtained. The modelling has been integrated in a collaborative
project between ethnomusicology, cognitive sciences and computer
science, aimed at the study of Tunisian Modal Music.
KEYWORDS: pattern extraction, time-series data, closed pattern,
periodic pattern, music analysis, tunisianmodal music
1 INTRODUCTION
This paper introduces a new methodology for repeatedpattern (or
motif) extraction in symbolic sequences,and is applied particularly
to the analysis of musicalscores. Among the different approaches
that can beconsidered for time-series data analysis, one domainof
research that has received much attention is theproblem of
extraction of motives, i.e. the discoveryof patterns appearing
frequently in time-series data[1, 2, 3, 4]. Indeed, motives may
characterize impor-tant aspects of the data, and help discovering
newassociation rules. In music too, repeated sequencesof notes are
easily perceived by listeners as importantstructures, forming the
words of the musical structure.
Lots of research have been carried out in this do-main and
numerous interesting solutions have beenproposed. One major problem
stems from the struc-tural redundancies logically resulting from
this task,which, if not carefully controlled, may provoke
com-binatorial explosion and infringe the quality of theresults.
Few researches have considered the patterndiscovery problem within
this general context. Theapproach presented in this paper follows
this ideaof closed pattern, which is defined here in a
multi-dimensional parametric space. Another combinatorialredundancy
problem, provoked by immediate succes-sion of same patterns, is
solved by introducing theconcept of cyclic pattern. The model has
been appliedto the automated motivic analysis of musical scores,and
in particular to the study of Arabic improvisationsplayed by
Tunisian masters.
Email: O Lartillot [email protected], M Ayari
[email protected]
Most music databases contain sound files of per-formance
recordings, which correspond to the way mu-sic is commonly
experienced. The underlying struc-ture of music, on the other hand,
is represented ina symbolic form – the score – that describes
musicalpieces regardless of the way they are performed. Thereexist
numerous digital formats of symbolic music rep-resentation (MIDI,
MusicXML, Humdrum, etc.). Thepattern discovery system described in
this paper isapplied uniquely to symbolic representation. A
directanalysis on the signal level would arouse
tremendousdifficulties. A pattern extraction task on the sym-bolic
level, although theoretically simpler, remains ex-tremely difficult
to carry out, and its automation hasnot been achieved up to now.
Indeed, computer re-searches on this subject hardly offer results
close tolisteners’ or musicologists’ expectations. Hence thepattern
discovery task is too complex to be under-taken directly at the
audio signal, and needs rathera prior transcription from the audio
to the symbolicrepresentations, in order to carry out the analysis
ona conceptual level.
2 AN INCREMENTAL MULTIDIMENSIONALMOTIVIC IDENTIFICATION
2.1 Definitions
Music is expressed along multiple parametric dimen-sions. This
paper will focus on two main dimensions(Figure 1):
• Melodic dimension (melo) defined by pitch dif-ferences between
successive notes. (In scores,pitches are represented by the
vertical positionof the notes.)
-
Joint Special Issue — Advances in end-user data-mining
techniques 17
• Rhythmic dimension (rhyt) defined by durationsbetween
successive notes, and expressed with re-spect to metrical unit. For
instance, in a 6/8 met-ric (whose metrical unit is the 8th note) a
dotted8th note correspond to the value 1.5.
Figure 1 shows a musical example in traditional mu-sical
notation. Below the score is indicated the cor-responding numerical
description of the musical ex-ample along the melodic and rhythmic
dimensions.For instance, the first value of the melodic
dimension(melo = +1) indicates that the second note of thescore is
located one step higher than the first note ofthe score on the
vertical dimension. In musical term,this means that the second
pitch is one step higherthan the first pitch. The first value of
the rhythmicdimension (rhyt = 1.5) indicates that the duration
ofthe first note (or, equivalently, the temporal distancebetween
the first and second note) is equal to 1.5 timesthe basic unit.
Figure 1: Multi-dimensional description of a musical se-
quence.
A repeated succession of descriptions forms a pat-tern, and the
occurrences of the pattern correspond toits repetitions in the
score. Figure 1 shows two rep-etitions of a pattern, which are
squared in the score.Each pattern occurrence is represented as a
chain ofstates, each successive state (a, b, c, d, e in figure 1)
cor-responding to each successive note of the occurrence.The
pattern itself – i.e., the abstract entity that uni-fies all these
occurrences – is represented at the lowerpart of the figure by the
same chain of state. Thetransitions between successive states,
indicated belowthe chain of states, describe the successive
intervalsbetween successive notes of the pattern.
2.2 Identification of similarities
Patterns are generally not exactly repeated but trans-formed in
multiple ways. These patterns should there-fore be detected through
an identification of theirdifferent occurrences beyond their
apparent diversi-ties. Current approaches follow two different
strate-gies. One is based on numerical similarity, and toler-ates a
certain amount of dissimilarity between com-pared parameters [5,
6]. The main drawback of thisstrategy arises from the impossibility
of fixing pre-cisely similarity thresholds, on which identification
de-cision are based, and hence insuring relevant analyses.
Reference cognitive studies [7], on the other hand,assert that
similarity does not come from numericaldistance minimization, and
propose instead an alter-native strategy based on exact
identification alongmultiple musical dimensions of various
specificity lev-els. Several approaches to pattern discovery
followthis second strategy of identification along differentmusical
dimensions [8, 9] and search for repetitionsalong each different
dimension and product of dimen-sions.
Nonetheless there exist patterns that are progres-sively
constructed along variable successive musical di-mensions. These
heterogeneous patterns cannot beidentified by traditional
approaches. For instance,each line of the score in figure 2
contains a repeti-tion of a same pattern: in the first half, both
melodicand rhythmic dimensions are repeated whereas, in thesecond
half, only the rhythmic dimension is repeated.The model presented
in this paper is able to discoverheterogeneous patterns.
3 COMBINATORIAL REDUNDANCY FIL-TERING
This section presents the basic problem of pattern dis-covery
and introduces the notion of closed pattern.
3.1 Formalisation
Let S =< a1a2 . . . aN > be a sequence of elements ofsome
set ai ∈ A. A subsequence Si,l of index i ∈ [1, N ]and of length l
∈ [1, N + 1 − i] is a sequence of theform:
Si,l =< aiai+1 . . . ai+l−1 > . (1)
A sub-sequence Si,k is included in another sub-sequence Sj,l,
noted Si,k ⊂ Sj,l when j 6 i and i+k 6j + l. A pattern of length l,
denoted P ∈ P(S), isdefined as a repeated sub-sequence:
P ∈ P(S) ⇐⇒ ∃(i, j) ∈ [1, N ]2, P = Si,l = Sj,l. (2)
The support of a pattern P , denoted σ(P ), is thenumber of
occurrences of the pattern, i.e.
σ(P ) =∣
∣
{
i ∈ [1, N ], Si,l = P}∣
∣. (3)
3.2 Maximal patterns and closed patterns
The task of discovering repeated patterns leads tocombinatorial
problems. Indeed each pattern of length
l contains∑l
i=1 i =l(l−1)
2 = O(l2) sub-patterns.
that would be considered as distinct patterns by anybrute force
algorithm. The problem can be avoidedby restricting the search to
the patterns of highsupport and/or to patterns of pre-specified
length[4, 8, 9, 10, 11]. These constraints, in return,
sig-nificantly reduces the richness of the analysis.
One common way to solve this problem consists infocusing on the
maximal patterns P of the sequenceS, denoted P ∈ M(S), which are
patterns of S not
-
18 Reviewed Article — ARIMA/SACJ, No. 36., 2006
Figure 2: Repetition of a heterogeneous pattern.
included in any other pattern of S [12, 13]:
P ∈ M(S) ⇐⇒
{
P ∈ P(S)�Q ∈ P(S), P ⊂ Q. (4)For instance, pattern aij (which
can simply be
denoted by its last state j) in figure 3 is a simplesuffix of
pattern abcde (or e). It does not need to beexplicitly represented,
since the set of its occurrences(or pattern class) can be directly
deduced from theclass of its superpattern e.
This heuristic enables a significant reduction ofthe number of
discovered pattern, but leads also to aloss of information. Indeed,
not all the sub-patternsmay be immediately reconstructed knowing
the max-imal patterns. In figure 4, for instance, the patternclass
of j cannot be directly deduced from the patternclass of e, and
should therefore be explicitly repre-sented in the final analysis.
This corresponds to theconcept of closed pattern [13]. A pattern P
will becalled closed, denoted P ∈ C(S), if and only if thereexists
no proper super-pattern Q of same support :
P ∈ C(S) ⇐⇒
P ∈ P(S)�Q ∈ P(S), { P ⊂ Qσ(P ) = σ(Q).
(5)
The concept of closed pattern offers a compactand lossless
description of the musical piece: it al-lows an exhaustive
description of the pattern config-urations and avoids any global
selection based, forinstance, on minimum support threshold, as in
tra-ditional approach for mining association rules [1, 2].Even
pattern with only two occurrences are includedin the description.
Some constraints, however, havebeen added to the framework, that
reduce the solutionspace following cognitive heuristics related to
musicperception. For instance, a constraint takes into ac-count the
limitations of short-term memory: namely,for a pattern to be
detected, the temporal distance be-tween at least two of its
occurrences should not exceeda given threshold.
4 MULTI-DIMENSIONAL CLOSED PAT-TERNS
The model presented in this paper looks for closedpattern in
musical sequences. For this purpose, thenotion of inclusion
relation between patterns found-ing the definition of closed
patterns needs to be gen-eralized to the multi-dimensional
parametric space ofmusic, defined in section 2.1.
4.1 Formalisation of the problem
Musical patterns are represented with the help ofa conceptual
framework that defines objects associ-ated with different kinds of
attributes [14]. These at-tributes consist not only of the
different musical di-mensions, but also of the different
sub-patterns andsuper-patterns. The objects of the pattern
descrip-tions are the successive notes of the musical
sequenceforming the set N (S). Each note ni ∈ N (S) relatesto a
specific temporal context, defined by the part ofthe musical
sequence concluded by this note ni, that isto say, the subsequence
< n1 . . . ni >.
Each note ni is described firstly by the differ-ent musical
characteristics of the preceding interval:−−−−→ni−1ni.
D0,pdesc
(ni) : desc(−−−−→ni−1ni) = p,
desc ∈ {melo, rhyt}, p ∈ desc.(6)
Each note ni is also described by the musical charac-teristics
of the older intervals:
Dj,pdesc
(ni) : desc(−−−−−−−→ni−j−1ni−j) = p,
desc ∈ {melo, rhyt}, p ∈ desc.(7)
Then the pattern description of the sequence Smay be expressed
as a formal context (N (S),D, I) [14]where :
• the set of objects is N (S): the set of notes in S,
• the set of attributes is D: the set of elementarymusical
descriptions defined by equations 6 and7,
• and I is the binary relation between N (S) andD, called
incidence, defined by:
(ni, δ) ∈ I ⇐⇒ δ(ni) is true. (8)
-
Joint Special Issue — Advances in end-user data-mining
techniques 19
Figure 3: On the score are squared the occurrences of two
patterns abcde and aij. The occurrences of these patterns are
represented below the score. Pattern descriptions are
represented over the score, using a tree representation justified
in
section 6.1. Pattern aij, suffix of abcde with same support (2
occurrences), is a non-closed pattern that does not need to
be explicitly represented.
Figure 4: Same convention than in figure 3. Pattern aij, whose
support (4 occurrences) is now greater than the support
of abcde or abcdefgh (2 occurrences each) is a closed patterns
and needs to be explicitly represented.
The derived description C′ of a set of notes C ⊂N (S) is defined
as the common description of all thesenotes:
C′ ={
δ ∈ D | ∀n ∈ A, (n, δ) ∈ I}
. (9)
The notes in C are therefore occurrences of a samepattern, which
is maximally described by C′.
The derived class D′ of a complex description D ⊂D is dually
defined as the set of notes complying withthis description:
D′ ={
n ∈ N (S) | ∀δ ∈ D, (n, δ) ∈ I}
. (10)
The pattern discovery task consists in finding exhaus-tive class
D′ sharing a same description D. The trou-ble is, lots of different
descriptions Di may lead tosame classes D′i.
4.2 A representation of patterns as formalconcepts
The derivators operations defined by equation 9 and10 establish
a Gallois connection between the powerset lattices on N (S) and D
[14]. The Gallois connec-tion leads to a dual isomorphism between
two closuresystems, whose elements, called formal concepts of
theformal context (S(S),D, I) corresponds exactly to theclose
patterns P = (C, D), verifying:
C ⊂ N (S), D ⊂ D, C′ = D, and D′ = C. (11)
For a close pattern P = (C, D), C is called the extentof D and D
the intent of C. We may simply call Cand D respectively the class
and the description of P .
Hence, for a set of patterns Pi = (D′i, Di) of same
class D′i = C, the close pattern P = (C, D) is de-scribed using
the derived operator C′ defined in equa-tion 9: it contains all the
elementary descriptions com-mon to all notes of the class C. In
other words, closedpatterns are described as precisely as
possible.
Closed patterns, or formal concepts, are naturallyordered by the
subconcept-superconcept relation de-fined by
(C1, D1) < (C2, D2) ⇐⇒ C1 ⊂ C2 ( ⇐⇒ D2 ⊂ D1).(12)
This subconcept-superconcept relation can also becalled
specificity relation. In the equation above, forinstance (C1, D1)
is more specific than (C2, D2) andvice versa.
In particular, a pattern is more specific than itssuffixes,
since the description of the suffixes are strictlyincluded in the
description of the pattern.
4.3 Illustration
For instance, pattern abcde (in figure 5) featuresmelodic and
rhythmic descriptions, whereas patternafghi only features its
rhythmic part. Hence pat-tern abcde can be considered as more
specific than
-
20 Reviewed Article — ARIMA/SACJ, No. 36., 2006
pattern afghi, since its description contains more in-formation.
The less specific pattern afghi is a closedpattern, since its third
occurrence is not an occurrenceof the more specific pattern abcde,
and is therefore notfiltered out.
5 CYCLIC PATTERNS
In this section, we present another important factor
ofredundancy that, contrary to closed patterns, has notbeen studied
in current general algorithmic researches.
5.1 Periodic sequences
Periodicity occurs when a given pattern is successivelyrepeated,
such as the repetition of the pattern < 1, 2 >in figure 6.
Periodicity leads to combinatory explo-sion, since all possible
periods (i.e. all the possiblerotations of one period, such as abc
and pqr in theexample) can be considered as patterns, as well as
allpossible concatenations of periods and their differentprefixes,
such as patterns abcd, pqrs, abcde, etc. onthe left panel of the
figure. These redundant struc-tural artefacts should be replaced by
a compact repre-sentation that would explicitly describe the
structuralproperties of such configuration. For this purpose,we
propose to model periodic sequence through cyclicgraphs. A cyclic
pattern chain (CPC) is constructedfrom an originally acyclic
pattern chain (APC) repre-senting one period of the cycle, where a
transition isadded from the last state to the first state. In
thisway, the whole local periodicity can be represented bya single
pattern occurrence chain where each succes-sive state is uniquely
linked to the successive phasesof the CPC. An example of CPC is
given on the rightpanel of figure 6.
Each successive state of a pattern chain is re-lated to each
successive prefix of the pattern occur-rence. For this reason,
concerning the pattern occur-rence chain representing the entire
periodicity, the firststates that represent the first period should
not be as-sociated with the CPC since they are already associ-ated
with the APC of that period. On the contrary,the states of the
pattern occurrence chain followingthe first period will be
associated to the CPC, sincethey represent a configuration that is
actually specificto the periodicity. In this way, the CPC may be
con-sidered as a child of the APC, as can be seen in thefigure.
This additional concept immediately solves the re-dundancy
problem. Indeed, each type of redundantstructure considered
previously are non-closed suffixof prefix of the long pattern
chain, and will thereforenot be represented any more. For instance,
in figure6, pattern abcd is a suffix of pattern b′ with same
sup-port. Pattern abcd is therefore non-closed and canbe discarded.
Idem for patterns abcde, abcdef , etc.Pattern pq, suffix of abc
with same support, is alsonon-closed. Pattern pqr, suffix of b′
with same sup-port, is non-closed too. Idem for patterns pqrs,
pqrst,etc. Hence only one pattern occurrence remains, that
cover the whole periodic sequence, as displayed on theright
panel of the figure.
But this compact representation will be possibleonly if the
initial period (corresponding to the APC) isconsidered and extended
before the other possible pe-riods. That is to say, in figure 6,
the APC abc shouldbe considered before pqr. This shows therefore
thatthe sequence needs to be scanned in a chronological
way. This justifies therefore the incremental approachfollowed
by the algorithmic realisation of the model-ing, presented in
section 6.1.
5.2 Related works
Researches are dedicated to the automated discoveryof periodic
patterns in time-series data [15, 16, 17, 18].But as the search is
focused on periodic patterns only,no interaction is proposed with
acyclic pattern dis-covery. Hence, although offering interesting
descrip-tions of time series data, they cannot be used in or-der to
solve the combinatory problem presented in theprevious paragraph.
In our approach, on the otherhand, the periodic pattern problem is
deeply articu-lated with the acyclic pattern discovery process,
in-suring the compactness of the results.
A simpler solution to the combinatory problemconsists in
forbidding overlapping between patterns[3]. But this heuristics
presupposes that time-seriesdata are segmented into one-dimensional
series of suc-cessive segments. Time-series data do not all fulfil
thisrequirement: musical sequences, in particular, maysometimes be
composed of multi-levelled hierarchy ofstructures. Another solution
is to control the combi-natorial explosion by selecting, once the
analyses com-pleted, patterns featuring minimal temporal
overlap-ping between occurrences [8]. But as the selection
isinferred globally, relevant patterns may be discarded.Besides
combinatorial redundancy remains problem-atic since the filtering
is carried out after the actualanalysis phase. Our focus on local
configurations en-ables a more precise filtering.
5.3 The figure/ground rule
Another kind of redundancy appears when occur-rences of a
pattern – such as pattern acd =< 1A, 2A >in figure 7 – are
superposed to a cyclic pattern (b′),such that the pattern acd is
more specific than the cy-cle period (b′ simply representing the
successive rep-etition of As). In this case, the intervals that
followthese occurrences are identical, since they are relatedto the
same state (b′) of the cyclic pattern. Logicallythe pattern could
be extended following the successiveextensions of the cyclic
patterns (leading to patterne, and so on). This phenomenon, which
frequentlyappears, leads to another combinatorial proliferationof
redundant structures if not correctly controlled byrelevant
mechanisms. On the contrary, following theGestalt Figure/Ground
rule, the pattern acd can beconsidered as a specific figure that
emerges above theperiodic background. Following the Gestalt rule,
thefigure cannot be extended (into d) by a description
-
Joint Special Issue — Advances in end-user data-mining
techniques 21
Figure 5: The rhythmic pattern afghi is less specific than the
melodico-rhythmic pattern abcde.
Figure 6: Multiple successive repetitions of pattern abc =<
1, 2 > induce a complex intertwining of non-perceived
structures (represented in grey, left panel) that can be
filtered out using cyclic patterns (right panel).
that can be simply identified with the background ex-tension.
This rules shows the interest of integratingcognitive rules into
the model, as these rules concernas much the perceptive adequacy of
the results thanthe computational efficiency of the process.
6 IMPLEMENTATION DETAILS
6.1 Incremental pattern construction
This paragraph describes the basic mechanism of pat-tern
extraction. As explained in section 5.1, it consistsin an online
process that progressively reads the musi-cal sequence in a
chronological order and that extractsa list of patterns. This list
of patterns can be repre-sented as a prefix tree, since two motives
with sameprefix can be considered as two different continuationsof
this prefix.
The basic principle of the algorithm refers to asso-ciative
memory, i.e. the capacity of relating items thatfeature similar
properties. The associative memory ismodeled through inverted lists
related to the differentmusical parameters (i.e. melodic and
rhythmic dimen-sions). A first set of tables store the intervals of
thepiece with respect to their values along each differentmusical
dimension. For instance, two tables (Figure3, line a) store the
intervals of the score according totheir melodic and rhythmic
values. The melodic tableshows that the first interval of each bar
shares samemelodic value melo = +1, and, the rhythmic
tableindicates another identity rhyt = 1.5.
Intervals sharing a same value form occurrencesof an elementary
pattern that simply represents this
particular interval parameter. The elementary pat-tern is
represented as a child (here b) of the root ofthe pattern tree (a).
Each time a new pattern is cre-ated, new tables (at the right of
node b) store all thepossible intervals that immediately follow the
occur-rences of the new pattern (b). When any identity isdetected
in these new tables, a new pattern is createdas an extension of the
previous one (c, as an exten-sion of b), and is represented as a
child in the patterntree, and so on. This algorithm enables a
progressivediscovery of the successive extensions of each
pattern,either homogeneous or heterogeneous, as defined insection
2.2: the selection of musical dimensions defin-ing each successive
extension of a pattern may vary.For instance, in Figure 8, the last
extension of patternabcde is simply melodic since the rhythm of the
lastinterval in each occurrence is different. Besides, ad-ditional
constraints have been integrated in order toinsure a minimal
continuity along these variable suc-cessive musical dimensions.
6.2 Avoiding redundant description of patternoccurrences
We have prolonged the attempt to optimize patterndescriptions by
adding a principle of maximally spe-cific descriptions of pattern
occurrences: when a pat-tern occurrence is discovered (pattern e in
Figure 5),all the occurrences of less specific patterns (pattern
i)are not superposed on it, since they do not bring addi-tional
information, and can be directly deduced fromthe most specific
pattern occurrence (e) and from thespecificity relation (between e
and i).
-
22 Reviewed Article — ARIMA/SACJ, No. 36., 2006
Figure 7: Pattern c is a specific figure, above a background
generated by the cyclic pattern b′.
Figure 8: Progressive construction of pattern abcde.
The less specific description should be taken intoaccount
implicitly though, because their extensionsmay sometimes lead to
specific descriptions. For in-stance (Figure 9), groups 1 and 3 are
occurrences ofpattern h, and groups 3 and 4 are occurrences of
pat-tern d. Since pattern d is more specific, the less spe-cific
pattern h does not need to be associated withgroup 4. However in
order to detect groups 2 and 5 asoccurrences of pattern l, it is
necessary to implicitlyconsider group 4 as an occurrence of pattern
h. Hence,even if pattern h, since less specific than d, was
notexplicitly associated with group 4, it had to be consid-ered
implicitly in order to construct pattern l. Implicitinformation is
reconstituted through a traversal of thepattern network along
specificity relations.
6.3 General and specific cycles
The integration of the concept of cyclic pattern in
themultidimensional musical space requires a generalisa-tion of
specificity relations, defined in previous section,to cyclic
patterns. A cyclic pattern C is considered asmore specific than
another cyclic pattern D when thesequence of description of pattern
D is included in thesequence of description of pattern C. For
instance, fig-ure 10 displays four different cycles, the less
specificcycle d′′ � f describes the alternation of 1 and 2, themost
specific cycle b′′ � g′ describes the alternation of1A and 2B, and
the two other cycles b′ � c′ and d′ � e′
are in-between in the specificy graph. All these fourcycles
forms therefore an oriented graph called specificgraph (SG) whose
root is the less specific cycle d′′ � f .7 GENERAL RESULTS
This model was first developed as a library of Open-Music [19],
called OMkanthus. A new version will beincluded in the next version
2.0 of MIDItoolbox [20],a Matlab toolbox dedicated to music
analysis. Themodel can analyse monodic musical pieces (i.e.,
con-stituted by a series of non-superposed notes) and high-light
the discovered patterns on a score.
7.1 Experiments
The model has been tested with different musical se-quences
taken from several musical genres (classicalmusic, pop, jazz,
etc.). Table 1 shows some results.The experiment has been
undertaken with version0.6.8 of OMkanthus on a 1-GHz PowerMac G4.
Amusicologist expert has validated the analyses. Theproportion of
patterns considered as relevant is dis-played in the table.
The analysis of a medieval song called Geisslerlied– sometimes
used as a reference test for formalisedmotivic analysis – gave
quite relevant results. Theanalysis has been actually carried out
on a slight sim-plification of the actual piece presented in [21],
exclud-
-
Joint Special Issue — Advances in end-user data-mining
techniques 23
Figure 9: Group 4 can be simply considered as occurrence of
pattern d. However, in order to detect group 5 as occurrence
of pattern l, it is necessary to implicitly infer group 4 as
occurrence of pattern h too.
Figure 10: More detailed analysis of the perceived cyclic
configurations.
Table 1: Results of analyses, either melodic (M) or
melodico-rythmic (M+R), performed by OMkanthus 0.6.8.
Musical sequence Anal. Pattern classes Comp.
Name Notes type Disc. Relv. Succ. time
Geisslerlied 108 M 6 5 83% 2.2 sec.medieval song
Au clair de la lune 44 M+R 21 5 24% 5.6 sec.folk song
Bach, Invention in D minor 283 M 49 34 69% 37.6 sec.BWV 775
Mozart, Sonata in A K331 36 M+R 14 10 71% 0.8 sec.1st theme, 1st
half, melody
The Beatles 390 M 14 10 71% 28.1 sec.Obla Di Obla Da
ing local motivic variations out of reach of the
currentmodelling.
The melodico-rhythmic analysis of the Frenchsong Au clair de la
lune posed problems: 21 patternswere discovered from a 44-note long
sequence. This isdue to the fact that the successive steps of
progressivegeneralisation or specification of cycles are
currentlymodelled using distinct intermediary cyclic patterns.The
inference of these redundant cyclic patterns willbe avoided in
further works.
The algorithm has been successfully applied on amelodic analysis
of a complete two-voice Invention by
J.S. Bach. Figure 11 shows the analysis of the 21 firstbars. The
cyclic patterns are represented by gradu-ated lines, the graduation
representing each return ofone possible phase. Due to the nature of
the cyclicpatterns, no preference is given by the model
betweendifferent possible phases of the same cycle. The rhyth-mic
analysis of the piece, on the contrary, failed, due tothe
alternation of sequences of either quarter notes or8th notes, which
will require a formalisation throughhierarchical pattern chains
(where successive states ofhigher-level patterns are linked to
distinct lower-levelpatterns).
-
24 Reviewed Article — ARIMA/SACJ, No. 36., 2006
Figure 11: Automated motivic analysis of J.S. Bach’s Invention
in D minor BWV 775, 21 first bars. The occurrences of
each pattern class are designated in a distinct way.
The analysis of The Beatles’ Obla Di Obla Damelody shows 14
relevant pattern classes, representingthe chorus, verses, phrases
and motives inside each ofthese structures. The 4 irrelevant
patterns are redun-dant patterns subsumed by the 14 relevant
ones.
In all these pieces, some patterns are consideredas irrelevant
because they cannot be perceived as suchby listeners. Additional
mechanisms should be addedto prevent these irrelevant inferences,
based on short-term memory, top-down mechanisms, etc.
7.2 About algorithm complexity
The algorithm complexity may be expressed first interms of
discovered structures: proliferation of re-dundant patterns, for
instance, would lead to com-binatorial explosion, since each new
structure needsproper processes assessing its interrelationships
withother structures, and inferring possible extensions.Hence a
maximally compact description insures in thesame time the clarity
and relevance of the results andthe limitation of combinatorial
explosion. Concerningtechnical implementation, the prototype needs
furtheroptimisations.
The overall computational modelling results in acomplex system
formed by a large number of highlydependent mechanisms. Without a
real synthetic vi-sion of the whole system, no general assessment
of theglobal complexity of the modelling has been achievedyet. The
complete rebuilding of the modelling cur-rently undertaken should
enable a better awarenessand control of complexity.
8 COGNITIVE STUDY OF TUNISIANMODAL MUSIC
The project, presented in this paper, of modellingof musical
pattern discovery processes has been in-tegrated into a more
general collaborative project be-tween computer science,
ethnomusicology and cogni-tive sciences. The main objective of this
project is todesign a cognitive modelling, using a complex
system,of the processes of music perception and understand-ing, in
order to understand the perceptive, musicaland computational
aspects of sequence segmentation
and patterns recognition.This study has been focused on Tunisian
Modal
Music, and particularly on Tba’, a modal system thatpresents
interesting configurations. A Tba’ is basedon a musical scale –
i.e. a set of pitches, such as(C, D, E, F, G, A, B) for instance –
subdivided intotwo or several sub-scales called genres. Each genre
ischaracterised by pivotal notes (more important thanothers) and
melodic profile. Hence in a specific genre,the pitches that it
contains are played in a specific or-der. Genres themselves are
hierarchically connectedone with the others. In the musical scale
of the Tba’,some of the notes play particular role in the mode:some
are mostly played at the beginning of the impro-visation, or at the
end of phrases. Finally, to each Tba’is also associated a set of
characteristic melodic pat-terns. Figure 12 presents an example of
Tba’ modalstructure.
The study has been focused on one particular im-provisation by
the Nay flute player Mohamed Saada,along the Tba’ Istikhbar Mhayyer
Sika. First, the im-provisation has been transcribed from an audio
recordinto a musical score. Then the resulting symbolic se-quence
has been analyzed by the modeling.
8.1 Psychological experiments
Psychological experiments have been carried out in or-der to
obtain a detailed description of listening strate-gies, and to
assess the role of cultural schemes in par-ticular. This study has
been focused in particular onthe determination of the patterns that
form the basicstructures of the musical genres. In order to
under-stand the impact of cultural knowledge on this partic-ular
task, two groups of subjects have been considered:one group formed
by European subjects unfamiliar toArabic modal music, and another
group formed byArabic subjects of various degree of expertise in
thismusic. Subjects have been asked to performed sev-eral tasks
successively: First of all, after hearing themusical piece, they
have to recognise the most salientmusical structures and, using
these structures, to re-duce the whole improvisation in order to
exhibit thedynamic macro-structure of the piece.
The experiments have been first carried out inParis on European
subjects, and then extended to
-
Joint Special Issue — Advances in end-user data-mining
techniques 25
Figure 12: Description of the Tba’ Mhayyer Sika D(Ré), in terms
of a sets of Genres and pivotal notes.
Arabic subjects in Tunisia. Results shows the
relativevariability and divergence of the judgements of Euro-pean
subjects. This is due to the fact that they can-not follow their
own cultural scheme when analysingArabic modal improvisations. They
have to rely in-stead on the structural characteristics of the
musicaldiscourse, and in particular the discovery of
repeatedpatterns. The experimental results of the listeningtests
have been used as a guideline in order to im-prove the model
presented in the previous sectionsand to take into account the
stylistic characteristicsof Arabic modal improvisations.
8.2 Improvement of the modelling
One major limitation of the first version of the mod-elling, as
presented in previous sections, is that onlyrepetition of sequences
of notes that are immediatelysuccessive could be detected. In music
in general,and in modal improvisation in particular,
repeatedpatterns are often ornamented: secondary notes canbe added
whose purpose is to emphasise the primarynotes of the initial
pattern. Figure 13 displays, for in-stance, a melodic phrase, and
one possible ornamen-tation. To some of the notes of the original
phrase areadded secondary notes (displayed with smaller size inthe
score) that are located in the neighbourhood of theprimary notes,
both in time and pitch dimensions.
In order to take into account these ornamentation,a set of
mechanisms have been added to the modelling.Solutions have been
proposed [6] based on optimalalignments between approximate
repetitions using dy-namic programming and edit distances. We have
de-veloped algorithms that automatically discover, fromthe rough
surface level of musical sequences, musi-cal transformations
revealing the sequence of pivotalnotes forming the deep structure
of these sequences.These mechanisms induce new connections
between
non-successive notes, transforming the syntagmaticchain of the
original musical sequence into a complexsyntagmatic graph. The
direct application of the pat-tern discovery algorithm on this
syntagmatic graphenables the detection of ornamented
repetitions.
8.3 Results of the computational modelling
The analysis of Mohamed Saada’s improvisation of Is-tikhbar
Mhayyer Sika is displayed in figure 14. Thediscovered structures
are represented below each lineof the score. Each line represents
an occurrence of thepattern, designated by a sign (1, 2, 3, 4, 5, +
and -)on the left of the line. The notes actually consideredby each
pattern occurrence are represented by squaresvertically aligned to
the notes. These squares repre-sents therefore the successive
states along the patternoccurrence chain, as shown in Figure 1.
Pattern ’-’ represents a simple sequence of notesof continuously
decreasing pitch heights, and pattern’+’ represents a sequence of
notes of continuously in-creasing pitch heights. Patterns 1 to 5
are sequencesrepeated several times in the improvisation. Eachblack
square represents the beginning of a new oc-currence, and each
white square one successive statealong the pattern chain. Grey
squares corresponds tooptional states that are not found in all the
occur-rences of the pattern. Finally, multiple branches des-ignates
multiple possible paths for one same patternoccurrence. The
improvisation is built on the specificmode Tba’ Mhayyer Sika D,
characterised by the useof a specific set of notes (D, E, F, G, A,
Bb, C) and aspecific melodic figure, which corresponds exactly
tothe pattern 2. The beginning of the improvisation isalso based on
the successive repetition of pattern 1,which corresponds to a
periodic melodic curve start-ing from note F and ending to the same
note F, whichis therefore a pivotal note of the improvisation.
This
-
26 Reviewed Article — ARIMA/SACJ, No. 36., 2006
Figure 13: A melodic phrase and an ornamented version of it.
Figure 14: Analysis of the beginning of Istikhbar Mhayyer Sika
improvised at the Nay flute by Mohamed Saada.
pattern correspond to the mode Mazmoum F shownin figure 12. The
second line of the improvisation ischaracterised by the successive
repetition of pattern3, which is a little melodic line
progressively trans-posed. Pattern 4 corresponds to another
importantmelodic profile associated to pattern 2. Finally thetwo
last lines of the improvisation are characterised bythe repetition
of pattern 5. Patterns 2 and 4 may beconsidered as stylistic
characterisations of the modeIstikhbar Mhayyer Sika whereas
patterns 1, 3 and 5shows the characteristics of the individual
style of theimproviser.
The integration of these new mechanisms is notcompletely
achieved. The application of the pattern
discovery algorithm in the general syntagmatic graphleads to
combinatorial explosion of redundant patternsnot fully controlled
yet, which will need further works.
9 CURRENT RESEARCHES
9.1 Addition of segmentation principles.
The structures currently found are based solely on pat-tern
repetitions. Segmentation rules based on Gestaltprinciples of
proximity and similarity [22, 8] need tobe added. Although this
rule plays a significant role inthe perception of large-scale
musical structures, thereis no common agreement on its application
to detailed
-
Joint Special Issue — Advances in end-user data-mining
techniques 27
structure, because it highly depends on the subjectivechoice of
musical parameters used for the segmenta-tions. The study will
focus in particular on the com-petitive/collaborative
interrelations between the twomechanisms, in particular the masking
effect of localdisjunction on pattern discovery.
9.2 From monody to polyphony.
Our approach is limited to the detection of repeatedmonodic
patterns. Music in general is polyphonic,where simultaneous notes
form chords and parallelvoices. Researches have been carried out in
this do-main [9], focused on the discovery of exact
repetitionsalong different separate dimensions. Our model willbe
generalised to polyphony following the syntagmaticgraph principle.
We are developing algorithms thatconstruct, from polyphonies,
syntagmatic chains rep-resenting distinct monodic streams. These
chains maybe intertwined, forming complex graphs along whichthe
pattern discovery algorithm will be applied. Pat-tern of chords may
also be considered in future works.
9.3 Applications to musical databases.
The automated discovery of repeated patterns can beapplied to
automated indexing of musical content insymbolic music databases.
This approach may begeneralised later to audio databases, once
robust andgeneral tools for automated transcription of musicalsound
into symbolic scores will be available. A newkind of similarity
distance between musical pieces maybe defined, based on these
pattern descriptions, of-fering new ways of browsing inside a music
databaseusing pattern-based similarity distance.
9.4 Applications to non-musical domains.
In future works, we plan to apply the algorithm tocase studies
other than music scores, such as DNAsequences.
ACKNOWLEDGMENTS
This modelling was designed by Olivier Lartillotpartly in the
context of a collaborative project, withcognitive
ethno-musicologist Mondher Ayari, com-puter music scientist Gérard
Assayag and cognitivescientists Stephen McAdams and Petri
Toiviainen, fo-cused on the study of arabic improvised music
withthe help of cognitive modelling. The project has beenfinanced
by the French CNRS within the context ofthe ACI Complex Systems for
Human and Social Sci-ences.
REFERENCES
[1] R. Agrawal, T. Imielinski and A. Swami. “Min-ing association
rules between sets of items in largedatabases”. In Proceedings of
the International Con-ference on Management of Data (SIGMOD 93).
1993.
[2] H. Mannila, H. Toivonen and A. I. Verkamo. “Effi-cient
Algorithms for Discovering Association Rules”.In Proceedings of the
AAAI Workshop on KnowledgeDiscovery in Databases. 1994.
[3] Y. Tanaka, K. Iwamoto and K. Uehara. “Discoveryof
Time-Series Motif from Multi-Dimensional DataBased on MDL
Principle”. Machine Learning, vol. 58,no. 2–3, 2005.
[4] J. Lin, E. Keogh, S. Lonardi and P. Patel. “FindingMotifs in
Time Series”. In Proceedings of the Interna-tional Conference on
Knowledge Discovery and DataMining. 2002.
[5] D. Cope. Computer and Musical Style. Oxford Uni-versity
Press, 1991.
[6] P.-Y. Rolland. “Discovering Patterns in Musical Se-quences”.
Journal of New Music Research, vol. 28,pp. 334–350, 1999.
[7] W. Dowling and D. Harwood. Music Cognition. Aca-demic Press,
1986.
[8] E. Cambouropoulos. Towards a General Computa-tional Theory
of Musical Structure. Ph.D. thesis, Uni-versity of Edinburgh,
1998.
[9] D. Meredith, K. Lemström and G. Wiggins. “Al-gorithms for
discovering repeated patterns in mul-tidimensional representations
of polyphonic music”.Journal of New Music Research, vol. 31, no. 4,
pp.321–345, 2002.
[10] N. Pasquier, Y. Bastide, R. Taouil and L.
Lakhal.“Discovering frequent closed itemsets for associationrules”.
In Proceedings of the International Conferenceon Database Theory.
1999.
[11] J. Pei, J. Han and R. Mao. “Closet: An efficientalgorithm
for mining frequent closed itemsets”. InProceedings of the
ACM-SIGMOD Workshop on Re-search Issues in Data Mining and
Knowledge Discov-ery. 2000.
[12] R. Agrawal and R. Skirant. “Mining Sequential Pat-terns”.
In Proceedings of the International Conferenceon Data Engineering.
1995.
[13] M. Zaki. “Efficient Algorithms for Mining ClosedItemsets
and Their Lattice Structure”. IEEE Trans-actions on Knowledge and
Data Engineering, vol. 17,no. 4, pp. 462–478, 2005.
[14] B. Ganter and R. Wille. Formal Concept
Analysis:Mathematical Foundations. Springer-Verlag, 1999.
[15] J. Han, W. Gong and Y. Yin. “Mining Segment-Wise Periodic
Patterns in Time-Related Databases”.In Proceedings of the
International Conference onKnowledge Discovery and Data Mining.
1998.
[16] J. Han, G. Dong and Y. Yin. “Efficient Mining of Par-tial
Periodic Patterns in Time Series Database”. InProceedings of the
International Conference on DataEngineering. 1999.
[17] S. Ma and J. Hellerstein. “Mining partially periodicevent
patterns with unknown periods”. In Proceedingsof the International
Conference on Data Engineering.2001.
[18] W. W. J. Yang and P. Yu. “InfoMiner+: MiningPartial
Periodic Patterns with Gap Penalties”. InProceedings of the IEEE
International Computer onData Mining. 2002.
-
28 Reviewed Article — ARIMA/SACJ, No. 36., 2006
[19] G. Assayag, C. Rueda, M. Laurson, C. Agon andO. Delerue.
“Computer Assisted Composition at Ir-cam: From PatchWork to
OpenMusic”. ComputerMusic Journal, vol. 23, no. 3, pp. 59–72,
1999.
[20] T. Eerola and P. Toiviainen. “MIR In Matlab: TheMIDI
Toolbox”. In Proceedings of the InternationalConference on Music
Information Retrieval. 2004.
[21] N. Ruwet. “Methods of Analysis in Musicology”. Mu-sic
Analysis, vol. 6, no. 1–2, pp. 4–39, 1987.
[22] F. Lerdahl and R. Jackendoff. A Generative Theoryof Tonal
Music. The MIT. Press, 1983.