Page 1
Theoretical andTheoretical and Methodological Fundaments of Methodological Fundaments of
Music AnnotationMusic [email protected]
Institute for Psychoacoustics and Electronic Music (IPEM)Dept. Of Musicology
University Ghent, Belgium
Promotor: Prof. Dr. Marc Leman
Page 2
Presentation overviewPresentation overview
1. Problem specification1. Problem specification
2. Masters’ thesis research2. Masters’ thesis research
3. Research perspectives3. Research perspectives
Page 3
1. Problem specification1. Problem specification
• Strong need for annotated music databases, in order to train and develop intelligent MIR systems
Page 5
1. Problem specification1. Problem specification
• Strong need for annotated music databases, in order to train and develop intelligent MIR systems
No general methodological framework for music annotation
No general theoretical framework for music annotation
Page 6
2. Masters’ thesis research2. Masters’ thesis research
• Theoretical research
• Methodological research
Page 7
2. Masters’ thesis research2. Masters’ thesis research
• Theoretical research
Page 8
2. Masters’ thesis research2. Masters’ thesis research
• Theoretical research
Music annotation = detailed description, via specific methods and techniques, of musical content
Musical content = those parameters & concepts which make people evaluate certain informationstreams as musical entities
Page 9
?
(Lesaffre, et al., 2003)
Page 10
2. Masters’ thesis research2. Masters’ thesis research
• Theoretical research
from an ecological point of view: music is not reducable to low level concepts
constant interaction between different abstraction levels
Page 12
2. Masters’ thesis research2. Masters’ thesis research
• Methodological research
Page 13
2. Masters’ thesis research2. Masters’ thesis research
• Methodological research
Manual low level music annotation for testing and training the MAMI-Melodytranscriber
High level music annotation of polyphonic music, through melody imitation
Page 14
2. Masters’ thesis research2. Masters’ thesis research
• Methodological research
Manual low level music annotation for testing and training the MAMI-Melodytranscriber
Page 15
2. Masters’ thesis research2. Masters’ thesis research
• Methodological research
Manual low level music annotation for testing and training the MAMI-Melodytranscriber
Annotationtool for Qbv:
- Automatic segmentation
- Automatic pitchannotation
Page 16
2. Masters’ thesis research2. Masters’ thesis research
• Methodological research
Manual low level music annotation for testing and training the MAMI-Melodytranscriber
Annotation method: Praat
Annotated queries: vocal queries + instrumental queries
Page 17
2. Masters’ thesis research2. Masters’ thesis research
• Methodological research
Manual low level music annotation for testing and training the MAMI-Melodytranscriber
Auditive evaluation (Sonar)
Statistical evaluation
Page 18
equal insertion deletionwhistling 92.9% 3.1% 4.0%guitar 77.8% 20.2% 2.0%gamba 79.3% 6.3% 14.4%violin 71.0% 24.6% 4.3%viola 85.1% 7.0% 7.9%voice 94.4% 5.6%flute 73.1% 15.2% 11.7%altoflute 73.5% 17.6% 8.8%sax 60.9% 39.1%
Page 19
Ranking test
0100200300400
500600700800900
wh
istl
ing
gu
itar
gam
ba
vio
lin
vio
la
voic
e
flu
te
alto
flu
te
sax
instrument
ran
kin
g
ADFFREQ
Page 20
2. Masters’ thesis research2. Masters’ thesis research
• Methodological research
Manual low level music annotation for testing and training the MAMI-Melodytranscriber
Improve performance
Page 21
Total
15.39%
Results for instrumental queries
Page 22
2. Masters’ thesis research2. Masters’ thesis research
• Methodological research
High level annotation of polyphonic music, through melody imitation
Page 23
2. Masters’ thesis research2. Masters’ thesis research
• Methodological research
High level annotation of polyphonic music, through melody imitation
How to extract different melodylines from polyphonic music?
Page 24
2. Masters’ thesis research2. Masters’ thesis research
• Methodological research
High level annotation of polyphonic music, through melody imitation
8 trained singers were asked to imitate main melodylines and bass lines (and other melodylines they considered important) from 10 popular songs
Page 25
2. Masters’ thesis research2. Masters’ thesis research
• Methodological research
High level annotation of polyphonic music, through melody imitation
- Good imitations of main melodylines
- Good imitations of other relevant melodylines, but no consistency in choosing other
relevant melodies
- Bad imitations of bass lines
Page 26
2. Masters’ thesis research2. Masters’ thesis research
• Methodological research
High level annotation of polyphonic music, through melody imitation
melody imitations serving as reference material for training MIR systems?
Page 27
3. Research perspectives3. Research perspectives
• What is musical content?
• How to handle this musical content in a MIR context?
• How to annotate musical audio and what are possible underlying theories?
developing annotated music databases