Intelligent real-time music accompaniment for constraint-free improvisation
Post on 14-Jul-2015
140 Views
Preview:
Transcript
Intelligent real-time music accompaniment forconstraint-free improvisation
Maximos A. Kaliakatsos–Papakostas1, Andreas Floros2, and Michael N.Vrahatis1
1Computational Intelligence Laboratory (CI Lab), Department of Mathematics,University of Patras, GR-26110 Patras, Greece
2Department of Audio and Visual Arts, Ionian University, GR-49100 Corfu, Greece
8 November 2012
Intelligent music accompaniment 1/37
Improvisation
Improviser produces novel melodies provided a harmonic context(constraint).
�harmony (constraint) �improviser
Infinite possibilities!
�possible improvisation 1
�possible improvisation 2
...
Intelligent music accompaniment 3/37 Music accompaniment
Constraint-free improvisation
The improviser produces melodies creating hers/his melodic context fromscratch.
improviser harmony
Intelligent music accompaniment 4/37 Intelligent, real-time, constraint free improvization
Intelligent accompaniment for constraint-freeimprovisation
How can the computer understand the improviser’s implied harmoniccontext, and generate proper harmonic response?
this is what thepaper discusses
improviser harmony
Intelligent music accompaniment 5/37 Intelligent, real-time, constraint free improvization
Intelligent accompaniment for constraint-freeimprovisation
How can the computer understand the improviser’s implied harmoniccontext, and generate proper harmonic response?
The computer needs tolisten (understand)
improviser harmony
Intelligent music accompaniment 6/37 Intelligent, real-time, constraint free improvization
Intelligent accompaniment for constraint-freeimprovisation
How can the computer understand the improviser’s implied harmoniccontext, and generate proper harmonic response?
and generate (compose)
improviser harmony
Intelligent music accompaniment 7/37 Intelligent, real-time, constraint free improvization
Music characteristics
Characteristics which the computer should be able to understand andreproduce
humanimproviser
rhythmmodule modulemoduletone intensity
tonelistener
rhythm
listener
intensitylistener
tonegenerator
rhythm
generatorintensity
generator
instrumentspecialization computer
performance
Intelligent music accompaniment 8/37 The proposed framework
Music characteristics
Characteristics which the computer should be able to understand andreproduce
humanimproviser
rhythmmodule modulemoduletone intensity
tonelistener
rhythm
listener
intensitylistener
tonegenerator
rhythm
generatorintensity
generator
instrumentspecialization computer
performance
Intelligent music accompaniment 9/37 The proposed framework
Music characteristics
Characteristics which the computer should be able to understand andreproduce
humanimproviser
rhythmmodule modulemoduletone intensity
tonelistener
rhythm
listener
intensitylistener
tonegenerator
rhythm
generatorintensity
generator
instrumentspecialization computer
performance
Intelligent music accompaniment 10/37 The proposed framework
Music characteristics
Characteristics which the computer should be able to understand andreproduce
humanimproviser
rhythmmodule modulemoduletone intensity
tonelistener
rhythm
listener
intensitylistener
tonegenerator
rhythm
generatorintensity
generator
instrumentspecialization computer
performance
Intelligent music accompaniment 11/37 The proposed framework
Music characteristics
Characteristics which the computer should be able to understand andreproduce
humanimproviser
rhythmmodule modulemoduletone intensity
tonelistener
rhythm
listener
intensitylistener
tonegenerator
rhythm
generatorintensity
generator
instrumentspecialization computer
performance
Intelligent music accompaniment 12/37 The proposed framework
Music characteristics
Characteristics which the computer should be able to understand andreproduce
humanimproviser
rhythmmodule modulemoduletone intensity
tonelistener
rhythm
listener
intensitylistener
tonegenerator
rhythm
generatorintensity
generator
instrumentspecialization computer
performance
Intelligent music accompaniment 13/37 The proposed framework
Tone module
Listen to the chords and their chroma complexity.
Train the logistic map1 to create note sequences with similarcharacteristics (with Differential Evolution).
humanimproviser
chordrecognition
PCPestimation
chord tones
listPCP tones
list
SIEestimation
global SIE
value
tone modulelistener
chord tones
list
PCP tones
list
global SIE
valueselect proper
set of tones
finaltone list
instrument
tonal range
list ofnotes
find properr valuewith DE
logisticmap
r value
soundingnote
tone modulegenerator
(a) listener (b) generator
1xn+1 = r xn (1 − xn)
Intelligent music accompaniment 14/37 The proposed framework
Rhythm module
Listen to the rhythm and compute its rhythm features.
Train an FL–system to create rhythms with similar features (withGenetic Algorithm).
humanimproviser
rhythm module
instrumentspecialization
computer
performance
listener
get rhythmfeatures FL-systems
rhythm modulegenerator
Intelligent music accompaniment 15/37 The proposed framework
Intensity module
Listen to the intensities of notes within a sliding time window.
Assign intensities to upcoming notes, with similar statisticalcharacteristics.
Intelligent music accompaniment 16/37 The proposed framework
Experimental setup and inquiries
Human input
Improviser: human guitar player.
The computer listens to the human performance through MIDI.
Computer output
Intelligent musicians: bass and piano player.
Performance assessmentHow well does the computer adapt to the improviser’s playing style?
Compare qualitative characteristics of the music created by the humanand the computer musicians.
How swiftly is the computer adapted?
Intelligent music accompaniment 17/37 Results
Experimental setup and inquiries
Human input
Improviser: human guitar player.
The computer listens to the human performance through MIDI.
Computer output
Intelligent musicians: bass and piano player.
Performance assessmentHow well does the computer adapt to the improviser’s playing style?
Compare qualitative characteristics of the music created by the humanand the computer musicians.
How swiftly is the computer adapted?
Intelligent music accompaniment 18/37 Results
Experimental setup and inquiries
Human input
Improviser: human guitar player.
The computer listens to the human performance through MIDI.
Computer output
Intelligent musicians: bass and piano player.
Performance assessmentHow well does the computer adapt to the improviser’s playing style?
Compare qualitative characteristics of the music created by the humanand the computer musicians.
How swiftly is the computer adapted?
Intelligent music accompaniment 19/37 Results
Experimental setup and inquiries
Human input
Improviser: human guitar player.
The computer listens to the human performance through MIDI.
Computer output
Intelligent musicians: bass and piano player.
Performance assessmentHow well does the computer adapt to the improviser’s playing style?
Compare qualitative characteristics of the music created by the humanand the computer musicians.
How swiftly is the computer adapted?
Intelligent music accompaniment 20/37 Results
Experimental setup and inquiries
Human input
Improviser: human guitar player.
The computer listens to the human performance through MIDI.
Computer output
Intelligent musicians: bass and piano player.
Performance assessmentHow well does the computer adapt to the improviser’s playing style?
Compare qualitative characteristics of the music created by the humanand the computer musicians.
How swiftly is the computer adapted?
Intelligent music accompaniment 21/37 Results
Experimental setup and inquiries
Human input
Improviser: human guitar player.
The computer listens to the human performance through MIDI.
Computer output
Intelligent musicians: bass and piano player.
Performance assessmentHow well does the computer adapt to the improviser’s playing style?
Compare qualitative characteristics of the music created by the humanand the computer musicians.
How swiftly is the computer adapted?
Intelligent music accompaniment 22/37 Results
Performance assessment metrics
Compare the human and computer generated music by comparing musicalattributes at short fixed time intervals.
Measuring musical attributes
Shannon Information Entropy of the Pitch Class Profile (tonal feature):with the acronym SIE
density of notes within a fixed time window (rhythmic feature)
syncopation of rhythm within 4 music measures (rhythmic feature)
symmetry of rhythm within 4 music measures (rhythmic feature)
intensity of notes within 4 music measures (intensity feature)
Intelligent music accompaniment 24/37 Results
Performance assessment metrics
Compare the human and computer generated music by comparing musicalattributes at short fixed time intervals.
Measuring musical attributes
Shannon Information Entropy of the Pitch Class Profile (tonal feature):with the acronym SIE
density of notes within a fixed time window (rhythmic feature)
syncopation of rhythm within 4 music measures (rhythmic feature)
symmetry of rhythm within 4 music measures (rhythmic feature)
intensity of notes within 4 music measures (intensity feature)
Intelligent music accompaniment 25/37 Results
Performance assessment metrics
Compare the human and computer generated music by comparing musicalattributes at short fixed time intervals.
Measuring musical attributes
Shannon Information Entropy of the Pitch Class Profile (tonal feature):with the acronym SIE
density of notes within a fixed time window (rhythmic feature)
syncopation of rhythm within 4 music measures (rhythmic feature)
symmetry of rhythm within 4 music measures (rhythmic feature)
intensity of notes within 4 music measures (intensity feature)
Intelligent music accompaniment 26/37 Results
Performance assessment metrics
Compare the human and computer generated music by comparing musicalattributes at short fixed time intervals.
Measuring musical attributes
Shannon Information Entropy of the Pitch Class Profile (tonal feature):with the acronym SIE
density of notes within a fixed time window (rhythmic feature)
syncopation of rhythm within 4 music measures (rhythmic feature)
symmetry of rhythm within 4 music measures (rhythmic feature)
intensity of notes within 4 music measures (intensity feature)
Intelligent music accompaniment 27/37 Results
Performance assessment metrics
Compare the human and computer generated music by comparing musicalattributes at short fixed time intervals.
Measuring musical attributes
Shannon Information Entropy of the Pitch Class Profile (tonal feature):with the acronym SIE
density of notes within a fixed time window (rhythmic feature)
syncopation of rhythm within 4 music measures (rhythmic feature)
symmetry of rhythm within 4 music measures (rhythmic feature)
intensity of notes within 4 music measures (intensity feature)
Intelligent music accompaniment 28/37 Results
Performance assessment metrics
Compare the human and computer generated music by comparing musicalattributes at short fixed time intervals.
Measuring musical attributes
Shannon Information Entropy of the Pitch Class Profile (tonal feature):with the acronym SIE
density of notes within a fixed time window (rhythmic feature)
syncopation of rhythm within 4 music measures (rhythmic feature)
symmetry of rhythm within 4 music measures (rhythmic feature)
intensity of notes within 4 music measures (intensity feature)
Intelligent music accompaniment 29/37 Results
Performance assessment metrics
Compare the human and computer generated music by comparing musicalattributes at short fixed time intervals.
Measuring musical attributes
Shannon Information Entropy of the Pitch Class Profile (tonal feature):with the acronym SIE
density of notes within a fixed time window (rhythmic feature)
syncopation of rhythm within 4 music measures (rhythmic feature)
symmetry of rhythm within 4 music measures (rhythmic feature)
intensity of notes within 4 music measures (intensity feature)
Intelligent music accompaniment 30/37 Results
Performance assessment metrics
SIE
guitarpianobass
Rhythm density
guitarpianobass
Rhythm syncopation
guitarpianobass
Rhythm symmetry
guitarpianobass
Intensities
guitarpianobass
Intelligent music accompaniment 31/37 Results
Performance assessment in improvisation 1
improvisation 1no delay delay
piano bass piano bassSIE 0.4280 0.5571 0.4240 0.5735
SIE (MA) 0.6516 0.7700 0.6829 0.8320density 0.4659 0.5321 0.7045 0.8557
density (MA) 0.5416 0.6053 0.7771 0.9064syncopation 0.1789 0.4802 0.2230 0.3418
syncopation (MA) 0.4542 0.6417 0.6551 0.7188symmetry -0.2060 -0.3752 -0.0050 0.4130
symmetry (MA) 0.1425 0.0314 0.4222 0.2274intensity 0.6696 0.6731 — —
Intelligent music accompaniment 32/37 Results
Performance assessment in improvisation 2
improvisation 2no delay delay
piano bass piano bassSIE 0.1949 0.1236 0.2601 0.0551
SIE (MA) 0.6189 0.4954 0.7198 0.5529density 0.5229 0.6316 0.7907 0.7220
density (MA) 0.7090 0.7133 0.8941 0.8520syncopation 0.0326 0.2958 0.0977 0.2191
syncopation (MA) 0.5455 0.5357 0.7203 0.6710symmetry -0.1455 0.0474 0.2701 0.0616
symmetry (MA) 0.2125 0.3113 0.4305 0.4902intensity 0.5459 0.5745 — —
Intelligent music accompaniment 33/37 Results
Performance assessment in improvisation 3
improvisation 3no delay delay
piano bass piano bassSIE 0.3686 0.2823 0.4133 0.2826
SIE (MA) 0.6455 0.4098 0.7308 0.4166density 0.6011 0.5285 0.8002 0.7011
density (MA) 0.7139 0.5964 0.8951 0.7687syncopation 0.0152 0.0764 -0.0298 0.1130
syncopation (MA) 0.4374 0.2382 0.6900 0.3164symmetry -0.1904 0.0927 0.1161 0.0235
symmetry (MA) 0.1348 0.2161 0.4947 0.2656intensity 0.6242 0.6345 — —
Intelligent music accompaniment 34/37 Results
Overall tonal impression
Comparing the pitch class profile of the entire recordings by each “musician”.
improvisation 1 improvisation 2 improvisation 3guitar piano bass guitar piano bass guitar piano bass
guitar 1.0000 0.9208 0.9160 1.0000 0.9583 0.9806 1.0000 0.8418 0.8490piano 0.9208 1.0000 0.9722 0.9583 1.0000 0.9754 0.8418 1.0000 0.9213bass 0.9160 0.9722 1.0000 0.9806 0.9754 1.0000 0.8490 0.9213 1.0000
Intelligent music accompaniment 35/37 Results
Concluding discussion
Summary
The presented system:
provides intelligent automatic accompaniment to a human improviserwithout any prior musical considerations
adapts to the human improviser’s tonal, rhythmic and intensityperforming style and composes novel music
Future improvements
examine whether the training process could be based on the MovingAverage time series of features
the listener submodules could enhanced by the introduction of newfeatures that incorporate further descriptive knowledge
the generator submodules could be further enhanced by utilizing moresophisticated intelligent and adaptive techniques
Intelligent music accompaniment 36/37 Conclusions
Concluding discussion
Summary
The presented system:
provides intelligent automatic accompaniment to a human improviserwithout any prior musical considerations
adapts to the human improviser’s tonal, rhythmic and intensityperforming style and composes novel music
Future improvements
examine whether the training process could be based on the MovingAverage time series of features
the listener submodules could enhanced by the introduction of newfeatures that incorporate further descriptive knowledge
the generator submodules could be further enhanced by utilizing moresophisticated intelligent and adaptive techniques
Intelligent music accompaniment 37/37 Conclusions
Concluding discussion
Summary
The presented system:
provides intelligent automatic accompaniment to a human improviserwithout any prior musical considerations
adapts to the human improviser’s tonal, rhythmic and intensityperforming style and composes novel music
Future improvements
examine whether the training process could be based on the MovingAverage time series of features
the listener submodules could enhanced by the introduction of newfeatures that incorporate further descriptive knowledge
the generator submodules could be further enhanced by utilizing moresophisticated intelligent and adaptive techniques
Intelligent music accompaniment 38/37 Conclusions
Concluding discussion
Summary
The presented system:
provides intelligent automatic accompaniment to a human improviserwithout any prior musical considerations
adapts to the human improviser’s tonal, rhythmic and intensityperforming style and composes novel music
Future improvements
examine whether the training process could be based on the MovingAverage time series of features
the listener submodules could enhanced by the introduction of newfeatures that incorporate further descriptive knowledge
the generator submodules could be further enhanced by utilizing moresophisticated intelligent and adaptive techniques
Intelligent music accompaniment 39/37 Conclusions
Concluding discussion
Summary
The presented system:
provides intelligent automatic accompaniment to a human improviserwithout any prior musical considerations
adapts to the human improviser’s tonal, rhythmic and intensityperforming style and composes novel music
Future improvements
examine whether the training process could be based on the MovingAverage time series of features
the listener submodules could enhanced by the introduction of newfeatures that incorporate further descriptive knowledge
the generator submodules could be further enhanced by utilizing moresophisticated intelligent and adaptive techniques
Intelligent music accompaniment 40/37 Conclusions
Concluding discussion
Summary
The presented system:
provides intelligent automatic accompaniment to a human improviserwithout any prior musical considerations
adapts to the human improviser’s tonal, rhythmic and intensityperforming style and composes novel music
Future improvements
examine whether the training process could be based on the MovingAverage time series of features
the listener submodules could enhanced by the introduction of newfeatures that incorporate further descriptive knowledge
the generator submodules could be further enhanced by utilizing moresophisticated intelligent and adaptive techniques
Intelligent music accompaniment 41/37 Conclusions
Concluding discussion
Summary
The presented system:
provides intelligent automatic accompaniment to a human improviserwithout any prior musical considerations
adapts to the human improviser’s tonal, rhythmic and intensityperforming style and composes novel music
Future improvements
examine whether the training process could be based on the MovingAverage time series of features
the listener submodules could enhanced by the introduction of newfeatures that incorporate further descriptive knowledge
the generator submodules could be further enhanced by utilizing moresophisticated intelligent and adaptive techniques
Intelligent music accompaniment 42/37 Conclusions
top related