Top Banner
Signal Processing Algorithms for Music, Marine Mammals and Speech Yannis Stylianou Outline of the talk Rhythmic Similarity of Music Whales Click Detections Measuring jitter References Signal Processing Algorithms for Music, Marine Mammals and Speech Yannis Stylianou University of Crete, Computer Science Dept., Multimedia Informatics Lab [email protected] AUTH 2008 June 23rd
58

Signal Processing Algorithms for Music, Marine Mammals …

Oct 22, 2021

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Signal Processing Algorithms forMusic, Marine Mammals and Speech

Yannis Stylianou

University of Crete, Computer Science Dept., Multimedia Informatics [email protected]

AUTH2008 June 23rd

Page 2: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

1 Rhythmic Similarity of Music

2 Whales Click Detections

3 Measuring jitter

4 References

Page 3: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Rhythmic Similarity of Music Based onDynamic Periodicity Warping

In collaboration with: Andre Holzapfel([email protected])

It was presented at ICASSP 2008, Las Vegas

References: [1]

Page 4: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

What is used for?

Organize your huge collection of songs according to theirrhythm.

Help ethnomusicologists to categorize and reveal musicalstructure of field recordings from some country.

Page 5: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Approaches to the problem

Beat spectra, cosine measure (J. Foote et al., 2002) [2]

Tempo based spectra (G. Peeters, 2005) [3]

Tactus based Rhythmic patterns (J. Paulus et al., 2002)[4]

We suggest the use of continuous periodicity spectra and awarping strategy to cope with large variations in tempo.

Page 6: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Approaches to the problem

Beat spectra, cosine measure (J. Foote et al., 2002) [2]

Tempo based spectra (G. Peeters, 2005) [3]

Tactus based Rhythmic patterns (J. Paulus et al., 2002)[4]

We suggest the use of continuous periodicity spectra and awarping strategy to cope with large variations in tempo.

Page 7: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Periodicity Spectra

Computation of onset strength signal, p(t) (D. Ellis,MIREX2006, beat tracking contest1)

Modeling of p(t)

p(t) =N∑

i=1

ei (t) ∗∑k∈Ki

δ(t − kT )

Periodicity Spectra:

P(f ) =

∣∣∣∣∣∣N∑

i=1

1

TEi (f )

∑k∈Ki

δ(f − k

T)

∣∣∣∣∣∣where f < 1000bpm (16.7Hz)

1www.music-ir.org/mirex2006/index.php/Audio Beat Tracking Results

Page 8: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Periodicity Spectra

Computation of onset strength signal, p(t) (D. Ellis,MIREX2006, beat tracking contest1)

Modeling of p(t)

p(t) =N∑

i=1

ei (t) ∗∑k∈Ki

δ(t − kT )

Periodicity Spectra:

P(f ) =

∣∣∣∣∣∣N∑

i=1

1

TEi (f )

∑k∈Ki

δ(f − k

T)

∣∣∣∣∣∣where f < 1000bpm (16.7Hz)

1www.music-ir.org/mirex2006/index.php/Audio Beat Tracking Results

Page 9: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Periodicity Spectra

Computation of onset strength signal, p(t) (D. Ellis,MIREX2006, beat tracking contest1)

Modeling of p(t)

p(t) =N∑

i=1

ei (t) ∗∑k∈Ki

δ(t − kT )

Periodicity Spectra:

P(f ) =

∣∣∣∣∣∣N∑

i=1

1

TEi (f )

∑k∈Ki

δ(f − k

T)

∣∣∣∣∣∣where f < 1000bpm (16.7Hz)

1www.music-ir.org/mirex2006/index.php/Audio Beat Tracking Results

Page 10: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Example of periodicity spectra

345 6900

0.01

0.02

0.03

bpm

345 6900

0.01

0.02

0.03

0.04

bpm

Two examples of periodicity spectra of Siganos dance: Upperpanel is a faster example of that in the lower panel. Windowlength is 8s.

Page 11: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Rhythm similarity based on DynamicPeriodicity Warping (DPW)

DPNORM PROJ

REFLINESIM

wDPW

ρ

Σ

S

dDPW

P1(f)

P2(f)

Page 12: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Example of DPW computation

Page 13: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Databases and baseline Distances

Databases:

D1: 698 songs from eight classes of ballroom dancesD2: 90 songs from six classes of Cretan dances

Baseline Distances

Cosine distance (inner product)Euclidean distanceCost of warping, (dCost) (J. Paulus et al., 2002)[4]Cosine distance after warping, dCosPost

Our measure: dDPW

Page 14: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Databases and baseline Distances

Databases:

D1: 698 songs from eight classes of ballroom dancesD2: 90 songs from six classes of Cretan dances

Baseline Distances

Cosine distance (inner product)Euclidean distanceCost of warping, (dCost) (J. Paulus et al., 2002)[4]Cosine distance after warping, dCosPost

Our measure: dDPW

Page 15: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Databases and baseline Distances

Databases:

D1: 698 songs from eight classes of ballroom dancesD2: 90 songs from six classes of Cretan dances

Baseline Distances

Cosine distance (inner product)Euclidean distanceCost of warping, (dCost) (J. Paulus et al., 2002)[4]Cosine distance after warping, dCosPost

Our measure: dDPW

Page 16: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Databases and baseline Distances

Databases:

D1: 698 songs from eight classes of ballroom dancesD2: 90 songs from six classes of Cretan dances

Baseline Distances

Cosine distance (inner product)Euclidean distanceCost of warping, (dCost) (J. Paulus et al., 2002)[4]Cosine distance after warping, dCosPost

Our measure: dDPW

Page 17: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Databases and baseline Distances

Databases:

D1: 698 songs from eight classes of ballroom dancesD2: 90 songs from six classes of Cretan dances

Baseline Distances

Cosine distance (inner product)Euclidean distanceCost of warping, (dCost) (J. Paulus et al., 2002)[4]Cosine distance after warping, dCosPost

Our measure: dDPW

Page 18: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Databases and baseline Distances

Databases:

D1: 698 songs from eight classes of ballroom dancesD2: 90 songs from six classes of Cretan dances

Baseline Distances

Cosine distance (inner product)Euclidean distanceCost of warping, (dCost) (J. Paulus et al., 2002)[4]Cosine distance after warping, dCosPost

Our measure: dDPW

Page 19: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Databases and baseline Distances

Databases:

D1: 698 songs from eight classes of ballroom dancesD2: 90 songs from six classes of Cretan dances

Baseline Distances

Cosine distance (inner product)Euclidean distanceCost of warping, (dCost) (J. Paulus et al., 2002)[4]Cosine distance after warping, dCosPost

Our measure: dDPW

Page 20: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Databases and baseline Distances

Databases:

D1: 698 songs from eight classes of ballroom dancesD2: 90 songs from six classes of Cretan dances

Baseline Distances

Cosine distance (inner product)Euclidean distanceCost of warping, (dCost) (J. Paulus et al., 2002)[4]Cosine distance after warping, dCosPost

Our measure: dDPW

Page 21: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Databases and baseline Distances

Databases:

D1: 698 songs from eight classes of ballroom dancesD2: 90 songs from six classes of Cretan dances

Baseline Distances

Cosine distance (inner product)Euclidean distanceCost of warping, (dCost) (J. Paulus et al., 2002)[4]Cosine distance after warping, dCosPost

Our measure: dDPW

Page 22: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

More on Cretan dances database (D2)

Table: Tempi of D2 and Listeners’ accuracy

Dance Tempo Range (♩) Listeners’ acc. (%)

Kalamatianos 116-142 93.3

Siganos 93-103 88.9

Maleviziotis 132-160 79.2

Pentozalis 123-182 45.6

Sousta 111-136 58.3

Chaniotis 58-79 88.5

Mean 75.6

Page 23: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Results on D1: Ballroom dances

Table: Classification Accuracies on D1

wkNN kNN

Cosine 85.5 (k=7) 84.5 (k=3)

Euclidean 83.8 (k=6) 82.7 (k=3)

dCost 72.4 (k=14) 70.7 (k=7)

dCosPost 70.7 (k=32) 69.2 (k=17)

dDPW 82.1 (k=11) 80.9 (k=20)

10 repetitions of 10-fold stratified cross-validation

Page 24: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Results on D2: Cretan dances

Table: Classification Accuracies on D2

wkNN kNN

Cosine 53.8 (k=1) 53.8 (k=1)

Euclidean 48.9 (k=1) 48.8 (k=1)

dCost 51.8 (k=18) 48.5 (k=8)

dCosPost 51.1 (k=19) 48.7 (k=12)

dDPW 69.0 (k=4) 64.4 (k=5)

10 repetitions of 10-fold stratified cross-validation

Page 25: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Whales Click detection using theTeager-Kaiser operator and PhaseSpectra

In collaboration with: Varvara Kandia([email protected])

Presented at:

ECS 2008 (The Netherlands),3rd Workshop on Detection and Classification of MarineMammals, Boston 20072nd Workshop on Detection and Classification of MarineMammals, Monaco 2006

References:[5][6][7]

Page 26: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Why to do it?

Localization and tracking with passive acoustics

Study animal behavior

Abundance estimation

Correlations with physiology (size of animals, soundproduction mechanism)

Page 27: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Examples of clicks from Sperm whales

Regular clicks:

0 500 1000 1500 2000 2500 3000 3500

−1

−0.5

0

0.5

1(a)

Time in ms

Am

plitu

de

Creak clicks:

0 50 100 150 200 250 300 350 400−0.04

−0.02

0

0.02

0.04(a)

Time in ms

Am

plitu

de

Page 28: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Examples of clicks from Beaked whales

Page 29: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Approaches/Softwares for clickdetection

Rainbow click (D. Gillespie, 1997)[8]

Moby click (O. Jake, 1996)[9]

Ishmael (D. Mellinger, 2001)[10]

Page 30: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Teager-Kaiser energy operator[5][6]

Definition for a discrete time signal

Ψ[s(n)] = s2(n)− s(n + 1)s(n − 1)

For a signal with 3 components: interference x [n],transient y [n], and noise u[n], so s[n] = x [n] + y [n] + u[n]:

Ψ[s(n)] = Ψ[x(n)] + Ψ[y(n)] + Ψ[u(n)] + T [n]

we may show that:

Ψ[s(n)] ≈ Ψ[y(n)] + w(n)

Page 31: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Teager-Kaiser energy operator[5][6]

Definition for a discrete time signal

Ψ[s(n)] = s2(n)− s(n + 1)s(n − 1)

For a signal with 3 components: interference x [n],transient y [n], and noise u[n], so s[n] = x [n] + y [n] + u[n]:

Ψ[s(n)] = Ψ[x(n)] + Ψ[y(n)] + Ψ[u(n)] + T [n]

we may show that:

Ψ[s(n)] ≈ Ψ[y(n)] + w(n)

Page 32: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Synthetic example

0 50 100 150 200150

200

250(a)

Time (ms)

Am

plitu

de

0 50 100 150 200−0.5

0

0.5

1(b)

Time (ms)

Am

plitu

de

Page 33: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Applied on clicks

From Sperm whales, Regular clicks: (a) Raw file, (b) after TK

0 500 1000 1500 2000 2500 3000 3500−1.5

−1

−0.5

0

0.5

1(a)

Time (ms)

Am

plitu

de

0 500 1000 1500 2000 2500 3000 3500−0.5

0

0.5

1(b)

Time (ms)

Am

plitu

de

Page 34: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Applied on clicks

From Sperm whales, Creak clicks: (a) Raw file, (b) after TK

0 50 100 150 200 250 300 350 400−0.04

−0.02

0

0.02

0.04(a)

Time (ms)

Am

plitu

de

0 50 100 150 200 250 300 350 400

0

0.5

1(b)

Time (ms)

Am

plitu

de

Page 35: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Comparison with Rainbow click

Det. Score: =Correctly detected hand labeled clicks

Total hand labeled clicks100

Table: Percentage (%) of correctly identified clicks per file.Tolerance of 2ms.

TK RBFile name clicks score (%) clicks score (%) clicks

F1 266 (0) 100 268 94.74 265

F2 944 (549) 60.17 986 15.68 781

F3 689 (414) 94.05 732 71.12 622

F4 529 (242) 99.81 528 75.05 440

F5 435 (155) 75.17 387 69.20 347

Page 36: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

In terms of ROC curves

1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 60

10

20

30

40

50

60

70

80

90

100Approximate ROC

Det

ectio

n R

ate

(%)

Tolerance (ms)

TKRB

Page 37: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Phase Spectrum[7]

Group delay:

τ(ω) = −dφ(ω)

dωor

τ(ω) =XR(ω)YR(ω) + XI (ω)YI (ω)

|X (ω)|2

where:

X (ω) = F(x [n]) = XR(ω) + jXI (ω)Y (ω) = F(nx [n]) = YR(ω) + jYI (ω)

Page 38: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Motivation

Page 39: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Motivation

Page 40: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Application on the Beaked whalesexample

Page 41: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Zoom on in an area of clicks

After applying an appropriate modulation and low-pass filteringto the original recordings.

Note: Triangles denote hand labels

Page 42: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Results on Beaked and Sperm Whales

Raw data/With TK

Species clicks Det (%) Corr (%) MAE (ms)

Beaked Whales 248 84.9/86.3 87.1/88.2 1.1/0.9

Sperm Whales 146 87.7/90.4 84.9/84.3 1.58/0.97

Det = Number of clicks correctly detectedTotal × 100

Corr = Total−Deleted−InsertedTotal × 100

Page 43: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

A Mathematical Model for AccurateMeasurement of Jitter

In collaboration with: Miltiadis Vasilakis([email protected])

It was presented at MAVEBA 2007, Florence

References: [11]

Page 44: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Jitter

Definition

Jitter is defined as perturbations of the glottal source signalthat occur during vowel phonation and affect the glottal pitchperiod.

Page 45: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Definitions

Let u[n] be the pitch period sequence.

Local jitter

1N−1

∑N−1n=1 |u(n + 1)− u(n)|

1N

∑n=1N u(n)

Absolute jitter

1

N − 1

N−1∑n=1

|u(n + 1)− u(n)|

Relative average Perturbation

1N−2

∑N−2n=1

|2u(n+1)−u(n)−u(n+2)|3

1N

∑n=1N u(n)

Page 46: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Definitions

Let u[n] be the pitch period sequence.

Local jitter

1N−1

∑N−1n=1 |u(n + 1)− u(n)|

1N

∑n=1N u(n)

Absolute jitter

1

N − 1

N−1∑n=1

|u(n + 1)− u(n)|

Relative average Perturbation

1N−2

∑N−2n=1

|2u(n+1)−u(n)−u(n+2)|3

1N

∑n=1N u(n)

Page 47: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Definitions

Let u[n] be the pitch period sequence.

Local jitter

1N−1

∑N−1n=1 |u(n + 1)− u(n)|

1N

∑n=1N u(n)

Absolute jitter

1

N − 1

N−1∑n=1

|u(n + 1)− u(n)|

Relative average Perturbation

1N−2

∑N−2n=1

|2u(n+1)−u(n)−u(n+2)|3

1N

∑n=1N u(n)

Page 48: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Our approach

1

ampl

itude

� ����� ��� ����� ��

time (samples)

P − ε P − εP + ε P + ε

Page 49: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

In mathematical terms

We model the glottal impulse train as:

p[n] =+∞∑

k=−∞δ[n − (2k)P] +

+∞∑k=−∞

δ[n + ε− (2k + 1)P]

We may show that its power spectrum is then:

|P(ω)|2 = H(ε, ω) + S(ε, ω)

Page 50: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

In mathematical terms

We model the glottal impulse train as:

p[n] =+∞∑

k=−∞δ[n − (2k)P] +

+∞∑k=−∞

δ[n + ε− (2k + 1)P]

We may show that its power spectrum is then:

|P(ω)|2 = H(ε, ω) + S(ε, ω)

Page 51: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Examples of power spectrum

On synthetic glottal signal

−40

−38

−36

−34

−32

−30

−28

−26

radian frequency (ω)

pow

er (

dB)

� ����� ����� ����� ����� ������ �

H(0, ω)

S(0, ω)

H(1, ω)

S(1, ω)

H(2, ω)

S(2, ω)

Page 52: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Examples of power spectrum

0 5 10 15 20−80

−70

−60

−50

−40

−30

−20harmonic & subharmonic parts of the power spectrum

frequency (kHz)

pow

er (

dB)

2.4 2.45 2.5 2.55 2.6

−55

−50

−45

a closer look at the first crossing

frequency (kHz)

pow

er (

dB)

0 5 10 15 20−80

−70

−60

−50

−40

−30

−20synthetic jitter signal (fs = 48kHz, ε = 5): power spectrum of a single frame

frequency (kHz)

pow

er (

dB)

H(ε, ω)

S(ε, ω)

|P(ω)|2

acceptedcrossing

rejectedcrossings

the circles indicatecrossings betweenthe harmonic andsubharmonic parts

Page 53: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Experiments

Goal: discriminate pathological from normal voices, based onjitter

Database: Massachusetts Eye and Ear Infirmary (MEEI)[12]

Sustained vowels,53 subjects with normal voice,657 subjects with a wide variety of pathological conditions

Jitter estimation methods:

PRAAT2007 (P. Boersma and D. Weenink) [13]Multi-Dimensional Voice Program (MDVP), (Kay-Pentaxelemetrics, 2007) [14]Our approach [11]

Page 54: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Experiments

Goal: discriminate pathological from normal voices, based onjitter

Database: Massachusetts Eye and Ear Infirmary (MEEI)[12]

Sustained vowels,53 subjects with normal voice,657 subjects with a wide variety of pathological conditions

Jitter estimation methods:

PRAAT2007 (P. Boersma and D. Weenink) [13]Multi-Dimensional Voice Program (MDVP), (Kay-Pentaxelemetrics, 2007) [14]Our approach [11]

Page 55: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

Results in ROC curves

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

False Positive Rate

Tru

e P

ositi

ve R

ate

MDVP JitaProposed method, fixed frame, sequence averageProposed method, variable frame, sequence averagePraat Jitter (local, absolute)

Page 56: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

A. Holzapfel and Y. Stylianou.

Rhythmic similarity of music based on dynamic periodicity warping.In IEEE ICASSP 2008.

Jonathan Foote, Matthew D. Cooper, and Unjung Nam.

Audio retrieval by rhythmic similarity.In Proc. of ISMIR 2002 - 3rd International Conference on Music Information Retrieval, 2002.

Geoffroy Peeters.

Rhythm classification using spectral rhythm patterns.In Proc. of ISMIR 2005 - 6th International Conference on Music Information Retrieval, pages644–647, 2005.

Jouni Paulus and A.P. Klapuri.

Measuring the similarity of rhythmic patterns.In Proc. of ISMIR 2002 - 3rd International Conference on Music Information Retrieval, 2002.

V. Kandia and Y. Stylianou.

Detection of creak clicks of sperm whales in low SNR conditions.In CD Proc. IEEE Oceans, Brest, France, 2005.

V. Kandia and Y. Stylianou.

Detection of sperm whale clicks based on the Teager-Kaiser energy operator.Applied Acoustics, 67(11-12):1144–1163, 2006.

V. Kandia and Y. Stylianou.

Detection of clicks based on group delay.Accepted in Canadian Acoustics, 2008.

D. Gillespie.

Page 57: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References

An acoustic survey for sperm whales in the Southern Ocean sanctuary conducted from the R/VAurora Australis.Rep. Int. Whal. Comm., 47:897–908, 1997.

O. Jake.

Acoustic Censusing of sperm whales at Kaikoura, New Zealand: An inexpensive method to countclicks and whales automatically.Master Thesis, University of Otago, Dunedin, New Zealand, 1996.

D. K. Mellinger.

Ishmael 1.0 Users Guide.NOAA, NOAA/PMEL/OERD, 2115 SE OSU Drive, Newport, OR 97365-5258, 2001.Technical Memorandum OAR PMEL-120.

M. Vasilakis and Y. Stylianou.

A mathematical model for accurate measurement of jitter.In MAVEBA 2007, Florence, Italy, 2007.

Kay Elemetrics.

Disordered Voice Database (Version 1.03), 1994.

Paul Boersma and David Weenink.

Praat: doing phonetics by computer (Version 4.6.24) [Computer program], 2007.

Kay Elemetrics.

Multi-Dimensional Voice Program (MDVP) [Computer program], 2007.

Page 58: Signal Processing Algorithms for Music, Marine Mammals …

SignalProcessingAlgorithmsfor Music,Marine

Mammals andSpeech

YannisStylianou

Outline of thetalk

RhythmicSimilarity ofMusic

Whales ClickDetections

Measuringjitter

References