Hidden Markov Model: Overview and Applications in MIR

1

Hidden Markov Model: Overview and Applications in MIR

Hidden Markov Model: Overview and Applications in MIR

MUMT 611, March 2005Paul Kolesnik

MUMT 611, March 2005Paul Kolesnik

2

ContentsContents

Introduction to HMM Overview of Publications Conclusion

Introduction to HMM Overview of Publications Conclusion

3

IntroductionIntroduction

Definition A structure that is used to statistically characterize the

behavior of sequences of event observations Extension of a model known as Markov chains “A double stochastic process with an underlying stochastic

process which is not observable, but can only be observed through another set of stochastic process that produces the sequence of observed symbols” (Rabiner and Huang 1986)

Concepts Any observable sequence can be represented as a succession

of states, with each state representing a grouped portion of the observation values and containing its features in a statistical form

Definition A structure that is used to statistically characterize the

behavior of sequences of event observations Extension of a model known as Markov chains “A double stochastic process with an underlying stochastic

process which is not observable, but can only be observed through another set of stochastic process that produces the sequence of observed symbols” (Rabiner and Huang 1986)

Concepts Any observable sequence can be represented as a succession

of states, with each state representing a grouped portion of the observation values and containing its features in a statistical form

4

IntroductionIntroduction

Concepts (ctd.) HMM keeps track of

What state will the sequence start in What state-to-state transitions are likely to take place What values are likely to occur in each state

Corresponding parameters Array of initial state probabilities Matrix of state-to-state transitional probabilities Matrix of state output probabilities

= (, A, B)

Concepts (ctd.) HMM keeps track of

What state will the sequence start in What state-to-state transitions are likely to take place What values are likely to occur in each state

Corresponding parameters Array of initial state probabilities Matrix of state-to-state transitional probabilities Matrix of state output probabilities

= (, A, B)

5

IntroductionIntroduction

Markov Model Example. - x — States of the Markov model - a — Transition probabilities - b — Output probabilities - y — Observable outputs

6

IntroductionIntroduction

Three Main HMM Problems Recognition

given an observation sequence and a Hidden Markov Model, calculate the probability that the model would produce this observation sequence.

Uncovering (Viterbi) given an observation sequence and a Hidden Markov Model,

calculate the optimal sequence of states that would maximize the likelihood of the HMM producing the observation.

Learning / Training given an observation sequence (or a set of observation

sequences and a Hidden Markov Model, adjust the model parameters, so that probability of the model is maximized.

Three Main HMM Problems Recognition

given an observation sequence and a Hidden Markov Model, calculate the probability that the model would produce this observation sequence.

Uncovering (Viterbi) given an observation sequence and a Hidden Markov Model,

calculate the optimal sequence of states that would maximize the likelihood of the HMM producing the observation.

Learning / Training given an observation sequence (or a set of observation

sequences and a Hidden Markov Model, adjust the model parameters, so that probability of the model is maximized.

7

IntroductionIntroduction

HMM History Basic concept developed by Markov Theory for practical implementation summarized by

Rabiner and Huang (1986) Applied in different fields to data stream

observation problems Common in speech recognition Has become increasingly popular in music

information retrieval applications

HMM History Basic concept developed by Markov Theory for practical implementation summarized by

Rabiner and Huang (1986) Applied in different fields to data stream

observation problems Common in speech recognition Has become increasingly popular in music

information retrieval applications

8

Overview of WorksOverview of Works

Automatic Segmentation for Music Classification using Competitive Hidden Markov Models

(2000) Battle, Cano – University Pompeu Fabra System classifies audio segments (automatic

segmentation into abstract acoustic events) can be applied to classify a database of audio sounds allows fast indexing and retrieval of audio fragments similar segment events are given the same label

Automatic Segmentation for Music Classification using Competitive Hidden Markov Models

(2000) Battle, Cano – University Pompeu Fabra System classifies audio segments (automatic

segmentation into abstract acoustic events) can be applied to classify a database of audio sounds allows fast indexing and retrieval of audio fragments similar segment events are given the same label

9

Overview of WorksOverview of Works

(2000) Battle, Cano (ctd.) First stage: parametrization, features obtained from

audio signals Mel-cepstrum analysis used to obtain feature vectors Main classification engine: HMM-based Traditional HMMs are not suited for blind learning Competitive HMMs used instead CoHMMs differ from HMMs only in training stage;

recognition is exactly the same

(2000) Battle, Cano (ctd.) First stage: parametrization, features obtained from

audio signals Mel-cepstrum analysis used to obtain feature vectors Main classification engine: HMM-based Traditional HMMs are not suited for blind learning Competitive HMMs used instead CoHMMs differ from HMMs only in training stage;

recognition is exactly the same

10

Overview of WorksOverview of Works

Melody Spotting Using Hidden Markov Models (2001) Durey, Clements–Georgia Institute of

Technology A melody-based database song retrieval system Uses melody spotting procedure adopted from word

spotting techniques in automatic speech recognition Humming, whistling, keyboard as input Main goal: develop a practical system for non-symbolic

music representation (audio) Word/melody-spotting: searching for a data segment in a

data stream

Melody Spotting Using Hidden Markov Models (2001) Durey, Clements–Georgia Institute of

Technology A melody-based database song retrieval system Uses melody spotting procedure adopted from word

spotting techniques in automatic speech recognition Humming, whistling, keyboard as input Main goal: develop a practical system for non-symbolic

music representation (audio) Word/melody-spotting: searching for a data segment in a

data stream

11

Overview of WorksOverview of Works

(2001) Durey, Clements (ctd.) Uses monophonic melodies, both audio and MIDI data Left-to-right, 5-state HMM to represent each available

note and a rest Frequency and time-domain features for feature vectors Constructs an HMM model from the input query Runs all of the feature vectors from the songs in the

database using Viterbi process A ranked list of melody occurances in database songs is

created System presented as a proof-of-concept

(2001) Durey, Clements (ctd.) Uses monophonic melodies, both audio and MIDI data Left-to-right, 5-state HMM to represent each available

note and a rest Frequency and time-domain features for feature vectors Constructs an HMM model from the input query Runs all of the feature vectors from the songs in the

database using Viterbi process A ranked list of melody occurances in database songs is

created System presented as a proof-of-concept

12

Overview of WorksOverview of Works

Indexing Hidden Markov Models for Music Retrieval

(2002) Jin, Jagadish – University of Michigan Music retrieval system Paper describes traditional MIR HMM techniques as

effective but not efficient Efficient mechanism is suggested to index the HMMs Each state is represented by an interval / inter onset

interval ratio Each transition is transformed into a 4-dimensional box

Indexing Hidden Markov Models for Music Retrieval

(2002) Jin, Jagadish – University of Michigan Music retrieval system Paper describes traditional MIR HMM techniques as

effective but not efficient Efficient mechanism is suggested to index the HMMs Each state is represented by an interval / inter onset

interval ratio Each transition is transformed into a 4-dimensional box

13

Overview of WorksOverview of Works

(2002) Jin, Jagadish (ctd.) All boxes are inserted into R-tree, an indexing

structure for multidimensional data HMMs are ranked by the number of boxes Most likely candidates are selected for

evaluation The evaluation uses the traditional forward

algorithm

(2002) Jin, Jagadish (ctd.) All boxes are inserted into R-tree, an indexing

structure for multidimensional data HMMs are ranked by the number of boxes Most likely candidates are selected for

evaluation The evaluation uses the traditional forward

algorithm

14

Overview of WorksOverview of Works

Chord Segmentation and Recognition using EM-Trained Hidden Markov Models (2003) Sheh, Ellis – Columbia University

Uses HMM for chord recognition, EM (Expectation-Maximization) Algorithm to train them

PCP (Pitch Class Profile) vectors used as features to train HMMs

HMM model for each chord type (ex. ‘A Minor’, etc.) System able to successfully recognize chords in

unstructured, polyphonic, multi-timbre audio

Chord Segmentation and Recognition using EM-Trained Hidden Markov Models (2003) Sheh, Ellis – Columbia University

Uses HMM for chord recognition, EM (Expectation-Maximization) Algorithm to train them

PCP (Pitch Class Profile) vectors used as features to train HMMs

HMM model for each chord type (ex. ‘A Minor’, etc.) System able to successfully recognize chords in

unstructured, polyphonic, multi-timbre audio

15

Overview of WorksOverview of Works

Effectiveness of HMM-Based Retrieval on Large Databases

(2003) Shifrin, Burmingham – University of Michigan Investigates performance of an HMM-based QBH

system on a large musical database VocalSearch system, part of MusArt project 50000-theme database (roughly 22000 songs) Uses <delta-pitch / Inter-onset Interval ratio> pair as

feature vectors

Effectiveness of HMM-Based Retrieval on Large Databases

(2003) Shifrin, Burmingham – University of Michigan Investigates performance of an HMM-based QBH

system on a large musical database VocalSearch system, part of MusArt project 50000-theme database (roughly 22000 songs) Uses <delta-pitch / Inter-onset Interval ratio> pair as

feature vectors

16

Overview of WorksOverview of Works

(2003) Shifrin, Burmingham (ctd) Compared perfect and imperfect queries,

simulated insertions and deletions Discovered Trends:

Longer queries have a positive effect on evaluation performance

All experiments show an early saturation point Performed well with imperfect queries on a large

database

(2003) Shifrin, Burmingham (ctd) Compared perfect and imperfect queries,

simulated insertions and deletions Discovered Trends:

Longer queries have a positive effect on evaluation performance

All experiments show an early saturation point Performed well with imperfect queries on a large

database

17

Overview of WorksOverview of Works

A HMM-Based Pitch Tracker for Audio Queries (2003) Orio, Sette – University of Padova

HMM-based approach to transcription of musical queries HMM used to model features related to singing voice A sung query is considered as an observation of an

unknown process – the melody the user has in mind Two-level HMM: event-level (using pitches as labels),

audio-level (attack-sustain-relst events) A simple model is presented, low recognition

percentages

A HMM-Based Pitch Tracker for Audio Queries (2003) Orio, Sette – University of Padova

HMM-based approach to transcription of musical queries HMM used to model features related to singing voice A sung query is considered as an observation of an

unknown process – the melody the user has in mind Two-level HMM: event-level (using pitches as labels),

audio-level (attack-sustain-relst events) A simple model is presented, low recognition

percentages

18

Conclusion Conclusion

HTML Bibliographyhttp://www.music.mcgill.ca/~pkoles

Questions

HTML Bibliographyhttp://www.music.mcgill.ca/~pkoles

Questions