Melodic Style Detection in Hindustani Music Amruta Vidwans Prateek Verma Preeti Rao Department of Electrical Engineering Indian Institute of Technology Bombay
Melodic Style Detection in Hindustani Music
Amruta Vidwans Prateek Verma
Preeti Rao
Department of Electrical Engineering Indian Institute of Technology
Bombay
Department of Electrical Engineering , IIT Bombay 2 of 25 of 40
OUTLINE
p Introduction n Objective
p Style Classification in Indian Classical Vocal Music n Literature Survey
n Previous Work
n Feature Description
n Database and Listening Tests
n Results and Extention to Turkish Music
p Style Classification In Flute Alap recordings n Database and Annotation
n Signal Characteristics of Discriminatory Ornaments
n Feature Design
p Conclusion and Future Work p References
2/27
Department of Electrical Engineering , IIT Bombay 3 of 25 of 40
INTRODUCTION Objective
p Melodic features to distinguish n Hindustani, Carnatic and Turkish music n Instrument playing styles
p Basis: Melody line alone suffices for listeners to reliably distinguish musical styles
p Applications: Extract important metadata automatically
3/27
Department of Electrical Engineering , IIT Bombay 4 of 25
Style Classification in Indian Classical Vocal Music
4/27
Department of Electrical Engineering , IIT Bombay 5 of 25 of 40
INTRODUCTION Literature Survey – Similar work done
p Timbral features n Features based on timbre such as MFCC, delta-MFCC, and spectral features used
by Parul et. al (ISMIR13) to distinguish Indian music genres using Adaboost, GMM etc.
p Timbral + Rhythmic features n Kini et. al. (NCC10) used these features to do genre classification of North Indian
Classical Music into Quawali, Bhajan, Bollywood etc. n Liu et. al. (ICASSP09) used timbral, wavelet coefficients and rhythmic features to
classify different styles viz. Arabic, Chinese, Japanese, Indian, Western classical p Emphasized that the diversity of Indian Classical Music is difficult to model
p Timbral +Melodic features n Salamon et al (ICASSP12) gave a large number of pitch based features in
addition to timbral features which improved the performance.
5/27
Department of Electrical Engineering , IIT Bombay 6 of 25 of 40
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Pre-processing
p Melody extraction n Difficulty: Presence of multiple instruments along with voice n Accuracy of the present state of the art automatic pitch trackers ~80% n Semi automatic approach for pitch detection is used to achieve best possible
performance
6/27
Carnatic raga Dwijavanti by artiste R Vedavalli
Hindustani raga Jaijaiwanti by artiste Rashid Khan
original resynthesized
Department of Electrical Engineering , IIT Bombay 7 of 25 of 40
p Localized contour shape-features n Previous study used Steady Note (SN) and Gamak Measure (GM) n A steady note: pitch segment of ‘N’ ms with standard deviation less than ‘J’
cents n Data driven parameters gave highest accuracy (N=400ms J=20cents) n SN: Normalized total duration of steady regions n GM: ratio of number of non steady regions(1sec) having oscillations in
3-7.5Hz to the total number of regions
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Feature Description
7/27
Department of Electrical Engineering , IIT Bombay 8 of 25 of 40
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Feature Description
p Primary Shape contour (PS) n Contour typology proposed by C. Adams [6] for categorizing melodic shapes in 15
types n Segments taken from silence to silence, assignment done using relation between
Initial, Final, Highest and Lowest pitch values.
8/27
Contour types 12 and 13 in alap section in raga Hindolam by Carnatic artistes (a) T. N. Sheshagopalan
(b)M. S. Subalaxmi
(a) (b)
12
13
Department of Electrical Engineering , IIT Bombay 9 of 25 of 40
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Feature Description
p Distance of highest peak in unfolded histogram from Tonic (DistTonic) n In unfolded histogram the Hindustani alaps are concentrated near the tonic,
Carnatic alap pitch distribution is closer to the upper octave tonic n Distance of the highest peak from the tonic in the unfolded histogram taken as
feature.
9/27
Rashid Khan raga Todi Sudha Raghunathan raga Subhapanthuvarali
Department of Electrical Engineering , IIT Bombay 10 of 25 of 40
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Feature Description
p Melodic Transitions (MT) n The overall progression of the melody can be characterized by this n Haar wavelet basis function used to represent the melody n Fifth level approximation is used
p Lower level approximation captures very minute variation whereas higher level gives a coarse representation.
n Normalized size of upward jumps(>1semitone) in concatenated pitch contour taken as feature.
10/27
Department of Electrical Engineering , IIT Bombay 11 of 25 of 40
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Database
p Choice of alap section used for labeling a track n Unmetered section always rendered in the start. n Database Description
p Widely performed raga pairs of same scale interval from Hindustani and Carnatic p Ragas belonging to different scales chosen p Renowned artists of various schools of music chosen
n Total of 120 alap sections equally distributed across both the styles
11/27
Scale Hindustani Raga (No. of clips)
Carnatic Raga (No. of clips )
Heptatonic Todi (12) Subhapanthuvarali (14)
Pentatonic Malkauns (18) Hindolam (12) Nonatonic Jaijaiwanti (10) Dwijavanthy (14)
Octatonic Yaman and Yaman Kalyan (20) Kalyani (20)
Department of Electrical Engineering , IIT Bombay 12 of 25 of 40
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Listening Tests
p Test Design n Hypothesis : Melody alone is sufficient to carry out style distinction n Audio clips re-syntheisized with constant timbre sound.
p Removes bias towards artist identity, pronunciation, voice quality. p Volume dynamics retained (Sum of vocal harmonics).
n Interface Description p User information in terms of training of subject as well as familiarity p Audios divided in 5 sets --12 clips of each Hindustani (H)-Carnatic (C) in each set. p Listening of 10 sec audio mandatory with option of pause, Skipping an audio not allowed p Decision label as H, C or NS (Not Sure) asked for each clip
12/27
Category Accuracy (no of participants) trained <3yrs 74.6% (10) trained 3-10yrs 79.7 % (8) trained >10yrs 89.6 % (2) Amateur 75.2 % (18) Listener 77.5 % (13) Overall Accuracy 76.9 % (51)
Department of Electrical Engineering , IIT Bombay 13 of 25 of 40
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Automatic classification and feature selection
p Quadratic classifier used on feature sets n 5 fold CV on 5 different random partitions to avoid any raga bias n Exhaustive search on all possible feature combinations for feature selection n Separability of parameterized distribution taken into account for small dataset
p Distribution of the Log Likelihood ratio (LLR) found for a feature set.
p F-ratio computed on distribution LLR as a confidence measure.
n Highest accuracy achieved is 96% for SN, GM, DistTonic, and PS feature set.
13/27
𝐿𝐿𝑅= ln (𝐿(𝑥∕H )/𝐿(𝑥∕C ) ) 𝐹 −𝑅𝑎𝑡𝑖𝑜 = ( 𝜇↓𝐻 − 𝜇↓𝐶 )↑2 /𝜎↓𝐻 ↑2 + 𝜎↓𝐶 ↑2
Department of Electrical Engineering , IIT Bombay 14 of 25 of 40
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Automatic classification and feature selection
p Confusion matrix
p Correlation with the listeners n Subjective test label across all listeners decided by 60% threshold for labeling H,
C or NS n For objective measure region around 10% of LLR values around 0 taken as NS n Cost value of -1,1 or 0 is assigned according to the agreement between
subjective and objective labels. n Highest value of correlation was found to be 0.79 across all the feature
combinations
14/27
Obs. C H NS True C 45 12 3 H 8 48 4
(a) (b) Confusion matrix for (a) listening tests (b) classifier output The horizontal labels correspond to true labels.
Obs. C H True C 58 2 H 3 57
Department of Electrical Engineering , IIT Bombay 15 of 25
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Extention to Turkish Music
0 15/27
Carnatic raga Dwijavanti by artiste R Vedavalli
Turkish makam Nihavent by artiste Hafiz Kemal Bey
Hindustani raga Jaijaiwanti by artiste Rashid Khan
Department of Electrical Engineering , IIT Bombay 16 of 25 of 40
p Experimental Setup and Results n 60 Taksims considered for classification using the melodic features n The features discussed before were applied to 3 classes. n Highest accuracy of 81.6 % was obtained using SN, GM, MT, DistTonic.
n Most confusion occurred for T and C which was validate via subjective tests
n No category of primary shape was discriminating Turkish-Carnatic clips
Obs. H C T True H 56 3 1 C 1 48 11 T 0 17 43
Obs. H C T NS True H 131 13 2 4 C 21 118 7 4 T 5 25 114 6
(a) (b) Confusion matrix for (a) Listening test output (b) Classifier output
16/27
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Extention to Turkish Music
Department of Electrical Engineering , IIT Bombay 17 of 25
Style Classification In Flute Alap recordings
17/27
Department of Electrical Engineering , IIT Bombay 18 of 25 of 40
INTRODUCTION
Example: Gayaki vs Tantrakari
18/27
Department of Electrical Engineering , IIT Bombay 19 of 25 of 40
p Gayaki vs Tantrakari styles n Gayaki : Vocal characteristics incorporated into instrument play.
p Singing voice is fluid, glides used often n Tantrakari : Instrumental characteristics (particularly plucked string)
p Melodic Leaps eg. Discreteness p Elements synonymous with pluck adapted in Flute : Tonguing p Fast Stroke pattern while playing melody adapted in Flute: Fast Tounging
p Ornament Production n Glide : Slowly lifting the finger- Rate controlling the glide shape n Vibrato : Periodic Pulsations in Air Flow n Fast Tounging : Tongue movement
p Using tongue to articulate notes differently in a melody p Synonymous with Sitar-rapid movements of right hand (Tantrakari)
n Blow Stop : p Blowing with stops while rendering a note p Periodic blowing patterns (1-2 or 1-1-2) with same/different note
INTRODUCTION
Tantrakari vs Gayaki Style
19/27
Department of Electrical Engineering , IIT Bombay 20 of 25 of 40
STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS
Database and Annotation
Artist Raga Duration(mins)
Hari Prasad Chaurasia Jhinjhoti (T-Metadata) 18:54
Hansadhwani 1:50
Yaman 2:29
Sindh Bhairav 3:30
Rupak Kulkarni Hemant (T-Metadata) 6:50
Ronu Majumdar Vibhas 5:39
Pannalal Ghosh Yaman 2:18
Shri 2:20
Pilu 1:30
Nityanand Haldipur ShuddhaBasant 2:46
Four Music Experts unaimously aggred on the two clips as tantrakari
20/27
Department of Electrical Engineering , IIT Bombay 21 of 25 of 40
STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS
Annotation Rules
p Five categories used for annotation n Tantrakari
p T1 : Blowing same notes with stops p T2 : Steady notes with abrupt movements p T3 : Discreteness with silences p T4 : Discreteness with rapid jumps p T5 : Fast Tounging with any pitch movement
n Gayaki p G1 : Oscillatory repetitive movements p G2 : Non- Oscillatory repetitive movements p G3 : Glides p G4 : Non-specific continuous pitch p G5 : Vibrato
p Final characteristic feature taken as the consensus of the feature agreed
by all the musicians
21/27
Department of Electrical Engineering , IIT Bombay 22 of 25 of 40
STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS
Database and Annotation
Artist Raga Duration(mins) f/T/G(in s)
Hari Prasad Chaurasia Jhinjhoti (T-Metadata) 18:54 352/465/267
Hansadhwani 1:50 43/14/33
Yaman 2:29 57/12/40
Sindh Bhairav 3:30 50/20/66
Rupak Kulkarni Hemant (T-Metadata) 6:50 124/102/111
Ronu Majumdar Vibhas 5:39 47/11/44
Pannalal Ghosh Yaman 2:18 39/3/66
Shri 2:20 45/0/29
Pilu 1:30 133/35/105
Nityanand Haldipur ShuddhaBasant 2:46 94/0/41
q G: Gayaki T : Tantrakari f: Niether Tantrakari
or Gayaki
q Alap-Jod-Jhala composition derived from Sitar. (It mainly contains Tantrakari elements)
q Two different style artists and their disciple chosen
22/27
Department of Electrical Engineering , IIT Bombay 23 of 25 of 40
STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS
Database and Annotation
Artist Raga Vibrato Fast
Tounging
Blow -
Stop
T3/T4 G3/G4
Hari Prasad Chaurasia Jhinjhoti(T) P P P P P
Hansadhwani P O P P P
Yaman O O O P P
Sindh Bhairav P O O P P
RupakKulkarni Hemant(T) O P P P P
RonuMajumdar Vibhas P O O P P
PannalalGhosh Yaman P O O P P
Shri P O O O P
Pilu P O O O P
NityanandHaldipur ShuddhaBasant P O O O P
23/27
Presence of Tantrakari and Gayaki elements
Department of Electrical Engineering , IIT Bombay 24 of 25 of 40
STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS Production and Acoustic Characteristics
p Blow-Stop & Fast Tonguing – Two of the most distinctive features n Energy Contour different due to difference in articulation n Higher rate of energy variation due to use of tongue. n Pitch movements do not influence energy movements. n Intensity of Tanpura almost constant
24/27
Department of Electrical Engineering , IIT Bombay 25 of 25 of 40
p Features capturing acoustic characteristics of Fast Tounging
n Sub-band Peak Ratio (SPR) =
n Sub-band FFT Ratio (SFR) =
STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS Feature Design
-- 2s segments of data across the two styles taken as data point
-- Ground Truth obtained using manual annotation
25/27
Classifier Accuracy Quadratic 74.07 Linear 73.15
Department of Electrical Engineering , IIT Bombay 26 of 25 of 40
p Conclusion n Successfully proposed melodic features to distinguish vocal music styles n Importance of energy based features brought out for flute style
classification. n Automatic classification results given with degree of separability
p Future Work n Extension of database to non-alap recordings in flute style classification
study. n Addition of new features to distinguish C and T to improve the
classification accuracy
26/27
Department of Electrical Engineering , IIT Bombay 27 of 25 of 40
1] P. Agarwal, H. Karnick and B. Raj, “A Comparative Study of Indian and Western Music Forms”, ISMIR 2013
2] S. Kini, S. Gulati and P. Rao, “Automatic genre classification of North Indian devotional music” , NCC 2010
3] Y. Liu, Q. Xiang, Y. Wang and L. Cai, “Cultural Style Based Music Classification of Audio Signals,” Acoustics, Speech and Signal Processing, ICASSP, 2009.
4] A. Kruspe et al. “Automatic classification of Music Pieces into Global Cultural Areas ”, In Proc. of 42nd International Conference on Semantic Audio, AES 2011
5] J. Salamon, B. Rocha and E. Gomez, "Musical Genre Classification using Melody Features extracted from polyphonic music signals", International Conference on Acoustics, Speech and Signal Processing, 2012
6] C. Adams, “Melodic Contour Typology” , Ethnomusicology 20.2 : 179-215 (1976) 7] S. Justin, S. Gulati and X. Serra, "A Multipitch Approach to Tonic Identification in
Indian Classical Music", ISMIR, 2012 8] C. Clemments, “Pannalal Ghosh and the Bānsurī in the Twentieth Century” , City
University of New York, New York City, September 2010
27/27
References