Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University "Culture-dependent emotional reactions to music Auditory roughness,

Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University

"Culture-dependent emotional reactions to music

Auditory roughness, cultural background, and musical tension-release judgments

Pantelis N. Vassilakis, Ph.D.DePaul UniversitySchool of Music


Prelude

Meaning is neither a property of a stimulus, nor the result of a physiological response to such stimulus, nor a contextual side-effect, nor a product of 'the mind.' Rather, it is a manifestation of the intersection among the stimulus as a physical entity, context, our physiology, and our mind and is partially received from, partially attached to the stimulus. Therefore, meanings and emotions associated with objects and events are not based on the isolated objects/events but, rather, on our relationship (as observers) to objects/events.


1. Auditory roughnessConcept & Models

2. SRA - Spectral AnalysisNew versus old methods

3. Roughness & musical tensionDependence on cultural background

At A Glance


Auditory Roughness

Auditory Roughness: Term introduced in the late 19th century by Helmholtz to describe the harsh, raspy sound quality of narrow harmonic intervals. E.g.

An acoustic/sensory dimension of dissonance. One of the perceptual manifestations of interference, expressed as a function of a complex signal’s spectral distribution; an attribute of timbre

_ Adding two sine signals with frequencies f1 and f2 results in a complex signal whose amplitude fluctuates between a minimum and a maximum value at a rate equal to |f1-f2|.

_ Amplitude fluctuation rate :a) < ~15 fluctuations/second → Beating (e.g. 5 fluctuations/second) b) between ~15 and ~75-150 fluctuations/second → Roughness (e.g. 35 fluct/sec)Regardless of origin (narrow harmonic intervals, fast trills, amplitude modulation, etc. ), such amplitude fluctuation rates give rise to the sensation of roughness.(gradual move of 2 sinusoids from a major second to a unison harmonic interval)c) > ~ 150 fluctuations/second → combination tones, envelope pitch, etc..


• Helmholtz (1885)

• Plomp & Levelt (1965)

• Kameoka & Kyriyagawa (1969a,b)

• Hutchinson & Knopoff (1978)

• Parncutt (1989)

• Daniel & Weber (1997)

• Sethares (1998)

• Leman (2000) • Pressnitzer, McAdams & Colleagues (1997, 1999a, 1999b, 2000) (auditory periphery mechanisms models)

• MacCallum & Einbond (2008)

Roughness calculation models


)()( 122121 ffsbffsb eeR

b1 = 3.5 b2 = 5.75

x* = 0.24

s1 = 0.0207 s2 = 18.96

211

*

sfs

xs

Roughness, frequency separation, and register


• General assumption:

Roughness is proportional to A1 * A2

Roughness and amplitude

• Previous experimental studies:

von Béckésy (1960)

Terhardt (1974)


Amplitude modulation depth versus degree of amplitude fluctuation


Proposed Roughness Calculation Model

Roughness estimation model (sine pairs): R = X * Y * Z (Vassilakis, 2001, 2005)

X = (Amin*Amax)0.1 Dependence of R on the absolute amplitude of the sines (Terhardt, 1974; Vassilakis, 2001)

Y = 0.5 [2Amin / (Amin+Amax )]3.11

Dependence of R on the relative amplitudes of the sines (von Béckésy, 1960; Terhard, 1974; Vassilakis, 2001)

Z = e-b1s(fmax - fmin) – e-b2s(fmax - fmin) [b1 = 3.5; b2 = 5.75; s = 0.24/(s1fmin + s2); s1 = 0.0207; s2 = 18.96]Dependence of R on relative (frequency difference of the sines) and absolute (frequency of the lower sine) frequencies of the added sines (Kameoka &

Kuriyagawa, 1969a&b; Plomp & Levelt, 1965; Sethares, 1998)


Roughness, phase, and signal envelope asymmetry

Pressnitzer and McAdams (1999a,b)


Comparison of 3 roughness calculation models


Comparison of 3 roughness calculation models

(Vassilakis)


Drawbacks of traditional FFT

Spectral Analysis

Frequency and time values returned are forced to fit onto the time-frequency grid defined by the analysis window

Frequency/temporal “smearing” and uncertainty on precise energy values


Frequency analysis and spectral peak pickingSTFT algorithm based on the Reassigned Bandwidth-Enhanced Additive Model (Fitz & Haken, 2002; Fitz et al., 2003)

Spectral Analysis

Dual STFT, fine-tuning spectral analysis results _ Frequency: time derivative of the argument (phase) of the complex analytic signal representing a given frequency bin _ Time: frequency derivative of the STFT phase, defining the local group delay (correction that pinpoints the precise excitation time)

Theory:

_developed by Kodera et al. (1976)

_expressed mathematically by Auger & Flandrin (1995)

_implemented to sound spectral analysis by Fulop & Fitz (2006a,b; 2007)


SRA online

http://musicalgorithms.ewu.edu/algorithms/roughness.htmlhttp://www.acousticslab.org/roughness

http://musicalgorithms.ewu.edu/algorithms/roughness.html

http://www.acousticslab.org/roughness


Musical tension / release & Roughness

_ tonal center_ dynamics_ pitch_ duration

_ orchestration_ performance techniques_ sensory dissonance(presence of auditory roughness)

Contrasts in terms of(Bigand, 1993, 1997; Bigand et al., 1996; Bigand and Parncutt, 1999; Krumhansl, 1996, 1997, 2002)

_ Auditory roughness contrasts constitute just one of the expressive tools contributing to the creation of musical tension/release patterns.

_ The assumption within the Western musical tradition that rough sounds should be avoided limits the range of roughness variations explored.

Performance practices outside the Western art musical tradition explore a much wider roughness range. (Vassilakis, 2005)


_ Interaction between drone and melody (India)_ Quechua Haraui songs (Peru)_ Xingu river Taquara flutes (Brazil)_ Mijwiz (two paired single-reed pipes - Middle East)_ Ganga songs (Bosnia)

Non-Western musical traditions & roughness


Limited possibilities for extensive exploration of dynamic, pitch, tonal, and timbral (other than roughness) contrasts.

Mijwiz (two paired single-reed pipes - Middle East) (Vassilakis et al. 2006)

For its expressive power, the mijwiz relies on extensive explorations of narrow harmonic intervals, corresponding to sophisticated manipulations of roughness degrees.

Pieces on the mijwiz are good candidates for examining

a. roughness and musical tension contrast relationships, as determined by listeners familiar with the style

A piece on the mijwiz: roughness/tension patterns

b. the relationship between cultural background and musical tension judgments.

• requires high pressure air-flow to sustain a steady tone

• is often performed using the circular breathing technique

• has narrow melodic range and limited harmonic menu

• has two reeds that are activated just by air pressure



Musical piece: Recording of a stylized improvisation on the mijwiz, performed by A. J. Racy

Roughness Profile EstimationBased on SRA, a custom application that automates frequency analysis and roughness calculation of complex signals, at user-specified time intervals.

Purpose: Compare the roughness profile of a musical piece on the mijwiz to patterns of tension and release identified by listeners



Participants could listen to the piece prior to the experiment as many times as necessary to familiarize themselves with it.

Tension / Release Judgments

Improviser & Western-trained musicians

Procedure

Experiment designed using MEDS (Kendall, 2002)

While listening to the piece, they were asked to tap the ‘M’ key on the keyboard whenever they sensed the musical tension increasing and the 'Z' key whenever they sensed the musical tension decreasing. They were instructed to continue tapping the appropriate key throughout the increase or decrease in tension and to tap no key if they did not sense a change in tension.


Roughness / Tesion Profile of the Improvisation on the Mijwiz

0

20

40

60

80

100

120

140

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72

Time (seconds)

Ro

ug

hn

es

s /

Te

ns

ion

lev

el

(arb

itrar

y un

its)

Roughness Profile (7-point moving average; 250ms)

Roughness Profile



0

20

40

60

80

100

120

140

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72

Time (seconds)

Ro

ug

hn

es

s /

Te

ns

ion

lev

el

(arb

itrar

y un

its)


Tension Profile - Improviserr=0.422

Roughness & Tension Profiles



0

20

40

60

80

100

120

140

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72

Time (seconds)

Ro

ug

hn

es

s /

Te

ns

ion

lev

el

(arb

itrar

y un

its)


Tension Profile - Improviser

Tension Profile - Listenersr=0.068

r=0.422

Roughness & Tension Profiles


Roughness/Tension Profiles - Results

Different listeners may employ different listening strategies that are related to musical expectations outlined by the different underlying musical traditions.

Improviser (Middle Eastern)

Roughness contrasts created through frequent moves between narrow and unison harmonic intervals constitute musical expressive cues that are widely accepted and employed within the Near Eastern musical tradition to communicate changes in musical tension.

Composers and performers incorporate these contrasts when setting up tension/release patterns and listeners look out for them, even if implicitly, when they organize perceptually and attach meaning to musical pieces.


For listeners raised within the Western musical tradition, auditory roughness contrasts play a limited role when it comes to identifying musical tension/release patterns, while tonal and temporal contrasts provide more significant musical cues and set up stronger tension/resolution expectations.

Listeners (Western-trained)





0

20

40

60

80

100

120

140

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72

Time (seconds)

Ro

ug

hn

es

s /

Te

ns

ion

lev

el

(arb

itrar

y un

its)




r=0.422



Ganga

Correct and good performances are governed by complex and systematic rules.

Consists of alternating solo and choral sections, with the choral sections involving moves between minor 2nd and major 2nd harmonic intervals, occasionally interrupted by short passages in unison.

Within its geographical territory, ganga is often valued for its distinct sonic effect rather than its semantic content.

Singing style common in Bosnia-Herzegovina and the Dalmatian Zagora regions of the Balkans.

The length and content of the lyrics vary greatly among performances. Singers are relatively free in their choice of words, which are often just vocalizations (Petroviç 1977).


Ganga rules

• Singers must sing loudly and maintain uniform strength among each other and through time.

• The voices must be as similar in timbre as possible, nasal, and without vibrato so that they blend.

• Singers must not move or dance while singing. They must stand tightly together, in an arch, turned slightly towards each other so that their voices will “collide” at the right point.


Ganga rules



• Singers must not move or dance while singing. They must stand tightly together, in an arch, turned slightly towards each other so that their voices will “collide” at the right point.


Ganga rules



• Singers must not move or dance while singing. They must stand very tightly together, in an arch, turned slightly towards each other so that their voices will “collide” at the right point.

• The two to three voice parts in the choral sections must be sung in minor

and major 2nd harmonic intervals with possible periodic insertions of unisons, always resolving on a major second.


Ganga songs are good candidates for examining

a. the relationship between auditory roughness profiles and patterns of musical tension, as determined by listeners familiar with the style

b. the relationship between cultural background and musical tension judgments.

• If the conditions outlined in the rules are not fulfilled, the ganga “will not be good.”

• Ganga represents a rare folk vocal genre where the sense of a song is related more to its sound than to its lyrics, with the conditions for a good song that makes sense being also conditions that guarantee the perceptual salience of roughness contrasts.

Ganga rules (cont.)


A ganga song: roughness/tension patterns

Musical piece: Odkad seke nismo zapjevale in “Bosnia: Echoes from an Endangered World – Music and Chant of the Bosnian Muslims”Ted Levin and Ankiça Petroviç, © 1993 Smithsonian Folkways Recordings

Roughness Profile CalculationBased on SRA, a custom application that automates frequency analysis and roughness calculation of complex signals, at user-specified time intervals.

Purpose: Comparison among the roughness profile of a ganga song and patterns of tension and release identified by i) a ganga expert and ii) musician listeners unfamiliar with the ganga style.

http://www.acousticslab.org/roughness


Participants could listen to the piece prior to the experiment as many times as necessary to familiarize themselves with it.

Tension / Release Judgments

Bosnian-born ganga scholar & 10 Western-trained musicians

Procedure

Experiment designed using MEDS (Kendall, 2002)

During the experiment, they were asked to tap the ‘M’ key on the keyboard whenever they sensed the musical tension increasing and the 'Z' key whenever they sensed the musical tension decreasing. They were instructed to continue tapping the appropriate key throughout the increase or decrease in tension and to tap no key if they did not sense a change in tension.

A ganga song: roughness/tension patterns


Roughness ProfileRoughness / Tesion Profile - Ganga song

0

10

20

30

40

50

60

70

80

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60

Time (seconds)

Ro

ug

hn

ess

/ T

ensi

on

lev

el

(arb

itrar

y un

its)



Roughness & Tension ProfilesRoughness / Tesion Profile - Ganga song

0

10

20

30

40

50

60

70

80

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60

Time (seconds)

Ro

ug

hn

ess

/ T

ensi

on

lev

el

(arb

itrar

y un

its)


Tension Profile (Bosnian Expert)



0

10

20

30

40

50

60

70

80

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60

Time (seconds)

Ro

ug

hn

ess

/ T

ensi

on

lev

el

(arb

itrar

y un

its)


Tension Profile (Bosnian Expert)

Tension Profile (Bosnian Expert) - Adjusted



0

10

20

30

40

50

60

70

80

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60

Time (seconds)

Ro

ug

hn

ess

/ T

ensi

on

lev

el

(arb

itrar

y un

its)



Tension Profile (Western-Trained Musicians)



0

10

20

30

40

50

60

70

80

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60

Time (seconds)

Ro

ug

hn

ess

/ T

ensi

on

lev

el

(arb

itrar

y un

its)

Roughness Profile (11-point moving average; 250ms)Tension Profile (Bosnian Expert) - AdjustedTension Profile (Western-Trained Musicians)Tension Profile (Western-Trained Musicians) - Adjusted


Roughness / Tesion Profile - Ganga song

0

10

20

30

40

50

60

70

80

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60

Time (seconds)

Ro

ug

hn

ess

/ T

ensi

on

lev

el

(arb

itrar

y un

its)



Tension Profile (Western-Trained Musicians) - Adjusted




0

10

20

30

40

50

60

70

80

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60

Time (seconds)

Ro

ug

hn

ess

/ T

ensi

on

lev

el

(arb

itrar

y un

its)




r=0.44

r=0.36r=0.82





Ganga scholar (Bosnian)

Roughness contrasts created through frequent moves among minor 2nd, major 2nd, and unison harmonic intervals constitute musical expressive cues that are widely accepted and employed within the ganga style to communicate changes in musical tension (“harmonious dissonance”).

Ganga singers incorporate these contrasts when setting up tension/release patterns and initiated listeners look out for them, even if implicitly, when they organize perceptually and attach meaning to the songs.


For listeners trained / raised within the Western musical tradition, auditory roughness contrasts play a limited role when it comes to identifying musical tension/release patterns, while tonal and temporal contrasts provide more significant musical cues and set up stronger tension/resolution expectations.

Listeners (Western-trained)





0

10

20

30

40

50

60

70

80

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60

Time (seconds)

Ro

ug

hn

ess

/ T

ensi

on

lev

el

(arb

itrar

y un

its)




r=0.44

r=0.36r=0.82




0

20

40

60

80

100

120

140

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72

Time (seconds)

Ro

ug

hn

es

s /

Te

ns

ion

lev

el

(arb

itrar

y un

its)




r=0.422




Both experiments presented indicate that, using familiar sound-parsing strategies on pieces created employing unfamiliar sound-organization tools, may leads to listener interpretations that differ from the performer’s expressive intent.

Musical tension and release are culture-specific concepts, guided by the equally culture-specific musical cues used to organize and recognize them.


References

Auger, F. and Flandrin, P. (1995). "Improving the readability of time frequency and time scale representations by the reassignment method," IEEE Transactions on Signal Processing 43: 1068-1089.

von Békésy, G. (1960). Experiments in Hearing. New York: Acoustical Society of America Press (1989).

Beyer, R.T. (1999). Sounds of Our Times. Two Hundred Years of Acoustics. New York: AIP Press; Springer.

Bigand, E. (1993). The influence of implicit harmony, rhythm and musical training on the abstraction of "tension-relaxation schemas" in tonal musical phrases. Contemporary Music Review 9(1&2): 123-137.

_ (1997). Perceiving musical stability: The effect of tonal structure, rhythm, and musical expertise. Journal of Experimental Psychology, Human Perception, and Performance 23(3): 808-822.

Bigand, E. and Parncutt, R. (1999). Perceiving musical tension in long chord sequences. Psychol. Res. 62(4): 237-254.


References (cont.)

Bigand, E., Parncutt, R., and Lerdahl, F. (1996). “Perception of musical tension in short chord sequences: The influence of harmonic function, sensory dissonance, horizontal motion, and musical training,” Perception & Psychophysics 58: 125-141.

Daniel, P. and Weber, R. (1997). “Psychoacoustical roughness: Implementation of an optimized model,” Acustica 83: 113-123.

Fitz, K. and Haken, L. (2002). “On the use of time-frequency reassignment in additive sound modeling,” Journal of the Audio Engineering Society 50(11): 879-893.

Fitz, K., Haken, L., Lefvert, S., Champion, C., and O'Donnell, M. (2003). "Cell-utes and flutter-tongued cats: Sound morphing using Loris and the Reassigned Bandwidth-Enhanced Model," Computer Music Journal 27(4): 44-65.

Fulop, S.A. and Fitz, K. (2006a). "Algorithms for computing the time-corrected instantaneous frequency (reassigned) spectrogram, with applications," J. Acoust. Soc. Am. 119(1): 360-371.


References (cont.)

Fulop, S.A. and Fitz, K. (2006b). "A spectrogram for the twenty-first century," Acoustics Today 2(3): 26-33.

Fulop, S.A. and Fitz, K. (2007). "Separation of components from impulses in reassigned spectrograms," J. Acoust. Soc. Am. 121(3): 1510-1518.

Helmholtz, H. L. F. 1885 [1954]. On the Sensations of Tone as a Physiological Basis for the Theory of Music. 2nd English edition. New York: Dover Publications. [Die Lehre von den Tonempfindungen, 1877. 4th German edition, trans. A. J. Ellis.]

Kameoka, A. and Kuriyagawa, M. (1969a). "Consonance theory, part I: Consonance of dyads," J. Acoust. Soc. Am. 45(6): 1451-1459.

Kameoka, A. and Kuriyagawa, M. (1969b). "Consonance theory, part II: Consonance of complex tones and its calculation method," J. Acoust. Soc. Am. 45(6): 1460-1469.

Kendall, R.A. (2002). Music Experiment Development System (MEDS) 2001B for Windows. Los Angeles: University of California Los Angeles, Department of Ethnomusicology, Program in Systematic Musicology.


References (cont.)

Krumhansl, C.L. (1996). Aperceptual analysis of Mozart’s Piano Sonata K. 282: Segmentation, tension, and musical ideas. Music Perception 13(3): 401-432.

_ (1997). An Exploratory Study of Musical Emotions and Psychophysiology. Canadian Journal of Experimental Psychology 51(4): 336-353.

_ (2002). Music: A Link Between Cognition and Emotion. Current Directions in Psychological Science 11(2): 45-50.

MacCallum, J. and Einbond, A. (2008). Real-Time Analysis of Sensory Dissonance. Berlin / Heidelberg: Springer.

Parncutt, R. (1989). Harmony: A Psychoacoustical Approach. Berlin: Springer-Verlag

Petroviç, A. (1977). Ganga: A Form of Traditional Rural Singing in Yugoslavia. Unpublished Dissertation. Belfast: University of Belfast; Department of Social Anthropology.

Plomp, R. and Levelt, W. J. M. (1965). "Tonal consonance and critical bandwidth," J. Acoust. Soc. Am. 38(4): 548-560.


References (cont.)

Pressnitzer, D. and McAdams, S. (1997). "Influence of Phase Effects on Roughness Modeling," ICMC: International Computer Music Conference, Thessaloniki, Greece, September 1997.

Pressnitzer, D. and McAdams, S. (1999a). "Two phase effects on roughness perception, ". J. Acoust. Soc. Am. 105(5): 2773-2782.

Pressnitzer, D. and McAdams, S. (1999b). Summation of roughness across frequency regions. In: Dau T, Hohmann V, and Kollmeier B (Eds) Temporal processing in the auditory system: Psychophysics, physiology and models of hearing, pages 105-108. World Scientific Publishing, Singapore.

Pressnitzer, D., McAdams, S., Winsberg, S., and Fineberg, J. (2000). " Perception of musical tension for non-tonal orchestral timbres and its relation to psychoacoustic roughness," Perception and Psychophysics, 62(1): 66-80.

Racy, A.J. (1994). A dialectical perspective on musical instruments: the East Mediterranean Mijwiz. Ethnomusicology 38(1): 37-58.

Sethares, W. A. (1998). Tuning, Timbre, Spectrum, Scale. London: Springer-Verlag.


References (cont.)

Terhardt, E. (1974). "On the perception of periodic sound fluctuations (roughness)," Acustica 30(4): 201-213.

Vassilakis, P.N. (2001). Perceptual and Physical Properties of Amplitude Fluctuation and their Musical Significance. Doctoral Dissertation. University of California, Los Angeles.

Vassilakis, P.N. (2005). Auditory roughness as means of musical expression. Selected Reports in Ethnomusicology 12: 119-144.

Vassilakis, P.N., Kendall, R.A., and Racy, A.J. (2006). The worlds of music: culture-dependent emotional reactions to an improvisation on the mijwiz. Proceedings of the 51st SEM: 197. University of Hawai’i at Manoa.

Vassilakis, P.N. and Fitz, K. (2007). SRA: A Web-based Research Tool for Spectral and Roughness Analysis of Sound Signals. Supported by a Northwest Academic Computing Consortium grant to J. Middleton, Eastern Washington University. [http://musicalgorithms.ewu.edu/algorithms/roughness.html] or [http://www.acousticslab.org/roughness]

Vassilakis, P.N. (2008). Culture-dependent Emotional Reactions to Music - DePaul University "Culture-dependent emotional reactions to music Auditory roughness,

Documents

music auditory roughness

music depaul university

fluctuationssecond roughness

sensation of roughness

roughness calculation

glance slide

amplitude modulation

amplitude fluctuation