Effects of Emotions on Configuration of the Speech ... · Effects of Emotions on Configuration of the Speech Articulators Tatsuya Kitamura Faculty of Intelligence and Informatics,

EffectsofEmotionsonConfigurationoftheSpeechArticulators

TatsuyaKitamuraFacultyofIntelligenceandInformatics,

KonanUniversity

1

BackgroundandObjectives

• Emotionsarelinkedtovariousphysicalandphysiologicalreactions.• Suchreactionsaffecttheconfigurationandphysicalpropertiesofthespeechorganswhetherornotapersonisspeaking.• Thechangesinthespeechorganscouldcontributetothevoicequalityofemotionalvoice.• WeproposeandverifyahypothesisonemotionalspeechproductionandverifybyMRImeasurements.• Sideeffecthypothesisofemotional speechproduction

2

“Emotion”inThisStudy

3

Affection

Emotion

Feeling

Primitiveemotion

Basicemotion

Hierarchicalhypothesisoffeelingbasedontheevolutionofthebrain(Fukuda,2008)

Socialfeeling

Intellectualfeeling

Pleasure/Displeasure

Joy,Anger,Fear,Disgust,Acceptance/Affection

Love,Jealousy

Affectionassociatedwithconceptssuchascultureandreligion

Acqu

ireddu

ring

evolutionaryhistory

SideEffectHypothesisofEmotionalSpeechProduction

4

Motorplanning

Phonation/Articulation

Speech

Phonemesequence

Emotion

Thehypothesissupposesthatthereisnotaspecificarticulationtargetforemotionalspeechproduction.

Words/Sentences

(Kitamura,2010)

Physicalandphysiologicalchanges

Changesinthevoicequality

EmotionalSpeech

HowtoVerifytheHypothesis

• Measurethespeechorganconfigurationwithandwithoutspeakingundercertainemotions.• Examinethesimilaritybetweenthedeformationofthespeechorganconfiguration.• Ifthedeformationhasthesametendencyfortheemotions,thearticulationcouldbeaffectedbytheemotionsdirectly.à Theresultscanbeinterpretedasindirectly

verifyingthehypothesis

5

Methods

• MRIscanner:Shimadzu-MarconiMAGNEXECLIPSE1.5PowerDrive250(ATRBAIC)• Participants:2Japaneseprofessionalactors(1male&1female)• Emotions:Neutral,Hotanger,Joy,and Sadness

6

Opticalmic.

MRIscanner

Dialogues

TheparticipantswereaskedtotakeontheroleofSpeakerBandutter/eh/.

Hotanger A: That was a complete lie.B: Eh, I can never forgive you.

Joy A: You’ve got a job offer?B: Eh, yes, I have.

Sadness A: You broke that expensive dish?B: Eh, yes, I did.

7

DataAcquisition

TheparticipantswereinstructedNOTtoprepareforthearticulationwhileexpressingtheemotionssilently.

8

Instructiontoexpressemotion

MRdataacquisition

/eh/ t/eh/

Soundrecording

nonspeaking speaking speaking

Results:NeutralandHotAnger(FemaleActor)

9Hotanger

Speaking

Speaking

Nonspeaking

Nonspeaking

Neutral

Thepharyngealandlaryngealcavitiesarenarrowed.

Thelaryngealheightishigherthanthatforneutral.

Results:JoyandSadness(FemaleActor)

10

Nonspeaking

Nonspeaking

Speaking

Speaking

Sadness

Joy Thelaryngealheightislowerthanthatforneutral.

Thepharynxandlarynxareblurred.

Thepharyngealandlaryngealcavitiesdilate.

Results:NeutralandHotAnger(MaleActor)

11Hotanger

Neutral

Speaking

Speaking

Nonspeaking

Nonspeaking

Thefrontofthetonguedorsumupandforward.

Thelaryngealheightislowerthanthatforneutral.

Results:JoyandSadness(MaleActor)

12

Speaking

Speaking

Nonspeaking

Nonspeaking Thepharyngealandlaryngealcavitiesdilate.

Sadness

Joy Thelowerjawispulledbackward.

Discussion

• Emotionsaffectthespeechorganconfigurationregardlessofwhetherornotthepersonisspeaking.• Sametendency ofchangesinthespeechorganconfigurationforthefouremotions.

• Limitations• Wemeasuredsimulatedemotionalstates.• Wecouldnotexcludethepossibilitythattheparticipantspreparedforthearticulationduringthenonspeakingstates.

13

Conclusions

• WeproposedthesideeffecthypothesisofemotionalspeechproductionandevaluateditbyMRIexperiments.• Theeffectsonthespeechorganconfigurationinthespeakingandnonspeakingstatesweresimilarforemotions.• Thehypothesiswassupportedindirectly.

14

AcknowledgementsThisstudywassupportedbySCOPE(071705001)oftheMinistryofInternalAffairsandCommunications,Japan,andJSPSKAKENHI(21300071,25280066,and25240026).