EffectsofEmotionsonConfigurationoftheSpeechArticulators
TatsuyaKitamuraFacultyofIntelligenceandInformatics,
KonanUniversity
1
BackgroundandObjectives
• Emotionsarelinkedtovariousphysicalandphysiologicalreactions.• Suchreactionsaffecttheconfigurationandphysicalpropertiesofthespeechorganswhetherornotapersonisspeaking.• Thechangesinthespeechorganscouldcontributetothevoicequalityofemotionalvoice.• WeproposeandverifyahypothesisonemotionalspeechproductionandverifybyMRImeasurements.• Sideeffecthypothesisofemotional speechproduction
2
“Emotion”inThisStudy
3
Affection
Emotion
Feeling
Primitiveemotion
Basicemotion
Hierarchicalhypothesisoffeelingbasedontheevolutionofthebrain(Fukuda,2008)
Socialfeeling
Intellectualfeeling
Pleasure/Displeasure
Joy,Anger,Fear,Disgust,Acceptance/Affection
Love,Jealousy
Affectionassociatedwithconceptssuchascultureandreligion
Acqu
ireddu
ring
evolutionaryhistory
SideEffectHypothesisofEmotionalSpeechProduction
4
Motorplanning
Phonation/Articulation
Speech
Phonemesequence
Emotion
Thehypothesissupposesthatthereisnotaspecificarticulationtargetforemotionalspeechproduction.
Words/Sentences
(Kitamura,2010)
Physicalandphysiologicalchanges
Changesinthevoicequality
EmotionalSpeech
HowtoVerifytheHypothesis
• Measurethespeechorganconfigurationwithandwithoutspeakingundercertainemotions.• Examinethesimilaritybetweenthedeformationofthespeechorganconfiguration.• Ifthedeformationhasthesametendencyfortheemotions,thearticulationcouldbeaffectedbytheemotionsdirectly.à Theresultscanbeinterpretedasindirectly
verifyingthehypothesis
5
Methods
• MRIscanner:Shimadzu-MarconiMAGNEXECLIPSE1.5PowerDrive250(ATRBAIC)• Participants:2Japaneseprofessionalactors(1male&1female)• Emotions:Neutral,Hotanger,Joy,and Sadness
6
Opticalmic.
MRIscanner
Dialogues
TheparticipantswereaskedtotakeontheroleofSpeakerBandutter/eh/.
Hotanger A: That was a complete lie.B: Eh, I can never forgive you.
Joy A: You’ve got a job offer?B: Eh, yes, I have.
Sadness A: You broke that expensive dish?B: Eh, yes, I did.
7
DataAcquisition
TheparticipantswereinstructedNOTtoprepareforthearticulationwhileexpressingtheemotionssilently.
8
Instructiontoexpressemotion
MRdataacquisition
/eh/ t/eh/
Soundrecording
nonspeaking speaking speaking
Results:NeutralandHotAnger(FemaleActor)
9Hotanger
Speaking
Speaking
Nonspeaking
Nonspeaking
Neutral
Thepharyngealandlaryngealcavitiesarenarrowed.
Thelaryngealheightishigherthanthatforneutral.
Results:JoyandSadness(FemaleActor)
10
Nonspeaking
Nonspeaking
Speaking
Speaking
Sadness
Joy Thelaryngealheightislowerthanthatforneutral.
Thepharynxandlarynxareblurred.
Thepharyngealandlaryngealcavitiesdilate.
Results:NeutralandHotAnger(MaleActor)
11Hotanger
Neutral
Speaking
Speaking
Nonspeaking
Nonspeaking
Thefrontofthetonguedorsumupandforward.
Thelaryngealheightislowerthanthatforneutral.
Results:JoyandSadness(MaleActor)
12
Speaking
Speaking
Nonspeaking
Nonspeaking Thepharyngealandlaryngealcavitiesdilate.
Sadness
Joy Thelowerjawispulledbackward.
Discussion
• Emotionsaffectthespeechorganconfigurationregardlessofwhetherornotthepersonisspeaking.• Sametendency ofchangesinthespeechorganconfigurationforthefouremotions.
• Limitations• Wemeasuredsimulatedemotionalstates.• Wecouldnotexcludethepossibilitythattheparticipantspreparedforthearticulationduringthenonspeakingstates.
13
Conclusions
• WeproposedthesideeffecthypothesisofemotionalspeechproductionandevaluateditbyMRIexperiments.• Theeffectsonthespeechorganconfigurationinthespeakingandnonspeakingstatesweresimilarforemotions.• Thehypothesiswassupportedindirectly.
14
AcknowledgementsThisstudywassupportedbySCOPE(071705001)oftheMinistryofInternalAffairsandCommunications,Japan,andJSPSKAKENHI(21300071,25280066,and25240026).
