Human speech sound production - Pennsylvania … speech sound production ... Larynx Anatomy View from above. flow ... Biomechanics of articulators (e.g., tongue, vocal folds, jaw)

Human speech sound production

Michael KranePenn State University

28‐Apr‐2017 FLINOVIA II 1

Acknowledge support from NIH 2R01 DC005642‐11

Vocal folds in action

Larynx Anatomy

View from above

flow

Vocal fold

“Body” layer stiff

“Cover” layer soft

Vocal fold structure

1 11 1 1 1

1

*

*

11 *

1 1

21 *

* * *

sgn( )'( , ) [ ] [ ]

2

sgn( )[ ]

2

1[ ]

jC o

C o j

t L R CC

t o

j

y

CcC C t

owall

tV

x yx y x ywhere t t t t t t

c c c

x ydVp x t F F

dt S

x yF

S

u dVc t

Speech aeroacoustic sources

yc1

yj1

yo1

x1

Source/Filter Decomposition of voiced sounds

SPL

f (Hz) Acoustic Transfer function

f (kHz)

Measured

sound power SPL

1.0 f (kHz) 3.0

Source spectrum

Source f0:Adult males ~ 120 Hz Adult females ~ 240 Hz Children ~ 200-400 Hz

Vocal tract filter in action

“It’s 10 below outside”

Categories of Speech Sounds

• Vowel – FIV of vocal folds (Voice) + vocal tract filter

• Consonant: Examples: CategorySustained jet  f         s   fricative

(voiced) (v)  (z) voiced fricative

Transient constriction/jet          t      k     p  stop(voiced) (d)   (g)  (b) voiced stop

Motivation

Medicine

How does physiology correlateto speech sounds?

• Surgical intervention• Speech therapy• Diagnosis

Speech technology

What is the best, most concise description of the speech signal?

• Speech synthesis -- “naturalness”

• Speech recognition• Speech coding

Questions addressable by Mechanics:1. Aeroacoustics of speech sound production2. Flow-induced vibration of vocal folds, palate, tongue3. Biomechanics of articulators (e.g., tongue, vocal folds, jaw)4. Control

(Who pays?)

Glottal jet aerodynamics

10

NIH R01 DC002654 – 2002‐ present

• Study phonatory aerodynamics in terms of their effect on:‐ laryngeal impedance  (2002‐2010)‐ aeroacoustic source mechanisms (2010‐2015)‐ energy utilization, efficiency (2015‐2020)

• Combination of physical model experiments, computer simulation, theoretical development

• Characterize efficacy of clinical measures of voice function which make reference to these  

Glottal jet aerodynamics

11

Tim Wei Mike Barry Ben Cohen

Rutgers University

GLOTTAL JET AERODYNAMICS

Scaled‐upPhysical model experiments

Glottal jet aerodynamics

12

Tim Wei Erica Sherman

Lori Lambert

Rensselaer Polytechnic/Univ. of Nebraska

GLOTTAL JET AERODYNAMICS

Life‐scalePhysical model experiments

Aeroelastic‐aeroacoustic simulation

Scaled‐upPhysical model experiments

Rensselaer Polytechnic

Lucy Zhang Xingshi Wang Jubiao Yang

Penn State UniversityDan Leonard Liz Campo Mike McPhail

Glottal jet aerodynamics

13

GLOTTAL JET AERODYNAMICS

Life‐scalePhysical model experiments

Aeroelastic‐aeroacoustic simulation

Patient data analysis

Scaled‐upPhysical model experiments

Bob Hillman Daryush Mehta

Mass. Gen. Hospital

Rensselaer PolytechnicLucy Zhang Jubiao Yang Feimi Yu

Penn State UniversityMike McPhailGage Walters

Tim Wei Dylan Rogers

Hunter Ringenberg

Univ. of Nebraska

Tim Wei Mike Barry

Rutgers University

Time-resolved glottal jet measurements

Water channel wall

Model vocal tract

Channel flow

Model vocal folds

Time-resolved glottal jet measurements

Laser light sheet

L= 12.7cm

28cm 56cm

10x scale‐up,  H2O working fluidMatch: Reh = hU/

f* = L / (2 U To)h = min glottal width           L = glottis length                 U = max glottal flow speed  To = time glottis open          = kinematic viscosity  

Re = 8000 f* = 0.040 f* = 0.035 f* = 0.018 f* = 0.010

f0 = 126Hz f0 = 109Hz f0 = 58Hz f0 = 30Hzinlet

Min. glottal width exit

contraction Jet region

flow

10 realizations of each f*

Time-resolved glottal jet measurements

28‐Apr‐2017 FLINOVIA II 17

f* = 0.035

Re = 8000

life scale f0 = 108Hz

(Krane, et al., 2007)

Time-resolved glottal jet measurements

High-frequency modulation due to jet vortex motion

Timing scales on Re

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

‐0.2 0 0.2 0.4 0.6 0.8 1

‐0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

‐0.2 0 0.2 0.4 0.6 0.8 1 1.2

t / To

Filtered

Single

u jet/ u

stea

dy

f* = 0.040

f* = 0.010

Ta / To

Ta / To Ta / To

Ta / To

Time-resolved glottal jet measurements

Exit velocity

Importance (?) of jet inertia

Contraction region

flow

Jet region

Whole glottis

Vocal fold

0

x

x

hppuudxt

txu

21

21

22 )(

2),(2

1

Estimate unsteady, convective accelerations from velocity measurements, computations

inlet

Min. glottal width exit

contraction Jet region

f* = 0.035 f0 = 109Hz life scale

Acceleration estimates

Whole Glottis

inlet

Min. glottal width exit

contraction Jet region

Acceleration estimates

28‐Apr‐2017 FLINOVIA II 22

Phonation aeroacoustic source

Penn State University

Dan Leonard Liz Campo Mike McPhail Gage Walters

28‐Apr‐2017 FLINOVIA II 23

Life‐scale physical model

Phonation aeroacoustic source

28‐Apr‐2017 FLINOVIA II 24

Life‐scale physical model

Phonation aeroacoustic source

Body layerstiff

Molded, 2‐layer vocal fold models

Cover layersoft

Installation in airway model

28‐Apr‐2017 FLINOVIA II 25

Life‐scale physical model

Phonation aeroacoustic source

28‐Apr‐2017 FLINOVIA II 26

0.01

0.1

1

10

100

0 500 1000 1500 2000

|pR

AD(f)

| (Pa

)

frequency (Hz)

0.01

0.1

1

10

100

0 500 1000 1500 2000

|p(f)

| (Pa

)

frequency (Hz)

DipoleInverse Filtered

0.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

90 100 110 120 130

σ(P D

) / σ

(PR

AD)

Fundamental Frequency (Hz)

Radiated sound spectrum Estimates of source spectra

Radiated sound intensitysource “intensity”

Phonation aeroacoustic source

28‐Apr‐2017 FLINOVIA II 27

Aeroelastic‐aeroacoustic simulation

Lucy Zhang Xingshi Wang Jubiao Yang Feimi Yu

Rensselaer Polytechnic

28

Aeroelastic‐aeroacoustic simulation

Phonation in an “infinite” duct

Flowbody force

PML PML

body forcePML PML

FlowEnforced Symmetry

No Enforced Symmetry

29

Phonation in an “infinite” duct

Aeroelastic‐aeroacoustic simulation

Pressure drive (input)

Acoustic output to vocal tract 

(output)

(output)

(storage)

(loss)(loss) (storage)

Acoustic loss to trachea (loss)

Aeroelastic‐aeroacoustic simulation

Phonation energy budget

Summary

Current Focus: Phonation Energetics, Efficiency • Characterize energy flow and relation to known geometric/mechanical “disorders”

• Evaluate clinical measures

• Develop new measure(s) for voice efficiency

• Apply to patient data