The Role of the Syllable in Speech Production Stefanie Shattuck-Hufnagel Speech Group RLE, MIT.

The Role of the Syllable in Speech Production

Stefanie Shattuck-Hufnagel

Speech Group

RLE, MIT

The Role of the Syllable in Speech Production in

American English Stefanie Shattuck-Hufnagel

Speech Group

RLE, MIT

Outline

• The syllable seems obvious as a unit– But, caveats

• Role of the syllable in production processing– Units of serial ordering– Phonological planning framework– Units of stored motor programs

• New ideas and methods

The syllable: obvious

• We know where they are and we can count ‘em

The syllable: obvious

• We know where they are and we can count ‘em– Tap to the rhythm

The syllable: obvious

• We know where they are and we can count ‘em– Tap to the rhythm– Imitate in reiterant speech

• Replace each syllable in the target utterance with /ma/

• Take a potato to Susie and Sasha

The syllable: obvious

• We know where they are and we can count ‘em– Tap to the rhythm– Imitate in reiterant speech

• Replace each syllable in the target utterance with /ma/

• Take a potato to Susie and Sasha• One male lion ran more than a mile

The syllable: obvious

• We know where they are and we can count ‘em– Tap to the rhythm– Imitate in reiterant speech– Caveat: sometimes the number is uncertain

The syllable: obvious

• We know where they are and we can count ‘em– Tap to the rhythm– Imitate in reiterant speech– Caveat: sometimes the number is uncertain

• We can manipulate ‘em– Language games

Language Manipulation Games in English

• Onset movement– Pig Latin

• An-kay oo-yay alk-tay ike-lay iss-they?

• Rhyme replacement– Ubbie dubbie

• Cub-an yub-ou tub-alk lub-ike thub-is?

– Op• Cop-an yop-ou top-alk lop-ike thop-is?

• Whole-syllable manipulation?– Uncommon in American English

The syllable: obvious

• We know where they are and we can count ‘em– Tap to the rhythm– Imitate in reiterant speech– Caveats: sometimes the number is uncertain sometimes boundaries are uncertain

• We can manipulate ‘em– Language games– Experimental manipulation tasks

The syllable: obvious

• We know where they are and we can count ‘em– Tap to the rhythm– Imitate in reiterant speech– Caveats: sometimes the number is uncertain sometimes boundaries are uncertain

• We can manipulate ‘em– Language games– Experimental manipulation tasks– Caveat: American English evidence is strongest for syllabic

subconstituents, not whole syllables

The syllable: obvious

• We know where they are and we can count ‘em– Tap to the rhythm– Imitate in reiterant speech– Caveats: sometimes the number is uncertain sometimes boundaries are uncertain

• We can manipulate ‘em– Language games– Experimental manipulation tasks– Caveat: American English evidence is strongest for subunits

• They are clear in the waveform display

Syllables clear in waveform

• With sonorant consonants

Sometimes less clear

• With stop bursts, fricatives

The syllable: obvious

• We know where they are and we can count ‘em– Tap to the rhythm– Imitate in reiterant speech– Caveats: sometimes the number is uncertain sometimes boundaries are uncertain

• We can manipulate ‘em– Language games– Experimental manipulation tasks– Caveat: American English evidence is strongest for subunits

• They are clear in the waveform display– Caveat: sometimes not so clear

The syllable: obvious

• We know where they are and we can count ‘em– Tap to the rhythm– Imitate in reiterant speech– Caveats: sometimes the number is uncertain sometimes boundaries are uncertain

• We can manipulate ‘em– Language games– Experimental manipulation tasks– Caveat: American English evidence is strongest for subunits

• They are clear in the waveform display– Caveat: sometimes not so clear

• They play a role in phonology

The syllable in phonology

• Phonotactic constraintse.g. No /tl/ onset

In what constituent?

• Positional allophonese.g. Non-aspirated voiceless stops

In what positions? Spin, lop, guppy

e.g. Glottalization of final /t/Final in what constituent?

The syllable in phonology

• Glottalization of word- or syllable-final voiceless stops– Particularly /t/

• fit, can’t

• Glottalization of word-medial /t/– syllable final?

• butler, subtler, Hitler, battling• cartwright

– syllable-initial?• Clinton, mountain

• Glottalization of selected word-medial /k/• technical

The syllable: obvious

• We know where they are and we can count ‘em– Tap to the rhythm– Imitate in reiterant speech– Caveats: sometimes the number is uncertain sometimes boundaries are uncertain

• We can manipulate ‘em– Language games– Experimental manipulation tasks– Caveat: American English evidence is strongest for subunits, not whole syl

• They are clear in the waveform display– Caveat: sometimes not so clear

• They play a role in phonology– Caveat: unclear how widespread; other accounts often possible

Outline

• The syllable seems obvious as a unit– But, caveats

• Role of the syllable in production processing– Units of serial ordering– Phonological planning framework– Units of stored motor programs

• New ideas and methods

Syllables in production processing

• Are they units of serial ordering?

Syllables in production processing

• Are they units of serial ordering?– Speech error evidence suggests that

phonological planning includes a serial ordering process for sublexical elements

Syllables in production processing

• Are they units of serial ordering?– Speech error evidence suggests that

phonological planning includes a serial ordering process for sublexical elements

– These elements can become misordered:• Features: tomato -> ponato• Segments: your car towed -> your tar cowed• Syllable subcomponents: borth and fack• Morphemes: intelephoning stalls

Syllables in production processing

• Are they units of serial ordering?– Speech error evidence suggests that

phonological planning includes a serial ordering process for sublexical elements

– These elements can become misordered:• Features: tomato -> ponato• Segments: your car towed -> your tar cowed• Syllable subcomponents: borth and fack• Morphemes: intelephoning stalls

– Are syllables one of those units?

Syllables in production processing

• Data source: • Large corpora of errors heard in spontaneous speech• UCLA SEC, MIT SEC, Spanish etc.

• Findings• Almost no unambiguous syllable-sized error units• Many syllable subcomponents:

– Onset: speak fast -> feak spast (But see sprit blain)– Nucleus: milk burning -> murk bilning– Coda: sit down -> sin dowt

• Does this imply syllable constituents as well?

Syllables in production processing

• MIT SEC• 10,000+ errors categorized for

– Error unit (feature, segment, string, syl, morph..)– Error type (exch, subst, addn, omis, blend)– For interaction errors: direction of influence from

source to target (anticipatory, perseveratory)

• Why categorize so extensively?– Document the extensive nature of ambiguity

Why categorizing is important

• Document the rampant ambiguity re– Error unit– Error type– Factors influencing interaction errors

• Direction of influence (source to target)• Position similarity of interacting elements• Position biases

• Unrecognized ambiguity in error patterns can lead to unwarranted assumptions in production planning models

Rampant ambiguity in speech error categorization

• He placed the highly paid players alone.He placed the highly [pled] players alone.

• Possible categorizations• Whole word substitution (paid -> played)• Anticipatory morpheme subst [ple] (from players)• Perseveratory string subst [ple-] (from placed)• Anticipatory onset subst [pl] (from players)• Persev onset subst [pl] (from placed)• Antic seg addition [l] (from players)• Antic seg addition [l] (from alone)• Persev seg addition [l] (from placed)

Why categorizing is important (1) Direction of influence

• Claim: Anticipations are more common than perseverations– Compare position preference for:

• Complete exchanges: shop talk -> top shalk• Complete anticipations: -> top talk• Incompletes: -> top---shop talk

– Results:• Exchanges 70-80% word-initial Cs• Anticipations 40-50% word-initial C’s• Incompletes: intermediate

Why categorizing is important (1) Direction of influence

• Incomplete errors include some incipient exchanges, some incipient antic substs

• Can estimate proportion of anticipations– They are actually rarer than perseverations

• Implications for models– Less support: models based on early activation of

later elements in the sentence– More support: models based on mis-selection

among similar candidate elements

Outline

• The syllable seems obvious as a unit– But, caveats

• Role of the syllable in production processing– Units of serial ordering– Phonological planning framework– Units of stored motor programs

• New ideas and methods

Syllables in production processing

• Do they form the planning framework for the sublexical serial ordering process?

Syllables in production processing

• Do they form the planning framework for the sublexical serial ordering process?

• Speech error evidence might help to answer this question

Syllables in production processing

• Do they form the planning framework for the sublexical serial ordering process?

• Speech error evidence might help to answer this question

• Do sublexical interaction errors obey a syllable position similarity constraint?

Why categorizing is important (2) Position similarity constraint

• ‘Syllable position’ similarity constraint on errors:‘Interacting error segments share syllable position’

• Onsets with onsets, nuclei with nuclei, etc.• Very few cases of onset-coda interactions• Surprisingly, these few cases are within-word: fish --> shif

Why categorizing is important (2) Position similarity constraint

• ‘Syllable position’ similarity constraint on errors:‘Interacting error segments share syllable position’

• Onsets with onsets, nuclei with nuclei, etc.• Very few cases of onset-coda interactions• Surprisingly, these few cases are within word: fish --> shif

• But is the syllable necessarily the domain?• find the park -> pind the fark• Onset of word, morpheme, syllable, foot; pre-stressed-V

Why categorizing is important (2) Position similarity constraint

• ‘Syllable position’ similarity constraint on errors:‘Interacting error segments share syllable position’

• Onsets with onsets, nuclei with nuclei, etc.• Very few cases of onset-coda interactions• Surprisingly, these few cases are within word: fish --> shif

• But is the syllable necessarily the domain?• find the park -> pind the fark• Onset of word, morpheme, syllable, foot; pre-stressed-V

• Caveat: For most interaction errors, other units would serve just as well to characterize the domain of the position similarity constraint

Why categorizing is important (2) Position similarity constraint

• Implications for models– If adopt the syllable-based view, leads to

syllable-based planning frames– But perhaps the planning frame is based

on another structure:• Word• Stress foot • Other?

Why categorizing is important (2) Position similarity constraint

• Implications for models– If adopt the syllable-based view, leads to

syllable-based planning frames– But perhaps the planning frame is based

on word structure, foot structure, other

• Elicitation experiment (Shattuck-Hufnagel 1992)

– Compare effects of word-position similarity vs. syllable-position similarity

What is the domain of the position similarity constraint?

• Most errors in natural corpora = ambiguous– Word-initial and stressed-syllable onset

• speak fast -> feak spast

– Word-medial and stressed-syllable nucleus• come back -> cam buck

– Word-final and stressed-syllable coda• blot up -> blop utt

What is the domain of the position similarity constraint?

• Most errors in natural corpora = ambiguous– Word-initial and stressed-syllable onset

• speak fast -> feak spast

– Word-medial and stressed-syllable nucleus• come back -> cam buck

– Word-final and stressed-syllable coda• blot up -> blop utt

• Very few can distinguish between word and str-syl• math review -> rath meview: Word position, not str-syl• may renew -> nay remew: Str-syl position, not word

What is the domain of the position similarity constraint?

• Elicitation stimuli• Share both word and str-syl position

peril fad foot parrot

What is the domain of the position similarity constraint?

• Elicitation stimuli• Share both word and str-syl position

peril fad foot parrot

• Share word but not str-syl position parade fad foot parole

What is the domain of the position similarity constraint?

• Elicitation stimuli• Share both word and str-syl position

peril fad foot parrot

• Share word but not str-syl position parade fad foot parole

• Share str-syl position but not word repeat fad foot repair

What is the domain of the position similarity constraint?

• Elicitation stimuli• Share both word and str-syl position

peril fad foot parrot

• Share word but not str-syl position parade fad foot parole

• Share str-syl position but not word repeat fad foot repair

• Share neither word nor str-syl position ripple fad foot rapid

What is the domain of the position similarity constraint?

• Results of elicitation experiment:– Most frequent interaction errors:

• Both shared onset positions: peril fad foot parrot

– Medium frequency of interaction errors:• One shared onset position: parade fad foot parole

repeat fad foot repair

– Negligible number of errors: • No shared onset positions: ripple fad foot rapid

What is the domain of the position similarity constraint?

• Interpretation of results– Some kind of shared position matters

• Not just presence of confusable pair in context

– Word onset and str-syl onset both plausible• Str-syl onset = pre-stressed-vowel position

– May be two separate similarity constraints• They are additive: significantly more errors if

target segments share both positions • i.e. not just shared syllable onset position

Implications of error data

• Evidence for syllabic subconstituents in production planning is reasonably strong

• Evidence for whole-syllable constituents is non-existent

• Evidence for syllable structure as a factor governing sublexical interaction errors is equivocal

Outline

• The syllable seems obvious as a unit– But, caveats

• Role of the syllable in production processing– Units of serial ordering– Phonological planning framework– Units of stored motor programs

• New sources of evidence

Stored syllable motor programs

• Proposed by Levelt, Roelofs and Meyer 1999– see also Crompton 1982, Browman and Goldstein

• Syllabification occurs during phonetic encoding– Retrieve unsyllabified lexical specifications– Form syllables for each prosodic word of a

particular utterance– Retrieve their stored motor programs

• Why within the prosodic word?– Rampant resyllabification within this constituent

Stored syllable motor programs

• Arguments for rampant resyllabification – Intuitions:

• British English: deciding = de.ci.ding

– Phonetic observations• British English: escort us = es.cor.tus• Noisy stop release characteristic of onset stops

– Candidate contexts: 1 per 6 words• FWds ~ 50% of texts; most of these FWds are V-init

monosyllables (Shattuck-Hufnagel and Veilleux 2000)

• Is this actually resyllabification in British English?• Does resyllabification occur in American English?

Stored syllable motor programs

• It’s not certain that resyllabification occurs in this context in British English: escort us

• Noisy release of voiceless stop is not in itself evidence of resyllabification

• Need to look more closely at the acoustics– Does this noisy release also occur utterance-

finally?• if so, then it’s not evidence of resyllabification

– Does it contain frication without aspiration?• if so, then it’s not produced as a typical onset stop

Typical onset release noise

• Transient, frication, aspiration

Onset noise

take potato

Stored syllable motor plans

• Argument for resyllabification across lexical word boundaries within prosodic words is weak– Acoustic evidence needed for Brit Engl– Acoustic analysis of Amer Engl does not indicate

such resyllabification (Shattuck-Hufnagel 2007)

Stored syllable motor plans

• Argument for resyllabification across lexical word boundaries within prosodic words is weak– Acoustic evidence needed for Brit Engl– Acoustic analysis of Amer Engl does not indicate

such resyllabification (Shattuck-Hufnagel 2007)

• Argument for phonetic encoding one PWd at a time is weak (Keating and Shattuck-Hufnagel 2002)

– There are phonological interactions between PWdsE.g. stress shift/early pitch accent

– Interaction errors typically occur between PWds

Stored syllable motor plans

• An alternative to LRM99’s ‘prosody last’ model with prosody built from the bottom up

• A ‘prosody first’ model: Develop the prosodic framework from the top down (Keating and Shattuck-Hufnagel 2002)

• Retrieve lexical information as needed at each level– number of words, stress pattern, segments

Stored syllable motor plans

• Prosodic shape of whole phrase or utterance is available to influence phonological/phonetic encoding

• Supporting evidence: phrase-level pitch accent pattern influences segmental error pattern (Croot and colleagues 2006)

• Suggests whole-phrase prosody is in place when phonological/phonetic coding occurs

Stored syllable motor plans

• LRM99’s arguments for phonological/phonetic encoding one PWd at a time, and building higher-level prosody on that structure, are not entirely persuasive

Stored syllable motor plans

• LRM99’s arguments for phonological/phonetic encoding one PWd at a time, and buildikng higher-level prosody on that structure, are not entirely persuasive

• However, their arguments for the retrieval stored syllable-sized motor plans are promising– See Cholin talk at this workshop, Sat 3:30

• Increasingly supported by additional results from new methods

Outline

• The syllable seems obvious as a unit– But, caveats

• Role of the syllable in production processing– Units of serial ordering– Phonological planning framework– Units of stored motor programs

• New sources of evidence

Syllables in production processing

• New sources of evidence– Syllable frequency effects (e.g. Carreiras and

Perea 2002)

– Syllable priming effects (e.g. Cholin, Schiller and Levelt 2004)

– ERP studies: locus and timing of these effects (e.g. Goslin, Grainger and Holcomb 2006)

Syllables in production processing

• What have we learned?– Apparent evidence for an active role for syllables and

syllable structure should be considered with caree.g. do syllabic subconstituents mean syllables?

e.g. do apparent syllable position effects require syllables?

– There may be alternative accountse.g. phonological context vs. syllable structure

e.g. word structure vs. syllable structure

– Evidence from a widening variety of methodse.g. production priming, syllable frequency, brain imaging

Acknowledgments

• Support from NIH, NSF • Alicia Patterson and MIT’s Undergraduate

Research Opportunity Program• My mentors: Merrill Garrett, Dennis Klatt, Ken

Stevens• My collaborators: Pat Keating, Alice Turk, Nanette

Veilleux• Victoria Fromkin, who rediscovered 1900s speech

error studies (Merringer, Freud), and showed us how to apply them to modern linguistic and psycholinguistic questions