Correlating Consonant Confusability and Neural Responses: An MEG Study Valentine Hacquard 1 Mary Ann Walter 1,2 1 Department of Linguistics and Philosophy,

Correlating Consonant Confusability and Neural Responses:

An MEG Study

Valentine Hacquard1

Mary Ann Walter1,2

1Department of Linguistics and Philosophy, KIT-MIT MEG Laboratory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA, 2National

Science Foundation Fellowship

Nasal Confusability

• Numerous behavioral studies have investigated error rates in identification of phonemes masked with noise.

• These show that nasal consonants are more

confusable with each other than oral consonants; e.g. m/n vs b/d (Miller & Nicely 1955, Wang & Bilger 1973).

Consequences

• Such perceptibility asymmetries motivate much current work in phonology and (it’s claimed) phonological processes (Hume 1998, Steriade 1999).

• Unsurprisingly, given this stance, Mohanan (1993) finds that nasals are particularly susceptible to place assimilation.

An Acoustic Model

These confusability and assimilation facts, as well as studies on offline similarity judgments (Hura et al. 1992), support a model of similarity based on acoustic, perceptual factors.

context-dependent, but inventory-independent

Although nasality itself is highly salient, persevering nasality masks the F2 transition into the following vowel, which is the primary cue for place of articulation.

Another Proposal

Frisch et al. (1997) and Frisch (1996) propose a metric in which similarity is computed according to natural classes:

Similarity = shared natural classes shared + unshared natural classes

context-independent, but inventory-dependent

English: b/d (= .29) > m/n (= .28), where 1 is identity.So, nasals equally or less similar to each other than orals.

The Question

• Does a difference in similarity correlate with a difference in an auditory brain response?

• If so, brain data can be used to substantiate proposals for similarity metrics, as well as their internal organization.

MAGNETOENCEPHALOGRAPHY (MEG)

MEG measures the magnetic fields (B) generated by electrical activity in the brain: specifically, by potentials in the apical dendrites of pyramidal cells in cortex

To input Coil of SQUID

cortex

Scalp

10 -4

10 -5

10 -6

10 -7

10 -8

10 -9

10 -10

10 -11

10 -12

10 -13

10 -14

10 -15

Earth field

Urban noise

Contamination at lung

Heart （ QRS ）

MuscleFetal heart

Spontaneous signal (-wave)

Signal from retina

Intrinsic noise of SQUID

Inte

nsity

of

mag

netic

si

gnal

(T)

Evoked signal

SCALE

The magnetic field of the brain, recorded with MEG, is 100 billion times weaker than the earth’s magnetic field!

MMF: mismatch field: automatic, auditory brain response evoked by a deviant stimulus following a sequence of standards, peaking ~180-250 ms post-stimulus onset.

MMF

M100

M100: automatic auditory evoked response that peaks

~100 ms post-stimulus onset

Origin of signal in auditory cortex

MMF LOCALIZATION

Note left-hemisphere concentration of the mismatch field with linguistic stimuli.

Properties of the MMFSharma & Dorman (1999) and Phillips et al. (2000) show that the same VOT span crossing a phonemic category boundary evokes a far greater MMF than one that doesn’t.

Näätänen et al. (1997) show that a small acoustic difference crossing a phonemic category boundary evokes a far greater MMF than a large one that doesn’t.

Phonological difference outweighs acoustic difference.

Do similarity distinctions matter when category is kept constant?

10 ms VOT spanwithin category

10 ms VOT spanacross category

MMF

A M P L I T U D E

50 HZ F2 spanwithin category

10 HZ F2 spanacross category

Procedure

Oddball paradigm: Conditions (8x30=240)

1) ba (400 ms) ba (400 ms) ba (400 ms) ba (400 ms) da deviant

2) da (400 ms) da (400 ms) da (400 ms) da (400 ms) da standard

3) da (400 ms) da (400 ms) da (400 ms) da (400 ms) ba deviant

4) ba (400 ms) ba (400 ms) ba (400 ms) ba (400 ms) ba standard

5) ma (400 ms) ma (400 ms) ma (400 ms) ma (400 ms) na deviant

6) na (400 ms) na (400 ms) na (400 ms) na (400 ms) na standard

7) na (400 ms) na (400 ms) na (400 ms) na (400 ms) ma deviant

8) ma (400 ms) ma (400 ms) ma (400 ms) ma (400 ms) ma standard

Subjects (n=16) made same-different button-press judgments.

Synthesized stimuli (4 tokens) were presented in six blocks of 40 trials, randomly ordered, with self-regulated breaks in between.

Predictions

• According to an acoustic-based similarity framework, the MMF-baseline gap should be larger for oral consonant pairs than for nasals.

• According to a natural-class-based one such as Frisch’s, it should be the opposite, or equivalent.

• If abstract phonological features are the only relevant factor in perceptibility at this stage, the gaps should be equivalent.

Behavioral Results – Error Rate

Error Rates

0

1

2

3

4

nasal oral

deviant

s tandard

• Deviants overall received significantly more errors than standards (p=.0009).

• No effect for manner was observed (p=.3538).

Cf. lack of masking noise or filtering, comparatively small number of trials.

Behavioral Results – Reaction Time Reaction Time

750

800

850

900

950

nasal oral

deviant

standard

Reaction times: Nasal vs. Orals

830840850860870880890

nasal oral

• RT to deviants overall was significantly faster than to standards (p<0001).

(Some subjects reported a waiting strategy in standard trials.)

• RT to nasals was significantly faster than to orals (p=.0197).

MMF Results

OralStandard

OralDeviant

single subject

Orals: Deviants vs Standards

00.

010.

02

-50 0 50 100 150 200 250 300

oral dvts

oralstds

Nasals: Dev iants vs Standards

00.0

10.0

2

-50 0 50 100 150 200 250 300

nasaldvtsnasalstds

MMF AMPLITUDE

500

600

700

800

900

1000

1100

nasal oral

standard

deviant

Deviants have significantly greater amplitude in MMF window than standards (p<.0001).

Orals have significantly greater amplitude in MMF window than nasals (p<.0001).

But nasals have significantly greater M100 amplitude than orals (p<.0001).

MMF Results

selected sensors RMS waves

NasalDeviant

OralDeviant

MMF Comparison: Nasal/Oralsingle subject

MMF Comparison-0

.01

00

.01

0.0

2

-50 0 50 100 150 200 250 300

nasdv-s t

oradv-s t

The MMF/baseline gap is significantly greater for oral consonant pairs than for nasals (p=.0399).

single subject

Conclusions• Oral mismatches elicit a stronger MMF than nasal

mismatches.• So oral consonants are perceived as more different

from each other than nasal ones at this stage.• Phonological categories are not the only relevant

factors in perception at this stage: acoustic similarity also plays a role.

• Finally, it is an acoustic-based similarity metric that appears to be operating at this time period, rather than a natural-class one, or feature-counting.

For Future Research

• Our results suggest that the MMF can be used to test proposals about degree of perceptual distance in acoustic-based similarity frameworks.

• Our next study will isolate Frisch-style similarity as a variable, testing the same phonological contrast with speakers of languages that have the contrast, but whose inventories differ in other ways.

THANK YOU !

Alec Marantz Diana Sonnenreich

Donca Steriade Karen Froud

Linnaea Stockall Pranav Anand

Ben Bruening, Elissa Flagg, Vivian Lin

you, the audience