Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. [email protected] ChialvoLab.Northwestern.Edu.

Ghost, phantoms and illusions are

the neural basis of tonal music

Dante R. Chialvo

Northwestern University. Chicago, [email protected] ChialvoLab.Northwestern.Edu

Si el ojo puede ser consideradouna cámara fotográfica,

el oido seguro que No es un microfono

Short Summary:

If the eye can be seen as a camera,

the ear surely is NOT a microphone

Resumen corto:

We look at mathematical models to determine how the auditory periphery (the 1st neuron) could solve complex problems usually attributed to central processors.

First, we solve a problem first investigated by Pythagoras, later by Ohm and von Helmholtz: how the brain determines the pitch of a complex sound. We will make three points:

1) The output of even a single noisy neuron driven by multiple frequencies are spikes spaced at the inverse of the “ghost” frequency we call “pitch”.

2) We find an expression to predict the frequency of the “ghost” resonance for some arbitrary inputs.

3) The ghost resonance suggests the presence of phantoms and other sensory illusions which will be discussed ten years from now.

Then, we look at the neural mechanism of consonance, the basis of tonal music, finding that can be mapped almost trivially to the same pitch problem we just solved.

Last, investigating ways to falsify the theory we found new results applying to another classic unsolved problem, the octave enlargement.

The problem: How do we perceive pitch of a complex tone ?

The big big picture

The general tendency in the field is:More complex acoustic attributes => much higher cortical

process.Notice that this chart starts at the brainstem…

Algo asi como que puede hacer una neurona por si sola?

Tres cosas preliminares:

1) Miremos como luce el input2) Que sabemos acerca de como responden las

neuronas- a señales periodicas sin ruido (deterministic)

3) Que sabemos acerca de como responden las neuronas- a señales periodicas ruidosos (stochastic)

Instead we look at generic dynamical properties of neurons

Time 1 2 3 4 5 6 7 8

The “fundamental” is not so fundamental!

x(t)

x(t)

x(t)

x(t)

x(t) = (cos f1 t + cos f2 t +cos f3t+ … cos fnt) /n

fn= q f0

Frequency

q

Spectrum

The “fundamental” is not so fundamental (Regardless of phase or harmonicity)

x(t)

x(t)

x(t)

x(t)

shifted

shifted

Frequency

Time

f

fRandom phase

Random phase

Equal phase

Equal phase

q

Spectrum

1 2 3 4 5 6 7 8

x(t) = (cos f1 t + cos f2 t +cos f3t+ … cos fnt) /n

fn= q f0 + f

N:M1:11:0

2:1

n:m

n+N:m+M

3:23:1

5:2 5:3 4:34:1…

Patterns of spikes are predicted by Farey’s series*

continue

*Chialvo, Nature 330, 1987

M/N 2:1

3:2

5:3

3:1

The devil staircase and the Farey series

1:11:0

2:1

no3:1

no no no4:1

The average pattern of spikes are period adding (N:1) sequences

etc:1

<1:1><2:1>

<3:1>

Inter-spike interval

Log

# S

pik

es

How we perceive pitch of a complex tone ?

Cual es el proceso neuronal mas simple que produce spike trains conteniendo el codigo del pitch de un tono complejo?

Respuesta corta:

The non dynamical threshold-crossing model

If x(t) > U th the system emits a ‘‘spike’’

“Ghost Stochastic Resonance” = noise intensity at the largest proportion of spikes with intervals ~ 1/ f0

The neuron respond to a frequency not present in the input (“ghost”)

f2 = 3*f0

f1 = 2*f0

Ghost resonance for mistuned two-frequencies signals

x(t)= A (cos f1 t + cos f2 t ) +

f1 = qfo + f

f2 = (q+1)fo + f

Neuron response to two-frequencies

tones for increasing f plotted as a

function of f1

with

k=2 k=3 k=4k=5

k=6fp ~ 1/(k+1/2)

The neuron response to mistuned complex tones can be generalized for even or odd number of tones

For inputs composed of N sinusoidal signals of frequencies:

ffNkffkfkf 000 )1(...,)1(,

the resonance occur at frequencies:

Two freq.Two freq. Three freq.Three freq.

Data, Theory and Numerics overimposed (no fitting)

Cariani P.A. and Delgutte B., J. Neurophysiol. 76, 1698-1716, 1996

Center Frequency f2 (Hz)625

100

200

300

150

250Es

t. P

itch

(Hz)

475 775

n=2

n=3

n=4n=5

n=6

How it compares with In Vivo Experiments (Cariani)

Our theory

Short list of recent Ghost Stochastic Resonance work:

• Molecular motors (ratchets) driven by 2F ( Fabio Marchesoni and colleagues).

• Models of climate changes (Holger Braun and colleagues).

• Lasers (Martin Buldu, Jordi Garcia-Ojalvo and colleagues).

• Electric fish sensory system (Longtin and colleagues)

Ghost resonance is a general phenomenon

Missing Fundamental

Ghost resonance in vision*

*K. Fujii et al, Psychological Research (2000) 64:149-154.

Another twist:

As seen, the intervals between spikes is determined by the “shape” in time of the complex tone,

Thus, what will be perceived for amplitude modulated white noise?

Shhhh…Shhhh…Shhhh…Shhhh…Shhhh…

From Ghosts to Phantoms

The input signal frequency spectrum

Input Signal

Notice that the input signal spectrum is flat

noise

1/fo

The input spectrum is flat

The ouput spikes at preferred intervals ~ 1/f0

White noise is perceived not as noise but as a periodic signal with 1/fo frequency

Obviamente si el oido fuese un microfono, debieramospercibir ruido....

White noise is perceived not as noise but as a periodic signal with 1/fo frequency

A short list of what we have learned:

1) The output of even a single noisy neuron driven by multiple frequencies are spikes spaced at the inverse of the “pitch”.

2) A simple expression predicts the pitch for many arbitrary inputs (which perfectly agrees with both psychophysics and neural data).

3) The mechanism (“ghost stochastic resonance”) suggests the presence of phantoms and other sensory illusions.

4) The perception of ghosts and phantoms clearly show that the ear is not a microphone.

Pythagoras discovered that two similar strings under the same tension sounded together sound pleasant if the length of the strings are in the ratio of two small integers …. (1:1, 2:1, 3:2… n:m )

Application

The problem:What is the neural mechanism of consonance

The first law of nature ruled by arithmetic of integers:

Consonance

Inconsistent!

According with this picture pure tones’ consonance is left undefined

A tendency is to think on terms of frequency

More coincident harmonics imply better consonance

1:1 > 2:1 > 3:2 > 4:3 > 45:32

Unison

Octave

Galileo (and independently Mersenne) explained consonance as the regularity of the intervals “such that the eardrum is not kept at perpetual torment”

La solucion: Tonos puros que suenan consonante tienen la misma altura y la neurona descarga a intervals de 1/altura

Interval is more telling than 1/Frequency

Unison

Octave

La solucion: Tonos complejos que suenan consonante tienen la misma altura y la neurona descarga a intervals de 1/altura

Conclusion 1:Two tones (simple or complex) sound consonance when they have the same pitch

Conclusion 2: Altua y consonance son la misma cosa para una neuroan

Conclusion 3:The model objectively judges both pitch and consonance (algo asi como un pichichometro o consonometro)

Conclusion 4: Dynamical properties of the auditory periphery can do highly non-trivial processing including the extraction of the pitch of arbitrarily complex tones and the judgment of consonance.

What else?

The problem: to be consonant higher octaves need a few more %

Psycho-acoustic Data Spike Trains Data

So called “enlargement of the octave”

The solution:

Enlargement of the octave can be explained again on the basis of dynamical properties of neurons, (i.e., incomplete recovery from the previous spike) but I am out of time…

Time to summarize:

BlahBlahlogy

By looking at the dynamics of toy neuron models we work out a long unsolved problem:

The brain (can) determine(s) the pitch of a complex sound by the combination of a linear superposition plus a nonlinear noisy threshold crossing process.

We find the expression predicting the frequency of the “ghost” resonance for some arbitrary inputs.

Phantoms and other illusions can be created in similar ways and surely are hiding new surprises.

Consonance can be judged even by a single neuron in exactly the same way than pitch.

The ear is not a microphone

Papers: www.chialvo.net

Papers: www.chialvo.net

How we perceive pitch of a complex tone ?

Labeling the time of each spike as tspike, average the forcing for t=tspike-t,tspike+t.

What do you see for the case of SR?Answer: The input periodic forcing

What do you see for the case of Ghost SR?Answer: The ghost

What do you see for the case of AM white noise?Answer: Nothing.

Replicated experimentally in semiconductor lasers with optical feedback

UPC (Barcelona): Javier Martin-Buldu , Jordi Garcia Ojalvo, Carme Torrent

and UIB (Mallorca): Claudio Mirasso

Ghost Resonance in a Semiconductor Laser with Optical Feedback