Top Banner
Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation
32

Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Dec 18, 2015

Download

Documents

Opal Wade
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Scaling Laws in Cognitive Science

Christopher KelloCognitive and Information Sciences

Thanks to NSF, DARPA, and the Keck Foundation

Page 2: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Background and Disclaimer

Cognitive Mechanics…

Fractional Order Mechanics?

Page 3: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Reasons for FC in Cogsci• Intrinsic Fluctuations

• Critical Branching

• Lévy-like Foraging

• Continuous-Time Random Walks

= Disabled synapse = Unblamed synapses

= Enabled synapse = Blamed synapses

1. Choose a disabled synapse

2. If , enable with probability ρ

3. Set to

B

?

?

itiV ,

~B~B

1. Choose an enabled synapse

2. If , disable with probability ρ

3. Set to

BB

Spike triggers axonal & dendritic processes

~B

Sequ

ence

dPo

isson

Poiss

on+S

TDP

Source Reservoir

6.9 6.902 6.904 6.906 6.908 6.91

x 105

0

10

20

30

40

50

60

70

80

90

100

Unit Time Interval

Sp

ike C

ou

nt

SequencePoissonPoisson+STDP

0 1 2 3 4 5

x 104Unit Time Interval

Sp

ike

Cou

nt

Sequence

Poisson

Poisson+STDP

CB on CB off

Page 5: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Lowen & Teich (1996), JASA

TN i

TN

TNTNTA

i

ii

2

21

TN i 1

Allan Factor Analyses Show Scaling Law Clustering

TTA

Intrinsic Fluctuations In Spike Trains

Page 6: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Intrinsic Fluctuations in LFPs

Beggs & Plenz (2003), J Neuroscience

Bursts of LFP Activity inRat Somatosensory Slice Preparations

Page 7: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Mazzoni et al. (2007), PLoS One

231 SSP

Burst Sizes Follow a 3/2 Inverse Scaling Law

Intrinsic Fluctuations in LFPs

Intact Leech Ganglia Dissociated Rat Hippocampus

Page 8: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Intrinsic Fluctuations in Speech

Am

pli

tud

e

Time

“Bucket” “Bucket” “Bucket” “Bucket”

12

9

3

6

3

0

Trial Number

Power (dB)-40

-20

0

20

40

60

Fre

qu

enc

y (K

Hz)

n n+1 n+2 n+3 n n+1 n+2 n+3

Buck Buck Buck Buck Ket Ket Ket Ket

Am

pli

tud

e

Time

“Bucket” “Bucket” “Bucket” “Bucket”

12

9

3

6

3

0

Trial Number

Power (dB)-40

-20

0

20

40

60

Fre

qu

enc

y (K

Hz)

n n+1 n+2 n+3 n n+1 n+2 n+3

Buck Buck Buck Buck Ket Ket Ket Ket

Page 9: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Intrinsic Fluctuations in Speech

0.15 kHz

6.05 kHz

13.15 kHz

Bucks KetsF

requ

ency

(K

Hz)

Inte

nsity

(st

anda

rdiz

ed)

Trial Number

0.15 kHz

6.05 kHz

13.15 kHz

Bucks KetsF

requ

ency

(K

Hz)

Inte

nsity

(st

anda

rdiz

ed)

Trial Number

Page 10: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Intrinsic Fluctuations in Speech

0.0 0.5 1.0 1.5 2.0

Alpha

0

30

60

90

120

150

Fre

qu

en

cy

M = 1.06SD = 0.26-0.85

Log f

Log

S(f)

S(f) ~ 1/fα

Page 11: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Scaling Laws in Brain and Behavior

• How can we model and simulate the pervasiveness of these scaling laws?

– Clustering in spike trains

– Burst distributions in local field potentials

– Fluctuations in repeated measures of behavior

Page 12: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Critical Branching• Critical branching is a critical point between

damped and runaway spike propagation

1~prepostc SN

1sub 1c 1super

Damped Runaway

pre

post

Page 13: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Spiking Network Model

PSPj,t : Ij,t = ωj

PSPk,t+τk

?, itiV

itiV ,

tjtt

titi IeVV i,

)'(',,

PSPk,t+τk

τk

Incoming PSP

Update Membrane(and floor at zero)

Crossed Threshold?(and not in refractory)

Reset Membrane

Outgoing PSPs forenabled synapses

ωkτk

ωk

LeakyIntegrate

&Fire

Neuron

Source

Sink

Rese

rvoir

Page 14: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Critical Branching Algorithm

= Disabled synapse = Unblamed synapses

= Enabled synapse = Blamed synapses

1. Choose a disabled synapse

2. If , enable with probability ρ

3. Set to

B

?

?

itiV ,

~B~B

1. Choose an enabled synapse

2. If , disable with probability ρ

3. Set to

BB

Spike triggers axonal & dendritic processes

~B

Page 15: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Critical Branching Tuning

0 1000 2000 3000 4000 5000 6000

Unit Time Interval X 10

Mea

n Lo

cal B

ranc

hing

Ra

tio

SequencePoissonPoisson+STDP

Tuning ON Tuning OFF

Page 16: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Spike TrainsSe

quen

ced

Poiss

onPo

isson

+STD

P

Source Reservoir

Page 17: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Allan Factor Results

100

101

102

103

100

101

102

Counting Time (T)

Alla

n F

acto

r A

(T)

Sequence

Poisson

Poisson+STDP

data5

data6

CB off

TN i

TN

TNTNTA

i

ii

2

21

TN i 1

TTA

Se

qu

en

ce

dP

ois

so

nP

ois

so

n+

ST

DP

Source Reservoir

Page 18: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Neuronal Bursts

6.9 6.902 6.904 6.906 6.908 6.91

x 105

0

10

20

30

40

50

60

70

80

90

100

Unit Time Interval

Spi

ke C

ount

SequencePoissonPoisson+STDP

Page 19: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Neuronal Avalanche Results

100

101

102

103

104

105

10-8

10-6

10-4

10-2

100

Size

P(S

ize)

SequencePoissonPoisson+STDPdata4data5

Page 20: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Simple Response Series

Predictable Cues Unpredictable Cues

Spik

e Co

unt

Sour

ceRe

serv

oir

Time

Page 21: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

1/f Noise in Simple Responses

Response Times Response Durations

10-4

10-3

10-2

10-1

100

10-1

100

101

102

Frequency

Po

we

r

Evenly Timed CuesRandomly Timed Cues

10-4

10-3

10-2

10-1

100

10-1

100

101

102

Frequency

Po

we

r

Evenly Timed CuesRandomly Timed Cues

Page 22: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Memory Capacity of Spike Dynamics

0 5 10 15 20 25 300.5

0.6

0.7

0.8

0.9

1

Time Lag

% C

orre

ct

BR ~ 1BR < 1 (~0.8)BR > 1 (~1.1)Random

0.7 0.8 0.9 1 1.10.68

0.69

0.7

0.71

0.72

0.73

0.74

Branching Ratio Bias

Mea

n %

Cor

rect

*Random

Page 23: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Critical Branching and FC

• The critical branching algorithm produces pervasive scaling laws in its activity.

FC might serve to:

– Analyze and better understand the algorithm

– Formalize the capacity for spike computation

– Refine and optimize the algorithm

Page 24: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Lévy-like Foraging𝑃 (𝑙 ) 𝑙−𝜇

1<𝜇<3

Animal Foraging

𝑃 (𝑡𝑖 ) (𝑡𝑖+1 )−𝜇

𝜇 2

Memory Foraging

𝑃 (𝑡𝑖 ) (𝑡𝑖+1 )−𝜇

𝜇 2

Page 25: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Lévy-like Visual Search

Page 26: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Lévy-like Visual Search

100

101

102

103

104

105

100

101

102

103

104

105

106

Tile Size

Alla

n F

act

or V

aria

nce

Natural

Artificial

Natural

Artificial

Image

Eye

100

101

102

103

10-6

10-5

10-4

10-3

10-2

10-1

100

Saccade Length

P(S

acca

de L

eng

th)

Natural

Artificial

Page 27: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Lévy-like Foraging Games

.05 .15 .25 .50

-2.2

-2.1

-2

-1.9

-1.8

-1.7Number of Resources Averaged

Resource Clustering

Slo

pe

Top 20 ScoresMiddle 20 ScoresBottom 20 Scores

25 50 100 150

-2.2

-2.1

-2

-1.9

-1.8

-1.7Degree of Clustering Averaged

Resource Quantity

Top 20 ScoresMiddle 20 ScoresBottom 20 Scores

Page 28: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

“Optimizing” Search with Levy Walks• Lévy walks with μ ~ 2 are maximally efficient

under certain assumptions

• How can these results be generalized and applied to more challenging search problems?

Page 29: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Continuous-Time Random WalksIn general, the CTRW probability density obeys

Mean waiting time:

Jump length variance:

Page 30: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Human-Robot Search Teams

• Wait times correspond to times for vertical movements

• Tradeoff between sensor accuracy and scope

• Human-controlled and algorithm-controlled search agents in virtual environments

Page 31: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Conclusions

• Neural and behavioral activities generally exhibit scaling laws

• Fractional calculus is a mathematics suited to scaling law phenomena

• Therefore, cognitive mechanics may be usefully formalized as fractional order mechanics

Page 32: Scaling Laws in Cognitive Science Christopher Kello Cognitive and Information Sciences Thanks to NSF, DARPA, and the Keck Foundation.

Collaborators

• Gregory Anderson• Brandon Beltz• Bryan Kerster• Jeff Rodny• Janelle Szary

• Marty Mayberry• Theo Rhodes

• John Beggs• Stefano Carpin• YangQuan Chen• Jay Holden• Guy Van Orden