Laura Gwilliams | NYU Perceptual decision making is supported by a hierarchical cascade in both biological and artificial neural networks Laura Gwilliams 11th July 2018
Laura Gwilliams | NYU
Perceptual decision making is supported by a hierarchical cascade
in both biological and artificial neural networks
Laura Gwilliams 11th July 2018
Laura Gwilliams | NYU
AI can categorise, too❖ Artificial intelligence has sought to solve a similar
problem in visual processing❖ Deep neural networks (DNNs) can label images
very accurately
Laura Gwilliams | NYU
AI and neural convergence❖ Correspondence has been found in terms of the representations employed by brains and DNNs
Yamins et al., 2014
Laura Gwilliams | NYU
AI and neural convergence
❖ Not so surprising, given that aspects of DNNs are modelled on vision neuroscience
❖ There is more to characterising a system than simply knowing the representations it uses:
❖ Architecture
❖ Computation
Laura Gwilliams | NYU
Research Question
What is the computational architecture of perceptual
decision making?
Laura Gwilliams | NYU
❖ 17 healthy adults❖ 306 channel MEG
❖ VGG19❖ 19-layer CNN❖ Image Classification
Time / Layer
Parallel Analysis
Laura Gwilliams | NYU
What are the underlying computations?
P ( l
ette
r )
0.5
0.45
0.55
P ( l
ette
r )
Linear Evidence
P ( l
ette
r )
Categorical Percept
Laura Gwilliams | NYU
P ( l
ette
r )
0.5
0.45
0.55
LinearCategorical
P ( l
ette
r )
0.5
0.45
0.55
P ( l
ette
r )
0.5
0.45
0.55
******h4 h h h h h h h4 h h h h h h
What are the underlying computations?
Laura Gwilliams | NYU
What are the underlying computations?
Time (s)
MEG
H
4
VGG19
4
H
Task-specific trained CNN
4
H
linear categorical
Time (s)
MEG
H
4
VGG19
4
H
Task-specific trained CNN
4
H
h
4
Time (s)
MEG
H
4
VGG19
Task-specific trained CNN
linear
Time (s)
MEG
H
4
VGG19
Task-specific trained CNN
h
4
Laura Gwilliams | NYU
Conclusions
❖ We replicate the finding that hierarchical representations converge between biological and artificial NNs
❖ Performance-optimised models only partially predict neural responses during perceptual decision making