Neuronal basis of natural textures coding in area V4 of the awake monkey: texture analysis P.Girard, C. Jouffrais, F. Arcizet, J. Bullier Insight2+ IST–2000-29688 3D shape and material properties for recognition
Jan 01, 2016
Neuronal basis of natural textures coding
in area V4 of the awake monkey: texture analysis
P.Girard, C. Jouffrais, F. Arcizet, J. Bullier
Insight2+IST–2000-296883D shape and material properties for recognition
Aim of the study (WP3)
Coding of material properties
In area V4 of awake macaque monkey
Performing a visual fixation task
Stimuli from the CURET database:12 textures + 12 scrambled textures
Frontal viewing direction
3 illumination directions (22.5, 45 and 67.6 deg.)
72 stimuli
Stimuli
Terrycloth Sand paper Plaster Aluminum foil
Salt crystalsRoof shinglePlaster (zoom)Lettuce leaf
Linen Concrete White bread Soleirolia plant
Experimental setup
Control of the experiment and real time analog and digital acquisition: CORTEX (courtesy of NIH)
5 independent microelectrodes (TREC)
Sorting software: MSD (Alpha-Omega)
Eye monitoring: IScan eye-tracker (120 Hz, 0.2 DVA)
Protocol
Mapping of the Receptive Field (RF)Hand-moved bars
M-sequences of black and white dots
Recording of response to the 72 stimuli (10 trials per stimulus)
Control: 36 original textures moved 1 deg apart
Database and statistics
Database:167 cells (42 with unshaped stimuli, 98 with shaped stimuli, 27 with new set of textures)
StatisticsANOVA 3-factors (Texture, Illum. Dir., Type)
Population (Rank analysis, MDS, comparison V4/IT)
V4 neuron sharply selective to textures
-0.5 0 0.5 1 1.50
1
-0.5 0 0.5 1 1.50
1
-0.5 0 0.5 1 1.50
1
-0.5 0 0.5 1 1.5
-0.5 0 0.5 1 1.5
-0.5 0 0.5 1 1.50
0.2
0.4
-0.5 0 0.5 1 1.5
-0.5 0 0.5 1 1.5
-0.5 0 0.5 1 1.50
0.2
-0.5 0 0.5 1 1.5
-0.5 0 0.5 1 1.5
-0.5 0 0.5 1 1.50
0.2
Plaster (zoom)Lettuce leaf
0.5s
100
0
Sp
ikes/
s
On Off
22.5 deg.
45 deg.
67.6 deg.
Texture
neuron selective to illumination direction
Example of a V4 cell whose discharge is systematically increased for a lighting direction of 67.6 deg.
22.5 deg.
45 deg.
67.6 deg.
Illum. dir.Error Bars show 95.0% Cl of Mean
Alu
min
um
Bre
ad
Con
cre
te
Leaf
Lin
en
Pla
nt
Pla
ster
Pla
ster
(z)
Sal
t
San
dpa
per
Shi
ngle
Terr
ycl
oth
Texture
0
20
40
60S
pik
es/s
.
V4 neuron selective to original and “moved” textures
Example of a V4 cell whose selectivity is the same for ‘original’ and ‘moved’ conditions. No response to scrambled sitmuli.
moved
original
scrambled
StimuliError Bars show 95.0% Cl of Mean
Alu
min
um
Bre
ad
Con
cre
te
Leaf
Lin
en
Pla
nt
Pla
ster
Pla
ster
(z)
Sal
t
San
dpa
per
Shi
ngle
Terr
ycl
oth
Texture
-10
0
10
20
30
Sp
ikes
/s.
Statistics
3 factors ANOVA (main effect + interaction, P<0.05) shows that:
82% of the cells are selective to textures
69% of the cells have a different response to original and random-phase textures
69% of the cells are selective to lighting direction
Multidimensional Scaling (MDS) – originals
Dim
ensi
on 2
MDS analysis performed on 68 cells. Original textures only, final configuration, 3 dimensions (Alienation:0.108, Stress: 0.099).
Correlations of neuronal responses with first,second,third and fourth order parameters
Median luminance
Rms contrast
skewness
kurtosis
Texture analysis
Is there a match between V4 cell population and a set of filters that could be used to classify the textures?
Are there other interesting parameters that characterize the textures and are coded in V4?
Texture analysis: methodology
Sets of 2D GABOR filters (several sizes, spatial frequencies and 8 orientations (0°:22.5:157.5°)
3 different types of quantification of outputs- thresholds
-energy
-Spectral histograms
Example of filter and computations (thresholds)
Size= 12 pixels, freq: 9.5 c/°, sigma 4 pixels, orientation 0
Size= 12 pixels, freq: 14 c/°, sigma 4 pixels, orientation 0
Cluster analysis based upon energy
N=56
filters: Size 12 pixels, freq: 2 to 28 c/°, sigma 3 pixels, orientations 0:22.5:157.5°
Spectral Histogram
N=29
C o m p a r i s o n o f M S D a n a l y s i s f o r f i l t e r s ( p a r a m e t e r s : s i z e = 1 2 , 2 4 , 3 6 p i x e l s , f r e q u e n c y = 4 . 7 6 c y c l e s / d e g , 8 o r i e n t a t i o n s ( N = 2 9 , c e n t r a l
s t i m u l i , t e x t u r e s * p < 0 . 0 0 1 )
a n d c e l l s .
filters (parameters: size=12, 24, 36 pixels, frequency= 4.76 cycles/deg,8 orientations and cells (N=29, central stimuli, textures* p<0.001).
MDS over different epochs after the stimulus onset filters: Size 12 pixels, freq: 2 to 28 c/°, sigma 3 pixels, orientations 0:22.5:157.5°
Snr : 1 possible dimensionN=27
filters: Size 12 pixels, freq: 2 to 28 c/°, sigma 3 pixels, orientations 0:22.5:157.5°
Conclusions
Coding of material properties in V4 and IT
Is this indeed texture classification or identification? We need expert advice to use better texture characterization (Spatial frequency…) or classification (Varma and Zisserman, Geusebroek and Smeulders)
Do neurons perform such expert classification?
Need to use a comparable behavioural task?