Neural population control via deep image synthesis · Pouya Bashivan*, Kohitij Kar*, James J. DiCarlo† Particular deep artificial neural networks (ANNs) are today’s most accurate

RESEARCH ARTICLE SUMMARY◥

NEUROSCIENCE

Neural population control via deepimage synthesisPouya Bashivan*, Kohitij Kar*, James J. DiCarlo†

INTRODUCTION: The pattern of light thatstrikes the eyes is processed and re-representedvia patterns of neural activity in a “deep” seriesof six interconnected cortical brain areas calledthe ventral visual stream. Visual neuroscienceresearch has revealed that these patterns ofneural activity underlie our ability to recog-nize objects and their relationships in theworld. Recent advances have enabled neuro-scientists to build ever more precise modelsof this complex visual processing. Currently, thebest such models are particular deep artificialneural network (ANN) models in which eachbrain area has a corresponding model layerandeachbrainneuronhas a correspondingmod-el neuron. Suchmodels are quite good at pre-dicting the responses of brain neurons, but theircontribution to an understanding of primatevisual processing remains controversial.

RATIONALE: These ANNmodels have at leasttwo potential limitations. First, because theyaim to be high-fidelity computerized copies ofthe brain, the total set of computations per-formed by thesemodels is difficult for humansto comprehend in detail. In that sense, eachmodel seems like a “black box,” and it is un-clear what form of understanding has beenachieved. Second, the generalization abilityof these models has been questioned becausethey have only been tested on visual stimulithat are similar to those used to “teach” themodels. Our goal was to assess both of thesepotential limitations through nonhuman pri-mate neurophysiology experiments in a mid-level visual brain area. We sought to answertwo questions: (i) Despite these ANN models’opacity to simple “understanding,” is the knowl-edge embedded in them already useful for a

potential application (i.e., neural activity con-trol)? (ii) Do these models accurately predictbrain responses to novel images?

RESULTS: We conducted several closed-loopneurophysiology experiments: After matchingmodel neurons to each of the recorded brainneural sites, we used the model to synthesizeentirely novel “controller” images based on themodel’s implicit knowledge of how the ventralvisual streamworks.We then presented those

images to each subject totest the model’s ability tocontrol the subject’s neu-rons. In one test, we askedthe model to try to con-trol each brain neuron sostrongly as to activate it

beyond its typically observed maximal activa-tion level. We found that the model-generatedsynthetic stimuli successfully drove 68% ofneural sites beyond their naturally observedactivation levels (chance level is 1%). In aneven more stringent test, the model revealedthat it is capable of selectively controlling anentire neural subpopulation, activating a par-ticular neuron while simultaneously inactivat-ing the other recorded neurons (76% successrate; chance is 1%).Next, we used these non-natural synthetic

controller images to ask whether the model’sability to predict the brain responses wouldhold up for these highly novel images. Wefound that the model was indeed quite ac-curate, predicting 54% of the image-evokedpatterns of brain response (chance level is0%), but it is clearly not yet perfect.

CONCLUSION: Even though the nonlinearcomputations of deep ANN models of visualprocessing are difficult to accurately summa-rize in a fewwords, they nonetheless provide ashareable way to embed collective knowledgeof visual processing, and they can be refinedby new knowledge. Our results demonstratethat the currently embedded knowledge alreadyhas potential application value (neural control)and that these models can partially generalizeoutside the world in which they “grew up.”Our results also show that these models arenot yet perfect and that more accurate ANNmodels would produce even more preciseneural control. Such noninvasive neural con-trol is not only a potentially powerful tool in thehands of neuroscientists but also could lead toa new class of therapeutic applications.▪

RESEARCH

Bashivan et al., Science 364, 453 (2019) 3 May 2019 1 of 1

Department of Brain and Cognitive Sciences, McGovernInstitute for Brain Research, and Center for Brains, Minds,and Machines, Massachusetts Institute of Technology,Cambridge, MA, USA.*These authors contributed equally to this work.†Corresponding author. Email: [email protected] this article as P. Bashivan et al., Science 364,eaav9436 (2019). DOI: 10.1126/science.aav9436

Collection of images synthesized by a deep neural network model to control the activity ofneural populations in primate cortical area V4.We used a deep artificial neural network tocontrol the activity pattern of a population of neurons in cortical area V4 of macaque monkeys bysynthesizing visual stimuli that, when applied to the subject’s retinae, successfully induced theexperimenter-desired neural response patterns.

ON OUR WEBSITE◥

Read the full articleat http://dx.doi.org/10.1126/science.aav9436..................................................

on May 2, 2019

http://science.sciencem

ag.org/D

ownloaded from

RESEARCH ARTICLE◥

NEUROSCIENCE

Neural population control via deepimage synthesisPouya Bashivan*, Kohitij Kar*, James J. DiCarlo†

Particular deep artificial neural networks (ANNs) are today’s most accurate models of theprimate brain’s ventral visual stream. Using an ANN-driven image synthesis method, wefound that luminous power patterns (i.e., images) can be applied to primate retinae topredictably push the spiking activity of targeted V4 neural sites beyond naturally occurringlevels. This method, although not yet perfect, achieves unprecedented independent controlof the activity state of entire populations of V4 neural sites, even those with overlappingreceptive fields. These results show how the knowledge embedded in today’s ANN modelsmight be used to noninvasively set desired internal brain states at neuron-level resolution, andsuggest that more accurate ANN models would produce even more accurate control.

Particular deep feedforward artificial neuralnetwork models (ANNs) constitute today’smost accurate “understanding” of the initial~200 ms of processing in the primate ven-tral visual stream and the core object re-

cognition behavior it supports [see (1) for thecurrently leading models]. In particular, visu-ally evoked internal “neural” representationsof these specific ANNs are remarkably similarto the visually evoked neural representations inmid-level (area V4) and high-level (inferior tem-poral) cortical stages of the ventral stream (2, 3)—a finding that has been extended to neural re-presentations in visual area V1 (4), to patterns ofbehavioral performance in core object recogni-tion tasks (5, 6), and to both magnetoencepha-lography and functional magnetic resonanceimaging (fMRI) measurements from the humanventral visual stream (7, 8). Notably, these priorfindings of model-to-brain similarity were notcurve fits to brain data; they were predictionsevaluated using images not previously seen bythe ANN models. This has been construed asevidence that these models demonstrate somegeneralization of their ability to capture key func-tional properties of the ventral visual stream.However, at least two important potential

limitations of this claim have been raised. First,because the visual processing that is executed bythemodels is not simple to describe, and becausethe models have only been evaluated in termsof internal functional similarity to the brain, per-haps they are more like a copy of, rather than auseful “understanding” of, the ventral stream.Second, because the images to assess similaritywere sampled from the same distribution as thatused to set themodel’s internal parameters (pho-

tograph and rendered object databases), it isunclear whether these models would pass astronger test of functional similarity—specifically,whether that similarity would generalize to en-tirely novel images. Perhaps themodels’ reportedapparent functional similarity to the brain (3, 7, 9)substantially overestimates their true functionalsimilarity.We conducted a set of nonhuman primate vis-

ual neurophysiology experiments to assess thefirst potential limitation by asking whether thedetailed knowledge that the models contain isuseful for one potential application (neural activ-ity control) and to assess the second potentiallimitation by asking whether the functional sim-ilarity of the model to the brain generalizes toentirely novel images. Specifically, we used oneof the leading deep ANN ventral stream models(i.e., a specific model with a fully fixed set ofparameters) to synthesize new patterns of lumi-nous power (“controller images”) that, when ap-plied to the retinae, were intended to control theneural firing activity of particular, experimenter-chosen neural sites in cortical visual area V4 ofmacaques in two settings: (i) neural “stretch,”inwhich synthesized images stretch themaximalfiring rate of any single targeted neural site wellbeyond its naturally occurringmaximal rate, and(ii) neural population state control, in whichsynthesized images independently control everyneural site in a small recorded population (here,populations of 5 to 40 neural sites). We testedthat population control by aiming to use suchmodel-designed retinal inputs to drive the V4population into an experimenter-chosen “one-hot” state in which one neural site is pushed tobe highly active while all other nearby sites aresimultaneously “clamped” at their baseline acti-vation level. We reasoned that successful exper-imenter control would demonstrate that at leastone ANN model can be used to noninvasivelycontrol the brain—a practical test of useful, causal“understanding” (10, 11).

We used chronic implanted microelectrode ar-rays to record the responses of 107 neural multi-unit and single-unit sites from visual area V4in three awake, fixating rhesus macaques de-signated as monkeys M, N, and S (nM = 52, nN =33, nS = 22). We first determined the classicalreceptive field (cRF) of each site with brieflypresented small squares (see methods). We thentested each site using a set of 640 naturalisticimages (always presented to cover the central8° of the visual field that overlapped with theestimated cRFs of all the recorded V4 sites), aswell as a set of 370 complex-curvature stimulipreviously determined to be good drivers of V4neurons (12) (location-tuned for the cRFs of theneural sites). Using each site’s visually evokedresponses (see methods) to 90% of the natu-ralistic images (n = 576), we created a mappingfrom a single “V4” layer of a deep ANN model(13) (the Conv-3 layer, which we had establishedin prior work) to the neural responses. We selec-ted the model layer that maximally predictedthe area V4 responses to the set of naturalisticimages using linear mapping with two-fold cross-validation (this model layer selection was alsoconsistent with similarity analysis using a repre-sentational dissimilaritymatrix; seemethods andfig. S8). The predictive accuracy of this model-to-brain mapping has previously been used as ameasure of the functional fidelity of the brainmodel to the brain (1, 3). Indeed, using the V4responses to the held-out 10% of the naturalisticimages as tests, we replicated and extended thatprior work. We found that the neural predictormodels correctly predicted 89% of the explain-able (i.e., image-driven) variance in the V4 neuralresponses (median over the 107 sites, each sitecomputed as themean over twomapping/testingsplits of the data; see methods).Besides generating a model V4–to–brain V4

similarity score (89% in this case), this mappingprocedure produces a potentially powerful tool:an image-computable predictor model of thevisually evoked firing rate of each of the V4neural sites. If truly accurate, this predictor mod-el is not simply a data-fitting device and not justa similarity scoring method; instead, it mustimplicitly capture a great deal of visual knowl-edge that may be difficult to express in humanlanguage but is hypothesized (by the model) tobe used by the brain to achieve successful visualbehavior. To extract and deploy that knowledge,we used a model-driven image synthesis algo-rithm (see Fig. 1 and methods) to generate con-troller images that were customized for eachneural site (i.e., according to its predictor model)so that each image should predictably and repro-ducibly control the firing rates of V4 neurons in aparticular, experimenter-chosen way. That is, weaimed to test the hypothesis that experimenter-delivered application of a particular pattern ofluminous power on the retinae will reliably andreproducibly cause V4 neurons to move to a par-ticular, experimenter-specified activity state (andthat the removal of this pattern of luminouspower will return those V4 neurons to their back-ground firing rates).

RESEARCH

Bashivan et al., Science 364, eaav9436 (2019) 3 May 2019 1 of 11

Department of Brain and Cognitive Sciences, McGovernInstitute for Brain Research, and Center for Brains, Minds,and Machines, Massachusetts Institute of Technology,Cambridge, MA, USA.*These authors contributed equally to this work.†Corresponding author. Email: [email protected]

on May 2, 2019

http://science.sciencem

ag.org/D

ownloaded from

Although there are an extremely large numberof possible neural activity states that an experi-menter might ask a controller method to try toachieve, we restricted our experiments to the V4spiking activity 70 to 170 ms after retinal powerinput (the time frame where the ANNmodels arepresumed to bemost accurate), andwe have thusfar tested two control settings: stretch controland one-hot-population control (see below). Totest and quantify the goodness of control, we ap-plied patterns of luminous power specified by thesynthesized controller images to the retinae of theanimal subjects while we recorded the responsesof the same V4 neural sites (see methods).Each experimental manipulation of the pat-

tern of luminous power on the retinae is col-loquially referred to as “presentation of animage.”However, here we state the precisemani-pulation of applied power that is under experi-

menter control and fully randomized with otherapplied luminous power patterns (other images)to emphasize that this is logically identical tomore direct energy application (e.g., optogeneticexperiments) in that the goodness of experi-mental control is inferred from the correlationbetween power manipulation and the neural re-sponse in exactly the sameway in both cases [see(11) for review]. The only difference between thetwo approaches is the assumedmechanisms thatintervene between the experimentally controlledpower and the controlled dependent variable(here, V4 spiking rate). These are steps that theANN model aims to approximate with stackedsynaptic sums, threshold nonlinearities, and nor-malization circuits. In both the control cases pres-ented here and the optogenetics control case,these intervening steps are not fully known butare approximated by a model of some type; that

is, neither experiment is “only correlational” be-cause causality is inferred from experimenter-delivered, experimenter-randomized applicationof power to the system.Because each experiment was performed over

separate days of recording (1 day to build all thepredictor models, 1 day to test control), onlyneural sites that maintained both a high signal-to-noise ratio and a consistent rank order ofresponses to a standard set of 25 naturalisticimages across the two experimental days wereconsidered further (nM = 38, nN = 19, and nS = 19for stretch experiments; nM = 38 and nS = 19 forone-hot-population experiments; see methods).

Stretch control: Attempt to maximizethe activity of individual V4 neural sites

We first defined each V4 site’s “naturally ob-served maximal firing rate” as that which was

Bashivan et al., Science 364, eaav9436 (2019) 3 May 2019 2 of 11

Central 8º Visual Field

Monkey MMonkey NMonkey S

Natural ImageResponses

StretchedResponses

B

Cat

C

A

One-Hot Population Responses

Neuron 1

Neu

ron

2

Neuron 1

Neu

ron

2

Neu

ron

2

Natural ImageResponses

Neuron 3

D

Neuron 1 (target) Responses

Neuron 3

Maximal Neural Drive (Stretch) One-Hot-Population Control

Neuron 3 Tria

l num

ber

Nor

mal

ized

Firi

ng R

ate

(a.u

.)

Time from image onset (ms)

0 100 200 3000

0 100 200 300

18

1

18

1

image ON

Neuron 1 (target) Responses

Fig. 1. Overview of the synthesis procedure. (A) Schematic illustration ofthe two tested control scenarios. Left: The controller algorithm synthesizesnovel images that it believes will maximally drive the firing rate of a targetneural site (stretch). In this case, the controller algorithm does not attempt toregulate the activity of other measured neurons (e.g., they might alsoincrease as shown). Right: The controller algorithm synthesizes images that itbelieves will maximally drive the firing rate of a target neural site whilesuppressing the activity of other measured neural sites (one-hot population).(B) Top: Responses of a single example V4 neural site to 640 naturalisticimages (averaged over ~40 repetitions for each image) are represented byoverlapping gray lines; black line at upper left denotes the image presentationperiod. Bottom: Raster plots of highest and lowest neural responses tonaturalistic images, corresponding to the black and purple lines in thetop panel, respectively. The shaded area indicates the time window over

which the activity level of each V4 neural site is computed (i.e., one valueper image for each neural site). (C) The neural control experiments aredone in four steps: (1) Parameters of the neural network are optimized bytraining on a large set of labeled natural images [Imagenet (35)] and thenheld constant thereafter. (2) ANN “neurons” are mapped to each recordedV4 neural site. The mapping function constitutes an image-computablepredictive model of the activity of each of these V4 sites. (3) The resultingdifferentiable model is then used to synthesize “controller” images for eithersingle-site or population control. (4) The luminous power patterns specifiedby these images are then applied by the experimenter to the subject’s retinae,and the degree of control of the neural sites is measured. AIT, anteriorinferior temporal cortex; CIT, central inferior temporal cortex; PIT, posteriorinferior temporal cortex. (D) Classical receptive fields of neural sites inmonkey M (black), monkey N (red), and monkey S (blue; see methods).

RESEARCH | RESEARCH ARTICLEon M

ay 2, 2019

http://science.sciencemag.org/

Dow

nloaded from

found by testing its response to the best of the640 naturalistic test images (cross-validated overrepeated presentations; see methods). We thengenerated synthetic controller images for whichthe synthesis algorithm was instructed to drivethe firing rate of one of the neural sites as high aspossible beyond that rate, regardless of the otherV4 neural sites. For our first stretch control ex-periment, we restricted the synthesis algorithmto operate only on parts of the image that werewithin the cRF of each neural site. For eachtarget neural site (nM = 21, nN = 19, and nS = 19),we ran the synthesis algorithm from five differentrandom image initializations. For 79% of neuralsites, the synthesis algorithm successfully foundat least one image that it predicted to be at least10% above the site’s naturally observed maximalfiring rate (see methods). However, in the interestof presenting an unbiased estimate of the stretchcontrol goodness for randomly sampled V4 neu-ral sites, we included all sites in our analyses,even those (~20%) that the control algorithmpredicted that it could not stretch. Visual inspec-tion suggested that the five stretch controllerimages generated by the algorithm for each neuralsite are perceptually more similar to each otherthan to those generated for different neural sites(see Fig. 2 and fig. S1), but we did not psycho-physically quantify that similarity.An example of the results of applying the

stretch control images to the retinae of onemonkey to target one of its V4 sites is shown inFig. 2A, along with the ANN model–predicted re-sponses of this site for all tested images. A closervisual inspection of this neural site’s “best” nat-ural and complex-curvature images within thesite’s cRF (Fig. 2A, top) suggests that it might beespecially sensitive to the presence of an angledconvex curvature in the middle and a set of con-centric circles at the lower left side. This is con-sistentwith extensive systematicwork in V4usingsuch stimuli (12, 14), and it suggests that we hadsuccessfully located the cRF and tuned our stim-ulus presentation to maximize the firing rate bythe standards of such prior work. Interestingly,however, we found that all five synthetic stretchcontrol images (red) drove the neural responsesabove the response to every tested naturalisticimage (blue) and above the response to everycomplex-curvature stimulus presented withinthe cRF (purple) (Fig. 2A).To quantify the goodness of this stretch con-

trol, we measured the neural response to thebest of the five synthetic images (again, cross-validated over repeated presentations; see meth-ods) and compared it with the naturally observedmaximal firing rate (defined above). We foundthat the stretch controller images successfullydrove 68% of the V4 neural sites (40 of 59)statistically beyond its maximal naturally observedfiring rate (P < 0.01, unpaired-samples t testbetween distribution of highest firing rates fornaturalistic and synthetic images; distributiongenerated from 50 random cross-validation sam-ples; see methods). Measured as an amplitude,we found that the stretch controller images ty-pically produced a firing rate that was 39%

higher than the maximal naturalistic firing rate(median over all tested sites; Fig. 2, B and C).Because our fixed set of naturalistic images

was not optimized to maximally drive each V4neural site, we considered the possibility that ourstretch controller was simply rediscovering imagepixel arrangements that are already known fromprior systematic work to be good drivers of V4neurons (12, 14). To test this hypothesis, we tested19 of the V4 sites (nM = 11, nS = 8) by presenting,inside the cRF of each neural site, each of 370complex-curvature shapes (14)—a stimulus set thathas been previously shown to contain image fea-tures that are good at driving V4 neurons whenplaced within the cRF. Because we were also con-cerned that the fixed set of naturalistic images didnot maximize the local image contrast withineach V4 neuron’s cRF, we presented the complex-curvature shapes at a contrast that was matchedto the contrast of the synthetic stretch controllerimages (fig. S4). Interestingly, we found that foreach tested neural site, the synthetic controllerimages generated higher firing rates than themost effective complex-curvature shape (Fig. 2D).Specifically, when we used the maximal responseover all the complex-curvature shapes as the ref-erence (again, cross-validated over repeated pre-sentations), we found that the median stretchamplitude was even larger (187%) than whenthe maximal naturalistic image was used as thereference (73% for the same 19 sites). In sum, theANN-driven stretch controller haddiscoveredpixelarrangements thatwere better drivers of V4 neuralsites than prior systematic attempts to do so.To further test the possibility that the relatively

simple image transformations might also achieveneural response levels that were as high as thoseelicited by the synthetic controller images, wecarried out extensive simulations to test the pre-dicted effects of a battery of alternative imagemanipulations. First, to investigate whether theresponse might be increased simply by reducingsurround suppression effects (15), we assessedeach site’s predicted response to its best natural-istic image, spatially cropped to match the site’scRF. We also adjusted the contrast of thatcropped image to match the average contrastof the synthetic images for the site (also mea-sured within the site’s cRF). Over all tested sites,the predictedmedian stretch control gain achievedusing these newly generated imageswas 14% lowerthan the original naturalistic set (n = 59 sites; seefig. S7). To explore this further, we optimized thesize and location of the cropped region of thenatural image (see methods). The stretch con-trol gain achieved with this procedure was 0.1%lower than that obtained for the original natural-istic images. Second,we tested response-optimizedaffine transformations of the best naturalisticimages (position, scale, rotations). Third, to placesome energy from multiple features of naturalimages in the cRF, we tested contrast blends of thebest two to five images for each site (seemethods).The predicted stretch control gain of each of thesemanipulations was still far below that achievedwith the synthetic controller images. In summary,we found that the achieved stretch control ability

is nontrivial, in that even at high contrast, itcannot be achieved by complex-curvature features,simple transformation on naturalistic images,combination of good naturalistic images, or op-timization of the spatial extent of the image (seemethods and fig. S7).

One-hot-population control: Attemptto activate only one of many V4neural sites

Similar to prior single-unit visual neurophysio-logy studies (16–18), the stretch control experi-ments attempted to optimize the response of eachV4 neural site individually, without regard to therest of the neural population. But the ANNmod-el potentially enables much richer forms of pop-ulation control in which each neural site mightbe independently controlled. As a first test ofthis, we asked the synthesis algorithm to try togenerate controller images with the goal of driv-ing the response of only one “target” neural sitehighwhile simultaneously keeping the responsesof all other recorded neural sites low (i.e., a one-hot-population activity state; see methods).We attempted this one-hot-population con-

trol on neural populations in which all sites weresimultaneously recorded (experiment 1, n = 38inmonkeyM; experiment 2,n = 19 inmonkey S).Specifically, we randomly chose a subset of neu-ral sites as “target” sites (14 in monkey M, 19 inmonkey S) and we asked the synthesis algorithmto generate five one-hot-population controllerimages for each of these sites (i.e., 33 tests inwhich each test is an attempt to maximize theactivity of one site while suppressing the activityof all othermeasured sites from the samemonkey).For these control tests, we allowed the controlleralgorithm to optimize pixels over the entire 8°diameter image (which included the cRFs of allthe recorded neural sites; see Fig. 3), andwe thenapplied the one-hot-population controller imagesto the monkey retinae to assess the goodness ofcontrol. The synthesis procedure predicted asoftmax score of at least 0.5 for 77% of populationexperiments (as a reference, the maximum soft-max score is 1 and is obtained when only thetarget neural site is active and all off-target neu-ral sites are completely inactive; see Fig. 3A foran example near 0.3).Although the one-hot-population controller im-

ages did not achieve perfect one-hot-populationcontrol, we found that the controller imageswere typically able to achieve enhancements inthe activity of the target site without generatingmuch increase in off-target sites (relative to na-turalistic images; see examples in Fig. 3A). Toquantify the goodness of one-hot-populationcontrol in each of the 33 tests, we computed aone-hot-population score on the responses of theactivity profile of each population (softmax score;see methods) and referenced that score to theone-hot-population control score that could beachieved using only the naturalistic images (i.e.,without the benefit of the ANN model andsynthesis algorithm). We took the ratio of thesetwo scores as the measure of improved one-hot-population control, and we found that the

Bashivan et al., Science 364, eaav9436 (2019) 3 May 2019 3 of 11

RESEARCH | RESEARCH ARTICLEon M

ay 2, 2019

http://science.sciencemag.org/

Dow

nloaded from

controller typically achieved an improvementof 57% (median over all 33 one-hot-populationcontrol tests; Fig. 3, B and C) and that thisimproved control was statistically significantfor 76% of the one-hot-population controltests (25 of 33 tests; P < 0.01, unpaired-samplest test).We considered the possibility that the im-

proved population control was a result of thenonoverlapping cRFs that would allow neuralsites to be independently controlled simply by

restricting image contrast energy to each site’scRF. To test this possibility, we analyzed a sub-sample of the measured neural population inwhich all sites had strongly overlapping cRFs(Fig. 3D). We considered a neural population ofsize 10 inmonkeyMand of size 8 inmonkey S forthis experiment with largely overlapping cRFs(Fig. 3D). In total, we performed the experimenton 12 target neural sites in two monkeys (four inmonkey M and eight in monkey S) and foundthat the amplitude of improved control was still

40% (Fig. 3D). Thus, a large portion of the im-proved control is the result of specific spatialarrangements of luminous power within the ret-inal input region shared by multiple V4 neuralsites that the ANNmodel has implicitly capturedand predicted and that the synthesis algorithmhas successfully recovered (Fig. 4).As another test of one-hot-population control,

we conducted an additional set of experiments inwhichwe restricted the one-hot control synthesisalgorithm to operate only on image pixels within

Bashivan et al., Science 364, eaav9436 (2019) 3 May 2019 4 of 11

Fig. 2. Maximal drive of individual neural sites (stretch). (A) Resultsfor an example successful stretch control test. Normalized activity level ofthe target V4 neural sites is shown for all of the naturalistic images (bluedots), complex-curvature stimuli (purple dots), and five synthetic stretchcontroller images (red dots; see methods). Best driving images withineach category and a zoomed view of the receptive field are shown at thetop. (B) Difference in firing rate in response to naturalistic (blue) andsynthetic images (red) for each neural site in three monkeys. Controller

image synthesis was restricted within the receptive field of the targetneural site. Error bars denote range of the data. (C) Histogram of increasein the firing rate over naturalistic images for cRF-restricted syntheticimages. (D) Histogram of increase in the firing rate over complex-curvaturestimuli. Black triangle with dotted black line marks the median of thescores over all tested neural sites. Red arrow highlights the gain in firing ratein each experiment achieved by the controller images. N indicates thenumber of neural sites included in each experiment.

RESEARCH | RESEARCH ARTICLEon M

ay 2, 2019

http://science.sciencemag.org/

Dow

nloaded from

the shared cRF of all neural sites in a subpopula-tion with overlapping cRFs (Fig. 3E).We comparedthis within-cRF synthetic one-hot-population con-trol with thewithin-cRF one-hot-population controlthat could be achievedwith the complex-curvatureshapes (because the prior experiments with thesestimuli were also designed to manipulate V4 re-sponses only using pixels inside the cRF). Wefound that for the same set of neural sites, thesynthetic controller images produced a very largeone-hot-population control gain (median 112%;Fig. 3E) and the control score was significantlyhigher than the best complex-curvature stimulusfor 86% of the neural sites (12 of 14).

Does the functional fidelity of the ANNbrain model generalize to novel images?

Besides testing noninvasive causal neural control,these experiments also aimed to ask whetherANN models would pass a stronger test of func-

tional similarity to the brain than prior work hadshown (2, 3)—specifically, whether thismodel-to-brain similarity would generalize to entirely novelimages. Because the controller images weresynthesized anew from random pixel arrange-ment andwere optimized to drive the firing ratesof V4 neural sites both upward (targets) anddownward (one-hot-population off-targets), weconsidered them to be a potentially novel set ofneural-modulating images that is far removedfrom the naturalistic images. We quantified andconfirmed this notion of novelty by demonstrat-ing that synthetic images were indeed statisticallyless similar to any of the naturalistic images thanthe naturalistic images were to themselves (mea-suring distances in pixel space, recorded V4 neu-ral population space, and model-predicted V4population space; see methods and fig. S6).To determine howwell the V4 predictormodel

generalizes to these novel synthetic images, for

each neural site we compared the predicted re-sponse to every tested synthetic image with theactual neural response, using the same similaritymeasure as prior work (2, 3), but now with zeroparameters to fit. That is, a good model-to-brainsimilarity score required that the ANN predictormodel for each V4 neural site accurately predictthe response of that neural site for all of manysynthetic images that are each very differentfrom those that we used to train the ANN (pho-tographs) and also very different from the imagesused to map ANN “V4” sites to individual V4neural sites (naturalistic images).Consistent with the control results (above),

we found that the ANN model accounted for54% of the explainable variance for the set ofsynthetic images (median over 76 neural sitesin three monkeys; fig. S3). Although the modeloverestimated the neural responses to synthesizedstimuli onmany occasions and themodel-to-brain

Bashivan et al., Science 364, eaav9436 (2019) 3 May 2019 5 of 11

Fig. 3. Neural population control.We synthesized controller images thataimed to set the neural population in a one-hot state (OHP) in which onetarget neural site is active and all other recorded neural sites are suppressed.(A) Two example OHP experiments (left and right). In each case, the neuralactivity of each of the validated V4 sites (see methods) in the recordedpopulation is plotted (most have overlapping cRFs), with the targetV4 site (dark blue or red) indicated by an arrow. Note that responses arenormalized individually on a normalizer image set to make side-by-sidecomparison of the responses meaningful (see methods). Top row: Activitypattern for the best (“best” in the sense of OHP control; see methods)naturalistic image (shown at right). Bottom row: Activity pattern producedby retinal application of the ANN model–synthesized controller image(shown at right). The red dashed oval marks the extended receptive field(2 SD) of each site. Error bars denote 95% confidence interval.

(B) Distribution of control scores for best synthetic and naturalistic imagesfor all 33 OHP full-image controller experiments (nM = 14, nS = 19). Controlscores are computed using cross-validation (see methods). Error barsdenote range of the data. (C) Histogram of OHP control gain (i.e.,improvement over naturalistic images) for results in (B); (i) and (ii) indicatethe scores corresponding to example experiments shown in (A). (D) Sameexperimental data as (C) except analyzed for subpopulations selected so thatall sites have highly overlapping cRFs [see cRFs below (C) to (E); black,monkey M; blue: monkey S]. (E) OHP control gain, where gain is relative tothe best complex-curvature stimulus in the shared cRF (see text) and thecontroller algorithm is also restricted to operate only in that shared cRF(n = 14 OHP experiments). In (C) to (E), N indicates the number ofexperiments in each setting; red arrow highlights the median gain in control(black triangle) achieved in each case.

RESEARCH | RESEARCH ARTICLEon M

ay 2, 2019

http://science.sciencemag.org/

Dow

nloaded from

similarity score was somewhat lower than thatobtained for responses to naturalistic images(89%), the model still predicted a substantialportion of the variance, given that all parameterswere fixed to make these “out of naturalisticdomain” image predictions. We believe this to bethe strongest test of generalization of today’sANNmodels of the ventral stream thus far, andit again shows that the model’s internal neu-ral representation is remarkably similar to thebrain’s intermediate ventral stream representa-tion (V4), although it is still not a perfect modelof the representation. We also note that be-cause the synthetic images were generated bythe model, we cannot assess the accuracy ofpredictions for images that are entirely “out ofmodel domain.”

How do we interpret these results?

Our results show that a deep ANN-driven con-troller method can be used to push the firingrates of most V4 neural sites beyond naturallyoccurring levels and that V4 neural sites withoverlapping receptive fields can be partly—butnot yet perfectly—independently controlled. Inboth cases, we show that the goodness of this

control is unprecedented in that it is superior tothat which can be obtained without the ANN.Finally, we find that with no parameter tuningat all, the ANN model generalizes moderatelywell (54%) to predict V4 responses to syntheticimages that are strikingly different from thereal-world photographs used to tune the ANNsynaptic connectivity and map the ANN’s “V4”to each V4 neural site. We believe that theseresults are the strongest test thus far of today’sdeep ANN models of the ventral stream.Beginning with the work of Hubel and Wiesel

(19, 20), visual neuroscience research has closelyequated an understanding of how the brain rep-resents the external visual world with an under-standing of what stimuli cause each neuron torespond the most. Indeed, textbooks and impor-tant recent results tell us that V1 neurons aretuned to oriented bars (20), V2 neurons are tunedto correlated combinations of V1 neurons foundin natural images (21), V4 neurons are tuned tocomplex-curvature shapes in both two and threedimensions (17, 22) and tuned to boundary infor-mation (12, 14), and inferior temporal (IT) neuronsrespond to complex object-like patterns (18) includ-ing faces (23, 24) and bodies as special cases (25).

Whereas these efforts have been essential tobuilding both a solid foundation and intuitionsabout the role of neurons in encoding visualinformation, our results show how they can befurther refined by current and futureANNmodelsof the ventral stream. For instance, we found thatsynthesis of only a few images leads to higherneural response levels thanwas possible by search-ing in a relatively large space of natural images(n = 640) and complex-curvature stimuli (n= 370)derived from those prior intuitions. This showsthat even today’s ANNmodels—which are clearlynot yet perfect (1, 6)—already give us new abilityto find manifolds of more optimal stimuli foreach neural site at a much finer degree of gra-nularity and to discover such stimuli uncon-strained by human intuition and the limits ofhuman language (see examples in fig. S1). Thisis likely to be especially important in middle andlater stages of the visual hierarchy (e.g., in V4and IT cortex), where the response complexityand larger receptive fields of neurons make man-ual search intractable.In light of these results, what can we now say

about the two important critiques of today’sANN models raised at the outset of this study

Bashivan et al., Science 364, eaav9436 (2019) 3 May 2019 6 of 11

Fig. 4. Example ofindependent controlof each neural siteon a subset of V4 neuralsites with highlyoverlapping cRFs.Controller images weresynthesized to try toachieve a one-hotpopulation overa population of eightneural sites (in eachcontrol test, the targetneural site is shown indark red and designatedby an arrow). Despitehighly overlapping receptivefields (center), most ofthe neural sites could beindividually controlled toa reasonable degree.Controller images areshown along with theextended cRF (2 SD) ofeach site (red dashedovals). Error barsdenote 95% confidenceinterval.

RESEARCH | RESEARCH ARTICLEon M

ay 2, 2019

http://science.sciencemag.org/

Dow

nloaded from

(understanding and generality)? In our view, theresults strongly mitigate both of those critiques,but they do not eliminate them. An importanttest of understanding is the ability to use knowl-edge to gain improved control over things ofinterest in the world, as we have demonstrated;however, we acknowledge that this is not theonly possible view, and many other notions of“understanding” remain to be explored to seewhether and how these models add value. Withrespect to generality, we found that even today’sANNmodels show good generalization to demon-strably novel images, so we believe these resultsclose the door on critiques that argue that cur-rent ANN models are extremely narrow in thescope of images they can accurately cover. How-ever, we note that although 54% of the explain-able variance in the generalization test wassuccessfully predicted, this is somewhat lowerthan the 89% explainable variance that is foundfor images that are “closer” to (but not identicalto) the mapping images. This not only reconfirmsthat these brain models are not yet perfect, butalso suggests that a single metric of model sim-ilarity to each brain area is insufficient to char-acterize and distinguish among alternativemodels[e.g., (1)]. Instead, multiple similarity tests atdifferent generalization “distances” could be use-ful, as we can imagine future models that showless decline in successfully predicted variance asone moves from testing images “near” the train-ing and mapping distributions (typically photo-graphs and naturalistic images) to “far” (such asthe synthetic images like those used here) to“extremely far,” such as images that cannot evenbe synthesized under the guidance of currentmodels and thus remained untested here.From an applications standpoint, the results

presented here show how today’s ANNmodels ofthe ventral stream can already be used to achieveimproved noninvasive population control (e.g.,

Fig. 4). However, the control results are clearlynot yet perfect. For example, in the one-hot pop-ulation control setting, we were not able to fullysuppress each and every one of the responses ofthe “off-target” neural sites while keeping thetarget neural site active (see examples in Figs. 3and 4). Post hoc analysis showed that we couldpartially anticipate which off-target sites wouldbe most difficult to suppress: They were typically(and not surprisingly) the sites that had highpatterns of response similarity with the targetsite (r = 0.49, P < 10–4; correlation between res-ponse similarity with the target neural site overnaturalistic images and the off-target activitylevel in the full image one-hot population experi-ments; n = 37 off-target sites). Such results raiseinteresting scientific and applied questions ofwhether and when perfect independent controlis possible at neuron-level resolution. Are ourcurrent limitations on control due to anatomicalconnectivity that restricts the potential populationcontrol, the nonperfect accuracy of the currentANN models of the ventral stream, nonperfectmapping of the model neurons to the individualneural site in the brain, the fact that we areattempting to control multi-unit activity, inade-quacy of the controller image synthesis algorithm,or some combination of all of these and otherfactors?Consider the synthesis algorithm: Intuitively,

each particular neural site might be sensitive tomany image features, but perhaps each site isonly sensitive to a few features that the otherneural sites are not sensitive to. This intuitionis consistent with the observation that, usingthe current ANNmodel, it wasmore difficult forour synthesis algorithm to find good controllerimages in the one-hot-population setting thanin the stretch setting (the one-hot-populationoptimization typically took more than twice asmany steps to find a synthetic image that is

predicted to drive the target neural site res-ponse to the same level as in the stretch setting),and visual inspection of the images suggests thatthe one-hot-population images have fewer iden-tifiable “features” (Fig. 5 and fig. S2). As the sizeof the neural population to be controlled is in-creased, it would likely become increasingly dif-ficult to achieve fully independent control, butthis is an open experimental question.Consider the current ANN models: Our data

suggest that future improved ANN models arelikely to enable even better control. For example,better ANN V4 population predictormodels gen-erally produced better one-hot-population con-trol of that V4 population (fig. S5). One thing isclear already: Improved ANNmodels of the ven-tral visual streamhave led to control of high-levelneural population that was previously out ofreach. With continuing improvement of thefidelity of ANN models of the ventral stream(1, 26, 27), the results presented here have likelyonly scratched the surface of what is possiblewith such implemented characterizations of thebrain’s neural networks.

MethodsElectrophysiological recordingsin macaques

We sampled and recorded neural sites acrossthe macaque V4 cortex in the left, right, and lefthemisphere of three awake, behaving macaques,respectively. In each monkey, we implanted onechronic 96-electrode microelectrode array (Utaharray), immediately anterior to the lunate sulcus(LS) and posterior to the inferior occipital sulcus(IOS), with the goal of targeting the central visualrepresentation (<5° eccentricity, contralaterallower visual field). Each array sampled from~25 mm2 of dorsal V4. On each day, recordingsites that were visually driven as measured byresponse correlation (Pearson r > 0.8) across

Bashivan et al., Science 364, eaav9436 (2019) 3 May 2019 7 of 11

ExampleSite 1

One

-Hot

Pop

ulat

ion

Str

etch

1°

ExampleSite 2

ExampleSite 3

ExampleSite 4

ExampleSite 5

ExampleSite 6

Fig. 5. Example controller images synthesized in stretch and one-hot population settings for six example target neural sites. Controller imageswere synthesized from the same initial random image but optimized for each target neural site and for each control goal (stretch or one-hotpopulation; see text). Visual inspection suggests that for each target site, the one-hot-population control images contain only some aspects of the imagefeatures in the stretch images.

RESEARCH | RESEARCH ARTICLEon M

ay 2, 2019

http://science.sciencemag.org/

Dow

nloaded from

split-half trials of a fixed set of 25 out-of-setnaturalistic images shown for every recordingsession (termed the normalizer image set) weredeemed “reliable.”We do not assume that each V4 electrode was

recording only the spikes of a single neuron.Hence, we use the term “neural site” throughoutthe manuscript. But we did require that thespiking responses obtained at each V4 sitemaintained stability in its image-wise “fingerprint”between the day(s) that the mapping imageswere tested (i.e., the response data used to buildthe ANN-driven predictive model of each site;see text) and the days that the controller imagesor the complex-curvature images were tested(see below). Specifically, to be “stable,” we re-quired an image-wise Pearson correlation of atleast 0.8 in its responses to the normalizer setacross recording days.Neural sites that were reliable on the experi-

mental mapping day and the experimental testdays, and were stable across all those days, weretermed “validated.” All validated sites were in-cluded in all presented results. (To avoid anypossible selection biases, this selection of vali-dated sites was done on data that were com-pletely independent from themain experimentalresult data.) In total, we recorded from 107 vali-dated V4 sites during the ANN-mapping day,including 52, 33, and 22 sites in monkey M (lefthemisphere), monkey N (right hemisphere), andmonkey S (left hemisphere), respectively. Of thesesites, 76 were validated for the stretch controlexperiments (nM = 38, nN = 19, nS = 19) and 57were validated for the one-hot-population con-trol experiments (nM = 38, nS = 19).To allow meaningful comparisons across re-

cording days and across V4 sites, we normalizedthe raw spiking rate of each site from each re-cording session (within just that session) by sub-tracting its mean response to the 25 normalizerimages and then dividing by the standard devia-tion of its response over those normalizer images(these are the arbitrary units shown as firingrates in Figs. 2A, 3A, and 4). The normalizer imageset was always randomly interleavedwith themainexperimental stimulus set(s) run on each day.Control experiments consisted of three steps.

In the first step, we recorded neural responses toour set of naturalistic images that were used toconstruct themapping function between theANNactivations and the recorded V4 sites. In a second,offline step, we used thesemapping functions (i.e.,a predictive model of the neural sites) to syn-thesize the controller images. Finally, in step 3, weclosed the loop by recording the neural responsesto the synthesized images. The time between step1 and step 3 ranged from several days to 3 weeks.

Fixation task

All images were presented while monkeys fix-ated a white square dot (0.2°) for 300 ms to ini-tiate a trial. We then presented a sequence of fiveto seven images, each ON for 100 ms followedby a 100-ms gray blank screen. This was followedby a water reward and an intertrial interval of500 ms, followed by the next sequence. Trials

were aborted if gaze was not held within ±0.5° ofthe central fixation dot during any point. Toestimate the cRF of each neural site, we flashed1° × 1° white squares across the central 8° of themonkeys’ visual field,measured the correspond-ing neural responses, and then fitted a 2DGaussian to the data. We defined 1 SD as thecRF of each site.

Naturalistic image set

We used a large set (N = 640) of naturalisticimages tomeasure the response of each recordedV4 neural site and every model “V4” neural siteto each of these images. Each of these imagescontained a 3D-rendered object instantiated ata random view overlaid on an unrelated naturalimage background; see (28) for details.

Complex-curvature stimuli

We used a set of images consisting of closedshapes constructed by combining concave andconvex curves (12). These stimuli are constructedby parametrically defining the number and con-figuration of the convex projections that con-stituted the shapes. Previous experiments withthese shapes showed that curvature and polarangle were quite good at describing the shapetuning (12). The number of projections variedfrom 3 to 5, and the angular separation betweenprojections was in 45° increments. These shapeswere previously shown to contain good drivers ofV4 neurons of macaque monkeys (12, 14). Thecomplex-curvature images were generated usingthe code generously supplied by the authors ofthat prior work (http://depts.washington.edu/shapelab/resources/stimsonly.php). The stimuliwere presented at the center of the receptivefield of the neural sites (detailed below).

Cross-validation procedure forevaluating control scores

To evaluate the scores from the neural responsesto an image set, we divided the neural responserepetitions into two randomly selected halves.We then computed the mean firing rate of eachneural site in response to each image in eachhalf. The mean responses from the first halfwere used to find the image that produces thehighest score (in that half) and the response tothat image was then measured in the secondhalf (this is the measurement used for furtheranalyses). We repeated this procedure 50 timesfor each neural site (i.e., 50 random half splits).For stretch and one-hot-population experiments,the score functions were the “neural firing rate”and “softmax score,” respectively. We computedeach score for the synthetic controller images andfor the reference images (either the naturalistic orthe complex-curvature sets; see text). The synthetic“gain” in the control score is calculated as the dif-ference between the synthetic controller score andthe reference score, divided by the reference score.

V4 encoding model

To use the ANN model to predict each recordedneural site (or neural population), the internal“V4-like” representation of the model must first

be mapped to the specific set of recorded neuralsites. The assumptions behind this mapping arediscussed elsewhere (9), but the key idea is thatany good model of a ventral stream area mustcontain a set of artificial neurons (“features”) thattogether span the same visual encoding space asthe brain’s population of neurons in that area(i.e., themodel “layer”mustmatch the brain areaup to a linear mapping). To build this predictivemap from model to brain, we started with aspecific deep ANNmodel with locked parameters.Here we used a variant of Alexnet architecturetrained on Imagenet (13), as we have previouslyfound the feature space at the output of Conv-3layer of Alexnet to be a good predictor of V4neural responses (we refer to this asmodel “V4”).We used the same training procedure as de-scribed in (13), except that we did not split themiddle convolutional layers between graphicsprocessing units (GPUs).In addition, the input imageswere transformed

using an eccentricity-dependent function thatmimics the known spatial sampling propertiesof the primate retinae (see below). We termedthis the “retinae transformation.” We had pre-viously found that training deep convolutionalANN models with retinae-transformed imagesimproves the neural prediction accuracy of V4neural sites (an increase in explained varianceby ~5 to 10%). The retinae transformation wasimplemented by a fisheye transformation thatmimics the eccentricity-dependent samplingperformed in primate retinae (code available athttps://github.com/dicarlolab/retinawarp). Allinput images to the neural network were pre-processed by randomly cropping followed byapplying the fisheye transformation. Parametersof the fisheye transformationwere tuned tomimicthe cones density ratio between the fovea and 4°peripheral vision (29).We used the responses of the recorded V4

neural sites in eachmonkey and the responses ofall the model “V4” neurons to build a mappingfrom the model to the recorded population of V4neural sites (Fig. 1). We used a convolutionalmapping function that significantly reduces theneural prediction error compared to other meth-ods such as principal component regression. Ourimplementation was a variant of the two-stageconvolutional mapping function proposed in (30)in which we substituted the group sparsity reg-ularization term with an L2 loss term to allowfor smooth (nonsparse) featuremixing. The firststage of themapping function consists of a learn-able spatial mask (Ws) that is parameterizedseparately for each neural site (n) and is used toestimate the receptive field of each neuron. Thesecond stage consists of a mixing pointwise con-volution (Wd) that computes a weighted sum ofall feature maps at a particular layer of the ANNmodel (Conv3 layer in our case). The mixingstage finds the best combination of model fea-tures that are predictive of each neural site’sresponse. The final output is then averaged overall spatial locations to form a scalar prediction ofthe neural response. Parameters are jointly op-timized to minimize the prediction error Le on

Bashivan et al., Science 364, eaav9436 (2019) 3 May 2019 8 of 11

RESEARCH | RESEARCH ARTICLEon M

ay 2, 2019

http://science.sciencemag.org/

Dow

nloaded from

the training set regularized by combination ofL2

and smoothing Laplacian lossesLLaplace (definedbelow). By factorizing the spatial and feature di-mensions, this method significantly improves thepredictivity of neural responses over the tradi-tional principal component regression. We inter-pret this improved predictive power as resultingfrom the fact that it imposes a prior on themodel-to-brain mapping procedure that is strongly inline with an empirical fact: that each neuron inarea V4 has a receptive field. That neuron is thusbest explained by linear combinations of simu-lated neurons that have similar receptive fields.

yn ¼X�

W ðnÞs � X

�h i�W ðnÞ

d þW ðnÞb ð1Þ

L2 ¼ lsXn

W ðnÞ2s þ ld

Xn

W ðnÞ2d ð2Þ

LLaplace ¼ ls

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXn

�W ðnÞ

s ∗L�2

s; L¼

0 �1 0�1 4 �10 �1 0

24

35

ð3Þ

Le ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXn

ðyn � yÞ2r

ð4Þ

L ¼ Le þ LLaplace þ L2 ð5Þ

We evaluated our model using two-fold cross-validation and observed that ~89% of the explain-able variance could be explained with our modelin three monkeys (EVM = 92%, EVN = 92%, EVS =80%). The addition of the retinae transformationtogether with the convolutional mapping func-tion increased the explained variance by ~13%over the naive principal component regressionapplied on features from themodel trainedwith-out the retinae transformation (EVM= 75%,EVN=80%, EVS = 73%). Ablation studies on data fromeach monkey suggested that on average about3 to 8% of the improvements were due to theaddition of the retinae transformation (see tableS1). For constructing the final mapping function,adopted for image synthesis, we optimized themapping function parameters on 90% of thedata, selected randomly.The resulting predictive model of V4 (ANN

features plus linearmapping) is referred to as themapped V4 encoding model and, by construc-tion, it produces the same number of artificial V4“neurons” as the number of recorded V4 neuralsites (52, 33, and 22 neural sites in monkeys M,N, and S, respectively).

Retinae transformation

To retain the resolution of the retinae-transformedimages as high as possible, we did not subsamplethe input image with a fixed sampling pattern.Instead, our implementation of the retinae sam-pling uses a backward function r = g(r′) thatmaps the radius of points in the retinae-transformed image (r′) to those in the inputimage (r). In this way, for every pixel in theoutput image, we can find the correspondingpixel in the input image using the pixel-mapping

function g. To formulate the pixel-mapping func-tion g, we take advantage of the known rate ofchange of cones density (r) in the primate re-tinae that exponentially decreases with eccen-tricity (29):

r ¼ 1

pd2¼ expð�ar′Þ ð6Þ

where d is the distance between nearby conesand r′ is the radial distance from the fovea in thetransformed image. From this, we can write d asa function of r′:

d ¼ 1ffiffiffip

p expar′

2

� �ð7Þ

The ratio between the cones density in the foveaand the outmost periphery, given the specificvisual field size in which the stimulus has beenpresented in the experiment, can be written as

rfrp

¼ expðar′maxÞ ð8Þ

where rf and rp are the cone densities at thefovea and periphery, respectively, and r′max is thehighest radial distance in the output image (e.g.,150 for an image of size 300). From Eq. 8, we cancalculate a as a function of rf, rp, and r′max:

a ¼ lnðrf=rpÞr′max

ð9Þ

The rf/rp ratio is known given the size of thevisual field in which the stimuli were presented(e.g., 10 for fovea to 4° in this study) and theoutput image size (e.g., 300 in this study).We cannow formulate the function g(r′) as the sum of allthe distances d up to radius r′ weighted by afactor b:

gðr′Þ ¼ bffiffiffip

pXr′�1

k¼0

dk ¼ bffiffiffip

pXr′�1

k¼0

expar

2

� �

¼ bffiffiffip

p 1� exp ar′2

� �1� exp a

2

� � ð10Þ

where b is found so that rmax/g(r′max) = 1. In ourimplementation we use Brent’s method to findthe optimal b value.

Finding the best representation in theANN model

We used linear mapping from model features toneural measurements to compare the representa-tion at each stage of processing in theANNmodel.For features in each layer of the ANNmodel, weapplied principal components analysis and ex-tracted the top 640 dimensions.We then fitted alinear transformation to the data using Ridgeregression method and computed the amountof explained variance (EV) by themapping func-tion. For each neural site we normalized the EVby the internal consistency of measurementsacross repetitions using two-fold cross-validation.The median normalized EV across all measured

sites was used to select the best representation inthe ANNmodel (fig. S8A). We also quantified thesimilarity of representations at each layer of theANNmodel and the neural measurements usingthe image-level representational dissimilarity matrix(RDM) that followed the samepattern as thatwhichwas obtained from linear mapping method (fig.S8B). RDMs were computed using the principalcomponents of the features at each layer in re-sponse to the naturalistic image set (n = 640).

Synthesized “controller” images

The “response” of an artificial neuron in themapped V4 encoding model (above) is a differ-entiable function of the pixel values f :J w�h�c→ℝthat enables us to use the model to analyze thepredicted sensitivity of neurons to patterns inthe pixels space. We formulate the synthesis op-eration as an optimizationprocedure duringwhichimages are synthesized to control the neural firingpatterns in the following two settings:1) Stretch: We synthesized controller images

that attempt to push each individual V4 neuralsite into its maximal activity state. To do so, wefollowed an approach first introduced in (31) anditeratively changed the pixel values in the direc-tion of the gradient that maximizes the firing rateof the corresponding model V4 neural site. Werepeated the procedure for each neural site usingfive different random starting images, therebygenerating five stretch controller images for eachV4 neural site.2) One-hot population: Similar to the stretch

scenario, except that here we chose the optimiza-tion to change the pixel values in a way that (i)attempts to maximize firing rate of the target V4neural site, and (ii) attempts to maximally sup-press the firing rates of all other recorded V4neural sites. We formalized the one-hot popula-tion goal in the following objective function thatwe then aimed to maximize during the imagesynthesis procedure:

S ¼ softmaxtðyÞ ¼ expðytÞXi

expðyiÞð11Þ

where t is the index of the target neural site,and yi is the response of the model V4 neuron ito the synthetic image.For each optimization run, we started from an

image that consists of random pixel values drawnfrom a standard normal distribution and opti-mized the objective function for a prespecifiednumber of steps using a gradient ascent algorithm(steps = 700). We also used the total variation(defined below) as additional regularization in theoptimization loss to reduce the high-frequencynoise in the generated images:

LTV ¼Xi; j

ð‖Iiþ1; j � Ii; j‖2 þ ‖Ii; jþ1 � Ii; j‖2Þ ð12Þ

During the experiments, monkeys were requiredto fixate within a 1° circle at the center of thescreen. This introduced an uncertainty on theexact gaze location. For this reason, images were

Bashivan et al., Science 364, eaav9436 (2019) 3 May 2019 9 of 11

RESEARCH | RESEARCH ARTICLEon M

ay 2, 2019

http://science.sciencemag.org/

Dow

nloaded from

synthesized to be robust to small translations ofmaximum 0.5°. At every iteration, we translatedthe image in random directions (i.e., jittering)with a maximum translation length of 0.5° ineach direction, thereby generating images thatwere predicted to elicit similarly high scores re-gardless of the translations within the range. Thetotal variation loss and the translation invarianceprocedure reduced the amount of high-frequencynoise patterns in the generated images commonlyknown as adversarial examples (32, 33). In addi-tion, at every iteration during the synthesis pro-cedure, we normalized the computed gradientsby its global norm and clipped the pixel valuesat –1 and 1.

Contrast energy

It has been shown that neurons in area V4 res-pond more strongly to higher contrast stimuli(34). To ask whether contrast energy (CE) wasthe main factor in stretching the V4 neural firingrates, we computed the contrast energy withinthe receptive field of the neural sites for all thesynthetic, naturalistic, and classic V4 stimuli. Con-trast energy was calculated as the ratio betweenthe maximum and background luminances. Forall images, the average luminance was used as thebackground value. Because the synthetic imagesconsisted of complex visual patterns, we alsocomputed the contrast energy using an alter-native method based on spectral energy withinthe receptive field. We calculated the averagepower in the cRF in the frequency range of 1 to30 cycles per degree. We ensured that for alltested neural sites, CE values within the cRFfor synthetic stretch controller images were lessthan or equal to the classic, complex-curvatureV4 stimuli (fig. S4).

cRF-cropped contrast-matchednaturalistic stimuli

For each neural site, we first produced a newnaturalistic image set by cropping the oldernaturalistic image set at the estimated cRF ofthe respective site. We then matched the con-trast of these naturalistic images (within the cRFof that neuron) to the average contrast across allfive synthesized images (generated for the sameneural site). We then computed the predictedneural responses to all these new cRF-masked,contrast-matched naturalistic images and eval-uated the stretch control gain achieved with thisset over the original naturalistic images. Thestretch control gain using these images showed a14% decrease in the median gain over all targetneurons. This meant that the original naturalisticimage set without the cRF masking and contrastmatching contained better drivers of the neuralsites measured in our experiments. We noticedthat masking the images with the estimated cRFwas responsible for most of the drop in the ob-served stretch control gain (11%; see fig. S7). Wealso noted that the contrast energy within thecRF was higher for best naturalistic images com-pared to synthetic images for most sites (medianratio of synthetics contrast to best naturalisticimages was 0.76 over all tested sites).

Monte Carlo mask optimizationWe estimated the optimal mask parameters for-mulated as a 2DGaussian function (i.e., m, s1, s2, r)for each neural site via Monte Carlo simulations(n = 500). We sampled each parameter from thecorresponding distribution derived from themea-sured neural sites in each monkey. For eachMonte Carlo simulation, we sampled the maskparameters fromtheabove-mentioneddistributionsand constructed a 2D mask. We then maskedthe naturalistic images with the sampled mask(cropped at 1 SD) andmatched image contraststo the average contrast of synthetic images pro-duced for each neural site within the mask. Foreach neural site, we chose the optimal mask pa-rameters that elicited themaximumaverage firingrate (predicted) across all images in the natural-istic set. Themaximum predicted output for eachneural site in response to these images was usedto evaluate the stretch control gain that showed anonsignificant gain over the naturalistic images.

Affine transformations of thenaturalistic image set

There might be simple image transformationsthat could achieve the same level of control asthat obtained by the synthetic images. To testthis, we conducted an additional analysis in whichwe randomly transformed the best naturalisticimage for each neural site using various affinetransformations (i.e., translation, scale, and rota-tion; n = 100) and calculated the predicted res-ponses to those images. We considered fourexperiments with the following transformations:(i) random scaling between 0.5 and 2, (ii) randomtranslation between –25 and 25 pixels in eachdirection, (iii) random rotation between 0° and90°, and (iv) mixture of all three transformations.For each experiment, we evaluated the stretchcontrol gain over the naturalistic image setachieved with these new images that showedsignificantly lower gains for all of the alternativemethods compared to our proposed model-basedmethod (see fig. S7).

Combining best driver images

Images that are good drivers of the measuredneurons could be combined together to formnew mixed images that might drive the neuronseven higher. To test this hypothesis, we com-bined the top naturalistic images for each neu-ron by taking the average pixel value over allselect images and matched the contrast (withincRF of each neural site) of themixed image to theaverage contrast across synthetic images gener-ated for each neuron. We tried various numbersof top images to create the mixed image (i.e., top2, 3, 4, and 5). We computed the predicted stretchcontrol gain using these mixed images over thenaturalistic image set and found that theseimages were considerably weaker drivers ofthe same neurons (see fig. S7).

Quantifying the noveltyof synthetic images

We hypothesized that if the synthetic stimuli areindeed novel, they should be less similar (i.e.,

correlated) to any of the naturalistic images thanthe naturalistic images are to themselves. Wecomputed the distances between synthetic andnaturalistic images in pixel space as well as inthe space of neural responses. To test this, wemeasured theminimumEuclidean distance (inthe space of measured neural responses) betweeneach synthetic image and all naturalistic imagesand compared them with minimum distancesobtained for naturalistic images. Figure S6 showsthe distribution of minimum distances of syn-thetic and naturalistic images to any naturalisticimages and illustrates the point that the responsesto synthetic images are significantly farther fromthe distribution of responses to naturalistic imagesthan expected from sampling within the natural-istic space (fig. S6, A, C, E, and F) or by applyingsimple image transformations on images sampledfrom that space (fig. S6, B and D). Therefore, wecan quantifiably call these images “out-of-domain”[Wilcoxon rank-sum test; Z(3798) = 30.8; P <0.0001]. We also computed the distances be-tween synthetic and naturalistic images in thepixel space using the correlation distance (1 – r)that showed a similar distinction between thetwo [Wilcoxon rank-sum test; Z(37120) = 29.3;P < 0.0001].

REFERENCES AND NOTES

1. M. Schrimpf et al., Brain-Score: Which Artificial NeuralNetwork for Object Recognition is most Brain-Like? bioRxiv[preprint]. 5 September 2018. pmid: 407007

2. D. L. K. Yamins, H. Hong, C. Cadieu, J. J. DiCarlo,in Advances in Neural Information Processing Systems 26,C. J. C. Burges et al., Eds. (Neural Information ProcessingSystems Foundation, 2013); https://papers.nips.cc/paper/4991-hierarchical-modular-optimization-of-convolutional-networks-achieves-representations-similar-to-macaque-it-and-human-ventral-stream.

3. D. L. Yamins et al., Performance-optimized hierarchical modelspredict neural responses in higher visual cortex. Proc. Natl.Acad. Sci. U.S.A. 111, 8619–8624 (2014). doi: 10.1073/pnas.1403112111; pmid: 24812127

4. S. A. Cadena et al., Deep convolutional models improvepredictions of macaque V1 responses to natural images.bioRxiv [preprint]. 11 October 2017. pmid: 201764

5. R. Rajalingham, K. Schmidt, J. J. DiCarlo, Comparison of ObjectRecognition Behavior in Human and Monkey. J. Neurosci. 35,12127–12136 (2015). doi: 10.1523/JNEUROSCI.0573-15.2015;pmid: 26338324

6. R. Rajalingham et al., Large-scale, high-resolution comparisonof the core visual object recognition behavior of humans,monkeys, and state-of-the-art deep artificial neural networks.J. Neurosci. 38, 7255–7269 (2018). doi: 10.1523/JNEUROSCI.0388-18.2018; pmid: 30006365

7. S. M. Khaligh-Razavi, N. Kriegeskorte, Deep supervised, butnot unsupervised, models may explain IT corticalrepresentation. PLOS Comput. Biol. 10, e1003915 (2014).pmid: 25375136

8. R. M. Cichy, A. Khosla, D. Pantazis, A. Torralba, A. Oliva,Comparison of deep neural networks to spatio-temporalcortical dynamics of human visual object recognition revealshierarchical correspondence. Sci. Rep. 6, 27755 (2016).doi: 10.1038/srep27755; pmid: 27282108

9. D. L. Yamins, J. J. DiCarlo, Using goal-driven deeplearning models to understand sensory cortex.Nat. Neurosci. 19, 356–365 (2016). doi: 10.1038/nn.4244;pmid: 26906502

10. J. Pearl, Causality (Cambridge Univ. Press, 2009).11. M. Jazayeri, A. Afraz, Navigating the Neural Space in

Search of the Neural Code. Neuron 93, 1003–1014 (2017).doi: 10.1016/j.neuron.2017.02.019; pmid: 28279349

12. A. Pasupathy, C. E. Connor, Shape representation in area V4:Position-specific tuning for boundary conformation.J. Neurophysiol. 86, 2505–2519 (2001). doi: 10.1152/jn.2001.86.5.2505; pmid: 11698538

Bashivan et al., Science 364, eaav9436 (2019) 3 May 2019 10 of 11

RESEARCH | RESEARCH ARTICLEon M

ay 2, 2019

http://science.sciencemag.org/

Dow

nloaded from

13. A. Krizhevsky, I. Sutskever, G. E. Hinton, in Advancesin Neural Information Processing Systems 25,F. Pereira et al., Eds. (Neural Information ProcessingSystems Foundation, 2012); https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.

14. A. Pasupathy, C. E. Connor, Population coding of shape in areaV4. Nat. Neurosci. 5, 1332–1338 (2002). doi: 10.1038/972;pmid: 12426571

15. J. R. Cavanaugh, W. Bair, J. A. Movshon, Nature andinteraction of signals from the receptive field center andsurround in macaque V1 neurons. J. Neurophysiol. 88,2530–2546 (2002). doi: 10.1152/jn.00692.2001;pmid: 12424292

16. E. T. Carlson, R. J. Rasquinha, K. Zhang, C. E. Connor, A sparseobject coding scheme in area V4. Curr. Biol. 21, 288–293(2011). doi: 10.1016/j.cub.2011.01.013; pmid: 21315595

17. D. A. Hinkle, C. E. Connor, Three-dimensional orientation tuningin macaque area V4. Nat. Neurosci. 5, 665–670 (2002).doi: 10.1038/nn875; pmid: 12068303

18. E. Kobatake, K. Tanaka, Neuronal selectivities to complexobject features in the ventral visual pathway of the macaquecerebral cortex. J. Neurophysiol. 71, 856–867 (1994).doi: 10.1152/jn.1994.71.3.856; pmid: 8201425

19. D. H. Hubel, T. N. Wiesel, Receptive fields, binocular interactionand functional architecture in the cat’s visual cortex. J. Physiol.160, 106–154 (1962). doi: 10.1113/jphysiol.1962.sp006837;pmid: 14449617

20. D. H. Hubel, T. N. Wiesel, Receptive fields and functionalarchitecture of monkey striate cortex. J. Physiol.195, 215–243 (1968). doi: 10.1113/jphysiol.1968.sp008455;pmid: 4966457

21. J. Freeman, C. M. Ziemba, D. J. Heeger, E. P. Simoncelli,J. A. Movshon, A functional and perceptual signatureof the second visual area in primates. Nat. Neurosci.16, 974–981 (2013). doi: 10.1038/nn.3402;pmid: 23685719

22. A. Pasupathy, C. E. Connor, Responses to contour features inmacaque area V4. J. Neurophysiol. 82, 2490–2502 (1999).doi: 10.1152/jn.1999.82.5.2490; pmid: 10561421

23. R. Desimone, T. D. Albright, C. G. Gross, C. Bruce, Stimulus-selective properties of inferior temporal neurons in themacaque. J. Neurosci. 4, 2051–2062 (1984). doi: 10.1523/JNEUROSCI.04-08-02051.1984; pmid: 6470767

24. D. Y. Tsao, W. A. Freiwald, R. B. Tootell, M. S. Livingstone,A cortical region consisting entirely of face-selective cells.Science 311, 670–674 (2006). doi: 10.1126/science.1119983;pmid: 16456083

25. I. D. Popivanov, J. Jastorff, W. Vanduffel, R. Vogels,Heterogeneous single-unit selectivity in an fMRI-defined body-selective patch. J. Neurosci. 34, 95–111 (2014). doi: 10.1523/JNEUROSCI.2748-13.2014; pmid: 24381271

26. J. Kubilius et al., CORnet: Modeling the Neural Mechanisms ofCore Object Recognition. bioRxiv [preprint]. 4 September 2018.pmid: 408385

27. K. Kar, J. Kubilius, K. M. Schmidt, E. B. Issa, J. J. DiCarlo,Evidence that recurrent circuits are critical to the ventralstream’s execution of core object recognition behavior. Nat.Neurosci. 10.1038/s41593-019-0392-5 (2019). doi: 10.1038/s41593-019-0392-5.

28. N. J. Majaj, H. Hong, E. A. Solomon, J. J. DiCarlo, SimpleLearned Weighted Sums of Inferior Temporal NeuronalFiring Rates Accurately Predict Human Core ObjectRecognition Performance. J. Neurosci. 35, 13402–13418(2015).doi: 10.1523/JNEUROSCI.5181-14.2015; pmid: 26424887

29. A. B. Watson, A formula for human retinal ganglion cellreceptive field density as a function of visual field location.J. Vis. 14, 1 (2014). pmid: 24982468

30. D. Klindt, A. S. Ecker, T. Euler, M. Bethge, in Advances inNeural Information Processing Systems 31, I. Guyon et al.,Eds. (Neural Information Processing Systems Foundation,2017); https://papers.nips.cc/paper/6942-neural-system-identification-for-large-populations-separating-what-and-where.pdf.

31. D. Erhan, Y. Bengio, A. Courville, P. Vincent, Visualizinghigher-layer features of a deep network (Departementd’Informatique et Recherche Opérationnelle, Université deMontréal, 2009); https://pdfs.semanticscholar.org/65d9/94fb778a8d9e0f632659fb33a082949a50d3.pdf.

32. M. D. Zeiler, R. Fergus, in European Conference on ComputerVision 2014 (Springer, 2014), pp. 818–833; https://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf.

33. I. J. Goodfellow, J. Shlens, C. Szegedy, Explaining andharnessing adversarial examples. arXiv 1412.6572 [stat.ML](20 March 2015).

34. K. Cheng, T. Hasegawa, K. S. Saleem, K. Tanaka, Comparison ofneuronal selectivity for stimulus speed, length, and contrast in theprestriate visual cortical areas V4 and MT of the macaquemonkey.J. Neurophysiol. 71, 2269–2280 (1994). doi: 10.1152/jn.1994.71.6.2269; pmid: 7931516

35. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, in 2009 IEEEConference on Computer Vision and Pattern Recognition (IEEE, 2009),pp. 248–255; www.image-net.org/papers/imagenet_cvpr09.pdf.

ACKNOWLEDGMENTS

We thank A. Pasupathy for generously providing the complex-curvature stimuli, and K. Schmidt, C. Shay, and S. Sanghavi fortechnical support. Funding: Supported by the IntelligenceAdvanced Research Projects Agency, U.S. National Eye Institutegrant R01-EY014970 (J.J.D.), Office of Naval Research grantMURI-114407 (J.J.D.), and Simons Foundation grant SCGB-542965(J.J.D.). Author contributions: P.B., K.K., and J.J.D. designedresearch; P.B. implemented the synthesis algorithm; K.K. and J.J.D.performed animal surgeries; K.K. performed neural recordings;P.B. and K.K. analyzed data; and P.B., K.K., and J.J.D. wrote thepaper. Competing interests: J.J.D. is an associate fellow of theCanadian Institute for Advanced Research (CIFAR). J.J.D. hasserved as a scientific advisor for, and has a financial interest in,Bay Labs Inc. Data and materials availability: The methods areclearly described, and the primary data are available athttps://github.com/dicarlolab/npc.

SUPPLEMENTARY MATERIALS

science.sciencemag.org/content/364/6439/eaav9436/suppl/DC1Figs. S1 to S8Table S1

4 November 2018; accepted 5 March 201910.1126/science.aav9436

Bashivan et al., Science 364, eaav9436 (2019) 3 May 2019 11 of 11

RESEARCH | RESEARCH ARTICLEon M

ay 2, 2019

http://science.sciencemag.org/

Dow

nloaded from

Neural population control via deep image synthesisPouya Bashivan, Kohitij Kar and James J. DiCarlo

DOI: 10.1126/science.aav9436 (6439), eaav9436.364Science 

, this issue p. eaav9436Sciencewas shown to reproduce the overall behavior of the animals' neural responses.and highly selective influence over the neuronal populations. Using novel and non-naturalistic images, the neural networkdesired effects in the macaque visual cortex. The manipulations showed very strong effects and achieved considerable population while keeping the others unchanged. They then analyzed the effectiveness of these images in producing theused it to construct images predicted to either broadly activate large populations of neurons or selectively activate one

took an artificial neural network built to model the behavior of the target visual system andet al.hypotheses? Bashivan To what extent are predictive deep learning models of neural responses useful for generating experimental

Predicting behavior of visual neurons

ARTICLE TOOLS http://science.sciencemag.org/content/364/6439/eaav9436

MATERIALSSUPPLEMENTARY http://science.sciencemag.org/content/suppl/2019/05/01/364.6439.eaav9436.DC1

REFERENCES

http://science.sciencemag.org/content/364/6439/eaav9436#BIBLThis article cites 27 articles, 8 of which you can access for free

PERMISSIONS http://www.sciencemag.org/help/reprints-and-permissions

Terms of ServiceUse of this article is subject to the

is a registered trademark of AAAS.Sciencelicensee American Association for the Advancement of Science. No claim to original U.S. Government Works. The title Science, 1200 New York Avenue NW, Washington, DC 20005. 2017 © The Authors, some rights reserved; exclusive

(print ISSN 0036-8075; online ISSN 1095-9203) is published by the American Association for the Advancement ofScience

on May 2, 2019

http://science.sciencem

ag.org/D

ownloaded from