-
Peter Gaffney / PHF 2009-2010
Connectionism and Ontological Realism
(Or “This is a tank.”)
The appearance of the mirror already introduced into the world
of perception an ironical effect of trompe-l’oeil, and we know what
malefice was attached to the appearance of doubles. But this is
also true of all the images which surround us: in general, they are
analyzed according to their value as representations, as media of
presence and meaning. The immense majority of present day
photographic, cinematic and television images are thought to bear
witness to the world with a naïve resemblance and a touching
fidelity. We have spontaneous confidence in their realism. We are
wrong. They only seem to resemble things, to resemble reality,
events, faces. Or rather, they really do conform, but their
conformity is diabolical.
Jean Baudrillard, “The Evil Demon of Images” (1987)
Representations are bodies too!
Gilles Deleuze and Félix Guattari, A Thousand Plateaus
(1987)
-
PART I: Computer vision and the problem of “investiture”
The problem
When you ask somebody in the field of computer vision about the
limitations of neural
networks, they are likely to tell you this story1: In the 1980s,
the Pentagon implemented a plan to
equip each of its tanks with a device that would scan the visual
field for enemy tanks and alert
the driver if it saw anything suspicious. The plan looked
feasible enough as far as hardware goes
(a camera connected to a computer), but when it came to writing
a program capable of
deciphering the complexity of the visual field in real time,
they found that conventional
algorithm-based data processing was ill suited to the task.
Making matters worse, the device
would have to be capable of detecting enemy tanks hiding behind
trees or other objects in the
visual field. So they opted for an artificial neural network, a
type of program that is designed to
mimic the way biological neurons approach the task of pattern
recognition (and other perceptual-
cognitive functions). We will see in a minute how this differs
from algorithmic symbol-based
programs, and how neural networks gain a key advantage when
employed in computer vision. In
the case of the tank-detection program, it is important to note
that neural networks, unlike
conventional programs, do not start out with a complete
understanding of how to execute a
particular task; they have to be trained. So the programmers
took 100 photos of tanks hiding
behind trees and another 100 of trees without tanks. They fed
half of all the photos into the
neural net and put the other half aside for testing the program
when training was complete. Each
time a photo was presented to the net, it was asked if there was
a tank hiding behind a tree.
1 I based my version of the story on the article “Neural Network
Follies” by Google software engineer Neil Fraser. The same article
is cited in Priddy and Keller, Artificial Neural Networks: An
Introduction (Bellingham, Washington: The International Society for
Optical Engineering, 2005), and Crochat and Franklin,
“Back-propagation neural network tutorial”
(http://ieee.uow.edu.au/~daniel/software/libneural).
Gaffney, "Connectionism and Ontological Realism" Page 2
-
Initially, the program was only able to produce random guesses,
since it did not even know what
it was supposed to do, let alone what features might make a tank
visually distinct from a tree.
Distinctions like these (the task itself, in essence) must be
“taught” to the neural net. This is done
by changing the way excitement flows from one “neuron” (or unit)
to another⎯that is, by
modifying the “weights” of all connections within the network
until it consistently produces the
desired distribution of electrical excitement for a given input.
Eventually, the program was able
to identify every photo in which a tank was hiding behind a
tree. It performed equally well when
it was given the 100 photos initially set aside (testing stage).
But the programmers wanted to be
sure, so they returned to the field and took 100 more photos.
This time, however, the answers
that the neural net generated appeared to be completely random.
What the programmers had
failed to consider is that the initial 100 photos with tanks had
been taken on a sunny day and all
the others when it was cloudy. Guided by no symbolic
representation on which to base its
deductions during the training stage, the neural net had been
looking all along at the color of the
sky. As Neil Fraser puts it, “The military was now the proud
owner of a multi-million dollar
mainframe computer that could tell you if it was sunny or
not.”
What does this story tell us? At first glance, it seems to
illustrate a fundamental flaw with
neural nets: we just do not know what they learn.2 But what
about the programmers? They had
neglected to control a key variable in the training period
(weather) that just happened to coincide
with the target pattern (hidden tanks). This may seem like an
obvious error⎯though, apparently,
not an uncommon one3⎯, but we can imagine other, subtler
patterns that a programmer might
2 It is worth looking at Neil Fraser’s exact words here: “This
story…is a perfect illustration of the biggest problem behind
neural networks. Any automatically trained net with more than a few
dozen neurons is virtually impossible to analyze and understand.
One can’t tell if a net has memorized inputs, or is ‘cheating’ in
some other way.” 3 Rolf Lakaemper, a computer vision and robotics
expert who first brought this story to my attention, added that
programmer errors of this magnitude happen all the time.
Gaffney, "Connectionism and Ontological Realism" Page 3
-
inadvertently feed to a neural network during the training
period, patterns that would not
necessarily be as obvious as the weather. We can even imagine
patterns that emerge as a
determining factor in the regular behavior of a neural net
without ever becoming detectable to
the programmer⎯until some catastrophic later moment when the two
patterns suddenly diverge.
It is plausible, for example, that birds partially hidden in the
photos of trees were not present in
the ones with tanks. Such a coincidence might initially help a
neural net produce the desired
output, until the day when birds are no longer afraid of tanks,
or some combination of season and
terrain means there are no birds to begin with.
But this is precisely what is so striking about neural nets. At
no point in the training
period can we say that they learn to recognize the target
object⎯indeed, we cannot say they
learn to recognize any object at all, not even the color of the
sky, so long as we define these
things first and foremost as symbolic representations, each one
with a finite sets of attributes.
What a neural network lacks is not the capacity to react
consistently to certain patterns, but the
power to treat intersecting patterns as discrete entities. We
have already gone too far when we
say that the tank-detection program can tell us if it is sunny
or not, since it could just as well be
some other pattern that intersects with fair weather, one that
is generally present⎯but not limited
to⎯the collection of attributes that we call a “sunny sky.” The
most we can say about a neural
network, in this sense, is that its numerous connections can be
modified to distribute electrical
impulses in a certain pattern every time it encounters some
other pattern (but which?) in the real
world. By the same token, we cannot say that responses produced
by the tank-detection program
during the final phase of testing were random in any sense of
the word. They only appear random
from the point of view of a programmer who is tracking the
features of a different object.
Gaffney, "Connectionism and Ontological Realism" Page 4
-
From basic neural nets to adaptive-subspace self-organizing maps
(ASSOM)
In their basic conception, neural networks designed for
pattern-recognition do not differ
greatly from conventional feature extraction and mapping. Early
designs for these networks
provided a new platform for processing complex data, one that
led to a new paradigm in
cognitive science and artificial intelligence, namely
connectionism. But it was not until the early
1980s that Teuvo Kohonen, a Finnish physicist working on the
problem of adaptive learning in
neural nets, developed a method for taking full advantage of its
novel characteristics.
Connectionism gets its name from the numerous synaptic-like
connections that join one layer of
units to another in a neural net. In a simple “feed-forward”
network (fig. 2), units on an input
layer are activated in a certain pattern (+1, -1, -1, +1). Each
one of these units sends a charge to
each other unit on the output layer, but the strength of that
charge (the amount of activation)
depends on the weight of each connection. It is by modifying
these weights that a programmer
can “train” a neural net to produce a desired pattern when it is
given a particular input. Since it is
the programmer who changes these weights (according to a
technique called “backpropagation”),
Gaffney, "Connectionism and Ontological Realism" Page 5
-
it is not clear at first why the proper weights are not simply
calculated in advance and
incorporated into the design of the machine. But this can be
explained in part by the fact that
many connectionists consider their research as a basis for
bridging the conceptual gap between
artificial intelligence and cognitive science. Neural nets
provide a functional model that explains
some of the same adaptive properties as real neurons in the
brain, and therefore bring new insight
to key issues in both fields.
But is there any practical advantage to be gained by reaching
the appropriate
configuration of weights over many (sometimes thousands) of
training cycles or “epochs”? One
answer has to do with the capacity for neural networks to
memorize and implement many
different patterns. A simple two-layered network designed by
James McClelland and David
Rumelhart was able to match four eight-digit patterns using
eight input units and eight output
units (Bechtel and Abrahamsen 1991, 2002: 93). This is
remarkable considering the similarities
between patterns, which consisted only in positive and negative
integer activation values, as in
the example above. Even more impressive is the network’s ability
to cope with distorted input
data. William Bechtel and Adele Abrahamsen built a two-layered
network but simulated
distortion by adding a random value between 0.5 and -0.5 to the
activation of each input. In only
50 epochs, the network produced the appropriate output for each
of the four patterns with a
margin of error of less than 0.2. In other experiments, they
reversed the sign of one of the units
or gave it an input pattern it had never seen before. In both
cases, the neural net performed well,
with nearly error-free output in the first case and plausible
generalized output in the second (i.e.
it chose the pattern from the four it had learned that most
resembled the new pattern).
Examples like these demonstrate how complex pattern-recognition
tasks can be carried
out with relatively few units simply by shifting the work of
computation to the connections
Gaffney, "Connectionism and Ontological Realism" Page 6
-
between them. But it is difficult to imagine how this principle
could be directly applied to real-
life pattern-recognition devices like the one designed to detect
tanks. For one, we have been
looking at models for neural networks in which the units are all
uniform, and uniformly
distributed. How would such a network approach the task of
selecting out those features that
belong to a tank or any other object, so that it could
distinguish it from other objects in the visual
field? The easiest solution is to design each input neuron to
detect a different feature and then
relay it to another layer where it can be incorporated into
higher representations: this is called
“feature extraction.” In 1959, Oliver Selfridge designed a
feature-extraction and processing
principle called “pandemonium” that is a forerunner of modern
Optical Character Recognition
(OCR) software. In the diagram below (fig. 3), we see how
competition among the feature
“demons” leads to correct output.
Gaffney, "Connectionism and Ontological Realism" Page 7
-
As mentioned earlier, Kehonen’s approach to pattern-recognition
in neural nets, called
adaptive-subspace self-organizing maps (ASSOM), differs in
several ways from the examples
above:
(1) Input units are not set up in advance to serve as
feature-specific (invariant)
detectors or “demons”; in their initial state, these units are
variable and
uniform (like the ones in fig. 2). It is only after many epochs
in the training
period that the neural net begins to acquire the capacity to
detect micro-
features in a visual field.
(2) Training is unsupervised; this means that programmers have
no part in changing
the weights of connections; instead, ASSOM nets are designed
with several
layers of self-organizing units whose only job is to record
patterns they receive
from the input layer. Weights are changed as a result of
competition among
these subspace representations.
(3) This “winner takes all” approach to competitive learning
also requires that units
have the capacity to exert an inhibitory effect on other groups
of subspace
units. This habituation to regular patterns in its environment
allows the neural
network to build successively more complex maps, but it also
means that some
patterns will be filtered out by more competitive
representations. This is meant
to simulate the way perceptual-cognitive faculties are able to
identify the same
patterns when they are partially hidden or transformed with
respect to
contingencies in the visual field. (Kehonen 1995, 2000, Kehonen
2003)
Kohonen’s explicit purpose in implementing the principle of
self-organization is to avoid any
model in which the mechanism for transforming patterns into
representations might be based on
Gaffney, "Connectionism and Ontological Realism" Page 8
-
a priori representations: “What we aim at is a model of
experimental data, not of the process
variables” (2003: 1178). This may sound like a subtle
distinction, but we have seen that it can
mean the difference between a device that detects enemy tanks
and one that just connects
ineffably with the world.
The symbolic approach
It will be useful to consider how a conventional symbol-based
processor would handle the
same pattern-recognition task as the tank-detection net before
moving on to some conclusions
about what this means for a new “ontological” definition of
perception. There are two questions
I’d like to pose in this part of my paper: (1) How do
symbol-based processors work?; (2) What
can conventional principles of pattern recognition tell us about
the (still ambiguous) notion of
symbolic representations, and what does this have to do with
what we will have reason to call
“investiture and decoding” (or Baudrillard’s “diabolical
conformity”)? In its most basic
conception, a conventional computer program functions by
manipulating symbols according to
an algorithm or sequence of instructions. In the case of a
program designed to identify an enemy
tank, such an algorithm might proceed from one proposition to
another following the decision
tree shown below (fig. 4). From the initial condition
(transmission of new data from camera to
computer), data flows through a series of “if…then…”
propositions that model the object of a
tank and lead the program to the appropriate safe/alert
response. The diagram in figure 4 is a
rather simplified version of what might be required to complete
the task of tank detection, but we
could always add more propositions to the sequence.
Gaffney, "Connectionism and Ontological Realism" Page 9
-
Rather than leading each part of the data through a series of
commands, it might be better to
cluster it into different tasks. Some of these might compare new
features to previous maps in
order to make subtler deductions. The diagram below (fig. 5)
shows how an actual program
enables a mobile robot to build maps of its environment
autonomously. This feature is called
“simultaneous localization and mapping” or SLAM:
Gaffney, "Connectionism and Ontological Realism" Page 10
-
What is important in the first example is that any instruction
added to the algorithm to further
qualify a “Yes” statement must be inserted between two
instructions in the sequence. Similarly,
an instruction or series of instructions used to qualify a “No”
would either proceed to a “Safe”
output or back to the next instruction in the main sequence.
Propositional logic thus shares its
essential structure with the syllogism, such as the famous “All
men are mortal; Socrates is a man;
therefore Socrates is mortal.” So long as the constituent
propositions are true, the final deduction
must also be true; it is a “truth-preserving” device. Of course,
we don’t need a data processor to
deduce that Socrates is mortal; this is already built into our
higher representations of Socrates, if
not immediately given in the attribute which tells us that
Socrates is already dead. But not all
syllogisms are so straightforward. We can add as many premises
to the syllogism as we want,
Gaffney, "Connectionism and Ontological Realism" Page 11
-
creating more stipulations in a chain of reasoning (one that
goes beyond the simple task of
pattern-recognition). Here’s a more complicated example:4
Instruction in natural language variables substituted If
industries are to be kept going, if B there must be a steady supply
of oil: then C If the United States is to prosper, if A its
industries must be kept going: then B therefore if the United
States is to prosper (therefore) if A there must be a steady supply
of oil. then C In order to secure a steady supply of oil, if C the
United States must go to war: then D therefore if the United States
is to prosper, therefore if A it must go to war. then D therefore
if it does not go to war, therefore if not D the United States will
not prosper. then not A
SLAM is another algorithm-based program, but the flowchart above
(fig. 5) does not
describe the attribution of symbolic values to objects in the
visual field. It only shows how the
robot extrapolates its position by comparing new visual data to
an existing map (localization),
and to modify the map whenever it diverges from the extrapolated
position (mapping). But let’s
say we were hired to modify this robot for an urban combat
environment where it must identify
and destroy enemy combatants. We would be in the same position
as the tank-detection
programmers, but with a much greater imperative to assign and
manipulate the proper symbolic
values for each target object. So long as we were using
algorithm-based processing, we would
need to develop a series of compounding stipulations similar to
the one above (if A and B and C
and D and E… then X, but only if Y = Z, etc.). This time, there
would be a much greater burden
4 This example is a modified version of the one found in Chapter
83 of R. W. Jepson’s “Clear Thinking”
(http://www.ourcivilisation.com/smartboard/shop/jepsonrw/chap83.htm).
Gaffney, "Connectionism and Ontological Realism" Page 12
-
on the side of the programmers, if not on the program itself, to
determine the thresholds between
targeted and non-targeted objects. Is an object in the visual
field acting hostile? Is it hiding a
weapon? Is it harboring insurgents? What is at issue here is not
the effectiveness of feature
extraction vis-à-vis some partially obscured object in the
visual field, but the capacity for
programmers to invest an image with whatever representations
will be necessary to complete a
given task. The real question is about human interest and
context, not computer vision.
In War in the Age of Intelligent Machines (1991), Manuel DeLanda
explores some of the
troubling implications of this issue in the case of military
automata:
The PROWLER [fig. 6]…is a small terrestrial armed vehicle,
equipped with a primitive
form of “machine vision” (the capability to analyze the contents
of a video frame) that
allows it to maneuver around a battlefield and distinguish
friends from enemies. Or at
least this is the aim of the robot’s designers. In reality, the
PROWLER still has difficulty
negotiating sharp turns or maneuvering over rough terrain, and
it also has been deployed
only for very simple tasks, such as patrolling a military
installation along a predefined
path. We do not know whether the PROWLER has ever opened fire on
an intruder
without human supervision, but it is doubtful that as currently
designed this robot has
been authorized to kill humans on its own…For now, the robot
simply makes the job of
its human remote-controller easier by preprocessing some of the
information itself, or
even by making and then relaying a preliminary assessment of
events within its visual
field. (1991: 1)
In its practical application, the computer vision of PROWLER and
other automata of its era is
little more than a surveillance camera. But DeLanda goes on to
point out that in the military
Gaffney, "Connectionism and Ontological Realism" Page 13
-
application of artificial intelligence, “it is precisely the
distinction between advisory and
executive capabilities that is being blurred” (2). He gives the
example of a war game designed to
help the Pentagon learn which sequence of actions comprise the
winning strategy; the problem is
that “SAM and IVAN [the simulated players of the game]…do not
have any problem triggering
World War III” (2).
Of course, this problem is not limited to algorithm-based
programs, nor does it first
emerge at the level of programming. As we have already seen, the
design of any pattern-
recognition device has its starting point in a network of
representations that belong to the
designers, to people further up the chain of command (those who
conceive and finance the
project), and eventually to a heterogeneous field of social
codes which determine the flows of
knowledge, investment capital, raw materials, and so on.
Likewise, computer-generated output is
only meaningful in relation to a broader set of representations
and to those who are interested in
its results. There is no such thing as a computer that sees, a
computer that knows, but this has
Gaffney, "Connectionism and Ontological Realism" Page 14
-
more to do with the agency behind it than anything in its
design. Such is the predicament John
Searle indicates when he writes: “What [a computer] does is
manipulate formal symbols. The
fact that the programmer and the interpreter of the computer
output use the symbols to stand for
objects in the world is totally beyond the scope of the
computer” (1980: 437). Searle takes the
more extreme view that a computer program is not capable of
processing actual images, only
other, non-computer symbol systems: the designations coded in a
natural language, for example,
together with the underlying cultural matrix that sanctions how
these designations are made.
What we have been calling pattern recognition thus consists in
projecting meaningful content
onto a visual field and then interacting with a social and
physical environment on that basis.
Programmers may very well have reason to be optimistic about the
growing power and
efficiency of computer vision technology. But this coincides
with growing concern among Searle
and others that there are ideological consequences of
symbol-manipulating technology. By
confusing intelligence and “investiture,” the symbolist approach
to cognitive science serves only
to reify our representations by pretending to find them “in the
outside world.”
Seeking and finding: the notion of “investiture”
It is precisely in this sense that Victor Burgin5 speaks of a
“photographic paradox”
(Thinking Photography). It is worth quoting these texts at
length here, since they illustrate how
closely the practical problems we have been discussing
resemble⎯because they derive
from⎯the more elusive mechanism of investiture:
5 It should be mentioned that many of Jean Baudrillard’s works
(The Evil Demon of Images, The Gulf War Did Not Take Place, among
others) trace the same development of a “diabolical conformity” in
the mediatization of the event. I have chosen to follow Victor
Burgin’s statement of the problem because it so closely resembles
the pattern-recognition problem.
Gaffney, "Connectionism and Ontological Realism" Page 15
-
The structure of representation⎯point-of-view and frame⎯is
intimately implicated in
the reproduction of ideology (the “frame of mind” of our
“points-of-view”). More than
any other textual system, the photograph presents itself as “an
offer you can’t refuse.”
The characteristics of the photographic apparatus position the
subject in such a way that
the object photographed serves to conceal the textuality of the
photograph
itself⎯substituting passive receptivity for active (critical)
reading. When confronted with
puzzle photographs of the “What is it?” variety (usually,
familiar objects shot from
unfamiliar angles) we are made aware of having to select from
sets of possible
alternatives, of having to supply information the image itself
does not contain. Once we
have discovered what the depicted object is, however, the
photograph is instantly
transformed for us⎯no longer a confusing conglomerate of light
and dark tones, of
uncertain edges and ambivalent volumes, it now shows a “thing”
which we invest with a
full identity, a being. With most photographs we see, this
decoding and investiture takes
place simultaneously, unselfconsciously, “naturally”; but it
does take place⎯the
wholeness, coherence, identity, which we attribute to the
depicted scene is a projection, a
refusal of an impoverished reality in favor of an imaginary
plenitude. (Burgin 146-147)
The problem with mechanical reproductions is not that they fool
the eye or cause us to confuse
reality with its likeness, but that they generate the comforting
illusion, as a function of their
realism, that symbolic representations spring readymade from the
visual field. This is what
makes photographs so persuasive. Similarly, if a computer is
able to decode an image, if it is able
to provide an appropriate answer every time it is asked “What is
it?” this is only because the
visual field has been prepared in advance to respond like a
code. The presence of an object is not
Gaffney, "Connectionism and Ontological Realism" Page 16
-
“recognized,” any more than a pattern is “decoded.” Rather,
these functions⎯truth itself, in
essence⎯arrive later on the scene, as the result of an
investment at the level of the symbolic.
Symbolist theories of human intelligence, taking their cues from
technological advances in the
field of computer vision, thus proceed by treating the world as
if it conformed to the rituals of
modern visual culture. What first confronts the eye as a
confusing jumble of dark and light is
subsequently brought into good order by the logical play of
representations. But this is not the
same thing as vision. It is only the happy coincidence of
meaning and presence, a game of hide
and seek like the one Nietzsche describes in his challenge to
the concept of truth: “When
someone hides something behind a bush and looks for it again in
the same place and finds it
there as well, there is not much to praise in such seeking and
finding. Yet this is how matters
stand regarding seeking and finding ‘truth’ within the realm of
reason” (2000: 157).
We should not view such criticisms as mere philosophical
wrangling. After all, it is by
the sanction of Allen Newell’s “unified theory of cognition”6
that SOAR, the symbolic cognitive
architecture that drives most of DARPA’s current designs for
automata, has begun to incorporate
what DeLanda calls “executive capabilities.” Recent thinking on
unmanned weapons points to
the tactical and psychological advantage to be gained by
deploying a multitude⎯or
“swarm”⎯of smaller automata against an enemy. DARPA’s newest
generation of unmanned
weapons will thus combine automatic targeting recognition with
other algorithms meant to
mimic the way birds move in flocks and ants forage for food
(Singer 2009: 232-233). As one
DARPA researcher puts it, these swarms might eventually reach
the size of “zillions and zillions
of robots” (234). What is troubling about DARPA’s multitude of
tiny robots is that each one will
6 It is notable that Newell argues strongly in favor of the view
that symbol systems have the necessary and sufficient means for
general intelligent action (“physical symbol system hypothesis”),
but that “any theory of what Soar will learn must occur within a
larger theory of the environments Soar inhabits, how tasks arise,
and how they get defined—in short, a total theory of Soar as an
agent in the world” (1994: 191).
Gaffney, "Connectionism and Ontological Realism" Page 17
-
really have to see. Unlike the PROWLER and other robots of its
generation whose deductions
were subject to the discretion of a human operator, the sheer
numbers of “Proliferated
Autonomous Weapons” (or PRAWNs) will make this kind of
intervention impractical. “Rather
than a controller back in a mothership furiously trying to point
and click at which target to
hit…the autonomous swarm would just figure it all out on its
own” (234).
There is no need to reconstruct here any of the possible
doomsday scenarios that such
automata bring to mind. What I would like to propose instead, in
light of this notion of
investiture, is a hypothesis about the ideological tension
between two moments that characterize
the social dimension of human perception, and that now plague
its technological counterparts
(including many neurally-inspired processors). In the first
moment, we find the profoundly
creative act that enables the mind, or perhaps a “collective
mind,” to invest the visual field with
an order of its own. Burgin sees this as the projection of a
pre-existing symbolic order onto a
chaotic collection of sensory data. We would do better, however,
to consider it in broader terms
as a connection between two or more patterns, with the result
that perceiver and perceived are
joined together in a determinate and productive relationship. We
may even say, following the
example of such thinkers as Henri Bergson and Gilles Deleuze,
that this simple intersection of
patterns constitutes the “actualization” of a world together
with an intelligence in it7⎯so that the
two sides, perceiver and perceived, are precisely what this
relationship produces. (This is a rather
confusing formula, and I will spend the second half of the paper
trying to sort out its
implications). If we look closely at the examples above, we see
a second moment as well, this 7 I am referring here to Bergson’s
notion that “The more consciousness is intellectualized, the more
matter is spatialized [Plus la conscience s’intellectualise, plus
la matière se spatialise]” (1959). This notion is at the center of
the Bergsonian method of intuition. Deleuze and Guattari follow the
same formula when they write that “it is by slowing down that
matter, as well as the scientific though able to penetrate it with
proposition, is actualized” (1994). In his work on the connection
between Bergson and Deleuze (Signature of the World), Eric Alliez
uses a more explicit formulation: “The diversion [détournement] of
Bergosnian (immediate) intuition is prepared both before and
beyond…the realism-idealism opposition, when the act of knowing
tends to coincide with the act that generates the real” (2004).
Gaffney, "Connectionism and Ontological Realism" Page 18
-
time characterized by a certain ideologically motivated sleight
of hand. Here, the process of
actualization that creates both intelligence and world is
carefully hidden from view. What was
actively produced in the first moment now appears to come
readymade from an “outside” world.
Symbolism, as a philosophical doctrine, thus coincides with a
visual culture (photographs,
television, cinema) that obscures the first moment by
reinforcing the second. Like scientific
realism, it has the potential to actively reduce a heterogeneous
complex of social codes to a
monolithic truth⎯a “royal science.”8 As Nietzsche puts it, there
is not much to praise in such
seeking and finding.
PART II: Connectionism and ontological realism
Does connectionism preclude representation?
The objection might be raised that the treachery of images is a
legitimate concern within
the limits of culture studies, perhaps even the other social
sciences, but has nothing to do with
computer vision or science proper. The very fact that an
automaton can interact with its
environment, the simple fact of applied science, should be taken
as sufficient proof that task-
oriented symbolic knowledge really works. This is effectively
the argument made by Newell and
Simon in their defense of the physical symbol hypothesis: “An
expression [composed of
symbols] designates an object if, given the expression, the
system can either affect the object
8 This concept is mapped out by Deleuze and Guattari in A
Thousand Plateaus: “The ideal of reproduction, deduction, or
induction is part of royal science…and treats differences of time
and place as so many variables, the constant form of which is
extracted precisely by the law…[royal science] implies the
permanence of a fixed point of view that is external to what is
produced” (372).
Gaffney, "Connectionism and Ontological Realism" Page 19
-
itself or behave in ways dependent on the object”1 (1976: 116).
What right do we have to
question the truth-value of symbolic representations? On what
grounds can we, like Burgin vis-à-
vis the photograph, describe them as no more than the effects of
an imaginary plenitude? These
are reasonable objections, and we will have no reason to
disagree with them.
The problem with symbolic knowledge is elsewhere. We spoke too
soon when we said
that it is too full, that it recognizes too many “things” in the
visual field. The problem is just the
opposite. In the symbolist conception, intelligence is reduced
to a mechanism for verifying the
identity of being (X=X). It is, from this point of view, an
epistemological machine, guided by its
truth-preserving structure and characterized by a passive
receptivity rather than an active
engagement of patterns in the visual field. This is quite
different from the case of the tank-
detection program, where the neural net failed precisely because
it was active—too active. It did
not know that the color of the sky was an invalid attribute of
the target object. It did not know
and it did not care: as an ASSOM-driven neural net, it was not
trained to identify symbolic
representations, only to produce an alert signal in the presence
of any pattern common to the
target samples used in the training period. The most we can say
about such a neural net is that it
makes connections with the visual field. But it does not do this
by investing visual data with pre-
existing symbolic value (even when operators use them as if they
do). It would seem then, that
there are two ways of thinking about perception. On the one
hand, it can involve the
reconstruction of “meaningless” visual data in a matrix of
symbolic knowledge. In the field of
culture studies, Burgin considers how visual media naturalize
this process, so that a photograph 1 Newell and Simon further
specify that a physical symbol system consists of “a set of
entities, called symbols, which are physical patterns that can
occur as components of another type of entity called an expression
(or symbol structure). Thus, a symbol structure is composed of a
number of instances (or tokens) of symbols related in some physical
way (such as one token being next to another). At any instant of
time the system will contain a collection of these symbol
structures. Besides these structures, the system also contains a
collection of processes that operate on the expressions to produce
other expressions: processes of creation, modification,
reproduction and destruction. A physical symbol system is a machine
that produces through time an evolving collection of symbol
structures. Such a system exists in a world of objects wider than
just these symbolic expressions themselves” (116).
Gaffney, "Connectionism and Ontological Realism" Page 20
-
or film conveys a reality that is already implicated in a matrix
of symbols by the time it is
perceived. Thomas Kuhn proposes a similar theory on scientific
breakthroughs, which, he
argues, do not augment an existing body of knowledge, but
radically dissolve and reorganize it
around the discovery of new patterns formerly dismissed as
statistical variance (concept of
“paradigm-induced gestalt switch”).2 Scientific observation,
like all acts of perception, is
constructive, instrumental and historically specific—but is not
for that matter less objective.
In each case, we are most likely to ask how symbolic
representations give meaning and
presence to sense experience. But what happens when we trace our
line of inquiry in the opposite
direction? What role does sense experience play in the
production of representations? These are
not two sides of the same coin, but opposing interpretations of
human intelligence that are
divided along the lines already discussed in this paper. On one
side, symbolists like Newell and
Simon advance the claim that “A physical symbol system has the
necessary and sufficient means
for general intelligent action” (1976: 116). Symbolism, for the
same reasons as computationalism
in cognitive science and psychology, regards the mind as a kind
of Turing machine which
interacts with the material world by performing purely formal
operations on symbols. On the
other side, connectionists like Donald Norman and David
Rumelhart argue that symbolic
representations and the rules which govern them do not play any
necessary role in perception and
cognition, or in any other cognitive task for that matter. In
the case of language acquisition, for
example, Norman and Rumelhart argue that “a system can appear to
obey and follow general
rules of language even though it does not have those rules
within it” (1981: 239). Their research
is particularly interesting since it provides evidence that even
when we consciously use symbol
2 In The Structure of Scientific Revolutions, Kuhn writes that
theoretical conflicts arising between existent and emergent
scientific paradigms are “terminated by a relatively sudden and
unstructured event like the [visual] gestalt switch” (122). It
should be noted that Kuhn is in some ways critical of the
comparison between scientific revolution and visual gestalt theory.
He does not go so far, for example, as Norwood Hanson’s Patterns of
Discovery: An Inquiry into the Conceptual Foundations of
Science.
Gaffney, "Connectionism and Ontological Realism" Page 21
-
systems, these systems are not necessarily governed by rules
(i.e. propositional logic). Neural
nets that are conditioned to perform basic linguistic tasks also
exhibit some very human-like
error patterns during the learning process, such as the
“U-shaped” acquisition of past tense
irregular verb forms11 (McClelland and Rumelhard 1988: 117).
DeLanda links the two sides of this debate to the famous
controversy over the 29 Eskimo
words for snow.12 Do Eskimos perceive 29 kinds of snow because
their linguistic categories “cut
up” reality in this way, or do the words derive from the sensual
experience of 29 kinds of snow?
In other words, does diversity in a system of symbolic
representations provide for diversity in
sense perception or is it the other way around? Hard line
connectionists like McClelland and
Rumelhard claim that representations do not provide the
conditions of possible experience. From
their point of view, there is no essential difference between
linguistic aptitude and any other
pattern-association/completion faculty. Representations emerge
as a secondary effect of
connections between one pattern and another; they are a symptom
rather than a cause of complex
behavior—perhaps only a convenient metaphor. Others in the field
question the usefulness, even 11 Children who correctly form
past-tense irregular verbs in an initial stage will often cease to
use them correctly while learning how to construct regular verb
endings. In a third stage, the child learns that irregular verbs
are a special case, and begins using them correctly once more. For
example, a child who initially says “She went to school” might
begin saying “She goed to school” during the initial stage of
learning the regular verb form. Does this three-stage, or U-shaped,
learning process occur because children learn to apply (and then
over-generalize) a new rule? Or is learning governed only by
exposure and usage, so that no explicit rule is ever internalized
or applied? Proceeding from the second hypothesis, McClelland and
Rumelhart constructed a neural network to mimic acquisition of the
past tense. Results suggest that it followed the same U-shaped
learning pattern as children who are learning a natural language
(McClelland and Rumelhart 1988: 117). 12 “Everyone knows that
Eskimos have 29 words for snow…The question is: Do Eskimos see 29
kinds of snow because they have 29 words for snow? That was the
position taken by linguists in the 20th century⎯words cut out
experience and in fact give form to experience; linguistic
categories shape reality, at least phenomenological reality⎯and
therefore their having 29 words for snow is the reason why they can
see 29 kinds of snow. That’s one position. The other position would
be the opposite: They have 29 words for snow because they can
touch, feel, smell, build igloos with, and do all kinds of
non-linguistic things with real snow, which comes in 29 kinds
because there are at least 29 mixtures of the solid and the
liquid….The two options remain quite different. In one, the 29
words are shaping perceptual reality literally. In the other,
perception is not linguistic. Perception is multi-modal: it’s about
sounds, smells, textures, images, all of which have their own
modality and can be combined in different ways. It’s also about
intervening causally in the world: to build igloos with that snow,
to hunt over that snow, to find your way out of a snow storm…”
(DeLanda 2007).
Gaffney, "Connectionism and Ontological Realism" Page 22
-
the validity, of continuing to use such a metaphor. Van Gelder,
a leading proponent of dynamical
systems theory, compares cognitive faculties to Watt’s steam
engine and centrifugal governor
(fig. 7): “when we see the relationship between engine
[flywheel] speed and arm angle, we see
that the notion of representation is just the wrong sort of
conceptual tool to apply…arm angle
and engine speed are at all times both determined by, and
determining, each other’s behavior.”
This relationship is “much more subtle and complex…than the
standard concept of
representation can handle” (van Gelder 1995: 353). Bechtel
disagrees. New findings by
connectionists challenge our conventional notion of how
representations work, but do not
preclude this notion altogether. In the case of Watt’s governor,
Bechtel argues, a special
representation of speed is necessary in order to connect one
system (referent) with another
(consumer): “Without the spindle arms and their linkage
mechanism, the valve has no access to
information about the flywheel speed. They were inserted in
order to encode that information in
a format that could be used by the valve-opening mechanism.”
Representations thus allow the
state of affairs in one medium to be transmitted in a format
that can be “understood” by another.
Gaffney, "Connectionism and Ontological Realism" Page 23
-
Deleuze and ontological realism
Put this way, however, there is no intrinsic difference between
a representation and an
optical device or an eye, a taste bud or an antenna. One of the
problems that arise whenever we
make a special case for representations is that we tend to get
hung up on the question of truth. Is
our representation of the world an accurate one? How do we know?
Can we know the world at
all? Such questions are of a purely epistemological value; they
help us evaluate the status of
knowledge with respect to the unstable boundaries of what is.
But they have limited value for
understanding the role of perception in constituting the real,
that is, in explaining how vision
(and other senses) actively construct, dissolve and articulate
patterns in a continuous coming to
be. If we put ourselves in the position of a self-organizing
neural net as it engages patterns in its
environment, we come to the realization that there is no basis,
much less practical reason, to
search elsewhere for “a world.” This activity is itself a world.
Yet it comprises only one of many
worlds that periodically intersect, disperse, emerge and
disappear again. We do not even have an
environment—an “outside”—only a milieu, assemblage or block of
becoming to which our own
becoming is inextricably connected. The matrix of social codes
identified by Burgin (and so
often vilified in culture studies) is just one more articulation
in a network made up of many
worlds—not possible worlds or “monads,” but real intersections
of abstract and material patterns.
As I have suggested elsewhere13, it is on these grounds that
Deleuze interprets the predictive
success of scientific observation and the models that derive
from it. From Deleuze’s point of
view, there is no need to question the truth or validity of such
models, especially as this would
lead us once more to a merely epistemological evaluation of the
problem. What Deleuze wants to
13 “Superposing Images: Deleuze and the Virtual after Bergson’s
Critique of Science,” in Peter Gaffney ed. The Force of the Virtual
(Minneapolis: University of Minnesota Press, 2010).
Gaffney, "Connectionism and Ontological Realism" Page 24
-
establish, starting with his monograph on Hume (Empiricism and
Subjectivity) and borrowing
heavily from Bergson (Bergsonism), is an ontology of becoming
that finds its root in all forms of
perception, conceived broadly as the creative force of the
virtual in the process of actualizing
itself. A neural net does not see things in the outside world;
rather, it introduces a very special
kind of eye, one that has the power to generate a set of
representations that will articulate one
system with another—to generate a world made out of patterns
that are themselves the
articulating feature of some other connection. Timothy Murphy
has proposed that quantum
physical models provide a “realist ontological” framework for
treating probably phenomena as
real events (1998: 213). In this paper, I have tried to show how
connectionism provides the same
kind of framework for an ontological realist account of
perception.
Conclusion
If a self-organizing neural net goes through the same stages as
human intelligence when
confronted with the task of pattern-recognition and completion,
then we must treat symbolic
representations as a strict redundancy. Surely representations
exist, and there is no doubt that
they play a central role in the way we shape our world. But they
are strange representations with
no beginning and no end, objective but not whole, real but not
corporeal. They do not refer to
things; they are things. We will not be prevented, for that
matter, from using representations in
the conventional manner (in this paper, for example). But it
will be difficult to proceed on the
understanding that they approximate a world that precedes them,
a world whose presence and
meaning confirms their validity, yet which remains just out of
reach. This kind of scientific or
Cartesian realism14 no longer applies when the act of seeking is
itself the constitutive feature of
14 With the term “Cartesian realism,” I am following the
definition proposed by Joseph Margolis: “any realism, no matter how
defended or qualified, that holds that the world has a determinate
structure apart from all constraints of
Gaffney, "Connectionism and Ontological Realism" Page 25
-
whatever might be found. We cannot therefore speak of what a
representation means, any more
than we can ask what a neural network learns. In each case,
representation functions only as a
connection between two or more systems, one system organizing
the other: a sunny sky system
organizing a neural net system, an incorporeal system organizing
a material system, or vice
versa. A vast system of connections whose boundaries are
continuously redrawn by the forces of
interest, desire and creativity: this is what we learn from the
example of the tank-detection
device. There is nobody to blame for its failure; there is no
failure. The human trainers could not
limit the task any more than the device could limit the means
for accomplishing it. In any case,
the story is apocryphal.15
human inquiry and that our cognizing faculties are nevertheless
able to discern those independent structures reliably.
‘Cartesianism’ serves as a term of art here, not confined to
Descartes’ doctrine. It ranges over pre-Kantian philosophy, Kant’s
own philosophy (quixotically), and over the views of such
contemporary theorists as Putnam and Davidson” (2006: 194). 15
Though it is cited by several other authors (see note 1), I have
found no published source which identifies the source of the story.
Notably, it is not mentioned in William Bechtel and Adele
Abrahamsen’s Connectionism and the Mind. Nick Bostrom and Milan M.
Ćirković (Global Catastrophic Risk) write, “This story, although
famous and oft-cited as fact, may be apocryphal…however, failures
of the type described are a major real-world consideration when
building and testing neural networks” (321).
Gaffney, "Connectionism and Ontological Realism" Page 26
-
WORKS CITED _______________________
Alliez, Eric. The Signature of the World: Or, What is Deleuze
and Guattari's Philosophy? (New York: Continuum, 2004), 36.
Baudrillard, Jean. “The Evil Demon of Images,” in Clive Cazeaux
ed. The Continental Aesthetics Reader (London: Taylor and Francis,
2000). Reprinted from The Evil Demon of Images. Trans. Paul Patton
and Paul Foss (Sydney: Power Institute of Fine Arts, 1987),
13-34.
Bechtel, William and Adele Abrahamsen. Connectionism and the
Mind, Second Edition (Malden, Massachusetts: Blackwell Publishers,
Inc., 2002).
Bergson, Henri. L’Évolution créatrice (Paris: Les Presses
universitaires de France, 1959), 132.
Bostrom, Nick and Milan M. Ćirković. Global Catastrophic Risk
(Oxford: Oxford University Press, 2008), 321.
Burgin, Victor. Thinking Photography (London: Macmillan
Education, 1982).
DeLanda, Manuel. War in the Age of Intelligent Machines (New
York: Zone Books, 1991), 1-2.
--------“The Origins of Artificial Intelligence.” 2007 Penn
Humanities Forum, University of Pennsylvania, Philadelphia. 7
November 2007.
Deleuze, Gilles and Félix Guattari, A Thousand Plateaus. Trans.
Brian Massumi (Minneapolis: University of Minnesota Press,
1987).
--------What Is Philosophy? Trans. Hugh Tomlinson and Graham
Burchell (New York: Columbia University Press, 1994), 118.
Dreyfus, Hubert L. “Artificial Intelligence,” American Academy
of Political and Social Science, Vol. 412 (March 1974), 21-33.
Fraser, Neil. “Neural Network Follies,”
http://neil.fraser.name/writing/tank)
Kehonen, Teuvo. Self-Organizing Maps (Berlin: Springer-Verlag,
1995, 2000).
Kehonen, Teuvo. “Self-Organized Maps of Sensory Events,” in
Philosophical Transactions: Mathematical, Physical and Engineering
Sciences, Vol. 361, No. 1807 (Jun. 15,
2003), 1177-1186.
Kuhn, Thomas. The Structure of Scientific Revolutions (Chicago:
University Of Chicago Press, 1996).
Gaffney, "Connectionism and Ontological Realism" Page 27
-
McClelland, J. L. and David Rumelhard. Explorations in parallel
distributed processing: a handbook of models (Cambridge,
Massachusetts: Harvard University Press, 1988), 117.
Murphy, Timothy. “Quantum Ontology: A Virtual Mechanics of
Becoming,” in Eleanor Kaufmann and Kevin Jon Heller eds. Deleuze
and Guattari: New Mappings in Politics, Philosophy and Culture
(Minneapolis: University of Minnesota Press, 1998), 211-229.
Newell, Allen and Herbert A. Simon. “Computer Science as
Empirical Inquiry: Symbols and Search,” Communications of the ACM.
vol. 19, No. 3 (March 1976), 113-126. Cited by Nils J. Nilsson,
“The Physical Symbol System Hypothesis: Status and Prospects,” in
Rolf Lungarella, Rolf Pfeifer, Josh Bongard eds., 50 Years of
Artificial Intelligence, LNAI (Berlin: Springer Berlin / Verlag,
2007), 10.
Newell, Allen. Unified Theories of Cognition (Cambridge,
Massachusetts: Harvard University Press, 1994).
Nietzsche, Friedrich. “On Truth and Lie in an Extra-Moral
Sense,” in Clive Cazeaux ed. The Continental Aesthetics Reader
(London: Taylor and Francis, 2000), 57.
Norman, Donald and David Rumelhart. “The LNR approach to human
information processing.” in Cognition, Volume 10, Issues 1-3
(July-August 1981), 239.
Rockmore, Tom. “Incommensurability Modestly Recovered,” in Tom
Rockmore and Joseph Margolis eds., History, Historicity and Science
(Hampshire: Ashgate, 2006).
Searle, John R. “Minds, Brains, and Programs,” Behavioral and
Brain Sciences, 3(3: 1980), 417-457.
Singer, P. W. Wired for War: The Robotics Revolution and
Conflict in the Twenty-First Century (New York: The Penguin Press
HC, 2009).
van Gelder, T. “What might cognition be, if not computation?”
The Journal of Philosophy, 92 (1995), 345-381. Cited by Bechtel and
Abrahamsen (1991, 2000: 275).
Gaffney, "Connectionism and Ontological Realism" Page 28
The problemInstruction in natural languagevariables
substituted