8/13/2019 Mculloch Pitts http://slidepdf.com/reader/full/mculloch-pitts 1/19 BULLETIN OF MATHEMATICAL BIOPHYSICS VOLUME 5 1943 A LOGICAL CALCULUS OF THE IDEAS IMMANENT IN NERVOUS ACTIVITY WARREN S. MCCULLOCH AND WALTER PITTS FROM THE UNIVERSITY OF ILLINOIS COLLEGEOF MEDICINI~ DEPARTMENT OF PSYCHIATRY AT THE ILLINOIS NEUROPSYCHIATRIC INSTITUTE AND TH E UNIVERSITY OF CHICAGO Because of the all-or-none character of nervous activity, neural events and the relations among them can be treated by means of propo- sitional logic. It is found that the behavior of every net can be described in these terms with the addition of more complicated logical means for nets containing circles; and that for any logical expression satisfying certain conditions one can find a net behaving in the fashion it describes. It is shown that many particular choices among possible neurophysiologi- cal assumptions are equivalent in the sense that for every net behav- ing under one assumption there exists another net which behaves un- der the other and gives the same results although perhaps not in the same time. Various applications of the calculus are discussed. I Introduction Theoretical neurophysiology rests on certain cardinal assump- tions. The nervous system is a net of neurons each having a soma and an axon. Their adjunctions or synapses are always between the axon of one neuron and the soma of another. At any instant a neuron has some threshold which excitation must exceed to initiate an im- pulse. This except for the fact and the time of its occurrence is de- termined by the neuron not by the excitation. From the point of ex- citation the impulse is propagated to all parts of the neuron. The velocity along the axon varies directly with its diameter from less than one meter per second in thin axons which are usually short to more than 150 meters per second in thick axons which are usually long. The time for axonal conduction is consequently of little impor- tance in determining the time of arrival of impulses at points un- equally remote from the same source. Excitation across synapses oc- curs predominantly from axonal terminations to somata. It is still a moot point whether this depends upon irreciprocity of individual syn- apses or merely upon prevalent anatomical configurations. To sup- pose the latter requires no hypothesis ad hoc and explains known ex- ceptions but any assumption as to cause is compatible with the cal- culus to come. No case is known in which excitation through a single synapse has elicited a nervous impulse in any neuron whereas any 115
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
I D E A S I M M A N E N T I N N E R V O U S A C T I V I T Y
W A R R E N S . M C C U L L O CH A N D W A L T E R P I T T S
FROM THE UNIVERSITY OF ILLINOIS COLLEGE OF MEDICINI~
DEPARTMENT OF PSYCHIATRY AT THE ILLINOIS NEUROPSYCHIATRIC INSTITUTE
AND THE UNIVERSITY OF CHICAGO
B e c a u s e o f t h e a l l- o r -n o n e c h a r a c t e r o f n e r v o u s a c t i v i ty , n e u r a le v e n t s a n d t h e r e l a t i o n s a m o n g t h e m c a n b e t r e a t e d b y m e a n s o f p r o p o -s i t io n a l l o g ic . I t i s f o u n d t h a t t h e b e h a v i o r o f e v e r y n e t c a n b e d e s c r i b e di n th e s e t e r ms w i t h t h e a d d i ti o n o f mo r e c o mp l i c a t e d l o g i c a l m e a n s f o rn e t s c o n t a i n i n g c i r c l e s ; a n d t h a t f o r a n y l o g i c a l e x p r e s s i o n s a t i s f y i n gc e r t a i n c o n d i ti o n s o n e c a n fi n d a n e t b e h a v i n g i n t h e f a s h i o n i t d e s c r i b e s.I t i s s h ow n t h a t m a n y p a r t i c u l a r c h oi ce s a m o n g p o s s ib l e n eu r o p h y si o lo g i -c a l a s s u m p t i o n s a r e e q u i v al e n t i n t h e se n s e t h a t f o r e v e r y n e t b e h a v -i n g u n d e r o n e a s s u m p t i o n t h e r e e x i s ts a n o t h e r n e t w h i c h b e h a v e s u n -d e r th e o t h e r a n d g i v e s t h e s a m e r e s u l t s a l t h o u g h p e r h a p s n o t i n t h es a m e t i me . Va r i o u s a p p l i c a t i o n s o f t h e c a l c u l u s a r e d i s c us s e d .
I In t r o d u c t i o n
T h e o r e t i c a l n e u r o p h y s i o l o g y r e s t s o n c e r t a i n c a r d i n a l a s s u m p -
t i o n s . T h e n e r v o u s s y s t e m i s a n e t o f n e u r o n s e a c h h a v i n g a s o m a
a n d a n a x o n . T h e i r a d j u n c t i o n s o r s y n a p s e s a r e a l w a y s b e t w e e n th e
a x o n o f o n e n e u r o n a n d t h e s o m a o f a n o t h e r . A t a n y i n s t a n t a n e u r o n
h a s s o m e t h r e s h o l d w h i c h e x c i t a t i o n m u s t e x c e e d t o i n i t i a t e a n i m -
p u l se . T h i s e x c e p t f o r t h e f a c t a n d t h e t i m e o f i t s o c c u r r e n c e i s d e -
t e r m i n e d b y t h e n e u r o n n o t b y t h e e x c i t a t i o n . F r o m t h e p o i n t o f e x -
c i t a t i o n th e i m p u l s e i s p r o p a g a t e d t o a ll p a r t s o f t h e n e u r o n . T h e
v e l o c i t y a l o n g t h e a x o n v a r i e s d i r e c t l y w i t h i t s d i a m e t e r f r o m l e s s
t h a n o n e m e t e r p e r s e c o n d i n t h i n a x o n s w h i c h a r e u s u a l ly s h o r t t o
m o r e t h a n 1 5 0 m e t e r s p e r s e c o n d i n t h i c k a x o n s w h i c h a r e u s u a l ly
l o n g . T h e t i m e f o r a x o n a l c o n d u c t i o n i s c o n s e q u e n t l y o f l i t t le i m p o r -
t a n c e i n d e t e r m i n i n g t h e t i m e o f a r r i v a l o f i m p u l s e s a t p o i n t s u n -
e q u a l l y r e m o t e f r o m t h e s a m e s o u rc e . E x c i t a t i o n a c r o s s s y n a p s e s o c -
c u r s p r e d o m i n a n t l y f r o m a x o n a l t e r m i n a t i o n s t o s o m a t a . I t i s s t i l l a
m o o t p o i n t w h e t h e r t h i s d e p e n d s u p o n i r r e c i p r o c i t y o f i n d i v i d u a l s y n -
a p s e s o r m e r e l y u p o n p r e v a l e n t a n a t o m i c a l c o n f ig u r a t i o n s . T o s u p -
p o s e t h e l a t t e r r e q u i r e s n o h y p o t h e s i s a d h o c a n d e x p l a i n s k n o w n e x -
c e p t i o n s b u t a n y a s s u m p t i o n a s t o c a u s e is c o m p a t i b l e w i t h t h e c a l-
c u l u s t o c o m e . N o c a s e i s k n o w n i n w h i c h e x c i t a t i o n t h r o u g h a s i n g l e
s y n a p s e h a s e l ic i t e d a n e r v o u s i m p u l s e i n a n y n e u r o n w h e r e a s a n y
n e u r o n m a y b e ex c i te d b y i m p u l s e s a r r i v i n g a t a s uf fi c i en t n u m b e r o f
n e i g h b o r i n g s y n a p s e s w i t h i n t h e p e r i o d o f l a t e n t a d d i ti o n , w h i c h l a s ts
l es s t h a n o n e q u a r t e r o f a m i l l is e co n d . O b s e r v e d t e m p o r a l s u m m a t i o n
o f i m p u l s e s a t g r e a t e r i n t e r v a l s is im p o s s i b l e fo r s i n g l e n e u r o n s a n d
e m p i r i c a l ly d e p e n d s u p o n s t r u c t u r a l p r o p e r t i e s o f t h e n e t. B e t w e e n
t h e a r r i v a l o f i m p u l s e s u p o n a n e u r o n a n d i t s o w n p r o p a g a t e d i m -
p u l s e t h e r e i s a s y n a p t i c d e l a y o f m o r e t h a n h a l f a m i l l is e c o n d . D u r -
i n g t h e f i r s t p a r t o f t h e n e r v o u s i m p u l s e t h e n e u r o n i s a b s o l u t e l y re -
f r a c t o r y t o a n y s t i m u l a t i o n . T h e r e a f t e r i t s e x c i t a b i l i t y r e t u r n s r a p -
i dl y, in s o m e c a s e s r e a c h i n g a v a l u e a b o v e n o r m a l f r o m w h i c h i t s in k s
a g a i n t o a s u b n o r m a l v a l ue , w h e n c e i t r e t u r n s s l o w l y t o n o r m a l . F r e -
q u e n t a c t i v i t y a u g m e n t s t h i s s u b n o r m a l i t y . S u c h s p e c if ic i ty a s i s
p o s s e s s ed b y n e r v o u s i m p u l s e s d e p e n d s s o l el y u p o n t h e i r t i m e a n d
p l a c e a n d n o t o n a n y o t h e r s p e c i f i ci t y o f n e r v o u s e n e r g i e s . O f l at e
o n l y i n h i b i t i o n h a s b e e n s e r i o u s l y a d d u c e d t o c o n t r a v e n e t h i s t h e s i s .
I n h i b i t i o n i s t h e t e r m i n a t i o n o r p r e v e n t i o n o f t h e a c t i v i t y o f o n e
g r o u p o f n e u r o n s b y c o n c u r r e n t o r a n t e c e d e n t a c t i v i t y o f a s e co n d
g r o u p . U n t i l r e c e n t l y t h is c o u l d b e e x p l a in e d o n t h e s u p p o s i t i o n t h a t
p r e v i o u s a c t i v i t y o f n e u r o n s o f t h e s e c o n d g r o u p m i g h t s o r a i s e t h e
t h r e s h o l d s o f i n t e r n u n c i a l n e u r o n s t h a t t h e y c o u ld n o l o n g e r b e e x -
c i te d b y n e u r o n s o f th e f i r s t g r o u p , w h e r e a s t h e i m p u l s e s o f t h e f ir s tg r o u p m u s t s u m w i t h t h e i m p u l s e s o f t h e s e i n t e rn u n c i a l s t o ex c i te t h e
n o w in h i b i t ed n e u r o n s . T o d a y , s o m e i n h i b i t i o n s h a v e b e e n s h o w n t o
c o n s u m e le s s t h a n o n e m i l l is e c o n d . T h i s e x c l u d e s i n t e r n u n c i a l s a n d
r e q u i r e s s y n a p s e s t h r o u g h w h i c h im p u l s e s in h i b i t t h a t n e u r o n w h i c h
i s b e i n g s t i m u l a t e d b y i m p u l s e s t h r o u g h o t h e r s y n a p s e s . A s y e t e x -
p e r i m e n t h a s n o t s h o w n w h e t h e r t h e r e f r a c t o r i n e s s i s r e l a t i v e o r a b -
s o lu t e. W e w i l l a s s u m e t h e l a t t e r a n d d e m o n s t r a t e t h a t t h e d i f f er e n c e
i s i m m a t e r i a l to o u r a rg u m e n t . E i t h e r v a r i e t y o f r e f r a c t o r i n e s s c a n
b e a c c o u n t e d f o r in e i t h e r o f t w o w a y s . T h e i n h i b i t o r y s y n a p s e
m a y b e o f s u c h a k in d a s to p r o d u c e a s u b s t a n c e w h i c h r a Ss es t h et h r e s h o l d o f th e n e u r o n , o r i t m a y b e s o p l a c e d t h a t t h e l oc al d i s t u r b -
a n c e p r o d u c e d b y i ts e x c i t a ti o n o p p o s e s t h e a l t e r a t i o n i n d u c e d b y t h e
o t h e r w i s e e x c i t a t o r y s y n a p s e s . I n a s m u c h a s p o s it i o n i s a l r e a d y k n o w n
t o h a v e s u c h e f f e c ts in t h e c a s e o f e l e ct r ic a l s t i m u l a t i o n , t h e f i r s t h y -
p o t h e s i s ~s t o b e e x c l u d e d u n l e s s a n d u n t i l i t b e s u b s t a n t i a t e d , f o r t h e
s e c on d i n v o lv e s n o n e w h y p o t h e s i s . W e h a v e , th e n , t w o e x p l a n a t i o n s
o f i n h i b i t i o n b a s e d o n t h e s a m e g e n e r a l p r e m i s e s , d i f f e r i n g o n l y in
t h e a s s u m e d n e r v o u s n e t s a n d , c o n s eq u e n t l y , in th e t i m e r e q u i r e d f o r
i n h ib i ti o n . H e r e a f t e r w e s h al l r e f e r t o su c h n e r v o u s n e t s a s equiva
lent in the extended sense. S i n ce w e a r e c o n c e r n e d w i t h p r o p e r t i e so f n e t s w h i c h a r e i n v a r i a n t u n d e r e q u iv a le n c e, w e m a y m a k e t h e
p h y s i c a l a s s u m p t i o n s w h i c h a r e m o s t c o n v e n i e n t f o r t h e c al cu l us .
L e t u s d e f i n e a t e m p o r a l p r o p o s i t i o n a l e x p r e s s i o n ( a T P E ) , d e s -
i g n a t i n g a t e m p o r a l p r o po s it io n a l f u n c t i o n ( T P F ) , b y t h e f o l lo w i n gr e c u r s i o n :
1. A ~p~ [z~] is a T P E , w h e r e p ~ i s a p r e d i c a t e - v a r i a b l e .
2 . I f S ~ a n d S~ a r e T P E c o n t a i n i n g t h e s a m e f r e e i n d iv i d u a l
v a r i a b l e , s o a r e SSI , S~vS~ , S I .S~ a n d S ~ . ~ S ~ .
3 . N o t h i n g e ls e i s a T P E .
T H E O R E M I
E v e r y n e t o f o r d e r 0 c ~ n b e s o l v e d i n t e r m s o f t e m p o r a l p r o p o s i -t iona l expres s ions .
L e t c~ b e a n y n e u r o n o f g~ w i t h a t h r e s h o l d t~ > 0 , a n d l e t c~ 1,
c ~ , . .. , c~, h a v e r e s p e c t i v e l y n ~ , n ~ : , . . . , n~p e x c i t a t o r y s y n a p s e s
u p o n i t. L e t c j~ , c i : , - - . , c~q h a v e i n h i b i t o r y s y n a p s e s u p o n i t. L e t ~
b e t h e s e t o f th e s u b c l a s s e s o f ( n ~ l, n ~ : , . ~ . , n~ ,) s u c h t h a t t h e s u m o f
t h e i r m e m b e r s e x c e e d s 6~. W e sh a l l t h e n b e a b l e t o w r i t e , i n a c c o r d -
a n c e w i t h t h e a s s u m p t i o n s m e n t i o n e d a b o v e ,
N~(z~) . - - . S t I~,~ ~ N j~ (z~ ) . ~ ,~ ~H N ~ ( z~ )} ( 1 )
w h e r e t h e 'F ~ ' a n d I I a r e s y n t a c t i c a l s y m b o l s f o r d i s j u n c t i o n s a n d
c o n j u n c t i o n s w h i c h a r e f in i te i n ea c h ca s e . S i n c e a n e x p r e s s i o n o f
t h i s f o r m c a n b e w r i t t e n f o r e a c h c~ w h i c h is n o t a p e r i p h e r a l a f f er -
e n t, w e ca n , b y s u b s t i t u t i n g t h e c o r r e s p o n d i n g e x p r e s s i o n i n ( 1 ) f o r
e a c h Ns, , o r N~ , w h o s e n e u r o n i s n o t a p e r i p h e r a l a f f e r e n t , a n d r e -
p e a t i n g t h e p r o c e s s o n t h e r e s u lt , u l t i m a t e l y c o m e t o a n e x p r e s s i o n
f o r N ~ i n t e r m s s o le ly o f p e r i p h e r a l l y a f f e r e n t N , s i n c e ~ i s w i t h o u t
c i rc l e s. M o r e o v e r , t h i s e x p r e s s i o n w i ll b e a T P E , s i n c e o b v i o u s l y ( 1 )
i s ; a n d J t f o l lo w s i m m e d i a t e l y f r o m t h e d e f i n it io n t h a t t h e r e s u l t o f
s u b s t i t u t i n g a T P E f o r a c o n s t i t u e n t p ( z ) i n a T P E i s a l s o o n e .
T H E O R E M I I .
E v e r y T P E i s r ea l i zab le by a ne t o f o rder z ero .
T h e f u n c t o r S o b v i o u s l y c o m m u t e s w i t h d i s j u n c t i o n , c o n j u n c t i o n ,
a n d n e g a t i o n . I t is o b v i o u s t h a t t h e r e s u l t o f s u b s t i t u t i n g a n y S ~ ,
r e a l iz a b l e i n th e n a r r o w s e n s e ( i . n .s .) , f o r th e p ( z ) i n a r e a l ~ i z a b l e e x -
p r e s s i o n 1 i s i t s e l f r e a l i z a b l e i .n .s . ; o n e c o n s t r u c t s t h e r e a l i z i n g n e t b yr e p l a c i n g t h e p e r i p h e r a l a f f e r e n t s i n t h e n e t f o r 1 b y t h e r e a l i z i n g n e u -
r o n s i n t h e n e t s f o r t h e S ~. T h e o n e n e u r o n n e t r e a l iz e s p l ( z l ) i .n .s . ,
d e r , f o r t h o s e c a s e s w h e r e t h e r e i s n o r e f e r e n c e t o e v e n t s i n d e f in i te l y
f a r i n t h e p a s t i n t h e s p e ci f ic a t i o n o f t h e c o n d it io n s . B y w a y o f e x -
a m p l e , w e m a y c o n s i d e r t h e c a s e o f h e a t p r o d u c e d b y a t r a n s i e n t
c o o l i n g .
I f a c ol d o b j e c t i s h e ld t o t h e s k i n f o r a m o m e n t a n d r e m o v e d ,
a s e n s a t i o n o f h e a t w i l l b e f e l t ; i f i t i s a p p l i e d f o r a l o n g e r t i m e , t h e
s e n s a t i o n w i l l b e o n l y of c o ld , w i t h n o p r e l i m i n a r y w a r m t h , h o w e v e r
t r a n s i e n t . I t i s k n o w n t h a t o n e c u t a n e o u s r e c e p t o r i s a f f ec t e d b y h e a t ,
a n d a n o t h e r b y c o ld . I f w e le t N 1 a n d N ~ b e t h e a c t i o n s o f t h e r e s p e c -
t i v e r e c e p t o r s a n d N ~ a n d N 4 o f n e u r o n s w h o s e a c t i v i t y i m p l i e s a s e n -
s a t i o n o f h e a t a n d c old , o u r r e q u i r e m e n t s m a y b e w r i t t e n a s
N ~ ( t ) : ~- : N l ( t - 1 ) . v . N ~ ( t - 3 ) . ~ N ~ ( t - 2 )
N 4 ( t ) . = - -. N ~ ( t - 2 ) . N ~ ( t - 1 )
w h e r e w e s u p p o s e f o r s i m p l i c i ty t h a t t h e r e q u i r e d p e r s i s t e n c e i n t h e
s e n s a t i o n o f c o ld is s a y t w o s y a a p t i c d e l ay s , c o m p a r e d w i t h o n e f o r
t h a t o f h e a t . T h e s e c o n d i t i o n s c l e a r l y f a l l u n d e r T h e o r e m I I I . A n e t
m a y c o n s e q u e n t l y b e c o n s t r u c t e d t o r e a li z e t h e m , b y t h e m e t h o d o f
T h e o r e m I I. W e b eg i n b y w r i t i n g t h e m i n a f a s h io n w h i c h e x h ib i t s
t h e m a s b u i lt o u t o f t h e i r c o n s t i t u e n t s b y t h e o p e r a t i o n s r e a l i z e d i n
F i g u r e s l a , b , c , d : i. e. , i n t h e f o r m~ t ) . - = . S ~ N 1 t ) v S E S N ~ t ) ) . ~ N ~ t ) ]}N 4 t ) . = - . S [ S Y ~ t ) ] . Y .~ t ) ) .
F i r s t w e c o n s t r u c t a n e t f o r th e f u n c t i o n e n c lo s e d i n th e g r e a t e s t
n u m b e r o f b r a c k e t s a n d p r o c e e d o u t w a r d ; i n t h i s c a s e w e r u n a n e t o f
t h e f o r m s h o w n i n F i g u r e l a f r o m cz t o s o m e n e u r o n c a , s a y , s o t h a t
Na t) . ~ . S N 2 ( t ) .
N e x t i n t r o d u c e t w o n e ts o f t h e f o r m s l c a n d l d , b o t h r u n n i n g f r o m
Ca a n d c : , a n d e n d i n g r e s p e c t i v e l y a t o4 a n d s a y c b . T h e n
N 4 ( t ) . ~ . S [ N ~ ( t ) . N~_ ( t ) ] . - ~ . S [ ( S N 2 ( t ) ) . N ~ ( t ) ] .
N b ( t ) . = --. S [ N r . ~ N ~ ( t ) ] . ~ . S [ ( S ~ [ 2 ( t ) ) . oo N ~ ( t ) ] .
F i n a l l y , r u n a n e t o f t h e f o r m l b f r o m Cl a n d c b t o c 8 , a n d d e r i v e
N~ t). ~-. S [ ~ t ) v Nb t)]9 ~ . S ( N ~ ( t ) v S [ ( S N : ( t ) ] . ~ N ~ ( t ) ) .
T h e s e e x p r e s s i o n s f o r N ~ ( t ) a n d N ~ ( t ) a r e t h e o n e s d e s i r e d ; a n d t h e
r e a l i z i n g n e t i n t o t o i s s h o w n i n F i g u r e l e .
T h i s i l l u si o n m a k e s v e r y c l e a r th e d e p e n d e n c e o f t h e c o r r e s p o n -d e n c e b e t w e e n p e r c e p t i o n a n d t h e e x t e r n a l w o r l d u p o n t h e s p ec if ic
s t r u c t u r a l p r o p e r t i e s o f t h e i n C e rv e n in g n e r v o u s n e t . T h e s a m e i l l u-
s io n , o f c o u r s e , c o u ld a ls o h a v e b e e n p r o d u c e d u n d e r v a r i o u s o t h e r
a s s u m p t i o n s a b o u t t h e b e h a v i o r o f t h e c u t a n e o u s r e c e p to r s , w i t h c o r -
r e s p o n 4 i n g l y d i f f e r e n t n e t s .
W e sh a l l n o w c o n s i d e r s o m e t h e o r e m s o f e q u i v a l e n c e : i .e ., t h e o -
r e m s w h i c h d e m o n s t r a t e t h e e s s e n t i a l i d e n t i ty , s a v e f o r t im e , o f v a r i -
o u s a l t e r n a t i v e l a w s o f n e r v o u s e x c i t a t i o n . L e t u s f i rs t d i s c u s s t h e
c a s e o f relative inhibition B y t h i s w e m e a n t h e s u p p o s i ti o n t h a t t h e
f i ri n g o f a n i n h i b i t o r y s y n a p s e d o e s n o t a b s o l u t e l y p r e v e n t t h e f i r i n g
o f t h e n e u r o n , b u t m e r e l y r a i s e s i t s t h r e s h o ld , s o t h a t a g r e a t e r n u m -
b e r o f e x c i t a t o r y s y n a p s e s m u s t f ir e c o n c u r r e n t l y t o f ir e i t t h a n w o u l d
o t h e r w i s e b e n e e d ed . W e m a y s u p p o s e , lo s i n g n o g e n e r a l i t y , t h a t t h e
i n c r e a s e .in t h r e s h o l d i s u n i t y f o r t h e f i r i n g o f e ac h s u c h s y n a p s e ; w e
t h e n h a v e t h e t h e o r e m :
T H E O R E M IV .
Relative and absolute inhibition are equivalent in the extended
sense
W e m a y w r i t e o u t a l a w o f n e r v o u s e x c i t a t i o n a f t e r t h e f a s h i o n
o f ( 1 ) , b u t e m p l o y i n g t h e a s s u m p t i o n o f r e l a t iv e i n h i b i t i o n i n s t e a d ;
i n s p e c t i o n t h e n s h o w s t h a t t h i s e x p r e s s i o n i s a TPE A n e x a m p l e o ft h e r e p l a c e m e n t o f re l a t i v e i n h ib i t io n b y a b s o l u t e is g i v e n b y F i g u r e
l f . T h e r e v e r s e r e p l a c e m e n t i s e v e n e a s i e r ; w e g i v e t h e i n h i b i t o r y
a x o n s a f f e r e n t t o c~ a n y s u ff ic i en t ly la r g e n u m b e r o f i n h i b i t o r y s y n -
a p s e s a p i e c e .
S e c on d , w e c o n s i d e r t h e c a se o f e x t in c t io n . W e m a y w r i t e t h i s
i n t h e f o r m o f a v a r i a t i o n i n t h e t h r e s h o l d 0~ ; a f t e r t h e n e u r o n c~ h a s
f i re d ; to t h e n e a r e s t i n t e g e r - - a n d o n l y t o t h i s a p p r o x i m a t i o n i s t h e
v a r i a t i o n i n t h r e s h o l d s i g n if i ca n t in n a t u r a l f o r m s o f e x c i t a t i o n - - t h i s
m a y b e w r i t t e n a s a s e q u e n c e 0~ + b j f o r ] s y n a p t i c d e l a y s a f t e r f ir i n g ,
w h e r e b s ---- 0 f o r ~ l a r g e e n o u g h , s a y ] - - M o r g r e a t e r . W e m a y t h e ns t a t e
THEOREM V
Extinction is equivalent to absolute inhibition
F o r , a s s u m i n g r e l a t i v e i n h i b i ti o n t o h o l d f o r t h e m o m e n t , w e
n e e d m e r e l y r u n M c i rc u i t s T ~ , T , , . .. T . c o n t a i n i n g r e s p e c t i v e l y 1,
2 , - . . , M n e u r o n s , s u c h t h a t t h e f i r i n g o f e a c h l in k i n a n y i s s u f fi c ie n t
t o f ir e t h e n e x t , f r o m t h e n e u r o n c~ b a c k t o i t , w h e r e t h e e n d o f th e
c i r c u i t T j h a s j u s t b j i n h i b i t o r y s y n a p s e s u p o n c , . I t is e v i d e n t t h a tt h i s w i l l p r o d u c e th e d e s i r e d r e s u l t s . T h e r e v e r s e s u b s t i t u t i o n m a y b e
a c c o m p l i sh e d b y t h e d i a g r a m o f F i g u r e l g . F r o m t h e t r a n s i t i v i t y o f
r e p l a c e m e n t , w e i n f e r t h e th e o r e m . T o t h i s g r o u p o f t h e o r e m s a l so
b e l o n g s t h e w e l l - k n o w n
THEOREM V I
F a c i l i t a t i o n a n d t e m p o r a l s u m m a t i o n m a y b e r e pl a ce d b y s p a t i a l
s u m m a t i o n
T h i s i s o b v i o u s : o n e n e e d m e r e l y i n t r o d u c e a s u i t a b l e se q u e n c e
o f d e l a y i n g c h a in s , o f in c r e a s i n g n u m b e r s o f s y n a p s e s , b e t w e e n t h e
e x c i t i n g ce ll a n d t h e n e u r o n w h e r e o n t e m p o r a l s u m m a t i o n i s d e s i r e d
t o h o ld . T h e a s s u m p t i o n o f s p a t i a l s u m m a t i o n w i ll t h e n g i v e t h e r e -
q u i r e d r e s u l t s . S e e e .g . F, g u r e l h . T h i s p r o c e d u r e h a d a p p l i c a t i o n i n
s h o w i n g t h a t t h e o b s e r v e d t e m p o r a l s u m m a t i o n i n g r o s s n e t s d o e s n o t
i m p l y s u c h a m e c h a n i s m i n C he i n t e r a c t i o n o f in d i v i d u a l n e u r o n s .
T h e p h e n o m e n a o f l e a rn i n g , w h i c h a r e o f a c h a r a c t e r p e r s i s t -
i n g o v e r m o s t p h y s i o l o g ic a l c h a n g e s i n n e r v o u s a c t i v i ty , s e e m t o r e -
q u i r e t h e p o s s i b i l i ty o f p e r m a n e n t a l t e r a t io n s i n t h e s t r u c t u r e o f n e ts .
T h e s i m p l e s t s u c h a l t e r a t i o n i s t h e f o r m a t i o n o f n e w s y n a p s e s o r
e q u i v a l e n t l oc al d e p r e ss i o n s o f t h r e sh o l d . W e s u p p o s e t h a t s o m e a x -
o n a l t e r m i n a t i o n s c a n n o t a t f ir s t e x c i te t h e s u c c e e d i n g n e u r o n ; b u t i f
a t a n y t i m e t h e n e u r o n f ir es , a n d t h e a x o n a l t e r m i n a t i o n s a r e s i m u l -t a n e o u s l y e x c i te d , t h e y b e c o m e s y n a p s e s o f t h e o r d i n a r y k i n d , h e n ce -
f o r t h c a p a b l e o f e x c i t i n g t h e n e u r o n . T h e l o ss o f a n i n h i b i t o r y s y n -
a p s e g i v e s a n e n t i r e l y e q u i v a l e n t r e s u l t . W e s h a ll t h e n h a v e
THEOREM V I I .
A l t e r a b l e s y n a p s e s c a n b e r e p l a c e d b y c i r cl e s
T h i s is ac c o m p l i s h e d b y t h e m e t h o d o f F i g u r e l i . I t i s a l so t o b e
r e m a r k e d t h a t a n e u ro n w h i c h b e co m e s a n d r e m a i n s s p o n t a n e o u s ly
a c t i v e c a n l i k e w i s e b e r e p l a c e d b y a c ir c le , w h i c h i s s e t i n to a c t i v i t yb y a p e r i p h e r a l a f f e r e n t w h e n t h e a c t i v i t y c o m m e n c e s , a n d i n h i b i te d
b y o n e w h e n i t ce a s es .
I H T h e T h e o r y : N e t s w i t h C i rc le s
T h e t r e a t m e n t o f n e t s w h i c h d o n o t s a t i s f y o u r p r e v io u s a s s u m p -
t i o n o f f r e e d o m f r o m c ir c le s is v e r y m u c h m o r e d if fi cu lt t h a n t h a t c a se .
T h i s i s l a r g e l y a c o n s e q u e n c e o f t h e p o s s i b i l i t y t h a t a c t i v i t y m a y b e
s e t u p in a c i r c u i t a n d c o n t i n u e r e v e r b e r a t i n g a r o u n d i t f o r a n in -
d e f in i te p e r i o d o f t im e , s o t h a t t h e r e a l iz a b l e P r m a y i n v o l v e r e f e r -
e n c e t o p a s t e v e n t s o f a n i n d e f in i te d e g r e e o f re m o t e n e s s . C o n s i d e rs u c h a n e t ~ , s a y o f o r d e r p , a n d l e t c l , c ~, . . . , c~ b e a c y c li c s e t o f
n e u r o n s o f g ( . I t i s f i rs t o f a ll c le a r f r o m t h e d e f i n it io n t h a t e v e r y N ,
W A R R E N S . M C C U L LO C H A N D W A L T E R P I T T S 1 2 7
h o l d s f o r s o m e n e t . T h e s e w i ll b e c al le d p r e h e n s i b l e c l a s s e s . L e t u s
d e f i n e t h e B o o l e a n r i n g g e n e r a t e d b y a c l a s s o f c la s s e s ~ a s t h e a g g r e -
g a t e o f t h e c l a s se s w h i c h c a n b e f o r m e d f r o m m e m b e r s o f ~ b y r e -
p e a t e d a p p l i c a t i o n o f t h e l o g ic a l o p e r a t i o n s ; i .e ., w e p u t
( , , ) = p ' ~ [ ( ~ , ~ ) : ~ ~ K
- - ) a e ~ : a , ~ . - - ' . - a , a . , a v ~ ] .
W e s h a l l a l s o d e f in e
a n d
B
" a G O 9 - - . ~ ( , , ) - ~ ' p . . . . , , ,
~ ( , , ) = p ' ~ [ ((~ , ~ ) : a s k - - , a s , ~ . - ) .
R . ( D = ~ ( K ) - ~ ' p ' - " , , ,
- a , a . f l , a v f l , S a e ~ ]
a 9 , t ) - - - - - ~ [ m ) . + t + l , t , m ) = ~ m ) ] .T h e c la s s ~( ,(K ) i s f o r m e d f r o m K i n a n a l o g y w i t h ~ ( K ) , b u t b y r e -
p e a t e d a p p l i c a t i o n n o t o n l y o f t h e l o g ic a l o p e r a t i o n s b u t a ls o o f t h a t
w h i c h re p l a c e s a c l a s s o f p r o p e r t i e s P s a b y S ( P ) s S a . W e s h a l l
t h e n h a v e t h e
I m M M A
P r l (P l , P~ , "" , Pro, z l ) is a T P E i f a n d o n l y if
z ~ ) p ~ , . . . , p , ~ ) E pm + ~) : p ~ §( 1 3 )
Pm +l Z l ) ------- r l (p~ , P2 , , P,~ , z; )
i s t r u e ; a n d i t i s a T P E n o t i n v o l v i n g ' S ' i f a n d o n l y i f t h is h o l d s w h e n
' ~ e ' is re p l a c e d b y '~ ' , a n d w e th e n o b t a i n
T H E O R E M I X .
A s e r i e s o f c l a s s e s a l , a ~.,
a n d o n l y i f
( E ra ) ( E n ) ( p ) n ( i ) ( 9 ) : . ( x ) m ~ ( x ) z 0 v ~ ( x ) - - 1 : -+ : ( E f t )
( E y ) m . ~ ( y ) - - O . f l s ' t ~ [~ ( ( E i ) . ? ~ - a ~ ) ) . v . ( x ) m .
~ ( x ) - - O . f l s ~ [ r ( ( E i ) . r ~ - a i ) ] : ( t ) ( ~ ) : ~ e a ~ .
q ( ~ , n t + p ) . - ~ . ( E f ) . f e f l . ( w ) m ( x ) t - 1 .
~ ( n ( t + l ) + p , n x + p , w ) - - ~ f ( n t + p , n x + p , v ~ ) .
9 . as i s a s e r i e s o f p r e h e n s i b l e c l a s s es i f
The proof here follows directly fro m the lemma. The condition is
necessary, since every net for which an expression of the form (4)can be wri tte n obviously verifies it, the ~ s being the charact erist ic
functions of the S~ and the p for each ~ being the class whose desig-
nation has the form H P r ~ H P r s , where P r ~ denotes a~ fo r all k. Con-~ea ie Sa
versely, we may write an expression of the form (4) for a net g(
fulfilling prehensible classes satisfying (14) by putting for the Pra
P r deno ting {he ~ s, and a P r , written in the analogue for classes of
the disjunctive normal form, and denoting the a corresponding to tha t
, conjo ined to it. Since eve ry S of the form (4) is clearly realizable,
we have the theorem.It is of some ,interest to consider the extent to which we can by
knowledge of the present determine the whole past of various specialnets: i.e., when we may construct a net the firing of the cyclic set of
whose neurons requires the peripheral afferents to have had a set of
past values specified by given functions ~ . In this case the classes
at of the last theorem reduced to unit classes; and the condition may
be transformed into
E m , n ) p ) n i , ~f) E l ) : . (x)m: ~(x) - - - -0.v.~(x)--1 :
r + p ) : -~ : w ) m x ) t - 1 . ~ n t + 1)+ p , n x + p , w ) ~ - ~ j n t + p , n x + p , w ) : .
u , v ) w ) m . g )~ n u + 1) + p, nu + p, w)
- - r l ) + p , n v + p , w ) .
On account of limitations of space, we have presented the above
argument very sketchily; we propose to expand it and certain of its
implications in a further publication.
The condition of the last theorem is fairly simple in principle,
though not in deta`il; its application to practical cases would, however,
require the exploration of some 22. classes of functions, namely themembers of ~({a~ , .. . , a~)). Since each of these is a possible fl ofTheorem IX, this res ult canno t be sharpened. But we may obtain a
sufficient condition for the realizability of an S which is very easily
applicable and probably covers most practical purposes. This is given
y
THEOREM X
Let us define a set K of S by the following recursion :
1. Any T P E and any T P E whose arguments have been replacedby members of K belong to K;
2 . I f P r l z l ) is a member of K, then z ~ ) z l . Prl(z~), E z ~ ) z l .
P r l z ~ ) , a n d C ~ . ( z l) . s b e l o n g to i t , w h e r e C ~ d e n o t e s t h e p r o p e r t y
o f b e i n g c o n g r u e n t t o m m o d u l o n , m < n .
3 . T h e s e t K h a s n o f u r t h e r m e m b e r s
T h e n e v e r y m e m b e r o f K i s r e a li z ab l e .
F o r , i f P r l ( z l) i s r e a l iz a b l e , n e r v o u s n e t s f o r w h i c h
N i z ~ ) . = -- . P r l z l ) . S N ~ z ~ )
N ~ z ~ ) . = . P r , z l ) v S N ~ z ~ )
a r e t h e e x p r e s s i o n s o f e q u a t i o n ( 4 ) , r e a li z e z ~ )z ~ . P r l z D a n d
E z ~ ) z l . P r ~ z D r e s p e c t i v e l y ; a n d a s i m p l e c i r c u i t , c ~ , c~ , . . . , e ~ ,o f n l i n k s , e a c h s u f f ic i e n t t o e x c i t e t h e n e x t , g i v e s a n e x p r e s s i o n
N m z ~ ) . - - . N I O ) . C ~
f o r t h e l a s t f o r m . B y i n d u c t i o n w e d e r i v e t h e th e o r e m .
O n e m o r e t h i n g i s t o b e r e m a r k e d i n c o n c l u s io n . I t i s e a s i l y
s h o w n : f ir s t, t h a t e v e r y n e t , i f f u r n i s h e d w i t h a t a p e , s c a n n e r s c o n -
n e c t e d to a f f e re n t s , a n d s u i t a b l e e f f e r e n ts t o p e r f o r m t h e n e c e s s a r y
m o t o r - o p e r a t i o n s , C all c o m p u t e o n l y su c h n u m b e r s a s c a n a T u r i n g
m a c h i n e ; s ec o n d , t h a t e a c h o f t h e l a t t e r n u m b e r s c a n b e c o m p u t e d b y
s u c h a n e t ; a n d t h a t n e t s w i t h c i r c le s c an b e c o m p u t e d b y s u c h a n e t ;a n d t h a t n e t s w i t h c i r c l es c a n c o m p u t e , w i t h o u t s c a n n e r s a n d a t a p e ,
s o m e o f t h e n u m b e r s t h e m a c h i n e c a n , b u t n o o th e r s , a n d n o t a l l o f
t h e m . T h i s is o f i n t e r e s t a s a f f o r d i n g a p s y c h o l o g i c a l j u s t i f i c a t i o n o f
t h e T u r i n g d e f in i ti o n o f c o m p u t a b i l i t y a n d i ts eq u i v a l e n ts , C h u r c h ' s
~ d e f i n ab i l i t y a n d K l e e n e ' s p r i m i t i v e r e c u r s iv e n e s s : I f a n y n u m b e r
c a n b e c o m p u t e d b y a n o r g a n i s m , i t is c o m p u t a b l e b y t h e s e d e f i ni ti o ns ,
a n d c o n v e r s e l y .
I V . C o n s e q u e n c e s
C a u s a l i t y , w h i c h r e q u i r e s d e s c r i p t i o n o f s t a t e s a n d a l a w o f n e c -
e s s a r y c o n n e c t i o n re l a t i n g t h e m , h a s a p p e a r e d i n s e v e r a l f o r m s i n
s e v e r a l s c ie n c e s , b u t n e v e r , e x c e p t i n s ta t i s t i c s , h a s i t b e e n a s i r r e -
c i p r o c a l a s i n t h i s t h e o r y . S p e c i f i c a ti o n f o r a n y o n e t im e o f a f f e r e n t
s t i m u l a t i o n a n d o f t h e a c t i v i t y o f al l c o n s t i t u e n t n e u r o n s , e a c h a n
a l l - o r - n o n e a f f a i r , d e t e r m i n e s th e s t a t e . S p e c i f i c a ti o n o f t h e n e r -
v o u s n e t p r o v i d e s t h e l a w o f n e c e s s a r y c o n n e c t i o n w h e r e b y o n e c an
c o m p u t e f r o m t h e d e s c r i p t io n o f a n y s t a t e t h a t o f t h e s u c c e e d i n g
s t a te , b u t t h e i n c lu s i o n o f d i s j u n c t i v e r e l a t i o n s p r e v e n t s c o m p l e t e d e -
t e r m i n a t i o n o f t h e o ne b e f o r e . M o r e o v e r , t h e r e g e n e r a t i v e a c t i v i t yo f C o n s t it u e n t c i r c le s r e n d e r s r e f e r e n c e i n d e f i n i t e a s t o t i m e p a s t .
T h u s o u r k n o w l e d g e o f th e w o r l d , i n c lu d i n g o u r s e lv e s , i s i n c o m p l e t e
a s to s p a c e a n d i n d e f in i t e a s t o t i m e . T h i s i g n o r a n c e , i m p l i c i t i n a l l
o u r b r a in s , i s t h e c o u n t e r p a r t o f t h e a b s t r a c t i o n w h i c h r e n d e r s o u r
k n o w l e d g e u s e fu l . T h e r o le o f b r a i n s i n d e t e r m i n i n g t h e e p is t e m i c
r e l a t io n s o f o u r t h e o r i e s t o o u r o b s e r v a t i o n s a n d o f t h e s e t o th e f a c t s
i s a ll to o c le a r, f o r i t is a p p a r e n t t h a t e v e r y i d e a a n d e v e r y s e n s a t i o n
is r ea l iz e d b y a c t i v i t y w i t h i n t h a t n e t, a n d b y n o su c h a c t i v i t y a r e t h e
a c t u a l a f f e r e n t s f u l ly d e t e r m i n e d .
T h e r e is n o t h e o r y w e m a y h o l d a n d n o o b s e r v a t i o n w e c a n m a k e
t h a t w i l l r e t a i n s o m u c h a s i ts o ld d e f e c t iv e r e f e r e n c e t o t h e f a c t s i f
t h e n e t b e a l t e re d . T i n i t u s , p a r a e s t h e a i a s , h a l l u c i n a t io n s , d e l u s io n s ,
c o n f u s i o n s an d d i s o r ie n t a t i o n s i n t e r v e n e . T h u s e m p i r y co n f i r m s t h a t
i f o u r n e t s a r e u n d e f i n e d , o u r f a c t s a r e u n d e f in e d , a n d t o t h e r e a l
w e c a n a t t r i b u t e n o t s o m u c h a s o n e q u a l i t y o r form. W i t h d e t e r m i -
n a t i o n o f t h e ne t , t h e u n k n o w a b l e o b j e c t o f k n o w l e d g e , th e t h i n g i n
i t s e l f , c e a s e s t o b e u n k n o w a b l e .
T o p s y c h o l o g y , h o w e v e r d ef in e d , sp e c i f ic a t io n o f t h e n e t w o u l d
c o n t r i b u t e a ll t h a t c o u l d b e a c h i e v e d i n t h a t f ie ld e v e n i f t h e a n a l y s i s
w e r e p u s h e d t o u l t i m a t e p s y c h i c u n i t s o r p s y c h o n s , f o r a p s y c h o n
c a n b e n o le s s t h a n t h e a c t i v i t y o f a si n g l e n e u r o n . S i n c e t h a t a c t i v i t y
i s i n h e r e n t l y p r o p o s i t i o n a l , a ll p s y c h i c e v e n t s h a v e a n i n t e n t io n a l , o r
s e m i o t i c , c h a r a c t e r . T h e a l l - o r - n o n e l a w o f t h e s e a c t i v i ti e s , a n dt h e c o n f o r m i t y o f t h e i r r e l a t i o n s t o t h o s e o f th e l og ic o f p r o p o s i t i o n s ,
i n s u r e t h a t t h e r e l a ti o n s o f p s y c h o n s a r e t h o s e o f t h e t w o - v a l u e d l o g ic
o f p r o p o s i t io n s . T h u s i n p s y c h o l o g y , i n t r o s p e c ti v e , b e h a v i o r i s t ic o r
p h y s i o lo g i c a l, t h e f u n d a m e n t a l r e l a ti o n s a r e t h o s e o f t w o - v a l u e d l og ic .
EXPRESSION FOR THE FIGURES
In the figure the neuron c~ is always marked with the numeral i upon the
body of the cell, and the corresponding action is denoted by N with i as sub-
script, as in the text.
F i gu r e l a N ~ ( t ) . ~ -- . N l ( t - - 1)
F i g u r e l b N z ( t ) . ~ . N l ( t - - 1 ) v N ~ ( t - - 1 )
Ffgure l c N ~ ( t ) . ~ . N ~ ( t - - 1 ) . N . e ( t - - 1 )
F i g u r e l d N 3( t ) . ~ . N ~ ( t - - 1 ) . o ~ N a ( t - - 1 )
Fig ure le N3(t) : ~ : N ~ ( t - - 1) . v. N ~ ( t - - 3). N2 t _ 2)
N 4 ( t ) . ~ . N 2 ( t - - 2) . N~ ( t - - 1 )
F i g u r e l f N 4 ( t ) : ~ : e ~ N ~ ( t - - 1 ) . N ~ ( t - - 1) vNa(t - - 1) .v .N ~( t -- 1).
N ~ ( t - - 1). N 3 ( t - - 1)
N 4 ( t ) : - ~ : c o N ~ ( t - - 2). N ~ ( t - - 2:) v N ~ ( t - - 2) . v 9 N ~ ( t - - 2).
N ~ ( t - - 2). N 3 ( t - - 2)
Figure lg N a ( t ) . - ~ . N ~ ( t - - 2). ~ N ~ ( t - - 3)F i g u r e l h N 2 ( t ) . - ~ . N ~ ( t - - 1) . N~ ( t - - 2 )
F igure l i N 3 ( t ) :~--: N ~ ( t - - 1) . v . N~ ( t - - 1 ) . ( E x ) t - - 1 . N ~ ( x ) . N , ( x )