NATIONAL PHYSICAL LABORATORY SYMPOSIUM No. 10 Mechanisation of Thought Processes Proceedings of a Symposium held at the National Physical Laboratory on 24th, 25th, 26th and 27th November 1958 VOLUME I LONDON : HER MAJESTY'S STATIONERY OFFICE 1959 (Reprinted 1962)
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
NATIONAL PHYSICAL LABORATORY
SYMPOSIUM No. 10
Mechanisation of Thought Processes
Proceedings of a Symposium held at
the National Physical Laboratory
on 24th, 25th, 26th and 27th November 1958
VOLUME I
LONDON : HER MAJESTY'S S T A T I O N E R Y O F F I C E
1 9 5 9
(Reprinted 1962)
SESSION 3
PAPER 6
P A N D E M O N I U M : A P A R A D I G M F O R L E A R N I N G
by
D?. 0. G. S E L F R I D G E
Cllver G. Selt'rlci~e was born I n London 10 M?;; 15£'C He studied 2t the Massachusetts Institute of Tfcci-iriology frorr. 192-1945, returning post- ~-rar i .da tc j .y ??or I94?-1950. Alter 2 y e a r s a t S! gna l Corzs Laticrstorl e s at Fort Mor.mouth. Ncv Jersey, he 2 o l n e d Lincoln Labora- t o r i e s In GX'JD 34, Corr'iunicatlon Techniques, of W:-.lch he
Is now Group Leader.
PANDEMONIUM: A PARADIGM FOR L 7 A R M M
DR. 0. G. SELFRIDGE
INTRODUCTION
WE a r e proposing he re a model o f a process which we claim can a d a p t i v e l y improve i t s e l f t o handle c e r t a i n p a t t e r n r ecogn i t ion problems which cannot be adequate ly s p e c i f i e d In advance. Such p rob ie r s a r e usua l when t r y i n g t o b u i l d a macnine t o I m i t a t e any one of a very l a r g e c l a s s of human d a t a process ing techniques. A speech t y p e w r i t e r I s a good example of soneth ing chat very many people have been t r y i n g unsuccess fu l ly t o b u i l d f o r some time.
We do n o t sugges t t h a t we have proposed a model vhlcn can l e a r n t o typewri te from merely h e a r i n g speech. Pandemonium does no t , however, seem on paper t o have the same k inds of l nne ren t r e s t r i c t i o n s o r i n f l e x i b i l i t y t h a t many previous proposals have had. Trie b a s i c motif Dehind ou r model i s the n o t i o n of p a r a l l e l process ing . This i s suggested on two grounds: f i r s t , i t i s o f t e n e a s i e r t o handle d a t a In a p a r a l l e l nanner, and, Indeed, I t is u s u a l l y the more " n a t u r a l " manner t o handle i t in; and, secondly, i t I s e a s i e r t o modify an assembly of quasi-independent modules than a machine a l l of whose p a r t s i n t e r a c t immediately and in a complex way.
We a r e n o t going to apologize f o r a f requent use 01 antnropomorphic o r ~ i o m o r p h i c terminology., They seem t o be u s e f u l words t o desc r ibe our not ions .
What we a r e descr ib l r ig i s a process , or , r a t h e r , a model of a process. We s h a l l no? desc r ibe all t P n reasons thac l e d t o i t s particular formulation, out we s h a l l give some reasons f o r noping t h a t i t does lr f a c t possess the f l e x i b i l i t y and a d a p t a b i l i t y t n a t we a s c r i b e t o i t .
THE PROBLEM ENVIRONMENT FOR LEARNING
PandeTionium i s a -nodel hrhicn vie nope can l e a r n t o recognize p a t t e r n s which nave n o t been s p e c i f i e d . We nean t n a t i n cue f o l i o Jing sense: we p resen t t o the model examples of p a t t e r n s taken from some s e t of theri, each time I n f o m i n g tne nodel which p a t t e r n we n i d j u s t presented. Then, a f t e r some time tne model guesses Cor rec t ly which p a t t e r n has J u s t been presented beft -d
we inform i t . For a l s r g e c l a s s of p a t t e r n r e c o m i t l o n ensembles there has never e x i s t e d any adequate w r i t t e n o r s t a t a b l e d e s c r i p t i o n of t he d i s t l n c - t i o n s between the pa t t e rns . The on ly requirement '/le can p l a c e on our modal I s t h a t ;-;e want i t t o benave i n tC.5 same '::ay t h a t Xen observably behave In. In an abso lu te sense t h i s I s a v e r y u n s a t i s f a c t o r y de l ' i n l t i on of any task:, but i t may be appa ren t t h a t i t I s the ':;a; In which most t a s k s a r e def ined f o r most men. Lucky is he whose job can 'oe e x a c t l y s p e c i f i e d i n words withouc any ambiguity or necessa ry Inferences . The example we s h a l l I l l u s t r a t e i n some d e t a i l is t r a n s l a t i n g from manually keyed Morse code Into, say, t ypewr i t t en messages. Now i t I s t r u e t h a t when one l e a r n s Morse code one l e a r n s t h a t a dash should b s e x a c t l y th ree t imes the l e n g t h of a dot and so on, but i t t u r n s ou t t h a t cl-.ls I s r e a l l y mos t ly I r r e l e v a n t . What m a t t e r s is only what tne v a s t army of people who use Morse code and with whom one I s going to have t o communicate unders tand and p r a c t i s e when they u s e i t . I t t u r n s ou t t h a t t h i s Is n e a r l y alw?ys ve ry d i f f e r e n t from school book Morse.
In the same way the only adequate d e f i n i t i o n of the p a t t e r n of a spoken word, o r one hand-written, must be In terms of the consensus of the people who a r e u s i n g i t .
We use the term p a t t e r n r e c o g n i t i o n In a broad sense t o inc lude n o t only t h a t d a t a process ing by which images a r e ass igned t o one o r another p a t t e r n . in some s e t of p a t t e r n s , b u t a l s o the p rocesses by which the p a t t e r n s and d a t a proc,essing a re developed b y the organism .er machine; we g e n e r a l l y c a l l t h i s l a t t e r " l e a r n i n g f f e
PANDEMONIUM, IDEALIZED AND PRACTICAL
We first c o n s t r u c t an i d e a l i z e d pandemonium (figa 1 ) . Each poss ib l e P a t t e r n o f the s e t , r ep resen ted by a demon In a box, computes h i s s i m i l a r i t y w i t h tne image s imul taneously on view t o a l l o f them. and g ives an output depending monotonlca l ly on t h a t s i m i l a r i t y . The d e c i s i o n demon on t o p makes a choice of t h a t p a t t e r n belonging t o the demon whose output was the l a r g e s t . *
* This I s an e x a c t c o r r e l a t e of a c c m u n i c a t . l o n s sys tem ,;.herein given a r e c e i v e d message M W and a n- int tier of poss ib l e t r a n s m i t t e d messages !'.(!), t h a t . " 1s
chosen, t h a t I s , deemed t o have been t r a n s m i t t e d , ~ i h l c h minimizes ~ v c T ) - ~ ~ ~ ( I ) I2dI. (Such a procedure I s opt imum under c e r t a i n c o n d i t i o n s ) . T h i s I n t e g r a l Is, a s I t were, t he squa re of a d i s t a n c e in a s i g n a l phase space - f ie .2 - and t h u s t h a t trans-ri l t te ' i ne s sage I s s e l e c t e d t h a t Is most s i n l l a r t o t he r e c e i v e d one.
PICKS THE LARGEST OUTPUT FROM THE
COGNITIVE DEMONS, WHO INSPECT THE
CATA OR IMAGE.
Each demon may, f o r example, be ass igned one l e t t e r of the a lphabet , so t h a t the task: of the A-demon I s t o shout a s loud of the amount of 'A-nessl t h a t he s e e s In the image.* Now i t w i l l u s u a l l y happen t h a t w i th a reasonable c o l l e c t i o n ol' c a t e g o r i e s - l i k e the l e t t e r s of the a lphabe t - the conputa- l i o n s cerformed by each o f t h e s e idea l cogn i t ive demons w i l l be l a r g e l y the same. In many Ins tances a p a t t e r n I s n e a r l y equ iva len t t o some l o g i c a l funct ion of a s e t of f e a t u r e s , each of which is i n d i v i d u a l l y comon t o perhaps s e v e r a l p a t t e r n s and whose absence is a l s o comon t o s e v e r a l o the r pa t t e rns . ?
t he re f o r anend our i d e a l i z e d Pandemcnium. The amended ve r s ion - fie. 3 - has scme profound advantages, ch ie f among which is i t s suscept- i b i l i t y t o t h a t kind of adap t ive self-improvement t h a t I c a l l learning.
The d i f f e r e n c e between f i g . 1 and f i g .3 is t h a t the common p a r t s of the computations t h a t each c o g n i t i v e demon c a r r i e s o u t i n f i g . 1 have in f i g . 3 been a s s igned i n s t e a d t o a h o s t of subdemons. A t t h i s s t a g e the organiza- t i c n has four l e v e l s . A t the bottom the d a t a demons se rve merely t o s t o r e and pass on the da t a . A t the nex t l e v e l the computational demons o r sub- demons perform c e r t a i n more o r l e s s complicated computations on the d a t a and pass t h e r e s u l t s of t hese up t o the n e x t l e v e l , the cogn i t ive demons who weigh the evidence, a s i t were. Each cogn i t ive demon computes a s h r i e k , and from a l l the s h r i e k s the h i g h e s t l e v e l demon of a l l , the dec i s ion demon, mere ly s e l e c t s the loudes t .
THE CONCEPTION OF PANDEMONIUM
We cannot ab I n i t i o know the i d e a l cons t ruc t ion of our Pandemonium. We t r y t o a s s u r e t h a t I t con ta ins the seeds of self-improvement. Of the f o u r l e v e l s i n fip. 3. a l l bu t the t h i r d , t he subdemons, which compute, a r e s p e c i f i e d by the task. For the t h i r d l e v e l , t he re fo re , we c o l l e c t a l a r g e number of poss ib l e u s e f u l func t ions , e l lmlna t lng a p r i o r i Only those which could n o t conceivably be r e l e v a n t , and make a reasonable S e l e c t i o n of the o t h e r s , be ing bound by economy and space. We then guess reasonable weights f o r them. The behaviour a t t h i s poin t may even De accep tab ly good, bu t Usual ly i t must be improved by means of the adap t ive changes we are about t o d iscuss .
* I t I s s o s s l b l e a l s o t o ph rase I t so t h a t t h e A-denon I s concutlng the d i s t a n c e In sone phase of t h e l r a g e from scne Idea l A; I t seems t o m ' Jnnecessar l ly p l a t o n i c t o p o s t u l a t e the e x i s t e n c e o r ' I d e a l 1 r e p r e s e n t a t i v e s o f l a t t e r n s , ana , Indeed, t h e r e a r e o f t e n good r easons f o r n o t d o l n s so .
/Â See, fo r example, J e r o n e Bruner, % s t u d y of Thlnkl~Ig*,
COGNITIVE DEMONS
COMPUTATIONAL DEMONS
DATA OR IMAGE DEMONS
The Evo lu t i on of Pandemonium
There a r e seve ra l k inds of adapt ive changes which we w i l l d i scuss fo r our 7 Pandemonlim. They a r e a l l e s s e n t i a l l y very s i m i l a r , b d t tney nay be p r o g r m e a and discussed separa te ly . The first nay be c a l l e d "Feature Weighting".
Altnough we have n o t y e t s p e c i f i e d what tne cogn i t ive denons conpute, the s o l e t a s k a t present is t o add a weignted SLCT o trie ou tpu t s o f a l l t3e computational demons o r s-Jbdenons; the weightings w i l l of course d i f f e r for
Flg.4. F i r s t hill c l imb ing technique: p i c k v e c t o r s a t rand= ( p o i n t s In the s p a c e ) , s co re tre'n, ard s e l e c t the one t h a t s c a r e s h ighes t .
the d i f f e r e n t cogn i t ive demons, bu t the weight ings w i l l be the only d i f f e r e n c e between them. Feature weight ing Cons i s t s of a l t e r i n g the weigy.ts ass igned to t he subdemons by t h e c o g n i t i v e demons s o a s t o maximize the sco re of the e n t i r e 7 Pandemonium. How then can we do t h i s ?
The Score
What we mean b y the sco re nere I s how well the machine i s doing tne tasA we want i t t o do. This presumes t h a t we a r e moni tor ing the machine and t e l l i n g i t when I t makes an e r r o r and so on, and f o r the r e s t of the d i scuss ion we s n a i l be assuming t'qat we have a v a i l a b l e some such running sco re . Now a t sone po in t we s n a i l be very i n t e r e s t e d i n Having the machine r J n .-.'Ithout t z a t k ind of d i r e c t supe rv i s ion , aKd the ques t ion n a t u r a l l y a r i s e s w r e t r e r the rnacr-ine car. mean ingfu l ly monitor i t s own perforinance. Je answer t h a t ques t ion t e n t a t i v e l y yes, b ' ~ t de l ay d i scuss ing i t t i l l a l d c e r s ec t ion .
~ l g - 5 . secozd n l l l - c l lmblng techniiiue: pick vectors until o f the--, (lpuxltter 4) outscores the previous ones. Then take sho re i ardoj-l steps, retracing any that decrease the sco re .
Feature i feightang and Hi. Ll-Cl t'mbing
The output of any cogn i t ive demon i s
so t h a t the complete s e t o f weights f c r a l l t he cogn i t ive de-nons over a i l the subdemons I s a vec to r ;
Now f o r some (unknown) s e t of weights A, the behaviour of t h i s whole Pandenonimi I s optimum, and the problem of f e a t u r e weighting I s t o f ' ina
MAIN PEAK
KS
Flg.6. General space showing f a l s e peaks . One of t he f a l s e peai ts ~ 3 1 1 t e Isolated f r o " ^ t h e na in o? t rue peak-
t h a t s e t . This l a y be desc r ibed a s a h i l l - c l i m b i n g ~ ? r r t l e m . We have a space (of A) and a funct ion on the space ( t h e s c o r e ) , which we piay cons ider an a l t i t u d e , and whicn \:e ,vlsh t o maximize by a proper s ea rch through A. one poss ib l e technique I s t o s e l e c t weight ing v e c t o r s a t random, s c o r e tnem, and f i n a l l y t o s e l e c t the vec to r t n a t s co red h i g h e s t ( s ee f\g. 4). I t w i l l u sda l ly , however, t u rn o u t t o be p r o f i t a b l e to t ake ~ d v a n t a g e of the c o n t i n u i t y p r o p e r t i e s of tne space, which u s u a l l y e x i s t In some sense, i n t r e following way: s e l e c t v e c t o r s a t random u n t i l you f ind one t n a t s co res perceptiblymorfc tran the o t rers , ?-an t h i s point take s n a i l randor s t e p s In a l l d i r e c t i o n s ( t h a t I s , add smal l random v e c t o r s ) m t i l you f i n d a d i r @ c t ' o n t h a t improves your score. When you f i n e sucn a s t ep , take i t and r epea t the process. Tnis I s I l l u s t r a t e d In fq. 5 , and i s the case of a b l i n d man t r y i n g t o climb a h i l l . There may ce, of course, TanJ f a l s e peaks on wnicn one may f i n d onese l f t rapped In sdcn a procedure {fig. E).
' Ee prol-'le-"; of r a i s e p e x s in searc.'-.in% tec.'.nIc.ues I s an o ld an;; f a n l l i a r one. In gene ra l , one may hope ti-.a: in spaces of ve ry ::i3;h dlxen- s i o n a l i t y t ; ~ e Interdependence c : m e componer.ts and the score is so g r e a t a s t o inE.<e ve ry Qnl lKely L!".? ex i s t ence of f a l s e peaks c o n c l e t e l y I s o l a t e d from Cue m i n o r tree peak;. I t n u s t be r e a l i z e d , however, t h a t :rils is a pu re ly exps r l -~ . e r . t a l ~ ' - i e s t l o n t h a t has to be answered s e p a r a t e l y :or every hil l-cl1rn~~i1-g s i t u a t i o n . I t 50:s tl-irn out In l- . i l l -cl in3ing siti-i?tlor.s tl-.at the choice of s t a r t i n g point I s oc ten very important . The main peak may b e
very prominent, but u n l e s s I t n a s wide-spread foot-3111s it nay r ace a very long t i n e before we eve r 3ezin to ga in a l t i t u d e .
T'nis w.vf b e descr ibed a s or.? of the proolems of tr?.lnirig, n m e l y , t o encourage the nac'nine o r organism t o s e t enough on the f o o t - 5 i l l s so t h a t smal l changes in h i s Darar.etprs d i l l produce n o t i c e a b l e Improvenent i n h i s a l t i t u d e o r score , One can desc r ibe l e a r n i n g s i t u a t i o n s where n o s t of the d i f f i c , ~ l t y o: toe t a s k l i e s ir. f i n d i n g a s y ',-:ay o f irriproving onef s score , s x n a s l e a r n i n g t o r i d e a un lcyc le , where I t takes longer t o s t a y on f o r a second than I t does t o i m ~ r c v e t n a t one second to a minute; and o t h e r s where I t is easy t o do a l i t t l e well and v e r y &rd t o do very wel l , sucn a s l e a r n i n g to p l a y c5ess. I t i s a l s o t r u e t h a t o f t e n tne n a i n geai? i s I ~ l a t e a u r a t h e r than an i s o l a t e d s?i;<eà That i s t o say, o p t i n a l behaviour 01' the inecLanIsm, once re?-ched, pi5y be r a t h e r i n s e n s i t i v e t o the change of some of the parameters,
Subdemon S e l e c t i o n
The second kind of adap t ive change t h a t we wish t o inco rpora t e i n t o ou r Pandemcnluii I s subdemon s e l e c t i o n . A t the conception c f our demoniac assembly vre c o l l e c t e d sonewhat a r b i t r a r i l y a l a r g e nunber of subdemons :;'rile?. we guessed w w l d be u s e f u l and a s s igned thex weights a l s o a r b i t r a r i l y . The f i r s t adapt lve c?ange, f ea tu re weighting. optimlze-i t hese weights. but we have no assurance a t a l l t ha t the p a r t i c u l a r subdemons we s e l e c t e d a re good ones. Sutdemon s e l e c t i o n gene ra t e s new subdemor.~ f o r t r i a l and e l i m i n a t e s i n e f f i c i e n t ones, m a t , i s , ones t h a t ao n o t much h e l p improve the score.
We gropose t o do t h i s i n i t i a l l y by two di:f'.ferent techniques, which may be c a l l e d "mutated f i s s i o a t l and Nconjugat ion" . The first po in t t o note i s t h a t I t I s p o s s i b l e t o a s s ign a worth to each of the s u b d e m s . I t may t e done in seve ra l bays, and we nay, f o r example, wr i t e the worth I$ of t?.e i t h demon
so t h a t tine worthy demons are those hose o u t p u t s a r e l i k e l y t o a f f e c t most s t r o n g l y the dec i s ions ;nade.
We assume t h a t f e a t u r e weight ing h a s a l r e a d y run s o long t h a t the behaviour of the machine has been approximately optimized, and t h a t s c o r e s and worths of xiachine and i t s demons have been obtained. First we e l imina te those subdemons wi th low worths. Next we gene ra t e new subdemons by muta t ing the s u r v i v o r s and reweight ing the assembly. A t p r e sen t we p l an t o p i c k one subdemon and a l t e r some of h i s parameters more o r l e s s a t random. This will u s u a l l y r e q u i r e t h a t we reduce the subdemon himsel f t o some canon ica l form so t h a t the random changes we i n s e r t do n o t have the e f f e c t of r e w r i t i n g the program s o t h a t i t w i l l n o t run o r so t h a t i t w i l l e n t e r a c losed loop wi thout any hope o f g e t t i n g o u t of it.*
Besides muta Led f i s s i o n , we a r e proposing another method of subdeinon improvement c a l l e d l tconjugat ionl l . ou r purpose here is two-Sold: f i rs t t o provide a l o g i c a l v a r i e t y in the func t ions computed by the subdexions, and. secondly, t o provide l e n g t h and complexity In them.
What we do is t h i s : glven two ' u s e f u l ' subdemons. we genera te a new subdemon which is the continuous analogue of one of t he t e n n o n t r i v i a l b i n a r y two-variable func t ions on them. For example, the product of two subdemon output S, corresponding t o the l o g i c a l product , would Suggest trie simultaneous presence of two f ea tu res . The t en n o n - t r i v i a l such func t ions a r e l i s t e d In Table I.
A . BVwA .3 A.?13v-A B Table 1. Non-trivial b i n a r y functions on two var iab les .
CORRECT DECISION '\ (FROM ME)
COMPUTING SUBDEMONS
EVEN -NUMBERED DURATIONS ARE MARKS LETHE ODD- NUMERED DURATIONS ARE SPACES
A S we mentioned before , the e n t i r e n o t i o n of Pandemonium ,-iids conceived 2 S CL i r a c t i c a l way o f au tomat i ca l ly improving data-processing f o r p a t t e r n recogni t ion . our i n i t i a l model t a s k I s to d i s t i n g u i s h :dots and dashes In manually keyed Horse code, so t h a t ou r Pandemonium can be I l l u s t r a t e d In fig. 7. Vote that . the f l s . c t i o n s and behaviour of a l l de.nons have 'ceen s p e c i f i e d except f o r the computing subdemons. We s h a l l r e i t e r a t e tnose spec1 f i ca t ions .
(1) The dec i s ion demon1 s output I s ' d o t 1 o r ' da sh ' according a s tr.e d o t demon's output is g r e a t e r o r l e s s than ine dasfi demon's. (2 ) The cogni t ive demons, d o t and dash, each compute a ',deignted sum of the ou tpu t s of some 150 computing subdeinons. I n i t i a l weights we 'nave ass igned a r b i t r a r i l y , bu t , we hope, reasonably. (5) The data-handling demons r ece ive d a t a In me form of du ra t ions , a1 t e rna t ive l ; c ? marks and spaces, and they pass them ?own the l i n e .
OUT PUT
0 1 - 5 '0 / ̂
Tne computing sxbdernons a r e cons t ruc t ed Tram on ly a ve ry few o p e r a t i o n a l funct ions , which a r e c a r e f u l l y non-binary. For example the subdemons d = d p and do dr , hive t h e i r ou tpu t s shown i n fzg.3. The operational , functions
L
f o l l o w 1 ) l = ' . This funct ion computes the degree of e q u a l i t y of some s e t o l v a r i a b l e s ( see ftg, 5). (2) ' ' , l , compute the aegree to which some v a r i a b l e i s l e s s than o r 1s g r e a t e r than some o the r v a r i a b l e (see f izh 8). (3) 'm&, ' m a x ' , compute the degree t o which some v a r i a b l e I s the l a r g e s t of an a r b i t r a r y s e t of v a r i a b l e s o r an a r b i t r a r y s e t of consec- u t i v e v a r i a b l e s , (4) '(lL , t A t s t o x 1222 degree t o ~.&iIcF. :i;e izh d ~ i ~ a ~ l o r ~ h s been I d e n t i f i e 6 as a dot o r dash. (5) "W computes 2n average o? socie s e t 01 v a r i a b l e : , (S) ', '/I is a family of t r ack iEg means. For example, i t ~'iis::.t co;r@lite
CONCUJSI ON
',&at I s h a l l present a t the meet ing In November w i l l be the d e t a i l s of the p rogres s of Pandemonium on the Yorse code t r a n s l a t i o n problem. The i n i t i a l emblem we have given the machine I s t o dIst ln:?uIsh d o t s and dashes. When the behaviour of the machine has Improved I t s e l f to the p o i n t where l i t t l e f u r t h e r Improvement seems t o be occurr ing , we s n a i l add th ree more cogn i t ive demons, the symbol space, the l e t t e r space, and t h e w r d space. Presumably a f t e r sone f u r t h e r time t h i s new Pandemonium will s e t t l e down t o some unlriprovable s t a t e . Then we s h a l l r e p l a c e the s e n i o r o r decision-making demn '.-,'l th a row of some f o r t y o r so c h a r a c t e r demons wi th a new decls ion- inaKing demon above them, l e t t i n g the new cogn i t ive demons f o r the c h a r a c t e r demons use a l l the I n f e r i o r demons, c o g n i t i v e and otherwise, f o r t h e i r Inputs. I t I s probably a l s o d e s i r a b l e t h a t previous dec i s ions be a v a i l a b l e f o r present dec i s ions , so t h a t a couple of new func t iona l ope ra t ions might be added. There need be l i t t l e concern about 10giCal C i r c u l a r i t y , because we have no requirement fo r l o g i c a l cons i s t ency , bu t a r e merely seeking agreeable Morse t r a n s l a t i o n .
How much of m e whole Drogram w i l l have been run and t e s t e d by Nover/oer I cannot be s u r e of. A t the p re sen t ( J u l y ) we have had some f a i r t e s t i n g o f h i l l - c l imb ing procedures.
ACKNOWLEDGEMENT
I should l i k e t o acknowledge the va luab le con t r ibu t ions o f d i scuss ions wi th many of my f r i e n d s , Inc luding e s p e c i a l l y M. Mlnsky, U. Neisser , F. Fr ick and J. Let tv ln .
DISCUSSION ON THE P A P E R BY DR. 0. G. SELFRIDCrF +
DR. J. W. BRAY: May I a s k Dr . S e l f r i d g e what n e th inks of t h i s approach t o h i s problem? Let X = dura t ion o f t n e l a s t s i g n a l , which may oe a d o t o r a dash, a s i g n a l space, l e t t e r space o r ward Space. Let y = 2, i f i n f a c t i t was a dasn, 1 I f dot , 0 i f s i g n a l space, -1 I f l e t t e r space and -2 i f word space. . , 3 ~
To form the polynomial:
t ake a number of obse rva t ions , a s he sugges ts , and l e t tne machine l e a r n the code by determining t h e c o e f f i c i e n t s Aoetc. by simple c u r v i l i n e a r regress ion . The du ra t ion o f p rev ious and subsequent s i g a l s ana the i n t e r - p r e t a t i o n given t o p rev ious s i g n a l s could be added a s f u r t h e r v a r i a b l e s on the r i g n t hand s ide .
HR. C. STRACHEY: A t Lne end o f me Paper you promise u s some f u r t h e r information about t h e l a t e s t s t a t e of t he programme. Could you l e t u s know what t h i s is?
DP- J. McCARTHY; I would l i k e t o speaK b r i e f l y about some of tne advantages of tne pandemonium model a s an a c t u a l mooel o f conscious oehavlour. In observing a bra in , one should maKe a d l ~ t i n ~ t i ~ n between t h a t a spec t of the behaviour which I s a v a i l a b l e consciously, and tflose Denavlours, no doubt equa l ly important , but whicn proceed unconsciously. I f one conceives of the Drain a s a pandemonl'.~n - a r o l l e c t l o n o f demons - pernaps vhat I s going on wi th in the demons can be regarded a s t h e unconscious p a r t of thought, and what the demons a r e p u b l i c l y shou t ing f o r each o x e r to hear , a s the conscious p a r t o f thought.
DR. D. M. MACKAY: Dr. Sel f r i d g e ' S 'pandemonium' h a s a c e r t a i n fa~mlly resemblance to a c l a s s o f mechanism considered i n some e a r l i e r papers (ref. l ) (tnough I had never suspected i t s demonic iVn$ l l ca t ions . ' ) .
In one of these , ( re f . 2) a f t e r d i scuss ing the gene ra l p r i n c i p l e t n a t you p e n a l i s e t h e unsuccessful , I po in t ed c u t t n a t the amount of i r&forna t ion
1. MACK.iY, D. M. Menta l i t y In Machii-ies. f'rcc. A r i s t . Soc. Sup f t . , 1932, 61. 2. KACKAY, D. M. The Ep i s t emol?g lca l Proclem f o r ~ u t o m a t a , ea. J. McCarthy and
C . E. Shannan Automata S t d e s , ??-inceton (1955). S e e a l s o &it. J . ?syc^iol. , 1956, 47, 30 and Advancement of Science, 1956, 392.
p e r t ~ i c ~ l ( t o use FT. S e l f r i d = e i s neta^iicr) is , e r / m a l l u n l e s s the p r o s a s i l i Z Y of success arc f a i l u r e a r e w a l . I f you ^av - a sb s t e? ,mere t n e r e a r e a v a s t n - i i l e r c f ~ c s s i ~ i l l t i e s co be e l i n i n d t e a t-y r a t n e r fee-dess t r i a l and e r r o r , tnt-n of coarse f a i l u r e occurs "idcn nore o f t e n than success. Tne s o l u t i o n I s J L e s t e d was t o for-n a Kind of s y n d i c a t e d l e a r n i n g p rocess In wnlcn a t f i rs t l a r - t nufl'sers of elements, d e s t i n e d e v e n t u a l l y f o r independence, snouln n s c n n ~ l e d toge the r s o a s to reduce tne d i v e r s i t y of response. A hand, f o r example, n l ~ l t n o t a t f i r s t nave M C ^ f i n -- :er s e p a r a t e l y c o n t r o l l a 3 l e 1 bu t c & ~ l c worc clumsily a s a bdnole. I n t h a t d a ~ , you can g r e a t l y decrease t r e amount of groping &ich i s necessa ry be fo re a success fu l .e. act e
Gay De l e s s . On t h i s p r i n c i p l e , i f pursded t o trie l i m i t , even the e a r l i e s t T r i a l s c o ~ l d nave a non-ned.lgic% chance o f s u c c e s s , (tt-ough success i n a
e r 1 small way^). Given t h i s easy s t W t , s i n p l e se l f -o rgan i s ing subrou t ines can b u i l d up
f a i r l y quickly. A s they i n c r e a s e t h e i r number ana t h e i r success , however, t he Idea I s t h a t the coupl ings between elements should g radua l ly t e d i s s o l v e d t o inc rease toe complexity of t h e problem. I f ~ O J keep uhe complexity i n c r e a s i n g s t e p b,y s t e p wi th the degree o f development o f success fu l I n t e r n a l n a t c n i n g sub-routines, tnen f u l l y adap t ive behaviour can oe enornously 'nore quickly developed than i f m e sys t en s t a r t s vvith tne f u l l r e p e r t o i r e to oe explored. My ques t ion I s why Dr. S e l f r i d g e n a s n o t i nco rpora t ed t h i s p r i n c i p l e in n i s l~andemonium' , so t n a t eacn ' k i c ~ ' ~ o u l a nave sometnin," n e a r e r CO one oli, of i ~ i f o n n a ~ i o n i n s t e a a of an - i l ~ i i o s ~ n e ~ l l g i b l e f r ac t ion .
38, ? C ORICE \ w r i t t e n con t r lou t l on \ " Tnls i s a very I n t e r e s t i n g anc s t i n i ~ l a t i n g paper. I nave one coniflent t o m x e , and t h a t i s on t h e d iscus- s ion of "Feature Wei&tIne and dill-ClirnDingH.
I t h i n k t h a t in t h i s d i scuss ion the a u t h o r has not Drought cut, one impor tant d i s t i n c t i o n between types of f lhi l l-cl imbingfl problems - t h a t between determinate and s t o c h a s t i c problems. Whereas i n a de terminate problem Lne " h i l l u i s de f ined by a s i n g l e func t ion o f many v a r i a b l e s
i n a s t o c h a s t i c problem I t I s de f ined by the mean va lue of a number of funct ions :
and a is a random v a r i a b l e whose value depends on t h e p a r t i c u l a r t r i a l made.
Now I would have thought t h a t n o s t o f the more i n t e r e s t i n g p a t t e r n recogni t i on problems a r e e s s e n t i a l l y s t o c h a s t i c : t h e obj e c t i v e is t o maxe
a macnine t n a t w i l l o b t a i n a s n i g n a p r o p o r t i o n as p o s s l . - ' l e o r c o r r e c t answers t o a s e r i e s o f q u e s t i o n s Itwhat i s t h i s p a t t e r n ? " r e f e r r i n g t o a r3ndoia s e r i e s o f p a t t . e r n s .
Tile a u t n o r h a s felven an e x c e l l e n t account o f some of trie d i f f i c u l t i e s o f d e t e r m i n a t e h i l l - c l i n b i n s and o f t n e t e c h n i a u e s f o r overcornin& thel", b u t s t o c h a s t i c h i l l - c l i m b i n ^ : p r e s e n t s a d d i t i o n a l p rob lems , and I thinK t h i s :,';ay become v e r y a p p a r e n t when a pandemonium is ~ u i l t t o a e a l w i t n a p r a c t i c a l t ask . I n p a r t i c u l a r , two prob lems which w i l l n e e d Invest!.-ation a re :
(1) how l a r g e a s a n p l e o f t r i a l v a l u e s f(x) and f d y ) , b e l o n g i n g t o two p o i n t s x and y i n -.the v e c t o r s p a c e b e i n g e x p l o r e d , w i l l be needed t o o b t a i n a s a t i s f a c t o r y e s t i m a t e o f [ f ( x ) - fi)] f o r t h e p u r p o s e s o f h i l l - c l l r f A I n g ?
(11) hovi iruch l o n g e r w i l l a p a r t i c u l a r s r c h a s ti c h i l l - c l i r l ' s i n g p r o c e d u r e t a k e than t h e c o r r e s p o n d i n g d e t e r m i n a t e p r o c e d u r e , a n a w i l l t h i s r a t i o p rove t o be t o o l a r g e , i n some p r a c t i c a l i n s t a n c e s , t o a l l o w t h e evolu- t i o n a r y development o f t k e pandemonium t o taKe p l a c e ? I s h o u l d be v e r y I n t e r e s t e d t o l e a r n i f t h e a u t h o r h a s c o n s i d e r e d t h e s e
prob lems and deve loped any s o l u t i o n s . Tne problem o f s t o c h a s t i c h i l l - cllrnbine; Is o f P r a c t i c a l i n t e r e s t o u t s i d e t h e c o n t e x t o r QiIs p a p e r , f o r I n s t a n c e i n c o n n e c t i o n w i t h t n e e v o l u t i o n a r y o p e r a t i o n of c n e n i c a l p r o c e s s e s .
DR. 0. G. SELFRIDGE ( I n r e p i / ^ ) : S i n c e I have been * o r k i n on Morse code, rihich I an do ing i n a d d i t i o n t o wording on l e a r n i n g , I nave i-net p robaoly 50 p e o p l e and when I say t h a t I am working on a macnine t o do m a n u a l l y keyed Morse code, they s a y "1 w i l l t e l l yoa how t o do m a t ' ' ana t h e y then p r o c e e a t o cone up w i t n scme scheme. A c t u a l l y , t n i s is t h e f i r s t t i n e I have n e a r d t n i s p a r t i c u l a r one, whereas I nave CoTie a c r o s s m e O t h e r s many t i m e s , so D r . Bray should be c o n g r a t u l a t e d on a new scheme f o r s o l v i n g Morse code. L e t me a s s u r e him s a t i t w i l l n o t w o r ~ , we have ~ r i e d it. , i & r s e code m a n u a i i y
r e c e i v e d I s n o t t h a t s i n p l e . Tne c o n t e x t n e c e s s a r y a p p a r e n t l y e x t e n d s a t l e a s t 10 l e t t e r s on e i t h e r s i d e - n o t J d s t one s i d e , bo th s i d e s . I n f a c t , t h e - e a l cyesc ion a b o u t d o i n s th i s - N ~ J I chose t lo rse code w2s e x a c t l y L Y A C 1. had h a d t h i s k i n d o f i n t e r a c t i o n s ¥,l t h eacn o t n e r . YOL. cannot do i t by l o o k i n & a t b i n a r y f u n c t i o n s o f t h e d u r a t i o n s , t h a t i s , p x p r e s i n g durn . t ions a s b i n a r y d i g i t s and m e n l o o k i n g on b inary f u n c t i o r s of l o t s of them. I f I taKe 10 l e t t e r s on e i t n e r s i d e , each h a s t h r e e marks, vou s h o u l d c o n s i a e r t h e s p a c e s a s w e l l , b a t l e a v i n g t h o s e o u t , t n a t i s t h r e e t id ies 20, m i c h Is 60 d u r a t i o n s , and i f we s p e c i f y t h e n t o one p a r t 11- 32, t h a t I s 300 b i t s , and Dinar;, f u n c t i o n s o f 300 b i t s cannot DS p i c k e d a t rdridGm. 30 t n e q u e s t i o n is t o take some s t e p s i n t n e r i g h t d i r e c t i o n ana tnen n o r e t l e . s t e p s a r e r i g h t enoi-i&h so t h a t you w i l l g e t enough i m p r o v e ~ e n t s , s o t h a t t h e f o l l o w i n g s t e p s w i l l g e t you even r r r e so.
~ d i t e d ve rs ions by the autnor *"is not a v i l l a o l e .
I mainta in t R 2 t L:-?re I s some n e r i t In s tud>ing s e l f - L ~ r o v e n e n t systems and i f I an goin; t o do t h a t I an going to s tudy systems where I Know m a t I want, r a t h e r tnan .?ore 3 i f f i c ~ l t problems, nowever a t t r a c t i v e they may be t o natfie:naLiclans. . - la ihenat ic ians 11x2 to work on '-insolved problems for the g r e a t e r g lory , and pro 'slens a l r e a d y solved l l ~ e tne prl::!e nun'oer theorem a r e l e f t t o graduate s tuden t s . I have a susp ic ion m a t John McCarthy might l a t e r s r i n g up some Important p o i n t s about d e s c r i p t i o n s , and here I s ee my p o i n t about so lv ing u s e f u l pro'::lens, because Morse code I s a u s e f u l problen. Tne ,+nole ques t ion I s a t '.+hat l e v e l you a r e g i n : to d e a l x i th d e s c r i p t i o n s o f your data. I n most proLleli-is, e s p e c i a l l y I n d e a l i n g wi th tne amount o f tr,ln;-s which people do, b ina ry func t ions a re J u s t n o t adequate. For one tnlnr; t ,here is too xucn data , and one of the f i r s t ques t ions you t a l k about In the condl t l oned r e f l e x - remember t h a t cond i t iona l p r o b a b i l i t y assumes *- t h a t you nave a l r eady reco/ni .sed m e s t imulus ; i f J-ou know t h a t the s t imu lus celongs to one of a smal l c l a s s of funct ions , I t I s p r e t t y easy, i s a man, cut mostly i n human problems you do not know mls.
D r . Mackay nade scme very Kind remarm. I would go f u r t h e r baCK than he a i d , a s he well knows, i n c r e d i t i n g a e i n o n o l o ~ . A l l o f u s have s inned In Aaam, we have ea t en 01" m e t r e e 0 1 the knowledge of good and e v i l , and the demonological a l l e g o r i I s a very o ld one, indeed. As for n i s remarks about t he syndica te approach, tne evo lu t ion of d i v e r s a t l v e o r ~ a n l s a t i o n , I t h ink t n a t i s an extremely accu ra t e and zood po in t . One of the tn lngs a s p e c i e s l e a r n s I s n o t only how t o survive but t o have e x a c t l y the rljdit v a r i a n t s In U s memters. The reason r:orseshoe c rabs have n o t changed much is n o t Deca-ise they a o n ' t adapt; triey can adapt , but t h e r e is no v a r i a t i o n , so tney can only aaa? t ve ry l i t t l e , end tney a re a s o u t a s good a s they can be; whereas m e r e a r e a l l Xlnas of people.
I n p e h d ' sna i r - f c m a t l o n ~i-ieory, I rea l1 r - consider t h a t speed i n the c l a s s i c sense of computation I s so completely i r r e l e v a n t to t h i s problem - m e numser of b inary ope ra t ions one does a second - t h a t i t does n o t i n t e r e s t me vcirj' nuch though i t i n t e r e s t s conputer des igners . I woula r a t h e r H A G to s e e l e t s o f o p e r a t i o n s >v!hicii could conceivably be aone I n p a r a l l e l , 'oein;; done s e o u e n t i a l l y , because t h i s i s the only machine we nave, snd t 3 t 5 I nope people w i l l nave machines whlcn w i l l w0rK In CaI'2llel. so t2a.t I want a machine t o do w i c e a s d i f f i c u l t a problem I merely c u l l d m i c e as D I G a machine, i n s t e a d o f l c t t i n e , one macnine worA twice as long.
Dr. Strachey a s ~ e d a ' x - ~ t r e s u l t s , ana they a re roughly a s I have ind ica t ed . Irprovement dces take p l ace ; the n i l l - c l i m b i n g does N C ~ K . The s u t - d e m n s e l e c t i o n I n tr.e prozramnes t n a t we nave d i d not do what we hoped, s u t tne %c-clexcns \\'ere e f f e c t i v e l y thrown our, ana tile ones which .';ere kept ; e r e 1ar;"ely concent ra ted around those aerions which r e l a t e those