Y - NASA c-, i b (THRU) ~ -- --r;-- - (ACCESSION NUMBER) -- $-- 3 9 ~~~ 1 ~ (PAGES) (CODE) c NASA CR-1035 iii 0 U _" a 0 a FREQUENCY CODED THRESHOLD " by Rob Roy and David HZnds tf) U 0 Prepared by RENSSELAER POLYTECHNIC INSTITUTE Troy, N. Y. for NATIONAL AERONAUTICS AND SPACE ADMINISTRATION WASHINGTON, D. C. MAY 1968
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Y
-
N A S A
c - , i b
(THRU) ~ -- --r;-- -
(ACCESSION NUMBER) -- $ - - 3 9 ~~~ 1 ~
(PAGES) (CODE) c
N A S A C R - 1 0 3 5
iii 0 U _"
a 0 a
FREQUENCY CODED THRESHOLD "
by Rob Roy and David HZnds
tf)
U 0
Prepared by RENSSELAER POLYTECHNIC INSTITUTE Troy, N. Y.
for
N A T I O N A L A E R O N A U T I C S A N D S P A C E A D M I N I S T R A T I O N W A S H I N G T O N , D . C. M A Y 1968
NASA CR-1035
FREQUENCY CODED THRESHOLD LOGIC UNIT
FOR PATTERN RECOGNITION APPLICATION
By Rob Roy and David Hinds
Distribution of this report is provided in the interest of information exchange. Responsibility for the contents r e s ides in the author o r organization that prepared it.
Prepared under Grant No. NGR-33-018-014 by RENSSELAER POLYTECHNIC INSTITUTE
Troy, N.Y.
for
NATIONAL AERONAUTICS AND SPACE ADMINISTRATION
For sa le by the Clearinghouse for Federal Scient i f ic and Technica l Information Springfield, Virginia 22151 - CFSTl price $3.00
Page Summary of t h e Error Charac te r i s t ics f o r Variat ions i n t h e System Design Parameters . . . . . . . . . . . 21
Table I
Table I1 Results of t h e Sample Character Recognition Problem ................................................ 45
LIST OF FIGURES
page Figure 1 Operational Model of a Pulse Rate Neuron .............. 5
Figure 2 Structure of t h e Threshold Logic Unit ................. 5
Figure 3 Dig i t a l Di f fe ren t ia l Analyzer ......................... 11
Figure 4 Binary Rate Mul t ip l ie r ................................ 13
Figure 5 Timing Diagram f o r t h e F i r s t 5 Stages of a Binary Hate Mul t ip l ie r .............................. 13
Figure 6 Error Charac te r i s t ics of t he Binary Rate Mul t ip l ie r ..w................................... 23
Figure 7 Input Equipment Diagram f o r a Single Input ............ 26
Figure 8 Weight Element Equipment Diagram f o r a Single Input ........................................... 28
Figure 9 Summation Element Equipment i)iagram .................... 31
Figure 10 Timing Diagram f o r t he Summation Element ............... 33
Figure 11 Decision Element Equipment Diagram ..................... 35
Figure 12 Timing Diagram f o r Operation i n t h e Class i f ica t ion Mode ................................... .3q
Figure 13 Timing Diagram f o r Operation i n t h e Training Mode .......................................... 40
Figure U, Control Equipment Diagram ............................. .41
Figure 15 Training Pa t t e rns Used i n Sample Character Recognition Problem .................................... 46
Pat te rns Used t o Test the Recognition Ab i l i t y of the Sample Design After Training was Complete ............................................... 47
Figure 16
v i
. .
ABSTRACT
I
I
i
t
I I t i I
I f
The object ive of t h i s project was the development of a f a s t ,
accurate and r e l i a b l e method of weight adaptation t o be used i n t r a inab le
pat tern recognition systems. The sensory nervous system was invest igated
as being a possible model f o r t h e proposed device. The speed with which
complex pa t t e rns a r e recognized and the binary nature of sensory systems
indicated t h a t t he model would be worthy of f u r t h e r consideration.
The concept of frequency coding, as found i n t h e sensory
nervous system, provides a method i n which modern d i g i t a l techniques
may be applied t o incorporate speed and r e l i a b i l i t y i n t o the design.
Coding the input pa t te rns as a pulse frequency, allows the use of binary
equipment t o instantaneously transmit an analog s igna l without quantiza-
t i o n e r r o r o r coding time delay. The concepts a r e then developed i n the
form of a Frequency Coded Threshold Logic Element.
The design presented i n t h i s project develops a discriminant
funct ion which i s a l i n e a r combination of weighted frequencies and computed
simply as a pulse count.
pulse frequency by using a voltage controlled as tab le multivibrator.
input pulse stream is then scaled using a Binary Rate Mult ipl ier , which
allows t h e storage of each weight as t h e contents of a binary reg is te r .
Summation and threshold detect ion rn accomplished by counting pulses i n
a b inary counter over a short period of time,at t he end of which a
c l a s s i f i c a t i o n i s made based on t h e contents of t h e counter. The design
presented here allows f o r both posi t ive and negative inputs and weights.
Further , provision i s made f o r an adaptable threshold by considering t h e
threshold as an addi t iona l input.
Briefly, each input voltage i s converted t o a
Each
v i i
The design i s applicable t o the current pa t t e rn recognition
It i s capable of speed and accuracy systems and t r a i n i n g procedures.
i n both c l a s s i f i c a t i o n and weight adaptation t h a t cannot be obtained
by using analog devices.
design a r e given i n de ta i l and require only a moderate expenditure of
equipment.
useful i n the construct ion of p rac t i ca l pa t te rn recognition systems.
The hardware requirements of t h e proposed
The design concepts presented i n t h i s project should prove
viii
PART (I.)
INTRODUCTION
A. Purpose
The purpose of t h i s pro jec t i s the inves t iga t ion of t h e
p r inc ip l e s on which t h e sensory nervous system functions, and t h e i r
appl icat ion t o pa t t e rn recognition problems.
system, w e mean t h e per ipheral nervous system through which environ-
mental information and body conditions are t ransmit ted t o t h e c e n t r a l
nervous system. It i s the purpose of the sensory system, such as t h e
visual, audi tory and k ines t e t i c systems, t o supply the animal with
spec i f i c pa t te rn information. The recognition, evaluation and eventual
ac t ion taken by the animal i s determined by the c e n t r a l nervous system.
By sensory nervous
We are considering the sensory nervous system as a model f o r a
pa t t e rn recognition device because of i t s information transmission
cha rac t e r i s t i c s . 'then one considers the mount of information and the
e f f i c i ency by which it i s transmitted from t h e eye t o t h e brain, it i s
evident t h a t t hese c h a r a c t e r i s t i c s would be advantages t o any pa t t e rn
recogni t ion problem.
Xe w i l l explore t h e bas ic concepts of t h e sensory nervous
system, and then apply them t o t h e current concepts i n t h e pa t te rn
recognition f i e ld .
recogni t ion device, employing t h e advantageous techniques of information
transmission exhibited by t h e sensory nervous system, t o improve t h e per-
formance of pa t t e rn recognition systems based on t h e Threshold Logic
concept .
The f i n a l result w i l l be the development of a pa t te rn
B. His tor ica l Review
The concept of a r t i f i c i a l i n t e l l i gence i n machines i s not new.
Rosenblatt 's FEHCEPTAON was among t he first attempts t o design a t r a in -
able pa t te rn c l a s s i f i c a t i o n machine.ll It was Rosenblatt 's idea t o
use the nervous system as a model f o r h i s design.
f i ca t ion systems, such as t h e Adaline and Madaline networks, have a l s o
been proposed and are w e l l covered in t h e l i t e r a t u r e . 13, 14
Other pa t te rn c l a s s i -
Pat tern recognition is based on the concept of t h e discriminant
function, which def ines t h e behavior of t h e categorizer .
considered t r a inab le when the discriminant function can be modified by an
i t e r a t i v e procedure. Nilsson presents an exce l len t review of t h e cur ren t
s t a t e of t r a inab le pa t te rn recognition devices. lo
comprehensive survey of t h e ea r ly work done i n t h e fie1d.l)
The machine i s
Hawkins presents a
There e d s t s ample l i t e r a t u r e ava i lab le i n t h e pa t te rn recogni-
t i o n f i e l d , and we s h a l l assume t h e reader is familiar with the fundamental
concepts of Threshold Logic. Included i n t h e Selected Reference, are a
number of publ icet ions dealing with spec i f i c c l a s s i f i c a t i o n systems and
techniques, t r a in ing methods and convergence proofs, and various imple-
mentations of analog memory devices.
t o discuss these papers.
used t o explain the work done here.
No attempt i s made i n t h i s pro jec t
We w i l l present only those concepts t h a t are
2
PART (11.)
THM)RY
A. Nervous System
Physiological research i n t o the neural s t ruc tu re of t he verte-
bra tes h a s not been ab le t o give a complete picture , o r explanation, of
t he nervous system. This i s pa r t ly due t o t h e complexity of t he system
l and p a r t l y t o the minute s t ruc ture of t h e components. The research repor t s
t h a t do exist a re of ten contradictory and inconclusive. The following
discussion i s a summary of the basic concepts accepted by t h e majority of
researchers . There a r e two d i s t i n c t types of nerve c e l l s known t o exis t .
They are characterized by the manner i n which they respond t o st imuli .
These responses can be c l a s s i f i ed as t h e !!all or nothing" pr inc ip le and
t h e "pulse rate" pr inciple .
nervous system.
number of incoming pulses a r r iv ing approximately a t the same time.
c e n t r a l nervous system is located i n t h e bra in and sp ina l s t ructure .
Operation i s dynamic i n nature, responding t o system changes ra ther than
s t a t i c conditions.
mental change rap id ly and in a complex manner.
method of operation seems t o l i e i n t h e v e r s a t i l i t y of t h e system i n
The first, is found i n t h e c e l l s of the c e n t r a l
A c e l l w i l l produce a pulse only i f the re i s a su f f i c i en t
The
It i s capable of responding t o an i n t e r n a l or environ-
The advantages of t he
handling a wide var ie ty of recognition and decision problems.
The Ifpulse ra te" pr inciple i s found i n c e l l s of t h e per ipheral
When stimulated, they w i l l generate a se r i e s or sensory nervous system.
of impulse spikes whose frequency i s proportional t o t h e in t ens i ty of t he
stimulus.
constant over a l l frequencies.
The amplitude of these impulses i s observed t o be r e l a t ive ly
The upper frequency a t which t h e c e l l s
3
w i l l operate is determined by the refractory period of
I n t h e human t h e maximum frequency i s about 300 pulses
Studies of t h e v i sua l systems of t h e ca t and
have shown the jmpulse frequency t o be proportional t o
t h e c e l l membrane.
1.2 second.
horseshoe crab
the c e l l membrane
potent ia l and resistance. 5,12 Incident energy, i n the form of l i g h t ,
chemically changes t h e c e l l res i s tance and the frequency of impulse
generation.
The output frequency i s a l i n e a r combination of t he input frequencies.
respective weighting of each input is done i n t h e synaptic junction of t h e
nerve c e l l and m y be e i t h e r posi t ive o r negative.
threshold below which the c e l l w i l l not f i r e i s questionable.
shown t h a t the r e t i n a l ganglion c e l l s of t he c a t discharge a t a constant
r a t e without s t i m l a t i o n .
decrease this r a t e , depending on where i n the v i sua l f i e l d the s t h u l u s
f a l l s . 5
structure.
impulses from the bipolar c e l l s .
u l a r bipolar c e l l , and then weighting the synaptic junction of t h i s c e l l
as an inh ib i tory input t o the r e t i n a l ganglion c e l l , an increase i n st imulus
energy w i l l be observed a s a decrease i n f i r i n g rate .
The function of t he ffpulse r a t e " neuron i s shown i n f igure 1.
The
However, existence of a
Kuff ler h a s
The e f fec t of st imulation may e i t h e r increase o r
T h i s phenomenon can be explained by considering the r e t i n a l
The r e t i n a l ganglion c e l l i s a t h i r d l aye r c e l l receiving
By st imulat ing the receptors of a par t ic -
Hubel has shown t h a t there exists a high degree of r i g i d
organization i n the v i sua l nervous system5
a l s o evident.
maintain pat tern recognition over a wide range of st imulus var ia t ion.
However, c e l l adaption i s
For t h e most part, it is a means of varying resolut ion t o
Perhaps, the best example i s the a b i l i t y of t h e eye t o recognize pa t t e rns
i n bright l i g h t as w e l l a s semi-darkness, w i t h t he only loss of inforWtiOn
being in the resolution of de t a i l s . The c e l l does t h i s by adapting
4
fl
f2 i = 2 Wifi out i=l
6 . JSff erent Nerve
Afferent Nerves
fn-l
fn
$-
Synaptic Junctions
r-
Category
Figure 1 Operational Model of a Pulse Hate Neuron
Weights
Figure 2 Struc ture of the Threshold Logic Unit
5
weights t o maintain t h e f i r i n g frequency approximately i n the center of
i t s linear range.
The role of t h e "pulse ra te" neuron i n the physiological nervous
system i s t h a t of information transmission.
a l l y t o t h e c e n t r a l nervous system, where it i s processed.
changes i n t h e environment t h a t t h e c e n t r a l nervous system a c t s on.
However, it is a constant monitoring of t h e environment t h a t i s ca r r i ed
out by t he sensory neuron.
Information i s sent continu-
It is the
B. Problem Definit ion
The main objec t ive of t h i s pro jec t is the deve lopen t of a
pat tern recognition device based on t h e pr inc ip le of t h e sensory nervous
system.
goal,
use the concept of frequency coding information t o develop a device appli-
cable t o t h e current pa t t e rn recognition techniques.
The remainder of this project repor t w i l l be devoted t o t h a t
We w i l l not attempt t o simulate a sensory neuron, but ra ther , w i l l
The bas ic element of a pa t te rn recogni t ion system, and t h e
simplest form of a complete system, i s t h e Threshold Logic Unit.
s ion o f t h e Threshold Logic Unit t o t h e more complex forms of pa t t e rn
recognition devices i s fairly straightforward. For this reason, and t o
keep t h e project t o reasonable proportions, w e w i l l confine our a t t e n t i o n
t o the developnent of a Frequency Coded Threshold Logic Unit.
Exten-
The Threshold Logic Unit functions, as previously discussed, by
generating a discriminant function, which i s a linear combination of t h e
weighted inputs.
value of t he discriminant function with a threshold value. I n order t o
allow f o r t h e adap tab i l i t y of t h e threshold, it i s considered as an
Class i f i ca t ion i s then accomplished by comparing t h e
6
addi t iona l input.
constant value, which we w i l l assume t o be zero.
discriminant functions are less than zero a re i n category one, and those
g rea t e r than zero are i n category t w o .
Unit i s shown i n figure 2.
This procedure permits the decision t o be based on a
Thus, those pa t te rns whose
The s t ruc tu re of a Threshold Logic
It can be considered as cons is t ing of four
d i s t i n c t parts; t he input sensor, t h e weight, t he s u n a t i o n junct ion and
t h e decis ion element.
Each p a r t w i l l be developed t o function s imi la r ly t o i t s
physiological counterpart i n the sensory nervous system.
t h e physiological motivation f o r t h e project, consideration has a l s o been
given t o t h e engineering aspects and appl icat ions f o r t h e Frequency Coded
Threshold Logic Unit.
recognition f i e l d , i t s design should use concepts and c3mponents within
t h e present state of p rac t i ca l engineering, be capable of reasonable speed
and accuracy, and should exhib i t d i s t i n c t advantages over t he present
ex is t ing methods and devices.
proposed device, de t a i l ed equipment requireinents, and f i n a l l y demonstrate
t he use of t h e Frequency Coded Threshold Logic U n i t by a computer simula-
t i o n of a pa t te rn recognition problem.
I n addi t ion t o
To be of p rac t i ca l s ignif icance i n t h e pa t te rn
We w i l l give a t h e o r e t i c a l discussion of t he
C. Desim Considerations
1. Sensor Implementation
The first s tage of t h e Threshold Logic U n i t , t h e sensor, con-
verts t h e fea tures of t h e pa t te rn i n t o input information i n a form
acceptable t o t h e system.
i s converted t o impulse frequency.
r a t h e r logarithmic, as Rushton has shown.12
aspect of t h e conversion process can be considered as a method of
I n t h e sensory nervous system, st imulus energy
The conversion is not l i nea r , but
However, t h i s non-linear
7
extract ing invar ien t features from t h e pattern. As such, it i s another
problem, and w e w i l l not consider it here.
our pat tern inputs a r e t h e desired f ea tu res and w i l l use a l i n e a r conver-
sion of input voltage t o frequency. We s h a l l f u r t h e r r e s t r i c t our allow-
We s h a l l assume, instead, t h a t
ab le pa t te rn inputs t o a specif ied voltage range centered about zero.
By converting t h e input voltage t o a pulse frequency, w e are
es sen t i a l ly coding t h e information contained i n each f ea tu re of t h e pat tern.
The coding i s ca r r i ed one s t e p fu r the r i n t h a t t he magnitude of t h e vol tage
will produce a pulse frequency independent of sign.
retained and used i n t h e determination of t h e discriminant function.
The s ign i s a l s o
2. Weight Imnlementation
Before considering t h e methods used t o weight t h e input
frequencies and sum t h e r e su l t s , we must look at the decis ion process i n
i t s ent i re ty . The decision of t h e Frequency Coded Threshold Logic Unit i s
based on t h e discriminant function,
n T-
i=l
where :
N W i - Are t h e weights Wth = Is t h e threshold weight f i
- Number of input l i nes
= Are t h e input f requencies
compared t o a threshold value.
weights as w e l l as pos i t ive and negat ive inputs.
and processing sign as w e l l as magnitude information.
We s h a l l al low both pos i t i ve and negative
This requi res r e t a in ing
I n order t o i d e n t i f y a spec i f i c pu lse frequency, t h e pulse t r a i n
must be observed f o r a t least one period. We propose t o observe each input
8
i pulse t r a i n f o r a time A T , ca l l ed t h e decision time. A decision, o r
I categorizat ion of an input pa t te rn w i l l occur at t h e end of each A T
k r t h e r , f i A T - n i and represents a specified number of pulses occuring
I interval . The discriminant function thus becomes,
l i-1
I This leads t o t h e formulation of t h e discriminant function,
We now have a l i n e a r combination of pulses ra ther than frequencies. The
threshold weight i s chosen as being independent of decision time and is
given as a pulse count. Considering t h e en t i re decision process i n t h i s
l i g h t , a l lows t h e weighting and summation processes to be implemented as
manipulations of a pulse count, r a the r than attempting t o maintain t h e
frequency concept throughout t h e device.
Perhaps, t he most important part of any t ra inable pa t te rn
recogni t ion system is the method of implementing the weights. The
physiological counterpart i s t h e synaptic junction of t h e nerve c e l l ,
where weighting each incoming nerve i s a complex electro-chemical process.
The effect of an incoming nerve can be excitory o r inhibatory. Correspondingly
t h i s proposal w i l l allow pos i t ive and negative values f o r t h e weights.
Among t h e major problems of pa t t e rn recognition systems, is input modifica-
t i o n and t h e associated storage and adeptab i l i ty of t h e modifying weights.
The input weighting process must be accurate, r e l i a b l e and rapid. Weight
9
storage must be constant over r e l a t i v e l y long periods of time and capable
of revers ib le adaptatim,at a reasonably rapid r a t e and by accurately
controlled amounts.
The proposal of t h i s project i s t o use t h e techniques of modern
d i g i t a l c i r c u i t r y t o develop an adaptive system meeting the above require-
ments. It i s i n t h i s respect t h a t t he concept of frequency coding becomes
advantageous.
coding t h e inputs.
theore t ica l r e s t r i c t i o n s on the number of leve ls . Prac t ica l ly , a maximum
would be 10 b i t s , or t o within one ten th of one percent. The weights are
allowed t o range from -1 t o 1 i n s t e p s of l/2", where n i s t h e number of
binary b i t s .
Binary devices can be used without f i r s t quantizing and
The weights are quantized, however, there are no
The accuracy with which a weight i s implemented i s discussed
under E r r o r and Accuracy Considerations. The accuracy t o which t h e weight
can be adopted i s l imi ted only by t h e quant izat ion leve l , s ince t h e weight
can be adapted exactly t o any desired level.
d i g i t a l weight i s superior t o any analog memory device cur ren t ly ava i lab le
f o r adaptive application.
I n these respects , t he
The proposal presented i n t h i s pro jec t requi res t h e weighting
element t o scale t h e incoming pulse t r a i n and also r e t a i n s ign information.
The sign is stored as a s ing le b i t , which i s t h e method used t o handle
signs throughout t h e design of t he device.
considered for implementing the sca l ing of t h e pulse t r a in .
was t h e D ig i t a l D i f f e ren t i a l Analyzer.2y6
There were two methods
The f irst
The s t ruc ture of Digi ta l M f f e r e n t i a l Analyzer i s shown i n
f igure 3.
t h e addition of t h e contents of t h e weight r e g i s t e r t o be added t o t h e
The device operates i n such a manner t h a t an input pu lse causes
10
n ax -
L.---I add - subtract
n O u t p u r
add - subtract
n I I
Figure 3 Digi ta l Di f fe ren t ia l Analyzer
11
contents of t h e storage reg is te r .
entered i n t h e storage r eg i s t e r .
t h e storage r e g i s t e r overflows.
determined by the number of input p u l s e s , a X , t h e weight, W, represented
by the contents of t h e Binary r eg i s t e r , and t h e number of s tages , n, i n
t h e regis ter . The r e s u l t i s t h a t t h e
input pulse t r a i n i s scaled by a f a c t o r of W/2”. Adaptation i s accomplished
by adding o r s u b t r a c t i n g ~ W b i t s , one a t a time, from t h e weight r eg i s t e r .
The Digi ta l M f f e r e n t i a l Analyzer i s used i n the design of incremental
computers and, therefore , has a wide va r i e ty of application.
t h a t the equipment required t o obtain t h i s v e r s a t i l i t y i s far too complex
f o r appl icat ion i n the Frequency Coded Threshold Logic Unit.
w i l l not be considered f u r t h e r i n t h i s design.
The result of t h i s addi t ion i s then
A n output pulse i s produced each time
The number of output pulses, A z , i s
The equation i a d 2 = (1/2”)xh’.
It i s fe l t
Hence, it
The second method, and t h e one chosen f o r t h i s appl icat ion, i s
t h e Bina ry Rate M ~ l t i p l i e r . ~ ’ ~ ’ ~ , ~ ’ ~ The bas ic s t ruc tu re of t h e Binary
Rate Mult ipl ier i s shown i n f igu re 4.
same as given f o r t h e Dig i t a l Di f fe ren t ia l Analyzer.
implementation i s qu i t e d i f fe ren t .
The equation f o r t h e device is t h e
However, t h e
The input pulse t r a i n is presented t o an n s tage b inary counter,
ca l l ed the input counter, where n i s t h e same as t h e number of b i t s used
i n t h e weight reg is te r .
each stage makes t h e t r ans i t i on f r o m t h e 1 t o t h e 0 s t a t e . The r i p p l e
i s accomplished by the generation of a carry pulse, as t h e s tage makes
t h e 1 t o 0 t r ans i t i on , which then a c t s as an input t o t h e next stage.
ca r ry pulse i s propagated u n t i l a t r a n s i t i o n from 0 t o 1 occurs.
The input pulses r i p p l e through the counter as
The
A t t h i s
point no carry pulse i s
s tage t o i t s associated
generated, but
sca l ing gate.
a pulse i s t ransmit ted from t h a t
A t each scaling ga te t h e r e appears
Input n Pulse
4 Stream
'I
LL
Scaling Control Signal
Input
1 1 I I I I I I 1 1 1 1 1 1 1 1 1 I I I I I I I I I I I , Scaling Gate 1
I I I 1 1 I I I I I I I I I b Scaling Gate 2
Figure 4 Binary Rate Mult ipl ier
Figure 5 Timing Diagram f o r t he F i r s t 5 Stages of the Binary Rate Mult ipl ier
a specific f r a c t i o n of t h e input pulse t r a i n , determined by t h e number
of input pulses required t o cause t h e 0 t o 1 t r a n s i t i o n of t h a t stage.
The kth stage w i l l sca le t h e pulse t r a i n by a f a c t o r of 1/2
takes 2k input pulses t o cause t h e 0 t o 1 t rans i t i on ,
k since it
The a c t u a l f r ac t ion of t h e input pulse t r a i n t h a t i s t ransmit ted
t o t h e output buffer i s determined by t h e combination of scal ing ga te s
t h a t are open. The condition of t h e scal ing ga te i s control led by the
associated s tage of t h e weight reg is te r .
weight r e g i s t e r permits t h e transmission, t o t h e output buffer , of t h e
pulses a r r iv ing a t t h e kth scal ing gate.
i s the number represented by t h e b i n a r y contents of t h e weight r e g i s t e r
divided by 2", for an
accomplished t h e desired scal ing of t he pulse count over t h e decis ion
in t e rva l .
A 1 on the kth s tage of t h e
The t o t a l sca l ing f rac t ion , then,
b i t weight. By scal ing the pulse train, we have
The timing diagram, f igu re 5 , shows t h e f r a c t i o n a l scaling
operation a t each s tage of the input counts.
it is evident t h a t t he re can be no simultaneous occurrence of pulses
a t the output buffer , t hus presenting no synchronization problems.
form of t h e Binary Rate Mul t ip l ie r used i n t h e f inal design has an
addi t ional s tage i n the weight r e g i s t e r and an assoc ia te sca l ing gate.
The addi t ional equipment is opt iona l and i s used t o t ransmit t h e overflow
pulse from t h e last s tage of t h e input counter t o the output.
t h e en t i r e input pulse t r a i n t o be t ransmit ted t o t h e output and
corresponds t o a maximum weight of unity.
pulsing t h e weight r e g i s t e r t o count up o r down.
is discussed i n conjunction with t h e t r a i n i n g procedure.
Fromthe timing diagram,
The
It allows
Adaptation i s accomplished by
The adaptat ion algorithm
14
3. Summation Element Implementation
When considering the implementation of the summation junction,
we again use the concept of pulse count t o determine the discriminant
function.
we need only feed t h e pulse stream from t h e output buffer of each Binary
Rate Mult ip l ie r i n t o a s ingle counter. The sign of the input i s used t o
determine whether t h e count i s up o r down.
w i l l be t h e signed value of t h e discriminate function a f t e r t h e decision
time has elapsed.
To obtain the summation of pulses occurring on a l l input l i nes ,
The contents of t h e counter
Provision must be made, however, f o r the simultaneous occurrence
of pulses on two o r more l ines . It w i l l be reca l led t h a t simultaneous
occurrence of pulses on a s ingle l i n e cannot occur.
e n t a i l s t h e synchronization of the imput l i n e s by delaying t h e pulses
This provision
and presenting each l i n e i n sequence t o e i the r t h e up o r down counter
buffer .
of inputs , including the threshold, a r e sequenced t o the counter at a
r a t e a t l e a s t equal t o t h e maximum allowable input frequency.
ensure t h a t no pulse w i l l be l o s t by the successive a r r i v a l of two pulses
on t h e same l i ne .
discussed later.
The only r e s t r i c t i o n i n t h i s procedure i s t h a t t h e e n t i r e s e t
Th i s w i l l
Actual equipment used f o r t h i s procedure w i l l be
4. Decision Element Implementation
The f i n a l s tage of t h e Threshold Logic Unit is t he decision
The decis ion process, whether neural o r e lectronic , e n t a i l s t he element.
comparison of t h e discriminant function w i t h a predetermined threshold
value.
cell, below which t h e c e l l w i l l not produce an impulse.
I n t h e nerve c e l l t h i s threshold i s t h e f i r i n g po ten t i a l of t h e
I n the decision
process it must be remembered t h a t , it i s t h e c e n t r a l nervous system and
t h e "on-off" pr inciple being considered.
problem, the threshold is determined by a f ixed value t o which the dis-
criminant funct ion i s compared.
I n the pattern recognition
I n t h i s pro jec t we propose t o use zero as the f ixed threshold
This allows making the f i n a l decision using only the s ign b i t of value.
the summation counter.
during t h e t r a in ing period.
weight as an addi t iona l input.
t he weight i s allowed t o be pos i t ive or negative and contains t h e same
number of b i t s as t h e o ther weights.
an equal e f f e c t on t h e discriminant funct ion i n comparison with t h e o the r
weights, the input frequency should be l imi ted t o t h e same range as t h e
o ther input lines.
There i s provision f o r threshold modification
This i s done by considering the threshold
The input frequency i s a constant and
In order t h a t t h e threshold exert
There i s a c l a s s i f i c a t i o n o r decis ion made only a t t h e end
of the decis ion in te rva l .
approximately constant over t h e decis ion in t e rva l .
r e s t r i c t i o n since decis ion times of less than a t e n t h of a second are
readily a t ta inable , with good accuracy, f o r a problem with 100 inputs .
Further, even i f several inputs should vary t h e e f f ec t ive input t o the
system w i l l b e an average of t h e a c t u a l input over t h e decis ion in te rva l .
This i m p l i c i t l y assumes t h a t t h e inputs are
T h i s i s not a ser ious
D. Training Procedure
The previous discussion has d e a l t with t h e pa t t e rn c l a s s i f i c a t i o n
aspect of t he pa t te rn recognition problem.
toward t r a in ing procedure and weight adaptation.
t ra ining pa t t e rns and t h e proper category have been determined p r i o r to
t ra ining, and t h a t they w i l l be presented i n sequence by an external agent,
We now tu rn our a t t e n t i o n
We w i l l assume t h a t t h e
16
ca l led t h e "teacher".
problem and will not concern us here.
w i l l introduce a new pa t te rn upon receiving a s igna l from t h e c l a s s i f i c a t i o n
device indicat ing t h a t t h e pr ior adaptation has been completed.
assumed t h a t t h e %eacherft w i l l continue the t r a in ing procedure u n t i l
s a t i s f i e d with t h e performance of t h e machine.
The method of presentation depends on t h e pa r t i cu la r
We need only assume t h a t the %eacherlf
It i s a l s o
As was previously s ta ted , the weights are stored as t h e contents
of a b inary r e g i s t e r and a r e modified by pulsing t h e r eg i s t e r t o count up
o r down.
adaptation of t h e weights, which i s consistent with current non-parametric
t r a i n i n g procedures.
correct ion r u l e described by Nilsson.
The following discussion will develop an algorithm f o r systematic
Specif ical ly , w e w i l l use t h e f ixed increment
10
Adaptation i s only allowed i f the c l a s s i f i c a t i o n of a t r a in ing
pa t te rn i s erroneous.
weights are modified using the following rule:
Should a pa t te rn be incor rec t ly c l a s s i f i ed , t h e
- W ' = W + c Z - where ,W i s t h e weight vector whose first n components are t h e input weights and n + 1 component i s the threshold weight.
- Y i s t h e augmented pa t t e rn vector whose first n components are t h e pat tern inpu t s and whose n + 1 component i s t h e r e l a t i v e input t o t h e threshold weight.
c i s the constant which deternines t h e increment of adaption.
For our purposes c w i l l be considered t h e nuhe r of b i t s each weight i s
adapted pe r vo l t input, and will be used t o determine the number of pulses
gated t o t h e adapt input of each weight. Whether t h e weight i s adapted
up or down is determined by the sign of the input and t h e category i n t o
which t h e pa t t e rn was erroneously c lass i f ied .
The s ign of t h e constant c is determined by the value of t h e
17
discriminant funct ion.
discriminant functions produced by e i t h e r pos i t ive or negative incorrect
c lass i f ica t ion . This allows t h e reinforcement of t h e weights which w i l l
The s ign Will be opposite t o the s ign of t h e
tend t o cor rec t t h e discriminant function f o r t h a t p a r t i c u l a r input
pattern, and decrease the e f f e c t of those inputs which tend t o adversly
effect t h e value of the discriminant function.
shown t o be convergent, but speed w i l l depend on t h e choice of C.
The procedure has been
10
E. Er ror and Accuracy Consideration
I n the preceding discussion we developed a discriminant funct ion
i n terms of a pulse count. I n order t o con t ro l t he accuracy of t h e
Frequency Coded Threahold Logic Unit, we w i l l derive a bound on t h e
m a x i m deviat ion of t h e ac tua l pulse count from t h e desired.
previously indicated t h a t t h e weights can be quantized t o any desired
accuracy and adapted t o any desired quant izat ion leve l .
consider t h e e r r o r introduced i n t h e process of determining t h e f i n a l
pulse count.
We have
Now we w i l l
We note t h a t t h e input frequency generation and weighting are
both done considering only t h e absolute value of t h e q u a n t i t i e s concerned.
Likewise, the e r r o r i n t h e sumnation process i s independent of sign. I n
discussing t h e max imum e r r o r bound, we need only consider t h e magnitude
of the maximum deviation.
developed here, s ince pos i t ive and negat ive e r r o r s will tend t o cancel i n
the f i n a l decision.
considered.
converter.
scal ing the pulse t r a i n .
process.
The ac tua l error w i l l be w e l l below t h e bound
There are th ree independent sources of error t o be
F i r s t , i s t h e l i n e a r i t y of t h e input vol tage t o frequency
Second, the e r r o r introduced by t h e Binary Rate Mul t ip l i e r i n
Final ly , t he e r r o r introduced by t h e summation
18
The accuracy of t h e pulse count is determined by t h e deviation
of t h e ac tua l pulse count from t he value of t h e discriminant function.
Total pulse deviat ion = 1 g ' ( f ) - pulse count i n the sunrmation counter
n 7
+ w t h
where g' (f) 2 Wifi . i-1
The bound developed here i s f o r a s ingle input l i n e and must be
mult ipl ied by t h e number of input l i n e s t o obtain t h e t o t a l deviation.
However, t h e f igure of most i n t e r e s t is not t h e pulse deviation, but
r a the r t h e percent deviation, measured as the pulse deviation with respect
t o - t h e maximum number of pulses possible. This f igure i s given as a per-
centage and i s independent of t h e number of input l i nes , The f igu re i s
important because it gives a measure of the precision of t h e f i n a l count,
which should be approximately t h e same as the precis ion with which the
weights are quantized.
Table 1 shows t he maximum e r ro r bounds for the spec i f i c
design indicated, i n pulse deviation per input l i n e and i n percent
deviation, f o r each s tage of t he device and f o r t he t o t a l design.
input e r r o r i s given as a percent deviation from l inea r i ty .
pulse deviat ion is dependent on the maximum input frequency, fma, and
t h e dec is ion in te rva l , A T .
The
The m a A m u m
It i s computed as follows:
M a x i m u m input pulse = deviat ion 100
% Deviation from l i n e a r i t g xfma x A T
The m a A m u m pulse deviation introduced by the Binary Rate
Mul t ip l i e r is dependent on the i n i t i a l state of t h e input counter, t he
contents of the weight r e g i s t e r and on the number of s tages used i n the
r e g i s t e r , Moshos has computed various bounds on t h e maximum pulse deviation
with respect t o these variable^.^ He shows t h a t f o r an a r b i t r a r y weight
and ini t ia l counter valuee,the max imum deviat ion depends on t h e number of
stages, n, i n t he following manner:
Maximurn Binary Hate Mul t ip l ie r 1 n pulse deviat ion / input l i n e = 9 + ' ? +
Figure 6 shows t h e e r r o r cha rac t e r i s t i c of t h e Binary Rate Mult ipl ier .
As i s indicated, t h e pulse scal ing i s exact after each 2" input pulses,
where n i s t h e number of stages. I n the computation of the maximum
percent deviat ions i n t a b l e 1, we have given t h e results as a funct ion
of cycles of 2" pulses.
minimum number of pulses equal t o 2" x t h e number of cycles.
Khat t h i s means i s t h a t each input l i n e sees a
We have
also computed t h e decision t h e , s ince the number of cycles i s considered
as t h e system parameter, r a the r than t h e decision time.
The summation process introduces a maximum deviat ion of one
pulse per input l i ne . This e r r o r a r i s e s when a pulse a r r i v e s a t t h e
input t o the summation buffer after t h a t input l i n e has been sequenced
i n t o the summation counter. This can occur only on t h e f i n a l input
sequencing i t e r a t i o n before t h e end of t h e decis ion period. Should t h i s
occur on any i t e r a t i o n p r io r t o t h e last, t h e pulse w i l l be picked up on
t h e next i t e r a t ion . For proper operat ion of t h e sumat ion element, each
input must be sequenced a t l e a s t once between each input pulse.
We compute t h e percent devia t ion by first ca l cu la t ing t h e t o t a l
range per input l i ne t h a t t h e summation counter can assume. For a weight
range of 0 t o 1, t h e maximum number of pulses per input l i n e is
M a x i m u m number of pulses fmax IC A T / input l ines
The percent deviat ion is:
Maximum percent deviation = MaximUm Pulse devia t ion / input l i n e x 100 Maximum number of pu lses / input l i n e
Table I Summary of Error Character is t ics f o r
Number of Binary Stages
B i t Weight (percent ) (pulses/l ine)
Binary Rate Mul t ip l ie r Deviation
Cycles of 2"
320 480 640 800 960
1120 1280
Counter I te ra t ion :
Decision l n t e va l clO-Tsec: 6.40 9.60
16.00 19.20
12.80
22.40 25.60 28.80 32.00
3.11
Percent Deviation Pulse Deviation
Input Total
2.40 6.51
4.00 8.11 4.80 8.91
1.60 5.71
3.20 7.31
5.60 9.71 6.40 10.51 7.20 11.31 8.00 12.11
Weight 97 65
.49
.39 32
.28 924 .22 19
Number of Binary Stages
8
Sumatior 31
.21
.16 9 13 .10 .OS9 .078 .069 .O63
B i t Weight (percent ) (pulse s / l ine)
39 3.45
B i n a r y Rate Mul t ip l ie r ' leviation
Tota: 1.78 1.36 1.4 1.01
93 87
.82
.79 76
iterations In te rva l ( 10-3 se c input
Cycles
8 9
10
?r of Stages
3
Numb Binary
Cycles Of 2"
2 3 4 5 6 7 8 9 10
B i t Weight (percent) (pulses/l ine)
19 3.77
Binary Rate Mul t ip l ie r Deviation
/Counter IDecision I Pulse
Counter I t e r a t i o n s
1280 1920 2560 3200 3840 k480 5120 5760 6400
Decision Pulse Deviation Percent Deviation I n t o y a l '
I I I S t a r t New 1 Reset Pulse - Input Counters, FF(A) I Decision I I I n t e mal I I L
I
I I
I I
I
1 t
1 Adapt Gate I I I I I
I I
I
I I
I
I
+ 4 L c
I\ T
Signal t o "Teacherll - Adaption Complete I m a I
I I
I i r
+I
A
I Signal from "Teacherll - New Pattern Ready I
I I
I
I I
I I
L F
I Signal t o "Teacher" - No Adaption Necessary
L I r
Figure 13 Timing Diagram f o r Operation
i n the Training Mode
40
r -I 1 1
o v
I
A
1 r
I I
1
r
41
PART ( I V . )
APPLICATIONS
A. General Discussion
The design presented i n t h i s project i s concerned with the
simplest trainable pa t te rn recognition system known, t h e Threshold Logic
Unit.
l i nea r discriminant func t ion can be used. There i s no r e s t r i c t i o n on
t h e s ize of t he problem, s ince any number of i npu t s can be considered
with no l o s s of accuracy, and only a s l i g h t decrease i n speed. Extension
of the concepts presented here can be made t o t h e multicategory c l a s s i f i -
cation problem.
b i t of t h e summation counter. The contents of t h e counter are a
measurement of t h e dis tance from the pa t t e rn point t o the decis ion hyper-
plane.
of the summation counter with a set of threshold values.
of each threshold weight could be accomplished by s tor ing it as t h e
contents of an adaptable r eg i s t e r , r a the r than as a pulse stream scaling
device.
sumation counter w i t h those of each threshold weight, e i t h e r by d i g i t a l
computation or by d i g i t a l t o analog conversion and analog comparison.
It i s only capable of handling two category problems i n which a
The two category decis ion i s made using only the s ign
For multicategory c l a s s i f i c a t i o n we need only compare conten ts
Implementation
Tne f i n a l decis ion would e n t a i l comparing t h e contents of t h e
The extension of t h e Frequency Coded Threshold Logic Unit t o
nonlinear c l a s s i f i ca t ion problems i s a l s o possible.
machine requires t h e analog mul t ip l ica t ion of t h e input vol tages t o
obtain t h e desired products, which are then used as t h e input vo l tages
t o the device.
proper interconnection of elementary Threshold Logic Units.
Generation of a 9
A piecewise l inear machine can be constructed by t h e
The development of t h i s extension was not attempted i n t h i s
42
project.
p r inc ip le and i t s appl ica t ion as a method of weight implementation.
design concepts presented here have several advantages over present ly
ex is t ing methods. The major advantage i s t h e accuracy and speed with
which t h e weights can be adapted.
equipment i n place of an analog memory device i s t h e key t o t h i s design.
I n addi t ion t o adaptation accuracy, t h e weight implementation, discriminant
The major concern w a s t h e development of t h e frequency coding
The
The use of high speed d i g i t a l
I funct ion computation and the decis ion process are a l l done with su f f i c i en t
precis ion f o r most appl icat ions.
The use of d i g i t a l equipment has added advantages. It i s
highly r e l i ab le , capable of miniatur izat ion and i s readi ly ava i lab le f o r
engineering appl icat ion.
recognition problem w i l l depend on t h e spec i f ic requirements of the
intended application. It i s f e l t t h a t t h e frequency coding pr inc ip le
has advantages which warrant i t s consideration as a su i tab le so lu t ion
t o many pa t t e rn recognition problems.
The f i n a l choice of equipment i n any pa t t e rn
B. Computer Simulation
The operation of t h e Frequency Coded Threshold Logic Unit was
The simmulation was then applied t o a simulated on a d i g i t a l computer.
s ing le two charac te r recognition problem.
operat ion and t h e two category aspect of the design, makes it more
appl icable t o a qua l i t y cont ro l problem than a character recognition
problem. However, t he simulation serves t o i l l u s t r a t e t he method by
which t h e frequency pr inc ip le c m be applied t o a wide va r i e ty of problem's.
The continual aspect of t h e
The continuous nature of t h e input var ia t ion i s used i n the
recogni t ion of t h e characters A and P.
our i npu t pat tern.
We use a 3 by 3 gr id t o determine
The percentage of each g r id sec tor t h a t i s blacked
43
out i s considered as t h e input feature . The sca le i s centered about a
50 percent blacking of t h e grid. Thus, no marks i n a g r id sec tor w i l l
correspond t o a maximum negative input, 50 percent t o zero vo l t s input
and t o t a l blackout w i l l correspond t o a maximum pos i t ive voltage. The
t r a in ing s e t used i s indicated i n f igu re 15.
The simulation program is wr i t ten t o generate t h e same e r r o r s
t h a t would be encountered i n the operation of t h e ac tua l device.
char t s are given i n t h e Appendix.
of design and t r a in ing parameters t o allow a c e r t a i n degree of versa-
t i l i t y i n simulation.
t h a t converts them t o t h e desired frequencies. The program operates on
t h e period r a the r than the frequency, since it i s t h e t i m e between pulses
t h a t c a r r i e s t h e information.
Flow
Provisions a r e made f o r t h e va r i a t ion
The input pa t te rns are presented t o a program
The r e s u l t s of t he character recognition problem are given i n
The r e s u l t s a r e not given t o ind ica t e t h e advantages of a t ab le 2.
pa r t i cu la r method of character recognition.
t he appl icat ion of the frequency coding pr inc ip le ,
r e su l t s a r e t h e speed with which the pa t t e rns would be c l a s s i f i e d i n
the ac tua l device, t h e accuracy of t h e weights, and t h e rate a t which
adaptation would occur.
pa r t i cu la r design ind ica ted i n the t ab le .
They serve only t o i l l u s t r a t e
The s ign i f i can t
These parameters are given i n t a b l e 2 f o r t h e
44
Table I1 Resul ts of t he Sample Character
Recognition Problem
System Design Parameters
Number of Inputs
Number of Binary Stages
Minimum Input Frequency
Maximum Input Frequency
Threshold Frequency
Number of Control Pulses / Decision In t e rva l
Fixed Increment Correction Constant
Relative Value of t h e Threshold Adaptation
Number of I t e r a t i o n s of the Training Set
Number of Pa t te rns Used i n the Training Set
Number of Pa t te rns Used i n t h e Trial Set
Recognition Results for t h e T r i a l Se t
Tota l Percent Recognition
Percent Recognition of t h e A
Percent Recognition of t h e P
Timing and Speed Considerations
Decision In t e rva l
Kaximum Time Required f o r one Complete Adaptation 5 B i t s a t a Frequency of 50000 cps
Tota l Time Required f o r t h e Training Frocedure 60 Adaptations
Time Required f o r t he Trial Class i f ica t ions
9
7
40000 cps
50000 cps
45000 CPS
480
5 pulses
.20
20
20
30
78 percent
60 percent
96 percent
.0096 s0c.
.OW1 sec.
3.84 sec.
.288 sec.
45
Category 1
Category 2
Figure 15 Training Pa t t e rns Used i n t h e Sample
Character Recognition Problem
46
,
figure 16 Pat terns Used t o Test the Recognition Abi l i ty
of t he Sample Design After Training was Complete
47
FART (V.)
SWdHy
The motivation f o r t h i s p ro jec t was the need f o r a f a s t ,
accurate and r e l i a b l e method of weight adaptation t o be used i n t ra in-
able pa t t e rn recognition systems.
invest igated as being a possible model f o r t h e proposed device.
speed with which complex pa t te rns a r e recognized and t h e binary nature
of sensory system indicated t h a t t h e model would be worthy of f u r t h e r
considerat ion.
The sensory nervous system was
The
Modern d i g i t a l techniques were applied i n an attempt t o
It was i n t h e incorporate speed and r e l i a b i l i t y i n t o the design.
application of t h e d i g i t a l techniques t h a t t h e frequency coding of
information, found i n t h e sensory nervous system, became important.
Coding the input pa t t e rns as a pulse frequency, allows t h e use of b inary
equipment t o instantaneously t r a n s n i t an analog s igna l without quantiza-
t i o n e r r o r or coding time delay.
form of a Frequency Coded Threshold Logic Xlement.
The concepts were then developed i n t h e
The device was shown; t o be appl icable t o t h e cur ren t pa t te rn
recognition systemand t r a i n i n g procedures, t o possess des i rab le speed
and accuracy cha rac t e r i s t i c s f o r both c l a s s i f i c a t i o n and adaptation, and
t o require only a moderate equipment expenditure.
concepts should prove use fu l i n the cons t ruc t ion of p r a c t i c a l pa t t e rn
recognition systems.
As such, t he design
48
&"ART (VI.)
LITEUTUm CITdD
1. Amstein, V., Mergler, H.W., and Singer, B. : IIDigital Linear Interpolat ion and the Binary Fiate Multiplier," Control Engineering, vol. 11, No. 6, pp. 79-83, June, 1964.
2. Braun, Edward L. : IIDesign Features of Current l l i g i t a l Di f fe ren t ia l Analyzers," Convention Record, p a r t 4, pp. 87-97, 1954.
3. Gordon, Sernard M. : "Adapting Digi ta l Techniques f o r Automatic Controls," E lec t r i ca l Manufacturing, vol. 54, ?io. 5, pp. 136-143, November, 1954.
4. Hawkins, J. : tlSelf-organixing Systems: A Review and Commentary,It Proceedings of the IRE, vol. 49, No. 1, pp. 31-48, January, 1961
5. Hubel, David H.: "The Visual Cortex of the Brain," Science American, vol. 209, pp. 54-62, November, 1963.
6. Mayorov, F.V.: Electronic Digital Integrating Computers: Digi ta l Different ia l Analyzers, American Elsevier Publishing Co.,Inc., New York, 1964.
7. Meyer, M.A. : W i g i t a l Techniques i n Analog Systems, Transactions - on Electronic Computers, vol. Ec-3, pp. 23-29, June, 1954.
8. Moshos, George J. : "Design of Real Time Computers Uti l iz ing Counting Techniques," NASA Technical Note TN D -3042, Washington, D.C., October, 1965.
9. Moshos, George J. : "Error Analysis of Binary Hate Mult ipl ier , It NASA Technical Note TN D -3124, Washington, D.C., 9ecember 1965.
10. Nilsson, Nils J. : Learning Machines: Foundation of Trainable Pattern- Classifying System, McGraw-Hill, New York, 1965.
11. Rosenblatt, Frank: Pr inciples of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, Spartan Books, ldashington, D.C., -- 1962.
12. Rushton, W.A.: "Peripheral Coding i n the Nervous System," Sensoq Comnicz t ion , Rosenblith, W.A., Ed., Niley and Sons, New York, 1961.
13. Tou, J.T., Wilcox, R.H.: Computer and Information Science Symposium - Northwestern University, 1963, Spartan Books, Jashington, D. C. 1964.
14. Widrow, B. and Hoff, M.E. : Iliidaptive Switching Ci rcu i t sp Stanford Electronics Laboratories Technical Report 1553-1, Stanford University, Stanford, California, June 30, 1960.
49
PART (VII.)
SELECTED FLEFJ3UNCES
1. , : IIAdaptive Memory Technical Document, Report No. HkDC-TDH-63-371, November, 1963, Information and Processing Lab., Rome Air Development Center, Gr i f f in A i r Force Base, N.Y..
2. Adrian, E.D.: The Mechanism of Nervous Action, University of Penn- sylvania Press, Philadelphia, 1932.
3. Crafts, H.S.: " A Magnetic Variable Gain Cornponent f o r Adaptive Networks, ' 1 Technical Report No. 1851-2 Stanford Electronics Lab., Stanford University, Stanford, Cal i f ; , December, 1962.
4. Oeutsch, Sid; : Wonjectures on Mammalian Neuron Networks f o r Visual Pa t te rn Recognition,I' Transactions on ,J stems Science Cybernetics, vol. SSC-2, No. 2, December, 196 3r
5. Enslein, Kurt, Ed.: Acquisit ion and Processinq Biolom and Medicine; Proceediws of the 1- Rochester Conference, Pargamon Press Inc., New York, 1962.
PART (VIII.)
APPENDIX
Flow Charts f o r t h e Computer Simulation of the Frequency Coded Threshold Logic Unit
C la r i f i ca t ion of Symbols
Input Parameters
Design Fmin Minimum input frequency
M a x i m u m input frequency sa Number of binary s tages I Number of input l i n e s N C I THI
Number of cont ro l pu lses per decision i n t e r v a l Relative value of t h e threshold input
Training C Fixed increment correct ion THC MAX
Relative value of t h e threshold adaptation Maximum allowable i t e r a t i o n s of t he t r a i n i n g s e t
Patterns TSETi Training pa t t e rns Index i over the input l i n e s CSETi: Trial pa t te rns J Number of pa t t e rns j over t h e pa t te rns
Subroutines
BRM CAT ADAPT
Simulation of t h e Binary Rate Mul t ip l ie r Simulation of t h e Categorization rout ine Simulation of t he adaptation rout ine
Symbols used i n t h e Simulation
wi , j 'th 'sign, j
c i , J PRES j
TW3 j
COUNT DTIME
Tmax Tmin T i NPUJSE
Weight r e g i s t e r Index i over t h e r e g i s t e r Threshold weight s tages Sign b i t f o r associated weight r e g i s t e r Input counter
Condition of f l i p f l o p (A) a t t h e s ta r t of a decis ion i n t e r v a l Time of a r r i v a l of t h e f i r s t input pulse after the start of a decision i n t e r v a l Contents of t h e sumat ion counter Decision i n t e r v a l Maximum input period
Eiinimwn input period Period of t he i t h input l i ne Number of pulses presented to the respect ive input counter over a decision in t e rva l
j over t h e input l i n e s
r S t a r t
Main Program
~~ ~
I n i t i a l i z e w n , i = 0 Pmsi = 0 Wth = O TMSi = O
n = l t o N i = l t o I
Operating Mode
I DTIMX = N C I * Tmin l H = Tfin/I
Design Parameters
')I Set
Adaption Counter = 0 L
52
, COUNT = 0
r Advance
Adaption Counter
Compute C n , i = 0
, j T i = TSETi n = l t o N i = l t o I
1 CALL BRM 5?
6 53
CALL AUAPT
Write Results of Adaption
Write Number of Adaptions i n Training Se t I t e r a t i o n m
Adaption Counter
I E n t i r e Training Set Correct ly Class i f ied
Write Maximum I t e r a t i o n s of t h e Training Set Exceeded
54
Set Operating Mode t o CLASSIFY
Pat terns t o be I Classif ied I
Se t Classif icat ion Counters = 0
I I
Compute T i = CSETi, j i = l t o I
u COUNT = 0
CALL BRM Y cor rec t incor rec t c l a s s i f i c a t i o n c l a s s i f i c a t i o n
55
V I
Write Correct Class i f ica t ion Pa t te rn j
Advance Correct Class i f ica t ion Counter
Write Number of Correct and Incorrect C las s i f i ca t ions
4
Write Incorrect C las s i f i ca t ion Pa t te rn j
1 Advance
Incorrect Class i f ica t ion Counter
I'
Enter Subroutine ADAPT
Compute I SG = PT,
+ Compute
NPULSE = SG/DT
Set INN = 1
ii Return
P
Set INN = 2
Set m = INN
v n = l
~ Compute =1 r~ W,,i = 0
I
<
m = INN i-1
Compute wn,i = 1 n = l t o N
I *
b 59
P Compute
w,,i = 0
I n = l t o N
Compute 1 wsi,gn,i = 0 I Compute 'sign,i = 1
I
I N N = m 9
60
4 Subroutine BRM
Compute
I Compute TREF = DTIME - Tmin
TIME < DTIME
61
1 n = l \
Compute cn,i = 1
Compute nn = N-n+l
, Y i
Compute Cn,i = 0
Compute PRESi = 0
\ f
-1 L A -
Compute COUNT = COUTN+l COUNT = COUNT-1
1 + + Compute
TIME = TIME + T.
62
1 I
Compute THESi = TIME - DTIME
I c
Return U
I Enter Subroutine CAT I c
Compute K T H = DTI&iYTHI/T~,
Compute COUNT = COUNT + KTHrcWth/Zn
Negative Posi t ive Negative Posi t ive
1
Set Set Category 1 Category 2
I I I
I b Return
64
Star t Input Program +
Read Design Parameters
I I Se t k = l
k = k + l
Pat tern Label Number of Pa t te rns
Write Pa t te rn Label
d Read Relat ive Input Pa t te rns on a Scale o f 0 - 100
I
Compute Compute PAT = T *VART PAT T -T *VAR . i,j c e t max i,j max c e t b