ED 042 358 TITLE INSTITUTION PUB DATE NOTE AVAILABLE FROM EDRS PRICE DESCRIPTORS ABSTRACT DOCUMENT RESUME EN 008 353 Programmed Instruction: A Brief of its Development and Current Status. Training Systems and Technology Series: Number 3. Civil Service Commission, Washington, D.C. Bureau of Training. May 70 57p. HC, Superintendent of Documents, U.S. Government Printing Office, Washington, D.C. 20402 (0-380-346, $0.55) EDRS Price MP-$0.25 BC Not Available from EDRS. Federal Programs, *Programed Instruction, Programed Materials, Trainers, *Training Techniques A non-technical synthesis of the major developments in programed instruction serves as .anintrodlIction-1: the contort of this paper. The paper inclddeSSoMe determination of the extent to which programed materials are being used by Federal agencies and how these uses compare with those found by Bryan and Nagay in an earlier study. The paper concludes with a chapter which gives guidelines to the trainer for the use and selection of programed materials. In appendices it lists programs in use by government training programs and provides background information on the methods of obtaining data for the study. A bibliography is appended. (JY)
58
Embed
Programmed Instruction: A Brief of its Development Civil ... · Origin and Development of Programmed Instruction. 3. Approaches to Programming. 5. The Role of Teaching Machines in
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
ED 042 358
TITLE
INSTITUTION
PUB DATENOTEAVAILABLE FROM
EDRS PRICEDESCRIPTORS
ABSTRACT
DOCUMENT RESUME
EN 008 353
Programmed Instruction: A Brief of its Developmentand Current Status. Training Systems and TechnologySeries: Number 3.Civil Service Commission, Washington, D.C. Bureau ofTraining.May 7057p.HC, Superintendent of Documents, U.S. GovernmentPrinting Office, Washington, D.C. 20402 (0-380-346,$0.55)
EDRS Price MP-$0.25 BC Not Available from EDRS.Federal Programs, *Programed Instruction, ProgramedMaterials, Trainers, *Training Techniques
A non-technical synthesis of the major developmentsin programed instruction serves as .anintrodlIction-1: the contort ofthis paper. The paper inclddeSSoMe determination of the extent towhich programed materials are being used by Federal agencies and howthese uses compare with those found by Bryan and Nagay in an earlierstudy. The paper concludes with a chapter which gives guidelines tothe trainer for the use and selection of programed materials. Inappendices it lists programs in use by government training programsand provides background information on the methods of obtaining datafor the study. A bibliography is appended. (JY)
CO
N! Programmed Instruction:
cpcl A Brief of its Development and
Current Status
PROCESS WITH MICROFICHE ANDPUBLISHER'S 'PRICES, 11,1CRO
FICHE REPRODUCTION ONLY.
Training Systems and Technolazy Series: No, HI
U.S. CIVIL SERVICE COMMISSION
BUREAU OF TRAINING
TRAINING SYSTEMS AND TECHNOLOGYSERIES PUBLICATIONS
No. I Instructional Systems and Technology : An Introduction, to the Field and Its Use in FederalTraining
No. II Application of a Systems Approach to Training: A Case StudyNo. III Programmed Instruction : A Brief of Its Development and Current Status
co
re\c,1-4'C)
w
PROCESS WITH MICROFICHE ANDPUBLISHER'S PRICES. MICRO-.FICHE REPRODUCTION ONLY.
.
Programmed Instruction: A Brief ofIts Development and Current Status
MAY 1970
PAMPHLET T-12
U.S. Civil Service CommissionBureau of TrainingTraining Systems and
Technology Division
For sale by the Superintendent of Documents, U.S. Government Printi, q OfficeWashington, D.C. 20402- Price 55 cents
TABLE OF CONTENTSPage
Introduction 1
Origin and Development of Programmed Instruction 3Approaches to Programming 5The Role of Teaching Machines in Programmed Instruction 8Fundamental Aspects of Programmed Instruction 8Applications in Federal Agencies.. 11The Next Step 19Summary 21Footnotes 22
ibi ug'i'aphy.. 23Appendices:
Appendix A.Sample of a Linear Program 25Appendix B.Sample of a Branching Program 26Appendix C.Survey Procedure and Scope 27Appendix D.Federal Government Organizations Surveyed 30Appendix E.Programs in Use 32
EXHIBITSPage
Exhibit A.The Programming Process 2Exhibit B.--Schematic DiagramLinear Program 5Exhibit C.Schematic DiagramBranching Program 6
Exhibit D.Schematic DiagramMultitrack Program 7
Exhibit E.Increase in the Use of PI 12Exhibit F.Frequency Distribution of Length of Programs 13Exhibit G.Sources of Programs in Use 15Exhibit H.Type of Instruction 16Exhibit I.Presentation Format 17Exhibit J.Locales at Which Programs Are Being Used 18
INTRODUCTION
Programmed instruction (PI) is no new phe-nomenon to Government trainers. A recent surveyconducted by the Bureau of Training found over2,300 programs currently in use in agency head-quarters, suborganizations and field offices. Thisrepresents a sixfold increase over the 382 programsin use 6 years ago as reported by Bryan andNagay in their article "The Use of ProgramedInstructional Materials in Federal GovernmentAgencies." 1
This evidence of comprehensive usage indicatesthat programmed instruction has established itselfas an accepted instructional method in Govern-ment-wide training, Although this usaon.is exten-sive in many agencies, in others its use is limitedand still in some is nonexistent. In order to extendthe usage of this approach, Government trainerswill need to equip themselves with a knowledgeof programmed instruction, its origins, applica-tions, and utilization in Federal Governmenttraining.
This paper, a part of the training systems andtechnology series, offers a concise, nontechnicalsynthesis of the major developments in PI. In ad-dition, the report includes some determination ofthe extent to which programmed materials are be-ing applied in Federal agencies and how theseapplications compare with the results compiledearlier in the study by Bryan and Nagay. Thepaper concludes with a chapter which gives guide-lines to the trainer for the use and selection of pro-grammed materials.
The current emphasis on the "systems a,p-ptrach" to training increases the need tounderstand the theoretical basis of programmedinstruction because it helped to foster the systemsapproach. The first paper of the series, Instruc-tional Systems and Technology: An Introductionto the Field and Its Use in Federal Training,described the systems approach as consisting ofseven essential steps :
1. Determining the need for training.2. Stating the objectives.3. Preparing measures of these objectives.
1
4. Identifying the knowledge alreadyachieved by the learners.
5. Selecting content and media to attain finalobjectives.
6. Implementing the selected methods.7. Evaluating procedures by measuring
performance.The steps of the programming process (exhibit
A) adhere to the principles of the systems ap-proach. Gabriel Ofiesh, a leader in the field ofeducational technology, has gone so far to statethat : "Prngrn ; +;,ns.rucon has einerged as thefirst valid system of educational and instructionaltechnology That our society has ever had." 2 At thevery least, it can be said that PI is a valid systemof instruction and has proven effective partiallydue to its reliance on the systems approach.
The product of the programming process is theprogram itself, the means by which the student isled through a series of questions, responses, andconfirmation of his responses until little by littlehe has progressed from small bits of relativelysimple knowledge to more complex principles.
Many writers have noted the similarity betweenthe PI approach and the tutorial system whichasks selective probing questions of a student, andthen lets him do the work of learning. The tutormust know his objectives and dissect the subjectmatter into small bits of information. Through alogical progression of elementary questions andanswers and through reward or correction ofanswers, the tutor brings the student to a grasp ofthe whole concept. Thus the procedure may belikened to the building of a picture puzzle wherethe complete configuration comes only through thecorrect placement of individual parts.
The manner of the program presentation maybe the programmed textbook, a teaching machine,a computer, or any variety of multimedia presen-tation. Regardless of the medium, the sameprogramming process applies.
The growing use of PI has generated much re-search, but often -without providing conclusiveevidence to many pressing questions. One impor-
Exhibit AELEMENTS OF THE PROGRAMMING PROCESS
DEFINITIONS ANDASSUMPTIONS
OBJECTIVES EVALUATION
PARADIGM OR MODEL INITIAL TESTING/SELECTION
REVISION
ORDERING ...,.....,CONSTRUCTION OF
"The selection of the subject to be pro-grammed requires an assumption by the pro-gram builder about the learner who will use theprogram. Next, he lists the objectives of the ma-terials to be programmed. These factors theninfluence the programmer's choice of a para-digm for the arrangement of the items.
He can now order the subject matter in thebest possible sequence and construct the itemsin accord with established learning techniques.As soon as short sequences have been completedthey can be tested and revised on an initialbasis. Later, after exposure to a large numberof students, the sequences can be analyzed forerrors and shortcomings. Besides, the sequencescan be evaluated for effectiveness in terms of
ITEMS
how the student learns from them, and thisinformation can be used for revision of theprogram. The complete experience has a cyclicalquality since it may be used as an aid in theselection of subsequent units of subject matterfor programming.3 "
The similarity of the programming process tothe systems approach can be seen in the analyti-cal procedure followed by each. However, thedetermination of the need for training as wellas the selection of programmed instruction asthe media must be made before the programm-ing process begins. Such steps as formingobjectives, identifying the learning group, andevaluation are inherent in the overall systemsapproach and the programming process forsuccessful outcome.
2
tant area being researched is the evaluation of theprogrammed instructional effort. Schramm re-ported in 1964 that of 36 reports which comparedconventional teaching methods to programmedmethods, 18 showed no significant differencebetween the two when measured by results on acriterion test. Seventeen, however, showed a sig-nificant superiority for PI, with only one showingsignificant superiority for traditional methods.'
Of course, such qualifying factors as idealphysical conditions in the classroom or the noveltythat new methodology presents to students, tendto affect the overall outcome. The crucial questionto ask, however, is how well and how effective thismethod of instruction is as a common means ofinstruction, not just in laboratory situations. ":Vhilethe results of research such as Schramm's may not
be taken as proof that PI is superior to conven-tional teaching methods, they may be regarded asindications that, at the very least, it is as effectiveas other methods and should be considered foroperational use along with any of the othermethods.
Despite its relatively recent emergence, pro-grammed instruction should not be regarded as anentirely new venture, without roots in an estab-lished theory of the psychology of learning.Human behavior has remained virtually constantthroughout recorded civilized history, however theability of educators to recognize and direct theinstructional process has been a comparativelyrecent development. Although PI is a relativelyrecent development, it is an application of wellfounded principles of learning.
ORIGIN Nn DEVELOPMENT OF PROGRAMMRD INSTRUCTION
One of the fundamental concepts on which PIis based is the concept of reinforcement in learning.This notion was introduced by Thomdike in theearly 1900's as his famous "law of effect". Statedbroadly, this law states that responses which arefollowed by satisfying states of affairs are selectedand fixated (learned), while those that are fol-lowed by unsatisfying states of affairs are elimi-nated. Very soon Thorndike dropped the latterpart dealing with the elimination of responses."Satisfying states of affairs" has come to mean"rewards ". Hence the law states that for responsesto be learned, they must be rewarded. Because ofthe theoretical ambiguity of the notion of rewardand the difficulty of defining it operationally, thelaw of effect has undergone many revisions in thehands of later looming theorists. Out of such ef-forts has emerged the modem notion of reinforce-ment, particularly as developed by B. F. Skinner,which has become so basic in PI, and to which weshall return later.
In the early 1920's, Sidney Pressey's observa-tions of classroom instruction led him to the de-velopment of a device designed to apply the "lawsof learning", defined by Thorndike, to classroomteaching. His original device was used for testing;however, later the teaching potential as well as thetesting capability of the machine was also demon-strated. He produced his device to provide im-mediate knowledge of correct responses andpractice of those responses. thus simulating a learn-ing situation of a tutor for each individual student.
3
Pressey's "teaching" machine worked in the fol-lowing manner: when a question appeared in thewindow of the machine, the student responded bypressing a button corresponding to the answer heselected. If the choice was correct, the machinerevealed a new question; if the answer was incor-rect the question remained, giving the student an-
ther chance to respond. The machine also recordedcorrect and incorrect answers, allowing the teacherto keep track of individual student scores.
Despite the continued efforts of Pressey andothers, the movement toward using teachingmachines did not catch on. Lysaught and Williamsoffer two explanations for the lack of success inthe program.
First no provision was made for the systematic pro-gramming of materials to be used in these machines, andsecond, the onset of the depression and Its impact on socialconditions and education offered an unfavorable environ-ment for an "industrial revolution" in the nation'sschools .5
The revival of interest in programmed instruc-tion did not come until the work of B. F. Skinneron operant conditioning in the 1950's. His firstmajor work, The Behavior of Organisms, pub-lished in 1938, presented his theory based on ani-mal studies.° However, it was not until 1954 thathe published his article "The Science of Learningand the Art of Teaching", which applied his find-ings to education."(
Historically, Skinner's ideas are a continuationof Thomdike's research on the law of effect.
Skinner's research focused upon a particular andvery important class of responses called operant&Operants are responses which are not made to anyknown external stimuli; rather, they are emittedby the learner and enable him to operate upon hisenvironment. Voluntary acts are an example ofoperants, while reflexes are not. Skinner developeda technique, called operant conditioning, wherebya response which a person emits could be strength-ened (made to appear more frequently) in thepresence of certain cues called discriminativestimuli. The technique involves strengthening thedesired response by a special class of events calledreinforcers. A reinforcer is defined as any eventfollowing an operant which increases the tendencyto emit the operant. Thus, to condition a response,one must discover a reinforcer for it, and arrange
thnt. 71101, Opp pverftri re.p?..110. is rif!cl,P, _
be followed by the reinforcer. In this way, makingthe response operates upon the environment thusproducing the reinforcer.
In cases where the desired response is not al-ready one which the learner can emit (not alreadyin his "response repertory", as Skinner puts it),Skinner developed a technique called "shaping"whereby the desired response can be synthesizedfrom other fragmentary or partial responses whichalready do exist in the learner's repertoryallby the use of an appropriate program ofreinforcement.
Skinner demonstrated shaping in his animalstudies. He showed that by reinforcing the subjectat first for making just some motion in the rightdirection of the desired final response, and laterfor coming a bit closer and so on, the subjecteventually attains the total correct behavior.Skinner illustrated this by instructing a pigeon topeck a key only when it was lit, not unlit. At firstthe pigeon was rewarded when his beak came nearthe lit key, and finally when he actually tapped thekey only when it was lit. This same principle ap-plies to a student learning a new process. Skinnerbelieved that the principles of shaping and dif-ferential reinforcement could not only be used toteach a simple task or lesson but to develop anentire curriculum of study.
Like Thorndike, Skinner believes that punish -merit does not eradicate the tendency to make awrong response, but rather leads only to a tempo-rary suppression of the response and certain at-tending emotional difficulties. Such a view ledSkinner to one of his biggest disagreements with
4
the educational community which, he claimed,operates almost totally on the basis of punishmentto "encourage" learning.
Note that Thorndike's old idea of "reward" hasnow become broadened into the notion of rein-forcement. While it is true that many reinforcerslook like rewards, many do not. Skinner himselfhas seen no need for the concept of reward, findingit both unnecessary and difficult to define satisfac-torily. Nor has he concerned himself much withthe theoretical questions of why reinforcers rein-force or what reinforcers really are psychologi-cally. His view was that reinforcers can be identi-fied only by empirical observation, but onceidentficd, can be used to train behavior. Certainreinforcers, however, are of sufficient cultural gen-erality that they can be used to reinforce a great7re ri,ty of rcspc.,:amat isiac;iley and praise would beexamples. But perhaps the most useful reinforcerOf all is a class which has been variously called"knowledge of results" or "feedback." Feedbackconsists of information given to the learner con-cerning his performance in learning. Reinforcersof this type are not only powerful but also are themost available and programable. For these reasonsfeedback is virtually the only reinforcer used todayin PI.
Skinner, in his own approach to designing auto-matic self-instruction devices, retained Pressey'suse of feedback as the principal reinforcer, butadded one important difference: the learner wasnot given the alternative responses to chooseamong, but instead was required to compose orconstruct his own response in spaces provided, andthen as a printed tape advanced, the desired re-sponse appeared for comparison with the com-posed one thus providing immediate feedback. Inbuilding his devices, Skinner also took advantageof a principle which he had discovered in his workon operant conditioning; namely, the longer thedelay between response and reinforcer, the less thereinforcing effect. And to avoid the possibility ofthe learner making incorrect responses (whichwould have to be unlearned), Skinner added thenotion of prompts or guides so that only the correctresponse had a high probability of being emitted.In this way the learner could innovate the correctresponse on his own, emit it with a high probabil-ity, and have it reinforced. With the publicationof these ideas in his classical article of 1954, thelagging revolution in educational technologyreached a new level of activity.
APPROACHES TO PROGRAMMING
Two basic methods for the programming of ma-terials for responses and reinforcement haveemerged from the research done during the past20 years. One is known as linear programming: "asequential development of the material throughwhich each student, regardless of his response,proceeds in exactly the same order".8 It is a directoutgrowth of the work of Skinner. The othermethod is branching or intrinsic programming. Asdefined by Norman Crowder, the chief figure iden-tified with its beginnings, intrinsic programmingis "the technique of using a student's choice of ananswer to a multiple choice question to determinethe next; material to which he will be exposed".9
The two methods have much in common but dis-agree on how to simulate the pedormance of anactual and effective tutor.
Linear ProgrammingThe format of the linear program emerges di-
rectly from the training techniques of Skinner.Information followed by a related question is pre-sented in small steps to the student. He respondsafter each step by composing and indicating' theanswer to a question. The student then receivesimmediate confirmation of the correctness or in-correctness of his response. Reinforcement for thelearner using programmed instruction comes fromthe feedback or knowledge of results given imme-diately after each response. The same elements :instruction, response, and reinforcement are re-peated in each step. (See exhibit B.) The sameinformation, presented in different contexts, pro-vides for maximum learning experience. Sinceonly correct responses are practiced and rein-forced, only correct responses are learned.
Exhibit B
Skinner developed a machine designed more toteach than to test as Pressey's had done. Insteadof using multiple choice questions and answers,Skinner allowed the student to construct his ownresponse, write it down, and then compare it withthe correct answer provided by the machine. Theindividual responses to the questions thus becamean entire complex of behavior or knowledge, thatis, the behavior was shaped according to the de-sires of the programmer or teacher. After the ap-pearance of Skinner's machine, his theory wasfurther applied in the development of pro-grammed textbooks and other media.
Intrinsic or Branching ProgrammingAlthough the intrinsic approach to programmed
instruction maintains the same concerns aboutlearning as the linear approach, it does not derivefrom a stated learning theory. Davey and McDon-nell in their book, Programmed Instruction, de-scribe Crowder's views on the theory of intrinsicprograming in the following statement :
Crowder says that the intrinsic programmers do not haveaccess to any "philosopher's stone" which provides anunderstanding of human learning. He concludes thatthere is no infallible process for bringing learning about,and suggests to predictably achieve a deCred result, onemust either have an infallible process tr, bring about theresult or one must have a means of determining whetherthe result has been achieved and of taking appropriateaction on the basis of that determination."
Crowder has applied his "theory" by devising a"means of determining whether the result has beenachieved"; i.e., the multiple choice question. Addi-tionally, and most important, the "taking of appro.priate action" means that the student's response topreviously presented material determines the next
THE LINEAR PROGRAM - SCHEMATIC DIAGRAM:=111
380-346 0- 70 - 25
step he takes; i.e., the program is determined in-trinsically. Exhibit C represents a simple sche-matic drawing of this concept.
In item 1 the student is presented with informa-tion. His choice of answers will lead him to frame2, 3, or 4. In this drawing, if the student has chosenthe correct answer he will proceed to frame 3where he will be presented with new information.If he choses a wrong answer in response to frame 1he is directed to either frame 2 or 4 dependingupon his response. The incorrect response framesin a simple program will give additional informa-tion to the student then direct him to return to theoriginal frame and select another answer. In amore complex program, an incorrect responsemight lead the student through an entire subpro-gram before returning to the next frame.
The major difference between linear and in-trinsic (or branching) programs is that linear pro-grams simply inform the student of the correctnessor incorrectness of his response while intrinsic pro-grams use the student's response to determinewhere he will proceed in the program. Accordingto the underlying theory of linear programming,
Exhibit C
the best learning situation is one in which no errorsoccur, and the program is designed to insure thatthe correct response is made with a high probabil-ity. If errors should occur, the student is simplyinformed of his error through feedback and hecontinues to the next frame. Intrinsic programm-ing, however, does not try to completely eliminateerrors. The student cannot go on to new informa-tion until he has demonstrated that he has under-stood or "learned" the concept already presentedto him. When he makes an error he is reinstructeduntil he has learned the material, thus taking intoaccount individual differences in the backgroundof the students and assuming that they will not allhave identical responses.
By testing the students' comprehension of theinformation presented, the intrinsic program canmake the steps larger than those of the linear pro-gram. The steps in the linear program must ofnecessity be very small so that no mistakes willoccur. The instrinsic can allow for mistakes to oc-cur because it provides a learning mechanism forerrors by reinstructing the student when errorsoccur and allowing him to make another choice.
THE BRANCHING PROGRAM - SCHEMATIC DIAGRAM
6
Further Developments of Programming Methods
The linear and branching methods while re-maining as the two basic programming techniqueshave been adapted for other forms of programs.These programs which are designed to achieve ad-ditional objectives in training sometimes combineelements of the two methods into one "new"technique. According to David Cram these de-velopments can be grouped into two categories :self-organizing and criterion programming." Aself-organizing program, originated by GordenPask, is in essence a complicated form of a regularbranching program. This type of program is runby a computer which adapts the program to theneeds of a student as shown by the type of answershe gives. It supposedly has an infinite number ofpossible branches available and is therefore easilyadaptable to differing abilities. Cram has identi-fied another method as criterion programming. Thename is derived from the process of pretesting thestudent on a criterion test to determine what hisneeds are. On the basis of the responses given onthe pretest, it is decided what type of materialsare best suited to an individual student and he isthen directed to movies, books, or other teachingdevices which will present the chosen material tohim.
A derivation of simple linear programming ismulti-track programming, a type of criterion pro-gramming, described by Davey and McDonnell."As in branching, this method is adaptable to theneeds and abilities of a range of students. Bymeans of a pretest the level of student ability is
Exhibit D
MULTI-TRACK:
determined. Dependent on these results he is di-rected to one of several linear programs (A, B,or C below), each of which presents the samematerial but does so in varying step sizes.
A superior student will be able to use a relativelyshort program with fairly large steps enablinghim to remain interested in the material. A poorerstudent will be led through the same informationin a longer 'program with very small steps in orderto minimize his misunderstanding or "gettinglost" in the program.
A paper by Susan Markle published in 1963reported another derivation of the-linear programwhich is quite adaptable." Taking a cue frombranching, it has provided frames which deter-mine how well the student has understood someof the previous material. If his answer is incorrectit will refer him back to the earlier section whichdealt with this information so that he will redothe program in this area and, hopefully, learn itthe second time around.
Another grouping of approaches to programm-ing varies according to the construction of theprogrammed sequence of materials. Among theseare "pragmatic", "ruleg", and "mathetics" systems.Pragmatic construction is done by the placementof behavioral aims into a logical order. Presenta-tion of material in terms of rules and examples isthe approach used by the ruleg system. Mathetics"is characterized, largely, by its concern over usingthe largest response units possible as the beginningpoint of instruction and by its concern with thecriterion of mastery".14 It is a very complex ap-
7
proach which systematically applies the principlesof reinforcement theory to the analysis and recon-struction of learning.
Determination of the best type of program to
be used in a given situation can only be determinedby the nature of the topic to be programmed andthe group of students for whom the program isintended.
THE ROLE OF TEACHING MACHINES IN PROGRAMMED INSTRUCTION
Along with the questions On what types of pro-grams are best for which audiences, there is somecontroversy on the role of the machine in pro-grammed instruction. The argument seems to bewhether a machine or a programmed text can do amore efficient and better teaching job. Most of the,statements have been inconclusive. For example:
There is no psychological principle of which I am awarethat would oppose the use of teaching machines. Con-versely, there are several machines which would be aneffective and efficier.t educational tool. Ultimately we haveto develop better theories of behavior, particularly thoseof transfer and symbolic processes, in order to make thebest use of teaching machines.iG
Mechner and Cook cited nine published studieswhich show no significant difference in effective-ness between text or machine presentation of pro-grams. Their conclusion is that ". . . no teachingmachine has yet been developed that administersa program more effectively than does a pro-grammed textbook." 10
Although Pressey's machine did not succeed instarting a mechanical revolution, Skinner's ma-chine began a wave of interest in this area. Infact, it. caught on so well that many manufacturersdeveloped a variety of new machines which werecapable of teaching students material in novelways. The problem was, howei-lr, that while allthe research and money went into the technicaldevelopment of the machines, no one botheredto write programs to put in them, and there wasno "software" for the student to learn by meansof these devices. More recently this disparity hasbeen recognized and many more programs havebeen developed, which at least provide a rationalefor the existence of the machines.
One of the main criticisms of machines is that
they destroy the flexibility which programs shouldhave. In most cases, early machines restrictedthe types of possible responses to multiple choice.On the other hand, with the adaptation of com-puters to this problem, much flexibility has beenregainedsuch as in the branching techniquewhere an infinite number of branches are avail-able to the student who can be guided by the com-puter to the proper frames. A device of Crowder'sknown as the "Tutor" uses film and other teachingaids to present a program adapted to an individualstudent as he progresses through the program.There are, however, the warnings of men such o sGilbert who remarked on the dangers of begin-ning with a machine and developing a programwhich will fit it : the ingenuity of the programmerand the quality of the program may be sacrificedin the process of fitting a program to a machine.17
Teaching or testing by machine offers some pre-vention against cheating which a text cannot pro-vide and also permits automatic scoring whichmay be of help to both teacher and student.
A machine can also provide greater control overthe stimulus presented. The comparative cost ofmachines and programmed texts have caused somecontroversy because with most machines the initialinvestment will be large. However, if they can beused a great deal, the per student cost will de.crease. Programmed texts are expensive to developand publish but in most cases the texts may beused many times before replacement is necessary.Although cost is definitely a factor in choosingbetween equally good programming, the reader iscautioned that cost should not be the sole factor indeciding which method is most effective in a par-ticular situation.
FUNDAMENTAL ASPECTS OF PROGRAMMED INSTRUCTION
So far this paper has covered the history ofprogrammed learning and the development of thetypes of programs. There continues to be much re-search and controversy revolving around basic
8
issues common to all types of programming tech-niques. Most of the controversy has centered onsome of the very basic issues : the nature and roleof stimulus, response, reinforcement, error, pac-
ing, and size of step. As well as mentioning someof the research which has been done, each of theseitems will be defined in relation to the whole con-cept of programmed instruction.
Stimulus and Response DevicesThe stimulus items in programmed instruction
can be either questions or incomplete sentences,and the response is the answer given by the stu-dent. Douglas Porter in his article "A Critical Re-view of a Portion of the Literature on TeachingDevices", classifies teaching devices into threetypes: stimulus devices, response devices, andstimulus-response devices.18 Stimulus devices arethose such as books, tapes, movies, or phonographrecords which provide the information or "learn-ing content" without any provision for determin-ing whether the student has learned anything orfor letting him know whether his impressions arecorrect. The second type, response devices, allowsthe student to practice responses, without provid-ing any stimulus information. An example of aresponse device is the typewriter.
The most effective type of device for teaching isthe third type, stimulus-response device. "Stimu-lus-response devices (or stimulus devices andresponse devices used in combination) are de-signed to present a sequence of stimuli (content)and provide the setting in which appropriate re-sponses may be made and rehearsed (process)." 1°These might be any one of a number of pro-grammed instruction devices: a machine such asPressey's punchboard, a programmed text or anyof the many "teaching machines" now being man-ufactured.
In SR devices the stimulus is usually a questionor incomplete sentence where the correct responseis to be made or chosen by the student. Opinionsand research findings vary on how much of a cueor hint should be given to the student to help himanswer the question. (Should one or several lettersof the missing word be provided or should therejust be a blank?) Another common form used toelicit the correct response is the analogy. Here thecorrect answer is strongly hinted at by the con-text of the frame. For example, "If water mustbe heated to make steam, we can reason that steammust be to make water." In this case it isunlikely that the student will miss the obviousanswer of "cooled". One of the other importantways of evoking a correct response depends onthe stimulus and knowledge provided in one of
9
the preceding frames. Any of these methods, ormore than one, may be used in a successfulprogram.
The form of the response required in a pro-gram has generated much research. Here theproblem has been whether the response shouldbe overt or covert and if overt, whether it shouldbe oral or written. Schramm reported that inmost studies which compared overt or consciouslymade responses to covert or "thought" responses,no significant difference in amounts of learningwas found; however, the covert response wasusually found to be quicker.20 "No significant dif-ference" was also the finding in most comparisonsof multiple choice and constructed response itemsalthough in individual cases, one might be moreeffective or faster than the other." This finding iscounter to what Skinner has espousedthat theact of the student constructing the response on hisown is one of the most valuable assets in .pro-grammed instruction and one of the elements thatmakes it an effective method.
ReinforcementIt seems generally agreed that systematic re-
inforcement is one of the features which makesPI function as well as it does. However, there isno generally accepted theory among psychologistsas to how reinforcers affect learning. Those of theSkinnerian school have little interest in this theo-retical issue. Their approach is practical andempirical : discover by experience and experimentwhich events can reinforce and then use them inaccordance with researched-derived principles ofprogramming. The drive-reduction theorists, onthe other hand, see a reinforcer as something whichsatisfies a drive or need. (It has been shown ex-perimentally, however, that some reinforcers donot reduce any known drive.) According to thisview, feedback reinforces because it satisfiessuch needs as the need for achievement, egogratification, social approval, etc.
These theoretical issues aside, it has been wellestablished experimentally that feedback is a ve^ypowerful reinforcer in human learning; in fact,it seems that no learning can occur without it.Most psychologists believe that feedback helpslearning in ways beyond simply reinforcing orrewarding correct responses. Feedback may alsoserve as an incentive and an instrument to pro-vide guiclric,e. Regardless of the psychologicalprocesses involved, it is clear that feedback, if
effectively programmed, is one of the most influen-tial and administratively manipulatable of allreinforcers.
Most psychologists agree that positive reinforce-ment is more effective than punishment in shapingbehavior and also that the immediacy of thereward is important. The traditional classroomsetting does not offer optimum conditions for theadministering of reinforcement. Teachers cannotgive individualized instruction nor can they pro-vide immediate reinforcement for completedassignments. Besides the obvious value of elimi-nating delays in time, programmed learning hasother advantages over traditional instruction.First, textbook reading often lacks positive rein-forcement because the student finds the materialuninteresting and has no way of actively respond-ing to what he reads. He often reads meaning-lessly only to get it done and avoid any unpleasantconsequences which may result if he does not showoutward progress. Second, traditional method-ology usually allows for reinforcement only whena student gives a totally correct answer. Pro-grammed instruction, on the other hand, rewardsthe student as he completes small steps in the totalanswer because it reinforces him as he progressestoward the total concept.
Size of StepSome other issues related to programming are
themselves somewhat interrelated. These includesize of step, error and pacing, which all arousesome controversy and are all important in the twomajor programming techniques. Size of step is theamount of information to be learned in each"frame" or unit.
Some researchers believe that size of step is oneof the crucial elements and that their size shouldbe very small so that there is no opportunity forthe learner to make errors which might lead to thereinforcement of incorrect responses. Skinner andthe linear programming process follow this ap-proach. Lumsdaine refers to a study by Evans,Glaser, and Homme in 1959 which shows that thesmaller the steps, the more effective the program.But, they also pointed out there must be a pointbeyond which the principle of small steps will nothold true.22
The format of the branching program enablessomewhat larger steps although it still adheres to
10
the principle of small steps. The remedial workwith erroneous responses means that the steps donot have to be so small that all errors are elimi-nated as in linear programming. Additionally,more complex intrinsic programs allow for variedotep size depending on the type of individual andhis ability to learn by providing alternate pathsthrough the material.
Whether the step size should be very small andfixed or whether the latitude of a branching pro-gram will be worthwhile will depend on the pro-jected subject matter and audience of the program.
ErrorThe role which error plays in PI is closely re-
lated to size of step. While linear programs try toavoid errors completely, branching programstreat errors diagnostically to give the student moreinformation on a topic which he misunderstood sohe can make a correct response. Intrinsic programsdo not allow the prolonged existence of error, thestudent cannot go on to new information untilhe has demonstrated comprehension of what hasalready been presented.
On the other hand the linear programs, whichhave no provision for the correction of error,attempt to eliminate error so that a SR bond isnot formed to an incorrect response. A few errors,however, will occur even in the best programs. Itis only where repeated errors appear on a singleframe or extensively throughout the program thatthere is need for concern.
PacingPacing is the speed which the student pro-
gresses through the program. Self-pacing, or thestudent determining his own rate of progressthrough the program, has been identified as one ofthe characteristics of PI.Summary
The reader has been presented with a concise,nontechnical synthesis of the major developmentsin PI. The origins, theory, approaches, and funda-mental aspects have been reviewed.
The next section cf the paper is a report on theextent to which programmed materials are beingapplied in Federal agency training programs. Thisreport should aid trainers when considering theuse of PI as a possible instructional method fortheir own training programs.
APPLICATIONS IN FEDERAL AGENCIESTo what extent is PI being used in the Federal
Government? The Bureau of Training conducteda survey of Federal activities during the summerof 1969 to answer this question. The survey fol-lowed the approach first used in 1963 by Bryanand Nagay of the Office of Naval Research. Theirsurvey was "of the extent to which programedinstructional materials are being used in the Fed-eral Government." 23
The current survey identified over 2,300 dif-ferent programs being used in some 121 head-quarters, suborganizations, and field offices. (Seeapp. E.) The compiled list of programs is notentirely inclusive of all programs in use in theFederal Government. An effort was made to becomprehensive but it would be difficult, if not im-possible, to identify every instance of use. Thetraining function in agencies is so widely dispersedthat it was difficult for all training activitiesthroughout headquarters and field offices to beidentified.
As was illustrated in the "Origins anal Develop-ment" section of this paper, programmed instruc-tion has been around for quite sometime but hascome into popular use rather recently. The AirForce reports that it began using PI as a regularinstructional technique in 1954, 15 years ago. By1962 the Army, Navy, Forest Service, and SocialSecurity Administration were also using PI. To-day its usage is widespread throughout Govern-ment agencies. Exhibit E illustrates the increasein the number of organizations using PI, particu-larly within the past 5 years.
Topics CoveredA look at the list ("Programs in Use"Appen-
dix E) will show that the topics of the programsare as diversified as the many functions for whichthe Federal Government must train its employees.The following list of program topics illustratesthis diversity :
Air Force Traffic ControlBlue Print ReadingBasic Computer SystemsDay and Night Storm Signals and Their
MeaningsEmployer-Employee RelationshipsFood Borne Disease InvestigationInstructor TrainingLaw EnforcementOrientation to War on Poverty
11
A great number of programs are of a technicalnature. This can be explained by the fact that themilitary, the largest user of PI, trains its men formany technical skills. Some of their programs are:A. -7 Aircraft Fuel System Familiarization, BombRelease Units, Jet Power Plant Designations, etc.Many of the programs used by organizations aredeveloped to meet unique agency needs. For ex-ample: Forest Service Orientation, Analysis ofTax Returns and Coverage of Exemptions. Thesecourses are used to train employees of the agencyon how to administer their specific job task ormission. Other programs, more general in nature,often cover basic skills. One finds the same coursebeing used throughout many agencies. Examplesare : Algebra, Basic Statistics, Effective Listening,English 3200 and Introduction to PERT amongothers.
Length of ProgramsJust as there is a great variance in the topics
of programs, there is variance in the length ofprograms, which range from less than one-halfhour to 270 hours. These figures represent onlythe average amount of time required to completethe program. For any given program, the comple-tion time will vary because the pace of pro-grammed instruction is determined by the student.
Data on program length was reported on 1,789programs. Although there is a wide variance inprogram length, most of the programs are rela-tively short-94 percent are 15 hours and less.This is comparable to the 1963 study in which 86percent of the programs average 15 hours andless.24 However when comparing the figures forthe number of programs of 1 hour and less thereis a significant difference. Sixty-four percent ofall the programs in the current study are 1 hourand less in length while the earlier study showedonly 11 percent to be 1 hour and less. (Seeexhibit F.)
Both surveys show a prevalance of relativelyshort programs, however there appears to be atrend toward even shorter programs. The shortprogram is generally characterized by the pre-sentation of a single concept which has severalapparent advantages :
(1) When using a short program, only the spe-cific needed parts of a course are presented.For example, rather than an entire programon English usage, there might be individual
Exhibit E
Number ofOrganizationsusing PI
INCREASE IN THE USE OF PI
( length of time using PI, by organization )N=34
SPRING 1969
15 14 13 12 11 10
Years using PI
Less wa.7 6 5 4 3 2 1 than
one,
12
Exhibit F
FREQUENCY DISTRIBUTION OF LENGTH OF PROGRAMS
( in Instructional Hours )
N=1789
Lengthin Hours
Number ofPrograms
Lengthin Hours
Number ofPrograms
1/2 and under 262 16 - 20 331/2 _ 3/4 169 20 - 25 23
3/4 - 1 136 26 - 30 6
1 589 31 - 35 4
2 237 36 - 40 17
3 85 41 - 45
4 44 over 45 26
5 38
6 26
7 12
8 17
9 6
10 25
11 12
12 9
13 1
14 6
15 6
380-396 0 - 70 - 3
13
programs on punctuation, commas, spelling,etc. The student will study only those spe-cific programs for which he has a need.
(2) The shorter program requires a shorter spanof concentration thus helping to eliminateboredom.The short program provides for easier useand coordination when it is used in a class-room setting. This is because the range ofcompletion time for a 1-hour program isless than that for a 20-hour program. Thusshort programmed segments can be coordi-nated well with other types of classroominstruction.
(3)
sources of ProgramsEach respondent was asked to indicate the
source or sources of programs for his organization.Of 40 responding organizations, 29 purchase com-mercially prepared programs (off-the-shelf), 18develop their programs in-house, and 11 have themdeveloped under contract. (See exhibit G.)
It should be noted that these figures do not rep-resent the number of programs from each of thesesources. If the data had been collected for eachprogram, it would probably be found that thegreatest number of programs were developed in-house. Those organizations which are the largestusers of PI have developed their own in-housecapacity whereas the smaller and newer users ofPI tend to purchase more off-the-shelf and contractprograms.
Type of InstructionForty-one organizations reported on how their
programs are used. (See exhibit H.) The mostcommon use is for standard instruction, that is"training in its usual senseregular traininganticipated in the lesson plan to which all traineesare routinely exposed." Seventy-one percent of therespondents use programmed instruction in thismanner. About half of the respondents use PI for
14
remedial instruction and for preparatory instruc-tion. Preparatory use is made of "programscovering material not previously learned andadministered prior to an established course," andremedial programs are used to overcome deficien-cies. Refresher programs are used as reviewsgiven before a standard course, and self-develop-ment are programs which "are not specific to anyparticular course, objective. They are usually vol-untary." 25 The use of programmed instruction forthese two purposes is somewhat less extensive.Each is used by 13 of the 41 respondents or 32percent.
Presentation FormatDiversity of media for presentation of pro-
grammed instruction is increasing according to theagencies. Initially there were machines such asthose developed by Pressey. Then the commonformat became, the programmed text which is cur-rently the most widely used media. All but oneof the organizations reporting on their use of PIstated that they use the programmed text. In addi-tion many agencies have added teaching machinesand an increasing amount of other media such astape-slide, tape-filmstrip and other devices. (Seeexhibit I.)Where Used
Programmed instruction in Federal Governmenttraining is most generally administered, "usuallyon a scheduled basis, to individuals during theirnormal school or working hours in a more or lessformal classroom setting." Sixty-six percent of theorganizations surveyed reported that PI is admin-istered as "honiework", that is "individuals aremade responsible for their own instruction andcomplete programs on a self-imposed schedule,either at home or in libraries, study halls, or thelike." 26 On a somewhat smaller scale, PI is admin-istered as correspondence courses, 27 percentof the organizations use it in this way. (Seeexhibit J.)
Exhibit 0
SOURCES OF PROGRAMS IN USE
( by Organization )N=40
Percent
CommercialPurchase29 of 40
In-house18 of 40
15
Contract11 of 40
Exhibit H
Percent
100
80
60
40
20
TYPE OF INSTRUCTION
( by Organization )
N=41
Standard29 of 41
Remedial Preparatory Refresher Self-Deveiopmt21 of 41 20 of 41
15
13 of 41 13 of 41
Exhibit I
Percent
PRESENTATION FORMAT
( by Organization )
N=41
Written Text Teaching Machine Tape Film40 of 41 16 of 41
17
10 of 41Other6 of 41
Exhibit J
LOCALES AT WHICH PROGRAMS ARE BEING USED
Percent
Classroom38 of 41
( by Organization )
N=41
Homework27 of 41
18
Correspondance11 of 41
THE NEXT STEPDoes programmed instruction have a place in
your training activities? The answer to this ques-tion is the next step.
The next step belongs to you. Hopefully thereading of this paper has led to some thoughts onyour training activities and whether the imple-mentation of programmed instruction is applic-able to your training needs.
First and foremost, however, you must deter-mine that the need for training really exists. Afterthis initial step follow the steps of the systemsapproach described in Instructional Systems andTechnology: An Introduction to the Field and ItsUse in Federal Training and summarized in theintroduction. You are then ready to select the bestmethod of instruction to attain your objectives.
In order to make a well-founded decision to useprogrammed. instruction, or any instructionalmethod, you must make a preliminary evaluationof a method and of in dividual programs usingthat method to determine if they will meet yourneeds.
The reader might ask, "How can I learn moreabout programmed instruction ?" This paper hasgiven an introduction. Sources for gaining furtherknowledge might be :
1. Books such as those listed in the bibliographyof this paper.
2. Courses such as those workshops sponsored byuniversities, given by the American Manage-ment Association or by the Civil ServiceCommission.
3. Visits to agencies which are already using PIin their training programs.
Making the Decision To Use PIConsideration of the use of PI should include a
review of some of the claimed advantages for pro-gramed materials listed below.27
1. First and probably most important is indi-vidualization of instruction. Students neednot all go to courses at the same time nor forthe same amount of time. Each is allowed towork at his own rate and only with that mate-rial which his need determines.
2. Programmed m aerials can reduce theamount of time required to teadh. Becauseprior determinatioi of objectives and studentability have been made, only essential ele-ments are taught. Questions asked by one
19
student need not be heard by all who under-stand the material.
3. Programmed materials are easily packagedand disseminated to dispersed training activi-ties. This eliminates duplication of coursesand the long-distance travel of students orteachers to and from courses.
4. A prepared program may be used by a singlestudent when he needs the training. it is notnecessary to wait until there are enough stu-dents to fill an entire class.
5. Programmed materials "can imnrove thelevel of performance and reduce the incidenceof failure among students." 28 Stuktmts aremeasured on what they actually learned asbased upon the objectives. Failure is reducedbecause the construction of the program de-mands that the student has "learned" materialin one frame before he can progress to thenext. Programmed materials provide a meansof evaluating results by the preestablishedcriterion test. Due to this testing, weaknessesin the program may be overcome thus en-hancing student learning.
Sources of ProgramsIf the advantages of PI are congruent with your
particular needs, you have three sources of pro-grams : off-the-shelf, contract, and in-house.
0 if-the-shellThere are many off-the-shelfprograms available covering almost every imagi-nable topic and their number is constantly increas-ing. Several references you can use are listedbelow.
1. Programmed Learning; a bibliography ofprograms and presentation devices, 4th ed., byCarl H. Hendershot (Bay City, Mich., 1967).
2. Programmed Instruction Guide, Northeast-ern University, Instructional Technology In-formation Center (ENTELEK, Inc., New-buryport, Mass.), Published every 6 months.
3. Appendix E of this paper, "Programs inUse".
There are a number of factors to consider in anoff-the-shelf program to insure that it will be effec-tive to meet the objectives and defined group oflearners. Questions that must be asked about theoff-the-shelf program under consideration includethe following."
1. Are the publishers list of objectives com-patible with yours ?
2. Is the target group of the publisher's pro-gram relatively the same as your group oflearners ? You need to know the educationallevel, job experiences, ages, and other relativefacts about the students involved.
3. Does the publisher provide validation data?The publisher should make available val ida-tion data of results on criterion tests. Youneed evidence to assure that the students havelearned from the program.
4. Is the cost comparable with other similarprograms ?
5. Do you need machines or other special equip-ment in order to use the program?
6. Is the format such that it can easily be up-dated or revised?
At this point you should have been able to nar-row down the number of programs. A more care-ful study of the internal progam characteristicsmust be made at this point. As well as review bythe trainer, it would be a good idea to have severalstudents work through the program and give anevaluation of it. Points to consider in workingthrough the program include:
1. The language and comprehension level.2. The scope, depth, accuracy, and currency of
the information presented.3. The techincal construction of the program
such as step size, login' ordering of steps,variety and relevency of responses, andfrequency of review.
I. The writing style and whether informationpresentation is interesting or dry and . dull.
5. The length of time required to complete theprogram.
Contract.A. second means of obtaining a pro-gram is through contract with a consulting orga-nization. If there are subjects which are unique toyour particular agency and are not covered in off-the-shelf publications, then a program must bedeveloped to meet your specific objectives. A lookat appendix E will illustrate that many of thecourses already in use by agencies zare of thisnature. The following are points to consider inselecting a consulting organization to develop acustom-tailored program.3°
1. Does the consultant have samples of hisprevious programs available for inspection?
2. Does he also have records of achievement andretention tests available for inspection ?
3. Are the lists of his customers available foryou to query?
20
4. What about the quality of his staff? Does itinclude reputable psychologists, along withsubject matter specialists and task analysts?It should. And are their credentials availablefor you to inspect?
5. Does the consultant ask for access to yourtechnical experts? And does he require sub-jects both for retesting and formal evaluationof his program? Any sound organization willmake these requests.
6. Are his costs in line with those presented byother reputable consulting organizations?
7. Is the consultant's organization close to yours,physically ? This last point is important be-cause your staff will have to work closely andcontinuously with the staff of the consultingorganization. No outside subject matterexpert knows your special problems. You willhave to provide a subject matter specialist towrite a statement of detailed objectives forthe program and to hold weekly conferencesto read and discuss frames.
In- house. The third alternative is in-house de-velopment. As with contract development, in-house-development is used for those programs whosesubjects are unique to an agency and not coveredin off-the-shelf publications. The development ofprograms in-house requires a programming staff aswell as the cooperation of the subject matterspecialist.
Developing your own programs is desirable if : 311. You have personnel on the payroll who can
be used as programmers. The initial dew op-ment of the programming capacity is a intand costly undertaking, however its existencemay be valuable for large-scale programdevelopment and revision.
2. There is the possibility that you will needextensive programming in many subjectareas.
3. You have gained enough previous experiencewith using PI to be able to evaluate the roleof PI in your organization.
Cos: One additional factor, ever present indecision-making, is that of cost. Actually there isno easy answer to how much programs cost. Thecost is influenced by a variety of factors. Thelength and complexity of the program, as well asthe medium and number of students using the pro-gram, are influencing factors on cost.
The cost of an off-the-shelf program is the leastcostly of any of the sources of programs. The price
per individual program can range from a fewdollars for a paperback programmed text to some-what more costly tape/slide or "teaching machine"programs.
The cost of custom programs is far more ex-pensive than those available off-the-shelf. Statedprices range from $1,000 to $3,000 per hour of in-struction.82 Again prices depend on the complexityof the program, the salaries of programmers andother factors. Generally on a per program basis,in-house programming would be less expensivethan contract but in a comparable price range. It isdifficult to determine the exact cost of in-houseprogramming particularly on a before-the-factbasis. Not only must the cost of the use of
SUM
Programmed instruction is not new. It has beenknown and used in a limited sense for many years.In recent years it has experienced remarkablegrowth in use. The underlying theory can be tracedback to Thorndike who in 1912 predicted the in-fluence that learning theory could have on methodsof instruction. Efforts to apply theory to instruc-tional methods involving teaching machines fol-lowed. These early machines operated mainly onthe principle of immediate reinforcement of cor-rect responses. In spite of the apparent success ofmachine-based instruction, PI did not developwidespread support until the 1950's when Skinnerbegan working in the field. His article, "The Sci-ence of Learning and the Art of Teaching", pub-lished in 1954, focused on the potential of PI andthe failure to exploit this potential. Unfortunately,the newly found interest in programming was tem-porarily misdirected because most efforts went intodeveloping machines instead of programs. By theearly sixties the excitement over the machines haddiminished and the demand for good programs re-sulted in a new emphasis on program construction.
Linear programmed instruction is usually con-sidered to be a direct outgrowth of Skinner's oper-ant conditioning theory. The central principles in-volved are the presentation of small pieces of ma-terial in an ordered fashion which lead the student
380-346 0 - 70 - 4
programmers and subject specialists be consideredbut also clerical costs and printing. The overallcost of training should be ascertained by not onlythe number of students who need training and howmuch effort is required to develop the program butalso' by how much value the desired outcome or per-formance has to the organization.
In the end, the choice to use programmed in-struction, the decision on the sources of the pro-gram and the program format can only come fromthe trainer who is aware of his needs and has madea careful analysis of them. If this is done, and hischoice is based on a solid understanding of the ma-terial with which lie is working, his chances ofhaving a successful outcome are very good.
MARY
to a predetermined goal of a specific behavior orpiece of knowledge by immediately reinforcing thecorrect responses which are given to each succeed-ing question.
Another approach, known as branching, differsfrom linear programming by providing for theskipping or repetion -f certain groups of stepsthe path of the student being determined by the re-sponses to the items. Not only can this make theprogram more interesting for a bright student, butit may also make it a more efficient teaching devicefor less bright students.
During the past few years, the differences be-tween these two approaches to programming havediminished as each has been modified by newdevelopments.
Programmed instruction has clearly proven it-self to be an effective method of instruction. Itsfurther use in the Federal Government will dependupon trainers having a grasp of the underlyingconcept and a working knowledge of the technique.
The reader is cautioned, however, that the im-plementation of the technique is not a cure-all fortraining problems. PI should not be used merelyfor the sake of replacing old "chalk and black-board" techniques with modern technology. Rather,a decision to use this method should come onlyafter systematic evaluation of the training needs.27
FOOTNOTESGlenn L. Bryan and John A. Nagay, "Use of Pro-
gramed Instructional Materials in Federal GovernmentAgencies" in Teaching Machines am,d Programmed Learn-ing, 11, ed. Robert Glaser (Washington, D.C., 1965), p.745.
° Gabriel Ofiesh, "The Emergence of Instructional Tech-nology" in Trends in Programmed Instruction, eds. Gabriel()flesh and Wesley C. Meierhenrg (Washington, D.C.,1964), p. 7.
Jerome P. Lysaught and Clarence M. Williams,Guide to Programmed Instruction (New York, 1968), pp.24-25.
Wilbur Schramm, The Research on Programmed In-struction; An Annotated Bibliography (Washington,1964), p. 5.
6 Lysaught and Williams, op. cit., pp. 165-167.°B. F. Skinner, The Behavior of Organisms (New York,
1938).B. F. Skinner, "The Science of Learning and the Art of
Lysaught and Williams, op. cit., p. 71.Norman A. Crowder, "Automatic Tutoring by Intrinsic
Programming" in Teaching Mach/Wes and ProgrammedLearning, eds. Robert Glaser and A. A. Lumsdaine (Wash-ington, D.C., 1960), p. 286.
" D. Mackenzie Davy and P. McDonnell, ProgrammedInstruction (London, 1965), p. 18.
I' David Cram, Explaining "Teaching Machines" andProgramming ( San Francisco, 1961), pp. 66-75.
" Davy and McDonnell, op. cit., p. 18." Susan Markle, "Programming '63: The Straight Line
Bends", Programmierter unterricht and Lehrmasehinen(Berlin, 1963), pp. 368-386.
"David J. Klaus, An Analysis of Programming Tech-niques in Glaser, p. 153.
" Howard II. Kendler, "Teaching Machines andPsychological Theory" in Automatic Teaching: The Stateof the Art, ed. Eugene Galantin (New York, 1959), p. 184.
1° Francis Mechner and Donald A. Cook, BehavioralTechnology and Manpower Development (New York,1964) , p. 36.
22
11 Thomas F. Gilbert, "On the Relevence of LaboratoryInvestigation to Self-Instructional Programming" inLumsdalne and Glaser, al. 475-485.
" Douglas Porter, "A Critical Review of a Portion of theLiterature on Teaching Devices" in Lumsdaine and Glaser,pp. 114-132.
"Ibid., p. 117.SPhram, op. cit., pp. 8-10.
"Edward B. Fry, "A Study of Teaching Machine Re-sponse Modes" in Lumsdaine and Glaser, pp. 469-474.
"A. A. Lumsdaine, "Teaching Machines : An Introduc-tory Overview" in Lumsdaine and Glaser, p. 17.
" Bryan and Nagay, op. cit., pp. 743-767.31 Ibid., 748.U Ibid., 752." Ibid."James W. Brown, Richard 13. Lewis, and Fred
Harcleroad, AV Instruction, Media and Methods (NewYork, 1969), pp. 114-115.
" Ibid., p. 115."Bureau of Business Prectices, Inc., What You Should
Know About Programmed Instruction (Waterford, Conn.,1964), pp. 24-25, and Brown, Lewis and Harcleroad,op. cit., p. 125.
Bureau of Business Practices, op. cit., p. 27.31 Ibid., p. 26."Francis Mechner and Donald Cool; "Behavioral Tech-
nology and Manpower Developm:Int" in Managing theInstructional Programming Effort, eds. Geary Rummler,Joseph Yaney, and Albert Schrader (Ann Arbor, Mich.,1967) p. 32.
U.S. Civil Service Commission, Bureau of Training,Instructional Systems and Technology: An Introduction tothe Field and Its Use in Federal Training, Training Sys-tems and Technology Series : No. I and Application of aSystems Approach to Training: A Case Study, TrainingSystems and Technology Series : No. II. (Washington,D.C., 1969.)
"Naval Air Training Command, Lesson No. 1.5.6. Pro-grammed Instruction,
"Cram, op. cit., p. 39.
BIBLIOGRAPHY
Brown, James W.; Lewis, Richard B.; Harcleroad,Fred F. AV Instruction, Media and Methods.New York : McGraw-Hill, 1969.
Bureau of Business Practices, Inc. What YouShould Know About Programmed Instruction.Waterford, Conn. Bureau of Business Practices,Inc., 1364.
Cram, David. Explaining "Teaching Machines"and Programming. San Francisco : Fearon Pub-lishers, 1961.
Davey, D. Mackenzie, and P. McDonnell. Pro-grammed Instruction. Lcndon : Institute of Per-sonnel Management, 1965.
Deterline, William A. An Introduction to Pro-grammed Instruction. American Institute forResearch, Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1962.
English, H. B. "How Psychology Can FacilitateMilitary TrainingA Concrete Example,"Journal of Applied Psychology, 26 (February1942), 3-7.
Galanter, Eugene, ed. Automatic Teaching: TheState of the Art. New York : John Wiley & Sons,Inc., 1959.
Glaser, Robert, ed. Teaching Machines and Pro-grammed Learning, II; Data and Directions.Department of Audiovisual Instruction, Na-tional Educational Association of U.S., 1965.
Hawley, William E. "Programmed Instruction",Training and Development Handbook, eds.Robert L. Craig and Lester R. Bittel, ASTD.New York : McGraw-Hill, 1967.
Hilgard, E. R. A Basic Reference Shelf on Learn-ing Theory. Stanford: ERIC Clearinghouse onEducational Media and Technology, Institutefor Communication Research, Stanford Univer-sity, 1967.
Hilgard, E. R., and G. H. Bower. Theories ofLearning, 3d ed. New York : Appleton-Century-Crofts, 1966.
Holland, James G., and B. F. Skinner. Analysisof Behavior. New York : McGraw-Hill, 1961.
Jacobs, Paul I., Milton H. Maier, and LawrenceM. Stolurow. Guide to Evaluating Self-Instruc-tional Programs. New York : Holt, Rinehart &Winston, Inc., 1966.
23
Kaufman, Roger A., Robert E. Corrigan, andCharles L. Nunnelly. "The Instructional Sys-terms Approach to Training." Human Factors(April 1966), 157-162.
Lumsdaine, A. A. "Teaching Machines and Self-Instructional Materials," Audiovisual Commu-nications Review, 7 (1959), 163-172.
Lumsdaine, A. A., and Robert Glaser, eds. Teach-ing Machines and Programmed Learning: asource book. Department of Audiovisual In-struction, National Education Association, 1960.
Lysaught, Jerome P., and Clarence M. Williams.A Guide to Programmed Instruction. NewYork : John Wiley & Sons, Inc., 1963.
Mager, Robert F. Preparing Instructional Objec-tives. Palo Alto, Calif. : Fearon Publishers, Inc.,1962.
Margulies, Stuart and Lewis D. Eigen, eds. Ap-plied Programmed Instruction. New York :John Wiley & Sons, Inc., 1962.
Markle, S. M. "Programming '63: The StraightLine Bends", Programmierter unterricht andLelbrmachinen. Report of International Confer-ence on Programmed Instruction and TeachingMachines. Berlin : Padagogische Arbeitstelle,Secretariat Padagogisches Zentrum, 1963, 368-386.
Mechner, Francis, and Donald A. Cook. Behavi-oral Technology and Manpower Development.Organization for Economic Cooperation andDevelopment. New York : Basic Systems Inc.,1964.
Murphy, John R., and Irving A. Goldberg. "Strat-egies for Using Programmed Instruction,"Harvard Business Review (May-June 1964),115-132.
Naval Air Training Command, Lesson No. 1.5.6.Programmed Instruction.
Ofiesh, Gabriel D., Programmed Instruction, AGuide for Management. New York : The Amer-ican Management Association, 1965.
Ofiesh, Gabriel D., and Wesley C. Meierhenry, eds.Trends in Programmed Instruction. Departmentof Audiovisual Instruct' ., National Educa-tion Association and National Society for Pro-grammed Instruction, 1964.
Rummler, Geary A., Joseph P. Yaney, and AlbertW. Schrader, eds. Managing the InstructionalProgramming re., /fort. Ann Arbor: Bureau ofIndustrial Relations, Graduate School of Busi-ness Administration, University of Michigan,1967.
Schramm, Wilbur, Programmed Instruction: To-day and Tomorrow. The Fund for the Advance-ment of Education, 1962.
Schramm, Wilbur. The Research. on ProgrammedInstruction: An Annotated Bibliography.Washington, D.C. : U.S. Office of Education,1964.
24
Skinner, B. F., The Behavior of Organisms, NewYork : Appleton-Century-Crofts, 1938.
Skinner, B. F. "The Science of Learning and theArt, of Teaching," Harvard Educational Re-view, 24 (1954), 86-97.
Staff, Psychological Test Film Unit, Army AirForce. "History, Organization, and. Research.Activities," Psychological Bulletin, 41 (July1044), 457-468.
Taber, Julian I., Robert Glaser, and HalmuthSchaefer. Learning and Programmed Instruc-tion. Reading, Mass.: Addison-Welsey, 1965.
Thorndike, E. L. Education. New York : Mac-Millan Co., 1912.
Appendix A.SAMPLE OF A LINEAR PROGRAM "
1. The objective of this section is to "LIST THE FIVE PRINCIPLES OFPROGRAMMING."
2. You are now beginning a lesson on programmed instruction. The principleof SELF-PACING as used in programmed instructions allows each traineeto work as slow or as fast as he chooses. Since you can control the amountof time you spend on this lesson, this program is using the principle ofself-
PACING 3. People naturally learn at different rates. A program that allows each traineeto control his own rate of learning is using the principle of
SELF-PACING 4. If a self-pacing program is to be successful, the information step size mustbe small. A program which is self-pacing would also apply the principleof small
STEPS 5. The average trainee will usually make correct responses if the correct sizestep of information is given. This is utilizing the principle of small
STEPS 6. A program that provides the trainee information in a step size which allowshim to be successful is applying the principle of
SMALL STEPS 7. A trainee knows the material being taught, but has to wait for the remainderof the class. What programming principle is being violated?
SELF - PACING 8. Two principles of programmed learning are:(1)(2)
1. SELF-PACING2. SMALL STEPS
NoTE.In this example for practical reasons the frames are arranged on 1 page rather than on succeeding pages.The answers should be covered until the preceding frame has been answered.
25
Appendix B.SAMPLE OF A BRANCHING PROGRAM "Page 39
Wrong answers, while not encouraged, are not avoided in a branching program as they are in alinear program, since they may be corrected before the learner moves on.
The assumption in branching programming is that a wrong response does not necessarily hinderthe learning of a correct response. The response is useful mainly in guiding the student through theprogram. Each response is used to test the success of the latest communication to the student, and inthat sense, it "lets the program know" where to take the student next.
The differences in approaches to programming, then, lie in what is considered to be the function ofthe overt or motor response. The advocates of branching programs believe that :
Page 41 The overt response is not fundamental to the learning process but it is useful as ameans of supplying feed-back to the program.
Page 44 The overt response is fundamental to the learning processPage 47 No answer is ever completely wrong.
Page 41Your answer is correct. The overt response, according to Dr. Crowder, is a measurer rather than a
fixer of learning.Inappropriate responses can be used to uncover misconceptions and areas of weakness and, therefore,
have great value.Since errors or misconceptions are corrected before the learner proceeds, the step size can be larger
and the main line (prime path) of the program may proceed more rapidly than in the linear program.There is always the possibility of having the main-line items proceed in large, fairly difficult steps
with shorter, easier stages moving side by side with the main line.Given a student with the necessary prerequisites, which of the following, in your opinion, would
be most dependent on his intelligence?Page 42 Whether he gets through the program or notPage 46 How long he takesPage 45 How much he knows about the subject upon finishing the program.
Page 44You've got them twisted around. The Skinnerian (constructed-response) programmers believe :The overt response is fundamental to the learning processlearning takes place when the overt
response is made.The branching programmers believe:The overt response is best used as a device for evaluation and direction, since learning may have
taken place by the time the response is made.(Please return to page 39 and select another alternative.)
Page 47I just threw this one in to bring you up short if you're getting tired.But don't be buffaloed into this kind of an opinionof course some things are amenable to right
and wrong answersbut not everything that we teach isand the branching technique allows us toprogram, for automated instruction, subject matter in which shades of meaning or interpretations areessential.
(Please return to page 39 and select another alternative.)
26
Appendix C.SURVEY PROCEDURE AND SCOPEThe survey conducted by Bryan and Nagay is
reported in Teaching Machines and ProgrammedLearning, II published by the National Educa-tion Association.1 A total of 125 organizationswere surveyed, reporting the use of 382 programs.Procedure
The format of the survey made by the Bureauof Training followed that of the earlier survey.The procedure was to develop a questionnairewhich covered various aspects of the use of PIin training activities and to contact as many Fed-eral Government organizations as possible bymeans of a telephone survey. Appendix A, "Fed-eral Government Organizations Surveyed", inBryan and Nagay served as a base for organiza-tions to contact. Initial calls were made to head-quarters of each agency asking for informationabout their use of PI or for recommended contactsin subordinate activities within the agency whomight have the desired information. Leads for con-tacts were also solicited when talking with those inthe subordinate activities. They were asked for ad-ditional contacts within their own agency as wellas their counterparts in other agencies. This proce-dure was followed to insure thorough coverage.
The Bureau of Training's list, "Federal Govern-ment Organizations Surveyed", is not identical tothe 1963 list due to changes in agency organizationand the establishment of new departments andactivities since that study was conducted.
As information yielding contacts were made, thesurveyors asked predetermined questions andanswers were recorded on standard questionnaireforms. In many cases additional information re-garding courses was sent to us by mail. In a fewcases the entire questionnaire was sent out, com-pleted, and returned by mail,
'Glenn L. Bryto. and John A. Nagay in "Use of Pro-grammed Instructional Materials in Federal GovernmentAgencies" in Teaching Machines and Programmed Learn-ing, II, ed. Robert Glaser (Washington, D.C., 1965)
27
Scope
Forty-seven departments and agencies reportedon the use of PI in their training activities in :121headquarters, suborganizationk and field offices.Of these 121 organizations, 69 ere directly con-tacted. An additional 52 organizations were re-ported on by agency headquarters. All organiza-tions contacted are listed in appendix D. Those notdirectly contacted are marked by an asterisk (*).
Of the 69 organizations directly contacted 61percent (42 organizations) reported that they arecurrently using programmed instructional mate-rials. Agency headquarters which reported ontheir own units reported only on those units wherePI is being used and they reported on these ac-tivities as a whole, therefore the 52 organizationsnot directly contacted are not counted individu-ally in tabulation of the questionnaires.
NOTE.Bryan and Nagay state that 125 organizationswere contacted in their survey, however they do not indi-cate how many were actually using PI in their trainingactivities.
The survey identified over 2,300 different pro-grams in use. Appendix E lists the "Programs inUse" in alphabetical order and gives a numericalcode indicating the agency using the program.This type of listing is given because the purpose isto acquaint the reader with the vast number andvariety of programs being used and illustrate thetypes of subject matter which can and have beenprogrammed for use in agency trainingprograms.
The "program titles" represent entire pro-grammed units or courses and courses of which asegment is programmed. For example, English3200 is a complete 40-hour programmed course,however, the course Position Classification and theManagement Process, a 40-hour course given by theCivil Service Commission, uses a several hour sup-plemental programmed unit.
(Page 1)
TELEPHONE SURVEY
Name: Address:
Position :
Agency :
Phone Number:
Do you use PI in your training programs?
A. YesIf so, how long have you been using it?
B. Nohave you ever used it: Yes NoIf so, why no longer?
*What percent of your total training is by PI ?
How were your courses developed? Were they obtained by :
A. Contract
B. Commercial purchase
C. In-house
If in-house, how were your programmers trained?
(Page 2)
What specific PI courses do you have?
Program title
Agency :
Length(bourn)
Percent ofPI
(Page 3)
What training objectives is your PI designed to meet, for example:A. Standard-regular trainingB. Remedial-to overcome deficiencyC. Refresher-quick review given before standard courseD. Preparatory-not previously learned prior to standard courseE. Self-development-not specific to any particular course objective
Where is it used:(a) Classroom:(b) Hom(-,work(c) Correspondence
28
PI can be presented in many forms ,which of the following do you use?A. Written textB. Teaching machinesC. Tape/filmD. Other audiovisual
*Have you tested for results?Yes No
If so how?Pretest /post -testOther
Do you anticipate further and new use of PI?Yes . No
Do you have printed information on your programs you could send us?Yes No
*In implementation, these questions proved to be ambiguous and replys inconclusive, therefore were not used inreporting on the survey.
29
Appendix D.FEDERAL GOVERNMENT ORGANIZATIONS SURVEYEDAgency for International DevelopmentAgriculture, Department of
Agricultural Research ServiceAgricultural Stabilization and Conservation
ServiceConsumer and Marketing ServiceFarmers Home AdministrationForest Service
Air Force, Department ofAerospace Defense Command*Air Training Command*Air Force Accounting and Finance Center*Air Force Logistics Command*Air Force Reserve*Air University*Military Airlift Command*Pacific Air Forces*Strategic Air Command*Tactical Air Command*USAF Academy*USAF Security Service*USAF Southern Ccrnmand*
Army, Department ofOffice of Civil DefenseOffice of Civil Defense Staff CollegeU.S. Army Training Centers*U.S. Army Medical Training Center*U.S. Army Adjutant General School*-U.S. Army Air Defense School*U.S. Army Armor School*U.S. Army Artillery and Missile School*U.S. Army Aviation School*U.S. Army Chaplain School and Center*U.S. Army Chemical Center and School*U.S. Army Civil Affairs School*U.S. Army Combat Surveillance and Elec-
tronics Warfare School*U.S. Army Command and General Staff
College*U.S. Army Engineer School*U.S. Army Finance School*U.S. Army Infantry School*U.S. Intelligence School*
'Not contacted directly.
30
U.S. Army Management School*U.S. Army Military Police School*U.S. Army Missile and Munitions Center and
School*U.S. Army Ordnance Center and School*U.S. Army Primary Helicopter School*U.S. Army Quartermaster School*U.S. Army Signal Center and School*U.S. Army Southeastern Signal School*U.S. Army Special Warfare School*U.S. Army Transportation School*U.S. Women's Army Corps School*
Atomic Energy CommissionBureau of the BudgetCentral Intelligence AgencyCivil Aeronautics BoardCivil Service Commission
ADP Management Training CenterCommunications and Office Skills Training
CenterFinancial Management and PPB Training
CenterGeneral Management Training CenterPersonnel Management Training CenterExecutive Seminar CentersPersonnel Division
Commerce, Department ofOffice of the SecretaryBureau of CensusNational Bureau of StandardsMaritime CommissionPatent OfficeEnvironmental Lcience Services Administra-
tionDomestic and International Business
District of Columbia GovernmentEqual Employment Opportunity CommissionExport-Import BankFederal Communications CommissionFederal Mediation and Conciliation AgencyFederal Power CommissionFederal Reserve BoardFederal Trade CommissionGeneral Accounting OfficeGeneral Services Administration
Government Printing OfficeHealth, Education, and Welfare, Department of
National Institutes of HealthPublic Health ServiceSocial Security Administration
Housing and Urban Development, Department ofInterior, Department of
Bureau of Indian AffairsInterstate Commerce CommissionJustice, Department of
Immigration and Naturalization ServiceBureau of PrisonsFederal Bureau of Investigation
Labor, Department ofLibrary of CongressNational Aeronautics and Space AdministrationNational Labor Relations BoardNational Science FoundationNational Security Agency
Commandant, Training SchoolNavy, Department of
Bureau of Medicine and Surgery*Bureau of Naval Personnel*Fleet Anti-Air Warfare Training Center*
31
Fleet Training Center*Landing Force Training Command Atlantic*Naval Air Advanced Training*Naval Air Basic Training*Naval Air Technical Training Center*Naval Air Technical Training Unit*Naval Post Graduate School*Service School Command*Training Command*
Peace CorpsPost Office, Department ofSecurities and Exchange CommissionSelective Service SystemSmall Business AdministrationSmithsonian InstituteState, Department ofTax Court of the United StatesDepartment of Transportation
Federal Aviation AdministrationTreasury, Department of
Internal Revenue ServiceUnited States Information AgencyUnited States Tariff CommissionVeterans Administration
Appendix
A-7 Aircraft Fuel System Familiarization 20
A-7 Aircraft Fuel System Operation 20
A-7A Aircraft Power Plant 20
A-7A Power Plant, Cold Section 20
A-7A Power Plant, Hot Section 20
ABO Grouping & Rh Typing 3
A.C. Bus System 3
A.C. Circuit Power Characteristics 20
A.C. Circuits, Characteristics, PIX-1A, P-1X-1B 20
A.C. Distribution System 3
A.C. Electricity 20
A.C. Fundamentals 20
A.C. Meters 20
A.C. TheoryAlternating Current and Voltage Characteristics 20
A.C. Theory, Related Mathematics 20
A.C. Time ConstantsRCL 3
ACL Form F (Aircraft PerformanceC-141 and
Air Transportation) 3
ADF Approach 4
ADF En route 4
ADF Flight 4
ADF Orientation and Tracking 4
ADF Procedures 4
ADFVOR 4
ADP Procedures Training 27-1
AID Files and Records System 1
AMSE Statistical Data Reporting 20
AN Nomenclature System, Advanced Developments 20
AN/APN-141(V), Power Supply Operation 20
ANM100A2 Series Bomb Tail Fuzes 20
ANM103A1 Bomb Nose Fuze 20
ANM123A1 Series Bomb Tail Fuzes 20
AN/PRC 8, 9, and 10 (Tuning and Calibration) 20
AN/SPM-4, Maintenance of FrequencyPower Meter 20
AN/UPN-12/5 Loran Receiver, Maintenance of 20
AN/WRT-1 Radio Transmitter, Maintenance of 20
ANWRT-2 Radio Transmitter, Maintenance of 20
ASW Plotting, Symbols for the DRT 20
Acceleration and Takeoff 3
Accelerometers 20
Accelerometers, Aircraft 3
Accident Control System for Supervisors 2-4
Accidents, Maintenance Induced 20
Accounting 2-4
E.PROGRAMS IN USE
82
Accounting Machine Operation and Wiring 402-3, 407 12-2
Accumulator, Air Flask and Pressure Regulator 3
Actuator 3
Addition, Subtraction, Multiplication and Division (ASMD) 12-2
Adjunctive Program on AR 711-16 4
Adjunctive Program for AR 735-35 4
Adjunctive Program for Excess Property Procedures 4
Adjunctive Program for Property Records Adjustment 4
Administration Responsibilities 4
Administrative Forms Preparation 4
Administrative Office Procedures 4
Administrative Specialist Course 4
Administrative Training 5
Advance Oversee Returnee 4
Adverse Weather Operation 4
Aerodynamics 20
Aerodynamics, Applied 3
Aerodynamics, Drag 20
AerodynamicsHelicopter (Introduction) 20
Aerodynamics, High speed, Parts I and 113
Aerodynamics of Sink Rate 3
Aerodynamics, T-28 20
Aeromedical Evacuation, Preparation of Patients for 3
Aeronautical Chart Symbols 4
Aids to Navigation, BuoysPart I 20
Air Conditioning Principles Review 20
Air Conditioning System, Cargo 3
Air Equipment Support Company 4
Air Flow Control Valves 3
Air Force Mission and Organization 3
Air Force Technical Order Form 781 3
Air Intelligence 20
Air Mass Weather 4
Air Masses 4, 20
Air Movement Planning 20
Air Navigation Computer 3
Air Navigation, Earth 20
Air Nak ltion (TACAN) 20
Air Navigation (VOR, Part I) 20
Air Navigation (VOR, Part II) 20
Air Route Traffic Control, Approach Control 20
Air Route Traffic Control, Departure Procedures 20
Air Route Traffic Control, En Route Procedures 20
Air Route Traffic Control VFR Operations Part 120
Air Route Traffic ControlVFR Operations, Part II 20
Air Service for Mail 22
Air Traffic Controller Training 26
Air Traffic Rules, Cruising Altitude Rules 20
Aircraft Accident Investigation 4
Aircraft Accident Prevention Program 4
Aircraft and Boat Crane 20
Aircraft and Squadron Designations and Missions 20
Aircraft Arresting Hooks, Maintenance of 20
Aircraft Bombs, Introduction to 20
Aircraft Carriers and Seaplane Tenders 20
Aircraft Carriers, Basic Fundamentals 20
Aircraft Chemical Tank 20
Aircraft Control and Protective Devices 20
Aircraft Control Cables, Maintenance of 20
Aircraft Electrical Conductors and Connectors 20
Aircraft Electrical Control 20
Aircraft Electrical Control and Protection Devices 20
Aircraft Familiarization 3
Aircraft Forms and Records 3
Aircraft Ground Handling Equipment 20
Aircraft Handling 20
Aircraft Hardware 3, 4
Aircraft Identification 3
Aircraft Inspections 4
Aircraft Instruments and Instrument Scan 20
Aircraft, Introduction to 20
Aircraft Jacks, Introduction to 20
Aircraft Maintenance Structure 4
Aircraft Mines and Torpedoes 20
Aircraft Nomenclature 20
Aircraft Performance 3
Aircraft, Preservation of 20
Aircraft, Protection Devices 20
Aircraft Rockets, Introduction to 20
Aircraft Statistical DataAircraft Accounting System 20
Aircraft Structures 4
Aircraft Systems Check 4
Aircraft Tires, Tubes and Wheels 20
Airdrop Equipment Repair and Suppy Company 4
Airdrop Supply Company 4
Airframe 4
Airport Surveillance Radar 4
Airport Traffic Control, Altimeter Setting Information 20
Airport Traffic Control, Airport Facilities 20
Airport Traffic Control,'Airport Lighting 20
Airport Traffic Control, Ground Traffic, Control of 20
Airport Traffic Control, Separation Minima 20
Airspeed (Indicated) and Machmeter Indicators 20
Airspeed (True) and Maximum Allowable Airspeed Indicators 3
Algebra 20,12 -2
Algebra, An Introduction to Verbal Problems in 3
33
Algebra, Boolean 3, 20
Algebra, College 3
Algebra, First Year 3
Algebra, Fundamentals, Part I 3
Algebra, Introduction to 3
Algebra, Language of 3
Algebra, Review of Basic 3
Algebra, Second Year 3
Algebra, Verbal Problems of 3
Algebraic Equations 20
Algebraic Expressions 3, 20
Alphabetic Interpreter Operation and Wiring/557 12-2
Alternating Current and Use of Multimeter, Introduction to 4
Alternating Current and Voltage Characteristics, PVIII--2 20
Alternating Current (Introduction to) 4
Alternators 4, 20
Altimeter Errors 4
Altimeters, Sensitive 3
Amebiasis: Laboratory Diagnosis, Introduction to the Course 12-3
Ammeter Use, Multimeter 3, 4
Ammeters 20
Ammeters and Voltmeters 20
Amphibious Operations, MEDS 80
Amplifiers, Audio Power 20
Amplifiers, Magnetic 20
Amplifiers, Multi-Element Tubes 20
Amplifiers, Triode 20
Analysis of Tax Returns 12-2
Angles 20
Antennas and Field Expedients for Antennas 4
Anti-Air Warfare, Display Methods 20
Anti-G Suit Valve 3
Anti-Skid Systems 3
Anti-Spin Differential 3
Anti-Streptolysin "0" Test 3
Anti-Submarine Warfare 20
Anti-Submarine Warfare, Evasive Steering 20
Applied Geometry 4
Approach, Final 3
Architect's Scale, The 4
Arithmetic and Whole Numbers, (Introduction to) 20