TS - AI Heuristic Frameworks 1990.pdf

7/28/2019 TS - AI Heuristic Frameworks 1990.pdf

1/12

M A N A G E R I A L A N D D E C I S I O N E C O N O M I C S , V O L . 1 1 , 3 6 5 - 3 7 5 ( 1 9 9 0 )

Artificial Intelligence, HeuristicFrameworks and Tabu SearchFred Glover

Center for Applied A rtificial Intelligence, G raduate School of Business, University ofColorado, Boulder, CO, USAThis paper examines some of the characteristics of Al-based heuristic procedures that haveemerged as frameworks for solving difficult optimization problems. Consideration of attributesshared to some degree by human prohlem solvers leads to focusing in greater detail on one of themore -ituccessful procedures, tabu search, which employs a flexible memory system (in contrastto 'memoryless^ systems , as in simulated annealing and genetic algorithms, and rigid memorysystems as in branch and bound and A* search). Specific attention is given to the short-termmemory component of tahu search, which has provided solutions superior to the best obtainedby other methods for a variety of problems. Our development emphasizes the principlesunderlying the interplay between restricting tbe searcb to avoid unproductive retracing of patbs(by tneans of tabu conditions) and freeing the search to explore otherwise forbidden avenues (byaspiration criteria). Finally, we discuss briefly the relevance of a supplementary framework,called target analysis, which is a method for determining good decision rules to enable heuristicsto perform more effectively.

INTRODUCTIONHeuristic approaches to optitnization problemsabound, and many claim some connection to artifi-cial intelligence. Generally speaking, good heuristicprocedures arc based on ideas that can trace theirorigins equally to the fields of artificial intelligenceand operation s research. (This is not too surprising,since the two fields emerged from common be-ginnings.) Nevertheless, whether due to differencesof emphasis or to differences in the supply of'congenial metaphors' , recent heuristic innovationshave tended frequently to become aligned with Al.Hence in order to investigate what is current inheuristic ideas it is app rop riate to examine proced-ures that have acquired some of the imprint of theAl domain.Given the proliferation of heuristic procedures inthis category, there is some challenge to identifyingthose that are more significant, or that at leastembody principles that have widespread utility.Currently, four methods that are perceived as affili-ated in some measure with the Al field have gainedprominence asframeworks for solving difficult prob-lems: neural netwo rks, simulated annealing, geneticalgorithms and tabu search.Neural networks have claimed intriguing suc-cesses in pattern-recognition applications, but have

generally performed less than impressively in op-timization settings. They have demonstrated theirprimary value for problem s whose structures can beexploited by processes likened to those of 'associ-ative memory', and appear less well adapted (so far)to the solution of optimization problems in broadercategories. While these approaches have a greatdeal that is inherently fascinating about them, parti-cularly concerning the directions in which they mayevolve, neural networks api>ear to show greatestpromise in partnership with other methods, and arecoming to rely on one of the other three heuristicframeworks to improve their effectiveness in anumber of applications.

Simulated annealing and genetic algorithmsdraw on analogies to phenomena in the physicaland biological sciences, respectively, and have theattractive feature of assured convergence underappropriate assumptions. It should be cautioned,however, that convergence in these procedurestakes a less than impressive form, couched in prob-abilistic assertions. In simulated annealing, for ex-ample, optimality is guaranteed to be achieved withprob ability 1 after an infinite num ber of iterations.The guarantee offered by genetic algorithms islikewise probabilistic and refers only to certainclasses of "undominated" solutions. Consequently,members of the Al and OR communities who are

0143 -6570 /90/050 365-1 l$O5.5O


2/12

366 F. GLOVERless religiously attached to mathematical demon-stration (e.g. who would like to see convergencetake place in the world we know) may prefer toplace greater stock in the empirical performance ofsuch approaches than in their theoretical under-pinnings. Fortunately, a number of instances ofefFective empirical performance have been docu-men ted, attesting to the value of these procedu res aspractical tools.

From an Al standpoint, one of the more inter-esting features of simulated annealing and geneticalgorithms is that they are virtually memoryless.Each operates by using a form of threshold to allowrandomly sampled moves to alter current solutions.and to initiate subsequent iterations of search.Memory has no role except as implicit in thestructure of solutions generated from one stage tothe next as a result of progressively applied screen-ing criteria.

Mathematically, there is a good reason for this.At our present level of mathematical developmentwe are unable to provide theorem-proof demon-strations for the behavior of systems that embodymemory, except in its most trivial or most rigidforms. Leading in popularity are the systems withno memory whatsoever, which simply involve rulesfor transforming a present state into a successor,closely followed in popularity by the 'highly struc-tured' memory systems exemplified by branch andbound. A* search, and their relatives.

Beyond the appeal of being susceptible to math-ematical analysis, memoryless systems have theattraction of appearing to effectively serve certainrealms of physics and biology (at a stage of evolu-tion that precedes the development of a complexbrain), while rigid memory systems appeal to no-tions of orderliness (as manifested in what may becalled a search-by-bookkeeping orientation).To the extent that Al motivates a notion ofintelligence that involves freer reign with the use ofmemory, however, we may conceive it worthwhileto explore frameworks that embrace more fiexiblememory structures. Mathematics may not yet beable to justify certain forms of intelligence, but wemay suspect that this should not compel us toabandon such forms of intelligence in buildingsolution methods.This latter view provides the perspective adop tedin this papera perspective that also underlies thedevelopment of the fourth heuristic framework,tabu search. Consequently, the class of approachesembodied within tabu search will be the chief focus

of the material that follows. In addition, we wibriefly offer a fifth framework for consideratiocalled target analysis., which is not a heuristisolution procedure but a form of leaming approacdesigned to determine good variants of such solution procedures.

TABU SEARCHTabu search is a higher-level method, or metastrategy, for solving optimization problems. Thtechnique is designed to be superimposed on anprocedure whose operation can be characterized aperforming a sequence of moves that lead thprocedure from one trial solution (or solution statto another. Each move is assumed to be selectefrom a set of currently available alternatives, and susceptible to being evaluated by one or morfunctions that measure its relative attractiveness isome local sense. When the solution produced bthe move is feasible, for example, the objectivfunction value itself provides such a measure.

The well-known hill-climbing heuristics fall withithe class of procedures susceptible to being embedded within tabu search. In general, a hill-climbinheuristic progresses from an initial feasible solutioalong a path that changes the objective functiovalue in a uniformly descending or ascending direction (for minimization or maximization, respecively) until no further im prove men t of the objectivfunction is possible by means of the availabmoves. At the stopping p oint, the solution obtaineis a local optimum that, for combinatorial problems, very rarely is also global (i.e. rarely the besolution across the entire range of feasible possibiities). In this context, tabu search provides a guidinframework for exploring the solution space beyonpoints where an embedded heuristic would becomtrapped at a local optimum.

The most basic form of tabu search consists ointroducing tabu restrictions that classify certamoves as forbidden, together with aspiration crteria capable of overriding the tabu status of mov(where appropriate). These activities have a timdependent dimension that can be implemented bmeans of a short-term memory function. Moelaborate tabu search procedures include intemediate and long-term memory functions to carout additional strategic operations.The success of the method has been noteworthAlthough, at present, tabu search is not nearly


3/12

TAB U SEAR C H 367widely studied as simulated annealing and geneticalgorithmsand, correspondingly, more remainsto be learned about the best ways to apply themethodthere are already a number of problemsettings where tabu search has been able to findsolutions superior to the best results previouslyobtained by any m ethod. In other cases, tabu searchhas been demonstrated to offer advantages in easeof implementation or in the flexibility to handleadditional considerations (such as constraints of aform not encompassed by the original problemformulation).

Applications of tabu search where superior per-formance and/or greater adaptability have beenreported cover a considerable spectrum, includingemployee scheduling (Glover and McMillan, 1986),machine scheduling (Laguna, 1989; Laguna et al,1989), maximum satisfiability problems (Hansenand Ja um ard , 1987), space planning and architec-tural design (Glover */ al , 1985), com puter channelbala ncin g (Glover, 1989a), cha ract er recogn ition(Hertz and de Werra, forthcoming), convoyschedu ling (Bovet, 1987), teleco mm unic ations pathassignment (Ryan et al, 1989), quadratic assign-ment problems (Skorin-Kapov, 1990), nonlinearcovering problems (Glover, 1990), traveling sales-man problems (Knox, 1989; Maiek et al, 1989a, b),flow-shop sequencing problems (Windmer andHertz, forthcoming), job-shop scheduling prob-lems (Eck, 1989), graph coloring and partitioningproblems (Hertz and de Werra, 1987; Wendelin,1988), maximum stable set problems (Friden et al,1989), and a variety of others (see, e.g., the surveys inGlover, 1989a, 1990; Hertz and de Werra, forth-coming).

In the domain of combinatorial problem solvingthere is still a long way to go, however, and the goalremains to do better tomorrow than today. Ack-nowledging this, the large body of positive resultsfor a procedure that is just beginning to be studiedsuggests the tabu search framework may offersome thing of value in a num ber ofthe areas c urrent-ly considered challenging. Viewed from the per-spective of research, the new types of memoryschemes appropriate to tabu search motivate thedevelopment of correspondingly new data struc-tures and processing methods. Tabu search alsoinvites new applications for cutting-plane theory(including the introduction of 'pseudo' cuttingplanes, which may be purged with the expiration ofa short-term memory tenure), and new quests forprobability theory in the versions of probabilistic

tabu search. Some ofthe features ofthe method areelaborated in the following sections.

BASIC ELEMENTSTabu search may be viewed as a nested hierarchy oflong-, intermediate- and short-term memory func-t ions, with the short-term function constituting thecore of the procedure. The short-term memorycomponent of the method operates by selectingmoves designed to progress quickly to a localoptimum (seeking those with the highest evalu-ations, subject to trade-offs involved in the effort ofidentifying such moves), and then to go beyond thelocal optimum by forbidding moves with certainattributes (m aking them tabu). N o concern is givento the fact that the best moves available may notimprove the current solution. Instead, the methodselects the moves with highest evalu ation s, from theset not classified as tabu, to drive the search intonew regions. The process generates a trajectory thatoften includes a large portion of high-quality solu-t ions, while periodically obtaining solutions betterthan the best found previously during the search.Each such pass ofthe short-term memory compon-ent continues until a specified number of iterationselapses since the best solution was last improved oruntil an overall cutoff limit is reached.

The intermediate and long-term memory func-tions of tabu search co-ordinate successive passes ofthe short-term memory component, or successiveintervals of a given pass, to achieve goals that maybe described as local intensification and globaldiversification ofthe search (Glover, 1989a, 1990).These memory functions operate as boosters toobtain solutions of still higher quality or to permitsolutions of a given quality to be obtained moreefficiently.Because of the central role of the short-term

mem ory com pone nt, we will focus on its opera tionsin the remainder of this section. To provide a fullerunderstanding of how this component operates, adiag ram ofa single pass of this proced ure is given inFig. 1, ada pted from Glo ver (1989b).The word 'solution' as used in this figure admits afiexible interpretation. It can represent what iscommonly called a trial solution, or even a partialsolution; i.e. it may not satisfy all constraints orspecify values for all variables. A form of evaluationcriterion is used that permits different solutions (ofwh atev er tyi>e) to be co m pa red . - J - :


4/12

368 F. GLOVERShort-Term Memory Component of Tabu Search

Step 1Begin with some Initial Solutio n

Designate it tbe Current Best Solution

Step 2.ffo throuRh a Sample set of Candidate Moves(If applied, each move would generate a newsolution from th e existing solution.}

iStep 3

Pick another move Evaluate the current moveDoes tbis move produce a higher evaluationtban any other so far found admissible(from tbe current Sample Set)?

YESStep

[potentialacceptance)

Step 7Check Sampling CriteriaShould another move fromSample Set be examined?(e.g., is there a "goodprobability" of bigberevaluation moves left)

Cbeck tabu statusIs tbe candidate movetabu?

Step 6Tabu

step 5

HO

Hove is admissibleStore as newcurrent best move

YES Cbeck Aspiration LevelDoes move satisfyaspiration criteria?

NOStep a Step 9 Step 10

Make tbe chosen best moveRecord th e resulting so-lution as tbe new CurrentBest Solution if it im-proves on tbe previousbest.

Stoppina CriterionHas a specifiednumber of iterationselapsed in total orsince the last Cur-rent Best Solutionwas found?

Nn .,update Tabu Lists andAspiration LevelsEstablish basis fornew Sample Set.

YESSTOP

F i g u r e 1 .

To give substance to the diagram, the functionsperformed by each of its steps will be identified. Forconcreteness, we describe how the procedure maybe applied to the traveling salesman problem. Theprinciples can readily be extrapolated to othercontexts.

Step 1. A natu ral form of a 'soluti on ' in thetraveling salesman setting is a tour, i.e. a cycle thatvisits each n ode ex actly once. An initial tour can begenerated in a variety of ways, randomly or otherwise (allowing artificial edges with high costs wherenecessary) as by a simple construction process tha


5/12

TABU SEARCH 369grows a progressively enlarging chain until the lastedge completes the tour.

Step 2. A Sample Set of candidate moves for thetraveling salesman problem may consist, for ex-ample, of the standard '2-OPT' moves, or swaps,which result by dropping any two nonadjacentedges and adding the unique pair of edges that willproduce a different tour. Where the set of suchmoves is large, the Sample Set may be chosen as asubset, as by subdividing the tour in various waysand allowing only those swaps that occur withingiven comp onen ts of the subdivision. (An app roachof creating sample sets by such subdivisions hasproved highly effective in applications of tabusearch to machine scheduling problems (Lagunaet ai, 1989).)

Step 3. The evaluation of the current candidatemove can be applied in the case of 2-OP T moves bycomparing the lengths of the two added and twodeleted edges to see if the resulting tour would beimproved. To take the 'YES bran ch' from this step,leading to Step 4, the move must have a higherevaluation than those in the Sample Set so far foundadmissible, where 'admissible' is defined as a resultof passing tests in subsequent steps. Until a movefrom the Sample Set is found that thus qualifies asadmissible, and therefore provides a basis for com-parison, all moves take the YES branch from theevaluation step (to see if they may in fact becomeadmissible).Step 4. This step embodies a key issue of theprocedure, which is to establish a basis for decidingif a move being examined should be classified astabu. T o create this basis, the most straightforwardapplication of tabu search maintains a tabu list thatrecords selected attributes of each move made. Forexample, each element of the tabu list for 2-OPTmoves could be a four-component vector whosefirst two entries identify the edges added by themove and whose last two entries identify the edgesdeleted.The tabu list embodies one of the primary short-term memory functions of the procedure, which itexecutes by recording only the t most recent moves,where / is the parameter that identifies the 'size' ofthe list. Considerable success has resulted fromstrategies that keep f at a fixed value. However,recent experimentation (Taillard, 1990) disclosesthat better performance results by varying t within achosen interval of values, remaining with a givenvalue for approximately 2t consecutive iterationsbefore choosing another.

To implement the procedure, an array denotedtabu-time (e) is created which identifies the (mostrecent) iteration when a move 'containing' a speci-fied attribute e was made. Attributes generally aredefined so that each has a natural complement or'reverse' attribute, (e.g., the complement of an at-tribute that corresponds to adding a particular edgeis the attribute that corresponds to dropping thatedge.) Then the repetition of a move containing anattribute e is avoided, along with other associatedmoves, by classifying e tabu (hence forbidding itsinclusion in future moves) as long as tabu-time (e)lies within t iterations of the current iteration. Thereversal of a move is avoided similarly by classifyingthe complement f of e tabu as long as tabu-time (f)does not exceed the difference 'current iteration~t\ Avoiding move reversals is frequently a moreeffective strategy than avoiding move repetitions,although in some contexts (Glover, 1989a) it isappropriate to prevent both reversals and repeti-tions, according to the type of move employed.

The attributes of moves that are chosen to berecorded in the elemen ts of the tabu list can be usedin a variety of ways to define tabu status. Suchdifferences are important and generate search pathswith different characteristics. Generally it seemsworthwhile to create a separate tabu list and tabulist 'size' t for each attribute class. For example,added edges and deleted edges can each have theirown lists, and the size of the list for added edges (toprevent them from being subsequently deleted)should normally be somewhat smaller than fordeleted edges (to prevent them from being sub-sequently added back), reffecting the fact that thenumber of edges contained in a traveling salesmantour is generally somewhat smaller than the n um berno t in the tour. Best ranges for t (as in the a ppro achthat varies t betw een selected limits) typically lie in apro per subset of the interval from n/3 to 3n, where nis related to problem dimension (such as the num-ber of nodes or edges of a graph, or a square root ofthis number). However, in some applications asimple choice of r in a range centered around 7seems to be quite effective. In any case, good rangesare characteristically easy to identify and highlyrobust. Dynamic tabu list strategies that employ"moving gaps" are emerging that may prove evenmore successful (Hubscher and Glover, 1990). Therelation between diflerent tabu list structures andthe types of restrictions for classifying moves tabu(as a function of selected move attributes) is anim po rtan t area for research. i i '


6/12

370 F. GLOVEROne goal of this application of short-term mem-ory is to avoid a solution path that duplicates asequence of solutions, i.e. to avoid cycling (looselydefined, since sometimes it is preferable to return toa previous solution to find an improved path lead-ing away). Because the manner of cycle avoidance

involved in the operation of the tabu list is highlyflexiblemuch m ore so than branch and boun d, forexampleit does not give an absolute theoreticalassurance that cycling is impossible. Nevertheless,from an empirical standpoint the tabu list performsthis function highly effectively for app ro pr ia tevalues of t. (More advanced types of tabu lists,theoretically m otivated, are now emerging (Glover,1990).)Step 5. Another key issue of tabu search ariseswhen the move under consideration has been found

to be tabu. If appropriate aspiration criteria aresatisfied, the move will still be considered admiss-ible in spite of its tabu classification. Roughlyspeaking, these criteria are designed to overridetabu status if a move is 'good enough'. The condi-tion 'good enough' must be sufficiently limiting tobe compatible with the goal of preventing thesolution process from cycling (in an appropriatesense).Based on this motivation, a simple form of anaspiration-level check is to permit tabu status to beoverridden if the solution produced would be betterthan the Current Best Solution. Another approachis to define a n a spir atio n level AiL) to be the lengthof the best tour that has ever been reached by amove from a tour of length L. Then, if a moveapplied to a tour of length L can produce a tourof length L* better than A{L) (i.e. L*


7/12


8/12

372 F. GLOVERtains only three elements) contains no entries, asindicated by the black dots in the cells. Thereafter,the entries in the ceils indicate both the tabu movesand, in parentheses, the aspiration values for XQwhich allow tabu status to be overridden. Theentries of the table may be explained as follows.

Iteration 1. Starting from an initial solution withXo = 60 , identified by the vector (1,0,0,0, 1), themethod examines the moves that change exactlyone X j value in this vector (from 0 to 1 or from 1 to0). The moves yielding the three best values for XQare show n in the table as X3= 1, x, = 0 and X2 = I,which, respectively, give XQ = 30, Xo = 40 andXQ = 85. (The first two moves imp rove the current XQvalue, while the third does not.) No moves are tabu,so the best move x, = I is selected, as indicated bythe (*) symbol. The reverse move x^^O becomestabu and is entered on the tabu list. The entryappears in the tabu list for Iteration 2 rather thanfor Iteration t, however, since this list shows thetabu status for moves at the start of each iteration.The aspiration value of 60 is recorded with the tabumove, because Xo = 6O occurs for the solution ofItera tion 1, and this is the value to beat if the m oveX3 = 0 is to be considered admissible.

Iteration 2. The solution obtained by setting Xj= 1 in Iteration 1 app ears in the colu mn for Iter-ation 2, identified by (1, 0, 1, 0, 1) with Xo = 30. Thebest move results by setting X3 = O to yield Xo^6O.(This discloses that a local optimum has beenreached, because the best move does not lead to asolution better than the present solution with XQ= 30). The tabu list shows, however, that the bestmove is tabu, and the associated aspiration value of60 is not surpassed by the value Xo = 60 of thismove. (The move u nder conside ration leads directlyback to the solution of Iteration I). Consequently,the symbol (T) appears in the cell to indicate themove's tabu status. In computer implementation,tabu status arrays such as the tabu-time (e) arraypermit such information to be determined directlywithout having to search a list of the form shownhere.The next best move, which yields Xo = 80 for Xj= 0, is not tabu and hence is selected. The reversemo ve, Xj = 1, is entered in the next available posi-tion on the tabu list (appearing in Iteration 3 todisclose the list condition at the start of that iter-ation), together with the aspiration value of 30,corresponding to the fact that XQ = 30 in the solu-tion of Iteration 2.

Iteration 3. The best available move once agaiis tabu, and hence is not taken. In this case thsecond-best move also appe ars on the tabu list. Thmove, X3 = 0, has an aspir ation value of 60, whiyielding Xo = 40 , and hence passes the aspiratiotest. The symbol (TA) identifies the move's admissbility in spite of being tabu, and this move iselected.

The tabu list is ujxlated as before, yielding thnew tabu list which appears on the column foIteration 4. This list contains two entries for Xj, onfor Xj = 0 and one for Xj = 1. This does not c urren tcreate a problem, since the entry for x_,=0 iirrelevant to the current solution (in which xalready is 0). In general, however, it would bpreferable to erase such an irrelevant entry, i.replacing it by a black dot in the table, to avoid thpossibility of future ambiguities.A superior me thod exists for updating aspiratiocriteria of the type used in this example when thtabu status of a move is overridden . An ap proxim ation to this approach is to subtract an amount Dfrom the aspiration values for all tabu moves recorded since the tabu move s that was overriddenwhere D is equ al to the aspira tion value for the tabmove recorded immediately after s, minus the aspiration value for s. If D is negative, then this steshould only be applied to the aspiration value fothe first tabu move after s.

Iteration 4. The solution reached at this iteratiois another local optimum, evidenced by the fact thathe best available move results in XQ = 60, as contrasted with the current Xo = 40 . The second-bemove, which is the highest evaluation move that inot tabu, is selected. Since the most recent tabu lisentry occurred in the last position, the entry for thcurrent move is made in tbe first position (to recortbe current tabu restriction X2 = 0). This removethe tabu restriction previously recorded in the firsposition, rendering it inapplicable.Iteration 5. The highest evaluation move, x^^ 1is admissible to be selected and hence becomes thcurrent choice. (The second-highest evaluatiomove is also admissible as a result of passing thaspiration test.) The solution associated with thimove yields Xo = 20 , which is better than any thufar produced, qualifying the solution as the newcurrent best.Additional columns of the table may be generated in a corresponding manner, given a subroutincapable of identifying evaluations (i.e. in this ex


9/12

TAB U SEAR C H 373

Table 2. Example problem (Solved by TabuSearch in Table I)Minimize 2O.t,-l-25x2-30x3-45x4-l-40xjSubject to

- X 2 +X tAll variables 0 1Penalty for each unit of constraint violation:70/unit for each of the first two constraintslOO/unit for each of the last two constraints

ample, the XQ values) that result for available m oves.Table 2 identifies the problem that was solved bythe process illustrated in Table 1. (Tbe solution w ithXo = 20 is in fact o ptim al for th is problem .) T histable was not provided earlier because of the diffi-culty of tracing relevant correspondences betweenthe solution process and the problem data, and theeffort of verifying the associated arithmetic calcu-lations. Table 2 is instructive at this point as ameans of disclosing how tabu search can be appliedto a problem of this type in more than one way.The initial solu tion of Table 1 is feasible for theproble m of Tab le 2. Th e solution proce ss illustratedin Table 1 is not based on requiring all subsequen tsolutions to be feasible, but instead allows consid-eration of infeasible moves, which are evaluated byimposing penalties on violating the constraints.These penalties, identified in Table 2, produce asolution trajectory in which one of the constraintsbecomes violated during the solution process.(Oth er penalties, leading to different trajectories,also could have been used.)Combinatorial optimizat ion problems are notalways conveniently structured to assure a feasiblepath will exist between all feasible solutions, andhence some method of allowing infeasible solutionsto be evaluated and visited is important. Besidesemploying evaluators that penalize infeasibilities invarious ways, tabu search also provides an ap-proach called strategic oscillation, which introducesadditional tabu restrictions to compel the search tocross feasibility boundaries to various depths(Glover, 1986, 1989a). The strategic oscillation ap-proach has been effective in applications that haveincluded the solution of p-median (lock box) prob-lems and large-scale employee scheduling problem s(Glover, 1989a; Glover and McMillan, 1986).

Fro m the perspective of the example problem thisapproach leads to consideration of a particularvariation that invites further exploration. The stra-tegic oscillation procedure often incorporates ashifting evaluation criterion that varies the em-phasis on feasibility and optimality considerations(at different depths and on different sides of feasibil-ity boundaries). A natural way to carry out this typeof process is to use adaptive penalty values for thedifferent problem constraints.

For any given set of such penalties the problem iseffectively transformed into an integer goal pro-gramming problem. Consequently, the use ofsuccessive (implicit) transformations of this sortgives rise to what may be called a tabu goal pro-gramming procedure, where the word *tabu' conveysnot o nly the use of tabu lists to avoid cycling but theadaptive m anipu lation of penalties accordin g to theobjectives of the strategic oscillation element oftabu search. (Effective use of such penalties shouldreduce the size of the tabu list that otherwise mightbe employed.) This type of procedure offers inter-esting possibilities to be investigated in integerprogramming applicat ions.

TARGET ANALYSISA careful consid eration of heuristic solution pro -cedures sooner or later encounters the challenge ofdetermining more advanced evaluation measuresthan embodied in objective function values pro-duced by available moves, as the preceding discus-sion underscores. In contexts where relaxationstrategies are used, as in obtaining LP solutions aspart of a method for solving integer programmingprob lem s, a true objective function value for a move(which results when the relaxed formulation isreplaced by the original) may not be known orreadily determined. A similar situation occurs incertain scheduling problems, where the implica-tions of making a move do not become visible untilthe schedule approaches a completed state.

Target analysis (Glover, 1986; Glover and G reen-berg, 1989; Glo ver an d Lag una, 1989; Glo ver et ai,1989) is a method that can be used to determinemore effective decision rules in such situations. Theprinciples of target analysis harmonize well withthose of tabu search, and the method also can beapplied in conjunction with many other procedures.


10/12

374 F. GLOVERIts main features may briefly be sketched by viewingthe approach as a five-phase procedure, as follows.Phas e 1 of target analys is is devo ted to apply ingexisting methods to determine optimal or near-optim al solutions to representative problem s from agiven class. This phase is straightforward in itsexecution, although a high level of effort may beexpen ded to assu re the solution s are of the specifiedquality.

Phase 2 uses the solutions produced by Phase 1as targets, which become the focus of a new set ofsolution passes. During these passes, each problemis solved again, this time scoring all available mov es(or a high-ranking subset) on the basis of theirability to progress effectively toward the targetsolu tion. (The scoring can be a simple classification,such as 'good' or 'bad'.) Choices may be biasedduring this phase to select moves that have highscores, thereby leading to the target solutions morequickly than the customary choice rules. Informa-tion generated during the solution effort, which maybe useful in inferring these scores, is stored for lateranalysis.

Phase 3 constru cts param eterized functions ofthe information recorded in Phase 2, with the goalof finding values of the parameters to create amaster decision rule. This rule is designed to choosemoves that score highly according to the outcomesof the second phase.Phase 4 generates a mathematical or statisticalmodel (such as a generalized goal programming ordiscriminate analysis model) to determine effectiveparameter values for the master decision rule. (Thesecond, third and fourth phases are not entirelydistinct, and may be iterative.) On the basis of theoutcomes of the Phase 4, the master decision rulebecomes the rule that drives the solution method.This rule itself may be evolutionary, i.e. it may usefeedback of outcomes obtained during the solutionprocess to modify its parameters for the problembeing solved.Phase 5 concludes the process by applying themaster decision rule to the original representativeproblems and to other problems from the chosensolution class to confirm its merit. (The process canbe repeated and nested to achieve further refine-ment.)Target analysis has an additional im porta nt func-tion. On the basis of the information generatedduring its application, and particularly during itsfinal confirmation phase, the method produces em-pirical frequency measures for the probabilities that

choices with high evaluations will lead to an optimal (or near-optimal) solution within a certainumber of steps. By this means, target analysis caprovide inferences concerning expected solutiobehavior, as a supplement to classical 'worst cascomplexity analysis. These inferences can aid thpractitioner by indicating how long to run a solution method to achieve a solution of desired qualit(and with a specified empirical probability).

In combination with an effective heuristic framework such as tabu search, target analysis app ears toffer a promising foundation for further advances isolving difficult optimization problems.Acknowledgement

This research has been supported in part by the NASA GranNAGQ-1388 of the Center for Space Construction and by thAir Force Office Scientific Research and the Office of NavResearch Contract # F49620-90-C-0033.

REFERENCESJ. Bovet, C. Co nstan tin and D. de Werra (1987). A convoscheduling problem. Research Report ORiVP 87/22Swiss Federal Institute of Technology in LausanneDecember.B. T. Eck (1989). Good solutions to job shop schedulinproblems via tabu search. Research Report, Department of Industrial Engineering and Operations Research, Columbia University, New York, May.U. Faigle and W. Kern (1989). Some convergence resultfor probabilistic tabu search. Research MemorandumN o . 882, Faculty of Applied Mathematics, Universitof Twente, The Netherlands, November.C. Friden, A. Hertz and D. de Werra (1989). Stabulus: technique for finding stable sets in large graphs wittabu search. Computing 42, 35-44.F. Glover (1986), Future paths for integer programminand links to artiticial intelligence. Computers and Opeations Research 13, No. 5, 533-49.F. Glover (1989a). Tab u search, part I. ORSA Journal oComputing 1, No. 3, 190-206.F. Glover (1989b). Candidate list strategies and tabsearch. CAAl Research Report, University of Coloado, Boulder, July.F. Glover (1990). Tabu search, part II. ORSA Journal oComputing 2, No. I , 4-32.F. Glover and H. J. Greenberg (1989). New approachefor heuristic search: a bilateral linkage with artificiaintelligence. European Journal of Operational Researc39 , No. 2, 119-30.F. Glover a nd M, Lagun a (1989). Target analysis timprove a tabu search method for machine schedulingTechnical Report, Advanced Knowledge ResearcGroup, US West Advanced Technologies. BouldeColorado .F. Glover a nd C. M cMillan (1986). The general em ploye

scheduling problem; an integration of managemen


11/12

TAB U SEAR C H 375science and artificial intelligence. Computers and Oper-ations Research 13, No. 5, 563-93.F . Glover, C. Mc Millan and B. Novick (1985). Interactivedecision software and computer graphics for architec-tural and space planning. Annals of Operations Re-search 5, C. Monma (ed) 557-73.R. Glover, D . Klingm an and N. V. Phillips (1989). Anetwork-related nuclear plant model with an intelli-gence branch-and-bound solution approach. Annals ofOR: Linkages with Artificial Intelligence.

P. Hansen and B. Jau ma rd (1987). Algorithms for themaximum satisfiability problem, RUTCOR ResearchReport RR # 43-87, Rutgers, NJ. June.A. Hertz and D. de Werra (1987). Using tabu searchtechniques for graph coloring. Computing 29 , 345-51.A. Hertz and D . de Werra (forthcoming). The tabu searchmetaheuristic: how we used it. Annals of Mathematicsand Artificial Intelligence.R. Hubscher and F. Glover (1990). Tabu search for binpacking. CAAI Research Report 90-4, University ofColorado, Boulder , September.J. Knox (1989). The Application of Tabu Search to theSymm etric Tra veling Salesman Problem, PhD thesis.Graduate School of Business, University of Colorado,July.M. Lag una 11989). A tabu search meth od for sch edulingjobs on parallel processors. Research Report, Depart-ment of Mechanical Engineering, University of Texas-Austin, May.M. Laguna, J. W. Barnes and F. Glover (1989).Scheduling job s w ith linear delay penalties and

sequence dependent setup costs using tabu search.Research Report, Department of Mechanical Engineer-ing, The University of Texas-Austin. April.M. Malek, M. Guruswamy, H. Owens and M. Pandya(1989a). Serial and parallel search techniques for thetraveling salesman problem. Annals of OR: Linkageswith Artificial Intelligence.M. M alek, M. Hea p, R. Ka pur and A. Mo urad (1989b), Afault tolerant implem entation of the traveling salesmanproblem. Research Report, Department of Electricaland Computer Engineering, University of Texas-Austin, May.J, Ryan, S. Oliveira and G, Stroud (1989). A parallelversion of tabu search and the path assignment prob-lem. Heuristics for Combinatorial Optimization 1-24,J. Skorin-Kapov (1990). Tabu search applied to thequadratic assignment problem. ORSA Journal on Com-puting 2: 1 , 33-45 .E. Taillard (1990). Robust ta boo search for the qua draticassignment problem. Research Report ORWP 90/10,EPFL, DMA, Lausanne, Switzerland, June.Ch. Wendelin (1988). Graph partitioning with the aid ofthe tabu method. Project Report, Institute fur HohereStudien, Vienna, Austria, June.M. Windmer and A. Hertz, (forthcoming). A new ap-proach for solving the flowshop sequencing problem.European Journal of Operational Research.D . W oodr ufTa nd E. Zemel (1990). Ha shing vectors fortabu search. Technical Report 90-08, Department ofI.E. and M.S., Northwestern University, July,


12/12

TS - AI Heuristic Frameworks 1990.pdf

Documents