LEARNING PLAYING STRATEGIES IN CHESS

Computational Intelligence, Volume 12, Number 1, 1996LEARNING PLAYING STRATEGIES IN CHESSEduardo Morales M.Instituto Tecnol�ogico y de Estudios Superiores de Monterrey { Campus Morelos,Apto. Postal C{99, Cuernavaca, Morelos, 62050, M�exico, email: [email protected] is believed that chess masters use pattern{based knowledge to analyze a position, followedby a pattern{based controlled search to verify or correct the analysis. In this paper, a �rst{ordersystem, called PAL, that can learn patterns in the form of Horn clauses from simple exampledescriptions and general purpose knowledge is described. It is shown how PAL can learn chesspatterns which are beyond the learning capabilities of current inductive systems. The patternslearned by PAL can be used for analysis of positions and for the construction of playing strategies.By taking the learned patterns as attributes for describing examples, a set of rules which decidewhether a Pawn can safely be promoted without moving the King in a King and Pawn vs. Kingendgame, is automatically constructed with a similarity-based learning algorithm. Similarly, aplaying strategy for the King and Rook vs. King endgame is automatically constructed with asimple learning algorithm, by following traces of games and using the patterns learned by PAL.Limitations of PAL in particular, and �rst{order systems in general, are exposed in domains wherea large number of background de�nitions may be required for induction. Conclusions and futureresearch directions are given. 1. INTRODUCTIONIt is believed that chess masters use pattern{based knowledge to analyze a posi-tion, followed by a pattern{based controlled search to verify or correct the analysis(Charness 1977; de Groot 1965). For example given the position of Figure 1, a chessplayer recognizes that:� A white Rook threatens the black Queen.� The black Bishop is pinned .� The white Queen is threatened by a black Pawn.� The white Knight can fork the black King, the black Rook and the black Knight.� Moving the foremost white Pawn can discover a threat , create a pin, and possiblya skewer .� : : :This analysis involves the recognition of concepts like, threat , pin, discovered threat ,fork , skewer , : : :, etc., which can then be used to choose a move (e.g., move the whiteKnight and check the black King). Previous work has shown how playing strategiescan be constructed following a pattern{based approach (Berliner 1977; Bramer 1980;Bratko 1982; Huberman 1968; Pitrat 1977; Wilkins 1979). However, a substantialprogramming e�ort needs to be devoted to the de�nition and implementation ofthe right patterns for the task. There have also been some attempts to learn chessconcepts from examples described with a set of attributes (Quinlan 1983; Shapiro1987). In this case too, most of the work is dedicated to the de�nition of adequateattributes to describe the examples and to construct the target concept. This workinvestigates whether chess patterns (such as those described above) that are powerfulenough for play can be acquired by machine learning techniques from simple exampledescriptions.The limited expressiveness of attribute{based learning systems has led to an in-creased interest in learning from �rst{order logical descriptions, despite the increasedcomplexity of the learning problem. First{order learning systems have used back-c 1996 Blackwell Publishers, 238Main Street, Cambridge,MA 02142, USA, and 108 Cowley Road, Oxford, OX4 1JF, UK.

2 Computational Intelligence0Z0l0ZrZopo0ZkZr0Z0ZnapZZ0Z0ZQZp0ZPZ0Z0ZZBZ0Z0ZNPO0Z0ZPOZ0ZRZRJ0Figure 1. An example positionground knowledge to induce concepts from examples. This is important as it allowsa simpler (and often more natural) way to represent examples. Background knowl-edge can help as well to reduce the inductive steps taken when learning particularconcepts. Furthermore, a basic core of background knowledge de�nitions can be usedto learn several concepts.Learning the chess concepts we are interested in is a non{trivial task beyond thecapability of existing systems. There are several technical issues that need to beaddressed:1. Chess requires a relatively large amount of background knowledge. This createssevere search problems when the background knowledge is used in learning.2. Chess concepts are inherently non{determinate.1 This is a problem for �rst{orderlearning mechanisms.3. Chess concepts are learned incrementally. This leads to a requirement thatlearned knowledge can be \recycled". The most e�ective way to do this is tohave the background knowledge in the same form as the induced knowledge.4. In many practical domains where the trainer does not understand the detailsof the learning mechanism, it is important that the learning mechanism is ro-bust with respect both to the examples presented and to their order. This isparticularly so with �rst{order systems which are particularly sensitive to suchvariations since their inductive steps are powerful and underdetermined.These issues are addressed by PAL.2 The central theme is that the notion of a pattern,de�ned as a set of relations between components of a state (such as a chess position),allows the e�ciency issues to be addressed without compromising the quality of theinduced knowledge.The patterns learned by PAL are powerful enough to be used in the analysis ofpositions and in the construction of playing strategies. In (Morales 1992) it is shownhow a set of rules for deciding whether a Pawn can safely promote in a King andPawn vs. King (KPK) endgame can be constructed with patterns learned by PAL. In1Clauses whose literals are not completely determined by instantiations of the head.2PAtterns and Learning.

Learning Playing Strategies in Chess 3(Morales, 1994) it is shown how a correct playing strategy3 for the winning side in theKing and Rook vs. King (KRK) endgame can be constructed with patterns learned byPAL. In both cases, the strategies were manually constructed by the author. In thispaper, it is shown how equivalent strategies can be automatically constructed withdi�erent learning mechanisms. It is also suggested how to integrate PAL's learningcapabilities for inducing patterns, with these learning mechanisms for constructingstrategies, into a single learning system.Section 2 brie y describes PAL and its generalization method. Section 3 showshow PAL is used to learn several chess concepts (such as those illustrated in Figure1). In section 4, learned patterns are used as attributes by a similarity{based learning(SBL) system, to construct a set of rules to decide whether a Pawn can be safelypromoted in a KPK endgame without doing any search. In section 5, a playingstrategy for the KRK endgame, using learned patterns, is automatically constructedby following traces of games. Section 6 has a comparison with related work. Finally,conclusions and future research directions are given in section 7.2. GENERALIZATION METHOD AND PALInductive Logic Programming (ILP) is a fast growing research area which com-bines logic programming with machine learning (Muggleton 1991). The general set-ting for ILP is, given a background knowledge K (in the form of Horn clauses) andsets of positive (E+) and negative (E�) examples, �nd a hypothesis H (another setof Horn clauses) for which K ^H ` E+ and K ^H 6` E�. That is, �nd a hypothesiswhich can explain the data in the sense that all the positive (E+) but none of thenegative (E�) examples can be deduced from the hypothesis and the backgroundknowledge. This inductive process can be seen as a search for logic programs overthe hypothesis space. Several constraints have been imposed to limit this space andguide the search. For learning to take place e�ciently, it is often crucial to structurethe hypotheses space. This can be done with a model of generalization. Search forhypotheses can then be seen as searching for more general clauses given a knownspecialized clause.Plotkin (1969; 1971a; 1971b) was the �rst to study in a rigorous manner thenotion of generalization based on �-subsumption. Clause C�-subsumes clause Di� there exists a substitution � such that C� � D (i.e., there exists a substitutionthat makes clause C a subset of clause D). Clause C1 is more general than clauseC2 if C1 �-subsumes C2. Plotkin investigated the existence and properties of leastgeneral generalizations (lgg) between clauses and the lgg of clauses relative to somebackground knowledge (rlgg). An lgg is a generalization that is less general, in termsof �-subsumption, than any other generalization. The lgg algorithm replaces all thedi�erent terms that have the same place within compatible literals4 by new variables.(See (Plotkin 1969) for more details.) For example, if we have two compatible literals:L1 = threat(white,rook,square(1,3),black,bishop,square(4,3))L2 = threat(white,queen,square(1,4),black,bishop,square(7,4))lgg(L1; L2) =threat(white,Piece1,square(1,Y),black,bishop,square(X,Y))3Meaning that regardless of the opponent's moves, the winning side can check-mate from any legalwinning-side to move position.4Literals with the same predicate name and the same number of arguments.

4 Computational IntelligenceTable 1. A plausible rlgg algorithm for a set of example clauses� given:{ a logic program (K){ a set of example clauses (SC )� Take an example clause (C1) from SC. Let �1;1 be a substitution groundingthe variables in the head of C1 to new constants and �1;2 grounding theremaining variables to new constants� Construct a new saturated clause (NC ) de�ned as:NC � C1�1;1 [ f:A1;1;:A1;2; : : :g whereK ^ C1body�1;1�1;2 j= A1;i, and A1;i is a ground atom� Set SC = SC �fC1g� while SC 6= f;g{ Take a new example clause (Cj) from SC. Let �j;1 be a substitutiongrounding the variables in the head of Cj to new constants, and �j;2grounding the remaining variables to new constants{ Construct a new saturated clause (C 0j) de�ned as:C0j � Cj�j;1 [ f:Aj;1;:Aj;2; : : :g whereK ^ Cjbody�j;1�j;2 j= Aj;k and Aj;k is a ground atom{ Set NC = lgg(C 0j,NC ){ Set SC = SC �fCjg� output NCThis generalization process is repeated between all the pairs of compatible literalswithin clauses. The lgg of two clauses C1 and C2 is de�ned as: fl : l1 2 C1 and l2 2C2 and l = lgg(l1; l2)g.More recently, Buntine (1988) de�ned a model{theoretic characterization of �-subsumption, called generalized subsumption for Horn clauses. (See (Buntine 1988)for more details.) Buntine also suggested a method for constructing rlgg 's usingPlotkin's lgg algorithm between clauses. The general idea of the rlgg algorithm isto augment the body of the example clauses with facts derived from the backgroundknowledge de�nitions (K) and the current body of the example clauses, and thengeneralize these augmented or saturated clauses using Plotkin's lgg algorithm.Following (Buntine 1988) the basis for a learning algorithm as shown in Table 1using rlgg is as follows: Let K be a logic program or theory, �i;1 be a substitutiongrounding the variables in the head of clause Ci to new constants and �i;2 ground-ing the remaining variables to new constants. If rlgg(C1; C2; : : : ; Cn) exists, it isequivalent w.r.t. K to the lgg(C 01; C 02; : : : ; C 0n), where for 1 � i � n,C0i � Ci�i;1 [ f:Ai;1;:Ai;2; : : :g (1)where K ^ Cibody�i;1�i;2 j= Ai;k, Ai;k is a ground atom, and the Ai;ks are all thepossible ground atoms deduced from the theory. Equation 1 can be rewritten as:C0i � Cihead�i;1 Cibody�i;1; Ai;1; Ai;2; : : : (2)

Learning Playing Strategies in Chess 5Let the resulting least general generalization of N saturated clauses be denoted as:GC(n) = lgg(C 01; C 02; : : : ; C 0n).Since lgg(C 01; C 02; : : : ; C 0n) � lgg(C 01; lgg(C02; : : : ; lgg(C0n�1; C 0n) : : :)) (Plotkin 1971b),then, GC(n) = lgg(C 0n; GC(n� 1)).This can be used for constructing the rlgg of a set of clauses. If a set of examples isdescribed with a set of clauses, a learning algorithm can accept a new example, con-struct a saturated clause with it and atoms derived from the background knowledge(logic program), and gradually generalize the clause by taking lgg 's of this clauseand subsequent clauses constructed from new examples until meeting a terminationcriterion. PAL's learning algorithm is based on this framework. A direct implemen-tation of it is impractical for all but the simplest cases, as it essentially involves thededuction of all ground atoms logically implied by the theory. (See (Niblett 1988) fora more thorough discussion on generalization.) However, practical implementationsof rlgg exist for theories without variables (as in Golem (Muggleton & Feng 1990)),theories without function symbols (as in Clint (De Raedt & Bruynooghe 1988))),and when only a �nite number of facts are deducible from the theory. The latteris achieved either by limiting the depth of the resolution steps taken to derive factsand/or by constraining the background knowledge de�nitions. This is the methodused in PAL. Even with a �nite set of facts, the lgg of two clauses can generate avery large number of literals. The main problem is that lgg produces very small gen-eralization steps and some heuristics are required to converge faster into a solutionto achieve practical results. PAL uses a pattern{based background knowledge repre-sentation to derive a �nite set of facts and applies a novel constraint which identi�esthe role of the components in di�erent example descriptions to reduce the complexityof the lgg algorithm.2.1. PALExamples in PAL are given as sets of ground atoms (e.g., descriptions of chesspositions stating the position of each piece in the board), and unlike other systems,the exact arguments involved in the target concept are not speci�ed in advance. Achess position can be completely described by a set of four{place atoms (contents/4)stating the side, name, and place of each piece in the board. For instance, con-tents(white, rook, square(2,3), pos1 ) states that a white Rook is at the second �leand third rank in position 1. Other pieces in a board position can be describedin the same way. In general, other descriptions can be used as well (see (Morales1992)). PAL uses example descriptions with its current background knowledge toinduce patterns in chess.In the context of chess, a pattern refers to relations between pieces and places inthe board. PAL induces pattern de�nitions with the following format:Head D1; D2; : : : ; Di; F1; F2; : : :where,� Head is the head of the pattern de�nition. Instantiations of the head are regardedas the patterns recognized by the system.

6 Computational Intelligence� The Di's are \input" predicates used to describe positions (i.e., contents/4) andrepresent the components which are involved in the pattern.� The Fi's are instances of de�nitions which are either provided as backgroundknowledge or learned by PAL, and represent the conditions (e.g., relations be-tween pieces and places) to be satis�ed by the pattern.In the context of chess, PAL can learn pattern de�nitions which are associatedwith a particular move. Only one move ahead is considered during the learning pro-cess; however, once a pattern is learned, it can be used to learn other patterns aswell. Make move is a predicate that changes the current state of the board descrip-tion. The make move predicate de�nes 1-ply moves; instantiations of this predicaterepresent the di�erent possible 1-ply moves. To learn such pattern de�nitions, theactual move of a piece is performed (changing the description of the board) and newinstantiations of patterns (ground atoms) are generated (deduced) after each move.For such patterns PAL induces concepts with the following format:Head D1; D2; : : : ; Dk;F1; F2; : : : ; Fm;MV1; F1;1; F1;2; : : : ; F1;n;MV2; F2;1; F2;2; : : : ; F2;p;...MVr; Fr;1; Fr;2; : : : ; Fr;s: (3)where,� MVi is an instance of the make move predicate representing a legal move withthe opponent's side not in check, and the Fi;j 's are instances of pattern de�nitionsthat change as a consequence of the move.Only moves which introduce a new predicate name or remove an existing predicatename after the move are considered. The complement5 of the new predicate is addedbefore the move.PAL starts with some pattern de�nitions as background knowledge and uses themto learn new patterns. For instance, the de�nition of being in check is given to PALas follows:in check(Side,KPlace,OPiece,OPlace,Pos) contents(Side,king,KPlace,Pos),contents(OSide,OPiece,OPlace,Pos),other side(Side,OSide),piece move(Oside,OPiece,OPlace,KPlace,Pos).This de�nition gets instantiated only with example descriptions with a King atKPlace and an opponent's piece (OPiece) at OPlace which could be moved to KPlace.Given an example description, PAL \collects" instantiations of its pattern{basedbackground knowledge de�nitions to construct an initial hypothesis clause. Thehead of the clause is initially constructed with the arguments used to describe the5The complement of P is :P and vice versa.

Learning Playing Strategies in Chess 7�rst example description. The initial head, in conjunction with the facts derivedfrom the background knowledge (instantiations of pattern{based de�nitions) and theexample description, constitutes an initial concept clause. This clause is generalizedby taking the lgg of it and clauses constructed from other example descriptions.Even with a �nite theory for chess, the large number of plausible facts derivablefrom it makes the �niteness irrelevant in practice. (Consider, for example, all thepossible legal moves for pieces in chess.) In PAL a fact F is relevant to exampledescription D if at least one of the ground atoms of D occurs in the derivation ofF . Since PAL constructs its clauses using pattern{based de�nitions, only a �nite setof relevant facts are considered. PAL is informed of which background knowledgede�nitions are considered as patterns.For instance, if there is only a white Bishop and a black Pawn in a chess position,described as follows:contents(white,bishop,square(2,3),pos1).contents(black,pawn,square(3,4),pos1).and PAL's only pattern{de�nition is legal move/5, then PAL produces the followingclause:tmp(white,bishop,square(2,3),black,pawn,square(3,4),pos1) contents(white,bishop,square(2,3),pos1),contents(black,pawn,square(3,4),pos1),legal move(white,bishop,square(2,3),square(1,4),pos1),legal move(white,bishop,square(2,3),square(3,2),pos1),legal move(white,bishop,square(2,3),square(4,1),pos1),legal move(white,bishop,square(2,3),square(3,4),pos1),legal move(white,bishop,square(2,3),square(1,2),pos1),legal move(black,pawn,square(3,4),square(2,3),pos1),legal move(black,pawn,square(3,4),square(3,3),pos1).This clause can be gradually generalized with other examples.2.2. Novel constraintsA common constraint in ILP systems, is to limit the size of the resulting gener-alized clause by requiring all the variables arguments to appear at least twice in theclause. (See, for example, (de Raedt & Bruynooghe 1988; Muggleton & Feng 1990;Rouveirol 1992).) In addition, PAL uses a novel constraint based on labeling thedi�erent components which are used to describe examples to guide and constrainedthe lgg algorithm. Every constant occurring in the atoms used to describe examplesis labeled with a unique constant symbol. For instance, in the example position givenabove, the Bishop and the Pawn are represented as follows (the following notationhas been adopted for presentation purposes: bl = black, wh = white, and (X,Y ) =square(X,Y )):6contents(wh,bishop,(2,3),pos1) �! contents(wh�,bishop� ,(2 ,3�),pos1)contents(bl,pawn,(3,4),pos1) �! contents(bl� ,pawn�,(3�,4 ),pos1)The labels are kept during the derivation process, so the system can distinguishwhich components are \responsible" for which facts derived from the background6The labels are used to identify particular pieces, so it is not necessary to label pos1.

8 Computational Intelligenceknowledge, by following the labels. Thus the instances of legal move/5 which involvethe Bishop use the same labels associated with the description of the Bishop, i.e.,7legal move(wh,bishop,(2,3),(4,1),pos1) �!legal move(wh�,bishop� ,(2 ,3�),(4,1),pos1)legal move(wh,bishop,(2,3),(3,4),pos1) �!legal move(wh�,bishop� ,(2 ,3�),(3�,4 ),pos1)...The predicates used to describe examples are part of the pattern de�nitions (e.g.,contents/4). When instantiations of these patterns are collected for each exampledescription, PAL identi�es the instantiations of the description predicates, i.e., thecomponents involved in the instantiation of the pattern. The lgg between compatibleliterals is guided by the associated labels to produce a smaller number of literals, aslgg 's are produced only between compatible literals with common labels. (A simplematching procedure is used for this purpose.) In chess this constraint means thatlegal moves of non{corresponding pieces are not considered in the lgg algorithm.For instance, with the white Bishop at (2,3) and the black Pawn at (3,4), thereare �ve legal moves for the Bishop and two for the Pawn. A second position where theBishop is changed to a black Pawn at square(4,2), and the Pawn to a white Knightat square(5,4), has one legal move for the Pawn and 8 for the Knight. The labelsused in the �rst position for the �rst piece are associated with the �rst piece of thesecond example. The lgg algorithm without labels produces 63 literals for the legalmove predicate (i.e., (5 + 2)� (1 + 8)). With labels it produces 21 (i.e., 5� 1 + 2�8). To recognize the corresponding pieces, the example generator (described below)knows which pieces are changed and associates their corresponding labels. Exampleswhich are manually provided or selected at random require that the correspondingcomponents are presented in the same order. This is equivalent to giving examplesas sets of predicates and not allowing any shift in the order of the arguments.2.3. Automatic example generatorPAL can experiment by automatically generating positive and negative examplesvalidated by the user. The example generator takes the current concept de�nitionand tries to fail on at least one of its literals by changing (perturbing) some of thearguments involved in the example description. For instance, changing the place of apiece, changing its side, or changing one piece for another. It uses domain dependentknowledge of the possible values of the arguments used to describe positions (e.g.,possible pieces, places, and sides). If the perturbation method generates a negativeexample, then the system analyzes which literals failed on that example and triesto construct a new example that will succeed on at least one of them. If a positiveexample is generated, PAL constructs a clause as de�ned above, and generalizes itusing the constrained lgg algorithm, with the current concept de�nition. To generateexamples, PAL �rst considers sets with the smallest number of arguments, i.e., itchanges the position of each piece at a time, before changing the positions of severalpieces at a time. When PAL cannot generate a new example with the current setof arguments, to produce a new generalization of the current de�nition, it continueswith a di�erent set of arguments. PAL stops when there are no more sets left, or7Note that square(4,1), in the �rst literal, is not labeled, since it is not a piece's position, whilesquare(3,4), in the second literal, takes the labels of the black Pawn.

Learning Playing Strategies in Chess 9when the user decides to terminate the process. A more detailed description is givenin (Morales 1991; 1992).The lgg algorithm constructs a single clause (the least general generalized clause)from a set of clauses. In order to learn disjunctive de�nitions (i.e., de�nitions whichrequire more than one clause), systems which use lgg must rely on the negativeexamples produced by the system and/or provided by the user. This is also true formost inductive learning algorithms. In PAL each new de�nition is checked against thecurrent negative examples. If a de�nition covers a negative example, it is rejected andthe example is stored. When the perturbation process �nishes or all the exampleshave been processed, the �nal de�nition is checked against the stored examples.Those examples which are covered are eliminated and those which are not coveredare tried again. PAL selects the �rst uncovered example and the whole processis repeated until all the positive examples have been covered without covering anynegative example. In this way, PAL learns disjunctive concepts by learning eachclause separately. Alternative methods for handling disjunctive concepts, like storingintermediate hypotheses and allowing some form of backtracking, are left for futureresearch. As the perturbation process cannot guarantee to produce an instance ofeach particular disjunct, the user must provide at least one example for each disjunctin advance, to be sure that all the disjuncts will be learned.3. LEARNING PATTERNS IN CHESSIn order to learn patterns in chess, PAL was provided with the following pattern{based background vocabulary.contents(Side,Piece,Place,Pos):Describes the position of each piece.other side(Side1,Side2 ):Side1 is the opponent side of Side2.sliding piece(Piece,Place,Pos):Piece is Bishop, Rook or Queen.in check(Side,Place,OPiece,OPlace,Pos):King in Place is in check by OPiece in OPlace.check mate(Side,Place,Pos):Side with King in Place is check mated.legal move(Side,Piece,Place,NPlace,Pos):Piece in Place can move to NPlace.stale(Side,Piece,Place,Pos):Piece in Place cannot move.make move(Side,Piece,Place,NPlace,Pos1,Pos2 ):Piece in Place moves to NPlace.The domains of the arguments used to describe chess positions and a representativeexample for each concept were also given.domain(piece,[pawn,knight,bishop,rook,queen,king]).domain(side,[black,white]).domain(place,[square(1,1),square(1,2),...,square(8,8)]).With the above information PAL was used to learn several chess concepts. (SeeTable 2.) For instance the following is a specialization of the concept of fork learned

10 Computational Intelligenceby PAL. Its interpretation is that a piece (P3) threatens another piece (P2) andchecks the King at the same time:fork(S1,king,(X1,Y1),S1,P2,(X2,Y2),S2,P3,(X3,Y3),Pos) contents(S1,king,(X1,Y1),Pos),contents(S1,P2,(X2,Y2),Pos),contents(S2,P3,(X3,Y3),Pos),other side(S2,S1),in check(S1,(X1,Y1),P3,(X3,Y3),Pos),legal move(S2,P3,(X3,Y3),(X2,Y2),Pos).An example of a pattern involving a 1{ply move (discovery check) is given below. Acheck by piece (P2) can be \discovered" after moving another piece (P1) to (X4,Y4).disc check(S1,P1,(X1,Y1),S1,P2,(X2,Y2),S2,king,(X3,Y3),Pos1) contents(S1,P1,(X1,Y1),Pos1),contents(S1,P2,(X2,Y2),Pos1),contents(S2,king,(X3,Y3),Pos1),other side(S1,S2),sliding piece(P1,(X1,Y1),Pos1),legal move(S1,P2,(X2,Y2),(X4,Y4),Pos1),not in check(S2,(X3,Y3),P1,(X1,Y1),Pos1),make move(S1,P2,(X2,Y2),(X4,Y4),Pos1,Pos2),in check(S2,(X3,Y3),P1,(X1,Y1),Pos2).The number of examples that PAL presents to the user can be reduced by takingadvantage of the symmetric properties of the board. In particular, some concepts inchess do not depend on any particular orientation of the board, and for each example7 equivalent examples can be generated, considering re ections along the horizontal,vertical and diagonal axes. PAL can take advantage of this knowledge to producefurther generalizations between all the \symmetric" examples before presenting theuser with a new example. (The exceptions are with concepts involving Pawns orcastling.) The user can tell the system which axes of symmetry to consider.Table 2 lists some of the concepts learned by PAL. The concept's name and itsarity are given in the �rst column. The number of examples produced by the examplegenerator (second column) is compared with those produced by PAL when additionalknowledge about symmetries is taken into account (third column). The minimumnumber of examples that a trained user (the author) needed to produce the samede�nitions was also recorded (fourth column).8 The additional background knowledgelearned by PAL, such as the concept of fork described above, and used to learn otherconcepts is given in the last column. The number of examples generated by PALcompares very favorably with the size of example space. For example, the examplespace for 3 pieces is approximately 108 examples. Using additional knowledge aboutthe symmetric properties of the board reduces the number of examples presented tothe user to almost one �fth on average.8Single{clause concept de�nitions can be learned with positive examples only, provided there is adequatebackground knowledge.

Learning Playing Strategies in Chess 11Table 2. Table of results for chess conceptsLearned Generated Generated Generated AdditionalConcept examples examples examples backgroundwith symmetry by the user knowledgethreat/7 23 + 8 � 10 + 7 � 2 + |fork/10 22 + 67 � 9 + 16 � 3 + |can check/7 20 + 18 � 2 + 1 � 2 + |can threaten/7 23 + 7 � 6 + 9 � 3 + threatcan fork/10 34 + 29 � 3 + 0 � 3 + forkdisc check/10 17 + 44 � 5 + 6 � 4 + |disc threat/10 26 + 48 � 4 + 2 � 3 + threatpin/10 22 + 42 � 4 + 3 � 3 + threatskewer/10 19 + 55 � 4 + 22 � 3 + threatAverage (Tot) 58.22 12.55 2.884. LEARNING CAN-RUN FOR THE KPK ENDGAMEThe patterns learned by PAL can be used to analyze chess positions. Using onlythe evaluation of patterns, we tried to automatically construct a decision procedureto judge whether a Pawn can safely be promoted in the KPK endgame withoutmoving the King (i.e., without doing any search). Although this may look like asimple task, it requires several interactions for de�ning the right patterns. (See(Bramer 1980; Morales 1992b).) The number of patterns to learn, required for thisstrategy, depends on the amount and \quality" of the background knowledge usedby PAL. A trivial strategy can be constructed if the de�nition of distance to thequeening square, considering the possible interaction between Kings, is included inthe background knowledge.The following background knowledge was provided to PAL to learn patterns forthis endgame. distance(Place1,Place2,Dist,Pos):Distance (Dist) between a piece in Place1 anda piece in Place2 .manh dist(Place1,Place2,Dist,Pos):Manhattan distance between two pieces.dist qs(Place,Dist,Pos):Distance (Dist) between a piece in Place andthe queening square.coordx(Place,X,Pos):The �le of a piece in Place.coordy(Place,Y,Pos):The rank of a piece in Place.In all 36 patterns were learned for this endgame. With them a strategy for de-ciding whether the Pawn can run using only the evaluation of patterns (i.e., without

12 Computational IntelligenceTable 3. Results from CN2 on the training and testing set.PREDICTEDACTUAL can run cannot run Accuracycan run 44,494 3 100%cannot run 259 23,294 99.1%doing any search) was manually constructed in two weeks (Morales 1992). A shortdescription of some patterns learned by PAL is given in appendix A.To automatically learn an equivalent strategy, a Similarity-Based Learning (SBL)algorithm, called CN2 (Clark & Niblett 1989), was used. The idea was to characterizethe di�erent positions with the patterns learned by PAL, and let the SBL algorithmlearn the appropriate decision rules. Given a set of examples described by attribute{value pairs with an associated class, SBL systems try to induce a set of rules ordecision trees to characterize all the examples of one class with a set of tests on theattributes. The basic operation of CN2 is that of generating a complex (a conjunct ofattribute tests) which covers (is satis�ed by) a subset of the training examples. Thiscomplex forms the condition part of a production rule `if condition then predictclass', where class is the most common class in the training examples which satisfythe condition. The search proceeds in CN2 by repeatedly specializing candidatecomplexes until one which covers many examples of a single class (C) and few ofother classes is located. Once a good complex is found, those examples it covers areremoved from the training set and the rule `if <complex> then predict C' is addedto the end of the rule list. This process iterates until no more satisfactory complexescan be found. (See (Clark & Niblett 1989) for more details.)All the possible starting positions with the Pawn's �le ranging from 1 to 4 and itsrank from 3 to 7 (roughly 78,000 positions), were generated. All the patterns learnedby PAL, used in the manual construction of a decision strategy, were evaluated oneach position and then used as attributes for the CN2 algorithm. (Values of \T"or \F" were assigned when a pattern was or was not applicable.) All the positionswere classi�ed as can run or cannot run using a simple search procedure. The �leand rank of each piece were also included as attributes. In all 42 attributes wereconsidered for each position (example).All the evaluated positions were randomly divided into two sets, 2/3 of the totalas a training set, and 1/3 as a testing set. With the training set, CN2 generated22 rules for can run and 32 rules for cannot run. (In 44,497 of the total positionsthe Pawn can run and in 23,553 it cannot run.) The rules for can run are given inappendix B. The results produced by CN2 on the training and testing set are givenin Table 3.Overall, 44,494 positions are correctly classi�ed as can run with 259 positions in-correctly classi�ed as can run. On the other hand, there are 23,294 positions correctlyclassi�ed as cannot run while only 3 incorrectly classi�ed as cannot run.With a larger set of examples in the training set, 100% accuracy can be obtainedon the cannot run class. This means that with the induced rules, any starting positioncan be correctly classi�ed as can run or cannot run using only the evaluation of staticpatterns.

Learning Playing Strategies in Chess 13Table 4. A plausible generalization algorithm.Given an initial set of attributes and some background knowledgeUse an SBL algorithm to generate a set of rulesUntil 100% accuracy is obtained in one classLet E+ be the set of misclassi�ed examplesLet E� be the rest of the examplesLet Patterns = []Until E+ is emptyLet C1 be an example taken from E+While C1 does not cover any example in E�Let C2 be another example taken from E+Let C1 = generalization (using PAL) of C1 and C2Add C1 to PatternsRemove all the examples from E+ covered by C1Add all the learned patterns as new attributesThis result may not be too surprising, as we were able to manually generateequivalent rules using the same patterns (with 19 rules for can run). However, itsuggests a way to integrate the classi�cation capabilities of an SBL algorithm, withthe learning capabilities of PAL. A plausible algorithm is described in Table 4. Theidea is to generate an initial set of rules using an SBL algorithm. If there is not 100%accuracy on one class, then learn a set of patterns to characterize all the misclassi-�ed examples. Augment the description of the examples with the new patterns (orattributes) and use the SBL algorithm again.Although a simple search procedure can achieve equivalent results, the signi�-cance of integrating both learning techniques is that they could be used to producepattern{based if{then rules on a more di�cult task where an easy{to{evaluate pro-cedure is not easy to produce. In addition, the identi�cation of an adequate set ofattributes to describe examples when using an SBL system is not an easy task. AnILP system, such as PAL, could be used to ease this process.A more fundamental problem is to decide on the starting background knowledgefor PAL. A clear indication of lack of adequate background knowledge is the inabilityto learn a pattern able to discriminate among positions belonging to di�erent classes.The identi�cation of which knowledge is required is a much harder open researchproblem.5. LEARNING A PLAYING STRATEGY FOR THE KRKENDGAMEPatterns learned by PAL are powerful enough to be used in the construction ofplaying strategies. (See (Morales 1994).) Using previously learned patterns, we triedto learn general playing rules for the KRK endgame from traces of games. For thispurpose a two phase process (learning and testing) was followed. In the learningphase, particular rules were induced by following the moves made by the winningside and looking at all the applicable/non{applicable patterns before and after themove. The hypothesis is that the identi�ed patterns before and after a move provide

14 Computational Intelligencethe required conditions for making that move, without using any evaluation function.Playing rules were constructed with the following format:rule(Side,Piece,Place,NPlace) :{condition1,condition2,: : :make move(Side,Piece,Place,NPlace),conditioni ,conditioni+1 ,: : :where,� make move is the move made by a chess player (human/machine).� conditionj before and after the move is an applicable or non-applicable pattern(patternj or not patternj)A rule is applicable if all the conditions before the move are satis�ed and a move canbe generated to ful�ll all the conditions after the move.As in the KPK experiment, all the patterns learned by PAL and used in a manualconstruction of a playing strategy for this endgame (16 learned patterns + givencheck mate) were considered. A description of some of these patterns appears inappendix C.Considering all the patterns involved before and after the move produces very longand speci�c rules with 34 conditions in all. This, in turn, produces strategies with alarge number of rules and long decision times to choose a rule to �re. One possibilityfor constructing more general, shorter rules is to eliminate all the patterns that areinvariant with respect to the move, using a similar strategy to PAL's when learning1-ply patterns. That is, keep only the patterns that change as a consequence of themove. The hypothesis is that the patterns that change can provide by themselvessu�cient conditions to decide on possible moves. This strategy, however, turned outto be too severe, because some of the patterns are worth maintaining after a move.For instance, if the Rook is dividing both Kings either vertically or horizontally, itis desirable to maintain, in many cases, that condition after a move made by thewinning side. Based on the manually constructed strategy, only three patterns wereallowed to persist before and after a move. (The rest of the invariant patterns wereeliminated.) The following three patterns were chosen:9� rook divs: the Rook divides both Kings either vertically or horizontally� rkK: the �les (or ranks) of the Rook, King and the opponent's King are either allin ascending or in descending order.� r edge: the Rook is on a border.During the learning phase, the system followed traces of games. For each movemade by the winning side, the system analyzed all the applicable patterns to constructplaying rules as described above. Each time a rule was constructed, all the constant9Although the decision of which patterns to keep before and after a move was taken directly from themanually constructed strategy, a clean criteria of when to keep invariant patterns (if any) needs to be properlyde�ned. This is left, however, for future research.

Learning Playing Strategies in Chess 15symbols were replaced by di�erent variables and the rules were sorted according totheir length. More speci�c (longer) rules were placed before more general ones.The author provided all the moves trying to simulate a very simple playing strat-egy where: (i) the opponent's King was never allowed to move more than one �le(rank) away from the Rook, and (ii) in all the starting positions the Rook was divid-ing both Kings either vertically or horizontally. After following a complete game, thesystem was switched into the testing phase, where the learned rules were tried fordi�erent initial con�gurations. Each time the strategy encountered a position whereno rule was applicable, the testing phase was stopped. That particular position wasgiven to the learning phase (where the user was asked for a \correct" move) and anew rule was learned for that position. The rule was generalized and sorted withthe rest of the rules. This process continued until several games were successfullyplayed without interruption. In all 20 rules were learned. (They are described in ap-pendix D.) Although the strategy is limited to certain starting positions, it is shownthat in principle a strategy for a simple endgame can be automatically learned fromtraces of games.The above learning strategy assumes that a complete set of patterns, to learn acorrect playing strategy, is provided in advance. An interesting research area is howto couple the above learning process with PAL, to learn \missing" patterns, whenan insu�cient set of them is given at the beginning. This may be why the learningstrategy without any invariant patterns was not successful. The main problem ishow to recognize whether the playing strategy is incorrect because the system hasnot seen a particular position or because it lacks a particular pattern. One indicationthat a pattern is missing, is when a learned rule produces a \wrong" answer on aparticular position. PAL can be used to learn a concept characterizing some aspectsof that position that could be included in the previous rule to produce \the right"answer. This, however, requires the user's intervention to recognize wrong answersand missing patterns; it is the subject of future research.As in the KPK example, a more fundamental question is which starting knowledgeshould PAL begin with to learn appropriate patterns. In (Morales 1994) the initialbackground knowledge is shown to a�ect the quality of the playing strategy. Forinstance, the concept of con�ned-area (the area where the opponent's King is con�nedby the Rook) can be used by PAL as starting knowledge to learn the concept ofsqueeze. With it, a more e�cient playing strategy based on squeezing the con�nedarea of the opponent's King can be constructed.6. RELATED WORKMany machine learning approaches have been used in chess. Quinlan used it asa test domain for his ID3 algorithm (Quinlan 1983). In a domain like chess, a wellde�ned set of attributes is hard to specify even for experts. Quinlan reports that twomonths of work were required to de�ne the 3{ply attributes for the King{Rook vs.King{Knight endgame (Quinlan 1983). Shapiro reports an estimated six weeks ofe�ort for the King{Pawn vs. King endgame (Shapiro 1987). A combination of PAL'slearning capabilities with an SBL algorithm could ease this process.Levinson and Snyder (1991) report a parameter{adjustment system, called Morph,which learns weighted patterns from traces of games. Morph is limited to patternswhich express attack/defense relations. For instance, it is unable to learn that twoRooks are on a diagonal or that a Rook is on an edge. All the patterns in Morph are

16 Computational Intelligenceconstructed from a �xed set of relations (links). Once a new pattern is learned, it can-not be used to construct other patterns (i.e., to be used as another link). Its graphrepresentation cannot express disjunctions or negations. This however is compen-sated for by a mechanism able to learn from traces of complete games. As in Morph,once a �xed set of patterns is chosen, a simple learning mechanism has been used tolearn a simple playing strategy without an evaluation function. It still needs to beassessed whether the same learning mechanism could be used on more complicatedgames.6.1. Explanation{Based Learning and ILPExplanation{Based Learning (EBL/G) (de Jong &Mooney 1986; Mitchell, Keller,& Kedar-Cabelli 1986) has also been used on chess (Flann & Dietterich 1989; Tade-palli 1989; Minton 1984). EBL/G uses as much knowledge as possible from thedomain. The basic idea of EBL/G is to start with a domain theory that de�nes atarget concept. The learning process repeatedly accepts a training example, appliesthe domain theory to prove that the example is an instance of the target concept, andthen extracts the weakest pre-conditions of that proof to form an e�cient \chunk"that provides an easy{to{evaluate, su�cient condition for the target concept.Perhaps the most successful EBL/G system for learning concepts in chess hasbeen Flann's WYL system (Flann & Dietterich 1989). It learned de�nitions forskewer, sliding{fork, knight{fork and general{king{fork, from a de�nition of check{with{bad{exchange. This illustrates the main di�erence between ILP and EBL/G.While in EBL/G all the learned concepts are specializations of a more generalgiven concept (i.e., all of the above concepts are specializations of check{with{bad{exchange), in ILP a new concept, not necessarily a specialization of a given concept,can be constructed from several background knowledge de�nitions. For instance, tolearn a restricted concept of fork, PAL used mainly instances of two concepts, legal{move and in check. The obvious objection to EBL/G is that it must start with astronger domain theory and at least a general (and very close) de�nition of the targetconcept de�nitions that we want to learn in the �rst place.6.2. Comparison with other ILP systemsTo achieve e�ciency some ILP systems, such as Golem (Muggleton & Feng 1990)or Foil (Quinlan 1990), have used a set of ground atoms as background knowledge.These systems, however, su�er from the problem of preparing such background facts.De�ning background facts for some concepts is a time consuming and di�cult process.It is sometimes unworkable for the systems to include all the background facts, evenif they are �nite, e.g., the concept of legal moves in chess. In such cases, appropriatesubsets need to be selected to maintain e�ciency, and that requires a prior knowledgeof the training example set. These systems cannot be used in an incremental way;once a concept is learned, ground facts need to be selected again if it is going to beused in the induction of a new concept.ILP systems that use non{ground clauses as background knowledge, such asITOU (Rouveirol 1992), Clint (de Raedt & Bruynooghe 1988; 1990), or RDT{MOBAL (Kietz & Wrobel 1992), have been applied to very limited domains, suchas family relations or the concept of \arch", despite the use of several constraints.CIA{Clint (de Raedt & Bruynooghe 1990) and RDT{MOBAL use schemas or mod-els to guide concept generation. Only clauses formed following a �xed set of models

Learning Playing Strategies in Chess 17can be learned. RDT{MOBAL and ITOU use a topology of concepts to restrict thehypothesis space. The introduction of pattern{based representation for backgroundknowledge and the introduction of a novel constraint based on labels have allowedPAL to learn concepts beyond the existing capabilities of current inductive systems.PAL, however, can only learn non{recursive concepts from background knowledgegiven as pattern{based de�nitions, and cannot include negation, unless a predicatename appears or disappears after a move.7. CONCLUSIONS AND FUTURE WORKPattern{based reasoning has been used by several computer systems to guidetheir reasoning strategies.10 For chess, in particular, a pattern{based approach hasbeen used with relative success in simple end{games (Bramer 1980; Bratko 1982; Hu-berman 1968) and tactically sharp middle games (Berliner 1977; Pitrat 1977; Wilkins1979). Our aim has been to learn chess patterns from simple example descriptionstogether with the rules of the game. To achieve this goal, we have used an InductiveLogic Programming (ILP) framework; it provides an adequate hypothesis languageand mechanism where patterns and examples can be simply expressed and, in princi-ple, learned in the presence of background knowledge. The limitations of current ILPsystems are more clearly exposed in domains like chess, where a large number of back-ground de�nitions can be required. Such systems typically have non{deterministicconcepts, some background knowledge de�nitions that can cover a very large numberof required facts, and concepts that can be several literals long. In PAL, examples aregiven as descriptions of states of a system (a description of the pieces in a chess board)and instances of patterns de�nitions are derived from such descriptions. Togetherthey are used to construct new pattern de�nitions, which can then be used in newexamples. In order to `recognize' instances of patterns from such state descriptions,we have introduced a pattern{based knowledge representation. This approach makesmore selective use of the background knowledge by considering only those de�nitionswhich apply to the current example description. The approach has been success-fully applied in chess, qualitative reasoning (Morales 1992), and music (Morales &Morales-Manzanares 1994), and it is expected to be useful in other domains wherea pattern{based approach can be applied. A novel mechanism for labeling the argu-ments of the components used in the state descriptions and the atoms derived fromthe background knowledge has been used for two main purposes. On one hand itguides and constrains the generalization process, as only compatible literals of thesame components are considered. On the other hand, it is used to construct the ex-ample space and guide the example generator by indicating which literals are a�ectedby which arguments.The patterns learned by PAL have been used in combination with other learningalgorithms to analyze positions and to construct a simple playing strategy. An SBLalgorithm, with the patterns learned by PAL as attributes, was used to learn a setof rules to decide whether a Pawn can safely promote in the KPK endgame withoutsearch. One of the bottlenecks in the construction of decision rules by SBL algorithmsis in the identi�cation of the right attributes. PAL's learning capabilities ease thisprocess in chess. A playing strategy can also be automatically constructed from10In particular, the conditions used in the production rules of most expert systems can be regarded asparticular patterns to match.

18 Computational Intelligencetraces of games using patterns learned by PAL and a simple learning algorithm.Our hypothesis is that the evaluation of patterns is su�cient for the construction ofplaying strategies without using an evaluation function. This however still needs tobe further tested.In addition to the integration of PAL with other learning mechanisms, there areseveral research areas which are being considered at the moment. In particular,we would like to extend the expressiveness of our pattern de�nitions to recursivepatterns. We have not dealt with noisy examples and/or incomplete knowledge. Al-though noise is not a real issue for chess, PAL has been used in other domains wherenoise can be an important factor. One important question is how much startingknowledge is required to learn a wide range of concepts. In the presence of noisein the training examples, the generated hypotheses must be allowed to accept somenegative examples and/or reject some positive examples. If both noise and incom-plete knowledge are considered, a balance between them must be established, torecognize whether the induced concept is incorrect due to noisy information or lackof knowledge. ACKNOWLEDGEMENTSThis research was partly supported by CONACyT (M�exico) and by ITESM{Campus Morelos. The author would like to thank Tim Niblett and Susan Epstein, inparticular, and the anonymous referees who provided helpful comments on an earlierdraft of this paper.ReferencesBerliner, H. J. 1977. A representation and some mechanisms for a problem-solvingchess program. In M. R. B. Clarke, editor, Advances in Computer Chess 1, EdinburghUniversity Press, Edinburgh, pages 7{29.Bramer, M. A. 1980. An Optimal Algorithm for king and pawn against king us-ing pattern-knowledge. In M. R. B. Clarke, editor, Advances in Computer Chess,Edinburgh University Press, Edinburgh, pages 82{96.Bratko, I. 1982. Knowledge-based problem-solving in AL3. In J. E. Hayes, D. Michie,& Y. H. Pao, editors, Machine intelligence 10, Ellis Horwood, U.K., pages 73{100.Buntine, W. 1988. Generalised subsumption and its applications to induction andredundancy. Arti�cial Intelligence 36(2):149{176.Charness, N. 1977. Human Chess Skill. In P. W. Fred, editor, Chess Skills in Manand Machine, Springer-Verlag, Berlin, pages 34{53.Clark, P. & Niblett, T. 1989. The CN2 induction algorithm. Machine Learning,3(4):261{283.

Learning Playing Strategies in Chess 19de Groot, A. D. 1965. Thought and Choice in Chess. Mouton, The Hague.de Jong, G. & Mooney, R. 1986. Explanation-based learning: an alternative view.Machine Learning, 1(2):145{176.de Raedt, L. & Bruynooghe, M. 1988. On interactive concept{learning and assimi-lation. In D. Sleeman & J. Richmond, editors, Proc. of the third european workingsession on learning (EWSL-88), Pitman, London, pages 167{176.de Raedt, L. & Bruynooghe, M. 1990. Indirect relevance and bias in inductiveconcept{learning. Knowledge Acquisition, 2:365{390.Flann, N. S. & Dietterich, T. G. 1989. A study of explanation-based methods forinductive learning. Machine learning, 4(2):187{226.Huberman, B. J. 1968. A program to play chess end games. CS-106, ComputerScience Department, Stanford University, Stanford, CA.Kietz, J. U. & Wrobel, S. 1992. Controlling the complexity of learning in logicthrough syntactic task{oriented models. In S. Muggleton, editor, Inductive LogicProgramming, Academic Press, London, pages 335{359.Levinson, R. & Snyder, R. 1991. Adaptive pattern-oriented chess. In Proceedings ofthe Ninth National Conference on Arti�cial Intelligence, AAAI Press - MIT Press,Boston, pages 601{606.Minton, S. 1984. Constraint{based generalization: learning game{playing plans fromsingle examples. In Proceedings of the National Conference on Arti�cial Intelligence,Kaufmann, Menlo Park, CA, pages 251{254.Mitchell, T. M., Keller, R. M., & Kedar-Cabelli, S. T. 1986. Explanation{basedgeneralization: a unifying view. Machine Learning, 1(1):47{80.Morales, E. 1991. Learning features by experimentation in chess. In Y. Kodrato�, ed-itor, Proceedings of the European Working Session on Learning, (EWSL-91), Springer-Verlag, Berlin, pages 494{511.Morales, E. 1992. First{order induction of patterns in chess. Ph.D. Thesis, TheTuring Institute - University of Strathclyde, Glasgow.Morales, E. 1994. Learning patterns for playing strategies. ICCA Journal, 17(1):15{26.Morales, E. & Morales-Manzanares, R. 1994. Learning counterpoint rules for analysisand generation. In Proceedings of the First Brasilian Symposium on Computer Music,(to appear).

20 Computational IntelligenceMuggleton, S. 1991. Inductive Logic Programming. New Generation Computing8:295{318.Muggleton, S. & Feng, C. 1990. E�cient induction of logic programs. In S. Arikaxa,S. Soto, S. Ohsuya, and T. Yokomari, editors, Proceedings of First InternationalWorkshop on Algorithmic Learning Theory (ALT90), Ohmsha, Tokyo, Japan, pages368{381.Niblett, T. 1988. A study of generalisation in logic programs. In D. Sleeman andJ. Richmond, editors, Proceedings of the Third European Working Session on Learn-ing (EWSL-88), Pitman, London, pages 131{138.Pitrat, J. 1977. A chess combination program which uses plans. Arti�cial Intelli-gence, 8:275{321.Plotkin, G. D. 1969. A note on inductive generalisation. In B. Meltzer & D. Michie,editors, Machine Intelligence 5, Edinburgh University Press, Edinburgh, pages 153{163.Plotkin, G. D. 1971a. A further note on inductive generalisation. In B. Meltzer &D. Michie, editors, Machine Intelligence 6, Edinburgh University Press, Edinburgh,pages 101{124.Plotkin, G. D. 1971b. Automatic methods of inductive inference. Ph.D. Thesis,Edinburgh University, Edinburgh.Quinlan, J. R. 1983. Learning e�cient classi�cation procedures and their applicationto chess end games. In R. S. Michalski, J. G. Carbonell, & T. M. Mitchell, editors,Machine Learning: an arti�cial intelligence approach, Tioga, Palo Alto, CA, pages463{482.Quinlan, J. R. 1990. Learning logical de�nitions from relations. Machine learning,5(3):239{266.Rouveirol, C. 1992. Extensions of inversion of resolution applied to theory com-pletion. In S. Muggleton, editor, Inductive Logic Programming, Academic Press,London, pages 63{92.Shapiro, A. D. 1987. Structured induction in expert systems, Turing Institute Pressin association with Addison-Wesley, Wokingham.Tadepalli, P. 1989. Planning in games using approximately learned macros. InB. Spatz, editor, Proceedings of the Sixth International Workshop on Machine Learn-ing, Morgan Kaufmann, San Mateo, CA, pages 221{223.Wilkins, D. 1979. Using patterns and plans to solve problems and control search.Technical ReportMEMO-AIM-329; STAN-CS-79-747, Stanford University, Arti�cial

Learning Playing Strategies in Chess 21Intelligence Laboratory, Stanford, CA.A. PATTERNS LEARNED BY PAL FOR THE KPK ENDGAMEThis appendix has some of the concepts learned by PAL for the KPK endgame.For presentation purpose, the following notation has been adopted: bl = black, wh= white, kg = king, pn = pawn, (X,Y) = square(X,Y)can run1/10: The distance to the queening square of the Pawn is less than thedistance of the opponent's King.can run1(wh,kg,(A,B),wh,pn,(C,D),bl,kg,(E,F),Pos) contents(bl,kg,(E,F),Pos),contents(wh,kg,(A,B),Pos),contents(wh,pn,(C,D),Pos),dist qs((E,F),H,Pos),dist qs((C,D),G,Pos),less than(G,H).distKqs1/10: the distance of the King to the queening square is one.distKqs1(wh,pn,(A,B),wh,kg,(C,D),bl,kg,(E,F),Pos) contents(wh,pn,(A,B),Pos),contents(wh,kg,(C,D),Pos),contents(bl,kg,(E,F),Pos),dist qs((C,D),1,Pos).closerk/10: the King is closer to its Pawn than the opponent's King.closerk(wh,pn,(A,B),wh,kg,(C,D),bl,kg,(E,F),Pos) contents(bl,kg,(E,F),Pos),contents(wh,kg,(C,D),Pos),contents(wh,pn,(A,B),Pos),distance((C,D),(A,B),H,Pos),distance((A,B),(E,F),G,Pos),less than(G,H).ks in line/7: both Kings are on the same rank.ks in line(wh,kg,(A,B),bl,kg,(C,B),Pos) contents(bl,kg,(C,B),Pos),contents(wh,kg,(A,B),Pos).B. INDUCED RULES FROM CN2This appendix has the (unordered) rules induced by CN2 for the class can run(with a Laplacian error estimate, Threshold = 0 and Star = 5).IF in front = F IF k7 divs = TAND can run1 = T AND sp same dist = F

22 Computational IntelligenceTHEN class = can run [28839 0] THEN class = can run [998 0]IF in front = F IF king divs = TAND distpqsk = T AND equal dist = TAND distKqs1 = T AND distKqs2 = TAND p divs ks8 = F AND closerk = FTHEN class = can run [412 0] THEN class = can run [162 0]IF can run4 = T IF opposition = TTHEN class = can run [881 0] AND can run6 = TTHEN class = can run [45 0]IF pawnY3 = FAND k7 divs = T IF pawnY3 = FTHEN class = can run [1032 0] AND in front = FAND kingY8 = FIF king divs = T AND distKqs1 = TAND can run6 = T AND closerk = FTHEN class = can run [118 0] THEN class = can run [906 0]IF king divs = T IF distKqs1 = TAND distpqsk = T AND can run61 = TAND opposition = T THEN class = can run [33 0]THEN class = can run [57 0] IF py = y5IF py = y5 AND distKqs1 = TAND king divs = T AND p divs ks8 = TAND distKqs1 = T THEN class = can run [7 0]THEN class = can run [205 0] IF ky = y5IF opposition = T AND king divs = TAND can run61 = T AND equal dist = TTHEN class = can run [21 0] AND alm opp = TTHEN class = can run [35 0]IF okx = x8AND in front = F IF px = x3AND distKqs1 = T AND okx = x7THEN class = can run [377 0] AND in front = FAND distKqs1 = TIF oky = y7 THEN class = can run [106 0]AND king divs = TAND distKqs1 = T IF py = y4AND pawnY4 = T AND kx = x5THEN class = can run [20 0] AND okx = x1AND distKqs1 = TIF corner patt = T THEN class = can run [11 0]THEN class = can run [3 0] IF in front = FIF py = y4 AND can run21 = TAND kx = x3 THEN class = can run [249 0]AND p divs ks8 = T

Learning Playing Strategies in Chess 23AND distkqs3 = TTHEN class = can run [1 0]C. KRK CONCEPTSThis appendix has some of the concepts learned by PAL for the playing strategy.The reader is referred for further details to (Morales 1992). For presentation purposes,the following notation has been adopted: kg = king, rk = rook, (X,Y ) = square(X,Y ).ThreatkR/7: The opposite King threatens our RookthreatkR(S1,kg,(X1,Y1),S2,rk,(X2,Y2),Pos1) contents(S2,rk,(X2,Y2),Pos1),contents(S1,kg,(X1,Y1),Pos1),other side(S2,S1),sliding piece(rk,(X1,Y1),Pos1),make move(S1,kg,(X1,Y1),(X2,Y2),Pos1,Pos2),not contents(S2,rk,(X2,Y2),Pos2).Opposition/8: The Kings are in front of each other with one square between them.The de�nitions for distkR/8 and in line/7 are similar.opposition(S1,kg,(X1,X2),S2,kg,(X1,Y2),Pos) contents(S2,kg,(X1,Y2),Pos),contents(S1,kg,(X1,Y1),Pos),other side(S1,S2),distance((X1,Y1),(X1,Y2),2,Pos),distance((X1,Y2),(X1,Y1),2,Pos).opposition(S1,kg,(X1,Y1),S2,kg,(X2,Y1),Pos) contents(S1,kg,(X1,Y1),Pos),contents(S2,kg,(X2,Y1),Pos),other side(S2,S1),distance((X2,Y1),(X1,Y1),2,Pos),distance((X1,Y1),(X2,Y1),2,Pos).Alm opp/7: King is \almost" in opposition with the opposite King.alm opp(S1,kg,(X1,Y1),S2,kg,(X2,Y2),Pos) contents(S2,kg,(X2,Y2),Pos),contents(S1,kg,(X1,Y1),Pos),other side(S1,S2),distance((X1,Y1), (X2,Y2),2,Pos),distance((X2,Y2), (X1,Y1),2,Pos),manh dist((X1,Y1),(X2,Y2),3,Pos),manh dist((X2,Y2),(X1,Y1),3,Pos).L patt/10: The 3 pieces form an L-shaped pattern with the opposite King in checkby the Rook.l patt(S1,kg,(X1,Y1),S2,kg,(X2,Y1),S2,rk,(X1,Y3),Pos) contents(S2,kg,(X2,Y1),Pos),

24 Computational Intelligencecontents(S2,rk,(X1,Y3),Pos),contents(S1,kg,(X1,Y1),Pos),other side(S1,S2),sliding piece(rk,(X1,Y3),Pos),in check(S1,(X1,Y1),rk,(X1,Y3),Pos),legal move(S2,rk,(X1,Y3),(X2,Y3),Pos),opposition(S1,kg,(X1,Y1),S2,kg,(X2,Y1),2,Pos),opposition(S2,kg,(X2,Y1),S1,kg,(X1,Y1),2,Pos).l patt(S1,kg,(X1,Y1),S2,kg,(X1,Y2),S2,rk,(X3,Y1),Pos) contents(S1,kg,(X1,Y1),Pos),contents(S2,kg,(X1,Y2),Pos),contents(S2,rk,(X3,Y1),Pos),other side(S2,S1),sliding piece(rk,(X3,Y1),Pos),in check(S1,(X1,Y1),rk,(X3,Y1),Pos),legal move(S2,rk,(X3,Y1),(X3,Y2),Pos),opposition(S2,kg,(X1,Y2),S1,kg,(X1,Y1),2,Pos),opposition(S1,kg,(X1,Y1),S2,kg,(X1,Y2),2,Pos).mv closerKk/7: The distance from the King to the opposite King decreases after amove. Similar de�nitions are given formv closer/away and squeeze (with area insteadof distance).mv closerKk(S1,kg,(X1,Y1),S2,kg,(X2,Y2),Pos1) contents(S1,kg,(X1,Y1),Pos1),contents(S2,kg,(X2,Y2),Pos1),other side(S1,S2),distance((X2,Y2),(X1,Y1),D1,Pos1),make move(S1,kg,(X1,Y1),(X3,Y3),Pos1,Pos2),distance((X2,Y2),(X3,Y3),D2,Pos2),less than(D2,D1).D. INDUCED RULES FOR THE KRK ENDGAMEThis appendix has some of the rules induced for the KRK endgame. The lengthof the rule and a short description of its purpose is given before each rule. The rule'snumber is the order in which the rule was generated. For presentation purpose, therules only include the name of the patterns involved.(17) Check{mater12 squeeze, k trap, mvKcloserR, rook divs, not l patt, opposition, not r edge, notcheck mate, kings same side,make move(Rook),not squeeze, not k trap, not mvKcloserR, not rook divs, r edge, l patt, check mate.(16) The Rook is threatened and its King can be moved to protect itr8 closerkKR, not distKR1, mvKcloserk, mvKcloserR, rook divs, not opposition,threatkR, kings same side,

Learning Playing Strategies in Chess 25make move(King),not closerkKR, distKR1, not mvKcloserk, not mvKcloserR, rook divs, opposition,not threatkR.(15) A wait move by the Rook since both Kings are almost in oppositionr17 squeeze, closerkKR, not closerKkR, mvKcloserR, rook divs, r edge, not rkK,kings same side, almost opposition,make move(Rook),not closerkKR, closerKkR, rook divs, r edge, rkK.(11) Force the opponent's King backwards when both Kings are in oppositionr3 squeeze, mvKcloserR, opposition, kings same side, rook divs, not l patt,make move(Rook),not squeeze, not mvKcloserR, not rook divs, l patt.(11) Both Kings are in opposition but too close to the Rook to force a backward move,so the Rook is moved to a borderr16 squeeze, distKR1, not mvKcloserR, rook divs, kings same side, not r edge,make move(Rook),not distKR1, mvKcloserR, rook divs, r edge.(10) Moves the King to be almost in opposition with the opponent's Kingr4 closerkKR, mvKcloserk, mvKcloserR, rook divs, kings same side, not almost {opposition,make move(King),not mvKcloserk, rook divs, almost opposition.(10) Moves the Rook away from the borderr11 squeeze, distKR1, not mvKcloserR, rook divs, r edge, kings same side,make move(Rook),not distKR1, mvKcloserR, rook divs.(10) The King is moved closer to the opponent's Kingr15 closerKkR, mvKcloserk, mvKcloserR, rook divs, kings same side, rkK,make move(King),not mvKcloserk, rook divs, rkK.(9) Force the opponent's King to be moved \backwards"r1 squeeze, closerkKR, opposition, not l patt, kings same side,make move(Rook),not squeeze, not closerkKR, l patt.(8) Moves the King to be almost in opposition with the opponent's Kingr5 closerkKR, mvKcloserR, rook divs, not almost opposition, kings same side,make move(King),rook divs, almost opposition.