Opmaker2: Efficient Action Schema Acquisition T.L.McCluskey, S.N.Cresswell, N. E. Richardson and M.M.West The University of Huddersfield,UK Email [email protected]@hud.ac.uk.

Opmaker2: Efficient Action Schema Acquisition

T.L.McCluskey, S.N.Cresswell, N. E. Richardson and M.M.West

The University of Huddersfield,UK

Email [email protected]

mailto:[email protected]

Knowledge Engineering and Intelligent Interfaces

Motivation – Intelligent Agents that can Plan?

The availability of knowledge via ontologies is growing it is not unreasonable that an agent will be able to acquire and refine

such knowledge with some degree of autonomy. A necessary precondition of the use of (current) automated planning

technology is that there exists detailed knowledge: action specifications and heuristics

the amount of effort needed to encode bug free, accurate action specifications and planning heuristics, and to maintain them, is significant.

for every agent that can perform planning, must we hand code and hand maintain its action descriptions and heuristics? No, if agents are to achieve this kind of

autonomy, then they should be capable of learning and refining action knowledge and heuristics.


Vision

Agent

ApplicationOntologies

PlanEngines

Learningphase

Agent capable of deliberative planning


Planning and Machine Learning – moved on?

Eg my work in 1987 (IJCAI-87, IWML-87) deriving planning heuristics (macros, operator choice, goal-orders) offline

YES – must derive heuristics + dynamic domain model


General Idea of Opmaker2 A knowledge base for planning can be divided into 3:- the easy (static) part of a domain model- The (hard) dynamic part- Planning heuristics

The idea of Opmaker is to learn the (hard) dynamic part and the planning heuristics from “training sequences” within the context of an existing (static) part of a domain model


Learning Problem - INPUT Input:

(a) knowledge of objects, and collections of similar objects making up distinct classes, possible states of a typical object of each class, and state invariants

(b) Knowledge of existing plans that other agents, or a trainer, has used [in terms of verbs and affected objects] with initial state and goal state for affected objects

the agent does not have an explicit specification of actions in such a way that it can reason about their synthesis (or the agent does have such a specification but needs to refine, maintain or evolve it).


Example of (b)Example training sequence from “Extended Tyre Domain”1. name: do_up; unchanging: wrench0,jack0, trim1; changing: hub1,nuts1

2. name: jack_down; unchanging:: nilchanging: hub1,jack0

3. name: tighten; unchanging: wrench0,hub1,trim1; changing: nuts1

4. name: apply_trim; unchanging: hub1; changing: trim1,wheel5


Learning Problem - OUTPUT

1. A full parameterised specification of actions which can be used to do planning

2. Planning heuristics.

3. Refinement of any existing parameterised specification of actions, and heuristics, that it currently holds.

Results:

Opmaker2 outputs 1, plus 2 in the form of HTN methods. Trying to avoid 3.


Opmaker vs Opmaker2

Opmaker2INPUT:Partial (static) Domain TheoryTraining sequences (.. extra Invariants)

OUTPUT:Primitive Operator SchemaHTN Schema

Opmaker (in GIPO)INPUT:Partial (static) Domain TheoryTraining sequencesIntermediate State Details

OUTPUT:Primitive Operator Schema

But hasn’t this been done before – by Opmaker(1) (2002) for example?

Painful part!


Method Overview

(1).use a set of heuristics and inferences to track the changing states (‘object trace’) of each changing object referred to within a training example; produce a set of candidate operators

(2). use domain invariants to reduce the set of candidate operators (to a singleton)

(3). use the techniques of the original Opmaker algorithm to generalise object references and create parameterised operator schema from the specific object transitions extracted in (i) from the training examples.


Method Overview for (1)

Track an object O in a

training sequence

Id1,Id2,…,Idk,…, Idn:

Where Idk includes the

last mention of

changing O

O.i

O.f

Method seeks to fill in state transitiondetails..

KNOWN

UNKNOWN

Id1

Idk


Method Overview for (1)

Track an object O in a

training sequence

Id1,Id2,…,Idk,…, Idn:

Where Idk includes the

last mention of

changing O

O.i

O.f

..by finding theObject’s trace includingany associations itmakes

Idk

Id1


Method Overview for (1) Considering a changing object O at training action with id

‘Idx’ .. If Idx is not the last change of O then we potentially have a choice of target state for O …

O current state = O.c Idx

Same stateState that has parameter of object type NOT in Idx

Potential new states


Back to Example id1: do_up; unchanging: wrench0,jack0, trim1; changing: hub1,nuts1 id2: jack_down; unchanging:: nilchanging: hub1,jack0 id3: tighten; unchanging: wrench0,hub1,trim1; changing: nuts1 Id4: apply_trim; unchanging: hub1; changing: trim1,wheel5


Method Overview for (1) - example Considering a changing object hub1 at training action ‘do_up’ ..

If do_up is not the last change of hub1 then we potentially have a choice of target state for hub1 …

hub1 current state = unfastened(hub1), jacked_up(hub1,jack0)

do_up

Same state

Potential new states (see PAPER for definitions)

S1 S2 S3 S4


Method Overview – result of step (1) Form sets of candidate operators for each Id, of form

- Id - x- Set of parameters- Set of prevail conditions (unchanging objects)- Set of object transitions


Method Overview – step (2)

use domain invariants to reduce the set of candidate operators (to a singleton)

Eliminate a candidate that is logically consistent with domain invariants


Method Overview – step (3)

use the techniques of the original Opmaker algorithm to generalise object references and create parameterised operator schema from the specific object transitions extracted in (i) from the training examples.

Generalise those left using OpMaker to create operator schema


Producing Heuristics- Our earlier paper (2002) on Opmaker indicated how the

training sequences can be made into ‘canned plans’ and used as components in planning using the Hybrid planner HyHTN

- Even earlier work (1997) showed how invariants are key to producing goal ordering heuristics

- In Opmaker2 we have implemented the former method and the canned plans are produced with the primitive operator schema


Experiments 3 Domains considered : Extended Tyre Domain, Hiking Domain, Blocks DomainEnough training sequences to include instances of all required

operator schemaExperimental Criteria:- A. ‘How many’ axioms have to be added to the static

domain model to make sure the operator set is unique?- B. Are the operator sets induced capable of being used in

solving planning problems?- C. With the canned plans introduced, is plan generation

time reduced?- D. Does the performance of the learning program vary a

great deal with different training sequences


Experimental Results

A. ‘How many’ axioms have to be added to the static domain model to make sure the operator set is unique?

- Tyre Domain - 8, Hiking Domain - 0, Blocks Domain - 4B. Are the operator sets induced capable of being used in solving planning

problems? Yes, in all 3 casesC. With the canned plans introduced, is plan generation time reduced?- Tyre Domain, Hiking Domain, Blocks Domain Yes, in all 3 cases (no graphs to show as yet..)D. Still much to do BUT LONG training sequences do not seem to cause a

problem – we tried a 22 element solution in the Blocks World and it produced a single set of operator schema

BUT result with blocks domain threw up some problems because of the multiple object references of the same class within one operator schema


Conclusions and Future Work Essentially, we have traded off effort to produce a detailed

“static” domain model vs effort to produce operator schema modelling actions because

(a) Static knowledge is more easily available

(b) Our experience of operator building in GIPO II showed that creating the parameterised specs to be the hardest part of domain encoding

Open Question for future work:

How to induce operator schema which require constraints mot appearing in the input object state definitions

How to guarantee that a set of invariants is strong enough to deliver a working set of operator schema?

Opmaker2: Efficient Action Schema Acquisition T.L.McCluskey, S.N.Cresswell, N. E. Richardson and M.M.West The University of Huddersfield,UK Email [email protected]@hud.ac.uk.

Documents

knowledge engineering

knowledge of objects

detailed knowledge

knowledge base

availability of knowledge

heuristics n

refining action knowledge

wheel5 slide