Environments and Sensing

Koen Hindriks Multi-Agent Systems

Environments and Sensing

Multi-Agent Systems Course

Koen HindriksDelft University of Technology, The Netherlands

Koen Hindriks Multi-Agent Systems 2011 2 Koen Hindriks Multi-Agent Systems 2

Outline for Today

Previous lecture: core GOAL language:

•Knowledge, beliefs, goals, actions, rules

Today:

•Agent Program Structure

•Macros

•Environments, sensing, and acting

•BW4T & Agent design guidelines

Koen Hindriks Multi-Agent Systems 2011 3 Koen Hindriks Multi-Agent Systems

AGENT PROGRAM STRUCTURE


An Agent is a Set of ModulesBuilt-in modules:• init module:

– Define global knowledge– Define initial beliefs & goals– Process “send once” percepts– Specify environment actions

• main module– Action selection strategy

• event module– Process percepts– Process messages– Goal management

User-defined modules.

init module{ knowledge{ … } beliefs{ %%% INITIAL BELIEFS ONLY IN INIT MODULE %%% } goals{ … } program{ %%% PROCESS “SEND ONCE” PERCEPTS HERE %%% } actionspec{ %%% SPECIFY ENVIRONMENT ACTIONS HERE %%% }} main module{ % OPTIONAL knowledge section % NO beliefs section HERE! % OPTIONAL goal section (not advised in ‘main’) program{ %%% ENVIRONMENT ACTION SELECTION HERE %%% }}

event module{ program{ %%% PROCESS PERCEPTS HERE %%% %%% PROCESS MESSAGES HERE %%% %%% PERFORM GOAL MANAGEMENT HERE %%% }}


GOAL Agent Cycle

Process events(= apply event rules)

Select action(= apply action rules)

Perform action(= send to environment)

Updatemental state

(= apply action specs + commitment strategy)

Also called reasoning or deliberation cycle.

GOAL’s cycle is a classic sense-plan-act cycle.

event module

main module


MACROS


Macro Definitions

instead of writing mental state conditions:

you may use macros in rules:

mental state condition is macro’s definition

program{ if bel(tower([Y|T])), a-goal(tower([X,Y|T])) then move(X,Y). if a-goal(tower([X|T])) then move(X,table).}

program{ #define misplaced(X) a-goal(tower([X|T])). #define constructiveMove(X,Y)

a-goal(tower([X,Y|T])), bel(tower([Y|T])).

if constructiveMove(X,Y) then move(X,Y). if misplaced(X) then move(X,table).}


Macro Definitions• Use to increase readability• Macro definitions must be placed inside the

program section, before any of the rules

• All variables in macro should occur in defining mental state condition.

• Other vars in condition are just placeholders.

program{ #define misplaced(X) a-goal(tower([X|T])). #define constructiveMove(X,Y)

a-goal(tower([X,Y|T])), bel(tower([Y|T])).

if constructiveMove(X,Y) then move(X,Y). if misplaced(X) then move(X,table).}


Macro Definitions

1.When is a block marked as obstructing? Informally explain the macro definition.

2.In which cases is the concept of an obstructing block useful?

knowledge{ … above(X,Y) :- on(X,Y). above(X,Y) :- on(X,Z), above(Z,Y).

tower([X]) :- on(X,table). tower([X,Y|T]) :- on(X,Y), tower([Y|T]).}

program{ #define inPosition(X) goal-a( tower([X|T]) ). #define obstructingBlock(X) a-goal( on(Y,Z) ), bel( above(X,Z); above(X,Y) ). …}

EXERCISE


Outline

• Environments and Perception

• Built-in Actions and Selecting Multiple Actions

• Environments and Durative Actions


This Lecture

Percepts

Action

events

actions goals

plans

beliefs

environment

agent


DEMO: TOWERWORLD


TowerWorld Environment

• The TowerWorld is:– Multi-agent:

user can also change the world– Fully observable:

the agent can see “everything”– Dynamic:

environment can change if agent does nothing

We need sensing / percepts.


SENSING AND PERCEPTION


Sensing

• Agents need sensors to:– explore the environment when they have

incomplete information (e.g. Wumpus World)– keep track of changes in the environment that are

not caused by itself

• GOAL agents sense the environment through a perceptual interface defined between the agent and the environment.– Environment generates perceptions


Sensing in the Tower World

Percepts are received by an agent in it’s percept base.

The reserved keyword percept is wrapped around the percept content, e.g. percept(block(a)).

Not automatically inserted into beliefs!


Inspecting the Percept BasePercept base is inspected using the bel operator, e.g.

bel( percept(on(X,Y)) ).

Often useful to check for difference with current beliefs:

1. Agent does not believe something it perceives:

bel( percept(on(X,Y)), not(on(X,Y)) )

2. Agent believes something it does not perceive:

bel( on(X,Y), not(percept(on(X,Y))) )

Why not: bel( not(percept(on(X,Y))), on(X,Y) )?

Evaluate with only: percept(on(a,b)) and on(a,c)

EXERCISE


Processing Percepts

• The percept base is refreshed, i.e. emptied and then filled again, every cycle of the agent.

• Agent has to decide what to do when it perceives something, i.e. receives a percept.

• Use percepts to update agent’s mental state:– Ignore the percept– Update the beliefs of the agent– Adopt a new goal


Built-in ActionsGOAL provides a number of built-in actions:• insert(<conjunction>)

example: insert(on(X,Y), not(on(X,Z)))

meaning: add on(X,Y) and delete on(X,Z)

• delete(<conjunction>)

example: delete(on(X,Y), not(on(X,Z)))

meaning: delete on(X,Y) and add on(X,Z)

• adopt(<conjunction of positive literals>)

example: adopt(on(a,b), on(b,c))

meaning: add new goal (if not already implied by a goal nor believed)

• drop(<conjunction>)

example: drop(on(b,a), not(on(c,table)))

meaning: remove all goals that imply <conjunction> from the goal base

NoteAll actions need to

be closed when they are executed.


Built-in Action InsertExample 1:

Example 2:

•Belief base affected, sometimes also goal base affected.•Example 2 preferred: Use insert only for inserting.

beliefs{ on(a,table). on(b,table). on(c,table).}goals{ on(a,b), on(b,table).}

beliefs{ on(a,b). on(b,table). on(c,table).}goals{ on(a,b), on(b,table).}

insert(on(a,b), not(on(a,table)))


beliefs{ on(a,table). on(b,table). on(c,table). on(d,table).}goals{ on(a,b), on(b,table).}

insert(on(d,table))


Built-in Action DeleteExample 1:

Example 2:

Only belief base affected, if used as in Example 2 (preferred). In that case, goal base always remains unchanged

beliefs{ on(a,b). on(b,table). on(c,table).}goals{ on(a,b), on(b,table).}


delete(on(a,b), not(on(a,table)))


beliefs{ on(b,table). on(c,table).}goals{ on(a,b), on(b,table).}

delete(on(a,table))


Multiple Actions in Rule• Using the + operator multiple actions can be combined

and performed in a single rule.

• The actions are executed in same reasoning cycle• Combined actions are executed in order (left-to-right)

• Example: rule to update moving state of BW4T robot.

Why not: insert( state(NewState) ) + delete( state(OldState) )?

If OldState = NewState information about state would be deleted!

program{ if bel(state(OldState), percept(state(NewState)) ) then delete( state(OldState) ) + insert( state(NewState) ). …..}

if <mental_state_condition> then <action1> + <action2> + … .

EXERCISE


Built-in Action AdoptExample 1:

Example 2(a) and (b):

•Only goal base affected, belief base remains unchanged.


beliefs{ on(a,table). on(b,table). on(c,table).}goals{ on(a,b), on(b,table). on(c,a).}

adopt(on(c,a))


does not execute

adopt(on(a,table))

adopt(on(a,b))

does not execute


Built-in Action DropExample 1:

Example 2:

•Only goal base affected, belief base remains unchanged.•Like insert and delete actions always executes.


beliefs{ on(a,table). on(b,table). on(c,table).}goals{ }

drop(on(a,b), on(b,table)))


beliefs{ on(a,table). on(b,table). on(c,table).}goals{ }

drop(on(a,b))


Drop Action• drop(on(b,a), not(on(c,table)))

• Is goal in goal base

dropped?

• Check: does goal

imply on(b,a), not(on(c,table)) ?

• A: Yes, so goal is removed by drop action.

knowledge{ block(X) :- on(X, Y). clear(X) :- block(X), not(on(Y,X)). clear(table). tower([X]) :- on(X, table). tower([X,Y|T]):- on(X,Y),tower([Y|T]).}goals{ on(a,table), on(b,a), on(c,b), on(d,table), on(e,table), on(f,e), on(g,f), on(h,g), on(i,h).}

EXERCISE


Changing Agent’s Mind: Goals

Changing One’s Mind Pattern:

• If a “better” goal G2 can be set, then replace a current goal G1 with that new goal G2.

• Example in Tower World Agent:program{ … if goal( holding(X) ), bel( not(clear(X) ), constructiveMove(Z,Y)

then drop( holding(X) ) + adopt( holding(Z) ). …}

program{ … if goal( G1 ), <INSERT_CONDITION_BETTER( G2 )>

then drop( G1 ) + adopt( G2 ). …}


Updating Agent’s Mental StateOne way to update beliefs with percepts:

• First, delete everything agent believes.

Example: remove all block and on facts.

• Second, insert new information about current state provided from percepts into belief base.

Example: insert block and on facts for every percept(block(…)) and percept(on(…)).

Assumes that environment is fully observable with respect to block and on facts.

Downside: not very efficient…


Percept Update PatternA typical pattern for updating is:Rule 1

If the agent

– perceives block X is on top of block Y, and– does not believe that X is on top of Y

Then insert on(X,Y)into the belief base.

Rule 2If the agent– believes that X is on top of Y, and– does not perceive block X is on top of block Y

Then remove on(X,Y)from the belief base.

Assumes full observability with respect to on facts.


Percepts and Event Module

• Percepts are processed in GOAL by means of event rules, i.e. rules in the event module.

• Event module is executed every time that agent receives new percepts.

event module{ program{ <… rules …> }}


Implementing Pattern Rule 1

Rule 1

If the agent

– perceives block X is on top of block Y, and

– does not believe that X is on top of Y

Then insert on(X,Y)into the belief base.

event module { program{ % assumes full observability. if bel( percept(on(X,Y)), not(on(X,Y)) ) then insert(on(X,Y)). … }}

INCORRECT!


Implementing Pattern Rule 1Rule 1

If the agent perceives block X is on top of block Y, and does not believe that X is on top of Y, then insert on(X,Y)into the belief base.

We want to apply

this rule for all

percept instances

that match it!

event module { program{ % assumes full observability. forall bel(percept(on(X,Y)), not(on(X,Y))) do insert(on(X,Y)). … }}

Content Percept Base percept(on(a,table))percept(on(b,table))percept(on(c,table))percept(on(d,table))…

insert(on(a,table))insert(on(b,table))insert(on(c,table))insert(on(d,table))…

Assuming emptybelief base


Implementing Pattern Rule 2Rule 2

If the agent

– believes that X is on top of Y, and

– does not perceive block X is on top of block Y

Then remove on(X,Y)from the belief base.

1. We want that all rules are applied!

By default the event module applies ALL rules in linear order.

2. Note that none of these rules fire if nothing changed.

event module { program{ % assumes full observability. forall bel(percept(on(X,Y)), not(on(X,Y))) do insert(on(X,Y)). forall bel(on(X,Y), not(percept(on(X,Y)))) do delete(on(X,Y)). }}


Event Module for Tower World

Rules for every type of percept, e.g., for Tower World:• blocks present;• whether gripper is holding a block or not;• a block being on top of another or on the table.

event module { program{ % assumes full observability. forall bel( block(X), not(percept(block(X))) ) do delete( block(X) ). forall bel( percept(block(X)), not(block(X)) ) do insert( block(X) ).

forall bel( holding(X), not(percept(holding(X))) ) do delete( holding(X) ). forall bel( percept(holding(X)), not(holding(X)) ) do insert( holding(X) ).

forall bel( on(X,Y), not(percept(on(X,Y))) ) do delete( on(X,Y) ). forall bel( percept(on(X,Y)), not(on(X,Y)) ) do insert( on(X,Y) ). }}


Initially… Agent Knows Nothing

• In most environments an agent initially has no information about the state of the environment, e.g. Tower World, Wumpus World, …

• Represented by an empty belief base:

• There is no need to include a belief base in this case in a GOAL agent.

• It is ok to simply have no belief base section.

beliefs{}


Processing Percepts

Types of percepts:

• “send always”

• “send once”, e.g., place(<PlaceID>)

• “send on change”, e.g., at(<PlaceID>)

• “send on change with negation”, e.g., in(<RoomID>)

How to handle these different type of percepts?

See previous slides


Pattern for Send Once

“Send once” percepts are sent only once when the agent first connects to an environment.

Use the init module to insert the beliefs you want to use in the agent’s belief base.

init module { … program{ forall bel( percept(place(X) ) do insert( place(X) ). … }}


Send on Change Percept

• “send on change” percepts are sent once when a change occurs:

• Note that pattern for “send always” percepts does not work for “send on change” due to second rule in that pattern.

event module { % send always pattern program{ … forall bel(on(X,Y), not(percept(on(X,Y)))) do delete(on(X,Y)). }}

State 1 State 2 State 3 State 4 State 5

state(traveling) state(arrived)-


Pattern for Send on Change“Send on change” percepts are sent only when

something in the environment has changed.

Rule: remove old belief and insert the new percept.

Put rule in event module.

event module { … program{ forall bel( state(Old), percept(state(New) )

do delete(state(Old)) + insert(state(New)). … }}

Could also have used ‘if’ here:At most one state percept received.


Send on Change with Negation“Send on change with negation” percepts:• Positive fact sent once when it becomes true in environment • Negative fact sent once when it becomes false in environment

Rule: insert positive and remove negative facts.

Put rules in event module.

event module { … program{ forall bel(percept(in(RoomID)) do insert(in(RoomID)). forall bel(percept(not(in(RoomID))) do delete(in(RoomID)). ... }}


Combining InformationIn BW4T it is most important to remember in which

place a colored block can be found.

Idea: combine at location with ‘color’ percept using a new ‘block’ predicate.

NB: make sure by earlier rule that at belief is correct!

event module { … program{ … forall bel( percept(color(BlockID, ColorID)), at(PlaceID) )

do insert( block(BlockID, ColorID, PlaceID) ).

forall bel( at(PlaceID), block(BlockID, ColorID, PlaceID), percept(not(color(BlockID, ColorID))) )

do delete( block(BlockID, ColorID, PlaceID) ). … }}


Summarizing• Two types of rules:

– if <cond> then <action>.

is applied at most once (if multiple instances chooses randomly)– forall <cond> do <action>.

is applied once for each instantiation of parameters that satisfy condition.

• Main module by default:– checks rules in linear order– applies first applicable rule (also checks action precondition!)

• Event module by default:– Checks rules in linear order– Applies all applicable rules (rules may enable/disable each other!)

• Program section modifiers: [order=random], [order=linear], [order=linearall], [order=randomall]

• Built-in actions: insert, delete, adopt, drop.


ACTIONS IN ENVIRONMENTS


Instantaneous versus Durative

• Instantaneous actions

Actions in the Blocks World environment are instantaneous, i.e. they do not take time.

Wumpus World actions are of this type as well.

• Durative actions

Actions in the Tower World environment take time. When a GOAL agent sends an action to such an environment, the action will not be completed immediately.


Durative ActionsDeciding on a next action typically gets more complicated when actions take time! Agents may decide to perform an action before finishing another.

Three cases:

1.Sending an action B while another action A is still ongoing overrides and terminates previous action A…

Example: actions in Tower World, goto action in UT2004.

2.Action B that is sent while another action A is still ongoing is performed in parallel… both actions are performed.

Example: shooting while running in UT2004.

3.Action B that is sent while another action A is still ongoing is simply ignored… nothing new happens.

Less frequent.


Durative Actions and Sensing

• While durative actions are performed an agent may receive percepts.

• Useful to monitor progress of action.

• Tower World Example:

Gripper may no longer be holding block because user removed it.

• BW4T Example:

Other robot is blocking entrance to room.


Specifying Durative Actions

• Tower World has two actions:

pickup and putdown block

both take time.

• How should we specify these actions?

delayed effect problem.

Solution:

– do not specify postcondition,

– make sure action effects are handled by event rules.


Specifying Durative ActionsTower World action specification:

Postcondition may also be “empty”: post { }

Better practice is to indicate that you have not forgotten to specify it by using post { true }.

actionspec{ pickup(X) { pre{ clear(X), not(holding(Y)) } post{ true } } putdown(X,Y) { pre { holding(X), clear(Y) } post { true } } nil { pre { true } post { true } }}


Multiple Environment Actions

• Warning: It may not be useful to combine environment actions.

• Environment actions may cancel each other out

• What happens with the following rule?

• Nothing… if gripper is already at “home position”

program{ … if goal( holding(X) ) then pickup(X) + nil. …}


Multiple Environment Actions

• Warning: It may not be useful to combine environment actions.

• Environment actions may cancel each other out

• What happens with the following rule?

• If robot is at “PlaceID” then maybe no pickup!

program{ … if bel( block(BlockID, ColorId, PlaceID) ), goal( holding(ColorId) ) then goTo(PlaceID) + pickUp(BlockID) + goTo(‘DropZone’) . …}


BW4T ASSIGNMENT


Beliefs:

Goals:

Setting Goals

• Adopt goals that agent will eventually believe.• For example, use atBlock(Block) en in(Room).

• No need to use drop action in this case!

in(‘RoomA1’)

in(‘RoomA1’)

goal adopted

-

beliefinserted

goalremoved


?? PARSING ERRORS ??

• Inspect PARSE INFO tab:Prolog Parser error at line 13, position 23 in robot.goal: Missing token; expected ENDTOKEN but got '{'.

• Pure Prolog code in knowledge section!

• Also check Console Tab for other errors.

• Stack overflow -> non-terminating Prolog rule!

knowledge{ … % Assignment 3.3: insert a predicate "nextColorInSeq(Color)“ nextColorInSeq(Color){ % ???? sequenceIndex(Q):- sequence(Q). % ???? } % ???? }


!! DOCUMENTATION !!

Document your code using COMMENTS!

• Solution can be provided in about 100 lines of code.

• But then add at least half that amount to document and explain your code


TESTING & SUBMITTING

• Test your multi-agent system on multiple maps!

• See doc for how to change maps. Your mas should work also on Banana map…!

• Hand in both .mas file and .goal file(s)!


Organisation

• Tutorial this week:– Assignment 3: BW4T agent

• Next week: modules & goal management

• Let us know any problems you have at:

[email protected]

Environments and Sensing

Documents