Endowing Agents with a Personal Sense of Time

cse@buff

alo

S.C. Shapiro

Endowing Agents with a Personal Sense of Time

Haythem O. Ismail & Stuart C. Shapiro

Department of Computer Science and Engineering

and Center for Cognitive Science

State University of New York at Buffalo

{hismail | shapiro}@cse.buffalo.edu

S.C. Shapiro

cse@buff

alo

Outline

• Introduction

• Subjective Time

• Time and Bodily Knowledge

• Time and External States

• Summary

S.C. Shapiro

cse@buff

alo

Uses of Timeby a Cognitive Agent

• Reason about time• Talk about time• Reason and act in time• Decide to act in timely fashion• Remember acts and when done• Sense current states• Be informed of current states• Be informed of past/future states• Understand & generate NL with correct tense & aspect.

S.C. Shapiro

cse@buff

alo

Cassie

• A Computational Cognitive Agent

• Based on SNePS– Logic-based– Network-based– Knowledge representation, reasoning, and acting.

S.C. Shapiro

cse@buff

alo

Embodied Cassie

• A Cognitive Robot– Hardware– or Software-Simulated

• Separate, but aligned– Body– Mind.

S.C. Shapiro

cse@buff

alo

GLAIR Architecture

Knowledge Level

Perceptuo-Motor Level

Sensory-Actuator Level

NL

Vision

Sonar

Motion

Proprioception

Grounded Layered Architecture with Integrated Reasoning

SNePS

S.C. Shapiro

cse@buff

alo

Symbol Grounding: Alignmentrobot

green

!

lex lexclassmod classhead

member

class

find lex

actionobject

“Find the green robot.”

B6

S.C. Shapiro

cse@buff

alo

Outline

• Introduction

• Subjective Time

• Time and Bodily Knowledge

• Time and External States

• Summary

S.C. Shapiro

cse@buff

alo

Deictic Center

• <I, YOU, NOW>• Variables whose values are SNePS terms

• Aspects of embodiedness

• *I: SNePS term representing Cassie

• *YOU: person Cassie is talking with

• *NOW: current time.

S.C. Shapiro

cse@buff

alo

Subjective Time

• NOW contains SNePS term representing current time.

• NOW moves when Cassie acts or perceives a change of state.

S.C. Shapiro

cse@buff

alo

B6

Representation of Time

find

lex

action object

B1

!

agentact

state time

NOW

!!before after before after

?????????????

I

S.C. Shapiro

cse@buff

alo

Movement of Time v.1t1 t2!before after t3!before after

NOW NOW NOW

S.C. Shapiro

cse@buff

alo

The Pacemaker

• PML process periodically increments variable COUNT.

• *COUNT = some PML integer.

• Reset to 0 when NOW moves.

• Provides bodily “feel” of passing time.

S.C. Shapiro

cse@buff

alo

Quantizing Time

Cannot conceptualize fine distinctions in time intervals.

So quantize, e.g. into half orders of magnitude (Hobbs, 2000).

S.C. Shapiro

cse@buff

alo

Movement of Time with Pacemaker

NOW COUNT n

hom

0

KL

PML

t1 t2q

!before after

time duration!

S.C. Shapiro

cse@buff

alo

Outline

• Introduction

• Subjective Time

• Time and Bodily Knowledge

• Time and External States

• Summary

S.C. Shapiro

cse@buff

alo

Modality Variables• Similar to Deictic Center.• E.g.: VISION, AUDITION, WHEELS, ARMS• *VISION = Holds(Lookat(Cassie, Stu), t3)

– if vision currently occupied by looking at Stu– t3 denotes the time during which Cassie will be

looking at Stu– *NOW is during t3

• Set at PML when bodily state starts/ceases.• One state may occupy multiple modalities.

S.C. Shapiro

cse@buff

alo

Knowing What You’re Doing

• When NOW moves– For each modality variable v– s.t. *v = Holds(s, t)– Make *NOW a subinterval of t

• So the agent believes it is now doing everything it is, in fact, doing.

S.C. Shapiro

cse@buff

alo

When you stop

• When state s ceases– For each modality variable vi– s.t. *vi = Holds(s, ti)

• Set vi to nil

– Move NOW– Believe each ti is before *NOW.

S.C. Shapiro

cse@buff

alo

When you start

• When state s starts– For each modality v that s occupies– set v to Holds(s, ti)– Move NOW.

S.C. Shapiro

cse@buff

alo

Outline

• Introduction

• Subjective Time

• Time and Bodily Knowledge

• Time and External States

• Summary

S.C. Shapiro

cse@buff

alo

The Problem of the Fleeting Now

How can you reason about “now”

if it never stands still?

S.C. Shapiro

cse@buff

alo

Motivating Joke

9:30:00 AM (Door-to-Door Salesman): May I interest you in a

brush?

9:30:02 AM (Homeowner): Not now.

9:30:03 AM (Salesman): Now?

S.C. Shapiro

cse@buff

alo

Fleeting Now Example 1

9:15:00: If the walk light is on now, cross the street.

= If the walk light is on at 9:15:00,

cross the street.9:15:01: Turn to look at walk light.9:15:02: The walk light is on at 9:15:02.

Should you cross the street?Yes, but why?

S.C. Shapiro

cse@buff

alo

Fleeting Now Example 2

12:15:00: “Is John having lunch now?”

12:15:02: Agent walks to John’s office.

12:17:00: Agent sees John at his desk, eating.

12:19:00: Agent reports “yes”.

Appropriate granularity.

S.C. Shapiro

cse@buff

alo

Fleeting Now Example 3

12:15:00: “Is John having lunch now?”

Agent knows John is at home without a phone.

Agent contemplates driving to John’s home.

Don’t bother---inappropriate granularity.

S.C. Shapiro

cse@buff

alo

The Vagueness of “now”

I’m now giving a talk.I’m now teaching a course.I’m now visiting Houston.I’m now living in Buffalo.The agent is now walking to John’s office.The agent is now seeing if John is eating lunch.

Multiple now’s at different granularities.

S.C. Shapiro

cse@buff

alo

NOW-MTF

NOW

Semi-lattice of times, all of which contain *NOW,any of which could be meant by “now”Finite---only conceptualized times of conceptualized states

Maximal Temporal Frame based on *NOW

S.C. Shapiro

cse@buff

alo

Moving NOW with MTF

NOW

S.C. Shapiro

cse@buff

alo

Typical Durations

“If the walk light is on now, cross the street.”Relevant duration is typical duration of walk lights.

“Is John having lunch now?”Relevant duration is typical duration of lunch.

Use quantized typical durations when updating NOW-MTFs.

S.C. Shapiro

cse@buff

alo

Using Appropriate Granularity

NOW

Lunch time

Lunch? Lunch!

Yes!

S.C. Shapiro

cse@buff

alo

Outline

• Introduction

• Subjective Time

• Time and Bodily Knowledge

• Time and External States

• Summary

S.C. Shapiro

cse@buff

alo

Summary• Distinguish body & mind, but align them.• Body (PML):

– What people and things look like.

– Primitive and routine actions.

– Time intervals.

– Pacemaker: Feel for elapsing time.

– Deictic Center variables.

– Modality variables.

• Mind:– Conceptualized people, things, actions, times, states.

S.C. Shapiro

cse@buff

alo

When Inquire about States

Put them into MTF

According to their typical duration.

S.C. Shapiro

cse@buff

alo

When NOW Moves

• Use Pacemaker to measure old NOW.• Include current actions in MTF.

• Include other states according to their typical durations.

S.C. Shapiro

cse@buff

alo

For More Information

http://www.cse.buffalo.edu/sneps/