Cse@buffalo S.C. Shapiro Development of a Cognitive Agent Stuart C. Shapiro Department of Computer Science and Engineering and Center for Cognitive Science.

S.C. Shapiro

cse@buff

alo

Development ofa Cognitive Agent

Stuart C. Shapiro

Department of Computer Science and Engineering

and Center for Cognitive Science

State University of New York at Buffalo

[email protected]

S.C. Shapiro

cse@buff

alo

Outline• Introduction• Intensional Representation• Logic for NLU and Commonsense Reasoning• Discussing Propositions and Sentences• Symbol-Grounding by Perception and Action • Representation and Use of Indexicals• A Personal Sense of Time• Summary

S.C. Shapiro

cse@buff

alo

Goal

• A computational cognitive agent that can:– Understand and communicate in English; – Discuss specific, generic, and “rule-like” information;– Reason;– Discuss acts and plans;– Sense;– Act;– Remember and report what it has sensed and done.

S.C. Shapiro

cse@buff

alo

Embodied Cassie

• A computational cognitive agent– Embodied in hardware– or Software-Simulated– Based on SNePS and GLAIR.

S.C. Shapiro

cse@buff

alo

SNePS• Knowledge Representation and Reasoning

– Propositions as Terms

• SNIP: SNePS Inference Package– Bi-Directional Inference

• SNeBR: SNePS Belief Revision

• SNeRE: SNePS Rational Engine

• Interface Languages– SNePSUL: Lisp-Like– SNePSLOG: Logic-Like– GATN for Fragments of English.

S.C. Shapiro

cse@buff

alo

GLAIR Architecture

Knowledge Level

Perceptuo-Motor Level

Sensory-Actuator Level

NL

Vision

Sonar

Motion

Proprioception

Grounded Layered Architecture with Integrated Reasoning

SNePS

S.C. Shapiro

cse@buff

alo

Interaction with CassieEnglish

(Statement, Question, Command)

(Current) Set of Beliefs[SNePS]

(Updated) Setof Beliefs[SNePS]

Actions[SNeRE]

(New Belief)[SNePS]

English sentence expressingnew belief answering question reporting actions

Answer[SNIP]

GATN Parser

GATN Generator

ReasoningClarification DialogueLooking in World

Reasoning

S.C. Shapiro

cse@buff

alo

Cassie, the FEVAHR

S.C. Shapiro

cse@buff

alo

Cassie in the Lab

S.C. Shapiro

cse@buff

alo

Garnet Simulation World

S.C. Shapiro

cse@buff

alo


S.C. Shapiro

cse@buff

alo

Entities, Terms, Symbols, Objects

• Cassie’s mental entity: a person named Bill

• SNePS term: B5• Object in world:

S.C. Shapiro

cse@buff

alo

Intensional Representation

Intensional entities are distincteven if coreferential.

“The morning star is the evening star.”

“George IV wondered if Scott was the author of Waverly.”

S.C. Shapiro

cse@buff

alo

McCarthy’s Telephone Number Problem

Mary's telephone number is Mike's telephone number.

I understand that Mike's telephone number is Mary's telephone number.

Pat knew Mike's telephone number.

I understand that Pat knew Mike's telephone number.

Pat dialed Mike's telephone number.

I understand that Pat dialed Mike's telephone number.

S.C. Shapiro

cse@buff

alo

Answering the Telephone Number Problem

Did Pat dial Mary's telephone number?

Yes, Pat dialed Mary's telephone number.

Did Pat know Mary's telephone number?

I don't know.

S.C. Shapiro

cse@buff

alo


S.C. Shapiro

cse@buff

alo

Logic for NLU &Commonsense Reasoning

Either Pat is a man or Pat is a woman or Pat is a robot.

I understand that Pat is a robot or Pat is a woman or Pat is a man.

Pat is a woman.

I understand that Pat is a woman.

What is Pat?

Pat is a woman and Pat is not a robotand Pat is not a man.

S.C. Shapiro

cse@buff

alo

Representation in FOPL?

Man(Pat) Woman(Pat) Robot(Pat)

but don’t want inclusive or

Man(Pat) Woman(Pat) Robot(Pat)+ +

T T T

F

TSo don’t want exclusive or either

S.C. Shapiro

cse@buff

alo

andor

andor(i, j){Pi, ..., Pn}

True iff at least i, and at most j of the Pi are True

S.C. Shapiro

cse@buff

alo


S.C. Shapiro

cse@buff

alo

Discussing Propositions

That Bill is sweet is Mary's favorite proposition.

I understand that Mary's favorite proposition is that Bill is sweet.

Mike believes Mary's favorite proposition.

I understand that Mike believes that Bill is sweet.

S.C. Shapiro

cse@buff

alo

Discussing SentencesHamlet said `To be or not to be'.I understand that Hamlet said `To be or not To be'.

Polonius said `Neither a borrower nor a lender be'.I understand that Polonius said `Neither a

borrower nor a lender be'.

Who said `To be or not to be'?Hamlet said `To be or not To be'.

S.C. Shapiro

cse@buff

alo

Representing Propositionsand Sentences

Propositions and sentences must be

First-class entities of the domain

Represented by terms

S.C. Shapiro

cse@buff

alo


S.C. Shapiro

cse@buff

alo

Symbol Grounding byPerception and Action

• Agent can demonstrate understanding

– Of properties and descriptions• By locating correct objects in the world

– Of verbs• By acting properly in the world

S.C. Shapiro

cse@buff

alo

Symbol-Grounding 1

Go to the green robot.

S.C. Shapiro

cse@buff

alo

Go to the green robot.

I went to the green robot.

I am near the green robot.

Symbol-Grounding 2

S.C. Shapiro

cse@buff

alo

Alignment

• Terms at GLAIR KR level

• Aligned with

• Descriptions and procedures

• At GLAIR PM level

S.C. Shapiro

cse@buff

alo


S.C. Shapiro

cse@buff

alo

Representation and Use of Indexicals

• Words whose meanings are determined by occasion of use

• E.g. I, you, now, then, here, there

• Deictic Center <*I, *YOU, *NOW>

• *I: SNePS term representing Cassie

• *YOU: person Cassie is talking with

• *NOW: current time.

S.C. Shapiro

cse@buff

alo

Analysis of Indexicals(in input)

• First person pronouns: *YOU• Second person pronouns: *I• “here”: location of *YOU.

S.C. Shapiro

cse@buff

alo

Generation of Indexicals

• *I: First person pronouns

• *YOU: Second person pronouns

• *NOW: used to determine tense.

S.C. Shapiro

cse@buff

alo

Come here.

Use of Indexicals 1

S.C. Shapiro

cse@buff

alo

Come here.I came to you, Stu.I am near you.

Use of Indexicals 2

S.C. Shapiro

cse@buff

alo

Who am I?Your name is ‘Stu’and you are a person.

Who have you talked to?

I am talking to you.Talk to Bill.

I am talking to you, Bill.Come here.

Use of Indexicals 3

S.C. Shapiro

cse@buff

alo

Come here.

I found you.I am looking at you.

Use of Indexicals 4

S.C. Shapiro

cse@buff

alo

Come here.

I came to you.I am near you.

I found you.I am looking at you.

Use of Indexicals 5

S.C. Shapiro

cse@buff

alo

Who am I?

I talked to Stuand I am talking to you.

Your name is ‘Bill’and you are a person.

Who are you?I am the FEVAHRand my name is ‘Cassie’.

Who have you talked to?

Use of Indexicals 6

S.C. Shapiro

cse@buff

alo


S.C. Shapiro

cse@buff

alo

A Personal Sense of Time

• *NOW contains SNePS term representing current time.

• *NOW moves when Cassie acts.

S.C. Shapiro

cse@buff

alo

B6

Representation of Time

find

lex

action object

B1

!

agentact

eventtime

NOW

!!before after before after

?????????????

I

S.C. Shapiro

cse@buff

alo

Movement of Timet1 t2!before after t3!before after

NOW NOW NOW

S.C. Shapiro

cse@buff

alo

Performing a Punctual Actt1 t3!before after

NOW NOW

t2!before after

!

time

event

S.C. Shapiro

cse@buff

alo

Performing a Durative Actt1

NOW

!before after t2

!

time

event

NOW

t3!

supintsubint

S.C. Shapiro

cse@buff

alo

Keeping Track of Time 1

Who are you looking at?I am looking at you, Stu.

Come here.

S.C. Shapiro

cse@buff

alo


Who are you looking at?I am looking at you.

Come here.

I came to you.I am near you.

Who are you looking at?I am looking at you.

S.C. Shapiro

cse@buff

alo


Find Bill.I found Bill.I am looking at Bill.

Who are you looking at?I looked at youand I am looking at Bill.

Who are you talking to?I am talking to you.

S.C. Shapiro

cse@buff

alo


Follow a red robot.I found a red robot.I am looking at a red robot.

S.C. Shapiro

cse@buff

alo


I went to a red robot.I am near a red robot.I am following a red robot.

Follow a red robot.I found a red robot.I am looking at a red robot.

S.C. Shapiro

cse@buff

alo



Who am I?Your name is ‘Stu’and you are a person.

Stop.I stopped.

S.C. Shapiro

cse@buff

alo


Who are you looking at?I looked at youand I looked at Billand I looked at a red robot.

Who are you following?I followed a red robot.


S.C. Shapiro

cse@buff

alo


S.C. Shapiro

cse@buff

alo

Goal

• A computational cognitive agent/robot

• That can communicate in natural language.

S.C. Shapiro

cse@buff

alo

Intensional Representation

• SNePS terms represent mental entities.

• May assert that two entities are coreferential.

• Relations/acts may be declared transparent.

S.C. Shapiro

cse@buff

alo

Logic for NLU and Commonsense Reasoning

• Designed logical connectivesand rules of inference

More appropriate than in standard FOPC.

S.C. Shapiro

cse@buff

alo

Discussing Propositionsand Sentences

• Propositions and sentences are first-class entities.

S.C. Shapiro

cse@buff

alo

Symbol-Grounding by Perception and Action

• Use of GLAIR architecture to connect entities with descriptions/functions used by sensors and effectors.

S.C. Shapiro

cse@buff

alo

Representation and Useof Indexicals

• Use of Deictic Center for parser to interpret indexicals as current referents

• And for generator to generate indexicals from current referents.

S.C. Shapiro

cse@buff

alo

A Personal Sense of Time

• *NOW is current time.

• Updated when Cassie acts.

S.C. Shapiro

cse@buff

alo

For More Information

• Personnel

• Manual

• Tutorial

• Bibliography

• ftp’able SNePS source code

• etc.

• http://www.cse.buffalo.edu/sneps/

Cse@buffalo S.C. Shapiro Development of a Cognitive Agent Stuart C. Shapiro Department of Computer Science and Engineering and Center for Cognitive Science.

Documents