Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona A Cognitive Architecture for Integrated.

Pat LangleyPat LangleySchool of Computing and InformaticsSchool of Computing and Informatics

Arizona State UniversityArizona State UniversityTempe, ArizonaTempe, Arizona

http://cll.stanford.edu/http://cll.stanford.edu/

A Cognitive Architecture for A Cognitive Architecture for Integrated Intelligent AgentsIntegrated Intelligent Agents

Thanks to D. Choi, K. Cummings, N. Nejati, S. Rogers, S. Sage, and D. Shapiro for their Thanks to D. Choi, K. Cummings, N. Nejati, S. Rogers, S. Sage, and D. Shapiro for their contributions. This talk reports research. funded by grants from DARPA IPTO and the contributions. This talk reports research. funded by grants from DARPA IPTO and the National Science Foundation, which are not responsible for its contents. National Science Foundation, which are not responsible for its contents.

The original goal of artificial intelligence was to design and The original goal of artificial intelligence was to design and implement computational artifacts that:implement computational artifacts that:

handled difficult tasks that require handled difficult tasks that require cognitivecognitive processing; processing;

combined many capabilities into integrated combined many capabilities into integrated systemssystems; ;

provided insights into the nature of mind and intelligence. provided insights into the nature of mind and intelligence.

Claim 1: Integrated Cognitive SystemsClaim 1: Integrated Cognitive Systems

Instead, modern AI has divided into many subfields that care Instead, modern AI has divided into many subfields that care little about cognition, systems, or intelligence. little about cognition, systems, or intelligence.

But the challenge remains and we need far more research on But the challenge remains and we need far more research on integrated cognitive systems. integrated cognitive systems.

Claim 2: Psychology and Design HeuristicsClaim 2: Psychology and Design Heuristics

how the system should represent and organize knowledgehow the system should represent and organize knowledge

how the system should use that knowledge in performancehow the system should use that knowledge in performance

how the system should acquire knowledge from experiencehow the system should acquire knowledge from experience

To develop intelligent systems, we must constrain their design, To develop intelligent systems, we must constrain their design, and findings about human behavior can suggest: and findings about human behavior can suggest:

This approach has led to many new insights and methods, but This approach has led to many new insights and methods, but few modern AI researchers take advantage of it. few modern AI researchers take advantage of it.

We need far more work that incorporates ideas from cognitive We need far more work that incorporates ideas from cognitive psychology into the design of AI systems. psychology into the design of AI systems.

The Fragmentation of AI ResearchThe Fragmentation of AI Research

action

perception

reasoning

learning

planninglanguage

The Domain of In-City DrivingThe Domain of In-City Driving

Consider driving a vehicle Consider driving a vehicle in a city, which requires: in a city, which requires:

selecting routesselecting routes obeying traffic lightsobeying traffic lights avoiding collisionsavoiding collisions being polite to othersbeing polite to others finding addressesfinding addresses staying in the lanestaying in the lane parking safelyparking safely stopping for pedestriansstopping for pedestrians following other vehiclesfollowing other vehicles delivering packagesdelivering packages

These tasks range from These tasks range from low-level execution to low-level execution to high-level reasoning. high-level reasoning.

QuickTime™ and aCinepak decompressor

are needed to see this picture.

Newell’s CritiqueNewell’s Critique

move beyond isolated phenomena and capabilities to develop move beyond isolated phenomena and capabilities to develop complete models of intelligent behavior; complete models of intelligent behavior;

demonstrate our systems’ intelligence on the same range of demonstrate our systems’ intelligence on the same range of domains and tasks as humans can handle; domains and tasks as humans can handle;

evaluate these systems in terms of generality and flexibility evaluate these systems in terms of generality and flexibility rather than success on a single class of tasks.rather than success on a single class of tasks.

However, there are different paths toward achieving such systems. However, there are different paths toward achieving such systems.

In 1973, Allen Newell argued “In 1973, Allen Newell argued “You can’t play twenty questions You can’t play twenty questions with nature and winwith nature and win”. Instead, he proposed that we: ”. Instead, he proposed that we:

A System with Communicating ModulesA System with Communicating Modules

action

perception

reasoning

learning

planninglanguage

software engineering / multi-agent systemssoftware engineering / multi-agent systems

action

perception

reasoning

learning

planninglanguage

short-termbeliefs and goals

A System with Shared Short-Term MemoryA System with Shared Short-Term Memory

blackboard architecturesblackboard architectures

Newell’s vision for research on theories of intelligence was that:Newell’s vision for research on theories of intelligence was that:

cognitive systems should make strong theoretical assumptions cognitive systems should make strong theoretical assumptions about the nature of the mind; about the nature of the mind;

theories of intelligence should change only gradually, as new theories of intelligence should change only gradually, as new structures or processes are determined necessary;structures or processes are determined necessary;

later design choices should be constrained heavily by earlier later design choices should be constrained heavily by earlier ones, not made independently. ones, not made independently.

Integration vs. UnificationIntegration vs. Unification

A successful framework is all about mutual constraints, and it A successful framework is all about mutual constraints, and it should provide a should provide a unifiedunified theory of intelligent behavior. theory of intelligent behavior.

He associated these aims with the idea of a He associated these aims with the idea of a cognitive architecturecognitive architecture, , which were also to incorporate results from psychology. which were also to incorporate results from psychology.

A System with Shared Long-Term MemoryA System with Shared Long-Term Memory

action

perception

reasoning

learning

planninglanguage

short-termbeliefs and goals

long-term memorystructures

cognitive architecturescognitive architectures

A Constrained Cognitive ArchitectureA Constrained Cognitive Architecture

action

perception

reasoning

learning

planninglanguage

short-termbeliefs and goals

long-term memorystructures

The IThe ICARUSCARUS Architecture Architecture

In this talk I will use one such framework In this talk I will use one such framework I ICARUSCARUS to illustrate to illustrate the advantages of cognitive architectures. the advantages of cognitive architectures.

IICARUSCARUS incorporates a variety of assumptions from psychological incorporates a variety of assumptions from psychological theories; the most basic are that: theories; the most basic are that:

These claims give IThese claims give ICARUSCARUS much in common with other cognitive much in common with other cognitive architectures like ACT-R, Soar, and Prodigy. architectures like ACT-R, Soar, and Prodigy.

1.1. Short-term memories are distinct from long-term stores Short-term memories are distinct from long-term stores

2.2. Memories contain modular elements cast as list structuresMemories contain modular elements cast as list structures

3.3. Long-term structures are accessed through pattern matchingLong-term structures are accessed through pattern matching

4.4. Cognition occurs in retrieval/selection/action cyclesCognition occurs in retrieval/selection/action cycles

5.5. Performance and learning compose elements in memoryPerformance and learning compose elements in memory

Cascaded Integration in ICascaded Integration in ICARUSCARUS

IICARUSCARUS adopts a adopts a cascadedcascaded approach to integration in which approach to integration in which lower-level modules produce results for higher-level ones. lower-level modules produce results for higher-level ones.

conceptual inference

skill execution

problem solving

learning

Each of these modules incorporates a variety of ideas that have Each of these modules incorporates a variety of ideas that have their origin in cognitive psychology. their origin in cognitive psychology.

concepts are distinct cognitive entities that support both concepts are distinct cognitive entities that support both categorization and inference;categorization and inference;

the majority of human concepts are grounded in perception the majority of human concepts are grounded in perception and action (Barsalou, 1999); and action (Barsalou, 1999);

many human concepts are relational in nature, describing many human concepts are relational in nature, describing connections among entities (Kotovsky & Gentner, 1996); connections among entities (Kotovsky & Gentner, 1996);

concepts are organized in a hierarchical manner, with more concepts are organized in a hierarchical manner, with more complex categories defined in terms of simpler ones. complex categories defined in terms of simpler ones.

Representing and Using ConceptsRepresenting and Using Concepts

Cognitive psychology makes claims about conceptual knowledge: Cognitive psychology makes claims about conceptual knowledge:

IICARUSCARUS adopts these assumptions about conceptual memory. adopts these assumptions about conceptual memory.

IICARUSCARUS Concepts for In-City Driving Concepts for In-City Driving

((in-rightmost-lane ?self ?clane)((in-rightmost-lane ?self ?clane) :percepts (:percepts ( (self ?self) (segment ?seg) (self ?self) (segment ?seg)

(line ?clane segment ?seg))(line ?clane segment ?seg)) :relations ((driving-well-in-segment ?self ?seg ?clane):relations ((driving-well-in-segment ?self ?seg ?clane)

(last-lane ?clane) (last-lane ?clane) (not (lane-to-right ?clane ?anylane))))(not (lane-to-right ?clane ?anylane))))

((driving-well-in-segment ?self ?seg ?lane)((driving-well-in-segment ?self ?seg ?lane) :percepts ((self ?self) (segment ?seg) (line ?lane segment ?seg)):percepts ((self ?self) (segment ?seg) (line ?lane segment ?seg)) :relations ((in-segment ?self ?seg) (in-lane ?self ?lane):relations ((in-segment ?self ?seg) (in-lane ?self ?lane)

(aligned-with-lane-in-segment ?self ?seg ?lane)(aligned-with-lane-in-segment ?self ?seg ?lane)(centered-in-lane ?self ?seg ?lane)(centered-in-lane ?self ?seg ?lane)(steering-wheel-straight ?self)))(steering-wheel-straight ?self)))

((in-lane ?self ?lane)((in-lane ?self ?lane) :percepts (:percepts ( (self ?self segment ?seg) (line ?lane segment ?seg dist ?dist))(self ?self segment ?seg) (line ?lane segment ?seg dist ?dist)) :tests (:tests ( (> ?dist -10) (<= ?dist 0)))(> ?dist -10) (<= ?dist 0)))

Structure and Use of Conceptual MemoryStructure and Use of Conceptual Memory

IICARUS CARUS organizes conceptual memory in a hierarchical organizes conceptual memory in a hierarchical manner.manner.

Conceptual inference occurs from the bottom up, starting from Conceptual inference occurs from the bottom up, starting from percepts to produce high-level beliefs about the current state. percepts to produce high-level beliefs about the current state.

Representing Short-Term Beliefs/GoalsRepresenting Short-Term Beliefs/Goals

(current-street me A)(current-street me A) (current-segment me g550)(current-segment me g550)(lane-to-right g599 g601)(lane-to-right g599 g601) (first-lane g599)(first-lane g599)(last-lane g599)(last-lane g599) (last-lane g601)(last-lane g601)(at-speed-for-u-turn me)(at-speed-for-u-turn me) (slow-for-right-turn me)(slow-for-right-turn me)(steering-wheel-not-straight me)(steering-wheel-not-straight me) (centered-in-lane me g550 g599)(centered-in-lane me g550 g599)(in-lane me g599)(in-lane me g599) (in-segment me g550)(in-segment me g550)(on-right-side-in-segment me)(on-right-side-in-segment me) (intersection-behind g550 g522)(intersection-behind g550 g522)(building-on-left g288)(building-on-left g288) (building-on-left g425)(building-on-left g425)(building-on-left g427)(building-on-left g427) (building-on-left g429)(building-on-left g429)(building-on-left g431)(building-on-left g431) (building-on-left g433)(building-on-left g433)(building-on-right g287)(building-on-right g287) (building-on-right g279)(building-on-right g279)(increasing-direction me)(increasing-direction me) (buildings-on-right g287 g279)(buildings-on-right g287 g279)

the same generic skill may be applied to distinct objects that the same generic skill may be applied to distinct objects that meet its application conditions; meet its application conditions;

skills support the execution of complex activities that have skills support the execution of complex activities that have hierarchical organization (Rosenbaum et al., 2001); hierarchical organization (Rosenbaum et al., 2001);

humans can carry out open-loop sequences, but they can also humans can carry out open-loop sequences, but they can also operate in closed-loop reactive mode; operate in closed-loop reactive mode;

humans can deal with multiple goals with different priorities, humans can deal with multiple goals with different priorities, which can lead to interrupted behavior. which can lead to interrupted behavior.

Skills and ExecutionSkills and Execution

IICARUSCARUS embodies these ideas in its skill execution module. embodies these ideas in its skill execution module.

Psychology also makes claims about skills and their execution: Psychology also makes claims about skills and their execution:

((in-rightmost-lane ?self ?line)((in-rightmost-lane ?self ?line) :percepts:percepts ((self ?self) (line ?line))((self ?self) (line ?line)) :start ((last-lane ?line)):start ((last-lane ?line)) :subgoals ((driving-well-in-segment ?self ?seg ?line))) :subgoals ((driving-well-in-segment ?self ?seg ?line)))

((driving-well-in-segment ?self ?seg ?line)((driving-well-in-segment ?self ?seg ?line) :percepts:percepts ((segment ?seg) (line ?line) (self ?self))((segment ?seg) (line ?line) (self ?self)) :start ((steering-wheel-straight ?self)):start ((steering-wheel-straight ?self)) :subgoals ((in-segment ?self ?seg):subgoals ((in-segment ?self ?seg)

(centered-in-lane ?self ?seg ?line)(centered-in-lane ?self ?seg ?line)(aligned-with-lane-in-segment ?self ?seg ?line)(aligned-with-lane-in-segment ?self ?seg ?line)(steering-wheel-straight ?self)))(steering-wheel-straight ?self)))

((in-segment ?self ?endsg)((in-segment ?self ?endsg) :percepts:percepts ((self ?self speed ?speed) (intersection ?int cross ?cross)((self ?self speed ?speed) (intersection ?int cross ?cross)

(segment ?endsg street ?cross angle ?angle))(segment ?endsg street ?cross angle ?angle)) :start ((in-intersection-for-right-turn ?self ?int)):start ((in-intersection-for-right-turn ?self ?int)) :actions:actions ((((steer 1)))steer 1)))

IICARUSCARUS Skills for In-City Driving Skills for In-City Driving

IICARUSCARUS Skills Build on Concepts Skills Build on Concepts

conceptsconcepts

skillsskills

Each concept is defined in terms of other concepts and/or percepts.Each concept is defined in terms of other concepts and/or percepts.

Each skill is defined in terms of other skills, concepts, and percepts.Each skill is defined in terms of other skills, concepts, and percepts.

IICARUS CARUS stores skills in a hierarchical manner that links to concepts.stores skills in a hierarchical manner that links to concepts.

Skill Execution in ISkill Execution in ICARUSCARUS

This process repeats on each cycle to give This process repeats on each cycle to give teleoreactiveteleoreactive control control (Nilsson, 1994) with a bias toward persistence of initiated skills. (Nilsson, 1994) with a bias toward persistence of initiated skills.

Skill execution occurs from the top down, starting from goals Skill execution occurs from the top down, starting from goals to find applicable paths through the skill hierarchy. to find applicable paths through the skill hierarchy.

humans often resort to means-ends analysis to solve novel humans often resort to means-ends analysis to solve novel problems (Newell & Simon, 1961); problems (Newell & Simon, 1961);

problem solving often occurs in a physical context and is problem solving often occurs in a physical context and is interleaved with execution (Gunzelman & Anderson, 2003); interleaved with execution (Gunzelman & Anderson, 2003);

efforts to overcome impasses during problem solving leads to efforts to overcome impasses during problem solving leads to incremental acquisition of new skills (Anzai & Simon, 1979); incremental acquisition of new skills (Anzai & Simon, 1979);

structural learning involves monotonic addition of symbolic structural learning involves monotonic addition of symbolic elements to long-term memory; elements to long-term memory;

learning can transform backward-chaining heuristic search into learning can transform backward-chaining heuristic search into informed forward-chaining execution (Larkin et al., 1980). informed forward-chaining execution (Larkin et al., 1980).

Ideas about Problem Solving and LearningIdeas about Problem Solving and Learning

Psychology also has ideas about problem solving and learning: Psychology also has ideas about problem solving and learning:

IICARUSCARUS reflects these ideas in its problem solving and learning. reflects these ideas in its problem solving and learning.

IICARUSCARUS Interleaves Execution and Problem Solving Interleaves Execution and Problem Solving

Executed plan

Problem

??

Skill Hierarchy

Primitive Skills

ReactiveExecution

impasse?

ProblemSolving

yesyes

nono

This organization reflects the psychological distinction between automatized This organization reflects the psychological distinction between automatized and controlled behavior. and controlled behavior.

A Successful Problem-Solving TraceA Successful Problem-Solving Trace

(ontable A T)

(on B A)

(on C B)

(hand-empty)

(clear C)

(unst. C B) (unstack C B) (clear B)

(putdown C T)

(unst. B A) (unstack B A) (clear A)

(holding C) (hand-empty)

(holding B)

A

B

C B

A C

initial stateinitial state

goalgoal

IICARUSCARUS Learns Skills from Problem Solving Learns Skills from Problem Solving

Executed plan

Problem

??

Skill Hierarchy

Primitive Skills

ReactiveExecution

impasse?

ProblemSolving

yesyes

nono

SkillLearning

(ontable A T)

(on B A)

(on C B)

(hand-empty)

(clear C)

(unst. C B) (unstack C B) (clear B)

(putdown C T)

(unst. B A) (unstack B A) (clear A)

(holding C) (hand-empty)

(holding B)

A

B

C B

A C

1

skill chainingskill chaining

Constructing Skills from a TraceConstructing Skills from a Trace

(ontable A T)

(on B A)

(on C B)

(hand-empty)

(clear C)

(unst. C B) (unstack C B) (clear B)

(putdown C T)

(unst. B A) (unstack B A) (clear A)

(holding C) (hand-empty)

(holding B)

A

B

C B

A C

1

2

skill chainingskill chaining

Constructing Skills from a TraceConstructing Skills from a Trace

(ontable A T)

(on B A)

(on C B)

(hand-empty)

(clear C)

(unst. C B) (unstack C B) (clear B)

(putdown C T)

(unst. B A) (unstack B A) (clear A)

(holding C) (hand-empty)

(holding B)

A

B

C B

A C

1 3

2

concept chainingconcept chaining

Constructing Skills from a TraceConstructing Skills from a Trace

(ontable A T)

(on B A)

(on C B)

(hand-empty)

(clear C)

(unst. C B) (unstack C B) (clear B)

(putdown C T)

(unst. B A) (unstack B A) (clear A)

(holding C) (hand-empty)

(holding B)

A

B

C B

A C

1 3

2

4

skill chainingskill chaining

Constructing Skills from a TraceConstructing Skills from a Trace

Learned Skills in the Blocks WorldLearned Skills in the Blocks World

(clear (?C)(clear (?C) :percepts :percepts ((block ?D) (block ?C))((block ?D) (block ?C)) :start :start ((unstackable ?D ?C))((unstackable ?D ?C)) :skills :skills ((unstack ?D ?C)))((unstack ?D ?C)))

(clear (?B)(clear (?B) :percepts :percepts ((block ?C) (block ?B))((block ?C) (block ?B)) :start :start ((on ?C ?B) (hand-empty))((on ?C ?B) (hand-empty)) :skills :skills ((unstackable ?C ?B) (unstack ?C ?B)))((unstackable ?C ?B) (unstack ?C ?B)))

(unstackable (?C ?B)(unstackable (?C ?B) :percepts :percepts ((block ?B) (block ?C))((block ?B) (block ?C)) :start :start ((on ?C ?B) (hand-empty))((on ?C ?B) (hand-empty)) :skills :skills ((clear ?C) (hand-empty)))((clear ?C) (hand-empty)))

(hand-empty ( )(hand-empty ( ) :percepts :percepts ((block ?D) (table ?T1))((block ?D) (table ?T1)) :start :start ((putdownable ?D ?T1))((putdownable ?D ?T1)) :skills :skills ((putdown ?D ?T1)))((putdown ?D ?T1)))

Hierarchical skills are Hierarchical skills are generalized traces of generalized traces of successful means-ends successful means-ends problem solvingproblem solving

Cumulative Curves for Blocks WorldCumulative Curves for Blocks World

Cumulative Curves for FreeCellCumulative Curves for FreeCell

Learning Skills for In-City DrivingLearning Skills for In-City Driving

We have also trained We have also trained IICARUSCARUS to drive in our to drive in our in-city environment. in-city environment.

We provide the system We provide the system with tasks of increasing with tasks of increasing complexity. complexity.

Learning transforms Learning transforms the problem-solving the problem-solving traces into hierarchical traces into hierarchical skills. skills.

The agent uses these The agent uses these skills to change lanes, skills to change lanes, turn, and park using turn, and park using only reactive control. only reactive control.

QuickTime™ and aCinepak decompressor

are needed to see this picture.

Skill Clauses Learning for In-City DrivingSkill Clauses Learning for In-City Driving

((parked ?me ?g1152)((parked ?me ?g1152) :percepts :percepts (( (lane-line ?g1152) (self ?me))(lane-line ?g1152) (self ?me)) :start :start ( )( ) :subgoals :subgoals (( (in-rightmost-lane ?me ?g1152)(in-rightmost-lane ?me ?g1152) (stopped ?me)) )(stopped ?me)) )

((in-rightmost-lane ?me ?g1152)((in-rightmost-lane ?me ?g1152) :percepts :percepts (( (self ?me) (lane-line ?g1152))(self ?me) (lane-line ?g1152)) :start :start (( (last-lane ?g1152))(last-lane ?g1152)) :subgoals :subgoals (( (driving-well-in-segment ?me ?g1101 ?g1152)) )(driving-well-in-segment ?me ?g1101 ?g1152)) )

((driving-well-in-segment ?me ?g1101 ?g1152)((driving-well-in-segment ?me ?g1101 ?g1152) :percepts :percepts (( (lane-line ?g1152) (segment ?g1101) (self ?me))(lane-line ?g1152) (segment ?g1101) (self ?me)) :start :start (( (steering-wheel-straight ?me))(steering-wheel-straight ?me)) :subgoals :subgoals (( (in-lane ?me ?g1152)(in-lane ?me ?g1152) (centered-in-lane ?me ?g1101 ?g1152)(centered-in-lane ?me ?g1101 ?g1152) (aligned-with-lane-in-segment ?me ?g1101 ?g1152)(aligned-with-lane-in-segment ?me ?g1101 ?g1152) (steering-wheel-straight ?me)) )(steering-wheel-straight ?me)) )

Learning Curves for In-City DrivingLearning Curves for In-City Driving

The architecture also supports the transfer of knowledge in that:The architecture also supports the transfer of knowledge in that:

skills acquired later can build on those learned earlier; skills acquired later can build on those learned earlier;

skill clauses are indexed by the goals they achieve; skill clauses are indexed by the goals they achieve;

conceptual inference supports mapping across domains. conceptual inference supports mapping across domains.

Transfer of Skills in ITransfer of Skills in ICARUSCARUS

We are exploring such effects in IWe are exploring such effects in ICARUSCARUS as part of a DARPA as part of a DARPA program on the transfer of learned knowledge. program on the transfer of learned knowledge.

Testbeds include first-person shooter games, board games, and Testbeds include first-person shooter games, board games, and physics problem solving. physics problem solving.

Transfer Effects in FreeCellTransfer Effects in FreeCell

On tasks with more cards, prior training aids solution probability.On tasks with more cards, prior training aids solution probability.

Transfer Effects in FreeCellTransfer Effects in FreeCell

However, it also lets the system solve problems with less effort. However, it also lets the system solve problems with less effort.

Cognitive architectures come with a programming language that:Cognitive architectures come with a programming language that:

includes a syntax linked to its representational assumptionsincludes a syntax linked to its representational assumptions

inputs long-term knowledge and initial short-term elementsinputs long-term knowledge and initial short-term elements

provides an interpreter that runs the specified programprovides an interpreter that runs the specified program

incorporates tracing facilities to inspect system behaviorincorporates tracing facilities to inspect system behavior

Architectures as Programming LanguagesArchitectures as Programming Languages

Such programming languages ease construction and debugging Such programming languages ease construction and debugging of knowledge-based systems.of knowledge-based systems.

For this reason, cognitive architectures support far more For this reason, cognitive architectures support far more efficient development of software for intelligent systems. efficient development of software for intelligent systems.

The programming language associated with IThe programming language associated with ICARUSCARUS comes with: comes with:

a syntax for concepts, skills, beliefs, and perceptsa syntax for concepts, skills, beliefs, and percepts

the ability to load and parse such programsthe ability to load and parse such programs

an interpreter for inference, execution, planning, and learningan interpreter for inference, execution, planning, and learning

a trace package that displays system behavior over timea trace package that displays system behavior over time

Programming in IProgramming in ICARUSCARUS

We have used this language to develop adaptive intelligent We have used this language to develop adaptive intelligent agents in a variety of domains. agents in a variety of domains.

An IAn ICARUSCARUS Agent for Urban Combat Agent for Urban Combat

QuickTime™ and aCinepak decompressor

are needed to see this picture.

IICARUSCARUS’ Memories and Processes’ Memories and Processes

Long-TermLong-TermConceptualConceptual

MemoryMemory

Short-TermShort-TermBeliefBelief

MemoryMemory

Short-TermShort-TermGoal MemoryGoal Memory

ConceptualConceptualInferenceInference

SkillSkillExecutionExecution

PerceptionPerception

EnvironmentEnvironment

PerceptualPerceptualBufferBuffer

Problem SolvingProblem SolvingSkill LearningSkill Learning

MotorMotorBufferBuffer

Skill RetrievalSkill Retrievaland Selectionand Selection

Long-TermLong-TermSkill MemorySkill Memory

Similarities to Previous ArchitecturesSimilarities to Previous Architectures

IICARUSCARUS has much in common with other cognitive architectures has much in common with other cognitive architectures like Soar (Laird et al., 1987) and ACT-R (Anderson, 1993): like Soar (Laird et al., 1987) and ACT-R (Anderson, 1993):

These ideas all have their origin in theories of human memory, These ideas all have their origin in theories of human memory, problem solving, and skill acquisition. problem solving, and skill acquisition.

1.1. Short-term memories are distinct from long-term stores Short-term memories are distinct from long-term stores

2.2. Memories contain modular elements cast as symbolic structuresMemories contain modular elements cast as symbolic structures

3.3. Long-term structures are accessed through pattern matchingLong-term structures are accessed through pattern matching

4.4. Cognition occurs in retrieval/selection/action cyclesCognition occurs in retrieval/selection/action cycles

5.5. Learning involves monotonic addition of elements to memoryLearning involves monotonic addition of elements to memory

6.6. Learning is incremental and interleaved with performanceLearning is incremental and interleaved with performance

Distinctive Features of IDistinctive Features of ICARUSCARUS

However, IHowever, ICARUSCARUS also makes assumptions that distinguish it from also makes assumptions that distinguish it from these architectures:these architectures:

Some of these assumptions appear in Bonasso et al.’s (2003) 3T, Some of these assumptions appear in Bonasso et al.’s (2003) 3T, Freed’s APEX, and Sun et al.’s (2001) CLARION architectures. Freed’s APEX, and Sun et al.’s (2001) CLARION architectures.

1.1. Cognition is grounded in perception and action Cognition is grounded in perception and action

2.2. Categories and skills are separate cognitive entitiesCategories and skills are separate cognitive entities

3.3. Short-term elements are instances of long-term structuresShort-term elements are instances of long-term structures

4.4. Inference and execution are more basic than problem solvingInference and execution are more basic than problem solving

5.5. Skill/concept hierarchies are learned in a cumulative mannerSkill/concept hierarchies are learned in a cumulative manner

Directions for Future ResearchDirections for Future Research

progressive deepening in forward-chaining searchprogressive deepening in forward-chaining search

graded nature of categories and category learninggraded nature of categories and category learning

model-based character of human reasoningmodel-based character of human reasoning

persistent but limited nature of short-term memoriespersistent but limited nature of short-term memories

creating perceptual chunks to reduce these limitationscreating perceptual chunks to reduce these limitations

storing and retrieving episodic memory tracesstoring and retrieving episodic memory traces

Future work on IFuture work on ICARUSCARUS should incorporate other ideas about: should incorporate other ideas about:

These additions will increase further IThese additions will increase further ICARUSCARUS’ debt to psychology. ’ debt to psychology.

Concluding RemarksConcluding Remarks

are embedded within a unified cognitive architecture;are embedded within a unified cognitive architecture;

incorporate constraints based on ideas from psychology; incorporate constraints based on ideas from psychology;

demonstrate a wide range of intelligent behavior; demonstrate a wide range of intelligent behavior;

are evaluated on multiple tasks in challenging testbeds. are evaluated on multiple tasks in challenging testbeds.

We need more research on integrated intelligent systems that: We need more research on integrated intelligent systems that:

For more information about the IFor more information about the ICARUSCARUS architecture, see: architecture, see:

http://cll.stanford.edu/research/ongoing/icarus/

End of PresentationEnd of Presentation

IICARUSCARUS’ Inference-Execution Cycle’ Inference-Execution Cycle

1.1. places descriptions of sensed objects in the perceptual buffer;places descriptions of sensed objects in the perceptual buffer;

2.2. infers instances of concepts implied by the current situation;infers instances of concepts implied by the current situation;

3.3. finds paths through the skill hierarchy from top-level goals;finds paths through the skill hierarchy from top-level goals;

4.4. selects one or more applicable skill paths for execution; selects one or more applicable skill paths for execution;

5.5. invokes the actions associated with each selected path. invokes the actions associated with each selected path.

On each successive execution cycle, the IOn each successive execution cycle, the ICARUSCARUS architecture: architecture:

IICARUSCARUS agents are agents are teleoreactiveteleoreactive (Nilsson, 1994) in that they are (Nilsson, 1994) in that they are executed reactively but in a goal-directed manner. executed reactively but in a goal-directed manner.

IICARUSCARUS’ Constraints on Skill Learning’ Constraints on Skill Learning

What determines the hierarchical structure of skill memory?What determines the hierarchical structure of skill memory?

The structure emerges the subproblems that arise during The structure emerges the subproblems that arise during problem solving, which, because operator conditions and problem solving, which, because operator conditions and goals are single literals, form a semilattice.goals are single literals, form a semilattice.

What determines the heads of the learned clauses/methods?What determines the heads of the learned clauses/methods?

The head of a learned clause is the goal literal that the The head of a learned clause is the goal literal that the planner achieved for the subproblem that produced it.planner achieved for the subproblem that produced it.

What are the conditions on the learned clauses/methods? What are the conditions on the learned clauses/methods?

If the subproblem involved skill chaining, they are the If the subproblem involved skill chaining, they are the conditions of the first subskill clause.conditions of the first subskill clause.

If the subproblem involved concept chaining, they are the If the subproblem involved concept chaining, they are the subconcepts that held at the subproblem’s outset. subconcepts that held at the subproblem’s outset.

Cumulative Curves for Blocks WorldCumulative Curves for Blocks World