RAAI Summer School 2019school-raai.org/wp-content/uploads/2019/07/Gudwin-RAAI... · 2019-07-08 · Peirce’s Modes of Being Firstness is the mode of being of that which is such as

Copyright@2019 DCA-FEEC-UNICAMP

RAAI Summer School 2019

Motivational Systems in Cognitive Architectures

Ricardo GudwinDCA-FEEC-UNICAMP

[email protected]://www.dca.fee.unicamp.br/~gudwin

Introduction

Intelligent Agents Systems capable of sensing

an environment, creating a model of it and acting intelligently on it

Environment

Perception

Actuation

Sensors

Actuators

Agent

Cognitive Architecture

The Problem: The Control of Intelligent Autonomous Agents

Introduction

Cognitive Architecture General purpose computational architectures providing the basic

infrastructure to the construction of artificial agents Cognitive models of human mind

• Implementing computational versions of cognitive abilities, such as sensing, perception, attention, emotions, memory, decision-making, reactive and deliberative behavior, motivations, learning and in some cases consciousness, language, meta-cognition and social cognition

40 years of Cognitive Architectures• 4CAPS, 4D/RCS, ACT-R, AIS, Apex, ART, Atlantis, BECCA, biSoar,

CERA-CRANIUM, Chrest, Clarion, CogNet, CogPrime, CoJACK, Copycat, Disciple, DUAL, Emotion Machine, Epic, ERE, FORR, Gat, GLAIR, GMU-BICA, Guardian, H-Cogaff, Homer, HTM, Icarus, Imprint, Leabra, LIDA, MAX, MECA, NARS, Nexting, Omar, Pogamut, Polyscheme, PRODIGY, PRS, Psi-Theory, R-CAST, RALPH-MEA, Recommendation Architecture, REM, SAL, Soar, Society of Mind, Subsumption architecture, Teton, Theo, Tosca, Ymir

CLARION

LIDA

MECA

SOAR

Introduction

Kinds of Architectures Symbolic (Rule-based) Connectionist (Neural Network) Hybrid

Software Implementations Framework Toolkit

Cognitive Cycle Cycle starting from sensory information and finishing in

actuations in the environment Behavior Generation

Sequence of processes within the cognitive cycle which ends generating the final agent’s behavior at the environment

Peirce’s Modes of Being

Firstness is the mode of being of that which is such as it is, positively and without reference

to anything else (CP 8.328; 1.295). The idea of First is predominant in the ideas of freshness, life, freedom (CP 1.302), novelty, creation, originality, potentiality, randomness.

Secondness is the mode of being of that which is such as it is, with respect to a second but

regardless of any third (CP 8.328; 1.296). The idea of second is predominant in the ideas of causation and of static force (CP 1.325), comparison, opposition, polarity, differentiation, existence (opposition to everything else).

Thirdness is the mode of being of that which is such as it is, in bringing a first and second

into relation to each other (CP 8.328;1.297). The idea of third is predominant in the ideas of generality, infinity, continuity, diffusion, growth, intelligence (CP 1.340), meaning, mediation and representation.

Applying Peirce’s Universal Categories to Behaviors

Kinds of Behaviors Original Behavior (firstness) Reactive Behavior (secondness) Motivated Behavior (thirdness)

Original Behaviors Random actions Random modifications of already known actions Random combinations of already known actions

Reactive Behaviors Depends only on sensory inputs, to what they react

• Behavior = function(sensory inputs) Don’t have internal states Are “blind” to what they cause in the world (mechanical reaction)

Motivated Behavior

Motivated Behavior Are not randomly generated behaviors neither pure reactions to

sensory inputs Have a “finality”, a future to be reached, a “goal”

Complex discussion in Philosophy and Biology Aristotle’s Final Cause, Teleology, Teleonomy

Cybernetics Rosenblueth, A., Wiener, N., & Bigelow, J. (1943). Behavior,

purpose and teleology. Philosophy of science, 10(1), 18-24.• Purposeful action: directed to the achievement of a goal – a final

condition in which the object affected by the action reaches a predetermined condition

• Negative Feedback Loop Control System Bratman, M. (1987). Intention, plans, and practical reason.

• Plans are created before their execution

Motivated Behavior

Desired Future’sRepresentation

Distance between CurrentPresent and Desired Future

Reactive Behavior

Sensory Inputs

Present’s Representation

ActuatorOutputs

Environment

+-Thirdness

Motivational Systems

Motivational Systems Module within a Cognitive Architecture responsible for

determining the behavior of intelligent agents, based on cognitive models of human motivations

Hull’s Drive Reduction Approach Based on the Motivation Theory from Clark Hull Identifies the origin of behavior in a set of internal needs,

ontogenetically inherited, providing motivation in the behavior generation

NEEDS DRIVES

AUTOMATICBEHAVIORNeeds Degree

of Dissatisfaction

Drive Reduction(provides needs satisfaction)

Motivational Systems

Maslow and the Hierarchy of Needs

Motivational Systems

The Problem in Maslow’s Proposal Arguments against the universality of the hierarchy

Source: (Kenrick et.al. 2010)

Motivational Systems

Maslow’s Considerations to be used As the number of needs increase, more difficult it is to find

actions simultaneously satisfying many needs Needs form a hierarchy, where some needs might have priority

over others• Needs with a bigger priority are satisfied first

Points still requiring investigation There might be unique hierarchies depending on the individual ?

(Free Will) Hierarchies might change during existence ? Is it possible to create new needs along the creature’s experience

?

Motivational Systems

Impulse Theory (Bausmeister 2016) Impulses are particular desires to perform particular behaviors on

particular occasions Contrary to drives, impules have a beginning and an end

Impulses Might imply an evolution of Hull’s standard drive mechanism:

instead of drive reduction behaviors, this mechanism generates corrective impulses, which, in being satisfied, reduce the corresponding drive

Impulses are dependent on drives Impulses depend on a “plan” for their satisfaction Impulses from different drives compete to each other to be

selected and executed

Motivational Systems

The Liking – Wanting Dichotomy

Wanting Happens in the present, but is directed to the future Specify a possible future to be reached Related to the drives/impulses mechanism

Liking Hedonic dimension – pleasure / displeasure Pleasure or Pleasures ? One or more dimensions ? Uniquely related to present, right now ! Related in some sense to a kind of emotional mechanism Related to the temporal difference on drives (Δdrive, or ∂drive/∂t)

• Evolution further introduced specialized dimensions• Goal: to serve as reinforcers in the drive reduction action learning

Models for Emotions

Whare are Emotions ? How emotions relate to motivations ? Big confusion in the literature !!!

Etymology Emovere – ex + movere = to put out

Origins Distinction between active and passive, action and passion The concept of “passion” was gradually refined to reach the

modern concept of emotion, used in psychology From “passion” as a mere passivity to something external to the

mind, the concepts of feeling, pleasure/displeasure as appraisive reactions to possible courses of action

From mere “passivity”, emotions become an action of mind guiding rational/intelligent behavior

Models for Emotions

The Modern Concept of Emotion (Dixon 2012) Appears just in XIX century, as an abstraction to different

concepts as e.g. appetites, passion, affect, feelings Created by the Scottish thinker Thomas Brown (1778-1820)

Main Perspectives Darwin: emotions as the result of evolutionary process –

behavioral expressions constructed during evolution William James: emotions as affects in the body, causing feelings

(Damásio) Cognitive (Arnold 1960): Emotions as appraisals Social-constructivist: emotions as the cultural byproduct of

learning social rules, caused by the interaction among human beings

Models for Emotions

The OCC Model (Ortony, Clore, Collins) Appraisal / Arousal Dychotomy Emotions are inner states with different intensities, related to

affects, i.e. appraisal reactions to situations considered to be good or bad, related to events, agents or environment objects

The OCC Model specifies 22 emotions, implemented in terms of local and global variables, with their values determined from rules applied to sensory information

Emotional parameters:• Emotion Triggering Potential• Emotion Triggering Threshold• Emotion Intensity

Models for Emotions

Damásio and the Somatic Marker Rationality with Emotions The need of a body in the process Distinction between Emotions and Feelings Emotion

• A body modification due to an external stimuli Feeling

• Cognitive counter-part of an emotion, appearing after the consciousness perception of the emotion

Primary Emotions• Experienced since childhood – innate mechanisms• Fear, joy, sadness, anger, etc.

Secondary Emotions• Learned along the life-cycle of an individual• Jealousy, Guilt, Pride, etc.

Models for Emotions

Simon and Sloman: Emotions as Alarms Process related to the motivational mechanism Simon (1967): Emotions as “interruptions”

• Critical situations might change creature’s goals, causing an interruption

• E.g.: the arousal of a predator, might cause the interruption of an animal current behavior and the generation of new goals, in order to escape from danger

• Interruptions might be caused by an imminent danger or a remarkable opportunity

Sloman• Emotions as attention filters (AFP:Attention Filter Penetration Theory)• Differently from an “interruption”, presuposes a momentaneous

disruption, with the possibility of returning to its past goal• Alarms: triggered by danger or by remarkable opportunities

Models for Emotions

Cañamero and the Modulation of Motivations Emotional System where emotions work as modulators, with the

definition of synthetic hormones affecting homeostatic processes associated to the needs of a creature

Motivational Mechanism:• Modifications in a creature’s body variables determine drives,

triggering an action aiming at the reduction of these drives (still without emotions)

Emotional Mechanism is independent from the Motivational one, but affects it

• Critical situations trigger the release of hormones which cause a modulation in the drives, making some of them more intense while others are attenuated.

Emotions: A Unified View

Independent Motivational and Emotional Systems Motivational System based on Drives AND Impulses Emotions as a result of an evolutionary process, where the

internal motivations of a creature are communicated to other creatures, by means of their expression in bodily modifications

Emotions Are not necessary for the regular working of the motivational

system, but are affected by it Just appear in the evolutionary scale in animals with a limbic

system (birds and mammals) Strategy to communicate emotional states can be useful in social

situations, like collaboration, cooperation and competition

Emotions: A Unified View

Source of confusion between Motivations and Emotions As emotions (in our view) are messages representing the

motivational states, they were misleaded by the own motivational states

And what about the Hedonic States (feelings) representing “Liking” ? They are used as “rewards” (or reinforcements) in the learning of

actions required for drive reduction in the motivational system Our hypothesis is that there might exist different kinds of

rewards, associated to the different creature’s needs

Emotions: A Unified View

Evolution of the Motivational System

Emotions: A Unified View

Evolution of the Motivational System

Conclusions

Summary of our Unified View Model of “Experience” as SDR (Sensors,Drives, Rewards) Experience integrated in the form of percepts embedding SDR Motivated actions learning using (reactive + original) actions plus

rewards (reinforcement learning) Use of affordances for the generation of impulses able to satisfy

the landscape of drives Choice among a repertoire of impulses based on a hierarchy of

needs (unique and dynamic to each creature, differently from Maslow)

Schedule of motivated actions to generate plans Emotions as a mechanism of communication of motivational

states, allowing the appearance of complex social behavior