Behavior Based Systems Behavior Based Systems Lezione 6.

Behavior Based SystemsBehavior Based Systems

Lezione 6

Properties of BehaviorsProperties of Behaviors Behaviors are feedback controllersBehaviors are feedback controllers Behaviors achieve specific tasks/goals (e.g., avoid-others, find-Behaviors achieve specific tasks/goals (e.g., avoid-others, find-

friend, go-home)friend, go-home) Behaviors are typically executed in parallel/concurrentlyBehaviors are typically executed in parallel/concurrently Behaviors can store state and be used to construct world Behaviors can store state and be used to construct world

models/representationmodels/representation Behaviors can directly connect sensors and effectors (i.e., take inputs Behaviors can directly connect sensors and effectors (i.e., take inputs

from sensors and send outputs to effectors) from sensors and send outputs to effectors) Behaviors can also take inputs from other behaviors and send outputs Behaviors can also take inputs from other behaviors and send outputs

to other behaviors (this allows for building networks)to other behaviors (this allows for building networks) Behaviors are typically higher-level than actions (go-home rather Behaviors are typically higher-level than actions (go-home rather

than turn-left-by-37.5-degrees)than turn-left-by-37.5-degrees) Behaviors are typically closed-loop but extended in timeBehaviors are typically closed-loop but extended in time When assembled into distributed representations, behaviors can be When assembled into distributed representations, behaviors can be

used to look ahead but at a time-scale comparable with the rest of the used to look ahead but at a time-scale comparable with the rest of the behavior-based systembehavior-based system

A Generic Classification of Robot BehaviorsA Generic Classification of Robot Behaviors Exploration/directional behaviors (heading based, wandering)Exploration/directional behaviors (heading based, wandering) Goal-oriented appetitive behaviors (discrete object attractor, Goal-oriented appetitive behaviors (discrete object attractor,

area attractor)area attractor) Aversive/protective behaviors (avoid stationary objects,dodge Aversive/protective behaviors (avoid stationary objects,dodge

moving objects, aggression)moving objects, aggression) Path following behaviors (road following, hallway navigation, Path following behaviors (road following, hallway navigation,

stripe following)stripe following) Postural behaviors (balance, stability)Postural behaviors (balance, stability) Social/cooperative behaviors (sharing, foraging, Social/cooperative behaviors (sharing, foraging,

flocking/herding)flocking/herding) Teleautonomous behaviors (influence, behavioral modification)Teleautonomous behaviors (influence, behavioral modification) Perceptual behaviors (saccades, visual search, ocular reflexes)Perceptual behaviors (saccades, visual search, ocular reflexes) Walking behaviors (gait control)Walking behaviors (gait control) Manipulator-specific behaviors (reaching)Manipulator-specific behaviors (reaching) Gripper/dextrous hand behaviors (grasping, enveloping)Gripper/dextrous hand behaviors (grasping, enveloping)

Key propertiesKey properties

Ability to act in real timeAbility to act in real time Ability to use representations to generate efficient Ability to use representations to generate efficient

(not only reactive) behavior(not only reactive) behavior Ability to use a uniform structure and Ability to use a uniform structure and

representationthroughout the system (so no representationthroughout the system (so no intermediate layer)intermediate layer)

Key Questions of Behavior-based Key Questions of Behavior-based RoboticsRobotics

What are the right behavioral building blocks for What are the right behavioral building blocks for robotic systems?robotic systems?

What really is a primitive behavior?What really is a primitive behavior? How are these behaviors effectively coordinated?How are these behaviors effectively coordinated? How are these behaviors grounded to sensors and How are these behaviors grounded to sensors and

actuators?actuators?

Design ParadigmsDesign Paradigms

Ethological guided/constrainedEthological guided/constrained Situated activitySituated activity Experimentally drivenExperimentally driven

Ethologically Guided/constrained Design Ethologically Guided/constrained Design MethodologyMethodology

Ethologically guided (inspired) design Ethologically guided (inspired) design methodologymethodology We can use and mimic models generated by We can use and mimic models generated by

biological scientists;biological scientists; we are free to take what we need and discard the rest.we are free to take what we need and discard the rest.

Ethologically constrained design methodologyEthologically constrained design methodology We deliberately constrain behavioral models to match We deliberately constrain behavioral models to match

those of the animal under study as well as we can.those of the animal under study as well as we can.

Design MethodologyDesign Methodology

Situated Activity Design MethodologySituated Activity Design Methodology

Situated activity-based design methodologySituated activity-based design methodology Situated activity means that a robot's actions are Situated activity means that a robot's actions are

predicated upon situations in which it finds itself.predicated upon situations in which it finds itself. Hence the perception problem is reduced to Hence the perception problem is reduced to

recognizing what situation(s) the robot is in and then recognizing what situation(s) the robot is in and then choosing one action (out of many) to undertake.choosing one action (out of many) to undertake.

Situated activity-based design Situated activity-based design methodologymethodology

Experimentally Driven Design Experimentally Driven Design MethodologyMethodology

Experimentally driven behaviors are invariably Experimentally driven behaviors are invariably created in a bottom-up manner. The basic created in a bottom-up manner. The basic operating premiseoperating premise is to endow a robot with a limited set of capabilities is to endow a robot with a limited set of capabilities

(competencies),(competencies), run experiments in the real world,run experiments in the real world, see what works, and what does not,see what works, and what does not, debug imperfect behaviors, and thendebug imperfect behaviors, and then add new behaviors iteratively until the overall system add new behaviors iteratively until the overall system

exhibits satisfactory performance.exhibits satisfactory performance. "Do whatever works.." methodology. Very much "Do whatever works.." methodology. Very much

like the software development write, run, crash, like the software development write, run, crash, debug, write,...debug, write,...

Experimentally Driven Design Experimentally Driven Design MethodologyMethodology

Types of Behavior NotationTypes of Behavior Notation

Stimulus-ResponseStimulus-Response Diagrams Functional Notation for BehaviorsDiagrams Functional Notation for Behaviors Finite State Acceptor DiagramsFinite State Acceptor Diagrams

Stimulus-response DiagramsStimulus-response Diagrams Stimulus-Response (SR) diagrams are the most Stimulus-Response (SR) diagrams are the most

intuitive and the least formal method of intuitive and the least formal method of expression.expression.

Stimulus-response Diagram for a Robot Stimulus-response Diagram for a Robot Going to a ClassroomGoing to a Classroom

Functional Notation for BehaviorsFunctional Notation for Behaviors

We can use functional notation to describe We can use functional notation to describe behaviorsbehaviors

b(s) = rb(s) = r where where bb is the behavior, is the behavior, ss the stimulus and the stimulus and rr the the

response.response. In a purely reactive system, time is not an In a purely reactive system, time is not an

argument for the behavior argument for the behavior bb..

Functional ExampleFunctional Example

coordinate-behaviorscoordinate-behaviors [ [ move-to-classroommove-to-classroom ( ( detect-classroom-locationdetect-classroom-location ),), avoid-objectsavoid-objects ( ( detect-objectsdetect-objects ),), dodge-studentsdodge-students ( ( detect-studentsdetect-students ), ), stay-to-right-on-pathstay-to-right-on-path ( ( detect-pathdetect-path ), ), defer-to-eldersdefer-to-elders ( ( detect-eldersdetect-elders ) ) ] = ] = motor-responsemotor-response

Finite State Acceptor DiagramsFinite State Acceptor Diagrams

Finite state diagrams (FSD) or finite state Finite state diagrams (FSD) or finite state acceptor (FSA) diagrams can be used to describe acceptor (FSA) diagrams can be used to describe aggregations and sequences of behaviors (also aggregations and sequences of behaviors (also called as behavior assemblages).called as behavior assemblages).

FSA are best to specify complex behavioral FSA are best to specify complex behavioral control systems where entire sets of primitive control systems where entire sets of primitive behaviors are swapped in and out of execution behaviors are swapped in and out of execution during the accomplishment of some high-level during the accomplishment of some high-level goal.goal.

In some cases formal languages like the In some cases formal languages like the Specification and Description language (SDL) Specification and Description language (SDL) can be used to create/generate code for real can be used to create/generate code for real implementationsimplementations

FSA representation for the classroom FSA representation for the classroom navigational tasknavigational task

Behavioral encodingBehavioral encoding

To encode the behavioral response that the To encode the behavioral response that the stimulus should evoke, we must create a stimulus should evoke, we must create a functional mapping from the stimulus plane to the functional mapping from the stimulus plane to the motor (response) planemotor (response) plane..

A behavior can be expressed as a triple (S,R,A behavior can be expressed as a triple (S,R,ββ ) ) where S denotes the domain of all interpretable where S denotes the domain of all interpretable stimuli. R denotes the range of possible stimuli. R denotes the range of possible responses, Behavior responses, Behavior ββ denotes the mapping denotes the mappingββ: S → R: S → R..

Two components of robot’s motor Two components of robot’s motor responseresponse

We can factor the robot's (motor) response into We can factor the robot's (motor) response into two orthogonal components:two orthogonal components: StrengthStrength::

Denotes the magnitude of the response.Denotes the magnitude of the response. The strength may be entirely independent of the strength of The strength may be entirely independent of the strength of

the stimulus yet modulated by exogenous factors like the stimulus yet modulated by exogenous factors like intentionintention

OrientationOrientation:: Denotes the direction of action for the response.Denotes the direction of action for the response. The realization of this directional component requires The realization of this directional component requires

knowledge of the robot's kinematics.knowledge of the robot's kinematics. It may or may not be dependent on the stimulus's strength.It may or may not be dependent on the stimulus's strength.

The instantaneous response The instantaneous response rr The instantaneous response rThe instantaneous response r (where r R)∈(where r R)∈ of a of a

behaviorbased reactive system can be expressed as a six-behaviorbased reactive system can be expressed as a six-dimensional vector:dimensional vector:

r =[x,y,z,θ,φ,ψ]r =[x,y,z,θ,φ,ψ] , , where the first three components represent where the first three components represent the translational degrees of freedom (x,y,z in three the translational degrees of freedom (x,y,z in three dimensional Cartesian coordinates), and the last three dimensional Cartesian coordinates), and the last three components encode the three rotational degrees of freedom. components encode the three rotational degrees of freedom. The dimensionality is often considerably less than six The dimensionality is often considerably less than six DOFs. 2D-mobile robot hasDOFs. 2D-mobile robot has: : r=[x, y, θ]r=[x, y, θ]..

The response r may be limited by the physical limitations of The response r may be limited by the physical limitations of the robot; the robot is non-holonomic, or has restrictions the the robot; the robot is non-holonomic, or has restrictions the way it can move. The non-holonomic constrains can be dealt way it can move. The non-holonomic constrains can be dealt withwith during the generation of the response orduring the generation of the response or by including them in β orby including them in β or after r has been computed, translating the desired after r has been computed, translating the desired

response to be within limitations.response to be within limitations.

The stimulus domainThe stimulus domain

S consists of the domain of all perceivable stimuli. S consists of the domain of all perceivable stimuli. Each individual stimulus s (s S) is represented as a ∈Each individual stimulus s (s S) is represented as a ∈binary tuple: s = (p,l), where p is a particular type or binary tuple: s = (p,l), where p is a particular type or perceptual class and l is a property strength. If l is perceptual class and l is a property strength. If l is nonzero, the stimulus specified by p is present to some nonzero, the stimulus specified by p is present to some degree.degree.

The presence of a stimulus is necessary but not The presence of a stimulus is necessary but not sufficient to evoke a motor response in a behavior-sufficient to evoke a motor response in a behavior-based robot.based robot.

A threshold value t is defined, above which a response A threshold value t is defined, above which a response is generated. Habituation and inhibition (and other is generated. Habituation and inhibition (and other exogenous factors) might influence the threshold t exogenous factors) might influence the threshold t value.value.

The behavioral mapping The behavioral mapping ββ For each individual active behavior we can formally For each individual active behavior we can formally

establish the mapping between the stimulus and establish the mapping between the stimulus and response range that defines a behavioral functionresponse range that defines a behavioral function β: β β: β (s) → r(s) → r..

β β can be defined arbitrarily, but it must be defined can be defined arbitrarily, but it must be defined over all relevant over all relevant pp in in SS. Where a specific stimulus . Where a specific stimulus threshold, threshold, tt, must be exceeded before a response is , must be exceeded before a response is produced for a specificproduced for a specific s = (p,l) s = (p,l); ; r =[0,0,0,0,0,0], l < t⎛ r =[0,0,0,0,0,0], l < t⎛β: (p,l) β: (p,l) r=some function(p, l),⎨ r=some function(p, l),⎨ otherwise ⎩ otherwise ⎩

The functional mapping between the strength of The functional mapping between the strength of stimulus and the magnitude (strength) and direction stimulus and the magnitude (strength) and direction of robotic motor response defines the design space for of robotic motor response defines the design space for a particular robot behaviora particular robot behavior..

Strength multiplierStrength multiplier

Associated with a particular behavior, b, may be a Associated with a particular behavior, b, may be a scalar gain value g, a strength multiplier. The scalar gain value g, a strength multiplier. The strength multiplier modifies the magnitude of the strength multiplier modifies the magnitude of the overall response r: r' = g r.overall response r: r' = g r.

The gain values are used to compose multiple The gain values are used to compose multiple behaviors by specifying their strengths relative to behaviors by specifying their strengths relative to one another. In the extreme case, g can be used to one another. In the extreme case, g can be used to turn off a behavior by setting it to zero, thus turn off a behavior by setting it to zero, thus reducing r' to 0.reducing r' to 0.

Learning and habituation can change the strength Learning and habituation can change the strength multiplier for a response.multiplier for a response.

Different types of behavioral mappings bDifferent types of behavioral mappings b

The behavioral mappings, b, of stimuli onto The behavioral mappings, b, of stimuli onto responses fall into three general categoriesresponses fall into three general categories:: NullNull: the stimulus produces no motor response.: the stimulus produces no motor response. DiscreteDiscrete: The stimulus produces a response from an : The stimulus produces a response from an

enumerated set of prescribed choices. Typically R is a enumerated set of prescribed choices. Typically R is a bounded set of stereotypical responses enumerated for bounded set of stereotypical responses enumerated for the stimulus domain S and specified by b.the stimulus domain S and specified by b.

ContinuousContinuous: The stimulus domain produces a motor : The stimulus domain produces a motor response that is continuous over R.response that is continuous over R.

Null EncodingNull Encoding

For all s, b, s → 0.For all s, b, s → 0.

A very useful behavior!A very useful behavior!

Discrete EncodingDiscrete Encoding We use the "Situated-action methodology"We use the "Situated-action methodology" One possibility is that the behavioral mapping b One possibility is that the behavioral mapping b

consists of a finite set of (situation,response) pairs. consists of a finite set of (situation,response) pairs. Sensing provides the index into a lookup table for Sensing provides the index into a lookup table for finding the appropriate situation and corresponding finding the appropriate situation and corresponding response.response.

Second possibility is to define the discrete encoding Second possibility is to define the discrete encoding by using rule- based systems. Here b is represented by using rule- based systems. Here b is represented as a collection of IF-THEN rules. The rules take the as a collection of IF-THEN rules. The rules take the general form:general form:

IF <antecedent> THEN <consequent>.IF <antecedent> THEN <consequent>.

If more than one rule applies, conflict resolution must If more than one rule applies, conflict resolution must be used to clarify the situation and to decide what be used to clarify the situation and to decide what rule to userule to use..

Continuous Functional encodingContinuous Functional encoding Continuous response allows a robot to have an Continuous response allows a robot to have an

infinite space of potential reactions to its world. This infinite space of potential reactions to its world. This might be a good thing or bad thing.might be a good thing or bad thing.

A mathematical function transforms the sensory A mathematical function transforms the sensory input into a behavioral reaction.input into a behavioral reaction.

Potential fields method is one of the most common Potential fields method is one of the most common (and promising) methods for implementing (and promising) methods for implementing continuous responsecontinuous response..

Here we borrow ideas from physics: Coulomb's Here we borrow ideas from physics: Coulomb's electrostatic attraction, gravitation fields. These fields electrostatic attraction, gravitation fields. These fields are typically combined through super-positioning to are typically combined through super-positioning to yield a single global fieldyield a single global field..

Potential fields have some problems; local minima Potential fields have some problems; local minima (robots get stuck) and oscillatory behaviors (robots (robots get stuck) and oscillatory behaviors (robots go in a circle)go in a circle)

Continuous Functional encoding (cont.).Continuous Functional encoding (cont.). Navigational templates (NATs) are one way to Navigational templates (NATs) are one way to

generate continuous response.generate continuous response. NATs are defined as arbitrary functions to NATs are defined as arbitrary functions to

characterize navigational space that do not characterize navigational space that do not necessary have any correlation with typical necessary have any correlation with typical potential field methods.potential field methods.

Example: spin-based obstacle treatment; Example: spin-based obstacle treatment; "whenever there is an obstacle we need to avoid, "whenever there is an obstacle we need to avoid, start circumventing the object".start circumventing the object".

By using NATs we can solve some local minima By using NATs we can solve some local minima problems found intraditional potential fields problems found intraditional potential fields methods.methods.