Intelligent Agents Lecture # 2&3

Intelligent AgentsLecture # 2&3

Objectives

2

Agents and environmentsRationalityPEAS (Performance measure, Environment,

Actuators, Sensors)Environment typesAgent types

Agents

3

An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators

Human agent: eyes, ears, and other organs for sensors; legs, mouth, and other body parts for actuators

Robotic agent: cameras and infrared range finders for sensors;various motors for actuators

Software agents or softbots that have some functions as sensors and some functions as actuators. Askjeeves.com is an example of a softbot.

Glossary

4

Percept– agents perceptual inputs

Percept sequence– History of everything the agent has perceived

Agent function– Describes agent’s behaviour – maps any percept to an action

Agent program– Implements agent function

Agents and environments

5

The agent function maps from percept histories to actions:

[f: P* A]The agent program runs on the physical

architecture to produce f agent = architecture + program

Example: Vacuum-cleaner Agent

6

• Environment: square A and B• Percepts: [location and content] e.g. [A, Dirty]• Actions: left, right, suck, and no-op• A simple agent function may be “if the current square is dirty, then suck or move to other square..”

Example: Vacuum-cleaner Agent

Agent Program !!!function REFLEX-VACUUM-

AGENT( [location,status]) returns actionif status = Dirty then return Suckelse if location = A then return Rightelse if location = B then return Left

7

Intelligent Agents

8

The fundamental faculties of intelligence are Acting Sensing Understanding, reasoning, learning

An Intelligent Agent must sense, must act, must be autonomous (to some extent).

It also must be rational.

AI is about building rational agents.

Rational Agent

9

An agent should strive to "do the right thing", based on what it can perceive and the actions it can perform.

What is the right thing?– Causes the agent to be most successful

– Rationality is not the same as Perfection – Rationality maximizes “Expected Performance”– Perfection maximizes “Actual Performance”

How to evaluate agent’s success? Performance Measure evaluates How.

• Performance measure – is a criteria to measure an agent’s behavior

e.g., performance measure of a vacuum-cleaner agent could be amount of dirt cleaned up, amount of time taken, amount of electricity consumed, amount of noise generated, etc

Performance measure according to what is wanted in the environment instead of how the agents should behave.

Rationality – Cont’d

10

What is rational - depends on four things:– The performance measure– The agent’s prior knowledge of the environment– The actions the agent can perform– The agent’s percept sequence to date

A rational agent is:For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and what ever built in knowledge the agent has.

Rational Agents – Cont’d

11

Rationality is distinct from omniscience (all-knowing with infinite knowledge)

Agents can perform actions in order to modify future percepts so as to obtain useful information (information gathering, exploration – an important part of rationality)

An agent is autonomous if its behavior is determined by its own experience (with ability to learn and adapt) – a rational agent should be autonomous ….!

Rational ⇒ exploration, learning, autonomy

Is our vacuum-cleaner agent rational?

12

Keeping in view four aspects to measure rationality:

The performance measure awards one point for each clean square at each time step over 10000 time steps.

The geography of the environment is known.- Clean squares stay clean and sucking cleansthe current square.

Only actions are left, right, suck, and NoOpThe agent correctly perceives its location and

whether that location contains dirt.

What’s your Answer -- Yes/No

Building Rational AgentsPEAS Description to Specify Task Environments

13

To design a rational agent we need to specify a task environment

a problem specification for which the agent is a solution

PEAS: to specify a task environmentP:Performance MeasureE: EnvironmentA: ActuatorsS: Sensors

PEAS: Specifying an automatedtaxi driverPerformance measure: ?Environment: ?Actuators: ?Sensors:?

14

PEAS: Specifying an automatedtaxi driverPerformance measure: safety, speed, legal, comfortable,

maximize profitsEnvironment: ?Actuators: ?Sensors: ?

15

PEAS: Specifying an automatedtaxi driverPerformance measure:safe, fast, legal, comfortable, maximize

profitsEnvironment: roads, other traffic, pedestrians, customersActuators: ?Sensors: ?

16

PEAS: Specifying an automatedtaxi driverPerformance measure: safe, fast, legal, comfortable, maximize

profitsEnvironment: roads, other traffic, pedestrians, customersActuators:steering, accelerator, brake, signal, hornSensors:?

17

PEAS: Specifying an automatedtaxi driverPerformance measure:safe, fast, legal, comfortable, maximize

profitsEnvironment:roads, other traffic, pedestrians, customersActuators:steering, accelerator, brake, signal, hornSensors:cameras, sonar, speedometer, GPS

18

PEAS: Specifying a part picking robot

19

Performance measure: Percentage of parts in correct bins

Environment: Conveyor belt with parts, binsActuators: Jointed arm and handSensors: Camera, joint angle sensors

PEAS: Specifying an interactive English tutor

20

Performance measure: Maximize student's score on test

Environment: Set of studentsActuators: Screen display (exercises,

suggestions, corrections)Sensors: Keyboard

Environment types/Properties of Task Environments

21

Fully observable (vs. partially observable): An agent's sensors give it access to the complete state of the environment at each point in time.

e.g a taxi agent doesn’t has sensor to see what other drivers are doing/thinking ….

Deterministic (vs. stochastic): The next state of the environment is completely determined by the current state and the action executed by the agent. (If the environment is deterministic except for the actions of other agents, then the environment is strategic)

Vacuum worlds is Deterministic while Taxi Driving is Stochastic – as one can exactly predict the behaviour of traffic

Environment types

22

Episodic (vs. sequential): The agent's experience is divided into atomic "episodes" (each episode consists of the agent perceiving and then performing a single action), and the choice of action in each episode depends only on the episode itself.

E.g. an agent sorting defective parts in an assembly line is episodic while a taxi driving agent or a chess playing agent are sequential ….

Static (vs. dynamic): The environment is unchanged while an agent is deliberating. (The environment is semidynamic if the environment

itself does not change with the passage of time but the agent's performance score does)Taxi Driving is Dynamic, Crossword Puzzle solver is static…

Environment types – cont’d

23

Discrete (vs. continuous): A limited number of distinct, clearly defined percepts and actions.

e.g. chess game has finite number of statesTaxi Driving is continuous-state and continuous-time

problem …

Single agent (vs. multiagent): An agent operating by itself in an environment.

An agent solving a crossword puzzle is in a single agent environment

Agent in chess playing is in two-agent environment

Examples

24

• The environment type largely determines the agent design• The real world is (of course) partially observable, stochastic, sequential, dynamic, continuous, multi-agent

Agent types

25

Four basic types in order of increasing generality:

Simple reflex agentsModel-based reflex agentsGoal-based agentsUtility-based agents

– All of these can be turned into learning agents

Simple reflex agents

26

Information comes from sensors - percepts Changes the agents current state of the

world These agents select actions on the basis of current percept

Triggers actions through the effectors The condition-action rules allow the agent to make

connection from percept to actionsif car-in-front-is-braking then brakeif light-becomes-green then move-forwardif intersection-has-stop-sign then stopIf dirty then suck

Simple reflex agentsCharacteristics

Such agents have limited intelligence…Efficient No internal representation for reasoning,

inference. No strategic planning, learning. Are not good for multiple, opposing, goals. Works only if correct decision can be

made on basis of current percept.

27

Simple reflex agentsFunction SIMPLE-REFLEX-AGENT(percept)

returns an actionstatic: rules, a set of condition-action rulesstate←INTERPRET-INPUT(percept)rule←RULE-MATCH(state, rule)action←RULE-ACTION[rule]return action

Will only work if the environment is fully observable otherwise infinite loops may occur.

28

Simple reflex agents

29

Model based reflex agents

30

These agents keep track of part of world it can’t see …..

To tackle partially observable environments.To update its state the agent needs two kinds of

knowledge: 1. how the world evolves independently from the agent; Ex: an overtaking car gets closer with time. 2. how the world is affected by the agent’s actions. Ex: if I turn left, what was to my right is now behind Me.Thus a model based agent works as follows:

information comes from sensors - percepts based on this, the agent changes the current state of the world based on state of the world and knowledge (memory), it triggers

actions through the effectors E.g.

Know about other car location in overtaking scenario for taxi driving agent When agent turns the steering clockwise, car turns right ….

Model based reflex agentsFunction REFLEX-AGENT-WITH-

STATE(percept) returns an actionstatic: rules, a set of condition-action rulesstate, a description of the current world state action, the most recent action.state←UPDATE-STATE(state, action, percept)rule←RULE-MATCH(state, rule)action←RULE-ACTION[rule]return action

31

Model-based reflex agents

32

Goal based agents

33

Current state of environments is not always enough ….e.g at a road junction, it can turn left, right or go straight ..

Correct decision in such cases depends on where taxi is trying to get to ….

Major difference: future is taken into accountCombining goal information with the knowledge of its

actions, the agent can choose those actions that will achieve the goal.

Goal-based Agents are much more flexible in responding to a changing environment; accepting different goals.Such agents work as follows:

information comes from sensors - percepts changes the agents current state of the world based on state of the world and knowledge (memory) and

goals/intentions, it chooses actions and does them through the effectors.

Goal-based agents

34

Utility based agents

35

Goals alone are not always enough to generate quality behaviourseg different action sequences can take the taxi

agent to destination (and achieving thereby the “goal”) – but some may be quicker, safer, economical etc ..

A general performance measure is required to compare different world states

A utility function maps a state (or sequence of states) to a real number to take rational decisions and to specify tradeoffs when:

goals are conflicting – like speed and safetyThere are several goals and none of which can be

achieved with certainty …

Utility-based agents

36

Learning agents

37

A learning agent can be divided into four conceptual components:Learning Element

Responsible for making improvements in Performance elementuses feedback from Critic to determine how performance

element should be modified to do better….Performance Element

Responsible for taking external actionsselecting actions based on percepts

CriticTells the learning element how well agent is doing w.r.t. to

fixed performance standards…Problem Generator

Responsible for suggesting actions that will lead to improved and informative experiences.

….

38

For a taxi driver agent:Performance element consists of collection of

knowledge and procedures for selecting driving actions

The Critic observes the world and passes the information to performance element – e.g. reaction/response of other drivers when the agent takes quick left turn from top lane …!!

Learning element then can formulate a rule to mark a “bad action”…

Problem generator identifies certain areas of behaviour improvement and suggest experiments – trying brakes on different road conditions etc…

The Learning element can make changes in “knowledge” components – by observing pairs of successive states allow an agent to learn (learn from what happens when strong brake is applied on a wet road …)

Learning agents

39

Summary

40

An agent is something that perceives and acts in an environment. The agent function specifies the action taken by the agent in response to any percept sequence.• The performance measure evaluates the behaviour of the agent in the environment. A rational agent acts to maximise the expected value of the performance measure.• Task environments can be fully or partially observable, deterministic or stochastic, episodic or sequential, static or dynamic, discrete or continuous, and single-agent or multiagent

Summary

41

Simplex reflex agents respond directly to percepts, whereas model-based reflex agents maintain internal state to track aspects of the world that are not evident in the current percept.

Goal-based agents act to achieve their goals, and utility-based agents try to maximize their own expected “happiness”.

All agents can improve their performance through learning