Rutgers CS440, Fall 2003 Lecture 2: Intelligent Agents Reading: AIMA, Ch. 2.

Rutgers CS440, Fall 2003

Lecture 2:

Intelligent Agents

Reading: AIMA, Ch. 2


What is an agent?

• An entity in an environment that perceives it through sensors and acts upon it through actuators.

Modified from "A Kalman Filter Model of the Visual Cortex", by P. Rao, Neural Computation 9(4):721--763, 1997

Environment Agent

Percepts

Actions

sensor

actuator

Agents: human, robot, softbot, thermostat, etc.

Agents act on environment to achieve a goal.


Agent function & program

• Agent’s choice of action is based on a sequence of percepts• Agent is specified by an agent function f that maps sequences

of percepts Y to actions a:

• Agent program implements agent function on a physical architecture

• “Easy” solution: table that maps every possible sequence Y to an action a

• Problem: not feasible

aYf

aaaA

yyyY

T

T

:

},...,,{

},...,,{

10

10


Example: Vacuum-cleaner world

• Percepts: location and contents, e.g., (A,dirty)• Actions:move, clean, do nothing:

LEFT, RIGHT, SUCK, NOP

A B


Vacuum-cleaner world: agent function

• What is the right function?• Can the function be implemented in a “short” program?


The “right” agent function – rational behavior

• Rational agent is the one that does the “right thing”: functional table is filled out correctly

• What is the “right thing”?• Define success through a performance measure, r• Vacuum-cleaner world:

– +1 point for each clean square in time T– +1 point for clean square, -1 for each move– -1000 for more than k dirty squares

• Rational agent:An agent who selects an action that is expected to maximize the performance measure for a given percept sequence and its built-in knowledge

• Ideal agent: maximizes actual performance, but needs to be omniscient. Impossible!

• Builds a model of environment.


Properties of a rational agent

• Maximize expected performance• Gathers information – does actions to modify future percepts• Explores – in unknown environments• Learns – from what it has perceived so far

(dung beetle, sphex wasp)• Autonomous – increase its knowledge by learning


Task environment

• To design a rational agent we need to specify a task environment = problem to which the agent is a solution

• P.E.A.S. = Performance measureEnvironmentActuatorsSensors

• Example: automated taxi driver

• Performance measure: safe, fast, legal, comfortable, maximize profits

• Environment: roads, other traffic, pedestrians, customers• Actuators: steering, accelerator, brake, signal, horn• Sensors: cameras, sonar, speedometer, GPS


More PEAS examples

• College test-taker• Internet shopping agent• Mars lander• The president• …


Properties of task environments

Solitaire Backgammon Internet shopping

Taxi

Observable (hidden)

Deterministic (stochastic)

Episodic (sequential)

Static (Dynamic)

Discrete (Continuous)

Single-agent (multi-agent)


Properties of task environments (cont’d)


Taxi

Observable (hidden)

Yes Yes No No



Static (Dynamic)






Taxi

Observable (hidden)

Yes Yes No No


Yes No Partly No


Static (Dynamic)






Taxi

Observable (hidden)

Yes Yes No No


Yes No Partly No


No No No No

Static (Dynamic)






Taxi

Observable (hidden)

Yes Yes No No


Yes No Partly No


No No No No

Static (Dynamic)

Yes Semi Semi No






Taxi

Observable (hidden)

Yes Yes No No


Yes No Partly No


No No No No

Static (Dynamic)

Yes Semi Semi No


Yes Yes Yes No





Taxi

Observable (hidden)

Yes Yes No No


Yes No Partly No


No No No No

Static (Dynamic)

Yes Semi Semi No


Yes Yes Yes No


Yes No No No


Structure of agents

• Goal of AI: give task environment, construct agent function, and design an agent program that implements agent function on a particular architecture

• Skeleton agent:

function SKELETON-AGENT(perceptt)

returns actiont

static: state, the agent’s memory of the world

statet = Update-State(statet-1,…,perceptt,actiont-1)

actiont = Choose-Best-Action(statet)

statet = Update-Memory(statet,actiont)

return action

ytat

st


Skeleton agent

• Graphical depiction (we will see more of it later in the semester)

at

st

yt

at+1

st+1

yt+1

… …


Agent types

• Simplest agent:Table-driven agent: for each percept sequence Y, has a table entry with associated action

function TABLE-DRIVEN-AGENT( percept )

returns action

static: sequence or percepts

percepts = Update-Percepts( percept )

action = Table( percepts )

return action

• Four basic types, in order of increasing complexity1. Simple reflex agent

2. Model-based reflex agent (reflex agent with state)

3. Goal-based agent

4. Utility-driven agent


Simple reflex agent

function REFLEX_VACUUM_AGENT( percept )

returns action

(location,status) = UPDATE_STATE( percept )

if status = DIRTY then action = SUCK;

else if location = A then return RIGHT;

else if location = B then return LEFT;

return action


Model-based reflex agent


Goal-driven agent


Utility-based agent


Learning agent

Any other agent

Rutgers CS440, Fall 2003 Lecture 2: Intelligent Agents Reading: AIMA, Ch. 2.

Documents

agent program

rutgers cs440

sequence of percepts

agent function f

dirty squares rational

learning slide

gps slide

feasible slide