Outline Special Topics in AI: Intelligent Agents and Multi ...profs.sci.univr.it/~farinelli/courses/ddrMAS/slides/ddr-intro.pdf · Multi-Agent Systems – Decentralized Coordination,

17/10/2012

1

Special Topics in AI: Intelligent

Agents and Multi-Agent Systems

Alessandro Farinelli

Course Presentation and Introduction

Outline

• Course Presentation

– Aims, schedule, exam modalities

• Intelligent agents

– AI, Intelligent agents, Rationality

• Multi-Agent Systems

– Main features, techniques, applications

Lecture Material

Artificial Intelligence – A Modern Approach

by Stuart Russell - Peter Norvig

Lecture slides and Info:

An Introduction to Multiagent Systems

by Michael Wooldridge

Multiagent Systems. 2nd Edition.

Gherard Weiss (Ed.)

Course Organization

Wed 17th Oct. 15:30 -- 17:30 Room M;

Tue 23rd Oct. 15:30 -- 17:30; Room H

Tue 30th Oct. 15:30 -- 17:30; Room H

Mon. 5th Nov. 16:00 -- 18:00; Sala Verde

Tue. 13th Nov. 15:30 -- 17:30; Room H

Tue. 20th Nov. 15:30 -- 17:30; Room H

Tue. 27th Nov. 15:30 -- 17:30; Room H

Tue. 4th Dec. 15:30 -- 17:30; Room H

Tue. 11th Dec. 15:30 -- 17:30; Room H

Tue. 18th Dec. 15:30 -- 17:30; Room H

17/10/2012

2

Course Aim

At the end of this course will be able to:

1. Understand main issues and research challenges for

Multi-Agent Systems

– Decentralized Coordination, Market Based Allocation,

Reasoning under uncertainty

2. Model and solve Decentralized Coordination problems

– DCOPs (exact and approx. methods)

3. Understand main models and solution techniques for

decision making under uncertainty

– MDP, POMDPs, Dec-MDPs

Course Program

1. Decentralized Coordination

– Modeling Decentralized Coordination as DCOPs

– DCOPs solution techniques (exact and approx.)

2. Market Based Allocation

– Auction Mechanisms, Combinatorial auctions, Sequential

auctions

3. Reasoning under uncertainty

– MDPs, POMDPs

– Probabilistic approaches for robot navigation

Exam modalities

• Students read, present to the class, and discuss a set of

selected papers.

• Student together with instructor choose papers

– Topics: Decentralized optimization, Market-Based Allocation,

Reasoning under uncertainty (robotics)

• Presentation:

– From 45mins to 1 hour + questions

– During the last three lessons (4th 11th 18th Dec.)

Outline

• Course Presentation

– Aims, schedule, exam modalities

• Intelligent agents

– AI, Intelligent agents, Rationality

• Multi-Agent Systems

– Main features, techniques, applications

17/10/2012

3

What is AI? Acting Humanly:Turing Test

Turing(1950) Computing Machinery and Intelligence

• Can machine think? � Can machine behave like humans?

• Operational test: the imitation game

Problem: not reproducible, constructive or amenable to

mathematical analysis

Thinking humanly: Cognitive

Science• Cognitive Neuroscience � theories of internal

activities of the brains

– Level of abstraction? Validation ?

• Available theories do not explain human-level

intelligence

Thinking rationally: Laws of

thoughts• Normative not descriptive

• Problems:

– Intelligence not always based on logical deliberation

– What are the purpose of thinking ? Which thoughts should

I have out of all the ones that I could have

17/10/2012

4

Acting rationally

• Do the right thing

– Action that maximizes some measure of performances

given current information

• Thinking should be in service of rational actions

– Thinking is not necessary (e.g., blinking reflex)

• Correct thinking (inference) does not always result in

rational actions

– Thinking is not sufficient

Rational agents

• Agent: entity that perceives and acts

• Rational agent

– A function from percept histories to actions

– For a given class of environments and tasks we seek the

agent with best performance (optimization problem)

Agents and Environments

• Agents: humans, softbots, thermostats, robots, etc.

• Agent function: maps perception histories to actions

• Agent program: implements the agent function on

the physical architecture

Rationality

• Given a performance measure for environment

sequences

• Rational agent: chooses actions that maximizes the

expected value given percept sequence

• RaDonal ≠ omniscient

– Perception may not supply all relevant info

• RaDonal ≠ clairvoyant

– Action outcome might be unexpected

• Hence RaDonal ≠ successful

• Rational => exploration, learning, autonomy,…

•

17/10/2012

5

Agent Types: Simple reflex Agent Agent Types: Goal-Based agents

Agent Types: Utility-Based Agent AI (recent) history

17/10/2012

6

AI Exciting Applications• Game Playing

– IBM’s Deep Blue (1997)

– Poker (Now) http://webdocs.cs.ualberta.ca/~games/poker/

• Autonomous Control

– Google self driving car

http://www.ted.com/talks/sebastian_thrun_google_s_driverle

ss_car.html

• Search and Recue/hostile environments

– RoboCup Rescue (http://www.robocuprescue.org/ )

• Human Agent Collectives

– Orchid project (http://www.orchid.ac.uk/project-aims/)

Example: Search and Rescue

LabRoCoCo http://labrococo.dis.uniroma1.it/wiki/doku.php

Outline

• Course Presentation

– Aims, schedule, exam modalities

• Intelligent agents

– AI, Intelligent agents, Rationality

• Multi-Agent Systems

– Main features, techniques, applications

Intelligent Agents

• Intelligent Agents: rational agent +

– Reactivity

– Pro-activeness

– Social ability � Multi-Agent systems

Rational Agent Intelligent Agent

17/10/2012

7

Multi-Agent Systems

• (Durfee and Lesser 1989): “loosely coupled network of

problem solvers that interact to solve problems that

are beyond the individual capabilities or knowledge

of each problem solver “

• Problem solvers: Intelligent agents

• (John Gage, Sun Microsystems)

“The network is the computer”

MAS Characteristics

(K. P. Sycara 1998)

1. Each agent has incomplete information or

capabilities for solving the problem and, thus, has

a limited viewpoint

2. There is no system global control

3. Data is decentralized

4. Computation is asynchronous

Example: cooperative foraging Why MAS?

• To solve problem that are too large for a single agent

– Problem decomposition

• To Avoid single point of failure in critical applications

– Disaster mitigation/urban search and rescue

• To model problem that are naturally described with

collectives of autonomous components

– Meeting scheduling, Traffic control, Forming coalition of

customers, …

17/10/2012

8

Main Research Areas in MAS

MAS

Coordination• Graphical models

SDM

• MDPs

KR

• ATL

Game Theory

• core stability

Applications of MAS I: Games, entertainment and education

Real Time Strategy (e.g. Starcraft, Age of Empires)

� group formation, task assignment, strategic planning

First Person Shooter (e.g. Half Life 2, Splinter Cell)

�character interactions

Applications of MAS II: Search and Rescue

UAVs cooperative image collection

Cooperative information gathering

Cooperative information gathering

Joint work with Stranders, Rogers, Jennings [IJCAI 09]

Limited

communication

17/10/2012

9

CIG: the model

• Monitor a spatial phenomena

• Model: scalar field

– Two spatial dimensions

– One temporal dimension

CIG: goal

• Minimise prediction

uncertainty

• Given a measure here

what is my uncertainty

over there

• Tools:

– Gaussian process

• Estimate uncertainty

– Entropy

• Measure information

Predictive

Uncertainty

Contours

Measures

CIG: Performance measure and

interactions

)( 1XH )|(12

XXH ),|(321

XXXH

1U2U 3U

),|()|()(),,(213121321

XXXHXXHXHXXXH ++=

∑=i

U1U 3U2U ++

CIG: Demo

17/10/2012

10

Cooperative Image Collection

Task Assignment for UAVs

Joint work with:Delle Fave, Rogers, Jennings

Interest points

Video Streaming

Coordination

CIC: Task utility

First assigned UAVs reaches task

38

Last assigned UAVs leaves task (consider battery life)

Priority

Urgency

Task completion

CIC: Interactions

2PDA

1UAV

1PDA

2UAV

2U 2

T

1T

3PDA

1U

3U

3T

1X

2X

CIC: UAVs Demo

17/10/2012

11

Applications of MAS III: Energy management

Mechanism design and Energy

trading

• Force demand to follow supply

Home Energy management

• Agents to decide load

scheduling and storage

Collective energy trading

• Buy and sell energy as

collectives

Intelligent agents for the smart grid

Electricity markets

Baseload: Carried by baseload stations with low cost generation, efficiency and safety

Baseload

Electricity markets Electricity markets

PeakLoad

Peakload: Carried by expensive, carbon-intensive peaking plants generators

17/10/2012

12

Electricity group purchasing

• Allow group purchasing among electricity

consumers

• Very popular successful cases

– Groupon, Groupalia

– UK Labour party initiative on collective electricity

purchase

Electricity Group Purchasing

+

• Virtual Electricity Consumer (VEC): A group of consumers that act in the market as a single energy consumer.

Group synergies

• Traditional group purchasing based on group size

• Group synergy: complementary energy

restrictions

• Flattened demand => Better prices

Social networks

• Social networks to support the

VEC formation and

management

• Look for potential partners

through its contacts

• VECs of friends of friends

17/10/2012

13

VEC formation as a coalitional

game

$ $$• Consumers are selfish

• Coalitional game:

–Agents join a coalition if

this is in their best interest

Challenges to address

• How do we evaluate a VEC

• How do we build feasible coalitions

• How do we form optimal and stable coalitions

Coalitional value metric

• Given an energy coalition:

• computes the total estimated payment

• optimizes the buying strategy among energy markets

Coalitional value metric

Solves a linear program for a coalition S:

Minimize

Subject toDay-aheadmarket price

Forwardmarket price

t-slot day-headquantity

forward quantity

Expected demand at slot time t

17/10/2012

14

Coalitional value metric

Ratio between markets

Forward marketquantity

Energy coalitions can buy a continuous amount even

when they are not expected to use it all hours of the day

Challenges to address

• How do we evaluate a VEC

• How do we build feasible coalitions

• How do we form optimal and stable coalitions

Coalition enumeration

• Feasible coalitions are restricted by the social

network graph

• Enumerate all connected sub-graphs

[Gutin et al 2008]

A

B C

A,B,C

A,B

A,C

B,C

Challenges to address

• How do we evaluate a VEC

• How do we build feasible coalitions

• How do we form optimal and stable coalitions

17/10/2012

15

Coalition Structure Generation

• Aim: identify the set of non-overlapping

coalitions with maximal value

• NP-Hard

• Binary integer problem formulation (IP)

A

B C

C = -1A,B = -6 > A,B,C=-8

Core-Stable Payoff Distribution

• Find core-stable payments

• agents have no economical incentive to

deviate from optimal coalitions

• Given optimal coalitions use LP formulation

A

B C

C = -1

A,B = -6-2.5, -3.5, -1 = -7> A,B,C= -8

-2.5

-3.5 -1

-2.5 > A=-4

-3.5 > B=-4

-1 > C=-4

Empirical evaluation

• Real energy profiles from houses in UK

• Energy consumption averaged over a month

• 20 agents

• Analyze average user gain and coalition structure:

• network structure (Random, Scalefree and Small-World)

• # friends acquaintances (#edges/#nodes)

• Different market conditions

User gain and stability

• Lower forward-market price => higher gain

• Higher network density => slightly higher gain, many unstable coalitions

• Similar considerations for small-world

17/10/2012

16

Structure of coalitions

• In M1,M2 coalitions of middle size for all configurations

• In M3 much larger

coalitions

• Coalition structure very sensitive to market prices

M1

M3

Scale Free

Conclusions

Intelligent Agents and MAS:

• “the network is the computer”

• Highly interdisciplinary fields

• Strong focus on building systems

• Many exciting applications