T12Distributed search and constraint handling

Distributed Constraint Handlingand Optimization

Alessandro Farinelli1 Alex Rogers2 Meritxell Vinyals1

1Computer Science DepartmentUniversity of Verona, Italy

2Agents, Interaction and Complexity GroupSchool of Electronics and Computer Science

University of Southampton,UK

Tutorial EASSS 2012 Valenciahttps://sites.google.com/site/

easss2012optimization/

https://sites.google.com/site/easss2012optimization/

https://sites.google.com/site/easss2012optimization/

Outline

Introduction

Distributed Constraint Reasoning

Applications and Exemplar Problems

Complete algorithms for DCOPs

Approximated Algorithms for DCOPs

Conclusions

Outline

Introduction





Conclusions

Constraints

• Pervade our everyday lives

• Are usually perceived as elements that limit solutions to theproblems we face

Constraints

From a computational point of view, they:

• Reduce the space of possible solutions

• Encode knowledge about the problem at hand

• Are key components for efficiently solving hard problems

Constraint Processing

Many different disciplines deal with hard computational problems thatcan be made tractable by carefully considering the constraints that

define the structure of the problem.

Planning Operational Automated Reasoning ComputerScheduling Research Decision Theory Vision

Constraint Processing in Multi-Agent Systems

Focus on how constraint processing can be used to addressoptimization problems in Multi-Agent Systems (MAS) where:

A set of agents must come to some agreement, typically via someform of negotiation, about which action each agent should take in

order to jointly obtain the best solution for the whole system.

A1 A2 A3

M1 M2

A2

Distributed Constraint Optimization Problems (DCOPs)

We will consider Distributed Constraint Optimization Problems (DCOP)where:

Each agent negotiates locally with just a subset of other agents(usually called neighbors) that are those that can directly influence

his/her behavior.

M1M2

A2

A1 A3

Distributed Constraint Optimization Problems (DCOPs)

After reading this chapter, you will understand:

• The mathematical formulation of a DCOP• The main exact solution techniques for DCOPs

• Key differences, benefits and limitations

• The main approximate solution techniques for DCOPs• Key differences, benefits and limitations

• The quality guarantees these approach provide:• Types of quality guarantees• Frameworks and techniques

Outline

Introduction





Conclusions

Constraint Networks

A constraint network N is formally defined as a tuple 〈X ,D,C〉 where:

• X = {x1, . . . ,xn} is a set of discrete variables;

• D = {D1, . . . ,Dn} is a set of variable domains, which enumerateall possible values of the corresponding variables; and

• C = {C1, . . . ,Cm} is a set of constraints; where a constraint Ci isdefined on a subset of variables Si ⊆ X which comprise thescope of the constraint

• r = |Si| is the arity of the constraint• Two types: hard or soft

Hard constraints

• A hard constraint Chi is a relation Ri that enumerates all the valid

joint assignments of all variables in the scope of the constraint.

Ri ⊆ Di1× . . .×Dir

Ri x j xk

0 11 0

Soft constraints

• A soft constraint Csi is a function Fi that maps every possible joint

assignment of all variables in the scope to a real value.

Fi : Di1× . . .×Dir →ℜ

Fi x j xk

2 0 00 0 10 1 01 1 1

Binary Constraint Networks

• Binary constraint networks are those where:

• Each constraint (soft or hard) is definedover two variables.

• Every constraint network can be mapped toa binary constraint network

• requires the addition of variables andconstraints

• may add complexity to the model

• They can be represented by a constraintgraph

x1

x2 x3

x4

F 1,2 F

1,3

F 2,4

F 1,4

Different objectives, different problems

• Constraint Satisfaction Problem (CSP)

• Objective: find an assignment for all the variables in the networkthat satisfies all constraints.

• Constraint Optimization Problem (COP)

• Objective: find an assignment for all the variables in the networkthat satisfies all constraints and optimizes a global function.

• Global function = aggregation (typically sum) of local functions.F(x) = ∑i Fi(xi)


When operating in adecentralized context:

• a set of agents controlvariables

• agents interact to find asolution to the constraintnetwork

A1

A2

A3

A4


Two types of decentralized problems:

• distributed CSP (DCSP)

• distributed COP (DCOP)

Here, we focus on DCOPs.

Distributed Constraint Optimization Problem (DCOP)

A DCOP consists of a constraint network N = 〈X ,D,C〉 and a set ofagents A = {A1, . . . ,Ak} where each agent:

• controls a subset of the variables Xi ⊆ X• is only aware of constraints that involve variable it controls

• communicates only with its neighbours

Distributed Constraint Optimization Problem (DCOP)

• Agents are assumed to be fully cooperative• Goal: find the assignment that optimizes the global function, not

their local local utilities.

• Solving a COP is NP-Hard and DCOP is as hard as COP.

Motivation

Why distribute?

• Privacy

• Robustness

• Scalability

Outline

Introduction





Conclusions

Real World Applications

Many standard benchmark problems in computer science can bemodeled using the DCOP framework:

• graph coloring

As can many real world applications:

• human-agent organizations (e.g. meeting scheduling)

• sensor networks and robotics (e.g. target tracking)

Outline

Introduction


Applications and Exemplar ProblemsGraph coloringMeeting SchedulingTarget Tracking



Conclusions

Graph coloring

• Popular benchmark

• Simple formulation• Complexity controlled with few parameters:

• Number of available colors• Number of nodes• Density (#nodes/#constraints)

• Many versions of the problem:• CSP, MaxCSP, COP

Graph coloring - CSP

• Nodes can take k colors• Any two adjacent nodes should have different colors

• If it happens this is a conflict

Yes! No!

Graph coloring - Max-CSP

• Minimize the number of conflicts

0 -1

Graph coloring - COP

• Different weights to violated constraints

• Preferences for different colors

0 -2-1

-3

-1

-2

Graph coloring - DCOP

• Each node:• controlled by one agent

• Each agent:• Preferences for different colors• Communicates with its direct neighbours in the graph

-1

-3

-1

-2A1

A2

A3

A4

• A1 and A2 exchangepreferences and conflicts

• A3 and A4 do notcommunicate

Outline

Introduction





Conclusions

Meeting Scheduling

Motivation:

• Privacy

• Robustness

• Scalability

Meeting Scheduling

In large organizations many people, possibly working in differentdepartments, are involved in a number of work meetings.

Meeting Scheduling

People might have various private preferences on meeting start times

Better after 12:00am

Meeting Scheduling

Two meetings that share a participant cannot overlap

Window: 15:00-18:00Duration: 2h

Window: 15:00-17:00Duration: 1h

DCOP formalization for the meeting scheduling problem

• A set of agents representing participants

• A set of variables representing meeting starting times accordingto a participant.

• Hard Constraints:• Starting meeting times across different agents are equal• Meetings for the same agent are non-overlapping.

• Soft Constraints:• Represent agent preferences on meeting starting times.

Objective: find a valid schedule for the meeting while maximizing thesum of individuals’ preferences.

Outline

Introduction





Conclusions

Target Tracking

Motivation:

• Privacy

• Robustness

• Scalability

Target Tracking

A set of sensors tracking a set of targets in order to provide anaccurate estimate of their positions.

T1

T2

T4

T3

Crucial for surveillance and monitoring applications.

Target Tracking

Sensors can have different sensing modalities that impact on theaccuracy of the estimation of the targets’ positions.

T1

T4

T3

T2

MODES

MODES

MODES

MODES

Target Tracking

Collaboration among sensors is crucial to improve systemperformance

T1

T2

T4

T3

DCOP formalization for the target tracking problem

• Agents represent sensors

• Variables encode the different sensing modalities of each sensor• Constraints

• relate to a specific target• represent how sensor modalities impacts on the tracking

performance

• Objective:• Maximize coverage of the environment• Provide accurate estimations of potentially dangerous targets

Outline

Introduction





Conclusions

Complete Algorithms

U Always find an optimal solution

D Exhibit an exponentially increasing coordination overhead

D Very limited scalability on general problems.

Complete Algorithms

• Completely decentralised• Search-based.

• Synchronous: SyncBB, AND/OR search• Asynchronous: ADOPT, NCBB and AFB.

• Dynamic programming.

• Partially decentralised• OptAPO

Next, we focus on completely decentralised algorithms

Decentralised Complete Algorithms

Search-based

• Uses distributed search

• Exchange individual values

• Small messages but. . . exponentially many

Representative: ADOPT [Modiet al., 2005]

Dynamic programming

• Uses distributed inference

• Exchange constraints

• Few messages but. . . exponentially large

Representative: DPOP [Petcuand Faltings, 2005]

Outline

Introduction



Complete algorithms for DCOPsSearch Based: ADOPTDynamic Programming DPOP


Conclusions

ADOPT

ADOPT (Asynchronous Distributed OPTimization) [Modi et al., 2005]:

• Distributed backtrack search using a best-first strategy

• Best value based on local information:

• Lower/upper bound estimates of each possible value of itsvariable

• Backtrack thresholds used to speed up the search of previouslyexplored solutions.

• Termination conditions that check if the bound interval is less thana given valid error bound (0 if optimal)

ADOPT by example

4 variables (4 agents): x1,x2,x3,x4 with D = {0,1}

x1

x2 x3

x4

F 1,2 F

1,3

F 2,4 F 1,

44 identical cost functions

Fi, j xi x j

2 0 00 0 10 1 01 1 1

Goal: find a variable assignment with minimal cost

Solution: x1 = 1, x2 = 0, x3 = 0 and x4 = 1giving total cost 1.

DFS arrangement

• Before executing ADOPT, agents must be arranged in a depthfirst search (DFS) tree.

• DFS trees have been frequently used in optimization becausethey have two interesting properties:

• Agents in different branches of the tree do not share anyconstraints;

• Every constraint network admits a DFS tree.

ADOPT by example

x1

x2 x3

x4

F 1,2 F

1,3

F 2,4 F 1,

4Fi, j xi x j

2 0 00 0 10 1 01 1 1

→

DFS arrangement

A1 (root)

A2 A3

A4

←ch

ild, p

aren

t→

←parent,child

→

←ch

ild, p

aren

t→

Cost functions

The local cost function for an agent Ai (δ (xi)) is the sum of the valuesof constraints involving only higher neighbors in the DFS.

ADOPT by example

A1 δ (x1) = 0

A2δ (x1,x2) = F1,2(x1,x2) A3 δ (x1,x3) = F1,3(x1,x3)

A4

δ (x1,x2,x4) = F1,4(x1,x4)+F2,4(x2,x4)

Initialization

Each agent initially chooses a random value for their variables andinitialize the lower and upper bounds to zero and infinity respectively.

A1x1 = 0,LB = 0,UB = ∞

A2x2 = 0,LB = 0,UB = ∞ A3 x3 = 0,LB = 0,UB = ∞

A4x4 = 0,LB = 0,UB = ∞

ADOPT by example

Value messages are sent by an agent to all its neighbors that arelower in the DFS tree

A1x1 = 0

A2x2 = 0 A3 x3 = 0

A4x4 = 0

x 1=

0←−−−

x1=

0

−−−→

x 2=

0←−−− x 1=

0←−−−

A1 sends three valuemessage to A2, A3 andA4 informing them that itscurrent value is 0.

ADOPT by example

Current Context: a partial variable assignment maintained by eachagent that records the assignment of all higher neighbours in the DFS.

A1

A2c2 : {x1 = 0} A3

c3 : {x1 = 0}

A4

c4 : {x1 = 0,x2 = 0}

• Updated by each VALUEmessage

• If current context is notcompatible with some childcontext, the latter is re-initialized(also the child bound)

ADOPT by example

Each agent Ai sends a cost message to its parent Ap

A1

A2 A3

A4

[0,∞

,c2]

[0,0,c3 ]

[0,0,c

4]

Each cost message reports:

• The minimum lower bound (LB)

• The maximum upper bound (UB)

• The context (ci)

[LB,UP,ci]

Lower bound computation

Each agent evaluates for each possible value of its variable:

• its local cost function with respect to the current context

• adding all the compatible lower bound messages received fromchildren.

Analogous computation for upper bounds

ADOPT by example

Consider the lower bound in the cost message sent by A4:

A1

A2 A3

A4

[0,0,c

4]

• Recall that A4 local cost function is:δ (x1,x2,x4) = F1,4(x1,x4)+F2,4(x2,x4)

• Restricted to the current contextc4 = {(x1 = 0,x2 = 0)}:λ (0,0,x4) = F1,4(0,x4)+F2,4(0,x4).

• For x4 = 0:λ (0,0,0) = F1,4(0,0)+F2,4(0,0) = 2+2 = 4.

• For x4 = 1:λ (0,0,1) = F1,4(0,1)+F2,4(0,1) = 0+0 = 0.

Then the minimum lower bound across variablevalues is LB = 0.

ADOPT by example

Each agent asynchronously chooses the value of its variable thatminimizes its lower bound.

A1

A2x2 = 0→ 1 A3

A4

[0,2,c

2]

x 2=

1

A2 computes for each possible value of itsvariable its local function restricted to thecurrent context c2 = {(x1 = 0)}(λ (0,x2) = F1,2(0,x2)) and adding lowerbound message from A4 (lb).

• For x2 = 0: LB(x2 = 0) = λ (0,x2 =0)+ lb(x2 = 0) = 2+0 = 2.

• For x2 = 1: LB(x2 = 1) = λ (0,x2 =1)+0 = 0+0 = 0.

A2 changes its value to x2 = 1 with LB = 0.

Backtrack thresholds

The search strategy is based on lower bounds

Problem

• Values abandoned before proven to besuboptimal

• Lower/upper bounds only stored for thecurrent context

Solution

• Backtrack thresholds: used to speed upthe search of previously exploredsolutions.

ADOPT by example

A1

x1 = 0→ 1→ 0

A2 A3

A4

A1 changes its value and the context withx1 = 0 is visited again.

• Reconstructing from scratch is inefficient

• Remembering solutions is expensive

Backtrack thresholds

Solution: Backtrack thresholds

• Lower bound previously determined by children

• Polynomial space

• Control backtracking to efficiently search

• Key point: do not change value until LB(currentvalue)> threshold

A child agent will not change its variable value so long as cost is lessthan the backtrack threshold given to it by its parent.

A1LB(x1 = 0) = 1

A2LB(x2 = 0)> 12 ? A3 LB(x3 = 0)> 1

2 ?

A4

t(x 1

=0)

=1 2

t(x1=

0)=

12

Rebalance incorrect threshold

How to correctly subdivide threshold among children?

• Parent distributes the accumulated bound among children• Arbitrarily/Using some heuristics

• Correct subdivision as feedback is received from children• LB < t(CONT EXT )• t(CONT EXT ) = ∑Ci t(CONT EXT )+δ

Backtrack Threshold Computation

A1

A2 A3

A4

(2)

t(x 1

=0)

=1

(1) L

B=

1

(2)12 t(x

1=

0)=

0

• When A1 receives a new lower boundfrom A2 rebalances thresholds

• A1 resends threshold messages to A2and A3

ADOPT extensions

• BnB-ADOPT [Yeoh et al., 2008] reduces computation time byusing depth-first search with branch and bound strategy

• [Ali et al., 2005] suggest the use of preprocessing techniques forguiding ADOPT search and show that this can result in aconsistent increase in performance.

Outline

Introduction



Complete algorithms for DCOPsSearch Based: ADOPTDynamic Programming DPOP


Conclusions

DPOP

DPOP (Dynamic Programming Optimization Protocol) [Petcu andFaltings, 2005]:

• Based on the dynamic programming paradigm.

• Special case of Bucket Tree Elimination Algorithm (BTE)[Dechter, 2003].

DPOP by example

x1

x2 x3

x4

Objective: find assignmentwith maximal value

F 1,2 F

1,3

F 2,4

F2,3F 1,4

Fi, j xi x j2 0 00 0 10 1 01 1 1

=>

DFS arrangement

x1

x2

x3x4

←P

2 ,child→

←ch

ild,P

4→

←P

3 ,child→

←ps

eudo

child

,PP 4→

←P

P3 , pseudochild

→

DPOP phases

Given a DFS tree structure, DPOP runs in two phases:• Util propagation: agents exchange util messages up the tree.

• Aim: aggregate all info so that root agent can choose optimalvalue

• Value propagation: agents exchange value messages down thetree.

• Aim: propagate info so that all agents can make their choice givenchoices of ancestors

Sepi: set of agents preceding Ai in the pseudo-tree order that areconnected with Ai or with a descendant of Ai.

x1 Sep1 = /0

x2 Sep2 = {x1}

x3 Sep3 = {x1,x2}x4Sep4 = {x1,x2}

←P

2 ,c hild→

←ch

ild,P

4→

←P

3 ,child→

←ps

eudo

child

,PP 4→

←P

P3 , pseudochild

→

Util message

The Util message Ui→ j that agent Ai sends to its parent A j can becomputed as:

Ui→ j(Sepi) = maxxi

( ⊗Ak∈Ci

Uk→i⊗⊗

Ap∈Pi∪PPi

Fi,p

)

All incoming messages Shared constraints withSize exponentialfrom children parents/pseudoparentsin Sepi

The ⊗ operator is a join operator that sums up functions with differentbut overlapping scores consistently.

Join operator

F2,4 x2 x42 0 00 0 10 1 01 1 1

AddProject

out x4F1,4 x1 x42 0 00 0 10 1 01 1 1

F1,4⊗F2,4x1 x2 x4

4 0 0 00 0 0 12 0 1 01 0 1 12 1 0 02 1 0 10 1 1 02 1 1 1

max{x4}(F1,4⊗F2,4)

x1 x2 x4

max(4,0)0 0 00 0 1

max(2,1)0 1 00 1 1

max(2,2)1 0 01 0 1

max(0,2)1 1 01 1 1

Complexity exponential to the largest Sepi.Largest Sepi = induced width of the DFS tree ordering used.

A1 (root)

A2max

x2(U3→2⊗U4→2⊗F1,2)

A3

maxx3

(F1,3⊗F2,3)

A4

12 max

x4(F1,4⊗F2,4)

U1→

2←−−−

U 4→2

−−−→

U3→

2

←−−−

U2→1 x10 101 5

U4→2 x1 x24 0 02 0 12 1 02 1 1

U3→2 x1 x24 0 02 0 12 1 02 1 1

Sep2

Sep3

Sep4

Value message

Keeping fixed the value of parent/pseudoparents, finds the value thatmaximizes the computed cost function in the util phase:

x∗i = argmaxxi

(∑

A j∈Ci

U j→i(xi,x∗p)+ ∑A j∈Pi∪PPi

Fi, j(xi,x∗j)

)

where x∗p =⋃

A j∈Pi∪PPi{x∗j} is the set of optimal values for Ai’s parent

and pseudoparents received from Ai’s parent.Propagates this value through children down the tree:

Vi→ j = {xi = x∗i }∪⋃

xs∈Sepi∩Sep j

{xs = x∗s}

A1x∗1 = max

x1U1→2(x1)

A2x∗2 = max

x2(U3→2(x∗1,x2)⊗U4→2(x∗1,x2)⊗F1,2(x∗1,x2))

A3

x∗3 = maxx3

(F1,3(x∗1,x3)⊗F2,3(x∗2,x3))

A4

x∗4 = maxx4

(F1,4(x∗1,x4)⊗F2,4(x∗2,x4))V

1→2

− −→

V 2→4

←−−

V2→

3

−−→

DPOP extensions

• MB-DPOP [Petcu and Faltings, 2007] trades-off message sizeagainst the number of messages.

• A-DPOP trades-off message size against solution quality [Petcuand Faltings, 2005(2)].

Conclusions

• Constraint processing• exploit problem structure to solve hard problems efficiently

• DCOP framework• applies constraint processing to solve decision making problems

in Multi-Agent Systems• increasingly being applied within real world problems.

References I

• [Modi et al., 2005] P. J. Modi, W. Shen, M. Tambe, and M.Yokoo. ADOPT: Asynchronousdistributed constraint optimization with quality guarantees. Artificial Intelligence Jour- nal,(161):149-180, 2005.

• [Yeoh et al., 2008] W. Yeoh, A. Felner, and S. Koenig. BnB-ADOPT: An asynchronousbranch-and-bound DCOP algorithm. In Proceedings of the Seventh International JointConference on Autonomous Agents and Multiagent Systems, pages 591Ð598, 2008.

• [Ali et al., 2005] S. M. Ali, S. Koenig, and M. Tambe. Preprocessing techniques foraccelerating the DCOP algorithm ADOPT. In Proceedings of the Fourth International JointConference on Autonomous Agents and Multiagent Systems, pages 1041Ð1048, 2005.

• [Petcu and Faltings, 2005] A. Petcu and B. Faltings. DPOP: A scalable method formultiagent constraint optimization. In Proceedings of the Nineteenth International JointConference on Arti- ficial Intelligence, pages 266-271, 2005.

• [Dechter, 2003] R. Dechter. Constraint Processing. Morgan Kaufmann, 2003.

References II

• [Petcu and Faltings, 2005(2)] A. Petcu and B. Faltings. A-DPOP: Approximations indistributed optimization. In Principles and Practice of Constraint Programming, pages802-806, 2005.

• [Petcu and Faltings, 2007] A. Petcu and B. Faltings. MB-DPOP: A new memory-boundedalgorithm for distributed optimization. In Proceedings of the Twentieth International JointConfer- ence on Artificial Intelligence, pages 1452-1457, 2007.

• [S. Fitzpatrick and L. Meetrens, 2003] S. Fitzpatrick and L. Meetrens. Distributed SensorNetworks: A multiagent perspective, chapter Distributed coordination through anarchicoptimization, pages 257- 293. Kluwer Academic, 2003.

• [R. T. Maheswaran et al., 2004] R. T. Maheswaran, J. P. Pearce, and M. Tambe.Distributed algorithms for DCOP: A graphical game-based approach. In Proceedings ofthe Seventeenth International Conference on Parallel and Distributed ComputingSystems, pages 432-439, 2004.

Outline

Introduction





Conclusions

Approximate Algorithms: outline

• No guarantees

– DSA-1, MGM-1 (exchange individual assignments)

– Max-Sum (exchange functions)– Max-Sum (exchange functions)

• Off-Line guarantees

– K-optimality and extensions

• On-Line Guarantees

– Bounded max-sum

Why Approximate Algorithms

• Motivations

– Often optimality in practical applications is not achievable

– Fast good enough solutions are all we can have

• Example – Graph coloring• Example – Graph coloring

– Medium size problem (about 20 nodes, three colors per

node)

– Number of states to visit for optimal solution in the worst

case 3^20 = 3 billions of states

• Key problem

– Provides guarantees on solution quality

Exemplar Application: Surveillance

• Event Detection

– Vehicles passing on a road

• Energy Constraints

– Sense/Sleep modes

– Recharge when sleeping– Recharge when sleeping

• Coordination

– Activity can be detected by single sensor

– Roads have different traffic loads

• Aim [Rogers et al. 10]

– Focus on road with more traffic load

time

duty cycle

Heavy traffic road small road

Good Schedule

Bad Schedule

Surveillance demo

Guarantees on solution quality

• Key Concept: bound the optimal solution

– Assume a maximization problem

– optimal solution, a solution

–

– percentage of optimality

• [0,1]

• The higher the better

– approximation ratio

• >= 1

• The lower the better

– is the bound

Types of Guarantees

Accuracy

Accuracy: high alpha

Generality: less use of

instance specific knowledgeBounded Max-

Sum

DaCSA

Instance-specific

Generality

MGM-1,

DSA-1,

Max-Sum

K-optimality

T-optimality

Region Opt.

Instance-generic

No guarantees

Centralized Local Greedy approaches• Greedy local search

– Start from random solution

– Do local changes if global solution improves

– Local: change the value of a subset of variables, usually one

-1-1 -1

-4-1

-1 -1-2

-4

0

0 -20

Centralized Local Greedy approaches

• Problems

– Local minima

– Standard solutions: RandomWalk, Simulated Annealing

-1 -1-1 -1

-2

-1 -1

-1 -1 -1 -1 -1 -1

Distributed Local Greedy approaches

• Local knowledge

• Parallel execution:

– A greedy local move might be harmful/useless

– Need coordination

-1-1-1 -1

-4

-1

-4

-1

-1

0 -2 0 -2 0 -20 -2

Distributed Stochastic Algorithm

• Greedy local search with activation probability to

mitigate issues with parallel executions

• DSA-1: change value of one variable at time

• Initialize agents with a random assignment and

communicate values to neighborscommunicate values to neighbors

• Each agent:

– Generates a random number and execute only if rnd less

than activation probability

– When executing changes value maximizing local gain

– Communicate possible variable change to neighbors

DSA-1: Execution Example

-1

P = 1/4

-1

-1

-1

rnd > ¼ ? rnd > ¼ ? rnd > ¼ ? rnd > ¼ ?

0 -2

DSA-1: discussion

• Extremely “cheap” (computation/communication)

• Good performance in various domains

– e.g. target tracking [Fitzpatrick Meertens 03, Zhang et al. 03],

– Shows an anytime property (not guaranteed)– Shows an anytime property (not guaranteed)

– Benchmarking technique for coordination

• Problems

– Activation probablity must be tuned [Zhang et al. 03]

– No general rule, hard to characterise results across domains

Maximum Gain Message (MGM-1)

• Coordinate to decide who is going to move

– Compute and exchange possible gains

– Agent with maximum (positive) gain executes

• Analysis [Maheswaran et al. 04]• Analysis [Maheswaran et al. 04]

– Empirically, similar to DSA

– More communication (but still linear)

– No Threshold to set

– Guaranteed to be monotonic (Anytime behavior)

MGM-1: Example

-1-1

0 -2-1 -1 0 -2

-1 -1

G = -2

G = 0 G = 2

G = 0

Local greedy approaches

• Exchange local values for variables

– Similar to search based methods (e.g. ADOPT)

• Consider only local information when maximizing

– Values of neighbors– Values of neighbors

• Anytime behaviors

• Could result in very bad solutions

Max-sum

Agents iteratively computes local functions that depend

only on the variable they control

X1 X2

Choose arg max

X4 X3 Shared constraint

All incoming

messages except x2

All incoming

messages

Choose arg max

Factor Graph and GDL• Factor Graph

– [Kschischang, Frey, Loeliger 01]

– Computational framework to represent factored computation

– Bipartite graph, Variable - Factor

)|()|()(),,( XXHXXHXHXXXH ++=

1x 2x

3x

1x 2x

3x

)|( 12 XXH

)( 1XH

)|( 13 XXH

)|()|()(),,( 13121321 XXHXXHXHXXXH ++=

Max-Sum on acyclic graphs• Max-sum Optimal on acyclic

graphs – Different branches are

independent

– Each agent can build a correct estimation of its contribution to the global problem (z functions)

1x 2x

)|( 12 XXH

)( 1XH

global problem (z functions)

• Message equations very similar to Util messages in DPOP– GDL generalizes DPOP [Vinyals

et al. 2010a]

3x

)|( 13 XXH

sum up info from other nodes

local maximization step

(Loopy) Max-sum Performance

• Good performance on loopy networks [Farinelli et al. 08]

– When it converges very good results

• Interesting results when only one cycle [Weiss 00]

– We could remove cycle but pay an exponential price (see DPOP)

– Java Library for max-sum http://code.google.com/p/jmaxsum/– Java Library for max-sum http://code.google.com/p/jmaxsum/

Max-Sum for low power devices

• Low overhead

– Msgs number/size

• Asynchronous computation

– Agents take decisions whenever new messages arrive

• Robust to message loss

Max-sum on hardware

Max-Sum for UAVs

Task Assignment for UAVs [Delle Fave et al 12]

Video Streaming

Interest points

Max-Sum

Task utility

Priority

Task completion

First assigned UAVs reaches task

24

Last assigned UAVs leaves task (consider battery life)

Urgency

Factor Graph Representation

2PDA

1PDA2UAV

U T

1T2X

1UAV

2U 2T

3PDA

1U

3U

3T

1X

UAVs Demo

Quality guarantees for approx.

techniques

• Key area of research

• Address trade-off between guarantees and

computational effort

• Particularly important for many real world applications• Particularly important for many real world applications

– Critical (e.g. Search and rescue)

– Constrained resource (e.g. Embedded devices)

– Dynamic settings

Instance-generic guarantees

Accuracy

Characterise solution quality without

running the algorithm

Bounded Max-

Sum

DaCSA

Instance-specific

Generality

Accuracy

MGM-1,

DSA-1,

Max-Sum

K-optimality

T-optimality

Region Opt.

Instance-generic

No guarantees

K-Optimality framework

• Given a characterization of solution gives bound on

solution quality [Pearce and Tambe 07]

• Characterization of solution: k-optimal

• K-optimal solution:• K-optimal solution:

– Corresponding value of the objective function can not be

improved by changing the assignment of k or less

variables.

K-Optimal solutions

1 1 1 1

1

1

1

1

2-optimal ? No3-optimal ? Yes

0 0 1 0

2

2

0

2

Bounds for K-Optimality

For any DCOP with non-negative rewards [Pearce and Tambe 07]

Number of agents Maximum arity of constraints

K-optimal solution

Binary Network (m=2):

K-Optimality Discussion

• Need algorithms for computing k-optimal solutions

– DSA-1, MGM-1 k=1; DSA-2, MGM-2 k=2 [Maheswaran et al. 04]

– DALO for generic k (and t-optimality) [Kiekintveld et al. 10]

• The higher k the more complex the computation

(exponential)(exponential)

Percentage of Optimal:

• The higher k the better

• The higher the number of

agents the worst

Trade-off between generality and solution

quality

• K-optimality based on worst case analysis

• assuming more knowledge gives much better bounds

• Knowledge on structure [Pearce and Tambe 07]

Trade-off between generality and

solution quality• Knowledge on reward [Bowring et al. 08]

• Beta: ratio of least minimum reward to the maximum

Off-Line Guarantees: Region

Optimality• k-optimality: use size as a criterion for optimality

• t-optimality: use distance to a central agent in the constraint graph

• Region Optimality: define regions based on general criteria (e.g. S-size bounded distance) [Vinyals et al 11]criteria (e.g. S-size bounded distance) [Vinyals et al 11]

• Ack: Meritxell Vinyals

3-size regions

x0 x1 x2 x3

x0

x1 x2

x3

x0

x1 x2

x3

x0

x1 x2

x3

x0

x1 x2

x3

1-distance regions

x0 x1 x2 x3

C regions

Size-Bounded Distance

• Region optimality can explore new regions: s-size bounded distance

• One region per agent, largest t-distance group whose size is less than s

3-size bounded distance

x0 x0than s

• S-Size-bounded distance – C-DALO extension of DALO for general

regions

– Can provides better bounds and keep under control size and number of regions

x1 x2

x3

x0

x1 x2

x3

x1 x2

x3

x0

x1 x2

x3

t=1

t=0 t=1

t=0

Max-Sum and Region Optimality

• Can use region optimality to provide bounds for Max-sum [Vinyals et al 10b]

• Upon convergence Max-Sum is optimal on SLT regions of the graph [Weiss 00]

• Single Loops and Trees (SLT): all groups of agents whose • Single Loops and Trees (SLT): all groups of agents whose vertex induced subgraph contains at most one cycle

x1

x0

x2

x3

x0

x1 x2

x3

x0

x1 x2

x3

x0

x1 x2

x3

x0

x1 x2

x3

Bounds for Max-Sum

• Complete: same as

3-size optimality

• bipartite

• 2D grids

Variable Disjoint Cycles

Very high quality guarantees if smallest cycle is large

Instance-specific guarantees

Characterise solution quality after/while

running the algorithm

Bounded Max-

Sum

DaCSA

Instance-specific

Generality

Accuracy

MGM-1,

DSA-1,

Max-Sum

DaCSA

K-optimality

T-optimality

Region Opt.

Instance-generic

No guarantees

Build Spanning tree

Bounded Max-SumAim: Remove cycles from Factor Graph avoiding

exponential computation/communication (e.g. no junction tree)

Key Idea: solve a relaxed problem instance [Rogers et al.11]

X1 F2 X3 X1 F2 X3

Run Max-SumCompute Bound

X1 X2

X2F1 F3 X2F1 F3

X3

Optimal solution on tree

Factor Graph Annotation

• Compute a weight for

each edge

– maximum possible impact of the variable on the

X1 F2 X3

w21 w23of the variable on the function

X2F1 F3

w21

w11

w12

w22

w23

w33

w32

Factor Graph Modification

X1 F2 X3

w21 w23

• Build a Maximum

Spanning Tree

– Keep higher weights

• Cut remaining

dependencies

– Compute

X2F1 F3

w11

w12

w22 w33

w32

W = w22 + w23

– Compute

• Modify functions

• Compute bound

Results: Random Binary Network

Optimal

Approx.

Lower Bound

Upper Bound

Bound is significant

– Approx. ratio is typically 1.23 (81 %)

Comparison with k-optimal

with knowledge on

reward structure

Much more accurate less general

Discussion

• Discussion with other data-dependent techniques

– BnB-ADOPT [Yeoh et al 09]

• Fix an error bound and execute until the error bound is met

• Worst case computation remains exponential

– ADPOP [Petcu and Faltings 05b]– ADPOP [Petcu and Faltings 05b]

• Can fix message size (and thus computation) or error bound and

leave the other parameter free

• Divide and coordinate [Vinyals et al 10]

– Divide problems among agents and negotiate agreement

by exchanging utility

– Provides anytime quality guarantees

Summary

• Approximation techniques crucial for practical applications:

surveillance, rescue, etc.

• DSA, MGM, Max-Sum heuristic approaches

– Low coordination overhead, acceptable performance

– No guarantees (convergence, solution quality)– No guarantees (convergence, solution quality)

• Instance generic guarantees:

– K-optimality framework

– Loose bounds for large scale systems

• Instance specific guarantees

– Bounded max-sum, ADPOP, BnB-ADOPT

– Performance depend on specific instance

References IDOCPs for MRS• [Delle Fave et al 12] A methodology for deploying the max-sum algorithm and a case study on

unmanned aerial vehicles. In, IAAI 2012

• [Taylor et al. 11] Distributed On-line Multi-Agent Optimization Under Uncertainty: Balancing Exploration and Exploitation, Advances in Complex Systems

MGM • [Maheswaran et al. 04] Distributed Algorithms for DCOP: A Graphical Game-Based Approach,

PDCS-2004

DSADSA• [Fitzpatrick and Meertens 03] Distributed Coordination through Anarchic Optimization,

Distributed Sensor Networks: a multiagent perspective.

• [Zhang et al. 03] A Comparative Study of Distributed Constraint algorithms, Distributed Sensor Networks: a multiagent perspective.

Max-Sum• [Stranders at al 09] Decentralised Coordination of Mobile Sensors Using the Max-Sum

Algorithm, AAAI 09

• [Rogers et al. 10] Self-organising Sensors for Wide Area Surveillance Using the Max-sum Algorithm, LNCS 6090 Self-Organizing Architectures

• [Farinelli et al. 08] Decentralised coordination of low-power embedded devices using the max-sum algorithm, AAMAS 08

References IIInstance-based Approximation• [Yeoh et al. 09] Trading off solution quality for faster computation in DCOP search algorithms,

IJCAI 09

• [Petcu and Faltings 05b] A-DPOP: Approximations in Distributed Optimization, CP 2005

• [Rogers et al. 11] Bounded approximate decentralised coordination via the max-sum algorithm, Artificial Intelligence 2011.

Instance-generic Approximation• [Vinyals et al 10b] Worst-case bounds on the quality of max-product fixed-points, NIPS 10

• [Vinyals et al 11] Quality guarantees for region optimal algorithms, AAMAS 11• [Vinyals et al 11] Quality guarantees for region optimal algorithms, AAMAS 11

• [Pearce and Tambe 07] Quality Guarantees on k-Optimal Solutions for Distributed Constraint Optimization Problems, IJCAI 07

• [Bowring et al. 08] On K-Optimal Distributed Constraint Optimization Algorithms: New Bounds and Algorithms, AAMAS 08

• [Weiss 00] Correctness of local probability propagation in graphical models with loops, Neural Computation

• [Kiekintveld et al. 10] Asynchronous Algorithms for Approximate Distributed Constraint Optimization with Quality Bounds, AAMAS 10

T12Distributed search and constraint handling

Education

constraint soft

hard constraint cih

soft constraint cis

hard problems

hard constraints

set of constraints

hard computational problems

soft constraints