Ant Colony Optimization - SUNY Polytechnic Instituteweb.cs.sunyit.edu/.../cs548SwarmIntelligence/ant_optimization.pdfLIACS Natural Computing Group Leiden University Ant Colony Optimization

LIACS Natural Computing Group Leiden University

Ant Colony Optimization

Slides largely based on:

Marco Dorigo and Thomas Stützle: Ant Colony

Optimization. MIT Press, Cambridge, MA, 2004.

LIACS Natural Computing Group Leiden University2

Ants (Formicidea)

• Exist already 110-130 million years

• Approximately 12,000 species exist today

• Exist on almost every landmass on earth

• Ants are social insects and form colonies

• The cooperation of ants in colonies makes them act as a

superorganism, producing phenomena governed by the

collective

A very successful family of creatures



The Social Phenomena of Ants

• Division of labor

Queens, drones, workers, soldiers

• Brood sorting

• Cooperative transport

• Foraging

• Nest defense

Interaction between simple agents causes complex behavior


Carpenter ant castes. From left to

right: queen, winged male, major

worker, minor worker

Interaction / Communication

• Ants cannot see, talk, or listen very well

• Interaction among ants or between ants and the environment is based on pheromones

• Pheromones: chemical factors that trigger a social response in members of the same species

• Indirect communication mediated by

modifications of the environment stigmergy


Ant Algorithms

• Models derived from the observation of real ants behavior

• Using the self-organizing principles that allow coordinated behavior of real ants in populations of artificial agents to solve computational problems

• Several aspects serve as inspiration:– Foraging

– Division of labor

– Brood sorting

– Cooperative transport

LIACS Natural Computing Group Leiden University5 LIACS Natural Computing Group Leiden University5


Ant Colony Optimization (ACO)

• Inspired by the foraging behavior of ant colonies

• Introduced by Marco Dorigo in the early 1990s, in the context of finding shortest paths in graphs

• Used for solving combinatorial optimization problems within a reasonable amounts of computation time, especially graph-based problems


The Foraging Behavior of Ants

The Double Bridge Experiments


The Foraging Behavior of Ants

• The foraging behavior of ants is based on indirect

communication mediated by trail pheromones

• While walking from food sources to the nest and vice

versa, ants deposit pheromones on the ground, forming

pheromone trails

• Ants can smell the pheromone and tend to choose,

probabilistically, paths marked by strong pheromone

concentrations



Double Bridge Experiments


J.L. Deneubourg, S. Aron, S. Goss and J.M. Pasteels (1990). The self-organizing exploratory pattern of the Argentine ant. Journal of Insect Behaviour, 3, 159-168.

S. Goss, S. Aron, J.L. Deneubourg and J.M. Pasteels (1989). Self-organized shortcuts of the Argentine ant. Naturwissenschaften, 76, 579-581

• A study on the pheromone trail-laying and –following behavior of ants

• A double bridge connects a nest of ants and a food source

• The ratio r = Llong / Lshort between the length of the two branches of the double bridge is varied

• Ants are free to move between the nest and the food

Double Bridge Experiments (1)


• Although in the initial phase random choices occurred,

eventually almost all the ants used the same branch

• Autocatalytic or positive feedback process is an example of

a self-organizing behaviour of the ants



• In most of the trials, after some time almost all the ants selected

the short branch

• Short branch

• Not all the ant use the short branch, but a small percentage may

take the longer one path exploration



• The short branch was only selected sporadically and the

colony was trapped on the long branch

• High pheromone concentration on the long branch and slow

evaporation of pheromone


The Foraging Behavior in a Nutshell

• Ants initially explore the area surrounding their nest randomly

• As soon as an ant finds a food source, it carries some of it back to the nest, depositing pheromones on its way back

• When choosing their way, ants prefer tend to follow strong pheromone concentrations, the pheromone trail will guide other ants to the food source

• The other ants will also find the food source, therewith reinforcing the pheromone trail

• Pheromones defuse over time, so when a food source is exhausted, the trail will no longer be reinforced and dissipates


Computer Simulated Foraging

Source:

• A 2D world consisting of a nest, a food

source and ants

• The ants move randomly

• When an ant stumbles upon the food

source, it

– Picks up some of it

– Goes back to the nest in a straight line

– Drops pheromones on its way back

• The pheromones slowly dissipates when

it is not reinforced


http://website.lineone.net/~john.montgomery/demos/ants.html




Artificial Ants on Graphs


Toward Artificial Ants

• The double bridge experiments show that ant colonies have a built-in optimization capability

• Three reasons why ants find the shortest path:– Earlier pheromone (the trail is completed earlier)

– More pheromone (higher ant density)

– Younger pheromone (less diffusion)

• What about setting out artificial ants on graphs and letting them solve shortest path problems?



Double Bridge Graph Model


• Node 1 and 2 represent the nest and the food respectively

• Assume the time to be discrete (t = 1, 2, ...)

• At each time step, each ant moves toward to a neighboring node

• Ants add pheromone to the visited arcs proportionally to the length

of those arcs

• Ants move on the graph by choosing the path probabilistically

FoodNest1 2s

l


Artificial Ants and Minimum Cost Paths


• A static and connected graph G = (N, A), N: nodes A: arcs

• Establish a minimum cost path between source and destination


A Simple Ant Colony Optimization

Algorithm


Simple ACO: Basics (1)

• Probabilistic forward ants and solution construction

– Forward ants build a solution by choosing probabilistically the next node to move to among those in the neighborhood of the graph node on which they are located

– Forward ants do not deposit any pheromone while moving

• Deterministic backward ants and pheromone update

– Backward ants use explicit memory to retrace the path

– Loop elimination

– Update the pheromones of the edges of the path that was taken based on the length of that path



Simple ACO: Basics (2)

• Pheromone updates based on solution quality

– Ants memorize the nodes they visited during the forward path, as well as the cost of the arcs traversed if the graph is weighted

– Ants evaluate the cost of the solutions they generated and use this to modulate the amount of pheromone they deposit while in backward mode

• Pheromone evaporation

– Simulated by applying an appropriately defined pheromone evaporation rule



Simple ACO: Main Loop

initialize pheromones

for each iteration do

for k = 1 to number of ants do

set out ant k at start node

while ant k has not build a solution do

choose the next node of the path

end while

end for

update pheromones

end for

return best solution found

22 LIACS Natural Computing Group Leiden University


Simple ACO: Tour Construction

• When ant k is located at a node vi the probability pjk of

choosing vj as the next node is:

With:

• Ni : the set of nodes that ant k can reach from vi (tabu list)

• ij : the amount of pheromone on edge (i,j)


i

i

Nm im

ij

k

Nj

Njp

ij

if0

if


Simple ACO: Pheromone Update (1)

The pheromone on each edge is updated as:

m

k

k

ij ij

0

ijijij )1(


With:

• : the evaporation rate of the old pheromone

• ∆ij : the new pheromone that is deposited by all ants on

edge (i,j) calculated as:


Simple ACO: Pheromone Update (2)


otherwise

TjiifLQ kkk

ij0

),(/

The pheromone that is deposited on edge (i,j) by ant k is

calculated as:

With:

• Q : a heuristic parameter

• Tk : the path traversed by ant k

• Lk : the length Tk calculated as the sum of the lengths of all

the edges of Tk


Example – Ants Walk (1)

(6,0.5)

1

6

4

2

35

(2,0.6)

(5,0.7)

(1,0.4)

(2,0.3)

(2,0.3)

(2,0.4)

(5,0.5)

(9,0.3)

(7,0.6)

Ant Path Path length

1

2

3

Edge lengthPheromones



(6,0.5)

1

6

4

2

35

(2,0.6)

(5,0.7)

(1,0.4)

(2,0.3)

(2,0.3)

(2,0.4)

(5,0.5)

(9,0.3)

(7,0.6)


1 {1,2,4,5,6} 6+2+5+2=15

2

3




(6,0.5)

1

6

4

2

35

(2,0.6)

(5,0.7)

(1,0.4)

(2,0.3)

(2,0.3)

(2,0.4)

(5,0.5)

(9,0.3)

(7,0.6)


1 {1,2,4,5,6} 6+2+5+2=15

2 {1,3,4,2,6} 6+2+1+9=18

3




(6,0.5)

1

6

4

2

35

(2,0.6)

(5,0.7)

(1,0.4)

(2,0.3)

(2,0.3)

(2,0.4)

(5,0.5)

(9,0.3)

(7,0.6)


1 {1,2,4,5,6} 6+2+5+2=15

2 {1,3,4,2,6} 6+2+1+9=18

3 {1,2,3,4,6} 2+5+2+7=16



Example – Pheromones Update

old ∆ij1 ∆ij

2 ∆ij3 ∆ij new

(1,2) 0.6 1/15 0 1/16 1/15 + 1/16 ≈ 0.129 0.6 * 0.9 + 0.129 = 0.669

(1,3) 0.5 0 1/18 0 1/18 ≈ 0.055 0.5 * 0.9 + 0.055 = 0.505

(2,3) 0.7 0 0 1/16 1/16 ≈ 0.063 0.7 * 0.9 + 0.063 = 0.693

(2,4) 0.4 1/15 1/18 0 1/15 + 1/18 ≈ 0.122 0.4 * 0.9 + 0.122 = 0.482

(2,6) 0.3 0 1/18 0 1/18 ≈ 0.055 0.3 * 0.9 + 0.055 = 0.325

(3,4) 0.3 0 1/18 1/16 1/18 + 1/16 ≈ 0.118 0.3 * 0.9 + 0.118 = 0.388

(3,5) 0.3 0 0 0 0 0.3 * 0.9 + 0 = 0.27

(4,5) 0.5 1/15 0 0 1/15 ≈ 0.067 0.5 * 0.9 + 0.067 = 0.517

(4,6) 0.6 0 0 1/16 1/16 ≈ 0.063 0.6 * 0.9 + 0.063 = 0.603

(5,6) 0.4 1/15 0 0 1/15 ≈ 0.067 0.4 * 0.9 + 0.067 = 0.427

Q = 1, = 0.1


Example – Reset for Next Iteration

(6,0.505)

1

6

4

2

35

(2,0. 669)

(5,0.693)(1,0.482)

(2,0.388)

(2,0.27)

(2,0.427)

(5,0.517)

(9,0.325)

(7,0.603)

Pheromones are updated and a new iteration can begin


Simple ACO: Experiments


• No evaporation leads to very poor performance

• Too much evaporation will prevent ants from learning at all

• Evaporation rate controls the balance exploration vs exploitation

ρ = 0

ρ = 0.1

ρ = 0.01

Ant System

The first real ACO algorithm for TSP



Ant System (AS)

• The first ACO algorithm proposed by Dorigo et al. in 1991

• Originally intended for TSP-like problems

• Encouraging initial results, but inferior to state-of-the-art

procedure Ant System for TSP

Pheromone Initialization

while (not terminate) do

for i = 1 to k do

Tour Construction

end

Update Pheromones

end

end


Ant System – Tour Construction

• When ant k is located at a node vi the probability pjk of choosing vj as

the next node is:

With:

• Ni : the set of nodes that ant k can reach from vi (tabu list)

• ij : the heuristical desirability for choosing edge (i,j)


• and : relative influence of heuristics vs. pheromone

i

i

Nm imim

ijij

k

Nj

Njp

ij

if0

if


Ant System - Pheromone Update

Pheromone update for each edge:

• : the evaporation rate of the „old‟ pheromone (0 < ≤ 1)

• ∆ijk : the pheromone deposited on edge (i,j) by ant k is calculated as:

otherwise0

),( if/1 k

k

k

ij

TjiC

• Tk : the path traversed by ant k

• Ck : the length of Tk (the sum of the lengths of all the edges of Tk)

k

ij

m

k

ijij

1

)1(


Ant System - Pheromone Initialization

• A good heuristic for pheromone initialization:

– Slightly higher than the expected amount of pheromone deposited

by the ants in one iteration

– A good setting:

m : the number of ants

Cnn: the length of a “reasonable tour” (e.g., a tour generated by the

nearest-neighbor heuristic)

• Heuristic desirability: ηij = 1 / dij, where dij is the length of edge (i,j)

• Common settings for α and β: α = 1, β = 5

nn

ij Cmji /),,( 0


The General ACO Metaheuristic


ACO general framework

Initialize pheromones

while termination conditions not met do

Construct ant solutions based on the pheromones

Update pheromones

Perform daemon actions (optional)

end while



Solution Construction (1)

• Start with an empty partial solution ps = Ø

• At each construction step

– Determine set of possible feasible solution components N(sp)

– Compute probabilities p(cij|sp) for each feasible

solution component cij N(sp) based on the amount

of pheromone ij at cij

– Select a solution component cij using the computed

probabilities

– Add a feasible solution component to sp



Solution Construction (2)

)(,][][

][][)|(

)(

pj

i

sNc

j

i

j

i

j

i

j

ipj

i sNcscp

pji

• The computation of the probability p(cij|sp) of choosing

solution component cij varies across the ACO variants

• The best known rule is the Ant System (AS) rule:

• Ni : the set of nodes that ant k can reach from sp (tabu list)

• ij : the heuristical desirability for choosing edge (i,j)


• and : relative influence of heuristics vs. pheromone



Pheromone update

scSs

ijijj

iupd

sf|

)()1(


The pheromone for each solution component is updated with:

With:

• : the evaporation rate

• Supd: the chosen set of good solutions


Local search / daemon actions

• In some cases, extra steps are needed or can be made to

improve the quality of the constructed solutions

• Performing a local search based on heuristical knowledge

to improve the quality of constructed solutions can speed

up the ACO

• In practice, ACO algorithms require an additional local

search method for good performance!



Some ACO Variants


ACO Algorithms

• Ant System (AS)

• Elitist Ant System (EAS)

• Rank-Based Ant System (ASrank)

• Min-Max Ant System (MMAS)

• Ant Colony System (ACS)



Elitist Ant System (EAS)

• An improved version of the Ant System (Dorigo 1992)

• Strong additional reinforcement of best found solution Tbs

(best-so-far)

• Pheromone update rule becomes:

With:

• With an appropriate setting for the parameter e the Elitist Ant System can find better tours in a lower number of iterations.

bs

ij

k

ij

m

k

ijij e 1

)1(

otherwise0

),(if/1 bsbs

bs

ij

TjiC


Rank-Based Ant System (ASrank)

• Proposed by Bullnheimer et al. in 1999

• Each ant deposits an amount of pheromone proportional to its rank

• The best-so-far ant always deposits the largest amount of

pheromone in each iteration

• Pheromone update:

with

• ∆ijr : the pheromone deposited on edge (i,j) by the r-th best ant

bs

ij

r

ij

w

r

ijij wrw

1

1

)(

otherwise0

),(if/1 rr

r

ij

TjiC

otherwise0

),(if/1 bsbs

bs

ij

TjiC


MAX-MIN Ant System (MMAS)

• Proposed by Stützle & Hoos, 1996-2001

• Stronger exploration of best solutions and additional

mechanism to avoid search stagnation

• Only iteration-best or best-so-far deposits pheromone:

• Additional limits to prevent stagnation:

– For all ij: min ≤ ij ≤ max

– Pheromones initialized to upper pheromone trail limit

– Pheromone trails are reinitialized each time the system

approaches stagnation (no improvement for a number of

consecutive iterations)

best

ijijij )1(


Ant Colony System (ACS)

• Proposed by Dorigo & Gambardella 1997

• Differs from AS in three points:

– More aggressive action choice rule

– Pheromone evaporation and pheromone deposit only

takes place on the arcs belonging to the best-so-far tour

– Each time an ant traverses an edge (i,j), it removes

some pheromone from the edge to increase exploration

of alternative paths


ACS – Tour Construction

Tour construction: located at city i, ant k moves to a city j according to

Only update pheromones edges contained in Tbs:

Ants eat away pheromones of traversed edges with:

otherwise

if][argmax 0

J

qqj ililNl k

i

bs

ijijij )1(

otherwise0

),(if/1 bsbs

bs

ij

TjiC

0)1( ijij

action obtained by using

the normal AS rule

A small constant

q ~ U(0,1): q0 denotes the

probability of using this

choice rule


Applications of ACO Algorithms


Problem Type Problem Name

Routing Traveling salesman

Vehicle routing

Scheduling Job shop

Project scheduling

Subset Multiple knapsack

Set covering

Machine learning Classification

Bayesian networks

Network routing Optical network routing


Advantages / Disadvantages

Advantages:

• Applicable to a broad range of optimization problems

• Can be used in dynamic applications (adapts to changes

such as new distances, etc.)

• Can compete with other global optimization techniques like

genetic algorithms and simulated annealing

Disadvantages:

• Mainly applicable for discrete problems

• Theoretical analysis is difficult



Further reading

General Ant Colony Optimization:

• http://en.wikipedia.org/wiki/Ant_colony_optimization

• http://www.aco-metaheuristic.org/

Simulation applications:

• http://www.nightlab.ch/antsim/

• http://website.lineone.net/~john.montgomery/demos/ants.html


Marco Dorigo and Thomas Stützle: Ant

Colony Optimization. MIT Press, Cambridge,

MA, 2004.

http://en.wikipedia.org/wiki/Ant_colony_optimization

http://www.aco-metaheuristic.org/



http://www.nightlab.ch/antsim/


Ant Colony Optimization - SUNY Polytechnic Instituteweb.cs.sunyit.edu/.../cs548SwarmIntelligence/ant_optimization.pdfLIACS Natural Computing Group Leiden University Ant Colony Optimization

Documents