Ant Colony Optimisation

Swarm intelligence

Swarm intelligence

• Collective system capable of accomplishing difficult tasks in dynamic and varied environments without any external guidance or control and with no central coordination.

• Achieving a collective performance which could not normally be achieved by an individual acting alone.

• Constituting a natural model particularly suited to distributed problem solving.

Swarm Algorithms

Inspiration from swarm intelligence has led to some highly successful optimisation algorithms.

– Ant Colony (-based) Optimisation – a way to solve optimisation problems based on the way that ants indirectly communicate directions to each other.

– Particle Swarm Optimisation — a different way to solve optimisation problems, based on the swarming behaviour of several kinds of organisms.

Optimization

• Given a graph with two specified vertices A and B, find a shortest path from A to B.

• Given a set of cities and pairwise distances, find a shortest tour.

• Given a sequence of amino acids of a protein, find the structure of the protein.

General optimization problem:given f:X ,ℝfind xεX such that f(x) is minimum

shortest path problem, polynomial

traveling salesperson problem,

protein structure prediction problem,

Ant Colony Optimization

• Real ants• Stigmergy• Autocatalyzation• Ant System• Ant Colony System

Ant Colony Optimization (ACO)

Overview

“Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete

optimization problems.”-Source: ACO website, http://iridia.ulb.ac.be/~mdorigo/ACO/about.html

Ant Colony Optimization (ACO)• In the real world, ants (initially) wander randomly, and upon finding food

return to their colony while laying down pheromone trails. If other ants find such a path, they are likely not to keep traveling at random, but instead follow the trail laid by earlier ants, returning and reinforcing it if they eventually find food

• Over time, however, the pheromone trail starts to evaporate, thus reducing its attractive strength. The more time it takes for an ant to travel down the path and back again, the more time the pheromones have to evaporate.

• A short path, by comparison, gets marched over faster, and thus the pheromone density remains high

• Pheromone evaporation has also the advantage of avoiding the convergence to a locally optimal solution. If there were no evaporation at all, the paths chosen by the first ants would tend to be excessively attractive to the following ones. In that case, the exploration of the solution space would be constrained.

• Thus, when one ant finds a good (short) path from the colony to a food source, other ants are more likely to follow that path, and such positive feedback eventually leaves all the ants following a single path.

• The idea of the ant colony algorithm is to mimic this behavior with "simulated ants" walking around the search space representing the problem to be solved.

• Ant colony optimization algorithms have been used to produce near-optimal solutions to the traveling salesman problem.

• They have an advantage over simulated annealing and genetic algorithm approaches when the graph may change dynamically. The ant colony algorithm can be run continuously and can adapt to changes in real time.

• This is of interest in network routing and urban transportation systems.

ACO Defined

• A heuristic optimization method for shortest path and other optimization problems which borrows ideas from biological ants.

• Based on the fact that ants are able to find shortest route between their nest and source of food.

Shortest Route

• Shortest route is found using pheromone trails which ants deposit whenever they travel, as a form of indirect communication

Ant Colony Optimization• Ant foraging – Co-operative search by

pheromone trails

When ants leave their nest to search for a food source, they randomly rotate around an obstacle


• Ant foraging – Co-operative search by pheromone trails

initially the pheromone deposits will be the same for the right and left directions



When the ants in the shorter direction find a food source, they carry the food and start returning back, following their pheromone trails, and still depositing more pheromone.



An ant will most likely choose the shortest path when returning back to the nest with food as this path will have the most deposited pheromone



For the same reason, new ants that later starts out from the nest to find food will also choose the shortest path.

Ant Colony Optimization• Ant foraging – Co-operative search by

pheromone trails

Over time, this positive feedback (autocatalytic) process prompts all ants to choose the shorter path

Naturally Observed Ant Behavior

All is well in the world of the ant.


Oh no! An obstacle has blocked our path!


Where do we go? Everybody, flip a coin.


Shorter path reinforced.

“Stigmergic?”

• Stigmergy, a term coined by French biologist Pierre-Paul Grasse, is interaction through the environment.

• Two individuals interact indirectly when one of them modifies the environment and the other responds to the new environment at a later time. This is stigmergy.

Stigmergy in Ants Ants are behaviorally unsophisticated, but collectively they

can perform complex tasks.

Ants have highly developed sophisticated sign-based stigmergy– They communicate using pheromones;– They lay trails of pheromone that can be followed by other ants.

• If an ant has a choice of two pheromone trails to follow, one to the NW, one to the NE, but the NW one is stronger – which one will it follow?

• Individual ants lay pheromone trails while travelling from the nest, to the nest or possibly in both directions.

• The pheromone trail gradually evaporates over time.• But pheromone trail strength accumulate with multiple

ants using path.

Pheromone Trails

Food sourceNest

Autocatalyzation

What is autocatalytic behavior?

E

D

CH

B

A

(b)

30 ants

30 ants

15 ants

15 ants15 ants

15 ants

t = 0

d = 0.5

d = 0.5

d = 1

d = 1

E

D

CH

B

A

(a)

E

D

CH

B

A

(c)

30 ants

30 ants

20 ants

20 ants10 ants

10 ants

t = 1

τ = 30

τ = 30

τ = 15

τ = 15

Initial state:no ants

Pheromone Trails

Autocatalyzation

This is why ACO algorithms are called autocatalytic positive

feedback algorithms!

Ant Colony OptimizationThe Ant System (AS)

Ant Colony Optimisation Algorithms: Basic Ideas

Ants are agents that:Move along between nodes in a graph.

They choose where to go based on pheromone strength (and maybe other things)

An ant’s path represents a specific candidate solution.

When an ant has finished a solution, pheromone is laid on its path, according to quality of solution.

This pheromone trail affects behaviour of other ants by `stigmergy’ …

E.g. A 4-city TSP

A B

C DPheromone

Ant AB: 10, AC: 10, AD, 30, BC, 40, CD 20

Initially, random levels of pheromone are scattered on the edges

E.g. A 4-city TSP

A B

C DPheromone

Ant AB: 10, AC: 10, AD, 30, BC, 40, CD 20

An ant is placed at a random node

E.g. A 4-city TSP

A B

C DPheromone

Ant AB: 10, AC: 10, AD, 30, BC, 40, CD 20

The ant decides where to go from that node,based on probabilitiescalculated from: - pheromone strengths, - next-hop distances.

Suppose this one chooses BC

E.g. A 4-city TSP

A B

C DPheromone

Ant AB: 10, AC: 10, AD, 30, BC, 40, CD 20

The ant is now at C, and has a `tour memory’ = {B, C} – so he cannot visit B or C again. Again, he decides next hop(from those allowed) basedon pheromone strengthand distance;suppose he choosesCD

E.g. A 4-city TSP

A B

C DPheromone

Ant AB: 10, AC: 10, AD, 30, BC, 40, CD 20

The ant is now at D, and has a `tour memory’ = {B, C, D}There is only one place he can go now:

E.g. A 4-city TSP

A B

C DPheromone

Ant AB: 10, AC: 10, AD, 30, BC, 40, CD 20

So, he has nearly finished his tour, having gone over the links:BC, CD, and DA.

E.g. A 4-city TSP

A B

C DPheromone

Ant AB: 10, AC: 10, AD, 30, BC, 40, CD 20

So, he has nearly finished his tour, having gone over the links:BC, CD, and DA. AB is added to complete the round trip.

Now, pheromone on the touris increased, in line with the fitness of that tour.

E.g. A 4-city TSP

A B

C DPheromone

Ant AB: 10, AC: 10, AD, 30, BC, 40, CD 20

Next, pheromone everywhereis decreased a little, to modeldecay of trail strength overtime

E.g. A 4-city TSP

B

C DPheromone

Ant AB: 10, AC: 10, AD, 30, BC, 40, CD 20

We start again, with another ant in a random position.

Where will he go?

Next , the actual algorithmand variants.

A

The ACO algorithm for the TSP[a simplified version with all essential details]

We have a TSP, with n cities.

1. We place some ants at each city. Each ant then does this:• It makes a complete tour of the cities, coming back to its starting city, using a

transition rule to decide which links to follow. By this rule, it chooses each next-city at random, but biased partly by the pheromone levels existing at each path, and biased partly by heuristic information.

2. When all ants have completed their tours, Global Pheromone Updating occurs.

• The current pheromone levels on all links are reduced (I.e. pheromone levels decay over time).

• Pheromone is lain (belatedly) by each ant as follows: it places pheromone on all links of its tour, with strength depending on how good the tour was.

Then we go back to 1 and repeat the whole process many times, until we reach a termination criterion.

• A generalized version of the pseudocode for the ACO algorithm with reference to the TSP is illustrated below:

ACO algorithm• The basic operational flow in Ant Colony Optimization is as follows:

Step 1: Represent the solution space by a construction graphStep 2: Initialize ACO parametersStep 3: Generate random solutions from each ant’s random walkStep 4: Update pheromone intensitiesStep 5: Goto Step 3, and repeat until convergence or a stopping condition is satisfied.

ACO algorithm• The process starts by generating m random ants (solution).

• An ant represents a solution string, with a selected value for each variable.

• An ant is evaluated according to an objective function.

• Accordingly, pheromone concentration associated with each possible route( variable value) is changed in a way to reinforce good solutions as follows:

46

: Pheromone trail of combination (i,j)

: Local heuristic of combination (i,j)

: Transition probability of combination (i,j)

: Relative importance of pheromone trail

: Relative importance of local heuristic

: Determines the relative importance of exploitation versus exploration

: Trail persistence

Parameters of ACO Algorithm

ij

ij

ijP

0q

ACO algorithm

lij is the length of the tour completed by ant i

ACO algorithm Implementation

above.

ACO Pseudo Code

Pseudo code for ACO is shown below:

Pheromone Concentration Calculations

Pheromone Concentration Calculation

Main parameters at glance

ACO Characteristics

• Natural algorithm since it is based on the behavior of real ants in establishing paths from their colony to source of food and back.

• Parallel and distributed since it concerns a population of agents moving simultaneously, independently and without a supervisor.

• Cooperative since each agent chooses a path on the basis of the information, pheromone trails laid by the other agents, which have previously selected the same path. This cooperative behavior is also autocatalytic, i.e., it provides a positive feedback, since the probability of choosing a path increases with the number of agents that previously chose that path.

• Versatile that it can be applied to similar versions of the same problem; for example, there is a straightforward extension from the traveling salesman problem (TSP) to the asymmetric traveling salesman problem (ATSP).

• Robust that it can be applied with minimal changes to other combinatorial optimization problems such as quadratic assignment problem (QAP) and the jobshop scheduling problem (JSP).

Similarities and Differences Between Real Ants and Artificial Ants

• Similarities1. Colony of cooperating individuals–Both real ant colonies and ant algorithms

are composed of a population, or colony of independent individual agents. They globally cooperate in order to find a good solution to the task under consideration

2. Pheromone trail and stigmergy–Like real ants, artificial ants change some aspects of their environment while walking. Real ants deposit a chemical substance called pheromone on the visited state. Artificial ants will change some numerical information of the problem state, locally stored, when that state is visited.

3. Local moves and the shortest path searching– both finds the shortest path between the origin and the destination.

4. Transition policy–Both real ants and artificial ones will build solutions by applying decision making procedures to move through adjacent states.

5. Deposited amount of pheromone–Amount of pheromone that an artificial ant will deposit is mostly a function of the quality of the discovered solution. In nature, some ants behave in a similar way, the deposited amount of pheromone is highly dependent on the quality of the discovered food source.

Similarities and Differences Between Real Ants and Artificial Ants

• Differences1. Artificial ants live in a discrete world. All their moves are jumps from one

discrete state to another adjacent one.

2. Artificial ants have memory, they could remember states that have been visited already.

3. Pheromone deposit methodology is significantly different between real and artificial ants. Timing in pheromone laying is problem dependent and often does not have similarities with the real ants pheromone deposit methodology.

4. To improve overall performance, Ant System algorithms could be enriched with some additional capabilities that cannot be found in real ant colonies. Most AS contains some local optimization techniques to improve solutions developed by ants.

ACO and model-based search (MBS) algorithm

• The ant colony optimization (ACO) is a meta-heuristic approach (problem-independent techniques, not greedy, even accept a temporary deterioration of the solution, for example, the simulated-annealing technique, which allows them to explore more thoroughly the solution space and thus to get a hopefully better solution) for combinatorial optimization problems inspired by the foraging behavior of the social insects, especially the ants.

• ACO algorithms belong to the class of model-based search (MBS) algorithm.

Ant Colony Optimization Algorithms

• Different ant colony optimization algorithms have been proposed. The original ant colony optimization algorithm is known as Ant System and was proposed in the early 90s.

• Table gives a list of successful variants of Ant Colony Optimization Algorithms.

Ant Colony Optimization Algorithms

• Figure gives a narrow overview on Ant Colony Optimization Algorithms.

• ACO is a class of algorithms, whose first member, called Ant System, was initially proposed by Colorni, Dorigo and Maniezzo.

• Many special cases of the ACO metaheuristic have been proposed.

• The three most successful ones are: 1. Ant System, 2. Ant Colony System (ACS), and 3. MAX-MIN Ant System (MMAS)

1. ACO - Ant System

2. ACO - Ant Colony System

• First major improvement over Ant System

• Differences with Ant System:1. Decision Rule - Pseudorandom proportional rule2. Local Pheromone Update3. Best only offline Pheromone Update

1. Pseudorandom proportional rule– Probability for an ant to move from city i to city j

depends on a random variable q uniformly distributed over [0, 1], and a parameter q0.

– If q ≤ q0, then, among the feasible components, the component that maximizes the product τilηβ

il is chosen, otherwise the same equation as in Ant System is used.

– This rule favours exploitation of pheromone information

2. Local pheromone update– The local pheromone update is performed by all ants

after each step.

– Each ant applies it only to the last edge traversed:τij= (1 − φ) · τij + φ · τ0

where φ (0, 1) is the pheromone decay coefficient∈τ 0 is the initial value of the pheromone

3. Best only offline pheromone update– Offline pheromone update equation

τij ← (1 − ρ) · τij + ρ · ∆τij best

Where 1/Lbest if best ant k travels on edge i, jτij

best = 0 otherwise

– Lbest can be set to the length of the best tour found in the current iteration or the best solution found since the start of the algorithm.

3. ACO - MAX-MIN Ant System

• Differences with Ant System:• Best only offline Pheromone Update• Min and Max values of the pheromone are explicitly

limited– τij is constrained between τmin and τmax (explicitly set by

algorithm designer).– After pheromone update, τij is set to τmax if τij > τmax and to

τmin if τij < τmin

Applications of Ant Colony Optimization• Traveling Salesman Problem, where a salesman must find the shortest route by

which he can visit a given number of cities, each city exactly once.

• Quadratic Assignment Problem, the problem of assigning n facilities to n locations so that the costs of the assignment are minimized.

• Job-Shop Scheduling Problem, where a given set of machines and set of job operations must be assigned to time intervals in such a way that no two jobs are processed at the same time on the same machine and the maximum time of completion of all operations is minimized.

• Vehicle Routing Problem, the objective is to find minimum cost vehicle routes such that:

(a) Every customer is visited exactly once by exactly one vehicle;(b) For every vehicle the total demand does not exceed the vehicle capacity;(c) The total tour length of each vehicle does not exceed a given limit;(d) Every vehicle starts and ends its tour at the same position.

Applications of Ant Colony Optimization• Shortest Common Super sequence Problem, where—given a set of strings over an

alphabet—a string of minimal length that is a super sequence of each string of the given set has to be found.

• Graph-Coloring Problem, which is the problem of finding a coloring of a graph so that the number of colors used is minimal.

• Sequential Ordering Problem, which consists of finding a minimum weight Hamiltonian path 2 on a directed graph with weights on the arcs and on the nodes, subject to precedent constraints among the nodes.

• Connection-Oriented Network Routing, where all packets of the same session follow the same path selected by a preliminary setup phase.

• Connectionless Network Routing where data packets of the same session can follow different paths (Internet-type networks).

Advantages of ACO

• Inherent parallelism

• Positive Feedback accounts for rapid discovery of good solutions

• Efficient for Traveling Salesman Problem and similar problems

• Can be used in dynamic applications (adapts to changes such as new distances, etc)

Disadvantages of ACO

• Theoretical analysis is difficult

• Sequences of random decisions (not independent)

• Probability distribution changes by iteration

• Research is experimental rather than theoretical

• Time to convergence uncertain (but convergence is gauranteed!)

71

Ant System (AS) – the earliest version of ACO

State Transition Probability

Pheromone Update Rule

Ulilil

ijijijP

)()()()(

NA

k

kij

oldij

newij

1

,Qkij

ij

kij d

Q k

kij L

Qor

72

ASelite

ASrank

eij

NA

k

kij

oldij

newij

1

eij

e

r

rij

oldij

newij

1

1

eeij L

Qe

rrij L

Qre )(

73

Ant-Q & Ant Colony System (ACS)

Local Updating (Online Updating)Global Updating (Offline Updating)

Vv

ililUl])()[(maxarg

0

0

qq

qq

Ulilil

ivivivP

)()()()(

0)1( oldij

newij

eij

oldij

newij )1(

Exploitation

Exploration

74

Max-Min Ant System (MMAS)

ANTS

eij

oldij

newij maxmin ij

Ulilil

ijijijP

])1([)1(

)1(0 LBLLBL

avg

kkij

NA

k

kij

oldij

newij

1

Methodology

Ant Colony Optimisation

Documents