Evolutionary Algorithms Nicolas Kruchten 4 th Year Engineering Science Infrastructure Option.

Evolutionary Algorithms

Nicolas Kruchten4th Year Engineering Science

Infrastructure Option

What are they?

• EAs are stochastic techniques used to search multidimensional spaces for points, according to some criteria.

• They operate by mimicking reproduction & selection processes

• More on this in a bit…

What are they good for?

• EAs are used to solve optimization problems of various types:

Find the vector x such that f(x) is min or max

• A huge variety of engineering or science problems can be reduced to, or expressed as, optimization problems

A black-box example

• Say you have a black box, with n knobs, and a light bulb, what set of knob positions will make the bulb glow brightest?

What if…

• n=1? n=10000?

• knobs are continuous? discrete? mixed?

• some combinations are invalid? (i.e. knob 1 must be less than knob 2)

• an analytical expression exists?

Traditional methods

• Direct Calculus

• Gradient descent or hill climbing

• Frank Wolfe (!)

• Simplex

• Simulated Annealing

• SPSA

• Etc.

Unfortunately they…

• Sometimes simply don’t apply

• Only work for specific problems

• Don’t scale well (NP-Hard, NP-Complete)

• Get ‘trapped’ in local optima

EAs…

• Don’t have all of these problems

• Scale reasonably well

• Work for a huge variety of problems

• Don’t require gradients but can use them

How EAs work

1) Pick a ‘population’ of feasible points

2) Generate new points from the better ones

3) Throw out the worse ones

4) That’s one ‘generation’: rinse & repeat

• Lo and behold! The points in your population are getting better and better!

Back to our example

• Our feasible points were sets of knob positions, we call these ‘chromosomes’

• Individual knob positions are ‘genes’

• The brightness we call the ‘fitness’, we’re looking for chromosomes with good fitness

How to get new points

• This is where the ‘genetic’ comes in

• Given some ‘parent’ chromosomes, how do we mix them up and get a ‘child’?

• Various schemes are possible depending on how we represent the parents (binary, real, integer etc)

Mutation

• If we only ever blend or mix and match genes, we can only ever explore combinations of our initial gene pool

• To explore new parts and find new optima, we periodically change, or mutate, genes.

• Again, various processes are possible.

Which parents to use

• Various schemes exist for selection:– Pick the best ones– Pick random ones– Pick them based on relative fitness– Pick them based on relative rank– Pick them based on how different they are– etc

Which points to keep

• Various schemes exist to decide which children and/or parents to keep at each generation– Keep only the children– Select from a pool of children and parents

• Best• Random• Rank• Fitness, etc

When to stop

• You can stop an EA whenever is appropriate, but typical convergence criteria include:– A number of generations– A given standard deviation of fitnesses– A period of time– Total convergence– Etc.

Putting it all togetherLet t = 0;

Initialize P(t)

Evaluate C(t)

Let t = t + 1;Assemble P(t)

from P(t-1) and C(t-1)

Stop?no

End

yes

Evaluate P(t)

Generate C(t) from P(t)

EA Limitations

• EAs are extremely flexible, but they have their own problems:

– Can be very slow, requiring many potentially long evaluations

– Are tough to design and configure, many many parameters

– No guarantee of global optimum

Some history

• You may have heard of GAs, which are a subset of EAs

• Much confusion and conflicting terminology exists

• A huge body of operators, combinations and approaches have been proposed

Evolutionary Programming

• Traditional:– Credited to Fogel, US, published in 66– Single parents (so not genetic)– Basically pure mutation

• Since then:– EP approaches themselves mutated

independently towards general EA practice

Evolutionary Strategies

• Traditional:– Credited to Rechenberg and Schwefel,

Switzerland, published in 73– Tiny populations (1 or 2)– Sophisticated adaptive mutation scheme

• Since then:– Adaptive mutation has gone mainstream

Genetic Algorithms

• Traditional:– Credited to Holland, US, published in 72– Binary coding– Keep only the children

• Since then:– Has grown to be loosely interchangable with

EA, with real-coded GA, crowding GA etc

State of the art

• No hard and fast rules about the best EA flavour for a given problem

• Some heuristics exist

• EAs have been successfully applied to many problems in various fields, but with few general results

Advanced EAs

• Parallelization

• Distribution– Master/Slave– Peered Demes (see parallelization)– Hybrid

• Hybridization

ITS Applications

• Model calibration is a great example of a challenging transportation-related optimization:

Minimize an error term byvarying some model parameters

ITS Example: GAID

• Genetic Adaptive Incident Detection

• Use of a GA to train a neural network: perfect example of ANN + EA = ITS

• The problem: automatically detecting incidents on the freeway, using traffic loop data

GAID: Basics

• GAID uses a probabilistic neural network (PNN) which takes 16d vectors and classifies them into two piles

• Traffic loop data -> incident/normal

• The PNN is defined by 16 parameters

GAID: Diagram

PNN = f(parameters)Loop Data

Incident

Normal

16Parameters

EA

GAID: Fitness

• Fitness: correct classifications/number of vectors

• We ‘quiz’ the PNN with known data to score it

• We are looking for the set of 16 real parameters which make GAID the most accurate

GAID: Initialization

• Initialization:– Pick population size: 50– Pick 50 random sets of 16 real parameters

between 0 and 1000– Score the chromosomes

GAID: Generation

• Generation:– Pick the child population (generation) size: 50– Select 2 parents at random– Each child gene is either mom or dad’s– Mutate every gene: add some uniformly

distributed random number between -50 and 50

– Score the children

GAID: 2d view

GAID: 2d view

GAID: 2d view

One alternative

GAID: Assembly

• Assembly of children and parents:

– Choose the 50 best from the combined pool of children and parents

• Convergence: repeat, stopping when the STD is less than 0.005

GAID: Summary

• Real representation, 16 genes• Popsize = Gensize = 50• Random Initialization• 2 parents, random selection• real crossover, creep mutation on all

genes• Crowding assembly, best selection• STD convergence