Introduction to Evolutionaryyp Computation What is an ...courses.nus.edu.sg/course/elepv/ee4306/Handouts/5.IntroEC.pdf · Evolutionary Algorithms Fuzzy Systems Evolution Strategies

1EE4306 Chapter 1 Distributed Autonomous Robotic Systems

Introduction to Evolutionary ComputationIntroduction to Evolutionary Computationy py p

COMPUTATIONALCOMPUTATIONALINTELLIGENCE

orSOFT COMPUTINGSOFT COMPUTING

NeuralNetworks

EvolutionaryAlgorithms

FuzzySystems

EvolutionStrategies

GeneticAlgorithms

GeneticProgramming

Dr. Prahlad Vadakkepat Semester II

Strategies Algorithms Programming


What is an optimization problem?What is an optimization problem?

Wh t th lWhat are the values of x and y such that z is equal to the

z = f (x, y)

z is equal to the value at the peak?

x y


y


2)( xxfy ]10,0[xwhereAn Optimization Problem

Find the value of x that gives the minimum of y

y020dy

0 when x min(f)

020 ⇒ xxdxdy

x

Exhaustive Search 00fx

0 when xmin(f)1000:100: ⇒ fx

.........4211

( )


10010.........


Consider, 10 parameters x1, x2, …, x10

210

22

21 ... xxxf

Find min(f)

x1 x2 x3 … x10 f0 10 0 10 0 10 0 10 0 10000 10 0 10 0 10 … 0 10 0 1000

Time required for the evaluation:All possible combinations * 0.01 s

1110 * 0.01 s = 3002 days = 8.22 years Not practical to evaluate ALL combinations!



Consider, 10 parameters x1, x2, …, x10

210

22

21 ... xxxf

Find min(f)

A string / chromosome Cost function orFitness function

x1 x2 x3 … x10 f0 10 0 10 0 10 … 0 10 0 1000



Conventional methods – single point search

Optimization problem: Finding a Po where

Gradient driven search)(min)(| iSPooo PfPfSPP

i

Gradient-driven search

f

nnfPP 1

P0

Pf nPP

nn P1

is a small positive


Pp

value – learning ratePn


Problems – 1. Function must be continuous/differentiable

2. Local optima

3 Performance depends on the starting point3. Performance depends on the starting point



Local search

A local hill-climbing technique

no differentiation is neededno differentiation is needed

N small short-sighted (too local): Difficult to improve the search;

N large exhaustive search: Time taken is too long;N large exhaustive search: Time taken is too long;

Problem with hill-climbing search - easy to be trapped at local optima.



Genetic Algorithm (GA) - The Ingredients

t t + 1t t + 1

l tiselection

t ti


mutation Crossover / Recombination


The Evolutionary Cyclecost

1fitness

Selection

fitness

MutationCrossover

e.g., Pcross = 0.7e.g., Pmutation = 0.01



Th i f th l tt h l’ d i ti l t th li d fitThe size of the roulette wheel’s edge or arc is proportional to the normalized fitnessTotal fitness is 360The spin is a random number generatorWheel is spun k times (k is the number of individuals in the population pool)Wheel is spun k times (k is the number of individuals in the population pool)An individual could be selected more than onceIt is possible that the best individual may not be selected



C b bilit h ft ill b f d If th iCrossover probability says how often will be crossover performed. If there isno crossover, offspring is exact copy of parents. If there is a crossover,offspring is made from parts of parents' chromosome. If crossover probabilityis 100% then all offspring is made by crossoveris 100%, then all offspring is made by crossover.

A random number is generated for each pair individual. If the number is lowerthan the crossover rate crossover is performed

One-point crossoverA single crossover point on both parents' organism

than the crossover rate, crossover is performed.

strings is selected. All data beyond that point in eitherorganism string is swapped between the two parentorganisms. The resulting organisms are the children:

Two-point crossoverTwo-point crossover calls for two points to be selectedon the parent organism strings. Everything betweenthe two points is swapped between the parentorganisms, rendering two child organisms:



Uniform CrossoverUniform Crossover

A random decision is made at each bit position in the string as to whether or not to crossover bits between the parent stringsIf th t t f th d b t iIf the output of the random number generator is above a threshold, the bits and exchanged



Mutation probability says how often will be parts of chromosomemutated.

If there is no mutation, offspring is taken after crossover (or copy) withoutany change. If mutation is performed, part of chromosome is changed. Ifmutation probability is 100% whole chromosome is changed if it is 0%mutation probability is 100%, whole chromosome is changed, if it is 0%,nothing is changed.

Mutation is made to prevent falling GA into local extreme, but it should notMutation is made to prevent falling GA into local extreme, but it should notoccur very often, because then GA will in fact change to random search.

It is possible that the roulette wheel selects the same or similar individualsThis could lead to a cross over between identical or similar genesThis could lead to a cross over between identical or similar genes. The next generation will have genes similar to their parents.This stagnation can be prevented by the mutation process



Operations Chromosomes Fitness

Initial population: Example of coded P1: 1 2 0 9 0 2 1 7 f(P1) = 5 %

parameter sets forming an initial

population with size 3. The performance of

each parameter set is simulated and then

P2: 4 0 0 3 0 1 6 1

P3: 0 1 6 4 1 8 0 1

f(P2) = 60 %

f(P3) = 35 %

(N.B. The above fitness

assigned a fitness. values are examples)

Reproduction: A simple scheme is to

allow the chromosomes to reproduce off-

P2: 4 0 0 3 0 1 6 1

P2: 4 0 0 3 0 1 6 1 Evolution in progress (No

spring according to their respective fitness.

Thus P1 has low probability of producing

children, P2 has a probability of producing

d P

P3: 0 1 6 4 1 8 0 1

need to re-calculate fitness

here).

two and P3 one.

Crossover: Some portion of a pair of

chromosomes is exchanged at the dotted

iti d l ifi d

P2: 4 0 0 3 0 1 6 1

P2’: 4 0 0 3 0 8 0 1

P ’ 0 1 6 4 1 1 6 1

No fitness calculations

needed here.

position randomly specified. P3’: 0 1 6 4 1 1 6 1

Mutation: The binary values of some

genes of some chromosomes are inverted.

Th l hi h h b h d

P2: 4 0 0 3 0 1 6 1

P2”: 4 0 1 3 0 8 0 1

P ’ 0 1 6 4 1 1 6 1

A new generation is now

formed and the fitness

d t b l t d f


The value which has been changed as an

example is highlighted by an underline.

P3’: 0 1 6 4 1 1 6 1 needs to be evaluated for

the next cycle.


The Basic Operation of a Simple GA

Make initial population;Evaluate the fitness of all initial chromosomes;REPEATREPEAT

Reproduce children from parents (selection);Apply crossover to some chromosomes;Apply crossover to some chromosomes;Apply mutation to some chromosomes;Preserve elite individuals;

UNTIL satisfactory result found Decode the best chromosome found.



f GReference Books on GA

T. Bäck, Evolutionary Algorithms in Theory and Practice, Oxford University Press, 1996.y

L. Davis, The Handbook of Genetic Algorithms, Van Nostrand & Reinhold, 1991.

D.B. Fogel, Evolutionary Computation, IEEE Press, 1995.

D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning Addison Wesley 1989Learning, Addison-Wesley, 1989.

J. Koza, Genetic Programming, MIT Press, 1992.

Z Michalewicz Genetic Algorithms + Data Structures = EvolutionZ. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs, Springer, 3rd ed., 1996.



Summary of GAPerformance is superior to other techniques on complexPerformance is superior to other techniques on complex

problems with:– lots of data/ many parameters (high-dimensional search– lots of data/ many parameters (high-dimensional search

space).– complex relationships among parameters.

Has great practical potentials simple to implement

complex relationships among parameters.– many (local) optima.

Has great practical potentials – simple to implement.

Is getting popular in many fields – Engineering, Comp. ScienceScience …

Gives high performance against low costs (with increasing ti d )


computing power everyday).


Details of Genetic AlgorithmsDetails of Genetic AlgorithmsStructure of population in a GA

Fitness function An individual

Chromosome 1Gene Gene Gene Genef1




Chromosome n - 3Gene Gene Gene Genefn-3



Chromosome nGene Gene Gene Genefn

l i i


population size = n


Genotype: This is the "internally coded, inheritable information" carried byall living organisms This stored information is used as a "blueprint" or setall living organisms. This stored information is used as a blueprint or setof instructions for building and maintaining a living creature. Theseinstructions are found within almost all cells (the "internal" part), they are

itt i d d l (th ti d ) th i d t th tiwritten in a coded language (the genetic code), they are copied at the timeof cell division or reproduction and are passed from one generation to thenext ("inheritable"). These instructions are intimately involved with allaspects of the life of a cell or an organism. They control everything fromthe formation of protein macromolecules, to the regulation of metabolismand synthesis.y



Phenotype: This is the "outward, physical manifestation" of the organism.These are the physical parts, the sum of the atoms, molecules,macromolecules cells structures metabolism energy utilization tissuesmacromolecules, cells, structures, metabolism, energy utilization, tissues,organs, reflexes and behaviors; anything that is part of the observablestructure, function or behavior of a living organism. An organism’sh t i ll f it b bl h t i ti hi h i fl dphenotype is all of its observable characteristics—which are influenced

both by its genotype and by the environment.

The food the flamingos eat makes their phenotype white or pink.

Differences in the genotypes can produce different phenotypes. In these house cats, the genes for ear form are different, causing one

f th t t h l d th


p yp pof these cats to have normal ears and the

other to have curled ears.


Designing a Representation- We have to come up with a method of representing an individual as a

genotype.When choosing a representation we have to bear in mind how the- When choosing a representation, we have to bear in mind how the genotypes will be evaluated and what the genetic operators might be.

Example: Binary Representation

CHROMOSOME / STRINGCHROMOSOME / STRING

Dr. Prahlad Vadakkepat Semester IIGENE

Bit or Binary number


Phenotypefor 1 parameter

8 bits Genotype

y

• IntegerInteger

• Real number

1*21*27 7 + 0*2+ 0*26 6 + 1*2+ 1*25 5 + 0*2+ 0*24 4 + 0*2+ 0*23 3 + 0*2+ 0*22 2 + 1*2+ 1*21 1 + 1*2+ 1*200

=128=128 + 32 + 2 + 1 = 163+ 32 + 2 + 1 = 163128 128 + 32 + 2 + 1 163+ 32 + 2 + 1 163

= 163 (integer)Phenotype:


= 163 (integer)


for 1 parameter: x

Genotype: Phenotype:

for 1 parameter: x

= 13.9609(real number)

yp Phenotype:Could be a real number, e.g. a number between

1 6 32 .5 2 0 .5 2 .5 1 3 .9 6 0 9x

(real number)gthe range of 2.5 to 20.5using 8 binary bits

max minmin max

2 .5 2 0 .5 2 .5 1 3 .9 6 0 92 5 5

xg y

max value for 8 bitsmin

What is x if the string is: 00000000 ?What is x if the string is: 11111111 ?


What is x if the string is: 11111111 ?


Initialization in GAInitialization in GA

Uniformly on the search space … if possible

Binary strings: 0 or 1 with probability 0 5– Binary strings: 0 or 1 with probability 0.5– Real-valued representations: Uniformly on a given

interval (OK for bounded values only)interval (OK for bounded values only).Seed the population with previous results or those from heuristics With care:heuristics. With care:– Possible loss of genetic diversity (dominated by the

good starting string)good starting string).– Possible unrecoverable bias (cannot converge to the

global optimum)


global optimum).


Evaluating an Individual

This is by far the most costly step for real-world applications; do not re evaluate unmodified individualsapplications; do not re-evaluate unmodified individuals.It might be a subroutine, a black-box simulator, or any

t l ( b t i t f ti iexternal process (e.g. robot experiment, function mapping -system identification, control, TSP etc.)C t i t h dli h t if th h t b kConstraint handling - what if the phenotype breaks some constraint of the problem:

penalize the fitness– penalize the fitness.– multi-objective optimization (formulate the constrains

as objectives to be optimized)


as objectives to be optimized).


Mutation OperatorsSome important points are:Some important points are:At least one mutation operator should allow every part of the search space to be reachedspace to be reached.The size of mutation is important and should be controllable.Mutation should produce valid chromosomes

1 1 1 1 1 1 1before

Mutation should produce valid chromosomes.

1 1 1 1 1 1 1before

1 1 1 0 1 1 1after

Mutation usually happens with b bilit ( 0 02) f

1 1 1 0 1 1 1after


probability pm (e.g., 0.02) for each gene.

mutated gene


Example: Mutation for order based representation (Swap)R d l l diff d h

7 83 41 2 6 5Randomly select two different genes and swap them.

7 83 46 2 1 57 83 46 2 1 5Example: Mutation for tree based representation

* *Randomly select one node and replaces it with another random node.

2 * *

Dr. Prahlad Vadakkepat Semester IIr r r r


Example: Recombination/Crossover for Discrete/Binary Representation

Whole Population:. . .

parentscut cut

1 1 1 1 1 1 1 0 0 0 0 0 0 0 parents

1 1 1 0 0 0 0 0 0 0 1 1 1 1 offspring

The child should inherit something from each parent. If this is not the case then the operator is a mutation operator.


Recombination should produce valid chromosomes.


Uniform Crossovera db fc e g h a b C Ed Hgfa db fc e g h

FD GE HCBA0 0 1 0 1 0 0 1

A B c eD hGF

a b C d g

Recombination for order based representation (example):Choose an arbitrary part from the first parent and copy this to the first childChoose an arbitrary part from the first parent and copy this to the first child

Copy the remaining genes (from the second parent) that are not in the copied part to the first child:copied part to the first child:

starting right from the cut point of the copied part

using the order of genes from the second parentusing the order of genes from the second parent

wrapping around at the end of the chromosome

Repeat this process with the parent roles reversed


Repeat this process with the parent roles reversed


Example: Recombination for order based representation

7 83 41 2 6 5Parent 1

78 16 5234

Parent 2

7 3 4 6 5

7 83 41 2 6 5 78 16 5234

7, 3, 4, 6, 5order

81 2 4, 3, 6, 7, 5

Child 1


7 85 41 2 3 6


Example: Recombination for tree-based representation

*2 ( )2 * 2 * (r * r )

r r*

+* (r + (l / r))

r /Two random sub-trees are

l t d f i


1 rselected for swapping.


Selection Strategy

We want to have some way to ensure that better yindividuals have a better chance of being parents than less good individuals.This will give us selection pressure which will drive the population forward.W h b f l i l d i di id lWe have to be careful to give less good individuals at least some chance of being parents – since they may include some useful genetic materialinclude some useful genetic material.



Example: Fitness proportionate selectionProbability of xi being selected for mating is: ff i

fi = fitness of xi = total fitness in the populationf

• Better (fitter) individuals have:more space

p p

– more space– more chances to be selected

BestWorst

Disadvantages:Worst

Danger of premature convergence because outstanding individuals take over the entire population very quickly.

l i h fi l h h


Low selection pressure when fitness values are near each other.


Example: Tournament selection1 Randomly pick k (tournament size) individuals from population;1. Randomly pick k (tournament size) individuals from population;2. Select “the best” among these k individuals;3. Repeat steps 1-2 (those picked before will not be picked again) until all p p ( p p g )

individuals have been picked;4. The above steps 1-3 will be performed for k times (each time with randomly

i k d i di id l ) th t th t t l b f i di id l l t d i l tpicked individuals), so that the total number of individuals selected is equal to the population size.

f=10 (k = 2)

f=7


f 7


2

2 3 3

2

2 3 3

k = 2

Better stringRandomly selected

9

23

48

1 9

5 4

1

4

9

23

48

1 9

5 4

1

4

Better string

Better stringChromosome b

5

6

7

81

10 6 10 105

6

7

81

10 6 10 10

ngs

Better string

Better string

number

8 7 78 7 7

Repeat 1 9 11 9 1

elec

ted

strinBetter string

Better stringRandomly selectedRepeat

9

23 3 6 39

23 3 6 3

Se

Better string

4

57

81

10

7 8

4 2

7

4

4

57

81

10

7 8

4 2

7

4

Better string

Better string


610

5 10 56

10

5 10 5Better string


Elitism in GAShould fitness be constantly improved?Should fitness be constantly improved?

Re-introduce in the population previous best-so-far (elitism).

For example keep the best 10% of parent population and re-For example, keep the best 10% of parent population and re-introduce them in the next generation by replacing the worst 10% children population.

In GA, elitism tends to improve on the final results.

Example ofConvergence trace



Stopping Criterion

The optimum is reached!Limit on CPU resources: maximum number of fitness evaluationsLimit on CPU resources: maximum number of fitness evaluations.Limit on the patience: after some generations without improvement.Never draw any conclusion from a single run – use statistical measures (e.g., average, standard deviation).– obtain results from a sufficient number of independent runs.


Introduction to Evolutionaryyp Computation What is an ...courses.nus.edu.sg/course/elepv/ee4306/Handouts/5.IntroEC.pdf · Evolutionary Algorithms Fuzzy Systems Evolution Strategies

Documents