1 Evolution in Computation James A. Foster Initiative for Bioinformatics And Evolutionary Studies (IBEST), MRCI, CSDS, and Computer Science University.

1

Evolution in Computation

James A. FosterInitiative for Bioinformatics

And Evolutionary Studies (IBEST),MRCI, CSDS, and Computer

ScienceUniversity of Idaho

2

Outline of Talk Evolutionary computation (EC)

Overview History

Applications Optimization Automatic programming Hardware design

Problems and promise Code bloat: survival of the fattest Robustness: why do evolved things endure? Hardness: when is EC not efficient? Randomness

3

Two Abstractions

Evolution: population based process in which individuals reproduce, inherit information via a noisy transmission process, interact with an environment which defines a fitness gradient on which selection operates.

Computation: information transformations induced by finite algorithms in discrete, well-defined steps.

Evolution and computation are abstractions, which can be: studied independently of their implementations, implemented in new ways

4

EC Overview

Individual solutions Solutions Translate

Evaluate fitness

Evolve: - Mate parents - Recombine - Mutate - Replace

Design challenges: - representation - fitness function - mutation operators - recombination details - selecting parents - replication details - when to stop

5

EC History ’50 A. Turing ’66 I. Rechenberg, H.P. Schwefel, P.

Bienert: fluid dynammics ’66 L. Fogel, A. Owens, M. Walsh:

machine intelligence ’75 J. Holland: modeling natural evolution ’85 J. Koza (and others): genetic

programming

6

Optimization Problems Minimize sorting network error Maximize “locomotion”* Optimizing a stock market portfolio Fuel optimal two stroke engine Drug design Disease diagnosis*Not done in my lab, but sweet stuff

7

Case Study: Sorting NetworksRobustness: an apparent intrinsic property of

evolved systems - ability to degrade gracefully with the presence of local failures. (Robustness for free)

Is this effect real, and how strong is it? What representations are best for developing

robustness in circuits? How does redundancy influence the robustness of

evolved circuits? How much is enough?

Method: evolve sorting networks, measure performance degradation with point failures in circuits

8

Sorting Nets: ExampleInputs Output

0

1

1

0

1

0

1

1

0

0

9

Sorting Nets: ExampleInputs

0

Output

01

1

1

0

1

0

0

1

0

1

1

1 1

1

0

0

0

1

1

0

10

CE (1,4) CE (0,3) CE (1,3) CE (1,2) CE (0,4)CE (0,2)CE (2,4) CE (3,4)

SN: Linear Individuals

001 100 000 011 001 011 001 010 010 100 000 010 011 100 000 100

11

CE (1,4) CE (0,3) CE (1,3) CE (2,4) CE (0,4) CE (2,3) CE (1,4) CE (1,2)

CE (0,3) CE (3,4) CE (0,2) CE (1,3) CE (2,3) CE (1,2) CE (0,3) CE (0,1)

SN: Crossover in Arrays

CE (1,4) CE (0,3) CE (2,3) CE (1,4) CE (1,2)

CE (0,3) CE (3,4)

CE (0,2) CE (1,3) CE (2,3)

CE (1,2) CE (0,3) CE (0,1)CE (1,3) CE (2,4) CE (0,4)

12

Number of bits correctly sorted, biased Number of bits correctly sorted, biased exponentially toward low order bits:exponentially toward low order bits:

SN: Fitness

kx

k

iii

i xNxCxf

1

0

)()(2)(

Where C(x)i and N(x)i are outputs of wire i from correct circuit C and evolved circuit N (resp.) on input x, and k is all k bit inputs.

13

SN: Robustness Bitwise stability (bs): percentage of

correctly sorted bits over all single point circuit failures

BS higher in evolved circuits than hand-designed ones

BS low for minimal circuits, high for circuits 5-10% larger than minimum, and decreases for larger circuits

14

SN: Robustness Size Data

Here is some data

15

Case Study: LocomotionKarl Sims, Thinking Machines Inc., evolved

“organism” descriptions and behaviors.

16

Evolving Software & Hardware Sorting circuits Robot control Machine code Growing chips

Image convolutions Feature extraction

17

CE (1,4)

CE (0,4)CE (0,3)

CE (1,3) CE (1,2)

CE (2,4) CE (0,2)

CE (3,4)

SN: Tree Representation

18

SN: Tree Crossover1. Randomly select one node on each tree

2. Swap Nodes and subtrees

19

SN: Representation Effects

Pass Through Stuck on Zero Stuck on One

MSN

Tree

Linear

* * * **** *** ** ***** ** *** **

All errors

20

SN: Evolving Physical Circuits

Cells ….. 64

Cel

ls …

.. 64

0110

0011

21

Evolving Circuits: More Results 17 electrical circuits evolved to date

which infringe on patents or rediscover prior best known designs

Filters, thermometers, amplifiers Arithmetic circuits Controllers (Higuchi’s prosthetic arm,

missile guidance) Image convolutions and classification

22

Case Study: Image ConvolutionsWe evolved FPGA circuits to produce

convolutions on laser diffraction patterns, pre-computed on special hardware.

23

Case Study: Regression

Problem: given target function, evolve symbolic functional representation to approximate it.

Fitness: square of errors on several points.

GP Demo by Hans U. Gerber, Swiss Federal Inst. Of Tech.

24

EC: Limitations & Advantages Robustness: inherent advantage of

evolution Survival of the fattest Hardness: limitations from

complexity theory, VLSI design, information theory

(Pseudo) randomness

25

Survival of the Fattest Variable size genomes grow without

correlation to fitness Protective neutral code Removal bias Combinatorics (in trees)

Given insufficient selective pressure for parsimony Only parsimony pressure helps “Liposuction” does not

Trees converge toward a fixed shape

26

Concluding Thoughts Bring more biology into engineering

and Computer Science—it works Bring more engineers and computer

scientists into biology—it works There are soooooo many interesting

questions, and soooooo much data And so very many interesting people Gee, this is fun!

27

Acknowledgements John Dickinson Jim Frenzel Deb Frincke Larry & Millie Johnston Steve McGrew Holly Wichman IBEST

BMDO DOD/OST NIH/NIGMS

John Cavalieri Bill Danielson Joe Dumoulin Brad Harvey Kosuke Imamura Jason Masner Mark Meysenburg Bart Rylander Jackie Shoaf