Algorithms and Architectures II Jens Myrup Pedersen (JMP ...kom.aau.dk/~rlo/lectures/algoAndArch08/mm3.pdfExamples: Convex hull (Graham’s Scan) • The problem asks to construct

Algorithms and Architectures II

Jens Myrup Pedersen (JMP), Rasmus Løvenstein Olsen (RLO)

Mm3: Algorithm examples: Genetic algorithms, advanced backtracking, min-max- February 25, 2008

Algorithms and Architectures II

1. Processor and Storage Architecture (JMP)2. Vehicular on-board networks and applications (RLO)3. Algorithm examples: Genetic algorithms, advanced backtracking (RLO)4. Complexity and complexity classes (RLO)5. Complexity and complexity classes 2 + algorithms for task scheduling (RLO)6. Advanced data structures 1 (JMP)7. Advanced data structures 2 (JMP)8. Graph algorithms 1 (JMP)9. Graph algorithms 2 (JMP)10. Guest Lecture by Rasmus H. Nielsen: Complex Problems in Network

Planning

NB: Slides originally prepared by Hans-Peter Schwefel, for D4-6: Lecture 2, Spring 2007

Program of today

• Introduction to algorithms• Binary trees • Backtracking • Spanning tree• Genetic algorithms• Assignments

Introduction to algorithms

• The word algorithms origins from a Persian mathmatician from 9th century: Abu Abdullah Muhammad ibn Musa al-Khwarizmi

• Algorithms has many uses, e.g. for light bulps

• Earliest known algorithm was doneby the Babylonians in 1600 BCfor factorisation and findingsquare roots

• Question: Today we have reallyfast computers, why do we needalgorithm and study of these?

• The Big O notation

Lamp does not work

Lamp pluggedIn?

Bulp burnedOut?

Buy new lamp

Plug in lamp

Replace bulp

g(x)=O(f(x))With a slight abuse of notation!

Program of today


Binary trees

Binary search tree property:Let x be a node in a binary search tree. If y is a node in the left subtree of x, then key[y] ≤ key[x]. If y is a node in the right subtree of x, then key[x] ≤ key[y]

5

3

2 5

7

8

23

7

58

4

Increasing key numbersDecreasing key numbers

Queries to binary trees

• Searching for a specific node (with key[y]) using a key x• If key x = key[y] then stop. Node has been identified• If key x < key[y] then search in the left branch• If key x > key[y] then search the right branch• If no more branches are left, then nothing has been found• Search time can be done in O(h) (h is height of tree)

• Maximum and minimum search• Minimum search: Follow the left branch until end • Maximum search: Follow the right branch until end • Search time can be done in O(h) (h is height of tree)

Operations on binary trees – Insert node

• Move down the tree comparing key x with key[y] until no more branches are available

• Inserted item is then a child of predecessor

12

5

2 9

18

15 19

1713

12

18

15

Operations on binary trees – Delete node

12

5

3 12

16

23

20

1810 13

6

7

Program of today


Backtracking – what is the problem?

• What if we do not have keys that allows us to do binary search?• A problem has been broken into a set of possible solution

and we need to find the best solution given some constraints

Examples: The Queens problem

• Consider a n by n chess board, and the problem of placing q queens on theboard without the queens threatening one another.

• The solution space is {1,2,3,…,n}n

• Trying all possible solutions implies nn cases, but realising that two queenscannot be on the same row/column, reduces this to n! cases!!

One queen threatensa lot of squares

Ø ØØØ Ø

Ø

Examples: Convex hull (Graham’s Scan)

• The problem asks to construct the shortest polygon which encloses a given set of points on a plan.

• Traverse the points in some order, adding them to the partial solution upon making a turn of less than 180 degrees, and backtracking whenmaking a larger turn.

• The algorithm requires O(n log n) time for sorting the points, and O(n) time to select an appropriate subset.

Intuitively

Start with extremepoint

Now, traverse throughPossible solutions

Backtracking

• A potential solution is represented by a set of vectors S={v1,v2,..,vn}

Is S a solution

For each v in S, do:

Is {v1,v2,..,vn, v}Acceptable? Try {v1,v2,..,vn, v}

Is S != Ø

Try(S)

Return S

Return S

Yes

Yes

Yes

No

No

No

Characteristics of the backtracking algorithm

• Backtracking can be used to traverse through a solution space• Identification of the problem is the firstmost important issue• Description and idenfitication of rules is necessary, but not necessarily

easy!• Can easily reduce the amount of iterations that needs to be done!!

Program of today


Graph definitions

• Graph G=(V,E)• V: set of vertices• E ⊆ VxV: Set of edges• Undirected graph: (v1,v2)∈E⇒(v2,v1)∈E

• Basic operations on graphs• On vertices: Add/Delete/Find• On edges: Add/Delete/Find

• Path := A sequence of edges• P(v1->vn)={(v1,v2), (v2,v3), (v3,v4),.., (vn-1,vn)}• Path length is n

• Application examples• Routing in computer networks• Representation and manipulation of (finite) automatas (parsers, compilers)• Representation of semantic relations• …

A word on greedy algorithm

• Greedy algorithms always picks the option that looks best at the given time

• They do not always lead to an optimal solution, but in many cases they do

The minimum spanning tree

• Problem: To connect a graph with minimum ”cost”

• Example applications: • Wiring PCB’s for electronic devices, • Network planning• Routing in ad hoc networks

∑∈

=Tvu

vuwTw),(

),()(

a

h

b

i

g

c

f

d

e

4

8

11

7

8

2

1

6

2

4

7

14

9

10

Growing a minimum spanning tree

• A is a subset of some minimum spanning tree, T• Generic algorithm for generating a minimum spanning tree

• Safe for A:• Problem is to find an edge that is safe for A

• Kruskal’s algorithm• Prim’s algorithm

1 A ← Ø2 while A does not form a spanning tree3 do find an edge (u,v) that is safe for A4 A ← A∪{(u,v)}5 return A

Kruskal’s algorithm

• ”Brute force” methodology to construct a minimum spanning tree• Idea is to build set of edges E’ in a greedy manner (’local’ decisions)• Algorithm

a

h

b

i

g

c

f

d

e

4

8

11

7

8

2

1

6

2

4

7

14

9

10

1 A ← Ø2 for each vertex v∈V[G]

do MAKE-SET(v)3 sort edges of E into increasing cost by w4 for each edge (u,v)∈E

do if FIND-SET(u)≠ FIND-SET(v)then A ← A∪{(u,v)}

UNION(u,v)5 return A

Prim’s algorithm

• ”Brute force” metode which grows the tree• All vertices not in the tree resides in a

min-priority queue, Q

• End goal, i.e. algorithm termination happenswhen

}}{:])[,{( QrVvvvA −−∈= π

a

h

b

i

g

c

f

d

e

4

8

11

7

2

1

6

2

4

7

14

9

10

8

1 for each u∈V[G]do key[u] ← ∞

π[u] ← NIL2 key[r]←03 Q←V[G]4 while Q≠Ø

do u←EXTRACT-MIN(Q)for each v∈Adj[u]

do if v∈Q and w(u,v)<key[v]then π[v]←u

key[v] ←w(u,v)}}{:])[,{( rVvvvA −∈= π

Properties of the two algorithms

• Running time for the algorithms• Kruskal’s algorithm O(E log V)• Prim’s algorithm (depending on how Q is implemented)

• Adjacency matrix: O(V2)• Binary heap: O(E log V)• Fibonacci heap: O(E+V log V)

• Prim’s algorithm contains at all time a tree, while Kruskal’s algorithm doesnot

Program of today


Genetic Algorithms

• Charles Darwin published in 1859The Origin of Species by Means of Natural Selection

• Survival of the fittest! • Stronger genes survives the battle of life• Weaker genes extinct

• It is the environment that dictates what is fit andwhat is not fit

• Probabilistic search algorithm

The travelling rock band (aka the travelling salesman) problem

The travelling rock band

• Constraints to be considered by the rock band• Minimize the time spend in the air/road• Minimize the cost of the tour in money• Well, the band is not very good so:

- They may not return to a city once visited!

• Example: three randomly picked tours• $17486.01 : Madrid, Vienna, Moscow, Berlin, Brussels, Munich, Milan,

Barcelona, London, Hamburg, Warsaw, Dublin, Kiev, Paris, Rome• $20198.92 : London, Rome, Brussels, Kiev, Hamburg, Warsaw, Barcelona,

Paris, Munich, Dublin, Vienna, Moscow, Madrid, Milan, Berlin • $17799.34 : Madrid, Milan, Kiev, Vienna, Warsaw, London, Barcelona,

Hamburg, Paris, Munich, Dublin, Berlin, Moscow, Rome, Brussels• The question: Are any of these the best trips (in relation to the requirements)?

Darwin put inside a computer

Surviving population ”=” Population ”+” EnvironmentNew population ”=” Surviving population ”+” new generation

Newpopulation

Selection/Environment press

Reproduction& Mutation

Start

Stop

A solution has been found

Describing the ”beast”

• Problem: How do we describe our creatures in our program?• Encoding of chromosomes

• Example of describing the tour of the rock band• Binary encoding for finding the right direction• The trip of the rock band could be

{London, Dublin, Paris, Brussels, Hamburg, Berlin, Warsaw, ….}

• is encoded as {1,0,2,3,8,4,5,...}

• or as binary{0001,0000,0010,0011,1000,0100,0101,...} ………

Brussels311

Paris210

London101

Dublin000

MeaningDecodedCode

Direction codes

Defining the fitness level

• The fitter a ”beast” is, the more likely it is to survive• Fitness level is individual to the problem needed solved• Example: The rock band tour

• Parameter is time of travelling• Definition of a matrix describing travel

time between cities• Parameter is price in e.g. Euros

• Definition of a matrix describing travelcost between cities

• A cost function could be defined as

• The fitness level could be defined as

00.5h2.5hWarsaw

0.5h01.5hBerlin

2.5h1.5h0London

WarsawBerlinLondon

0100750Warsaw

1000500Berlin

7505000London

WarsawBerlinLondon

i1i

i timeprice ⋅+⋅= ∑=

t

N

p wwC

CF 1

=

Setting up a population of ”beasts”

• Initiating the algorithm by defining N numbers of ”beasts” (genes)• Gene values can be based on

• Random selection among valid chromosomes• Qualified initial guesses• A combination of both random and qualified guesses

• This is our initial population

Newpopulation


Reproduction

Start

Stop

Who will survive our environment

• First thing to do is to calculate the fitness of each ”beast”• If there exists solutions such that our goal has been reached (more or

less), then stop• Elitism is a concept that

• Ensures that the best ”beasts” will always pass on to the reproduction• Eliminates the worst of the worst ”beasts”• However, be careful with elitism:

• The population may easily becometoo homogene to make furtherprogress

Newpopulation


Reproduction

Start

Stop

Reproduction is a stochastic process

• Who should pair with whom?• It is basically a game of roullette

Newpopulation


Reproduction &mutation

Start

Stop

1

2

3

4

5

6

7

8

9

Drrrrr rrrr rrrTik tik tik tik tik

• Surface area is determined by thefitness of individual ”beasts”compared to the total sum• -> higher fitness value, the more likely it will be reproduced

Reproduction in a nutshell

• Once two pairs of ”beasts” have been found, they make childrens• Example of how a son/daughter ”beast” is created

• Other cross-over aproaches can also be applied, e.g.• Partially mapped cross-over• Alternating position cross-over• Maximal preservation cross-over• And many other types

Beast A{10,11,10,01,01,01}

Beast B{11,11,10,01,10,11}+

{01,10,11,10,11,10}Beast C (new)

Mutant children• Mutation is used to ensure diversity in the population• Randomly altering a gene, example:

• Similar, mutation can be applied in dedicated ways to produce valid ”beasts”

• Invalid mutations should be removed or as minimum handled in some way• Mutation rate, m, determines how often/how likely a mutation occurs

Beast A{10,11,10,01,01,01}

Beast B{11,11,10,01,10,11}+

{01,10,11,10,11,10}Beast C (new)

{01,11,11,10,11,00}Beast C (new and mutated)

Challenges, traps and disadvantages with Genetic Algorithms

• Genetic algortihms are useful if• A given problem can be specified in terms of a set of parameters and a

cost/fitness value• If there are too many parameters to fully explore the parameter space

within reasonable time

• However,• GA’s may not apply to all types of problems• It may be difficult to define the best way of describing the ”beasts”• It may be difficult to define the fitness functions• It is not guaranteed that the optimal solution for a given problem is

found

Program of today


Assignments

• Project Discussions (15min): Discuss the potential application of graphs in your project. • What type of graphs (directed/un-directed) and what type of edge-

marking may be useful? • What tasks/algorithms would you need for your project to work with

those graphs?

• How many queens can safely be put on a• 4x4 square chess board?• 8x8 square chess board?• 16x16 square chess board?• How is the results affected if you can put other types of pieces on the

chess board, e.g. one or two ponds?

Assignments

• Genetic algorithms: Design and implement an algorithm that is capable of doing simple arithmetics using n numbers• Use the operators +,-,/,*• Use the numbers 0,1,2,3..,8,9• Example: ((3*7+1+0+1+3)/2)+9=15• Experiment with e.g. and see how these affects the algorithm

• different genome types and length, • mutation rate, • selection process for reproduction, • amount of genomes in pool

Assignments

• Assume you are running a transport enterprise in Norway with n cities in your operational scope. The cities are connected by (one-way) streets, but due to the many tunnels there are different height limitations for vehicles. The height limitations are given by a positive cost function on the directed graph G=(V,E). We are interested to find for each pair (vi,vj) of cities the maximum height of the vehicle so that it can reach city vj from vi (possibly also on longer paths).

Assume that the graph including cost function is represented by a matrix A=(aij) as follows: aij=-1 if (vi,vj) ∉E, aij=c(vi,vj) otherwise

References

• GA: Online toturial at http://fog.neopages.org/helloworldgeneticalgorithms.php

• http://ai4r.rubyforge.org/geneticAlgorithms.html#chromosome-impl• Backtracking problems (online): http://www.cse.ohio-

state.edu/~gurari/course/cis680/cis680Ch19.html#QQ1-51-128• Introduction to Algorithms by T.H.Cormen, C.E.Leiserson, R.L.Rivest and

C.Stein• Chapter 12, p. 253-255• Chapter 23, p. 561-579

Algorithms and Architectures II Jens Myrup Pedersen (JMP ...kom.aau.dk/~rlo/lectures/algoAndArch08/mm3.pdfExamples: Convex hull (Graham’s Scan) • The problem asks to construct

Documents