Applications of Dynamic Programming and Heuristics to the Traveling Salesman Problem ERIC SALMON & JOSEPH SEWELL
Jan 04, 2016
Applications of Dynamic Programming and Heuristics to the Traveling Salesman ProblemERIC SALMON & JOSEPH SEWELL
Traveling Salesman Problem
Distances between n cities are stores in a distance matrix D with elements dij where i, j = 1 … n and the diagonal elements dii are zero. A tour can be represented by a cyclic permutation π of { 1, 2, …, n} where π(i) represents the city that follows city i on the tour. The traveling salesman problem is then the optimization problem to find a permutation π that minimizes the length of the tour denoted by:
Hasler & Hornik (Journal of Statistical Software, 2007 V. 23 I. 22)
Heuristic Algorithms for TSP
A heuristic is a technique for solving a problem quicker when other methods are too slow or for finding an approximate solution to a problem.
Random Search
Generate random permutation for a tour
Genetic Algorithm
Mimic evolution to arrive at a tolerable tour
Simulated Annealing
Find a solution by moving slowly towards a global optimum without being trapped in local optimums
Genetic Algorithm
A genetic algorithm is a search heuristic that mimics the process of natural selection.
Simulated Annealing
Name inspired from metal work
Heating and cooling an object to alter its properties
While the algorithm is ‘hot’, it is allowed to jump out of its local optimums
As the algorithm ‘cools’ it begins to hone on the global optimum
Simulated Annealing (cont.)
Dynamic Programming
A method for solving complex problems by breaking them down into simpler sub-problems
Exploits sub-problem overlap
Example: Finding Fibonacci numbers.
F(n) = F(n-2) + F(n-1)
To find F(n) you must also compute F(n-2) and F(n-1)
These values will be recomputed for each F(n) you want to find
Using Dynamic Programming, every computed value would be stored which would then be looked up before computation.
Computing Fibonacci (naïve)
fib(n)
if n <= 2 : f = 1
else : f = fib(n-1) + fib(n-2)
return f
Dynamic Programing: Fibonacci
array = {}
fib(n):
if n in array: return array[n]
if n <= 2 : f = 1
else: f = fib(n-1) + fib(n-2)
array[n] = f
return f
Branch & Bound
An algorithm design for optimization problems.
Enumerations are possible solutions
Candidate partial solutions are child nodes from the root
Before enumerating child node, this branch is checked against upper/lower bounds compared to optimal solution
In the case of TSP this would be total distance up to that node
If this value is greater than the bound, discard entire branch
No added distance would ever decrease total distance
Continue enumeration through tree until solution found
Branch & Bound
is an algorithm design paradigm for discrete and combinatorial optimization problems. A branch-and-bound algorithm consists of a systematic enumeration of candidate solutions by means of state space search: the set of candidate solutions is thought of as forming a rooted tree with the full set at the root. The algorithm explores branches of this tree, which represent subsets of the solution set. Before enumerating the candidate solutions of a branch, the branch is checked against upper and lower estimated bounds on the optimal solution, and is discarded if it cannot produce a better solution than the best one found so far by the algorithm.
http://en.wikipedia.org/wiki/Branch_and_bound
Branch & Bound
A branch-and-bound procedure requires two tools. The first one is a splitting procedure that, given a set S of candidates, returns two or more smaller sets S1, S2, … whose union covers S. Note that the minimum of f(x) over S is min{v1, v2, …}, where each vi is the minimum of f(x) within Si. This step is called branching, since its recursive application defines a search tree whose nodes are the subsets of S.
http://en.wikipedia.org/wiki/Branch_and_bound
Branch & Bound
The second tool is a procedure that computes upper and lower bounds for the minimum value of f(x) within a given subset of S. This step is called bounding.
The key idea of the BB algorithm is: if the lower bound for some tree node (set of candidates) A is greater than the upper bound for some other node B, then A may be safely discarded from the search. This step is called pruning, and is usually implemented by maintaining a global variable m (shared among all nodes of the tree) that records the minimum upper bound seen among all sub-regions examined so far. Any node whose lower bound is greater than m can be discarded.
http://en.wikipedia.org/wiki/Branch_and_bound
Branch & Bound
The recursion stops when the current candidate set S is reduced to a single element, or when the upper bound for set S matches the lower bound. Either way, any element of S will be a minimum of the function within S.
http://en.wikipedia.org/wiki/Branch_and_bound
Branch & Bound on TSP
Given:
Branch & Bound on TSP