Lecture: 5 Optimization Methods & Heuristic Strategies
Ajmal Muhammad, Robert ForchheimerInformation Coding Group
ISY Department
Outline
Introduction to Optimization & Intractable Problems Linear Programming Heuristic Metaheuristic Linear Programming Formulation
Shortest path Problem
Max Flow Problem
Wavelength Assignment Problem
Optimization Problem
Problem that aims to find the best solution from all feasible solutions
The best solution can be the minimum or maximum solution
Examples:
Find the route from point A to point B that takes the shortest timesDetermine how a production factory can maximize its profit using limited materials
The Travelling Salesman Problem (TSP)
A classic combinatorial optimization problem
Given n cities on a map, find the shortest route that visits all the cities once, and starts and ends at the same city
Starting and ending in Paris, whichroute allow us to visit major citieswith the least amount of travelling ?
The 1-Dimensional Bin Packing Problem
Given n items of different (1D) sizes, and given some fixed- capacity bins, pack the items into a minimum number of bins
Is this the minimal number of Bins needed for these items?
TSP Growth Rates A route around a map can be represented as a permutation of the n
cities:
E.g. for 5 cities, [B,A,C,E,D] means “start at city B, then go to city A, then city C, then E, then D, and return to city B”
Given n cities, there is a total of n! permutations Some permutations represent the same routes - there are actually
½(n-1)! different routes in total To return the provably optimal solution to any instance of the TSP
the algorithm needs to check the majority of – if not all – possible routes
The number of routes grows exponentially quickly making the problem intractable:
Linear Programming (LP)
An optimization problem in which the objective function and all the constraints are expressed as linear functions
even if just one of them is not a linear function, the problem is not an LP problem
Linear programming problem
Nonlinear programming problem
LP Formulation
Matrix expression for LP formulation
Nomenclature in LP Problem
Boundary is a constraint that express the upper or lower bound of an inequality or equality
Feasible region is an area delineated by the boundaries
Corner point is an intersection of the boundaries
Example
Let z= x+y max zRewrite as y= -x + z
Solution by Moving y = -x +z
Simplex Method
Heuristics
• Provide a shortcut to solve difficult problems.• Used under limited time and/or information to make a
decision.• Problem-dependent techniques, i.e., adapted to the
problem at hand to take advantage of the particularities of the problem.
• Lead to a good decision most of the time.
Heuristics are good when Making a spur-of-the-moment decision Having limited information and cannot obtain more The decision is not that important
Heuristics
Heuristics are bad whenHaving plenty of time and information to make an important decisionNeed to be right 100% of the time
Examples of heuristicsDeciding to eat at restaurant B rather than restaurant A only because B has more cars in its parking lotWhen the level in the fuel tank drops to 1/2 tank or less, always filling up the fuel tank at the very next re-filling station
Metaheuristic
Meta: in an upper level
Heuristic: to find
A metaheuristic is formally defined as an iterative generation process which guides a subordinate heuristic by combining intelligently different concepts for exploring and exploiting the search space, learning strategies are used to structure information in order to find efficiently near-optimal solutions.
A metaheuristic is a function that consists of heuristic subroutines. It is a compound heuristic.
Fundamental Properties of Metaheuristics
Metaheuristics are strategies that guide the search process to avoid being trapped in confined areas of the search space
The goal is to efficiently explore the search space in order to find (near)-optimal solutions
Techniques which constitute metaheuristic algorithms range from simple local search procedures to complex learning processes
Metaheuristic algorithms are approximate and usually non-deterministic
Metaheuristics are not problem-specific
Metaheuristic Examples
Genetic algorithms (1960s/1970s), Evolutionary strategy (Rechenberg & Swefel 1960s), Evolutionary programming (Fogel et al. 1960s).
Simulated annealing (Kirkpatrick et al. 1983), Tabu search (Glover 1980s), Ant colony optimization (Dorigo 1992), Genetic programming (Koza 1992), Particle swarm optimization (Kennedy & Eberhart 1995), Differential evolution (Storn & Price 1996/1997),
Harmony search (Geem et al. 2001), Honeybee algorithm (Nakrani & Tovey 2004), ..., Firefly algorithm (Yang 2008), Cuckoo search (Yang & Deb 2009), ...
Genetic Algorithms: TSP Example
Genetic Algorithms: TSP Example
Genetic Algorithms: TSP Example
Genetic Algorithms: TSP Example
Genetic Algorithms: TSP Example
Ant Colony Optimization (in one Slide)
A. Ants in a pheromone trail between nest and food; B. an obstacle interrupts the trail; C. ants find two paths to go around the obstacle; D. a new pheromone trail is formed along the shorter path.
LP Formulation for Shortest Path Problem
Network is represented by directed graph G(V,E), where V is the set of vertices (nodes) & E is the set of links
Link from node i to node j is expressed by (i,j); dij is the link cost of (i,j)
Xpqij : traffic volume from node p to node q routed through (i,j)
Max flow Problem
Network model with link capacities
Max flow routing from node 1 to node 6
ILP formulation for max flow problem ?Assignment
Wavelength Assignment (WA) Problem
Optical path requests
Graph construction for coloring
LP Formulation for Graph Coloring
Let W be a set of λ, where W = {λ1,λ2,··· ,λ|W|}
Let xλv and yλ, be binary variables.
If λ is assigned to a path corresponding v, xλv = 1, otherwise xλ
v = 0.
If λ is used at least one time yλ=1, otherwise yλ=0.
Heuristic: Largest Degree First (LDF)
Wavelength assignment process of LDF
Step 1: select the uncolored vertex with largest degree.
Step 2: choose the min indexed colorfrom the colors that are not used by the adjacent vertices.
Step 3: color the selected vertex using the color chosen in step 2.
Step 4: if all vertices are colored, LDF stops. otherwise, LDF returns to step 1.