9/28/2016 1 ARTIFICIAL INTELLIGENCE (CSCU9YE ) LECTURE 4: OPTIMISATION AND SEARCH (LOCAL SEARCH) Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ OUTLINE 1. Optimisation problems Optimisation & search Two Examples The Knapsack Problem The Traveling Salesman Problem TSP (next lecture) 2. Optimisation methods Heuristics and metaheuristcis Single point algorithms Population-based algorithms, such as genetic and other evolutionary algorithms (next lecture 2
14
Embed
ARTIFICIAL INTELLIGENCE (CSCU9YE ) LECTURE 4: OPTIMISATION … · 2016-09-28 · strategies to solve problems Heuristics for solving optimization problems, G. Poyla (1945) A method
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
9/28/2016
1
ARTIFICIAL INTELLIGENCE (CSCU9YE )
LECTURE 4: OPTIMISATION AND
SEARCH (LOCAL SEARCH)
Gabriela Ochoa
http://www.cs.stir.ac.uk/~goc/
OUTLINE
1. Optimisation problems
Optimisation & search
Two Examples
The Knapsack Problem
The Traveling Salesman Problem TSP (next lecture)
2. Optimisation methods
Heuristics and metaheuristcis
Single point algorithms
Population-based algorithms, such as genetic and other
• Generally different flavours and methods for solving them
• Have become divergent 9
OPTIMISATION/SEARCH ALGORITHMS
Optimisation algorithms
Exact
Special purpose
Generate bounds: dual ascent,
Langrangean relax
General purpose
Branch and bound
Cutting planes
Approximate
Special purpose
Approximation Greedy /
Constructive Heuristics
Meta and Hyper heuristics
Single point Population based
• Guarantee finding optimal solution
• Useful when problems can be solved in
Polynomial time, or for small instances
• Do not Guarantee finding optimal solution
• For most interesting optimisation problems no
polynomial methods are known
Approximation algorithms:
• An attempt to formalise heuristics (emerged from the field of theoretical computer science)
• Polynomial time heuristics that provide some sort of guarantee on the quality of the solution 10
9/28/2016
6
THE KNAPSACK PROBLEM
Given a knapsack of capacity W, and a number n of items, each with
a weight and value. The objective is to maximise the total value of
the items in the knapsack
maximise
4x1+2x2+x3+10x4 +2x5
subject to
12x1+2x2+x3+4x4+x5 15
x1,x2,x3,x4,x5 {0, 1}
Xi = 1 If we select item i
0 Otherwise
• Binary representation [11010]
Maximise Subject
to
• Search space size = 2n
• n = 100, 2100 ≈ 1030
11
REAL-WORLD EXAMPLE OF THE KNAPSACK
PROBLEM
Consider a cargo company, that has an airplane
and need to carry packages.
Customers state the weight of the cargo item
they would like delivered, and the amount they
are prepared to pay.
The airline is constrained by the total amount of
weight the plane is allowed to carry.
The company must choose a subset of the
packages (bids) to carry in order to make the
maximum possible profit, given the weight limit
that they must respect. 12
9/28/2016
7
LOCAL SEARCH METHODS FOR
OPTIMISATION
Exhaustive enumeration, only possible for small
problems
Random search
Hill-climbing
Steepest (Best) ascent
First (Next)ascent
Multi-start hill-climbing
Iterated local search
Simulated annealing
13
EXHAUSTIVE ENUMERATION
Try out all possible ways of packing/leaving out the items
For each way, it is easy to calculate the total weight carried and the total value carried
Consider the following knapsack problem instance: 3
1 5 4
2 12 10
3 8 5
11
Where: The first line gives the number of items. The last line gives the capacity of the knapsack. The remaining lines give the index, value and weight of each item.
14
9/28/2016
8
KNAPSACK, FULL ENUMERATION
Items Value Weight Feasible?
000 0 0 Yes
001 8 5 Yes
010 12 10 Yes
011 20 15 No
100 5 4 Yes
101 13 9 Yes
110 17 14 No
111 25 19 No
15
Optimal!!
THE NOTION OF NEIGHBOURHOOD
S x
N(x)
A search space S, a potential
solution x, and its neighbourhood
N(x)
Region of the search space that is “near” to some particular
point in that space
Define a distance function dist on the search space S
Dist: S x S → R
N(x) = {y Є S: dist(x,y) ≤ ε }
Examples: • Euclidean distance, for search spaces
defined over continuous variables
• Hamming distance, for search spaces
defined over binary strings
• Hamming distance between two strings is
the number of positions at which the
corresponding symbols are different. 16
9/28/2016
9
DEFINING NEIGHBOURHOODS: BINARY
REPRESENTATION
1-flip mutation: flipping a single bit in the given bit
string
For strings of length n, every solution has n neighbours
Example: 1 1 0 0 1 → 0 1 0 0 1,
Python implementation
17
HILL-CLIMBING SEARCH
Like climbing a mountain in thick fog with amnesia
18
Best Improvement (gradient descent, greedy hill-climbing): Choose maximally
improving neighbour
First Improvement: Choose the first found improving move.
Local optimum: No other solution in the neighbourhood has better fitness
9/28/2016
10
HILL-CLIMBING SEARCH IN PYTHON
19
HILL-CLIMBING SEARCH
Problem: depending on initial state, can get stuck in local maxima
20
9/28/2016
11
TERMINOLOGY AND DATES
Heuristic: Greek word heuriskein, the art of discovering new
strategies to solve problems
Heuristics for solving optimization problems, G. Poyla (1945)
A method for helping in solving of a problem, commonly informal
“rules of thumb”, educated guesses, or simply common sense
Prefix meta: Greek for “upper level methodology”
Metaheuristics: term was introduced by Fred Glover (1986).
Other terms: modern heuristics, heuristic optimisation, stochastic
local search
21
WHAT IS A METAHEURISTIC?
Extended variants of local search
General-purpose solvers, usually applicable to a large
variety of problems
Use two phases during search
Intensification (exploitation): focus the applications of