This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Solving Minimax Problems using an HeuristicPattern Search Algorithm
Isabel A.C.P. Espírito Santo and Edite M.G.P. Fernandes
For sufficiently large αj , the minimizer of the minimax problemcoincides with solution of inequality problem.
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 6 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Optimality conditions
Stationary point (Xu, 2001)
x∗ is a stationary point to the minimax problem, if there existelements λ∗j ≥ 0, j = 1, . . . ,m such that∑m
j=1 λ∗j∇Fj(x∗) = 0 and∑m
j=1 λ∗j = 1,
andλ∗j = 0 if Fj(x∗) < maxF1(x∗), . . . , Fm(x∗).
Theorem (Xu, 2001)
If x∗ is a local minimum of minimax problem, then it is a stationarypoint. Conversely, if f(x) is convex and x∗ is a stationary point,then it is a global minimum to the minimax problem.
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 7 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Difficulties
Minimax problems are difficult to solve through traditionalgradient based algorithms.
First derivatives of f(x) are discontinuousat points where f(x) = Fj(x)
for two or more values of j in the set 1, . . . ,m,even if all the functions Fj(x) have continuous first derivatives.
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 8 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Tools of Heuristic Pattern Search Method
Our proposal:1 uses a derivative-free method, known as pattern search
method, as outline in Lewis & Torczon (1999);2 is based on the Hooke and Jeeves moves - exploratory move +
pattern move - Hooke & Jeeves (1961);3 and uses an heuristic move - a random descent walk - Hedar
& Fukushima (2004)
to obtain high accuracy solutions
⇒ the pattern move is followed by a random descent walk, when asuccessful iterate is encountered.
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 9 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Outline
1 Introduction
2 Pattern search method - Hooke and Jeeves
3 Heuristic Pattern Search
4 Numerical Experiments and Conclusions
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 10 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Hooke and Jeeves pattern search method
Let xk ∈ IRn be the iterate at iteration k;Let ∆k be the step length;
The Hooke and Jeeves (HJ) method performs two types of moves:the exploratory move carries out a coordinate search - asearch along the coordinate axes - about a selected iterate,with a step size ∆k;when xk is a successful iterate, the pattern move - apromising direction - is defined by xk − xk−1.
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 11 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Hooke and Jeeves moves
When iterate xk is successful ⇒ pattern move, followed by anexploratory move about xk + (xk − xk−1):
in IR2
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 12 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Hooke and Jeeves moves
When pattern move is unsuccessful ⇒ exploratory move about xk:
in IR2
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 13 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Outline
1 Introduction
2 Pattern search method - Hooke and Jeeves
3 Heuristic Pattern Search
4 Numerical Experiments and Conclusions
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 14 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Heuristic pattern move
Heuristic pattern search method performs two types of moves:
the exploratory moveis a coordinate search about a selected iterate, with a step size∆k;
a two-stage moveis a pattern move followed by an approximate descentrandom search - defined by xk − xk−1 + dk - when xk is asuccessful iterate.
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 15 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Approximate descent random search
Let pk = xk + (xk − xk−1)based on two points y1 and y2, randomly generated from theneighborhood of pk;an approximate descent search for f at pk is
dk = − 1∑2j=1 |∆fj |
2∑i=1
(∆fi)pk − yi
‖pk − yi‖,
where ∆fj = f(pk)− f(yj).
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 16 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Heuristic descent move
When iterate xk is successful ⇒ a pattern move followed by adescent random search dk at xk + (xk − xk−1):
in IR2
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 17 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Heuristic descent move
When pattern move is unsuccessful ⇒ exploratory move about xk:
in IR2
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 18 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Constraining for feasibility
To maintain iterate in Ω, a reflexion into the feasible region iscarried out - componentwise (for i = 1, . . . , n)
xk,i =
li + (li − xk,i) if xk,i < li
xk,i if li ≤ xk,i ≤ ui
ui − (xk,i − ui) if xk,i > ui
If a component of an iterate still is out of the bounds thenxk,i = (li + ui)/2.
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 19 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Outline
1 Introduction
2 Pattern search method - Hooke and Jeeves
3 Heuristic Pattern Search
4 Numerical Experiments and Conclusions
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 20 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Details
Algorithms coded in C programming language with AMPL interfaceto read problems coded in AMPL:
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 23 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Table of results - HJ pattern search / Heuristic patternsearch
P - Problem; f∗ - best known solution in the literature;
HJ pattern search (deterministic method):Nit− number of iterations to achieve the desired accuracyNfe− number of objective function evaluationssolution - obtained solution according to termination conditions.
Heuristic pattern search (stochastic method) - each problemwas run 100 times:
AvNit− average number of iterations, over the 100 runsAvNfe− average number of function evaluations, over the 100 runssolution - best of the solutions found in the 100 runsaverage - average of the solutions found in the 100 runs.
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 24 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Tests with the heuristic pattern search algorithm
With the heuristic pattern search we solve1 each problem 100 times, using the 6 values of γ∆ - total of
600 runs for each problem;2 for each γ∆, choose the best run (over the 100 runs) - with
solution closest to f∗;3 choose the γ∆ that gives solution closest to f∗;
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 25 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 30 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
Conclusions and Future Work
We presented a derivative-free pattern search search methodthat incorporates an heuristic descent random walk, when asuccessful iterate is found:
1 it improves solution accuracy;
2 it solves difficult non-differentiable problems - bound minimaxproblems;
3 it is easy to implement.
Future development: extend heuristic pattern search to equality andinequality constrained problems, using an augmented Lagrangianfunction - penalty multiplier method.
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 31 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions
References
A.-R. Hedar and M. Fukushima, Heuristic pattern search and its hybridizationwith simulated annealing for nonlinear global optimization, OptimizationMethods and Software, 19 (2004) 291–308.
R. Hooke and T. A. Jeeves, Direct search solution of numerical and statisticalproblems, Journal on Associated Computation, 8 (1961) 212–229.
E. C. Laskari, K. E. Parsopoulos and M. N. Vrahatis, Particle swarmoptimization for minimax problems, Proceedings of IEEE 2002 Congress onEvolutionary Computation, ISBN: 0-7803-7278-6, 1576–1581, 2002.
R. M. Lewis and V. Torczon, Pattern search algorithms for bound constrainedminimization, SIAM Journal on Optimization, 9 (1999) 1082–1099.
L. Lukšan and J. Vlček, Test problems for nonsmooth unconstrained and linearlyconstrained optimization, TR 798, ICS, Academy of Science of the CzechRepublic, January 2000.
V. Torczon, On the convergence of pattern search algorithms, SIAM Journal onOptimization, 7 (1997) 1–25.
S. Xu, Smoothing method for minimax problems, Computational Optimizationand Applications, 20 (2001) 267–279.
IACP Espírito Santo and EMGP Fernandes Minimax Problems using an Heuristic Pattern Search 32 / 33
IntroductionPattern search method - Hooke and Jeeves
Heuristic Pattern SearchNumerical Experiments and Conclusions