Global Optimization Toolbox

8/2/2019 Global Optimization Toolbox

1/12

G lobal Optim ization Toolbox 3 .1Solve multiple maxima, multiple minima, and nonsmooth optimization problems

Global Optimization Toolbox provides methods that search for global solutions to problems that contain multiplemaxima or minima. Itincludes global search, multi start, pattern search, genetic algorithm, and simulatedannealing solvers. You can use these solvers to solve optimization problems where the objective or constraintfunction is continuous, discontinuous, stochastic, does not possess derivatives, or includes simulations orblack-box functions with undefined values for some parameter settings.Genetic algorithm and pattern search solvers support algorithmic customization. You can create a custom geneticalgorithm variant by modifying initial population and fitness scaling options or by defining parent selection,crossover, and mutation functions. You can customize pattern search by defining polling, searching, and otherfunctions.

100 -100

100

Plotof different types of problems Global Optimization Toolbox can solve: nonsmooth (top), single global minimumnested inside multiple local minima (left), multiple local minima with no global minimum (right).

Key FeaturesInteractive tools for defining and solving optimization problems and monitoring solution progressGlobal search and multi start solvers for finding single or multiple global optimaGenetic algorithm solver that supports linear, nonlinear, and bound constraints

MathWorks A C D e r J a r o , i ! r 9 ' J h 8 'p a co o f t 1n g in e tJ .r in g a n d s c r. i3 'n c e


2/12

Multiobjective genetic algorithm with Pareto-front identification, including linear and bound constraintsPattern search solver that supports linear, nonlinear, and bound constraintsSimulated annealing tools that implement a random search method, with options for defining annealingprocess, temperature schedule, and acceptance criteriaParallel computing support in multi start, genetic algorithm, and pattern search solversCustom data type support in genetic algorithm, multiobjective genetic algorithm, and simulated annealingsolvers

" I~ 2S' ++: : 5 2 D '~" . ,o\--:--; ' ; ,-----:c-----;;;-----o;:c--~---c:;-;;;--:c-----:.I l ' l ! r a l i o o

CI l l f I ? " l L " : i - ~S.;:~ ~.5)1j71!-(l(l7

Opt'eonl~Iy( p1P:.IWIDr.h~P3:LI,r.lSe~rt.,- s O M ! . r _

~ I > .M o I! S l O P I I ~ ~ ! ~ ! ! ! ! ! ! ! I.-,"~''''~CEEDc~~.fI:i r.~~""""1M.~m",n.r..,.,~...." !~_~~~~OIm . ..>o~tM:.",g.>,...btrun~i~

1ZI~U.:tW1"'-IZI"""" '. ORn;lio:Q:


3/12

Capture and reuse work performed in the GUI using MATLAB code generationYou can also customize the solvers by providing your own algori thm options and custom functions. Multistartand global search solvers are accessible only from the command line .

.;~--st-n-~ i-"-'-Fu- n-rn-u-"- ---------- ---------- ---------- ~~~

'C . . . - . : ; ; ! tD i n I : ; ; :li1w,,~lks: J!t;~ b,;~l 1 - i e . 3 r e > ! l L B 1 i b " ~ : A~,: ~ bl!!ll, : ~ Iriti.! illpop,A tioo:_'o """,c~""",c~H o rh a r c ,o o :; .1 r


4/12

~-" "'_ , r : ~ " ": : : _ : : : : : ; ' " '= - - -- = = :J - - '~Qi~!~

~~~,~OOn t . t


5/12

While an optimization is running, you can change some options to refine the solution and update performanceresults in genetic algorithm, multiobjective genetic algorithm, simulated annealing, and pattern search solvers. Forexample, you can enable or disable plot functions, output functions, and command-line iterative display duringrun time to view intermediate results and query solution progress, without the need to stop and restart the solver.You can also modify stopping conditions to refine the solution progression or reduce the number of iterationsrequired to achieve a desired tolerance based upon run-time performance feedback.Global Search and Multistart SolversThe global search and multi start solvers use gradient-based methods to return local and global minima. Bothsolvers start a local solver (in Optimization Toolbox) from multiple start ing points and store local and globalsolutions found during the search process.The global search solver:

Uses a scatter-search algorithm to generate multiple starting pointsFilters nonpromising start points based upon objective and constraint function values and local minimaalready foundRuns a constrained nonlinear optimization solver to search for a local minimum from the remaining startpoints

The multi start solver uses either uniformly distributed start points within predefined bounds or user-defined startpoints to find multiple local minima, including a single global minimum if one exists. The multi start solver runsthe local solver from all starting points and can be run in serial or in parallel (using Parallel ComputingToolbox"). The multistart solver also provides flexibility in choosing different local nonlinear solvers. Theavailable local solvers include unconstrained nonlinear, constrained nonlinear, nonlinear least-squares, andnonlinear least-squares curve fitting.

Nonlinear Optimizat ion with the Global Search Solver 3:57Local and global minima of the peaks function are found.

Nonlinear Regression with the Multistart Solver 4: 16The best-fitparameters for an exponential model are found.

Genetic Algorithm SolverThe genetic algorithm solves optimization problems by mimicking the principles of biological evolution,repeatedly modifying a population ofindividual points using rules modeled on gene combinations in biologicalreproduction. Due to its random nature, the genetic algori thm improves your chances of finding a global solution.It enables you to solve unconstrained, bound-constrained, and general optimization problems, and it does notrequire the functions to be differentiable or continuous.

MathWorks A C D e r J a r o , i ! r 9 ' J h 8 'p a co o f t 1n g in e tJ .r in g a n d s c r. i3 'n c e 5


6/12

The following table shows the standard genetic algorithm options provided by Global Optimization Toolbox.

Step Genetic Algorithm OptionCreation Uniform, feasibleFitness scaling Rank-based, proportional, top (truncation), shift linearSelection Roulette, s tochastic uniform selection (SUS), tournament, uniform, remainderCrossover Arithmetic, heuristic, intermediate, scattered, single-point, two-pointMutation Adaptive feasible, Gaussian, uniformPlotting Best fitness, best individual, distance among individuals, diversity of population,

expectation of individuals, max constraint, range, selection index, stoppingconditions

Global Optimization Toolbox also lets you specify:Population sizeNumber of elite childrenCrossover fractionMigration among subpopulations (using ring topology)Bounds, linear, and nonlinear constraints for an optimization problem

You can customize these algorithm options by providing user-defined functions and represent the problem in avariety of data formats, for example by defining variables that are integers, mixed integers, categorical, or complex.You can base the stopping criteria for the algori thm on time, stalling, fitness limit, or number of generations. Andyou can vectorize your fitness function to improve execution speed or execute the objective and constraintfunctions in parallel (using Parallel Computing Toolbox).



7/12

.J O pt in m .: a ti on T o ol Q@~F il e H el p

I!}MgratorC,llI\"Ir~I:5;LkiM"neq Ji!Ii~e~: A:c=J b:c=JL i '1 e -MeoqL I al t i eS : ,II,-eq:c=J bE!q:c=JB : D 1 S 1 d ' ~ : lower:=Jupper: c=J~rdnBtr2lint1i.VJcI:om '------------'

I!AJqor1t1m~t~ngs~Hyi) I i: : I -Aslct ior1

HybndMctio" I L " " ' c : : = ' ~ , , - - - - ~v lO~IXl: i: eU :; o; ;: d~ I: [J

O,~ry, ,--------,

D l J S ! ! " r...-.dom si.lIi~s from pr~;:llJ"lIl~~~(a'~litilW.!I~o(); 100

PI ( Jo t f t l l e f ' J a l :

~g,~t~~~

DE~,1""d:afurr

Ds.;ore.iver~t)'p~l'I'lifatiOi'li~rirIgS , ,. . ;: td ' m , , ~ h y b li d n . .. :. c ti o rr .Op~mr!!IjN W'n\!"J!tI!:dObB[1iv~ f u n d io o v d \ Je : ~.9ao500:roo7! JS4.5Qp~m . .. l ; oo terrW.;!ted: ms: .Wnurn ~mboer o! gene'a~~e:E:e~dF M I N I , . . i \ I C 'Loc.! lrn ...-wIl ' l I iS~h!:,

D;;";ore:>

OM~'W:Str.antD"'""""

,~F!r:I;i,jtI{IIi:W

D"'' ' ' 'YDC"' ' ' ' 'c : ~ : :telleiofdi

furJLJff"~tlpp@dbS :aus@th!!~j@of l i !!C .JrTent!t !pi!18! ; I ha n ~"e d'e faultl ,o '~lweor the ,rep:size IdErllr'll:e B@8t:3 ,OC63M@iin:J.0148

ilM1gn~ Iueao ntness

f lnaloolll:

GenerationCUIl I;I Il I S~~Indil'idwalL ~ l~ : = : = ~: : : I2 3 4 5 6 a 9 10Ncmber ot vsriebles ( 10)

Output that shows solutions reached when using only the genetic algorithm (right, bar chart) and when using thegenetic algorithm with a gradient-based solver from Optimization Toolbox (finalpoint in Optimization Tool, left).Combining algorithms can produce more accurate results while reducing the number of function evaluations requiredby the genetic algorithm alone.

Multiobjective Genetic Algorithm SolverMultiobjective optimization is concerned with the minimization of multiple objective functions that are subject toa set of constraints. The multi objective genetic algorithm solver is used to solve multi objective optimizationproblems by identifying the Pareto front-the set of evenly distributed nondominated optimal solutions. You canuse this solver to solve smooth or nonsmooth optimization problems with or without bound and linearconstraints. The multiobjective genetic algorithm does not require the functions to be differentiable orcontinuous.

MathWorks A C D e r J a r o , i ! r 9 ' J h 8 ' p a c o o f t 1 ng i ne t J. r in g an d sdi3'nce 7


8/12

The following table shows the standard multi objective genetic algorithm options provided by Global OptimizationToolbox.

Step Multiobjective Genetic Algorithm OptionCreation Uniform, feasibleFitnessscaling Rank-based, proportional, top (truncation), linear scaling, shiftSelection TournamentCrossover Arithmetic, heuristic, intermediate, scattered, single-point, two-pointMutation Adaptive feasible, Gaussian, uniformPlotting Average Pareto distance, average Pareto spread, distance among individuals,

diversity of population, expectation of individuals, Pareto front, rank histogram,selection index, stopping conditions

Global Optimization Toolbox also lets you specify:Population sizeCrossover fractionPareto fractionDistance measure across individualsMigration among subpopulations (using ring topology)Linear and bound constraints for an optimization problem

You can customize these algorithm options by providing user-defined functions and represent the problem in avariety of data formats, for example by defining variables that are integers, mixed integers, categorical, or complex.You can base the stopping criteria for the algori thm on time, fitness limit, or number of generations. And you canvectorize your fitness function to improve execution speed or execute the objective functions in parallel (usingParallel Computing Toolbox).



9/12

WM'M, i ! l l 1 I l i I 1 1 ! ! 1'" . . . .,....,."',~"~!!ll'~".-:I~I..,..~i IIt I' .!o .:;:.nIifQ ~lio: ....... , MultiQbjectiveGenetic AlgorithmF ~ l io o n o = jd;w_ Solver

il l isW1llco,r.e-.pundslutMiQ~"-~ ':~obj furn:lionCk ttu E ' lq ) o 3 nd t h~ : : l ~c t il R l

Ac==J b: c==J i~e~~~~""g~o~di"\l to ~ 1 l \ J '~I!QII;JI",," . . . . ,_ 'C==J b e q = c = = J C>i 1 ! K P ! ~ ~ ~ e l ! ! < ! . . . .~I;MlW"P:~

L_~~~ O~Fr'C===C prO~Iim S i t t u - p ~nJ:ll ' : : i i < S -UC I . S Pm b ff ln

Op ' t l , H I I ! ; -S p o~ i ly o i> M Il 5 r lJ r' n ~ G e o e 1 1 ! ( ;AJgoilrithm~ar. 'E"'' ,"ooo1i11ron

0......."DSouppioJoJ

O~_P",~:~di."""~ D-..,.J~~=~=-~~""~~""~~~~~~~~~~ID~o..!tc.:n~ t:==::D~~"'l'ID~- 1 n _ : 1 iDci.l!ltco!n~ c : : : : = =

!; p!!ralll'!!'i:.er.s. tp, - 0 .13 . ;:c ~ ~ :!! = '!.e"g",~ ("-, ..

"4 - i = 1:r.;fl~) ".:I1.l(;l(.al:l::l~Ali);.~.a"5~~iIJ.(A~i).~:b)I;

I ~ F 0 1Multiobiective genetic algorithm defined in the Optimization Tool (top), used to identify the Pareto front containingdisconnected regions (middle) for the Kursawe function (bottom).

Pattern Search SolverGlobal Optimization Toolbox contains three direct search algorithms: generalized pattern search (GPS),generating set search (GSS), and mesh adaptive search (MADS). While more traditional optimization algorithmsuse exact or approximate information about the gradient or higher derivatives to search for an optimal point,these algori thms use a pattern search method that implements a minimal and maximal positive basis pattern. Thepattern search method handles optimization problems with nonlinear, linear, and bound constraints, and does notrequire functions to be differentiable or continuous.



10/12

The following table shows the pattern search algorithm options provided by Global Optimization Toolbox. Youcan change any of the options from the command line or the Optimization Tool.

Pattern Search Option Description

MathWorks

Polling methods Decide how to generate and evaluate the points in a pattern and the maximumnumber of points generated at each step. You can also control the polling orderof the points to improve efficiency.

Choose an optional search step that may be more efficient than a poll step. Youcan perform a search in a pattern or in the entire search space. Global searchmethods, like the genetic algorithm, can be used to obtain a good starting point.

Search methods

Mesh Control how the pattern changes over iterations and adjusts the mesh forproblems that vary in scale across dimensions. You can choose the initial meshsize, mesh refining factor, or mesh contraction factor. The mesh acceleratorspeeds up convergence when it is near a minimum.

Cache Store points evaluated during optimization of computationally expensiveobjective functions. You can specify the size and tolerance of the cache that thepattern search algorithm uses and vary the cache tolerance as the algorithmproceeds, improving optimization speed and efficiency.

Nonlinear constraint algorithmsettings

Specify a penalty parameter for the nonlinear constraints as well as a penaltyupdate factor.

Thrs torn cmrBSfI :mds ~~tll~~"-~,.~~,,-~e~~e" Iuoc~ionCllGk~~e~laJld lh~ seLtioo below~OITe5J11~ding10 ~om ta~kFrllb emSeil.lpil1d~esl.llt~.~ CM$lmi~~", ;:!"r i~Mr ~~ , o a w IM~L!

.~~~'M10~~MMlill,,~: : ; ; : : : : : : . 0 .

co)e:li> o!M.:I!OtI.-.-GZi'M- c :p tm ' ' ' il i ' 1 . . . . . . . . . , ' ' ' " ' ' ' ' ' ' ' ' ' , . . . , . ; . , . ; ' ' ' ' ' ' . . . , . , _ , . T ~ .

!J;It~!o:\ol~rM~~~~"'Iii-;;- ~1"''''~lii-i;j.


11/12

Simulated Annealing SolverSimulated annealing solves optimization problems using a probabilistic search algorithm that mimics the physicalprocess of annealing, in which a material is heated and then the temperature is slowly lowered to decrease defects,thus minimizing the system energy. By analogy, each iteration of a simulated annealing algori thm seeks toimprove the current minimum by slowly reducing the extent of the search.The simulated annealing algori thm accepts all new points that lower the objective, but also, with a certainprobability, points that raise the objective. By accepting points that raise the objective, the algorithm avoids beingtrapped in local minima in early iterations and is able to explore globally for better solutions.Simulated annealing allows you to solve unconstrained or bound-constrained optimization problems and does notrequire that the functions be differentiable or continuous. From the command line or Optimization Tool you canuse toolbox functions to:

Solve problems using adaptive simulated annealing, Boltzmann annealing, or fast annealing algorithmsCreate custom functions to define the annealing process, acceptance criteria, temperature schedule, plottingfunctions, simulation output, or custom data typesPerform hybrid optimization by specifying another optimization method to run at defined intervals or atnormal solver termination

i"il~ Ii lJ!lp

Pt~bern

MathWorks

Problem Setup and Results OptkJons ~t - , ~ o ' " , - , ~ ; ~ , , " " = ' ' ' = '' '= ' = , , " = ' = ~ S = i ' = ' '' = ' '= ' = 8 0 = ' ' = ' '= " ' = " = " = " = " " = = : : : ; : : ' : : ;- ] 1 r c lJ B l J S ~ " ' ; ; ; f l ? ; ; : i n Q ; : ; " ' ; ; ; " ' ; ; ] i ; : O= = = = = = = = = = = I ~ ~t -' .a ){ ir er ao or lG : 0 Lr se dE fa Li t: ln F

05",,,,,, r-------,Ob jec t ive func tion: : ; , 1 @ d = = ' ~ J O O = ' : : :5 ~ = a > , = = = = = = = = , : V IStiilrl~int [ O . O J

OS~d~: L_ ~o Use-()efaLlt:InFOS~d~: r-----------,0Usec): : faLlt: le-605~d~; ,-------------,

011se(iefa~t:-Inf05",,~, ,-------------,

S i a l neretcre: 0L I s e - defa~t: 5 0 0 " n u r r i J e r O I V i i lr i a b l ' e s

Con~~..n:b;:60 ' .mds :

R u n s o lv e r a n d v ie w r es u lt s

~Currer i1iter a b o n = I D II C lee r Resu i t s IOp~n!'atl~i"IrlJring.~ b n i~ a l :i Q n I ; sl l in a t~ c ; I.Objective fun~~an ", .;lbJ~~Il.~OOO3B311165B34BOp~rr.illO!ltion'!Irrln!t~d; d la rl g. ! T I best func~o.nv,!'lue I!:S~mn(lptons,ToIfllrJ. EiAnnecl'n~l~rameters 1

AJlIre:~l ing fmchoru I F~s:tanne~ling " ,I

OSpecity:

ReM lnea l ing g,iel"!.'ai!: @lJsedefuult:l00o Spe d fy :

I n i ' I B l temperefurer @lJ:se-default:l00

I E": b"~M\o,"~lYIleriZAI:~ptc n[e p r ob a b lt y f u nc t iD n : I :!rnulatod ,annfflin~ o i' Ir re p1 :< lr w :e ' v I

100 100

10 0

Using simulated annealing to solve a challenging problem that contains flat regions between basins.

A C D e r J a r o , i ! r 9 ' J h 8 'p a co o f t 1n g in e tJ .r in g a n d s c r. i3 'n c e 1 1


12/12

Solving Optimi:z:ationProblems Using Parallel ComputingYou can use Global Optimization Toolbox in conjunction with Parallel Computing Toolbox to solve problemsthat benefit from parallel computation. By using built-in parallel computing capabilities or defining a customparallel computing implementation of your optimization problem, you decrease time to solution.Built-in support for parallel computing accelerates the objective and constraint function evaluation in geneticalgorithm, multi objective genetic algorithm, and pattern search solvers. You can accelerate the multi start solver bydistributing the multiple local solver calls across multiple MATLAB workers or by enabling the parallel gradientestimation in the local solvers.

Demonstration of Using the Genetic Algorithm with Parallel Computing ToolboxA custom parallel computing implementation involves explicitly defining the optimization problem to use parallelcomputing functionality. You can define either your objective function or constraint function to use parallelcomputing, letting you decrease the time required to evaluate the objective or constraint.

Nonlinear Regressionwth the Multistart Solver 4: 16The best-fit parameters for a semi-empirical model are found using parallelcomputing.

Resou rcesP ro du ct D etails, D em o s, an d S yste m R eq u ire m en tswww.mathworks.com/products/global-optimizationTr ia l S o f twar ewww.mathworks.com/trialrequestSoleswww.mathworks.com/contactsalesT ec hn ic a l S u p po rtwww.mathworks.com/support

O nlin e U se r C o m m un itywww.mathworks.com/matlabcentralT ra in i n g S e rv ic e swww.mathworks.com/trainingT hird -P arty P ro du cts a nd S erv ic eswww.mathworks.com/connectionsWo rld wid e C o nta ctswww.mathworks.com/contact

2011 The MathWorks, Inc . MATLABand Simulink a re reg iste red trademarks ofThe MathWorks, Inc . See www.mathworks.com/trademarks
http://www.mathworks.com/products/global-optimizationhttp://www.mathworks.com/trialrequesthttp://www.mathworks.com/contactsaleshttp://www.mathworks.com/supporthttp://www.mathworks.com/matlabcentralhttp://www.mathworks.com/traininghttp://www.mathworks.com/connectionshttp://www.mathworks.com/contacthttp://www.mathworks.com/trademarkshttp://www.mathworks.com/trademarkshttp://www.mathworks.com/contacthttp://www.mathworks.com/connectionshttp://www.mathworks.com/traininghttp://www.mathworks.com/matlabcentralhttp://www.mathworks.com/supporthttp://www.mathworks.com/contactsaleshttp://www.mathworks.com/trialrequesthttp://www.mathworks.com/products/global-optimization

Global Optimization Toolbox

Documents