A Hybrid Optimization Approach for Global Exploration

A Hybrid Optimization Approachfor Global Exploration2005 713Satoru HIWAIntelligent Systems Design Laboratory

Here comes your footer

OptimizationOptimization problem consists of:Objective function: we want to minimize or maximize.Design variables: affect the objective function value.Constraints: allow the design variables to take on certain values but exclude others.Mathematical discipline that concerns the finding of minima or maxima of functions, subject to constraintsReal-world applicationsOptimization techniques have been applied to various real-world problems.e.g.)Structural designElectric device design

Here comes your footer

Problem Solving by OptimizationThere are many good optimization algorithms.Each method has its own characteristics.It is difficult to choose the best method for the optimization problem.It is important to select and apply the appropriate algorithms according to the complexities of the problems.It is hard to solve the problem with only one algorithm when the problem is complicated.Hybrid optimization approach, which combines plural optimization algorithms, should be necessary.Purpose of the research:To develop an efficient hybrid optimization algorithm

Here comes your footer

Hybrid Optimization ApproachIt provides the high performance which cannot be accomplished with only one algorithm.Hybrid optimization algorithmWe have to determine what kinds of solutions are required.Desired solutions may vary depending on the user:One may require the better result within a reasonable time.The other may want not only the optimum, but also the information of the landscape.Optimization strategyFirst, how the optimization process is performed should be determined.To develop an efficient hybrid optimization algorithm

Here comes your footer

Optimization StrategyBy this, we can obtain not only the optimum point, but also the information of the landscape.Many optimization algorithms are designed only to derive an optimum.To explore the search space uniformly and equallyWhy is the strategy needed?

Here comes your footer

Why is the Strategy Needed?When we solve real-world optimization problems;Usually, the landscape and the optimum are unknown.In this case, the obtained results should be reliable.Genetic Algorithms (GAs) are powerful techniques to obtain the global optimum.Probabilistic algorithm inspired by evolutionary biologyExample of optimization by GAs:ProblemGAs

Here comes your footer

Why is the Strategy Needed?When we solve real-world optimization problems;Usually, the landscape and the optimum are unknown.In this case, the obtained results should be reliable.Genetic Algorithms (GAs) are powerful techniques to obtain the global optimum.Probabilistic algorithm inspired by evolutionary biologyExample of optimization by GAs:ProblemGAs

Here comes your footer

Why is the Strategy Needed?When we solve real-world optimization problems;Usually, the landscape and the optimum are unknown.In this case, the obtained results should be reliable.Genetic Algorithms (GAs) are powerful techniques to obtain the global optimum.Probabilistic algorithm inspired by evolutionary biologyExample of optimization by GAs:ProblemGAsUnknownThe result is not reliable.Unexplored area exists.Is real optimum in the area?

Here comes your footer

Why is the Strategy Needed?When we solve real-world optimization problems;Usually, the landscape and the optimum are unknown.In this case, the obtained results should be reliable.Genetic Algorithms (GAs) are powerful techniques to obtain the global optimum.Probabilistic algorithm inspired by evolutionary biologyExample of optimization by GAs:ProblemGAsUnknownThe result is not reliable.Unexplored area exists.Is real optimum in the area?The strategy is not achieved only by GAs.

Here comes your footer

Why is the Strategy Needed?When we solve real-world optimization problems;Usually, the landscape and the optimum are unknown.In this case, the obtained results should be reliable.Genetic Algorithms (GAs) are powerful techniques to obtain the global optimum.Probabilistic algorithm inspired by evolutionary biologyExample of optimization by GAs:ProblemGAsUnknownReliability can be evaluated.The strategy is achieved.The landscape is grasped.

Here comes your footer

Optimization AlgorithmsThe strategy is not achieved only by GAs.Other algorithm, which provides more global search, is needed.However, the globally-intensified search converges slowly compared to GAs or local search algorithms.the much time is consumed in exploring the entire search space.There are tradeoff between the search broadness and the convergence rate.It is necessary to balance the global and local search.GAsDIRECT: explores search space globally.SQP: is high-convergence local search method.Both global and local search algorithms are hybridized.

Here comes your footer

DIRECTDeterministic, global optimization algorithmIts name comes from DIviding RECTangles.Search space is considered to be a hyper-rectangle (box).Each box is trisected in each dimension.Center point of each box is sampled as solution.Boxes to be dividedare mathematically guaranteed to be promising.are called potentially optimal boxes.

Here comes your footer

Characteristics of the DIRECT searchPotentially optimal boxes potentially contain a better value than any other box.DIRECT divides the potentially optimal boxes at each iteration.

Here comes your footer

Characteristics of the DIRECT searchExample: 2-dimensional Schwefel FunctionSome Local optima exist far from the global optimum.DIRECT explores the search space uniformly and equally.DIRECT also detects the promising area.Global OptimumLocal Optima

Here comes your footer

Characteristics of the DIRECT searchExample: 2-dimensional Schwefel FunctionSome Local optima exist far from the global optimum.DIRECT explores the search space uniformly and equally.DIRECT also detects the promising area.Global OptimumLocal Optima

Here comes your footer

Characteristics of the DIRECT searchExample: 2-dimensional Schwefel FunctionSome Local optima exist far from the global optimum.DIRECT explores the search space uniformly and equally.DIRECT also detects the promising area.Global OptimumLocal Optima

Here comes your footer

Genetic Algorithms (GAs)Heuristic algorithms inspired by evolutionary biology.Solutions are called individuals, and genetic operators (Crossover, Selection, Mutation) are applied.Real-coded GAsIndividuals are represented by real number vector.Although GAs are global optimization algorithm, the search broadness is inferior to DIRECT.GAs are used as more locally-intensified search than DIRECT.ParentsChildrenIndividuals

Here comes your footer

Sequential Quadratic Programming (SQP)Gradient-based local search algorithmThe most efficient method in nonlinear programmingBy using gradient information, SQP rapidly converges to the optimum.AdvantageHigh convergenceDisadvantageSQP is often trapped to the local optima, for the problem which has many local optima.

Here comes your footer

Hybrid Optimization AlgorithmPerform the DIRECT search.Execute GAs.Improve the best solution obtained in GAs search by SQP.Idea of the proposed hybrid optimization approachGlobal explorationby DIRECTLocally-intensifiedsearch by GAsFine tuningby SQPProcedure of the proposed algorithm

Here comes your footer

Hybrid Optimization AlgorithmPerform the DIRECT search.Execute GAs.Improve the best solution obtained in GAs search by SQP.Idea of the proposed hybrid optimization approachGlobal explorationby DIRECTOptimumLocally-intensifiedsearch by GAsFine tuningby SQPProcedure of the proposed algorithm

Here comes your footer

How to Combine DIRECT and GAsGAs utilize the center points of the potentially optimal boxes in DIRECT as their individuals.Number of potentially optimal = number of individualsNumber of potentially optimal differs at each iteration.Number of individuals are determined according to the complexities of the problems.(e.g. In N-dim. space, N10 individuals are recommended.)

DIRECT stopped.GAs start.

Here comes your footer

How to Combine DIRECT and GAsGAs utilize the center points of the potentially optimal boxes in DIRECT as their individuals.Number of potentially optimal = number of individualsNumber of potentially optimal differs at each iteration.Number of individuals are determined according to the complexities of the problems.(e.g. In N-dim. Space, N10 individuals are recommended.)

DIRECT stopped.GAs start.Number of potentially optimal boxes should be adjusted according to the number of individuals.

Here comes your footer

How to Combine DIRECT and GAsIf the number of potentially optimal is smaller than Ni,randomly generated individuals are added.

If the number of potentially optimal is larger than Ni,a certain number of potentially optimal boxes are selected.Box selection rules are proposed and applied.Ni: Number of individuals in GAs

Here comes your footer

Box Selection Rules for DIRECTDIRECT sometimes performs an local improvement.In the hybrid optimization, it is not necessary for DIRECT to perform locally-intensified search.Proposed rules reduce the crowded boxes.Distance from the box with best function value is calculated.A certain number of boxes far from the best point are selected.The rules are applied at each iteration in DIRECT search.Idea of selecting the boxes to be divided

Here comes your footer

Box Selection Rules for DIRECTDIRECT sometimes performs an local improvement.In the hybrid optimization, it is not necessary for DIRECT to perform locally-intensified search.Proposed rules reduce the crowded boxes.Distance from the box with best function value is calculated.A certain number of boxes far from the best point are selected.The rules are applied at each iteration in DIRECT search.Idea of selecting the boxes to be dividedThe number of potentially optimal boxes is reducedwithout breaking the global search characteristics of DIRECT.Potentially optimal boxes near the best point are discarded, and locally-biased search is prevented.

Here comes your footer

Experiments10-dimensional Schwefel functionA lot of local optimum exist.The function value of the global optimum is zero.Target problemNumerical example is shownto verify whether the proposed method achieve the proposed strategy to explore the search space uniformly and equally.The proposed hybrid optimization algorithmis applied to the benchmark problem.is compared to the search only by GAs.

Verification of effectiveness of the hybrid approach

Here comes your footer

Results and DiscussionsSearching abilityAverage values of function value and the number of function evaluations are shown.

Proposed hybrid algorithm obtains better function value than that of GAs, with less function evaluations.

Average of 30 runsHybridGAsFunction value9.0710-85.58102Function evaluations129,373279,703

Here comes your footer

Results and DiscussionsTo see whether the proposed strategy is achievedSearch histories of DIRECT and GAs in the hybrid algorithm are checked.History in 10-dimensional space is projected into 2-dimensional plane.Although 45 plots exist, 4 typical examples are picked. (x1, x2, , x10) (x1, x2), (x1, x3),

Here comes your footer

Search History of DIRECT(x1, x2)(x3, x6)(x2, x5)(x7, x9)

Here comes your footer

Search History of DIRECT

Here comes your footer

Search Histories of DIRECT and GAs

Here comes your footer

Search Histories of DIRECT and GAsThe proposed strategy is achieved.

Here comes your footer

Conclusionsoptimization strategy is proposed:To explore the search space uniformly and equallyOptimization algorithms used for the strategy:DIRECT, GAs, and SQPHybrid optimization approach is proposed.Modification to DIRECTBox selection rules are proposed and applied.

Hybrid optimization algorithmIt achieved the proposed strategy. It provided the efficient performance than the search only by GAs.

Here comes your footer

Paper ListMitsunori Miki, Satoru Hiwa, Tomoyuki HiroyasuSimulated Annealing using an Adaptive Search VectorProceedings of IEEE International Conference on Cybernetics and Intelligent Systems 2006 (Bangkok, Thailand)Proceeding of International ConferenceThe Science and Engineering Review of Doshisha University LED Vol.46 No.3 pp 9-182005Oral Presentation (in Japan) 72006 62006 82005

Here comes your footer

Here comes your footer

Lipschitzian Optimization [Shubert 1972]It requires the user to specify the Lipschitz constant K . abx1Slope = KSlope = +Kabx1x2abx1x2x3K is used as a prediction of the maximum possible slope of the objective function over the global domain. K+K

Here comes your footer

DIRECT (one-dimensional)abBox 1Box 1Box 2Box 3Box 1Box 2Box 3Box 4Box 5

Here comes your footer

DIRECT (one-dimensional)abBox 1Box 1Box 2Box 3Box 1Box 2Box 3Box 4Box 5Box 4Box 1Box 5Box 2Slope = K1Slope = K2Slope = K

Here comes your footer

DIRECT Potentially Optimal BoxesDIRECT divides all potentially optimal boxes.Potentially optimal boxes are defined by:Identification of potentially optimal boxesA hyper box j is potentially optimal if there exists some such thatcj: center point of the box jdj: distance from the center point to vertices

Here comes your footer

DIRECT Potentially Optimal BoxesDIRECT divides all potentially optimal boxes.Identification of potentially optimal boxesSearch space

Here comes your footer

DIRECT Potentially Optimal BoxesDIRECT divides all potentially optimal boxes.Search spacecjBox jIdentification of potentially optimal boxes

Here comes your footer

DIRECT Potentially Optimal BoxesDIRECT divides all potentially optimal boxes.Search spaceCenter - vertex distance (dj)f (cj)cjBox jIdentification of potentially optimal boxes

Here comes your footer

DIRECT Potentially Optimal BoxesDIRECT divides all potentially optimal boxes.Center - vertex distance (dj)f (cj)cjBox j( 0, fmin -| fmin | )fminIdentification of potentially optimal boxes

Here comes your footer

DIRECT Potentially Optimal BoxesDIRECT divides all potentially optimal boxes.Center - vertex distance (dj)f (cj)cjBox j( 0, fmin -| fmin | )fminMake the convex hull which contains all points.Identification of potentially optimal boxes

Here comes your footer

DIRECT Potentially Optimal BoxesDIRECT divides all potentially optimal boxes.Center - vertex distance (dj)f (cj)cjBox j( 0, fmin -| fmin | )fminBoxes on the lower part of convex hull is selected as potentially optimal.Identification of potentially optimal boxes

Here comes your footer

DIRECT Potentially Optimal BoxesDIRECT divides all potentially optimal boxes.Center - vertex distance (dj)f (cj)cjBox jBoxes on the lower part of convex hull is selected as potentially optimal.Search spaceIdentification of potentially optimal boxes

Here comes your footer

Genetic Algorithms (GAs)Global search algorithm inspired by evolutionary biology.Solutions are called individuals, and genetic operators (Crossover, Selection, Mutation) are applied.Real-Coded GAs (RCGAs)Individuals are represented by real number vector.Crossover operator significantly affects the searching ability.Simplex Crossover (SPX)One of the efficient crossover operator for RCGAs.Generates offspring in a simplex, which is formed by n+1 individuals in n-dimensional spaceRCGAs using the SPX operatorhas both global and local searchcharacteristics.RCGAs using the SPX operator are used.

Here comes your footer

GAs and SQPGradient-based local search algorithmBy using gradient information, SQP rapidly converges to the optimum.GAs (Genetic Algorithms)SQP (Sequential Quadratic Programming)Heuristic algorithm inspired by evolutionary biology.Solutions are called individuals, and genetic operators (Crossover, Selection, Mutation) are applied.ParentsChildrenIndividuals

Here comes your footer

Stopping Criterionis terminated when the size of the best potentially optimal box is less than certain value prescribed.A certain depth of search space exploration is obtained.DIRECTare terminated when their individuals converged.Spread of the individuals in design variable space:xmax xmin < thresholdGAsSQPcontinues its search until the improvement of solution becomes a minute value.

Here comes your footer

Stopping Criterion (DIRECT)is terminated when the longest side length of the best potentially optimal box is less than 10-3.A certain depth of search space exploration is obtained.DIRECT

Here comes your footer

Stopping Criterion (GAs)GAsare terminated when their individuals converged.Spread of the individuals in design variable space:Spreadi = xmax xminxmax : the maximum value of i-th design variables in all individuals.xmin : the minimum value of i-th design variables in all individuals.If Spreadi is smaller than 10-3 feasible range for all dimensions, GAs are terminated.Spread1Spread2Population converged

Here comes your footer

Results (of each algorithm)

Function valueDIRECT(po 52)GAs(Ind 100)SQPHybridGAs only(Ind 100)Average3.52x10-21.23x10-49.07x10-89.07x10-85.58x102St. Dev.0.001.29x10-49.16x10-89.16x10-81.82x102

Num. of eval.DIRECTGAsSQPHybridGAs onlyAverage1352911579350129373279703St. Dev.0930018930722402

Here comes your footer

How to Combine DIRECT and GAsGAs utilize the center points of the potentially optimal boxes in DIRECT as their individuals.If Npo > NindBox selection rules are applied.If Npo < Nind Randomly generated individuals are added to GAs.

DIRECT stopped.GAs start.

Here comes your footer

Modification to DIRECTSelect two boxes, with the smallest size and with the largest from the set of potentially optimal boxes.For each boxes, calculate the distance from two box.Sort the boxes by the distance in descending order, and select N boxes from them.Box selection rulesThe number of potentially optimal boxes is reducedwithout breaking the global search characteristics of DIRECT.Potentially optimal boxes near two boxes are discarded, and locally-biased search is prevented.

Here comes your footer

Modification to DIRECTSelect two boxes, with the smallest size and with the largest from the set of potentially optimal boxes.For each boxes, calculate the distance from two box.Sort the boxes by the distance in descending order, and select N boxes from them.Box selection rulesThe number of potentially optimal boxes is reducedwithout breaking the global search characteristics of DIRECT.Potentially optimal boxes near two boxes are discarded, and locally-biased search is prevented.

Here comes your footer

Potentially optimal boxes(when DIRECT was terminated)(x1, x2)(x3, x6)(x2, x5)(x7, x9)

Here comes your footer

History of the search only by GAs(x1, x2)(x3, x6)(x2, x5)(x7, x9)

Here comes your footer

History of the search only by GAs(x1, x2)(x3, x6)(x2, x5)(x7, x9)GAs were trapped to the local optima.

Here comes your footer

My name is Satoru Hiwa.Now I will talk about my study, A Hybrid Optimization Approach for Global Exploration.Optimization is a mathematical discipline that concerns the finding of minima or maxima of functions, subject to constraints.

Optimization problem consists of objective function, design variables, and constraints. Objective function is the function we want to minimize or maximize. Design variables is the variable affect the objective function value. Constraints is the condition allow the design variables to take on certain values but exclude others.

Optimization techniques have been applied to various real-world problems, such as Structural design, and Electric device design.When we solve the problems, there are many good optimization algorithms, and each method has its own characteristics. So it is difficult to choose the best method for the optimization.

Therefore, it is important to select and apply the appropriate algorithms according to the complexities of the problems.

However, it is hard to solve the problem with only one algorithm when the problem is complicated. In this case, hybrid optimization approach, which combines plural optimization algorithm, should be necessary.

So the purpose of my research is to develop an efficient hybrid optimization algorithm.

The hybrid optimization algorithm integrates the plural optimization algorithms, and provides the high performance which cannot be accomplished with only one algorithm.

To develop an efficient hybrid optimization algorithm, first, we have to determine what kinds of solutions are required, because the desired solutions may vary depending on the user.

For example, one may require the better result within a reasonable time, and the other may want not only the optimum, but also the information of the landscape.

Therefore, the optimization strategy, how the optimization process is performed should be determined.In this research, I propose new optimization strategy, to explore the search space uniformly and equally.

By this strategy, we can obtain not only the optimum point, but also the information of the landscape.

However, many optimization algorithms are designed only to derive an optimum.

So I explain why the strategy is needed in the next slide.When we solve real-world optimization problems, usually, the landscape and the optimum are unknown.

In this case, the obtained results should be reliable.

In recent years, it is said that Genetic Algorithms (GAs) are powerful techniques to obtain the global optimum.

The figure shows the example of optimization by GAs. This is the landscape of the problem, and the area with better function values are represented by white color.In this problem, the global optimum is shown here, and some local optima exist like this.

In this case, GAs obtain the global optimum.However, you can see that unexplored area exists in the GAs search. In many real-world problems, the landscape is unknown.So, we want to know whether the real optimum is in the unexplored area. That is, the result is not reliable.Therefore, the strategy is not achieved only by GAs.On the other hand, if the search space is explored uniformly and equally, the landscape is roughly grasped, and we can evaluate the reliabilities of the obtained results.From the results, it is found that the strategy is not achieved only by GAs.Thus, other algorithm, which provides more global search, is needed.However, the globally-intensified search converges slowly compared to GAs or local search algorithms because the much time is consumed in exploring the entire search space.

That is, there are tradeoff between the search broadness and the convergence rate.I assume that, it is necessary to balance the global and local search.

Therefore, in this research, both global and local search algorithms are hybridized. GAs, DIRECT and SQP are used for the hybrid algorithm.DIRECT is a deterministic, global optimization algorithm, and its name comes from DIviding RECTangles".

In DIRECT, search space is considered to be a hyper-rectangle (box). Each box is trisected in each dimension, and center points of each box is sampled as solution. Boxes to be divided are mathematically guaranteed to be promising, and they are called potentially optimal boxes.

DIRECT divides the potentially optimal boxes at each iteration. By this, DIRECT explores globally, and can obtain the global optimum.For example, on 2-dimensional Schwefel Function, which has some local optima exist far from the global optimum.This is result of the DIRECT search. You can see that DIRECT explores the search space uniformly and equally.Moreover, DIRECT also detects not only the global optimum, but also local optima.

GAs are heuristic algorithm inspired by evolutionary biology. In GAs, solutions are called individuals, and genetic operators (Crossover, Selection and Mutation) are applied.

In this research, Real-coded GAs, whose individuals are represented by real number vector, are used.

Although GAs are global optimization algorithm, the search broadness is inferior to DIRECT.So in the proposed hybrid approach, GAs are used as more locally-intensified search than DIRECT.

SQP is a gradient-based local search algorithm, and is the most efficient method in nonlinear programming. By using gradient information, SQP rapidly converges to the local optimum.However, SQP is often trapped to the local optima, for the problem which has many local optima.These figures illustrates the idea of the proposed hybrid optimization approach.First, global exploration is performed by DIRECT, and then GAs perform the locally-intensified search. Finally, fine tuning of the solution is made by SQP.

However, there are some difficulties in combining DIRECT and GAs.In the proposed approach, GAs utilize the center points of the potentially optimal boxes in DIRECT as their individuals. When DIRECT stopped, the potentially optimal boxes are identified, and the center points of them are used as individuals of GAs.

In this case, the number of potentially optimal boxes equals the number of individuals.However, the number of potentially optimal boxes differs at each iteration.On the other hand, in GAs, the number of individuals are determined according to the complexities of the problems.So, in this research, the number of potentially optimal boxes should be adjusted according to the number of individuals.

Here, Ni represents the number of individuals.

In the proposed algorithm, if the the number of potentially optimal boxes is smaller than Ni, randomly generated individuals are added to GAs. On the other hand, if the number of potentially optimal boxes is larger than Ni, a certain number of potentially optimal boxes are selected as individuals.

For this, box selection rules are proposed and applied.

DIRECT sometimes performs an local improvement, but in the hybrid optimization, it is not necessary for DIRECT to perform locally-intensified search.

So the proposed rules reduce the crowded boxes, like this.In the proposed rules, for each box, distance from the box with best function value is calculated, and then a certain number of boxes far from the best point are selected.Here, the proposed rules are applied at each iteration in DIRECT search.By this, potentially optimal boxes near two boxes are discarded, and locally-biased search is prevented. Therefore, the number of potentially optimal is reduced without breaking the global search characteristics of DIRECT.

Verification of the effectiveness of the proposed hybrid optimization approach is made through the numerical experiments.

Numerical example is shown to verify whether the proposed method realize the proposed strategy - to explore the search space uniformly and equally.

The proposed hybrid optimization algorithm is applied to the benchmark problem, and it is compared to the search only by GAs.

The target problem is 10-dimensional Schwefel function.

This table shows the result of the experiment.In this table, the average of the function value and the number of function evaluations in 30 runs are shown.The function value becomes better when it is close to zero.

From this result, the hybrid algorithm obtains better function value than that of GAs, with less function evaluations.Next, to see whether the proposed strategy is achieved by the hybrid algorithm, the search history of DIRECT and GAs in the hybrid algorithm are checked.

However, the problem is 10-dimensional, so history in 10-dimensional space is projected into 2-dimensional plane. Although 45 plots exist, 4 typical examples are picked.These figures represent the history of the DIRECT search. This figure represents the landscape of Schwefel function. From the figures, it is found that DIRECT detects not only the global optimum, but also the local optima.Next, the search history of GAs are shown.Red points represent the search history of DIRECT.You can see that GAs obtain the global optimum by starting the search from the promising area detected by DIRECT.Moreover, it can be seen that DIRECT covers the unexplored area in GAs search.From the results, it is found that the proposed strategy is achieved.I would like to conclude the presentation.

In this study, the Hybrid optimization approach is proposed.To design the hybrid algorithm, it is important to define the optimization strategy.So I proposed the new optimization strategy, to explore the search space uniformly and equally.Moreover, DIRECT, GAs and SQP are introduced for the strategy.

Additionally, to integrate the DIRECT and GAs, box selection rules are proposed for DIRECT.

From the experimental results, the proposed hybrid algorithm realized the the proposed strategy, and provided the efficient performance than the search only by GAs.Genetic Algorithms (GAs) are global search algorithm inspired by evolutionary biology. In GAs, solutions are called individuals, and genetic operators (Crossover, Selection, Mutation) are applied.

In particular, GAs, whose individuals are represented by real number vector, are called Real-Coded GAs (RCGAs). In RCGAs, crossover operator significantly affects the searching ability.

Simplex Crossover (SPX) is one of the efficient crossover operator for RCGAs. SPX generates offspring in a simplex, which is formed by n+1 individuals in n-dimensional space.

Since RCGAs using the SPX operator has both global and local search characteristics, in this research, RCGAs using the SPX operator are used.

GAs are heuristic algorithm inspired by evolutionary biology. In GAs, solutions are called individuals, and genetic operators (Crossover, Selection and Mutation) are applied.

SQP is a gradient-based local search algorithm. By using gradient information, SQP rapidly converge to the local optimum.Stopping criterion of each algorithm is shown here.DIRECT is terminated when the size of the best potentially optimal box is less than certain value prescribed. That is, a certain depth of search space exploration is obtained.

On the other hand, GAs are terminated when their individuals converged.SQP continues its search until the improvement of solution becomes a minute value.Stopping criterion of each algorithm is shown here.DIRECT is terminated when the size of the best potentially optimal box is less than certain value prescribed. That is, a certain depth of search space exploration is obtained.

On the other hand, GAs are terminated when their individuals converged.SQP continues its search until the improvement of solution becomes a minute value.Stopping criterion of each algorithm is shown here.DIRECT is terminated when the size of the best potentially optimal box is less than certain value prescribed. That is, a certain depth of search space exploration is obtained.

On the other hand, GAs are terminated when their individuals converged.SQP continues its search until the improvement of solution becomes a minute value.However, there are some difficulties in combining DIRECT and GAs.In the proposed approach, GAs utilize the center points of the potentially optimal boxes in DIRECT as their individuals. When DIRECT stopped, the potentially optimal boxes are identified, and the center points of them are used as individuals of GAs.

In this case, the number of potentially optimal boxes equals the number of individuals. However the number of potentially optimal boxes differs at each iteration. Thus, If the number of potentially optimal is large, we have to select the certain number of boxes from them.

For this, the box selection rules are proposed and applied.

The box selection rules are described here.First, select two boxes, with the smallest size and with the largest from the set of potentially optimal.Next, for each potentially optimal boxes, calculate the distance from two box.Finally, sort the boxes by the distance in descending order, and select N boxes from them.

By this, potentially optimal boxes near two boxes are discarded, and locally-biased search is prevented. Therefore, the number of potentially optimal is reduced without breaking the global search characteristics of DIRECT.

The box selection rules are described here.First, select two boxes, with the smallest size and with the largest from the set of potentially optimal.Next, for each potentially optimal boxes, calculate the distance from two box.Finally, sort the boxes by the distance in descending order, and select N boxes from them.

By this, potentially optimal boxes near two boxes are discarded, and locally-biased search is prevented. Therefore, the number of potentially optimal is reduced without breaking the global search characteristics of DIRECT.

Next, for the comparison, the history of the search only by GAs are shown.From the figure, you can see that GAs were trapped to the local optima in some dimensions.