Recent Advances in Data-driven Evolutionary Computationstaff.ustc.edu.cn/~ketang/PPT/DDEC201406.pdf · Recent Advances in Data-driven Evolutionary Computation Ke Tang USTC-Birmingham
Post on 26-Mar-2018
227 Views
Preview:
Transcript
Recent Advances in Data-driven Evolutionary Computation
Ke Tang
USTC-Birmingham Joint Research Institute in Intelligent Computation and Its Applications (UBRI) School of Computer Science and Technology
University of Science and Technology of China
June 2014 @ IWSIS&IWMWC
1
Outline
• A data-driven perspective on Evolutionary Algorithms • History-Based Topological Speciation (HTS)
• Population-based Algorithm Portfolios based on Estimated Performance Matrix (EPM-PAP)
• Summary
1 1
Outline
• A data-driven perspective on Evolutionary Algorithms • History-Based Topological Speciation (HTS)
• Population-based Algorithm Portfolios based on Estimated Performance Matrix (EPM-PAP)
• Summary
2 2
A data-driven perspective on EAs
• Evolutionary Algorithms (EAs): Algorithms that mimic natural evolution.
• There are quite a few famous branches of EAs – Genetic Algorithms – Genetic Programming – Evolution Strategies – Evolutionary Programming – Particle Swarm Optimizer – Ant Colony Optimization – Differential Evolution – etc.
3 3
A data-driven perspective on EAs
• Despite of the different historical background, most EAs share a similar framework.
• The Framework of EAs
4 4
c�Xin Yao 3'
&
$
%
What Is an Evolutionary Algorithm?
(OK, you can open your eyes and wake up now.)
1. Generate the initial population P (0) at random, and seti← 0;
2. REPEAT
(a) Evaluate the fitness of each individual in P (i);
(b) Select parents from P (i) based on their fitness in P (i);
(c) Generate offspring from the parents using crossover andmutation to form P (i + 1);
(d) i← i + 1;
3. UNTIL halting criteria are satisfied
A data-driven perspective on EAs
• Most EAs are Stochastic Population-Based Generate-and-Test algorithms.
• An EA iteratively sample in a solution space, and thus can be viewed as a data-generating process.
5 5
Sampling
x1 … xD fitness Individual 1 …
Individual n
A data-driven perspective on EAs
• The data consist a lot of information: – The candidate solutions (individuals) – Their corresponding fitness – The “source” of an individual (e.g., an individual was
generated by applying which operator to which parents).
• Data-driven Evolutionary Computation: To exploit the data generated by an EA during its search process to enhance the EA’s performance.
6 6
A data-driven perspective on EAs
• Data-driven Evolutionary Computation is a methodology rather than a new idea, and has been adopted in many existing works, e.g., – Surrogate Assisted Evolutionary Algorithms – Various parameter adaptation/self-adaptation schemes
• The key issues of DDEC: – What information is to be exploited from the data, and how. – How will an EA benefits from the obtained information.
7 7
Outline
• A data-driven perspective on Evolutionary Algorithms • History-Based Topological Speciation (HTS)
• Population-based Algorithm Portfolios based on Estimated Performance Matrix
• Summary and Discussions
8 8
Outline
• A data-driven perspective on Evolutionary Algorithms • History-Based Topological Speciation (HTS)
• Population-based Algorithm Portfolios based on Estimated Performance Matrix (EPM-PAP)
• Summary
9 9
HTS - Background
10 10
• Traditional optimization – Finds a single global optimum of a problem
• Challenge brought by a multimodal problem – There might be more than one optima that are (roughly) equally good. – Finds multiple optima of a problem.
• Why?
– Provides the user with a range of choices (more informed decisions) – Reveals insights into the problem (inspire innovations)
HTS - Background
• When employing EAs to find multiple optima, a procedure called speciation is usually required.
• Speciation: partitioning a population into a few species. – Niche: an region of attraction on the fitness landscape – Species: a group of individuals occupying the same niche – Species seed: the best (fittest) individual of a species
11 11
HTS - Background
• A typical speciation procedure
12 12
HTS - Background
13 13
• Most speciation methods relies on a sub-algorithm to determine whether two individuals are of the same species.
• Speciation methods – Distance-based: Determines whether two individuals are of the same
species according to their distance – Topology-based: Determines whether two individuals are of the same
species according to the fitness landscape topography
Speciation
Distance-Based Topology-Based
Hill-Valley Recursive Middling
Distance-based Speciation
14 14
• Two individuals are assigned to the same species if their distance is smaller than a predefined threshold called niche radius.
• Introduce an additional parameter (i.e., niche radius), which is difficult to tune.
• Make strong assumptions, i.e., equally sized and spherically shaped niches.
Topology-based Speciation
15 15
– Hill-Valley (HV)
– Recursive Middling
– Make weaker assumptions – Sampling new points in order to capture the landscape topography, – When more FEs are spent on speciation, less are available for the
evolutionary algorithm to converge. – Not very attractive especially when fitness evaluation is costly.
History-based Topology Speciation
16 16
• Research question: Could topology-based speciation be FE-free, so that their benefits can be better appreciated?
• Approach: Capture landscape topography based exclusively on search history.
History-based Topology Speciation
17 17
• Topology-based speciation methods can be interpreted from the perspective of a sequence of points. – is an infinite sequence and cannot be tested directly. – RM method “approximates” by sampling a few points on ab.
• Basic idea: Approximate only using history data/points.
• What is a “good” approximation? bad bad good
XX
X
History-based Topology Speciation
18 18
• Conceptually, HTS follows a two-step procedure 1. Construct a finite discrete approximate sequence 2. Test the approximate sequence to reach a final decision (trivial)
• More formally, the problem of finding the best approximation
can be stated as:
• where
History-based Topology Speciation
19 19
History-based Topology Speciation
20 20
HTS: Experiments
21 21
• Compared methods – HTS – The traditional distance-based method
• DIS: niche radius = 𝜎 computed using Deb and Goldberg’s formula
• DIS-: niche radius = 1/5 𝜎
• DIS+: niche radius = 5 𝜎
– HV • HV1: sample size = 1 • HV3: sample size = 3 • HV5: sample size = 5
– RM – The fictional idealized method RM*
• Works the same as RM but with all FE cost being artificially neglected • Used to obtain some kind of performance upper-bound for topology-based
speciation
HTS: Experiments
22 22
• Different methods are integrated into the same evolutionary framework for comparison – Crowding Differential Evolution with Species Conservation
• Benchmark functions – F1—F6: 6 two-dimensional functions with various properties
Number of optima: 2—10 – F7—F10: MMP functions in 4, 8, 16, 32 dimensions, respectively
Number of optima: 48 – F11: An composition of 50 random 32-dimensional shifted rotated
ellipsoidal sub-functions coupled via the max operator Number of optima: 50
• The goal is to find all optima of the benchmark functions
HTS: Experiments
23 23
• Performance Measure: The distance error of the last generation is then used to measure the performance of the algorithm
• Win/Draw/Lose of HTS versus every other method
– Both t-test and Wilcoxon rank-sum tests are used – Consider a difference to be statistically significant if it is asserted so by both
tests at the 0.05 significance level – A draw is counted when no statistically significant difference is observed
DIS- DIS DIS+ HV1 HV3 HV5 RM RM* 9/0/2 2/5/4 4/5/2 9/0/2 9/0/2 9/0/2 9/0/2 1/3/7
Outline
• A data-driven perspective on Evolutionary Algorithms • History-Based Topological Speciation (HTS)
• Population-based Algorithm Portfolios based on Estimated Performance Matrix (EPM-PAP)
• Summary
24 24
PAP - Background � A scenario frequently encounter in the real world: Ø A number of optimization problems Ø A time budget T Ø A number of optimization algorithms (e.g., GA, ES, EP, EDA,
DE, PSO…) We want to obtain the best (or as good as possible) solutions for all the problems with T.
25
PAP - Background � Ø Intuitively, the total time budget T can used for two purposes
(1) to identify the best algorithm (2) to search for the best solution
Ø In general, the more time we spent on (2), the better solution it
will achieve.
Ø Different problem may favor different algorithm. Finding the best algorithm for a problem can be very time consuming.
26
PAP - Background � General thoughts:
Ø Arbitrarily pick an algorithm for every problem? – T will solely be used to search for solutions – too risky
Ø Carefully identify the best algorithm for each problem? – A lot of time will be used for algorithm selection – The time left for searching for good solutions might be insufficient.
Ø Try to find a single algorithm suitable for all problems? – Sounds like a good trade-off – Advantages of having a set of different algorithms are not fully utilized.
27
PAP - Background � Ø How about establishing a good “portfolio” of algorithms (e.g.,
a combination of multiple algorithms) for all problems?
Advantages:
• Making use of advantages of different algorithms, rather than putting all the eggs (time) into a single basket (algorithm).
• Hopefully not too time-consuming since only one portfolio is needed for all problems.
28
PAP - Background �
Ø Algorithm Portfolios “invests” limited time in multiple algorithms to fully utilize the advantages of these algorithms to maximize the expected utility of a problem solving episode.
Ø Analogy to Economics: One allocates his/her money to different financial assets (stocks, bonds, etc.) in order to maximize the expected returns while minimizing risks.
Ø Population-based Algorithm Portfolios (PAP) – Conceptually similar to Algorithm Portfolios – Aims to solve a set of problems rather a single one – Focuses on population-based algorithms (e.g., EAs)
29
PAP - Background �
Ø The General Framework of PAP
30
Select the cons:tuent algorithms from a pool of candidate algorithms
Construct a concrete PAP instan:a:on with the cons:tuent algorithms
Apply the PAP instan:a:on to each problem
Output the best solu:on obtained for each problem
PAP - Background �
Which candidate algorithms should serve as constituent algorithms depends on the way of building a PAP instantiation: Ø A PAP instantiation maintains multiple sub-populations.
Ø Each sub-population is evolved with a constituent algorithm.
Ø Information is shared among sub-populations by activating a migration scheme periodically.
31
PAP - Background �
Pseudo-code of a PAP instantiation:
32
PAP - Background �
Ø The above pseudo-code looks very simple.
Ø But…if combining different EAs in this way won’t lead to any advantage over using a single algorithm. Why bother seeking the so-called constituent algorithms?
33
PAP - Background �
Ø A preliminary experiment – 4 Candidate EAs: CMA-ES, G3PCX, SaNSDE, wPSO – 11 PAP instantiations (with 2, 3, and 4 constituent algorithms)
Ø For each PAP instantiation – compare it to its constituent algorithms alone to verify whether there would be any advantage of PAP over its constituent algorithms. – compare it to G-CMA-ES to verify whether portfolio of some “weak” algorithms could outperforms the state-of-the-art.
Ø Benchmark problems – 13 numerical problems from classical benchmark suite – 14 numerical problems from CEC2005 benchmark suite – Dimension: 30
34
PAP - Background �
Ø Total Fitness Evaluations (FEs) for each problem: 300,000
Ø 30 independent runs on each problem
Ø For the convenience of implementation, all constituent algorithms of a PAP instantiation evolve with the same number of generations.
Ø Parameters of constituent algorithms are not fine-tuned, but set according to rules or default values suggested in the original publications.
Ø migration_interval=MAX_GEN/20, migration_size=1
35
PAP - Background �
Ø Wilcoxon Test Results (Significance level 0.05): “w-d-l” stands for “win-draw-lose”
36
PAP - Background �
Ø Observations: Combining EAs in the simple way does lead to performance improvement.
Ø Is that due to the fine-tuning of the migration parameter?
Ø Consider four values for each of the two migration parameters and test 16 pairs of parameters. – migration_interval: MAX_GEN divided by 20, 30, 40, 50 – migration_size: 1, 2, 4, 8
Ø Sensitivity Analysis – for each pair, 30 runs were executed on all 27 benchmark functions. – for each instantiation, count the number of the benchmark functions on which all 16 pairs of parameters made no difference. – Kruskal-Wallis one-way analysis of variance by ranks with significance level 0.05 was employed.
37
PAP - Background �
Ø Results
Ø Performance of a PAP instantiation is not very sensitive to the migration parameters, but relies on the constituent algorithms.
38
EPM-PAP �
On Choosing Constituent Algorithms
Ø Let F = {fk | k = 1, 2, … , n} be a given problem set, A = {aj |j = 1, 2, ..., m} be a set of candidate EAs, choosing constituent algorithm for PAP is formulated as seeking the algorithm subset = {ai |i = 1, 2, ..., l} of A that leads to the best overall performance on F, as given in Eq. (1)
Ø A most straightforward approach: enumerate all possible subset and employ a procedure like statistical racing to find the best one.
Even more time consuming than selecting a single algorithm!
39
),,~(maxarg~~
TFAUAAA
opt⊆
=
EPM-PAP �
Ø Recall that we expect that a good PAP instantiation to under-perform a candidate EA (say, ai) with small probability.
Ø Assuming independence between constituent algorithms, the above statement can be written for an algorithm j on problem fk as:
Ø Averaging over all problems and all candidate EAs, we get (1)
40
Rjk = (1−Pi, jk )
i=1
l
∏
∑∑∏= = =
−=m
j
n
k
l
i
kjiPmn
R1 1 1
, )1(1
EPM-PAP �
What is Estimated Performance Matrix
• A matrix that records the performance of each candidate EA.
• For each aj, the corresponding EPM, denoted by EPMj, is an r-by-n matrix.
• This matrix can be obtained by running aj on each of the n problems for r times.
• Each element of EPMj is the objective value of the best solution that aj obtained on a problem in a single run.
• Since each element of EPMj is obtained with a small portion of T, it can be viewed as a conservative estimate of the solution quality achieved by running aj with T on the same problem.
41
EPM-PAP �
Ø With the help of some statistical tests, EPMs provide all information that is needed to calculate Eq. (1)
Good news
Ø No need to compare the performance of all possible subsets with a tedious procedure like statistical racing.
Ø Estimating the performance of a single candidate EA is sufficient for constituent algorithm subset selection.
42
EPM-PAP �
Detailed steps for Choosing Constituent Algorithms
1. Apply each candidate EA aj to each problem for r independent runs. The final population obtained in each run is stored.
2. Construct EPM for each aj based on the quality of the best solution it obtained in each run.
3. All possible subset of A is enumerated and the corresponding R is calculated using Eq. (1) and the EPMs.
4. The subset with the smallest R is selected as the constituent algorithms for PAP.
43
EPM-PAP: Experiments � Ø 4 Candidate EAs: CMA-ES, G3PCX, SaNSDE, wPSO
Ø Benchmark problems – 13 numerical problems from classical benchmark suite used in [6] – 14 numerical problems from CEC2005 benchmark suite [7] – Dimension: 30
44
EPM-PAP: Experiments � Ø Total Fitness Evaluations (FEs) for each problem: 400000,
800000, and 1200000, respectively.
Ø 25 independent runs on each problem
Ø For the convenience of implementation, all constituent algorithms of a PAP instantiation evolve with the same number of generations.
Ø Parameters of constituent algorithms are not fine-tuned.
Ø migration_interval=MAX_GEN/20, migration_size=1
Ø PAP with 2 and 3 constituent algorithms are considered.
45
EPM-PAP: Experiments � Ø Wilcoxon Test Results (Significance level 0.05): “w-d-l” stands
for “win-draw-lose”
46
Time Budget
SaNSDE
wPSO G3PCX CMA-ES
F-Race Intra-AOTA
EPM-PAP-2
T1 8-14-5 17-10-0 21-6-0 8-13-6 9-14-4 6-15-6
T2 7-14-6 16-10-1 20-7-0 9-14-4 7-15-5 5-18-4
T3 6-15-6 17-9-1 21-6-0 10-14-3 7-14-6 6-18-3
EPM-PAP-3
T1 9-11-7 19-7-1 21-5-1 10-10-4 10-13-4 5-17-5
T2 8-17-2 17-9-1 20-7-0 9-12-6 9-12-6 5-20-2
T3 9-16-2 17-10-0 21-6-0 9-14-4 9-14-4 6-20-1
EPM-PAP: Experiments �
Ø Performance ranking of all possible EPM-PAP-2 and EPM-PAP-3
47
PAP Rank Time Budget = T1 Time Budget = T2 Time Budget = T3 with 2 constituent algorithms
1 SaNSDE + CMA-ES SaNSDE + CMA-ES SaNSDE + CMA-ES 2 wPSO + CMA-ES wPSO + CMA-ES wPSO + CMA-ES 3 SaNSDE + wPSO SaNSDE + wPSO SaNSDE + wPSO 4 SaNSDE + G3PCX SaNSDE + G3PCX SaNSDE + G3PCX 5 G3PCX + CMA-ES G3PCX + CMA-ES G3PCX + CMA-ES 6 wPSO + G3PCX wPSO + G3PCX wPSO + G3PCX
with 3 constituent algorithms
1 SaNSDE+wPSO+CMA-ES SaNSDE+wPSO+CMA-ES SaNSDE+wPSO+CMA-ES 2 SaNSDE+G3PCX+CMA-ES SaNSDE+G3PCX+CMA-ES SaNSDE+G3PCX+CMA-ES 3 SaNSDE+wPSO+G3PCX SaNSDE+wPSO+G3PCX wPSO+ G3PCX+CMA-ES 4 wPSO+ G3PCX+CMA-ES wPSO+ G3PCX+CMA-ES SaNSDE+wPSO+G3PCX
EPM-PAP: Experiments �
Ø Success Rates of the EPM-based selection procedure: How likely did it select the best constituent algorithm subset?
48
Time Budget SR1 SR2
EPM-PAP-2 T1 40% 88%
T2 56% 100%
T3 72% 100%
EPM-PAP-3 T1 16% 84%
T2 36% 88%
T3 56% 100%
Outline
• A data-driven perspective on Evolutionary Algorithms • History-Based Topological Speciation (HTS)
• Population-based Algorithm Portfolios based on Estimated Performance Matrix (EPM-PAP)
• Summary
49 49
Summary
• Data-driven EAs make use of data analytics approach to gain useful information from the data generated during search.
• Two recent examples of DDEC have been introduced.
• In HTS, history data are utilized to determine whether two individuals belong to the same species.
• In EPM-PAP, history data are utilized to select constituent algorithms for PAP.
• Different context in EC may induce significantly different data analytics problem, where a lot of work could be done.
50 50
Collaborators
• Collaborators – Mr. Lingxi Li (HTS)
– Dr. Fei Peng (EPM-PAP)
– Prof. Xin Yao (EPM-PAP)
– Prof. Guoliang Chen (EPM-PAP)
51 51
References
HTS • L. Li and K. Tang, “History-Based Topological Speciation for Multimodal
Optimization,” IEEE Transactions on Evolutionary Computation, in press (DOI: 10.1109/TEVC.2014.2306677).
EPM-PAP
• F. Peng, K. Tang, G. Chen and X. Yao, “Population-based Algorithm Portfolios for Numerical Optimization,” IEEE Transactions on Evolutionary Computation, 14(5): 782-800, October 2010.
• K. Tang, F. Peng, G. Chen and X. Yao, “Population-based Algorithm Portfolios with automated constituent algorithms selection,” Information Sciences, in press (DOI: 10.1016/j.ins.2014.03.105).
52 52
Thanks for your time! Q&A?
53 53
top related