-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 649 #1
36Parallel HybridMultiobjective
Metaheuristics onP2P Systems
N. MelabEl-Ghazali TalbiM. MezmazB. Wei
36.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .36-64936.2 Parallel
Hybrid MOMs and P2P Computing . . . . . . . . . .36-650
Multiobjective Optimization Parallelism andHybridization P2P
Computing for Parallel MOOptimization
36.3 A Model for P2P Coordination . . . . . . . . . . . . . . .
. . . . . . . . . .36-652Model Description Implementation on Top of
XtremWeb
36.4 Application to BPFSP and Experimentation . . . . . . . . .
. .36-655Problem Formulation A GeneticMimetic Algorithm forSolving
BPFSP Parallel Hybrid AGMA Deployment andExperimentation Deployment
and Fault Tolerance Experimental Results
36.5 Conclusion and Future Work . . . . . . . . . . . . . . . .
. . . . . . . . . . .36-661References . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . .36-663
36.1 Introduction
Metaheuristics allow to provide near-optimal solutions of
NP-hard complex problems in a reasonable time.They fall into two
complementary categories: evolutionary algorithms (EAs) that have a
good explorationpower, and local searches (LSs) characterized by
better intensification capabilities. The hybridization ofthe two
categories permits to improve the effectiveness (quality of
provided solutions) and the robustnessof the metaheuristics [11].
Nevertheless, as it is CPU time consuming it is not often fully
exploited inpractice. Indeed, experiments with hybrid
metaheuristics are often stopped before the convergence isreached.
Nowadays, Peer-to-Peer (P2P) computing [8] and grid computing [5]
are two powerful waysto achieve high performance on long-running
scientific applications. Parallel hybrid metaheuristics usedfor
solving real-world multiobjective problems (MOPs) are good
challenges for P2P and grid computing.However, to the best of our
knowledge no research work has been published on that topic.
36-649
2006 by Taylor & Francis Group, LLC
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 650 #2
36-650 Handbook of Bioinspired Algorithms and Applications
In this chapter, we contribute with the first results on
parallel hybrid multiobjective metaheuristicson P2P systems. The
design and deployment of these optimization methods require a
middleware thatallows cooperation between parallel tasks. In
addition, the traditional parallel models and
hybridizationmechanisms have to be re-thinked and adapted to be
scaled up. Moreover, these require to be fault-tolerantto allow
long-running problem resolutions. We particularly focus here on the
island model and themultistart model.
Recently, few middlewares [1,4,13] allowing to exploit P2P
systems have emerged. These middlewaresare well suited for
embarrassingly parallel applications such as multi parameter
simulations. However,they are limited regarding the parallelism as
they do not allow direct cross-peer (or cross-task) commu-nication.
Our contribution is to propose a Linda-like [7] coordination model
and its implementationon top of XtremWeb [4]. This is a
Dispatcher/Worker oriented middleware, in which the
Dispatcherdistributes application tasks submitted by clients to
volunteer worker peers at their request. In addition,the considered
middleware provides fault-tolerance mechanisms that are costly in a
highly volatile P2Penvironment. Indeed, a work unit is re-started
from scratch each time it fails. Another contribution of
thischapter is to deal with the fault-tolerance issue at
application level. We propose a check-pointing approachfor the two
parallel models quoted above.
To be validated the proposed approaches have been experimented
on the Bi-criterion PermutationFlow-Shop Problem (BPFSP) [12]. The
problem consists roughly to find a schedule of a set of jobson a
set of machines that minimizes the makespan and the total
tardiness. Jobs must be scheduled inthe same order on all machines,
and each machine cannot be simultaneously assigned to two jobs.In
Reference 2, a hybrid MultiObjective Metaheuristic (MOM) has been
proposed to solve this problem.In this chapter, we extend this work
with two P2P-based fault-tolerant parallel models: the island
andmultistart models. Our extended version allows to fully exploit
the hybridization and provides clearlybetter results. This
constitutes another contribution of this chapter.
This chapter is organized as follows: Section 36.2 presents
briefly parallel hybrid multiobjective optimiz-ation (MOO). Section
36.3 highlights the requirements of MOO and describes the proposed
coordinationmodel and its implementation on top of XtremWeb.
Section 36.4 presents the experimentation of themodel and its
implementation through a parallel hybrid metaheuristic applied to
the BPFSP, and analyzesthe preliminary experimental results.
Finally, Section 36.5 concludes the chapter.
36.2 Parallel Hybrid MOMs and P2P Computing
36.2.1 Multiobjective Optimization
An MOP consists generally in optimizing a vector of nbobj
objective functions F(x) = ( f1(x), . . . , fnbobj(x)),where x is
an d-dimensional decision vector x = (x1, . . . , xd) from some
universe called decision space.The space the objective vector
belongs to is called the objective space. F can be defined as a
cost functionfrom the decision space to the objective space that
evaluates the quality of each solution (x1, . . . , xd) byassigning
it an objective vector (y1, . . . , ynbobj), called the
fitness.
Unlike single-objective optimization problems, an MOP may have a
set of solutions known as the Paretooptimal set rather than a
unique optimal solution. The image of this set in the objective
space is denoted asPareto front. Graphically, a solution x is
Pareto optimal if there is no other solution x such that the
pointF(x ) is in the dominance cone of F(x). This dominance cone is
the box defined by F(x), its projections
36.2.2 Parallelism and Hybridization
In Reference 3, different parallel models have been
distinguished: those associated with LSs and thosededicated to EAs.
Three major parallel models for EAs are presented: the island
(a)synchronous cooperativemodel, the parallel evaluation of the
population, and the distributed evaluation of a single solution.The
parallel models for LSs are mainly: the parallel exploration of
neighboring candidate solutions and
2006 by Taylor & Francis Group, LLC
on the axes and the origin (Figure 36.1).Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 651 #3
Parallel Hybrid Multiobjective Metaheuristics on P2P Systems
36-651
Pareto solutionDominated solution
f 2
f1
FIGURE 36.1 Example of nondominated solutions.
the multistart model. In this chapter, we focus only on the
coarse-grained models: the island model and themultistart model.
Due to the communication delays, fine-grained models are often
inefficient when theyare deployed in a large-scale network.
In the island (a)synchronous cooperative model, different EAs
are simultaneously deployed and cooper-ate for computing better and
robust solutions. They exchange, in an asynchronous way, the
genetic stuffto diversify the search. The objective is to allow to
delay the global convergence, especially when the EAsare
heterogeneous regarding the variation operators. The migration of
individuals follows a policy definedby few parameters: the
migration decision criterion, the exchange topology, the number of
emigrants, theemigrants selection policy, and the
replacement/integration policy.
The multistart model consists in simultaneously launching
several local searches. They may be hetero-geneous, but no
information is exchanged between them. The results would be
identical as if the algorithmswere sequentially run. Very often
deterministic algorithms differ by the supplied initial solution
and/orsome other parameters. This trivial model is convenient for
low-speed networks of workstations.
Combinations of different metaheuristics often provide very
powerful search methods. In Reference 11,two levels and two modes
of hybridization are distinguished: Low and High levels, and Relay
and Cooper-ative modes. The low-level hybridization consists in
replacing an internal function (e.g., an operator) ofa given
metaheuristic by another metaheuristic. In high-level hybrid
algorithms, the different metaheur-istics are self-containing,
meaning no direct relationship to their internal working is
considered. Relayhybridization means a set of metaheuristics is
applied in a pipeline way. The output of a metaheuristic(except the
last) is the input of the following one (except the first).
Conversely, teamwork hybridization is acooperative optimization
model. Each metaheuristic performs a search in a solution space,
and exchangessolutions with others. In this chapter, we address the
high-level hybridization mechanism in the relay andcooperative
modes.
36.2.3 P2P Computing for Parallel MO Optimization
In this chapter, we focus on Dispatcher/Worker-oriented P2P
middlewares such as XtremWeb [4] andSETI@Home [1]. In such systems,
clients can submit their jobs to the Dispatcher. A set of volatile
workers(peers) request the jobs from the Dispatcher according to
the cycle stealing model. Then, they execute thejobs and return the
results to the Dispatcher to be collected by the clients. In these
middlewares, even acentral server (the Dispatcher) is required for
controlling the peers (workers) they are considered as P2Psoftware
environments. Indeed, an important part of these systems is
executed on these peers with a highautonomy.
One of the major limitations of P2P computing environments is
that they are well suited for embarrass-ingly parallel (e.g.,
multiparameter) applications with independent tasks. In this case,
no communicationis required between the tasks, and thus peers. The
deployment of parallel hybrid metaheuristics thatneed
cross-peer/task communication is not straightforward. The
programmer has the burden to manage
2006 by Taylor & Francis Group, LLC
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 652 #4
36-652 Handbook of Bioinspired Algorithms and Applications
and control the complex coordination between the workers. To
deal with such problem existing middle-wares must be extended with
a software layer that implements a coordination model. Several
interestingcoordination models have been proposed in the literature
[6,9]. In this chapter, we focus only on one ofthe most popular of
them, that is, Linda [7], as our proposed model is an extension of
this model.
In the Linda model, the coordination is performed through
generative communications. Processesshare a virtual memory space
called a tuple-space (set of tuples). The fundamental data unit, a
tuple,is an ordered vector of typed values. Processes communicate
by reading, writing, and consuming thesetuples. The eval operation
is particularly useful in a P2P environment as it allows to spawn
tasks to beexecuted on volunteer peers. A small set of four simple
operations allows highly complex communicationand synchronization
schemes:
Out(tuple): puts tuple into tuple-space. In(pattern): removes a
(often the first) tuple matching pattern from tuple-space.
rd(pattern): is the same as in(pattern), but does not remove the
tuple from tuple-space. Eval(expression): puts expression in
tuple-space for evaluation. The evaluation result is a tuple
left
in tuple-space.
Nevertheless, Linda has several limitations regarding the design
and deployment of parallel hybridmetaheuristics for P2P systems.
First, it does not allow rewriting operations on the tuple space.
Due tothe high communication delays in a P2P system, tuple
rewriting is very important as it allows to reducethe number of
communications and the synchronization cost. Indeed, in Linda a
rewriting operationis performed as an in or rd operation followed
by a local modification and an out operation.The operations in/rd
and out involve two communications and a heavy synchronization.
Therefore,the model needs to be extended with a rewriting
operation. Furthermore, the model does not supportgroup operations
that are useful for efficiently writing/reading Pareto sets in/from
the tuple-space. Finally,nonblocking operations that are very
important in a P2P context are not supported in Linda. In the
nextsection, we propose an extension of the Linda model that allows
to meet these requirements.
36.3 A Model for P2P Coordination
36.3.1 Model Description
Designing a coordination model for parallel MOO requires the
specification of the content of the tuple-space, a set of
coordination operations and a pattern matching mechanism. The
tuple-space may becomposed of a set of Pareto optimal solutions and
their corresponding solutions in the objective space.For the
parallel island model of the multiobjective metaheuristics, the
tuple-space contains a collection of(parts of) Pareto optimal sets
deposited by the islands for migration. The mathematical
formulation ofthe tuple-space (Pareto Space or PS) is the
following:
PS =
PO, with PO = {(x , F(x)), x is Pareto optimal}.
In addition to the operations provided in Linda, parallel P2P
multiobjective optimization needs otheroperations. These operations
fall into two categories: group operations and nonblocking
operations. Groupoperations are useful to manage multiple Pareto
optimal solutions. Nonblocking operations are necessaryto take into
account the volatile nature of P2P systems. In our model, the
coordination primitives aredefined as follows:
in, rd, out and eval : These operations are the same as those of
Linda defined in Section 36.2.3. ing(pattern): Withdraws from PS
all the solutions matching the specified pattern. rdg(pattern):
Reads from PS a copy of all the solutions matching the specified
pattern. outg(setOfSolutions): Inserts multiple solutions in
PS.
2006 by Taylor & Francis Group, LLC
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 653 #5
Parallel Hybrid Multiobjective Metaheuristics on P2P Systems
36-653
update(pattern, expression): Updates all the solutions matching
the specified pattern by the solutionsresulting from the evaluation
of expression.
inIfExist, rdIfExist, ingIfExist, and rdgIfExist : These
operations have the same syntax thanrespectively in, rd, ing, and
rdg but they are non blocking probe operations.
The update operation allows to locally update the PS, and so to
reduce the communication and syn-chronization cost. The pattern
matching mechanism depends strongly on how the model is
implemented,and in particular on how the tuple-space is stored and
accessed. For instance, if the tuple-space is storedin a database
the mechanism can be the request mechanism used by the database
management system.More details on the pattern matching mechanism of
our model are given in the next section.
36.3.2 Implementation on Top of XtremWeb
XtremWeb [4] is a Java P2P project developed at Paris-Sud
University. It is intended to distribute applica-tions over a set
of peers, and is dedicated to multiparameter applications that have
to be computed severaltimes with different inputs. XtremWeb manages
tasks following the Dispatcher/Worker paradigm (seeFigure 36.2).
Tasks are scheduled by the Dispatcher to workers only on their
specific demand since theymay adaptively appear (connect to the
Dispatcher) and disappear (disconnect from the Dispatcher).
Thetasks are submitted by either a client or a worker, and in the
latter case, the tasks are dynamically generatedfor parallel
execution. The final or intermediate results returned by the
workers are stored in a MySQLdatabase. These results can be
requested later by either the clients or the workers. The database
stores alsodifferent information related to the workers and the
deployed application tasks.
XtremWeb is well suited for embarrassingly parallel applications
where no cross-peer communicationoccurs between workers, and these
can only communicate with the Dispatcher. Yet, many parallel
dis-tributed applications particularly parallel MOMs need
cooperation between workers. In order to freethe user from the
burden of managing himself/herself such cooperation we propose an
extension of themiddleware with a software layer.
a coordination API and its implementation at the worker level
and a coordination request broker (CRB).The PS is a part of the
MySQL database associated with the Dispatcher. Each tuple or
solution of the PSis stored as a record in the database.
...
...
XtremWeb workers
XtremWeb dispatcher
XtremWeb clients
Internet
Get results
Submit work
Get a work unitSend results
FIGURE 36.2 Global architecture of XtremWeb.
2006 by Taylor & Francis Group, LLC
The software layer is an implementation of the proposed model
composed of two parts (see Figure. 36.3):
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 654 #6
36-654 Handbook of Bioinspired Algorithms and Applications
XtremWeb worker
XtremWeb dispatcher
XtremWeb databaseinformation
Paretospace (PS)
execlp(..., CRB_Stub,ING, ARGS_FILE, ...);
Work unit
CRB_Stub
CRB_Skeleton
ParetoSpace
Manager
switch(){}
RMI callof ing
switch(OP) {...
case ING:RMI call of ing(pattern);...
};
...
...
s=ing(pattern);
switch(OP) { ...
case ING: local call of ing(pattern); ...
};
switch(){}...
...
ing(pattern){}...
...
SELECT * FROM PSWHERE pattern is
matched
FIGURE 36.3 Implementation of the coordination model on top of
XtremWeb.
From the worker side the coordination API is implemented in Java
and in C/C++. The C/C++version allows the deployment and execution
of C/C++ applications with XtremWeb (written in Java).The
coordination library must be included in these programmer
applications. From the Dispatcher side,the coordination API is
implemented in Java as a PS manager. The CRB is a software broker
allowing theworkers to transport their coordination operations
calls to the Dispatcher, and has two components: onefor the worker
(CRB stub) and another for the Dispatcher (CRB skeleton). The role
of the CRB stub is totransform the local calls to the coordination
operations performed by the tasks executed by the worker intoRMI
calls. The role of the CRB skeleton is to transform these RMI calls
into local calls to the coordinationoperations performed by the PS
Manager. These local calls are translated into MySQL requests
addressedto the PS.
To illustrate the implementation of the coordination layer on
top of XtremWeb, let us consider the scen-ario presented in Figure
36.3. The work unit performed by an XtremWeb worker calls the ing
(template)coordination operation. In the C++ version of the
coordination API, the implementation of each coordin-ation
operation makes the system call execlp() with appropriate
parameters to plug in the CRB_Stub Javaobject. In our scenario, the
major parameters are the number ING designating the operation and
the fileARGS_FILE containing the arguments specified in the
template parameter. CRB_Stub translates the inglocal call into an
RMI call to the CRB_Skeleton Java object. This latter translates
the RMI call into a localcall to the ing operation implemented in
the PS Manager class. The implementation of the
coordinationoperation consists in a MySQL select request addressed
to the PS part of the XtremWeb informationdatabase.
Note that the method declarations for the coordination
operations in the PS Manager class contain theJava synchronized
keyword. Hence, the system associates a unique lock with the
instance of the PS Managerclass. Whenever control enters a
synchronized coordination operation, other calls to a
synchronizedcooperation method are blocked until the PS Manager
object is unlocked. In the next section, the proposedcoordination
model is applied to parallel hybrid MOMs.
2006 by Taylor & Francis Group, LLC
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 655 #7
Parallel Hybrid Multiobjective Metaheuristics on P2P Systems
36-655
M3
M2
M1 J2
J2
J4 J5 J1 J3J6
J5 J1
J1
J4 J6 J3
J2 J4 J5 J6 J3
FIGURE 36.4 Example of permutation flow-shop with 6 jobs and 3
machines.
36.4 Application to BPFSP and Experimentation
36.4.1 Problem Formulation
The Flow-Shop problem is a scheduling problem [12] that has
received a great attention given its import-ance in many industrial
areas. The problem can be formulated as a set of N jobs J1, J2, . .
. , JN to bescheduled on M machines. The machines are critical
resources as each machine cannot be simultaneouslyassigned to two
jobs. Each job Ji is composed of M consecutive tasks ti1, . . . ,
tiM , where tij represents thejth task of the job Ji requiring the
machine mj . To each task tij is associated a processing time pij ,
and eachjob Ji must be achieved before a due date di .
In this chapter, we focus on the BPFSP where jobs must be
scheduled in the same order on all themachines (see Figure 36.4).
Therefore, two objectives have to be minimized:
Cmax: Makespan (Total completion time) T : Total tardiness
The task tij being scheduled at time sij , the two objectives
can be formulated as follows:
f1 = Cmax = max{siM + piM |i [1, . . . , N ]},f2 = T =
Ni=1[max(0, siM + piM di)].
The Pareto front PF associated with BPFSP may be formulated as
follows:
y , x PF , (m(x) m(y)) or (t (x) t (y)),
where x and y are solutions of the MOP, and m(x) (respectively t
(x)) is the value of x corresponding tothe makespan (respectively
tardiness) criterion.
36.4.2 A GeneticMimetic Algorithm for Solving BPFSP
In single objective optimization, it is well known that GAs
provide better results when they are hybridizedwith LS algorithms.
Indeed, the GA convergence is too slow to be really effective
without any hybridization[10]. In Reference 2, a hybrid
GeneticMimetic algorithm named AGMA has been proposed for
solving
AGMA combines a genetic algorithm (GA) and a mimetic algorithm
(MA). In this chapter, we do notgive the details and parameters of
the two algorithms, and if needs be, the reader is referred to
Reference 2.The GA uses mainly two parameters: an archive (Pareto
Front) PO of nondominated solutions, and aprogression ratio PPO of
PO. At each generation, these two parameters are updated. If no
significantprogression is noticed (PPO < , where is a fixed
threshold), an intensified search process is triggered.
generation. The application of MA returns a Pareto Front PO that
serves to update the Pareto Front POof the GA.
2006 by Taylor & Francis Group, LLC
Figure 36.5.
The intensification consists in applying MA (see Algorithm 36.2)
to the current population during one
BPFSP. The simplified pseudocode of the algorithm is presented
in Algorithm 36.1 and illustrated in
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 656 #8
36-656 Handbook of Bioinspired Algorithms and Applications
Algorithm 36.1 AGMA Algorithm
Create an initial populationwhile run time not reached do
Perform a GA generation with adaptive mutationUpdate PO and
PPOif PPO < then
Perform a generation of MA on the population (Algorithm 2)Update
PO and PPO
end ifUpdate selection probability of each mutation operator
end while
POP
Genetic algorithm Mimetic algorithm
POPCrossover
PO*
PO* POP
Ppo* < a
Neighbors
FIGURE 36.5 Illustration of AGMA.
Algorithm 36.2 MA algorithm
while MA run time not reached doSelect randomly a set P of
solutions from the current populationApply the crossover on P to
generate a set P of new solutionsCompute the nondominated set PO
from P while New solutions found do
Create the neighborhood N of each solution of POLet PO be the
nondominated set of N
PO
end whileend while
Mimetic algorithm consists in selecting randomly a set of
solutions from the current population ofthe GA. A crossover
operator is then applied to these solutions and new solutions are
generated. Amongthese new solutions only nondominated ones are
maintained to constitute a new Pareto Front PO. AnLS is then
applied to each solution of PO to compute its neighborhood. The
nondominated solutionsbelonging to the neighborhood are inserted
into PO.
36.4.3 Parallel Hybrid AGMA
Different parallel models have been sketched and analyzed in
Section 36.2. The fine-grained parallel modelscould not be
exploited efficiently in a P2P environment due to the communication
delays. In BPFSP, themodel based on parallel evaluation of each
solution is fine-grained and is not likely to lead to better
2006 by Taylor & Francis Group, LLC
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 657 #9
Parallel Hybrid Multiobjective Metaheuristics on P2P Systems
36-657
performance. Indeed, the evaluation of each objective has a low
cost. Therefore, it is useless to evaluatein parallel the two
objectives and evaluate each of them in parallel. Conversely, it is
useful to exploit thefollowing parallel models: (1) the island
model that consists in performing in parallel several
cooperativeAGMAs; (2) the parallel evaluation of the population of
each AGMA; (3) the multistart model that consistsin applying in
parallel an LS on each solution of the Pareto Front PO in MA. The
parallel evaluation ofthe neighborhood of each solution could not
be efficient for the same reason as the parallel evaluation ofeach
solution.
We have limited our implementation to the coarse-grained
parallel models, that is, the island modeland the multistart model.
Figure 36.6 illustrates the parallel hybrid AGMA exploiting these
two models.
The island model : Due to its exorbitant cost in terms of CPU
time on large-size instances of BPFSP,the island model has not been
exploited in Reference 2. Indeed, the exploitation of the model
onlarge-size BPFSP is possible only on large-scale P2P networks or
grids. In our implementation(see Figure 36.6), the parameters of
the model are the following: The different cooperative AGMA
POP POPCrossover
Neighbors
PO*
PO* POP
Ppo* < a
PO*
XtremWeb workers
XtremWebinterface
Multistartmodel
Islandmodel
Geneticalgorithm Mimetic
algorithm
FIGURE 36.6 Illustration of parallel AGMA.
2006 by Taylor & Francis Group, LLC
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 658 #10
36-658 Handbook of Bioinspired Algorithms and Applications
exchange their whole archives PO, and the number of emigrants is
dynamic. At its arrival, theimmigrant archive is merged with the
local one. Migrations occur periodically (each a fixed numberof
iterations). The migration topology is the random one, meaning the
destination island is selectedrandomly.
The multistart model : The multistart model is exploited during
the execution of MA. Each solutionof the Pareto Front PO computed
by the algorithm represents the initial solution of an LS
methodthat calculates its neighborhood. The different LSs are
executed in parallel according to the MasterSlave model. The
master, that is, the algorithm MA merges with PO the neighborhoods
returnedby the different slaves and computes the new PO that
contains the nondominated solutions.
36.4.4 Deployment and Experimentation
In this section, we present the deployment scheme of the
different parallel hybrid models on the XtremWebarchitecture, and
the preliminary experimental results obtained on the application
presented above.
36.4.5 Deployment and Fault Tolerance
A deployment scheme may be defined as a function that consists
in embedding the different componentsof the parallel models on the
different components of the P2P architecture. Different deployment
schemaof the island and multistart models on XtremWeb are possible.
Indeed, the AGMA algorithms of the islandmodel can be deployed
either as XtremWeb clients or workers. For the multistart model,
the master canbe either a client or a worker, and the slaves are
necessarily deployed as workers. For our experimentation,the
deployment scheme is illustrated in Figure 36.7.
The island model is deployed on three XtremWeb clients, and each
client runs the AGMA algorithm.During the hybridization phase
(execution of MA), the LSs initiated on the Pareto Front PO are
submittedas tasks to the Dispatcher that launches them on the
workers at their request. The multistart model is thusdeployed on a
client and a set of workers.
In XtremWeb, the fault tolerance issue is tackled at Worker and
Dispatcher levels. When a worker failsthe work unit being executed
is re-started from scratch. If the Dispatcher crashes it is
re-started usingits information database. The problem with such
solution is that in a highly volatile environment a large
Dispatcher
LSNeighbors
MySQL
LS11 LS25 LSm4
LSNeighbors
Worker1 Worker2 WorkerN
Client1 Client2 Clientm
LSNeighbors
...
PO*PO*
FIGURE 36.7 Deployment schema of parallel hybrid AGMA on top of
XtremWeb.
2006 by Taylor & Francis Group, LLC
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 659 #11
Parallel Hybrid Multiobjective Metaheuristics on P2P Systems
36-659
amount of CPU time is wasted as the system spends its time in
re-starting work units performed by theworkers. Therefore, we
propose a check-pointing approach at the client level that allows
to solve moreefficiently the fault-tolerance problem. Indeed, the
problem data and intermediate results are periodicallystored. If
the Dispatcher fails the application is restored and re-started
from the last checkpoint. In caseof worker failure the work unit is
re-started using the intermediate results. The check-pointing
operation(storing) is performed after each LS and/or every 100
generations. The second condition is necessary whenno LS has been
launched during the last 100 generations. This means that a
significant progression of thePareto Front has been observed at
each generation, what excludes any resort to the hybridization.
36.4.6 Experimental Results
In our experiments, we consider the BPFSP instance 200 jobs on
10 machines. The parameters of theisland model are fixed as the
following: migrations occur every 10 generations, the number of
emigrantsis fixed at each migration operation to 20 the size of the
archive PO is upper than 20, and the wholearchive otherwise, and
the population size of each AGMA is 100.
The application has been deployed during working days
(nondedicated environment) on the educationnetwork of the
PolytechLille engineering school. The experimentation hardware
platform is composed of120 heterogeneous Linux Debian PCs. The
characteristics of these PCs are presented in Table 36.1.
Three parallel hybrid versions of AGMA are experimented,
evaluated and compared:
Version 1 is that proposed in Reference 2, and exploits only the
multistart model. This meansthat the GA is executed on a single
machine and the hybridization phase is deployed on a
parallelmachine (IBM-SP2) according to the MasterSlave model (Push
mode, i.e., work distribution isinitiated by the master).
Version 2 is the same as Version 1 except that the hybridization
is deployed in a distributed wayon a set of XtremWeb workers
according to the cycle stealing paradigm (Pull mode, i.e.,
workdistribution is initiated by the workers).
Version 3 is not considered in Reference 2, and is a combination
of the multistart and island models.
according to the island model. Each AGMA is an implementation of
Version 2.
are approximately the same, but Version 2 has the advantage to
be fault tolerant.The execution of Version 1 is stopped after 80
LSs as it is not fault tolerant. Conversely, with Version 2
350 LSs. The execution lasted one week, and 10 failures have
been observed and as many check-pointingoperations have been
performed. As a result, the Pareto Front obtained with 350 LSs is
clearly better thanthat obtained with 80 LSs using Version 1 or
Version 2. One has to note that such results are possible onlywith
a scalable and fault tolerant version of the algorithm.
TABLE 36.1 Experimentation Hardware Platform
Processor Number
AMD Duron(tm) Processor 14Celero (Coppermine) 14Intel(R)
Celeron(R) CPU 2.00 GHz 8Intel(R) Celeron(R) CPU 2.20 GHz
28Intel(R) Celeron(R) CPU 2.40 GHz 21Intel(R) Celeron(R) CPU 1400
MHz 7Pentium III (Katmai) 28
Total 120
2006 by Taylor & Francis Group, LLC
As illustrated in Figure 36.7, three AGMA algorithms are
deployed on client machines and cooperate
Figure 36.8 illustrates the Pareto Fronts obtained with the
versions 1 and 2 after 80 LSs. The two fronts
long-lasting executions are possible. For instance, Figure 36.9
shows that the execution goes on up to
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 660 #12
36-660 Handbook of Bioinspired Algorithms and Applications
40000 45000 50000 55000 60000 65000 7000010850
10900
10950
11000
11050
11100
11150
11200
11250M
akesp
an
Version 1 (80 LS)Version 2 (80 LS)
Tardiness
FIGURE 36.8 Pareto Fronts obtained with Version 1 and Version 2
(80 LSs).
35000 40000 45000 50000 55000 60000 65000 7000010850
10900
10950
11050
11000
11100
11200
11150
11250
11300
11350
Mak
esp
an
Version 1 (80 LS)Version 2 (350 LS)
Tardiness
FIGURE 36.9 Pareto Fronts with Version 1 (80 LSs) and Version 2
(350 LSs).
2006 by Taylor & Francis Group, LLC
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 661 #13
Parallel Hybrid Multiobjective Metaheuristics on P2P Systems
36-661
42000 44000 46000 48000 50000 52000 54000 56000 58000 60000
62000 6400010850
10900
10950
11000
11050
11100
11150
11200
11250M
akesp
an
Version 2 (80 LS)Version 3 (80 LS)
Tardiness
FIGURE 36.10 Pareto Fronts with Version 2 (80 LSs) and Version 3
(80 LSs).
Figure 36.10 allows to compare the Pareto Fronts obtained with
Version 2 and Version 3 and todemonstrate the contribution of the
island model to the effectiveness. With 80 LSs, the Pareto
Frontobtained using Version 3 is better than that obtained using
Version 2. More experiments with more LSsare in progress.
Version 3. Figure 36.11 (Part B) is a zoom up of Figure 36.11
(Part A) on the 50,000 first time units. Itshows that the MA (thus
LS) is frequently solicited and this lasts long. Figure 36.11 (Part
C) illustrates theevolution in time of the number of deployed
workers at the beginning of the execution (zoom out on thefirst 350
time-units). The maximum number of workers is 60 because during the
starting phase the ParetoFront contains a small number of
solutions. The number of workers decrease to 0 when the GA
succeedto improve the Pareto Front without calling MA.
One has to note that the spectrum blackens with the time (from
left to right). This means that the GAsolicits more and more the
MA, that is, the LS because it never enhances again the Pareto
Front, in otherwords the GA converges. On the other hand, the local
search lasts less and less time. Therefore, even theintensification
(by LS) does not contribute to enhance the effectiveness, meaning
that the AGMA con-verges. Through this experimentation, we have
learned more on the convergence of the AGMA algorithm.Therefore,
one can note that P2P computing allows topush farthe limits in
terms of computing resourcesto better evaluate the contribution of
the hybridization but also its limitations.
36.5 Conclusion and Future Work
The hybridization of metaheuristics having complementary
behaviors allows to enhance the effectivenessand robustness in
combinatorial optimization [11]. However, its exploitation on
industrial applications ispossible only by using a great computing
power. Large-scale parallelism based on the use of
computationalgrids and/or P2P systems is recently revealed to be a
good way to get at hand such computing power andexploit
hybridization. To our best of knowledge, no research work has been
published on parallel hybrid
2006 by Taylor & Francis Group, LLC
Figure 36.11 (Part A) shows the oscillation between GA and MA
(or LS) over time obtained with
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 662 #14
36-662 Handbook of Bioinspired Algorithms and Applications
.
TimeMA(LS)
GA
Time
Time
(Part A)
(Part C)
5000040000100005000 3000020000 45000350002500015000
500000450000350000 400000300000000mps
0
(Part B)
0 50 100 150 200 250 300 350
60
50
40
30
20
10
0
Number of workers
FIGURE 36.11 Oscillation spectrum between the GA and MA (thus
LS).
metaheuristics on P2P systems. Nowadays, existing P2P computing
middlewares are inadequate for thedeployment of parallel
cooperative applications. Indeed, these need to be extended with a
software layerto support the cooperation. In this chapter, we have
proposed a Linda-like cooperation model that hasbeen implemented on
top of XtremWeb.
In Reference 2, a hybrid metaheuristic (AGMA) has been proposed
and experimented on BPFSP.The performed experiments on large-size
instances such as 200 jobs on 10 machines are often stoppedwithout
the convergence is reached. The full exploitation of the
hybridization needs a large amount ofcomputational resources and
the management of the fault-tolerance issue. We have proposed a
fault-tolerant hybrid parallel design of the AGMA combining two
parallel models: the multistart model and theisland model. The
algorithm has been implemented on our extended version of
XtremWeb.
The first experiments have been performed on the education
network of the PolytechLille engineeringschool. The network is
composed of 120 heterogeneous Linux PCs. The preliminary results,
obtainedafter several execution days, demonstrate that the use of
P2P computing allows to fully exploit the benefitsof hybridization.
Indeed, the obtained Pareto Front is clearly better than that
obtained in Reference 2.
2006 by Taylor & Francis Group, LLC
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
-
CHAPMAN: C4754_C036 2005/8/6 14:14 page 663 #15
Parallel Hybrid Multiobjective Metaheuristics on P2P Systems
36-663
On the other hand, the deployment of the island model allows to
improve the effectiveness. Beyond theimprovement of the
effectiveness, the parallelism on P2P systems allows to push far
the limits in termsof computational resources. As a consequence, it
permits to better evaluate the benefits and limitationsof the
hybridization. Such result has to be confirmed again on a larger
P2P network and larger instancesof the problem.
References
[1] D.P. Anderson, J. Cobb, E. Korpela, M. Lepofsky, and D.
Werthimer. SETI@home: An experimentin public-resource computing.
Communications of the ACM, 45: 5661, 2002.
[2] M. Basseur, F. Seynhaeve, and E.-G. Talbi. Adaptive
mechanisms for multi-objective evolution-ary algorithms. In
Congress on Engineering in System Application CESA 03, Lille,
France, 2003,pp. 7286.
[3] S. Cahon, N. Melab, and E.-G. Talbi. ParadisEO: A framework
for the reusable design of paralleland distributed metaheuristics.
Journal of Heuristics, 10: 353376, 2004.
[4] G. Fedak, C. Germain, V. Neri, and F. Cappello. XtremWeb:
Building an experimental platformfor Global Computing. Workshop on
Global Computing on Personal Devices (CCGRID2001), IEEEPress, May
2001.
[5] I. Foster and C. Kesselman. The Grid: Blueprint for a New
Computing Infrastructure. MorganKaufmann, San Fransisco, CA,
1999.
[6] D. Gelernter and N. Carriero. Coordination languages and
their significance. Communications ofthe ACM, 35: 97107, 1992.
[7] D. Gelernter. Generative communication in Linda. ACM
Transactions on Programming Languagesand Systems, 7: 80112,
1985.
[8] A. Oram. Peer-to-Peer: Harnessing the Power of Disruptive
Technologies. OReilly & Associates, 2001.[9] G.A. Papadopoulos
and F. Arbab. Coordination models and languages. Advances in
Computers:
The Engineering of Large Systems, Academic Press, 1998, p.
46.[10] E.-G. Talbi, M. Rahoual, M.-H. Mabed, and C. Dhaenens. A
hybrid evolutionary approach
for multicriteria optimization problems: Application to the Flow
Shop. In E. Zitzler et al.,Eds., Evolutionary Multi-Criterion
Optimization, Vol. 1993 of Lecture Notes in Computer
Science,Springer-Verlag, Heidelberg, 2001, pp. 416428.
[11] E.-G. Talbi. A Taxonomy of Hybrid Metaheuristics. Journal
of Heuristics, 8: 541564, 2002.[12] V. Tkindt and J.-C. Billaut.
Multicriteria Scheduling Theory, Models and Algorithms.
Springer-
Verlag, Heidelberg, 2002.[13] J. Verbeke, N. Nadgir, G. Ruetsch,
and I. Sharapov. Framework for peer-to-peer distributed com-
puting in a heterogeneous, decentralized environment. In
Proceedings of the Third InternationalWorkshop on Grid Computing
(GRID 2002), Baltimore, MD, January 2002, pp. 112.
2006 by Taylor & Francis Group, LLC
Dow
nloa
ded
by [I
ndian
Insti
tute o
f Tec
hnolo
gy, K
anpu
r], [T
ATHA
GATA
BHA
DURI
] at 0
8:34 0
1 Jun
e 201
5
Chapter 36: Parallel Hybrid Multiobjective Metaheuristics on P2P
Systems36.1 Introduction36.2 Parallel Hybrid MOMs and P2P
Computing36.2.1 Multiobjective Optimization36.2.2 Parallelism and
Hybridization36.2.3 P2P Computing for Parallel MO Optimization
36.3 A Model for P2P Coordination36.3.1 Model Description36.3.2
Implementation on Top of XtremWeb
36.4 Application to BPFSP and Experimentation36.4.1 Problem
Formulation36.4.2 A GeneticMimetic Algorithm for Solving
BPFSP36.4.3 Parallel Hybrid AGMA36.4.4 Deployment and
Experimentation36.4.5 Deployment and Fault Tolerance36.4.6
Experimental Results
36.5 Conclusion and Future WorkReferences