Slide 1
Introduction to multi-objective optimizationWe often have more
than one objectiveThis means that design points are no longer
arranged in strict hierarchyThere are points that are clearly
poorer than others because all objectives are worseIn optimization
jargon we call these points dominatedPoints that are not dominated
are called non-dominated or Pareto optimal
In many engineering problems we have more than one objective.
Typical objectives in structural optimization may be weight, cost,
failure load or probability of failure.
Once we have more than one objectives we cannot arrange points
in strict hierarchy. For example, I worked on a problem of
designing composite laminates, where we could combine two
materials, glass epoxy and graphite epoxy. The former was cheaper
and heavier than the latter.
While some designs cannot be ranked compared to other designs,
some can. If we have the two objectives of weight and cost, and one
design is both lighter and cheaper than the other, it is clearly
better. We say that the poorer design is dominated.
Points in design space that are not dominated are called
non-dominated, or Pareto optimal. The set of all Pareto-optimal
points is called the Pareto front.
The Italian economist Vilfredo Pareto (1848-1923) was one of the
leaders of the Lausanne School. Although only mildly influential
during his lifetime, his "tastes-and-obstacles" approach to general
equilibrium theory were resurrected during the great "Paretian
Revival" of the 1930s and have guided much of economics since (from
Wikipedia)
Chapter 11 of Belegundu and Chandrupatla,s Optimization Concepts
and Applicatons in Engineering, is placed in the resource section
of the course for additional material.
1Definition of Pareto optimalityFor a problem with m objective
functions, a design variable vector x* is Pareto optimal if and
only if there is no vector x in the feasible space with the
characteristics
This slide provides the formal definition of Pareto optimality.
Given an optimization problem with m objectives, a feasible x* is
Pareto-optimal or non dominated, if we cannot find another feasible
x that dominates it. That is, we cannot find an x that has a lower
value for one objective and is at least as good for all the
others.2Problem dominationWe have two objectives that are to be
minimized. The following are the pairs of objectives at 10 design
points. Identify the dominated ones.
(67,71) (48,72) (29,88) (-106, 294) (32,13), (-120,163),
(103,-30) (-78,114) (-80,143) (75,-171) Solution in notes page
(67,71) (48,72) dominated by (32,13)
(-106, 294) dominated by (-120,163)
(103,-30) dominated by (75,-171)3Work choice exampleItem
(task)Pay ($)Time (min)Fun index1133225231224321Minimize time and
maximize fun so that you make $100. Will need between 33.3 to 100
items. Time can vary from 66.7 minutes to 300. Fun can vary between
33.3 and 300.
Choosing what line of work you want to engage in is often a
matter of maximizing your income, maximizing your free time, and
maximizing your job satisfaction. Here is a problem that captures
these considerations. However, to reduce the number of objectives
to two, we assume that you need to make $100 a day from a mix of
four different tasks defined in the table.4Comparing tasksItem
(task)Pay ($)/hourTotal time (hours)Total
fun12053002244.171003303.331004901.1133.3task 2 is dominated by
task 3. Tasks 1,3,4 are Pareto optimal
One gets more insight into the selection process by looking at
the amount of time required to make $100 from one task. This is
shown in the table, showing that as we progress we reduce time and
fun at the same time. Except from the second task, which is clearly
inferior (or dominated) to task 3 which produces the same fun in
less time.5Multi-objective Formulation
ItemPay Time Fun 1133225231224321
We now pose the problem as an optimization problem, defining the
design variable xi as the number of times we repeat task i. The
constraint on the required $100 is written in a standard and
normalized form.
The common sense constraints derive from the extreme case of
using only one task to satisfy the dollar requirement. Thus, if we
use only Task 1, which pays $1, we will need to repeat it 100
times. If we also do some other tasks, x1 will be smaller than 100,
hence the common sense constraint on x1.
6Solution by enumerationIn the range of three variables
calculate time and fun for all combinations that produce exactly
$100.Pareto front is upper boundary and it is almost a straight
line.Matlab code given in notes page.
function [ fun,time,pay,npoint,x] = multobjfun()%Fun
maximization and time minimization in task selection% Go over all
selections of tasks 1,2,3,4 from multi-optimization% lecture that
yield a pay of $100.npoint=0;for i1=0:100for i3=0:100 for i4=0:34
payc=i1+i3+3*i4; if payc==100 npoint=npoint+1;
pay(npoint)=i1+i3+3*i4; fun(npoint)=3*i1+2*i3+i4;
time(npoint)=3*i1+2*i3+2*i4; x(1,npoint)=i1; x(3,npoint)=i3;
x(4,npoint)=i4; end endendend end
7Pay-fun ProblemFormulate the problem of maximizing fun and pay
for five hours (300 minutes) including only non-dominated
variables. SolutionObtain the Pareto front analytically or by
enumeration and writes its equation (fun as a function of pay).
Solution.8More efficient solution methodsMethods that try to avoid
generating the Pareto frontGenerate utopia pointDefine optimum
based on some measure of distance from utopia pointGenerating
entire Pareto frontWeighted sum of objectives with variable
coefficientsOptimize one objective for a range of constraints on
the othersNiching methods with population based algorithms
In most problem we cannot solve the problem by enumeration, or
by covering the design space by a dense grid. For a more efficient
approach, there are methods that avoid generating the Pareto front,
and instead try for a predetermined compromise.
One example is to start by generating a utopia point, which is
obtained by minimizing each function individually and then taking
the best values of all objectives. Then a single objective can be
generated based on some measure of the distance from the utopia
point, as will be shown in the example on the next slide.
For generating the Pareto front three approaches are popular.
The first takes a weighted average of the objectives for a range of
weights. For example, 0.1 times the first objective plus 0.9 times
the second, then 0.2 times the first plus 0.8 times the second, and
so on. This method may fail to produce the entire front if it is
not convex.
A second method, which I recommend is to optimize one objective
for a series of constraints on the other objective, which we will
look into for the task example.Finally, multi-objective genetic
algorithms will be studied later on in the course.9The utopia point
is (66.7,300).One approach is to use it to form a compromise
objective as shown above.It gives time=168 minutes, fun=160
(x3=76,x4=8)
For our problem, the best time is 66.7 minutes, and the highest
fun index is 300, so the utopia point is (66.7, 300). If we have
the entire front, we may decide to select the nearest point to the
utopia point, but that would be difficult to generate without
knowing the entire front.
Instead, we may use the utopia values to normalize a combined
objective, as shown in the equation on the right. Searching the
vectors of time and fun obtained by enumeration, we find that the
compromise objective is minimized for time=168, and fun index=160,
corresponding to x1=0, x3=76, x4=8.10Series of constraints
The next slides provides a Matlab segment for solving this
optimization problem using the function fmincon, but without the
requirement of integer variables.How will you change the
formulation so that a single Matlab run will give us results for
any required earning level R? Solution in the notes page.The
recommended approach for generating the Pareto front is to minimize
one objective with a range of constraints on the other objectives.
Here we selected to minimize the time for a series of constraints
on the fun. We know that the best time is obtained by using only
x4, and then the fun is 30, while the highest fun is objtained when
we use only x1, and then it is equal to 300.
So in the next slide we will show a Matlab segment that solves
this problem for a series of constraints on the fun ranging from 30
to 300 in increments of 5.
If the required earning (pay) is R instead of 100, that is
x1+x3+3x4R
we can normalize the design variables as yi=100xi/R, then we can
use the same problem formulation as before, and just multiply the
solution by R/100.11Matlab segmentx0 = [10 10 10];for fun_idx =
30:5:300 A = [-1 -1 -3; -3 -2 -1]; b = [-100;-fun_idx]; lb =
zeros(3,1); options = optimset('Display','off');
[x,fval,exitflag,output,lambda] =
fmincon('myfun',x0,A,b,[],[],lb,[],[],options);
pareto_sol(fun_idx,:) = x; pareto_fun(fun_idx,1) = fval;
pareto_fun(fun_idx,2) = 3*x(1) + 2*x(2) + x(3);End
function f = myfun(x)f = 3*x(1) + 2*x(2) + 2*x(3);
We will study the algorithm behind fmincon later in the
semester. For the time being you may want to look at the
documentation for fmincon, in order to clarify the Matlabe
statements above.12Pareto-front ProblemGenerate the Pareto front
for the pay and fun maximization using a series of constraints, and
also find a compromise point on it using the utopia point.
SolutionWhat is responsible for the slope discontinuity in the
Pareto front on Slide 10? Solution in notes pageBefore the
discontinuity point, the Pareto optimal correspond to x1=0, while
after the discontinuity point, the Pareto optimal correspond to
x4=0. This is because in the first part of Pareto front, time
minimization is more dominant, x3 and x4 are better choices since
their time/pay ratio is lower than for x1 . And in the second part
of Pareto front, fun maximization becomes more dominant, x1 and x3
are better choices since their fun/pay ratio is higher than
x4.13