Slide 1
Graphical optimizationSome problems are cheap to simulate or
test.Even if they are not, we may fit a surrogate that is cheap to
evaluate.Relying on optimization software to find the optimum is
foolhardy.It is better to thoroughly explore manually.In two
dimensions, graphical optimization is a good way to go.For problems
with more variables, it may still be good to identify two important
variables and proceed graphically.In higher dimensions, two
dimensional cuts can help understand the relationship between local
optima.
1ExamplePlot and estimate minimum ofIn the rangeCan do contour
plot or mesh plot
x=linspace(-2,0,40); y=linspace(0,3,40);[X,Y]=meshgrid(x,y);
Z=2+X-Y+2*X.^2+2*X.*Y+Y.^2;cs=contour(X,Y,Z);
cs=surfc(X,Y,Z);xlabel('x_1'); ylabel('x_2');
zlabel('f(x_1,x_2)');2Then zoomZooming by reducing the range does
not change the estimate of the position (-1,1.5), but the value is
now below 0.8.
x=linspace(-1.5,-0.5,40);
y=linspace(1,2,40);[X,Y]=meshgrid(x,y);Z=2+X-Y+2*X.^2+2*X.*Y+Y.^2
Once we know the approximate position of the minimum we can zoom
on a narrower range using the Matlab sequence:
x=linspace(-1.5,-0.5,40);y=linspace(1,2,40);[X,Y]=meshgrid(x,y);Z=2+X-Y+2*X.^2+2*X.*Y+Y.^2
The figures do not change our estimate of the position of the
minimum (-1,1.5), but update our estimate of the value of the
minimum (below 0.8).3In higher dimensionsOften different solutions
of optimization problems are obtained from multiple starting points
or even from same starting point.It is tempting to ascribe this to
local optima.Often it is the result of algorithmic or software
failure.Generate line connecting two solutions
4ExampleConsider Two candidate local optima (0,0,0) and
(0.94,0.94,0.94).Line connecting first and second point indicates
that they may be local optima.
alpha=linspace(0,1,101);x1=[0 0 0]; x2=[0.94 0.94
0.94];x=x2(1)*alpha+x1(1)*(1-alpha);
y=x2(2)*alpha+x1(2)*(1-alpha);z=x2(3)*alpha+x1(3)*(1-alpha);f=abs(sin(2*pi.*x).*sin(2*pi.*y).*sin(2*pi.*z))+x.*sqrt(y).*z.^2;plot(alpha,f);
5Two-dimensional cut
6ExampleOptimize g=@(x)
(sin(2*pi*x(1))*sin(2*pi*x(2))*sin(2*pi*x(3)))+x(1)*sqrt(x(2))*x(3)^2;
X=fminsearch(@(x) g(x), [0,0,0])
7
Unconstraiend optimization in Matlab
[X,gval,exitflag]=fminsearch(@(x) g(x), [0,0,0])X =0.2492 0.2496
-0.2485gval =-0.9923exitflag =1[X,gval,exitflag]=fminsearch(@(x)
g(x), [0.5,0.5,0.5])X =0.2487 0.7498 0.2473gval =-0.9867exitflag =1
1 Maximum coordinate difference between current best point and
other points in simplex is less than or equal to TolX, and
corresponding difference in function values is less than or equal
to TolFun. 0 Maximum number of function evaluations or iterations
reached. -1 Algorithm terminated by the output function.
Can set convergence criteria in optimset.
8Alternative using derivatives[X,gval,exitflag]=fminunc(@(x)
g(x), [0.24,0.24,-0.24])Warning: Gradient must be provided for
trust-region algorithm; using line-search algorithm instead. >
In fminunc at 383 Local minimum found.Optimization completed
because the size of the gradient is less than the default value of
the function tolerance.
X =0.2492 0.2496 -0.2484gval = -0.9923exitflag =1
[X,gval,exitflag]=fminunc(@(x) g(x), [0,0,0])Warning: Gradient
must be provided for trust-region algorithm; using line-search
algorithm instead. > In fminunc at 383Initial point is a local
minimum.Optimization completed because the size of the gradient at
the initial point is less than the default value of the function
tolerance.
X = 0 0 0gval = 0exitflag = 1
10 dimensional constrained exampleAerodynamic design of
supersonic transport.Objective function, take-off gross weight is a
cheap and simple function of design variables.Design variables
define geometry of wing and fuselage.Constraints include range,
take-off and landing constraints, and maneuvrability
constraints.The constraints are all uni-modal functions, but in
combination, they create a complex, non-convex feasible domain.
To complement the previous simple example we also look at an
example where multiple optima are not due to waviness of the
objective functions but due to the fact that the feasible domain
(the domain where all constraints are satisfied) is not convex
(that is two points in the domain can be connected by a line that
passes outside the domain). The example is taken from: Knill, D.L.,
Giunta, A.A., Baker, C.A., Grossman, B., Mason, W.H., Haftka, R.T.,
and Watson, L.T., Response Surface Models Combining Linear and
Euler Aerodynamics for Supersonic Transport Design, Journal of
Aircraft, 36(1), pp. 75-86, 1999
The objective function is the take-off weight of a supersonic
transport, the design variables define the shape of the wing and
fuselage, and there is a large number of constraints such as range,
take-off clearance constraints, and maneuverability
constraints.
All the constraints are approximated by quadratic polynomials,
so they are not wavy, and the objective function is also unimodal
(not wavy). However the constraints still create different local
optima at vertices of the feasible domain. 10Simple constraints
create non-convex design space.Can thathappen withlinear
constraints?
Two local optima and a third point
11
Problems