Optimisation • The general problem: Want to minimise some function F(x) subject to constraints, a i (x) = 0, i=1,2,…,m 1 b i (x) 0, i=1,2,…,m 2 where x is a vector of length n. • F( ) is called the objective function. • a i ( ) and b i ( ) are called the constraint functions.
23
Embed
Optimisation The general problem: Want to minimise some function F(x) subject to constraints, a i (x) = 0, i=1,2,…,m 1 b i (x) 0, i=1,2,…,m 2 where x.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Optimisation
• The general problem:Want to minimise some function F(x) subject to constraints,
ai(x) = 0, i=1,2,…,m1
bi(x) 0, i=1,2,…,m2
where x is a vector of length n.
• F( ) is called the objective function.• ai( ) and bi( ) are called the constraint
functions.
Special Cases
• If n=1 there is just one variable, and we have the univariate case (as opposed to the multivariate case).
• If ai(x) and bi(x) are linear functions then we have linear constraints (as opposed to nonlinear constraints).
• If m2=0 we have equality constraints only.
• If m1=0 we have inequality constraints only.
• If m1=m2=0 we have the unconstrained case.
Techniques
• The techniques used to solve an optimisation problem depends on the properties of the functions F, ai, and bi.
• Important factors include:– Univariate or multivariate case?– Constrained or unconstrained problem?– Do we know the derivatives of F?
Example Linear Problem• An oil refinery can buy light crude at £35/barrel and
heavy crude at £30/barrel. • Refining one barrel of oil produces petrol, heating oil,
and jet fuel as follows:
Petrol Heating oil Jet fuel
Light crude 0.3 0.2 0.3
Heavy crude 0.3 0.4 0.2
• The refinery has contracts for 0.9M barrels of petrol, 0.8M barrels of heating oil and 0.5M barrels of jet fuel.
• How much light and heavy crude should the refinery buy to satisfy the contracts at least cost?
Problem Specification
• Let x1 and x2 be the number of barrels (in millions) of light and heavy crude that the refinery purchases.
• Cost (in millions of £): F(x) = 35x1 + 30x2
• Constraints:
0.3x1 + 0.3x2 0.9 (petrol)
0.2x1 + 0.4x2 0.8 (heating oil)
0.3x1 + 0.2x2 0.5 (jet fuel)
x1 0, x2 0 (non-negativity)
This is called a “linear program”
Graphical Solution
• Minimum of F lies on boundary of feasible region.• F varies linearly on each section of the boundary.• Can get the solution by looking at the intersection points
of the constraints forming the boundary.
Feasible region
x2
x14321
1
2
(1, 2)
Solution
(x1,x2) F(x)
(0,3) 90
(2,1) 100
(4,0) 140
• So minimum cost is for x1 = 0 and x2 = 3.
Recall that:
F(x) = 35x1 + 30x2
Unconstrained Univariate Case• We seek to minimise f(x).
• If x* minimises f(x) then:i. f (x*) = 0 (first order condition)
ii. f (x*) 0 (second order condition)
f(x) = (x-1)(x-1)+2
Example• Minimise f(x) = x2 + 4Cos(x)
• Solve: f (x) = 2x – 4Sin(x) = 0
• y = fzero(@(x)(2*x-4*sin(x)),2)
• Gives y = 1.8955
Bisection Method
• Suppose we have already bracketed the zero in the interval [a,b]. Then:
1. Evaluate f at mid-point c=(a+b)/2.2. If f(c) is zero then quit.3. If f(a) and f(c) have the same sign then set a=c;
• Problems may arise if the initial estimate is not “sufficiently close” to the zero.
• Consider f(x)=ln(x).
e1
If 0<x1<e then Newton’s method will converge.
If x1e it will fail.
Linear Interpolation Methods
• Newton method requires first derivative at each iteration.• The bisection method doesn’t use the magnitudes of f at
each end of the interval.
• Suppose we use f(an) and f(bn) and finds a new estimate of the zero by approximating the function between an and bn by a straight line.
f(bn)
bn
an
xnf(an)
)()(
)(nn
nnnnn afbf
abafax
Secant Method
• The secant method is a linear interpolation method that generates approximations to the zero starting with x0 and x1 according to:
)()(
)(1
11
nn
nnnnn xfxf
xxxfxx
xn-1
xn xn+1 xn+2
Problem with divergence!
Method of False Position
• To avoid possible divergence problem with the secant method we keep the zero bracketed in an interval (a,b), as in the bisection method.
)()(
)(afbf
abafac
• If f(c) = 0 we are finished.• If f(a) and f(c) have the same sign we replace
a by c; otherwise, we replace b by c.
Golden Section Method
• A function is unimodal on an interval [a,b] if it has a single local minimum on [a,b].
• The Golden Section method can be used to find the minimum of function F on [a,b], where F is unimodal on [a,b].
• This method is not based on solving F(x)=0.
• We seek to avoid unnecessary function evaluations.
• If u>v then x* must lie in [a,x], and if uv then x* must lie in [y,b].• Case 1: If u>v then new interval is [a,x] and length is x-a=(b-a). At the next step we need to know
F at:
a + (x-a) = a + 2(b-a)
a + 2(x-a) = a + 3(b-a)• But we already know F at a + 2(b-a) from the previous step so we can avoid this function
evaluation.
Golden Section Method
• Divide interval [a,b] at x and y as follows:
x = a + (b-a); u = F(x)
y = a + 2(b-a); v = F(y) 6180339887.015
2
1
Golden Section Method
• Case 2: If uv then new interval is [y,b] and length is b-y=(b-a). At the next step we need to know F at:
y + (b-y) = a + 22(b-a)
y + 2(b-y) = a + 2(1+)(b-a) = a + (b-a)• But we already know F at a + (b-a) from the previous
step so we can avoid this function evaluation.• In both cases we get a new interval that is times the
length of the current interval, and each iteration requires only one function evaluation.
• After n iterations the error is bounded by (b-a)n/2