ch7

Instructor

Walid Morsi Ibrahim

Part II: Roots and Optimization

Fall 2010ENGR 1400 Information Technology for Engineers1

Roots


Quadratic formula

These values of x are called the roots or zeros of the equation.

Many functions are not solved so easily, thus we will look at the numerical

methods, for example: Bungee Jumper

If you want to solve for m, you cant bring m to the left hand side.

Subtract v(t) from both sides

And then find the roots, i.e. the value of m that makes f(m) = 0

Solution:

Optimization


Is the determination of optimal values.

For the figure shown, values of x that maximizes or

minimizes f(x)

Finding the Roots


Graphical methods: plot the function and see where the zero

crossings are.

We can zoom in on the plot for increased precision but there

is a great deal of human intervention.

Therefore it is very slow. More automated methods are

needed.

General Rules for Finding the Roots

Graphically


Consider xl and xu are the lower and upper limit points.

If both are positive or negative no root

Even number of roots

If they have different sign odd number of roots

Exceptions


Bracketing Methods and Initial Guess


Bracketing methods are based on two initial guesses, one on

either side of the root.

Advantage: always works

Disadvantage: converges slowly

Open methods have an initial guess but do not need to

bracket the root.

Advantage: more quicker

Disadvantage: may not work if the solution diverges

Increment Search


This method searches for an interval with a sign change.

It sates that: if f(x) is real and continuous in the interval from

xl to xu and f(xl) and f(xu) have opposite sign that is:

f(xl) f(xu) < 0

Then there is at least one real root between xl and xu

Limitations


If the increment length is too small then the search becomes

time consuming.

If the increment length is too large then some roots may be

missed.

Bisection Method


Start with initial bounds and cut the problem in half again

and again.

Ex: use bisection method to determine the drag coefficient

needed so that an 65 kg bungee jumper has a velocity of 35

m/s after 4.5 s of free fall. g = 9.81 m/s2. start with xl =

0.2, xu = 0.3 and iterate until < 2%.

Solution


Solution Cont.


Example on Bisection Method


Looking at the graph the root

Can be estimated to be at 145

Now try the same problem


f(50) = -ve and f(200) = +ve

Bracketing [50,200]

xl has to be updated. New interval

xu has to be updated. New interval

Run M-file bisection to get final value of the

root at the 8th iteration 142.7376

Graphical representation of Bisection

Method


See M-file bisectnew: Given xl = 40 and xu = 200, Sol mass = 142.7376

False Position Method


Rather than bisecting the interval, it locates the root by

joining f(xl) and f(xu) with a straight line. The intersection of

this line with the x axis represents an improved estimate of

the root.

Derivation of the FP Method


Referring to the figure:

[f(xu) - f(xl)]/[xu - xl ]= m [f(xu) - f(xl)]/[xu - xl ] = f(xu) /[xu - xr ]

And [f(xu) - 0]/[xu - xr ]= m

[xu - xr ] = [f(xu) [xu - xl ]]/ [f(xu) - f(xl)]

- xr = - xu + [f(xu) [xu - xl ]]/ [f(xu) - f(xl)]

xr = xu - [f(xu) [xu - xl ]]/ [f(xu) - f(xl)]

xr = xu - [f(xu) [xl - xu]]/ [f(xl) - f(xu)]

Disadvantages


The shape of the function influences the new root estimate

The figure shows slow convergence of the FP method

f(x)

root

x


Open Methods

Open Methods


In bracketing methods the root has to be located between an

upper and lower bound: xl and xu

In open methods the starting value (initial estimate) does not

need to bracket the root.

Although the solution converges more quickly , sometimes it

may diverge.

Comparison


Simple Fixed-point Iteration


Rearranging the function f(x) = 0 so that x is in the left hand

side of the equation: x = g(x)

Then given an initial estimate xi the new value of x can be

calculated: xi+1 = g(xi)

The process continue until the error criteria is met:

= abs([xi+1 – xi]/[xi+1]) x 100%

Example


Pros and Cons


Simple and easy formula

Slow convergence

Convergence depends mainly on the location of the initial

guess.

Divergence may occur

Newton-Raphson


The most widely used.

Considering the figure:

Example


Example


Four Cases of Poor Convergence


Function file Newton Raphson


Secant Method


If the derivative used in NR is not available then a finite

difference derivative can be used

Secant method requires two initial values xi and xi-1 to yield a

new value xi+1

An alternative approach uses one initial value and a small

perturbation fraction (modified secant method)

Convergence and Divergence


Example


Example Cont.


Trade-off in choosing the perturbation

value


If the perturbation value is chosen too small, round off error

could be increased especially in the denominator.

If the perturbation value is chosen too large, then the

solution may diverges and the algorithm becomes inefficient

MATLAB Function: fzero


Methods that we have seen so far areeither:

1. Slow but reliable (bracketing methods)

2. Fast but possibly unreliable (openmethods)

As a trade off, better results could beobtained using the fzero function.

Fzero is a combination of the bisection(reliable), secant (fast) and inversequadratic interpolation (very fast).

Note the inverse quadratic interpolationuses parabola instead of straight line.

Secant method

Inverse quadratic

interpolation

MATLAB Function: fzero


Syntax:

Disadvantages: root has to cross the x axis. If it touches the x

axis then it is not valid.

Ex: y = x.^2

Initial guess

Guesses that bracket a sign change

Polynomials


Are nonlinear algebraic equations of the general form:

MATLAB function: roots

Syntax:

Is a column vector

Containing the rootsIs a row vector containing

The polynomial coefficients

Example


Use MATLAB to find the roots of the following polynomial:

Sol:

Evaluate the polynomial at x = 1as substitute by x =1 in f5(x)

Polynomials


To get the polynomial from its roots:


Optimization

What is optimization?


It is the process of creating something that is as effective as possible.

Some examples of optimization:

1. Finding the minimum or maximum

2. Minimizing the fuel consumption

3. Maximizing the data rate

One popular method for optimization is to solve for the root of the problem:

Since at minimum or maximum the slope should be zero

Is it maximum or minimum?


We need to check the sign of that root:

1. If the sign is negative, then it is maximum

2. If the sign is positive then it is minimum.

Optimization types


One dimensional: One dependent variable f(x) based on one

independent variable x and we are searching for x*

Two dimensional: one dependent variable f(x,y) based on two

independent variables x and y and we are searching for x* and

y*

One Dimensional Optimization


Global optimum represents the very best solution

Local optimum is not the very best

Since optimization is based on finding roots then the

optimization methods are similar to the idea of bracketing

used in bisection method

Golden Section Search


Euclid’s definition of the golden ratio:

Select l1 and l2 such that:

[l1 + l2]/ l1 = l1 / l2Multiplying by

l1 / l2 = Ф gives:

Ф2 - Ф – 1 = 0

Solving for Ф

Ф = [1 + ]/2 = 1.618033 golden ratio

Like bisection method we need a lower and an upper bound of an interval containing a single minimum (unimodal)

l1 + l2

l1 l2

How to proceed with the Golden

Search?


We need to identify two points

between xl and xu say x1 and x2

if f(x2) < f(x1) therefore the

minimum lies between x1 and xu

Example:


Example Cont.


Since

For next iteration: new values (or updated values)

The process is repeated until the 8th iteration, optimum value

is:

MATLAB Function fminbnd


Similar algorithm to fzero

Find the minimum of a function of one variable within fixed

interval.

fminbnd for optimization (one dimensional) combines slow

dependable (golden section) and faster possible unreliable

(parabolic interpolation).

Syntax:

Location value of the min of f(x) Bounds of the interval being searched

Limitations: limited to one dimensional-only handles real

variables-slow convergence

Multi-dimensional Optimization


MATLAB function fminsearch

Syntax:

Unconstrained optimization

Start with an initial estimates

Only handles real values

If complex then it has to be split into two parts.

ch7

Documents

information technology

xu fxu xl xu fxl fxu

xu fxu xu xl fxu fxl

fxu xu xr xu xr

fxl fxu

new interval xu

fxu fxlxu

s of free fall