Roots of Equations - UNC Charlotte · Newton-Raphson Method Zeros of ... Solve for the variables and a Jacobi style iteration ... Introduction to Numerical Methods 27 Roots of Equations.

Roots of Equations

� Direct Search, Bisection Methods

� Regula Falsi, Secant Methods

� Newton-Raphson Method

� Zeros of Polynomials (Horner’s, Muller’s methods)

� EigenValue Analysis

ITCS 4133/5133: Introduction to Numerical Methods 1 Roots of Equations

Motivation

� Solution to many scientific and engineering problems

� Equations may be non-linear

� Equations expressible in the form F (x) = 0

� Values of x that satisfy the above equation are termed “roots”

� Example:Quadratic Equation

ax2 + bx + c = 0

with roots

x1 =−b +

√b2 − 4ac

2a

x2 =−b−

√b2 − 4ac

2a


Example Functions


Direct Search Method

Trial and Error Approach: Evaluate f (x) over some range of x at multiplepoints

� Specify a range within which the root is assumed to occur

� Subdivide range into smaller, uniformly spaced intervals

� Search through all subintervals to locate the root.


Direct Search:Searhing an interval

� Evaluate at end points of interval, and check if f (x) has oppositepolarity.


Direct Search:Disadvantages

⇒ Computationally expensive (linear search)

⇒ Intervals have to be very small, for high precision

⇒ If intervals are too large, roots can be missed.

⇒ Multiple roots cannot be handled:

f (x) = x2 − 2x + 1


Bisection Method

� Assumes a single root within the interval of interest.

� Similar to binary search: evaluates function at middle of interval andmoves to interval containing the root.

� Requires fewer calculations than direct search method.


Bisection Method:Algorithm


Bisection Method:Example


Error Analysis

Error measures can be an absolute value,

εd = |xm,i+1 − xm,i|

or, a percent error

εr =

∣∣∣∣xm,i+1 − xm,i

xm,i+1

∣∣∣∣ 100

True Accuracy is known only if true solution (root xt) is known

εr =

∣∣∣∣xt − xm,i

xt

∣∣∣∣ 100


Regula Falsi,Secant Methods

� Bisection makes no use of shape of function to determine root.

� Use a straightline approximation to f (x)(in contrast to the midpoint)to find the approximate root location.

� Start with 2 points, (a, f (a)), (b, f (b)) that satisfies (f (a).f (b) < 0.


Regula Falsi Method

� Next approximation: where the straightline approximation crossesthe x-axis:

f (b)

(b− s)=

f (b)− f (a)

(b− a)

s = b− b− af (b)− f (a)

f (b)


Regular Falsi: Algorithm


Regular Falsi: Example(Cube Root of 2)


Secant Method

� Similar to Regula Falsi method, with a slight modification.

� New estimate of xi+1 of the root, based on f (xi) and f (xi−1)

� Especially useful when its difficult to obtain derivatives analytically.

x2 = x1 −x1 − x0

y1 − y0y1

and

xi+1 = xi −xi − xi−1

yi − yi−1yi


Secant Method (contd)

f (xi−1)

xi+1 − xi−1=

f (xi)

xi+1 − xi

Solving for xi+1 gives

xi+1 = xi −f (xi)[xi−1 − xi]

f (xi−1 − f (xi)

= xi −f (xi)

f(xi−1−f(xi)xi−1−xi

= xi −f (xi)

f ′(xi)

which is similar in form to the Newton’s method.

⇒ Secant method doesnt require testing for interval selection

⇒ Generally converges faster, but not guaranteed.ITCS 4133/5133: Introduction to Numerical Methods 16 Roots of Equations

Secant Method:Algorithm


Secant Method:Example(Square Root of 2)


Newton-Raphson Method

� Motivation: Rate of convergence of bisection method is slow.

� Newton’s iteration method uses the linear portion of the Taylor series

f (x1) = f (x0) +df

dx∆x

where ∆x = x1 − x0,dfdx

is the derivative w.r.t x.

f (x1) = 0 = f (x0) +df

dx(x1 − x0)

when x1 is the root of the function.

x1 = x0 −f (x0)

df/dx= x0 −

f (x0)

f ′(x0)(derivative f ′ evaluated at x0)


Newton-Raphson Method (contd)

Iteratively,

xi+1 = xi −f (xi)

f ′(xi)

Accuracy

� If f (x) is linear, first iteration produces exact solution

� If f (x) is non-linear, accuracy depends on importance of the trun-cated non-linear terms of the Taylor series

Graphical Interpretation

Approximate f ′(xi) over [xi, xi+1] as

f ′(xi) =f (xi − 0)

xi+1 − xi

from which the iteration is derived.ITCS 4133/5133: Introduction to Numerical Methods 20 Roots of Equations

Newton-Raphson Method: Algorithm


Newton-Raphson Method: Example


Newton-Raphson Method:Non-Convergence

� f ′(x) approaches zero, exception (division by zero).

� Solution oscillates between two different solutions

f (xi)/f′(xi) = −f (xi+1/f

′(xi+1)


Solving Non-Linear Equations

� Addresses root finding to two or more variables

� Approach: Solve for the variables and a Jacobi style iteration tech-nique

Example

x3 − 3x2 + xy = 0

4x2 − 4xy2 + 3y2 = 0

Solve for x and y,

x = (3x2 − xy)1/3

y =4x2 + 3y2

4x

1/2

Use initial estimates for x and y to begin the iteration using most recentvalues of x and y.ITCS 4133/5133: Introduction to Numerical Methods 24 Roots of Equations

Muller’s Method

� Extension of the Secant Method; uses a quadratic approximation.

� Starts with 3 initial approximations x0, x1, x2 and consider the nextapproximation x4 as the intersection of the parabola connecting(x0, f (x0)), (x1, f (x1)), (x2, f (x2)) with the x-axis

� In general, less sensitive to starting values than Newton’s method.


Muller’s Method(contd)

Start with

P (x) = a(x− x2)2

+ b(x− x2) + c

As the quadratic passes through (x0, f (x0)) , (x1, f (x1) and (x2, f (x2)

f (x0) = a(x0 − x2)2

+ b(x0 − x2) + c

f (x1) = a(x1 − x2)2

+ b(x1 − x2) + c

f (x2) = a.02 + b.0 + c = c

We can solve for a, b, c

a =(x1 − x2)[f (x0)− f (x2)]− (x0 − x2)[f (x1)− f (x2)]

(x0 − x2)(x1 − x2)(x0 − x1)

b =(x0 − x2)

2[f (x1)− f (x2)]− (x1 − x2)

2[f (x0)− f (x2)]

(x0 − x2)(x1 − x2)(x0 − x1)c = f (x2)ITCS 4133/5133: Introduction to Numerical Methods 26 Roots of Equations

Muller’s Method (contd.)

We can solve the quadratic P (x) for the next estimate, x3.

P (x) = P (x3 − x2) = a(x3 − x2)2

+ b(x3 − x2) + c = 0

resulting in

x3 − x2 =−b±

√b2 − 4ac

2a

The above form can cause roundoff errors, when b2 ≫ 4ac. Insteaduse

x3 − x2 =−2c

b±√b2 − 4ac

Muller’s method chooses the root, that agrees with the sign of b to makethe denominator large, and hence closest root to x2.


Eigenvalue Analysis

Eigen values, denoted by λ, are values for the matrix system

[A− λI]X = 0

having a non-zero solution vector X.

⇒ A and I are n× n⇒ I is the identity matrix.

⇒ Applications: Analysis of sediment deposits in river beds, analyzing flutterin airplane wings.

Solve

|A− λI| = 0

resulting in an nth order polynomial, to be solved for the λs.


Eigenvalue Analysis(contd.)

For a 3× 3 system,

|A− λI| =

∣∣∣∣∣∣a11 − λ a12 a13a21 a22 − λ a23a31 a32 a33 − λ

∣∣∣∣∣∣ = 0

resulting in

λ3 + b2λ2 + b1λ + b0 = 0


Analyis: Root Finding Methods

� Need to understand errors

� Need to understancd convergence rates

Convergence Rate

� If |en+1| approaches K ∗ |en|p as =⇒∞, then the method is of order p

� Typically convergence rates are valid only close to the root.

� Can study convergence rates experimentally or theoretically.


Convergence Rate: Fixed Point Iteration

xn+1 = g(xn)

If x = r is a solution of f (x) = 0, then f (r) = 0, r = g(r). Thus,

xn+1 − r = g(xn)− g(r) =g(xn)− g(r)

(xn − r)(xn − r)

We can use the mean value theorm (assuming g(x), g′(x) are continuous)

xn+1 − r = g′(ξn) ∗ (xn − r)

where ξn ∈ (xn, r). Similarly, the error can be defined as

ei+1 = g′(ξn) ∗ ei


Convergence Rate: Fixed Point Iteration

|ei+1| = |g′(ξn)| ∗ |ei|

Suppose |g′(x)| < K < 1 for all x ∈ (r − h, r + h), then if x0 is in thisinterval, then the iterates converge, as

|en+1| < K ∗ |en| < K2 ∗ |en−1| < K3|en−2| < .... < Kn+1 ∗ |e0|

which implies fixed point iteration is of order 1.


Convergence Rate: Newton’s Method

xn+1 = xn −f (xn)

f ′(xn)= g(xn)

◦ Has the same form as the fixed point iteration.

◦ Thus, successive iterates converge if |g′(x)| < 1

Assume a simple root at x = r,

g′(x) = 1− f ′(x) ∗ f ′(x)− f (x) ∗ f ′′(x)

[f ′(x)]2 =

f (x) ∗ f ′′(x)

[f ′(x)]2

Thus, if

f (x) ∗ f ′′(x)

[f ′(x)]2 < 1

in an interval around the root r, the method will converge for any x0.



If r is a root of f (x) = 0, then r = g(r), xn+1 = g(xn), so

xn+1 − r = g(xn)− g(r)

Expanding g(xn) using Taylor series, expanding about (xn − r),

g(xn) = g(r) + g′(r) ∗ (xn − r) +g′′(ξ)

2(xn − r)2

with ξ ∈ (xn, r)

g′(r) =f (r) ∗ f ′′(r)

[f ′(r)]2 = 0

thus

g(xn) = g(r) +g′′(ξ)

2(xn − r)2



g(xn) = g(r) +g′′(ξ)

2(xn − r)2

If xn − r = en, we have

en+1 = xn+1 − r = g(xn)− g(r) =g′′(ξ)

2en

2

Thus, each error in the limit is proportional to the second power of the previouserror, resulting convergence of order 2, or quadratic convergence.


Convergence Rate: Bisection Method

� Method based on intermediate value theorem: root exists in the interval[a, b], if f (a) and f (b) are of opposite sign.

� Error at stage k is at most (b− a)/2k

� Error estimates(on the average) are reduced by half at each iteration

|en+1| = 0.5 ∗ |en|

� Bisection is linearly convergent.


Analysis: Regula Falsi, Secant Methods

� Rate of Convergence: Must investigate

|xk − x∗||xk−1 − x∗|p

for large values of p.

� It can be shown that errors are reduced as follows:

en+1 =g′′(ξi, ξ2)

2(en)en−1

� Convergence rate can be shown to be about 1.62.


Roots of Equations - UNC Charlotte · Newton-Raphson Method Zeros of ... Solve for the variables and a Jacobi style iteration ... Introduction to Numerical Methods 27 Roots of Equations.

Documents