Top Banner
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion Nonlinear Systems CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Nonlinear Systems 1 / 27
44

Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

May 20, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinear Systems

CS 205A:Mathematical Methods for Robotics, Vision, and Graphics

Doug James (and Justin Solomon)

CS 205A: Mathematical Methods Nonlinear Systems 1 / 27

Page 2: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Part III: Nonlinear Problems

Not all numerical problemscan be solved with \ in

Matlab.

CS 205A: Mathematical Methods Nonlinear Systems 2 / 27

Page 3: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Question

Have we already seen anonlinear problem?

minimize ‖A~x‖2such that ‖~x‖2 = 1←− nonlinear!

CS 205A: Mathematical Methods Nonlinear Systems 3 / 27

Page 4: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Question

Have we already seen anonlinear problem?

minimize ‖A~x‖2such that ‖~x‖2 = 1←− nonlinear!

CS 205A: Mathematical Methods Nonlinear Systems 3 / 27

Page 5: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Root-Finding Problem

Given: f : Rn→ Rm

Find: ~x∗ with f (~x∗) = ~0

CS 205A: Mathematical Methods Nonlinear Systems 4 / 27

Page 6: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Root-Finding Applications

I Collision detection (graphics,astronomy)

I Graphics rendering (ray intersection)

I Robotics (kinematics)

I Optimization (line search)

CS 205A: Mathematical Methods Nonlinear Systems 5 / 27

Page 7: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Issue: Regularizing Assumptions

f (x) =

{−1 when x ≤ 01 when x > 0

g(x) =

{−1 when x ∈ Q1 when x 6∈ Q

CS 205A: Mathematical Methods Nonlinear Systems 6 / 27

Page 8: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Issue: Regularizing Assumptions

f (x) =

{−1 when x ≤ 01 when x > 0

g(x) =

{−1 when x ∈ Q1 when x 6∈ Q

CS 205A: Mathematical Methods Nonlinear Systems 6 / 27

Page 9: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Typical Regularizing AssumptionsContinuous

f(~x)→ f(~y) as ~x→ ~y

Lipschitz

‖f(~x)− f(~y)‖2 ≤ c‖~x− ~y‖2 for all ~x, ~y (same c)

Differentiable

Df(~x) exists for all ~x

Ck

k derivatives exist and are continuous

CS 205A: Mathematical Methods Nonlinear Systems 7 / 27

Page 10: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Typical Regularizing AssumptionsContinuous

f(~x)→ f(~y) as ~x→ ~y

Lipschitz

‖f(~x)− f(~y)‖2 ≤ c‖~x− ~y‖2 for all ~x, ~y (same c)

Differentiable

Df(~x) exists for all ~x

Ck

k derivatives exist and are continuous

CS 205A: Mathematical Methods Nonlinear Systems 7 / 27

Page 11: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Typical Regularizing AssumptionsContinuous

f(~x)→ f(~y) as ~x→ ~y

Lipschitz

‖f(~x)− f(~y)‖2 ≤ c‖~x− ~y‖2 for all ~x, ~y (same c)

Differentiable

Df(~x) exists for all ~x

Ck

k derivatives exist and are continuous

CS 205A: Mathematical Methods Nonlinear Systems 7 / 27

Page 12: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Typical Regularizing AssumptionsContinuous

f(~x)→ f(~y) as ~x→ ~y

Lipschitz

‖f(~x)− f(~y)‖2 ≤ c‖~x− ~y‖2 for all ~x, ~y (same c)

Differentiable

Df(~x) exists for all ~x

Ck

k derivatives exist and are continuous

CS 205A: Mathematical Methods Nonlinear Systems 7 / 27

Page 13: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Today

f : R→ RCS 205A: Mathematical Methods Nonlinear Systems 8 / 27

Page 14: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Property of Continuous Functions

Intermediate Value TheoremSuppose that f : [a, b]→ R is continuous and

that f (a) < u < f (b) or f (b) < u < f (a).

Then, there exists z ∈ (a, b) such that f (z) = u

CS 205A: Mathematical Methods Nonlinear Systems 9 / 27

Page 15: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Reasonable Input

I Continuous function f (x)

I `, r ∈ R withf (`) · f (r) < 0 (why?)

CS 205A: Mathematical Methods Nonlinear Systems 10 / 27

Page 16: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Bisection Algorithm

1. Compute c = `+r/2.

2. If f (c) = 0, return x∗ = c.

3. If f (`) · f (c) < 0, take r ← c. Otherwise

take `← c.

4. Return to step 1 until |r − `| < ε; then

return c.

CS 205A: Mathematical Methods Nonlinear Systems 11 / 27

Page 17: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Bisection: Illustration

`

r

cx

f(x)

x∗

f(x) > 0

f(x) < 0

CS 205A: Mathematical Methods Nonlinear Systems 12 / 27

Page 18: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Two Important Questions

1. Does it converge?

Yes! Unconditionally.

2. How quickly?

CS 205A: Mathematical Methods Nonlinear Systems 13 / 27

Page 19: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Two Important Questions

1. Does it converge?Yes! Unconditionally.

2. How quickly?

CS 205A: Mathematical Methods Nonlinear Systems 13 / 27

Page 20: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Two Important Questions

1. Does it converge?Yes! Unconditionally.

2. How quickly?

CS 205A: Mathematical Methods Nonlinear Systems 13 / 27

Page 21: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Convergence Analysis

Examine Ek with|xk − x∗| < Ek.

CS 205A: Mathematical Methods Nonlinear Systems 14 / 27

Page 22: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Bisection: Linear Convergence

Ek+1 ≤ 12Ek

for Ek ≡ |rk − `k|

CS 205A: Mathematical Methods Nonlinear Systems 15 / 27

Page 23: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Fixed Points

g(x∗) = x∗

Question:Same as root-finding?

CS 205A: Mathematical Methods Nonlinear Systems 16 / 27

Page 24: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Fixed Points

g(x∗) = x∗

Question:Same as root-finding?

CS 205A: Mathematical Methods Nonlinear Systems 16 / 27

Page 25: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Simple Strategy

xk+1 = g(xk)

CS 205A: Mathematical Methods Nonlinear Systems 17 / 27

Page 26: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Convergence Criterion

Ek ≡ |xk − x∗|= |g(xk−1)− g(x∗)|

≤ c|xk−1 − x∗|if g is Lipschitz

= cEk−1

=⇒ Ek ≤ ckE0

→ 0 as k →∞ (c < 1)

CS 205A: Mathematical Methods Nonlinear Systems 18 / 27

Page 27: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Convergence Criterion

Ek ≡ |xk − x∗|= |g(xk−1)− g(x∗)|≤ c|xk−1 − x∗|

if g is Lipschitz

= cEk−1

=⇒ Ek ≤ ckE0

→ 0 as k →∞ (c < 1)

CS 205A: Mathematical Methods Nonlinear Systems 18 / 27

Page 28: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Convergence Criterion

Ek ≡ |xk − x∗|= |g(xk−1)− g(x∗)|≤ c|xk−1 − x∗|

if g is Lipschitz

= cEk−1

=⇒ Ek ≤ ckE0

→ 0 as k →∞ (c < 1)

CS 205A: Mathematical Methods Nonlinear Systems 18 / 27

Page 29: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Alternative Criterion

Lipschitz near x∗ with goodstarting point.

e.g. C1 with |g′(x∗)| < 1

CS 205A: Mathematical Methods Nonlinear Systems 19 / 27

Page 30: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Alternative Criterion

Lipschitz near x∗ with goodstarting point.

e.g. C1 with |g′(x∗)| < 1

CS 205A: Mathematical Methods Nonlinear Systems 19 / 27

Page 31: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Convergence Rate of Fixed Point

When it converges...Always linear (why?)

Often quadratic! (→ board)

CS 205A: Mathematical Methods Nonlinear Systems 20 / 27

Page 32: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Convergence Rate of Fixed Point

When it converges...Always linear (why?)

Often quadratic! (→ board)

CS 205A: Mathematical Methods Nonlinear Systems 20 / 27

Page 33: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Approach for Differentiable f (x)

x

f(x)

x0

x1

x2

CS 205A: Mathematical Methods Nonlinear Systems 21 / 27

Page 34: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Newton’s Method

xk+1 = xk −f (xk)

f ′(xk)

Fixed point iteration on

g(x) ≡ x− f (x)

f ′(x)

CS 205A: Mathematical Methods Nonlinear Systems 22 / 27

Page 35: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Newton’s Method

xk+1 = xk −f (xk)

f ′(xk)

Fixed point iteration on

g(x) ≡ x− f (x)

f ′(x)CS 205A: Mathematical Methods Nonlinear Systems 22 / 27

Page 36: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Convergence of Newton

Simple Root

A root x∗ with f ′(x∗) 6= 0.

Quadratic convergence in this case! (→ board)

CS 205A: Mathematical Methods Nonlinear Systems 23 / 27

Page 37: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Convergence of Newton

Simple Root

A root x∗ with f ′(x∗) 6= 0.

Quadratic convergence in this case! (→ board)

CS 205A: Mathematical Methods Nonlinear Systems 23 / 27

Page 38: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Issue

Differentiation is hard!

CS 205A: Mathematical Methods Nonlinear Systems 24 / 27

Page 39: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Secant Method

xk+1 = xk −f (xk)(xk − xk−1)

f (xk)− f (xk−1)

Trivia:Converges at rate 1+

√5

2 ≈ 1.6180339887 . . .

(“Golden Ratio”)

CS 205A: Mathematical Methods Nonlinear Systems 25 / 27

Page 40: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Secant Method

xk+1 = xk −f (xk)(xk − xk−1)

f (xk)− f (xk−1)

Trivia:Converges at rate 1+

√5

2 ≈ 1.6180339887 . . .

(“Golden Ratio”)

CS 205A: Mathematical Methods Nonlinear Systems 25 / 27

Page 41: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Secant Method

xk+1 = xk −f (xk)(xk − xk−1)

f (xk)− f (xk−1)

Trivia:Converges at rate 1+

√5

2 ≈ 1.6180339887 . . .

(“Golden Ratio”)

CS 205A: Mathematical Methods Nonlinear Systems 25 / 27

Page 42: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Hybrid Methods

Want: Convergence rate of secant/Newton with

convergence guarantees of bisection

e.g. Dekker’s Method: Take secant step if it is

in the bracket, bisection step otherwise

CS 205A: Mathematical Methods Nonlinear Systems 26 / 27

Page 43: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Hybrid Methods

Want: Convergence rate of secant/Newton with

convergence guarantees of bisection

e.g. Dekker’s Method: Take secant step if it is

in the bracket, bisection step otherwise

CS 205A: Mathematical Methods Nonlinear Systems 26 / 27

Page 44: Nonlinear Systems - Computer Graphicsgraphics.stanford.edu/.../nonlinear_systems.pdf · Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion

Single-Variable Conclusion

I Unlikely to solve exactly, so we settle for

iterative methodsI Must check that method converges at allI Convergence rates:

I Linear: Ek+1 ≤ CEk for some 0 ≤ C < 1I Superlinear: Ek+1 ≤ CEr

k for some r > 1I Quadratic: r = 2I Cubic: r = 3

I Time per iteration also important

Next

CS 205A: Mathematical Methods Nonlinear Systems 27 / 27