Top Banner
2.4 Error Analysis for Iterative Methods 1
9

2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfย ยท Newtonโ€™s Method as Fixed Point Problem Solve ๐‘ฅ= r by fixed point method. We write the problem as an equivalent

Aug 03, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfย ยท Newtonโ€™s Method as Fixed Point Problem Solve ๐‘ฅ= r by fixed point method. We write the problem as an equivalent

2.4 Error Analysis for Iterative Methods

1

Page 2: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfย ยท Newtonโ€™s Method as Fixed Point Problem Solve ๐‘ฅ= r by fixed point method. We write the problem as an equivalent

Definition โ€ข Order of Convergence Suppose {๐‘๐‘›}๐‘›=0

โˆž is a sequence that converges to ๐‘ with ๐‘๐‘› โ‰  ๐‘ for all ๐‘›. If positive constants ๐œ† and ๐›ผ exist with

lim๐‘›โ†’โˆž

|๐‘๐‘›+1 โˆ’ ๐‘|

|๐‘๐‘› โˆ’ ๐‘|๐›ผ= ๐œ†

then {๐‘๐‘›}๐‘›=0โˆž is said to converges to ๐’‘ of order ๐œถ with asymptotic

error constant ๐€. An iterative technique ๐‘๐‘› = ๐‘”(๐‘๐‘›โˆ’1) is said to be of order ๐œถ if the sequence {๐‘๐‘›}๐‘›=0

โˆž converges to the solution ๐‘ = ๐‘”(๐‘) of order ๐œถ. โ€ข Special cases

1. If ๐›ผ = 1 (and ๐œ† < 1), the sequence is linearly convergent 2. If ๐›ผ = 2, the sequence is quadratically convergent 3. If ๐›ผ < 1, the sequence is sub-linearly convergent (undesirable, very slow) 4. If ๐›ผ = 1 and ๐œ† = 0 or 1 < ๐›ผ < 2, the sequence is super-linearly convergent

โ€ข Remark: High order (๐›ผ) โŸน faster convergence (more desirable) ๐œ† is less important than the order (๐›ผ)

2

Page 3: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfย ยท Newtonโ€™s Method as Fixed Point Problem Solve ๐‘ฅ= r by fixed point method. We write the problem as an equivalent

Linear vs. Quadratic

3

Suppose we have two sequences converging to 0 with:

lim๐‘›โ†’โˆž

|๐‘๐‘›+1|

|๐‘๐‘›|= 0.9, lim

๐‘›โ†’โˆž

|๐‘ž๐‘›+1|

|๐‘ž๐‘›|2= 0.9

Roughly we have: ๐‘๐‘› โ‰ˆ 0.9 ๐‘๐‘›โˆ’1 โ‰ˆ โ‹ฏ โ‰ˆ 0.9๐‘› ๐‘0 ,

๐‘ž๐‘› โ‰ˆ 0.9|๐‘ž๐‘›โˆ’1|2 โ‰ˆ โ‹ฏ โ‰ˆ 0.92๐‘›โˆ’1 ๐‘ž0 , Assume ๐‘0 = ๐‘ž0 = 1

Page 4: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfย ยท Newtonโ€™s Method as Fixed Point Problem Solve ๐‘ฅ= r by fixed point method. We write the problem as an equivalent

Fixed Point Convergence โ€ข Theorem

Let ๐‘” โˆˆ ๐ถ[๐‘Ž, ๐‘] be such that ๐‘” ๐‘ฅ โˆˆ [๐‘Ž, ๐‘] for all ๐‘ฅ โˆˆ ๐‘Ž, ๐‘ . Suppose ๐‘”โ€ฒ is continuous on (๐‘Ž, ๐‘) and that 0 < ๐‘˜ < 1 exists with |๐‘”โ€ฒ(๐‘ฅ)| โ‰ค ๐‘˜ for all ๐‘ฅ โˆˆ ๐‘Ž, ๐‘ .

If ๐‘”โ€ฒ(๐‘) โ‰  0, then for all number ๐‘0 in [๐‘Ž, ๐‘], the sequence ๐‘๐‘› = ๐‘”(๐‘๐‘›โˆ’1) converges only linearly to the unique fixed point ๐‘ in ๐‘Ž, ๐‘ .

โ€ข Proof: ๐‘๐‘›+1 โˆ’ ๐‘ = ๐‘” ๐‘๐‘› โˆ’ ๐‘” ๐‘ = ๐‘”โ€ฒ ๐œ‰๐‘› ๐‘๐‘› โˆ’ ๐‘ , ๐œ‰๐‘› โˆˆ (๐‘๐‘›, ๐‘)

Since {๐‘๐‘›}๐‘›=0โˆž converges to ๐‘, {๐œ‰๐‘›}๐‘›=0

โˆž converges to ๐‘.

Since ๐‘”โ€ฒ is continuous, lim๐‘›โ†’โˆž

๐‘”โ€ฒ(๐œ‰๐‘›) = ๐‘”โ€ฒ(๐‘)

lim๐‘›โ†’โˆž

|๐‘๐‘›+1โˆ’๐‘|

|๐‘๐‘›โˆ’๐‘|= lim

๐‘›โ†’โˆž๐‘”โ€ฒ ๐œ‰๐‘› = |๐‘”โ€ฒ(๐‘)| โŸน linear convergence

4

Page 5: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfย ยท Newtonโ€™s Method as Fixed Point Problem Solve ๐‘ฅ= r by fixed point method. We write the problem as an equivalent

Speed up Convergence of Fixed Point Iteration

โ€ข If we look for faster convergence methods, we must have ๐‘”โ€ฒ ๐‘ = 0

โ€ข Theorem Let ๐‘ be a solution of ๐‘ฅ = ๐‘” ๐‘ฅ . Suppose ๐‘”โ€ฒ ๐‘ = 0 and ๐‘”โ€ฒโ€ฒ is continuous with ๐‘”โ€ฒโ€ฒ ๐‘ฅ < ๐‘€ on an open interval ๐ผ containing ๐‘. Then there exists a ๐›ฟ > 0 such that for ๐‘0 โˆˆ ๐‘ โˆ’ ๐›ฟ, ๐‘ + ๐›ฟ , the sequence defined by ๐‘๐‘›+1 =๐‘” ๐‘๐‘› , when ๐‘› โ‰ฅ 0, converges at least quadratically to ๐‘. For sufficiently large ๐‘›

๐‘๐‘›+1 โˆ’ ๐‘ <๐‘€

2|๐‘๐‘› โˆ’ ๐‘|2

Remark:

Look for quadratically convergent fixed point methods which ๐‘” ๐‘ = ๐‘ and ๐‘”โ€ฒ ๐‘ = 0.

5

Page 6: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfย ยท Newtonโ€™s Method as Fixed Point Problem Solve ๐‘ฅ= r by fixed point method. We write the problem as an equivalent

Newtonโ€™s Method as Fixed Point Problem

Solve ๐‘“ ๐‘ฅ = 0 by fixed point method. We write the problem as an equivalent fixed point problem:

๐‘” ๐‘ฅ = ๐‘ฅ โˆ’ ๐‘“ ๐‘ฅ solve: ๐‘ฅ = ๐‘”(๐‘ฅ) ๐‘” ๐‘ฅ = ๐‘ฅ โˆ’ ๐›ผ๐‘“ ๐‘ฅ solve ๐‘ฅ = ๐‘” ๐‘ฅ , ๐›ผ is a constant

๐‘” ๐‘ฅ = ๐‘ฅ โˆ’ ๐œ™ ๐‘ฅ ๐‘“ ๐‘ฅ solve ๐‘ฅ = ๐‘” ๐‘ฅ , ๐œ™ ๐‘ฅ is differentiable

Newtonโ€™s method is derived by the last form:

Find differentiable ๐œ™ ๐‘ฅ with ๐‘”โ€ฒ ๐‘ = 0 when ๐‘“ ๐‘ = 0.

๐‘”โ€ฒ ๐‘ฅ =๐‘‘

๐‘‘๐‘ฅ๐‘ฅ โˆ’ ๐œ™ ๐‘ฅ ๐‘“ ๐‘ฅ = 1 โˆ’ ๐œ™โ€ฒ๐‘“ โˆ’ ๐œ™๐‘“โ€ฒ

Use ๐‘”โ€ฒ ๐‘ = 0 when ๐‘“ ๐‘ = 0 ๐‘”โ€ฒ ๐‘ = 1 โˆ’ ๐œ™โ€ฒ ๐‘ โˆ™ 0 โˆ’ ๐œ™ ๐‘ ๐‘“โ€ฒ ๐‘ = 0

๐œ™ ๐‘ = 1/๐‘“โ€ฒ(๐‘)

This gives Newtonโ€™s method

๐‘๐‘›+1 = ๐‘” ๐‘๐‘› = ๐‘๐‘› โˆ’๐‘“(๐‘๐‘›)

๐‘“โ€ฒ(๐‘๐‘›)

6

Page 7: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfย ยท Newtonโ€™s Method as Fixed Point Problem Solve ๐‘ฅ= r by fixed point method. We write the problem as an equivalent

Multiple Roots

โ€ข How to modify Newtonโ€™s method when ๐‘“โ€ฒ ๐‘ = 0. Here ๐‘ is the root of ๐‘“ ๐‘ฅ = 0.

โ€ข Definition: Multiplicity of a Root A solution ๐‘ of ๐‘“ ๐‘ฅ = 0 is a zero of multiplicity ๐‘š of ๐‘“ if for ๐‘ฅ โ‰  ๐‘, we can write ๐‘“ ๐‘ฅ = ๐‘ฅ โˆ’ ๐‘ ๐‘š๐‘ž ๐‘ฅ , where lim๐‘ฅโ†’๐‘ ๐‘ž(๐‘ฅ) โ‰  0.

โ€ข Theorem ๐‘“ โˆˆ ๐ถ1[๐‘Ž, ๐‘] has a simple zero at ๐‘ in (๐‘Ž, ๐‘) if and only if ๐‘“ ๐‘ = 0, but ๐‘“โ€ฒ(๐‘) โ‰  0.

โ€ข Theorem The function ๐‘“ โˆˆ ๐ถ๐‘š[๐‘Ž, ๐‘] has a zero of multiplicity ๐‘š at point ๐‘ in (๐‘Ž, ๐‘) if and only if 0 = ๐‘“ ๐‘ = ๐‘“โ€ฒ ๐‘ = ๐‘“โ€ฒโ€ฒ ๐‘ = โ‹ฏ = ๐‘“ ๐‘šโˆ’1 ๐‘ , but ๐‘“ ๐‘š (๐‘) โ‰  0

7

Page 8: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfย ยท Newtonโ€™s Method as Fixed Point Problem Solve ๐‘ฅ= r by fixed point method. We write the problem as an equivalent

Newtonโ€™s Method for Zeroes of Higher Multiplicity (๐‘š > 1)

Define the new function ๐œ‡ ๐‘ฅ =๐‘“(๐‘ฅ)

๐‘“โ€ฒ(๐‘ฅ).

Write ๐‘“ ๐‘ฅ = ๐‘ฅ โˆ’ ๐‘ ๐‘š๐‘ž(๐‘ฅ), hence

๐œ‡ ๐‘ฅ =๐‘“(๐‘ฅ)

๐‘“โ€ฒ(๐‘ฅ)= ๐‘ฅ โˆ’ ๐‘

๐‘ž ๐‘ฅ

๐‘š๐‘ž ๐‘ฅ + ๐‘ฅ โˆ’ ๐‘ ๐‘žโ€ฒ ๐‘ฅ

Note that ๐‘ is a simple zero of ๐œ‡ ๐‘ฅ .

โ€ข Apply Newtonโ€™s method to ๐œ‡ ๐‘ฅ to give:

๐‘ฅ = ๐‘” ๐‘ฅ = ๐‘ฅ โˆ’๐œ‡ ๐‘ฅ

๐œ‡โ€ฒ ๐‘ฅ

= ๐‘ฅ โˆ’๐‘“ ๐‘ฅ ๐‘“โ€ฒ ๐‘ฅ

๐‘“โ€ฒ ๐‘ฅ 2 โˆ’ ๐‘“ ๐‘ฅ ๐‘“โ€ฒโ€ฒ ๐‘ฅ

โ€ข Quadratic convergence: ๐‘๐‘›+1 = ๐‘๐‘› โˆ’๐‘“ ๐‘๐‘› ๐‘“โ€ฒ ๐‘๐‘›

๐‘“โ€ฒ ๐‘๐‘›2โˆ’๐‘“ ๐‘๐‘› ๐‘“โ€ฒโ€ฒ ๐‘๐‘›

8

Page 9: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfย ยท Newtonโ€™s Method as Fixed Point Problem Solve ๐‘ฅ= r by fixed point method. We write the problem as an equivalent

Drawbacks:

โ€ข Compute ๐‘“โ€ฒโ€ฒ(๐‘ฅ) is expensive

โ€ข Iteration formula is more complicated โ€“ more expensive to compute

โ€ข Roundoff errors in denominator โ€“ both ๐‘“โ€ฒ(๐‘ฅ) and ๐‘“(๐‘ฅ) approach zero.

9