Top Banner
2.4 Error Analysis for Iterative Methods 1
9

2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfΒ Β· Newton’s Method as Fixed Point Problem Solve π‘₯= r by fixed point method. We write the problem as an equivalent

Aug 03, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfΒ Β· Newton’s Method as Fixed Point Problem Solve π‘₯= r by fixed point method. We write the problem as an equivalent

2.4 Error Analysis for Iterative Methods

1

Page 2: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfΒ Β· Newton’s Method as Fixed Point Problem Solve π‘₯= r by fixed point method. We write the problem as an equivalent

Definition β€’ Order of Convergence Suppose {𝑝𝑛}𝑛=0

∞ is a sequence that converges to 𝑝 with 𝑝𝑛 β‰  𝑝 for all 𝑛. If positive constants πœ† and 𝛼 exist with

limπ‘›β†’βˆž

|𝑝𝑛+1 βˆ’ 𝑝|

|𝑝𝑛 βˆ’ 𝑝|𝛼= πœ†

then {𝑝𝑛}𝑛=0∞ is said to converges to 𝒑 of order 𝜢 with asymptotic

error constant 𝝀. An iterative technique 𝑝𝑛 = 𝑔(π‘π‘›βˆ’1) is said to be of order 𝜢 if the sequence {𝑝𝑛}𝑛=0

∞ converges to the solution 𝑝 = 𝑔(𝑝) of order 𝜢. β€’ Special cases

1. If 𝛼 = 1 (and πœ† < 1), the sequence is linearly convergent 2. If 𝛼 = 2, the sequence is quadratically convergent 3. If 𝛼 < 1, the sequence is sub-linearly convergent (undesirable, very slow) 4. If 𝛼 = 1 and πœ† = 0 or 1 < 𝛼 < 2, the sequence is super-linearly convergent

β€’ Remark: High order (𝛼) ⟹ faster convergence (more desirable) πœ† is less important than the order (𝛼)

2

Page 3: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfΒ Β· Newton’s Method as Fixed Point Problem Solve π‘₯= r by fixed point method. We write the problem as an equivalent

Linear vs. Quadratic

3

Suppose we have two sequences converging to 0 with:

limπ‘›β†’βˆž

|𝑝𝑛+1|

|𝑝𝑛|= 0.9, lim

π‘›β†’βˆž

|π‘žπ‘›+1|

|π‘žπ‘›|2= 0.9

Roughly we have: 𝑝𝑛 β‰ˆ 0.9 π‘π‘›βˆ’1 β‰ˆ β‹― β‰ˆ 0.9𝑛 𝑝0 ,

π‘žπ‘› β‰ˆ 0.9|π‘žπ‘›βˆ’1|2 β‰ˆ β‹― β‰ˆ 0.92π‘›βˆ’1 π‘ž0 , Assume 𝑝0 = π‘ž0 = 1

Page 4: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfΒ Β· Newton’s Method as Fixed Point Problem Solve π‘₯= r by fixed point method. We write the problem as an equivalent

Fixed Point Convergence β€’ Theorem

Let 𝑔 ∈ 𝐢[π‘Ž, 𝑏] be such that 𝑔 π‘₯ ∈ [π‘Ž, 𝑏] for all π‘₯ ∈ π‘Ž, 𝑏 . Suppose 𝑔′ is continuous on (π‘Ž, 𝑏) and that 0 < π‘˜ < 1 exists with |𝑔′(π‘₯)| ≀ π‘˜ for all π‘₯ ∈ π‘Ž, 𝑏 .

If 𝑔′(𝑝) β‰  0, then for all number 𝑝0 in [π‘Ž, 𝑏], the sequence 𝑝𝑛 = 𝑔(π‘π‘›βˆ’1) converges only linearly to the unique fixed point 𝑝 in π‘Ž, 𝑏 .

β€’ Proof: 𝑝𝑛+1 βˆ’ 𝑝 = 𝑔 𝑝𝑛 βˆ’ 𝑔 𝑝 = 𝑔′ πœ‰π‘› 𝑝𝑛 βˆ’ 𝑝 , πœ‰π‘› ∈ (𝑝𝑛, 𝑝)

Since {𝑝𝑛}𝑛=0∞ converges to 𝑝, {πœ‰π‘›}𝑛=0

∞ converges to 𝑝.

Since 𝑔′ is continuous, limπ‘›β†’βˆž

𝑔′(πœ‰π‘›) = 𝑔′(𝑝)

limπ‘›β†’βˆž

|𝑝𝑛+1βˆ’π‘|

|π‘π‘›βˆ’π‘|= lim

π‘›β†’βˆžπ‘”β€² πœ‰π‘› = |𝑔′(𝑝)| ⟹ linear convergence

4

Page 5: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfΒ Β· Newton’s Method as Fixed Point Problem Solve π‘₯= r by fixed point method. We write the problem as an equivalent

Speed up Convergence of Fixed Point Iteration

β€’ If we look for faster convergence methods, we must have 𝑔′ 𝑝 = 0

β€’ Theorem Let 𝑝 be a solution of π‘₯ = 𝑔 π‘₯ . Suppose 𝑔′ 𝑝 = 0 and 𝑔′′ is continuous with 𝑔′′ π‘₯ < 𝑀 on an open interval 𝐼 containing 𝑝. Then there exists a 𝛿 > 0 such that for 𝑝0 ∈ 𝑝 βˆ’ 𝛿, 𝑝 + 𝛿 , the sequence defined by 𝑝𝑛+1 =𝑔 𝑝𝑛 , when 𝑛 β‰₯ 0, converges at least quadratically to 𝑝. For sufficiently large 𝑛

𝑝𝑛+1 βˆ’ 𝑝 <𝑀

2|𝑝𝑛 βˆ’ 𝑝|2

Remark:

Look for quadratically convergent fixed point methods which 𝑔 𝑝 = 𝑝 and 𝑔′ 𝑝 = 0.

5

Page 6: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfΒ Β· Newton’s Method as Fixed Point Problem Solve π‘₯= r by fixed point method. We write the problem as an equivalent

Newton’s Method as Fixed Point Problem

Solve 𝑓 π‘₯ = 0 by fixed point method. We write the problem as an equivalent fixed point problem:

𝑔 π‘₯ = π‘₯ βˆ’ 𝑓 π‘₯ solve: π‘₯ = 𝑔(π‘₯) 𝑔 π‘₯ = π‘₯ βˆ’ 𝛼𝑓 π‘₯ solve π‘₯ = 𝑔 π‘₯ , 𝛼 is a constant

𝑔 π‘₯ = π‘₯ βˆ’ πœ™ π‘₯ 𝑓 π‘₯ solve π‘₯ = 𝑔 π‘₯ , πœ™ π‘₯ is differentiable

Newton’s method is derived by the last form:

Find differentiable πœ™ π‘₯ with 𝑔′ 𝑝 = 0 when 𝑓 𝑝 = 0.

𝑔′ π‘₯ =𝑑

𝑑π‘₯π‘₯ βˆ’ πœ™ π‘₯ 𝑓 π‘₯ = 1 βˆ’ πœ™β€²π‘“ βˆ’ πœ™π‘“β€²

Use 𝑔′ 𝑝 = 0 when 𝑓 𝑝 = 0 𝑔′ 𝑝 = 1 βˆ’ πœ™β€² 𝑝 βˆ™ 0 βˆ’ πœ™ 𝑝 𝑓′ 𝑝 = 0

πœ™ 𝑝 = 1/𝑓′(𝑝)

This gives Newton’s method

𝑝𝑛+1 = 𝑔 𝑝𝑛 = 𝑝𝑛 βˆ’π‘“(𝑝𝑛)

𝑓′(𝑝𝑛)

6

Page 7: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfΒ Β· Newton’s Method as Fixed Point Problem Solve π‘₯= r by fixed point method. We write the problem as an equivalent

Multiple Roots

β€’ How to modify Newton’s method when 𝑓′ 𝑝 = 0. Here 𝑝 is the root of 𝑓 π‘₯ = 0.

β€’ Definition: Multiplicity of a Root A solution 𝑝 of 𝑓 π‘₯ = 0 is a zero of multiplicity π‘š of 𝑓 if for π‘₯ β‰  𝑝, we can write 𝑓 π‘₯ = π‘₯ βˆ’ 𝑝 π‘šπ‘ž π‘₯ , where limπ‘₯→𝑝 π‘ž(π‘₯) β‰  0.

β€’ Theorem 𝑓 ∈ 𝐢1[π‘Ž, 𝑏] has a simple zero at 𝑝 in (π‘Ž, 𝑏) if and only if 𝑓 𝑝 = 0, but 𝑓′(𝑝) β‰  0.

β€’ Theorem The function 𝑓 ∈ πΆπ‘š[π‘Ž, 𝑏] has a zero of multiplicity π‘š at point 𝑝 in (π‘Ž, 𝑏) if and only if 0 = 𝑓 𝑝 = 𝑓′ 𝑝 = 𝑓′′ 𝑝 = β‹― = 𝑓 π‘šβˆ’1 𝑝 , but 𝑓 π‘š (𝑝) β‰  0

7

Page 8: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfΒ Β· Newton’s Method as Fixed Point Problem Solve π‘₯= r by fixed point method. We write the problem as an equivalent

Newton’s Method for Zeroes of Higher Multiplicity (π‘š > 1)

Define the new function πœ‡ π‘₯ =𝑓(π‘₯)

𝑓′(π‘₯).

Write 𝑓 π‘₯ = π‘₯ βˆ’ 𝑝 π‘šπ‘ž(π‘₯), hence

πœ‡ π‘₯ =𝑓(π‘₯)

𝑓′(π‘₯)= π‘₯ βˆ’ 𝑝

π‘ž π‘₯

π‘šπ‘ž π‘₯ + π‘₯ βˆ’ 𝑝 π‘žβ€² π‘₯

Note that 𝑝 is a simple zero of πœ‡ π‘₯ .

β€’ Apply Newton’s method to πœ‡ π‘₯ to give:

π‘₯ = 𝑔 π‘₯ = π‘₯ βˆ’πœ‡ π‘₯

πœ‡β€² π‘₯

= π‘₯ βˆ’π‘“ π‘₯ 𝑓′ π‘₯

𝑓′ π‘₯ 2 βˆ’ 𝑓 π‘₯ 𝑓′′ π‘₯

β€’ Quadratic convergence: 𝑝𝑛+1 = 𝑝𝑛 βˆ’π‘“ 𝑝𝑛 𝑓′ 𝑝𝑛

𝑓′ 𝑝𝑛2βˆ’π‘“ 𝑝𝑛 𝑓′′ 𝑝𝑛

8

Page 9: 2.4 Error Analysis for Iterative Methodszxu2/acms40390F12/Lec-2.4.pdfΒ Β· Newton’s Method as Fixed Point Problem Solve π‘₯= r by fixed point method. We write the problem as an equivalent

Drawbacks:

β€’ Compute 𝑓′′(π‘₯) is expensive

β€’ Iteration formula is more complicated – more expensive to compute

β€’ Roundoff errors in denominator – both 𝑓′(π‘₯) and 𝑓(π‘₯) approach zero.

9