Extrapolation Models for Convergence Acceleration and Function ’ s Extension David Levin Tel-Aviv University MAIA Erice 2013.

Extrapolation Models for Convergence Acceleration and Function’s Extension

David LevinDavid Levin

Tel-Aviv UniversityTel-Aviv University

MAIA Erice 2013

Extrapolation – Given values at some domain,

estimate values outside the domain. Prediction, Forecasting, Extension, Continuation

Extrapolation to the limit: Infinite series, Infinite Integrals

Convergence Acceleration

Models

Extension of Univariate and Bivariate Functions

Given N terms of an infinite series

can we estimate the infinite sum?

We must assume that the unknown terms can be determined by the given terms, i.e., we must assume the existence of a model, a prediction model!

A general model :

For evaluating using values ,

a natural model would be a differential equation

or a difference equation.

1 2 3{ , , ,..., }Na a a a

1n

n

a

1 2 1[ , ,..., ] 0n n n mM a a a

0

( )f x dx

( ), [0, ]f x x N

( )[ ( ), '( ),..., ( )] 0mM f x f x f x

1.5 20

1

( )sin(2 )n

n J nx nx

Q: What is a good model?A: A model which covers a large class of series, and can

be used for extrapolation.

Q: What about linear models? With constant coefficients?A: Exact for rational functions!

Leading to Pade Approximations.

Q: What about linear models with varying coefficients?A: They cover a very large class of series in applied math.

Q: How to use linear models for convergence acceleration?Q: How to use linear models for function’s extension?

linear model with constant coefficients

Given N terms of an infinite series let us assume that the unknown terms can be predicted by a linear model with constant coefficients:

The coefficients of the model can be found by fitting this model to the given terms

Assuming such a model is equivalent to assuming that the terms of the series, as function of their index, are sums of exponentials (including polynomials).

1 2 3{ , , ,..., }Na a a a

11

(1)m

n m i n ii

a p a

{ }ip

11

, 2 ,..., 1m

n m i n ii

a p a n N m N m

Using the model for approximating

Apply to the model we obtain

If we know we can find out S .

The resulting approximation to S is the same as Wynn’s algorithm, and it also gives the Pade approximant in the case of power series.

Pade Approximation is very good for series whose terms approximately satisfy a linear model with constant coefficients – i.e., sum of exponentials.

What about other series, e.g ,.

Which model is appropriate here ?

11

m

n m i n ii

a p a

1

nn

S a

n k

11 1

( ) ,m k

k m i k i k ji j

S S p S S where S a

{ }ip

1.5 20

1

( )sin(2 )n

n J nx nx

A model with varying coefficients:

Definition:

Where have asymptotic expansions

Examples:

( ){ } mna B

11

( ) (2)m

n m i n ii

a p n a

( ){ } ,ik

i ip A k Z

,0

( ) ~ , (3)ik ji i j

j

p n n n as n

1.5 2

0 ( )sin(2 )na n J nx nx

1.5 (1) (2) (2)0{ } B , { ( )} B , {sin(2 )} B ,n J nx nx

2 (3) (6)0{ ( )} B { } BnJ nx a

( 1)( )( ) ( ) ( ) ( ) 2 2{ } , { } { } , { } , { }m m

m k m k mkn n n n n n na b a b a b a

B B B B B

A model with varying coefficients (cont.)

Assuming , i.e., with

leads to the transformation of series (Levin-Sidi 1980).

Application of the transformation does not require knowledge of

It involves solving a linear system for the approximation to the infinite sum S.

( ){ } mna B 1

1

( )m

n m i n ii

a p n a

( ){ } ikip A

( )md ( )md { ( )}ip n

It follows that

Truncating the asymptotic expansions of we form a system of linear equations for S .

For details and analysis see: Practical Extrapolation Methods, by A. Sidi

1( )

0 1

( ), ,m k

i m in n i k j i

i j

S S a q n where S a q A

{ ( )}iq n

Linear Models for Double series:

Here, even if we have the model, we cannot compute the terms of the series based upon a finite number of terms.

This is due to the ODE – PDE difference:An ODE + initial conditions define the solution

But a PDE requires boundary conditions to determine the solution in a domain!

Yet, such models are related to multivariate Pade approximations, and toother rational approximations

,, 1

m nm n

S a

, , , ,1 1

, 1 (4)k

n k m i j n i m j kj i

a p a p

Extension of functionsGiven function values in a domain D, we would like to extend it so that the extension continues the behavioral trends of the function. As in convergence acceleration, we assume a model, and use it for the extension:

Ideally, we would like to find ‘THE’ differential equation which the function fulfills on D, and then use it to extend the function outside D.

Since we assume discrete data on D, maybe noisy, we shall look instead for a difference equation which ‘best’ describes the behavior of the function on D.

We shall discuss different models, and how to use them for extension.

Linear Model – Constant Coef. – Univariate case Given function values in [a,b]:

Assume f is bandlimited, and let

By Nyquist–Shannon sampling theorem sampling distance d is sufficient for

reconstructing f. We use sequences of ‘mesh size ’

To these sequences we fit a Linear Constant Coefficients Model of order m

using least-squares minimization to find the model coefficients:

NabhNiihaxfx iNiii /)(,,...,0,,},{ 0

],[)( BBfspectrum B

nhd2

1

]1

[

1)1( }{, n

N

jnjifnhd

m

knkmiki fpf

1)1(

min2

1)1(

m

knkmiki

N

mni

fpf

All sequences satisfying are of the form (*)

where are roots of (if the roots are simple)

The extension algorithm:

1 .Find the model coefficients by least-squares minimization

2 .Find the roots

3 .Define the approximation on [a,b] and its extension by fitting

This is nothing but Prony’s method (1795)

Q: Is it applicable to varying coefficients models? or to the 2D case?

m

knkmiki fpf

1)1(

j

m

j

ijni cf

1

)(

}{ j1

1

)(

km

kk

m pp

}{ j

)1....()(1

dgolwcxg xj

m

jj

Example 1 – Fitting Exponentials 1

1)2sin(2)cos(8.)(

x

xxxf x

}652.0542.0,908.0414.0,864.0,273.0{}{;1;6 jdm

To enable varying coefficients models, and for 2D applications, we suggest an algorithm which does not require solving the difference equation :

Denote a sequence satisfying a model

We look for which approximates the given data, and is smooth

The algorithm: Find which minimizes the functional

Q: How to choose the parameter ?

Mgg i }{ M

Mg

Mgg i }{

2

0

2 ||||)( i

N

iii

pp gfggF

Example 2 – Fitting Smooth Approximation-Extension

1

1)2sin(2)cos(8.)(

x

xxxf x

0001.0;2;1;6 pdm

Example 3 – Approximation-Extension using varying coefficients

)sin(1

)2cos(5)( 5.1

2xx

x

xxf

0001.0;2;1;6 pdm

1

1)(,))(())(1(

6

1)1(7

xxufxuqpfxuq

knkmiikkii

Bivariate case: 2D Linear Models – Constant Coef.Given function values in [a,b]x[a,b]:

Assume f is bandlimited, and let

By Nyquist–Shannon sampling theorem sampling distance d is sufficient for

reconstructing f. We use sequences of ‘mesh size ’

To these sequences we fit a Linear Constant Coefficients Model M of order mxm

Note: This model includes bivariate exponentials, and much more.

Unlike the 1D case, the dimension of is not finite.

NabhNjijhbihayxfyx jiijji /)(,,...,0,),(),(},,,{ 2],[)( BBfspectrum

Bnhd

2

1

}{, )1(,)1( njnkifnhd

.1,0 ,1,

)1(,)1(

mm

m

knjnkik pfp

}{ Mg

Example 4 – 2D Approximation-Extension

We look for which approximates

the given data, and is smooth:

The optimization problem is heavy!

0000.15024.01743.01989.0

2967.04422.00797.02253.0

4032.03966.00796.03700.0

7606.00964.01662.00539.0

P

Mg

Observation: Let

And let

then

---------------------------------------------------------------------------------------------------------------------

E.g., Choosing as a cubic tensor product B-spline, we can approximate well the unknown function by

Using this observation we can now work with smooth functions satisfying linear models with constant coefficients, avoiding the high cost of the above optimization approach of finding minimizing

2

),(

),(,),(),(2

RyxjyixcyxgZjiij

2,

,,, ),(,0..,}{ ZjicpeiMc jki

klkji

2

,, ),(,0),(..,}{ RyxykxgpeiMg

klk

Mjispang )},({

2

0

2 ||||)( i

N

iii

pp gfggF

Mgg i }{

We can even define basis functions spanning

within a given domain.

Example of a basis function:

Mjispan )},({

Example 5 – 2D Approximation-Extension using spline basis

0000.13254.02081.01863.0

4786.00200.00840.01204.0

0669.01838.02556.00480.0

3483.01560.00772.01505.0

P

Mjispang )},({

Example 6 – Blending between models

From noisy cos(2x) into exp(-2x)

Thank you!