Explorations into Spectral Methods for Stochastic Di ... · Our goal, therefore, is to decompose a function ˙as for our heat equation into Hermite polynomials for a xed parameter

Explorations into Spectral Methods for Stochastic

Differential Equations

Joachim Worthington

June 20, 2012

1 Introduction

The goal of this paper is study the viability of constructing solutions to stochastic dif-ferential equations using ultraspherical methods. Although we will focus on the heatequation, it is hoped that the methodology followed in this paper can be extended tomore general problems. We will use the Golub-Welsch algorithm to develop quadraturewith Hermite polynomials to carry out our approximations. After recasting the problemin coefficient space, we can employ Monte Carlo methods in (hopefully) more efficientways.

2 The Heat Equation and Stochastic Calculus

The standard heat equation is given by

∂u

∂t=

1

2∇2u (1)

We will simplify this to the one-dimensional heat equation, so ∇2u = ∂2u∂x2

. For anybounded continuous function σ(x),

u(x, t) =1√2πt

∫ ∞−∞

σ(y)e−(x−y)2

2t dy (2)

is a solution to the heat equation.We define the (one-dimensional) Wiener process as a stochastic process satisfying

1. W0 = 0

2. Increments Wt −Ws are independent of Ws (the Markov property)

3. Wt −Ws ∼ N (0, t− s)

1

We can further define W xt = Wt + x, the Wiener process starting at x. Then, by

applying Ito integration, we can derive that

u(x, t) = E[σ(W xt )] (3)

solves the heat equation. We can recast this in terms of stochastic differentials, so

du = σ(Wt)dWt (4)

and it is in this form that we shall work (see [1] and [2] for further information and amore thorough derivation, respectively). This motivates our exploration of a fast methodfor solving stochastic differential equations (or SDE) in the form of equation 4, althoughit is hoped that the techniques used can be applied to more general SDEs.

3 Hermite Polynomials and Stochastic Calculus

We now consider the Hermite Polynomials, which we define as

Hn(x; c) = (−c)nex2/2cDne−x2/2c (5)

These polynomials are orthogonal with respect to the weight function e−x2/2c, and satisfythe recursion relationship

Hn+1(x; c) = xHn(x; c)− cnHn−1(x; c) (6)

(as demonstrated in the exam for this course, with minor modifications). The derivativesof these are given by

d

dxHn(x; c) = nHn−1(x; c) (7)

Using these, we can write MATLAB functions to recursively generate values of HermitePolynomials (see H.m and HDash.m in the appendix)

What is most useful to us, though, is the identity that∫Hn(W (u);u)dWt =

Hn+1(W (t); t)

n+ 1(8)

In this sense, the Hermite polynomials in stochastic calculus are analogous polynomialsof the form pn(x) = xn in deterministic calculus, where∫

pn(x)dx =pn+1(x)

n+ 1(9)

Our plan is to decompose given function into Hermite polynomials, and then use thisidentity. (For proofs of the identities in this section, see [3]).

2

4 Spectral Methods with Hermite Polynomials

Our goal, therefore, is to decompose a function σ as for our heat equation into Hermitepolynomials for a fixed parameter

σ(x) =

∞∑n=0

αcnHn(x; c) (10)

for some coefficients αcn. We call the transformation of σ into coefficient space H, sothat

H[σ] =

αc0αc1αc2...

(11)

To be able to calculate such a transformation, we need first that Hn(x; c) form anorthogonal basis of the space we are concerned with.

Theorem 4.1. Let v be the Gaussian measure with 0 mean and variance c. ThenHn(x; c) form an orthogonal basis of L2(v). Furthermore for the inner product on L2(v)we have

< Hn(x; c), Hn(x; c) >c =∫∞−∞Hn(x; c)Hn(x; c)dv(x)

=∫∞−∞Hn(x; c)Hn(x; c)e−x2/2cdx

= n!cn

The orthogonality follows from the work in the exam for this course. The otherresults are proved in [3].

Thus, for fixed c, we can calculate H[σ] by the formula

αcm =1

m!cm

∫ ∞−∞

σ(x)Hm(x; c)e−x2/2cdx (12)

To implement this numerically, though, we need to calculate the integrals. To do this,we will use the Golub-Welsch algorithm to develop a quadrature rule for our Hermitepolynomials with a parameter.

5 The Golub-Welsch algorithm

Recall that we have the following recurrence relationship

Hn+1(x; c) = xHn(x; c)− cnHn−1(x; c) (13)

We fix c. Then we have a three-term recurrence relationship, with An = 1, Bn = 0and Cn = cn. We can thus use the Golub-Welsch algorithm. (The following follows theprocedure in the course notes)

3

As An = 1, these polynomials are already monic. So our next step is to convert toorthonormal polynomials, which we shall denote h:

hn(x; c) =Hn(x; c)

||Hn(x; c)||(14)

This gives the following recurrence for h:

an+1hn+1(x; c) = xhn(x; c)− anhn−1(x; c) (15)

where an =√Cn =

√cn.

We thus construct the n× n Jacobi matrix:

Jn =

0√c 0√

c 0√

2c

0√

2c 0√

3c. . .

. . .. . .√

(n− 2)c 0√

(n− 1)c

0√

(n− 1)c 0

=√c

0 1 0

1 0√

2

0√

2 0√

3. . .

. . .. . .√

n− 2 0√n− 1

0√n− 1 0

(16)

The eigenvalues of this matrix are roots of hn, and thus also of Hn.We construct MATLAB codes to generate the Jacobi matrix (Jacobi.m) and the

eigenvalues (HermitePoints.m), given values for n and c. These codes are included inthe Appendix. We can test that these functions perform as we expect quite simply:

1 EDU>> H(HermitePoints(4,6),4,6)2

3 ans =4

5 1.0e−10 *6

7 −0.1818989403545868 0.0159161572810269 0.002273736754432

10 0.00852651282912111 0.06593836587853712 −0.181898940354586

This confirms that the points found are indeed roots of the corresponding Hermitepolynomial, to a high degree of accuracy.

4

Our next step is to calculate the weights. To do so, we will need the first moment∫ ∞−∞

w(x)dx =

∫ ∞−∞

e−x2/2cdx =√

2cπ (17)

which follows easily by making the substitution y = x/√2c and noticing that the resulting

integral is just that of the error function which has integral√π (see Appendix B for a

proof of this).Now, if vj is an eigenvector of J corresponding to the eigenvalue xj , we can calculate

the corresponding weight as

wj =(eT1 vj)

2√

2cπ

vTj vj

(18)

We write new code to calculate these weights, as well as a code to calculate both atonce (HermiteWeights.m and HermiteQuadrature.m respectively).

Using these weights and points, we have developed the quadrature rule∫ ∞−∞

f(x)e−x2/2cdx ≈n∑k=1

wkf(xk) (19)

We implement the calculation of the n-point approximation to∫∞−∞ f(x)e−x2/cdx in

HermiteIntegral.m.At this point, we note that the quadrature rules have a fairly simple relationship to

the parameter: if

w1

w2...

are the weights and

x1x2...

are the quadrature points for the

parameter value of 1, for some other c we have the weights and quadrature points

√c

w1

w2...wn

,√c

x1x2...xn

(20)

As such, if we need to calculate the quadrature rules for a variety of values of c, we cancalculate them once and construct the rest using this relationship. Figure 1 shows a fewexample quadrature rules: the quadrature points are plotted, at height correspondingto their weight.

We can test this quadrature rule, and the results of this are shown in Figure 2. Forthese examples we have set c = 0.5, to test the integral

∫∞−∞ f(x)e−x

2dx (this choice

was made so that exact integrals could be checked for comparison; for other values of cthe results are identical up to a transformation). For f(x) a polynomial, the quadraturerule performs very well, as we would expect. Our n-point method should be exact forpolynomials of degree up to 2n− 1 (although numerical errors do creep in).

5

−10 −5 0 5 10

0

0.5

1

1.5

2

2.5

c=2, n=5

Wei

ght

Quadrature Points−10 −5 0 5 10

0

0.5

1

1.5

2

2.5

c=2, n=10

Wei

ght

Quadrature Points

−10 −5 0 5 10

0

0.5

1

1.5

2

2.5

c=3, n=5

Wei

ght

Quadrature Points−10 −5 0 5 10

0

0.5

1

1.5

2

2.5

c=3, n=10

Wei

ght

Quadrature Points

Figure 1: Calculating the Hermite polynomial quadrature rules for different values of c and n.As we can see, there is a linear relationship between the weights and nodes as the parameter cis varied.

6

0 5 10 15 20 25 30 35 4010

−20

10−15

10−10

10−5

100

105

1010

1015

Number of quadrature points n

Err

or

0 50 100 150 20010

−7

10−6

10−5

10−4

10−3

10−2

10−1

100

101

Number of quadrature points n

Err

or

Figure 2: Testing the error of our quadrature rule based on Hermite Polynomials. Top: estimat-

ing∫x20e−x2

dx (setting c = 0.5). Below: estimating∫

e−x2

25x2+1dx.

7

6 Calculating the Hermite transform

We now turn our attention back to the Hermite polynomials. We have the followingformula for our coefficients

αcm =1

m!cm

∫ ∞−∞

σ(x)Hm(x; c)e−x2/2cdx (21)

We can now approximate these coefficients very easily as

αcm ≈1

m!cm

k∑0

σ(xk)Hm(xk; c)wk (22)

This is implemented in HermiteCoefficient.m and HermiteTransform.m (see Appendix).We also include HermiteSeries.m, which takes a set of coefficients and returns the cor-responding function (a linear sum of Hermite polynomials). We use this to test theapproximations, the results of which are shown in Figure 3. The code to calculate theseis included as ApproxTest.m. (Note that we have only run this to a few quadraturepoints; as mentioned, the implementation here is very computationally expensive).

It is worth noting at this point that our implementations are very inefficient; inparticular our calculation of the Hermite polynomials themselves is done in an extremelytime consuming way. The terse programs included with this paper are for the purposeof demonstration and are not intended to be practical. The first and easiest way tospeed up the program would be to use dynamic programming; the quadrature rules, thepolynomials etc can be precomputed, or at least stored when they are computed forreuse.

This numerical method gives us the Hermite transform, as required. Our next stepis to recast our SDE in terms of transforms and operators.

7 Calculating the SDE in operator space

We now turn out attention back to the SDE

u(x; t) =

∫σ(W x

s )dWt (23)

Using the above, we can take the Hermite transform of σ:

u =∫σ(Ws)dWt

=∫

(∑∞

k=0 αckHk(Ws; s)) dWt

=∑∞

k=0 αck

(∫Hk(Ws; s)dWt

)=

∑∞k=0 α

ckHk+1(W (t);s)

k+1

=∑∞

k=1

αck−1

k Hk(W (t); s)

(24)

8

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2−25

−20

−15

−10

−5

0

5

10

x

y

σHermite Approximation

−2 −1.5 −1 −0.5 0 0.5 1 1.5 20

2

4

6

8

10

12

x

y

σHermite Approximation

Figure 3: The approximation to σ by Hermite functions. For the top plot, σ(x) = x3 − x2,c = 0.5, the quadrature rule uses 10 nodes, and we calculate 10 coefficients. For the lower plot,σ(x) = ex, c = 0.5, the quadrature rule uses 15 nodes, and we calculate 15 coefficients.

9

−1 −0.5 0 0.5 1−20

0

20

40

60

80

100

x

u(x;

t)

−1 −0.5 0 0.5 1−20

−15

−10

−5

0

5

10

15

20

x

u(x;

t)

Figure 4: Realisations of u(W xt ; t) for a given σ at a fixed t. Top: σ(x) = x3 − x2, t = 1, 10

points are taken for our Hermite transform, and 20 quadrature points are used. Note that c = t,as in the original problem. Twenty sample paths are shown. Below: σ(x) = |1 − x2|, with thesame parameter values. Note that, as we would expect, the samples are bunched around anequilibrium at u(x; t) = 0.

10

We can rewrite this in operator space. The differentiation operator is very simple:

D =

0 1

23

4. . .

(25)

Then the equation becomesH[u] = DH[σ] (26)

u = H−1 [DH[σ]] (27)

We can now use the above to approximate this numerically:

un = H−1n[(PnDPTn )Hn[σ]

](28)

where Hn is the Hermite transformation truncated to n values.The advantage here is that this computation can be carried out in coefficient space

for a given function σ, and then a variety of values of W xt can be substituted in. This

gives a very efficient way of carrying out Monte Carlo sampling; rather than having tocalculate the integral for every possible sample path, we can simply substitute BrownianMotion sample paths into the expression we’ve already calculated. This substitution willrun very fast, and so many simulations can be carried out very quickly. Note that thismethod requires new calculations for different values of t; this is reasonable, as we willoften be sampling at a few fixed points in time. The code to make such calculations isincluded in the appendix as MonteCarlo.m, and the results are shown in Figure 4. Asexpected, this is much faster than integrating many sample paths, and the main overheadis our poorly coded functions to calculate the H polynomials. As earlier stated, this couldbe fixed in a more robust program.

8 Conclusion

The methods developed in this paper demonstrate a way to use Hermite polynomialsto significantly speed up Monte Carlo sampling for a given stochastic differential equa-tion. In particular, it allows for large volumes of sample paths to be considered whileminimising computational overhead. Although the examples given here are for a verynarrow set of problems, it is hoped that the general process of decomposing drift termsinto Hermite polynomials could be extended to a larger class of stochastic differentialequations. The methods could also be extended to multi-dimensional Wiener processes.In particular, we could apply this to problems with stochastic volatility, which is a verylarge area of research in financial mathematics at the moment.

11

A MATLAB codes

H.m

1 function [ y ] = H( x,c,n )2 %The nth Hermite polynomial with parameter c at point x3

4 if (n==0)5 y=1;6 elseif (n==1)7 y=x;8 else9 y=x.*H(x,c,n−1)−c.*(n−1).*H(x,c,n−2);

10 end;11

12 end

HDash.m

1 function [ y ] = HDash( x,c,n )2 %The derivative of the nth Hermite polynomial3 % with parameter c at point x4

5 y=H(x,c,n−1).*n;6

7 end

Jacobi.m

1 function [ J ] = Jacobi( c, n )2 %The Jacobi matrix for the n−point quadrature rule3

4 a=sqrt(c*(1:n−1));5 J=diag(a,1)+diag(a,−1);6

7 end

HermitePoints.m

1 function [ x ] = HermitePoints( c, n )2 %Returns the nodes for the quadrature rule3 %With parameter c and n nodes4

5 J=Jacobi(c,n);6 x=eig(J);7

8 end

12

HermiteWeights.m

1 function [ w ] = HermiteWeights( c, n )2 %Returns the weights for the quadrature rule3 %With parameter c and n nodes4

5 J=Jacobi(c,n);6 [v,x]=eig(J);7 w=zeros(n,1);8 e=ones(n,1);9 for count=1:n

10 ev=v(:,count);11 w(count)=((e'*ev)ˆ2*sqrt(2*c*pi))/(ev'*ev);12 end;13

14 end

HermiteQuadrature.m

1 function [ x, w ] = HermiteQuadrature( c, n )2 %Returns the nodes and weights for the quadrature rule3 %With parameter c and n nodes4

5 J=Jacobi(c,n);6 [v,x]=eig(J);7

8 w=zeros(n,1);9 e=ones(n,1);

10

11 for count=1:n12 ev=v(:,count);13 w(count)=((e'*ev)ˆ2*sqrt(2*c*pi))/(ev'*ev);14 end;15

16 x=diag(x);17

18 end

HermiteIntegral.m

1 function [ int ] = HermiteIntegral( f, c, n )2 %Calculates the approximation to the integral f*eˆ(−xˆ2)3 %Using the Hermite quadrature rule, with parameter c4 %and n approximation points5

6 [x,w]=HermiteQuadrature( c, n );7 int=w(1:n)'*f(x(1:n));8

9 end

13

HermiteCoefficient.m

1 function [ alpha m ] = HermiteCoefficient( sigma, m, c, n )2 %Calculates the coefficient of the mth Hermite polynomial for the3 %expansion of the function sigma, using c as the Hermite parameter4 %and using the n point quadrature rule5

6 [x,w]=HermiteQuadrature( c, n );7 alpha m=(w(1:n)'*(sigma(x(1:n)).*H(x(1:n),c,m)))/(factorial(m)*cˆm);8

9 end

HermiteTransform.m

1 function [ alpha ] = HermiteTransform( sigma, m, c, n )2 %Calculate the coefficients for the Hermite transformation3 %Calculates m coefficients of sigma, with Hermite parameter c,4 %and uses the n point Hermite quadrature5

6 alpha=zeros(m,1);7 for count=0:m−18 alpha(count+1)=HermiteCoefficient(sigma,count,c,n);9 end;

10

11 end

ApproxTest.m

1 mesh=0.01;2

3 c=0.5;4 quadpts=10;5 coeffs=10;6

7 sigma=@f;8 alpha=HermiteTransform(sigma,coeffs,c,quadpts);9

10

11 figure;12 hold on13 x=−3+mesh:mesh:3;14 y1=zeros(length(x));15 y2=zeros(length(x));16 for count=−3+mesh:mesh:317 y1(round((count+3)/mesh+1))=f(count);18 y2(round((count+3)/mesh+1))=HermiteSeries(count,alpha,c);19 end;20

21 plot(x(10:length(x)−10),y1(10:length(x)−10),'k');22 plot(x(10:length(x)−10),y2(10:length(x)−10),'r');

14

MonteCarlo.m

1 %For a given sigma, take a number of sample paths of u at a set time2

3 m=10;4 c=0.5;5 n=20;6 sigma=@f;7

8 t=1;9 %Take the samples from time = 1

10

11 %The differentiation operator12 D=diag(1:m−1,1);13

14 alpha=HermiteTransform(sigma,m,c,n);15 alpha=D*alpha;16

17 mesh=0.01;18 k=1;19 x=zeros(2/mesh+1);20 y=zeros(2/mesh+1);21

22

23 %The sample point at time t24

25 figure;26 hold on;27 samples=20;28

29 for iter=1:samples30 W=sqrt(t)*randn();31 for count=−1:mesh:132 x(k)=count;33 y(k)=HermiteSeries(W+count,alpha,c);34 k=k+1;35 end;36 plot(x,y,'k');37 end;

B Calculating the Integral of a Gaussian

To calculate∫∞−∞ e

−x2 , we consider the integral

I =

∫R2

e−x2−y2dA

We can calculate this as

I =∫∞−∞

∫∞−∞ e

−x2−y2dydx

=∫ 2π0

∫∞0 e−r

2rdrdθ (in polar coordinates)

15

If we now let t = r2, thenI = 2π

∫∞0

1/2e−tdt= −π(e−∞ − e0)= π

However, we can also consider I as

I =∫∞−∞ e

−x2dx∫∞−∞ e

−y2dy

=(∫∞−∞ e

−x2dx)2

and thus conclude that ∫ ∞−∞

e−x2dx =

√π

References

[1] S.R. Varadhan PDEs for Finance, section 7: Brownian Motion New York University

[2] D. Bell Brownian motion and the heat equation University of North Florida

[3] H. Kuo Introduction to Stochastic Integration Springer

[4] N. Privault, W. Schoutens Krawtchouk Polynomials and Iterated Stochastic Integra-tion University of Stuttgart

[5] S. Olver Numerical Complex Analysis course notes University of Sydney

16