short course on Subsurface stochastic modelling and geostatistics

1

Stochastic Modelling and

Geostatisticsby

Dr. Amro Elfeki

2

Time Table• 02/10/2002 Introduction and probability theory of single and

multi-variate • 09/10/2002 Some termenology, real-domain and spectral

domain representation of stochastic processes.

• 16/10/2002 Stochastic models for site characterization: Discrete models

• 23/10/2002 Stochastic models for site characterization: Continuous models (1).

• 30/10/2002 Stochastic models for site characterization: Continuous models (2).

• 06/11/2002 Stochastic differential equations and methods of solution.

• 13/11/2002 MonteCarlo method.• 20/11/2002 Kriging and conditional simulations.

3

Time Series

0

20

40

60

80

100

120

140

160

1-1-

1978

1-3-

1978

1-5-

1978

1-7-

1978

1-9-

1978

1-11

-197

8

1-1-

1979

1-3-

1979

1-5-

1979

1-7-

1979

1-9-

1979

1-11

-197

9

Series1

4

Space Series

5

Outcrop (1)

6

Outcrop (2)

7

Boreholes

8

Site Characterization

• Deterministic Approach.

• Stochastic Approach.

9

Deterministic Approach

0 200 400 600 800 1000 1200 1400 1600-15

-10

-5

0

10

Stochastic Approach

• The word stochastic has its origin in the Greek adjective στoχαστικoς which means skilful at aiming or guessing .

• The stochastic approach is used to solve differential equations with stochastic parameters.

• This approach is a tool to evaluate the effects of

spatial variability of the hydrogeological parameters on flow and transport characteristics in porous formations.

11

Why we need the Stochastic Approach?

• The erratic nature of the hydrogeological parameters observed at field data.

• The uncertainty due to the lack of information about the subsurface structure which is known only at sparse sampled locations.

12

Concept of Stochastic Simulation

• Generation alternative equiprobable images of spatial distributions of objects or nodal values.

• Each alternative distribution is a stochastic simulation.

13

Types of Stochastic Models for Subsurface Characterization

D isc re te M od e ls C on tin u ou s M od e ls H yb rid M od e ls

S toch as tic M od e ls

14

Scales of Natural Variability

15

Definition of The Stochastic Process

A stochastic process can be defined as:

“a collection of random variables”.In a mathematical form:

the set {[x, Z(x,ζi)], x Rn }, i = 1,2,3...,m.

Z(x,ζ) is stochastic process, (random function), x is the coordinates of a point in n-dimensional space, ζ is a state variable (the model parameter), Z(x,ζi) represents one single realization of the stochastic process, i= 1,2,...,m (i: number of a realization of the stochastic process Z), Z(x0,ζ) = random variable, i.e., the ensemble of the realizations of the stochastic process Z at x0, and Z(x0,ζi)= single value of Z at x0. For simplification the variable ζ is generally omitted and the notation of this stochastic process is Z(x).

16

Realizations of A Stochastic Process

17

Uni-dimensional Stochastic Process

0 10 20 30 40Space or T im e Scale

-2.0

0.0

2.0

Par

amet

er

A stochastic process in which the variation of a property of a physical phenomenon is represented in one coordinate dimension is called uni-dimensional stochastic process. The coordinate dimension can be time as in time series, or space as in space series.

18

Spatial Random Fields

0 20 40 60 80 100 120 140 160 180 200

-40

-20

0

-3.3 -2.3 -1.3 -0.3 0.7 1.7 2.7

Y=Log (K )

Random fields are multi-dimensional stochastic processes.

19

Comparison in Terminology between

Statistical and Stochastic Theories Statistical Theory Stochastic Theory

Sample Realization

Population Ensemble

20

Probabilistic Description of Stochastic

Processes • Single Random Variable (Univariate).

• Multi Random Variables (Multivariate): Random Vector

21

Probability Distribution Function (Cumulative Distribution Function)

}{Pr)( zZ = zP Single random variable

where Pr{A} is a probability of occurrence of an event A, and P(A) is a cumulative distribution function of the event A, z is a value in deterministic sense.

The distribution function is monotonically nondecreasing.

1)(0)( = +P = -P

22

Probability Density Function (PDF)

dzzdP =

zz+zZ<z = zp

z

)(}Pr{lim)(0

The density function, p(z), of random variable Z is defined by,

Inversely, the distribution function can be expressed in terms of the density function as follows

z

-

dz zp = zP ')'()(

p(z) is not a probability, but must be multiplied by a certain region Δz to obtain a probability. P(z) is dimensionless, but p(z) is not. It has dimension of [z-1].

23

Graphical Representation of pdf and cdf

24

Log-normal Density

0.......,21)(

2

2)ln(

xex

xf x

xx

z

25

Derivation of a pdf from a Time Series Z(t)

t

zz+dz

dt1 dt2 dt3

T

3

1.1lim}Pr{lim)(

ii

T0z0z

tTz

= z

z+zZ<z = zp

3

1

1lim}Pr{i

iTt

T = z+zZ<z

p(z).z = prob. That Z(t) lies between z and z+z.

26

Joint Probability Density Function

z...zP

zzzz+zZ< z,...,z+zZ<zp

n

n

n

nnnn

zn

z

121

1111 )( = ...

}ΔΔ{ Prlim = )(

0..

01

zz

Here, z without index is a vector. Inversely, the joint distribution function can be expressed in terms of the joint density as follows,

nz

n

z

'dz'...dzpdpP-

1--

)'(... = )'( = )(1

zz'zzz

ijX

Y0

ZZ

1

p

2 3

27

Bivariate Normal pdf

28

Joint Probability Distribution Function

Consider Z as a random vector defined in a vector form as {Z1,Z2,…,Zn}T, where, Z1, Z2,… and Zn are single random variables. z is described by the joint distribution function of Z as,

)(Pr).( zZ ,...,zZ ,zZ = z .,,.z ,zP nn2211n21

1)(0)( = ...,+ ,+ ,+ ,+P

= ...,- ,- ,- ,-P

29

Graphical Representation of jpdf and jcdf

30

Marginal Probability Density Function The marginal probability density function is defined as follows,

dz...dzdz...dz dz p = zp ni+i-

n

i 1121

1

)(...)( z

-

-

dzzzp =zp

dzzzp =zp

1212

2211

),()(

),()(

In case of bivariate pdf:

31

Marginal Probability Distribution Function The marginal probability distribution function of a component Z1 of the random vector Z is obtained from the joint density function by the integration,

')(...

),,,Pr()(

12

1

dz dz...dz p =

<Z<-...<Z<-z<Z = zPz

n

n2111

z

The term between brackets is the marginal probability density function of the component Z1, and

P(z1) is called the marginal distribution function of the component Z1 of the random vector Z.

32

Conditional Probability Density Function The conditional density function of component Zn of the random vector Z given that the random components at n-1, n-2,…,1 have specified values is defined by

zz,..., z|z P = z,..., z | zp

n

n-nn-n

)()( 1111

the function p(zn zn-1,…,z1) can be expressed in a more convenient form as follows,

)()(),(

121121

z,...,z, zpp= z,...z, z | zp

n-n-n-n

z

where, p(z), is the joint density function of all the components of the vector Z. It can also be written,

)()( 321 z,..., z, z, z = pp nz

33

Conditional Probability Distribution Function (Cont.)

).()()(

z ,,..z ,zpz ..., ,z ,z ,zp = z,...z ,z | zp1-n21

n32112-n1-nn

If Z1, Z2,..., and Zn are independent random variables then the joint density function is the multiplication of the marginal density function of the individual random components. This can be expressed as follows,

)()()()( zp ...zp . zp = z ,...,z ,z ,zp n21n321

So, in conclusion, for independent random variables the following holds,

)()( zp= z,...z ,z | zp n12-n1-nn

34

Conditional Prob. Distribution Function

The conditional distribution function of one component Zn of a random vector Z given that the random components at n-1, n-2,…,1 have specified values is defined by,

}ΔΔPr{).(

11111111

121

z+zZ<z,..., z+zZ< z | zZ

= z.., ,z, z | zP

n-n-n-n-nn

n-n-n

where, the definition Prob{AB} is the conditional probability of event A given that, event B has occurred and is defined by,

}Pr{}{Pr}{Pr

B B A = B | A

where, AB is the conjunctive event of A and B.

35

Statistical Properties of Stochastic Processes

• Spatial or Temporal Properties.MeanVarianceCovariance

• Ensemble Properties.MeanVarianceCovariance

36

Spatial Average (Mean)

dZ| v |

= Zv ii )(

)(1x

xx

where, v(x) is the specified length, area or volume (for one, two or three dimensional space respectively) centred at x of measure v and index i is the i-th realization.

n

jjii Z

n Z

1

)(1x

where, n is the number of discrete points discretizing the volume v, index j is the j-th point on volume v.

Z

xv

Z

xZ 1

Z 2 Z i Z n

37

Spatial Mean Square Value

)(

22 )(1

x

xxv

ii dZ | v |

= Z


n

jjii Z

n Z

1

22 )(1x


Z

xv

Z

xZ 1

Z 2 Z i Z n

38

Spatial Variance



dZ - Z | v |

= Z - Z = S = ZVarv

iiiiZi i )(

222 )(1

)(][x

xxx

n

ji

2Z Z

-n S i

1i

2Z - )(

11

x j

)( 2ii

22Z Z - Z S i

39

Spatial Covariance

The covariance is a measure of the mutual variability of a pair of realizations; or in other words, it is the joint variation of two variables about their common mean.

)xv(

iiiiii dZ-Z +Z-+Z| v |

=Z,ZCov xxxsxsxxsx )()()()(1))()((

Z - Z Z - Zn

Z,ZCovn

j=iiiiii

)(

1

)()()()()(

1))()((s

jjjj xxs+xs+xs

xs+x

where, n(s) is the number of points with lag s.

40

Ensemble Statistical Properties

(Mathematical Expectation) The average of statistical properties over all possible realizations of the process at a given point on the process axis.

41

Ensemble Average (Mean)

dzz pz = ZE

)()()}({ xx oo

xo is the coordinate of a given point on the space axis, p(z) is probability density function of the process Z(x) at location xo, and E{.} is the expected value operator.

Zm

ZE m

ii

1

)(1)}({ xx oo

m is the number of realizations.

42

Ensemble Mean Square Value

dzz pz = ZE

)()()}({ 22 xx oo

m

iiZ

m ZE

1

22 )(1)}({ xx oo

43

Ensemble Variance

dzz pZ-E Z= Z-EZ=E σ Z

)()}({)()}({)( 222)( xxxx ooooxo

m

iiZ Z - E Z

m-σ1

22)( })({)(

11

xx ooxo

})({})({ 222)( xx ooxo

ZE - Z=E σ Z

44

Ensemble Covariance

dzdz, zzp=, ZZCov

)()())()(())()(( xs+xxs+xxs+x

p(z(x+s),z(x)) is the joint probability density function of the process Z(x) at locations x+s and x.

m

ijjjj Z - E Z.Z - E Z

m, Z+ZCov

1

})({)(})({)(1))()(( xxs+xs+xxsx

45

Some Terminology

• Stationarity (Statistical Homogeneity).

• Non-stationarity.• Intrinsic Hypothesis.• Ergodicity.

46

Stationarity

The stochastic process is said to be second-order stationary (weak sense) if:

1) The mean value is constant at all points in the field, i.e., the mean does not depend on the position. μ = ZE Z)}({ x

2) The covariance depends only on the difference between the position vectors of two points (xi-xj)= sij the separation vector, and does not depend on the position vectors xi and xj themselves. )()}({)()}({)())()(( sxxxxxx jjiiji =CovZ-EZ Z-EZ=E, ZZCov

σ = = Cov)Z(Var Z2)0(x

ijX

Y0

ZZ

1

p

2 3

47

Non-StationarityA stochastic process is called non-stationary, if the moments of the process are variant in space, i.e., from one position to another.

48

Example of Non-stationarity

0 20 0 40 0 600 80 0 1 00 0 1 20 0 14 0 0 16 0 0 1 800 2 00 0-4 00

-2 00

0

-10 -8 -6 -4 -2 0 2 4 -5 .0 -3 .0 -1 .0 1.0 3 .0

0 20 0 40 0 600 80 0 1 00 0 1 20 0 14 0 0 16 0 0 1 800 2 00 0-4 00

-2 00

0

0 .0 0.8 1 .5 2 .3 3.0

0 20 0 40 0 60 0 80 0 10 0 0 12 0 0 14 0 0 16 0 0 180 0 20 0 0-400

-200

0

- 8 - 6 - 4 - 2 0 2 4

0 2 0 0 4 0 0 6 0 0 8 0 0 1 0 0 0 1 2 0 0 1 4 0 0 1 6 0 0 1 8 0 0 2 0 0 0H o r i z o n t a l D i s t a n c e ( m )

-4 00

-2 00

0

Dep

th (m

)

1 2 3 4

L og (H yd raulic C on d uc tiv ity m /day) L o g (H ydra u lic C o nd u c tivity m /d a y)

L og (H yd raulic C on d uc tiv ity m /day)L og (H yd ra ulic C on d uctiv ity m /da y)

(a ) N o n-S ta tio n a rity in T h e M ea n .

(b ) N o n -S ta tio na r ity in T h e V ar ia n ce .

(c ) N o n -S ta tio n arity in C o rrela tio n L en g ths.

(d ) G lo ba l N o n - S ta tio n a rity .

G eo lo g ica l S truc tu re .

0 20 0 40 0 60 0 80 0 10 0 0 12 0 0 140 0 16 0 0 180 0 20 0 0-40 0

-20 0

0

49

Intrinsic Hypothesis

The intrinsic hypothesis assumes that even if the variance of Z(x) is not finite, the variance of the first-order increments of Z(x) is finite and these increments are themselves second-order stationary. This hypothesis postulates that:

(1) the mean is the same everywhere in the field; and (2) for all distances, s, the variance of the increments,

{Z(x+s)-Z(x)} is a unique function of s so independent of x. A stochastic process that satisfies the stationarity of order two also satisfies the intrinsic hypothesis, but the converse is not true.

)(2)()(

0})()({2 sxsx

xsx

γ = -Z+ZE

= -Z+ZE

(s) is called the semi-variogram,

50

Intrinsic Hypothesis (cont.)

From practical point of view:

1. The intrinsic hypothesis is appealing, because it allows the determination of the statistical structure, without demanding the prior estimation of the mean.

2. For a stationary random process, where both a covariance and a sime-variogram are exist, it is easy to show the relationship between them as, )()0()( ss - Cov = Covγ

51

Comparison between Intrinsic Hypothesis and Second-order Stationarity

Intrinsic Hypothesis Second-order Stationarity

Less strict than 2nd order stationarity

More strict than Intrinsic hypothesis

Variogram Correlogram

If the phenomenon does not have a finite variance, the variogram will never have a horizontal asymptotic value.

0})()({ = -Z+ZE xsx

-Z+ZEγ 2)()(21)( xsxs

Z = ZE })({ x

Z+ZECov ZZ )(.)()( xsxs

finitebemustσ Z2

52

Ergodicity

Ergodicity is a statistical property which implies that:

(spatial statistics) are equivalent to (ensemble statistics).

This equivalence is achieved when the size of the space domain is sufficiently large or tends to infinity.

It is theoretical defined, but practically impossible.

Z E Z i )}({ xo

σ S Z2

Z i

2)( xo

53

Real (Lag) Domain Representation

of Stochastic Processes

Properties of stationary stochastic processes may be represented in a lag domain:

- auto-correlation function of the lag s, or - cross-correlation function of s.

Correlogram: represents the correlation coefficients between the values of the process versus the lag s.

54

Spatial Auto-Correlation It is a measure of the spatial correlation structure of a process.

σ

, Z+ZCov = ρZ

ZZ 2

)()()( xsxs

The auto-correlation function has the following properties:

)(- = )(0)(

1)0(

ZZ ss

ZZ

ZZ

ZZ

= =

Z - Z Z - Zn

Z,ZCovn

j=iiiiii

)(

1

)()()(

1))()((s

jj xs+xs

xs+x

55

Calculation of Auto-Correlation Function

56

High and Poor Correlations

57

Some Auto-Correlation of Series

58

2D Isotropic Exponential Auto Correlation

59

Statistical Isotropy and Anisotropy

A multi-dimensional stochastic process is said to be

Isotropic, if the process does not have a preferred direction, i.e., the variability in the process is the same in all directions.

Anisotropic, if the variability changes from one direction to another.

Isotropic Anisotropic

60

Integral ScaleThe integral scale Iz of autocorrelation function is defined as,

0

)( ss dρ = I ZZz

which implies that the average distance over which the process is autocorrelated in space. For practical applications, the integration is calculated over a certain limits [0, So] where, So is the smallest value of s at which the autocorrelation function becomes practically zero.

0 5 10 15 20 25

0

0.2

0.4

0.6

0.8

1

61

Correlation Scale (Range)

The correlation scale is defined as the distance over which the process is autocorrelated in space. It is calculated as the distance at which the autocorrelation function tends to zero. There are various ways, some authors suggest the threshold value taken as e-1 to others.

In case of 1D of linear auto-correlation,

sif =sρ

sifλ

| s | = sρ

ZZ

ZZ

0)(

1)(

one finds that the integral scale is related to the correlation length by the formula,

2 = I z

62

Correlation Range

63

Spatial Cross-Correlation

)())()(()(

22 21σ σ

, Y+ZCov = ρYZ

/ZYxsxs

The spatial cross-correlation represents a relation between two stochastic processes. It defines the degree of which two stochastic process are correlated as a function of separation lag.

positive or negative or zero

64

A Typical Variogram

65

Sime-Variogram Models

Sime-variogram models

]5.05.1[)( 3ssCsγ

ssCsγ )sin(1)(

s Csγ .)(

]1[)( se Csγ

]1[)(2ss e Cγ

66

Variogram Example Calculation

-Z+ZEγ 2)()(21)( xsxs

67

Uncorrelated, Orthogonal, Independent Random Vectors

)().(),(

0

jiji

jTi

jTij

Ti

ZpZ pZZpiftIndependen

ZZE ifOrthogonal

ZE Z E ZZE ifedUncorrelat

-Independence is a stronger condition than uncorrelatedness.

68

Spectral (Frequency) Domain Representation of Stochastic Processes

Properties of stochastic processes can be represented in the frequency domain, relating:

“ the square of amplitude of each sine or cosine component fitting the process versus ordinary frequency or its angular frequency or wave number”.

In this respect, the stochastic process is considered as made up of oscillations of all possible frequencies.

The diagram used for this presentation is called priodgram.

69

Decomposition of a Random Signal

0 1 2 3 4 5 6Frequency

0

20

40

60

80

10 0

(Am

plitu

de)^

2

70

Auto-Power (Variance) Spectral Density Function (Auto-PSD)

The term power is commonly seen in the literature. Its origin comes from the field of electrical and communication engineering:

power dissipated in an electrical circuit is proportional to the mean square voltage applied.

The adjective spectral denotes a function of frequency.

The concept of density comes from the division of the power (variance) of an infinitesimal frequency interval by the width of that interval.

The power spectrum describes the distribution of power (variance) with frequency of the random processes, and as such is real and non-negative.

71

(Variance) Spectral Density Function

The auto-power spectrum (spectral density function) for a process Z(x) is given by,

||z L

= z.zL

= SL

*

LZZ )(

1lim)()(1lim)( 2ωωωω

where, z(ω) is Fourier transform of the process Z(x), which is expressed as,

deZπ

= z -i

xxω xω)(21)(

and z*(ω) is the conjugate of z(ω) and ω is the angular frequency vector.

72

Calculation of Power Spectrum from a Signal

73

Properties of the Spectral Density Function

)()(

)(

0)(2

ωω

ωω

ω

S S

σ = dS

S

ZZZZ

Z- ZZ

ZZ

74

Cross-Power (Variance) Spectral Density

Function (Cross-PSD) The cross-PSD is defined between a pair of stochastic process. Cross-PSD is in general complex. The magnitude of the cross-PSD describes whether frequency components in a process are associated with large or small amplitudes at the same frequency in another process, and the phase of the cross-PSD indicates the phase lag or lead of one process with respect to the other one for a given frequency component. This expressed mathematically as,

)()(1lim)( ωωω y.z L

=S *

LZY

where, y*(ω) is the conjugate of y(ω), and y(ω) is Fourier transform of the process Y(x)

75

Relation between AutoCovariance Functions and AutoSpectral Density Functions

The covariance functions and spectral density functions are Fourier transform pairs. This can be expressed in mathematical forms using

Wiener-Khinchin relationships,

σ = dS = C

deS = C

deCπ

= S

Z- ZZZZ

-

iZZZZ

-

-iZZZZ

2)()0(

)()(

)(21)(

ωω

ωωs

ssω

sω

sω

76

Relation between Auto Correlation and Power Spectrum (examples)

77

Relation between Cross-Covariance Functions and Cross-Spectral Density

Functions

For cross-PSD and cross-correlation these relations are,

-

iZYZY

-

-iZYZY

deS =C

deC π

= S

ωωs

ssω

sω

sω

)()(

)(21)(

78

Summary of A Random Variable

79

Monte-Carlo Sampling

80

Generation of a Random Variable

81

Models of The Stochastic Approach

D isc re te M od e ls C on tin u ou s M od e ls H yb rid M od e ls

S toch as tic A p p roach

82

Mosaic Facies (Discrete) Models

Types of discrete models:• Object-based Models.• Sequential based Models:

-Markov Chains in 1-D, 2-D etc.-Markov Random Fields.-Sequential Indicator Simulation Models. -Random Lines Models.

In this approach one is aiming to construct - formation geological units, its geometric characteristics. - lithologies.- units dimensions (length, thickness, and width), - orientations and frequency of occurrence, etc..

83

Object-based Models

84

Sequential-based Models

-Markov Chains in 1-D.-Markov Chain in 2-D.

85

Theory of One-dimensional Markov ChainSS S

i0 1 i+ 1i-1 N2

l k q

,:)Pr()Pr(

p S Z | S ZS Z ,..., S Z ,S Z ,S Z | S Z

lkl1-iki

p0r3-in2-il1-iki

................

..

1

21

11211

nnn

lk

n

pp

pp

ppp

p 1,...,01

ppn

klklk

w p kN

lkN

)(lim1,0

...,

1

1

n

kkk

klkl

n

l

ww

n ,1 k ,w p wMarginal prob.

Transition prob.

86

Example on One-dimensional Markov Chain

87

Coupled Markov Chain

i0 1 i+ 1i-1 N2

S lS f

i0 1 i+ 1i-1 N2

S kS q

X-Chain

Y-Chain

kflmmilif1+ik1+i p S Y ,S X | S Y ,S X ,)Pr(

},...,,,...,,,....{. 2121, nnmflkkflm ssssssppp Transition probabilities

mllm www .Marginal probabilities

88

Coupled Chain on a Lattice Dark Grey (Boundary Cells)Light Grey (Previously Generated Cells)W hite (Unknown Cells)

i-1 ,j i,ji,j-1

1 ,1

N x ,N y

N x ,1

1 ,N y

N x ,j

nkp . p

p . p SZSZSZ p

p . p C

SYSYSXSXCS Z ,S Z | S Z

SYSYSXSXS Z ,S Z | S Z

SYSYS Z | S Z

SXSXS Z | S Z

f

vmf

hlf

vmk

hlk

mjiljikjiklm

n

f

vmf

hlf

mjkjlikimjiljikji

mjkjlikimjiljikji

mjkjmjikji

likiljikji

,...1.),|Pr(:

)Pr()Pr()(Pr

)Pr()Pr()(Pr

)Pr()(Pr

)Pr()(Pr

1,,1,,

1

1

111,,1,

111,,1,

11,,

1,1,

89

Coupled Chain Prob. In Two-State ModelTwo-State Model

1

1 22

1

2 11

1

2 22

1

1 11

2

1 11

2

1 22

2

1 21

2

1 12

2

2 11

2

2 22

2

2 21

2

2 12

1

1 21

1

1 12

1

2 12

1

2 21

P(ij,lk) = P(i,l) P (j,k) , i,j,l,k = 1,2 h v

The Transition Probabilities o f The Coupled C hain

k= l k<> lP(11,11) P(11,22) P (11,21) P(11,12) P(12,11) P(12,22) P (12,21) P(12,12)P(21,11) P(21,22) P (21,21) P(21,12) P(22,11) P(22,22) P (22,21) P(22,12)

90

Example of Two-State ModelNumerical Exam ple of Two-State Model

The Transition Probabilities of The Coupled Chain

k=l k<>lP(11,11) =0.250 P(11,22) =0.250 P(11,21) =0.250 P(11,12) =0.250 P(12,11) =0.250 P(12,22) =0.250 P(12,21) =0.250 P(12,12) =0.250P(21,11) =0.125 P(21,22) =0.375 P(21,21) =0.375 P(21,12) =0.125 P(22,11) =0.125 P(22,22) =0.375 P(22,21) =0.375 P(22,12) =0.125

P(ij,lk) = P(i,l) P(j,k) , i,j,l,k = 1,2 h v

The Horizontal Chain The Vertical Chain 1 2 1 2 1 0.50 0.50 1 0.50 0.50 2 0.25 0.75 2 0.50 0.50

Normalized Transition Probabilities of The Coupled Chain

k=l k<>lP(11,11) =0.500 P(11,22) =0.500 P(11,21) =0.000 P(11,12) =0.000 P(12,11) =0.500 P(12,22) =0.500 P(12,21) =0.000 P(12,12) =0.000P(21,11) =0.250 P(21,22) =0.750 P(21,21) =0.000 P(21,12) =0.000 P(22,11) =0.250 P(22,22) =0.750 P(22,21) =0.000 P(22,12) =0.000

P(ij,kk) = P(i,f) P(j,f)

P(i,k) P(j,k)i,j,k =1,2

f

91

One-dimensional Markov Chain Conditioned on Future States

i0 1 i+ 1i-1 N2

S kS l S q

)1(

)(

)1(

)(

1

11

111

1

11

1

111

1

11

1

)(Pr

)Pr().|(Pr)(Pr).|(Pr).|(Pr

)(Pr

),(Pr),(Pr).|(Pr

)(Pr

),(Pr),(Pr),|(Pr

)(Pr

),(Pr),,Pr(

)(Pr

)(Pr

iNlq

iNkqlk

qlk

iNlq

lkiN

kqqNliki

liliqN

lilikikiqNqNliki

qNli

kilikiqNqNliki

qNli

kilikiliqNqNliki

qNli

qNkiliqNliki

qNliki

ppp

p

ppp

S Z ,S Z | S Z

SZS Z S Z S Z S Z S Z S Z S Z

S Z ,S Z | S Z

S Z S Z S Z S Z S Z S Z

S Z ,S Z | S Z

S Z S Z

S Z S Z S Z S ZS Z S Z ,S Z | S Z

S Z S Z S ZS ZS Z S Z ,S Z | S Z

S Z ,S Z | S Z

92

Coupled Markov Chain (application) Two-dimensional Cross-sectional Panel of the Fluvial Succession of the Medial

Area of the Tόrtola Fluvial System, Spain

Length of The Section (m) = 648. Depth of The Section (m) = 115.Sampling interval in X-axis (m) = 9. Sampling interval in Y-axis (m) = 2.5

Horizontal Transition Probability Matrix State 1 2 3 4 5 6 7 8 1 0.893 0.009 0.005 0.000 0.000 0.000 0.000 0.093 2 0.000 0.796 0.011 0.000 0.000 0.000 0.000 0.194 3 0.000 0.000 0.989 0.000 0.000 0.000 0.000 0.011 4 0.006 0.000 0.013 0.885 0.000 0.000 0.000 0.096 5 0.074 0.000 0.000 0.074 0.593 0.037 0.000 0.222 6 0.000 0.013 0.000 0.000 0.000 0.946 0.000 0.040 7 0.040 0.000 0.000 0.000 0.000 0.000 0.940 0.020 8 0.007 0.006 0.002 0.007 0.005 0.005 0.001 0.968

Vertical Transition Probability Matrix State 1 2 3 4 5 6 7 8 1 0.591 0.000 0.000 0.000 0.014 0.000 0.042 0.353

2 0.011 0.753 0.097 0.000 0.000 0.000 0.000 0.1403 0.032 0.000 0.623 0.000 0.000 0.238 0.000 0.1073 0.000 0.025 0.000 0.662 0.013 0.000 0.000 0.2995 0.111 0.000 0.000 0.074 0.519 0.000 0.000 0.2966 0.000 0.000 0.026 0.032 0.006 0.084 0.000 0.8517 0.120 0.000 0.000 0.100 0.000 0.000 0.360 0.420

8 0.029 0.008 0.039 0.017 0.003 0.031 0.010 0.863

1 2 3 4 5 6 7 8

93

Coupled Markov Chain (application cont.)

0 50 100 150 200 250 300-80

-60

-40

-20

0

0 50 100 150 200 250 300

-80

-60

-40

-20

0

1 2 3 4 5 6 7 8

0 50 100 150 200 250 300

-80

-60

-40

-20

0

94

Application of C_CMCM

1

2

3

4

5

6

7

8

0 50 100 150 200 250 30 0-80

-60

-40

-20

0

0 50 10 0 150 200 250 300-80

-60

-40

-20

0

0 50 10 0 150 200 250 300-80

-60

-40

-20

0

0 50 100 150 200 250 300-80

-60

-40

-20

0

0 50 100 150 200 250 300-80

-60

-40

-20

0

0 50 100 15 0 200 250 300-80

-60

-40

-20

0

95

Random Lines Model

96

Random Sets Mosaic Model

97

Continuous Models

- Multi-variate Method.

- Nearest Neighbour Method.

- Turing Bands Method.

98

Multivariate Normal Method

ijX

Y0

ZZ

1

p

2 3

99

Nearest Neighbour Method

100

Turning Bands Method

101

Comparison Between Various Methods

102

Hybrid Models

103

Stochastic Differential Equations (SDEs)

Stochastic differential equation (SDE) = Differential equations for random functions (stochastic processes)

= Classical differential equation (DE) +

Random functions, coefficients, parameters and boundary or initial values

Ω = yΦyxK

y +

xΦyxK

x yyxx

0),(),(

104

Solving SDEs

Analytical Approaches G reen 's F u n c tion A p p roach

P ertu rb a tion M eth odS p ec tra l M eth od

Num erical ApproachesM on teC arlo M eth od

S o lvin g S D E s

105

Spectral Method

The dependent variable and parameter in a stochastic differential equation are represented in terms of its mean or expected value denoted with an angle brackets, and some fluctuations around the mean denoted by a prime, as follows:

Φ + ΦΦ = where, Y is written as the perturbed parameter, Y is the mean or expected value of the parameter, E{Y}, Y´ is a perturbation around the mean value of the parameter, so E{Y´}= 0. Similarly, Φ is the perturbed variable, Φ is the mean or expected value of the variable, E{Φ}, and Φ` is a perturbation around the mean value of the variable, E{Φ`}= 0.

Y + YY =

106

Spectral Method (Cont.1)

Assumptions:1. The perturbations are relatively small compared to the mean

value, so that second order terms involving products of small perturbations can be neglected.

2. The stochastic inputs parameters and the outputs variables are second order stationary so that they can be expressed in terms of the representation theorem.

Procedure:1. Introducing the expressions into the differential equation.2. Taking the expected value of the equation results in two new

equations, one for the first moment (mean) and the other for the perturbations.

3. The first is a deterministic differential equation, which can be solved analytically to get the solution for the mean of the dependent variable as a function of the mean of the parameter.

4. The second equation is transformed in the spectral domain by using Fourier-Stieltjes representation theorem.

107


)(kkx dZ e = Y Yi

)(kkx dZ e = Φ Φi

5. The following integral transformation is used,

Where k is wave number vector, x is space dimension vector, Z(k) is a random function with orthogonal increments, i.e., non-overlapping differences are uncorrelated and dZ(k) is complex amplitudes of the Fourier modes of wave number k. The spectral density function SYY(k) of Y’ is related to the generalized Fourier amplitude, dZY by

0)}()({ 2121 k k if , = kdZ.kdZE *YY

k = k if dk, kS = kdZ.kdZE YY*YY 21121 )()}()({

The asterisk, *, denotes the complex conjugate.

108


6. By using the above representation and substituting them into the stochastic differential equation of perturbation, one can get the spectrum of the variable as a function of the spectrum of the parameter.7. The spectral density function is the Fourier transform of its auto-covariance function, which can be expressed mathematically as follows:

ssk ks dC eπ

= S ΦΦ-i

ΦΦ )(21)(

where s = lag vector of the auto-covariance function. 8. By using Wiener-Khinchin theorem, one can write,

kks ks dS e = C ΦΦ-i

ΦΦ )()(

)0(2ΦΦΦ = Cσ

109

Perturbation Method

The parameter,Y, (e.g. conductivity) and the variable, Φ, (e.g. head) can be expressed in a power series expansion as,

......Yβ + Y + βYY = o 22

1

......β + + β = o 22

1where, β is a small parameter (smaller than unity). These expressions are introduced in the differential equations of the system to get a set of equations in terms of zero- and higher-order expressions of the factor β. The equation that is in terms of zero β corresponds to the mean head. The equation that is in terms of first-order of β corresponds to the head perturbation. In practice, only two or three terms of the series are usually evaluated.

110

MonteCarlo Method1. Assumption of the pdf of the model parameters or joint pdf. The

pdfs are based on some field tests and/or laboratory tests. 2. Generation of random fields of the hydrogeological parameters to

represent the heterogeneity of the formation.3. By using a random number generator, one generates a realization

for each one of these parameters. The parameter generation can be correlated or uncorrelated depending on the type of the problem.

4. With this parameter realization a classical numerical flow or/and transport model is run and a set of results is obtained.

5. Another random selection of the parameters is made and the model is run again, and so on.

6. It's necessary to have a very large number of runs, and the output model results corresponding to each input is obtained which can be represented mathematically by the stochastic process Φ(x,ζi).

7. Statistical analysis of the ensemble of the output (i.e. Φ(x,ζi) for i = 1,2,...m, can be made to get the mean, the variance, the covariance or the probability density function for each node with a location x in the grid.

111

Comparison between Analytical and MonteCarlo Methods

Item Analytical MonteCarloSolution defined over a

continuumdefined over a grid.

Stationarity of the variables

input and output variables should be stationary

no need for stationarity assumption.

Probability distribution of input variables

no need to define PDF of the input variable in some applications.

the PDF of the input variables must be known.

Handling variability limited to small variability.

not limited to small variability.

112

Comparison between Analytical and MonteCarlo Methods (cont.1)

Item Analytical MonteCarloLinearity versus non-linearity

based on linearized theories or weakly-nonlinearity.

it can address both cases.

Outcome of the method

closed form solution of moments.(limited only for the first two moments)

numerical values used to calculate moments of the independent variables. (One can calculate the complete PDF).

113


Item Analytical MonteCarloSpatial structure of the variability

simple forms of auto-covariance models

simple and compound (nested) forms of auto-covariances.

Sources of errors number of simplifying assumptions such as, the form of mean and covariance function, the geometry of the domain and the boundary conditions.

sampling (finite number of realizations) and discretization errors are introduced because of approximation of the governing equations.

Time and computer effort

limited (to calculate the values).

time consuming.

114


Item Analytical MonteCarloperforming conditioning to field measurements

difficult easy

handling more than one stochastic variable

if it is possible, it is too difficult.

it is easy to handle more than one variable.

115

Kriging

116

Kriging Example

117

Conditional Simulations

From a practical point of view, it is desirable that the random fields not only reproduce the spatial structure of the field but also honour the measured data and their locations.

This requires an implementation of some kind of conditioning, so that the generated realizations are constrained to the available field measurements.

118

Conditional Simulations (cont.)

119

Methods of Conditioning

D irec t M eth od s"M etrica l M eth od s "

In d irec t M e th od s"K rig in g M eth od "

M eth od s o f C on d it ion in g

120

Indirect Conditioning by Kriging

(1) A kriged map is generated from the field data with the sampled locations which will be smoother than reality. (2) An unconditional simulated field is generated by TBM from the data which reproduces the spatial structure of the underlying random function.(3) Allocation of the unconditional values (pseudo measurements) at the sites of measurements is done on the simulated map in step 2.(4) Another kriged map is generated from the pseudo measurements.(5) A pseudo error is calculated by subtracting the kriged map in step 4 from the unconditional simulation in step 2.(6) The conditional simulation map is generated by adding the pseudo error in step 5 to the kriged map in step 1. So,

)Z - Z( + Z = Z kususkdcs

Zcs is the required conditional simulation, Zkd is the kriged map from the real data, Zus is the unconditional simulation, Zkus is the kriged map with the pseudo measurements.

121

Example of Conditioning by Kriging

122

Correlation Matrix

123

Fractals

124

Pdf of Sine wave

125

short course on Subsurface stochastic modelling and geostatistics

Engineering