STATISTICS Joint and Conditional Distributions Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering National Taiwan University.

STATISTICS Joint and Conditional

Distributions

Professor Ke-Sheng ChengDepartment of Bioenvironmental Systems Engineering

National Taiwan University

Joint cumulative distribution function

• Let be k random variables all defined on the same probability space ( ,A, P[]). The joint cumulative distribution function of , denoted by , is defined as

for all .

kXXX ,,, 21

kXXX ,,, 21 ),,(,,1 kXXF

],,[ 2211 kk xXxXxXP ),,( 21 kxxx

04/21/23 2Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Discrete joint density



Marginal discrete density• If X and Y are bivariate joint discrete

random variables, then and are called marginal discrete density functions.

)(Xf )(Yf

}:{

, ),()(ki xxi

iiYXkX yxfxf

}:{

, ),()(ki yyi

iiYXkY yxfyf

0),( yxf XY x y

XY yxf 1),(


Continuous Joint Density Function• The k-dimensional random variable

( ) is defined to be a k-dimensional continuous random variable if and only if there exists a function such that

for all .

• is defined to be the joint probability density function.

kXXX ,, 21

0),,(,,1 kXXf

k

x x

kXXkXX duduuufxxFk

kk 11,,1,,

1

11),,(),,(

),,( 21 kxxx

),,(,,1 kXXf


0),,( 1,,1kXX xxf

k

1),,( 11,,1

kkXX dxdxxxfk

],,,[ 222111 kkk bXabXabXaP

k

b

a

b

a kXX dxdxxxfk

kk

11,,

1

11

),,(


Marginal continuous probability density function

If X and Y are bivariate joint continuous random variables, then and are called marginal probability density functions.

)(Xf )(Yf

dyyxfxf XYX ),()(

dxyxfyf XYY ),()(


Conditional distribution functions for discrete random variables

• If X and Y are bivariate joint discrete random variables with joint discrete density function

, then the conditional discrete density function of Y given X=x, denoted by

or , is defined to be

),( XYf

)|(| xf XY )(| xXYf

)(

),()|(| xf

yxfxyf

X

XYXY


}:{

|| )|(]|[)|(yyj

jXYXY

j

xyfxXyYPxyF


Conditional distribution functions for continuous random variables

• If X and Y are bivariate joint continuous random variables with joint continuous density function , then the conditional probability density function of Y given X=x, denoted by or , is defined to be

),( XYf

)|(| xf XY )(| xXYf

)(

),()|(| xf

yxfxyf

X

XYXY



dyyxfxf

dyxf

yxfdyxyf

XYX

X

XYXY

),()(

1

)(

),()|(|

1)(

)(

xf

xf

X

X



Stochastic independence of random variables


Expectation of function of a k-dimensional discrete random variable



Conditional Expectation

04/21/23 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

18

xY X

XY

X

XYXY

dyxf

yxfyxXYE

xf

yxfxyf

|

|

)(

),(|

)(

),()|(


19

][

)(

),(),(

)()(

),(

]|[|

|

|

|

YE

dyyyf

dydxyxfydxdyyxyf

dxxfdyxf

yxfy

xYEEXYEE

Y

Y

Y X

XY

X xY

XY

X

X xY X

XY

xXYX

)]|([][ XYEEYEX

Expectation of a Random Sum of Random Variables

• Let N be a random variable which can assume positive integer values 1, 2, 3....

• Let Xi be a sequence of independent random variables which are also independent of N and have a common mean E[X] independent of i. Then the expectation of the sum of N Xi’s can be expressed as


20

][][][ XENESE

N

iiXS

1Homework problem

Covariance


YXXYEYXCov ][),(


• If two random variables X and Y are independent, then .0),( YXCov

Therefore,

YXXYEYXCov ][),(

YXYX

YX

XY

dyyyfdxxxf

dxdyyfxxyf

dxdyyxxyfXYE

)()(

)()(

),()(

.0),( YXCov


• However, does not imply that two random variables X and Y are independent.

0),( YXCov


A measure of linear correlation:Pearson coefficient of correlation

YXXY

YXCovYXCorrel

),(

),(

11 XY04/21/23 25Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental

Systems Engineering, National Taiwan Univ.

Covariance and Correlation Coefficient • Suppose we have observed the following data.

We wish to measure both the direction and the strength of the relationship between Y and X.








Examples of joint distributions

• Duration and total depth of storm events. (bivariate gamma, non-causal relation)

• Hours spent for study and test score. (causal relation)


Bivariate Normal Distribution

• Bivariate normal density function

1

2

1

21

2

1)(),(

zz

ZXY

T

ezfyxf


• Conditional normal density

2

22|1

)()(

2

1exp

)1(2

1)|(

Y

XX

YY

Y

XY

xy

xyYf

)(| yf xXY




Bivariate normal simulation I. Using the conditional density


38

2

22|1

)()(

2

1exp

)1(2

1)|(

Y

XX

YY

Y

XY

xy

xyYf



(x,y) scatter plot


Histogram of X


Histogram of Y


Bivariate normal simulation II. Using the PC Transformation





(x,y) scatter plot


Histogram of X


Histogram of Y


Multivariate normal simulation using R

• The mvtnorm package in R • dmvnorm• rmvnorm• pmvnorm• qmvnorm


51

Conceptual illustration of Bivariate gamma simulation


STATISTICS Joint and Conditional Distributions Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering National Taiwan University.

Documents

spatial modeling

remote sensing hydrology

discrete joint density

statistics joint

discrete random variablesif

marginal discrete densityif

dimensional random variable

probability space