Review of Probability Concepts ECON 4550 Econometrics Memorial University of Newfoundland Adapted from Vera Tabakova’s notes SECOND.

ECON 4550Econometrics Memorial University of Newfoundland

Adapted from Vera Tabakova’s notes

SECOND

B.4.1 Mean, median and mode

For a discrete random variable the expected value is:

1 1 2 2[ ] n nE X x P X x x P X x x P X x

1 1 2 2

1

[ ] ( ) ( ) ( )

( ) ( )

n n

n

i ii x

E X x f x x f x x f x

x f x xf x

Where f is the discrete PDF of x

For a continuous random variable the expected value is:

The mean has a flaw as a measure of the center of a probability distribution in that it can be pulled by extreme values.

E X xf x dx

For a continuous distribution the median of X is the value m such that

In symmetric distributions, like the familiar “bell-shaped curve” of the normal distribution, the mean and median are equal.

The mode is the value of X at which the pdf is highest.

( ) .5P X m P X m

[ ( )] ( ) ( )x

E g X g x f x

E aX aE X

E g X g x f x axf x a xf x aE X

Where g is any function of x, in particular;

E aX b aE X b

1 2 1 2E g X g X E g X E g X

The variance of a discrete or continuous random variable X is the expected value of

2g X X E X

The variance

The variance of a random variable is important in characterizing the

scale of measurement, and the spread of the probability distribution.

Algebraically, letting E(X) = μ,

22 2 2var( ) [ ]X E X E X

The variance of a constant is?

Figure B.3 Distributions with different variances

2var( ) var( )aX b a X

2 2

2 22 2

var( )

var

aX b E aX b E aX b E aX b a b

E a X a E X a X

3

3

4

4

E Xskewness

E Xkurtosis

[ ( , )] ( , ) ( , )x y

E g X Y g x y f x y

( ) ( )E X Y E X E Y

, , ,

, ,

x y x y x y

x y y x x y

E X Y x y f x y xf x y yf x y

x f x y y f x y xf x yf y

E X E Y

( ) ( ) ( )E aX bY c aE X bE Y c

, ,

if and are independent.

x y x y

x y

E XY E g X Y xyf x y xyf x f y

xf x yf y E X E Y X Y

( , ) ( )( )X Yg X Y X Y

Figure B.4 Correlated data

If X and Y are independent random variables then the covariance and

correlation between them are zero. The converse of this relationship

is not true.

cov( , ) XY X Y X YX Y E X Y E XY

cov ,

var( ) var( )XY

X Y

X Y

X Y

Covariance and correlation coefficient

The correlation coefficient is a measure of linear correlation between the variables

Its values range from -1 (perfect negative correlation) and 1 (perfect positive correlation)

cov ,

var( ) var( )XY

X Y

X Y

X Y

Covariance and correlation coefficient

If a and b are constants then:

2 2var var( ) var( ) 2 cov( , )aX bY a X b Y ab X Y

var var( ) var( ) 2cov( , )X Y X Y X Y

var var( ) var( ) 2cov( , )X Y X Y X Y

If a and b are constants then:

varX Y varX varY 2 x y

So:

Why is that? (and of course the same happens for the caseof var(X-Y))

var var( ) var( ) 2cov( , )X Y X Y X Y

If X and Y are independent then:

2 2var var( ) var( )aX bY a X b Y

var var( ) var( )X Y X Y

var var var varX Y Z X Y Z

If X and Y are independent then:

var var var varX Y Z X Y Z

Otherwise this expression would have to include all the doubling of each of the (non-zero) pairwise covariances between variables as summands as well

4

1

1 .1 2 .2 3 .3 4 .4 3 Xx

E X xf x

22

2 2 2 21 3 .1 2 3 .2 3 3 .3 4 3 .4

4 .1 1 .2 0 .3 1 .4

1

X XE X

B.5.1 The Normal Distribution

If X is a normally distributed random variable with mean μ and

variance σ2, it can be symbolized as 2~ , .X N

2

22

1 ( )( ) exp ,

22

xf x x

Figure B.5a Normal Probability Density Functions with Means μ and Variance 1

Figure B.5b Normal Probability Density Functions with Mean 0 and Variance σ2

A standard normal random variable is one that has a normal

probability density function with mean 0 and variance 1.

The cdf for the standardized normal variable Z is

~ (0,1)X

Z N

( ) .z P Z z

[ ]X a a a

P X a P P Z

[ ] 1X a a a

P X a P P Z

[ ]a b b a

P a X b P Z

A weighted sum of normal random variables has a normal

distribution.

21 1 1

22 2 2

~ ,

~ ,

X N

X N

2 2 2 2 21 1 2 2 1 1 2 2 1 1 2 2 1 2 12~ , 2Y YY a X a X N a a a a a a

2 2 2 2

1 2 ( )~m mV Z Z Z

2( )

2( )

[ ]

var[ ] var 2

m

m

E V E m

V m

Figure B.6 The chi-square distribution

A “t” random variable (no upper case) is formed by dividing a

standard normal random variable by the square root of an

independent chi-square random variable, , that has been

divided by its degrees of freedom m.

( )~ m

Zt t

Vm

~ 0,1Z N

2( )~ mV

Figure B.7 The standard normal and t(3) probability density functions

An F random variable is formed by the ratio of two independent chi-

square random variables that have been divided by their degrees of

freedom.

1 2

1 1( , )

2 2

~ m m

V mF F

V m

Figure B.8 The probability density function of an F random variable

Slide B-35Principles of Econometrics, 3rd Edition