II. Characterization of Random Variables. © Tallal Elshabrawy 2 Random Variable Characterizes a random experiment in terms of real numbers Discrete Random.

II. Characterization of Random Variables

© Tallal Elshabrawy 2

Random Variable

Characterizes a random experiment in terms of real numbers

Discrete Random Variables The random variable can only take a finite number of

values

Continuous Random Variables The random variable can take a continuum of values


Probability Mass Function

Only Suitable to characterize discrete random variables

P X x


Cumulative Distribution Function

1 2 1 2

1 2 1 2 2 1

F 0 F 1 0 F 1

F is monotone non-decreasing function

For < F F

For < P =F F

X X X

X

X X

X X

x

x

x x x x

x x x X x x x

F P X x X x

Properties of FX x


Probability Density Function

Used to characterize Continuous Random Variables

2

1

1 2 2 1

1 2 2 1

1 2

df F

df 0

F f d f d 1

P P P

P F F

P f d

X X

X

x

X X X

X X

x

X

x

x xx

x

x y y x x

x X x X x X x

x X x x x

x X x x x


Uniform Random Variable

xf x

a b

a b

1

0

1f

0

0

F

1

X

X

x a

x a x bb a

x b

x a

x ax a x b

b ax b

fX x

FX x

1

b a


Gaussian Random VariableMany physical phenomenon can be modeled as Gaussian Random Variables most popular to communication engineers is … AWGN Channels

2

22 2

2

1~N , f

2

x m

XX m x e

mean

standard deviation

2

2

2

2

2

2

1F d 1 Q

2where

1Q d

2

y mx

X

t

x

x mx e y

x e t


Exponential Random Variable

Commonly encountered in the study of queuing systems

1f 0

x

bX x e x

b

F 1 0

x

bX x e x


How to Characterize a Distribution

Client: Tell me how good is your network?

Salesman: Well, P(Delay<1)=0.1, P(Delay<2)=0.3,P(Delay<3)=0.2,……

Client: HmmmSo what does this really mean?

Salesman: How can I explain this?


Mean of Random Variables

Client: Tell me how good is your network?

Salesman: Well, The average delay per packet is 1 sec

Client: HmmmSo what does this really mean?

Salesman: If you need to send 100 packets, they will most likely take 100 seconds


Mean of a Random Variable

E P X x X x

E f d

XX x x x

Discrete Random Variable

Continuous Random Variable


Consider a Network where the delay ‘D’ is either 1 or 5 secondsi.e., P[D = 1] = 0.3, P[D =5] = 0.7

P[D = 0, 2, 3, 4] = 0, P[D = 6, 7, 8, 9, …] = 0

What is the mean delay? Let assume 100 packets, then most likely

30 packets will be delayed for 1 sec

70 packets will be delayed for 5 sec Therefore 100 packets will most likely take 30x1+70x5 = 380 sec Average Delay = 380/100 = 3.8 sec E[D] = 1xP[D=1]+5xP[D=5] = 3.8 sec

Example


Moments of a Random Variable



E P n nX x X x

E f d

n nXX x x x


Central Moments

E E E P n n

X X x X X x

E E E f d

n n

XX X X X x x




Variance

Variance is a measure of random variable’s randomness around its mean value

2

22

Var =E E

Var =E E

X X X

X X X

Std = VarX X


Conditional CDF

Define FX|A[x] as the conditional cumulative distribution function of the random variable X conditioned on the occurrence of the event A, then

P ,F P

P

X A

X x Ax X x A

A

P ,P

P

A BB A

A

Remember Bayes’s Rule

1 2 1 2

1 2 1 2 2 1

F 0 F 1 0 F 1

F is monotone non-decreasing function

For < F F

For < P =F F

X A X A X A

X A

X A X A

X A X A

x

x

x x x x

x x x X x A x x

Properties of FX A x


Conditional CDF: Example

Consider a uniformly distributed random variable X with CDF

0 0

F 0 1

1 1

X

x

x x x

x

Calculate the conditional CDF of X given that X<1/2. In other words we would like to compute FX|X<1/2[x]

x

FX[x]

0 1

1

1 2

P , 1 2F P 1 2

P 1 2

X X

X x Xx X x X

X

1 2

0 P , 1 2 0

F 0 0

X X

x X x X

x x



0 0

F 0 1

1 1

X

x

x x x

x


x

FX[x]

0 1

1

1 2

P , 1 2F P 1 2

P 1 2

X X

X x Xx X x X

X

1 2

0 1 2 P , 1 2 P

PF 2 0 1 2

P 1 2

X X

x X x X X x

X xx x x

X




0 0

F 0 1

1 1

X

x

x x x

x


x

FX[x]

0 1

1

1 2

P , 1 2F P 1 2

P 1 2

X X

X x Xx X x X

X

1 2

1 2 P , 1 2 P 1 2

P 1 2F 1 1 2

P 1 2

X X

x X x X X

Xx x

X




0 0

F 0 1

1 1

X

x

x x x

x


x

FX[x]

0 1

1

1 2

0 0

F 2 0 1 2

1 1 2

X X

x

x x x

x x

FX[x]

0 1/2

1



Exercise

For some random variable X and given constants a, b such that a<b

0

F FF

F F

1

X XX a X b

X X

x a

x ax a x b

b a

x b

P ,F P

P

X a X b

X x a X bx X x a X b

a X b


Conditional PDF

Define fX|A[x] as the conditional probability density function of the random variable X conditioned on the occurrence of the event A, then

df F

dX A X Ax x

x

2

1

1 2

f 0

F f d f d 1

P f d

X A

x

X A X A X A

x

X Ax

x

x y y x x

x X x A x x

Properties of fX A x


Conditional PDF: Example


0 0

f 1 0 1

0 1

X

x

x x

x

Calculate the conditional PDF of X given that X<1/2. In other words we would like to compute fX|X<1/2[x]

x

fX[x]

0 1

1

x

fX|X<1/2[x]

1

2

1 2

0 0

f 2 0 1 2

0 1 2

X X

x

x x

x

1/2


Exercise

For some random variable X and given constants a, b such that a<b

0

F FF

F F

1

X XX a X b

X X

x a

x ax a x b

b a

x b

df F

d X a X b X a X bx xx

0

f ff

F F P

0

X XX a X b

X X

x a

x xx a x b

b a a x b

x b


Conditioning on a Characteristic of Experiment

Conditioning does not necessarily have to be on the numerical outcome of an experiment

It is possible to have qualitative conditioning based on a characteristic of an experiment

Example: Consider a random variable X that represents the score of students in a given course Conditioning based on experiment outcome

The distribution of grades given it is greater than 80% (i.e., FX|X>80[x])

Conditioning based on experiment characteristic The distribution of grades given the gender of students (i.e., FX|M[x])



Consider a set of N mutually exclusive events A1, A2,…, AN. Suppose we know FX|An[x] for n=1, 2, …, N. Then

1

1

1

F P P P

F P F P

f f P

n

n

N

X n nn

N

X nX An

N

X nX An

x X x X x A A

x X x x A

x x A

The unconditional CDF/PDF is basically the conditioned CDF averaged across the probability of occurrence of conditioning events

Example:

For a bit b sent over a communication channel and the received voltage r

P[r<0]=P[r<0|b=1]*P[b=1]+P[r<0|b=0]*P[b=0]



Consider a set of N mutually exclusive events A1, A2,…, AN. Suppose we know FX|An[x] for n=1, 2, …, N. Then

P PP

P

n n

n

X x A AA X x

X xDiscrete Random Variable

For a continuous random variable P [X=x|An]=0, P [X=x]=0 resulting in an undetermined expression



Consider a set of N mutually exclusive events A1, A2,…, AN. Suppose we know FX|An[x] for n=1, 2, …, N. Then for a continuous random variable

P PP

P

0

P f P f

f P f PP

f f

0

f PP

f

n

n n

n

n nn

n XX A

n nX A X A

nX X

nX A

nX

x X x A AA x X x

x X x

x X x A x x X x x

x A x AA x X x

x x

x AA X x

x


Conditional Expected Value

The expected value of a random variable X conditioned on some event A

E P X AX A x x Discrete Random Variable

E f d

X AX A x x x Continuous Random Variable

II. Characterization of Random Variables. © Tallal Elshabrawy 2 Random Variable Characterizes a random experiment in terms of real numbers Discrete Random.

Documents

tallal elshabrawy

random experiment

gaussian random variables

uniform random variable

exponential random variable

standard deviation slide

probability density

probability mass function