c Stanley Chan 2020. All Rights Reserved. ECE 302: Lecture 4.7 Gaussian Random Variable Prof Stanley Chan School of Electrical and Computer Engineering Purdue University 1 / 22
c©Stanley Chan 2020. All Rights Reserved.
ECE 302: Lecture 4.7 Gaussian Random Variable
Prof Stanley Chan
School of Electrical and Computer EngineeringPurdue University
1 / 22
c©Stanley Chan 2020. All Rights Reserved.
Outline
Overall schedule:
Continuous random variables, PDF
CDF
Expectation
Mean, mode, medianCommon random variables
UniformExponentialGaussian
Transformation of random variables
How to generate random numbers
Today’s lecture:
Definition of Gaussian
Mean and variance
Skewness and kurtosis
Origin of Gaussian2 / 22
c©Stanley Chan 2020. All Rights Reserved.
Definition
Definition
Let X be an Gaussian random variable. The PDF of X is
fX (x) =1√
2πσ2e−
(x−µ)2
2σ2 (1)
where (µ, σ2) are parameters of the distribution. We write
X ∼ Gaussian(µ, σ2) or X ∼ N (µ, σ2)
to say that X is drawn from a Gaussian distribution of parameter (µ, σ2).
3 / 22
c©Stanley Chan 2020. All Rights Reserved.
Interpreting the mean and variance
-10 -5 0 5 10
0
0.1
0.2
0.3
0.4
0.5
= -3
= -0.3
= 0
= 1.2
= 4
-10 -5 0 5 10
0
0.1
0.2
0.3
0.4
0.5
= 0.8
= 1
= 2
= 3
= 4
µ changes, σ = 1 µ = 0, σ changes
Figure: A Gaussian random variable with different µ and σ
4 / 22
c©Stanley Chan 2020. All Rights Reserved.
Proving the mean
Theorem
If X ∼ N (µ, σ2), then
E[X ] = µ, and Var[X ] = σ2. (2)
E[X ] =1√
2πσ2
∫ ∞−∞
xe−(x−µ)2
2σ2 dx
(a)=
1√2πσ2
∫ ∞−∞
(y + µ)e−y2
2σ2 dy
=
(b)=
(c)= µ.
5 / 22
c©Stanley Chan 2020. All Rights Reserved.
Proving the variance
Theorem
If X ∼ N (µ, σ2), then
E[X ] = µ, and Var[X ] = σ2. (3)
Var[X ] =1√
2πσ2
∫ ∞−∞
(x − µ)2e−(x−µ)2
2σ2 dx
(a)=
σ2√2π
∫ ∞−∞
y2e−y2
2 dy , by letting y =
=σ2√2π
(−ye−
y2
2
∣∣∣∞−∞
)+
σ2√2π
∫ ∞−∞
e−y2
2 dy
=
= σ2
6 / 22
c©Stanley Chan 2020. All Rights Reserved.
Standard Gaussian PDF
Definition
A standard Gaussian (or standard Normal) random variable X has a PDF
fX (x) =1√2π
e−x2
2 . (4)
That is, X ∼ N (0, 1) is a Gaussian with µ = 0 and σ2 = 1.
Figure: Definition of the CDF of the standard Gaussian Φ(x).
7 / 22
c©Stanley Chan 2020. All Rights Reserved.
Standard Gaussian CDF
Definition
The CDF of the standard Gaussian is defined as the Φ(·) function
Φ(x)def= FX (x) =
1√2π
∫ x
−∞e−
t2
2 dt. (5)
The standard Gaussian’s CDF is related to a so-called error functionwhich is defined as
erf(x) =2√π
∫ x
0e−t
2dt. (6)
It is quite easy to link Φ(x) with erf(x):
Φ(x) =1
2
[1 + erf
(x√2
)], and erf(x) = 2Φ(x
√2)− 1.
8 / 22
c©Stanley Chan 2020. All Rights Reserved.
CDF of arbitrary Gaussian
Theorem (CDF of an arbitrary Gaussian)
Let X ∼ N (µ, σ2). Then,
FX (x) = Φ
(x − µσ
). (7)
We start by expressing FX (x):
FX (x) = .
Substituting y = t−µσ , and using the definition of standard Gaussian, we
have∫ x
−∞
1√2πσ2
e−(t−µ)2
2σ2 dt =
∫ x−µσ
−∞
1√2π
e−y2
2 dy =
9 / 22
c©Stanley Chan 2020. All Rights Reserved.
Other results
P[a < X ≤ b] = Φ
(b − µσ
)− Φ
(a− µσ
). (8)
To see this, note that
P[a < X ≤ b] = P[X ≤ b]− P[X ≤ a] = Φ
(b − µσ
)− Φ
(a− µσ
).
Corollary
Let X ∼ N (µ, σ2). Then, the following results hold:
Φ(y) = 1− Φ(−y).
P[X ≥ b] = 1− Φ(b−µσ
).
P[|X | ≥ b] = 1− Φ(b−µσ
)+ Φ
(−b−µ
σ
)10 / 22
c©Stanley Chan 2020. All Rights Reserved.
Skewness and Kurtosis
Definition
For a random variable X with PDF fX (x), define the following centralmoments as
mean = E[X ]def= µ,
variance = E[(X − µ)2
]def= σ2,
skewness = E
[(X − µσ
)3]
def= γ,
kurtosis = E
[(X − µσ
)4]
def= κ.
11 / 22
c©Stanley Chan 2020. All Rights Reserved.
Skewness
What is skewness?
E[(
X−µσ
)3].
Measures how asymmetrical the distribution is.Gaussian has skewness 0.
0 5 10 15 200
0.1
0.2
0.3
0.4
positive skewness
symmetric
negative skewness
Figure: Skewness of a distribution measures how asymmetric the distribution is.In this example, the skewness are: orange = 0.8943, black = 0, blue = -1.414.
12 / 22
c©Stanley Chan 2020. All Rights Reserved.
Kurtosis
What is kurtosis?
κ = E[(
X−µσ
)4].
Measures how heavy tail is. Gaussian has kurtosis 3.Some people prefer excess kurtosis κ− 3. Gaussian has excesskurtosis 0.
-5 -4 -3 -2 -1 0 1 2 3 4 5
0
0.2
0.4
0.6
0.8
1
kurtosis > 0
kurtosis = 0
kurtosis < 0
Figure: Kurtosis of a distribution measures how heavy tail the distribution is. Inthis example, the (excess) kurtosis are: orange = 2.8567, black = 0, blue =-0.1242. 13 / 22
c©Stanley Chan 2020. All Rights Reserved.
Skewness and Kurtosis
Random variable Mean Variance Skewness Excess kurtosisµ σ2 γ κ− 3
Bernoulli p p(1− p) 1−2p√p(1−p)
11−p + 1
p − 6
Binomial np np(1− p) 1−2p√np(1−p)
6p2−6p+1np(1−p)
Geometric 1p
1−pp2
2−p√1−p
p2−6p+61−p
Poisson λ λ 1√λ
1λ
Uniform a+b2
(b−a)212 0 −6
5Exponential 1
λ1λ2 2 6
Gaussian µ σ2 0 0
Table: The first few moments of commonly used random variables.
14 / 22
c©Stanley Chan 2020. All Rights Reserved.
Example: Titanic
On April 15, 1912, RMS Titanic sank after hitting an iceberg. This haskilled 1502 out of 2224 passengers and crew. A hundred years later, wewant to analyze the data. On https://www.kaggle.com/c/titanic/
there is a dataset collecting the identities, age, gender, etc of thepassengers.
Statistics Group 1 (Died) Group 2 (Survived)
Mean 30.6262 28.3437Standard Deviation 14.1721 14.9510Skewness 0.5835 0.1795Excess Kurtosis 0.2652 -0.0772
15 / 22
c©Stanley Chan 2020. All Rights Reserved.
Example: Titanic
Mean and standard deviation cannot tell the difference.
Skewness and kurtosis can tell the difference.
0 20 40 60 80
age
0
10
20
30
40
0 20 40 60 80
age
0
10
20
30
40
Group 1 (died) Group 2 (survived)
Figure: The Titanic dataset https://www.kaggle.com/c/titanic/.
16 / 22
c©Stanley Chan 2020. All Rights Reserved.
Origin of Gaussian
Where does Gaussian come from?
Why are they so popular?
Why do they have bell shapes?
What is the origin of Gaussian?
When we sum many independent random variables, the resultingrandom variable is a Gaussian.This is known as the Central Limit Theorem. The theorem appliesto any random variable.Summing random variables is equivalent to convolving the PDFs.Convolving PDFs infinitely many times yields the bell shape.
17 / 22
c©Stanley Chan 2020. All Rights Reserved.
The experiment of throwing many dices
(a) X1 (b) X1 + X2
(c) X1 + . . .+ X5 (d) X1 + . . .+ X100
Figure: When adding uniform random variables, the overall distribution isbecoming like a Gaussian.
18 / 22
c©Stanley Chan 2020. All Rights Reserved.
Sum of X and Y = Convolution of fX and fY
Example: Two rectangles to give a triangle:
We will show this result in a later lecture:
(fX ∗ fX )(x) =
∫ ∞−∞
fX (τ)fX (x − τ)dτ.
19 / 22
c©Stanley Chan 2020. All Rights Reserved.
If you convolve infinitely many times
Then in Fourier domain you will have
F {(fX ∗ fX ∗ . . . ∗ fX )} = F{fX} · F{fX} · . . . · F{fX}.
-10 -8 -6 -4 -2 0 2 4 6 8 10-0.5
-0.25
0
0.25
0.5
0.75
1
1.25
(sin x)/x
(sin x)2/x
2
(sin x)3/x
3
Figure: Convolving the PDF of a uniform distribution is equivalent to multiplyingtheir Fourier transforms in the Fourier space. As the number of convolutiongrows, the product is gradually becoming Gaussian.
20 / 22
c©Stanley Chan 2020. All Rights Reserved.
Origin of Gaussian
What happens if you convolve a PDF infinitely many times?
You will get a Gaussian.This is known as the central limit theorem.
Why are Gaussians everywhere?
We seldom look at individual random variables. We often look atthe sum/average.Whenever we have a sum, Central Limit Theorem kicks in.Summing random variables is equivalent to convolving the PDFs.Convolving PDFs infinitely many times yields the bell shape.This result applies to any random variable, as long as they areindependently summed.
21 / 22
c©Stanley Chan 2020. All Rights Reserved.
Questions?
22 / 22