Statistics for Financial Engineering
Part1: ProbabilityInstructor: Youngju Lee
MFE, Haas Business SchoolUniversity of California, Berkeley
Overview of Class
Part1: Probability – March 23rd, 2006 Part2: Statistics – March 25th, 2006 Class will be organized as
Definitions Some comments about from definition Problems Applications in financial engineering – I will give short
examples how I apply these concepts in my real life and practice since I assume you do not have any idea about financial engineering as of now.
Probability
1. Probability
2. Random Variables – Discrete and Continuous
3. Distribution and Probability Density
4. Moments and Moments Generating Function
5. Stochastic Independence
6. Basic Limit Theorem
Probability
Definition1
A probability function (P) is a function which assigns to each event A a number denoted by P(A), called the probability of A and satisfies the following requirements. a) P is non-negative; P(A) 0 for every event A b) P us normed: that is P(S) = 1 c) P is additive: that is, for every collection of pairwise(or mutually) disjoint events, we
have P( jj
A ) = ( )jj
P A
c-1) P is finite additive
Probability
Some consequences of definition 1
a) P(0) = 0
b) P(n
jj
A ) = ( )n
jj
P A
c) P( cA ) = 1 – P(A) d) 1 2 1 2, ( ) ( )A A P A P A
e) 0 ( ) 1P A
f) 1 2 1 2 1 2( ) ( ) ( ) ( )P A A P A P A P A A
g) P is sub additive: 11
( ) ( )j j jj
P A P A
Probability
Try this
Twenty balls numbered from 1 to 20 are mixed in an urn and two balls are drawn successively and without replacement. If 1x and 2x are the numbers written on the first and second ball drawn,
respectively, what is the probability that
i. 1 2 8x x
ii. 1 2 5x x
Probability
In finance world?
This is the very basic concept of everything. – States, Monte-Carlo simulations and Binomial Trees, etc.
Conditional Probability
Definition 2
Let A be an event such that ( ) 0P A . Then the conditional probability, given A, is the function denoted by ( | )P A and defined for every event B as follows:
( )( | )
( )
P A BP B A
P A
( | )P B A is called the conditional probability of B given A.
Conditional Probability
Some consequences from definition 2
a) (Multiplicative Theorem) Let , 1, 2,3...,jA j n be events such that 11( ) 0n
j jP A then 11 1 2 1 1 1 2 2 2 1 1( ) ( | ) ( | ) ( | ) ( )n
j j n n n nP A P A A A A P A A A A P A A P A
b) (The total probability theorem) Let , 1, 2,3...,jA j n be a partition of S with
( ) 0jP A , all j. Then for B A , we have ( ) ( | ) ( )j jj
P B P B A P A
c) (Bayes Fomular) If , 1, 2,3...,jA j n is a partition of S and ( ) 0jP A and if
( ) 0P B then ( | ) ( )
( | ) 0( | ) ( )
j jj
i ii
P B A P AP A B
P B A P A
Conditional Probability
Try this.
Suppose that a test for diagnosing a certain heart disease is 95% accurate when applied to both those who have the disease and those who do not. If it is known that 5 of 1000 in a certain population have the disease in question, compute the probability that a patient actually has the disease if the test indicates that he does. (try to explain by intuitive reasoning.)
Conditional Probability
In finance world?
Fancy empirical model – Regime Switch Model
Independence
Definition 3
The events A and B are said to be independent if ( ) ( ) ( )P A B P A P B .
Definition4: The events , 1, 2,...,jA j n are said to be mutually or completely independent
if the following relationships hold. 1 2 1( ... ) ( )... ( )n nP A A A P A P A .
Independence
Some consequences from definition 3
a) If the events 1 2, ,... nA A A are independent, so are the events ' ' '1 2, ,... ,nA A A where '
jA is CjA or jA , j = 1,2,…n.
Independence
Try this. – Easy!
Six fair dice are tossed once. What is the probability that all six faces appear?
Seven fair dice are tossed once. What is the probability that every face appears at least once?
Independence
In finance world?
Is there any independent event in the financial world or at least in practice?
Random Variables
Definition 4
Random variable is a function which assigns to each sample point s S a real number, the value of the r.v. at s.
Discrete Random Variable
Definition 5: Binomial distribution is associated with binomial experiments – success or fail
( ) {0,1,2..., }
( ) ( )
0 1, 1
x n x
X S n
nP X x f x p q
x
p q p
Discrete Random Variables
Definition 6: Poisson distribution
( ) ( )!
x
P X x f x ex
Discrete Random Variables
Definition 7: Discrete uniform distribution
( ) {0,1,2..., 1}
1( ) ( )
0,1,2...
X S n
P X x f xn
x n
Discrete Random Variables
Definition 8: Hyper-geometric distribution
( ) {0,1,2..., }
( ) ( )
X S r
m n
x r xP X x f x
m n
r
Discrete Random Variables
Definition 9: Negative binomial distribution
( ) {0,1,2...}
1( ) ( )
0 1, 1 , 0,1,2....
r x
X S
r xP X x f x p q
x
p q p x
Discrete Random Variables
Definition 10: Multi-nominal distribution
'1 2
1
1 21 1 1
1 2
1
( ) { ( , ,...., ) : 0, 0,1, 2... , }
( ) ( ) ...! !... !
1
k
k j jj
r x x xk
k
k
jj
X S x x x x x j k x n
nP X x f x p p p p
x x x
p
Continuous Random Variables
Definition 11: Normal distribution
Rx
xxfRSX
,
2exp
2
1)(,)(
2
2
Continuous Random Variables
Some consequences from definition 11
Normal distribution is symmetric. Normal distribution has maximum value at mean.
Continuous Random Variables
Try this.
Let X to be distributed as N(0,1) and for a < b, let p P a X b . Then use the
symmetry of the p.d.f. f in order to show that:
1. For 0 ,a b p b a
2. For 0 , 1a b p b a
3. For 0 ,a b p a b
4. For 0, 2 1c P c X c c
Continuous Random Variables
In finance world?
Everything is assumed normal distribution in financial engineering. To check normality,
Use K-S test or Normal Probability Plot. I will cover this later.
Continuous Random Variables
Definition 12: Gamma distribution
0,0
0,0
0,1
)(
),0()(
1
x
xexxf
SXx
Where
0
1 dyey y . This integral is known as the Gamma function.
Continuous Random Variables
Definition 13: Chi-square distribution
0
0,22/1
1)( 212/
21
r
xexr
xfxr
Continuous Random Variables
Definition 14: Negative exponential distribution
0,0,0
0,)(
x
xexf
x
Continuous Random Variables
Definition 15: Continuous uniform distribution
else
xxf
RSX
,0
,1)(
)(
Continuous Random Variables
Definition 16: Beta distribution
0,0
,0
10,1)(
)(
11
else
xxxxf
RSX
Continuous Random Variables
Definition 17: Cauchy distribution
0,,
1)(
)(
22
RRx
xxf
RSX
Continuous Random Variables
Definition 18: Lognormal distribution
0,0
0,2
loglogexp
2
1)(
)(
2
2
x
xx
xxf
RSX
Continuous Random Variables
Definition 19: Bi-variate normal distribution
2
2
22
2
22
1
11
2
1
112
2
221
21
21
1
12
1,
xxxxq
exxfq
D.F. and P.D.F.
Definition 20: The distribution function
The distribution function F of a random variable X satisfies the following properties.
RxxF ,1)(0
F is non-decreasing F is continuous from the right
xxF
xxF
,1)(
,0)(
D.F. and P.D.F.
Some consequences from definition 20
Let X be an ),( 2N distributed r.v. and set
xY . Then Y is an r.v. and its
distribution is )1,0(N
Let X be an )1,0(N distributed r.v. then 2XY is distributed as .21
Let X be a ),( 2N distributed r.v. then the r.v. 2
x
is distributed as .21
D.F. and P.D.F.
Try this. – It is better to know what logistic distribution is.
Show that the following function F is a d.f. (Logistic distribution) and derive the corresponding p.d.f., f.
( )
1( ) , , 0,
1 xF x x R R
e
D.F. and P.D.F.
In finance world?
You probably want to remember some consequences from last slide. We use this all the time to make trading signals.
D.F. and P.D.F.
Definition 21: Joint distribution function
The joint distribution function of 1 2,X X X is 1 2 1 1 2 2, ,F X X P X x X x
D.F. and P.D.F.
Definition 22: Quantile of a distribution
Let X be an r.v. with d.f. F and consider a number p such that 0 < p < 1. A pth quantile of the r.v.X or of its d.f. F is a number denoted by px and having the following property:
pxXP p and pxXP p 1 .
D.F. and P.D.F.
Definition 23: Mode
Let X be an r.v. with p.d.f. f. Then a mode of f, if it exists, is any number which maximize f(x).
D.F. and P.D.F.
Try this.
Let X be an r.v. with p.d.f. f symmetric about a constant c then show c is a median of f.
D.F. and P.D.F.
In finance world?
Moments
Definition 24: Moments of random variables
For n=1,2,…, the nth moment of g(x) is denoted by nxgE )( and is defined by
nn
nn
x
n
n
dxdxdxxxxfxxxg
xfxg
xgE...)...,()...,(
)()(
)(212121
The first moment is called mean and the difference between the second moment and the square of the first moment is called variance.
Moments
Some consequences from definition 24The basic properties of mean
|)(||)]([|
)()(,
])([])([
)(
XgEXgE
YEXEYX
dxgcEdxcgE
ccE
The basic properties of variance
222
222
2
)]([])([)]([
)]([])([
0
XgEXgEXg
XgcdXcg
c
Moments
Try this.
A roulette wheel has 38 slots of which 18 are red, 18 black, and 2 green. Suppose a gambler is placing a bet of $M on red. What is the gambler’s expected gain or loss and what is the standard deviation?
Moments
Try this. But do not calculate!
Let X be an r.v. taking on the values -2,-1,1,2 each with probability 0.25. Set Y=X*X and compute the following quantities. EX, Var(X), EY and Var(Y).
Moments
In finance world?
I do not think you can be in finance industry without talking about Sharpe ratio a lot. (mean/sd)
We also need to look at skewness and kurtosis.
Stochastic Independence
Definition 25: Stochastic independence
The r.v.’s ,...2,1, jX j are said to be independent if, for sets kjRB j ,...2,1, , it holds
)(,...,1, 1 jjkjjj BXPkjBXP
Stochastic Independence
Some consequences from definition 25
Let 21 , XX have the bivariate normal distribution. Then 21 , XX are independent if and only if they are uncorrelated.
Let jX be ),( pnB j , j=1,2,…,k and independent. Then
k
jjXX
1
is B(n,p), where
k
jjnn
1
.
Let jX be )( jP , j=1,2,…,k and independent. Then
k
jjXX
1
is )(P , where
k
jj
1
Stochastic Independence
Some consequences from definition 25
Let jX be ),( 2jjN , j=1,2,…,k and independent. Then
k
jjXX
1
is ),( 2N , where
k
jj
1
and
k
j
j
1
22
Let jX be 2
jr , j=1,2,…,k and independent. Then
k
jjXX
1
is 2r where
k
jjrr
1
Let jX be ),( 2
jjN , j=1,2,…,k and independent. Then
22 /kS is 21 k where
k
jj XX
kS
1
22 1
The Central Limit Theorem
Definition 26: Central Limit Theorem
Let 1 2, ... nX X X be i.i.di r.v. with mean and variance 2 . Then
10,1
n
n
n XN
S