L05 Binary Choice Models CIE555 0205&10

8/9/2019 L05 Binary Choice Models CIE555 0205&10

1/36

CIE 555: Discrete Choice Analysis

Instructor: Qian Wang

02/05&02/10, 2015CIE 555: Discrete Choice Analysis L051

Lecture 5 Binary Choice Models


2/36

Outline

2

Binary Choices

Binary Choice Models

Probability of A Choice

Estimation of Binary Choice Models


3/36

An Example of Binary Choices

EZ-Pass payment methods

Choice problem: choose toll payment methods

given the time of day pricing

Decision makers: travelers

Choice alternatives

EZ-Pass vs. Cash

Choice rule: utility

maximization

3

http://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_crossbay_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_verrazano_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_goethals_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_batterytunnel_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_lincoln_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_throgs_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_triborough_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_henryhudson_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_lincoln_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_throgs_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_triborough_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_henryhudson_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_triborough_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_henryhudson_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_henryhudson_on.gifhttp://localhost/var/www/apps/Class%20work/multivariate%20analysis/projects/document.changer','document.changer','images/map/nycrolls/roll_henryhudson_on.gif


4/36

Binary Choices

Choice set: C n={alter. 1, alter. 2}

Choice tree

Examples Travel decisions: travel or not

Payment types: paying tolls by E-ZPass or cash

4

Alter. 1 Alter. 2

Travel decision


5/36

Binary Choice Models Utility function

Systematic component

Where: = the coefficient of independent variable k ;

= independent variable k for alternative i perceived by

decision maker n.5

nnn V U 111

Systematic utility

Error

nnn V U 222

Alter. 1:

Alter. 2:

Kn K nnn x x xV 112211101 ... Alter. 1:

Kn K nnn x x xV 22222112 '...'' Alter. 2:

', k k

ikn x


6/36

Notations of the Parameters Conceptually: the parameters should vary by

alternatives and decision makers

Ways to deal with the variations of parameters:

Regarding the variations by alternatives:

Try the same parameters at first; if the variable prove tobe significant, vary the parameters by alternatives

Regarding the variations by decision makers:

Split the whole population to several homogeneous

groups, and then specify the choice models for each

group; OR Treat the parameters as random variables that vary

across individuals (random parameter utility functions)

6


7/36

Error Terms

We assume the distributions of the error terms inorder to calculate the probability of a choice

7


8/36

Probability of A Binary Choice

The probability of choosing alternative i (i =1 or 2)

8

}]2,1{,Pr[

}]2,1{,Pr[)|(

jV V

jV V C i P

jninin jn

jn jnininn

Let: in jnn

We get:

}]2,1{,Pr[)|( jV V C i P jninnn

Its distribution is the key to

calculate the probability


9/36

Linear Probability Model

Assuming the difference of errors follows anuniform distribution

9

-L L

Density function of the error

difference

in jnn

L2

1

:)( n f

L L L

Lor L

f n

nn

n

,2

1

,0

)(

0


10/36

Linear Probability Model (Cont.)

The probability of choosing alter. i :

10

jnin V V

L

jnin

jnin

jnin

jnin

nnn

LV V

LV V L L

LV V

LV V

d f i P

,1

,2

,0

)()(

-L L

Probability of choosing alter. i

jnin V V

:)(i pn

0.5

1

0

045


11/36

Linear Probability Model (Cont.)

The value of L:

The probability only depends on the ratio of the

utility difference to L but not L

We can arbitrarily set L=1/211

LV V L L

LV V i p jnin

jnin

n

,2

)(

Let: 11, LV V jnin

We get:

11,

2

1

2

)(

L

L Li pn


12/36

Binary Probit Model

In reality, few cases match the uniformdistributions

Since the error terms represent the combination

of various sources of randomness, by the central

limit theorem, their distributions would tend to be

normal

Note: central limit theorem states that the re-

averaged sum of a sufficiently large number of

identically distributed independent random

variables, each with finite mean and variance, will

be approximately normally distributed

12


13/36

Binary Probit Model (Cont.)

Assuming the difference of errors follows annormal distribution with mean as zero and

variance as

13

]2

1exp[

2

1)(

2

nn f

)( n f (Density function)

0n

2


14/36

Binary Probit Model


14

)(

]2

1exp[

2

1

]2

1exp[

2

1

)()(

/)(2

2

jnin

V V

V V

n

V V

nnn

V V

d

d

d f i P

jnin

jnin

jnin

Standardized cumulative normal

distribution function


15/36

Binary Probit Model

The probability curve

The value of :

We choose15


jnin V V

:)(i pn

0.5

1

1


16/36

Binary Logit Model

Assuming the difference of errors follows anlogistic distribution

It is “probitlike”: it resembles the normal

distribution in shape but has heavier tails

It is analytically convenient

16


17/36

Binary Logit Model (Cont.)

The probability density function

17

nnn

n

e

e f

,0,)1(

)(2

Where:= a positive scale

parameter

s = set as 1 for the binary

logit case


18/36



18

jnin

in

jnin

jnin

n

n

jnin

V V

V

V V

V V n

V V

nnn

ee

e

e

d ee

d f i P

)(

2

1

1

)1(

)()(


19/36


The probability curve

19


jnin V V

:)(i pn

0.5

1


20/36


Replace V by the utility functions

This indicates that the scale parameter ( )

cannot be distinguished from the overall scales of

parameters ( ) For convenience, we assume that

Furthermore, this implies that

20

)(

)(

1

1

1

1)(

jnin

jnin

X X

V V n

e

ei P

1

3/)var( 2 n


21/36

Extreme Cases of the Linear, Probit

and Logit Models

21


jnin V V

:)(i pn

0.5

1 0,0, L

L,,0


22/36

Other Binary Choice Models

Arctan probability model

Right-truncated exponential model

Left-truncated exponential model

Reference: Section 4.2 of the textbook (page 73)

22


23/36

Estimation of Binary Choice Models Model estimation procedure

23

Specifying utility functional

forms

Estimating parameters

Validation

Application (Forecasting)

Calibration:


24/36

Methods to Estimate Parameters

Maximum likelihood estimation Likelihood: the probability that the whole

population/sample of decision makers make certain

choices

Methodology: find the best parameter values thatmaximize the logarithm likelihood

24


25/36

Maximum Likelihood Estimation Likelihood function:

The log likelihood becomes

25

)(

1

2

1,21

)ln(

)ln()...,,(

i N n

ni

N

n i nini K

p

p y LL

N

n i

y

ni K ni p L

1

2

1

,21 )...,,(

y ni =1 if decision maker n

chooses alternative i ; 0,

otherwise

Probability for decisionmaker n to choose

alternative i

The group of decision makers

who chose alternative i


26/36

Maximum Likelihood Estimation

Step 2: Find the optimal solutions of theparameters by maximizing the likelihood function

Step 2.1: Find the POTENTIAL optimal solutions by

solving the first-order conditions (first-order

derivatives):

26

)...,,(max ,21 K LL

Given unknown variables: K ,...,, 10

K k i p

i p y

LL N

n i n

k nin

k

,...,0,0)(

/)(

1

2

1


27/36

Maximum Likelihood Estimation Step 2.2: Pick the optimal solution by checking the

Hessian matrix of the likelihood function (second-

order condition)

If it is a negative semidefinite matrix: the likelihood function

is concave with a UNIQUE optimal solution

If not: calculate the likelihood value for each solution from

the first-order condition, and choose the solution resulting in

the maximum value of likelihood

27

][

2

l k

LL Hessian


28/36

Example: Estimation of Binary

Logit Models

In the binary Logit Models, the probabilities are:

Step 1: the log likelihood function

28

jnin

jn

jnin

in

X X

X

n X X

X

nee

e j p

ee

ei p

''

'

''

'

)(,)(

N

n X X

X

in X X

X

in

K

jnin

jn

jnin

in

eee y

eee y

LL

1''

'

''

'

,21

)]log()1()log([

)...,,(


29/36


Logit Models (Cont.)

Step 2: Maximum likelihood problem

Step 2.1: Find the POTENTIAL optimal solutions by

solving the first-order conditions (first-order

derivatives):

29

)...,,(max ,21 K LL

Given unknown variables: K ,...,, 10

K k xi p y LL

N

n

nk nin

k

,...,1,0)]([1

f


30/36


Logit Models (Cont.) Step 2.2: Pick the optimal solution by checking the

Hessian matrix of the likelihood function (second-

order derivatives)

The Hessian matrix is negative semidefinite (check page 85

in the textbook to see why) the solution from the first-

order condition is the ONLY optimal solution

30

N

nnl nk nn

l k x xi pi p

LL

1

2

))(1)((

P i D i d f h Fi


31/36

Properties Derived from the First-

Order Conditions

For the derivatives corresponding to the constantterm:

31

0)]([1

0

0

N

n

nnin xi p y LL

=1

}2.,1.{,)(

0)]([

11

1

alter alter ii p y

i p y

N

n

n

N

n

in

N

n

nin

Which is equivalent to:

Property 1

Where: N = the total number of the observations in the full sample

P ti D i d f th Fi t


32/36

Properties Derived from the First-

Order Conditions (Cont.)

For the derivatives corresponding to thealternative-specific dummy variables:

32

}''{,0)]([1

K k xi p y LL N

n

nk nin

k

'

1

'

1

'

1

)(

}''{,0)]([

N

n

n

N

n

in

N

n

nin

i p y

K k i p y

Which is equivalent to:

Property 2

0

1

nk x

For a subset of the sample whose x nk equal 1

For the remainder of the sample

Where: N’ = the number of observations in the subset of the sample

C t ti l A t f


33/36

Computational Aspects of

Maximum Likelihood

The parameters are estimated by solving a groupof NOLINEAR equations from the first-order

conditions

Iterative procedures are required to obtain thesolutions, e.g., the Newton-Raphson algorithm

(check the page 82 in the textbook)

33

K k xi p y

LL N

nnk nin

k ,...,1,0)]([1

Nonlinear functions of the unknown parameters


34/36

Learning Outcomes

What is a binary (binomial) choice situation What are the three typical choice models to deal

with the situations, and how they are derived

based on what assumptions

Binary-uniform

Binary-logit

Binary-probit

Be able to apply the appropriate method to deal

with a real-world binary choice problem

34


35/36

References

35

Chapter 4 of the text book Holguín-Veras, J., & Wang, Q. (2011). Behavioral

investigation on the factors that determine

adoption of an electronic toll collection system:

Freight carriers. Transportation research part C:Emerging technologies, 19(4), 593-605.


36/36

Next Class

36

Lab 1: use LIMDEP to estimate binary choicemodels

Bring your laptop with the installed software to the

class

Lab 1 assignment will be given

L05 Binary Choice Models CIE555 0205&10

Documents