MSH3 Generalized linear model Ch. 6 Count data models Contents §4 Count data model 123 §4.1 Introduction: The Children Ever Born Data ....... 123 §4.2 The Poisson Distribution ................. 124 §4.3 Log-Linear Models ..................... 126 §4.4 ML estimation ....................... 127 §4.5 Goodness of fit and hypothesis test ............ 128 §4.6 Children ever born example ................ 129 §4.6.1 Grouped Data and the Offset ........... 129 §4.6.2 The Deviance Table ................ 130 §4.6.3 The Additive Model ................ 132 §4.7 Distributions with over- and underdispersion ....... 135 §4.7.1 Negative binomial distribution ........... 135 §4.7.2 Generalized Poisson distribution .......... 139 §4.8 Model for excess zero ................... 141 §4.8.1 Zero-inflated Poisson model ............ 142 §4.8.2 Hurdle model .................... 143 §4.9 Poisson mixture model ................... 143 §4.10Model selection example .................. 148 SydU MSH3 GLM (2016) Second semester Dr. J. Chan 122
33
Embed
MSH3 Generalized linear model Ch. 6 Count data …SydU MSH3 GLM (2016) Second semester Dr. J. Chan 122 MSH3 Generalized linear model Ch. 6 Count data models x4 Count data model x4.1
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
MSH3 Generalized linear model Ch. 6 Count data models
Contents
§4 Count data model 123
§4.1 Introduction: The Children Ever Born Data . . . . . . . 123
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 122
MSH3 Generalized linear model Ch. 6 Count data models
§4 Count data model
§4.1 Introduction: The Children Ever Born Data
The following table, adapted from Little (1979), comes from the Fiji
Fertility Survey, typical in the reports of the World Fertility Survey.
Each cell in the table shows the mean, the variance and the number of
observations.
The unit of analysis is the individual woman, the response is the number
of children she has borne, and the 3 discrete predictors are the duration
since her first marriage, the type of place of residence (Suva the capital,
other urban and rural), and her educational level (none, lower primary,
upper primary, and secondary or higher).
Data such as these have traditionally been analyzed using ordinary lin-
ear models with normal errors. The key concern, however, is not the
normality of the errors but rather the assumption of constant variance.
0.0 0.5 1.0 1.5 2.0
−0.
50.
00.
51.
01.
52.
02.
5
log(mean)
log(
var)
Figure 6.1: The Mean-variance Relationship for the CEB Data
The plot of variance against mean in log scale for all cells with at least
20 observation shows clearly that, the assumption of constant variance is
not suitable and the relationship is close to proportional. Thus, a Poisson
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 123
MSH3 Generalized linear model Ch. 6 Count data models
regression models is more suitable than a ordinary linear models.
Table 6.1: Number of Children Ever Born to Women of Indian RaceBy Marital Duration, Type of Place of Residence and Educational Level(Each cell shows the mean, variance and sample size)
Marr. Suva Urban RuralDur. N LP UP S+ N LP UP S+ N LP UP S+
> #BFGS is a quasi-Newton method also known as variable metric algorithm
> OI1<-solve(mreg$hessian)
> se1<-sqrt(diag(OI1))
> par1=mreg$par
> rbind(par1,se1)
[,1] [,2] [,3]
par1 0.04184276 0.28891875 12.59239
se1 0.39073410 0.03941281 11.06966
> AIC1=mreg$value*2+length(par1)*2
> AIC1
[1] 83.76235
> library(MASS) #alternative way using glm.nb
> summary(glm.nb(no~time, link = log))
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.04179 0.34097 0.123 0.902
time 0.28892 0.03308 8.734 <2e-16 ***
---
(Dispersion parameter for Negative Binomial(12.5919) family taken as 1)
AIC: 83.762
Theta: 12.59
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 138
MSH3 Generalized linear model Ch. 6 Count data models
Std. Err.: 9.81
Warning while fitting theta: alternation limit reached
The AIC of 83.762 is lower than 85.358 for Poisson regression in Chapter 1 showingimproved model fit after allowing for model complexity. Equivalently, one mayconsider logistic regression for pi. Then E(Yi) = exp(ln r+xiβ) implying that β1remains unchange and β0 drops by ln r.
§4.7.2 Generalized Poisson distribution
A random variable Y follows a GP(r, λ) distribution if its pmf is given by
fGP (y) =
λ(λ+ ry)y−1 exp[−(λ+ ry)]
y!, y = 0, 1, . . . ;
0, y > s when r < 0
where λ, |r| < 1 and s ≥ 4 is the largest natural number such that λ+ rs > 0 toensure λ + ry > 0 when r < 0. The two parameters r and λ are subject to thefollowing constraints:
1. λ ≥ 0,
2. max
−1,−λ
s
< r < 1.
The mean and variance for Y are given by respectively
E(Y ) =λ
1− rand V ar(Y ) =
λ
(1− r)3=
E(Y )
(1− r)2. (1)
The shape of GP distribution is controlled by r. A negative, zero and positive rindicates underdispersion, equidispersion and overdispersion, respectively.
col = c("red","blue","gray50","gray50","magenta","magenta"),
lty = c(1,1,1,2,1,2),lwd=c(2,2,2,2,2,2),cex=1)
The AIC improves further from 83.762 (NB) to 82.366 (GP) and the estimates arenearly the same as those of Poisson regression. The following diagram comparesall fits for mean and variance.
2 4 6 8 10 12 14
010
2030
40
3 month periods: Jan−March 1983 to April−June 1986
Num
ber
of A
IDS
dea
ths
linear meanPoisson meanNB meanNB varGP meanGP var
linear meanPoisson meanNB meanNB varGP meanGP var
§4.8 Model for excess zero
The model concerns a random event containing excess zero counts.
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 141
MSH3 Generalized linear model Ch. 6 Count data models
§4.8.1 Zero-inflated Poisson model
Proposed by Lambert (1992), the model contains two mixture components fortwo zero generating processes: a degenerate distribution that generates structuralzeros and a Poisson, Negative Binomial or GP distributions that generates counts,some of which may be zero. With a Poisson mixture, it is given by:
Pr(Y = 0) = π + (1− π)e−λ
Pr(Y = y) = (1− π)λye−λ
y!, y ≥ 1
where λ is the expected Poisson count and π is the probability of extra zeros. Anexample is the number of certain insurance claims which is zero-inflated by thosepeople who have not bought any insurance and thus are unable to claim. Themean and variance are
where y is the sample mean and s2 is the sample variance. The maximum likeli-hood estimator for λ can be found by solving the following equation
y(1− e−λ) = λ(
1− n0n
)(2)
for λ where n0n is the observed proportion of zeros. This can be solved by iterations
or using the R module uniroot, and the maximum likelihood estimator for π isgiven below (proofs in assignment):
π = 1− y
λ.
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 142
MSH3 Generalized linear model Ch. 6 Count data models
§4.8.2 Hurdle model
Proposed by Mullahy (1986), the model contains two mixture components: adegenerate model for measuring whether the outcome overcomes a hurdle (say zerofor zero-inflated data) and a truncated model like Poisson or negative binomial(NB) distributions explaining those outcomes that pass the hurdle. The model isgiven by:
Pr(Y = 0) = π
Pr(Y = y) = (1− π)λye−λ
y!(1− e−λ), y ≥ 1
The loglikelihood function is
`(π, λ) = n0 ln π + n1 ln(1− π) + lnλ
n1∑i=1
yi − n1λ−n1∑i=1
yi!− n1 ln(1− e−λ)
where n0 is the count of 0 and n1 is the count of yi > 0. Hence the MLE areπ = n0
n and λ is the solution to (2).
§4.9 Poisson mixture model
Assume that there are m subjects each with n repeated measurement Yij. If
subject i comes from group k at probability πk, k = 1, . . . , G such thatG∑k=1
πk = 1,
then Yij ∼ P (µjk) where µjk = exp(xjβk). Then the observed data loglikelihood
`(y|θ) =m∑i=1
log
[G∑k=1
πk
(n∏j=1
e−µjkµyijjk
yij!
)](3)
where θ = (βT1 , . . . ,βTG, π1, . . . , πG−1) and y is a vector of yij. Define the group
membership Iik = 1 if subject i belongs to group k. Then an estimate of Iik is
Iik =
πk
(n∏j=1
e−µjkµyijjk
yij!
)G∑
k′=1
πk′
(n∏j=1
e−µjk′µyijjk′
yij!)
) . (4)
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 143
MSH3 Generalized linear model Ch. 6 Count data models
Parameters can be estimated using the ML method by maximizing `(y|θ) in (3)with respect to θ applying optim again. Other R package for mixture modelssay flexmix can’t handle group membership for subject (cluster) rather thanindividual observation. Alternatively, one may use the EM method:
E-step: estimate the unobserved group membership Iik in (4) in E-step and
πk =m∑i=1
Iik/n.
M-step: estimate β = (βT1 , . . . ,βTG) using the complete data likelihood
`c(y|β, I) = ln
m∏i=1
G∏k=1
[πk
(n∏j=1
e−µjkµyijjk
yij!
)]Iik=
m∑i=1
G∑k=1
Iik ln πk +m∑i=1
G∑k=1
Iik
[n∑t=1
(−µjk − yij lnµjk − ln yij!)
]where I is a vector of Iik. We consider a Cannabis offences data in Sydney andfit a 2-group mixture model using the ML method.
> ym=read.table("CannabisData2.txt")
> y1 #high
V1 V2 V3 V4 V5 V6 V7 V8
11 7 8 11 16 9 9 3 9
12 22 17 12 20 22 13 19 15
13 18 9 13 16 17 22 17 26
14 18 13 5 17 24 12 9 6
15 47 9 13 15 17 8 14 5
17 18 22 13 16 34 35 5 16
25 22 15 9 18 15 12 8 7
28 30 5 13 10 18 5 3 8
29 30 25 26 19 14 16 13 5
42 9 11 7 26 6 5 3 4
43 5 4 21 5 13 7 6 4
> y2 #low
V1 V2 V3 V4 V5 V6 V7 V8
1 6 2 4 2 3 1 1 2
2 2 1 1 2 0 1 1 2
3 7 5 6 4 7 8 5 5
5 8 3 6 8 8 2 4 1
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 144
MSH3 Generalized linear model Ch. 6 Count data models
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 146
MSH3 Generalized linear model Ch. 6 Count data models
> gp2=mi[ind==0]
> y1=ym[gp1,]
> y2=ym[gp2,]
> ym1=apply(y1,2,mean)
> ym2=apply(y2,2,mean)
> n1=length(gp1)
> n2=length(gp2)
> c(n1,n2)
[1] 11 31
1 2 3 4 5 6 7 8
010
2030
40
time
Num
ber
of c
anna
bis
offe
nces
high fitted meanhigh obs meanhigh obs varlow fitted meanlow obs meanlow obs var
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
0.00
0.05
0.10
0.15
0.20
0.25
t=2t=4t=6t=8
pmf of the 2 groups for time=2,4,6,8
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
0.00
0.05
0.10
0.15
high level grouplow level groupoverall
pmf of the 2 groups and overall for time=1
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 147
MSH3 Generalized linear model Ch. 6 Count data models
§4.10 Model selection example
Maxell (1961) discusses a 5 × 4 table giving the number of boys with 4 differentratings for disturbed dreams in 5 different age groups. The higher the rating themore the boys suffer from disturbed dreams.
RatingAge 4 3 2 15-7 7 3 4 78-9 13 11 15 10
10-11 7 11 9 2312-13 10 12 9 2814-15 3 4 5 32
One way of modeling the data is to let the number in the (i, j)-th cell be Yij andassume that Yij ∼ Poisson(µij).
Given∑5
i=1
∑4j=1 Yij = 223, the joint conditional distribution of the Yij’s is multi-
nomial. We look for a pattern in the µij relating to the factors age and rating.We use log link.
The saturated model Ω : lnµij = µ0 + αi + βj + (αβ)ij
where αi is the age group effect, α1 = β1 = 0,
βj is the dream rating effect,
(αβ)ij is the interaction effects and (αβ)1j = 0 = (αβ)i1.
To test if β1 = β2 = β3, i.e., rating can be collapsed to 2-levels, the change indeviance, as compared to the saturated model with 0 deviance and 0 d.f., is 4.2581which distributed as χ2
(Dispersion parameter for poisson family taken to be 1)
Null deviance: 94.6068 on 19 degrees of freedom
Residual deviance: 4.2581 on 10 degrees of freedom #n-p=20-1-4-1-4
AIC: 105.37
Number of Fisher Scoring iterations: 4
For residual checks in GLM, the mean and variance are functionally related ininterpreting the residuals plot. To help in this problem, we generally use thescaled residuals of some form:
1. Ordinary residuals: Ri = Yi − µi but it is useful only in normal case.
1. Little, J.A. (1979) The General Linear Model and Direct Standardization AComparison. Sociological methods and research, 7, 475-501.
2. Lambert, D. (1992) Zero-Inflated Poisson Regression, with an Application toDefects in Manufacturing. Technometrics, 34: 1-14.
3. Mullahy, J. (1986) Specification and testing of some modified count datamodels. Journal of Econometrics, 33, 341-365.
SydU MSH3 GLM (2016) Second semester Dr. J. Chan 152
MSH3 Generalized linear model Ch. 6 Count data models
Appendix
Definition: A stochastic process N(t), t ≥ 0 is a counting process if N(t)represents the total number of events that have occured by time t. Note: N(t) ≥ 0.If s < t, N(s) ≤ N(t).
Definition: A stochastic process is independent increment the number of eventsin disjoint intervals are independent.
Definition: A stochastic process is stationary increment if the distributions ofN(t1 + s)−N(t2 + s) and N(t1)−N(t2) are equal.
Lemma: The counting process N(t), t ≥ 0 is a Poisson process with rate λ if
1. N(0) = 0
2. The process has stationary and independent increment.
3. Pr(N(h) = 1) = λh+ o(h)
4. Pr(N(h) ≥ 2) = o(h)
where f(x) = o(x) if f(x)x → 0 as x→ 0 and o(h)± o(h) = o(h).