ST745: Survival Analysis: Hypothesis testing and ...

ST745: Survival Analysis:Hypothesis testing and

confidence intervals

Eric B. Laber

Department of Statistics, North Carolina State University

March 17, 2015

I remember that one fateful day when Coach took measide. I knew what was coming. ”You don’t have totell me,” I said. ”I’m off the team, aren’t I?” ”Well,”said Coach, ”you never were really ON the team. Youmade that uniform you’re wearing out of rags andtowels, and your helmet is a toy space helmet. Youshow up at practice and then either steal the ball andmake us chase you to get it back, or you try to tacklepeople at inappropriate times.” It was all true what hewas saying. And yet, I thought something is brewinginside the head of this Coach. He sees something inme, some kind of raw talent that he can mold. Butthat’s when I felt the handcuffs go on. —Philip Glass

Then and now

I Last time we discussed graphical methodsI QQ-plots

I PP-plots

I Linearized plots

I Today we’ll discussI A bottomless hole

I Inference with parametric models

I Prediction intervals

Warm-up

I Explain to your stat buddy

1. What is a likelihood ratio test?

2. How can you invert a hypothesis test to form a confidenceinterval?

3. What is a prediction interval? How does this difference from aconfidence interval?

4. Who was Philip Glass?

I True or false:I (T/F) A likelihood ratio test is uniformly most powerful for

simple hypotheses?

I (T/F) The ratio of two (scaled) independent chi-squaredrandom variables has an F-distribution

I (T/F) The most expensive Donkey cheese in the world sells forseveral thousand dollars a pound

Inference and prediction intervals

I Quantities estimated from data should (must!) beaccompanied by measures of uncertainty

I Hypothesis tests and confidence intervals regardpopulation-level parameters, assess uncertainty and strengthof evidence

I Prediction intervals reflect both estimation uncertainty andvariability in the generative distribution

Shaking off the rust

I Suppose T1, . . . ,Tn are i .i .d . with density f (t; θ)I Inference and hypothesis testing typically concern θ

I Prediction regards uncertainty in a new draw Tn+1 from f (t; θ)

I Suppose X1, . . . ,Xn are i .i .d . N(µ, 1), with your stat buddy:

1. Give a 95% confidence interval for µ

2. Construct a test of H0 : µ = µ0 vs. Ha : µ 6= µ0 with α Type Ierror

3. Give a 95% prediction interval for Xn+1

Confidence intervals

I Let {(Ti , δi )}ni=1 denote censored observation times, survivaldistn f (t; θ)

I ML approach for CIS:I Construct MLE θ̂nI Construct observed fisher information I (θ̂)

I Use asymptotically normality of MLE

I 1/2(θ̂n)(θ̂n − θ) ≈ N(0, 1),

to obtain θ̂n ± z1−α/2I−1/2(θ̂n)

The example you love to hate

I Let {(Ti , δi )}ni=1 denote observation times subject tonon-informative right-censoring

I Assume the true survival distribution is exponential withdensity f (t; θ) = θ−1 exp−t/θ construct a 95% CI for θ

I Answer: on board

I Problem! The log-LH is not approximated well by a quadraticin small-samples when the amount of censoring is large

I Recall precision varies inversely with number of uncensoredobservations (Why?)

I Normal approximation may be poor

The example you love to hate cont’d

I Local approximation of log-LH by a quadratic, T ∼ exp(1)

0.0 0.5 1.0 1.5 2.0

−20

0−

150

−10

0−

50

Log−LH with n=100, Σδi = 10

θ

l(θ)

0.4 0.6 0.8 1.0 1.2 1.4 1.6

−30

−28

−26

−24

−22

Log−LH with n=100, Σδi = 25

θ

l(θ)

0.6 0.8 1.0 1.2 1.4

−53

−52

−51

−50

−49

−48

−47

Log−LH with n=100, Σδi = 50

θ

l(θ)

Inverting a test: review

I Consider testing the null H0 : θ = θ0 vs. Ha : θ 6= θ0

I Let Tn(θ0) be a test statistic and c1−α an α-level criticalvalue so that the test rejects when Tn(θ0) > c1−α

I E.g., H0 : µ = µ0 vs. Ha : µ 6= µ0 andTn(µ0) = n(X̄n − µ0)2/σ̂2, and c1−α = χ2

n−1,1−α

I If θ∗ is the ‘true’ value of θ then

P (Tn(θ∗) ≤ c1−α) ≥ 1− α,

thus {θ : Tn(θ) ≤ c1−α} is a valid (1− α)× 100% CI for θ∗

(Why?)

Inverting a test: likelihood

I Likelihood ratio statistic is

Λ(θ) = 2`(θ̂)− 2`(θ)

I If θ∗ is the true parameter value then Λ(θ∗) is asymptoticallyχ2p where p = dim θ

I (1− α)× 100% LH ratio CI for θ{θ : Λ(θ) ≤ χ2

p,1−α}

I Often preferred to the standard MLE interval in small samples

LH ratio CI for an exponential RV

I {(Ti , δi )}ni=1 observation times subj to non-informativeright-censoring, assume f (t; θ) = θ−1 exp{−t/θ}

I With your stat buddy: compute a LR CI for θ

LH ratio CI for an exponential RV cont’d

I See lrtCI.R

Sprott’s adjustment

I Sprott showed that the transformation φ = θ3 has asymmetrizing effect on the log-LH

I Idea: Construct usual ML based interval for φ using

I1/21 (φ̂)(φ̂− φ) ≈ N(0, 1),

where I1(φ̂) is the observed Fisher info for φ, then solve for aCI for θ

I This method performs similarly to the LRT and does notextend generally so we will not consider it further

Exact methods the exponential distribution

I Under certain censoring schemes exact confidence intervalsare possible

I Type II censoring

I Testing with replacement

I These are somewhat specialized examples but you shouldknow they exist as they are useful when applicable (see pp.153-154)

Likelihood ratio test

I Goal is to test null hypothesis H0

I Restricted MLE is θ̃ = arg maxθ satisfies H0 L(θ)

I Ex. X1, . . . ,Xn ∼ N(µ, σ2) compute the restricted MLE underH0 : µ = 1

I Ex. X1, . . . ,Xn ∼ N(µ, σ2) compute the restricted MLE underH0 : µ = σ

I LRT statisticΛ = 2`(θ̂)− 2`(θ̃),

when H0 is true Λ ≈ χ2d where d is difference in the degrees of

freedom between restricted and unrestricted models

LRT example

I Suppose we observe right-censored failure times from mdifferent groups that we believe have exponential lifetimedistns.

I Observed data {(Tij , δij)}, j = 1, . . . , ni , i = 1, . . . ,m

I Want to test H0 : θ1 = · · · = θmI With your stat buddy: derive unrestricted and restricted MLEs

I Test statistic Λ = 2`(θ̂)− 2`(θ̃) reject H0 if Λ > χ2m−1,1−α

(Why?)

LRT example cont’d

I See lrtFourMachines.R

Other distributions and censoring mechanisms

I As long as we have a likelihood we can apply these methods

I The book gives additional details for other distributionsI Gamma, Inverse-gaussian

I In more general settings numerical methods must be used tomaximize the LH

I Since we know how to compute the LH for other censoringmechanisms (e.g., left, interval) we can apply the proposedmethods directly

I Don’t forget about bootstrap CIs!1

1Bootstrap tests are also possible but we won’t discuss them in this class.See Bickel and Ren for info.

Prediction intervals

I Confidence intervals quantify uncertainty in estimatedcoefficients, the do not include uncertainty about futureobservations

I Give prognosis to newly diagnosed patient

I Predict failure time of new manufacturing process

I Implementing redundancies

I . . .

Prediction intervals cont’d

I Let X1, . . . ,Xn,Xn+1 be iid , a (1− α) level prediction intervalfor Xn+1 based on X1, . . . ,Xn are functions L(X1, . . . ,Xn) andU(X1, . . . ,XN) so that

P {L(X1, . . . ,Xn) ≤ Xn+1 ≤ U(X1, . . . ,Xn)} ≥ 1− α

I E.g., in the warm-up example X1, . . . ,Xn,Xn+1 iid N(µ, 1),L(X1, . . . ,Xn) = X̄n − z1−α/2

√1 + 1/n, and

U(X1, . . . ,Xn) = X̄n + z1−α/2

√1 + 1/n

Pivots

I G (X1, . . . ,Xn,Xn+1) that does not depend on θ is called pivot

I X̄n − Xn+1 is a pivot the N(µ, 1) example

I Solve for a, b such that

P (a ≤ G (X1, . . . ,Xn,Xn+1) ≤ b) ≥ 1− α,

then {x : a ≤ G (X1, . . . ,Xn, x) ≤ b} is (1− α)× 100% PI

I In survival problems obtaining a pivot can be difficult, instead:I Plug-in intervals

I Projection intervals

I Calibrate the interval using simulation

Plug-in intervals

I Compute θ̂ and take percentiles of distn f (t; θ̂)

I Asymptotically correct when θ̂ is consistent

I Ignores uncertainty in estimated θ

I Ex. {(Ti , δi )}ni=1 censored lifetimes assumed exp(θ)I Solve S(t; θ) = 1− p for t to obtain tp(θ) = − log(1− p)θ2

I Approximate prediction interval(− log(1− α/2)θ̂,− log(α/2)θ̂

)I See approxatePI.R

2Here we’re using f (t; θ) = θ−1 exp{−t/θ}

Projection intervals

I Plugin PIs ignore uncertainty in θ̂ and thus may beanti-conservative

I Projection intervals incorporate uncertainty in θ̂ but can beconservative

I Let C1−α(θ0) be a (1− α)× 100% plugin PI using assumedparameter θ0

I Let ζ1−η be a (1− η)× 100% confidence region for θ

I Projection interval ⋃θ0∈ζ1−η

C1−α(θ0),

is a valid 1− α− η PI (Why?)

Calibration

I Investigate the performance of PI procedure view α as tuningparameter

I Choose α so that the coverage is near nominal levels insimulated examples