Appendix A1 Pro le Likelihoods for Fitted ... - stacks.cdc.gov · 40 50 60 Adult One-dose Effectiveness (%) Two doses Two-Dose 20K Mixed Mixed 20K Pre-Vaccination Scenarios 0 10 20

Appendix

A1 Profile Likelihoods for Fitted Parameters.

0 0.005 0.01 0.015 0.02 0.025

ka

-530

-520

-510

-500

-490

-480

-470

-460

-450

Negative L

og L

ikelih

ood

0 0.005 0.01 0.015 0.02 0.025 0.03 0.035

kc

-530

-520

-510

-500

-490

-480

-470

-460

Negative L

og L

ikelih

ood

0 0.2 0.4 0.6 0.8 1 1.2 1.4

W 10-5

-526

-524

-522

-520

-518

-516

-514

-512

-510

-508

-506

Ne

ga

tive

Lo

g L

ike

liho

od

0 0.2 0.4 0.6 0.8 1 1.2 1.4

I 10-5

-550

-500

-450

-400

-350

-300

-250

-200

-150

Ne

ga

tive

Lo

g L

ike

liho

od

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07-525

-524

-523

-522

-521

-520

-519

-518

-517

Ne

ga

tive

Lo

g L

ike

liho

od

Figure A1: Profile likelihood plots for estimated parameters top row: ka (left) ,kc (right), second row: βW(left), βI (right), and third row: ξ. Note: The βW and ξ ranges were extended to capture the 95% confidenceintervals.

1

A2 Cumulative Cases of Alternative Seeding Scenarios.

Table A2: Cumulative Cases of Alternative Seeding Scenarios.

Vaccination Scenario Cases Attack Rate (per 1,000 people)Baseline: No vaccination 395.4 8.7

Pre-Vaccination

Maela 0.1 0.002

One-dose only 239.8 5.3

Two-dose only 233.5 5.2

Mixed 236.6 5.2

first come, first served 237.8 5.3

2

A3 Cumulative Cases of Alternative Seeding Scenarios.

0 100 200 300 400

days

0

50

100

150

200

250

Ca

se

s

Adults

nv

p1d

p2d

pMixed

pfcfs

Pre-Vaccination Scenarios - Alternative Seeding

0 100 200 300 400

days

0

50

100

150

200

Ca

se

s

Children

nv

p1d

p2d

pMixed

pfcfs

Figure A3: Alternative seeding scenario: one actual case. Cumulative cholera cases in adults and children, fordifferent pre-vaccination scenarios. Where ‘nv’ is the baseline, no vaccination scenario; ‘p1d’ is the one-dosescenario; ‘p2d’ is the two-dose scenario; ‘pmixed’ is the mixed scenario.

3

A4 Maela Vaccination Coverage Calculations.

Table A4: OCV coverage data from the 2013 vaccine campaign. The “% Coverage” columnindicates the percent coverage for the entire population (all included and excluded subjects).The “Number of Individuals” column indicates the initial conditions used in the model, cal-culated from the coverage percentages.

Class % Coverage Number of IndividualsNon-vaccinated infectious adults - seeding (Ia) 0% 72

Non-vaccinated adults (Sa) 0% 7,856

Once-vaccinated adults (Va) 22.3% 6,208

Twice-vaccinated adults (V Va) 49.3% 13,765

Non-vaccinated infectious children - seeding (Ic) 0% 53

Non-vaccinated children (Sc) 0% 3,021

Once-vaccinated children (Vc) 18.7% 3,241

Twice-vaccinated children (V Vc) 63.6% 11,018

4

A5 Pre-vaccination with Reduced Vaccine Effectiveness Among Chil-dren

0 10 20 30 40 50 60 70 80 90

Total Doses (in thousands)

0

10

20

30

40

50

60

Ad

ult

On

e-d

ose

Eff

ective

ne

ss (

%) First Come, First Served

FCFS 20K

0 10 20 30 40 50 60 70 80 90

Total Doses (in thousands)

0

10

20

30

40

50

60

Ad

ult

On

e-d

ose

Eff

ective

ne

ss (

%) Two doses

Two-Dose 20K

0 10 20 30 40 50 60 70 80 90

Total Doses (in thousands)

0

10

20

30

40

50

60

Ad

ult

On

e-d

ose

Eff

ective

ne

ss (

%) Mixed

Mixed 20K

Pre-Vaccination Scenarios

0 10 20 30 40 50 60 70 80 90

Total Doses (in thousands)

0

10

20

30

40

50

60

Ad

ult

On

e-d

ose

Eff

ective

ne

ss (

%) One dose

One-Dose 20K

50

100

150

200

250

300

350

Cumulative Cases

Figure A5: Varying one-dose vaccine effectiveness and total doses for different pre-vaccination scenarios withreduced vaccine effectiveness estimates among children.

5

A6 Reactive vaccination with Reduced Vaccine Effectiveness AmongChildren

0 10 20 30 40 50 60

Delay (days)

0

20

40

60

80

Adult

One-d

ose E

ffic

acy (

%)

First Come, First Served

0 10 20 30 40 50 60

Delay (days)

0

20

40

60

80

Adult

One-d

ose E

ffic

acy (

%)

Two doses

0 10 20 30 40 50 60

Delay (days)

0

20

40

60

80

Adult

One-d

ose E

ffic

acy (

%)

Mixed

Reactive Vaccination Scenarios

0 10 20 30 40 50 60

Delay (days)

0

20

40

60

80

Adult

One-d

ose E

ffic

acy (

%)

One dose

150

200

250

300

350

Cumulative Cases

Figure A6: Varying one-dose vaccine effectiveness and delay in campaign implementation for different reactivevaccination scenarios, with reduced vaccine effectiveness estimates among children.

6

A7 2013 Forecasting Results with Partially Immune Population.

Without OCV Campaign With OCV Campaign

0 10 20 30 40 50

Week

10-2

100

102

Cum

ula

tive O

bserv

ed C

ases

best-fit parameters median 75% & 25% quantiles

0 10 20 30 40 50

Week

10-2

10-1

100

Cum

ula

tive O

bserv

ed C

ases

best-fit parameters median 75% & 25% quantiles

Figure A7: 2013 forecast with a single actual case in adults and children as seeding. These plots show apartially immune population with no OCV campaign on left and with OCV campaign on right.

7

A8 2014 Forecasting Results

0 10 20 30 40 50

Week

10-2

100

102

Cum

ula

tive O

bserv

ed C

ases

best-fit parameters median 75% & 25% quantiles

0 10 20 30 40 50

Week

10-2

100

102

Cum

ula

tive O

bserv

ed C

ases

Cumulative Total Cases

Figure A8: 2014 forecast with a single actual case in adults and children as seeding. First row: Fullysusceptible population with OCV campaign. Second row: Partially immune population with OCV campaign.

8

A9 Model equations and additional details

A9.1 Simplified Model Equations

The simplified age-structured model equations with no vaccination are below. This model was usedfor the identifiability analysis to calculate R0 and was fit to the Maela outbreak data.

Force of Infection Equations

λa = βaaIa + βcaIc + βwaW

λc = βacIa + βccIc + βwcW(1)

Non-Vaccinated Adults

Ṡa =Ma2− λaSa − µaSa

İa = λaSa − γIa − µaIa

Ṙa =Ma2

+ γIa − µaRa

(2)

Non-Vaccinated Children

Ṡc = B +Mc2− λcSc − µcSc

İc = λcSc − γIc − µcIc

Ṙc =Mc2

+ γIc − µcRc

(3)

Environmental Pathogen

Ẇ = ξ(λw −W )λw = Ia + σIc

(4)

A9.2 Full Model Equations

The full age-structured model equations separated by non-vaccinated, once-vaccinated, and twice-vaccinated individuals are below. If fitted, parameter values are from the simplified model (above)and the remaining non-fitted values (e.g., vaccine effectiveness) are from the literature, see Table1. This model was used to examine the different counterfactual vaccination scenarios.

9

Force of Infection Equations

λa = βaaIa + βcaIc + βwaW + βaaIVa + βcaIVc + βaaIV Va + βcaIV Vc

λva = (1− VE1)(βaaIa + βcaIc + βwaW + βaaIVa + βcaIVc + βaaIV Va + βcaIV Vc)λvva = (1− VE2)(βaaIa + βcaIc + βwaW + βaaIVa + βcaIVc + βaaIV Va + βcaIV Vc)λc = βacIa + βccIc + βwcW + βacIVa + βccIVc + βacIV Va + βccIV Vc

λvc = (1− VE1)(βacIa + βccIc + βwcW + βacIVa + βccIVc + βacIV Va + βccIV Vc)λvvc = (1− VE2)(βacIa + βccIc + βwcW + βacIVa + βccIVc + βacIV Va + βccIV Vc)

(5)

Non-Vaccinated Adults

Ṡa = −µaSa − λaSa +Ma/2− νa1Saİa = λaSa − µaIa − γIaṘa = γIa − µaRa +Ma/2− νa1Ra

(6)

Once-Vaccinated Adults

˙SV a = −λvaSVa − µaSVa + νa1Sa − νa2SVa˙IV a = λvaSVa − µaIVa − γIVa˙RV a = γIVa − µaRVa + νa1Ra − νa2RVa

(7)

Twice-Vaccinated Adults

˙SV V a = −λvvaSV Va − µaSV Va + νa2SVa˙IV V a = λvvaSV Va − µaIV Va − γIV Va˙RV V a = γIV Va − µaRV Va + νa2RVa

(8)

Non-Vaccinated Children

Ṡc = −µcSc − λcSc +Mc/2 +B − νc1Scİc = λcSc − µcIc − γIcṘc = γIc − µcRc +Mc/2− νc1Rc

(9)

Once-Vaccinated Children

˙SV c = −µcSVc − λvcSVc + νc1Sc − νc2SVc˙IV c = λvcSVc − µcIVc − γIVc˙RV c = γIVc − µcRVc + νc1Rc − νc2RVc

(10)

10

Twice-Vaccinated Children

˙SV V c = −λvvcSV Vc − µcSV Vc + νc2SVc˙IV V c = λvvcSV Vc − µcIV Vc − γIV Vc˙RV V c = γIV Vc − µcRV Vc + νc2RVc

(11)

Environmental Pathogen

λw = Ia + σIc + IVa + σIVc + IV Va + σIV Vc

Ẇ = ξ(λw −W )(12)

Total Population Sizes

Na = Sa + Ia +Ra + SVa + IVa +RVa + SV Va + IV Va +RV Va

Nc = Sc + Ic +Rc + SVc + IVc +RVc + SV Vc + IV Vc +RV Vc(13)

A9.3 Forecasting Model equations

The age-structured model equations used for the forecasting scenarios are below.

Non-Vaccinated Adults

Ṡa = MaMsusc − βI(Ia + Ic)Sa − βWWSa + 2αV 1a − µaSaİa = (Sa + (1− VE1D)V 1a + (1− VE2D)V 2a)(βI(Ia + Ic) + βWW )− γIa − µaIa

(14)

Non-Vaccinated Children

Ṡc = McMsusc +B − βI(Ia + Ic)Sc − βWWSc + 2αV 1c − µcScİc = (Sc + (1− VE1D)V 1c + (1− VE2D)V 2c)(βI(Ia + Ic) + βWW )− γIc − µcIc

(15)

Immune Adults

Ṙa = γIa − 2αRa − µaRa (16)

Immune Children

Ṙc = γIc − 2αRc − µcRc (17)

11

Partial Immune/Vaccinated Adults

˙V 1a = Ma(1−Msusc)− ((1− VE1D)V 1a)(βI(Ia + Ic)− βWW )+2α(V 2a − V 1a)− µaV 1a

˙V 2a = ((1− VE2D)V 2a)(−βI(Ia + Ic)− βWW ) + 2α(Ra − V 2a)− µaV 2a

(18)

Partial Immune/Vaccinated Children

˙V 1c = Mc(1−Msusc)− ((1− VE1D)V 1c)(βI(Ia + Ic)− βWW )+2α(V 2c − V 1c)− µcV 1c

˙V 2c = ((1− VE2D)V 2c)(−βI(Ia + Ic)− βWW ) + 2α(Rc − V 2c)− µcV 2c

(19)

Environmental Pathogen

Ẇ = ξ(Ia + Ic −W ) (20)

Identifiability Analysis

Identifiability analysis addresses the question of whether the model parameters can be estimatedfrom a given data set [1]. Identifiability is typically broken into two broad categories—(1) struc-tural identifiability, which examines theoretical identifiability from the structure of the model andmeasured variables, and (2) practical identifiability, which addresses how a model’s identifiabilityproperties are affected by real-world data issues such as noise and sampling frequency.

Structural Identifiability Analysis

To examine the structural identifiability of our simplified age-structured model, we used the differential-algebra based approach developed in [1–6]. Determining the structural identifiability of the modelis a prerequisite to determining if there is a unique solution for a set of unknown model parameters[2]. Structural identifiability can be framed as evaluating whether the model parameters can beestimated uniquely, when the data is assumed to be ‘perfect’ (i.e., noise-free and measured forall time points). Establishing structural identifiability is a prerequisite for successful parameterestimation from real-world, noisy data. When parameters are not individually identifiable, groupsof parameters typically form identifiable combinations that can be uniquely determined.

In the differential algebra approach, the unmeasured state variables (e.g. SA, SC , etc.) areeliminated, leaving equations only the measured variables, their derivatives, and the parameters,denoted the input-output equations. In this case, the measured variables are cholera incidenceamong adults and children. The identifiability from cholera incidence was more easily analyzedusing the prevalence approximation, which as γ is assumed to be known, yields the same structuralidentifiability results as the standard incidence. We assumed the demographic parameters, initialpopulation sizes, and recovery rate are known from data as described above and defined in Table 1,and the remaining parameters (βij ’s, k’s, α, σ, and ξ) were considered unknown. A Gröbner-basis

12

approach was then used to test whether the unknown model parameters in Equations (1) – (4) areidentifiable from the measured data, with all calculations performed in Mathematica Version 10.

Similar to the original SIWR model [1], the waterborne transmission parameters and α were notseparately identifiable for our model, instead forming the identifiable combination β̄w =

αξ βW . To

address this, we define W̄ = ξαW . Rewriting the model equations in terms of these new variablesyields the following equation for environmental pathogen:

˙̄W = ξ(λw − W̄ )

with all other equations remaining the same except replacing W with W̄ and the identifiablecombination β̄w. Once re-scaled, all unknown model parameters (βI , β̄W , σ, ξ, and the k’s) werestructurally identifiable. From this point forward (and for the parameter estimation and otheranalyses), we use only the rescaled versions of βW and W , and thus we will omit the bar notation.

Practical Identifiability

Initially, even though structural identifiability was considered, we obtained extremely similar fits fora wide range of transmission parameter values, suggesting that there were practical unidentifiabilityissues wherein the reporting parameters (ka and kc) and adult and child transmission parameterscan partially compensate for one another to yield the same overall apparent cholera incidences. Forthe sake of parsimony [7], we set all human-human transmission parameters equal to each other,denoted βI , and separately we set all human-water transmission parameters equal to each other,denoted βW . Similarly, σ, the relative shedding rate for adults and children, was also relativelypractically unidentifiable, and so we set shedding to be equal for both classes.

To examine practical identifiability and parameter uncertainty, we plotted profile likelihoodsof each fitted parameter. Profile likelihoods are a numerical approach to evaluating parameteruncertainty and identifiability [8]. Profiles are generated by fixing the profiled parameter to a seriesof values, while fitting the remaining parameters that are being estimated. Typically, the minimumnegative log likelihood (or equivalently the maximum likelihood) values are plotted for each value ofthe profiled parameter, forming the profile likelihood for that parameter. The minimum representsthe best-fit value of the profiled parameter and is determined by parameter estimation. If the profileis flat, the parameter cannot be uniquely determined and is considered unidentifiable. However,even if the profile is structurally identifiable, the curvature may be quite shallow, so that a particularminimum cannot practically be distinguished - this is denoted practical unidentifiability. Confidenceintervals can be determined from the profile likelihood by setting a significance-based threshold onthe likelihood based on a χ2 distribution [8]. Once the threshold is set, all parameters correspondingto likelihood values below the threshold fall within the confidence interval. The results from theprofile likelihood plotting can be seen in Figure A1.

Sensitivity Analysis: Initial Seeding from Observed to Actual

As another sensitivity analysis, we changed our initial seeding from one observed case to one actualcase for the Maela and pre-vaccination scenarios. Because vaccination occurred before any outbreak,administration of the vaccine was not affected by case detection. Overall, we see the same patternof results, shown in Figure A3 and Table A2. The two-dose scenario sees the largest reduction incases followed by the mixed, first come, first served, and one-dose scenarios. Since the number ofinitial infected individuals is lower, the total cumulative case counts are as well. Of note is thatthe reduction in cases is greater for pre-vaccination scenarios than for the baseline non-vaccinationscenario.

13

Maela Vaccine Coverage Calculations

Among included individuals (pregnant women and infants < 1 year were excluded), the OCVcampaign covered 51% of adults and 68% of children with two doses, and another 23% of adultsand 20% of children with one dose. We made the following adjustments to determine total Maelacoverage among both included and excluded individuals for the forecasting scenarios:

• Once-vaccinated adults:(Va − V Va)((Na− Pregnant women)/(Na))Na((0.74-0.51)*(27901-910)/27901)*27901 = 6207.9

• Twice-vaccinated adults:(V Va)((Na− Pregnant women)/(Na))Na(0.51*(27901-910)/27901)*27901 = 13765.4

• Once-vaccinated children:(Vc − V Vc)((Nc− infants under 1 year old)/(Nc))Nc((0.88-0.68)*(17332-1129)/17332)*17332 = 3240.6

• Twice-vaccinated adults:(V Vc)((Nc− infants under 1 year old)/(Nc))Nc(0.68*(17332-1129)/17332)*17332 = 11018

Table A4 shows the total number of individuals by class used to simulate the OCV campaign.

Forecasts for the 2013 Cholera Season

In the 2013 forecasting results, we see a larger spread of total case numbers for runs in the scenariowithout the OCV campaign compared to the scenario with the OCV campaign. The partiallyimmune population runs generally have lower case counts when comparing to the fully susceptiblepopulation. Furthermore, for the scenarios that consider the OCV campaign, we see that the vastmajority of runs having case counts close to 0. For details see Figures 8 and A7.

Forecasts for the 2014 Cholera Season

The 2014 forecasting results are quite similar to the 2013 runs for the fully susceptible populationcompared to the partially immune population with more runs resulting in 0 total cases for thepartially immune population. As population immunity wanes between 2013 and 2014 we get ahigher proportion of larger outbreaks for the 2014 forecasting scenarios, but the vast majority ofruns remain close to 0 for both the fully susceptible and partially immune populations. For detailssee Figure A8.

References

[1] Eisenberg MC, Robertson SL, Tien JH. Identifiability and estimation of multiple transmissionpathways in cholera and waterborne disease. Journal of Theoretical Biology. 2013;324:84–102.

[2] Audoly S, Bellu G, D’Angio L, Saccomani MP, Cobelli C. Global identifiability of nonlinearmodels of biological systems. IEEE Transactions on Biomedical Engineering. 2001;48(1):55–65.

14

[3] Evans ND, White LJ, Chapman MJ, Godfrey KR, Chappell MJ. The structural identifiabilityof the susceptible infected recovered model with seasonal forcing. Mathematical Biosciences.2005;194(2):175–197.

[4] Chapman JD, Evans ND. The structural identifiability of susceptible–infective–recovered typeepidemic models with incomplete immunity and birth targeted vaccination. Biomedical SignalProcessing and Control. 2009;4(4):278–284.

[5] Bellman R, Åström KJ. On structural identifiability. Mathematical Biosciences. 1970;7(3-4):329–339.

[6] Chis OT, Banga JR, Balsa-Canto E. Structural identifiability of systems biology models: acritical comparison of methods. PloS One. 2011;6(11):e27755.

[7] Stoica P, SÖDERSTRÖM T. On the parsimony principle. International Journal of Control.1982;36(3):409–418.

[8] Raue A, Kreutz C, Maiwald T, Bachmann J, Schilling M, Klingmüller U, et al. Structural andpractical identifiability analysis of partially observed dynamical models by exploiting the profilelikelihood. Bioinformatics. 2009;25(15):1923–1929.

15

Profile Likelihoods for Fitted Parameters.Cumulative Cases of Alternative Seeding Scenarios.Cumulative Cases of Alternative Seeding Scenarios.Maela Vaccination Coverage Calculations.Pre-vaccination with Reduced Vaccine Effectiveness Among ChildrenReactive vaccination with Reduced Vaccine Effectiveness Among Children2013 Forecasting Results with Partially Immune Population.2014 Forecasting ResultsModel equations and additional detailsSimplified Model EquationsFull Model EquationsForecasting Model equations