Biochemical Network

8/13/2019 Biochemical Network

1/41

Biological modellingModel calibration

Application projectsBayesian inference

Summary and conclusions

Introduction to Biochemical Network Modelling

Darren Wilkinson1,21School of Mathematics & Statistics

2Centre for Integrated Systems Biology of Ageing and NutritionNewcastle University, UK

SAMSI Undergraduate Workshop, 2nd3rd March, 2007

Darren Wilkinson SAMSI Undergraduate Workshop Biochemical Network Modelling


2/41




Overview

Biological network modelling

Model calibration

Application projects modelling and inference

(Bayesian inference)

Round-up



3/41




IntroductionModellingStochastic kinetics

Computational Systems Biology (CSB)

Much of CSB is concerned with building models of complexbiological pathways, then validating and analysing thosemodels using a variety of methods, including time-course

simulationMost CSB researchers work with continuous deterministicmodels (coupled ODE and DAE systems)

There is increasing evidence that much intra-cellularbehaviour (including gene expression) is intrinsically

stochastic, and that systems cannot be properly understoodunless stochastic effects are incorporated into the models

Stochastic models are harder to build, estimate, validate,analyse and simulate than deterministic models...



4/41





Modelling

Start with some kind of picture or diagram for a mechanism

Turn it into a set of (pseudo-)biochemical reactions

Specify the rate laws and rate parameters of the reactionsRun some stochastic or deterministic computer simulator ofthe system dynamics

Study the dynamics in a variety of ways to gain insight into

the systemRefine the model structure and/or parameters after comparingsimulated dynamics with experimental observations



5/41

Bi l i l d lli


6/41





Biochemical reactions

Simplified view:

Reactions

g+P2 g P2 Repressiong g+r Transcriptionr r+P Translation2P P2 Dimerisation

r mRNA degradationP Protein degradation

But these arent as nice to look at as the picture...


Biological modelling


7/41





Petri net representation

Simple bipartite digraph representation of the reaction network useful both for visualisation and computational analysis




8/41





Matrix representation of the Petri net

Reactants (Pre) Products (Post)Species g P2 g r P P 2 g P2 g r P P 2

Repression 1 1 1Reverse repression 1 1 1

Transcription 1 1 1Translation 1 1 1

Dimerisation 2 1

Dissociation 1 2mRNA degradation 1Protein degradation 1

But still need rate laws and reaction rates...




9/41





Mass-action stochastic kinetics

Stochastic molecular approach:

Statistical mechanics arguments lead to aMarkov jumpprocessin continuous time whose instantaneous reaction rates

are directly proportional to the number of molecules of eachreacting species

Such dynamics can be simulated (exactly) on a computerusing standarddiscrete-event simulationtechniques

Standard implementation of this strategy is known as theGillespie algorithm (just discrete event simulation), butthere are several exact and approximate variants of this basicapproach




10/41





Lotka-Volterra system

Reactions

X 2X (prey reproduction)X+Y 2Y (prey-predator interaction)

Y (predator death)

X Prey, Y PredatorWe can re-write this using matrix notation for thecorresponding Petri net




11/41

g gModel calibration




Forming the matrix representation

The L-V system in tabular form

Rate Law LHS RHS Net-effecth(, c) X Y X Y X Y

R1 c1x 1 0 2 0 1 0R2 c2xy 1 1 0 2 -1 1R3 c3y 0 1 0 0 0 -1

Call the 3 2 net-effect (orreaction) matrix A. The matrix S=A

is thestoichiometry matrixof the system. Typically both aresparse. The SVD ofS (orA) is of interest for structural analysis ofthe system dynamics...




12/41





Petri net invariants

A P-invariant is a non-zero solution to Ay= 0 (ie. y is in thenull-space ofA)

P-invariants correspond toconservation lawsin the network,

and lead to rank-degeneracy ofAA T-invariant is a non-zero, non-negative (integer-valued)solution to Sx= 0 (ie. x is in the null-space ofS)

T invariants correspond to sequences of reaction events thatreturn the system to its original state

The SVD ofS (orA) characterises the null-space ofS and A

The Lotka-Volterra model is of full rank (so no P-invariants),and has one T-invariant,x= (1, 1, 1)




13/41





The Gillespie algorithm

1 Initialise the system at t= 0 with rate constants c1, c2, . . . , cv andinitial numbers of molecules for each species, x1, x2, . . . , xu.

2 For each i= 1, 2, . . . , v, calculate hi(x, ci) based on the currentstate, x.

3 Calculate h0(x, c)

vi=1hi(x, ci), the combined reaction hazard.

4 Simulate time to next event,t, as an Exp(h0(x, c)) randomquantity, and put t :=t+t.

5 Simulate the reaction index, j, as a discrete random quantity withprobabilitiesh

i(x, c

i) / h

0(x, c), i= 1, 2, . . . , v.

6 Update xaccording to reaction j. That is, put x :=x+S(j), whereS(j) denotes the jth column of the stoichiometry matrix S.

7 Output x and t.

8 Ift


14/41

Model calibrationApplication projects

Bayesian inferenceSummary and conclusions


The continuous deterministic approximation

If the discreteness and stochasticity are ignored, then byconsidering the reaction fluxes it is straightforward to deducethe mass-action ordinary differential equation (ODE) system:

ODE ModeldXt

dt =Sh(Xt, c)

Analytic solutions are rarely available, but good numericalsolvers can generate time course behaviour

Slight complications due to rank-degeneracy ofS

Also spatial versions reaction-diffusion kinetics PDEmodels computationally intensive (slow)


Biological modellingM d l lib i I d i


15/41




The Lotka-Volterra model

Time

[Y]

0 20 40 60 80 100

0

5

10

15

[Y1]

[Y2]

0 2 4 6 8

0

5

10

15

20

25

[Y1]

[Y2]

Time

Y

0 5 10 15 20 25

100

200

300

400 Y1

Y2

50 100 150 200 250 300 350

100

200

300

4

00

Y1

Y2


Biological modellingM d l lib ti I t d ti


16/41




Key differences

Deterministic solution is exactly periodic with perfectlyrepeating oscillations, carrying on indefinitely

Stochastic solution oscillates, but in a random, unpredictable

way (wandering from orbit to orbit in phase space)Stochastic solutionwillend in disaster! Either prey orpredator numbers will hit zero...

Either way, predators will end up extinct, soexpectednumberof predators will tend to zero qualitatively differentto the

deterministic solution

So, in general the deterministic solution does not providereliable information about either the stochastic process or itsaverage behaviour


Biological modellingModel calibration Introduction


17/41




Simulated realisation of the auto-regulatory network

0.0

0.5

1.0

1.5

2.0

Rna

0

10

30

50

P

0

200

400

600

0 1000 2000 3000 4000 5000

P2

Time




18/41



IntroductionLikelihood-based fully Bayesian inferenceLikelihood-free Bayesian inference

Model calibration

In its most basic form, model calibrationis concerned withtuning the parameters of a computer model in order to

make the output obtained by running it consistent withexperimental observations

In practice, this is only one aspect of the problem, as therewill typically be a range of parameter values consistent withobservations, and so the calibration exercise is part of a

broader analysis, also concerning modelvalidityand parameteridentifiabilityandconfounding




19/41




Simple example: linear birth-death process

Birth-death reactions

X X 2X

X X

Deterministic solution: Xt=X0exp{( )t}

This is a function of ( ) only!Stochastic solution is more interesting, and depends on both and ...




20/41




Birth-death realisations

0 1 2 3 4 5

0

10

20

30

40

50

60

t

X




21/41



Likelihood-based fully Bayesian inferenceLikelihood-free Bayesian inference

Issues with the birth-death process

Stochastic variation: random distribution at each time point,correlations between time points, random time to extinction,

etc.Parameter identification: if a deterministic model is fitted, onecan onlyeveridentify ( ) never and separately

Information aboutboth and in the data...

Needboth and for reliable stochastic simulation

Cant fit parameters using a deterministic model, then run astochastic simulation...




22/41




Birth-death realisations

0 1 2 3 4 5

0

20

40

60

t

X

lambda=0, mu=1

0 1 2 3 4 5

0

20

40

60

t

X

lambda=3, mu=4

0 1 2 3 4 5

0

20

40

60

t

X

lambda=7, mu=8

0 1 2 3 4 5

0

20

40

60

t

X

lambda=10, mu=11




23/41




Fully Bayesian inference

In principle it is possible to carry out rigorous statisticalinference for the parameters of the stochastic process model

Fairly detailed experimental data are required eg.quantitative single-cell time-course dataderived from live-cell

imagingThe standard procedure uses GFP labelling of key reporterproteins together with time-lapse confocal microscopy, butother approaches are also possible

The statistical theory underlying the inference algorithms is

fairly technical the techniques are developed and illustratedin a sequence of papers. The main findings are summarised in:Golightly, A. & Wilkinson, D. J. (2006)Bayesian sequentialinference for stochastic kinetic biochemical network models,Journal of Computational Biology, 13(3):838851.



A li i jIntroductionLik lih d b d f ll B i i f


24/41




Likelihood-free MCMC for Bayesian inference

It is possible to develop a generic framework for Bayesianinference for model parameters applicable to bothdeterministic and stochastic models using the ideas oflikelihood-free MCMC, which sacrifices some computational

efficiency for considerable reduction in implementationcomplexity

It exploitsforward simulationfrom the computer model

Such an approach requires a very large number of simulation

runs, and is therefore most easily applied to fast simulators(simple models)

Forslow simulators(complex models), HPC facilities can beexploited in order to build a fastemulatorof the slowsimulator



25/41


Application projectsAgeingComplex modelling


26/41



Complex modellingBayesian calibration

Network theory of ageing





27/41




Modelling large biological systems

BBSRC/MRC/DTI Grant (+ Unilever)

BASIS Biology of Ageing e-Science Integration and

Simulation(4/023/06) Kirkwood, Wilkinson, Boys, Gillespie,Proctor, Shanley

Modelling large complex systems with many interactingcomponents

SBML model database (SBML encoded for discrete stochastic

simulation)Discrete stochastic simulation service (and results database)

Distributed computing infrastructure for routine use (webportal and web-service interface for GRID computing)





28/41




SBML The Systems Biology Markup Language

SBML is an XML-based language for encoding andexchanging quantitative biochemical network models

Encodes species, initial amounts, reactions, rate laws, etc.

Original specification (Level 1) aimed mainly at continuousdeterministic models

Current specification (Level 2) perfectly capable of encodingdiscrete stochastic models in an unambiguous way

Many tools for working with SBML models (model builders,

deterministic and stochastic simulators, etc.)

Issues with testing correctness of stochastic simulators, andcorrectly encoding discrete stochastic models usingoff-the-shelf model-building tools





29/41




Computer model technology

BASIS features service-oriented architecture (SOA)

Controls access to models, data and computational resourcesRepresents and encodes complex models using XMLtechnology (SBML in this case)Simulation engine that can handle a broad class of modelswithout recompilationDatabases for models and simulation outputWeb interface for human-interaction

SOAP web-services API for programmatical access

Do we need a standard API for biological simulation services?





30/41

pp p jBayesian inference


p gBayesian calibration

BASIS Software www.basis.ncl.ac.uk

UK e-Science GRID Pilot Project

C

Simulation code

GSL

Scientific

library

Postgres

Database

Condor

Jobsched

libSBML

SBML

library

R

Data

analysis

Networkvisualise

Python

Main BASIS API

Python SOAP Web Services interface (SSLbased)

PythonSpyce/PSP

and CGI

scripts

Java

Axis

Tomcat

Apache web server

WSSecurity WSs

Web client

Debian GNU/Linux (sarge)

SOAP client (WSSecurity) SOAP client (SSL)

Software architecture used to implement the BASISsystem

GraphViz





31/41

pp p jBayesian inference


p gBayesian calibration

Example: Chaperones and their role in ageing

C. J. Proctor, C. Soti, R. J. Boys, C. S. Gillespie, D. P.Shanley, D. J. Wilkinson, T. B. L. Kirkwood (2005)Modellingthe actions of chaperones and their role in ageing,

Mechanisms of Ageing and Development, 126(1):119-131.Several versions of this model in the BASIS public modelrepository, each with a unique ID each can be copied,modified and simulated

eg. urn:basis.ncl:model:518





32/41


Bayesian calibration

Outline CaliBayes architecture



33/41

Biological modellingModel calibrationApplication projects

B i i f

AgeingComplex modellingB i lib ti


34/41



An example posterior distribution

0.00 0.01 0.02 0.03 0.04

0.1

0

0.1

5

0.2

0

0.2

5

vd

vdr

vd

Density

0.000 0.010 0.020 0.030

0

20

40

60

80

100

vd

Density

0.05 0.15 0.25

0

5

10

15

20

25



Ba esian inference

AgeingComplex modellingBa esian calibration


35/41



Extensions

Bayesian inference naturally integrates data from multiplesources, and may be assimilated simultaneously or sequentiallydepending on the context

The architecture requires slight modification for complex

models, as then the simulator is replaced by an emulator, builtoff-line using HPC facilities

The framework can also be adapted to tackle experimentaldesign questions such as: Given a limited budget, and ourcurrent state of knowledge, what are the best new

experiments to carry out in order to learn most about the

model parameters of greatest interest?

It is also possible to extend the framework to compareevidence for competing models for the same process



Bayesian inference

MCMCFuture directionsEmulators


36/41


Emulators

MCMC-based fully Bayesian inference for fastcomputer

models

Before worrying about the issues associated withslowsimulators, it is worth thinking about the issues involved incalibratingfast deterministicandstochasticsimulators, basedonly on the ability to forward-simulatefrom the model

In this case it is often possible to construct MCMC algorithmsfor fully Bayesian inference using the ideas of likelihood-freeMCMC(Marjoram et al 2003)

Here an MCMC scheme is developed exploiting forwardsimulation from the model, and this causes problematiclikelihood terms to drop out of the M-H acceptanceprobabilities



Bayesian inference



37/41


Emulators

Future directions

In the presence of measurement error, the sequentiallikelihood-free scheme is effective, and is muchsimpler than amore efficient MCMC approach

The likelihood-free approach is easier to tailor to non-standard

models and data

The essential problem is that ofcalibrationof complexstochastic computer models

Worth connecting with the literature on deterministic

computer modelsForslowstochastic models, there is considerable interest indeveloping fastemulatorsand embedding these into MCMCalgorithms



Bayesian inference



38/41


Emulators

Building emulators for slowsimulators

UseGaussian process regressionto build an emulator of a slowdeterministic simulator

Obtain runs on a carefully constructed set of design points

(eg. a Latin hypercube) easy to exploit parallel computinghardware here

For a stochastic simulator, many approaches are possible

(Mixtures of) Dirichlet processes (and related constructs) arepotentially quite flexible

Can also model output parametrically (say, Gaussian), withparameters modelled by (independent) Gaussian processesWill typically want more than one run per design point, inorder to be able to estimate distribution



Bayesian inference

Biological computer modelsProblemsReference


39/41

ayes a e e ceSummary and conclusions

e e e ce

Why are Systems Biology models interesting examples of

computer models?Models

Diverse class of models: fast/slow,spatial/non-spatial,deterministic/stochastic,discrete/continuous time/stateseven modelling the same biological process!Many parametersStructural uncertaintyGenuine interest in the (posterior distribution of the)parameters not just in prediction

DataHigh-dimensionalDiverse: high-resolution time-course data, coarse populationaveraged data, endpoint data,distributional data, individualspecific parameters/data, covariatesMultiple distinct sources of data for a given model



Bayesian inference



40/41

ySummary and conclusions

Interesting methodological problems

Calibration of fast and slowstochasticsimulators, usingindividual, averaged and distributional data

Dealing withheterogeneity cellcell, tissuetissue, or

organismorganismEmulationof slow stochastic simulators good models andfitting procedures

Experimental designfor stochastic computer models trade

offs between repetition and space-filling, etc.Utilising fast stochastic or deterministic approximatesimulators for a slow stochastic simulator



Bayesian inference



41/41


Further information

Stochastic Modelling for Systems BiologyAn accessible introduction to stochastic modellingof complex genetic and biochemical networks.Covers: biological modelling, biochemical reac-

tions, Petri nets, SBML, stochastic processes, sim-ulation algorithms (including Gillespie), case stud-ies, MCMC, and Bayesian inference for networkdynamics. ISBN: 1-58488-540-8

Contact details...

email: [email protected]: http://www.staff.ncl.ac.uk/d.j.wilkinson/

http://localhost/var/www/apps/conversion/tmp/scratch_10/[email protected]://www.staff.ncl.ac.uk/d.j.wilkinson/http://www.staff.ncl.ac.uk/d.j.wilkinson/http://localhost/var/www/apps/conversion/tmp/scratch_10/[email protected]

Biochemical Network

Documents