Nature-inspired Smart Info Systems Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks1 Ronald L. Westra, Ralf L. M. Peeters,

Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks 1

Nature-inspired Smart Info Systems

Ronald L. Westra, Ralf L. M. Peeters,

Department of Mathematics

Maastricht University

Robust Identification of Piecewise LinearGene-Protein Interaction Networks

NISIS conference, Albufeira, October 4, 2005



1. Background and problem formulation

2. Modeling and identification of gene/proteins interactions

3. The implications of stochastic fluctuations and deterministic chaos

5. Example 1: Application on fission yeast expression data

5. Example 2: Application on artificial reaction model

5. Example 3: Application on Tyson-Novak model for fission yeast

6. Conclusions

Items in this Presentation



Questions:

* Can we identify (= reconstruct) gene regulatory networks from

time series of observations of (partial) genome wide and protein

concentrations?

* What is the influence of intrinsic noise and deterministic chaos

on the identifiability of such networks?

1. Problem formulation



Relation between mathematical model and phys-chem-biol reality

Macroscopic complexity from simple microscopic interactions

Approximate modeling as partitioned in subsystems with local

dynamics

Modeling of subsystems as piecewise linear systems (PWL)

PWL-Identification algorithms: network reconstruction from

(partial) expression and RNA/protein data

Experimental conditions of poor data: lots of gene but little data

The role of stochasticity and chaos on the identifiability

Problems in modeling and identification



2. Modeling the Interactions between Genes and Proteins

Prerequisite for the successful reconstruction of gene-protein networks is the way in which the dynamics of their interactions is modeled.



2.1 Modeling the molecular dynamics and reaction kinetics as Stochastic Differential Equations

Biochemical reactions and kinematic rate equations:

this is a microscopic reality:(in)elastic collisions, electrostatic forces, “binding”

this is a statistic average.true only under some conditions



Conditions for modeling reactions as rate equations

The law of large numbers. In inhomogeneous mixtures or in slow

reactions as in gene-, RNA-, and protein-interactions this will not

(always) be true. Hence; the problem is stochastic.

The Maxwell velocity distribution should apply, otherwise details

of the velocity distribution will enter. This condition is not met for

macromolecules in a cytoplasm.

The distribution of the internal degrees of freedom of the

constituents, like rotational and vibrational energies, must have

the same ’temperature’ as the Maxwell velocity distribution,

otherwise it will influence the rate of the collisions that result in a

chemical reaction. This condition is not met by gene/RNA/protein

interactions.



Gene-Protein Interaction Networks asPIECEWISE Linear Models

The general case is complex and approximate

Strongly dependent on unknown microscopic details

Relevant parameters are unidentified and thus unknown



Modeling of PWL Systems as subspace models

Global dynamics:

Local attractors (uniform, cycles, strange)

Basins of Attraction

Each BoA is a

subsystem Σi

“checkpoints”State space

Σ1

Σ2Σ3

Σ4

Σ5

Σ6



Modeling of PWL Systems as subspace models

State vector moves through state space

driven by local dynamics (attractor, repeller) and inputs

in each subsystem Σ1

the dynamics is governed by the local equilibria.

approximation of subsystem as linear statespace model:

State space



PWL Systems as flexible networks

For different biological processes the subsystem defines a different network structures

GG44

GG11

PP55

PP44

PP33

GG33

GG66

Σ2

GG22

GG11

PP22

PP11

PP33

GG33

Σ1



3. Identification of Interactions between Genes and Proteins



The identifiability of Piecewise Linear models from Microarray Timeseries

Sequence of genome-wide expression profiles at consequent instants become more realistic with decreasing costs …



Problems concerning the identifiability of Piecewise Linear models

1. Due to the huge costs and efforts involved in the experiments, only a limited number of time points are available in the data. Together with the high dimensionality of the system, this makes the problemseverely under-determined.

2. In the time series many genes exhibit strong correlation in their time-evolution, which is not per se indicative for a strong coupling between these genes but rather induced by the over-all dynamics ofthe ensemble of genes. This can be avoided by persistently exciting inputs.



Problems concerning the identifiability of Piecewise Linear models

3. Not all genes are observed in the experiment, and certainly most of the RNAs and proteins are not considered. therefore, there are many hidden states.

4. Effects of stochastic fluctuations on genes with low transcription factors are severe and will obscure their true dependencies.



Such are the problems relating to the identifiability of piecewise linear systems:

Are conditions for modeling rate equations met?

High stochasticity and chaos

Are piecewise linear approximations a valid metaphor?

Problems with stochastic modeling



The identification of PIECEWISE linear networks by L1-minimization

K linear time-invariant subsystems {Σ1, Σ2, .., ΣK}Continuous/Discrete time



4.2 The identification of PIECEWISE linear networks by L1-minimization

Weights wkj indicate membership of observation #k

to subsystem Σj :



Rich and Poor data

poor data: not sufficient empirical data is available to reliably estimate all system parameters, i.e. the resulting identification problem is under-determined.



(un)known switching times,regular sampling intervals,rich / poor data,

Identification of PWL models with known switching times and regular sampling intervals from rich data

Identification of PWL models with known switching times and regular sampling intervals from poor data



1. unknown switching times,regular sampling intervals,poor data, known state derivatives

This is similar to simple linear case



This can thus be written as:



with:



with:

The approach is as follows:

(i) initialize A, B, and W,

(ii) perform the iteration:1. Compute H1 and H2, using the simple linear system approach 2. Using fixed W, compute A and B,3. Using fixed A and B, compute W

until: (iii) criterion E has converged sufficiently – or a maximum number of iterations.



Linear L1-criterion:



With linear L1-criterion E1 the problem can be formulated as LP-problem:

LP1: compute H1,H2 from simple linear case

LP2: A and B, using E1-criterion and extra constraints for W, H1,H2,

LP3: compute optimal weights W, using E1-criterion with constraints for W, H1,H2, A and B



2. unknown switching times,regular sampling intervals,poor data, unknown state derivatives

Use same philosophy as mentioned before



Subspace dynamics and linear L1-criterion :



System parameters and empirical data :



Quadratic Programming problem QP :

Problem: not well-posed: i.e.: Jacobian becomes zero and ill-conditioned near optimum



Therefore split in TWO Linear Programming problems:



In case of sparse interactions replace LP1 with:



Performance of the robust Identification approach

Artificially produced data reconstructed with this approach

Compare reconstructed and original data

Here some results …



a: CPU-time Tc as a function of the problem size N, b: Number of errors as a function of the number of nonzero entries k,

M = 150, m = 5, N = 50000.



a: Number of errors versus M, b: Computation time versus M

N = 50000, k = 10, m = 0.



a: Minimal number of measurements Mmin required to compute A free of error versus the problem size N,

b: Number of errors as a function of the intrinsic noise level σA

N = 10000, k = 10, m = 5, M = 150, measuring noise B = 0.



The influence of increasing intrinsic noise on the identifiability.



4. The Implications of Stochastic fluctuations and Deterministic Chaos

4.1 Stochastic fluctuations : * some experimental and numerical results



the evolution of the expression of two coupled genes. The genes, with expressions x1 and x2, are coupled as:

with zero-mean Gaussian stochastic noise. Influence of increasing intrinsic noise level. The time steps dt relate to the strength of the noise.

Stochastic fluctuations



Influence of stochastic fluctuations on the evolution of the expression of two coupled genes.



Noise-induced control in single-cell gene expression

i. In experimental work on E. coli, Elowitz and Swain found that low intracellular copy numbers of molecules can limit the precision of gene regulation.

They found that: genotypic identical cells exhibit

substantial phenotypic variation this variation arises from stochasticity

in gene expressionthis variation is essential for many

biological processesprime factors in stochasticity are:

transcription rate, regulatory dynamics, and genetic factors .



ii. In stochastic simulations on Drosophila and Neurospora, Goldbeter and Gonze found that robust circadian oscillations can emerge at the cellular level, even when only a few tens of mRNA and protein molecules are involved. This shows how autoregulation processes at the cellular level allow the emergence of a coherent biological rhythm out of molecular noise.



iii. Steuer found that the addition of noise to a deterministic simulation model of the cell-cycle in fission yeast (Tyson-Novak model) could explain several experimental findings, such as the existence of quantized cycle times in double-mutant wee1−cdc25 cells.

Moreover, he found that his stochastic model led to the emergence of noise induced oscillations.



4. The Implications of Stochastic fluctuations and Deterministic Chaos

4.1 Stochastic fluctuations : * some experimental and numerical results

4.2 Deterministic chaos: * some remarks



Gene-Protein system with coupling strength λ

Consider a gene with expression x(t) that is coupled to an stimulating protein with density a(t).

cxtx

fta

txtatx)(

)1(

))(1).((.)1(

With f a sigmoid function.

Now consider the limit gene states x(t) as function of λ



Feigenbaum bifurcation as λ increases



Complex deterministic chaotic behaviour



Deterministic Chaos in a Gene-Protein system

This chaotic behaviour is beneficial for identification as it provides many independent data points to explore the dynamics in the basin of attraction.

In this way, chaos acts as the persistently exciting inputs in the linear-convergent case.



Applications



Example 1: how to apply this method on current data sets

Spellman et al. data for cell-cycle of fission yeast :

Components: 6179 genes measured for 18-24 irregular time instants

Processing: fuzzy C-means, gene annotation with Go term finder and Fatigo, net recontruction with identification algorithm



Spellman et al. data for cell-cycle of fission yeast :

Processing:

Selection of most up/down-regulated genes: 3107 from 6179

Clustering: fuzzy C-means: best outcome 23 clusters

Gene annotation with Go term finder (4th level) and Fatigo, both for biological process and cellular component

Net recontruction with identification algorithm on 23 clusters



Centroids after clustering 23 clusters



Gene ontology



Gene ontologyCluster 1GO Term Finder: The genes are involved in spindle pole during the cell cycle, with relations to microtubuli and chromosomal structure.FatiGO: The main cellular component is the chromosome.

Cluster 2GO Term Finder: The genes are involved in proliferation and replications, especially bud neck and polarized growth.FatiGO: The results found by the GO Term Finder are confirmed. …………….

Cluster 22GO Term Finder: Only a few annotations are found and there are many unknown genes. The genes are involved in respiration and reproduction. The main cellular components are the actin/cortical skeleton and the mitochondrial inner membrane.FatiGO: No further clear annotations are found.

Cluster 23GO Term Finder: The genes are involved in RNA processing. The main cellular components are the nucleus, the RNA polymerase complex and the ribonucleoprotein complex.FatiGO: The main cellular component is the ribonucleoprotein complex.



Reonstructed network



Example 2: artificial data of hierarchic/sparse network

Artificial reaction network with:

Components: 2 master genes with high transcription rates 3 slave genes with low transcription rates 4 agents (= RNA or proteins).

Processes: stimulation, inhibition, transcription, and reactions between ‘agents’



Dynamics:

– large hierarchic and sparse network

– implicit relation between genes with expression x

through agents (= proteins, RNA) with concentration a – system near equilibrium and small perturbations

– inputs: persistent excitation u



Dynamics:

– implicit system dynamics:

– linear statespace model makes gene interaction explicit:



Dynamics:

– estimate gene-gene interaction matrix A from empirical data:



reactions



reactions



reactions



Matlab-simulation

y(1) = - 0.03*x(1) + 0.2*(1-x(1))*a(2)^2 - 0.2*x(1)*a(3) ;y(2) = - 0.05*x(2) + 0.3*(1-x(2))*a(1) - 0.1*x(2)*a(4) ;y(3) = - 0.02*x(3) + 0.1*(1-x(3))*a(2) - 0.1*x(3)*a(1) ;y(4) = - 0.01*x(4) + 0.2*(1-x(4))*a(1)*a(2) - 0.2*x(4)*a(3)^2;y(5) = - 0.02*x(5) + 0.3*(1-x(5))*a(3) - 0.1*x(5)*a(1);y(6) = - 0.02*a(1) + 0.4*x(1) - 0.2*a(1)*a(2) - 0.1*a(1)*a(3)^3;y(7) = - 0.01*a(2) + 0.15*x(2) - 0.2*a(1)*a(2);y(8) = - 0.01*a(3) + 0.2*a(1)*a(2) - 0.1*a(1)*a(3)^3;y(9) = - 0.05*a(4) + 0.9*a(1)*a(3);

rate equations



Real network structure: implicit

2211

aa

33

44

55

dd

bb

cc

pp

gg gene

agent



Real network structure: explicit

2211

33 44 55

slave slaveslave

mastermaster









2211

33 44 55

Reconstructed network structure: low noise

master master

slave slave slave



2211

33 44 55

Reconstructed network structure: moderate noise

slave slaveslave

mastermaster



Reconstructed network structure: high noise (an example)

2211

33 44 55

slave masterslave

slavemaster



Example 3: influence of noise

Artificial model for infection with deterministic chaos and no intrinsic noise :

Components: 2 agents (= RNA or proteins).




Example 3: data of Tyson-Novak math. model for cell cycle

Tyson-Novak model for cell-cycle of fission yeast :

Components: 9 agents (= RNA or proteins).




The deterministic Tyson-Novak model.



The stochastic Tyson-Novak model.



Example: stochastic Tyson-Novak model



Example: stochastic Tyson-Novak model



5. Conclusions

The piecewise linear system is an attractive metaphor for modeling dynamic gene-protein interactions

Robust identification can efficiently reconstruct network structures of PWL systems for ‘poor’ data

Stochastic fluctuations mostly affect slave genes with low transcription rates

Strongest links (e.g. master genes) are most resistant to increasing noise



Discussion …

GG22

GG11

PP22

PP11

PP33

GG33

GG44

GG11

PP55

PP44

PP33

GG33

GG66

Σ1 Σ2

Nature-inspired Smart Info Systems Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks1 Ronald L. Westra, Ralf L. M. Peeters,

Documents