Improved Gene Expression Programming to Solve the Inverse Problem for Ordinary Differential Equations Kangshun Li Professor, Ph.D Professor, Ph.D College.

Improved Gene Expression Programming to Solve the Inverse Problem for Ordinary Differential Equations

Kangshun Kangshun LiLi Professor, Ph.DProfessor, Ph.D

College of Information, College of Information,

South China Agricultural University, ChinaSouth China Agricultural University, China

Hong KongHong Kong

December 6, 2014December 6, 2014

[email protected]

Outline of My Talk

Introduction

Inverse problems for ODEs

Improved GEP for the inverse problem of ODEs

Experiments

Conclusions and future research

Outline of My Talk

Introduction



Experiments

Conclusions

1. Introduction

Dynamic systems Their dominant features are complicated or non-linear.

They often change over time.

How to predict them?

Stock MarketWeather Forecast Population Trends

Features of such dynamic systems It’s difficult to find the functional relations among variables in the

complicated changing processes. It’s possible to find out the change rate or differential coefficient of

some variables.

1. Introduction

Ordinary Differential Equations (ODEs)

121213

232

11

1 xxxxxedtdx

exxdtdx

xdtdx

tx

t 　　　　　　　　

　　　　　　　　　　　　

1. Introduction

Inverse problems How to establish the ODEs based on previous data.

tt

t

t

etex

tex

ex

2

12

3

22

1

　　　

　　　　　

t 1x 2x 3x

0.00 1.000000 1.000000 1.000000

0.01 1.010050 1.030403 1.040555

0.02 1.020201 1.061627 1.082236

0.03 1.030455 1.093692 1.125074

0.04 1.040811 1.126619 1.169095

Canonical problem

Inverse problem

121213

232

11

1

　　　　　　　　

　　　　　　　　　　　　

xxxxxedtdx

exxdtdx

xdtdx

tx

t

1. Introduction

Example

121213

232

11

1 xxxxxedtdx

exxdtdx

xdtdx

tx

t 　　　　　　　　

　　　　　　　　　　　　

1. Introduction

Challenges of solving inverse problems With a few observed data, it’s difficult to create ODEs. It’s difficult to determine the model structure. It’s difficult to adjust parameters.

Outline of My Talk

Introduction



Experiments

Conclusions

A dynamic system can be expressed by: , and t denotes time.

A series of observed data collected at times . .

2. Inverse problems for ODEs

txtxtx n,,, 21

11211

11211

00201

,,,

,,,

,,,

mnmm

n

n

txtxtx

txtxtx

txtxtx

X

Approaches to solving inverse problems of ODEs: Linear modeling Autoregressive model

Moving Average model

Autoregressive Moving Average model

Pre-selected based on experience

Faced with complex data, it’s hard to select the right differential equation model.

Evolutionary modeling Genetic Programming (GP)

Gene Expression Programming (GEP)

2. Inverse problems for ODEs

Non-linear dynamical systems

Outline of My Talk

Introduction



Experiments

Conclusions

GEP Based on genome and phenomena.

Refer to the gene expression rule in the genetics.

Have advantages of both GP and GA.

GEP chromosome Q ×+×a×Q a a ba b b a a b a b a a b

× stands for the multiplication operation.

Q represents square root operation.

Segment without underline belongs to the Head.

Underlined segment is the Tail.

3. Improved GEP for the inverse problem of ODEs

An example of GEP coding Each gene describes an ODE.

A chromosome describes an ODE group.

- + * x1 ^ 2 x3 x2 2 x1 x2 + + x3 x1 x2 2 4 x1 x2 x3 8 + + 3 x1 ^ t 3 6 x1 x2 x3

|————gene1———| ————gene2——— | ———gene3——— |


-

+ *

x1 ^ 2 x3

x2 2

+

+ x3

x1 x2

+

+ 3

x1 ^

t 3

T1 T2 T3

3

2

31

3

3212

3221

1

txdtdx

xxxdtdx

xxxdtdx

The flowchart of GEP algorithm for the ODEs inverse problem


Share the same evolution framework with other evolutionary algorithms!

Initialize population Set control parameters

termination symbol

Functional set head

head length: 8

tail lengths ： 9

gene number: 3

Create initial population

genetic ： *+-1q*+3201321023

chromosome： *+-1q*+3201321023*-*1+*+*202312032*+*1q*+3210301323

population size ： 50


Fitness evaluation and chromosomes ranking


Calculate genes Traditional method

Convert the chromosome into the expression tree, and then solve it

via stacks.

Our approach

Gene Read & Compute Machine (GRCM) algorithm.

The procedure of converting the chromosome into the expression

tree can be avoided.


An example of GRCM algorithm


+ - sin a b c d e f

+ - sin a b c

+ - sin a b c

P

+ - sin(c) a b

P

+ a-b sin(c)

P

(a-b)+sin(c)

P

Generate training data and prediction data The Runge-Kutta is adopted in this phase, which is a iteration method for

simulating the ODE solutions. The RK4 formula is shown:


njKKKKtxtx

tKtxKtxKtxfK

tKtxKtxKtxfK

tKtxKtxKtxfK

ttxtxtxfK

valuesoriginalaretxtxtx

tiiiijiji

tjtnjntjtjii

tjtnjntjtjii

tjtnjntjtjii

jjnjjii

n

,3,2,1,)22(6

1)()(

)2

1,

2

1)(,,

2

1)(,

2

1(

)2

1,

2

1)(,,

2

1)(,

2

1(

)2

1,

2

1)(,,

2

1)(,

2

1(

,,,

,,

4,3,2,1,1

3,3,223,114,

2,2,222,113,

1,1,221,112,

211,

00201

Construction of fitness function It is constructed by the differences between X and X*, i.e. ∆=‖X-X* ‖


Genetic operators Selection

The Roulette selection is adopted, which means that he better the fitness, the greater probability an individual is reproduced to the next generation.

Mutation

The Head can be mutated into any function or terminal symbol, while the Tail can only be mutated into the terminal symbol

Transportation

Insertion Sequence Transposition

Root Insertion Sequence Transposition

Gene Transposition


Reconstruction

Single point restructuring

Double-point restructuring

Gene restructuring

Termination conditions

The maximum number of generations is reached.

The fitness of the best individual reaches a predefined value, or it is unchanged for a predefined number of generations.


Outline of My Talk

Introduction



Experiments

Conclusions

Four different datasets are used.

GP and the basic GEP are involved in the comparison.

Three different metrics are compared. Training standard deviation

Prediction

Running time

4. Experiments

Four different datasets are used.

GP and the basic GEP are involved in the comparison.

Three different metrics are compared. Training standard deviation

Prediction

Running time

4. Experiments

Datasets

4. Experiments

Results

4. Experiments

Running time

Almost performs similar with the standard GEP.

Significantly less than the GP algorithm.

Stability

Better than using the GP algorithm, particularly for complex problems.

Prediction accuracy

Better than standard GEP for each dataset

Also be superior to the standard deviation of GP algorithm

4. Experiments

An improved GEP is proposed to solve the inverse problem of ODE

Overcome the shorting of evolution operations in the recessive segment.

Provide a better way to model dynamic systems.

5. Conclusions and future research

Stock MarketStock Market

Weather Forecast Population Trends

Thank you!Thank you!Q&AQ&A

Improved Gene Expression Programming to Solve the Inverse Problem for Ordinary Differential Equations Kangshun Li Professor, Ph.D Professor, Ph.D College.

Documents

inverse problems of

inverse problem of odesshare

odesimproved gep

inverse problemswith

x1 x2 x3 x1 x2

ba b b

x1 x2 x3gene1 gene2

differential coefficient