Robust Pareto Design of GMDH-type Neural Networks for Systems with Probabilistic Uncertainties N. Nariman-zadeh, F. Kalantary, A. Jamali, F. Ebrahimi Faculty of Engineering, The University of Guilan
Jan 04, 2016
Robust Pareto Design of GMDH-type Neural Networks for Systems with Probabilistic Uncertainties
N. Nariman-zadeh, F. Kalantary, A. Jamali, F. Ebrahimi
Faculty of Engineering,
The University of Guilan
•System identification techniques are applied in many fields in order to model and predict the behaviors of unknown and/or very complex systems based on given input-output data
•GMDH is a self-organizing approach by which gradually complicated models are generated based on the evaluation of their performances on a set of multi-input-single-output data
•In order to obtain more robust models, it is required to consider all the conflicting objectives, namely, training error (TE), prediction error (PE) in the sense of multi-objective Pareto optimization process
•For multi-objective optimization problems, there is a set of optimal solutions, known as Pareto optimal solutions or Pareto front
Introduction
•System Identification Techniques Are Applied in Many Fields in order to Model and Predict the Behaviors of Unknown and/or Very Complex Systems Based on Given Input-Output Data.
•Group Method of Data Handling (GMDH) Algorithm is Self-Organizing Approach by which Gradually Complicated Models are Generated Based on the Evaluation of their Performances on a set of Multi-Input-Single-Output Data Pairs (i=1, 2, …, M)
X1
X2
Xn
Y1
.
.Ym
Modelling Using GMDH-type Networks
The classical GMDH algorithm can be represented as set of neurons in which different pairs of them in each layer are connected through a quadratic polynomial and thus produce new neurons in the next layer.
G1
G2
G4
G6
X1
X2
X3
X4
A Feedforward GMDH-Type Network
G3
G5
Input Layer
Output Layer
Hidden Layer(s)
Modelling Using GMDH-type Networks
A Generalized GMDH Network Structure of a Chromosome
a
c
b
d
ad
bc
adbc
a d b c b c b c
LayerHiddenNeuronofLength 2.
Application of Genetic Algorithm in the Topology Design of GMDH-type NNs
a
c
b
d
ad
bc
adbc
a d b c d d d d
Application of Genetic Algorithm in the Topology Design of GMDH-type NNs
Crossover operation for two individuals in GS-GMDH networks
Application of Singular Value Decomposition to the Design of GMDH-type Networks
SVD is the method for solving most linear least squares problems that some singularities may exist in the normal equations YA a
The SVD of a matrix, , is a factorization of the matrix into the product of three matrices, matrix , diagonal matrix with non-negative elements (Singular Values), and orthogonal matrix such that :
M NA R M NU R
N NW R N NV R
. . TA U W V YUw
diagV T
j
..)1
(.a
Genetic Algorithms and Multi-objective Pareto Optimization
Genetic algorithms are iterative and stochastic
optimization techniques.
In the optimization of complex In the optimization of complex real-world problems, there are real-world problems, there are
several objective functions to be several objective functions to be optimized simultaneously.optimized simultaneously.
In the optimization of complex In the optimization of complex real-world problems, there are real-world problems, there are
several objective functions to be several objective functions to be optimized simultaneously.optimized simultaneously.
There is There is no single optimal solutionno single optimal solution as the best because objectives as the best because objectives
conflict each other.conflict each other.
There is There is no single optimal solutionno single optimal solution as the best because objectives as the best because objectives
conflict each other.conflict each other.There is a set of optimal solutions, There is a set of optimal solutions,
well known as well known as Pareto optimal Pareto optimal solutions solutions oror Pareto front. Pareto front.
There is a set of optimal solutions, There is a set of optimal solutions, well known as well known as Pareto optimal Pareto optimal
solutions solutions oror Pareto front. Pareto front.
Modelling error
Multi-objective optimization
Modelling error
Multi-objective optimization
Difference between robust optimization and traditional optimization
Design Variable
Obj
ectiv
e F
unct
ion
Feasible
Infeasible
Optimal solution
Robust optimal solution
Random variable
0.25
0.50
0.75
1.00
CDF
p
x
XX dxxfxXxF Pr
xf X
dxxfxxdFXE XX
dxxfXExX X
2
For the discrete sampling:
N
iixN
XE1
1
N
ii XEx
NX
1
22
11
Stochastic Robust Analysis
•Modelling and prediction of soil shear strength, Su , based on 5 input parameters, namely, SPT number (Standard Penetration Test) N′, effective overburden stress s/
0, moisture content percent W , LL liquid limit, and PL plastic limit of fine-graded clay soil
•The data used in this study were gathered from the National Iranian Geotechnical Database, which has been set up in the Building and Housing Research Centre (BHRC)
•The database has been established under a mandate from the Management and Planning Organization (MPORG), which supervises the professional activities of all of the consultancy firms in Iran
Comparison of actual values with the evolved GMDH model corresponding to optimum point C (nominal table)
Training set Prediction set
Point Network’s structure TE PE Mean of TE Mean of PE Variance of TE Variance of PE
A bbaebcacbcaeacee 133.12 48.49 323.76 161.49 174862.64 42019.59
B bcaebacdbcbbadde 79.20 260.15 73785.2 17844.7 3.8e11 3.3e9
C bcaebccdbdbcaccd 89.79 75.30 28366.5 709.8 3.7e10 2.6e6
Objective functions and structure of networks of different optimum design points
Point Network’s structure TE PE Mean of TE
Mean of PE Variance of TE Variance of PE
A bbaebcacbcaeacee 133.12 48.49 323.76 161.49 174862.64 42019.59B bcaebacdbcbbadde 79.20 260.15 73785.2 17844.7 3.8e11 3.3e9C bcaebccdbdbcaccd 89.79 75.30 28366.5 709.8 3.7e10 2.6e6D abeecddd 132.79 237.59 234 .61 248.03 178.77 1174.283
Objective functions and structure of networks of different optimum design points
Y1
Y4
Y3
Y5Y2
Point CPoint D
The structure of network corresponding to point C and D
Y1=-5.94+ 0.65 N’ + 0.76 σ0’ -0.0083 N’2 - 0.0019 σ0
’2 + 0.0013 N’ σ0’
Y2= 25.42 - 2.76w + 1.86LL - 0.019w2 - 0.045LL2 + 0.11w(LL)
Y3= 16.99 + 0.82Y2 - 1.27LL - 0.0015Y22 + 0.016(LL)2 + 0.015(Y2)(LL)
Y4= 10.16 + 0.74Y1 - 0.22PL - 0.019Y12 - 0.034PL2 + 0.056(Y1)(PL)
Y5= 16.12 + 0.83Y4 - 0.64Y3 - 0.0004Y42 + 0.0060Y32+ 0.0036(Y4)(Y3)
Conclusion
• A multi-objective genetic algorithm was used to optimally design GMDH-type neural networks from a robustness point
of view in a probabilistic approach.
• Multi-objective optimization of robust GMDH models led to the discovering some important trade-off among those
objective functions.
• The framework of this work is very promising and can be generally used in the optimum design of GMDH models in
real-world complex systems with probabilistic uncertainties.
Thanks for your attention…