Complex system

Colon Tumor Classification using various Neural Network Models

coupled with Multi-Objective Evolutionary Optimization Schemes

Anirudh Munnangi

Chandrasekar Venkatesh

Ahmed Sageer Cheriya Melat

This project is an implementation of Neural Networks, with hybrid evolutionary algorithm optimizing multiple objectives for classification.

Optimizing three objectives: Pareto non dominated sorting genetic algorithm based optimization is done on norm of the weights, mean norm square errors and complexity of the network.

The evolutionary algorithm is applied to Radial Basis Function Networks (RBFNs) and on Multi-Layer Perceptron networks (MLPs), these algorithms are applied to classify real world two class colon tumor data.

Abstract

Data is real world two class colon tumor data

Data set consist of 62 data points

Data points have feature space of 2000

By principal component analysis method, the dimension of data points is reduced to 47 feature space

Data processing

Important Concepts

• Radial Basis Function Networks• Multi Layer Perceptron• Pareto Optimality• Genetic Algorithm

Radial Basis Function Networks

Radial Basis Function NetworksThe following equation defines the process which is followed by the RBFN map.

𝑌 2

𝑌 1

1

1

W20

W21

W22

Wm2

Wm1

W12

W10W11

∑

∑

X1

Xn

Input: 47 dimensional points

Output: 2 bits representing class

Number of hidden layers: 1Number of neurons in each layer: 20 or more

Multi Layer Perceptron

Multi Layer Perceptron

Xn

W2n

∑

∑

∑

𝑌 2

𝑌 1

W21

W1n

Wmn

Wm1

W11

∑

∑

X1

Input: 47 dimensional points Output: 2 bits

representing class

Number of hidden layers: 1Number of neurons in each layer: 20 or more

Pareto Optimality

Pareto Optimality

The solution is pareto optimal if

Objective functions considered in this project: - Mean norm square error

- Complexity map of the neural network

- norm of all the weights in the network

Genetic Algorithm

Genetic Algorithm

• Population size = 20• Each member represents Mean norm square error, complexity

of the network and norm of weights• For each iteration

• Perform Non dominated sort (Pareto optimization)• Choose fittest, crossover and form child population• Combine parent and child population, perform non-

dominated sort• Form new population

Program Flow

2.6 2.8 3 3.2 3.4 3.6 3.8 4 4.2 4.4 4.60

10

20

30

40

50

60

f3=||w||

f1=

Mean n

orm

Square

Err

or

Convergence of Pareto Front

NSGA using RBFN

Optimized RBFN with the best trade-off between objectives

0 5 10 15 20 25 30 35 40 452

2.5

3

3.5

4

4.5

5

5.5

6

6.5

7Convergence of Pareto Front

Objective function f2

Obje

ctive f

unction f

1

RBFN with NSGA

f1=MSE f2=Active hidden kernels

f1=MSE f2=||w||f1=Active hidden kernels f2=||w||

Performance in different runs

0 10 20 30 40 50 60 702

3

4

5

6

7

8

9

10Convergence of Pareto Front

f3=||w||

Mea

n N

orm

Squ

are

Err

or

RBFN NSGA

MLP NSGA

Comparative analysis of the performance of RBFN Vs MLP

INFERENCE• For the data set we used, optimized RBFN model seems to perform

better than MLP.

• Pareto optimization for multiple objectives works well with will all objectives being achieved.

FURTHER WORK• Can try more objectives, different parameters to optimize• Try optimizing other network models apart from RBFN and MLP, maybe

try optimizing for a combination of networks.

REFERENCES

1. Sultan Noman Qasem, Siti Mariyam Shamsuddin, Azlan Mohd Zain “Multi-objective hybrid evolutionary algorithms for radial basis function neural network design” Knowledge-Based Systems 27 (2012) 475–497 25 November 2011

2. Sultan Noman Qasema, Siti Mariyam Shamsuddina, “Memetic Elitist Pareto Differential Evolution algorithm based Radial basis function networks for classification problems” Neurocomputing; Applied Soft Computing 11 (2011) 5565–5581- 6 May 2011

3. Aimin Zhou, Bo-Yang Qu, Hui Li, Shi Zheng Zhao, Ponnuthurai Nagaratnam Suganthan, Qingfu Zhang, “Multiobjective evolutionary algorithms: A survey of the state of the art” Swarm and Evolutionary Computation 1 (2011) 32–49; 16 March 2011

4. Illya Kokshenev, Antonio Padua Braga, “An efficient multi-objective learning algorithm for RBF neural network. Neurocomputing 73(2010)2799–2808, 22August2010

5. Sultan Noman Qasem and Siti Mariyam Shamsuddin, Radial Basis Function Network based on time variant multiobjective particle swarm optimization for medical disease analysis.

6. Jonathan E. Fieldsend and Sameer Singh, “Pareto Evolutionary Neural Networks” IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 16, NO. 2, MARCH 2005

7. Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and T. Meyarivan, “A Fast and Elitist Multiobjective Genetic Algorithm- NSGA-II” IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 6, NO. 2, APRIL 2002

8. David Lahoz and Pedro Mateo, “Neural Network Ensembles for Classification Problems using Multiobjective Genetic Algorithms”9. U. Alon, et al. "Broad Patterns of Gene Expression Revealed by Clustering Analysis of Tumor and Normal Colon Tissues Probed by

Oligonucleotide Arrays", PNAS, 96:6745-6750, 1999