Department of Computer Science and Engineering, National Institute ofTechnology, Warangal – 506004 Learning Bayesian Classifiers Using Differential Evolution algorithm for Variable Ordering Project Guide: Dr. S. G. Sanjeevi (Head of the Department) – Associate Professor 12/31/2011
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Shruti B – 8772 Mouli C R K – 8792 Divya B V – 8773
PREVIOUS EXPERIMENTS Genetic algorithms like VOGA and VOGA+ have been used for optimizing the learning of BC from data
process by means of the identification of a suitable VO. In these genetic algorithms each element of thepopulation is a possible ordering and their fitness is the K2 metric (g value). Evolutionary algorithms with
canonical crossover and mutation have also been used to find an appropriate VO.
Evolutionary Algorithms (EAs):
EAs are computational models that solve a given problem by maintaining a changing population of
chromosomes, each with its own level of ‘fitness’. A fitness function is used to measure the quality of
each chromosome. Genetic algorithms are most popular models of EAs. Differential Evolution
algorithms are also a class of Evolutionary Algorithms.
VOGA (Variable Ordering Genetic Algorithm):
What is VOGA?
The main idea in the proposed method is to use a GA and the class variable information to optimize the
variable ordering (VO) which will be used as an input to learn a BC from data. In this sense, we fix the
class variable as the first one in the VO. Subsequently, the GA is used trying to find the best ordering for
the remaining variables. Our method uses a GA in which the chromosomes represent possible variables
ordering. The variables identification (ID) is codified as an integer number. Therefore, each chromosome
has (n – 1) genes, where n is the number of variables (including the class variable) and each gene is
instanced with a variable ID. Thus, each possible ordering may form a chromosome. The fitness function
is given by the Bayesian score (g function) defined in K2 algorithm.
How it is implemented?
VOGA generates a random initial population. Each chromosome is evaluated by the K2 algorithm whose
function g is used as fitness function. The best chromosomes are selected, and using crossover and
mutation operators the next generation is generated. The process is repeated and for each generation
the best ordering is stored. If there is no improvement after 10 generations, the algorithm locks up and
returns the best found ordering. The flowchart summarizes the process all.
Shruti B – 8772 Mouli C R K – 8792 Divya B V – 8773
Experiment:
Seven domains were used in our simulations. Two well-known Bayesian Network domain (Engine Fuel
System and Asia) and five benchmark problems from the U. C. Irvine repository1 were used in the VO
and classification task, namely, Balance, Breast – w, Congressional Voting Records (Voting), Vehicle and
Iris. The following table summarizes the data set features.
Asia Balance Breast – w Engine Iris Vehicle Voting
AT 8 5 10 9 5 19 17
IN 15000 625 683 15000 150 846 232
CL 2 3 2 2 3 4 2
Datasets Description with dataset name (Data), number of attributes plus class (AT), number of instances (IN) and number of classes (CL).
The experiments were conducted following the steps below.
1. Initially, the datasets had been used as input to the K2 algorithm. The VO was the original one
given in the file. The Bayesian score (g) obtained to each dataset was stored.
2. The same datasets used in step 1 had been used as input to VOGA and VOGA+. The Bayesian
score (g) obtained to each dataset and the number of generations necessary to reach the
solution were stored.
Results achieved in steps 1 and 2 are presented in the following tables respectively.
Asia Balance Breast – w Engine Iris Vehicle Voting
K2 -33610 -4457 -8159 -33809 -2026 -10357 -1749
VOGA -33610 -4457 -8159 -33755 -2026 -10006 -1727
VOGA+ -33608 -4457 -8159 -33755 -2026 -9956 -1724Bayesian Score (g function) of each achieved Bayesian Network Structure. The best results in each dataset are in bold face.
Analyzing results presented in the above Table, it is possible to infer that, as far as the Bayesian score (g
function) is concerned, in all performed experiments, VOGA produced results at least as good as the
Shruti B – 8772 Mouli C R K – 8792 Divya B V – 8773
ones produced by K2 and in 3 out of the 7 datasets VOGA improved the results obtained using K2. In
addition, VOGA+ performed at least as well as VOGA and in 3 out of the 7 datasets VOGA+ improved the
results obtained using VOGA.
Another interesting issue revealed in Table 2 is that datasets having higher number of attributes, namelyVehicle (19 attributes) and Voting (17 attributes) favored the proposed method (VOGA), mainly when
using the enhanced version VOGA+.
Asia Balance Breast – w Engine Iris Vehicle Voting
VOGA 11 11 11 13 11 11 38
VOGA+ 19 11 11 12 11 15 6
Number of generations needed until convergence.
When the number of generations is concerned, in 4 (Balance, Breast-w, Engine and Iris) out of the 7
datasets VOGA and VOGA+ presented (mostly) the same results. The other 3 datasets (Asia, Vehicle and
Voting) revealed that, when the number of generations was not the same for VOGA and VOGA+, the
Bayesian score obtained by the later one was always better.