BioHEL System Our approach Results Summary Post-processing Operators for Decision Lists María A. Franco Supervisor: Jaume Bacardit University of Nottingham, UK, ICOS Research Group, School of Computer Science [email protected]June 12, 2012 María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 1 / 29
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
BioHEL SystemOur approach
ResultsSummary
Post-processing Operators forDecision Lists
María A. Franco
Supervisor: Jaume BacarditUniversity of Nottingham, UK,
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 1 / 29
BioHEL SystemOur approach
ResultsSummary
Motivation
Goal of my PhD projectTo enhance evolutionary learning systems based on IRL(BioHEL) to work better with large scale datasets.
How have we been doing this?Analysing the weaknesses of the system in differentdomains [Franco et al., 2012a]
Improving the execution time by means of GPGPUs[Franco et al., 2010]
Developing theoretical models that allow us to adaptparameters within the system [Franco et al., 2011]
Improving the quality of the final solutions by means oflocal search (memetic operators) [Franco et al., 2012b]
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 2 / 29
BioHEL SystemOur approach
ResultsSummary
Motivation
Goal of my PhD projectTo enhance evolutionary learning systems based on IRL(BioHEL) to work better with large scale datasets.
How have we been doing this?Analysing the weaknesses of the system in differentdomains [Franco et al., 2012a]
Improving the execution time by means of GPGPUs[Franco et al., 2010]
Developing theoretical models that allow us to adaptparameters within the system [Franco et al., 2011]
Improving the quality of the final solutions by means oflocal search (memetic operators) [Franco et al., 2012b]
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 2 / 29
BioHEL SystemOur approach
ResultsSummary
Motivation
Goal of my PhD projectTo enhance evolutionary learning systems based on IRL(BioHEL) to work better with large scale datasets.
How have we been doing this?Analysing the weaknesses of the system in differentdomains [Franco et al., 2012a]
Improving the execution time by means of GPGPUs[Franco et al., 2010]
Developing theoretical models that allow us to adaptparameters within the system [Franco et al., 2011]
Improving the quality of the final solutions by means oflocal search (memetic operators) [Franco et al., 2012b]
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 2 / 29
BioHEL SystemOur approach
ResultsSummary
Motivation
Goal of my PhD projectTo enhance evolutionary learning systems based on IRL(BioHEL) to work better with large scale datasets.
How have we been doing this?Analysing the weaknesses of the system in differentdomains [Franco et al., 2012a]
Improving the execution time by means of GPGPUs[Franco et al., 2010]
Developing theoretical models that allow us to adaptparameters within the system [Franco et al., 2011]
Improving the quality of the final solutions by means oflocal search (memetic operators) [Franco et al., 2012b]
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 2 / 29
BioHEL SystemOur approach
ResultsSummary
Motivation
Goal of my PhD projectTo enhance evolutionary learning systems based on IRL(BioHEL) to work better with large scale datasets.
How have we been doing this?Analysing the weaknesses of the system in differentdomains [Franco et al., 2012a]
Improving the execution time by means of GPGPUs[Franco et al., 2010]
Developing theoretical models that allow us to adaptparameters within the system [Franco et al., 2011]
Improving the quality of the final solutions by means oflocal search (memetic operators) [Franco et al., 2012b]
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 2 / 29
BioHEL SystemOur approach
ResultsSummary
Motivation
Goal of this workTo improve the quality of the decision lists by means of localsearch (memetic operators)
Decision lists are a widespread paradigm in rule learning,guided local search and supervised learning.
ExamplePittsburgh Learning Classifier SystemsRule induction systems in mainstream machine learning(PART, CN2, JRip)
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 3 / 29
BioHEL SystemOur approach
ResultsSummary
Motivation
Goal of this workTo improve the quality of the decision lists by means of localsearch (memetic operators)
Decision lists are a widespread paradigm in rule learning,guided local search and supervised learning.
ExamplePittsburgh Learning Classifier SystemsRule induction systems in mainstream machine learning(PART, CN2, JRip)
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 3 / 29
BioHEL SystemOur approach
ResultsSummary
Outline
1 BioHELAttribute List Knowledge RepresentationStructure of the solutionsWhat is the problem?
2 Our approach: Post-processing the rulesSwappingPruningCleaning
3 Results
4 SummaryWhere to go from here?
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 4 / 29
BioHEL SystemOur approach
ResultsSummary
BioHEL SystemAttribute List Knowledge RepresentationStructure of the solutionsWhat is the problem?
Introduction to the BioHEL System
BIOinformatics-oriented Hierarchical Evolutionary Learning- BioHEL [Bacardit et al., 2009]
BioHEL is an evolutionary learning system that employsthe Iterative Rule Learning (IRL) paradigmBioHEL was especially designed to cope with large scaledatasets
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 5 / 29
BioHEL SystemOur approach
ResultsSummary
BioHEL SystemAttribute List Knowledge RepresentationStructure of the solutionsWhat is the problem?
Attribute List Knowledge Representation
Meta-representation to handle large amount of discreteand continuous attributes fast [Bacardit and Krasnogor, 2009].
ALKR Classifier Example
numAtt
predicates
class
whichAtt
3
0
0.70.5
1
0.3
offsetPred 0
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 6 / 29
BioHEL SystemOur approach
ResultsSummary
BioHEL SystemAttribute List Knowledge RepresentationStructure of the solutionsWhat is the problem?
Swapping is very slow... It depends on the number of instancesand number of rules generated.
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 25 / 29
BioHEL SystemOur approach
ResultsSummary
Where to go from here?
Summary and next steps
SummaryThe operators manage to reduce the number of rules andexpressed attributes in 30% in some cases.
Next stepsApply the CL and PR operators during the learning processInvestigate other measures of similarities among rulesApply these operators over other systems
Different representations
CUDA accelerated operators?
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 26 / 29
BioHEL SystemOur approach
ResultsSummary
Where to go from here?
Summary and next steps
SummaryThe operators manage to reduce the number of rules andexpressed attributes in 30% in some cases.
Next stepsApply the CL and PR operators during the learning processInvestigate other measures of similarities among rulesApply these operators over other systems
Different representations
CUDA accelerated operators?
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 26 / 29
BioHEL SystemOur approach
ResultsSummary
Where to go from here?
References I
Bacardit, J., Burke, E., and Krasnogor, N. (2009).Improving the scalability of rule-based evolutionary learning.Memetic Computing, 1(1):55–67.
Bacardit, J. and Krasnogor, N. (2009).A mixed discrete-continuous attribute list representation for large scale classification domains.In GECCO ’09: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, pages1155–1162, New York, NY, USA. ACM Press.
Franco, M., Krasnogor, N., and Bacardit, J. (2012a).Analysing biohel using challenging boolean functions.Evolutionary Intelligence, 5:87–102.10.1007/s12065-012-0080-9.
Franco, M. A., Krasnogor, N., and Bacardit, J. (2010).Speeding up the evaluation of evolutionary learning systems using GPGPUs.In GECCO ’10: Proceedings of the 12th annual conference on Genetic and evolutionary computation, pages1039–1046, New York, NY, USA. ACM.
Franco, M. A., Krasnogor, N., and Bacardit, J. (2011).Modelling the initialisation stage of the alkr representation for discrete domains and gabil encoding.In Proceedings of the 13th annual conference on Genetic and evolutionary computation, GECCO ’11, pages1291–1298, New York, NY, USA. ACM.
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 27 / 29
BioHEL SystemOur approach
ResultsSummary
Where to go from here?
References II
Franco, M. A., Krasnogor, N., and Bacardit, J. (2012b).Postprocessing operators for decision lists.In GECCO ’12: Proceedings of the 14th annual conference comp on Genetic and evolutionary computation,page to appear, New York, NY, USA. ACM Press.
Venturini, G. (1993).SIA: a supervised inductive algorithm with genetic search for learning attributes based concepts.In Brazdil, P. B., editor, Machine Learning: ECML-93 - Proceedings of the European Conference on MachineLearning, pages 280–296. Springer-Verlag.
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 28 / 29
BioHEL SystemOur approach
ResultsSummary
Where to go from here?
Questions or comments?
María A. Franco. University of Nottingham Post-processing Operators for Decision Lists 29 / 29