Fuzzy-UCS: Preliminary Results Albert Orriols-Puig 1,2 Albert Orriols Puig Jorge Casillas 2 Ester Bernadó-Mansilla 1 1 Research Group in Intelligent Systems Enginyeria i Arquitectura La Salle, Ramon Llull University 2 Dept. Computer Science and Artificial Intelligence University of Granada
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Fuzzy-UCS: Preliminary Results
Albert Orriols-Puig1,2Albert Orriols PuigJorge Casillas2
Ester Bernadó-Mansilla1
1Research Group in Intelligent SystemsEnginyeria i Arquitectura La Salle, Ramon Llull University
2Dept. Computer Science and Artificial IntelligenceUniversity of Granada
Motivation
Michigan-style LCSs for supervised learning. Eg. XCS and UCS
Evolve online highly accurate models– Evolve online highly accurate models
– Competitive to the most-used machine learning techniques• (Bernadó et al, 02; Wilson, 02; Bacardit & Butz, 04; Butz, 06; Orriols & Bernadó, 07)
Main drawback: Intepretability of the rule sets– Number of rules or classifiers
• Reduction schemes (Wilson, 02; Fu & Davis, 02; Dixon et al., 03)
Intervalar representation of continuous attributes: [l u ]– Intervalar representation of continuous attributes: [li, ui] . Semantic-free variables
Slide 2GRSI Enginyeria i Arquitectura la Salle
Framework
Genetic Fuzzy SystemsChange the rule representation to fuzzy rules– Change the rule representation to fuzzy rules
– Provide a robust, flexible, and powerful methodology to deal with noisy imprecise and incomplete datanoisy, imprecise, and incomplete data.
Michigan-style Learning Fuzzy-Classifier Systems (LFCS)– (Valenzuela-Radón, 91 & 98)
– (Parodi & Bonelli, 93)
– (Furuhashi, Nakaoka & Uchikawa, 94)
– (Velasco, 98)
– (Ishibuchi, Nakashima & Murata, 99 & 05): First LFCS for pattern classification
– (Casillas, Carse & Bull, 07) Fuzzy-XCS
Slide 3GRSI Enginyeria i Arquitectura la Salle
Aim
Propose Fuzzy-UCSAccuracy based Michigan style LFCS– Accuracy-based Michigan-style LFCS
– Supervised learning scheme
– Derived from UCS (Bernado & Garrell, 2003)
• Introduction of a linguistic fuzzy representation
• Modification of all operators that deal with rules
– We expect:We expect:• Achieve similar performance than UCS
• Higher interpretability since we would deal with linguistic rules• Higher interpretability since we would deal with linguistic rules
• Lower number of fuzzy rules in the final population
E– Eg:IF x1 Є [l1, u1] ^ x2 Є [l2, u2] … ^ xn Є[ln, nn] THEN class1
– Matching instance e: for all ei: li ≤ ei ≤ ui
– Set of parameters:• Accuracy
• Fitness
• Numerosity• Numerosity
• Experience
• Correct set size
Slide 6GRSI Enginyeria i Arquitectura la Salle
Description of UCS1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p4. Conclusions
Environment
M t h S t [M]P bl i t
Stream ofexamples
Population [P]
1 C A acc F num cs ts exp3 C A acc F num cs ts exp5 C A acc F num cs ts exp
Match Set [M]Problem instance+
output class
1 C A acc F num cs ts exp2 C A acc F num cs ts exp3 C A acc F num cs ts exp4 C A acc F num cs ts exp
Population [P]
Classifier
6 C A acc F num cs ts exp…
correct set4 C A acc F num cs ts exp5 C A acc F num cs ts exp6 C A acc F num cs ts exp
…
ClassifierParameters
UpdateMatch set generation
Correct Set [C]
correct setgeneration
ExperienceCorrectacc #
=Selection, Reproduction, mutation
Deletion 3 C A acc F num cs ts exp6 C A acc F num cs ts exp
…
Correct Set [C]
p
νaccFitness =Genetic AlgorithmIf there are no classfiers in [C], covering is triggered
Slide 7GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCSp y
Describe the different components1. Rule representation and matching1. Rule representation and matching
2. Learning interaction
3 Di t3. Discovery component
4. Fuzzy-UCS in test mode
Slide 8GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCS1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p y
Rule representation
4. Conclusions
Rule representation– Linguistic fuzzy rules
– E.g.: IF x1 is A1 and x2 is A2 … and xn is An THEN class1
Disjunction of linguistic
All i bl h th ti
Disjunction of linguistic fuzzy terms
– All variables share the same semantics
– Example: Ai = {small, medium, large}
IF x1 is small and x2 is medium or large THEN class1
– Codification:
Slide 9GRSI Enginyeria i Arquitectura la Salle
IF [100 | 011] THEN class1
Description of Fuzzy-UCS1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p y
How do we know if a given input is small, medium or large?
4. Conclusions
g p , g– Each linguistic term defined by a membership function
Belongs to medium with a degree of 0 8Belongs to medium with a degree of 0.8
Belongs to small with a degree of 0 2
ei
Belongs to small with a degree of 0.2
Triangular-shaped membership functions
Attribute valuemembership functions
Slide 10GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCS1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p y
Matching degree uAk(e) [0,1]
4. Conclusions
g g A ( ) [ , ]
k: IF x1 is small and x2 is medium or large THEN class1
Example: (e1, e2)
0 8
0.2 0.2
0.8
T-conorm: bounded sum
e1 e2
max ( 1, 0.8 + 0.2) = 1
T-norm: productk
Slide 11GRSI Enginyeria i Arquitectura la Salle
uAk(e) = 1 * 0.2 = 0.2
Description of Fuzzy-UCS1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p y4. Conclusions
The role of matching changes:• UCS: A rule matches or not an example (binary function)• Fuzzy-UCS: A rule matches an example with a certain degree
Slide 12GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCS1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p y4. Conclusions
Each classifier has the following parameters:1 Weight per class w1. Weight per class wj
• Soundness with which the rule predicts the class j.
• The class value is dynamic and corresponds to the class j with higher w• The class value is dynamic and corresponds to the class j with higher wj
2. Fitness:• Quality of the rule
3. Other parameters directly inherited from UCS:• numerosity
• correct set size
• experience
Slide 13GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCSp y
Describe the different components1. Rule representation and matching1. Rule representation and matching
2. Learning interaction
3 Di t3. Discovery component
4. Fuzzy-UCS in test mode
Slide 14GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCS1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p y4. Conclusions
Learning interaction:– The environment provides an example e and its class c
– Match set creation: all classifiers that match with uAk(x) > 0
– Correct set creation: all classifiers that advocate cCorrect set creation: all classifiers that advocate c
– Covering: if there is not a classifier that maximally matches e• Create the classifier that match the input example with maximumCreate the classifier that match the input example with maximum
degree.
• Generalize the condition with probability P#
A2A1 A3For each variable:
Slide 15GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCS1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p y4. Conclusions
Parameters’ UpdateExperience:– Experience:
– Sum of correct matching per class j cmj:
Slide 16GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCS1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p y4. Conclusions
Parameters’ UpdateUse cm to update of the weights per each class:– Use cm to update of the weights per each class:
• Rule that only matches instances of class c:
• wc = 1
• For all the other classes j: wj = 0
– Calculate the fitness
Pressuring toward rules that correctly match instances of
only one classonly one class
Slide 17GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCSp y
Describe the different components1. Rule representation and matching1. Rule representation and matching
2. Learning interaction
3 Di t3. Discovery component
4. Fuzzy-UCS in test mode
Slide 18GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCS1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p y4. Conclusions
Discovery componentSteady state niched GA– Steady-state niched GA
– Roulette wheel selectionInstances that have a highergmatching degree have more
opportunities of being selected
Slide 19GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCS1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p y4. Conclusions
Discovery componentCrossover and mutation applied on the antecedent– Crossover and mutation applied on the antecedent
• 2 point crossoverIF [100 | 011] THEN classIF [100 | 011] THEN class1IF [101 | 100] THEN class1
• Mutation:
– Expansion IF [100 | 011] THEN class1 IF [101 | 011] THEN class1p
– Contraction
[ | ] 1 [ | ] 1
IF [100 | 011] THEN class1 IF [100 | 001] THEN class1
– Shift IF [100 | 011] THEN class1 IF [010 | 011] THEN class1
Slide 20GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCSp y
Describe the different components1. Rule representation and matching1. Rule representation and matching
2. Learning interaction
3 Di t3. Discovery component
4. Fuzzy-UCS in test mode
Slide 21GRSI Enginyeria i Arquitectura la Salle
Description of Fuzzy-UCS1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p y4. Conclusions
Class inference of a test example eWinner rule– Winner rule
• Inference: Select the rule that maximizes uAk(e) · Fk
• Reduction: Only keep in the final population that rules that maximize uA
k(e) · Fk at least for one training example
– Average vote• Inference: All experienced rules vote for the class they predict. The
most voted class is returned.
R d ti O l k i d l ith iti fit i th• Reduction: Only keep experienced rules with positive fitness in the final population
Slide 22GRSI Enginyeria i Arquitectura la Salle
Experimental Methodology1. Description of Fuzzy-UCS2. Experimental Methodology3. Results4 C l i
p gy4. Conclusions
Methodology– Compare Fuzzy-UCS to UCS, C4.5, and SMO.