Page 1
Department of Computer Science,
University of Waikato, New Zealand
Eibe Frank
◼ WEKA: A Machine
Learning Toolkit
◼ The Explorer
• Classification and
Regression
• Clustering
• Association Rules
• Attribute Selection
• Data Visualization
◼ The Experimenter
◼ The Knowledge
Flow GUI
◼ Conclusions
Machine Learning with WEKA
Page 2
10/15/2018 University of Waikato 2
WEKA: the bird
Copyright: Martin Kramer ([email protected] )
Page 3
10/15/2018 University of Waikato 3
WEKA: the software
◼ Machine learning/data mining software written in
Java (distributed under the GNU Public License)
◼ Used for research, education, and applications
◼ Complements “Data Mining” by Witten & Frank
◼ Main features:
◆ Comprehensive set of data pre-processing tools,
learning algorithms and evaluation methods
◆ Graphical user interfaces (incl. data visualization)
◆ Environment for comparing learning algorithms
Page 4
10/15/2018 University of Waikato 4
WEKA: versions
◼ There are several versions of WEKA:
◆ WEKA 3.0: “book version” compatible with
description in data mining book
◆ WEKA 3.2: “GUI version” adds graphical user
interfaces (book version is command-line only)
◆ WEKA 3.3: “development version” with lots of
improvements
◼ This talk is based on the latest snapshot of WEKA
3.3 (soon to be WEKA 3.4)
Page 5
10/15/2018 University of Waikato 5
@relation heart-disease-simplified
@attribute age numeric
@attribute sex { female, male}
@attribute chest_pain_type { typ_angina, asympt, non_anginal, atyp_angina}
@attribute cholesterol numeric
@attribute exercise_induced_angina { no, yes}
@attribute class { present, not_present}
@data
63,male,typ_angina,233,no,not_present
67,male,asympt,286,yes,present
67,male,asympt,229,yes,present
38,female,non_anginal,?,no,not_present
...
WEKA only deals with “flat” files
Page 6
10/15/2018 University of Waikato 6
@relation heart-disease-simplified
@attribute age numeric
@attribute sex { female, male}
@attribute chest_pain_type { typ_angina, asympt, non_anginal, atyp_angina}
@attribute cholesterol numeric
@attribute exercise_induced_angina { no, yes}
@attribute class { present, not_present}
@data
63,male,typ_angina,233,no,not_present
67,male,asympt,286,yes,present
67,male,asympt,229,yes,present
38,female,non_anginal,?,no,not_present
...
WEKA only deals with “flat” files
Page 7
10/15/2018 University of Waikato 7
Page 8
10/15/2018 University of Waikato 8
Explorer: Exploring the data
◼ Data can be imported from a file in various
formats: ARFF, CSV, C4.5, binary
◼ Data can also be read from a URL or from an SQL
database (using JDBC)
◼ Pre-processing tools in WEKA are called “filters”
◼ WEKA contains filters for:
◆ Discretization, normalization, resampling, attribute
selection, transforming and combining attributes, …
Page 9
10/15/2018 University of Waikato 9
Page 10
10/15/2018 University of Waikato 10
Page 11
10/15/2018 University of Waikato 11
Page 12
10/15/2018 University of Waikato 12
Page 13
10/15/2018 University of Waikato 13
Page 14
10/15/2018 University of Waikato 14
Page 15
10/15/2018 University of Waikato 15
Page 16
10/15/2018 University of Waikato 16
Page 17
10/15/2018 University of Waikato 17
Page 18
10/15/2018 University of Waikato 18
Page 19
10/15/2018 University of Waikato 19
Page 20
10/15/2018 University of Waikato 20
Page 21
10/15/2018 University of Waikato 21
Page 22
10/15/2018 University of Waikato 22
Page 23
10/15/2018 University of Waikato 23
Page 24
10/15/2018 University of Waikato 24
Page 25
10/15/2018 University of Waikato 25
Page 26
10/15/2018 University of Waikato 26
Page 27
10/15/2018 University of Waikato 27
Page 28
10/15/2018 University of Waikato 28
Page 29
10/15/2018 University of Waikato 29
Page 30
10/15/2018 University of Waikato 30
Explorer: building “classifiers”
◼ Classifiers in WEKA are models for predicting
nominal or numeric quantities
◼ Implemented learning schemes include:
◆ Decision trees and lists, instance-based classifiers,
support vector machines, multi-layer perceptrons,
logistic regression, Bayes’ nets, …
◼ “Meta”-classifiers include:
◆ Bagging, boosting, stacking, error-correcting output
codes, locally weighted learning, …
Page 31
10/15/2018 University of Waikato 31
Page 32
10/15/2018 University of Waikato 32
Page 33
10/15/2018 University of Waikato 33
Page 34
10/15/2018 University of Waikato 34
Page 35
10/15/2018 University of Waikato 35
Page 36
10/15/2018 University of Waikato 36
Page 37
10/15/2018 University of Waikato 37
Page 38
10/15/2018 University of Waikato 38
Page 39
10/15/2018 University of Waikato 39
Page 40
10/15/2018 University of Waikato 40
Page 41
10/15/2018 University of Waikato 41
Page 42
10/15/2018 University of Waikato 42
Page 43
10/15/2018 University of Waikato 43
Page 44
10/15/2018 University of Waikato 44
Page 45
10/15/2018 University of Waikato 45
Page 46
10/15/2018 University of Waikato 46
Page 47
10/15/2018 University of Waikato 47
Page 48
10/15/2018 University of Waikato 48
Page 49
10/15/2018 University of Waikato 49
Page 50
10/15/2018 University of Waikato 50
Page 51
10/15/2018 University of Waikato 51
Page 52
10/15/2018 University of Waikato 52
Page 53
10/15/2018 University of Waikato 53
Page 54
10/15/2018 University of Waikato 54
Page 55
10/15/2018 University of Waikato 55
Page 56
10/15/2018 University of Waikato 56
Page 57
10/15/2018 University of Waikato 57
Page 58
10/15/2018 University of Waikato 58
Page 59
10/15/2018 University of Waikato 59
Page 60
10/15/2018 University of Waikato 60
Page 61
10/15/2018 University of Waikato 61
Page 62
10/15/2018 University of Waikato 62
Page 63
10/15/2018 University of Waikato 63QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.
Page 64
10/15/2018 University of Waikato 64QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.
Page 65
10/15/2018 University of Waikato 65QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.
Page 66
10/15/2018 University of Waikato 66
Page 67
10/15/2018 University of Waikato 67
Page 68
10/15/2018 University of Waikato 68
Page 69
10/15/2018 University of Waikato 69
Page 70
10/15/2018 University of Waikato 70
Page 71
10/15/2018 University of Waikato 71
Page 72
10/15/2018 University of Waikato 72
Page 73
10/15/2018 University of Waikato 73
Q uick Tim e™ and a TIFF (LZW) dec om press or are needed to s ee this p ic ture.
Page 74
10/15/2018 University of Waikato 74
Page 75
10/15/2018 University of Waikato 75
Page 76
10/15/2018 University of Waikato 76
Page 77
10/15/2018 University of Waikato 77
Page 78
10/15/2018 University of Waikato 78
QuickTime™ and a TIFF (LZW) decompressor are needed t o see this picture.
Page 79
10/15/2018 University of Waikato 79
QuickTime™ and a TIFF (LZW) decompressor are needed t o see this picture.
Page 80
10/15/2018 University of Waikato 80
Page 81
10/15/2018 University of Waikato 81
QuickTime™ and a TIFF (LZW) decompressor are needed t o see this picture.
Page 82
10/15/2018 University of Waikato 82
Page 83
10/15/2018 University of Waikato 83
Page 84
10/15/2018 University of Waikato 84
Page 85
10/15/2018 University of Waikato 85
Page 86
10/15/2018 University of Waikato 86
Page 87
10/15/2018 University of Waikato 87
Page 88
10/15/2018 University of Waikato 88
Page 89
10/15/2018 University of Waikato 89
Page 90
10/15/2018 University of Waikato 90
Explorer: clustering data
◼ WEKA contains “clusterers” for finding groups of
similar instances in a dataset
◼ Implemented schemes are:
◆ k-Means, EM, Cobweb, X-means, FarthestFirst
◼ Clusters can be visualized and compared to “true”
clusters (if given)
◼ Evaluation based on loglikelihood if clustering
scheme produces a probability distribution
Page 91
10/15/2018 University of Waikato 91
Page 92
10/15/2018 University of Waikato 92
Page 93
10/15/2018 University of Waikato 93
Page 94
10/15/2018 University of Waikato 94
Page 95
10/15/2018 University of Waikato 95
Page 96
10/15/2018 University of Waikato 96
Page 97
10/15/2018 University of Waikato 97
Page 98
10/15/2018 University of Waikato 98
Page 99
10/15/2018 University of Waikato 99
Page 100
10/15/2018 University of Waikato 100
Page 101
10/15/2018 University of Waikato 101
Page 102
10/15/2018 University of Waikato 102
Page 103
10/15/2018 University of Waikato 103
Page 104
10/15/2018 University of Waikato 104
Page 105
10/15/2018 University of Waikato 105
Page 106
10/15/2018 University of Waikato 106
Explorer: finding associations
◼ WEKA contains an implementation of the Apriori
algorithm for learning association rules
◆ Works only with discrete data
◼ Can identify statistical dependencies between
groups of attributes:
◆ milk, butter bread, eggs (with confidence 0.9 and
support 2000)
◼ Apriori can compute all rules that have a given
minimum support and exceed a given confidence
Page 107
10/15/2018 University of Waikato 107
Page 108
10/15/2018 University of Waikato 108
Page 109
10/15/2018 University of Waikato 109
Page 110
10/15/2018 University of Waikato 110
Page 111
10/15/2018 University of Waikato 111
Page 112
10/15/2018 University of Waikato 112
Page 113
10/15/2018 University of Waikato 113
Page 114
10/15/2018 University of Waikato 114
Explorer: attribute selection
◼ Panel that can be used to investigate which
(subsets of) attributes are the most predictive ones
◼ Attribute selection methods contain two parts:
◆ A search method: best-first, forward selection,
random, exhaustive, genetic algorithm, ranking
◆ An evaluation method: correlation-based, wrapper,
information gain, chi-squared, …
◼ Very flexible: WEKA allows (almost) arbitrary
combinations of these two
Page 115
10/15/2018 University of Waikato 115
Page 116
10/15/2018 University of Waikato 116
Page 117
10/15/2018 University of Waikato 117
Page 118
10/15/2018 University of Waikato 118
Page 119
10/15/2018 University of Waikato 119
Page 120
10/15/2018 University of Waikato 120
Page 121
10/15/2018 University of Waikato 121
Page 122
10/15/2018 University of Waikato 122
Page 123
10/15/2018 University of Waikato 123
Explorer: data visualization
◼ Visualization very useful in practice: e.g. helps to
determine difficulty of the learning problem
◼ WEKA can visualize single attributes (1-d) and
pairs of attributes (2-d)
◆ To do: rotating 3-d visualizations (Xgobi-style)
◼ Color-coded class values
◼ “Jitter” option to deal with nominal attributes (and
to detect “hidden” data points)
◼ “Zoom-in” function
Page 124
10/15/2018 University of Waikato 124
Page 125
10/15/2018 University of Waikato 125
Page 126
10/15/2018 University of Waikato 126
Page 127
10/15/2018 University of Waikato 127
Page 128
10/15/2018 University of Waikato 128
Page 129
10/15/2018 University of Waikato 129
Page 130
10/15/2018 University of Waikato 130
Page 131
10/15/2018 University of Waikato 131
Page 132
10/15/2018 University of Waikato 132
Page 133
10/15/2018 University of Waikato 133
Page 134
10/15/2018 University of Waikato 134
Page 135
10/15/2018 University of Waikato 135
Page 136
10/15/2018 University of Waikato 136
Performing experiments
◼ Experimenter makes it easy to compare the
performance of different learning schemes
◼ For classification and regression problems
◼ Results can be written into file or database
◼ Evaluation options: cross-validation, learning
curve, hold-out
◼ Can also iterate over different parameter settings
◼ Significance-testing built in!
Page 137
10/15/2018 University of Waikato 137
Page 138
10/15/2018 University of Waikato 138
Page 139
10/15/2018 University of Waikato 139
Page 140
10/15/2018 University of Waikato 140
Page 141
10/15/2018 University of Waikato 141
Page 142
10/15/2018 University of Waikato 142
Page 143
10/15/2018 University of Waikato 143
Page 144
10/15/2018 University of Waikato 144
Page 145
10/15/2018 University of Waikato 145
Page 146
10/15/2018 University of Waikato 146
Page 147
10/15/2018 University of Waikato 147
Page 148
10/15/2018 University of Waikato 148
Page 149
10/15/2018 University of Waikato 149
Page 150
10/15/2018 University of Waikato 150
The Knowledge Flow GUI
◼ New graphical user interface for WEKA
◼ Java-Beans-based interface for setting up and
running machine learning experiments
◼ Data sources, classifiers, etc. are beans and can
be connected graphically
◼ Data “flows” through components: e.g.,
“data source” -> “filter” -> “classifier” -> “evaluator”
◼ Layouts can be saved and loaded again later
Page 151
10/15/2018 University of Waikato 151
Page 152
10/15/2018 University of Waikato 152
Page 153
10/15/2018 University of Waikato 153
Page 154
10/15/2018 University of Waikato 154
Page 155
10/15/2018 University of Waikato 155
Page 156
10/15/2018 University of Waikato 156
Page 157
10/15/2018 University of Waikato 157
Page 158
10/15/2018 University of Waikato 158
Page 159
10/15/2018 University of Waikato 159
Page 160
10/15/2018 University of Waikato 160
Page 161
10/15/2018 University of Waikato 161
Page 162
10/15/2018 University of Waikato 162
Page 163
10/15/2018 University of Waikato 163
Page 164
10/15/2018 University of Waikato 164
Page 165
10/15/2018 University of Waikato 165
Page 166
10/15/2018 University of Waikato 166
Page 167
10/15/2018 University of Waikato 167
Page 168
10/15/2018 University of Waikato 168
Page 169
10/15/2018 University of Waikato 169
Page 170
10/15/2018 University of Waikato 170
Page 171
10/15/2018 University of Waikato 171
Conclusion: try it yourself!
◼ WEKA is available at
http://www.cs.waikato.ac.nz/ml/weka
▪ Also has a list of projects based on WEKA
▪ WEKA contributors:
Abdelaziz Mahoui, Alexander K. Seewald, Ashraf M. Kibriya, Bernhard
Pfahringer , Brent Martin, Peter Flach, Eibe Frank ,Gabi Schmidberger ,Ian H.
Witten , J. Lindgren, Janice Boughton, Jason Wells, Len Trigg, Lucio de
Souza Coelho, Malcolm Ware, Mark Hall ,Remco Bouckaert , Richard Kirkby,
Shane Butler, Shane Legg, Stuart Inglis, Sylvain Roy, Tony Voyle, Xin Xu,
Yong Wang, Zhihai Wang