Types of Cost
Nov 29, 2014
Types of Cost
What are the types of cost that are involved in Inductive Learning?
In real-world applications, there are many different types of cost.
Most Machine Learning (ML) literature largely ignores all types of cost.
The only exception is Constant Error Cost.
Why is this important?
Many ML papers ignore many of the cost types. By ignoring methods of cost, ML does not work
most effectively in real life situations. A taxonomy may help to organize the literature on
cost-sensitive learning. Motivation is to inspire researchers to investigate all
types of cost in inductive concept learning in more depth.
Taxonomy
Cost of Misclassification of Errors Cost of Tests Cost of Teacher Cost of Intervention Cost of Unwanted Achievements Cost of Computation Cost of Cases Human-Computer Interaction Cost Cost of Instability
Constant Error Cost
1 2 … i1 0 1 1 12 1 0 1 1… 1 1 0 1j 1 1 1 0
1 2 … i1 1 0 0 02 0 1 0 0… 0 0 0 0j 0 0 0 1
Error-Rate Accuracy
Cost of Misclassification of Errors
Constant Error Cost Conditional Error Cost
Individual Case Time of Classification Classification of Other Cases Feature Value
Cost of Tests
Constant Cost Test Conditional Cost Test
Prior Test Selection Prior Test Results True Class of Case Test Side-Effects Individual Case Time of Test
Other Costs
Cost of Teacher Constant Conditional
Cost of Intervention Constant Conditional
Cost of Unwanted Achievements Constant Conditional
Cost of Instability
Cost of Computation
Static Complexity Size Complexity Structural Complexity
Dynamic Complexity Time Complexity Space Complexity
Training Complexity Testing Complexity
Cost of Cases
Batch Learner Incremental Learner
Human-Computer Interaction Cost (HCIC)
Data Engineering Parameter Setting Analysis of Learned Models Incorporating Domain Knowledge
Results
Presentation of Taxonomy Serves as a platform for organization of
literature on cost-sensitive learning Inspires research into under-investigated
types of cost.
Weak/Strong Points
STRONG – Interesting idea for incorporating different, mostly unconsidered costs into classification methods.
STRONG – May be more pragmatic in real-world scenarios.
STRONG – Good domain examples. WEAK – Lacks formalized support for the points in
the paper. WEAK – Sections of the paper were imbalanced. WEAK – No empirical evidence to support methods.
Suggestions for Improvements
Gather some empirical data to support the costing methods.
Recommend better ways for use of costing methods (rather than adding more classes).
Perhaps different weighting based on feature? Incorporation of a weighted cost matrix for
predictions.
Conclusions
Turney presents some interesting ideas for various costing methods.
Although these methods are not well supported, the ideas behind them will hopefully drive research in the area of costing methods for inductive concept learning.
This will possibly result in support for the methods.