Large-scale CelebA face attribute dataset • 200K celebrity images, each with 40 attributes • Highly imbalanced: average positive class rate 23% • Total accuracy = + + Balanced accuracy = 1 2 + Edge detection on BSDS500 dataset • Retrieve from 2M edge label patches with long-tail distribution Learning Deep Representation for Imbalanced Classification Chen Huang 1,2 , Yining Li 1 , Chen Change Loy 1 , Xiaoou Tang 1 1 The Chinese University of Hong Kong 2 SenseTime Group Limited {chuang, ly015, ccloy, xtang}@ie.cuhk.edu.hk Data imbalance is common in visual classification 1. Motivation Wearing hat Not wearing hat … Minority class Majority class Face attribute example Deep embedding: Class-level cluster- & class-level constraint Study traditional re-sampling and cost-sensitive learning scheme 2. Main Idea Triplet embedding Class 1 minority Class 2 majority Class 2 majority Quintuplet embedding Class 1 minority … Cluster j Cluster 2 Cluster 1 Cluster 1 , + < ( , ( − )) < ( , ( −− )) < ( , ( )) – an anchor + – the anchor’s most distant within-cluster neighbor − – the nearest within-class neighbor of the anchor, but from a different cluster −− – the most distant within-class neighbor of the anchor – the nearest between-class neighbor of the anchor Triple-header hinge loss Network architecture • Equal class re-sampling & class costs assignment in batches Training step 3. Large Margin Local Embedding (LMLE) s.t.: ● Clustering by k-means ● Generate quintuplets from cluster & class membership ● Re-sample batches equally from each class ● Forward their quintuplets to CNN to compute loss ● Back-propagation Feature-based clustering Feature learning/updating Every 5000 iterations CNN CNN CNN CNN CNN Triple-header hinge loss Mini- batches Training samples … Embedding Quintuplet Shared parameters Large margin cluster-wise kNN: fast & imbalance resistant 4. Cluster-wise kNN search 10 -8 -6 -4 -2 0 2 4 6 8 -8 -6 -4 -2 0 2 4 6 8 - 1 1 Class 1: cluster 1 Class 1: cluster 2 Class 2: cluster 1 Class 2: cluster 2 Class 2: cluster 3 Class 2: cluster 4 Class 2: cluster 5 15 -10 -5 0 5 10 C2 C3 C4 C5 C1 C2 C1 -8 DeepID2 Triplet Our LMLE 5. Results Total acc. Balanced acc. Triplet-kNN 83 72 Anet 87 80 LMLE-kNN 90 84 Ground truth SketchToken ODS 0.73 DeepContour ODS 0.76 LMLE-kNN ODS 0.78 6. Conclusion Cluster- & class-level quintuplets preserve both locality across clusters and discrimination between classes, irrespective of class imbalance Large margin classification by fast cluster-wise kNN search