Learning Robust Global Representations by Penalizing Local Predictive Power Haohan Wang, Songwei Ge, Eric P. Xing, Zachary C. Lipton School of Computer Science, Carnegie Mellon University ! ImageNet-Sketch Dataset & Experiments • First out-of-domain data set at the ImageNet validation set scale • 1000 classes, with 50 testing images in each • Used as test data set to test the model’s generalization ability when trained on standard ImageNet train set. • Performance: • Analysis: Accuracy AlexNet DANN* InfoDrop HEX PAR Top 1 0.1204 0.1360* 0.1224 0.1292 0.1306 Top 5 0.2480 0.2712* 0.2560 0.2654 0.2627 AlexNet-PAR AlexNet Predic;on Confidence Predic;on Confidence stethoscope 0.6608 hook 0.3903 tricycle 0.9260 safety pin 0.5143 Afghan hound 0.8945 swab (mop) 0.7379 red wine 0.5999 goblet 0.7427 ! Patch-wise Adversarial Regularization (PAR) ! Highlights ! Empirical Results • Notations • top layers: f(•;θ) • patch classifier: h(•;ϕ) • bottom layers: g(•;δ) • Patch-wise Adversarial Regularization • Training heuristics • first train the model conventionally until convergence • then train the model with regularization • Variants • PAR: • 1-layer classifier • 1x1 local patch • first layer • PAR B • 3x3 local path • PAR M • 3-layer classifier • PAR H • higher layer • Engineering-wise • One set of parameters • Implemented efficiently through convolution • Out-of-domain CIFAR10 • Test with ResNet-50 • 4 out-of-domain settings created: • Greyscale, NegativeColor, RandomKernel, RadiamKernel • Best performance in comparison to standard methods • PACS experiment • Test with AlexNet (consistent with previous state-of-the-art) • Best average performance in domain-agnostic setting • Best performance in Sketch domain in comparison to any method ! Contact • Novel method for out-of-domain robustness • with domain-agnostic setting (more industry-friendly) • simple and intuitive regularization, architecture-agnostic • New vision data set for large scale out-of- domain robustness testing • ImageNet validation set scale ! Motivation • Neural networks are not robust enough! • Models with high accuracy can easily fail when tested with out-of-domain data • One reason is that the models are exploiting predictive local signals, ignoring the global picture • Penalize model’s tendency in predicting through local signals • [email protected] @HaohanWang • [email protected] • resource links