UNIFORM CONVERGENCE MAY BE UNABLE TO EXPLAIN GENERALIZATION IN DEEP LEARNING VAISHNAVH NAGARAJAN 1 ZICO KOLTER 1,2 1 Computer Science Department, Carnegie Mellon University 2 Bosch Center for Artificial Intelligence, Pittsburgh THE HIGH LEVEL MESSAGE THE GENERALIZATION PUZZLE & UNIFORM CONVERGENCE (U.C) PROGRESS SO FAR Our 1st finding: GENERALIZATION BOUNDS ↑ WITH TRAINING SET SIZE Key element in proof: TIGHTEST UNIFORM CONVERGENCE CONCLUSIONS AND FUTURE WORK REFERENCES PROOF INTUITION Our 2nd finding: PROVABLE FAILURE OF UNIFORM CONVERGENCE FAILURE OF U.C. IN A HYPERSPHERE EXAMPLE
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
UNIFORM CONVERGENCE MAY BE UNABLE TO EXPLAIN GENERALIZATION IN DEEP LEARNINGVAISHNAVH NAGARAJAN1 ZICO KOLTER1,2
1Computer Science Department, Carnegie Mellon University 2Bosch Center for Artificial Intelligence, Pittsburgh
THE HIGH LEVEL MESSAGE
THE GENERALIZATION PUZZLE & UNIFORM CONVERGENCE (U.C)
PROGRESS SO FAR
Our 1st finding: GENERALIZATION BOUNDS ↑ WITH TRAINING SET SIZE
Key element in proof: TIGHTEST UNIFORM CONVERGENCE
CONCLUSIONS AND FUTURE WORK
REFERENCES
PROOF INTUITION
Our 2nd finding: PROVABLE FAILURE OF UNIFORM CONVERGENCE
FAILURE OF U.C. IN A HYPERSPHERE EXAMPLE
Related Documents
