hello, world EKR Everything you need to know to survive the upcoming robot apocalypse! What is Deep Learning?
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
hello, world
EKR
Everything you need to know to survive the
upcoming robot apocalypse!
What is Deep Learning?
Whether we are based on carbon or on silicon makes no fundamental difference; we should each be treated with appropriate respect.
Arthur C. Clarke CBE FRAS 2010: Odyssey Two
With artificial intelligence we are summoning the demon
Elon Musk CEO & CTO, SpaceX CEO, Tesla Motors Chairman, Solar City
The development of full artificial intelligence could spell the end of the human race
Stephen Hawking CH CBE FRS FRSA Director of Research, Center for Theoretical Cosmology, Cambridge
hello, world
EKR
With great power comes great
responsibility. How does this stuff work?
That’s Scary!
1+1 = ?A. 0 B. 1 C. 2 D. None of the Above
1+1 = ?A. 0 B. 1 C. 2 D. None of the Above
2*2 = ?A. 2 B. 22 C. 4 D. A and B
2*2 = ?A. 2 B. 22 C. 4 D. A and B
Where is the maximum of this function?
A.
B.
C.
D.
x
y
Where is the maximum of this function?
A.
B.
C.
D.
x
y
hello, world
EKR
How does such a simple thing get to
such a level as “summoning the
demon?”
That’s Just Math!
Going deeper with ConvolutionsGoogle used a new variant of convolutional neural network called “Inception” for classification, and for detection the R-CNN [5] was used. The results and the approach that Google’s team took are summarized here [2, 3]. Google’s team was able to train a much smaller neural network and obtained much better results compared to results obtained with convolutional neural networks in the previous year’s challenges.
1: Computers can recognize objects
Show and Tell: A Neural Image Caption GeneratorAutomatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing. In this paper, we present a generative model based on a deep recurrent architecture that combines recent advances in computer vision and machine translation and that can be used to generate natural sentences describing an image.
2: Computers can write image captions
Human-level control through deep reinforcement learningWe tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters.
3: Computers can play video games
4: Computers can evolve better models
Mar I/O: Evolving Neural Networks through Augmenting TopologiesNeural networks can be combined with other machine learning techniques to solve complex problems like model selection. Mar I/O is a program made of neural networks and genetic algorithms that kicks butt at Super Mario World.
5: Computers can dream
Twitch Deep Dream VisualizationA convolutional neural network trained to recognize objects in images can be run backwards to dream. Over the spring of 2015, this dreaming neural network was streamed online.
hello, world
EKR
Let’s figure out how to design and use this
stuff?
That’s impressive.
x00
x01
x02
y00Σ φw00
w01
w02
x00
x01
x02
yΣ φ
Input Layer
w
w
w
x
x
x
yΣ φ
Weighted Sum
w00
w01
w02
x
x
x
yΣ φ
Transfer Function
w
w
w
x
x
x
y00Σ φ
Output Layer
w
w
w
Neuron Objective: Choose w00, w01, w02
so that is y00 close to y. Minimize the “approximation error.” By doing that, you’ve learned to reproduce the original function.
y00 = φ(w00 * x00 + w01 * x01 + w02 * x02)
Simple Example: Draw a line through these points by choosing an appropriate value for w.
y = w * x
x
y y y y
A. B. C. D.y = 0.25 * x y = 0.50 * x y = 1.00 * x y = 3.00 * x
x
y y y y
A. B. C. D.y y y = 1.00 * x y
Simple Example: Draw a line through these points by choosing an appropriate value for w.
y = w * x
x
y y y y
A. B. C. D.y y y = 1.00 * x y
Option C. minimizes the classification error of the example. This value of w “learns” how to separate green dots from blue ones!
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
What about more complex functions? Use more neurons!
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
φφ
φφ
φφ
φφ
φφ
φφ
φφ
φ
φφ
φφ
φφ
φφ
φφ
φφ
φφ
φφ
φφ
φφ
φ
φφ
φφ
φφ
Big networks can become very complex and hard to train, many layers deep with billions of parameters.
That’s where the term deep learning comes from.
CNNConvolutional
Neural Network
RNNRecurrent Neural
Network
L-STMLong Short-Term
Memory
Q
DQNDeep Q-Network
C
NTM
MR
W
Neural Turing Machine
hello, world
EKR
Why is this so important now if the
idea has been around for decades?
I understand!
Recall Moore’s Law
1970 1975 1980 1985 1990 1995 2000 2005 2010 20151,000
10,000
100,000
1,000,000
10,000,000
100,000,000
1,000,000,000
10,000,000,000
4004
8088
Pentium
Pentium 4
Pascal
80386
Pentium 3
KeplerMaxwell
TeslaNahalem
Core i7
• 2880 CUDA Cores
• 7.1 Billion Transistors
• 15 SMX units
• > 1 TFLOP FP64
• 1.5M L2 Cache
• 384-bit DDR5
GK110 / GTX Titan X, 980, …
hello, world
EKR
Exponentially faster computers have made
zombie algorithms from the 1950s
apocalpyse-capable! What do we do?
Thanks Gordon!
Make Autonomous Weapons?
Autonomous weapons select and engage targets without human intervention. They might include, for example, armed quadcopters that can search for and eliminate people meeting certain pre-defined criteria, but do not include cruise missiles or remotely piloted drones for which humans make all targeting decisions. Artificial Intelligence (AI) technology has reached a point where the deployment of such systems is — practically if not legally — feasible within years, not decades, and the stakes are high: autonomous weapons have been described as the third revolution in warfare, after gunpowder and nuclear arms.
In summary, we believe that AI has great potential to benefit humanity in many ways, and that the goal of the field should be to do so. Starting a military AI arms race is a bad idea, and should be prevented by a ban on offensive autonomous weapons beyond meaningful human control.
Stephen Hawking Director of research at the Department of Applied Mathematics and Theoretical Physics at Cambridge, 2012 Fundamental Physics Prize laureate for his work on quantum gravity Elon Musk SpaceX, Tesla, Solar City Steve Wozniak, Apple Inc., Co-founder, member of IEEE CS Jaan Tallinn co-founder of Skype, CSER and FLI Frank Wilczek MIT, Professor of Physics, Nobel Laureate for his work on the strong nuclear force Max Tegmark MIT, Professor of Physics, co-founder of FLI Daniel C. Dennett, Tufts University, Professor, Co-Director, Center for Cognitive Studies, member of AAAI Noam Chomsky MIT, Institute Professor emeritus, inductee in IEEE Intelligent Systems Hall of Fame, Franklin medalist in Computer and Cognitive Science Barbara Simons IBM Research (retired), Past President ACM, ACM Fellow, AAAS Fellow Stephen Goose Director of Human Rights Watch's Arms Division Anthony Aguirre, UCSC, Professor of Physics, co-founder of FLI Lisa Randall, Harvard, Professor of Physics Martin Rees Co-founder of CSER and Astrophysicist
Stuart Russell Berkeley, Professor of Computer Science, director of the Center for Intelligent Systems, and co-author of the standard textbook “Artificial Intelligence: a Modern Approach" Nils J. Nilsson, Department of Computer Science, Stanford University, Kumagai Professor of Engineering, Emeritus, past president of AAAI Barbara J. Grosz Harvard University, Higgins Professor of Natural Sciences, former president AAAI, former chair of IJCAI Board of Trustees Tom Mitchell CMU, past president of AAAI, Fredkin University Professor and Head of the Machine Learning Department Eric Horvitz, Microsoft Research, Managing director, Microsoft Research, past president of AAAI, co-chair of AAAI Presidential Panel on Long-term AI Futures, member of ACM, IEEE CIS Martha E. Pollack University of Michigan, Provost, Professor of Computer Science & Professor of Information, past president of AAAI, Fellow of AAAS, ACM & AAAI Henry Kautz, University of Rochester, Professor of Computer Science, past president of AAAI, member of ACM Demis Hassabis, Google DeepMind, CEO Yann LeCun, New York University & Facebook AI Research, Professor of Computer Science & Director of AI Research Oren Etzioni, Allen Institute for AI, CEO, member of AAAI, ACM Peter Norvig, Google, Research Director, member of AAAI, ACM Geoffrey Hinton University of Toronto and Google, Emeritus Professor, AAAI Fellow Yoshua Bengio, Université de Montréal, Professor
Engage with deep learning, it’s powerful; please don’t make evil killer robots. #ekr
Chris Friel [email protected]
@cfriel
Related Documents
