SXSW

hello, world

Everything you need to know to survive the

upcoming robot apocalypse!

What is Deep Learning?

Whether we are based on carbon or on silicon makes no fundamental difference; we should each be treated with appropriate respect.

Arthur C. Clarke CBE FRAS 2010: Odyssey Two

With artificial intelligence we are summoning the demon

Elon Musk CEO & CTO, SpaceX CEO, Tesla Motors Chairman, Solar City

The development of full artificial intelligence could spell the end of the human race

Stephen Hawking CH CBE FRS FRSA Director of Research, Center for Theoretical Cosmology, Cambridge

hello, world

With great power comes great

responsibility. How does this stuff work?

That’s Scary!

1+1 = ?A. 0 B. 1 C. 2 D. None of the Above

2*2 = ?A. 2 B. 22 C. 4 D. A and B

Where is the maximum of this function?

hello, world

How does such a simple thing get to

such a level as “summoning the

demon?”

That’s Just Math!

Going deeper with ConvolutionsGoogle used a new variant of convolutional neural network called “Inception” for classification, and for detection the R-CNN [5] was used. The results and the approach that Google’s team took are summarized here [2, 3]. Google’s team was able to train a much smaller neural network and obtained much better results compared to results obtained with convolutional neural networks in the previous year’s challenges.

1: Computers can recognize objects

Show and Tell: A Neural Image Caption GeneratorAutomatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing. In this paper, we present a generative model based on a deep recurrent architecture that combines recent advances in computer vision and machine translation and that can be used to generate natural sentences describing an image.

2: Computers can write image captions

Human-level control through deep reinforcement learningWe tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters.

3: Computers can play video games

4: Computers can evolve better models

Mar I/O: Evolving Neural Networks through Augmenting TopologiesNeural networks can be combined with other machine learning techniques to solve complex problems like model selection. Mar I/O is a program made of neural networks and genetic algorithms that kicks butt at Super Mario World.

5: Computers can dream

Twitch Deep Dream VisualizationA convolutional neural network trained to recognize objects in images can be run backwards to dream. Over the spring of 2015, this dreaming neural network was streamed online.

hello, world

Let’s figure out how to design and use this

stuff?

That’s impressive.

y00Σ φw00

yΣ φ

Input Layer

yΣ φ

Weighted Sum

yΣ φ

Transfer Function

y00Σ φ

Output Layer

Neuron Objective: Choose w00, w01, w02

so that is y00 close to y. Minimize the “approximation error.” By doing that, you’ve learned to reproduce the original function.

y00 = φ(w00 * x00 + w01 * x01 + w02 * x02)

Simple Example: Draw a line through these points by choosing an appropriate value for w.

y = w * x

y y y y

A. B. C. D.y = 0.25 * x y = 0.50 * x y = 1.00 * x y = 3.00 * x

y y y y

A. B. C. D.y y y = 1.00 * x y

Simple Example: Draw a line through these points by choosing an appropriate value for w.

y = w * x

y y y y

A. B. C. D.y y y = 1.00 * x y

Option C. minimizes the classification error of the example. This value of w “learns” how to separate green dots from blue ones!

Σ φ y00

What about more complex functions? Use more neurons!

Σ φ y00

Feed-forward Pass

Σ φ y00

Feed-forward Pass

Σ φ y00

Feed-forward Pass

Σ φ y00

Feed-forward Pass

Σ φ y00

Feed-forward Pass

Σ φ y00

Feed-forward Pass

Σ φ y00

Feed-forward Pass

Σ φ y00

Feed-forward Pass

Σ φ y00

Feed-forward Pass

Σ φ y00

Feed-forward Pass

Σ φ y00

Feed-forward Pass

Σ φ y00

Feed-forward Pass

Σ φ y00

Backpropagation Pass

Σ φ y00

Big networks can become very complex and hard to train, many layers deep with billions of parameters.

That’s where the term deep learning comes from.

CNNConvolutional

Neural Network

RNNRecurrent Neural

Network

L-STMLong Short-Term

Memory

DQNDeep Q-Network

Neural Turing Machine

hello, world

Why is this so important now if the

idea has been around for decades?

I understand!

Recall Moore’s Law

1970 1975 1980 1985 1990 1995 2000 2005 2010 20151,000

10,000

100,000

1,000,000

10,000,000

100,000,000

1,000,000,000

10,000,000,000

Pentium

Pentium 4

Pascal

Pentium 3

KeplerMaxwell

TeslaNahalem

Core i7

• 2880 CUDA Cores

• 7.1 Billion Transistors

• 15 SMX units

• > 1 TFLOP FP64

• 1.5M L2 Cache

• 384-bit DDR5

GK110 / GTX Titan X, 980, …

hello, world

Exponentially faster computers have made

zombie algorithms from the 1950s

apocalpyse-capable! What do we do?

Thanks Gordon!

Make Autonomous Weapons?

Autonomous weapons select and engage targets without human intervention. They might include, for example, armed quadcopters that can search for and eliminate people meeting certain pre-defined criteria, but do not include cruise missiles or remotely piloted drones for which humans make all targeting decisions. Artificial Intelligence (AI) technology has reached a point where the deployment of such systems is — practically if not legally — feasible within years, not decades, and the stakes are high: autonomous weapons have been described as the third revolution in warfare, after gunpowder and nuclear arms.

In summary, we believe that AI has great potential to benefit humanity in many ways, and that the goal of the field should be to do so. Starting a military AI arms race is a bad idea, and should be prevented by a ban on offensive autonomous weapons beyond meaningful human control.

Stephen Hawking Director of research at the Department of Applied Mathematics and Theoretical Physics at Cambridge, 2012 Fundamental Physics Prize laureate for his work on quantum gravity Elon Musk SpaceX, Tesla, Solar City Steve Wozniak, Apple Inc., Co-founder, member of IEEE CS Jaan Tallinn co-founder of Skype, CSER and FLI Frank Wilczek MIT, Professor of Physics, Nobel Laureate for his work on the strong nuclear force Max Tegmark MIT, Professor of Physics, co-founder of FLI Daniel C. Dennett, Tufts University, Professor, Co-Director, Center for Cognitive Studies, member of AAAI Noam Chomsky MIT, Institute Professor emeritus, inductee in IEEE Intelligent Systems Hall of Fame, Franklin medalist in Computer and Cognitive Science Barbara Simons IBM Research (retired), Past President ACM, ACM Fellow, AAAS Fellow Stephen Goose Director of Human Rights Watch's Arms Division Anthony Aguirre, UCSC, Professor of Physics, co-founder of FLI Lisa Randall, Harvard, Professor of Physics Martin Rees Co-founder of CSER and Astrophysicist

Stuart Russell Berkeley, Professor of Computer Science, director of the Center for Intelligent Systems, and co-author of the standard textbook “Artificial Intelligence: a Modern Approach" Nils J. Nilsson, Department of Computer Science, Stanford University, Kumagai Professor of Engineering, Emeritus, past president of AAAI Barbara J. Grosz Harvard University, Higgins Professor of Natural Sciences, former president AAAI, former chair of IJCAI Board of Trustees Tom Mitchell CMU, past president of AAAI, Fredkin University Professor and Head of the Machine Learning Department Eric Horvitz, Microsoft Research, Managing director, Microsoft Research, past president of AAAI, co-chair of AAAI Presidential Panel on Long-term AI Futures, member of ACM, IEEE CIS Martha E. Pollack University of Michigan, Provost, Professor of Computer Science & Professor of Information, past president of AAAI, Fellow of AAAS, ACM & AAAI Henry Kautz, University of Rochester, Professor of Computer Science, past president of AAAI, member of ACM Demis Hassabis, Google DeepMind, CEO Yann LeCun, New York University & Facebook AI Research, Professor of Computer Science & Director of AI Research Oren Etzioni, Allen Institute for AI, CEO, member of AAAI, ACM Peter Norvig, Google, Research Director, member of AAAI, ACM Geoffrey Hinton University of Toronto and Google, Emeritus Professor, AAAI Fellow Yoshua Bengio, Université de Montréal, Professor

Engage with deep learning, it’s powerful; please don’t make evil killer robots. #ekr

Chris Friel cfriel@gmail.com

@cfriel

SXSW

convolutional neural

smaller neural network

world ekr

network architecture

deep learning

neural image caption

deep qnetwork agent

deep reinforcement

Documents

SXSW Redux

SXSW PanelPicker

Director Mark Cousins on I am Belfast ( ... · ©2015-2016....

SXSW Bites: The Best of SXSW 2014 Made Easy to Digest

SXSW Presentation

Insider’s Guide to SXSW Interactive 2015: Following the...

SXSW submission

SXSW Finance 2.0

Digital Urbanites @SxSW

SXSW Takeaways

@SXSW @Verego

SXSW Pokes

Bing @ SXSW

ThisWay SXSW

Resumo sxsw 2014

SXSW HOTSHEET: 360i Recaps the Best of SXSW Interactive 2014