Deep Water - GPU Deep Learning for H2O - Arno Candel

DeepWaterGPUDeepLearningforH2O

ArnoCandel,PhDChiefArchitect,Physicist&Hacker,H2O.ai

@ArnoCandel

TheOpenTour,Dallas,TXOct262016

Overview

Machine Learning (ML)

Artificial Intelligence (A.I.)

Computer Science (CS)

H2O.ai

Deep Learning (DL)hot hot hot hot hot

ASimpleDeepLearningModel:ArtificialNeuralNetwork

heartbeat

blood pressure

oxygensend to regular care

send to intensivecare unit (ICU)

IN: data OUT: prediction

nodes : neuron activations (real numbers) — represent features arrows : connecting weights (real numbers) — learned during training

: non-linearity x -> f(x) — adds model complexity

from 1970s, now rebranded as DL

BriefHistoryofA.I.,MLandDL

John McCarthyPrinceton, Bell Labs, Dartmouth, later: MIT, Stanford

1955: “A proposal for the Dartmouth summer research project on Artificial Intelligence”

with Marvin Minsky (MIT), Claude Shannon (Bell Labs) and Nathaniel Rochester (IBM)

http://www.asiapacific-mathnews.com/04/0403/0015_0020.pdf

A step back: A.I. was coined over 60 years ago

1955proposalfortheDartmouthsummerresearchprojectonA.I.

“We propose that a 2-month, 10-man study of artificial intelligence be carried out during the summer of 1956 at Dartmouth College in Hanover, New Hampshire. The study is to proceed on the basis of the conjecture that every aspect of learning and any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for one summer.”

Step1:GreatAlgorithms+FastComputers

http://nautil.us/issue/18/genius/why-the-chess-computer-deep-blue-played-like-a-human

1997: Playing Chess (IBM Deep Blue beats Kasparov)

ComputerScience 30customCPUs,60billionmovesin3mins

“No computer will ever beat me at playing chess.”

Step2:MoreData+Real-TimeProcessing

http://cs.stanford.edu/group/roadrunner/old/presskit.html

2005: Self-driving CarsDARPA Grand Challenge, 132 miles (won by Stanford A.I. lab*)

Sensors&ComputerSciencevideo,radar,laser,GPS,7Pentiumcomputers

“It takes a human to drive a car!”

*A.I. lab was established by McCarthy et al. in the early ‘60s

Step3:BigData+In-MemoryClusters

2011: Jeopardy (IBM Watson)

In-MemoryAnalytics/ML4TBofdata(incl.wikipedia),90servers,16TBRAM,Hadoop,6millionlogicrules

https://www.youtube.com/watch?v=P18EdAKuC1U https://en.wikipedia.org/wiki/Watson_(computer)

Note: IBM Watson received the question in electronic written form, and was often able to press the answer button faster than the competing humans.

“Computers can’t answer arbitrary questions!”

“Computers don’t understand our language!”

2014: Google(acquired Quest Visual)

DeepLearning ConvolutionalandRecurrent

NeuralNetworks,withtrainingdatafromusers

Step4:DeepLearning

• Translate between 103 languages by typing • Instant camera translation: Use your camera to translate text instantly in 29 languages • Camera Mode: Take pictures of text for higher-quality translations in 37 languages • Conversation Mode: Two-way instant speech translation in 32 languages • Handwriting: Draw characters instead of using the keyboard in 93 languages

Step5:AugmentedDeepLearning

2014: Atari Games (DeepMind)

2016: AlphaGo (Google DeepMind)

DeepLearning +reinforcementlearning,treesearch,

MonteCarlo,GPUs,playingagainstitself,…

Go board has approx. 200,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000 (2E170) possible positions.

trainedfromrawpixelvalues,nohumanrules

“Go is too complex for computers to master!”

ThingsareChangingQuickly

Yesterday:SmallData(<GB) Today:BigData(TeraBytes,ExaBytes)

Data+Skillsaregoodforbusiness

Data+MachineLearningAREthebusiness

• conceptuallysimple• solvesmanyproblems• benefitsfrombigdata• super-humanresults

WhyDeepLearning?

DeepLearningisintheCenterofThisRevolution

WhyDeepLearning?

• hardtointerpret—butsolutionsexist

• lotsofarchitecturalchoices—requireluckyPhDs,open-sourcehelps

• lotsofhyper-parameters—AutoMLcandothetuningforyou

• slowtotrainonbigdata—dedicatedhardwarehelps(GPUclusters)

• rapidlychanginglandscape—DeepWaterunifiesopen-sourceAPIs

However:

H2OpioneeredEasy-To-UseOpenSourceDeepLearning

GradientBoosting Machine

Generalized LinearModeling

DeepLearning

Distributed RandomForest

Allalgorithmsaredistributedandscalable

H2OpioneeredEasy-To-UseOpenSourceDeepLearning

H2ODeepLearningIsWidelyUsed

http://www.kdnuggets.com

H2OandTensorFlowaretied

usageofDeepLearningtoolsinpastyear

DeepWateropenstheFloodgatesforstate-of-the-artDeepLearning

H2ODeepLearning:simplemulti-layernetworks,CPUs

H2ODeepWater:arbitrarynetworks,CPUsorGPUs

Limitedtobusinessanalytics,statisticalmodels(CSVdata)

Largenetworksforbigdata(e.g.image1000x1000x3->3minputsperobservation)

1-5layersMBs/GBsofdata

1-1000layersGBs/TBsofdata

DeepWater:THEopen-sourceDeepLearningPlatform

H2Ointegratesthetop open-sourceDLtools

NativeGPUsupport isupto100xfasterthan

EnterpriseReadyEasytotrain,compareanddeploy,interactiveandscalable,fromFlow,R,Python,Spark/Scala,Java,REST

NewBigDataUseCases(previouslyimpossibleordifficultinH2O)

Image-socialmedia,manufacturing,healthcare,…Video-UX/UI,security,automotive,socialmedia,…Sound-automotive,security,callcenters,healthcare,…Text-NLP,sentiment,security,finance,fraud,…TimeSeries-security,IoT,finance,e-commerce,…

DeepWater:BestOpen-SourceDeepLearning

EnterpriseDeepLearningforBusinessTransformation

Scala Spark

DeepWaterArchitecture

H2OJava

Execution Engine

TensorFlow/mxnet/Caffe C++

GPU CPU

TensorFlow/mxnet/Caffe C++

GPU CPU

H2O RPC

Data Science API: R, Python, Flow GUIREST API Web server

H2OJava

Execution Engineother comm.

Node 1 Node N

Scala Spark

Step1:LeverageResearchCommunityCode,DataandModels

World’sbestImageClassifier(Google+Microsoft,Aug2016)

https://research.googleblog.com/2016/08/improving-inception-and-image.html

open-sourceimplementation

BuildyourownmodelsintheDeepWaterHands-OnSessionToday!

Step2:TrainModelswithFamiliarAPIs

https://github.com/h2oai/h2o-3/blob/master/h2o-py/tests/testdir_algos/deepwater/pyunit_inception_resnet_v2_deepwater.py

PretrainedImageClassifier-StandaloneScoringServerDeployedinSeconds

http://deepwater.h2o.ai/classy-demo/

Step3:ModelComparisonandRapidDeployment

SteamCompareModels

ScoringServerBuildsmarterapplicationsanddataproducts

High-PerformanceMulti-GPUTrainingwithCaffe

Trainingona16-GPUEC2instancewithCaffe

java -cp target/dependency/*:target/classes deepwater.examples.ImageNet

CaffelaunchedfromJava:

https://aws.amazon.com/blogs/aws/new-p2-instance-type-for-amazon-ec2-up-to-16-gpus/

All16GPUsarebusytraining

H2Ointegrationisinprogress

Outlook

FinishTensorFlowintegration(C++/Python/Java):PackagePythononthebackendtocreatetrainablegraphs

RoadmapforDeepWater(Q42016):

FinishCaffeintegration(pureC++/Java):OptimizedMulti-GPUtraining(NVIDIANCCL)

AddmorecapabilitiestoH2ODeepWater:Text/NLP,TimeSeries,LSTM,AutoEncoder,FeatureExtraction,Input/Outputshapemapping,etc.

Addmulti-GPUsupportformxnet

LearnmoreaboutDeepWater&GPUBackends

11:05-11:30DmitryLarko—CreditCardDefaultPrediction

2:40-3:00FabrizioMilo—TensorFlowinternals,Wide&Deepmodels

3:00-4:00ArnoCandel—Hands-OnWorkshopwithImageClassification,CreditCardDefaultPrediction,BenchmarkingandHyper-ParameterSearchinFlow,PythonandR

BreakoutTracksonDeepWater&GPUBackends

DeepWaterLiveDemo

LiveDemo