Environment for training models

Environment for training modelsDmitry Spodarets

AI Rush

Who am I

Dmitry Spodarets• Founder and CEO at FlyElephant

• PhD candidate at Odessa National University

• Lecturer at Odessa Polytechnic University • Organizer of technical conferences about AI,

BigData, HPC, JS, Web Technologies …

Agenda

•Data Science Tools Survey Results• Computing resources• Clouds (AWS & Azure)• Containers (Docker, Singularity)• FlyElephant platform for Data Science

Data Science Tools Survey

220datascientist

Datasets

0

10

20

30

40

50

60

70

lessthan1MB

1.1to10MB

11to100MB

101MBto1GB

1.1to10GB

11to100GB

101GBto1

Terabyte

1.1to10TB

11to100TB

101TBto1

Petabyte

1.1PBto10

Petabyte

11to100PB

over100PB

Datasets

Datasets

Tools for collecting data

Python 45

R 26

Spark 18

SQL 15

Excel 13

Kafka 11

Pandas 10

custom 8

Hadoop 5

Numpy 5

SAS 5

Tools for storing data

PostgreSQL 37

CSV 31

MySQL 21

Hadoop 16

Excel 15

HDFS 15

Mongodb 15

MyServer 12

Oracle 11

Hive 8

Programming languages

Python 151

R 88

SQL 37

Java 32

Scala 22

bash 17

C++ 17

JavaScript 15

C# 13

vba 8

C 6

Libraries

Pandas 88

Numpy 68

scikit-learn 48

scipy 26

dplyr 20

matplotlib 20

ggplot2 15

keras 14

SPARK 13

xgboost 13

Tensorflow 12

Tools for the visualization of data

matplotlib 66

ggplot 40

seaborn 33

Excel 22

Tableau 22

R 19

plotly 13

bokeh 12

d3 11

Clouds

aws 77

none 41

azure 25

google 24

digital ocean 9

OpenStack 7

Watson 1

The Jupyter Notebook

Jupyter Lab

Computing resources

Computing resources

Computing resources

NVIDIADGX-1Deep Learning Supercomputer170/3TFLOPS(GPUFP16/CPUFP32)

nvidia tesla p100~5 TeraFLOPS

~3TeraFLOPS

Image Training Performance on GoogLeNet

251,77425,38

569,1467,73

791,96

1230,63

0

200

400

600

800

1000

1200

1400

1GPU(1.86X) 2GPUs(1.87X) 4GPUs(2.2X)

TeslaK80 TeslaP100

http://www.nvidia.com/object/caffe-benchmarks.html

images

traine

dpe

rsecon

d

1080 vs Titan X vs K80 vs P100

0,25

8,8

0,3

10,1

2,9

8,7

5,3

10,6

0

2

4

6

8

10

12

FP32(Singleprecision) FP64(Doubleprecision)

1080 TitanX K80 P100

http://www.nvidia.com/

TFLO

PS

Problem

Effective parallelization of algorithms

NVIDIA Deep Learning SDK

Computing power (Intel)

• Intel Math Kernel Library (Intel MKL)Natively supports C, C++ and Fortran Development. Cross-language compatible with Java, C#, Python and other languages.

• Intel Data Analytics Acceleration Library (Intel DAAL)Includes Python, C++, and Java APIs and connectors to popular data sources including Spark and Hadoop.

• Intel MPI LibraryNatively supports C,C++ and Fortran development

Books

Clouds

Clouds

P2-series N-series 16X K80 4X K80X1-series H-Series

128 vCPU / 1952 GB 16 vCPU / 224 GBC4-series

36 vCPU / 60 GBaws.amazon.com/marketplace/ azuremarketplace.microsoft.com

Azure CLI

1. sudo pip install azure-cli2. az login3. az group create --name GroupName --location EastUS4. az vm create --resource-group GroupName --name MyVM --image

Canonical:UbuntuServer:16.04-LTS:latest --size Standard_NC6 --storage-sku Standard_LRS --admin-username user --ssh-key-value ~/.ssh/id_rsa.pub

5. az vm deallocate --resource-group GroupName --name MyVM6. az vm start --resource-group GroupName --name MyVM7. az vm list-ip-addresses --resource-group GroupName --name

MyVM8. az vm delete --resource-group GroupName --name MyVM9. az group delete --name GroupName

Data Science images in Azure Marketplace

Data Science images in AWS Marketplace

Containers

Docker

Docker (Dockerfile)

FROM gcr.io/tensorflow/tensorflow

MAINTAINER Dmitry Spodarets <[email protected]>

RUN apt update && apt -y upgrade && apt -y install git curl wget

CMD /run_jupyter.sh

Docker (build.sh)

#!/bin/bashfunction docker_build {

docker build -t $1 ./$1; docker tag $1 registry.flyelephant.net/$1 docker push registry.flyelephant.net/$1 docker rmi $1 registry.flyelephant.net/$1

}case $1 in all)

for i in `cat build.list`; do docker_build $i;

done ;;

*) docker_build $1;;

esac

Docker Hub

https://hub.docker.com/

Docker

1. docker images

2. docker run --memory 512m --cpus="2" --name mycont registry.flyelephant.net/tensorflow

3. docker exec -i -t mycont bash

4. docker ps

5. docker stats

6. docker stop CONTAINER ID

7. docker start CONTAINER ID

8. docker rm CONTAINER ID

Docker Machine

• Amazon Web Services

• Digital Ocean

• Exoscale

• Generic

• Google Compute Engine

• IBM Softlayer

• Microsoft Azure

• Microsoft Hyper-V

• OpenStack

• Oracle

• VirtualBox

• Rackspace

• VMware Fusion

• VMware v

• Cloud Air

• VMware vSphere

docker-machine create --driver azure --azure-subscription-id subscription-id --azure-resource-group resourcename --azure-ssh-user user --azure-size machine-name

docker-machine ssh machine-name

Singularity

Singularity - Containers for Science

• First public release in April 2016, followed by a massive uptake•HPC Wire Editor’s choice: Top Technologies to Watch for 2017• Simple integration with resource managers, InfiniBand, GPUs, MPI, file

systems, and supports multiple architectures (x86_64, PPC, ARM, etc..)• Limits user’s privileges (inside user == outside user)•No root owned container daemon•Network images are supported via URIs and all require local caching:

○ docker:// - This will pull a container from Docker Hub

○ http://, https:// - This will pull an image or tarball from the URL, cache and run it

○ shub:// - Pull an image from the Singularity Hub

Singularity - Usage Examples

$ python ./hello.pyHello World: The Python version is 2.7.5$ sudo singularity exec --writable /tmp/debian.img apt-get install python…$ singularity exec /tmp/debian.img python ./hello.pyHello World: The Python version is 2.7.12

Webinar"IntroductiontoSingularity"https://youtu.be/h5rDnCA3NJA

Contributors to Singularity

Network Based Computing LabOhio State University

• High-Performance Big Data (HiBD)http://hibd.cse.ohio-state.edu/

• High-Performance Deep Learning (HiDL)http://hidl.cse.ohio-state.edu/

FlyElephant

FlyElephant platform for Data Science

We automate Data Scienceand help teams to work efficiently.

Computing resources

Ready-computing infrastructure

Collaboration& Sharing

Fast Deployment

Expert Community

Ready-computing infrastructure

Jupyter orother IDE

Automatic running of tasks

Server orCluster

Our resources

• Public Clouds: Azure & AWS.• Private cloud based on OpenStack.• HPC-clusters based on SLURM.• Docker-clusters based on Swarm / Singularity.

• Tools and languages: R, Python, Java, Scala, C/C++, Julia, OpenFOAM, Octave, PyFR,

Scilab, GROMACS, MATLAB, Intel MKL, FlowVision, ANSYS, COMSOL, AVL, Hadoop, Spark, H2O, Anaconda, Octave, scikit-learn, Tensorflow, Theano, Caffe, etc.

FlyElephant US 1 Cloud (P100, K80, Titan X, FPGA (Xilinx))

• HPC HUB 1: 80 nodes (2 × Xeon E5-2680v2 (20 cores), 64GB RAM, IB FDR) and 240TB storage.• HPC HUB 2: 100 nodes (2 × Xeon E5-2670v2 (20 cores), 256GB RAM, IB FDR) and 240TB storage.• HPC HUB 3: 150 nodes (2 × Xeon E5-2650v2 (16 cores), 128GB RAM, 2 × Tesla K80, IB FDR) and 240TB storage.

Advania, CESGA, TACC(17), HLRS (14), LANL(10)

Dmitry Spodarets

[email protected]