Edge-to-Cloud Compute with MxNet - NVIDIA · 2017-05-29 · Linking the Pi to AWS # Load the network parameters model = load_model.ImagenetModel('synset.txt', 'squeezenet_v1.1') #

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Aran Khanna, Software Developer, AWS

Miro Enev, Solutions Architect, NVIDIA

Edge-to-Cloud Compute with MxNet

AWS & NVIDIA

Presented @ GTC 2017 &

Overview

Edge-To-Cloud Compute Models

Advantages of Hub & Spoke

Case Study / Demo

References & Tools

IIoT & AI

AI/DL is primed to disrupt the Industrial Sector.

NVIDIA IoT

Build

Operate

Maintain & Repair

Design

Robots & Smart Manufacturing

Predictive Maintenance

Generative Design

Inspection & Efficiency

NVIDIA IoT

Amazon IoT & GreenGrass

Amazon IoT Software & NVIDIA Hardware

Key Functions• Data Ingest

• Compressed Inference

• Full Inference / Trained Model Query

• Model Training

Deployment Models

Cloud <-> Edge

Cloud <-> Hub <-> Edge

Emerging Deployment Model

Cloud

Elastic GPU [ Tesla ]

Hub/Gateway

Jetson [ Tegra ]

Edge

Sensors [ RasPi ]

Trends: Latency, Cost, Reliability, Privacy/Security Policy

Handoff

Use Cases Of ML at The Edge

Transportation AR & VR Industrials

Smart Home Robotics Body Cameras

In This Demo I will:

-Show you why you should care about Machine Learning at

the Edge

-Walk through the value of running deep learning

applications with MXNet on a set of edge devices.

-Demonstrate how AWS services make managing deep

learning at the edge easier than ever.

MXNet Everywhere

Low End:

Raspberry Pi

- 32 Bit ARMv7

- ARM NEON

- Raspbian

High End:

NVIDIA Jetson

- 64 Bit ARM Arch64

- CUDA

- Ubuntu

Easy as 1…2...3...

1.

git clone

https://github.com/

dmlc/mxnet.git

--recursive

3.

cd python &&

sudo python

setup.py install

2.

cd mxnet &&

make

ImageNet Models

- Well studied

- Form the base models for

most image tasks

- Widely available pre-trained

models

Try it Yourself:

http://data.mxnet.io/models

Running the Model

# Load the network parameters from the cloud

def load_model(model_url=“http://data.mxnet.io/models/imagenet/squeezenet”):

urllib.urlretrieve(model_url, ‘model.zip’)

unzip(model.zip)

sym, arg_params, aux_params = mx.model.load_checkpoint(’Image_Model', 0)

mod = mx.mod.Module(symbol=sym, context= mx.cpu())

mod.bind(for_training=False,data_shapes=[('data', (1,3,224,224))])

mod.set_params(arg_params, aux_params)

return mod

# Predict on an Image

def predict(image_path, mod):

img = cv2.cvtColor(cv2.imread(filename), cv2.COLOR_BGR2RGB)

img = cv2.resize(img, (224, 224))

img = np.swapaxes(img, 0, 2)

img = np.swapaxes(img, 1, 2)

img = img[np.newaxis, :]

mod.forward(Batch([mx.nd.array(img)]))

return mod.get_outputs()[0].asnumpy()

Making This Production Ready With AWS

Management

- Rollout changes

- Coordinate local

devices

- Handle network

and hardware

failure

Aggregation

- Gather and

centralize data

from the Edge

- Connect to

existing systems

in the cloud

Logging

- Monitor

performance

- Analyze uptime

- Diagnose

failures

Enter AWS IoT and AWS Greengrass

On The Pi: SqueezeNet Object Classifier

Iandola et. Al

Linking the Pi to AWS

# Load the network parameters

model = load_model.ImagenetModel('synset.txt', 'squeezenet_v1.1')

# Run Predictions on Images Captured With the Local Camera

def Callback(client, userdata, message):

args = json.load(message.payload)

predictions, img = model.predict_from_cam(args.model)

myAWSIoTMQTTClient.publish(”sdk/test/predict”, predictions,1)

if(predictions[0][0] < 0.8):

myAWSIoTMQTTClient.publish(”sdk/test/escalate”, img, 1)

myMQTTClient.connect()

myMQTTClient.subscribe(”sdk/test/control", 1, Callback)

On The TX1: ResNet-152 Object Detector

Szegedy et. Al

Linking the TX1 to AWS

# Load the network parameters

model = load_model.ImagenetModel('synset.txt', ’resnet-v2')

pi_model = load_model.ImagenetModel('synset.txt', 'squeezenet_v1.1')

# Run Predictions on Images Captured With the Local Camera

def Callback(client, userdata, message):

args = json.load(message.payload)

predictions = model.predict_from_img(args.img. args.model)

myAWSIoTMQTTClient.publish(”sdk/test/predict”, predictions, 1)

# Fine tune the squeezenet model to for redeployment to Pi

if(predictions[0][0] > 0.8):

pi_model.update_model(args.img, predictions)

myMQTTClient.connect()

myMQTTClient.subscribe(”sdk/test/escalate", 1, Callback)

Local Learning Loop

Poorly

Classified

Picture

Updated

Model

Fine Tune Model With

Accurate Classification

Thank You!Miro Enev – [email protected]

Aran Khanna – [email protected]