Google Cloud infrastructure in Conrad Connect by Google & waylay

Google Cloud Platform 2

Organize the world’s information and make it universally accessible and useful.Google’s Mission

2

“

Google Cloud Platform 3

1Bseven

users

products with

Google Global Cache (GGC) edge nodes

Points of presence (>100)

Network fiber

FASTER (US, JP, TW) 2016

Unity (US, JP) 2010

SJC (JP, HK, SG) 2013

Monet (US, BR) 2017

Google Network More than a collection of data centers

Google Cloud Platform 5

We’ve built our IoT stack on the sametechnology and tools we use at Google

Confidential & ProprietaryGoogle Cloud Platform 5

Google Cloud Platform 6

Eliminate Infrastructure from IoT Data Analytics

Using Google’s Serverless Cloud loT

Platform

Ensure Secure Device Connections with Industry-Standard

Protocols

Lower your Total Cost of Ownership with No

Capex or Ongoing Maintenance Required

A Global IoT Service that Scales

Automatically

Confidential & ProprietaryGoogle Cloud Platform 6

Google Cloud infrastructure in Conrad Connect

© 2017 waylay.io | All Rights Reserved IoT Convention Europe , June 15th. 2017

Veselin Pizurica, CTO waylay

“Waylay connects Internet-of-Things solutions with IT applications and cloud

services empowering you to build automations, system integrations and new

applications faster than ever before.”

Ever growing ecosystem

Scaling, scaling ….

~ 100K #devices

~ 15M API’s per day

~ few billion metrics per week

IoT “mixed” with API: challenges Most of people think of API and scaling in terms of speed/ crunching the big numbers.The biggest problem is not high throughput, but what if some API’s are slow in response?

Have a look at the next slide….

API’s and response times

Other API challenges People think that having REST is all it takes to call yourself an API provider…Since Conrad Connect is B2C, API vendors should in principle be only OAuth2, but that is not always the case.

Have a look at the next slide….

What sort of API’s we see?● REST GET/POST● REST webhooks● Event streams, channels● Multiplexed websockets● “Mobile” like API (sockets) ….

And we need to consume data both for rules and storage… and that is not the same requirement!

Storage vs. rules execution● For storage: you want to batch data, which means that

the preference is that the client polls for data -> “traffic shaping” client side (we are the client)

● For rules: you want to execute rules as soon as the device has new data, which means that the preference is that the server forwards the data to the client as soon as it receives it from the end device

Integration API patterns? Pub/sub event streams:1. server to one or few “channels” (streams, sockets…)2. server to many endpoints (for every device) - typical for “mobile API”

WebhooksAll fine, unless there are way too many events.If too many we prefer “buffered” events (arrays), but then server needs to batch for limited time (otherwise is not OK for rules)

HTTP PollingGood for data collection/storage, not so great for rules…

How to build a scalable system?● Every component must be exposed as a microservice

● Every component must be stateless

● Every component must scale linearly – adding N components

provide N times scalability of a single component – sharding

becomes “trivial” with simple balancer/proxy in front

Conrad Connect architecture

collector NVendorXdata

events

VendorYPLUGS

collector 1

PLUGS

actuation

data

events

OAuth2

REST

Visualization of a serverless event-driven architecture

Yes - this is a reality

Serverless architecture challenges

● Release/build management?● Configuration?● Deployment?● Security?● Monitoring?● Failover?● Central Logging?

Kubernetes to the rescue!

CONTAINER ENGINEKubernetes clusters, managed by Google

● Docker support, private container registry● Configure Pods/Replicas using YAML● Automatically scale up and down your application based

on the resource utilization (CPU, memory, ...).● Logging & Monitoring (Stackdriver)● Container Engine is fully managed by Google reliability

engineers

Kubernetes clusters in Conrad Connect

THANK YOU!