From the Internet of Things to Intelligent Systems A Developer's Primer - Garibay - Redmond

From the Internet of Things toIntelligent Systems:

A Developer’s Primer

Rick G. GaribayVP, Distinguished Engineer

Level: Intermediate

About Me• VP, Distinguished Engineer leading the Development

Platform Group at Neudesic• Working on IoT, Intelligent Transportation and Hospitality

& Gaming• Microsoft MVP, Microsoft Azure• Co-Author, “Windows Server AppFabric Cookbook” by

Packt Pub.• Chairman, Co-Founder Phoenix Connected Systems User

Group (PCSUG.org)• twitter: @rickggaribay• blog: http://rickgaribay.net• email: [email protected] | [email protected]

What is the Internet of Things?

The Internet of Things: By theNumbers

B5075212

This change is happening morerapidly than anyone imagined.

This change could bringtremendous opportunity to youremployer, clients, industry and

you as a technologist.

The Internet of Things is the network ofphysical objects that contain

embedded technology to communicateand interact with their internal states

or the external environment.

OEM Revenue Opportunity | MarketForecast CY17

Auto & Trans Retail Manufacturing Healthcare Energy Computing Telecom Consumer

$7 B $16 B $197 B $3 B $27 B $908 B $179 B $356 B System Revenue

IntelligentSystems

1.7T$

SmartProducts

Grid

Renewables

Oil/Gas/CoalRecovery and

Distribution

Pointsof Sale

Restaurants

Hotels

FuelStations

Patients

Clinics

Hospitals

NursingHomes

MobileCare

SafetySecurity

ComfortLighting

Automation

ManufacturingIntegration and

AutomationRemoteServicing

Predictive andReactive

Maintenance

Water

Waste

PollutionControl

Fire

Emergency

PublicSafety

LawEnforcement

LettersPackages

Containers

Tanks Bulkware

Games

Events

Sports

TelevisionStreaming

Traffic Buses

Cars

Trucks

Trains

Vessels

Aircraft

BikesSmartEnergy

SmartRetail

SmartMobilitySmart

Mobility

SmartLogisticsSmart

Logistics

SmartFactorySmart

Factory

SmartCitiesSmartCities

SmartEntertain-

ment

SmartEntertain-

ment

SmartHealth-

care

SmartBuildingHome

SmartBuildingHome

EventVelocity

Device telemetry• Thermostats report data

every 15 minutes• Cars send telemetry data every minute

Applicationtelemetry• Application perf counters are measured

every second per server• Mobile app telemetry is captured for

every action on your app!

App andoperational events• Halo game engine estimate 1,000,000

messages/second

IoT Device Taxonomy

Large

Mobile

Micro

Small

POS terminal, ATM, MRIx86, PC-like, apps

Industry handheld, POS tabletARM and x86, shell experience, apps

Gateways, wearables, panels, carsARM and x86, diverse hardware, no shell

Controllers, fixed-use, sensors, actuatorsARM, constrained hardware, headless

5mm 5mm

Raspberry Pi

Intel Galileo

Gadgeteer Fez Spider

Arduino Uno

Netduino Plus 2

Spark

Beyond the garage, the true significanceof IoT is the foundation it provides for

providing insights that enable newbusiness capabilities.

FromInformation Technology

toOperational Technology

IT

Servers,Applications,

Systems

IT

Servers,Applications,

Systems

OT

Devices,Telemetry,

Command &Control

OT

Devices,Telemetry,

Command &Control

Data-Driven Insight + Action at a Distance

Data-Driven Insight

• Data –> Information –> Insight ($+)– Make more efficient use of resources (reduce

cost, environmental impact)• Example: Power management in buildings and data centers• Smart Parking

– Provide more targeted products and services(increase revenue, social impact)

• Example: Preventive maintenance, optimal usage analytics for expensive machines

• “Things” = a rapidly expanding source of rawmaterial for the Insight pipeline

Action at a Distance• Data isn’t the only raw material being unlocked by the IoT

– The ability to act remotely – automatically and intelligently– Remote control is a source of efficiency– Enables new forms of customer interaction and engagement

• IoT extends customer engagement opportunities to physical products

• Taking engagement with customers beyond the point of sale– Preventive maintenance– Best practices guidance– Proactive sales– Remote servicing

• From CRM to PRM – “Product Relationship Management”

From IoT to Intelligent Systems

Large

Mobile

Micro

Small

M2M/Device to

Cloud

Various Network Protocols

2G2G

Wi-FiWi-Fi

Bluetooth/BLE

Bluetooth/BLE

RFIDRFID

GPRSGPRS SMSSMS

3G3G

LTELTE

WiMaxWi

Max

ZigBeeZigBee

Various Application Protocols

AMQPAMQP MQTTMQTT

CoAPCoAP

CustomCustom

HTTPHTTP

……

MQ Telemetry Transport (MQTT)• Born out of IBM MQ Series messaging middleware product• Compact binary protocol – min. 7 byte overhead per message sent• No structured message – message bodies are byte arrays• Simple topic name based pub/sub messaging model

– Send to topic name, e.g., “/a/b/c/d” or “/a/b/e/f”– Subscribe to topic name, e.g., “/a/b/c/d” or use wildcard, e.g., “/a/b/#”

• Reliable – fire-and-forget to reliable, exactly-once delivery• Two innovative, device-oriented features:

– Retain – mark a message to be delivered to new subscribers on connection– Last will and testament – register message to be sent on abrupt disconnect

• Not general purpose – lacking key features, e.g., flow control• Standardization in progress through OASIS

Constrained Application Protocol(CoAp)

• Embedded web transfer protocol (coap://)

• Asynchronous transaction model

• UDP binding with reliability and multicast support

• GET, POST, PUT, DELETE methods

• URI support

• Small, simple 4 byte header

• DTLS based PSK, RPK and Certificate security

• Subset of MIME types and HTTP response codes

• Built-in discovery

• Optional observation and block transfer

Advanced Message Queuing Protocol1.0 (AMQP)

• Efficient – binary connection-oriented protocol• Reliable – fire-and-forget to reliable, exactly-once delivery• Portable data representation and structured message definition• Flexible – peer-peer, client-broker, and broker-broker topologies• Broker-model independent – no requirements on broker internals• Rich flow control – multiplex multiple data streams over a connection• OASIS Standard (Oct 2012); International Standardization in progress

– Somewhat controversial…

Message Types

Voluntaryinformation flow

from device toanother system.

Requests forinformation fromdevice to other

systems.

Instructions fromother systems to

a device.

Information flowfrom other

systems to thedevice.

Telemetry Inquires Commands Notifications

Default Connectivity Model

• Connectivity (IPv6 + VPN)– Give every device a routable IP address– Devices expose services for control/query

operations– Command Source is either on premise or remote,

enabled by a bridge of some sort.– Remote access is enabled within the VPN’s

routing domain

Default Connectivity Model

Connections arecommand source

initiated.

Connections arecommand source

initiated.

Device exposes aservice/API

Device exposes aservice/API

Command SourceCommand Source

DEMO 1Device Commands with the Default IoT Connectivity Model

Default Connectivity Model Challenges

• Addressability– Requires network-layer intervention– Doesn’t work for devices that are loosely connected (roaming, frequently offline)

• Security– By default, every protocol that can be routed over Ethernet can flow – and between any

two nodes– SSL/TLS is not an option on many small devices.– VPN controls access to IP addresses and ports, not application endpoints (lack of

granular authorization)– Many devices are not VPN-capable due to resource/bandwidth constraints

• Efficient scale– VPN infrastructure is expensive and costly to maintain– Does not address device management.

• Think 1K, 10K, 100K+ devices

On-Premise Brokered DeviceCommunications

• Connectivity (IPv6 + VPN)– Give every device a routable IP address.– Devices participate in pub-sub messaging on-

prem or via VPN using industry standard protocollike MQTT.

– Command Source is either on premise or remote,enabled by a bridge of some sort.

– Remote access is enabled within the VPN’srouting domain.

On-Premise Brokered DeviceCommunications

Device subscribes tobroker via TCP, etc.Device subscribes tobroker via TCP, etc.

Device BrokerDevice Broker

Typically a socketconnection.

Typically a socketconnection.

Messaging happens onpremise, attack surface

minimized.

Messaging happens onpremise, attack surface

minimized.

MQTT, etc.Command SourceCommand Source

Must be on premise orsomehow bridged.

Must be on premise orsomehow bridged.

DEMO 2Brokered Commands with MQTT & RabbitMQ

On-Premise Brokered DeviceCommunications Challenges

• Addressability– Device and broker are intimately connected.– Doesn’t work for devices that are loosely connected (roaming, frequently offline).

• Security– SSL/TLS is not an option on many small devices.– Many devices are not VPN-capable due to resource/bandwidth constraints.

• Efficient scale– VPN infrastructure is expensive and costly to maintain.– External commands require some kind of a gateway service.– Does not address device management.

• Think 1K, 10K, 100K+ devices

Service Assisted Communications• Devices connect via open standard protocols

– AMQP 1.0 and HTTP supported natively by the Service Bus– MQTT, CoAP and others can be implemented via custom gateway/adapter model– Sockets secured via TLS (or a lightweight variant)

• Each device has a dedicated Inbox/Outbox on the Gateway– Device sends telemetry/alerts and routes service invocations via its Outbox– Device receives commands and queries from its Inbox– Correlated request/reply patterns can be implemented on top of these two messaging channels– The device knows, and has access to, only its own specific inbox/outbox endpoints (URI’s)

BackendComponents

BackendComponents

Cloud GatewayCloud Gateway

InboxInbox

OutboxOutbox

Com

man

d AP

ICo

mm

and

API

Prot

ocol

Hea

dPr

otoc

ol H

ead

Service-Assisted Communications

Connections aredevice-initiated and

outbound

Connections aredevice-initiated and

outbound

NAT/FirewallDevice (Router)

NAT/FirewallDevice (Router)

IP NAT

Cloud GatewayCloud GatewayCommand SourceCommand Source

Port mapping isautomatic, outbound

Port mapping isautomatic, outbound

Device does not listenfor unsolicited trafficDevice does not listenfor unsolicited traffic

No inbound ports open,attack surface is

minimized

No inbound ports open,attack surface is

minimized

Access-controlledcommand API

Secure, managed hostingplatform

Access-controlledcommand API

Secure, managed hostingplatform

DNSmyapp.cloudapp.net

IoT Cloud Platform “Stack” – AbstractModel

Non-IPCapableDevices

IPCapableDevices Cl

oud

Gat

eway

CustomCode

Clou

d Pl

atfo

rmSe

rvic

es

EnterpriseSystems

Third-Party Dataand Services

A B C D E F

Fiel

dG

atew

ay

Azure Hosting Options

Non-IPCapableDevices

IPCapableDevices Cl

oud

Gat

eway

CustomCode

Clou

d Pl

atfo

rmSe

rvic

es

EnterpriseSystems

Third-Party Data

and Services

A B C D E F

Web SitesWeb Sites

Mobile ServicesMobile Services Cloud ServicesCloud Services

External CodeExternal Code

VM RolesVM Roles

Fiel

dG

atew

ay

Azure Platform Services

Non-IPCapableDevices

IPCapableDevices Cl

oud

Gat

eway

CustomCode

Clou

d Pl

atfo

rmSe

rvic

es

EnterpriseSystems

Third-Party Dataand Services

A B C D E F

HD InsightHD Insight

Azure DatabasesAzure Databases Table/Blob StorageTable/Blob Storage

BizTalk ServicesBizTalk Services

Service BusService Bus

Fiel

dG

atew

ay

Media ServicesMedia Services

Azure – IoT Cloud Gateway

Non-IPCapableDevices

IPCapableDevices Cl

oud

Gat

eway

CustomCode

Clou

d Pl

atfo

rmSe

rvic

es

EnterpriseSystems

Third-Party Dataand ServicesFi

eld

Gat

eway

A B C D E F

Serv

ice

Bus

A/B Se

rvic

e Bu

s

A/B

Cust

omGW

Rol

e

Pattern 1: Device Direct Pattern 2: Custom Gateway

Telemetry Routing with the AzureService Bus

Split the stream Enable parallel processing

Implement different Q QoS levels Level and balance the load

Topic SubsFilters

Service Bus

Device 2

Receiver 2b

Device 1

Device 3Receiver 2a

Alerts

Data

Receiver 1 AlertProcessor

StoragePre-processor

Routing Commands with the AzureService Bus

TopicSubs Filters

Service Bus

Device 2

Device 1

Device 3Sender 2

Model A

Device 3

Sender 1

Model TModel T

Model A

Target individuals or groups Set delivery timeouts (TTL)

Deal with spotty connectivity Traverse NATs/firewalls

securely

DEMO 3Service-Assisted Device-Direct Commands over Azure Service Bus

Service Assisted Custom/CloudGateway Challenges

• Connectivity– Addressability (non-IP devices, firewalls/NATs, online/offline, roaming)– Heterogeneity (OS/firmware, power/network constraints, protocols)– Security (identity, authorization, privacy, data integrity)– Efficient Scale (millions of devices per tenant, at a reasonable cost)

• Messaging– Telemetry (collection, filtering, routing, throughput, per-message QoS)– Notifications (targeting devices/device groups within large populations)– Command/Query and Inquiries (correlation, sessions/batching)

• Data Analytics and Visualization– Its all about the data!

Additional Key Considerations

• Device Provisioning• Security• Performance• Scale• Redundancy

Service Bus MessagingService Bus Messaging

Device Gateway Accelerator –Reference Architecture (Reykjavík)

1. Custom ProtocolGateway

2. Telemetry Pump andAdapters

3. Command Gateway

4. Provisioning Serviceand Metadata Store

Custom Protocol Gateway HostCustom Protocol Gateway Host

MQTTMQTT CoAPCoAP ……

Telemetry/RequestRouter

Telemetry/RequestRouter

Notification/CommandRouter

Notification/CommandRouter

AdaptersAdapters Command API HostCommand API Host

ProvisioningService

DeviceMetadataand Key

Store

HD

Insi

ght

HD

Insi

ght

BizT

alk

BizT

alk

Orle

ans

Orle

ans

Azur

e St

orag

eAz

ure

Stor

age

Azur

eD

bsAz

ure

Dbs

Serv

ice

Bus

Serv

ice

Bus HTTP

HTTP

DevicesDevices

AMQP

11

22 33

44

ConfigurationConfiguration

HTTP

Your

Pro

cess

Your

Pro

cess

Device Gateway – Partition Topology

• The “Partition” is a set of resources dedicated to a specific devicepopulation (or subset thereof).

• The “Master” role manages partition deployment and deviceprovisioning into the partitions.

PartitionMaster

PartitionRepo

PartitionRepo

Command TopicsCommand Topics

Service Bus Standard ProtocolService Bus Standard Protocol Custom ProtocolCustom Protocol

Device RepoDevice Repoin0000in0000 inFFFFinFFFF…in0001in0001 in0002in0002

AMQPAMQP HTTPHTTP MQTTMQTT Custom Protocol HostCustom Protocol Host

Protocol AdaptersProtocol Adapters

diagdiagallall diagdiagallall diagdiagallall diagdiagallall

Telemetry Pump/RouterTelemetry Pump/RouterN Instances

TelemetryAdapter

TelemetryAdapter

TelemetryAdapter

TelemetryAdapter

TelemetryAdapter

TelemetryAdapter

DeploymentRuntime

DeploymentRuntime

out0000out0000 outFFFFoutFFFF…out0001out0001 out0002out0002

s000

1s0

001

s000

2s0

002

s03E

7s0

3E7

s000

1s0

001

s000

2s0

002

s03E

7s0

3E7

s000

1s0

001

s000

2s0

002

s03E

7s0

3E7

s000

1s0

001

s000

2s0

002

s03E

7s0

3E7

g0000/rte0000g0000/rte0000

g0000/rte0001g0000/rte0001

out0

out0

out1

out1

out2

out2

n Groups of m Routers

out0

out0

out1

out1

out2

out2

g0001/rte0000g0001/rte0000

g0001/rte0001g0001/rte0001

out0

out0

out1

out1

out2

out2

out0

out0

out1

out1

out2

out2

ProvisioningRuntime

ProvisioningRuntime

Ingestion Topics (Telemetry)Ingestion Topics (Telemetry)

CommandAPI Host

CommandAPI Host

Device Gateway – Customer Topology

• Global coverage achieved by spreading partitions across multiple Azure regions• Reference architecture supports up to 1000 distinct partitions• Number and distribution of partitions driven by data volumes, business continuity, legal

and proximity considerations

DEMO 4Provisioning and Exercising Reykjavik

Device

Device

EventHub Azure

EventProcessing

AzureStorage

Azure

Customer Apps

HDInsight

BI Systems

3rd Party Solutions

Data Flow

SQLAzure

EventHub

BasicDevice

Registry

Command & Control

Microsoft Azure Stack for IoT

Device(Non-ISS)

Device(Non-ISS)

EventHub

AzureStorage

Rich Device Registry & Object Model of “Things”Rich Device Registry & Object Model of “Things”

Azure

ISS

Customer Apps

HDInsights

BI Systems

3rd Party Solutions

Data Flow

ISS Solution built on Azure

SQLAzure

EventHub

BasicDevice

Registry

ISS

Secu

rity,

Priv

acy

&Sh

arin

g Co

ntro

ls

IoT Rule TemplatesIoT Rule Templates

Natural LanguageQuery

Natural LanguageQuery

ISSAgents

ISSAgents

ISSAgents

ISS Solution

Sing

le A

ccou

nt, P

er d

evic

e Bi

lling

,et

c.Si

ngle

Acc

ount

, Per

dev

ice

Billi

ng,

etc.

Command & Control

AzureEventProcessing

ISS

Port

alIS

SPo

rtal

Protocol reach to devices andplatforms

Windows

AzureService BusQueuesTopicsEvent hubs

/azure-sdk-for-python/

/azure-sdk-for-php/

/azure-sdk-for-node/

/azure-sdk-for-java/

/azure-sdk-for-ruby/

HTTP(S)HTTP(S)

https://github.com/windowsAzure/AMQP 1.0AMQP 1.0

Other platforms

AMQP1.0AMQP1.0

Embedded

Event Hub is a pub-sub ingestorservice

– Variety: > million publishers with HTTP/AMQP– Velocity: > million EventData data ingress/second– Volume: > GB/s ingress, concurrent consumers– Security: SAS based, unique token per publisher– Buffer: Consumer provides its cursor/offset– Durable: Between 1 and 30 days retention– Latency: 50ms end-to-end durable– Cheap: Competitive pricing, PaaS service so pay-as-

you-go

Event Hub for IoT: Big DataIngestion

Event Hub

More on ISS & Event Hub• //build 2014: Windows and the Internet of Things: http://bit.ly/1ijTeyW• Internetofyourthings.com• Azure Service Bus Event Hubs: http://bit.ly/eventhub

References• Internet of Things with Azure Service Bus: http://bit.ly/1m4MMME• Windows and the Internet of Things: http://bit.ly/1ijTeyW• Subscribe!: http://channel9.msdn.com/Blogs/Subscribe• Service Assisted Communications:

http://vasters.com/clemensv/CategoryView,category,Architecture.aspx• Internet of Things & Azure Service Bus: http://bit.ly/1jFf5k5 and

http://bit.ly/1jFf5k5• M2MQTT Library for .NET MF: http://m2mqtt.codeplex.com/• Special thanks to Clemens Vaster, Markus Horseman and Todd Holmquist-

Sutherland on the Microsoft Azure M2M team.• Demo code: https://github.com/rickggaribay/IoT

More on Reykjavik/Device Gateway

• //build 2014: Internet of Things with Azure Service Bus:http://bit.ly/1m4MMME

• Neudesic is currently offering industry-specific briefings on IoT.

• We are very interested in working with early adopters or those seeking tomodernize their existing IoT investments.

http://neudesic.com/iotInvitation code: VSLRedmond

twitter: @rickggaribayblog: http://rickgaribay.netemail: [email protected] | [email protected]

About Me• VP, Distinguished Engineer leading the Development

Platform Group at Neudesic• Working on IoT, Intelligent Transportation and Hospitality

& Gaming• Microsoft MVP, Microsoft Azure• Co-Author, “Windows Server AppFabric Cookbook” by

Packt Pub.• Chairman, Co-Founder Phoenix Connected Systems User

Group (PCSUG.org)• twitter: @rickggaribay• blog: http://rickgaribay.net• email: [email protected] | [email protected]