Chandler Howell's AWS Chicago user group presentation "IoT Security in AWS"l

Building Secure IoT Ecosystems

With AWS

Topic: IoT/Security/TOC/#

{{ “Message”:“Background,The Ecosystem,Precursors,AWS Overall,Infrastructure,Multi-Tenancy,Transport,Clients,LWT,Resources”}

}

IoT/Security/Background/WhoAmI

I’m Chandler HowellDirector of Engineering at [email protected]@chandlerhowell on Twitter

Nexum is a Network & Security Reseller & Consultancy

Headquartered in ChicagoPresence East of the Mississippi Riverhttp://nexuminc.com

Iot/Security/Background/Why

Hopefully, to have wasted your time

Started talking IoT Security in 2015How it suckedWhy it suckedWhy it mattered

IoT/Security/Ecosystem/#

AWSAWS IoTInfrastructure

TransportRESTful HTTPWebSocketMQTT

ThingsSensorsDevices

ClientsAppsAPI’sWeb portals

Gotta Secure ‘em All!

Iot/Security/Precursors/HowToFAIL

What am I hoping you’ll avoid?

FAILURE

Harming peopleBeing in the newsDevice recalls/updatesGetting suedCorporate bankruptcyBringing bad products to market

IoT/Security/Precursors/BakedIn

Security happens before the Things

Baked in, not painted on

IoT/Security/Precursors/Challenge

I’m here to pose questions

It’s up to you to answer them

IoT/Security/Precursors/Policies

You’re in the Data Business nowWhat data do you collect?How long do you retain it?Who owns the data you collect?

Do you have a published Privacy Policy?Have you calibrated your business model with it?Does it cover both Internal & Third Party Use, Sharing and Disclosure?

Do you have a published Service Level Agreement?

Do you have Incident Response & Disaster Recovery Plans?

IoT/Security/Precursors/Assumptions

If you must assume, assume things will go wrong.

* Connectivity will fail* Network* Device Association

* Vulnerabilities will be found* Devices will be compromised* Keys/Credentials will be compromised* Retail (client)* Wholesale (server keys)

* Data might be breached or destroyed

How will each of these affect...your customers?...your product?...your company?

IoT/Security/Precursors/Challenges

How will you…* Push updates to Things

* When would you force an update?* Cryptographically verify those updates

* Track versions* Deal with version incompatibilities

* Deal with potential downtime

IoT/Security/Precursors/Transport

* How much data can you lose to outages?* How much data can you queue?

* If you can’t publish to a Thing...* What telemetry is lost?* What functionality is lost?

* How will you handle network limits* Blocking MQTT* Blocking HTTPs* Blocking un-inspected SSL/TLS

IoT/Security/AWS

Expect to use LOTS of services* Manage access through IAM Policies & Roles* Segment where it makes sense* Consider CRUD needs for all access * (Create, Read, Update, Delete)

AWS Provides some features by default* DDoS Protection

IoT/Security/AWS/EC2

* This is a great place to get pwned* “Traditional” IT brings Traditional

problems

* ALL SERVERS SHOULD BE EPHEMERAL* “Pets vs. Cattle”

* Amazon Inspector is your friend* https://aws.amazon.com/inspector/ * Security Scanning* Can be automated with Lambda & SNS* Ticketing, e.g. into Jira

IoT/Security/AWS/Monitoring

You can’t find what you don’t look for

Log & alert changes to:* Running instances* Policies* IAM Roles* Accounts* Security Groups* Billing Events* Workload spikes* Errors & Exceptions

IoT/Security/AWS/HowToWin

* Harden the environment* Delete the Root Access Keys* Enable Multi-Factor/Strong Authentication

* Adhere to the Principle of Least Privilege* Don’t just hand out Permissions* RTFM on Policies & Permissions* AWS Provides Sample Policies

* Don’t forget your processes* User Management* IT Inventory & Asset Management* Vulnerability & Configuration Management

IoT/Security/AWS/HowToWin/2

Minimize your Attack Surfaces* Only expose Public Services to the Internet* Segment where it makes sense* Limit internal access to Production

* Make use of AWS’ IAM & RBAC* RBAC is a de facto inter-service firewall

Use real remote access* Direct Connect* Layer 3 VPN* Bastion Host/Jump Box* Assuming you absolutely HAVE to access

instances directly

IoT/Security/AWS/Infrastructure/PSA

Public Service Announcement:

Don’t put your infrastructure raw on the Internet!

Don’t be a…MongoDB or ElasticSearch

Mass hack victimVtech Hack VictimCloudPets Hack Victim

IoT/Security/AWS/MultiTenancy

Assume you need multi-tenancy…* How distinct must the segregation be?* Separate accounts* Cumbersome, but most effective

* Separate data stores* Do-able, but shifts complexity into the

business logic* Common data stores with key fields* Best option if it is an option

IoT/Security/Multi-Tenancy/IAM

* Not complete or by default in all services

* Fine-grained access control through AWS Identity & Access Management (IAM)

But…some Control Plane calls are not (yet)e.g. “list-things” will show all devices,

not just a tenant’s devices* Wrap that API to filter to just the

tenant via device registry

IoT/Security/Multi-Tenancy/HowToWin

* Define Requirements up front* How much segregation is enough?

* Review each service’s capabilities* Make sure you solve before you commit

* Include tests for cross-tenant failure* Can you CRUD resources you should not be

able to?

IoT/Security/AWS/Lambda

* This is where the magic happens* Good Magic* like working code

* Bad Magic* AppSec vulnerabilities

* Resources like OWASP apply here, too* http://www.owasp.org* Open Web Application Security Project* SANS Also has great AppSec training* http://sans.org

IoT/Security/AWS/Transport/+

* 3 Options within AWS* HTTP (RESTful)* WebSocket* Message Queue Telemetry Transport (MQTT)

IoT/Security/Transport/HTTP

* HTTP* POST to a RESTful API* Only scales so far* Included for completeness

* <AWS IoT Endpoint>/topics/<url_encoded_topic_name>?qos=1* Uses AWS Signature Version 4* Add either a Query String param or an

Authorization: headerGET https://iam.amazonaws.com/?Action=ListUsers&Version=2010-05-08 HTTP/1.1Authorization: AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20150830/us-east-1/iam/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924a6f2b5d7content-type: application/x-www-form-urlencoded; charset=utf-8host: iam.amazonaws.comx-amz-date: 20150830T123600Z

IoT/Security/Transport/WebSocket

* WebSocket tunnels MQTT over HTTP(s)* Good for passing firewalls* Runs over port 443* Uses the HTTP UPGRADE verb

* Also AWS uses Signature Version 4* URL Format:

wss://<endpoint>.iot.<region>.amazonaws.com/mqtt

* Best if you have a hub forwarding traffic

* Either no MQTT allowed* or older, crypto-incapable devices

IoT/Security/Transport/MQTT

* Publish-Subscribe protocol* 1st implementation 1999* Designed for high-latency, low bandwidth* Lightweight* Bandwidth* CPU

* Can be secure, but can be Not Secure, too

IoT/Security/Transport/MQTT/QoS

* 3 Quality of Service (QoS) Options* 0 – At most once * Best Effort* No retry, no acknowledgement

* 1 – At least once* Retry until acknowledgement received* May result in multiple deliveries

* 2 – Guaranteed single delivery* Full send-ack transaction & queueing

IoT/Security/Transport/MQTT/Authentication

* Multiple Options* No Authentication (Don’t do this)* Topic-based Pseudo-auth (Another NOPE)* Username/Password (Dont’ do this either)* X.509 Certificates (Do this)* AWS makes this easy

IoT/Security/Transport/MQTT/HowToFail

* Authentication* No Authentication* Weak Authentication

* Not encrypting Traffic* MQTT+TLS For The Win* (Unless absolutely necessary)

IoT/Security/Transport/MQTT/HowToWin

* Use X.509 Certificates for Authentication* Always use TLS if possible* Use AWS IAM to define device roles* Follow Principle of Least Privilege* Test for Information Leakage* e.g. aws iot list-devices in multi-

tenant environments* If you have insecure legacy devices, use

a broker for secure upstream transport

(Yes, that’s the username & password being sent in the clear!)

IoT/Security/Transport/MQTT/PSA

Public Service Announcement:

Username & Password are even less of your friend than usual

IoT/Security/Clients

* Researcher focus has largely been on clients* Soft targets* Riddled with Amateur Hour vulnerabilities* Weak machines * Under their physical control* Fewer legal issues

* Ecosystem testing still the realm of authorized testers* They don’t generally publish* So less data to to assume against

IoT/Security/Clients/HowToFail

* A few pitfalls to avoid* Use of no/default credentials* Re-use of keys or credentials* Hard coding credentials* Assuming a friendly deployment environment* Running unnecessary services* Especially network services

* Not using signed/secure images

IoT/Security/Clients/HowToWin

* Incorporate security into your design* Threat Model* Educate yourself on AppSec

* Scan/Attack your services & device ports* Dynamic Analysis tools

* Run Static Analysis tools on your source code* Or at least include failure-mode tests

* Consider credential storage on the client* How hard is credential (key) compromise?* What do those keys get you?* Do you leak credentials, e.g. Wi-Fi?

IoT/Security/LWT

I’ve asked you a lot of questions, so I guess it’s only fair to let you ask me some.

IoT/Security/Resources

* Security & Identity for AWS IoT * https://docs.aws.amazon.com/iot/latest/developerguide/iot-security-identity.html

* Things I wish I’d known before I started working with AWS* https://wblinks.com/notes/aws-tips-i-wish-id-known-before-i-started/* especially this change monitoring script

* https://s3.amazonaws.com/reinvent2013-sec402/SecConfig.py

* AWS Security Blog Post: Automatic Remediation with AWS Inspector* https://aws.amazon.com/blogs/security/how-to-remediate-amazon-inspector-security-findings-automatically/

* MQTT Security Fundamentals* http://www.hivemq.com/mqtt-security-fundamentals/

* AWS IoT Protocols* https://docs.aws.amazon.com/iot/latest/developerguide/protocols.html

* How to bridge Mosquitto MQTT Broker to AWS IoT* https://aws.amazon.com/blogs/iot/how-to-bridge-mosquitto-mqtt-broker-to-aws-iot/

* Multi-Tenant Storage with DynamoDB* https://aws.amazon.com/blogs/apn/multi-tenant-storage-with-amazon-dynamodb/

* AWS Access Management* https://docs.aws.amazon.com/IAM/latest/UserGuide/access.html