Leveraging the Cloud: Getting the more bang for your buck

Getting the most bang for your buck

Nate Lindstrom Director of Network Operations at Desk.com

www.linkedin.com/in/nwlindstrom

§  We make it easy for you to support customers right from the browser, via email, phone, chat web, Facebook and Twitter

§  We provide a hosted, cloud-based SaaS

help desk platform for SMBs

Cloudy Change Management Trust but verify

Cloud Change Management Trust but verify

Process requirements

Formal, documented change management

ISO 27001 compliance

SOX section 404 compliance

Safe Harbor certification

Process Requirements

§  Formal, documented change management

§  ISO 27001 compliance

§  SOX section 404 compliance

§  Safe Harbor certification

Single file change process

§  Changes can be made rapidly and safely

§  Unauthorized changes reverted by the CMS or flagged by CloudPassage Halo FIM

RFC created

RFC closed

Make pull

request

Change applied to

staging

Effects observed

Change applied to

production

FIM updated

SME reviews request

Under the hood

§  Chicken-and-egg problem for new instances

§  Puppet determines role based on hostname

§  Hostname isn’t set on new instances

Under the hood

Chicken-and-egg problem for new instances

Puppet determines role based on hostname

Hostname  isn’t  set  on  new instances

How we start instances

node/^web\d+.desk\.com$/ inherits production_app { include web}

Puppet

AMI

Script

web01.desk.com nginx

Name=web01.desk.com

How we monitor instances

S3 Bucket

web01.desk.com cron

Effective monitoring

§  Icinga is the most comprehensive open source monitoring solution available

Effective monitoring

Icinga is the most comprehensive open source monitoring solution available

Secret change process

§  “Secret” as in production secrets, like passwords

RFC created

RFC closed

Make pull

request

Change applied to

production

FIM updated

SME reviews request

Under the hood

§  Storing production secrets in plain text is BAD

§  Sending decryption key over same channel as encrypted data is BAD

Under the hood

Storing production secrets in plain text is bad

Sending decryption key over same channel as encrypted data is bad

Secure repositories TechO

ps Everyone

Puppet git

Repo

Prod Credentials

Non-Prod Credentials

Full Access Pull Request Only

GnuPG GnuPG

Secure distribution

Puppet git

Repo

Instance Credentials Puppet

git  

Secrets  

GnuPG Key  

AMI  

What the cloud means to us More typing, less driving

What the cloud means to us More typing, less driving

Physical asset tracking

§  If you came to doubt the accuracy of your CMDB, you could always fall back on a physical inventory

§  Almost always, anyways

Physical asset tracking

If you came to doubt the accuracy of your CMDB, you could always fall back on a physical inventory

Almost always, anyway

Virtual asset tracking §  When you don’t have any physical assets it’s even

easier to “lose” instances

§  “Lost” instances can silently consume big $$$

Virtual asset tracking When  you  don’t  have  any  physical  assets  it’s  even  easier  to  “lose”  instances

“Lost”  instances  can  silently  consume  big  $$$

How an instance can be lost §  Provisioning script loses connectivity during launch

§  Instance fails to upload existence information to S3

S3 Bucket

CMDB

Instance

Updates Launches

Provisioning Sot

Minimizing lost instances

§  Your CMDB may not see your lost instances consuming $$$, but Cloudyn does

§  Cloudyn makes it easy to maintain an efficient and lean cloud presence

Minimizing lost instances

Your CMDB may not see your lost instances consuming $$$, but Cloudyn does

Cloudyn makes it easy to maintain an efficient and lean cloud presence

JIT capacity Let your servers order more servers

JIT capacity Let your servers order more servers

Auto Scale architecture §  Everything should scale horizontally

Auto Scale architecture

Everything should scale horizontally

Auto Scale in action §  Loosely-couple tiers provide greatest flexibility

§  Scale up quickly, scale down slowly Auto Scale in action

Loosely-coupled tiers provide greatest flexibility

Scale up quickly, scale down slowly

Web Web Web Web Web Web Web

App App App App App App

ELB

ELB

Traf

fic In

crea

sing

Traffic Decreasing

Auto Scaling control

§  Scalr makes managing dynamic environments in the cloud easy and painless

Auto Scaling control

Scalr makes managing dynamic environments in the cloud easy and painless

Whole-unit troubleshooting

Don’t sweat the small stuff

Whole-unit troubleshooting Don’t  sweat  the  small stuff

Think in clusters §  If one instance is having problems, replace it

§  If many instances are having problems, dig deeper

§  Use the 1, 2, 3 rule for determining response

ELB  

Instance Instance Instance Instance Instance

Architecture for failure

Build it to land gracefully

Architecting for failure Build it to land gracefully

AWS

Expect failure §  Make use of regions and availability zones

§  Avoid storing sessions on any one server

§  The cloud is inherently unreliable, but your app doesn’t need to be

us-west-1

Us-west-1a

us-east-1

Us-west-1b

Security awareness

False security is worse than no security

Security awareness False security is worse than no security

Cloud isn’t private §  Multitenancy means the cloud is never truly

private

§  Build security in from the very beginning

§  Apply defense in depth

ELB   Web

Internet

ELB   App DB

Security groups are limited §  An instance’s security

groups cannot ever be changed

§  Security groups can only limit inbound (ingress) traffic

§  Security groups cannot restrict outbound (egress) traffic

Security groups are limited An  instance’s  security  groups cannot ever be changed

Security groups can only limit inbound (ingress) traffic

Security groups cannot restrict outbound (egress) traffic

Comprehensive security

§  CloudPassage Halo allows the implementation of comprehensive security with minimal effort

Comprehensive security

CloudPassage Halo allows the implementation of comprehensive security with minimal effort

The cloud… §  Is not a data center

§  Is only as secure as you make it

§  Is very expensive if not managed well

§  Works best with lots and lots of litter servers

§  Will occasionally fail

The cloud...

Is not a data center

Is only as secure as you make it

Is very expensive if not managed well

Works best with lots and lots of little servers

Will occasionally fail

THANK YOU!