VMworld 2015 San Francisco - INF5432 - Infrastructure as Code - Ban Snowflake Deployments

1© Copyright 2015 EMC Corporation. All rights reserved.

INFRASTRUCTURE AS CODEBAN SNOWFLAKE SERVERS


2© Copyright 2015 EMC Corporation. All rights reserved. 2© Copyright 2015 EMC Corporation. All rights reserved.

Jonas RoslandDeveloper AdvocateEMC {code}

@jonasrosland

Loves retro gamingTalks a lot

3© Copyright 2015 EMC Corporation. All rights reserved. 3© Copyright 2015 EMC Corporation. All rights reserved.

Reid VandewieleSolutions EngineerPuppet Labs®

@reidmv

Biking, HikingCode



WHAT IS DEVOPS?


AN IDEA OR COLLABORATIVE CULTURE/PHILOSOPHY BETWEEN

TECHNICAL TEAMS


UNIFIED PROCESSES AND TOOLS FOR FASTER END-TO-END DELIVERY

OF QUALITY SOFTWARE


NOT A JOB DESCRIPTION!


SO WHY SHOULD YOU CARE?


YOU CAN'T WAIT 6 MONTHS FOR A FEATURE UPGRADE/BUG FIX/ETC


SOLVE CONFIGURATION DRIFT ISSUES QUICKLY


KNOW EXACTLY WHAT'S WRONG, WHERE AND WHY

13© Copyright 2015 EMC Corporation. All rights reserved.http://46zwyrvli634e39iq2l9mv8g.wpengine.netdna-cdn.com/wp-content/uploads/2013/06/DevOps-infinity-loop2.png


LAST BUT NOT LEAST, IMPROVE TEAM COMMUNICATION


SO WHAT'S THE RESULT OF DEVOPS?


IT'S REALLY BIG ACTUALLY...


DEVOPS PRACTICES AND IT PERFORMANCE IMPACT

ORGANIZATIONAL PERFORMANCE

http://www.slideshare.net/realgenekim/2014-state-of-devops-findings-velocity-conference


LET'S START WITH

INFRASTRUCTURE AS CODE


WHAT DO WE DEFINE AS INFRASTRUCTURE?





• Deploy, monitor and connect together all the pieces needed to run services for the organization• Processes and tools for faster end-to-end delivery of

quality services• Automation comes built-in

WHY DEFINE INFRASTRUCTURE AS CODE?


• Desired state specified in text files• Autonomic (self-corrects to desired state)• State should be known through monitoring• Remove snowflake servers

HOW DO WE DO THIS?


• Easy to read and edit• Shareable• Can use standard version control like Git or SVN• Becomes executable documentation

WHY STORE THEM IN TEXT FILES?


node 'www2' {

class { 'apache': } # use apache module

apache::vhost { 'awesomewebsite.com': # define vhost

port => '80',

docroot => '/var/www/html'

}

}

SIMPLE EXAMPLE


user { root: ensure => present, password => '$ecretP@ssw0rd',}

CHANGE ROOT PASSWORD


MANUALLY CONFIGURED ENVIRONMENTS ARE LIKE A HOUSE

OF CARDS IN A CHINA SHOP

- NEAL FORD


• Deploying, provisioning and scaling automatically is virtually impossible if every server is unique• Adds friction between the requestor and the deployer• Mistakes happen• We're all human

SNOWFLAKE SERVERS


WHAT CAN HAPPEN IF YOU DON'T TREAT

INFRASTRUCTURE AS CODE?


UP TO 60% OF FAILURES ARE CAUSED BY HUMAN ERROR, NOT

HARDWARE FAILURE


EXAMPLE - KNIGHT CAPITAL GROUP

https://infocus.emc.com/wp-content/uploads/2012/08/KCG.jpg


• Manual deployment of new trading software• 7 of 8 servers correctly updated• Old function still alive on the 8th server led to…

KNIGHT CAPITAL GROUP - WHAT HAPPENED?


$440 MILLION LOSSIN 45 MINUTES


SO


TREAT YOUR INFRASTRUCTURE AS CODE


BARE METAL APP STACK



• Razor– Text / JSON configuration– Bare-metal provisioning– Policy based

• Puppet– Text configuration manifests– Software configuration management



OS – DNS – DHCP - DB

Storage – Network - Compute

INFRASTRUCTURE AS CODERA

ZOR


OS – DNS – DHCP - DB

Storage – Network - Compute

App Queue Business Logic

PUPP

ETINFRASTRUCTURE AS CODE


STARTING ENVIRONMENT:INFRASTRUCTURE AS CODE

One Razor ServerOne Puppet Server


LIVE DEMO




Boot PXE To Razor

OS Installed Puppetize! Decommissio

n

One-time Provisioning

Lifetime Management


Boot PXE To Razor

OS Installed Puppetize! Decommissio

n

One-time Provisioning

Lifetime Management



• Self documenting infrastructure• You now have source code for how anything in your

datacenter is setup• Executable documentation

RECAP OF THE BENEFITS


ALRIGHT


THEN WHAT?


HOW ABOUTAUTOMATING SOME MORE?


• Jenkins CI• Travis CI• Codeship• CloudFoundry• OpenShift

DEPLOYMENT AUTOMATION


• Docker• Framework for application container management• HUGE community

CONTAINERS


• Splunk• Loggly• LogInsight• ELK Stack

– ElasticSearch– LogStash– Kibana

LOG MANAGEMENT


• vCenter Operations• New Relic• AppDynamics• DataDog

PERFORMANCE MANAGEMENT


• Nagios• Sensu• VictorOps• PagerDuty

MONITORING


• Slack• HipChat• Trello• Asana

COMMUNICATION


• Github• Quip

COLLABORATION


• And by smartly we mean cooperatively between Devs and Ops• Will help operators understand what's running and

happening in the infrastructure• Will give developers insight into the underlying

infrastructure and the way it behaves under load

USE THESE TOOLS SMARTLY


• Start talking about it• Choose low-hanging fruit• Learn, make wrong decisions, learn again• Involve the broader team

– Give books, presentations, blog posts etc.

SO HOW DO WE GET STARTED?


QUESTIONS?Jonas Rosland

Developer Advocate @ EMC {code}@jonasrosland

VMworld 2015 San Francisco - INF5432 - Infrastructure as Code - Ban Snowflake Deployments

Technology