Devopsdays State of the Union Amsterdam 2014

Devopsdays State of the Union

John Willis VP of Customer Enablement

Statelessnetworks @botchagalupe

Devops State of the Union

• The 2014 Devops Survey

• Software Defined Everything

• Consumable Composable Infrastructure

#Sometimes presentations write themselves #I was trying to think of what are the latest and greatest things going on that have an impact on us #devops folk… ## last week was a good week for some cool disruptions…

Devops State of the Union

• The 2014 Devops Survey

• Software Defined Everything

• Consumable Composable Infrastructure

The Big Fat Rethink

#Sometimes presentations write themselves #I was trying to think of what are the latest and greatest things going on that have an impact on us #devops folk… ## last week was a good week for some cool disruptions…

Devops Survey

#2013 was a good first start… It proved out the actions of high performance orgs #2014 might not the perfect but…. it’s pretty awesome improvement in that the data is starting correlate high perf w/competitive advantage and in some cases financial improvements #Meta Points: (this is really good data) ##9200 vs 4000 ##Empl Size = 41% in the 500 to 10k+ range ##Dept = Almost 60% are in the right place ##Servers = Roughly 60% >100 - were significant orgs ##20% are rather large orbs (>2k which would be a good indicator of enterprise)

Devops Survey

• Organizational Performance

• IT Performance

• Organizational Culture

• Job Satisfaction

#In summary the report kind of describes this idea of “Org Perf” #Also points out IT Performance = Competitive Advantage (up to 2x) ##A smaller subset of the respondents volunteered co / pub traded name/ of that subset the findings were that there was a 2.5x of ones that fell into the high IT perf categories #Organizational Culture = IT Performance ( One of the strongest predictors of IT Perf ) ##High trust, ##Cross Functional Collaboration, ##Shared Responsibilities, ##Learning Organization #Job Satisfaction is a key indicator for Organizational Performance #We’ve know this all along .. this is devops.. # We already know this as #devops

Devops Survey

• Organizational Performance

• IT Performance

• Organizational Culture

• Job Satisfaction= Devops

#In summary the report kind of describes this idea of “Org Perf” #Also points out IT Performance = Competitive Advantage (up to 2x) ##A smaller subset of the respondents volunteered co / pub traded name/ of that subset the findings were that there was a 2.5x of ones that fell into the high IT perf categories #Organizational Culture = IT Performance ( One of the strongest predictors of IT Perf ) ##High trust, ##Cross Functional Collaboration, ##Shared Responsibilities, ##Learning Organization #Job Satisfaction is a key indicator for Organizational Performance #We’ve know this all along .. this is devops.. # We already know this as #devops

Devops Survey

Devops Competitive Advantage

=

#However, this survey is the first form of proof that we are right… the data… # We can now use this data to start the “proof” that we are on to something. #We need a lot more data….

#

The Big Fat Rethink

#

Software Defined Everything

• Software Defined Data Center

• Software Defined Computing

• Software Defined Storage

• Software Defined Networking

#SDDC has also been referred to as Converged Infrastructure !# SDC - ## First order abstractions: Hypervisors on physical, then/now LXC, libContainer (containers) - Docker! - Micro-task virtualization! ## Second order abstractions are distributed clusters (Cloud, Big Data ) .. well defined API’s !#SDS ## First order abstractions virtualized storage hardware ## Second order abstractions ( Ceph, Swift, Swiftstack, S3).. well defined API’s (can you download the software) !#SDN ##First order abstractions VLAN .. again virtualizing a single box.. switch L2) ##Second order abstractions Tunnels … VXLAN… .. well defined API’s ## SDN offers further service abstractions … ### .. adding functionality into the network routing fabric for example LB (SDAS), FW (SDSec) becomes smart routing !# Major points… ## all three abstractions move one step away from the hardware. Second order abstractions are not directly coupled hardware. ## All three have high order programming interfaces (API’s). #These hight order highly programable abstractions changing the landscape of IT

SDN is the Decoupling of Control From the Data Plan

http://networkstatic.net

SDN is the decoupling of the control plane from the data plane... physically removing… !Network gear has traditionally been a black box w/coupled arch D,C,M !Data Plane.. packet-in packet-out .. nano second .. wire speed Control Plane.. Local distributed brains of the network (BGP, OSPF, MPLS), How the forwarding state gets to the data place .. milliseconds/days !Stanford Seminar - Software-Defined Networking at the Crossroads http://www.youtube.com/watch?v=WabdXYzCAOU !Nicira Notes: Nicira Networks was acquired by VMware for $1.26B. (7/23/12) Posted 7/23/12 at 6:14pm via techcrunch.com !Nicira Networks added Alan Cohen as VP, Marketing. (10/1/11) Posted 10/21/11 at 4:42am !Nicira Networks received $26M in Series C funding. (2/3/11) Posted 2/4/11 at 6:31pm via sec.gov !Nicira Networks added John Vrionis as Investor. (2/1/11) Posted 9/4/12 at 9:08am !

Openflow is to SDN as HTTP is to Web

http://networkstatic.net

OpenFlow is not magic, just a low level primitive with momentum for FIB programming (forwarding plane programming). Explain the flow table... Interesting thing this can be a device as ( a router, switch, firewall, Nat and even a Load Balancer). ... now it starts getting very interesting... The OpenFlow Switch and Controller can communicate via the OpenFlow protocol, !Service chains (firewalls, Load balancers, IDS, L3 Switching) !

Software Defined Everything

• Disaggregation of Hardware/Software

• X86 Compute Servers

• IP Based Storage

• Bare Metal Switches

# We saw disaggregation of compute about 15 years ago (Sun/IBM). # ISCSI over Fiber Channel .. # The latest and greatest is in networking… Cumulus Networks # OCP

!

!

• Software Defined Data Center

• Rethink Declarative

• Rethink Source of Truth

• Rethink Layer 7 Semantics

Software Defined Everything

# We need to start to rethink declarative and desired state infrastructure. # We are somewhat Myopic on compute? ## It’s not just server config, packaging and vm provisioning anymore. ### How do you create declarative desired state converged infrastructure (SDE) ### Storage config, network configurations? ### Declarative Network topologies ### How are SDN flow tables declared (SDN gui’s/API abstractions). Frenetic/Pynetic ### When do we get/have converged Cloud API abstractions (e.g., a jClouds that abstracts SDC,SDS and SDN) ### What does provisioning look like for containers.. #### With things like Mesos ad Fleet the new Kubernetes) ## What is the original desired state and what is the ongoing operational state of truth .. who wins? !#Do our new declarations start looking more like policies. ## I need service A ### it needs n amount of nodes with x amount of cpu/mem per node. ### How is storage allocated for this service (object storage/block storage) ### Does it need virtual perimeters for Networking (VXLAN) and Security (FW) and ### How is this application service request queuing managed (LB SDAS) and also QOS !# infrastructure - is there a common definition of a node? Is there a single source of truth for all “nodes” ## DNS is not enough… ## Service Discovery/Mapping … go based derivatives of zookepper. Etcd/confd .. ### Consul .. someones needs to give Mitchell some Demerol and slow this kid down..

Composable Infrastructure

Composability is a system design principle that deals with the inter-relationships of components. A highly composable

system provides recombinant components that can be selected and assembled in various combinations to satisfy

specific user requirements. !

Wikipedia

# Derek Collison (creator of CloudFoundry) says we are on the dawn of Composeable Infrastructure. Actually Google has been doing this very well of a number of years now. # The big difference in with things like docker containers are consumable…

• Bare Metal (8 weeks)

• Virtualization ( 2 weeks)

• IaaS ( 2 minutes)

• PaaS ( 1 minutes)

• Containers ( 500 ms)

Consumable Composable Infrastructure

#

• Docker

!

• Commoditized Containers

• Portable Images

• Using a Git Like Workflow

Composeable Infrastructure

# How to describe docker to a friend… !## Very easy to install and run LXC containers… ## Docker binaries are portable across multiple visualization infrastructures. ### Truly a create once run anywhere… (BM, VBOX, Vsphere, AWS, GCE, CF) ## My personal favorite is the Git like workflow… (docker pull,push, diff commit) ### The idea is you get the image from a repo, you change it, commit and push it). ### Docker uses “copy on write file system so you can do interesting workflows… !#commit, diff, pull, push

Virtualization Docker

Composeable Infrastructure

# From the #docker site…

• Orchestration

• Mesos

• Kubernetes

• CloudOS

• CoreOS

• Atomic

Composeable Infrastructure

# Openstack is to KVM as Mesos/Kubernetes is to Docker !## Mesos mostly developed by ex google guys who went to Twitter ### Google had the borg—>omega — (OSS) ->Kubernetes !## Distributes container workloads across multiple physical boxes with no/OSS overhead !# Will the real JEOS please stand up…

Racks of the Future

Cumulus Networks !

Opendaylight !

Mesos Server !

Docker Registry !

ETC/D

Cumulus Networks !

OVSDB !

Openvswitch !

Mesos Client !

Docker Engine !

CoreOS

# Left hand side is the management rack… (think Openstack Controller Node) # Right is the compute node .. (see what’s missing… i.e, hypervisor)

John Willis @botchagalupe

Some Images licensed under Bigstock.com