Keynote ECSE 2011: Development and Operations - Two Worlds Collide

07.09.11

Development and Operations Two Worlds Collide Eberhard Wolff Architecture and Technology Manager adesso AG

About me ►  Eberhard Wolff

►  Architecture & Technology Manager at adesso

►  adesso is a leading IT consultancy in the German speaking region

►  Speaker

►  Author

►  Blog: http://ewolff.com

►  Twitter: @ewolff

►  http://www.slideshare.net/ewolff

►  [email protected]

Eberhard Wolff: Development and Operations - Two Worlds Collide

Development and Operations

Development

►  Projects

►  Software development

►  Requirements

►  …

Operations

►  Keeps system functional in production

►  Install / upgrade hardware / software

►  Service Level Agreements

►  …

07.09.11

Ubiquitous concept - separated in almost any organization Sometimes separated up to the CIO level

Software


Why Separated? ►  Adam Smith

►  Division of labor

►  Standardization

►  Specialization

►  Industrialization

►  Productivity gains

►  Really? Considering the typical IT zoo…

07.09.11 4 Eberhard Wolff: Development and Operations - Two Worlds Collide

Development and Operations

Development ►  Concerned with implementing features

and fixing bugs

►  Care about the evolution of systems

►  Prefer technologies with high productivity

►  Care the most about programming languages frameworks etc.

►  Area of expertise: Code, Languages etc

Operations ►  Concerned with availability &

performance

►  Care about stability (called at 3 am)

►  Prefer stable & established technologies

►  Care the most about operating system database servers applications servers etc.

►  Area of expertise: OS, monitoring etc

07.09.11 Eberhard Wolff: Development and Operations - Two Worlds Collide

Story ►  Large Java project, not too long ago

►  Decision: Use JDK 1.4 >  Experience - foundation for another project >  Was already outdated by that time >  But: No support any more

►  Called in to solve a problem

►  Result: A performance issue in the JDK

►  JDK 1.5 >  Many improvements for developers >  Specific problem is fixed >  So: Better for operations, too!

07.09.11 6

Separation leads to wrong decision


Another Story ►  Development decided against using Enterprise Java Beans (EJBs)

►  Operations: “We will buy an application server with EJB support.”

►  …just in case

►  More complex, more costly

►  Not their turf: Frameworks decided in development

►  Same organization: Operations created several frameworks >  To make applications easier to operate >  Because the framework is concerned with specific infrastructure

07.09.11 7

Separation not enforced


Performance: Different Perspectives

07.09.11 8

Database

Operating System Hardware

JVM Code

Development

Operations

Classes Garbage Collection JVM options

File I/O Memory OS locks CPU load

Database Know How?

Java Know How?

Java Database Driver No holistic view on performance and

architecture Eberhard Wolff: Development and Operations - Two Worlds Collide

Separation? ►  Operations usually automates

>  Develops shell scripts etc >  Automated software deployment

►  Operations = developers for infrastructure code >  Deployment scripts >  Monitoring >  …

►  Will become more due to Virtualization and Cloud >  Less hardware >  More need for automation >  Throw-away systems

►  Database administrators >  Care about availability of databases >  And support development


Lean Production ►  Eliminate waste!

►  E.g. transport aka hand over

►  Hand over from development to operations is waste

►  Documentations needs to be created

►  Issues might not surface until operations takes over

►  Lean Production is influential to Agile Software Development


Completely Different Approach ►  Amazon ca. 2006 according to Werner Vogels (CTO)

►  Each teams builds a services

►  A service has a business purpose

►  Each service has its own database and technology stack

►  Virtual machines are the only common ground

►  Teams provide operations and development

►  No boundaries between ops and dev

►  Freedom technology-wise


Benefits Of Approach ►  Can use the best technology for the job

►  Trade off between operations and development inside the team

►  Same ideas as Service Oriented Architecture

►  i.e. decompose IT by services

►  More flexibility

►  Compose services to new services

►  IT aligned to business

►  Easier for business to understand IT structure

►  Easier to create domain knowledge in development

►  No finger pointing between operations / development if service is not available


Other Example: Flick Photo Sharing


Other Example: Etsy


DevOps ►  John Allspaw at Flickr and

Etsy pioneered DevOps

►  DevOps = Development + Operations

►  It has a name!

►  Non trivial examples

►  Common ownership of features and performance

►  Cultural shift

►  Less finger pointing


DevOps Collaborations ►  Ops can find problems in network, OS, I/O

►  Development can create to show exceptions and parameters in the application

►  Development knows application level metrics (registrations, revenue etc)

►  Ops can monitor, plot and create triggers on the data

►  A web site with disabled functions is better than no web site at all

►  Development can create configurations to disable parts of the application for Ops

►  A holistic view on applications


DevOps And Business Agility ►  Fully automated build and deployment

►  Deployment can be triggered by several people, not just higher level management

►  Etsy and Flickr do several deployments per day!

►  Non-trivial system

►  While often a deployment is a project by itself

►  Enabler: Virtualization >  Easy to create a new environment >  Easy to fall back to older environment

►  Enabler: Modern software distribution >  Fully automated

►  Cloud has similar approaches

►  Much easier to get new versions out

►  More business agility


From Continuous Integration to Continuous Deployment ►  Integrating the work of multiple teams can be painful

►  If it hurts do it more often

►  Continuous Integration >  A server constantly takes code from the version control >  …for all projects >  Compile >  Do the automated tests

►  Result: Integration not a problem

►  Deploying software can be painful

►  Continuous Delivery: Deploy as often as you can

►  Automated


From Agility To DevOps ►  Agility

>  Create runnable software during each iteration

►  “Works on my machine”

►  “Could be put into production – but isn’t.”

►  DevOps >  Software is deployed automatically >  Software is actually put into production

07.09.11 19

Requirements Engineering Implementation Test Deployment

Agility DevOps


Conclusion: Create Running Software ►  Separation between Operations and Development is

ubiquitous

►  Idea: Industrialization, standardization …

►  Can lead to suboptimal compromises

►  Creates Hand Over / Waste

►  Different approach: Join Operations and Development “DevOps”

►  Amazon, Flickr and Etsy pioneered the approach

►  Better collaboration

►  More business agility

►  Logical next step after Agile Processes

►  Enterprise ITs start to adopt

►  Open: Impact on cost and processes


Keynote ECSE 2011: Development and Operations - Two Worlds Collide

Technology

eberhard wolff

web site

finger pointing

holistic view

worlds collide

business agility

create