DevOps Means Business
Presented by
Nicole Forsgren Velasquez | Jez Humble | Nigel Kersten | Gene Kim
Findings From Puppet Labs State Of DevOps 2014
Velocity Conference, Santa Clara
June 26, 2014
Puppet Labs Webinar 2014 State of DevOps
Puppet Labs Webinar 2014 State of DevOps
Agenda
● Demographics● Key Findings● Survey Methodology and Analysis● IT Performance● Organizational Culture● Job Satisfaction● Big Surprises● Future Plans● Q&A
Gene Kim
Multiple award-winning CTO, researcher and co-author of The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win. First time Google Slides user.
Presenters
Nicole Forsgren Velasquez
Professor at the Huntsman School of Business at Utah State University. IT impacts, knowledge management, and analytics consultant and researcher. Stats ninja.
Jez Humble
Principal at Thoughtworks, Lecturer at UC Berkeley, co-author of Continuous Delivery, and accomplished procrastinator.
Nigel Kersten
CIO at Puppet Labs and responsible for IT/Operations there. Almost fully recovered sysadmin.
Puppet Labs Webinar 2014 State of DevOps
Puppet Labs Webinar 2014 State of DevOps
9,200+ Respondents from 110 countries, across all industries
Demographics
@botchagalupe#devops survey early observations #4 Retail, Banking,
Finance and Healthcare respondents LT 15% total (typically the risk averse
sectors)
27% of
responses from companies with 500 to 9,999 employees
Size of Organization
Puppet Labs Webinar 2014 State of DevOps
Puppet Labs Webinar 2014 State of DevOps
Size of Infrastructure
51% of
respondents said their infrastructure included <500 Servers
@botchagalupe#devops survey early observations #1
Only 40% of respondents have more than 500 servers
Puppet Labs Webinar 2014 State of DevOps
Departments
16% identified asDevOps Department
2012
2013
2014
DevOps Grows Up
What is devops?
DevOps adoption is accelerating.
Holy cow! DevOps works!
Puppet Labs Webinar 2014 State of DevOps
Why I Care: IT Performance Matters
Puppet Labs Webinar 2014 State of DevOps
Cross-population Studies
Tradeshows
Industry Press
Blogosphere
Vendors
Consultants
Industry Analyst
Anecdote
Peer Recommendation
Prior Experience
ROI Story Best Practice
Benchmark
Population Study
Academic Study
Intuitive Analytical
We Are Here
High-performing IT Orgs are More Agile
Puppet Labs Webinar 2014 State of DevOps
30x 8,000xmore frequent deployments
faster lead times than their peers
High-performing IT Orgs are More Reliable
Puppet Labs Webinar 2014 State of DevOps
2x 12xthe change success rate
faster mean time to recover (MTTR)
Firms with high-performing IT orgs are winning
more likely to exceed profitability, market share & productivity goals
Puppet Labs Webinar 2014 State of DevOps
2xhigher market capitalization growth over 3 years*
50%
How We Collected Responses
Puppet Labs Webinar 2014 State of DevOps
Snowball Sampling
Questions We Asked
Puppet Labs Webinar 2014 State of DevOps
Likert-type questions give us a more nuanced measurement.
Latent Constructs
Puppet Labs Webinar 2014 State of DevOps
Some things can’t be measured directly, like happiness.
“Happiness Statistics” by Andrew Tarvin is licensed under CC BY 2.0
Quantifying IT Performance
Puppet Labs Webinar 2014 State of DevOps
Deploy
30X more frequently with
50% fewer failures
● Deploy Frequency
● Mean Time to Recover (MTTR)
● Lead Time for Changes
Categorizing Companies: High, Medium, Low IT Performance
Puppet Labs Webinar 2014 State of DevOps
Deploy Frequency
Mean Time to Recover (MTTR)
Lead Time for Changes
MTTR
Lead Time
Deploy Frequency
High-performing IT orgs
Medium-performing IT orgs
Low-performing IT orgs
Puppet Labs Webinar 2014 State of DevOps
Practices Correlated with IT Performance Metrics
Throughput Metrics Stability Metrics
Deployment Frequency● Continuous Delivery● Version Control
Mean Time to Recover (MTTR)● Version Control● Monitoring System and
Application Health
Lead Time for Changes● Version Control● Automated Testing
Change Fail Rate ● Not strongly correlated with
specific practices.● Significant differences between
groups with high, medium and low change fail rates. High performing IT orgs have 50% lower change fail rates than their peers.(mention Roy Rapoport!)
Puppet Labs Webinar 2014 State of DevOps
Practices Correlated with IT Performance Metrics
Throughput Metrics Stability Metrics
Deployment Frequency● Continuous Delivery● Version Control
Mean Time to Recover (MTTR)● Version Control● Monitoring System and
Application Health
Lead Time for Changes● Version Control● Automated Testing
Change Fail Rate ● Not strongly correlated with
specific practices.● Significant differences between
groups with high, medium and low change fail rates. High performing IT orgs have 50% lower change fail rates than their peers.(mention Roy Rapoport!)
Puppet Labs Webinar 2014 State of DevOps
Practices Correlated with IT Performance Metrics
Throughput Metrics Stability Metrics
Deployment Frequency● Continuous Delivery● Version Control
Mean Time to Recover (MTTR)● Version Control● Monitoring System and
Application Health
Lead Time for Changes● Version Control● Automated Testing
Change Fail Rate ● Not strongly correlated with
specific practices.● Significant differences between
groups with high, medium and low change fail rates. High performing IT orgs have 50% lower change fail rates than their peers.(mention Roy Rapoport!)
Puppet Labs Webinar 2014 State of DevOps
Practices Correlated with IT Performance Metrics
Throughput Metrics Stability Metrics
Deployment Frequency● Continuous Delivery● Version Control
Mean Time to Recover (MTTR)● Version Control● Monitoring System and
Application Health
Lead Time for Changes● Version Control● Automated Testing
Change Fail Rate ● Not strongly correlated with
specific practices.● Significant differences between
groups with high, medium and low change fail rates. High performing IT orgs have 50% lower change fail rates than their peers.(mention Roy Rapoport!)
Puppet Labs Webinar 2014 State of DevOps
Top Predictors of IT Performance• Peer-review of production changes (vs.
external change approval)• Version control of all production artifacts• Proactive monitoring of the production
environment• High trust culture• Win-win relationship between Dev, Ops and
Infosec• High job satisfaction
Puppet Labs Webinar 2014 State of DevOps
Feedback Loops
Source: Elisabeth Hendrickson: Care And Feeding Of Feedback Loops
Puppet Labs Webinar 2014 State of DevOps
Organizational Culture
15% 52% 33%
Puppet Labs Webinar 2014 State of DevOps
Organizational Culture Resources● http://www.thisamericanlife.org/radio-archives/episode/403/nummi
● http://sloanreview.mit.edu/article/how-to-change-a-culture-lessons-from-nummi/
● Read Toyota Kata by Mike Rother
Puppet Labs Webinar 2014 State of DevOps
Job Satisfaction
Job satisfaction is the # 1 predictor of organizational performance!
Top Predictors of Job Satisfaction
★ High-trust organizational culture
★ Climate of learning★ Win-win relationships
between ops, dev and infosec teams
★ Proactive monitoring and autoscaling
★ Use of version control for all production artifacts
★ Automated testing
Gene• Version control of the environment is more
important than version control of the code!• There’s never been a better time to know
statistics
Jez• Whether or not you have an integration or
stabilization phase has zero impact on IT perf• Peer review is more effective than CAB
Surprises
Puppet Labs Webinar 2014 State of DevOps
Can Large Orgs Be High Performers?
Puppet Labs Webinar 2014 State of DevOps
Yes.
But orgs with 10,000+ employees 40% less likely to be high performing vs.
500 employee orgs…
Can Large Orgs AdoptThese Practices?
Puppet Labs Webinar 2014 State of DevOps
Yes!(Automated testing,
Continuous integration, proactive monitoring…)
The only practice not being adopted is Peer Review vs.
Change Approval!
Nicole• DevOps practices and IT performance impact
organizational performance• Change fail rate wasn’t part of IT performance
Nigel• Forming new DevOps teams and giving people
DevOps titles is successful in practice
Surprises
Puppet Labs Webinar 2014 State of DevOps
• Collect more stock tickers!
• 2014-15 research goals
• What do you want to know?• Put your ideas and problems you want to
explore in this Google+ Community:http://bit.ly/FutureStateOfDevOps
Puppet Labs Webinar 2014 State of DevOps
Future Plans
O&Abit.ly/puppetquestions
Sept 3-4 | FlowConFlowCon brings together technologists and industry leaders passionate about innovation through continuous delivery, continuous design, and lean product development. Register at http://flowcon.org/
Sept 23-24 | PuppetConf Join us for the IT automation event of the year. Register now to save $249. http://puppetconf.com
Oct 21-23 | DevOps Enterprise SummitA conference for horses, by horses, featuring Macy’s Disney, GE Capital, and more. Register at http://devopsenterprise.io/
Nov 9-14 | LISA ‘14Systems engineering and ops professionals share real-world knowledge about designing, building & maintaining critical systems. https://www.usenix.org/conference/lisa14
Puppet Labs Webinar 2014 State of DevOps
Upcoming Events
Resources• Download the 2014 State of DevOps Report:
http://puppetlabs.com/2014-devops-report
• Pre-order Jez’s forthcoming book Lean Enterprise: Adopting Continuous Delivery, DevOps, and Lean Startup at Scale: http://amzn.com/1449368425
•Sign up for Gene’s newsletter to receive a sample chapter of his forthcoming book DevOps Cookbook: http://itrevolution.com/blog