Accenture Architecture Services DevOps-as-a-Service: Towards Automating the Automation DevOps Enterprise Summit November 8, 2016 Keith Pleas DevOps Architecture Sr. Mgr. [email protected]
Accenture Architecture Services
DevOps-as-a-Service:Towards Automating the AutomationDevOps Enterprise SummitNovember 8, 2016
Keith PleasDevOps Architecture Sr. [email protected]
2
Accenture’s Global DevOps PracticeWe support a large variety of industry standard tools – both open source and vendor/proprietary tools.
250 Clients on Accenture DevOps Platform
+ 100s more Clients with DevOps services
4400 DevOps Trained
1700+ DevOps experts
350 DevOps Certified
250+ DevOps Factory FTE
7 countries
15+ Years heritage of high-
performance CI/CD & DevOps
2000+ Members of DevOps community on Yammer sharing strategy, knowledge & collaboration
759 People deployed to AT&A
Operations Domain alone
10%+Cost reduction
across IT delivery
New features in hours not days
DevOps Leader• Accenture developed ADOP DevOps platform (now open-sourced)• DevOps Managed Services(DCSC)• Development Lifecycle Management• Our in-house developed Cloud Platform• DevOps maturity assessment• IP/Frameworks: Methods and Tools
Services• Assessment and Strategy• Application Architecture • DevOps Implementation
– Continuous Integration– Continuous Delivery– Continuous Testing
• Organizational Change• IT operating model transformation• Managed Service
Components• Build & deployment automation• Software configuration management• Test automation• Environment management & monitoring• Cloud environment provisioning• Development & deployment architecture
Key principles• Test early and often• Improve continuously• Automate everything• Cohesive teams• Strong source control• Deliver small increments• Experiment frequently without regret
Enabling DevOps Professionals
Global Expert Network
Toronto
Chicago
Wilmington
Sao Paulo
London
Málaga
Mumbai
Bangalore
Mauritius
Dalian
Manila
Prague
Atlanta
Note: Other geographic and industry focused delivery centres exist
Courses/TrainingsDevOps Introduction
• Introduction to various DevOps concepts and how to apply them
• Illustrative examples/case studies
DevOps Academy • Intensive 2 day classroom course• Continuous Integration and Delivery• Infrastructure as Code
Multiple technology specific trainings
Copyright © 2016 Accenture All rights reserved.
DevOps and Accenture
2009 2015201420132012201120102007 200820062005
Agile ConferencePatrick Dubois & Andrew Clay ShaferAgile Infrastructure
Qcon: Daniel SchauenbergEtsy 50 Deploys per day
Velocity ConferenceJohn Alspaw & Paul Hammond: Flikr Deploy10 time per day
DevOps Days Belgium #DevOps
Accenture DCSC Shared Service Created
2016
Copyright © 2016 Accenture All rights reserved. 3
Copyright © 2016 Accenture All rights reserved. 4
The Joel Test (August 9, 2000)A friend from my distant past
• Do you use source control?• Can you make a build in one step?• Do you make daily builds?• Do you have a bug database?• Do you fix bugs before writing new code?• Do you have an up-to-date schedule?• Do you have a spec?• Do programmers have quiet working conditions?• Do you use the best tools money can buy?• Do you have testers?• Do new candidates write code during their interview?• Do you do hallway usability testing?
www.joelonsoftware.com/articles/fog0000000043.html
What is DevOps?Is it culture?
Copyright © 2016 Accenture All rights reserved.
Copyright © 2016 Accenture All rights reserved. 6
Our Challenges: Two Lenses• Technology– Automation – Software-defined Infrastructure– Automate the Automation – Software-defined DevOps– Templates, tools and stacks– Enterprise-specific concerns: licensing, security, support– Technical debt
• People – Do what you do best - leverage others– Open Source culture– New paradigm for engaging Dev and Ops
Copyright © 2016 Accenture All rights reserved.
Focus on Technology
7
Copyright © 2016 Accenture All rights reserved. 8
Roger’s Curve with Moore’s “Chasm”How innovative are you?
Copyright © 2016 Accenture All rights reserved. 9
Modern IT methodologies emphasize collaborationDevOps in context
IDEA PLAN DESIGN BUILD DEPLOY TEST RELEASE OPERATE
DevOps
Continuous Delivery
Continuous Integration
Execution Architecture Development Architecture Testing Strategy Ops Arch
Design Thinking
Lean Manufacturing
Agile Development
Copyright © 2016 Accenture All rights reserved. 10
What about those “non-functional” requirements?Auditability, consistency, maintainability, modifiability, operability, recoverability, repeatability, resilience, reusability, robustness, scalability, security, supportability, scalability, testability, usability
• How are you managing your own continuous delivery artifacts?• Are all of your build pipeline settings stored in source code control?• Can you rebuild your pipeline via automation?• Can you easily scale your pipeline?• Can you quickly onboard resources and make them productive?• Could you port your pipeline to a new build environment?• How are you managing your DevOps technical debt?
Copyright © 2016 Accenture All rights reserved.
Our first “enabler”
11
12
DevOps Maturity Model – Framework
Copyright © 2016 Accenture All rights reserved.
DELIVERY ORGANISATION AND METHODOLOGY
• Release Frequency• Delivery Organisation• Delivery Process & Methodology• Metrics & Governance
LEAN DELIVERY GOVERNANCE & PROCESSES
• Governance Approach• Continuous Improvement• Product Management• Delivery & Change Control
AUTOMATED RELEASE OF SOFTWARE • Source Code Version Control• Deployment Process• Deployment Artefact Management
CONTINUOUS INTEGRATION • Traceability• Continuous Integration• Code Quality & Unit Test
CONTINUOUS DELIVERY (INCLUDING AUTOMATED QA)
• Test Strategy• Test Automation• Test Data Management• Deployment Pipelines
AUTOMATED OPERATIONS • Monitoring• Organisational Considerations• Resilience
SOFTWARE DEFINED INFRASTRUCTURE & CLOUD
• Environments• Infrastructure Suitability• Responsiveness
PLATFORM/APPLICATION ARCHITECTURE • Architecture• Impact of Component Failure• Business Enablement
Architecture
Methodology
Lean Governance & Process
Continuous Delivery Pipelines
SW Defined Infrastructure
Automated Release of SW
Automated Operations
Continuous Integration
13
DevOps Maturity Model - Overview
Copyright © 2016 Accenture All rights reserved.
REPEATABLE CONSISTENT OPTIMISEDAD-HOC LEADING2 3 41 5
DELIVERY ORGANISATION AND METHODOLOGY
Release scope poorly defined; subject to catastrophic and ad hoc change requests
Releases duration exceeds business need; releases face disruptive change
Release cadences well defined but exceeds business need; Requirements are stable;
Release on demand, multi-speed releases; time-box meets to business need (eg: monthly)
Small changes pushed through the pipeline ; Continuous deployment enables innovation
LEAN DELIVERY GOVERNANCE & PROCESSES
No defined or consistent applied delivery approach
Delivery approach is well defined, but largely manually governed
Industrialised project delivery; governance integrated into delivery and product management established
Blame free culture embedded in governance that uses real-time data rather than manual inputs
Continuous Delivery / Deployment supporting agile business change with automated governance
AUTOMATED RELEASE OF SOFTWARE
Mostly manual deployments Some CIs automated, environment tailoring required, no enterprise tools
Fully automated Single-touch deployments into environments.
Functioning environments can be build from nothing programmatically.
Zero-touch zero- downtime deployments
CONTINUOUS INTEGRATION
No consistent use of version control – builds cannot be traced back to source code
Source code consistently managed in VCS; releases traceable to source
Developers integrate changes by checking into trunk on regular basis (daily)
Build is typically green – if build breaks developers do not make other changes until resolved
Build is typically green – if build breaks the CI tooling automatically reverses the failed change
CONTINUOUS DELIVERY (INCL AUTOMATED QA)
Fully manual test scripts Testers run a harness / suite Test harness / suite run automatically for some envs
Automated test suites enforce a quality gate
Tests run as functional monitoring
AUTOMATED OPERATIONS
No monitoring tools Tools in place, but not configured beyond basic OS checks
Functional monitoring in place and DR is available
Environment and application health monitored and proactively managed
Service level monitoring (perf, usage) integrated with infrastructure and self-healing available
SW DEFINED INFRASTRUCTURE & CLOUD
Environments are managed ad-hoc without consistent blueprint
Environments follow standards but are manually created
Environments can be created automatically and follow common blueprint
SW defined infrastructure that closely resembles Production throughout SDLC
Utility based computing that leverages auto-scaling for several SDLC aspects
PLATFORM/APPLICATION ARCHITECTURE
Big monolithic application architecture
Architecture based on platforms, business requires IT for every change
Resilient Architecture minimising single point of failures
Modular and scalable architecture, that allows for some business self service configuration
Microservice based architecture supported by end-to-end ownership in teams
14
DevOps Maturity Model - Overview
Copyright © 2016 Accenture All rights reserved.
REPEATABLE CONSISTENT OPTIMISEDAD-HOC LEADING2 3 41 5
Copyright © 2016 Accenture All rights reserved.
Our second “enabler”
15
Copyright © 2016 Accenture All rights reserved. 16
Structure, Standardize, AutomateThe complete pipeline to operations with ADOP
IDE• Eclipse• X-code• …
Git / Gerrit• Source code
repository• Code branch
management• Code review
Jenkins• Continuous
Integration• Multi-
platform builds
Code Analysis• Code quality• Coding
standards rules
• Compliance
Deploy• Dev app store• Release
repository• Environments
Test• Component• Behavioural• Functional• Security• Performance
Development Source Control
Code Review Build Code
Quality Deploy Testing A/B Test in Production
Please visit github.com/Accenture for more information.
Operations• Search &
Analytics• Log Processing
& Analysis• Visualization
17Copyright © 2016 Accenture All rights reserved.
Fully automated solution for implementing ADOP
https://hub.docker.com/
Availability Zone
AWS region EU (Ireland)AWS
AWS cloud
ChefServer
Public security group
Public subnet
Internet
AWSAWS Cloud
AWS CloudFormation
CF template json
VPC
Single ADOP VM
ADOP Provisioning
Gerrit ELK
Nexus
Sonar
Nginx Jenkins
Platform Extensions
Load Cartridge TomcatEnvironment
CloudInit
18
Fully distributed, highly available, auto-scaling, and self-healingADOP platform
Copyright © 2016 Accenture All rights reserved.
19
ADOP CartridgesAdditional enablers
Copyright © 2016 Accenture All rights reserved.
Dedicated versus Managed ADOPDedicated Instance
Single tenancy
Supported by project or by DCSC
Installed on-cloud or on client infrastructure
Supports cartridge/blueprint configuration for various technologies
Managed Service
Multitenancy
Provided as a managed service by DCSC
Embeds ADLM and Innersource
Costs recovered by service-charge
Hosted primarily in AWS
Configurable to support project requirements
Clients Internal Projects
Users
Jira/Confluence 148 7 10,800
Innersource 69 2,820 10,600
Usage Jan 2016
20Copyright © 2016 Accenture All rights reserved.
Dedicated versus Managed ADOP Tools and ServicesDedicated Instance
On Premise
(Roadmap)
(Beta)
Selenium
AOWP
Digital/01
(Beta)
(Roadmap)
+Others
AVS
Hybrid
Managed Service
(Roadmap) (Beta)
Compatibility(Roadmap)
21Copyright © 2016 Accenture All rights reserved.
Copyright © 2016 Accenture All rights reserved. 22
DCSC’s model• Service Catalog of pre-configured PMO tools
and reporting, enabling rapid mobilization and low cost run
• A number of processes benefit from a Business Process Management tool (Activiti), enabling an automated chasing and reporting
• Services under continuous improvement, rolling out the latest technology to projects
How to do shared services?
Founding principle for myPMO Services: Structure....Standardize…. Automate
To reduce local team effort
Copyright © 2016 Accenture All rights reserved. 23
• The spectrums: standardization, governance, processes• Consolidating and simplifying the tools portfolio & licensing• Maturity assessments and coaching clients• Experiment with ways to obtain better feedback from clients e.g. periodic surveys, feedback channels, user groups, social media
• Geo-location• Metrics• Security, including VPN tunnels across trust boundaries– Detection (e.g. Security Monkey) rather than imposing restriction
What are the tradeoffs with shared-services?
Copyright © 2016 Accenture All rights reserved. 24
• Proof of concepts and MVPs• Onboarding and training• Starting point for your own DevOps stack
• Starting point for your own shared services
How might you use ADOP?
Copyright © 2016 Accenture All rights reserved.
Focus on People
25
Copyright © 2016 Accenture All rights reserved. 26
Just one example from Jenkins World…
Copyright © 2016 Accenture All rights reserved. 27
Benefits of a shared service model
• Re-use of assets between teams• Sharing of knowledge between teams• Flexible workloads e.g. person X from project A spends and afternoon helping person Y on project B when needed
• Creating strong culture that fosters shared values and creates a great and meaningful place to work
• Being able to have our own culture and identity that we can control
Focus on the People
Copyright © 2016 Accenture All rights reserved. 28
Lessons learned from a decade of DCSC
• Avoiding the HiPPO (Highest paid person’s opinion)• Specialization, cross-training, silos, and scalability• Trust versus Empowerment– johnclapham.wordpress.com/2016/04/22/inflicting-trust/
• Relationships and staying connected with a lack of face to face• Ensure internal product teams don’t end up in a silo and lose touch• Flexibility for mobility of individuals between roles and meet their needs balanced with staying on engagement through realization
Focus on the People
Copyright © 2016 Accenture All rights reserved.
Where do we go from here?
29
Copyright © 2016 Accenture All rights reserved. 30
• Ever-increasing complexity• Work harder, work smarter, or get help• Structure, standardize, automate
Evolution: Fact or Theory?
Copyright © 2016 Accenture All rights reserved. 31
Large media clientComplex DevOps tools landscape
Log Aggregation and Op. Dashboards
Programme and Project Management
ConfigurationManagement
Test Automation
RequirementsManagement
Functional Testing
Continuous Integration
Continuous Delivery
ArtefactManagement
SourceControl
Change Management
Defect Management
Continuous Integration/Build Build Orchestration
JenkinsJenkins
Risks and Issues
Continuous Integration Build
DeploymentOrchestration
Jenkins
Code Validation Functional and API Tests Load Injection
Design DocumentationTeam Dashboards MI and Reporting
Security Tests
Travis CI
AWSS3
Github
Jenkins
Slack and Lita
ZapCharles
Box
Composer
BlazemeterPostmanSelenium
Team Communications
Slack
Netsparker
Development and Prototyping
Copyright © 2016 Accenture All rights reserved. 32
Let’s look at those NFRs again…
• Versioning, packaging, dependencies• Chef and Puppet had a head start and showed us the way• Cloud Formation templates• Security settings• Certificates• ….
Where are our DevOps artifacts?
Auditability, consistency, maintainability, modifiability, operability, recoverability, repeatability, resilience, reusability, robustness, scalability, security, supportability, scalability, testability, usability
Copyright © 2016 Accenture All rights reserved. 33
In general the same way
• Repositories• DSLs (Domain-Specific-Languages)– Example: Gradle’s Groovy-based DSL for builds
• Jenkins “Pipeline as Code”- jenkins.io/doc/book/pipeline-as-code/• A new opportunity for automated testing
How do we manage these?
Copyright © 2016 Accenture All rights reserved. 34
Ah yes, software factories…did that end well?
“Why I Hate Frameworks” – Benji Smith, September 30, 2005
• A hammer• A universal hammer• A hammer factory• A hammer-factory-building factory• A general-purpose tool-building factory factory factory
Modification - the fatal flaw of MDA and software factories and…
Automating DevOps
discuss.joelonsoftware.com/?joel.3.219431.12 34
Copyright © 2016 Accenture All rights reserved.
Putting it all together
35
Copyright © 2016 Accenture All rights reserved. 36
• Technology– Do you have standards for your DevOps practices?– Do you rationalize your DevOps tools?– Are you considering your DevOps technical debt?
• People– Are you attracting or building DevOps talent?– Are you making the best use of the talent you have?
Does DevOps-as-a-Service make sense to you?
Questions to consider
Copyright © 2016 Accenture All rights reserved.
• www.Accenture.com/DevOps
• ADOP Wiki and quickstart: accenture.github.io/adop-docker-compose/docs/quickstart/• Mark Rendell DOES15 - www.youtube.com/watch?v=8WRRi6oui34• “The Joel Test” - www.joelonsoftware.com/articles/fog0000000043.html• “Why I hate frameworks” - discuss.joelonsoftware.com/?joel.3.219431.12• DevOps Maturity Model- [email protected]• John Clapham “Inflicting Trust”: johnclapham.wordpress.com/2016/04/22/inflicting-trust/• Accenture DevOps blog: www.accenture.com/us-en/blogs/blogs-accenture-devops• Mark Rendell (DCSC) blog: markosrendell.wordpress.com• Mirco Hering (Maturity Model) blog: notafactoryanymore.com
Resources
37