Transforming Enterprise Release Management in Elastic Beanstalk using Jenkins, Docker and Jenkins Job Builder

Transforming enterprise release management in Elastic Beanstalk using

Jenkins, Docker and Jenkins Job Builder

Yves Hwang

Internet & Mobility Architect Statoil Fuel & Retail

@yveshwang http://macyves.wordpress.com

I live in Oslo, Norway

I come from Perth, WA

I am passionate about devops and automation

Statoil Fuel & Retail (SFR)• Large I.T. application portfolio

• Start of digital and I.T. transformation

• “Bank is I.T., I.T. is bank” - ING, H. Kolk (Chief Architect)

• Attract and retain entrepreneurial engineers

• Microservice, small technical teams, release frequently

• Start of our I.T. journey, product development mindset, lifecycle changes

•

Statoil Fuel & Retail (SFR)• Technology strategy focused on

• Autonomy

• Decentralisation

• Parallelism

• Isolation

• Idempotent

• Statelessness

• Fit for the organisation

Statoil Fuel & Retail (SFR)• Innovation fuels growth and improves time to market

• Autonomous team drives and improves efficiency; lowers cost

• Competency lowers cost and thrives for operational excellence

what is an enterprise i.t.

You know you are an enterprise I.T. when…

• Lots of teams, people and technology are established, supporting lots of diverse applications and platforms

• Teams are working in different ways – some agile and product aligned, some waterfall and technology aligned

• Lots of change is flowing through the system but also a lot of business, meaning that uptime and stability are essential

• Communicate via tickets and everything is in silos

You know you are an enterprise I.T. when…

• Lots of meetings, not much coding

• Strong focus on Information Technology Infrastructure Library (ITIL)

• CMMI level 5 becomes a selection criteria

• Manual process FTW!

… and it gets tougher• Our monolithic J2EE application servers take an hour to start, making Continuous

Delivery difficult, how do I build a business case?

• Heavily planned and released in a waterfall style. How do I decouple our systems from one another?

• We’ve just spent 5 years improving our ITIL maturity and feel that we are now in a good place from an operational perspective. Where does DevOps leave that and where do we go from here?

• Should I invest in release automation tooling, infrastructure as code or cloud first?

There is a light at the end of the tunnel :)

• Turn IT from a cost centre to a profit centre that delivers competitive advantage

• Respond to an increased volume of change in support of their digital initiatives and competitive pressures

• Becoming more customer and product centric with reduced cycle time from customer feedback to implementation

• Moving away from big, risky, monolithic software releases towards a continuous stream of change, enabling reduced cycle time and lightweight innovation

devops transformation

Some anchoring principles• Automate all the things to cater to large volumes of releases and

operations. Orchestration of common platforms with toolchains

• Test automation, invest in quality and take pride in craftsmanship

• Manage environment consistency, infrastructure as code

• Optimise for development experience and drive innovation through freedom, trust and responsibility

These are the anchors• Jenkins! :)

• Git! :)

• Jenkins job builder!

• Docker!

• Elastic beanstalk!

Jenkins is awesome!

Elastic Beanstalk• AWS Linux instance running nginx and docker

• Deploy Docker images or Dockerfile

• Elastic load balance

• Highly available platform

• Rolling updates, 0 downtime

• Hackable platform

Docker• Infra as code, tested throughout development

• Concept of immutable service

• Available, fast

• No more “this works on my machine!”

• Full stack test driven development

Jenkins Job Builder• OpenStack

• DSL for Jenkins jobs, calls out to Jenkins API

• YAML based

• Hooks in perfectly in Git post-hook

Jenkins Job Builder• Templating the build flow and being able to test it!

• YAML makes it readable, variable supported, common routines are re-used

• Scalable

• Exhaustive list of supported plugins in YAML

So what does this YAML look like

• job, job-template, job-group

• Modules

• publishers, scm, properties

• Very well documented

• http://docs.openstack.org/infra/jenkins-job-builder/

Few commands to work with• $ jenkins-jobs --conf jenkins-job-builder-localhost.ini test example/vagent2.yaml -o output/

• $ jenkins-jobs --conf jenkins-job-builder-localhost.ini test example/vagent2.yaml

• $ jenkins-jobs --conf jenkins-job-builder-localhost.ini delete-all

• # deletes all jobs

• $ jenkins-jobs --conf jenkins-job-builder-localhost.ini --flush-cache update example/vagent2.yaml

• # sync jobs

Short demo!

Setup Jenkins Job Builder with Jenkins: self updating Jenkins

• Use Jenkins to perform self update based on Git

• Create a first tiny build pipeline for a micro service API

• https://github.com/yveshwang/jenkins-docker-2step

• https://github.com/yveshwang/jenkins-config-example

• https://github.com/yveshwang/api-example

Putting it all together

Releasing to different Elastic Bean Stalk Environment

• test/dev —> prep —> shop

• 3 different git branches, containing different CloudFormation configuration for EB

• Same build pipeline template :)

• Release management is completely git based, cherry-picking and merging of branches

Jenkins and automation anchors our enterprise

transformation

https://macyves.wordpress.com/category/release-management/

Thanks!