Patterns & Antipatterns in Docker Image Lifecycle

PATTERNS AND ANTIPATTERNSIN DOCKER IMAGE LIFECYCLE

whoami• Yoav Landman• Co-founder & CTO at JFrog• @_yoav_

JFrog Xray

JFrog + Docker

Poll Time!DOCKER

DOCKER EVERYWHERE

Poll Time!Can do the tutorialPoCing, playing etc.Production, baby!

Y U NO

DOCKER PRODUCTION?

I LIKE YOU

BUT I DON’T TRUST YOU

CONTINUOUSIMAGE

INTEGRITY

Who’s using Docker and nothing else?

DO WE HAVE AN EXISTING PATTERN?

CAN WE ADAPT IT?

CI/CD PIPELINES?

DOING THESE FOR YEARS.

The Promotion Pyramid

Development builds

Dev Integration tests

Integr. tests

StagingPre-Prod

Prod

Frequency of builds

Build

/Dep

loy

time

Num

ber of binaries

Pipeline: Quality Gates and Visibility

Source: Agile ALM, Michael Hüttermann, Manning Publications Co.

http://www.flickr.com/photos/jemsweb/4363548805



$docker build

DOCKER BUILD

ALL THE THINGS!

Simple!

FAST AND CHEAP BUILDS

NOT ALWAYS THE WAY TO GO

This is why

Let’s fix it!

Let’s fix it (again)!

Let’s fix it (again)!

Docker content mistrust

I DON’T ALWAYS BUILD PROMOTION PIPELINES

BUT WHEN I DO, IT’S WITH IMMUTABLE AND STABLE BINARIES

What’s up with the gates?!

What’s up with the gates?!

What’s up with the gates?! - QA shouldn’t test dev images

What’s up with the gates?! - QA shouldn’t test dev images - non-QA’ed images shouldn't be staged

What’s up with the gates?! - QA shouldn’t test dev images - non-QA’ed images shouldn't be staged - non-QA’ed, non-staged or dev images shouldn’t end up in production!

Not so fast…

Trumped-up limitations

The Anatomy of Docker Tag

Wait a second, how can I have more than one

repository per host now?!

How can we support this?

https://host:8081/artifactory/docker-dev/busybox

https://host:8081/artifactory/docker-staging/busybox

https://host:8081/artifactory/docker-qa/busybox

https://host:8081/artifactory/docker-prod/busybox

“One registry per host isought to be enough for

anybody.”

https://www.reddit.com/r/theydidthemath/comments/1x37rx/request_how_much_alcohol_is_needed_to_get_a_whale

Panic!

Virtual hosts/ports to the rescue

https://host:8081/artifactory/docker-dev/busybox

Context name

Virtual repository nameTag name

https://host:port/v2/busybox

server { listen 5001;

server_name 192.168.99.100; if ($http_x_forwarded_proto = '') { set $http_x_forwarded_proto $scheme; } rewrite ^/(v1|v2)/(.*) /artifactory/api/docker/docker-dev/$1/$2; … }}

But then you realize…Wait a second, now I need to pull, retag and push for

every step?!

WHAT DO WE WANT?

MINIMIZE THE NUMBER OF REPOS DOCKER INTERACT

WITH

HOW CAN WE DO IT?

VIRTUAL REPOSITORIES!

Virtual RepositoriesLocal Repository

Virtual RepositoryRemote Repository

Docker Hub

What We’ll DO? - Minimize number of repositories docker interacts with

What We’ll DO? - Minimize number of repositories docker interacts with - deploy to virtual (backed by dev repository)

What We’ll DO? - Minimize number of repositories docker interacts with - deploy to virtual (backed by dev repository) - promote within artifactory

What We’ll DO? - Minimize number of repositories docker interacts with - deploy to virtual (backed by dev repository) - promote within artifactory- Resolve from virtual (production-ready images)

Virtual Repositories FTWdocker-dev-local

docker-prod-localdocker-virtual

Dev

docker-hub-remote

Docker Hub

resolve

resolve

resolve

resolve

resolve



Dev

docker-hub-remote

Docker Hub

resolve

resolve

resolve

resolve

resolve

deploy

deploy



Dev

docker-hub-remote

Docker Hub

resolve

resolve

resolve

resolve

resolve

deploy

deploy

promote



Dev

docker-hub-remote

Docker Hub

resolve

resolve

resolve

resolve

resolve

deploy

deploy

promote

Prod

resolve

resolve

Anatomy of a container

Our LayersApplication: • .war file

Framework: • JDK8 + Tomcat

Base: • ubuntu-trusty

Framework build- Verified base image- Add system dependencies

from artifactory- JDK- Tomcat

Framework build

Own it!

Minimal Framework build DockerfileFROM ubuntu:14.04MAINTAINER [email protected]

Application build- Base==Framework image- Run app build- Add app files to base- Done!

Application build Dockerfile

FROM yourorg-docker.jfrog.io/myorg/framework:latestMAINTAINER [email protected]

ADD https://yourorg.jfrog.io/java-release-local/…/app-[RELEASE].war /var/lib/tomcat7/webapps/app.war

?




? ?




? ?

???

Framework Pipeline

Application Pipeline

Framework Pipeline


Framework Pipeline


Install it! Installing with Docker compose Great for micro services located

on same host

End users have Docker installed

Don’t want to run/install docker-compose Or any other installer

Docker compose and docker client can introduce incompatibilities

The Solution- Create An “Installer” Image- Provide variables for:

- Where to pull from- Docker Daemon to use

- Have it run docker compose- Install onto Client’s Docker!

Installer Image

Installer’s run.sh Just run docker-compose with

the right command start, stop, up, down, restart…

Check calling script version compatibility!

app.sh (user script)

Set the repo to pull from

Set up script andapplication versions

Determine the docker daemon for docker-compose to use

Run the installer image

The installer patternDocker pulls and runs the

installer image

app.sh

Executesrun.sh

installer image

Runs docker-compose

run.sh

Pulls down and installs

micro-services

Docker daemon Docker registry

docker-compose

HIGH QUALITY(software and information) SPEED LOW COST

(automation)

Fast releases > Modular > Automation

Conclusions: Release Fast or Die!

Thank you!

Patterns & Antipatterns in Docker Image Lifecycle

Software