Best Practices for Managing Docker Versions as presented at JavaOne 2017

@jbaruch #oraclecode

Docker Version –

Best Practices


About me

»@jbaruch

»Developer Advocate @JFrog

»We might still have some t-shirts left!


Poll Time!


Poll Time!

»Heard about Docker

»Can do the tutorial

»PoCing, playing etc.

»Production, baby!





JFrog Artifactory + Docker


»There are 2 hard problems in computer science:

cache invalidation,

naming things,

and off-by-1 errors.


Naming Things

with Docker


Who

Cares

About

Versions

Anyhow

?


Why Do We Care about versions?

»Pipeline Automation/Orchestration

»Traceability/Communication

»Maintainability/Storage Management


Those are old news,

we have been using versions for years

»Docker is different !!

»Versions are Tags


Docker Manifest and Tags

ac-image:1.0 ac-image:1.0-beta ac-image:1.0-qa

Manifest

sha256:252564..

Manifest

sha256:462564..

ac-image:1.1

OS: SHA2

Framework: SHA2

Application: SHA2: 3

Layers


Tags

Manife

st


So let’s talk about tags in detail

»Mutable and Dynamic in nature

»Example: my-image:5.0

»And Latest is not latest

»Confusion !!


my-image:5.0

OS layer 1.0

Framework layer 2.0

Application layer 2.0

OS layer 1.1

Framework layer 2.1

Application layer 2.1

Yesterday Today


The case for `latest`And other mutable tags


Convenient promotion


Automation:

Docker Image Promotion Process across

silos


Promotion

ac-image:1.0-

devac-image:1.0-qa

ac-image:1.0-

release

Manifest

sha256:252564..

OS: SHA2

Framework: SHA2


L

a

y

e

r

s

T

a

g

s


Pull, retag, push for every stage? It’s

nuts!!!


We got you covered


The case for immutable

tags


Traceability!

»We always know what this image is

»Trace it back to CI

»Trace it back to source


Traceability: Classic Approaches

»Version Names based On Git Hashes

»Version Names based on Jenkins Builds

»Version Names based on the packaged software

version number

»Version names with a unique date

(the build timestamp)


So which one is

better?


Static vs. Dynamic Tags

»Static Tags:⋄Assigned at image CREATION

⋄Reflect metadata that traces the tag to its build and/or contents

»Dynamic Tags:⋄Reflect an image you should use, the current known-good version

⋄Classic example is ubuntu:trusty

⋄Remember, ‘latest’ isn’t actually latest, just one tagged as such!


Love and Hate the dynamic versions

Pros Cons

Versions express promotion state Consistency

No version – trash! Traceability


Best Practices – Best of two worlds!


Double Tag

»Push every image with a static tag

»Create a second, dynamic tag for the same

image as appropriate⋄Just a like a sym-link

⋄Tag name is not traceable by itself, because the “dynamic”

tag may have moved since client pulled.

⋄Docker Inspect, Search, Find image for traceability

»Cleanup static tags based on some policy


Static and Dynamic Tags

ac-image:1.0-

2017-04-01-

111

ac-image:latest1-

betaac-image:latest1

Manifest

sha256:252564..

Manifest

sha256:462564..

ac-image:1.0-

2017-04-03-

222

OS: SHA2

Framework: SHA2


L

a

y

e

r

s Application: SHA2: 4

T

a

g

s

ac-

image:latest1


The orphans


Another problem!

»So, if we change tags during promotion, what

happens to not promoted images?

»They loose their version!!!

(WTF, what does that mean?)


The unlucky ones

»In Java we call it SNAPSHOT-s.

»In Docker – images without versions.

»They are huge!



You don’t really have unlimited space


How Docker Registries

Work


Docker Distribution and DTR

»Checksum based storage ⋄Multiple tags with the same manifest does not use up additional disk

space

»Deleting a tag does NOT delete the layers from

storage⋄Actual Delete based on SHA2 reference ONLY

⋄Tag is effectively deleted if you re-push with a same tag

⋄Old Manifest remains, and still referenceable by SHA2


Deleting a tag does NOT delete

the layers from storage

ac-image:1.0 ac-image:1.0-beta ac-image:1.0-qa

Manifest

sha256:252564..

Manifest

sha256:462564..

ac-image:1.1

OS: SHA2

Framework: SHA2


L

a

y

e

r


T

a

g

s


Actual DELETE is based on SHA2

reference ONLY

ac-image:1.0 ac-image:1.0-qa

Manifest

sha256:252564..

Manifest

sha256:462564..

ac-image:1.1

OS: SHA2

Framework: SHA2


L

a

y

e

r


T

a

g

s

Delete via

SHA2


Docker GC

»Required to delete layers with no manifests

»Required to clear up disk space

»Stop the world


Docker Distribution, Best Practice

»Delete layers without tags up front.⋄HEAD Tag get SHA2 of manifest

⋄Delete SHA2

⋄After deleting reference, run GC to clean up


Docker Distribution, Best Practice

»(Not OOB) To clean up a registry with manifests

that have no tag⋄Command used to find these layers:

comm -23 <(find . -type f -name "link" | grep

"_manifests/revisions/sha256" | grep -v "\/signatures\/sha256\/" | awk -F/

'{print $(NF-1)}' | sort) <(for f in $(find . -type f -name "link" | grep

"_manifests/tags/.*/current/link"); do cat ${f} | sed 's/^sha256://g';

echo; done | sort) | wc –l


JFrog Artifactory

»We delete any layer that is not referenced by a

tag immediately

»We delete manifests that is not referenced by a

tag

»We have an API to copy/move a docker image or

change its tag without using the docker client to

pull/push a second time

»Configure the max number of tags per image


Built-in cleanup

ac-image:1.0 ac-image:1.0-qa

Manifest

sha256:252564..Manifest

sha256:462564..

ac-image:1.1

OS: SHA2

Framework: SHA2


L

a

y

e

r


T

a

g

s

Delete via

SHA2

Manifest

sha256:462564..

ac-image:1.2


Delete Tag


Recap

»Dynamic versions are good for promotion

»Static versions are good for traceability

»Retagging should be done in the registry

»All unreferenced objects should be collected


Q&A and Links

»@jbaruch

»jfrog.com/shownotes

»We’re hiring!

Best Practices for Managing Docker Versions as presented at JavaOne 2017

Technology