Marriage with docker

M A R R I A G E W I T H D O C K E RD U S A N K A T O N A

Since 01/2014

W H O A M I ?

• team lead & scrum master in GoodData

• agile enthusiast

• docker enthusiast

• keen traveller

S E A R C H I N G F O R A P E R F E C T W O M A N

• run arbitrary (ETL) ruby scripts on the platform

• isolate runs - from host & other scripts

T H E O T H E R W O M A N

• Openshift

• too heavyweight for us

• Amazon/Google containers service

• we don’t want to move data between Rackspace and other cloud providers

H O N E Y M O O N

• docker 0.7.6

• we don’t need to use plain LXC (call scripts with root)

• use docker REST API from our application

• create one image with RubySDK

• memory & CPU limits per container

S O B E R I N G U P

• setup private docker registry

• how do we limit disk usage

• how do we limit log size produced by scripts

• how do we limit access to internal network

• how do we monitor containers

• random kernel panics (2.6.32 kernel)

• disk usage -> LXC hooks create loop device and mount it to the container

• log size -> container writes to named pipe, application reads it and limit the size

• access to internal network -> turn off iptables on docker deamon, use our own

T O U G H T I M E S

• network problems - 20-30% scripts failed due to lost packets

• unstable client library (docker-java) - threading issues

• high load/fork bombs - resolved in docker 1.6 with ulimit per container (only for native driver :( )

• allow users to run multiple versions on our server

• easy rollout/rollback

• in production since January 2014 (for limited audience)

• ~1000 containers spawned a day

• some fixes backported to 2.6.32 kernel -> no more kernel panics

R E L AT I O N S H I P

F I N A L W O R D

• Docker is perfectly usable for development & testing infrastructure

• be cautious when using docker in production

• have a bunch of OPS engineers at your hand