Scale Big With Docker February 2014—Docker 0.8.0
Jan 27, 2015
Scale Big With Docker
February 2014—Docker 0.8.0
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
Deploy everything
● webapps● backends● SQL, NoSQL● big data● message queues● … and more
Deploy almost everywhere
● Linux servers● VMs or bare metal● Any distro● Kernel 3.8 (or RHEL 2.6.32)
Deploy reliably & consistently
● If it works locally, it will work on the server● With exactly the same behavior● Regardless of versions● Regardless of distros● Regardless of dependencies
Deploy easily
● Don't learn a new CM tool● Reuse components and recipes● Don't install anything fancy locally
(One single local VM to handle all your apps)
Deploy at scale
● Build once● Deploy anywhere● Deploy quickly
– Images are lightweight– Docker runtime itself is lightweight– Fast start/stop times
● Manage Docker with its REST API
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
Problem: shipping goods? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
Solution:the intermodal shipping container
Problem: shipping code
djangoweb frontend ? ? ? ? ? ?
node.jsasync API ? ? ? ? ? ?
background workers ? ? ? ? ? ?
SQL database
? ? ? ? ? ?
distributed DB, big data
? ? ? ? ? ?
message queue
? ? ? ? ? ?
mylaptop
your laptop
QA staging prod on cloud VM
prod on bare metal
Solution:the Linux container
High level approach:it's a lightweight VM
● I can SSH into it● I have root access● I can apt-get/yum install packages ● I don't see the host system (or other containers)
« Machine Container »
Low level approach:it's chroot on steroids
● I can run a single program● container = isolated process(es)● share kernel with host● no device emulation → no overhead
« Application Container »
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
One-time setup
● On your servers (Linux)– Packages (Ubuntu, Debian, Fedora, Gentoo, Arch...)
– Single binary install (Golang FTW!)
– Easy provisioning on Rackspace, Digital Ocean, EC2, GCE...
● On your dev env (Linux, OS X, Windows)– Vagrantfile
– boot2docker (25 MB VM image)
– Natively (if you run Linux)
●
The Docker workflow 1/2
● Work in dev environment(local machine or container)
● Other services can be perfect copies of prod● Whenever you want to test « for real »:
– Build in seconds
– Run instantly
The Docker workflow 2/2
● Push your build to a registry (public or private)● Run it in CI/CD● Run it in production● Rejoice
Something goes wrong?Just re-run previous version (it's still there!)
I'm starting a new project!
● Each service will be in its own container(s)● Build an image for each service● Pin dependencies (packages etc.) accurately● Link services together
→ Recommended workflow!
I'm Dockerizing some code!
● Making a single service runnable on Docker● Create a « Dockerfile »
(Think Makefile, Vagrantfile...)● If it's on GitHub, create a Trusted Build
(will automatically build ready-to-run images)● In any case, register it on index.docker.io
I'm migrating to Docker!
● Easy if there is only one service per server– Dockerize the service
– Containerize administrative functions (logs...)
● Otherwise, try to split services● You don't need to move everything to Docker● Don't use containers as VMs, even if you can!
I'm already using Chef/Puppet/Salt/Ansible/...!
● Plan A: generic images– Put your configuration management tool in an image
– Run that image just like you would run a VM
– Great for mixed environments
● Plan B: specialized images– Put your configuration management tool in an image
– Run that image, freeze the result, run it!
– Faster and safer
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
Authoring imageswith run/commit
1) docker run ubuntu bash
2) apt-get install this and that
3) docker commit <containerid> <imagename>
4) docker run <imagename> bash
5) git clone git://.../mycode
6) pip install -r requirements.txt
7) docker commit <containerid> <imagename>
8) repeat steps 4-7 as necessary
9) docker tag <imagename> <user/image>
10) docker push <user/image>
Authoring imageswith run/commit
● Pros– Convenient, nothing to learn
– Can roll back/forward if needed
● Cons– Manual process
– Iterative changes stack up
– Full rebuilds are boring, error-prone
Authoring imageswith a Dockerfile
FROM ubuntu
RUN apt-get -y updateRUN apt-get install -y g++RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe ...RUN apt-get install -y libmozjs185-dev libicu-dev libtool ...RUN apt-get install -y make wget
RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf-RUN cd /tmp/apache-couchdb-* && ./configure && make install
RUN printf "[httpd]\nport = 8101\nbind_address = 0.0.0.0" > /usr/local/etc/couchdb/local.d/docker.ini
EXPOSE 8101CMD ["/usr/local/bin/couchdb"]
docker build -t jpetazzo/couchdb .
Authoring imageswith a Dockerfile
● Almost nothing to learn● Rebuilds are easy● Caching system makes rebuilds faster● Single file to define the whole environment!
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
Docker: the community
● Docker: >330 contributors● Docker inc.: <20 engineers ● latest milestone (0.8): >120 contributors● ~50% of all commits by external contributors● GitHub repository: >1400 forks
...and counting!
Coming Soon
● Network acceleration● Container-specific metrics● Consolidated logging● Plugins (compute backends...)
Those things are already possible,but will soon be part of the core.
Docker 1.0
● Multi-arch, multi-OS● Stable control API● Stable plugin API ● Resiliency● Signature● Clustering
Thank you! Questions?
http://docker.io/
http://docker.com/
https://github.com/dotcloud/docker
@docker
@jpetazzo