Dockerize Your IT ! 1 Centrale Nantes Information Technology Department Yoann Juet Dec, 2018
Dockerize Your IT !
1
Centrale NantesInformation Technology Department
Yoann JuetDec, 2018
A Brief History of Containers
2
1979 2000 2001 2002
2006 2008
UNIXCHROOT
BSDJAIL
LINUXVSERVER
LINUXNAMESPACES
2005
LINUXOPENVZ
LINUXCGROUPS
LINUXLXC
2013
DOCKER
2015
KUBERNETES
Docker Containers != Virtual Machines
3
VMs have a full copy of an OS, leading to cpu, memory, network overhead ; very good isolation between VMs
Containers share host ressources (kernel, cpu, memory, network) ; good isolation between Containers
Docker architecture
4
● docker : the Docker user CLI● dockerd : engine daemon
○ Create image, pass it to containerd
● containerd : runtime daemon○ Core container runtime for
Docker○ Manage the complete
container lifecycle (stop, start, transfer, supervision, storage, network)
● This model gives the ability to restart or upgrade Docker Engine without breaking the running containers
Docker images,
5
An image is a read-only group of layers of other images. It includes everything an application needs to run: binaries, libraries, config files…
Each image is made of a base image (e.g. debian, ubuntu, alpine) plus a collection of diffs - intermediate images/layers - that adds the required features (e.g. emacs, apache).
Images can be stored on public, private repos, on any host machine that has previously pulled the package from a repo
Docker Containers
6
A container is a running instance - read-write - of an image
You can run containers on Linux, Windows 10, Windows Server 2016, Cloud (AWS, Google…)
Containers should be as ephemeral as possible. You should expect them to go down at any time and lose all data stored inside:
● Don’t store data in containers● Don’t run more than one process in a single
container● Use custom created volumes or system
mounts
7
Docker Image != Docker Container
App Image
App 1Container
Public or PrivateRegistry
Remember…
● A Docker Image is similar to a read-only template● A Docker Container is an writable instance of a
Docker Image ● Each Docker Container has its own read-write
layer - thus its own data - that sits on one or more Docker imageApp 2
Container
pull
r/w Instance
Commit
r/w
Inst
ance
push
8
Lightweight, low resources consumptions
Isolation model, default set of capabilities
Low attack surface
Networking can be tricky
Fast boot, removal, reproducibility : the same code runs everywhere
Orchestration complexity
Large ecosystem - lot of official and unofficial images -
Ideal for development team (test, pre-prod, prod)
Docker Containers - Pros and Cons
9
Docker repository (version 18.09)user@host:~$ sudo apt-get update && apt-get install apt-transport-https ca-certificates curl gnupg2 software-properties-common
user@host:~$ sudo curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
user@host:~$ sudo apt-key fingerprint 0EBFCD88
pub rsa4096 2017-02-22 [SCEA]
9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88
uid [ unknown] Docker Release (CE deb) <[email protected]>
sub rsa4096 2017-02-22 [S]
user@host:~$ sudo add-apt-repository "deb https://download.docker.com/linux/debian stretch stable"
user@host:~$ sudo apt-get update && apt-get install docker.ce
user@host:~$ sudo usermod -aG docker <user> && newgrp docker
Installing Docker Community Edition (CE)On debian 9 (stretch) host - from the netinst image -
10
Check for correct installationuser@host:~$ docker info...Server Version: 18.09.0Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true...Runtimes: runcDefault Runtime: runcKernel Version: 4.9.0-8-amd64Operating System: Debian GNU/Linux 9 (stretch)...Docker Root Dir: /var/lib/docker...Product License: Community Engine
Installing Docker Community Edition (CE)On debian 9 (stretch) host - from the netinst image -
11
Search for an official nginx image...user@host:~$ docker search nginxNAME DESCRIPTION STARS OFFICIAL AUTOMATEDnginx Official build of Nginx. 10386 [OK] jwilder/nginx-proxy Automated Nginx reverse proxy for docker con… 1473 [OK]richarvey/nginx-php-fpm Container running Nginx + PHP-FPM capable of… 652 [OK]...
Download the image from the official Docker Registryuser@host:~$ docker pull nginxUsing default tag: latestlatest: Pulling from library/nginxa5a6f2f73cd8: Pull complete 1ba02017c4b2: Pull complete 33b176c904de: Pull complete Digest: sha256:5d32f60db294b5deb55d078cd4feb410ad88e6fe77500c87d3970eca97f54dbaStatus: Downloaded newer image for nginx:latest
Run Your First Docker ApplicationLet’s say an nginx http server !
In the OFFICIAL column, OK indicates an image built by the company/community behind the project
12
Verify user@host:~$ docker image lsREPOSITORY TAG IMAGE ID CREATED SIZEnginx latest 568c4670fa80 46 hours ago 109MB
Run a container based on this imageuser@host:~$ mkdir ~/mydir && echo "Hello, France!" > ~/mydir/index.html && docker run --name myweb -v ~/mydir:/usr/share/nginx/html -p 8080:80 -d nginx8807f82f280b6dde9848029c414b22b9ac7b77362b39d7c58c94e15d2eedc905
user@host:~$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES8807f82f280b nginx "nginx -g…" 40 seconds ago Up 39 seconds 0.0.0.0:8080->80/tcp myweb
Run Your First Docker ApplicationLet’s say an nginx http server !
13
Connect to your Appuser@host:~$ wget -q localhost:8080 -O /dev/stdout Hello, France!
Enter in your Appuser@host:~$ docker exec -it myweb /bin/bashroot@8807f82f280b:/# ls /etc/nginx/ && exitconf.d fastcgi_params koi-utf koi-win mime.types modules nginx.conf ...
Stop and Remove your Appuser@host:~$ docker rm -f mywebmyweb
root@host:/tmp# docker ps --allCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Run Your First Docker ApplicationLet’s say an nginx http server !
Focus on Docker Images...
14
docker push image[:TAG]docker image push image[:TAG]
docker imagesdocker image ls [--all]List all installed images
Pull an image from a registry docker pull imagedocker image pull image
Push an image to a registry
Remove an imagedocker rmi imagedocker image rm imagedocker image prune [--all]
Backup and restore an image (tarball)docker save -o archive.tar imagedocker load < archive.tar
Inspect the content of an image docker inspect image
Focus on Docker Containers...
15
docker stop containerdocker container stop container
docker ps [--all]docker container ls [--all]List all containers
[re]Start an inactive container docker [re]start containerdocker container [re]start container
Stop an active container
[stop and] Remove a container docker rm [-f] containerdocker container rm [-f] container
Execute a command in an active container / Get a shell console
docker exec container commanddocker exec -it container /bin/bash
Inspect the content of a container docker inspect container
Focus on Docker Containers...
16
docker create imagedocker run image
docker commit container [REPO[:TAG]]Create an image from a container
Show the logs of a container docker logs container
Create a new container
Backup and restore a container (tarball) docker export container > archive.tardocker import - container < archive.tar
17
1 - Get alpine imageuser@host:~$ docker pull alpine
2 - Run ituser@host:~$ docker run -d -it --name myapp alpine3 - Edit a file in the containeruser@host:~$ docker exec myapp \sh -c 'echo "Hello, France!" > /root/msg.txt'4 - Stop the containeruser@host:~$ docker stop myapp5 - Start the containeruser@host:~$ docker start myapp6 - Checkuser@host:~$ docker exec myapp less /root/msg.txtHello, France
What about my data ?
7 - Stop then remove the containeruser@host:~$ docker rm -f myapp8 - Run again a containeruser@host:~$ docker run -d -it --name myapp alpine9 - Checkuser@host:~$ docker exec myapp less /root/msg.txtmore: can't open '/root/msg.txt': No such file or directory
Remember that each container has is own read-write layer
When you instantiate an image, the new container starts with a clean filesystem
18
Run it user@host:~$ echo "Hello, Nantes!" > ~/msg.txt && docker run -d -it -v ~/msg.txt:/root/hop.txt --name mydata alpineCheckuser@host:~$ docker exec mydata ls /roothop.txtuser@host:~$ docker exec mydata less /root/hop.txtHello, Nantes!
What about my data ?
Run another docker based on alpineuser@host:~$ docker run -d -it -v ~/msg.txt:/root/hop.txt --name mydatb alpineEdit the messageuser@host:~$ echo "Hello, Centrale!" > ~/msg.txtCheckuser@host:~$ docker exec mydata less /root/hop.txtHello, Centrale!user@host:~$ docker exec mydatb less /root/hop.txtHello, Centrale!
Sharing data in your docker host with containers
Use the -v option : -v [HOST-DIR]:[CONTAINER-DIR]It creates a bind mount exposing the host directory/host file to the container directory/file
19
Get information about volumesuser@host:~$ docker inspect mydata... "Mounts": [ { "Type": "bind", "Source": "/home/user/msg.txt", "Destination": "/root/hop.txt", "Mode": "", "RW": true, "Propagation": "rprivate" } ],...
What about my data ?
By default, Docker mounts the volume in read-write mode
Sharing data in your docker host with containers
Instruction to share data in read-only modeuser@host:~$ docker run -d -it -v ~/msg.txt:/root/hop.txt:ro --name mydata alpine
20
Run the yoyo container user@host:~$ docker run -d -it -v /sharing --name yoyo alpineuser@host:~$ docker inspect yoyo... "Mounts": [ { "Type": "volume", "Name": "fdc5bd0de90688d590b38f9f931eb011c4de9c4032d49...", "Source": "/var/lib/docker/volumes/fdc5bd0de90...a54b/_data", "Destination": "/sharing", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" } ],
What about my data ?
If the host directory/file is omitted, a data container (read-write mode per default) is created
The volume specified, here /sharing, is created inside the container
user@host:~$ touch /var/lib/docker/volumes/fdc5bd0d.../_data/msg.txt
user@host:~$ docker exec yoyo ls /sharingmsg.txt
Sharing data between containers
21
Run the dong container user@host:~$ docker run -d -it --volumes-from yoyo --name dong alpineuser@host:~$ docker inspect dong... "Mounts": [ { "Type": "volume", "Name": "fdc5bd0de90688d590b38f9f931eb011c4de9c40...4a54b", "Source": "/var/lib/docker/volumes/fdc5bd0de90...54b/_data", "Destination": "/sharing", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" } ],
What about my data ?
user@host:~$ docker exec yoyo ls /sharingMsg.txt
user@host:~$ docker exec dong ls /sharingMsg.txt
user@host:~$ docker exec dong touch /sharing/msg2.txt
user@host:~$ docker exec yoyo ls /sharingmsg.txtMsg2.txt
user@host:~$ ls /var/lib/docker/volumes/fdc5bd0de90688d5.../_datamsg2.txt msg.txt
Sharing data between containers
22
user@host:~$ docker image ls --allREPOSITORY TAG IMAGE ID CREATED SIZEalpine latest 196d12cf6ab1 2 months ago 4.41MBuser@host:~$ docker run -d -it --name ding alpineuser@host:~$ docker exec -it ding /bin/sh/ # apk update && apk upgradefetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gzv3.8.1-133-g80b45d6920 [http://dl-cdn.alpinelinux.org/alpine/v3.8/main]v3.8.1-133-g80b45d6920 [http://dl-cdn.alpinelinux.org/alpine/v3.8/community]OK: 9546 distinct packages availableOK: 4 MiB in 13 packages/ # exituser@host:~$ docker commit ding alpine:201812sha256:4acef3925b22e668e0e1755a3fa5ab6004889cacee28c8a487c5754ea105be70user@host:~$ docker image ls --allREPOSITORY TAG IMAGE ID CREATED SIZEalpine 201812 4acef3925b22 27 seconds ago 5.71MBalpine latest 196d12cf6ab1 2 months ago 4.41MB
Committing Changes in a ContainerTurning a container into an image
23
user@host:~$ docker diff dingC /libC /lib/apkC /lib/apk/db...C /var/cacheC /var/cache/apkA /var/cache/apk/APKINDEX.adfa7ceb.tar.gzA /var/cache/apk/APKINDEX.efaa1f73.tar.gzC /rootA /root/.ash_history
Committing Changes in a Container
First column: A means that the directory/file was added, C means that a change was made, D means that it was removed.
You see all changes applied to the read-write layer, the container itself - useful before executing the commit instruction
Display container changes
Docker Networking Services
24
Bridge Host
Share host interfaces
No more network isolation between the host and containers
Should be used very carefully
Macvlan Bridge
Unique MAC address
Allows you to configure slave/sub-interfaces of a parent, physical ethernet interface, each with its
own unique MAC address
None
No network !
Overlay
Multiple hosts with only L3 connectivity
Combines local bridges and VXLAN, GRE to
overlay container-to-container
Useful to offer network connectivity between containers on multiple
hosts usingtheir own IP addresses
25
user@host:~$ docker run --name alice -p 8080:80 -d nginxuser@host:~$ docker run --name bob -p 8081:80 -d nginx
user@host:~$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESffe72798ac2d nginx "nginx -g 'daemon of…" … … 0.0.0.0:8081->80/tcp bob1a2f2f83e044 nginx "nginx -g 'daemon of…" … … 0.0.0.0:8080->80/tcp alice
Going deeper in docker bridge default mode
Expose unique ports on the host - here 8081 and 8080 -
26
user@host:~$ brctl showbridge name bridge id STP enabled interfacesdocker0 8000.02422d1a2630 no veth856faee
vetha53c999user@host:~$ ip add...3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:2d:1a:26:30 brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 valid_lft forever preferred_lft forever7: vetha53c999@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default link/ether be:71:06:64:64:8f brd ff:ff:ff:ff:ff:ff link-netnsid 111: veth856faee@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default link/ether b2:6c:95:08:43:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 2
...
Going deeper in docker bridge default mode
27
user@host:~$ docker inspect bob "NetworkSettings": { "Bridge": "",
... "Gateway": "172.17.0.1", "IPAddress": "172.17.0.4", "IPPrefixLen": 16, "MacAddress": "02:42:ac:11:00:04",
...}
user@host:~$ docker inspect alice "NetworkSettings": { "Bridge": "",
... "Gateway": "172.17.0.1", "IPAddress": "172.17.0.3", "IPPrefixLen": 16, "MacAddress": "02:42:ac:11:00:03",
...}
Going deeper in docker bridge default mode
Docker assigns a dynamic IP address for both containers
28
user@host:~$ iptables -L -n -v ...Chain DOCKER (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT tcp -- !docker0 docker0 0.0.0.0/0 172.17.0.3 tcp dpt:80 0 0 ACCEPT tcp -- !docker0 docker0 0.0.0.0/0 172.17.0.4 tcp dpt:80...
user@host:~$ iptables -t nat -L -n -vChain DOCKER (2 references) pkts bytes target prot opt in out source destination 1 84 RETURN all -- docker0 * 0.0.0.0/0 0.0.0.0/0 0 0 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 to:172.17.0.3:80 0 0 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8081 to:172.17.0.4:80
Going deeper in docker bridge default mode
29
user@host:~$ docker image ls --allREPOSITORY TAG IMAGE ID CREATED SIZEalpine latest 196d12cf6ab1 2 months ago 4.41MB
user@host:~$ vi /tmp/DockerfileFROM alpine:latestRUN apk update && apk upgradeENTRYPOINT ["/bin/echo","Hello, France!"]
user@host:~$ cd /tmp && docker build . -t hello
Sending build context to Docker daemon 18.94kBStep 1/3 : FROM alpine:latest ---> 196d12cf6ab1Step 2/3 : RUN apk update && apk upgrade ---> Running in 116d94cb96fafetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gzv3.8.1-115-ge3ed6b4e31 [http://dl-cdn.alpinelinux.org/alpine/v3.8/main]v3.8.1-112-g45bdd0edfb [http://dl-cdn.alpinelinux.org/alpine/v3.8/community]OK: 9546 distinct packages availableOK: 4 MiB in 13 packagesRemoving intermediate container 116d94cb96fa ---> fd105816bcb9Step 3/3 : ENTRYPOINT ["/bin/echo","Hello, France!"] ---> Running in 8756eb9e1d37Removing intermediate container 8756eb9e1d37 ---> b4277a54d78bSuccessfully built b4277a54d78bSuccessfully tagged hello:latest
Build your own image
30
user@host:~$ docker image ls --allREPOSITORY TAG IMAGE ID CREATED SIZE<none> <none> fd105816bcb9 5 minutes ago 5.71MBhello latest b4277a54d78b 5 minutes ago 5.71MBalpine latest 196d12cf6ab1 2 months ago 4.41MB
Sending build context to Docker daemon 18.94kBStep 1/3 : FROM alpine:latest ---> 196d12cf6ab1Step 2/3 : RUN apk update && apk upgrade ---> Running in 116d94cb96fafetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gzv3.8.1-115-ge3ed6b4e31 [http://dl-cdn.alpinelinux.org/alpine/v3.8/main]v3.8.1-112-g45bdd0edfb [http://dl-cdn.alpinelinux.org/alpine/v3.8/community]OK: 9546 distinct packages availableOK: 4 MiB in 13 packagesRemoving intermediate container 116d94cb96fa ---> fd105816bcb9Step 3/3 : ENTRYPOINT ["/bin/echo","Hello, France!"] ---> Running in 8756eb9e1d37Removing intermediate container 8756eb9e1d37 ---> b4277a54d78bSuccessfully built b4277a54d78bSuccessfully tagged hello:latest
31
user@host:~$ vi /tmp/DockerfileFROM alpine:latestRUN apk update && apk upgradeRUN apk add opensshENTRYPOINT ["/bin/echo","Hello, France!"]
user@host:~$ cd /tmp && docker build . -t hello
Sending build context to Docker daemon 5.632kBStep 1/4 : FROM alpine:latest ---> 196d12cf6ab1Step 2/4 : RUN apk update && apk upgrade ---> Using cache ---> fd105816bcb9Step 3/4 : RUN apk add openssh ---> Running in c0fbeb4c8ea8fetch http://dl-cdn.alpinelinux.org/./APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/./APKINDEX.tar.gz(1/6) Installing openssh-keygen (7.7_p1-r3)...(6/6) Installing openssh (7.7_p1-r3)Executing busybox-1.28.4-r1.triggerOK: 8 MiB in 19 packagesRemoving intermediate container c0fbeb4c8ea8 ---> 3ba8527195d0Step 4/4 : ENTRYPOINT ["/bin/echo","Hello, France!"] ---> Running in ead00e7c9adaRemoving intermediate container ead00e7c9ada ---> 1d451e42e9e2Successfully built 1d451e42e9e2Successfully tagged hello:latest
Build your own image
32
Sending build context to Docker daemon 5.632kBStep 1/4 : FROM alpine:latest ---> 196d12cf6ab1Step 2/4 : RUN apk update && apk upgrade ---> Using cache ---> fd105816bcb9Step 3/4 : RUN apk add openssh ---> Running in c0fbeb4c8ea8fetch http://dl-cdn.alpinelinux.org/./APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/./APKINDEX.tar.gz(1/6) Installing openssh-keygen (7.7_p1-r3)...(6/6) Installing openssh (7.7_p1-r3)Executing busybox-1.28.4-r1.triggerOK: 8 MiB in 19 packagesRemoving intermediate container c0fbeb4c8ea8 ---> 3ba8527195d0Step 4/4 : ENTRYPOINT ["/bin/echo","Hello, France!"] ---> Running in ead00e7c9adaRemoving intermediate container ead00e7c9ada ---> 1d451e42e9e2Successfully built 1d451e42e9e2Successfully tagged hello:latest
root@host:/tmp# docker image ls --allREPOSITORY TAG IMAGE ID CREATED SIZE<none> <none> 3ba8527195d0 5 minutes ago 11MBhello latest 1d451e42e9e2 5 minutes ago 11MB<none> <none> b4277a54d78b 24 hours ago 5.71MB<none> <none> fd105816bcb9 24 hours ago 5.71MBalpine latest 196d12cf6ab1 2 months ago 4.41MB
root@host:/tmp# docker image pruneWARNING! This will remove all dangling images.Are you sure you want to continue? [y/N] yDeleted Images:deleted: sha256:b4277a54d78bc870ba2c1e971f076da6a324cc73f97dead788eb0823592c9e5e
Total reclaimed space: 0B
Build your own image
Thanks for your attention