Linux Container Technology inside Docker with RHEL7 Etsuji Nakai Senior Solution Architect and Cloud Evangelist Red Hat K.K v1.0 2015/06/22
Aug 04, 2015
Linux Container Technologyinside Docker with RHEL7
Etsuji NakaiSenior Solution Architect
and Cloud EvangelistRed Hat K.K
v1.0 2015/06/22
2
Linux Container Technology inside Docker with RHEL7
Contents
What is Docker? Container Technology inside Docker Architecture of Kubernetes References
What is Docker?
4
Linux Container Technology inside Docker with RHEL7
Quick Demo!
5
Linux Container Technology inside Docker with RHEL7
Dockerfile
① Auto-build Docker images
OS Image
Application Library / Framework
Application Binary
Describe steps to build an image
Dockerimage
Everything you need to run applicationis included in the image
② Upload and publish images
③ Download and run
What you can do with Docker
Container Technologyinside Docker
7
Linux Container Technology inside Docker with RHEL7
"Linux Container" is a Linux kernel feature to contain a group of processes in an independent execution environment.
Linux kernel provides an independent application execution environment for each container including:– Independent filesystem.– Independent network interface and IP address.–Usage limit for memory and CPU time.
Linux Kernel
Use
r Pr
oces
s
・・・
Physical Host / VMPhysical Host / VM
OS
ContainerNo Container
Use
r Pr
oces
s
Use
r Pr
oces
s
User Space
Linux Kernel
Use
r Pr
oces
s
Use
r Pr
oces
s
User Space
Use
r Pr
oces
s
Use
r Pr
oces
s
User Space
・・・
What is container technology?
Container
8
Linux Container Technology inside Docker with RHEL7
Container supports separation of various resources. They are internally realized with different technologies called "namespace."– Filesystem separation → Mount namespace (kernel 2.4.19) – Hostname separation → UTS namespace (kernel 2.6.19)– IPC separation → IPC namespace (kernel 2.6.19)– User (UID/GID) separation → User namespace (kernel 2.6.23〜kernel 3.8)– Processtable separation → PID namespace (kernel 2.6.24) – Network separation → Network Namespace (kernel 2.6.24)– Usage limit of CPU/Memory → Control groups
Linux container is realized with integrating these namespace features. There are multiple container management tools such as lxctools, libvirt and docker. They may use different parts of these features.
Under the hood
9
Linux Container Technology inside Docker with RHEL7
Processes in all containers are executed on the same Linux kernel. But, inside a container, you can see processes only in the container.– This is because each container has its own process table. On host linux, which is outside
containers, you can see all processes including ones in containers.
Process table
# ps -efUID PID PPID C STIME TTY TIME CMDroot 1 0 0 09:49 ? 00:00:00 /bin/sh /usr/local/bin/init.shroot 47 1 0 09:49 ? 00:00:00 /usr/sbin/httpdapache 49 47 0 09:49 ? 00:00:00 /usr/sbin/httpdapache 50 47 0 09:49 ? 00:00:00 /usr/sbin/httpd...apache 56 47 0 09:49 ? 00:00:00 /usr/sbin/httpdroot 57 1 0 09:49 ? 00:00:00 /bin/bash
# ps -efUID PID PPID C STIME TTY TIME CMD...root 802 1 0 18:10 ? 00:01:20 /usr/bin/docker -d --selinux-enabled -H fd://...root 3687 802 0 18:49 pts/2 00:00:00 /bin/sh /usr/local/bin/init.shroot 3748 3687 0 18:49 ? 00:00:00 /usr/sbin/httpd48 3750 3748 0 18:49 ? 00:00:00 /usr/sbin/httpd...48 3757 3748 0 18:49 ? 00:00:00 /usr/sbin/httpdroot 3758 3687 0 18:49 pts/2 00:00:00 /bin/bash
Processes seen inside container
Processes seen outside container
10
Linux Container Technology inside Docker with RHEL7
Process table
fork/exec
PID namespace
In the example of previous page, docker daemon fork/exec-ed the initial process "init.sh" and put it in a new "PID namespace." After that, all processes fork/exec-ed from init.sh are put in the same namespace.
Inside container, the initial process has PID=1 independently from the host. Likewise, child processes of it have independent PID's.
PID=1
bash
/bin/sh /usr/local/bin/init.sh
httpd
httpd
・・・
#!/bin/sh
service httpd startwhile [[ true ]]; do /bin/bashdone
init.sh
docker daemon
11
Linux Container Technology inside Docker with RHEL7
Filesystem
A specific directory on the host is bind mounted as a root directory of the container. Inside container, that directory is seen as a root directory, very similar mechanism to the "chroot jail."
When using traditional container management tools such as lxctools or libvirt, you need to prepare the directory contents by hand.– You can put minimum contents for a specific application such as application binaries
and shared libraries in the directory.– It's also possible to copy a whole root filesystem of a specific linux distribution to
the directory.– If necessary, special filesystems such as /dev, /proc and /sys are mounted in the
container by the management tool.
Mount namespace
/ |--etc |--bin |--sbin...
/export/container01/rootfs/ |--etc |--bin |--sbin ...
bind mount
12
Linux Container Technology inside Docker with RHEL7
Filesystem
Container
App
licat
ion
Directory Tree
Mounted on the host
Assign as / filesystem
With Docker, you don't need to prepare the directory tree by hand.
Docker image is mounted on the host and used as root filesystem of the container.
Docker Image
13
Linux Container Technology inside Docker with RHEL7
Network namespace
Network
Container uses Linux's "veth" device for network communication.– veth is a pair of logical NIC devices connected through a (virtual) crossover cable.
One side of the veth pair is placed in a container's network namespace so that it can be seen only inside the container. The other side is connected to a Linux bridge on the host.– A device name in the container is renamed such as "eth0." By means of the namespace, network
settings such as IP address, routing table and iptables are independently configured in the container。
– The connection between the bridge and a physical network is up to the host configuration.
Host LinuxvethXX
eth0
docker0
eth0
IP masquerade
Physical network
Docker creates a bridge "docker0" and packets from containers are forwarded with IP masquerade.– Packets from the physical network targeted to specified
ports are forwarded to the container using the port forwarding feature of iptables.
172.17.42.1
14
Linux Container Technology inside Docker with RHEL7
Network
Example container network for 3-tier application running on the same host.
Accessing to the external IP of the host
Container:Web Server
REST_PORT_5555_TCP_ADDR
eth0
DB_PORT_3306_TCP_ADDR
Container:App Server
eth0
Container:Database
eth0
Linux bridge(docker0)
External IP
Port 80 Port 5555 Port 3306
Port 80
15
Linux Container Technology inside Docker with RHEL7
Network
Example container network for 3-tier application running on different hosts.
REST_PORT_5555_TCP_ADDR
eth0
External IP
REST_PORT_5555_TCP_ADDR
eth0
External IP
eth0
External IP
Container:Web Server Container:App Server Container:Database
Port 80 Port 5555 Port 3306
Architecture of Kubernetes
17
Linux Container Technology inside Docker with RHEL7
Server configuration
etcd
・・・
Backend Database(KVS)
Kubernetes MasterKubernetes Node (Minion)
・・・
Scale-out cluster
Docker Docker Docker
Add more minionsif necessary.
Docker Registry
Kubernetes manages multiple nodes (minions) from a single master.– Clustering of multiple masters is not available now. You may use active-standby
configuration with standard HA tools for high availability.– etcd (KVS) is used as a backend database. It can be configured as a scale-out cluster.
18
Linux Container Technology inside Docker with RHEL7
Network configuration
etcd KubernetesMaster
DockerRegistry
Configured asan overlay network.
・・・
Physical network is simple. Kubernetes works just by connecting all servers to a single service network.
However, you need to create an internal network for container communication using an overlay network.– You may use Flannel, Open vSwitch, etc. as an overlay technology.
Service network192.168.122.0/24
Minion
docker0
Minion
docker0
Internal network10.1.0.0/16
19
Linux Container Technology inside Docker with RHEL7
Internal network details
The internal network needs to be prepared independently from Kubernetes.– Flannel is the most convenient tool for this purpose.
Flannel configures an internal network as follows:– Assign non-overlapping subnets to the Linux bridge (docker0) of each minion. (eg.
10.1.x.0/24 with x=1,2,3,...)– Create a virtual interface "flannel.1" which works as a gateway to other minions.– Linux kernel on each minion transfers packets from/to flannel.1 using the VXLAN
encapsulation. (Flannel daemon "flanneld" provides necessary information for VXLAN processing to the kernel.)
flannel.1
docker0
10.1.1.0/24
10.1.1.0
etn0
10.1.1.1
Gateway to10.1.0.0/16
Encapsulation flannel.1
docker0
10.1.2.0/24
10.1.2.0
etn0
10.1.2.1
Gateway to10.1.0.0/16
minion01 minion02
10.1.0.0/16
flanneld flanneld
20
Linux Container Technology inside Docker with RHEL7
External access
etcd KubernetesMaster Minion Docker
RegistryMinion
API requests Image upload
・・・
Service access
There are following cases for the external access.– API requests are sent to the master.– Services running on containers are accessed from minions' external IPs via proxy
mechanism.– Docker registry is an independent component from Kubernetes. You may use a
registry server running on a container.
Service network
Internal network
21
Linux Container Technology inside Docker with RHEL7
Baremetal / VM ・・・
Docker
Baremetal / VM
Docker
Kubernetes
Platform as a Service
・・・
Execution Resource
ContainerManagement
Container Orchestration
UI, Monitoring,Image build workflow,
etc.
RHEL Atomic Host
OpenShift 3.0
Beyond Kubernetes: OpenShift v3
Container
Container
Container
Container
・・・ ・・・
References
23
Linux Container Technology inside Docker with RHEL7
References
Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1– http://www.slideshare.net/enakai/architecture-overview-rubbernecks-with-red-
hat-enterprise-linux-71
Inside Docker for Fedora20/RHEL7– http://www.slideshare.net/enakai/docker-technology-v18e
OpenShift 3 Technical Architecture– https://docs.google.com/presentation/d/1Isp5UeQZTo3gh6e59FMYmMs_V9QIQeBel
mbyHIJ1H_g/pub
OpenShift v3 Internal networking details– http://www.slideshare.net/enakai/openshift-45465283
EMPOWER PEOPLE,
EMPOWER ENTERPRISE,
OPEN INNOVATION.