Container networking with MidoNet Pino de Candia, August 2015
Container networking with MidoNet
Pino de Candia, August 2015
About Midokura● Global startup with about 50+ employees● Started in 2010 with goal of building a cloud, but turned
its focus to network virtualization.● Engineering in Barcelona, Tokyo, Tel Aviv● Sales and Marketing in San Francisco● Network Virtualization for OpenStack, Docker,
Eucalyptus, and ESXi● Customers in web scale, enterprise, higher ed and
service provider segments
● Containers made easy● Daemon based● multiple backends
Docker Networkinglibcontainer(builtin):● NAT Bridge● None● Host● ContainerOverlay
NAT Bridge
Advantages:● Isolation from underlay● SimpleDrawbacks:● no easy cross-host● no advanced networking
Host Mode
Advantages:● Directly on the underlay● Full capabilitiesDrawbacks:● Lack of isolation
ContainerAdvantages:● Isolation from the underlay● Full communication containersDrawbacks:● no easy cross-host● no advanced networking
Overlay - FlannelAdvantages:● Overlay for the hosts● Each docker0 gets a routable /24● UDP and VXLAN backends● Provides configuration file for
dockerdDrawbacks:● Lack of flexibility● no advanced networking
Enter Neutron (and MidoNet)
Bring advanced networking to Containers● Neutron is the ONLY “standard” network
virtualization APIBring containers to OpenStack
Bare MetalServer
Bare MetalServer
Neutron/MidoNet transforms...
VM
VM
VM
VM VM
VM VM
VM
VM
VM
VM
VM VM
VM VM
VM
VM
VM
VM
VM VM
VM VM
VM
VM
VM
VM
VM VM
VM VM
VM
IP Fabric
Bare MetalServer
Bare MetalServer
that into this...
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM VM
VM
VM
VMVM
VM
VMVM
VM
VM
VM
VM
VM
VM
VM
FW
LB
FW
LB
Internet/WAN
FW
How to extend Docker networking?
Current way:● wrap around the docker API/cli, or● Hook to the event stream and react
Future way:● libnetwork (core or remote)
Wrapping Docker● Advanced networking as extra commands/params● Write a wrapping API● Write wrapping cli● Make sure users target the wrappers● Examples
○ Socketplane(cli)○ Weave(cli)○ Nova-docker(both)
How Nova plugs a Container’s NIC...1. the user calls nova-api to launch instance with a --nic parameter
2. nova-scheduler chooses the host and messages nova-compute
3. nova-compute delegates to the virt driver for nova: nova-docker
4. the driver pulls the docker image from glance into the local registry
5. the driver uses docker-py library to talk to docker and create the container
6. nova-compute asks the virt driver to plug and attach the nic.
7. the driver checks the port type and calls plug_ovs or plug_midonet, which creates a veth pair, puts one side in the container and one in overlay.
8. the driver sets the ip/mac config for the container side (netlink).
nova-docker wraps docker
How Nova plugs a VM’s NIC...1. the user calls nova-api to launch instance with a --nic parameter.
2. nova-scheduler chooses the host and messages nova-compute.
3. nova-compute delegates to the virt driver for libvirt
4. the driver prepares an xml definition of the VM, including the tap device needed for the port (depends on the porty type)
5. libvirt creates the VM and tap
6. the driver calls the plug method for the port type (libvirt already knows how to plug taps for MidoNet and OVS Neutron plugins).
7. The VM is created and gets the ip configuration thanks to cloud init and/or dhcp.
libnetwork (refactored libcontainer)Refactor libcontainer networking into a separate libraryExtends docker API/CLIWritten in Go by Docker for dockerMultiple drivers:
● bridge● host● null● overlay*● remote
libnetwork EntitiesDriverSandboxNetwork● Create● Delete
Endpoint● Create● Join● Leave● Delete
libnetwork “remote”
● Open to external vendors● Containerized● RPC dockerd<-->remote driver● Implements the driver entities
libnetwork neutron driver: kuryr
kuryr● Gives Swarm an overlay● Maps libnetwork entities to Neutron● Gets IPAM from Neutron● Vendor interface plugging configurable● Extra entities via labels● Opens Docker to advanced networking● Open Source and open to contributors
Contribute at https://github.com/openstack/kuryr
OpenStack Magnum