Vbrownbag container networking for real workloads

Matt Johnson #vBrownBag at Cisco LIVE! 2015 9th June 2015

Container networking for real workloads. Why Default isn’t Best

[email protected] @mattdashj

Presentation recording available at: https://www.youtube.com/watch?v=z-GD-ctjCRc

2 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public

Hard to say ‘container’ nowadays without…

So we’ll start our story here.

3 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public

Quick Overview: Docker Networking

host docker0 172.17.42.1/16

veth049e89e

eth0

172.17.0.21/16

veth049e88f veth049e87af

eth0

192.168.1.10

eth0

192.168.1.10

eth0

192.168.1.10

host eth0

docker run --net=bridge (default) docker run --net=host

Container shares the host’s IP address and MAC address. vEth pair linked into container from docker0 bridge. DNAT provides inbound ports to containerized apps.

•  Default IP address is dynamically allocated from unused range (ie 172.17.0.0/16.)

can be set with –fixed-cidr on docker daemon •  Default GW for containers is the IP addr of the host docker0 interface

can be set with –bip on docker daemon

eth0

172.17.0.21/16

eth0

172.17.0.21/16 containers

3

iptables (NAT) iptables (NAT)

4 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public

Why NAT? -  Works out of the box on a host with a single IP

Dockers’ rise to prominence due to User Experience. Not new tech.

- LXC, Existed - Containers, Nothing new (Solaris Zones Anyone?)

The difference is it had never been as easy to use/explain/build/share.

Not without it’s limits: -  Container IP tied to Host.

-  Only one container can have a ‘common’ port per host. -  Careful manual port assignment.

-  Dynamic assignment with support in the app to support it. -  App’s need to be told to read ‘real’ network info from $ENV’s

6 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public

Extending Networking: Kubernetes Pod - a co-located group of Docker containers with shared volumes. They're the smallest deployable units that can be created, scheduled, and managed with Kubernetes.

Service - provide a single, stable name and address for a set of pods. They act as basic load balancers.

Label - are used to organize and select groups of objects based on key/value pairs.

Replication Controller - are used to organize and select groups of objects based on key/value pairs.

master server apiserver

etcd

controller manager

scheduler

skydns

pod C11

C12

C13

minion

pod C21

C22

C23

pod C41

C42

C43

minion

pod

C14

C15

pod

C44

minion

pod

C24

pod

C44

C45

minion

pod

C24

pod C11

C12

C13

minion

pod C21

C22

C23

service(IP addr/DN) replication Controller service(IP addr/DN)

cluster

6

7 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public

Extending Networking: Kubernetes

7

Tunnels are great… Right?

9 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public

Tunnels

We don’t have to deal with whatever the underlying

network is.

10 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public

Tunnels Although we do have to deal

with…

Ramps! On/Off the overlay. Potential bottleneck / scalability issues

An Interesting L2 Design.

Subject to implementation, may lack multicast support, treat broadcast strangely etc.

Debug overhead.

I only have 50 VM’s, but which one of my 1,225 tunnels (full mesh) is causing the issue.

11 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public

Tunnels Although we do have to deal

with…

MTU and Frag. avoidance. It is a tunnel after all!

Security.

Subject to implementation, spanning ARP really isn’t ideal. Lots of containers running

privileged.

12 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public

Future Docker Networking: libnetwork The Container Network Model (CMN)

12

Docker Container Docker Container Docker Container

Network Sandbox Network Sandbox Network Sandbox

Endpoint Endpoint Endpoint Endpoint

Backend Network Frontend Network

•  Network Sandbox -- isolated environment where container network configuration lives •  Endpoint – network interface tied to a specific network

•  Network – a uniquely identifiable collection of Endpoints that are able to communicate with each other

A pluggable interface. Expected to first ship in Docker 1.7. Distributed bridge plugin under development.

13 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public

Future Docker Networking: Overlay/L2 Centric The Container Network Model (CMN)

13

14 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public

So wouldn’t it be great if…

We could network containers without NAT,

Without requiring a ‘tunnels everywhere’ approach, Without on or off ramps, With full IPv4 and IPv6 support,

And at Internet-scale?

15 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public

We Can! .. If we don’t treat containers as Unique/Special

16 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public

Scalable DataCenter topologies have existed for quite a while…

And no, this isn’t about to turn into a Cisco sales pitch… promise!

18 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public

Standards Use Networking standards

tried and tested by the rest of the industry. Already used

every day at unprecedented Scale.

Don’t reinvent the wheel.

Integrate and benefit from

existing network team skillsets.

19 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public

Proposed solution with Docker / Container Networking

20 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public

Containers as a first class citizen of the real network •  Devolve routing down into each host. •  BGP as the saleable, proven control plane. •  Standard, proven linux networking stack. •  Containers have IP’s..

•  No NAT. •  DNS works properly. •  No tunnels necessary. •  Support for legacy applications & clustering. •  Support for multicast. •  IPv4 & IPv6 Support.

•  Standard network protocols that the rest of the industry uses •  Service providers and Enterprises can easily integrate.

21 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public

Buy Today…

And we’ll throw in container mobility for free!

IP’s no longer bound to the IP of the host, If a workload moves, the IP can go with it.

Containerized data services will benefit

massively from this.

22 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public

So where is this work going? Validated ideas through other container Industry supporters. Not the only people thinking this way. Implementation is happening now! Which brings me onto a side note…

Matt Johnson #vBrownBag at Cisco LIVE! 2015 9th June 2015

This isn’t a real title.

The ‘Nothing really to do with networking’ part of the talk

@mattdashj

Snowflakes!

25 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public

GLUEFlakes!

Snowflakes!

27 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public

Avoid GlueFlakes!

There’s something to this whole ‘user experience’

thing!

MicroServices Infrastructure release 0.3 TODAY!

we’re changing the name to something less sucky soon too!

28 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public

•  Cisco Microservices Environment based on Mesos. •  Fully open source.

•  Framework support in Mesos leveraged to deploy scalable data, monitoring and assurance services all-in-one.

•  We’ve designed, written and tested the glue.

•  ‘Dogfooding’ internally. We’ve built it to solve real problems.

•  Contribute vs Creating a new GlueFlake.

•  ‘Out of the Box’ environment for real-life workloads.

Enable the automatic deployment of development dependencies!

29 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public

So back to that network thing…

You guessed it… It’s going into MicroServices Infrastructure. Preconfigured to work with data and marathon workloads out of the box.

/ciscocloud/microservices-infrastructure

Roadmap updates and official announcements soon…

Thanks for Listening!

Questions?