Kubernetes networking Introduction to overlay networks, communication models and implementation [April 26 2016] [ Murat Mukhtarov ] Zendesk
Jan 07, 2017
Kubernetes networking
Introduction to overlay networks, communication models and implementation
[April 26 2016][ Murat Mukhtarov ]
Zendesk
Contents
2
● Overlay networking introduction○ Overlay concept○ Overlay network example: VXLAN
● Linux namespaces● Kubernetes networking○ Kubernetes and Docker networking comparison○ Service abstraction○ Inter POD communication○ Flannel example
● Q&A and links
Overlay networks: it is not a new paradigm
3
Stacking nature of networking allows encapsulation of different protocol stack at each network layer:- Datalink could be encapsulated in datalink, IP,
transport (e.g. UDP, IPSEC)- IP could be encapsulated in IP, MPLS, Datalink- MPLS allows simple stacking for complex
service-oriented topologies
Overlay networking concept
4
Overlay networks - encapsulation of the full tcp/ip stack including layer 2 inside transport network (UDP datagrams)
Overlay example: VXLAN
5
VXLAN overlay networking technology that allows to send Ethernet traffic encapsulated into UDP datagrams
over IP/GRE networks. Detailed description of VXLAN networking could be found in RFC7348
24 bit VNI field is VXLAN address field that could be
compared with 802.1q tag for Ethernet frames or MPLS
label.
Bare in mind MTU value when using VXLAN
Linux network namespaces
6
Network namespaces is a part of containerization technology that used by Linux kernelNetwork namespaces allows:
○ To create linux container network isolation instances (namespaces)
○ With own routing table, virtual interfaces, L2 isolation
● The tool that is used to operate with network ns: iproute2
● Network namespaces are stored in ○ /var/run/netns
● There two types of network namespaces:○ Root namespace [ ip link ]○ Non-root namespace [ ip netns .. ip link ]
Kubernetes: service abstraction
8
Service-oriented model:- POD represents group of containers in the
same namespace- Service represents load-balancing group of
PODs- Service could be mapped to overlay
network number, e.g. VXLAN number
Kubernetes networking model provides flexibility in terms of:- IP addressing: routable address per pod- Organizing networks: Flannel, OVS and etc.- Customising plugins: e.g. CNI- Docker0 serves as L3 (IP) gateway for PODs
behind it
Inter POD communication with overlay: general principle
9
Communication can be implemented as L2 and L3, it totally depends on chosen network model.
This particular example assumes that DHCP server resides somewhere in the network where overlay subnets are terminated.
Further flexibility could be achieved with CNI.
Inter POD communication: other examples
10
Flannel OVS
https://github.com/coreos/flannel#flannel http://kubernetes.io/docs/admin/ovs-networking/
Flannel example explanation
11
● Flannel creates overlay network on your choice:○ UDP or VXLAN encapsulation
● Flannel creates interface ○ flannel.VNI
● where VNI is number that you specified in json payload.
● Flannel interface is being assigned with ip address 10.1.X.0/16 address, where X - is random subnet number.
● Docker0 interface is assigned with address 10.1.x.1/24 all hosts behind docker0 are assigned with 10.1.x.2 - 254 addresses with 24 bit mask.
● Nodes are talking each other on switched 192.168.1.0/24 network
Apparently scaling can be questionable if we have more than 150-200 nodes in the network
Links:Kubernetes network design
https://github.com/kubernetes/kubernetes/blob/release-1.2/docs/design/networking.md
Kubernetes with OVShttp://kubernetes.io/docs/admin/ovs-networking/
Kubernetes with Flannelhttps://github.com/coreos/flannel#flannel
Calico BGP projecthttps://github.com/projectcalico/calico-containers
BaGPipe BGP CNI plugin:https://github.com/murat1985/bagpipe-cni