Kubernetes CRI containerd integration by Lantao Liu (Google)

Kubernetes Container Integration - CRI-Container14 Sept, 2017

Lantao Liu <Random-Liu@github> Google Kubernetes

Abhinandan Prativadi <abhinandanpb@github> Docker

Contents

● Container Runtime Interface (CRI)

● CRI & Containerd

● CRI-Containerd

● Q & A

Contents

● Container Runtime Interface (CRI)

● CRI & Containerd

● CRI-Containerd

● Q & A

Container Runtime Interface

● What is Container Runtime Interface (CRI) ?

○ A gRPC interface and a group of libraries

○ Enables Kubernetes to use a wide variety of container runtimes

○ Introduced in Kubernetes 1.5

Kubelet

CRIgRPCclient

CRI shim

CRIgRPC

server

container runtime

containercontainer

containercontainer

container

Container Runtime Interface

● CRI Runtimes

○ cri-containerd: https://github.com/kubernetes-incubator/cri-containerd

○ cri-o: https://github.com/kubernetes-incubator/cri-o

○ Docker (Upstream):

https://github.com/kubernetes/kubernetes/tree/master/pkg/kubelet/dockershim

○ frakti: https://github.com/kubernetes/frakti

○ rktlet: https://github.com/kubernetes-incubator/rktlet

○ virtlet: https://github.com/Mirantis/virtlet

● CRI Tools https://github.com/kubernetes-incubator/cri-tools○ critest: CRI Validation Test Suite

○ crictl: CRI Command Line Tool

Contents

● Container Runtime Interface (CRI)

● CRI & Containerd

● CRI-Containerd

● Q & A

CRI & Containerd

● The scope of containerd 1.0 aligns with the requirement of CRI.

Name CRI Requirement Containerd 1.0 Scope

Container Lifecycle

Management

Create/Start/Stop/Delete/Lis

t/InspectIn

Image Management Pull/List/Inspect In

Networking

K8s handles pod and

service network, container

runtime SHOULD NOT

provide extra network

solution.

OutNo concrete network solution.

User can setup network

namespace, and put container

into it.

CRI & Containerd

● Continued

Name CRI Requirement Containerd Scope

Volumes

K8s manages volumes.

Container runtime SHOULD

NOT provide extra volume

support.

OutNo volume management. User

can setup host path, and

mount into container.

Persistent Container

Logging

K8s has specific

requirements for persistent

container logging, namely

format and path. Container

runtime SHOULD NOT

persist an unmanageable

log.

OutNo persistent container

logging. Container stdio is

provided as FIFOs, which can

be redirected/decorated as is

required.

CRI & Containerd

● Continued

Name CRI Requirement Containerd Scope

Metrics

K8s expects container

runtime to provide container

metrics (CPU, Memory,

Writable Layer Size etc.)

and image filesystem usage.

InContainerd provides these

metrics as part of the API.

CRI & Containerd

● Other alignments with Kubernetes:

○ Decentralized container management - containerd-shim.

■ Live restore.

■ Charge container management overhead to corresponding pod.

○ Decoupled image and container management.

■ Support other image formats (e.g. tarball)

○ Extensible image management:

■ Client-driven Image Download

■ Snapshotter

○ Support OCI image/runtime spec.

○ CNCF project.

○ ...

Contents

● Container Runtime Interface (CRI)

● CRI & Containerd

● CRI-Containerd

● Q & A

CRI-Containerd

● cri-containerd: A containerd based implementation of CRI.

○ https://github.com/kubernetes-incubator/cri-containerd

○ Kubernetes incubator project.

○ Started in April 2017.

containercontainerKubelet dockershim dockercontainer

d containercontainer

CRI

containercontainerKubelet cri-containerd

containerd

containercontainer

CRI

dockershim

cri-containerd

CRI-Containerd Architecture

Pod B

Pod A Cgroups

Pod A Namespaces

cri-containerd

sandbox container

containerd shim

image service

runtime service

Kubelet

gRPCClient

containerd shim

container A

ocicni

CRI-Containerd Status

● Dependencies:

○ Kubernetes: >= v1.7

○ containerd: v1.0.0-beta.0 (daily/weekly update)

○ CNI: v0.6.0 (Spec Version: 0.3.1)

● All features in CRI (K8s 1.8) supported other than 3 missing features being added soon:

○ Seccomp. Under review.

○ Metrics. Kubelet and containerd changes merged, cri-containerd support in 1-2 weeks.○ Mount Propagation (K8s 1.8): Under review.

● 37/37 CRI validation tests passing (Per-PR test)

● 180/182 node e2e tests passing (Per-PR test)○ Running the same set of K8s PR node e2e test.

○ 2 test failures:

CRI-Containerd Plan for 2017

● 1.0.0-alpha.0 by the end of September.○ Feature Complete.

○ All Kubernetes PR node e2e tests passing.

○ Use kubeadm to bring up Kubernetes cri-containerd cluster.

○ Release tarball for distribution and installation.

● Q4: Additional testing, bug fixes and documentation.○ Test: Setup FULL SET of node/cluster e2e test in Kubernetes test infra.

■ Suites: Slow, Serial, Reboot, Performance etc.

■ OS: Ubuntu, COS (Container-Optimized OS) etc.

○ Ease of use:

■ Documentation.

■ Polish kubeadm integration.

■ kube-up.sh integration.

○ 1.0.0-beta.0 by the end of 2017.

Demo

● Demo Focus:

○ Installation

○ Cluster lifecycle

○ Networking

○ Namespace management

○ Mount Propagation

○ Pod operations

○ Sample Demo App

Recap

● CRI is the standard way to integrate Container Runtime with

Kubernetes.

● New containerd matches CRI and Kubernetes’ requirement very well.

● CRI-Containerd 1.0.0-alpha.0 is releasing by the end of September. It

will be feature complete.

Links

● Github: https://github.com/kubernetes-incubator/cri-containerd

● Slack: https://kubernetes.slack.com/messages/sig-node

● Mailing List: https://groups.google.com/forum/#!forum/kubernetes-

sig-node

● Maintainers:

○ Lantao Liu <[email protected]>

○ Abhi Prativadi <[email protected]>

○ Mike Brown <[email protected]>

Q & A