Sf bay area Kubernetes meetup dec8 2016 - deployment models

Copyright © 2015 Mirantis, Inc. All rights reserved

www.mirantis.com

Kubernetes deployment models(modelling complex applications in K8S)

Dec8 2016Piotr Siwczak(https://www.linkedin.com/in/psiwczak)


Agenda

● Challenges in managing complex microservice architectures

● What’s missing in K8S to manage complex microservice architectures efficiently

● K8S AppController as an enhancement to handle complex architectures

● Demo of AppController

● Q&A about AppController


Challenges in managing microservice architectures


Linux as a microservice apps platform

Collection of small, independent programs acting together to form larger systems.

Programs communicate over standardized protocols/API-s

Abstracts computing resources (kernel)


Unix/Linux simplified architecture

Kernel

Libraries

Init system Interactive shell

Userspace apps

Resource access

Orchestration

User functionality


The role of init system

SysVInit … Upstart … Systemd

Init makes sure that apps start in proper order and deps for them are handled

e.g.

Network subsystem -> Iptables -> ssh


Apps - combined K8S resources

Kubernetes clients (kubectl...)

Kubernetes resources (pod, service…)

Unix/Linux vs K8S

Kernel

Libraries

Init system Interactive shell

Userspace apps

?


Do we have init equivalent in k8s?

Original photo by https://www.flickr.com/people/waferboard/ License: CC BY v2.0, Title: “wholesome stirfry”

“Containers start in parallel after volumes are

mounted, leaving no opportunity for

coordination between containers...”

https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/container-init.md

https://www.flickr.com/people/waferboard/

https://creativecommons.org/licenses/by/2.0/legalcode

https://www.flickr.com/photos/waferboard/3845324613/in/photolist-6RNi3e-a7CQ1i-bXA1Cb-a7FMXW-a7FHgN-5HMcuh-o1JfR8-bEPxrg-8FV8Au-bEsYWV-i1DT9-9MqUiK-anmmQc-bfzLbR-5JsMuP-9LNuyi-4QcG7h-7nn6Xx-4w56bJ-pajWBi-oYw9Dt-23mSwf-a3HY7H-a1UmEF-gbuRB-4qHDSS-by9Ktx-ebtkfu-dPpVaw-nGMssa-5CHCry-48mjRR-5cGaaS-qYXYbL-9yr2o8-4Atoik-99jiUB-iLLJfi-a6sEs4-qY1pWs-8jniNj-4Fmdky-dJCLeR-aA4suz-pNnyFi-pSrkMW-a9Vn2Q-x9SKM-sfg9b-dfTwbA





...to make a good dish one needs to follow steps

Original photo by https://www.flickr.com/people/waferboard/ License: CC BY v2.0, Title: “wholesome stirfry”

● heat oil● add garlic● add veggies and sauce● add meat

https://www.flickr.com/people/waferboard/

https://creativecommons.org/licenses/by/2.0/legalcode

https://www.flickr.com/photos/waferboard/3845324613/in/photolist-6RNi3e-a7CQ1i-bXA1Cb-a7FMXW-a7FHgN-5HMcuh-o1JfR8-bEPxrg-8FV8Au-bEsYWV-i1DT9-9MqUiK-anmmQc-bfzLbR-5JsMuP-9LNuyi-4QcG7h-7nn6Xx-4w56bJ-pajWBi-oYw9Dt-23mSwf-a3HY7H-a1UmEF-gbuRB-4qHDSS-by9Ktx-ebtkfu-dPpVaw-nGMssa-5CHCry-48mjRR-5cGaaS-qYXYbL-9yr2o8-4Atoik-99jiUB-iLLJfi-a6sEs4-qY1pWs-8jniNj-4Fmdky-dJCLeR-aA4suz-pNnyFi-pSrkMW-a9Vn2Q-x9SKM-sfg9b-dfTwbA


K8S challenge for complex apps

(T3) wordpress depl/service

(T2) mysql depl/service

(T1) mysql password

password

db dns name & password

kubectl create -f mysql-pass.yaml

kubectl create -f mysql-deployment.yaml

kubectl create -f wordpress-deplyment.yaml


Is lack of deps really a problem for K8S?

We all know microservices are supposed to orchestrate themselves and tolerate failures

...but…


...deps seem to be needed

https://blog.xebialabs.com/2015/04/13/before-you-go-over-the-container-cliff-with-docker-mesos-etc-points-to-consider/

“"A common definition for a microservice we often hear mentioned is an “independently-deployable unit”, and indeed it is good practice to design your

microservices so they can start up successfully without requiring all kinds of other components to be available. But in the vast majority of cases, “no microservice is an

island”...






Docker-compose and Mesos application groups are here and being used:

https://docs.docker.com/compose/gettingstarted/

https://mesosphere.github.io/marathon/docs/application-groups.html







https://github.com/vishnubob/wait-for-it





https://github.com/mesosphere/kubernetes-mesos/issues/119

http://stackoverflow.com/questions/27701994/specify-order-dockers-run-on-kubernetes-pod

https://github.com/kubernetes/kubernetes/issues/29804









https://github.com/Mirantis/k8s-AppController

AppController




● way to express dependencies between K8S objects,

● thus allowing to deploy complex, multi-tier applications in fully automated fashion

● k8s object dependency graph:● definitions (nodes)● dependencies (edges)

AppController - really short summary...


Before…

● kubectl create -f t1.yaml● check status…● kubectl create -f t2.yaml● check status…● kubectl create -f t3.yaml● ….

After…

● kubectl create -f graph.yaml

● k8s-appcontroller ac-run


AppController architecture

Kubernetes

k8s-appcontroller pod

kubeac binary k8s API extensions

3rd party resources:dependency

definition


workflow

AppController


AppController workflow - definitions

Definition

Standard K8S resource

(deployment, pod, service…)

Definition



Definition



Definition



kubectl create -f definitions.yaml


AppController workflow - definitions

apiVersion: appcontroller.k8s/v1alpha1

kind: Definition

metadata:

name: secret-mysql-pass

secret:

apiVersion: v1

data:

password.txt: cXdxd3F3

kind: Secret

metadata:

creationTimestamp: 2016-12-06T16:56:02Z

name: mysql-pass

namespace: default

secret/mysql-pass


(secret)

Objects are not created in k8s until triggered by AppController!


Definitions - summary

Definition:

● “node” in the graph● wrapper over regular k8s resource● defers the creation of the resource until triggered (in

contrary to “kubectl create -f” which creates the resource immediately


AppController - dependencies

Definition



Definition



Definition



Definition



start end

kubectl create -f deps.yaml


AppController - dependencies

apiVersion: appcontroller.k8s/v1alpha1

kind: Dependency

metadata:

name: mysql-pass--to--mysql-deployment

parent: secret/mysql-pass

child: deployment/mysql

secret/mysql-pass


(secret)

deployment/mysql


(deployment)


Dependencies - summary

Dependency:

● “edge” in the graph● links definitions together● provides the sense of dependency between definitions


AppController - application rollout

Definition



Definition



Definition



Definition



start end

kubectl exec k8s-appcontroller ac-run

kubectl exec k8s-appcontroller kubeac get-status


AppController workflow - summary

● “wrap” regular k8s resources into definitions > defs.yaml● load defs.yaml to k8s

● create dependencies between resources > deps.yaml● load deps.yaml into k8s

● trigger the app deployment from AppController application (kubectl exec k8s-appcontroller ac-run)


MySQL + Wordpress

Demo


def_db_password

AppController - wordpress deployment

secret: db_password

def_db_deploymtdeployment:

mysql-deployment

def_db_service

service:db_service

def_wp_deploymt

deployment:wordpress-dep

loyment

def_wp_service

service:wordpress-ser

vice

DB_PASS

DB_HOSTNAME, DB_PASS


Questions/Answers


Q&A

Q: What if I abort the deployment in the middle - how does AppController recover

from partially provisioned graph?

A:AppController will check the status of already provisioned resources. Will only

provision the ones which are absent


Q&A

Q:Can I run multiple AppControllers on a single K8S ?

A:You can run 1 AppController per namespace


Q&A

Q:What K8S resources can be currently wrapped into definitions?

A:- Daemonset

- Job- Petset- Pod

- Replicaset- Service

- ConfigMap- Secrets

- Deployments


Q&A

Q:How is provisioning of resources validated?

A:Status of the k8s resource is checked.

AppController implements also some checks of its own (e.g. for replicasets readiness probe is based on “success factor” or all resources ready. Success

factor is a part of appcontroller and for services we are checking service selector and see if the backends are ready - e.g. replica sets)


Q&A

Q:What’s on the roadmap?

A:Graph notifications, reactions, error handling

More resources supported

Usability improvements

Better documentation (incl. real-life complex examples)


Q&A

Q:How AppController is different from Init Containers?

A:Supports more complex deployments (complex graphs)

Handles deps not only between containers

Checks resource states (no need to implement custom probes in the container)

Keeps debug logs in one place

Can react to changes in the graph


Recording

https://www.youtube.com/watch?v=7GSwSTtBAYo&utm_content=38600000





Thank you!

Sf bay area Kubernetes meetup dec8 2016 - deployment models

Technology