Kubernetes - training micro-dragons without getting burnt

Kubernetes - training micro-dragons without getting

burnt

Amir MoghimiSenior Consultant in managed services

Sixtree, Australia

Traditional approach

Micro-services approach

Train microservicesMicroservices architecture magnifies the need for:● Fairly homogenous build artifacts● Standard running platform● Configuration and secret management● Service Discovery

Polyglot programming● Pick right tool for the job● Multiple teams with different expertise/perspectives● Keep developers busy learning new language(s)

Homogenous build artifactsBuild artifacts:

● Java Jar and War files● Ruby Gems and Rails apps● Node packages and apps● Go binaries

Containerise everything (Docker):

● Universally deployable artifact

DockerfileFROM debian:jessie

ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/jre

RUN apt-get update \

&& apt-get install -y \

openjdk-8-jre-headless

COPY my-app.jar /my-app.jar

ENV MY_APP_CONF_VAR super-cool-default-value

CMD [“java”, “-jar”, “/my-app.jar”]

docker build -t registry/image_name .docker push registry/image_name

Configure and rundocker run -d \

-e REDIS_NAMESPACE='staging' \

-e POSTGRES_ENV_POSTGRES_PASSWORD='foo' \

-e POSTGRES_ENV_POSTGRES_USER='bar' \

-e POSTGRES_ENV_DB_NAME='mysite_staging' \

-e POSTGRES_ADDR='docker-db-1.us-east-1.rds.amazonaws.com' \

-e SITE_URL='staging.mysite.com' \

-p 80:80 \

--restart=on-failure:10 \

--name container_name \

registry/image_name \

image_command cmd_arg1 cmd_arg2

Configuration hell● Application config

○ Env vars, config files, cmd line args

● Runtime environment config○ Web server, JVM

● Runtime dependencies config○ Volumes, logging, monitoring, stats

Configuration management● Train your app:

○ 12-factor app

● Configuration in a containerised world:

○ Log to stdout○ Port mappings (from host to container)○ SaaS blob storage (mount volumes only if providing a storage service)○ Service discovery (Consul, Eureka, DNS)○ Secrets (ideally only in memory but how?)○ Environment Variables for everything else

Configuration management tools● Docker compose

○ State management? Templating? Secrets? Service discovery? Cluster-level volumes?

● Ansible Role (classic host-based approach + docker module)

● Kubernetes (container PaaS)

Kubernetes key resources● Namespace● Pod (container)● Replica Set● ConfigMap● Secret● Service● Deployment

Kubernetes Master

API Server Replica Set

kubelet Node

Pod

Container

Pod

Container

kubelet Node

Pod

Container

Kubernetes Cluster

= Label

= Resource

= Process

Replica Set (Replication Controller)apiVersion: v1

kind: ReplicationController

metadata:

name: my-nginx-replica-set

spec:

replicas: 3

selector:

app: dragon-web

template:

metadata:

name: nginx-pod

labels:

app: dragon-web

spec:

containers:

- name: nginx-container

image: nginx

env:

- name: LOG_LEVEL

value: INFO

ports:

- containerPort: 80

apiVersion: v1kind: Pod

kubectl create -f my-nginx-replica-set.yml

ConfigMapapiVersion: v1

kind: ConfigMap

metadata:

name: dragon-config

labels:

environment: non-prod

data:

dragon.how.much: very

dragon.type: fast

apiVersion: v1kind: Podmetadata: name: dragon-podspec: containers: - name: dragon-container image: dragon-image env: - name: DRAGON_LEVEL valueFrom: configMapKeyRef: name: dragon-config key: dragon.how.much - name: DRAGON_TYPE valueFrom: configMapKeyRef: name: dragon-config key: dragon.type

SecretapiVersion: v1

kind: Secret

metadata:

name: my-secret

type: Opaque

data:

password: MWYyZDFlMmU2N2RmCg==

username: my_admin

apiVersion: v1

kind: Podmetadata:

name: secret-user-pod

Spec:

volumes:

name: secret-vol

secret:

secretName: my-secret

containers:

- name: nginx-container

image: nginx

volumeMounts:

name: secret-vol

mountPath: /etc/my-access-keys

readOnly: true

Service{

"apiVersion": "v1",

"kind": "Service",

"metadata": {

"name": "my-service"

},

"spec": {

"selector": {

"app": "dragon-web"

},

"ports": [{

"protocol": "TCP",

"port": 80,

"targetPort": 80

}]

}

}

Service discovery● DNS

○ Take extra care when playing with fire○ No control over client○ Time sensitive protocol○ Use only if you have a reliable DNS service, i.e. AWS Route53

● Provided environment variables○ MY_DROGON_SERVICE_HOST=10.0.0.11

MY_DROGON_SERVICE_PORT=8080○ Create services before using them in pods○ Only works per namespace

● Kubernetes REST API○ GET /api/v1/namespaces/{namespace}/services/{service_name}

DNS HAZARD

DeploymentapiVersion: extensions/v1beta1

kind: Deployment

metadata:

name: nginx-deployment

spec:

replicas: 3

template:

metadata:

labels:

app: nginx

spec:

containers:

- name: nginx

image: nginx:1.7.9

ports:

- containerPort: 80

DeclarativeServer-sideRevision trackingEasy rollback

Project structuremy-dragon-microservice/

src/kube-resources/

default-configmap.ymldefault-secret.ymlservice.ymldeployment.yml

pipeline-resources/build.shtest.shdeploy.sh # kubectl apply -f ../kube-resources

pipeline.ymlDockerfile

Environments configdev-1/

namespace.ymlconfigmap.ymlsecret.yml

dev-2/… dev-N/qa-1/

namespace.ymlconfigmap.ymlsecret.yml

qa-2/… qa-N/prod-1/

namespace.ymlconfigmap.ymlsecret.yml

prod-2/… prod-N/

Demo