Continuous Delivery of Cloud Applications with Docker Containers and IBM Bluemix

ContinuousContinuousDelivery of CloudDelivery of Cloud

ApplicationsApplicationswith Docker Containers

and IBM Bluemix

Florian Georg

IBM Cloud Advisor

@florian_georg | #ibmcloudadvisor

[email protected]

IBM CloudIBM CloudAdvisorsAdvisors

World-wide Expert Team"All things Cloud"

Call us, we won't charge you

Continuous Delivery,Continuous Delivery,Containers,Containers,PaaSPaaS

Benefits ofBenefits ofContinuousContinuous

DeliveryDeliveryValidated Learningbuild the right product Increase stability, reducerisk Real project progress(done-done)

Deliver SoftwareDeliver Software

Dark LaunchingCanary ReleasesFeature TogglesGreen/Blue Deployments...

Why Containers ?Why Containers ?Portable Workloads (Workstation to Cloud)Proven technology, lightweight, fast, easy to useSeparation of Concerns + common interfaces

CodeProcessesPackage Managers...

Lifecycle ManagementLoggingMonitoringNetworking

Dev Ops

Linux-based, isolatedinstances

Common commands for alltypes of images / containers

Why a CloudWhy a CloudPaaS ?PaaS ?

Build software, not tech stacksSecurity + SpeedElastic scaling and billingCompose applications servicesEnable modern ArchitecturePatterns (Microservices etc.)

"The entire history of

is that of the rise inlevels of abstraction."

- Grady Booch

softwareengineering

https://developer.ibm.com/bluemix/docs/actionable-architecture-building-web-application-hosting-containers/

http://www.cloud-council.org/

Web App Reference ArchitectureWeb App Reference Architecture

CloudCloudFoundryFoundryOverviewOverview

World's largest publicCloudFoundry instanceIBM invested > 1 Billion$100+ Services(IBM / 3rd Party)Significant free tiers - e.g. runsmall apps completely free,forever http://bluemix.nethttp://bluemix.net

IBM ContainerIBM ContainerServicesServices

IBM Containers (beta)IBM Containers (beta)

Private Docker RegistryTrusted Images (Node, Java Liberty)Container Groups (scale, failover)Monitoring, logging (logstash)Run on Bare-Metal Servers (latency)Public IPsBind to Bluemix CloudFoundry PaaSServices

http://ice.mybluemix.net/

Web DashboardWeb Dashboard

Scoping according to CloudFoundry concepts (org/space)Manage containers/container groups through dashboard

"ice""ice" Command Line tool Command Line tool

Install & usage docs: https://www.ng.bluemix.net/docs/starters/container_cli_ov.html#container_cli_ov

Win, Linux, Macsome prereqs

Python 2.7CloudFoundryCLI ("cf")Docker

# install latest version (3.0) of CLI > sudo pip install https://static-ice.ng.bluemix.net/icecli-3.0.zip[...]

> ice infoDate/Time : 2015-06-17 11:43:30.491651Debug mode : FalseCCS host/url : https://api-ice.ng.bluemix.net/v2/containersRegistry host : registry-ice.ng.bluemix.netBluemix api host/url : https://api.ng.bluemix.netBluemix Org : Florian Georg - demo (f36d3cd8-70ba-4d96-a571-f46a451bdaaf)Bluemix Space : dev (291b7833-0fb5-484e-95af-5bfc0e252080)ICE CLI Version : 3.0 481 2015-06-16T21:08:26CCS API Version : 2.0 1120 2015-06-16T16:52:41Repository namespace : faxgContainers limit : UnlimitedContainers usage : 5Containers running : 4CPU limit (cores) : UnlimitedCPU usage (cores) : 6Memory limit (MB) : 2048Memory usage (MB) : 1536Floating IPs limit : 2Floating IPs allocated : 1Floating IPs bound : 1

Login Login

# login and (optionally) select a target space for running containers> ice login --space dev[...]

# Set repository namespace. This needs to be done only once> ice namespace set myNamespace

Authenticate to cloud service + docker registrySSO with your Bluemix accountCloudFoundry Scoping Model

Image registry per "Organization"Containers run in "Spaces"

You must create & set a unique repository namespace once "ice namespace set <myNamespace>"

Pull Docker Image Pull Docker Image

# pull image from remote registry> ice --local pull registry-ice.ng.bluemix.net/ibmnode

# you could also copy from public registry like dockerhub (ice >= 3.0)> ice cpi ansi/mosquitto registry-ice.ng.bluemix.net/faxg/mosquitto

Image naming:

registry-ice.ng.bluemix.net/<trustedImageName>registry-ice.ng.bluemix.net/<namespace>/<imageName>

(public trusted images currently 'ibmnode' and 'ibmliberty')

"ice cpi" is combined pull - tag - push

Change, Commit & Tag Image Change, Commit & Tag Image

# run a container and make some changes> ice --local run registry-ice.ng.bluemix.net/ibmnode apt-get update -y[...]

# commit changes to container (id #1234...) into a new (local) image> ice --local commit 1234 myAppImage

# tag the new image with into your remote repository > ice --local tag myAppImage registry-ice.ng.bluemix.net/myNamespace/myAppImage

Before pushing images to your remote repository, you must tag themlike this:ice --local tag <localImageName> registry-ice.ng.bluemix.net/<Namespace>/<remoteImageName>

Push image & run in the CloudPush image & run in the Cloud

# push the tagged image to your remote repository on Bluemix> ice --local push registry-ice.ng.bluemix.net/myNamespace/myAppImage

# run a new container on the cloud> ice run --name myAppContainer --memory 256 myNamespace/myAppImage echo "Some command"

# retrieve stdout / logs> ice logs myAppContainer

Containers have "t-shirt sizes" (small, medium, large...)combination of memory and local storageSpecify --memory <MB> from CLI, or use web console

Request & Bind Public IPRequest & Bind Public IP

# request a public IP> ice ip requestSuccessfully obtained ip: "129.41.232.25"

# bind floating IP to container> ice ip bind 129.41.232.25 myAppContainer

You have a quota of public IP adressesUnbound IPs are "floating" in your pool until releasedContainers also have a private IP on the same network

Create & mount storageCreate & mount storage

> ice volume create myStorage> ice run --name containerWithVolume --volume myStorage:/var/images myNamespace/myAppImage

Permanent storage (as opposed to container "storage" !)does not get deleted with containermust be mounted on container startup ("run")

Bind to CloudFoundry ServicesBind to CloudFoundry Services

> ice run --name myAppContainer --bind myDB myNamespace/myAppImage

Platform injects VCAP_SERVICES environment variableUse Bluemix Services in your containerYou need a CloudFoundry "proxy app" that binds to the services

Create Container GroupsCreate Container Groups

# Create a load-balancing container group (open Port 80, auto-recovery, 2 containers)> ice group create -p 80 --auto --desire 2 --name myGroup myNamespace/myImageName

# create a route (http://groupRoute.mybluemix.net) for this container group> ice route map --hostname groupRoute --domain mybluemix.net myGroup

Container groups load-balance incoming traffic to a set ofcontainersRestart crashed containers with --auto--desire <numContainers>Map external URL route

Key FeaturesKey Features

Private registry w/ access controlsPush / Pull images between on-prem and off-prem registriesDocker CLI compatible (reuse existing tools)Load-balance + auto-recovery (container incl. running services)Easy public IP bindingContainer-level logging & monitoringContainer-level private, secure networking (IP-based)Container-level attachable storage Bind containers to Bluemix PaaS ServicesIntegrated CI/CD + operation lifecycleIntegration with Bluemix Router (DNS, load-balance etc.)

Delivery PipelineDelivery Pipelinewith Green/Blue(*) Deployments

(*) called "red_black" in our scripts

IBM DevOps Services (IDS)IBM DevOps Services (IDS)Delivery PipelineDelivery Pipeline

Cloud IDE (Code, Build, Tracking):Cloud IDE (Code, Build, Tracking):http://hub.jazz.nethttp://hub.jazz.net

Stage 1: BuildStage 1: Build

Triggered on push to connected Git repo/branchYou need a DockerfileYou provide the imageName to useTags the new image into your private Docker repository Version tag will be set to the build# (from Jenkins)(you would want to add some unit testing job here)

# FULL_REPOSITORY_NAME is something like # "ice-registry.ng.bluemix.net/<Namespace>/<imageName>:<version>"

ice build --pull --tag ${FULL_REPOSITORY_NAME} ${WORKSPACE}

Pseudocode

Stage 2: StagingStage 2: Staging

Triggered on successful "Build" stageGets artifacts + variables from build stageClones deployscripts from external repoClean deploy as single container into staging spaceUpdates internal IDS inventory

# Clone deployscripts into workspacegit_retry clone https://github.com/Osthanes/deployscripts.git deployscripts

# Deploy "clean" as single container into staging space# /bin/bash deployscripts/deploycontainer.sh :

# clean up previous containersice rm -f ${CONTAINER_NAME}_${PREVIOUS_VERSION}

# start new containerice run --name ${CONTAINER_NAME}_${VERSION} ${ICE_ARGS} ${IMAGE_NAME} wait_for_startup()

# reclaim floating IP & bind to current container versionice ip unbind ${FLOATING_IP} ${CONTAINER_NAME}_${PREVIOUS_VERSION}ice ip bind ${FLOATING_IP} ${CONTAINER_NAME}_${VERSION}

# update IDS inventory[...] Pseudocode

Stage 3: ProductionStage 3: Production

Triggered manually or autostarts a new container groupzero downtime "cut-over" by re-mapping URL routeYou may keep some "old" groups running for rollback

# Clone deployscripts into workspacegit_retry clone https://github.com/Osthanes/deployscripts.git deployscripts

# Deploy "red_black" as container group into production space# /bin/bash deployscripts/deploygroup.sh :

# create new container group & wait until upice group create --name ${GROUP_NAME}_${VERSION} ${PARAMS} ${IMAGE_NAME}wait_for_startup()

# map DNS route to new group, zero downtime cut-over ice route map --hostname $HOSTNAME --domain $DOMAIN ${GROUP_NAME}_${VERSION}

# clean up version-2 container groupice group rm -f ${GROUP_NAME}_${VERSION - 2}

# update IDS inventory[...]

Pseudocode

DemoDemo(*)(*)

(*) https://www.youtube.com/watch?v=5NRHVtguODM

Confused about Confused about "The Cloud" ?"The Cloud" ?