GovCMS: Lagoon ahoy!amazee.io... · History of amazee.io STORY TIME 2010 2011 2012 2013 2014 2015 2016 2017 2018 Need hosting in CH Configuration Mgmt “How do you host?” amazee.io

GovCMS: Lagoon ahoy!Auckland - May 2019

Agenda

1. Introduction to Lagoon

2. Introduction to the development platform

3. Demonstration

4. Q&A

Open Platform Technology: Lagoon,

Containerisation & Openshift

History of amazee.ioSTORY TIME

2010 2011 2012 2013 2014 2015 2016 2017 2018

Need hosting in CH Configuration Mgmt “How do you host?” amazee.io

History of LagoonSTORY TIME

... 2014 2015 2016 2017 2018

Local Docker

Decoupled!?

Open Source FTW!

Vagrant 😢🧐💡

Docker in Production!

Aug 2017: Lagoon is Open SourceSTORY TIME

Open Sourcevs.

Proprietary Hosting

Proprietary Hosting

Proprietary Hosting

✅ Drupal

Open Sourceness of Drupal Hosting

❌ Hosting Platform

✅ PHP

✅ Nginx

✅ MySQL

✅ Operating System

✅ Linux Kernel

● All Docker Images

● All Service Configurations

● Build & Deploy Scripts

● Test Infrastructure

● Apache 2.0 License

Lagoon is fully Open Source

● OpenShift

● Kubernetes

● Alpine Docker Images

● Node.js, Golang, PHP

● Elasticsearch, Kibana, Logstash, RabbitMQ, OpenSSH

● Varnish, Nginx, MariaDB, Postgres, Redis, Solr, MongoDB,

Lagoon uses Open Source

GovCMS 2.0: TL;DR;

GovCMS 2.0: Too Short; Want More.

Lagoon OpenShift / Kubernetes

Local

Develop with docker-compose ● Reads

docker-compose.yml

● Build Images

● Setup OpenShift Projects

● Push Images to OS Registry

● Configure Resources

Container Orchestration

Git push

Webhooks Notifications

● Monitor Deployments

Microservices

Runs inside OpenShift

Lagoon deploys Lagoon

Lagoon is fully tested

1. Ansible

2. Starts new OpenShift & Lagoon

3. Creates fresh git repo

4. Commits Node.js code with random string

5. Let's Lagoon deploy from that git repo

6. Checks if deployed App contains random string

7. Rinse and Repeat for:

○ Drupal 7, Drupal 8, MariaDB, Postgres, Elasticsearch, Drush, and and and

Lagoon runs Docker

● Whatever you can put into Docker

○ It deploys with Lagoon

● Existing Templates for

○ Drupal (Varnish, Nginx, PHP, Solr, Elasticsearch, Redis, MariaDB, Postgres)

○ Wordpress, Silverstripe, TYPO3 ...

○ PHP Frameworks (Symfony, Laravel)

○ ELK (Elasticsearch, Logstash, Kibana)

Summary of Lagoon BenefitsA FEW REASONS TO USE LAGOON

● Lagoon is Battle tested

● Lagoon is and leverages Open Source

● Lagoon is fully automated tested

● Lagoon allows congruent local development

● Lagoon is and runs Docker

Wait! Containers? Images? Pods? Registries? OpenShift? Kubernetes?

Container ImagesUNDERSTANDING CONTAINERS

● Container Image contains an application and its dependencies

● Independent on Kernel Version (runs everywhere)

● Logically similar to VM Image but with very different implementation

● Immutable. Rebuild instead of Update an existing one

Container ImagesUNDERSTANDING CONTAINERS

● Container Images are based on another Container Image

PHP Container Image Drupal Container ImageAlpine Container Image

basedon

basedon

ContainersUNDERSTANDING CONTAINERS

● Containers are created from Container Images

● Many Containers per Images possible

● Creation of Container takes Milliseconds

Container Image Container

Container

Container

Container Image RegistriesUNDERSTANDING CONTAINERS

● Container images are pushed to a Container Image Registry

Container Image

Container Image Registry

Container OrchestratorMANAGING CONTAINERS

● Container Orchestrator manages the container lifecycle over a distributed set of worker nodes

● Images are pulled from a Container Image Registry

● The container Orchestrator provides:

○ Service Discovery○ Self healing○ Networking○ Storage○ Routing○ Resource allocation

Container Image Registry

PodsMANAGING CONTAINERS

● Pods keeps Containers together that should stay together

● Example: Nginx & PHP (both have Drupal Code in them)

PHP Container

Nginx/PHP Pod

Nginx Container

MariaDB Pod

MariaDB Container

ApplicationsMODELLING APPLICATIONS AS SETS OF CONTAINERS

● Applications are composed of one or more Containers● Container Orchestrator understands dependencies● The services provided by the Container Orchestrator allow the Containers to

○ Find and communicate with each other○ Share storage○ Etc.

Environments aka Namespaces MAPPING CONTAINERS INTO ENVIRONMENTS USING NAMESPACES

● Kubernetes: NameSpace / OpenShift: Project● A NameSpace contains one or more Pods and Resources● A NameSpace can be thought of as an environment

ScalingFLEXIBLE CONTROL OF WORKLOAD SIZE AND REDUNDANCY

● Pods can be automatically scaled

● Redundancy & Performance at the same time

CI/CDA NEW PARADIGM FOR DEVELOPMENT AND DEPLOYMENT

● Each Git Branch can be deployed as an Environment/NameSpace

● Same for Pull-Requests

ClustersFULL CLUSTER FOR FULL HA AND PROPER SLAS

● Masters manage Pods, Resources and Nodes (Orchestrators)● Etcd is the Master Database● Scaling happens on Worker/Compute Node

OpenShift vs KubernetesOPENSHIFT = ENTERPRISE KUBERNETES

Kubernetes OpenShift

“Vanilla Linux Kernel” “Linux Distribution”

No Support Enterprise Support

“Self Service” All Inclusive

Upgradeable Full Upgrade Support

Moves FastMoves more predictable

Maintaining GovCMS project

Required

● Docker

● pygmy - multi-purpose Docker helper

Optional

● Ahoy - workflow helper

Host dependenciesWHAT NEEDS TO BE INSTALLED ON HOST TO DEVELOP A GOVCMS PROJECT

● Docker images are built with all development tools installed during the image build process

● Development tools:

○ Parallel PHP lint

○ PHP Code Sniffer

○ Behat

○ PHPUnit

Toolset is inside containersDEVELOPMENT ENVIRONMENT HAS ALL TOOLS INBUILT

1. CI configuration stored in image repository

2. Runs in GitLab on code changes in `master` branch and PRs

3. Publishes images to Dockerhub

GovCMS images CI AUTOMATED IMAGE TESTING AND PUBLISHING

GovCMS image releaseCONTROLLED IMAGE RELEASE WORKFLOW

● Pushing tagged commit triggers deployment of ‘beta’ images to Dockerhub

● After manual review, images promoted to ‘latest’

● `ahoy release`:

1. Pulls latest ‘beta’ images from Dockerhub

2. Tags images as ‘latest’

3. Pushes images to Dockerhub

Improved DXSMOOTH DEVELOPMENT WORKFLOW

Ahoy wraps complex CLI calls into simplified commands:

build Build project.cli Start a shell inside CLI container.drush Run drush commands in the CLI service container.install Install the profile.lint Lint code.login Login to a website.logs Show Docker logs.make Install local dependencies.ps List running Docker containers.pull Pull latest docker images.push Push all docker images.release Push all docker images.restart Restart Docker containers.run Run command inside CLI container.stop Stop Docker containers.test Run all tests.test-behat Run Behat tests.test-phpunit Run PHPUnit tests.up Build project.

Maintaining SaaS site

SaaS development platform contains all

necessary tools to efficiently develop, test and

deploy websites into preview and production

environments in a consistent and efficient way.

SaaS development platformBATTERIES INCLUDED

● Environment configuration is provided with SaaS repository

● Easy to debug production issues

● No more “... but it works on my machine ...”

Identical environments

SaaS project structurePROJECT SCAFFOLDING

.├── .ahoy.yml | Ahoy configuration.│├── .docker | Docker image files.│ ├── Dockerfile.cli │ ├── Dockerfile.nginx│ └── Dockerfile.php│├── .gitlab-ci.yml | GitLab CI configuration.│├── .lagoon.yml | Lagoon deployment configuration.│├── docker-compose.yml | Docker-compose configuration.│├── files | Project files.│ └── .gitkeep│├── tests | Behat and PHPUnit tests.│ └── .gitkeep│└── themes | Project themes. └── .gitkeep

Authoring SaaS themesADDING CODE TO SAAS SITE

● SaaS site get own private GitLab repository

● SaaS site use GovCMS images

● themes directory mounted into GovCMS container when environment is built

● While developing locally, changes are synchronized to container

● Code linting available in the GovCMS image

● Behat and PHPUnit configuration available in the GovCMS image

● Custom Behat and PHPUnit tests can be added to tests directory

1. CI configuration stored in GovCMS image to enforce predefined steps

2. Consistent across all SaaS sites

3. Runs in GitLab for all sites when code changes on all branches and PRs

4. Performs code standards check and runs per-site tests

5. Triggers Lagoon deployment on success

SaaS CI AUTOMATED SAAS TESTING AND DEPLOYMENT

Improved DXSMOOTH DEVELOPMENT WORKFLOW

Ahoy wraps complex CLI calls into simplified commands:

build Build project.cli Start a shell inside CLI container.drush Run drush commands in the CLI service container.install Install the profile.lint Lint code.login Login to a website.logs Show Docker logs.make Install local dependencies.ps List running Docker containers.pull Pull latest docker images.push Push all docker images.release Push all docker images.restart Restart Docker containers.run Run command inside CLI container.stop Stop Docker containers.test Run all tests.test-behat Run Behat tests.test-phpunit Run PHPUnit tests.up Build project.

Exactly the same Ahoy commands as for GovCMS project

Demonstration

Questions and discussions