Tech Talk: Startup Camp Berlin 2015 - DevOps at LeanIX
Post on 16-Jul-2015
1009 Views
Preview:
Transcript
DevOps in prac-ce – How to keep focus on your product?
André Christ Startup Camp Berlin, March 2015
Hint: The former capital ;-‐)
2 Photo: Michael Sondermann/Bundesstadt Bonn
About
3
André Christ • Co-‐Founder & Co-‐CEO • Responsible for IT & Finance • Before: DHL, HTGF, Startups
LeanIX GmbH • SaaS-‐Provider for IT Architecture Mgmt • Self-‐Funded, Founded 2012 in Bonn • > 20 Customers in US, Europe and Australia
What’s your level of experience with these tools?
4
“Build, ship, and run distributed applicaZons”
“Building/tesZng so\ware projects conZnuously”
“IT automaZon engine that automates applicaZon
deployment, configuraZon management, ….”
Ansible and Docker clearly connected to the DevOps “Hype” …
5 Source: Google Trends
DevOps Docker Ansible
… while Jenkins is really, really mature and widely adopted by all company sizes
6 Source: Google Trends
DevOps Docker Ansible Jenkins
Agenda
7
Docker
Ansible
Jenkins
Mo-va-on – How to keep focus on your product?
Summary / Learnings
Challenge 1: Serve high-‐profile customers worldwide with very limited resources
8
Challenge 2: Scale the plahorm with the right tools for the job
2012 2014 2013
“MVP” SSO & Micro services
Professional Build Process
Service-‐ & Quality Ensurance
2015
Single Page JS App
Challenge 3: B2B-‐SaaS requires flexible deployment models – Cloud and On-‐Premise
10
Different cloud environments
• Amazon Webservices (US & Australia) • Strato (Germany) • Further providers planned (e.g. Azure) • …
Individual plaPorms per company
• Ubuntu Linux • Redhat • Windows • ….
11
Write proper documenta-on and manuals?!
Does not solve our issues*
• High effort to test • Not repeatable • Outdated very soon • Prohibits scale • Not customer friendly (e.g. on premise)
*) Does not mean we don’t write proper documentaZon where needed J
ProducZon Servers US ProducZon Servers US
Our solu-on: Highly automated build, test and deploy tool chain
12
Develop Build Test Deploy
Staging Server
ProducZon Servers Europe
Developer Machines
Real-‐life example: LeanIX Synclog micro service
13
Track synchroniza-on results: Success, errors etc. of all LeanIX 3rd party integraZons
Synclog Microservice
Synclog Architecture: RESTful micro service with database and Single Page App GUI
DB
Synclog REST-‐API
Synclog GUI Angular.JS Nginx Webserver
Postgres
Java Dropwizard
API
Agenda
14
Docker
Ansible
Jenkins
MoZvaZon – How to keep focus on your product?
Summary / Learnings
How does Docker compare to virtual machines?
15
Server
Host OS
Hypervisor
Server
Host OS
Docker Engine
Guest OS
Bins/Libs
App A
Guest OS
Bins/Libs
App B
Svc 1 Svc 2
Bins/Libs
Virtualisa-on Docker Advantages
• Significantly faster deployment
• Much less overhead • Easier migraZon
• Faster restart & cleanup • Fits perfect to micro services architectures
Topics to be adressed
• IsolaZon / Security • Requires Linux Kernel (Workarounds via boot2docker)
• Maturity of tools
VM VM
Container Container
Svc 3
Container
Bins/Libs
What is a Docker container and image?
16
Docker Client
Host Docker Registry
Docker Deamon
Container 1
Container 2
Container 3
Image 1
Container N
Image 2
Image 3
Image 4
Docker API
Private running stopped
Simple example: Run a Database
17
Docker Client
Host Docker Registry
Docker Deamon
mydb
leanix-‐synclog
postgres:9.4
nginx
leanix-‐synclog
// Start a Postgres DB $ docker pull postgres:9.4 $ docker run –name mydb postgres:9.4 $ docker stop mydb // Create a new image $ docker build –-tag=“leanix/leanix-synclog” . $ docker push leanix/leanix-synclog
Docker API
…
Private running stopped
Docker Registry (Docker Hub)
18
• 100 „Official“ images • > 45.000 public images • SubscripZon model for private images (similar to GitHub)
Our example is cut into 3 containers
19
DB
Synclog REST-‐API
Synclog GUI Angular.JS Nginx Webserver
Postgres
Java Dropwizard
Example Docker Images
postgres:9.4
leanix-‐synclog
leanix-‐synclog-‐client
Synclog service in context of enZre leanIX architecture
20
Reverse Proxy (nginx)
eam
(Core App)
mtm
(User Mgmt) synclog
expo
rt
web
hooks
noZfi
caZo
ns
images
….
Requests
• Auth between Services: oAuth2 • Single Sign on (with customer) SAML 2.0
Create custom docker image based on exisZng images
21 Source: Docker
FROM dockerfile/java:oracle-java7 # Copy JAR and config into container COPY target/leanix-synclog-0.1.0 / COPY config.yml / COPY run.sh / RUN chmod +x /run.sh # Expose port 9100 and run java process EXPOSE 9100 WORKDIR / # Run command CMD ["./run.sh"]
leanix-‐synclog/Dockerfile
$ docker build –-tag=“leanix/leanix-synclog” . $ docker push leanix/leanix-synclog
How to Ze everything together? docker-‐compose (was: fig)
22
api: build: . links: - db ports: - "9100:9100” db: image: postgres
docker-‐compose.yml
$ docker-compose up $ docker-compose stop
#!/bin/bash # Replace address sed -i "s/localhost:5432/ $DB_1_PORT_5432_TCP_ADDR:5432/" config.yml java -jar leanix-synclog-0.1.0.jar db migrate config.yml java -jar leanix-synclog-0.1.0.jar server config.yml
leanix-‐synclog/run.sh
Docker UI – Get visual overview and controls for Docker (made with Angular)
23
$ docker run -d -p 9999:9000 --name dockerui --privileged -v /var/run/docker.sock:/var/run/docker.sock dockerui/dockerui
Keep an eye on: Giant swarm Docker orchestraZon made easy
24
{ "app_name": "leanix-‐synclog", "services": [{ "service_name": "leanix-‐synclog", "components": [ { "component_name": "leanix-‐synclog", "image": ”leanix/leanix-‐synclog", "ports": [ ”80/tcp" ], "domains": { "synclog-‐andrechrist.giganZc.io": "80" }, "dependencies": [ { "name": "db_1", "port": 5432}] }, { "component_name": "db_1", "image" : "postgres:9.4", "ports": ["5432/tcp"] } ] } ] }
$ swarm login $ swarm up $ swarm logs <ID> $ swarm stop $ swarm scaleup <COMP>
swarm.json
Agenda
25
Docker
Ansible
Jenkins
MoZvaZon – How to keep focus on your product?
Summary / Learnings
Jenkins Overview
26
Simple
Extensible
Scalable
Flexible
Open Source
Community
Good Web GUI
ConZnuous IntegraZon (CI) Server
Thousands of plug-‐ins
Distributed BUILD System
ConfiguraZon of Jobs
MIT License
Very acZve (since 2011)
Jenkins Job: leanix-‐synclog
27 Image Source: robots.com
Trigger: GitHub Push 1. Builds, e.g. using
maven (Java) 2. Executes Unit Tests 3. Creates Docker image 4. Pushes Docker image
to Docker Hub 5. Deploys to staging
system (via Ansible)
Agenda
28
Docker
Ansible
Jenkins
MoZvaZon – How to keep focus on your product?
Summary / Learnings
Ansible Overview
29
YAML Syntax
Library of modules
Switch from ssh push to pull
EncapsulaZon by roles
MIT License
Strong Growth (since 2012)
OrchestraZon of Admin-‐Tasks
Simple
Extensible
Scalable
Flexible
Open Source
Community
Automate repeatable admin tasks, e.g. install packages, set configs, ...
Image Source: robots.com
Ansible Architecture
30
Tasks
Install Package
Copy file
Apply template
...
Deploy synclog Apply template
Run docker
Roles
init
...
docker
Playbooks
Provision Server
leanix-‐synclog
...
postgres
Hosts
SVR-‐DE-‐TEST
SVR-‐DE-‐DEV
SVR-‐DE-‐WEB-‐1
SVR-‐US-‐WEB-‐1
via ssh
...
Ansible Example: Provision servers
31
# Provisions the frontend servers --- - hosts: frontend sudo: true roles: - {role: 'init'} - {role: 'docker'} - {role: 'updates'}
provision_server.yml
$ ansible_playbook provision_server.yml –I hosts/prod -v
hosts/prod
[frontend] srv-de-web-1.leanix.net srv-de-web-2.leanix.net srv-us-web-1.leanix.net srv-us-web-2.leanix.net [backend] srv-de-app-1.leanix.net srv-de-app-2.leanix.net srv-us-app-1.leanix.net srv-us-app-2.leanix.net
Ansible Example: Configure system
32
[…] - name: Install System Packages apt: pkg={{ item }} state=latest with_items: ["curl", "wget", "python-software-properties", "software-properties-common", "daemon", "supervisor"] - name: German kb command: loadkeys de changed_when: false - name: Set hostname on boot to short name from inventory list template: src=hostname.j2 dest=/etc/hostname owner=root group=root mode=0644 register: hostname_file […]
roles/init/tasks/main.yml
Ansible Example: Deploy service
33
[…] - name: Start leanix synclog Docker container shell: docker run -d
--name {{ item.1.name }} -p {{ ansible_eth1.ipv4.address }}:{{ item.1.synclog_port }}:{{SERVICE_PORT }} -e SERVICE=synclog -e PROXY_SERVICE={{ PROXY_SERVICE }} -e SWAGGER_BASEPATH={{ SWAGGER_BASEPATH }}
leanix/leanix-synclog with_indexed_items: SYNCLOG_SERVICES when: synclog_running.results[{{ item.0 }}].rc != 0 […]
roles/init/tasks/main.yml
Agenda
34
Docker
Ansible
Jenkins
MoZvaZon – How to keep focus on your product?
Summary / Learnings
Key Issues & SoluZon Approaches
35
• Docker: Usage of boot2docker on non Linux (OS X, Win)
• Docker: Container orchestraZon on producZon systems with distributed docker deamons
• Ansible: v1 not running on Win
• Ansible: For some tasks, special python libs need to be on the target host
Contribute to helper tools ;-‐) e.g. b2d-‐sync (node.js tool)
Create custom soluZons based on Ambassador pa~ern and etcd orchestrated by Ansible
Use Vagrant or get rid of Windows ;-‐)) Check in tasks, if libraries are already installed
Summary
36
20 days
3 hours
< 30 min
Effort for setup, learning and trouble shooZng for Ansible & Jenkins
Ramp-‐Up of a new employee unZl first commit
Time unZl a fresh Server is setup with all required services
14 days For changing our Vagrant based environment to docker & docker-‐compose
37
THANKS! Follow us @leanix_net
Download Ansible Example Code github.com/leanix/codetalks_2014_demo
We are hiring!!! • Frontend Developers • Backend Developers • MarkeZng & Sales
www.leanix.net
Backup
38
Growth and DigitalizaZon leads to increasing IT complexity
39
-me
Growth / Digitaliza-on
20-‐30 ApplicaZons
50-‐80 ApplicaZons
> 150 ApplicaZons
Lacking transparency of the IT landscape is a strong barrier for innovaZon
40
35 b€ overspent due to redundant applicaZons
Source: Detecon / McKinsey, 15% of applicaZons can be consolidated
Inefficient processes due to head monopoly
leanIX is a smart plahorm which helps companies to opZmize their IT landscape
41
Fact Sheets & Tagging
Context-‐based Search
API, Import & Export
Comments & Threads
IT Inventory Collabora-on PlaPorm Interac-ve Repor-ng
AcZvity Stream & NoZficaZons
SubscripZon
Print & Export (PDF)
Best PracZce Reports
InteracZve AdapZon
leanIX connects the dots in the ecosystem of modern IT management
42
Collabora-on-‐ & Document Mgmt
IT Service Management & Help Desk
Requirements-‐ & Business Process Mgmt
Project PorPolio-‐ and Ressource-‐Management
Selected products to illustrate
Out-‐of-‐the-‐box IntegraZonen
REST API & SDKs
developer.leanix.net
Partnering / IntegraZon
43
Sync
top related