Top Banner
Continuous Delivery of Infrastructure The case of the growing startup

BlaBlaCar and infrastructure automation

Jul 16, 2015



  • Continuous Delivery of Infrastructure

    The case of the growing startup

  • Nicolas Blanc - BlaBlArchitect

    SinfomicSinfomic (1999)






  • 3/49


    Introduction Why Continuous Delivery of Infrastructure Automate Configuration Hardware Industrialization Automate OS Install Hardware availability Continuous delivery

  • Introduction

  • 80 202020



    Trusted Ridesharing

  • People Powered Travel

    A travel search engine

    A trusted community


  • Fast growth


  • 8/49

    Why Infrastructure Automation ?

  • 9/49

    Few numbers of BlaBlaCar today

    2 Data Centers in Paris

    242 servers 109 virtual servers (VMWare) 110 EC2 instances during working day

    20 instances the rest of the time

  • Our Continuous Delivery at the top.

    Jealous from the development team !

  • 11/49

    We have a strong agility culture

    Push our main app on production 10x a day Developers make the push themselves

    App can be pushed only after a Bamboo's merge Bamboo merge only after thousands of tests

  • 12/49

    tools to manage them all

  • Old Infrastructure Overview

  • 14/49

    Physical Server

    3 racks on a single DC

    12 servers per rack

    More than 20 differents

    hardware configurations

  • 15/49

    Cloud (Amazon EC2)

    Only 10 to 20 instances always running

    Used for elasticity

    10 to 20 instances running for test (Bamboo)

  • 16/49


    Strong Team who designed a very strong network Rmi (@shtouff)

    Not a problem for us to scale the network

  • Automate the Configuration

  • 18/49

    Choosing the right tool

  • 19/49

    Improve our Service Deployment

    Started to work on a front web server Ease configuration deployment Ease a new installation

    Then worked on all servers First just for configuration Then all service deployment

  • Hardware Industrialization

  • 21/49

    Haute Couture

    Before our automation process: One server configuration by service Multiple services

  • 22/49

    Haute Couture - Caveats

    Tailored server are complicated to design Hard to upgrade

  • 23/49

    Next Step - Virtualization

    Create a virtualization platform 4 big servers A big disk array

  • 24/49

    Virtualization - Caveats

    I/Os Performances compare to physical server Network Performances

  • 25/49

    Cloud: Why on premise in 2015 ?

    Data privacy concerns Limited elasticity required Cost efficiency History & Culture

  • 26/49

    Now Standard Servers

    Define a single 1U server Dual-CPU (12 cores) 128 Go of RAM Disk configuration vary on 3 flavors:

    Only 2 SSD mirrored (RAID) Fast disk (SSD) needed Very big capacity (lot of disks)

  • Automate OS Installation

  • 28/49

    Automate debian installation

    All our infra is debian based PXE ipmi

    preseed (thx to @jbfavre)

  • 29/49

    Virtualization => Template

    Work well on EC2 (only option for instance install)

    Never used in VMWare Same install as physical server debian based

    (thanks to Chef)

  • 30/49

    Define our needs to choose a tool

    IP allocation Bootstrap chef-client PXE preseed debian Can install either a physical, virtual or cloud server Configure the BMC interface

    HP (iLo) Dell (DRAC) SuperMicro

  • 31/49

    Because Simon (@s_ourea) chose it ! More likely to be maintained Big community

    Fits our needs

  • Hardware Availability

  • 33/49

    We need more

    Chef / Foreman, great tools to: Install OS Install all services and configure them

    No easy solution when you need a specific hardware on never used server

  • 34/49

    Collins !

    Nice tool to manage our hardware inventory. A lot of good reasons to choose it:

    Simon (@s_ourea) chose it Fit our needs

    Linked with


  • Hey ? What about the initial setup of a physical server ?

  • 36/49

    Install servers into DC Limited added value in the team racking servers Time consuming DCs can be far away

    Simon(@s_ourea) & Rmi(@shtouff) pushed a new idea Buy servers only when we can use a full rack Write an installation documentation Choose a company to rack

  • 37/49

    Our procedure to have a new rack

    Buy a new rack into the DC

    Our network team installs 2 switchs in the rack.

    They add links to our core network equipments

    Hey! Let's ask to our provider to rack and setup all servers, following Simon's documentation

  • 38/49

    The typical life of a server

    Discovery, and go into Collins

    Another team takes it and installs it via Foreman

    Chef is called via Foreman

    Chef is always running

  • Tests, and Continuous delivery really ?

  • 40/49

    Chef and tests

    Yes, we test all of our

    infrastructure's services !

    Thanks to Chef and all of it's community:KitchenCI + vagrant + serverspec, chefspec

  • 41/49

    Chef - Cookbook life

    Each cookbook in it's own git repository

    First test and merge with Bamboo

    Push in production with a new version number

    All servers converge to the desired state

  • 42/49

    What do you test ?

    Everything! (one day perhaps) Server install Bootstrapping Convergence Clustering /!\ (not automatic) Version upgrade ( one day !) Coffee quality

  • 43/49

    Continuous delivery, really ?

    New cookbook versions pushed 10x times / day

    Provision several new servers / VM / instances by day

    Buy a rack full of servers, and install them in less than a day

  • So cool ! But...What will you do in the future ?

  • 45/49



  • 46/49

    What we will continue to do

    Test our infra (with or without container) Industrialize our processes Spread knowledge to all other teams Dream solutions, and finally build them

    Listen to Simon (@s_ourea)

  • Conclusion

  • 48/49

    What to keep in mind

    Involve overall company

    Support company growth

    Don't get attached to tools

    Keep a small team spirit

  • That's all folks!

    Thanks you!

    Nicolas Blanc(@thewhitegeek)

    Slide 1Slide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24Slide 25Slide 26Slide 27Slide 28Slide 29Slide 30Slide 31Slide 32Slide 33Slide 34Slide 35Slide 36Slide 37Slide 38Slide 39Slide 40Slide 41Slide 42Slide 43Slide 44Slide 45Slide 46Slide 47Slide 48Slide 49

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.