Top Banner
LAB ZERO INNOVATIONS, INC. PIER 9, THE EMBARCADERO SAN FRANCISCO, CA 94111 T:415.839.6861 F:415.839.6901 INFO@LABZERO.COM HTTP://LABZERO.COM Don’t Hate, Automate. Lessons learned from implementing Continuous Delivery. Matt Wilson, CTO Lab Zero

Don't hate, automate. lessons learned from implementing continuous delivery

Jun 25, 2015



Solano Labs

This presentation on Continuous Delivery is from the November 2013 Automated Testing San Francisco meetup that took place at Constant Contact. The author/presenter is Matt Wilson, CTO of Lab Zero. Matt has advised clients at various industries including consumer brands, non-profits, start-ups, and financial services on Agile development, web application development, and other technology leadership challenges. This overview on Continuous Delivery highlights some of the best practices that Lab Zero has distilled, based on their many client engagements.

About Matt Wilson:
Matt is an enthused agile developer, architect, and consultant. He enjoys building elegant web services in Ruby. He believes that high-fives are underrated and measures the success of his day by how many he's seen.

Prior to joining Lab Zero, Matt's work history includes: Co-founder/Architect at, Architect at Kodak Gallery, Developer at Westwave Communications, Engineer at Motorola, and Developer at Coldwell Banker.
About Lab Zero:

Lab Zero Innovations, Inc. provides web application development and technology leadership consulting. Our client relationships include staff augmentation, pure software development, project management, system integration, advisor/leadership roles. Contact us about your next project.
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
  • 1. Dont Hate, Automate. Lessons learned from implementing Continuous Delivery. Matt Wilson, CTO Lab ZeroLAB ZERO INNOVATIONS, INC. PIER 9, THE EMBARCADERO SAN FRANCISCO, CA 94111 T:415.839.6861 F:415.839.6901 INFO@LABZERO.COM HTTP://LABZERO.COM

2. Overview Who we areOur evolution toward Continuous D*What weve learned along the wayQuestions & AnswersLAB ZERO INNOVATIONS, INC. 3. Who We Are Product development, design, and strategy team based in SF. Early-stage entrepreneurs and startup founders:,,, echosign. com, Held senior product design & dev roles at Apple, Google, Yahoo, Kodak, Accenture. Our clients are often startups and Fortune 500s. Founded in 2008Many of our clients hire us just to build out their Continuous D* rigs.LAB ZERO INNOVATIONS, INC. 4. Our path to Continuous D* and what weve seen 5. In the beginning there was Just a build We had code and compiled it. We even had some unit tests for some areas of the code. The official build ran nightly but we certainly werent using it for fast feedback. Then we started deploying with automation. cap production deploy Those were some fancy pants. Then we started running our tests on every check in. Then we needed to automate the validation of our app running in different environments Then we started automating the creation of our environments. Just cant have enough.LAB ZERO INNOVATIONS, INC. 6. Necessity struck We have to start new projects very often. The teams we collaborate with want to see progress often. Projects started without a clear delivery process were often slower to deliver over time. At some point manually running our deployment scripts every time we wanted to share our work seemed like too much hassle. We need to know that the feature works in the QA environment.We figured out how to get Continuous D* running on our projects in less than a day.LAB ZERO INNOVATIONS, INC. 7. Our Webapp Build Rspec: Unit tests and generates code coverage reports.Jasmine + PhantomJS: Javascript unit testing.Cucumber: We write our ATDD suites in Cucumber and use Capybara to actually drive browsers.Brakeman: Static security analyzer to help us find obvious security vulnerabilities.Capistrano: Automated deployments of RoR, PHP, Java... applications. LAB ZERO INNOVATIONS, INC. 8. Our Mobile Build Xcodebuild, Ant+javac: iOS and Android command line tools.OCUnit, JUnit: iOS and Android unit testing frameworks.Cucumber: Using Calabash to actually drive the native devices and emulators.TestFlight: Deploy every passing build to TestFlight for easy distribution to team devices.LAB ZERO INNOVATIONS, INC. 9. The story starts at your laptop and ends in production.LAB ZERO INNOVATIONS, INC. 10. TDD the first d is for driven 11. Build Failed 12. When the light is green, the trap is clean. --Ghostbusters 13. Its hard to keep the light green or meaningful if tests are written afterwards. Build your culture of automation awareness. There is a correlation between writing tests and running them. Those that write them tend to run them more often. Developers and QA both need to run tests before committing. If developers arent running the tests because they take too long, its worth the investment to speed them up.CD can fall apart quickly if the team doesnt react early to failed builds.LAB ZERO INNOVATIONS, INC. 14. Speed will matter when youre doing it right 15. How long does it take to deliver a feature to QA?LAB ZERO INNOVATIONS, INC. 16. Ouch, my tests! We want to run as many tests as we can but once weve invested in a fair amount of browser and emulator testing they get really slow. When they get slow we become reluctant to run them. Tests that dont run, cant find bugs. The longer you wait to run them all, the less likely they are to still pass. Tests that are broken and remain unfixed become dead weight. Releasing under-tested products often means were asking too much of our QA teams and usually wont find bugs until they are in production.LAB ZERO INNOVATIONS, INC. 17. Usual suspects for slowness in the pipe Heavy use of fixtures and db cleaning (factory-girl and database-cleaner) Consider using mocks/stubs instead of dbs when possible. Android emulator Get a real device and plug it into your CI server. Assets compiled on each server at deploy time Pre-compile your static assets on the build server and deploy archives instead.LAB ZERO INNOVATIONS, INC. 18. Parallulala, parallaluh, parallelize them Selenium tests are just too slow to not do this Remote browser tests will need you to run multiple tests at once. Especially when you start considering running your tests in more than one browser/OS combo. To do this well you should lean on the PaaSs out there like Solano CI (beaucoup parallelization-fu), Sauce Labs (browsers and mobile emulators, can be used from Solano), Travis CI, etc... Unit tests will need it too If youre doing TDD, you likely already have this problem.LAB ZERO INNOVATIONS, INC. 19. Continuous D* wants to use your test automation across many environments 20. Youll need a few environments to pull this off. CI Server Run tests here first to find anything that should prevent us from deploying the app. Unit tests, static analysis, DevInteg First deployed spot for the app, the environment will be volatile and will serve little use other than for CI. May also serve as QA server if the numbers of commits are low. QA1, 2, 3.. You may need one of these for each product team. Staging Runs in production and is often used as a place to validate the app in production before going live. Production Live. LAB ZERO INNOVATIONS, INC. 21. We want to get the features out, but they have to be verified in QA and Staging before we go live.LAB ZERO INNOVATIONS, INC. 22. Co-own yer ATDDwith QA 23. Force multiplier Share the same ATDD code Devs and QA should collaborate on their functional testing investment. Many organizations separate these groups activities which leads to redundant investments. Devs need to lead in the spirit of TDD Relying only on QA for functional automation causes flapping red->red->green->red->green builds. It works better when you work together Devs can do the heavy lifting on writing reusable steps and tests. QA can add more cases if they see fit. Devs should expect to train and support QA on the tools. Allow QA to audit With our QA hats on, lets have an open conversation about what should be tested and what is missing. This is how we can focus our beams to make the light brighter.LAB ZERO INNOVATIONS, INC. 24. Running testsjust one thing your CI server can do 25. Robots, ACLs and users, what could go wrong? Empower folks with a magic button There are likely things that your team is always asking someone else to do for them. De-spof it by making a job that folks can click whenever needed, even from their phones. Zomg seriously. Database Refreshes Nearly every webapp project needs to be able to reset/refresh the dev/qa/staging databases easily. Production deploy jobs This is your parents liquor cabinet: make sure the kids dont get into it, but make it easy to open when needed.LAB ZERO INNOVATIONS, INC. 26. Thank You! Questions? twitter/aim/yim/skype: w1150nLAB ZERO INNOVATIONS, INC. PIER 9, THE EMBARCADERO SAN FRANCISCO, CA 94111 T:415.839.6861 F:415.839.6901 INFO@LABZERO.COM WWW.LABZERO.COM