Don't hate, automate. lessons learned from implementing continuous delivery

• 1. Dont Hate, Automate. Lessons learned from implementing Continuous Delivery. Matt Wilson, CTO Lab ZeroLAB ZERO INNOVATIONS, INC. PIER 9, THE EMBARCADERO SAN FRANCISCO, CA 94111 T:415.839.6861 F:415.839.6901 INFO@LABZERO.COM HTTP://LABZERO.COM

2. Overview Who we areOur evolution toward Continuous D*What weve learned along the wayQuestions & AnswersLAB ZERO INNOVATIONS, INC. 3. Who We Are Product development, design, and strategy team based in SF. Early-stage entrepreneurs and startup founders: earfl.com, meez.com, gofish.com, echosign. com, veodia.com. Held senior product design & dev roles at Apple, Google, Yahoo, Kodak, Accenture. Our clients are often startups and Fortune 500s. Founded in 2008Many of our clients hire us just to build out their Continuous D* rigs.LAB ZERO INNOVATIONS, INC. 4. Our path to Continuous D* and what weve seen 5. In the beginning there was Just a build We had code and compiled it. We even had some unit tests for some areas of the code. The official build ran nightly but we certainly werent using it for fast feedback. Then we started deploying with automation. cap production deploy Those were some fancy pants. Then we started running our tests on every check in. Then we needed to automate the validation of our app running in different environments Then we started automating the creation of our environments. Just cant have enough.LAB ZERO INNOVATIONS, INC. 6. Necessity struck We have to start new projects very often. The teams we collaborate with want to see progress often. Projects started without a clear delivery process were often slower to deliver over time. At some point manually running our deployment scripts every time we wanted to share our work seemed like too much hassle. We need to know that the feature works in the QA environment.We figured out how to get Continuous D* running on our projects in less than a day.LAB ZERO INNOVATIONS, INC. 7. Our Webapp Build Rspec: Unit tests and generates code coverage reports.Jasmine + PhantomJS: Javascript unit testing.Cucumber: We write our ATDD suites in Cucumber and use Capybara to actually drive browsers.Brakeman: Static security analyzer to help us find obvious security vulnerabilities.Capistrano: Automated deployments of RoR, PHP, Java... applications. LAB ZERO INNOVATIONS, INC. 8. Our Mobile Build Xcodebuild, Ant+javac: iOS and Android command line tools.OCUnit, JUnit: iOS and Android unit testing frameworks.Cucumber: Using Calabash to actually drive the native devices and emulators.TestFlight: Deploy every passing build to TestFlight for easy distribution to team devices.LAB ZERO INNOVATIONS, INC. 9. The story starts at your laptop and ends in production.LAB ZERO INNOVATIONS, INC. 10. TDD the first d is for driven 11. Build Failed 12. When the light is green, the trap is clean. --Ghostbusters 13. Its hard to keep the light green or meaningful if tests are written afterwards. Build your culture of automation awareness. There is a correlation between writing tests and running them. Those that write them tend to run them more often. Developers and QA both need to run tests before committing. If developers arent running the tests because they take too long, its worth the investment to speed them up.CD can fall apart quickly if the team doesnt react early to failed builds.LAB ZERO INNOVATIONS, INC. 14. Speed will matter when youre doing it right 15. How long does it take to deliver a feature to QA?LAB ZERO INNOVATIONS, INC. 16. Ouch, my tests! We want to run as many tests as we can but once weve invested in a fair amount of browser and emulator testing they get really slow. When they get slow we become reluctant to run them. Tests that dont run, cant find bugs. The longer you wait to run them all, the less likely they are to still pass. Tests that are broken and remain unfixed become dead weight. Releasing under-tested products often means were asking too much of our QA teams and usually wont find bugs until they are in production.LAB ZERO INNOVATIONS, INC. 17. Usual suspects for slowness in the pipe Heavy use of fixtures and db cleaning (factory-girl and database-cleaner) Consider using mocks/stubs instead of dbs when possible. Android emulator Get a real device and plug it into your CI server. Assets compiled on each server at deploy time Pre-compile your static assets on the build server and deploy archives instead.LAB ZERO INNOVATIONS, INC. 18. Parallulala, parallaluh, parallelize them Selenium tests are just too slow to not do this Remote browser tests will need you to run multiple tests at once. Especially when you start considering running your tests in more than one browser/OS combo. To do this well you should lean on the PaaSs out there like Solano CI (beaucoup parallelization-fu), Sauce Labs (browsers and mobile emulators, can be used from Solano), Travis CI, etc... Unit tests will need it too If youre doing TDD, you likely already have this problem.LAB ZERO INNOVATIONS, INC. 19. Continuous D* wants to use your test automation across many environments 20. Youll need a few environments to pull this off. CI Server Run tests here first to find anything that should prevent us from deploying the app. Unit tests, static analysis, DevInteg First deployed spot for the app, the environment will be volatile and will serve little use other than for CI. May also serve as QA server if the numbers of commits are low. QA1, 2, 3.. You may need one of these for each product team. Staging Runs in production and is often used as a place to validate the app in production before going live. Production Live. LAB ZERO INNOVATIONS, INC. 21. We want to get the features out, but they have to be verified in QA and Staging before we go live.LAB ZERO INNOVATIONS, INC. 22. Co-own yer ATDDwith QA 23. Force multiplier Share the same ATDD code Devs and QA should collaborate on their functional testing investment. Many organizations separate these groups activities which leads to redundant investments. Devs need to lead in the spirit of TDD Relying only on QA for functional automation causes flapping red->red->green->red->green builds. It works better when you work together Devs can do the heavy lifting on writing reusable steps and tests. QA can add more cases if they see fit. Devs should expect to train and support QA on the tools. Allow QA to audit With our QA hats on, lets have an open conversation about what should be tested and what is missing. This is how we can focus our beams to make the light brighter.LAB ZERO INNOVATIONS, INC. 24. Running testsjust one thing your CI server can do 25. Robots, ACLs and users, what could go wrong? Empower folks with a magic button There are likely things that your team is always asking someone else to do for them. De-spof it by making a job that folks can click whenever needed, even from their phones. Zomg seriously. Database Refreshes Nearly every webapp project needs to be able to reset/refresh the dev/qa/staging databases easily. Production deploy jobs This is your parents liquor cabinet: make sure the kids dont get into it, but make it easy to open when needed.LAB ZERO INNOVATIONS, INC. 26. Thank You! Questions?matt@labzero.com twitter/aim/yim/skype: w1150nLAB ZERO INNOVATIONS, INC. PIER 9, THE EMBARCADERO SAN FRANCISCO, CA 94111 T:415.839.6861 F:415.839.6901 INFO@LABZERO.COM WWW.LABZERO.COM