FINE-TUNING OF AGILE DEVELOPMENT
Isa Goksu, Cengiz Han
C o n t i n u o u s D e l i v e r y i n P r a c t i c e
ABOUT US
§ Isa Goksu, @IsaGoksu Tech Principal
§ Cengiz Han, @hancengiz Tech Lead
2 © 2014 ThoughtWorks, Inc. All rights reserved.
RECOGNIZE THIS?
§ Single line of change takes a week to deploy
§ Slow Feedback (10 weeks after delivering the features, receiving your first feedbacks)
§ You started a new project, defect from old project is bugging you
§ Code Freeze?
§ Sleepless nights during deployments
3 © 2014 ThoughtWorks, Inc. All rights reserved.
“How long would it take your organization to deploy a change that involves just one single line of code?”
Mary and Tom Poppendieck
4 © 2014 ThoughtWorks, Inc. All rights reserved.
THE CONTINUOUS DELIVERY MATURITY MODEL
5 © 2014 ThoughtWorks, Inc. All rights reserved.
Where to start?
6 © 2014 ThoughtWorks, Inc. All rights reserved.
1
7
make it always production ready
© 2014 ThoughtWorks, Inc. All rights reserved.
TRUNK-BASED DEVELOPMENT
8
§ One and only one branch
§ Continuously Integrate with the team
§ All the time production ready
§ At least 1-commit per day per pair/dev
§ Quality increase
FEATURE TOGGLING
§ Helps you to have production-ready code all the time
§ Gives you the flexibility of trying out multiple features
§ QA can test individual features or group of features
§ Business can get fast feedback thru feature previewing
9 © 2014 ThoughtWorks, Inc. All rights reserved.
10
Features developed in an iteration
Enabled features in production
© 2014 ThoughtWorks, Inc. All rights reserved.
11
Many more..
C# - FeatureSwitcher
Golang - flag
Python - Waffle
Ruby - Feature-Toggles
Node.js - feature-flags
© 2014 ThoughtWorks, Inc. All rights reserved.
FEATURE BRANCHING
§ Avoid it! § CI is a problem
§ Long-lived
§ Merging hell
§ Works on § Small teams
§ Short-lived
12 © 2014 ThoughtWorks, Inc. All rights reserved.
2
13
build quality in
© 2014 ThoughtWorks, Inc. All rights reserved.
TEST STRATEGY
§ Make the QA the key participant of your flow
§ QAs are not just testers (assuring quality)
§ Enforces you to deliver vertical slices
§ Consider using new approaches like Specification by Example, Generative Testing, Parametric Testing)
§ Automation is a must
14 © 2014 ThoughtWorks, Inc. All rights reserved.
15 © 2014 ThoughtWorks, Inc. All rights reserved.
16 © 2014 ThoughtWorks, Inc. All rights reserved.
3
17
keep everything in source control
© 2014 ThoughtWorks, Inc. All rights reserved.
STORING DB CHANGES IN SCM
§ Start with a clean DB
§ Use proven tools (dbdeploy, liquidbase, etc)
§ Make sure deltas are incremental, small and immutable
§ Have rollback scripts ready and auto rollback if possible, or do only roll-forward
§ Fail directly if any delta fails
§ Run each delta in order
§ Stored together 18 © 2014 ThoughtWorks, Inc. All rights reserved.
19
DBDeploy Metadata
Baseline Database
Apply Deltas
Test
Fail Fast
Apply Deltas Apply Deltas
Apply Deltas Apply Deltas
© 2014 ThoughtWorks, Inc. All rights reserved.
20 © 2014 ThoughtWorks, Inc. All rights reserved.
INFRASTRUCTURE AS CODE
§ Determinism vs. Indeterminism
§ Treat your infrastructure as you are coding
§ Helps you to have system consistency
§ Use proven tools (puppet, chef, ansible, bcm, cfengine, etc)
§ Have a test machine/vm to try 21 © 2014 ThoughtWorks, Inc. All rights reserved.
22
System Config.
SCM Puppetmaster
WebServers AppServers OtherServers WebServers AppServers OtherServers
© 2014 ThoughtWorks, Inc. All rights reserved.
23
Puppet example manifest for
Apache HTTP server
© 2014 ThoughtWorks, Inc. All rights reserved.
TEST IT BEFORE GET IT OUT
24 © 2014 ThoughtWorks, Inc. All rights reserved.
4
25
repeatable, risk-free deployments
© 2014 ThoughtWorks, Inc. All rights reserved.
UNIFIED WAY
§ Build one binary and use it everywhere
§ Deploy the same way to all environments
§ Unattended deployments
§ Separate the things that change from the thing that don’t
26 © 2014 ThoughtWorks, Inc. All rights reserved.
VIRTUALIZATION
§ Use one
§ Containers vs Hypervisors
§ Availability, Fail-safe
§ Fast provisioning
§ You can automate it!
27 © 2014 ThoughtWorks, Inc. All rights reserved.
PACKAGING
28
RPM | CHOCOLATEY
sudo rpm –ivh awesome-app-1.0.rpm Cinst awesome-app –version 1.0
© 2014 ThoughtWorks, Inc. All rights reserved.
5
29
everybody is responsible
© 2014 ThoughtWorks, Inc. All rights reserved.
MTBF, MTRS
30
§ Failure is inescapable just like getting sick § Self-healing systems (auto-scale, retries, etc)
© 2014 ThoughtWorks, Inc. All rights reserved.
IMMUNE SYSTEMS
§ Zero access policy (phoenix or immutable servers)
§ Monitoring (graphite, riemann, munin, nagios, etc)
§ Logging (ELK clusters, Graylog2, syslog, etc)
§ Application Status / Healthcheck / Configuration Endpoints
31 © 2014 ThoughtWorks, Inc. All rights reserved.
32
Status Endpoint Healthcheck Endpoint
GET http://my.service/status
GET http://my.service/healthcheck
© 2014 ThoughtWorks, Inc. All rights reserved.
SUMMARY
§ DONE means released!
§ Automate everything
§ Be always production ready
§ If anything fails, stop the delivery line
§ Keep everything in SCM
§ Build binaries only once
§ Use precisely the same mechanism to the every environment
33 © 2014 ThoughtWorks, Inc. All rights reserved.
Q/A Ping us later @IsaGoksu, @hancengiz
34 © 2014 ThoughtWorks, Inc. All rights reserved.