How Automation Reveals Technical Debt

© 2013 IBM Corporation

How Automation Reveals Technical Debt


Eric’s BioI’m a DevOps Evangelist at UrbanCode where I helps customers get the most out of their build, deploy and release processes.

I have automation experience throughout the application life-cycle in roles as a developer, test automation engineer, and support engineer. For the last 9 years, I’ve been focused on CI, CD and DevOps

Eric [email protected]@EricMinick

mailto:[email protected]


Technical Debt


Why do we accumulate technical debt?

We leverage technical debt to deliver more faster.

De-leveraging is rarely accounted for in project planning.

Green-Shifting*

Scope Time

Resources

* http://www.drdobbs.com/191600661


Why do we care? Paying interest

Now Later Later Still Much Later0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Debt PaidInterest PaidValue Delivered

Debt


Why do we care? Or paying our debts

Now Later Later Still Much Later0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Debt PaidInterest PaidValue Delivered

Debt


Pay it now, or pay more later

NowLa

ter

Late

r Still

Muc

h La

ter

Debt Paid

Interest Paid

Value De-livered

NowLa

ter

Late

r Still

Muc

h La

ter

Debt Paid

Interest Paid

Value De-livered


Why should we care?

Baggage that slows the team

Lack of automated tests lengthen QA cycles

Fear of merging work

Unrefactored code slow to work with

Slow build / deploy processes delay learning and release pace

Quality issues

Lack of tests results in buggier code

Releases are error prone and lead to unnecessary outages


The limits of what we know

Known Knowns: Bugs confirmed and tracked

Known Unknowns: Undiscovered bugs

Unknown unknowns: One of our project teams is using a GPL’d library making their product impossible to ship


Where Automation Helps

“Testing” for debt: Automated tests, code scans and reports can help identify (and quantify) problems.

Automation as a learning experience: The act of automating brings surprises.


Testing for Debt

Continuous Integration (multi-component)

Code Inspection

Watching Trends


Testing for Debt: Continuous Integration

On code commit, build and test the software

Roll up changes to build-time or runtime dependencies and test those too to identify API incompatibilities


Testing for Debt: Code Inspection

Code Reviews – Rapidly detect issues

Static Code Analysis – Tool based checks for bugs, code duplication, code theft, & non-compliance with dev standards.


Testing for Debt: Code Inspection

Code Reviews – Rapidly detect issues

Static Code Analysis – Tool based checks for bugs, code duplication, code theft, & non-compliance with dev standards.


Testing for Debt: Watching Trends


More visualizations: Sonar

http://nemo.sonarsource.org/dashboard/index/327690?did=6


An example safety net

Continuous build & unit test

Nightly slow tests / code scans

Emails identifying new issues – ideally tied against source code changes

Regular inspection of trends

Bugs / Stories entered around issues


Automation as a learning experience

Implementing the safety net helps us discover unacknowledged debt


Tests? What tests?


Automation Examples: The Build

Build time dependencies not understood

Build scripts missing or incomplete

“Magic build server” anti-pattern

http://www.urbancode.com/html/resources/webinars/Role_of_Binary_repositories_in_Software_Configuration_Management.html


Automation Examples: Deployment

Deployment scripts scarce

“Special Instructions” with most deployments indicate non-repeatable process

Environmental differences handled poorly

Separation of duties less than real


Automation Example: Test Automation

Requirements less well understood

Existing tests are few, stale, un-optimized

Application not architected for testability


Expect the unexpected when automating

At scale, Green-Shifting, has hidden issues

Include these discoveries in ROI estimations for automation (positively and negatively)

This is a happy side effect


Start making decisions


Direct and indirect automation benefits

Direct: We tested for problems and found them.

Indirect: In attempting to be more efficient, we automated, and accidently discovered problems.


Automating for the team

Provide a “safety net” to detect and recognize issues.

Diagnose and repair lack of repeatability in build-deploy-test

Quantify accumulating debt in support of fighting scope creep

Team level tooling is fine


Automating the enterprise

Benefits extend beyond aggregate team level benefits

Central Automation and Reporting gets us:

–Identify who can use shared configuration

–Who has tests, who doesn’t

–Who is using what tools

–Build / deploy failure rates


Stories from customers


Favorite Examples: Deployment Failures

Debt: High rate of deployment failures a problem

Interest: QA productivity is getting hurt & lengthened time to market

Goal: Reduce failure rate from 40% to 5%

Approach: Avoid manually fixing a deployment. Fix the automation and redeploy.

Enforcement: Monthly / weekly spreadsheet of success to CIO with a six month plan.


Favorite Examples: “End of Day” Commits

Debt: Developers commit breaking changes at the end of the day

Interest: Code base broken in morning, or other people stay up late to fix it.

Goal: Avoid other devs staying late to clean up

Approach: Report on failures, and react to negative patterns as a team.

Enforcement: Social pressures



Favorite Examples: Low numbers of tests

Problem: Unit testing discipline breaking down over time

Goal: Maintain high or improving coverage

Approach: Standard coverage tools and an emphasis on upward trends.

Enforcement: Trending report

on monitor over CIO’s door


No hiding!No greenshifting


Summary

We accumulate technical debt as we race to deliver more, faster.

This causes us to eventually release less, slower, with worse quality

Automation directly and indirectly helps us find issues.

There are benefits at both team and enterprise levels.


More References

http://urbancode.com/resources

Enterprise CD Maturity Model

Lean Build & Deployment Automation

Managing Release Risks with Metrics

Blogs.urbancode.com

Twitter.com/UrbanCode

facebook.com/IBMUrbanCodeProduc

Slideshare.net/Urbancode

http://urbancode.com/resources

http://blogs.urbancode.com/

http://slideshare.net/Urbancode



Yes, we sell products for this

uBuild

–Build management and continuous integration

uDeploy

–Deployment and release automation

uRelease

–Release management tool for planning and executing big releases

... And IBM’s amazing portfolio of CLM, testing tools, service virtualization, provisioning, etc, etc.


Questions?

[email protected]@EricMinick, @UrbancodeSlideshare.net/Urbancode


How Automation Reveals Technical Debt

Technology

ibm corporation tests

ibm corporation testing

ibm corporation stories

ibm corporation direct

ibm corporation technical

ibm corporation favorite

ibm corporation erics

later debt