Managing Software Debt - Federal Reserve Bank

Managing Software Debt

Continued Delivery of High Value as Systems Age

Chris SterlingVP of EngineeringAgileEVM Inc.

Web: www.AgileEVM.comEmail: [email protected]

Blog: www.GettingAgile.com Follow Me on Twitter: @csterwa

Hash Tag for Presentation: #swdebt

1Thursday, October 28, 2010

http://www.SterlingBarton.com

http://www.SterlingBarton.com

mailto:[email protected]


http://www.GettingAgile.com

http://www.GettingAgile.com

© 2009-2010,

Chris Sterling – AgileEVM Inc.

Developer of AgileEVM ( www.AgileEVM.com ), a project portfolio decision support tool

Technology Consultant, Agile Consultant and Certified Scrum Trainer

Consults on software technology, Agile technical practices, Scrum, and effective management techniques

Innovation Games® Trained Facilitator

Open Source Developer and Consultant

Software technology, architecture, release management, monitoring, and design consulting for Agile Teams

Publishing book with Addison-Wesley called “Managing Software Debt” - due out Dec 2010

2

Email: [email protected]

Web: http://www.sterlingbarton.comBlog: http://www.gettingagile.com

Follow me on Twitter: @csterwa


http://www.AgileEVM.com






http://chrissterling.gettingagile.com/




http://www.gettingagile.com




© 2009-2010,

Brent Barton - AgileEVM Inc.

President, AgileEVM Inc.

More than 15 years software development in many roles as both employee and consultant for organizations from small start ups to multinational corporations

Former CTO. Active Agile Coach, Mentor, Certified Scrum Trainer

Actively involved in Agile Rollouts from small Product companies to very large IT organizations

Scrum Articles

“AgileEVM – Earned Value Management in Scrum Projects”, IEEE

“Implementing a Professional Services Organization Using Type C Scrum”, IEEE

“Establishing and Maintaining Top to Bottom Transparency Using the Meta-Scrum”, AgileJournal

“All-Out Organizational Scrum as anInnovation Value Chain”, IEEE

3

www.AgileEVM.com Email: [email protected]

Web: http://www.sterlingbarton.comBlog: http://www.gettingagile.comFollow me on Twitter: @brentbarton














© 2009-2010,

Topics Being Covered

Problems Found with Aging Software

Software Debt Explained

Technical Debt

Quality Debt

Configuration Management Debt

Design Debt

Platform Experience Debt

The Wrap Up

A Story of What is Possible

4


© 2009-2010,

Problems Found with Aging Software

Software gets difficult to add features to as it ages

Business expectations do not lessen as software ages

Software must remain maintainable and changeable to meet needs of business over time

5


© 2009-2010,

Lack of emphasis on software quality attributes contributes to decay

6


© 2009-2010,

The “Rewrite”, “NextGen” or “Like-to-like Migration”

“It will be easy since we worked on the original version” - although we understand the domain we will be fighting with new features, technology, tools, and processes

“We don’t have any other options” - Refactoring and test automation are potential alternatives to like-to-like migrations.

7


© 2009-2010,

Limited Platform Expertise

Risk and costs increase as expertise becomes more limited for aging software platforms.

8


© 2009-2010,

Costs for Release Stabilization Increase Over Time

0

125000

250000

375000

500000

Release 1

Release 2Release 3

Release 4Release 5

Release 6

Cost of Fixing Defects Cost for Feature Dev


© 2009-2010,

Extreme Specialization

Knowledge and capability to maintain legacy software decays with time

Costs to maintain rarely used software platforms are higher

Leads to waiting for people in specialized roles to finish their tasks in support of development effort

10


Software Debt

11

Creeps into software slowly and leaves organizations with liabilities


© 2009-2010,

Software Debt Creeps In

12


© 2009-2010,


13


© 2009-2010,


14


© 2009-2010,

Managing Software Debt – an Overview

15


© 2009-2010,

Managing Software Debt

16

It is impossible to stop software debt from creeping into our software

Managing debt in software is based on putting frequent feedback mechanisms in place for code, quality assurance, configuration management, design, and organization of people on project team

Feedback mechanisms should be frequent, automated, and easy to use to support their continued use or modified to new needs


© 2009-2010,

Types of Software Debt

Technical Debt

Quality Debt


Design Debt


17


Technical Debt

18

Issues in software that will impede future development if left unresolved


© 2009-2010,

* Ward Cunningham on “Technical Debt”

Technical Debt includes those internal things that you choose not to do now, but which will impede future development if left undone. This includes deferred refactoring.

Technical Debt doesn’t include deferred functionality, except possibly in edge cases where delivered functionality is “good enough” for the customer, but doesn’t satisfy some standard (e.g., a UI element that isn’t fully compliant with some UI standard).

* Ward Cunningham - “Technical Debt” - http://c2.com/cgi/wiki?TechnicalDebt

19


http://c2.com/cgi/wiki?TechnicalDebt

http://c2.com/cgi/wiki?TechnicalDebt

© 2009-2010,

My Definition of “Technical Debt”

“Technical debt is the decay of component and inter-component

behavior when the application functionality meets a minimum standard

of satisfaction for the customer.”

20


© 2009-2010,

Regression Costs - Manual vs. Automated

21


© 2009-2010,

Principles of Executable Design

22

The way we design can always be improved.

We’ll get it “right” the third time.

We will not get it “right” the first time.

Design and construct for change rather than longevity.

Lower the threshold of pain.

If we are not enhancing the design then we are just writing a bunch of tests.


Quality Debt

A lack of quality will lessen the value per feature added over time

23


© 2009-2010,

Accrual of Quality Debt with Releases

24


© 2009-2010,

Break/Fix Only Prolongs the Agony

25


© 2009-2010,

Effect of Project Constraints on Quality

26


© 2009-2010,

Effect of Project Constraints on Quality

26


© 2009-2010,

Acceptance Test-Driven Development

27


A Fit Case Study

Cost reduction using Fit for test automation and data conversion

28


© 2009-2010,

Manual Regression Testing

Testing was taking 75 person hours during 2 full test runs consisting of:

Comprehensive manual regression testing

Data conversion and validation

Cost for testing was $17,000 each iteration

29


© 2009-2010,

Introducing Fit into Testing Process

After 8 iterations team had introduced healthy amount of Fit fixtures and automated tests

Reduced 70+ hour test runtime down to 6 hours which now included:

Fit automated regression testing

Data conversion and validation automated with Fit fixtures

Reduced cost of testing each iteration from $17,000 to $7,000

30



Unpredictable and error-prone release management

31


© 2009-2010,

Traditional Source Control Management

32


© 2009-2010,


32

Main Branch


© 2009-2010,


32

Main Branch

Version 1Branch

Integrate forVersion 2

CodeComplete


© 2009-2010,


32

Main BranchDebt

Death March

Version 1Branch


CodeComplete


© 2009-2010,


32

Main BranchDebt

Death March {Debt accrues quickly within stabilization periods

Version 1Branch


CodeComplete


© 2009-2010,

Flexible Source Control Management

33


© 2009-2010,


33

Main Branch


© 2009-2010,


33

Main Branch

Version 1


© 2009-2010,


33

Main Branch

Version 1 Version 2


© 2009-2010,


33

Main Branch

Version 1 Version 2{Not Easy! Must have proper infrastructure to do this.


© 2009-2010,

Continuous Integration

34


© 2009-2010,

Scaling to Multiple Integrations

35


© 2009-2010,

The Power of 2 Scripts: Deploy and Rollback

36

Deploy Rollback


Design Debt

Design decays when not attended to so design software continually

37


© 2009-2010,

* Abuse User Stories

38

Implement Securityfor User Information

* From “User Stories Applied” presented by Mike Cohn Agile 2006


© 2009-2010,

* Abuse User Stories

38

Implement Securityfor User Information

As a Malicious Hacker I want to steal credit card information so that I can make fraudulent charges

* From “User Stories Applied” presented by Mike Cohn Agile 2006



Silos of knowledge and increased specialization will increase cost of maintenance over time

39


© 2009-2010,

How to Combat Platform Experience Debt

Ignore it (I do not suggest this!) Surround existing functionality with automated functional tests

Wrap platform interfaces with adapters

Transfer knowledge of platform to more people

Rewrite on more current platform

Move thin slices of functionality to newer platform

Start platform upgrade discussions and rearrange teams into known effective team

40

~OR~


© 2009-2010,

Team Configuration Patterns

Virtual Team Pattern

Integration Team Pattern

Component Shepherd Pattern

Team Architect Pattern

41


© 2009-2010,


EnterprisePlanning

42


© 2009-2010,


Pros

Share architecture ideas and needs across teams

Based on verbal communication

Cons

Usually singles out special Team Member role

Could lead to top-down architecture decisions

IT may gain extensive influence and begin to run Product Backlog prioritization for architecture needs

43


© 2009-2010,


44

Feature Development

Integrate Features

All features are implemented

and integrated every iteration


© 2009-2010,


Pros

Reduces complexity on Feature Teams

Forces delivery from Integration Team instead of interface and deployment designs

Cons

Perpetuates specialized roles

Don’t always work on highest value Product Backlog items

45


© 2009-2010,


46


© 2009-2010,


Pros

Share more knowledge within organization to minimize platform experience debt

Work on highest value Product Backlog items

Cons

Not always optimal as using individual knowledge

Difficult to learn multiple systems across Teams

47


© 2009-2010,

Feature Team Pattern

48


© 2009-2010,

Feature Team Pattern

Pros

Team owns architecture decisions

Decisions are made close to implementation concerns

Cons

May not have appropriate experience on Team

Team could get “stuck” on architecture decisions

49


What is possible?

High quality can be attained and enables accelerated feature delivery.

50


© 2009-2010,

A Story: Field Support Application

2000+ users access application each day

Application supports multiple perspectives and workflows from Field Support Operations to Customer Service

Team of 5 people delivering features on existing Cold Fusion platform implementation

Migrating to Spring/Hibernate in slices while delivering valuable features

36 2-week Sprints, 33 production releases, and only 1 defect found in production

So, what was the defect you say? Let me tell you…

51


Lets wrap this up...

What should I take away from this?

52


© 2009-2010,

Principles for Managing Software Debt

Maintain one list of work

Emphasize quality

Evolve tools and infrastructure continually

Improve system design always

Share knowledge across the organization

And most importantly, get the right people to work on your software!

53


Thank you

Questions and Answers

54


© 2009-2010,

Chris Sterling – AgileEVM Inc.

Developer of AgileEVM ( www.AgileEVM.com ), a project portfolio decision support tool

Technology Consultant, Agile Consultant and Certified Scrum Trainer

Consults on software technology, Agile technical practices, Scrum, and effective management techniques

Innovation Games® Trained Facilitator

Open Source Developer and Consultant

Software technology, architecture, release management, monitoring, and design consulting for Agile Teams

55

Email: [email protected]

Web: http://www.sterlingbarton.comBlog: http://www.gettingagile.com

Follow me on Twitter: @csterwa
















Managing Software Debt - Federal Reserve Bank

Technology

impede future

agile technical

managing software

software debt

chris sterling

agile consultant

software technology

quality debt