Architecture in an Agile Organization Managing our Software as a Valuable Asset while Delivering Incrementally Chris Sterling Principal Consultant, Certified.

Architecture in an Agile OrganizationManaging our Software as a Valuable Asset whileDelivering Incrementally

Chris SterlingPrincipal Consultant, Certified Scrum Trainer, and Agile Coach

SolutionsIQ

Mickey PhoenixSenior Software Development Engineer

SolutionsIQ

2Copyright © 2008 SolutionsIQ. All rights reserved.

Architecture in an Agile Organization

• As companies begin to embrace Agile methods, such as Scrum and Extreme Programming (XP), questions about architecture begin to emerge.– How is architecture implemented in an Agile organization? – What is the software architect's role with Agile development? – How should architecture be discussed with business stakeholders?

• Too often, Agile software development methods are tagged as "architecturally weak" or "disconnected from the realities of delivering large systems." However, leading software architects are pushing through the noise to rapidly evolve the Agile architecture discipline.

• This session will focus on the experiences of two experts and the approaches they have taken to better align businesses with architecture goals.

• Presented by:– Chris Sterling, Principal Consultant and Certified Scrum Trainer at SolutionsIQ– Mickey Phoenix, Senior Software Development Engineer at SolutionsIQ

2


Topics to Discuss

•Self-Organizing Project Teams

•Practices versus Guidelines

•Managing Software Debt

•Coherent Architecture Strategy

•Where can we start?

3


Questions for You...

•What does “architecture” mean to you?

•What issues are you having currently with architecture?

•Why do we worry about architecture?

4

Self-organizing Project Teams


* “The best architectures,

requirements, and designs emerge from

self-organizing teams”

* The Agile Manifesto - http://www.agilemanifesto.org

http://www.agilemanifesto.org/


* Self-organizing Project Teams

•Autonomy: top management provides guidance, money, and morale support

•Self-transcendence: team seems on never-ending quest for “the limit” within guidelines set forth by top management

•Cross-fertilization: team consisting of varying functional specializations, thought processes, and behavior patterns

* From Harvard Business Review Jan/Feb 1986“The New New Product Development Game” by Hirotaka Takeuchi and Ikujira Nonaka


Self-organizing Doesn’t Happen Overnight

Practices versus Guidelines


•Practices: techniques, methodologies, procedures, and processes that are used to get the job done

•Guidelines: aim to streamline particular processes according although, by definition, following a guideline is never mandatory. Guidelines are an essential part of the larger process of governance to make delivery more predictable, and presumably of higher quality

Practices versus Guidelines


Example Practices

•Code Review

•Threat Modeling

• Filling out Document Templates

•Test-Driven Development

•Pair Programming

•Continuous Integration

11

Practices are not bad but may impede a team’s self-organization.


Example Guidelines

•75% Code Coverage

•Audit Trail for all System User Interactions

•Have all Source Control Check-ins Reviewed

•Each Team Member Checks in an Artifact every Day

•Applicable Features Pass Security Audit Policies

13

Managing Software Debt


•Software gets more difficult to add features to as it ages

•Business expectations do not lessen as software ages

•Software must remain maintainable and changeable to meet needs of business over time

• Lack of emphasis on quality attributes to keep software maintainable

The Problem...


๏ Like-to-like migrations๏ Limited experts capable to work on a

system๏ Long release stabilization periods๏ Increasing duration of costly regression

test phases๏ Higher costs for people to work on

legacy software

Software Asset Liabilities


Like-to-Like Migrations

• “It will be easy since we worked on the original version” - although we understand the domain we will be fighting with new features, technology, tools, and processes

• “We don’t have any other options” - Refactoring and test automation are potential alternatives to like-to-like migrations.

17


Limited Expertise

Risk increase as expertise becomes more limited for an aging software system.


Long Stabilization Periods


Costly Regression Test Phases


Higher Costs for People

• Legacy software expenses increase with time

• High demand for limited reserve of experienced programmers (ie. COBOL)

• Specialized tools have a limited following with higher bill rates

21


Software Debt Creeps In

Shows a relatively new system with little debt accrued.


An aging software system slowly incurs significant debt in multiple functional areas.



An older system has accrued significant debt in all functional areas and components.



Types of Software Debt

•Technical Debt

•Quality Debt

•Configuration Management Debt

•Design Debt

•Platform Experience Debt

25

Technical Debt

• Issues in software implementation that will impede future development if left unresolved


• Technical Debt includes those internal things that you choose not to do now, but which will impede future development if left undone. This includes deferred refactoring.

• Technical Debt doesn’t include deferred functionality, except possibly in edge cases where delivered functionality is “good enough” for the customer, but doesn’t satisfy some standard (e.g., a UI element that isn’t fully compliant with some UI standard).

* Ward Cunningham - “Technical Debt” - http://c2.com/cgi/wiki?TechnicalDebt

* Technical Debt

http://c2.com/cgi/wiki?TechnicalDebt

Quality Debt

• A lack of quality, either technical or functional, will minimize value per feature added increasingly over time


Accrual of Software Debt


Break/Fix Only Prolongs It

A Fit Case Study

• Cost reduction using Fit for test automation and

• data conversion


•Testing was taking 75 person hours during 2 full test runs consisting of:

Comprehensive manual regression testing

Data conversion and validation

•Cost for testing was $17,000 each iteration

Manual Regression Testing


•After 8 iterations team had introduced healthy amount of Fit fixtures and automated tests

•Reduced 70+ hour test runtime down to 6 hours which now included:

Fit automated regression testing

Data conversion and validation automated with Fit fixtures

•Reduced cost of testing each iteration from $17,000 to $7,000

33

Introducing Fit into Testing Process


Bug or Enhancement?

In Bug Tracking System

In Product Backlog


• Came from important customer.

• Must be fixed and deployed within 2 weeks.

The Situation: A Bug


• Must be in next release which is 1 month away.

• Involves multiple user stories currently estimated to be delivered in 3 weeks based on velocity of Team.

The Situation: New Features


The Situation: Should we work on bugs or enhancements first?• Fix bug and deploy within 2 weeks

• or• Finish 3 weeks of enhancements to deploy in

next release

• The Catch: The release is due to go out in 1 month and doing both will not fit into this

timeframe.

37


•Put all work into Product Backlog

•Make tradeoff decisions based on

reality of the situation

Maintain One List of Work

Definition of Done

• Define what is involved in delivering a potentially shippable product increment each iteration


Definition of Done


Capture Resulting Artifacts

•Be careful capturing delivered artifacts in Definition of Done• Should capture resulting artifacts

• Minimize capture of transitory documentation

• Minimize capture of practices

• Different for each Team but may include enterprise guidelines when appropriate

41

Configuration Management Debt• Debt in this area can be

unpredictable, prone to error, and cause poor decision-making on quality attributes of a system


Traditional Source Control Management

Main BranchMain BranchDebtDebt

Death March

{Debt accrues quickly within stabilization periods

Version 1Branch

Integrate forVersion 2

CodeComplete


Main BranchMain Branch

Version 1 Version 2

{Not Easy! Must have proper

infrastructure to do this.

Flexible Source Control Management


Continuous Integration


Scaling to Multiple Integrations

Design Debt

• Design decays when not attended to so put effort into maintaining your software by designing all the time


•Refactoring: a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.*

•Merciless: having or showing no [mercy - showing great kindness toward the distressed]

•Relieve your distressed code through kindness and disciplined restructuring

* From http://www.refactoring.com/

Merciless Refactoring

http://www.refactoring.com/


•Refactoring cannot be done effectively without automated tests surrounding code

•Start by creating automated test which fails

• If difficult to create at unit level look at automated acceptance tests from functional perspective

•Over time look for ways to create automated unit tests

Automate Testing to Support Refactoring

Design Debt• When the cost of adding new

functionality to an existing system is more than writing it from scratch


* Architecture Risk Themes

• Observed that twice as many risk themes are risks of “omission” as are risks of “commission”. That is, risk themes identify decisions or investigations that were never made rather than those that were made and could lead to undesirable consequences.

• No discernible relationship between articulated business and mission goals of a system and risk themes from an ATAM evaluation of that system

• No discernible relationship between domain of system being evaluated and risk themes associated with development of that system

* From “Risk Themes Discovered Through Architecture Evaluations” by Len Bass, Robert Nord, William Wood, and David Zubrow

Proper Architecture Design Provides Value• Software is a business asset and our job is

to produce the greatest Return on Investment (ROI) by delivering high value quickly


* Abuse User Stories

Implement Securityfor User

Information

As a Malicious Hacker I want to steal credit card information so

that I can make fraudulent charges

* From “User Stories Applied” presented by Mike Cohn Agile 2006

Platform Experience Debt• How many of you are writing

Legacy Code?


How to Combat Platform Experience Debt

• Ignore it (I do not suggest this!)

•Make interface to underlying system

•Transfer learning of platform to more people

•Rewrite system on more current platform

•Slice functionality to more current platform over time


Architecture Team Patterns

•Virtual Architect Pattern

• Integration Team Pattern

•Component Shepherd Pattern

•Team Architect Pattern

56


EnterprisePlanning

Virtual Architect Pattern


•Pros

• Share architecture ideas and needs across teams

• Based on verbal communication

•Cons

• Usually singles out special Team Member role

• Could lead to top-down architecture decisions

• IT may gain extensive influence and begin to run Product Backlog prioritization for architecture needs

Virtual Architect Pattern


Feature Development

Integrate Features

All features are All features are implemented and implemented and integrated every integrated every

iterationiteration

Integration Team Pattern


•Pros

• Reduces complexity on Feature Teams

• Forces delivery from Integration Team instead of interface and deployment designs

•Cons

• Perpetuates specialized roles

• Don’t always work on highest value Product Backlog items

Integration Team Pattern


Component Shepherd Pattern


•Pros

• Share more knowledge within organization to minimize platform experience debt

• Work on highest value Product Backlog items

•Cons

• Not always optimal as using individual knowledge

• Difficult to learn multiple systems across Teams

Component Shepherd Pattern


Team Architect Pattern


•Pros

• Team owns architecture decisions

• Decisions are made close to implementation concerns

•Cons

• May not have appropriate experience on Team

• Team could get “stuck” on architecture decisions

Team Architect Pattern

Coherent Architecture Strategy


Product Domain Model

•Domain-Driven Design

•XP Metaphor

•System of Names

•Glossary of Terms

66


COTS (Commercial Off-The-Shelf)

•COTS does 80% of what we need usually

•We conduct custom configurations to enable rest of what we need

•Automate functional tests for all customizations

•Automated functional tests can be executed against future upgrades to see what “broke”

67


Automated Tests

New New FeaturFeatur

ee

SystemSystem

Design ChangesDesign Changes

AutomatAutomateded

RegressiRegressionon

Test RunTest Run

Where do we start?• Knowing areas to improve on is

one thing. How can we make progress from where we are today?


• Focus on guidelines rather than practices to influence quality of team deliveries

• Maintain one list of work

• Emphasize quality

• Evolve tools and infrastructure continually

• Improve system design always

• Check in project artifacts more often (Continuous Integration)

• Automate builds from compile to deploy

• Share knowledge across organization

• Hire the right people to work on your software!

Some Suggestions


Principles of Agile Software Quality

•The system always runs

•No code is written without a failing test

•Zero post-iteration bugs

* From “Flawless Iterations” presented by Alex Pukinskis at Agile 2005

71

Thank you• Questions and Answers


Chris Sterling

• Principal Consultant, Certified Scrum Trainer and Agile Coach at SolutionsIQ

• Consults on enterprise architecture and Agile software development practices across a spectrum of industries

• Founder of the International Association of Software Architects (IASA) Puget Sound chapter

• Email: [email protected]• Blog: http://chrissterling.gettingagile.com

mailto:[email protected]

http://chrissterling.gettingagile.com/


More from SolutionsIQ at Agile2008

Domain-Specific Testing Languages (DSTLs)DSTLs help you keep the core testing tool simple while creating automated test scripts the customer can easily read, verify, and use as requirements- Rand Huso, Senior Software Development Engineer

Narrative Testing: Tools for Story Test-Driven DevelopmentIncrease your customers’ confidence in testing by leveraging script-based

testing Tools and DSTLs to express Story Tests in the user’s own language.- Mickey Phoenix, Senior Software Development Engineer

Panel Discussion: Troubleshooting Distributed Agile Team Projects Leading Agile experts Esther Derby, Hubert Smits, Tamara Sulaiman, Samir

Shahjoin Monica Yap to share their experiences working with distributed Agile teams.- Monica Yap, Engagement Manager, ScrumMaster, Agile Coach

Punctuated Continuity: Using Ritual and Ceremony to Avoid Process FatigueLearn techniques that can be employed to keep repetitive Agile routinesinvigorating, pulled from actual experiences with teams practicing XP and

Scrum. - Michael Tardiff, Agile Team Lead and Coach

74

http://www.solutionsiq.com/about-us/our-people.php


More from SolutionsIQ at Agile2008

• Swarming: The Birds and the Bees and Agile• Discuss the fascinating set of swarming behaviors in the animal

world that resonate strongly with some of the central tenets of Agile development.

- Dhaval Panchal, Agile Coach, Analyst, Certified Scrum Practitioner

Assembling a Real-Time Collaborative Development Platform in the Cloud

• SolutionsIQ CEO demonstrates a whole platform for Agile development featuring mashups of SaaS and open-source tools for development and continuous integration

- Charlie Rudd, Chairman and CEO

75


Thank You!

• Come to the SolutionsIQ booth at Agile 2008– Pick up a free Agile t-shirt, and– Schedule one-on-one sessions with SolutionsIQ speakers

• Visit solutionsiq.com/agile2008 for additional Agile 2008 materials and related content from SolutionsIQ

About SolutionsIQSolutionsIQ offers a full spectrum of services to develop software and fulfill technical talent needs, while improving your Agile knowledge and capabilities. Clients include AT&T (Cingular), Amazon, Corbis, Expedia, Federal Home Loan Bank, InfoSpace, Key Bank, Nike, Nordstrom, Regence Blue Shield, Safeco, US Bank, and Washington State University. A Microsoft Gold Certified Partner, SolutionsIQ is also a member of the Java Community Process, Scrum Alliance, Software Association of Oregon, and Washington Technology Industry Association. Learn more at www.SolutionsIQ.com.

76

http://www.solutionsiq.com/agile2008

http://www.solutionsiq.com/

http://www.solutionsiq.com/


http://www.ScrumNoir.com

Previous Issues

http://www.ScrumNoir.com/

Architecture in an Agile Organization Managing our Software as a Valuable Asset while Delivering Incrementally Chris Sterling Principal Consultant, Certified.

Documents

software ages software

overnight slide

managing software debt

software maintainable

ikujira nonaka slide

agile development

agile architecture discipline

aging software system