Agile.Next – Accelerating Product Agilityyowconference.com.au/slides/...AgileNextAcceleratingProductAgility.pdf · Agile.Next – Accelerating Product Agility Dave Thomas Bedarra

2013-09-15

1

©2004 Bedarra Research Labs. All rights reserved.

Agile.Next – Accelerating Product Agility

Dave Thomas

Bedarra Research Labs, YOW! Conferences, Carleton University, Queensland University of Technology,

Lodestone Foundation, Depth First

Outline

The Demand for Greater Agility

Products are from Venus - Projects are from Mars

Business Agility – Streamline Your Organization

Technical Agility - Cutting Code Faster

© 2012 Bedarra Research Labs. All rights reserved.

2013-09-15

2


Benefits of Agile Development

Improved Predictability! §  Simplified work flow § Short iterations

§  Collaborative estimation §  Simplified visual progress tracking

Improved Quality! § Improved requirements communication

§ Integrated teams § Test driven development § Continuous integration

Some Improved Productivity § Reduced meetings § Reduced rework

§  Improved Response to change § Improved Communication

§ Reduced Stress


“We’ve given you Objects, Agile, Tools, Hardware, New Offices, Mentors, Great Compensation because we believe you need these to be your best and have great confidence in your abilities to meet our competitive challenges. Here is what here is what the business needs from you. 1. We need to be much faster in selecting and executing high value projects. This means much better estimates of $ and time to value based on our capacity. 2. We need to turn up the dials on productivity while maintaining quality and features.” …. THIS IS YOUR MISSION…THIS TAPE WILL SELF DESTRUCT ….

CEO – All Hands Product Challenge “As long as we keeping do software the same way as everyone else

were have no competitive advantage.”

2013-09-15

3

Better, Faster, and Cheaper!!!

Typical Implications! 1.  Cut out any complicated

functionality even if it is important

2.  Rush it through with missing functionality and too many defects

3.  Deploy a Prototype, followed by a support nightmare

4.  Over staff (in source or outsource deliver a mess late)


Cycle Time!

Software Value Chain

Better, Faster, Cheaper – A New Road?

We must adapt and improve our way of building software to meet the challenge.

1.  Advance – use alternative techniques to communicate, design, estimate, build, test and deploy.

2.  Refactor our organization to streamline and enable more concurrency and reduce cycle time without reducing quality. We need leverage what works and not be constrained by current best practices. If it is slow it has to go!

3.  Explore and Experiment – we need to envision alternatives and evaluate them quickly before betting too much on any approach. We need to fail vast to maximize ROI and time.

4.  Focus on Value – Target resources and innovations to where they will make a difference.


2013-09-15

4

Change is easy, as long as someone else is doing it

A Natural-Born ‘Leader’ Emerges...apparently self

organization needed his leadership

“Who him? ... Oh, he’s the Product Owner you’ve been hired to replace…he failed to

feed the developers on demand

March Sprint 1 2 3 4, Sprint! …..


Leadership Matters!

Has integrity/stands by values, Leads by example, Executes with transparency, Has the wisdom of experience , Articulates the

vision, Says when they are wrong, Judges fairly, promotes trust, Delegates authority, Coaches versus directs, Promotes

constructive diversity, Makes timely consistent decisions, Sense of humour, Positively motivates and educates, Always tries to

find the best wrong answer, Has time to listen


“Agile Leadership is Fragile” Nigel Dalton

Sustained Leadership is still the Largest Challenge

2013-09-15

5

•  Respect isn’t optional

•  Colored Hats ; What’s your color; Who stole my cheese; Psychology of Computer Programming

•  Working with difficult people

•  How different people Learn and Change

•  Respect the Organizational API – Lead with Humility

•  Dilbert invisible to our Dilbert?

•  Embrace competent strong opponents

•  Diversity Imperative – Human, Geographic and Technical


Software Sociology Considered Essential

Specialists are Essential, Special People are Not!

Specialist •  Expert in one or more technical or business areas •  Deep experience in one or more technical or business areas •  Generalist understanding of other areas

Special Person •  I can’t be with the team because …. •  I don’t use the same wrong tools, process because … •  That is now how we do it in UX, DB, OPS, … •  I only do x, y or z because I’m special


2013-09-15

6

Collective Ownership

Collective Ownership (more than Pairs)

4 eyes on the code •  shares knowledge •  reduces risk •  reduces stress •  prevents defects Achieved by pairing, reviews, inspections...


Executive

Technical Director

Team Leader

Individual Contributors…

Individual Contributor…

Management Ladder

Lean Software Organization Technical Ladders, Playing Coaches and Communities

Distinguished Engineer

Principal Engineer

Outstanding Contributor

Technical Ladder


Management Architects

Leads

Customer Product

Mgr. Tools

Process Infrastructure

Platforms Coaches

Release Deployment

Support Test

Driven Development

Products

Learning COPs

Align Compensation with Work Products and Goals

2013-09-15

7

Lean – Ouch! Thinking?


Lean is a top down process to improve the software value chain by reducing waste, improving flow and continuous improvement

Agile is a bottom up team centered process for improving development predictability and quality.

Agility is the ability of the business to respond to rapidly changing demands.


Lean Thinking – The Zen of Agile

2013-09-15

8

Lack of common vocabulary, rhythm and tools

Too many meetings – 1/meetings is a productivity metric

Lack of transparency

…


Lean: Examples of Software Waste

Lack of understanding (requirements, architecture, design)

Lack of training (requirements, architecture, design, development, test)

Defects (requirements, architecture, design, programs, tests)

Unmanaged supplier risks

Unnecessary fire drills (feature requests disguised as defects)

Excess repair vs. timely replacement

Manual testing

Unnecessary process artifacts

Big Up Front (Analysis, Architecture, Design,

Too many any dependencies

Gold Plating – Requirements, Code, Tests, Tools/Frameworks

Lean – KISS (Keep It Simple Stupid) Staff your team with people who have a track recorder for timely

quality delivery

Fix price your consultants and make sure the delivery with acceptance tests and reward with fast acceptance payment

Reward Tangible Requirements with Timely Quality Deliver

Business ensures the value is as stated by deliverables into their performance plan

Avoid dependencies on anything especially infrastructure, reorgs, space, expertise, tasks forces ...

Reduce meetings to increase available work effort and make the ones

Be a wildly optimistic downside planner, prepare for everything major to fail, second guess your own plan


2013-09-15

9

Projects are from Mars - Products are from Venues


Projects are about apps and resource; Products are about Features IT sees feature teams == project teams; Products require component and feature teams; Projects App is all we need, reuse is optional and unlikely; Product Architecture and Reuse is Essential Project focus is my App ; Products all about Interfaces, Dependencies Project apps can evolve : Products need Design to Last Projects resources pooled ;Products own their $, backlogs, teams … Projects delivered incrementally ; Products need a schedule and deliverables for major functionality

IT Project versus Product Architectures

Component Break Down Structure (Software Bill Of Materials)

Application Layer

Service Layer

Platform Layer HW/SW Encapsulation, Drivers, Protocols

Means subsystem

Means component, framework or library

App Tier SOC Tier Data Tier

My Project

My

Pro

ject

My Project

2013-09-15

10

Lean and Agile Values and Principles

Product Development Activities At-A-Glance

Envisioning

Product Owner Team Common Work Practices

Definition Development DevOps

Prototypes/Models

Requirements Backlog

Risk Backlog

Team… Team…

GUI Guidelines

Architecture

Product Backlog

Release Backlogs

Team…

Team…

Team…

Team…

Potentially Shippable Product

Team Release Backlog

Shippable Code Increments

Sprint Release Backlogs

CI&T

CI&T

©2006-2007 Bedarra Research Labs and Object Mentor

Visualize Flow – Put It On The Wall You Can Only Improve what You Can Measure

Artifact Status Reporting Status of artifacts such as personas, problems, use scenarios, stories

Subjective Reporting Qualitative data entered by teams at retrospectives

CI Environment Reporting Status of stories written, stories completed, tests written, tests passed and tests failed

Query & Reporting

Cumulative Flow

Retrospective Reports

Unit and Story Tests


2013-09-15

11

Velocity and Burndown So XP 2000!


Cumulative Flow Diagram

Complete backlogs ensure that all work is visible, not just new feature work

Use Complete Backlogs

Development Work Items A list of the work items that need to be completed by the Scrum team during the Sprint that contribute to the Product being created. Examples include: -- Writing code and tests -- Writing stories -- Writing acceptance criteria and tests -- High-level software design -- Wireframes - Design and Code Reviews -- Software or GUI prototyping -- Feature planning and estimating -- Release engineering tasks -- Beta test plans -- Documentation plans -- Localization plans

Backlog

Non Development Work Items

A list of any work items that impacts the productivity of the team outside development such as: -- Training courses -- Task forces -- Performance reviews and evaluations -- Answering questions from other teams -- Vacations and holidays

2013-09-15

12

Portfolio Management – Managing to Capacity Program Feature Team

P1 F1 Blue

F2 Blue

F3 Red

F4 Red

F5 Red

F6 Red

P2 F7 Yellow

F8 Green

F9 Green

F10 Purple

F11 Purple

P3 F12 White

F13 White

F14 White

F15 White

Component F16 Orange

F17 Orange

F18 Orange

Company Backlog

Program Backlogs

Team Backlogs

Program

Feature

Epic

Story

Task

MRI Mechanical Control

Table Movement

Horizontal Movement

Medical Imaging

Position w/ Joy Sticks

It’s All in Backlogs! Evidence for Action Lumpy Stories – sizes, estimates, clarity

Story Rework Due To Changing Requirements

Uncompleted Stories

Story Rework Due to Integration Problems

Missing Important Functionality

Done but not usable, poor performance

Done doesn’t mean acceptance tested?

More stories keep getting added to the backlog

Lack of business Value/Tangibility

Blocked on Another Team

Blocked waiting on Product Owner

All resources 100+% allocated


2013-09-15

13

Coo! Is that Scrum or Kanban Wall? No It is Our Wall!


My really cool feature …

What is on the back of the card?

Tangible Requirements

•  easily expressed by Business

•  easily understood by IT

•  acceptance easily understood by Business

•  acceptance easily understood by IT

•  acceptance criteria => acceptance test (AT)


Story Acceptance Criteria

2013-09-15

14

10% of overall effort

Envisioning

Competitive Delta Analysis

Practices Brainstorming & visioning Competitive analysis (SWOT) Delta analysis QFD Customer studies Hardware, platform & component evals Prototyping/modeling

Deliverables Requirements backlog Risk backlog Analysis & Verification Reports Prototypes/Models Look-and-Feel Guidelines

Requirements Backlog

Risk Backlog

Customer Field Studies & Interviews

Technology Evaluations

Prototypes & Models

GUI Guidelines

Market & Product Analysis Brainstorming

& Visioning

QFD House of Quality

Prototyping Deliverables Acceptance Criteria

Envisioning develops a clear product vision /roadmap to deliver the right product to the right market using the right technology through consultation with users

and choosers.

Low-Fidelity Envisioning Prototype Example Low-fidelity prototype

§  Initially rough and then later refined drawings §  Interactive branching allowed walkthrough §  User model, task model, task flows §  3 structure and navigation alternatives §  2 visual form alternatives

Concept iterations §  6 iterations (expanding from 8 to 48 screens) §  3 sprints

§  3 internal / 2 external customer sessions

Detail iterations §  3 iterations (148 screens) §  8 sprints §  3 internal / 1 external customer sessions

Investment §  Less than 2% of overall effort

2013-09-15

15

AIL Feature Planning Flow

Development Envisioning Definition

35 stories

25 stories

20 stories

80-100 ids

60-80 days

55-60 ids

5 stories ? ids

4 stories ? ids

5 stories ? ids

20 stories

15 stories

15 stories

? ids

? ids

? ids

15 Key Stories Architecture Diagram GUI Prototype Customer Feedback

80 Clear User Stories 14 Unclear User Stories Feature Breakdown Component Breakdown Dependency Chart Release Backlogs

6 Weeks 2 Weeks

80 User Stories Initially 63 Additional stories after second round of envisioning

45 Weeks

35 stories

25 stories

20 stories

+ 23 later on

+15 later on

+ 25 later on

2 Weeks 2 Weeks

1st Iteration

2nd Iteration

Artefacts

Feature Containing Red, Green & Gold Epics

4 4 5

Story Points Yes but Not End of Estimate Story


Estimates – Do Them Fast and Often

Collectively Owned By Team

§  2 or 3 point estimates to include uncertainty

§  All items in backlog

§  AIL uses Ideal Days

§  Planning Poker (wide band Delphi)

1 Point

2 point

3 point

Low, Medium, High

Relative Points

Ideal Days

2013-09-15

16

Part* Based Estimates – Managing Risk for Large Projects

Activity Examples Worst Best Expected Likely Java Code (classes, methods)

Interface

Pricing Table (table size)

Process (steps)

Product/Data Definition (Tables/Fields)

Rules (Decision Table (conditions/rules)

Forms (screens, widgets)

Printed Forms (page types, widgets)

* Also called Activity

Part and Story Estimates – Managing Risk for Large Projects

Story Estimates

Part Estimates

Risk Window

2013-09-15

17

Essential Architecture == Interfaces (APIs) The essence of the product and skeleton for evolution

•  APIs versioned in the code base

•  Should ALWAYS be able to create the architecture pictures from the code

•  Allow the architecture to emerge during envisioning through key classes and interfaces

•  architect is a role not a job •  ability to effectively communicate •  competent architects understand the technology/team and business

(NOT post technical) •  playing coaches competent smell and read code

•  Try Extreme Design


Showing Progress Through Architecture Create a working ‘thin slice’ through the architecture

Also called tracer bullets, essential use cases

Allows you to test functionality and show progress to stakeholders

Example: ‘S1, S45, S22 and S18, we can show a simple search…’

Database

Component

Component

Server

Client

Platform Layer Service Layer Application Layer

Database

S1

S22

S45

S18

2013-09-15

18

Feature and Component Team Coordination .

Component C

Component B

Component A

Feature Epic

Component Epics

Component C

Feature A Priority 1

Feature B Priority 2

Feature C Priority 3

Component A Component B

Feature A Work

Feature C Work

Feature B Work

Feature A&B Work

Feature C Work

Feature A Work

Faster, Better, Cheaper

How can we reduce the software cycle time?


Needs Solution

Development

2013-09-15

19

Answer - Ship Less Code!


KLOCS (1000 lines of code) Kill! Dependencies Strangle OO Frameworks inject dependencies Data Driven Always More Flexible Than Code

Use Less Objects and Less Code !

Object Refactoring harder than Changing Data and Functions

Many Apps have few if any domain objects!


Table Driven Programming

Rules Decision Table Calculation Spreadsheet Data Validation Domain and Range Table Mapping Lookup Table Flow Data, Work Flow, Message Events, Matches State Table Process, Reports Input-Output Table Acceptance Criteria BDD Domain Models Entity-Attribute Dictionary

Enable Customer Self Service Atom feeds versus APIs Query versus custom Expose Scriptable Services Self Described Data

2013-09-15

20

Table Driven Programming A picture is 1000 words, a table 200 and a diagram 50

Advantages

•  Easily understood by Business, BA, Dev and QA

•  Easy to create, refactor and extend using Excel

•  Modularity through structured tables

•  Consistency /Completeness Checking

•  Easy to version and Diff

•  Efficient Automated Data Driven implementation

•  Data Driven means changes can be “hot deployed” to a running application

Applications Insurance, Banking, Taxation, Healthcare, ATC, Real-time...


Reduce Integration Time and $$$

•  Put ATOM/RSS feeds on our legacy/partner systems – journal files, events …

•  REST and JSONify your services

•  Use ODBC as a simple interface to complex server systems

•  Use a simple MashUp tool to deliver a integrated application view


2013-09-15

21

Script to Save Time and $$$ •  More and more applications are disposable at least in part –

make it work and get it out there, scale it later if you need to ( e.g Facebook …)

•  Use Ruby (IronRuby, JRuby) , Python(Jython, IronPython) , PHP, Groovy, JavaScript, Clojure… to get development productivity

•  Leverage cloud services (map reduce, cloud DB)

•  Leverage core internal and external services via REST/ODBC


Enabling Loose Coupling

•  All APIs are value based and where possible stateless •  Isolation of services in separate processes/machines

•  Simple Pipes and Filters when possible

•  RSS/ATOM feeds from events/updates/logs

•  Query interface such as ODBC

•  Occasionally Disconnected – replication and sync; event source..

•  Simple efficient implementations using co-routines..

•  Orchestration/Composition using Scripting Process •  Service Bus, Messaging.. Node.js, Erlang, Actors

•  FP thinking encourages value orientation and composition


2013-09-15

22


Service Architecture and Design

Improved Testing Value

•  Unit Tests << Component Tests << Acceptance Tests

•  BDD – Software By Example … Programming By Example

•  Look at Randomized Testing aka Quick Check

•  Introduce Design Rule Checking •  Lint, JSHint, Naked Strings •  Type Inference •  Pluggable Types


“Testing shows the

presence not absence of

bugs”*

2013-09-15

23

We all know that testing costs a lot and takes time, especially when working in a changing complex environment. Lets not bother!

What it takes 1.  Modular micro service architecture §  instance deployment and tear down §  loosely data coupling

2.  Simple Functionality Deployment

3.  Stringent Monitoring for Deviant Behavior

4.  Applications – Telecom, Finance, Web, Sensors …


Code-Deploy-Monitor – Testing Considered Harmful!

Let the Hardware Do The Work! $49,000 buys a computer 1 TB RAM with 40 TB disk and 32 cores!

$200 buys 1000 4 GB cpus on Amazon for 1 hour!

•  Automated Build and Test

•  All interesting data is in memory!

•  Inexpensive Data Conversion/Translation

•  Data Compression and Encryption is “cheaper” on multi-core

•  Data Base is an oxymoron

•  Speed and Memory enable Simpler Algorithms

•  Enable End User Computing


2013-09-15

24

Successful Software Development is about a Winning Culture

Software is a team sport, and like all team sports practice, constructive peer feedback, and coaching are essential.

Winning teams need to implicitly know the moves of each player, as well as the movements of the team as whole.

The ultimate expression of process is a culture where building software is more like playing jazz. People Just Do It!

Agile.Next – Accelerating Product Agilityyowconference.com.au/slides/...AgileNextAcceleratingProductAgility.pdf · Agile.Next – Accelerating Product Agility Dave Thomas Bedarra

Documents