ATDD In The Enterprise Context

1© 2008 - 2013 Leffingwell, LLC. and Scaled Agile, Inc. All rights reserved.

© 2008 - 2013 Leffingwell, LLC,& Scaled Agile, Inc. All rights reserved.

Scaled Agile Framework™ is a trademark of Leffingwell, LLC.

Acceptance Test-Driven

Development in the

Enterprises ContextBridging the Gap Between

Business Goals and Code

Alex [email protected]

Training . Certification . Community

[email protected]



The Scaled Agile Framework (SAFe)

The Scaled Agile Framework is a proven, publicly-facing framework

for applying Lean and Agile practices at enterprise scale

Synchronizes alignment,

collaboration and delivery

Well defined in books

and now on the web

Scales successfully to large

numbers of practitioners and

teams

Core values:

1. Code Quality

2. Program Execution

3. Alignment

4. Transparency

®

http://ScaledAgileFramework.com

http://scaledagileframework.com/


Contributors

Principal

Contributors

Drew Jemilo

Alan Shalloway

Colin O’Neill

CommunityEnterprise

Adopters

Associate

Methodologist

Acknowledgements

Alex Yakyma

Creator and Chief

Methodologist

Dean Leffingwell


Code Quality

You can’t scale crappy code

Agile Architecture

Continuous Integration

Test-First

Refactoring

Pair Work

Collective Ownership

Code Quality Provides:

Higher quality products and

services, customer

satisfaction

Predictability and integrity of

software development

Development scalability

Higher development velocity,

system performance and

business agility

Ability to innovate


Software Development Reality?

Even small items experience long lead times as a result of

misinterpreted or misimplemented requirements and requirements

implemented on top of them

ANALYZE

REQSCODE TEST

L E A D T I M E : 2 M O N T H S

New Functionality TOTAL EFFORT ESTIMATE: 5 IDEAL DAYS

HAPPILY WORK ON OTHER

BACKLOG ITEMS

FIND

PROBLEMFIX

FIX

DEPENDENT

FUNC-LITY

RE-TEST

Problems with flow cause:

Highly variable outcome

Disproportional growth of defects

Knowledge depreciation and lack of improvement

Lower motivation and pride of ownership


We Adopt Lean Perspective

Lean focuses on establishing a fast, sustainable flow of value

VALUE

VALUE

VALUE

VALUE

VALUEVALUE

IDEA

CUSTOMER

Lean achieves flow via:

Optimizing the system as a whole

Building quality in

Continuously reducing lead time

Continuous improvement mindset

VALUE STREAM WITHIN THE ORGANIZATION

VALUE


Improvement

Can we improve the flow by optimizing these individual steps?

Requirements

Design

Code

Integrate

Test

Deploy


Acceptance Test-Driven Development

Specify

Requirements, Code, Lean Thinking

Organize Bind Implement Verify


The Great Breakthrough: Agile Development

Iterative incremental development is intended to introduce more

reliable method and measure of progress

“Deliver working software frequently, from a

couple of weeks to a couple of months, with a

preference to the shorter timescale.”

“Working software is the primary measure of

progress.”

http://agilemanifesto.org

http://agilemanifesto.org/


But There Are Problems…

How many points does the star below have?


Two Gaps that Cause Inefficiency

Business Team

Code

1) what business wants vs. what team understands, and

2) what team wants to build an what they actually build have

critical effect on productivity


Telephone Game

Normally communication methods assume certain level of inherent

ambiguity

Different contexts imply different interpretation of the same thing

Natural language is tricky on its own


Assume Less about Requirements

“For up to 12 aircraft, the small display format shall be used.

Otherwise, the large display format shall be used.”

Adapted from:

Erik Kamsties and Barbara Paech, Taming Ambiguity in Natural Language Requirements, http://prof.kamsties.com/download/icssea2000.pdf

“The product shall show the weather for the next twenty-four

hours.”

http://prof.kamsties.com/download/icssea2000.pdf


Assume Less About Software…

The “Arnold Schwarzenegger” example

8 days of indexing

~10, 000, 000 web pages with information about Arnold

Schwarzenegger

Search result for “Arnold Schwarzenegger”: nothing!

Assumptions about your software are very likely false

(The system simply wasn’t recognizing Schwarzenegger as a last name)



Specify



The Team within the Program


Operating with Valuable Backlog Items

Effectively defining and splitting Epics, Features and User Stories is

extremely important at scale

EPIC

Feature

…Story

…

…

Story

Pass 1

Initial

functionality

Pass 2

Everything

else


Proper Tooling Matters

Teams need tools for effective automation of the requirements

backed up by Continuous Integration system



Specify





Simple Solution for Ambiguity

The source for ambiguity is abstraction. Disambiguate with Concrete

Examples!

Is a valid

point

Is NOT a

valid point

A lot less misunderstanding

Examples drive attention

Allow to spot gaps, missing

cases or rules

Makes us think harder


Examples Clarify Ambiguous Requirements

“The customer enters a card and a numeric personal code. If it is not valid then the ATM

rejects the card.” – Referential ambiguity (“it” refers to the “card” or the “code”?)

– Given card “1111 1111 1111 1111” has pin code of “1234” when I enter “2222” then ATM rejects

the card

– Given card “1111 1111 1111 1111” has pin code of “1234” when I enter “1234” then ATM

accepts the card

“For up to 12 aircraft, the small display format shall be used. Otherwise, the large display

format shall be used.” – Lexical ambiguity (“up to 12” – including or excluding 12?)

Adapted from:

Erik Kamsties and Barbara Paech, Taming Ambiguity in Natural Language Requirements, http://prof.kamsties.com/download/icssea2000.pdf

# of aircrafts Use small display

1 YES

11 YES

12 NO

http://prof.kamsties.com/download/icssea2000.pdf


A “Whole Gang” Exercise

The whole gang should be involved in a specification workshop,

working through the edge cases and resolving ambiguities

Using BVIR is also important: everybody can see and reason

about the requirements

• Developers

• Testers

• PO

• SMEs

• other stakeholders

(whenever

achievable)


Specification Workshop and Cadence

First workshop is done a week before the iteration starts

So the team has time “to sleep on it”

An bind some examples to code

Specification Workshop is typically a part of Backlog Grooming and

Sprint Planning

ITERATION N ITERATION N + 1

Initial Spec

WorkshopContinued

Individual

Clarification

… …


What Does It Mean To Be Lean?

ATDD is not just about implementing the requirements correctly

We are not just building software, we are enabling the customer to

achieve user goals

Deliver VALUE, not just software

Scrum:

Working Software

Automation:

Certain coverage with tests/examples

Goal-oriented:

Smart coverage of “goal enablers” first


Analyzing Stories with User Goals

User Story

User Goal 2

User Goal 1

Scenario

Scenario

Scenario

Scenario

…


The Right Size of Stories

Splitting stories is itself a requirements discovery process



Specify





Binding Examples to Code

Requirements bound to code become a “Living Documentation”,

always correct and always up to date

Abstract Reqs Concrete Examples

SYSTEM


Example of a Binding

Example of functionality in Gherkin bound to code


Gherkin

Gherkin is a Business Readable, Domain Specific Language created

especially for behavior descriptions

http://docs.behat.org/guides/1.gherkin.html


Binding First

Binding may require creating new interfaces

Or modifying the existing ones

And / Or changing logic behind them

Binding is created before the functionality gets implemented


!Binding via UI…

Binding via UI

Expensive – hard to implement a binding

Brittle – hard to maintain

Error prone – binding may often be incorrect


Design for Binding!

Binding is simpler when…

UI is “detachable” from the logic

Less logic on data source side

System scenarios are the result of collaborating domain

objects


Deterministic Environment

Deterministic environment means that the test results depend only on the system

under test

Immediate target state:

– Separate environment

– Data is thin, manually maintained and kept under version control

– Configurations are set automatically at every test run (or most of the users are

aggressively “de-rooted”)

Inability to create a deterministic test environment is a very common

reason for failure at test automation

Image adapted from http://commons.wikimedia.org

http://commons.wikimedia.org/


Develop the Skill Gradually

Many teams fail because they try it all-at-once and either drop the

idea soon or “adapt” it away from ATDD to support what they can do

Sprint 1 Sprint 2 Sprint 3 Sprint 4

Few stories, few automated examples

More stories, more examples

Most of the stories, some fully covered

All stories fully covered



Specify





Execution – Don’t Waterfall Sprints

This is an inter-sprint waterfall

This is a intra-sprint waterfall

Sprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5

Sprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5

This is a proper sprintSprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5

Analysis Coding Testing

CodingAnalysis Testing

Analysis Coding Testing

Analy

sis

An

aly

sis

An

aly

sis

Co

din

g

Co

din

g

Codin

g

Te

stin

g

Te

stin

g

Te

stin

g

A A AC C CT T T

A A AC C CT T T

A A AC C CT T T


Vertical Story Slices Eliminate Sprint Waterfall

Story

Example

As a user I can

login to the

system

…Login with

correct credentials

(successful path

only)

…Error handling

for incorrect

credentials

…Client-side

validation of

input


ATDD Fosters Programming by Intention

(aka Top-Down Programming) The process of coding to first reflect the

desirable external behavior and interfaces and then to add actual logic

1

2

3

Define new entities and

properties/methods in a usage context

Create empty entities and

property/method declarations

Add actual logic to satisfy

the examples


Vertical Slices Derived from Examples

Slice 1

Slice 2

Slice 3

Slice 4

Slice 5



Specify





Reliable Measure of Progress

How many older examples aren’t passing now?

Are we building something on top of incorrect functionality?

Are we really done?

The number of non-passing examples represents the reliable

measure of team progress within the iteration

?

How reliable is Burn-Down Chart


Code That Is Less Than Adorable…

Cover existing code gradually as the new stories occur

Cover some ancillary functionality beyond the scope of the story

If there is a dependency that prohibits from binding – carefully break it

Follow SRP to make this refactoring logical and understandable to the

others

In case the code is hard to write a binding against – it is time to

refactor

SYSTEM

…


Collective Ownership of Bindings

Write bindings together (same when you

refactor)

Apply certain coding standards to

bindings

Debug together, it’s fun!

Rotate over the functionality

Bindings, just like any other code, require collective understanding in

order to be effectively maintained


Manual Testing…

Manual testing of bindings is very useful. Sometimes simply failing the

example by modifying it or the binding helps identifying hidden problems…

especially after recent changes in examples or the code.

This should break it, right?

…and it does


The Flow of Work

It is important that the team controls the actual flow of work, operates

under reasonable limits of the work in process

Ready for

Spec

Workshop

Specified Committed Binding Implementing Verified Accepted


Organizing Living Documentation

Gherkin features will be hard to follow unless properly structured into

a meaningful hierarchy

Organize around:

System use-cases

User goals

User personas

Domain aspects

Requirement areas

AVOID:

Organizing around user

stories, features or

epics

More than two levels of

hierarchy



Specify





The Team on the Train

At scale, multiple teams are required to build system-level solutions

Solution


A program splits into…


The challenge with Value Threads

Only one team can bind meaningful tests

All teams are required to disambiguate the

requirements

All teams are required to design a thing

All teams are required to estimate, plan and track

One team may create tests, another test data and

another one knows nothing about the other two


System as a whole…



ScaledAgileFramework.comBrowse the Scaled Agile Framework

Read Agile Software Requirements:

Lean Requirements Practices for

Teams, Programs, and the Enterprise

Get Training, Certification and Courseware

from Scaled Agile Academy

Get help on implementation strategy,

and customizable Scaled Agile Process

and Scaled Agile Framework

Learn how to launch Agile Release Trains

with the Agile Release Train Quickstart

Get help from the experts and the extensive

service delivery partner community

at Scaled Agile Partners

Join the Scaled Agile Framework community

DeanLeffingwell.com/book-agile-software-requirements/

ScaledAgileAcademy.com

ScaledAgile.com

ScaledAgile.com/launch-agile-release-train

ScaledAgilePartners.com

Community.ScaledAgileAcademy.com


Further Reading

http://scaledagileframework.com

Gojko Adzic, Bridging The Communication Gap:

Specification by Example and Agile Acceptance Testing.

Matt Wynne, Aslak Hellesoy, The Cucumber Book:

Behaviour-Driven Development for Testers and

Developers.

Ken Pugh, Lean-Agile Acceptance Test-Driven

Development: Better Software Through Collaboration.

https://github.com/techtalk/SpecFlow/wiki/Documentation

http://scaledagileframework.com/

https://github.com/techtalk/SpecFlow/wiki/Documentation

ATDD In The Enterprise Context

Technology

agile practices

scaled agile framework

trademark of leffingwell

requirements design

organization value

software development

reqs code test

code alex yakyma