Fllow con 2014

Transforming Traditional Enterprise Software Development Processes“Appling DevOps and Agile Principles at Scale”

Gary GruverSeptember 3, 2014

- FW no longer a bottleneck for the business

- Development costs reduced from $100M/yr. to $55M/yr.

- ~140% increase in the number of products under development

- Capacity for innovation increased from ~5% to ~40%

Requirements Scheduling DevelopmentIntegration & Qualification

Waterfall Development Model

Resources

ScheduleScope

Waterfall Approach: DoEnough up front planningTo lock in all three corners

Requirements Scheduling DevelopmentIntegration & Qualification

Waterfall Development Model

Resources

ScheduleScope

Waterfall Approach: DoEnough up front planningTo lock in all three corners

Waterfall Reality: Discoveriesduring development and difficultieswith integration/qualification resultsin frequent and costly schedule slips

Agile Development Model

Drops of working Complete code onA fixed interval

Plan

Qual Dev

Plan

Qual Dev

Plan

Qual Dev

Plan

Qual Dev

Drops of working Complete code onA fixed interval

• Fully qualified code drops on a fixed schedule• Frequent small integrations• Short planning and development cycles• Delivering to a well prioritized product backlog

Scrum Agile≠

Water

Scrum

Fall

Transforming Traditional Enterprise Software Development Processes

Enterprise Level Continuous

Improvement

Planning & Prioritized

Backlog

Applying DevOpsPrinciples at

Scale

Business Objectives

Business Objectives (Don’t “Do Agile”)

Define your value proposition

Understand yourcost & cycle-time

drivers

Either automate, eliminate, or engineer out the drivers that aren’t key to the value prop

Mini-milestone Objectives

Cascading Objectives to Track Progress

ConversationsLearnings

Agile Adjustments

Having real-time metrics is essential for the speed of agile & aligning the org. But don’t manage by metrics.

Use the metrics tounderstand where to have conversations about what is not getting done.

Interative Approach to Agile Management

Long Term Predictability for SW SchedulesDo we really need the predictability of our current planning processes?Are our current planning processes really that accurate?50% of all SW is never used or does not meet the business objectives!

100%

Acc

ura

cy

Planning Investment

One of the biggest challenges with Agile Planning at the enterprise level is getting the organization to accept the uncertainty in SW development and appreciate the flexibility and opportunity.

Locking in Capacity over time

100%

Cap

acit

y

Time Horizon

Long-range commitments < ~50% of Capacity

Mid-range commitment < ~80% of Capacity

Delivery

Transforming Traditional Enterprise Software Development Processes

Enterprise Level Continuous

Improvement

Planning & Prioritized

Backlog

Applying DevOpsPrinciples at

Scale

Business Objectives

Business Objectives

Increase the quality and frequency of feedback

Reduce the time and resources between release branch & production

Improve deployment repeatability/env. stability

Testing in an Operational like environment as close to dev. as possible

Release testing considerations

Cost of Failure

Ease of Deployment

Architectural

Test Env =Production

DevOps unique challenges

SimulatorsEmulators

Embedded SW/FW

Upgrade Compatibility

Patches

Packaged SW

Continuous Delivery

Web/SaaS

Ensuring the architecture/build system is ready

Comp A v1 Comp B v1

Comp D v1 Comp E v1 Comp F v1

Comp C v1Comp B v2Comp B v1Comp A v2

Comp D v2 Comp E v2

Comp C v2

Comp F v2

Automated system test architecture

• Component based approach to the automated test architecture

Data Magic

Navigation

Page Objects

Finding the offending codeWhat Code?

When? Are you sure it wasn’t Bob?

Building up a Large SW System

Inte

rfac

e Te

st S

imu

lato

r

Agile Comp 1

Agile Comp 2

Agile Comp 3

Agile Comp 4

Agile Comp 5

Agile Comp 6

Legacy Waterfall

IT 1

Legacy Waterfall

IT 2

Legacy Waterfall

IT 3

Agile Comp 1

Auto-revert/Gated Commits

Continuous Integration and Test System

L1 Sim(10-14x/day)

L2 Sim(12x/day)

L3 Emu(6x/day)

L4 Sim(1x/day)

STAGE 1

STAGE 2

Embedded System Pipeline

Cost of Testing

Drive test escapes upstream

1 2 3 4 5 60%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Sim. Quality bar

Emulator

Sim. Regression

Product

ProductEmulatorSimulator

Building up the Enterprise

Gating at the Application or Artifact

Inte

rfac

e Te

st S

imu

lato

rIn

terf

ace

Test

Sim

ula

tor

Gating at the SCM

Gating at the SCM

Developing on Trunk

CD - Picking the right tool for the job

Orchistrator

Trigger ScriptedEnv

Deployment EDD Auto

Testing

Scripting Environments - VariancesDev/QA PRE-PROD PERFORMANCE PRODUCTION

Common Script Common Script Common ScriptCommon Script

Environment descriptors

Dev/QA PRE-PROD PERFORMANCE PRODUCTION

Common

ScriptCommon ScriptCommon ScriptCommon Script

Environment descriptors

Deploy Workflow Complexity

Dev/QA PRE-PROD PERFORMANCE PRODUCTION

Common

ScriptCommon ScriptCommon ScriptCommon Workflow

Environment descriptors

Evolutionary Database

-Add deprecate-Lazy Instantiation

-Don’t alter or Modify-It is like crossing the beams

Finding the offending component quickly improves production

Configure Servers/Routing Device1:

Deploy Code2:

Run a System Test3: ? ? ? ? ? ? ? ? ? ? ? ?

Finding the offending component quickly improves production

Configure Servers/RoutingDevice and Validate Data1:

Deploy Code & Validate Successful Deployment2:

Run a System Test3: Code Validated

Transforming Traditional Enterprise Software Development Processes“Applying DevOps and Agile Principles at Scale

E-mail: [email protected]: largescaleagile.comTwitter: @GRUVERGary