PuppetConf 2016: There is No “I” in DevOps – Bart Driscoll, Dell EMC

Dell Technologies - Internal Use - Confidential1

There is no “I” in DevOps @ ScaleChange Culture in the Enterprise

Your Speaker

IT leader and evangelist within EMC Global ServicesPassionate about driving IT Performance through lean and agileFocused on application development and deployment operations (aka. DevOps)

R. Bart DriscollDirector, DevOps Services @bartdriscoll

https://infocus.emc.com/author/bart_driscoll/bart.driscoll@dell.com

Austin, TX

There is no “I” in DevOps @ ScaleChange Culture in the Enterprise

Change Culture, Embracing DevOps

C H A N G E

I SPOSSIBLE

Change Culture, Embracing DevOps

1. MEASURE2. RADIATE3. EVOLVE

C H A N G E

I SPOSSIBLE

THREE KEYS:

• Previous experience

• Conference or training

• Personal network

• Blogs, articles, and books

• Other

DEVOPS

PROTOTYPEDEVOPS

It will never work for my team!

Why should I change? Or care?

We already tried something like this before and it failed.

How is this better than what I am doing

today?

But, what about…

You can do that in PROD!

Cool! But, there is no money to scale?

Great! Too busy to help.

Yikes. Not ready for this.

Show me the ROI.

DEVOPS

Tools?

People?

Process?

Valley of Despair

Valley of DespairEARLY

PROTOTYPE

Missing the 3 Keys

MEASURE RADIATE

EVOLVE

Cannot overcome enterprise inertia.

Cannot unlock culture.

Culture is the way of life in an enterprise.

• Behaviors• Beliefs• Values• Symbols

Comprised of …… passed along by communication and imitation from one generation to the next

Generally accepted without thinking

about it.

It is the underlying, undocumented operating fabric.

HARDW H Y I S I T S O

T O C H A N G E ?

Culture is like a knit sweater. It is both pliable and resilient

to change.

Otherwise, it would shred and destroy

the enterprise at the first sign of stress.

“Valley of Despair”Unravel

Re-stitc

It Takes Time

Overcoming InertiaUnraveling Enterprise Culture

MEASURE RADIATE EVOVLEDevelop a resilient,

scalable & extensible

system that adapts to change

Create a proven

destination & pathway for success with strong feed-back loops

Build experts and champions

to guide, mentor, and model new

culture

Why is Measuring Important

• Proof

• Visibility

• Feedback

My Favorite - Title Insurance Company

12,936 Hours of rework per year $970,200

Averaged 98 defects related to deployment

per release

4 Major releases a year for their

primary application

33 person hours to fix, test,

redeploy each deployment

defect

Average Chargeback

$75/person hour

264 Hours of rework per year $19,800

Averaged 2 defects related to deployment

per release

4 Major releases a year for their

primary application

33 person hours to fix, test,

redeploy each deployment

defect

Average Chargeback

$75/person hour

4 98 33x x

4 2 33x x

Why is Measuring Important

• Proof

• Visibility

• Feedback

Dell Technologies - Internal Use - Confidential25But what do we measure?

Andrew Clay Shafer @ Spring Conference 2016

Alignment

ENTERPRISE

IT DEPARTMENT

PRODUCT/APPLICATION

TEAM/INDIVIDUAL

Processes& Models

3-5 Year Objectives/Outcomes

AnnualObjectives/Outcomes

Improvement Targets

Improve Earnings Per Share

e Effi

r pilo

s or l

icatio

f pilo

Application Architecture PracticesPlatform/Server Management ProcessDatabase Development LifecycleRelease Management ProcessDeployment & Provisioning ProcessSoftware Development Lifecycle

AffectedProcesses &

Practices

S.M.A.R.T.Improvement Targets

Deploy Every 28 DaysTeam Level Metrics to Change Culture

Application Architecture Practices

Platform/Server Management Process

Database Development Lifecycle

Release Management Process

Deployment & Provisioning Process

Software Development Lifecycle

# of collaboration with Ops/Infra/Dev/QA on standards# automated tests created to validate standards

# code commits per week (configurations)# automated tests created per change (unit, integrate, capacity)# of conversations, calls, or webex with Infra, Ops, QA, and Dev

# code commits per week (database/schema)# automated tests created per change (unit, func., regression)# of conversations, calls, or webex with DBA, Ops, QA, and Dev

LATER (still manual promote to PROD)

# code commits per week (pipeline)# of collaboration with Ops/Infra on configuration changes

# code commits per week (application/tests)# automated tests created per story (unit, functional, regression)# of conversations, calls, or webex with ProdMgr, QA, and Dev

Metrics Tips

• Make sure you have (or easily can get) the data• Make sure it is measuring the desired outcome• Once new behavior is habit (aka. part of culture) remove

metric and add a new one; you can always bring it back if needed

• Make a big deal of achieving and beating metrics• Always stay aligned to goal and higher level goals/metrics

Overcoming InertiaMinimize the Valley of Despair

MEASURE RADIATE EVOVLEDevelop a resilient,

Create a proven

culture

Dell Technologies - Internal Use - Confidential31John Shook, 2010 MIT Sloan Management Review

Different Approach

CULTURE

VALUES AND ATTITUDES

WHAT WE DO

TRADITIONAL MODEL:Change thinking to change behavior

Distributethen

Inspect

Provethen

Radiate

DEVOPS MODEL:Change behavior to change thinking

APP GOAL+ +CD PIPELINE

The Pilot Team

Develop a Support system for changeCreate Champions

EXTERNAL COACH(es)

INTERNAL CHAMPION(s)

TECHNICAL SME(s)

Industry ExpertiseBroad Experience

‘Outsider’ Perspective

• Recognized leader• Politically savvy• ‘Insider’ Perspective• Pragmatic thought-leadership

• Known technical expert• Deep subject matter expert• Innovator/Early Adopter

• • •

• Cross-functional• Collaborative• Transparent• Iterative• Courageous• Retrospective

Tips for RadiationStart with the right team

“Why should we change?”

“Can’t do that because …?” “Impossible!”

“Can we do that?”

“Cool!”

CHAMPIONS

Tips for Radiation

• Be clear about why?• Do more and teach more• Expect failure, drive out fear• Collaborate and listen more;

compete less (internally)• Measure outcomes

Overcoming InertiaMinimize the Valley of Despair

MEASURE RADIATE EVOLVEDevelop a resilient,

Create a proven

culture

Organizations which design systems ... are constrained to produce designs which are copies of the communication structures of

these organizations

System Mirror and Enforce CultureConway’s Law of System Design

Conway, Melvin E. (April 1968), "How do Committees Invent?“, Datamation

AUTOMATION ALONE

DOESN’T MAKE YOU GO FASTER!

Organizations which design systems ... are constrained to produce designs which are copies of the communication structures of

these organizations

System Mirror and Enforce CultureConway’s Law of System Design

Conway, Melvin E. (April 1968), "How do Committees Invent?“, Datamation

IterateIt is an evolution, not a revolution

* Illustration is from Jeff Patton

LEARNADAPT

Model System HolisticallyOrganize around value creation

TOOLS &PLATFORMS

STANDARDS &PRACTICES

PROCESS &STRUCTURE

SKILLS &TEAMS

Development & Delivery Pipeline

Map tools to workflowCreate Pipelines

Deployment Environments Management & Monitoring

Version Control Build Tools Continuous Integration Tools Test Frameworks

TestManager

ACCEPTANCE STAGINGPRE-COMMIT DEV/COMMIT PRODUCTION

Configuration Managment Artifact Repositories

API FRAMEWORK & LIBRARIES

DevOps Workflow

API FRAMEWORK & LIBRARIES

Think LayersDefine Policy, Standards, Stage Gates, Workflows, etc. bottoms-up

APPLICATION SPECIFIC

ARCHITYPE/PORTFOLIO

LINE-OF-BUSINESS

ENTERPRISE-WIDE

Dependent systems; shared standardsAnd Scale

CRM Functionality Removed from Navigator (Client/Contact/Activity), just Deal Management Remains.

CEDTX Data

Navigator Services

DEALSTX Data

GCIB MSCRMMS Dynamics CRM 2015

DACTX Data

GWB MSCRMMS Dynamics CRM 2015

The next Phase of the Sales Platform:

Navigator: All CRM functionality will be removed from Navigator which will retain only Deal Management functionality.

Informatica will no longer push Clients/Entities and will start pushing read-only Deal data.

CRM Integration Engine will add support for Client/Entity data and will not longer sync Activities or Contacts with Navigator. Its data services will be the provisioning point for this data.

GPDS will be refactored to not update CDB or the legacy CED Orgs. Downstreams must stop using legacy CED Orgs.

Client Plans must move to GCIB MSCRM.

Client/Entity updates are gathered from a CED MQ topic (must research what MRDS plans).

Sales ADS: Theoretically, if GWB MSCRM contains all contacts and activities, it could serve as the aggregation point for that data. However, the Sales ADS will eventually become the single provisioning point and we will incrementally build it to that point. Until GDH is replaced and its downstreams refactored, its data will still flow to GDH.

CRM Integration Engine

FAST Mart

SalesADS

Real-time inserts, updates, deletes: Activities, Contacts, Clients/Entities.

Deal & Admin Codes

DealsAssociates

GDH ProfitClient Financials

1x/month

4x/day

SOSRef DataClient 360

4x/day

GCIB Reporting Lines, Ref

Manual Process

Web Service Direct Call

Informatica ETL

Legend

Out: Plug-inIn: CRM API

MS Dynamics MarketingCloud

Dynamics Marketing Connector

Azure Service Bus

DACClient/Server

NavigatorWeblogic Portal

GCBRTX Data

GDHSales Referrals

Sales Data4x/day

GCB Referrals

GPBS MQ Topic

Sync Data and Aggregated

Contacts, Activities, Clients, Client Plans

, Clie

Tips for Evolution

• Iterate, iterate, iterate• Be holistic by design• API-driven architectures• Define in layers starting with the application• Use dependencies to guide scaling

1. MEASURE2. RADIATE3. EVOLVE

C H A N G E

I SPOSSIBLE

THREE KEYS:

What can you start doing next week?

• MEASURE– Create a team metric or product metric to start changing behavior– Challenge everyone to create a metric that aligns with enterprise goals

• RADIATE– Write a blog, white paper, or submit code to open source project– Create a meet-up, brown bag session– Train some one on your team

• EVOLVE– Revisit your current automation platform

• What are the inputs? Outputs?• How can you make upstream/downstream tasks easier?

Thank you!

PuppetConf 2016: There is No “I” in DevOps – Bart Driscoll, Dell EMC

Technology

Puppet at GitHub - PuppetConf 2013

Package manages and Puppet - PuppetConf 2015

PuppetConf track overview: Puppet Applied

Driscoll Complaint

R10K Workshop - PuppetConf 2014

Driscoll bi sig_15_jun2010

Puppet Labs Partner Network - PuppetConf 2014

Infrastructure as Software - PuppetConf 2014

Martin Driscoll Catalog

Understanding OpenStack Deployments - PuppetConf 2014

Puppet Design Patterns - PuppetConf

Sensu and Sensibility - Puppetconf 2014

Juju + Puppet (Puppetconf 2011)

PuppetConf track overview: Puppet 4

John C. Driscoll

PuppetConf track overview: Windows