Managing the Evolution of Software Systems: The Old, the New and What's Consistently True by Joe Hessmiller

1

Managing the Evolution of Software Systems

The Old, the New and What’s Consistently True:


By Joe Hessmiller


Why Focus on Maintenance?

End User Call Support

WorkRequests

Admin.& Other Costs

IncidentHandling


Maintenance Is Where the Money Is

• Maintenance and Support Makes Up 50-80% of Software Management Budgets

• And, 100% of the Software Actually Used By the Organization

Maintenanceand Support

Development

SoftwareBudget

65%

SoftwareSupportingBusiness

100%


Software Development and Support Has Enormous Room for Improvement

The Old


1969 NATO Report on Software Engineering

• Documented problems in – Requirements, Design, Coding– Estimates, Monitoring

Progress, Communication– Productivity (26:1), Reliability

(Bugs) – Hardware Dependencies,

Reuse – Maintenance Costs

• Sound familiar??


A Generation After NATO Report• 34% of Projects Completed On-Time, On-Budget

• 48% of Required Features Missing in “Successfully Completed” Projects

• 43% Average Cost Overrun

• 82% Average Schedule Overrun

“So many software projects fail in some major way that we have had to redefine success to keep everyone from becoming despondent...” Source: Tom DeMarco in the book, Controlling Software Projects

Source: Standish Group, Chaos Report, 2003


Individual Performance Variation

• Performance Differences– 10 to 1 Difference in Productivity with

Same Level of Experience• Sackman, Erikson and Grant, 1981• Curtis, 1981• Mills, 1983• Demarco and Lister, 1985• Curtis, et al, 1986• Card, 1987• Calett and McGarry, 1989


Team Performance Variation

“Productivity of the 90th percentile teams is four times higher than that of the 15th

percentile teams”

Source: Barry Boehm. Software Engineering Economics. Englewood Cliffs, NJ: Prentice Hall


What’s Possible?

• Software quality can improve at 15% to as much as 40% PER YEAR

• Software development productivity can improve at rates of 5% to 20% PER YEAR

• Software maintenance productivity can improve at rates of 10% to 40% PER YEAR

- Capers Jones, Calculating the Value of PI, 2000


What’s Been Accomplished?

• HP– 10X reduction in defect rate, average time to fix a defect was cut in

half in one business unit, reduced time to market by 5X over 5 year period in another business unit

• IBM Toronto– 10X reduction in delivered defect rates, productivity up by 240%,

rework reduced by 80%

• General Dynamics Decision Systems Division– rework percent dropped from 23.2% to 6.8%; customer reported

defect rate dropped from 3.20 to 0.19 defects/KSLOC; productivity rose 2.9X


CAI Experience with Productivity Improvement


Productivity Increased in Wide Variety of Scope-Headcount-Cost Situations


Lessons from Modern Manufacturing for Application Development and MAINTENANCE

The New


Why Manufacturing?

• Importance of Thinking “Outside the Box”

• Applicability of Modern Manufacturing Lessons to Software Development and Maintenance

• Participate in the Continuous Evolution of “Production” Ideas


Manufacturing Productivity Outpaces Services

Real Output per Full-Time Equivalent Worker, 1977-2001


First: Think Out of the Box

The definition of insanity is doing the same thing over and over and expecting different results.

- Benjamin Franklin


Thinking Outside of the BoxThe Fremont Story

• 1982 - GM shutters Fremont assembly plant– Worst quality and productivity in GM network– Absenteeism of 20%

• 1984 - GM-Toyota JV (NUMMI)– Hired 5000 former workers (including union leadership)– Implements TPS (later called “Lean”)

• 1986 – Fremont Plant is: – one of the most efficient plants in North America– producing highest rated car in terms of quality in GM– Absenteeism of 3%


Evolution of Manufacturing

CraftProduction

Flexible,High Cost Per Unit

MassProduction

Low Cost Per Unit,Rigid

LeanProduction

Flexible,Low Cost Per Unit

•19th Century – Craft Production

•20th Century – Mass Production

•21st Century – Lean Production


Software “Shortcut” to Lean

CraftProduction

Flexible,High Cost Per Unit

MassProduction

Rigid,Low Cost Per Unit

LeanProduction

Flexible,Low Cost Per Unit

•19th Century – Craft Production

•20th Century – Mass Production

•21st Century – Lean Production

Lessons


What IS Lean Thinking?The Five Lean Enterprise

Principles– Value Focus on What Adds Value for the

Customer

– Value Stream Understand How Value Is Created

– Flow Maximize Speed to Value

– Pull Work on Just-In-Time Basis

– Perfection Continuously Improve Performance

Drive Out Waste


Lean Makes Powerful Impact in Administrative Areas

“ We were doubly shocked when we realized they [144 Japanese firms benchmarked] were also four times more productive per person [than peers in North American companies] in the administrative area.”

- George Koenigsaecker, CME, 2003


Why Lean Works

In reality, companies had come to recognize that combinations of efficiency and flexibility allowed them to meet various and changing customer needs more effectively than simply maximizing one dimension.”

- Michael Cusumano, Japan's Software Factories, 1991


Process, Metrics and People

What’s Consistently True


High Performance Organization Characteristics

“High performance IT organizations across all industries share common process characteristics:– Processes are clear;– The link between process and performance

goals is well understood; and– The performance of processes is well

measured and properly supported.”

Page and Pearson, Transforming the IT Workforce, Outlook 2004, Number 2



Process


Process Should Support Different Motivation and Management Systems

• Douglas McGregor (1960) – Theory X

• Average person dislikes work and will avoid it if possible• People must be forced to work toward organizational goals• People prefer to be directed and avoid responsibility

– Theory Y• Effort in work is natural• People will apply self-control in pursuit of organizational objectives• People seek responsibility

• William Ouchi (1981)– Theory Z

• Implicit, Informal Control• Explicit, Formalized Measures


Why Is Standard Process Important?

Enables …• Common Language to Communicate

Activity, Status, Issues



Enables …• Common Language

• Quality Plans and Improvement Strategies– Discrete Components for Performance

Specification and Impact Assessment– Meaningful Process Metrics




• Quality Plans and Improvement Strategies

• Tool Usage Benefit Maximization– Deployment of Tool Across Organization– Feedback/Tuning Experience




• Quality Plans and Improvement Strategies

• Tool Usage Benefit Maximization

• Successful Partnering– Common Expectations – Clear Interaction Process








What Are the Maintenance Processes?

End User Call Support

WorkRequests

Admin.& Other Costs

IncidentHandling


What Are the Maintenance Processes?

ISO 1764, IEEE 1219


Key Process Areas Mapped

Work Request (WR) Process Flow

Work Request Process FlowLegend

Print WeeklyTime Audit

Time EntryPrint Time

Sheets

Time SheetTracking

Documentation

Update WRQC/CMPL. Dates

Update WRIssue Tracking

Work RequestIssue Form

User AcceptanceTest (Date)

Update WRScope Change

Work RequestScope Change

WR Design &Specification

Program QC(System Test)

Code/UnitTesting

SpecificationQC

W/R Received

Print Customer Sat.Feedback FormW/R Received

Customer Sat.Feedback FormImplementation

ProductionNotification

ApplicationDocumentation

ApplicationBulletin

DB ChangeRequest

Assign DevelopResource

Update WR FinalEstimate

Final Estimate

IdentifyResources

Development

Function -PointAnalysis

Distribute WRReport

DevelopProject Plan

Print WRStatus Report

Log InFeedback

DetermineSolutions

User Returns Cust.Sat. Feedback Form

Update WRPriority

DefineRequirements

IdentifyResources

PrioritizePreliminary

Estimate

Close WR(Checklist Rev.)

MonitorProduction

ProgramTurnover

Create WRTasks/Estimate

MoveSource/Object

ProductionMove Request

UserApproval

User GroupMeeting

Main ProcessFunction

Manual ProcessFunction

Automated ProcessFunction

AcknowledgeWR Receipt

WR Log-InTracking

WR Received

Update WRClose. Date

555453a

52

48403615 20

35

31

51

50

49

47

46

45a

44

43

42

41

39

38

37

34

33

32

30

29

28

27

26

25

2419

17

16

23

22

21

4

10

13

12

7

6

11

9

8

52

3

1

P02

Create WREstimate Task

WR Estimate &Notify Memo

Assign Est.Resources

EstimateApproval

14

18

IncidentNotification

Capture IncidentData

PrioritizeIncident

Classify Incident

Non-ABENDIncident

Identify OutsideResources

Identify Resources

Notify Resources

NotifyResources

AssignResources

Close Incident

P04User Approval

(Close)

Update IncidentTracking Time-Entry

Program Chng.WR Steps 28-45

CorrectiveAction

Incident Log-InTracking

Incident Log-InTracking

IdentifyResources

Capture IncidentData

Batch ProgramJob Abort

ABEND/Job Abort

On-Line Program

Update IncidentTracking

Tracking

No ActionRequired

IncidentInvestigation

CorrectiveAction

AssignResources

CloseIncident

User Approval(Close)

Incident IssueForm

Update IncidentIssue Tracking

EscalationProcess

1 20

2

28 29

27

26

25

3231

376

38

39

21

30

22

2423

19

18

367

1687

176

1211

13

14

1510

9

8

7

53

6

4

Production Incident Process Flow

Production Incident ProcessFlow Legend

Legend


Manual Process Function

Main Process Function

Figure X.XFigure X.X

Incident IssueForm

Update IncidentIssue Tracking

EscalationProcess

33

34

35

Program Chng.WR Steps (28-45)

Log-In TicketTracking

AssignResources

Consulting/Investigation

User Approval(Close)

Time EntryUpdate Ticket

Tracking

Close CallSupport Ticket

IdentifyResources

Fill Out CallSupport Ticket

Enhancement(Work Request)

ClassifySupport

IncidentNotificationConsulting

Night SupportAnalyst

After Hours

Call Support

EscalationProcess

Update SupportIssue Tracking

Support IssueForm

Job Monitor

Business Hours

1

42

21

1920

18

14

15

17

16

13

12

11

10

9 8

7 6

35

Support Process Flow

Support Process FlowLegend


Manual Process Function

Main Process Function

P03


Work Request Process MappedWork Request (WR) Process Flow

Work Request Process FlowLegend

Print WeeklyTime Audit

Time EntryPrint Time

Sheets

Time SheetTracking

Documentation

Update WRQC/CMPL. Dates

Update WRIssue Tracking

Work RequestIssue Form

User AcceptanceTest (Date)

Update WRScope Change

Work RequestScope Change

WR Design &Specification

Program QC(System Test)

Code/UnitTesting

SpecificationQC

W/R Received

Print Customer Sat.Feedback FormW/R Received

Customer Sat.Feedback FormImplementation

ProductionNotification

ApplicationDocumentation

ApplicationBulletin

DB ChangeRequest

Assign DevelopResource

Update WR FinalEstimate

Final Estimate

IdentifyResources

Development

Function -PointAnalysis

Distribute WRReport

DevelopProject Plan

Print WRStatus Report

Log InFeedback

DetermineSolutions

User Returns Cust.Sat. Feedback Form

Update WRPriority

DefineRequirements

IdentifyResources

PrioritizePreliminary

Estimate

Close WR(Checklist Rev.)

MonitorProduction

ProgramTurnover

Create WRTasks/Estimate

MoveSource/Object

ProductionMove Request

UserApproval

User GroupMeeting

Main ProcessFunction

Manual ProcessFunction


AcknowledgeWR Receipt

WR Log-InTracking

WR Received

Update WRClose. Date

555453a

52

48403615 20

35

31

51

50

49

47

46

45a

44

43

42

41

39

38

37

34

33

32

30

29

28

27

26

25

2419

17

16

23

22

21

4

10

13

12

7

6

11

9

8

52

3

1

P02

Create WREstimate Task

WR Estimate &Notify Memo

Assign Est.Resources

EstimateApproval

14

18


Process Step Map



Metrics


What Maintenance Management Measures

• Volume of Service – Measured by the number of units of services and the number of services delivered per unit of time.

• Quality – Measured by defect rates, standards compliance, technical quality, services availability, and service satisfaction.

• Responsiveness – Measured by time to acknowledge, time to implement, and services backlog.

• Efficiency – Measured by cost per unit of service delivered








Quality Management - Performance Over Time


Customer Satisfaction Management


Quality Management - Rework Management


Drill Down into Root Causes – Repeated Data Errors within the Order Processing System

Identify Incident Causes - Data Errors: 34 Hours

Quality Management -Defect Management


Productivity Management - Efficiency Management


Productivity Management - Effectiveness Management


Productivity Objective: More Time Focused on Value Adding Activities

Analysis of Total Year-to-Date Hours

Admin, 57

Call Support, 113

Incidents, 104

Work Requests, 381

Admin

Call Support

Incidents

Work Requests

Analysis of Total Year-to-Date Hours

Admin, 47

Call Support, 102

Incidents, 229

Work Requests, 173

Admin

Call Support

Incidents

Work Requests

Analysis of Previous Month Hours Analysis of Previous Month Hours



People


Motivation is KEY to Productivity Improvement

• Most productivity studies have found that motivation has a stronger influence on productivity than any other factor.

- Barry Boehm, 1981


Achievement and RecognitionRecognition of Achievement is VERY Important to Developers

Accurate, Objective Metrics Meet People’s Needs.


Motivating Developers

• The Work Itself• Achievement• Advancement• Compensation• Recognition

- Cougar, Zawacki, Opperman, MIS Quarterly


What Motivates Developers?


Getting People to Use the System

• Threaten Them• Incent Them

– Make Job Easier Through Automation

– Ensure Recognition Through Metrics

How to Herd Cats








Assignment Management


Task Management


Estimation Management


Issue Management


Time Management


Change (Scope) Management


Quality Management – Step Level


Setting Achievable Goals, Objectively Measuring Performance

How to Implement

Managing the Evolution of Software SystemsIm

pro

ved

Cap

abi

lity

Le

vel

VISIBILITYClassify and Capture

Work, Metrics, Resources and Time

Implement Workflow Processes

Streamline & Craft Work Processes

How They Did It: Three-Phase Implementation Model

CONTROL

OPTIMIZATION


Visibility

• To “Shine a Light” on What is Being Done By Whom and How

• To Capture Performance Metrics; Baseline, Operational and Systemic


Control

• To Produce Meaningful Process and Product Management Metrics

• To Ensure That Performance Meets Customer Expectations (and React Quickly and Effectively To Variation )

• To Provide a Stable Platform for Process Improvement Experiments and Improvement Deployment


Optimization

• To Identify Efficiency Improvement Opportunities

• To Identify Effectiveness Improvement Opportunities


What We Need: Maintenance and the Three Principles

IN CS WR

Visibility What is causing error?

What problems are being

experienced?

What are people working

on?

ControlAre errors

being closed within SLAs?

Are problems being resolved within SLAs?

Are WR being completed

within SLAs/Estimates

?

Optimization

What errors are recurring?

Are the corrective

actions working?

What calls are recurring? Are

corrective actions

working?

What’s causing us to miss

estimates? Are corrective

actions working?


Next Steps

A Best Practice Assessment is Your First Step Toward Realizing the Benefits of Best Practices Organization…

– Visibility – Control – Optimization


Best Practice Assessment Process

Where Are We? 1. Identify Current State

Is Current State = Best Practice? 2. Analyze Gap

Where Do We Want To Be? 3. Develop Future State Vision

How Should We Get There? 4. Prepare Recommendation

The Best Practice Assessment Helps You Answer …


World Leader in IT Process and Productivity

CAI Profile


CAI Company Profile

• Founded 1980

• 2000+ Employees

• Offices in US, Europe, Canada, and Philippines

• 70% of Revenue from Managed Services Partnerships

• Fortune-1000 Customer Base

• World-Class Solution Centers

• Long-term Strategic Partnership Focused



CAI Product Offerings

• Application Support & Maintenance– Managed Maintenance

• Application Development– Construction Management

• Desktop Services & Help Desk – Help Desk, Workstation Support, IMAC

• Staff Augmentation - T&M Consulting


The CAI Value PropositionCAI enables its clients to:• Track and measure Track and measure the rightthe right activities in the IT function activities in the IT function

• Compare IT performance vs. benchmark dataCompare IT performance vs. benchmark data

• Optimize IT performance, using Optimize IT performance, using datadata to make decisions to make decisions

• Achieve double-digit % gains in IT productivity Achieve double-digit % gains in IT productivity

• Establish objective measures of IT worker performanceEstablish objective measures of IT worker performance

This is made possible through the use of:• A unique metrics-based methodology – “Managed Maintenance” A unique metrics-based methodology – “Managed Maintenance”

• The proprietary TRACERThe proprietary TRACER®® software tool and data repository software tool and data repository


Embrace Change

Closing Thoughts


Thoughts on ChangeIt is not necessary to change. Survival is not mandatory.

W. Edwards Deming

Beware becoming change-averse change agents.Barry Boehm, SEI Symposium 2000

In theory, there is no difference between theory and practice. In practice there is. Lawrence Peter Berra


Excellent IT Reference Sitewww.itmpi.org

Managing the Evolution of Software Systems: The Old, the New and What's Consistently True by Joe Hessmiller

Technology

software quality

software projects source

software engineering

organization software

software management

productivity improvement

communication productivity

defectsksloc productivity