Software Project Management Tools Draft 5.0 - July 18, 2008 ...

1

Software Project Management Tools

Draft 5.0 - July 18, 2008

Capers Jones

Keywords: Software project management, software sizing, software cost estimating, software

schedule planning, software quality estimating, software risk analysis, software progress

tracking, software litigation.

Abstract

Software projects have been notoriously difficult to manage well. Cost overruns, schedule

delays, and outright cancellations of software projects have been common occurrences for more

than 50 years. Poor quality when delivered remains a chronic issue. Although software project

management tools are not a panacea, when used by capable managers they do simplify the

complexity of software project management and raise the odds of successful completions of

large software projects. Software project management tools have many divisions and

subcategories. However they can be broadly classified into two major areas of focus: 1) Tools

for planning and estimating software projects before they begin; 2) Tools for tracking and

reporting software project status and costs while projects are underway. Beneath this division

are 12 topical areas and scores of project management activities.

Capers Jones

Chief Scientist Emeritus, Software Productivity Research LLC

Web: http://www.spr.com

Email: [email protected]

Copyright © 2008 by Capers Jones. All rights reserved.

2

Software Project Management Tools

The Tasks of Software Project Managers

This article discusses the tasks of project management and the available tools that support those

tasks. From the frequent failure of large software projects and from analysis of the depositions

and testimony of litigation for software failures, it is apparent that software project managers are

often not well trained in managerial tasks, and often fail to utilize appropriate tools to support

critical tasks such as estimating and progress tracking.

Before discussing the tools of software project management, it is appropriate to consider the

many tasks that software project mangers are likely to confront in carrying out their jobs. The

two major divisions of software project management include planning and estimating before

projects start, and tracking status and expenses while software projects are underway. Beneath

this broad division there are a dozen major functional areas and scores of individual tasks, as

shown below:

1) Software Project Sizing

a. Size in function points

b. Size in lines of code

c. Size in data base contents

d. Numbers of screens

e. Numbers of reports

f. Numbers of interfaces

g. Numbers of files

h. Prediction of requirements creep

i. Prediction of requirements churn

2) Software Project Schedule Planning

a. Project schedules

b. Activity schedules

3

c. Task schedules

d. Critical paths

e. Team assignments

f. Individual assignments

g. Requirements creep

h. Requirements churn

i. Requirements slip

3) Software Project Cost Estimating

a. Estimating method selection

b. Estimating tool selection

c. Project cost estimating

d. Activity cost estimating

e. Task cost estimating

f. Project earned value calculation

g. Creeping requirements cost prediction

h. Churned requirements cost prediction

i. Slipped requirements cost prediction

j. Complexity estimating

4) Software Project Risk Estimating

a. Schedule delays

b. Cost overruns

c. Cancellation

d. Poor quality

e. Poor customer satisfaction

4

f. Litigation for breach of contract

g. Loss of key personnel

h. Potential security problems

5) Software Project Quality Estimating

a. Defect potentials

b. Defect removal efficiency

c. Bad fix injections

d. Error-prone modules

e. Bad test cases

f. Cyclomatic and essential complexity

g. Number of defect removal activities

h. Number of test stages

i. Number of test cases per stage

6) Software Project Outsource Contract Analysis

a. Requirements scope in contract

b. Requirements changes in contract

c. Quality terms in contract

d. Incentives for early completion

e. Penalties for late completion

f. Penalties for poor quality

g. Required tracking and reporting

7) Software Project Commercial Off-the-Shelf (COTS) Acquisition

a. COTS Acquisition Analysis

b. Due diligence prior to major acquisitions

5

c. Warranty provisions

d. Installation schedules

e. Installation defects

f. Learning curves

g. Loss of performance during learning

8) Software Personnel Management

a. Specialist selection

b. Hiring of new employees

c. Appraisals of existing employees

d. Promotions

e. Demotions

f. Terminations

g. Awards

9) Software Project Measurement, Tracking, and Control

a. Financial tracking

b. Earned value tracking

c. Balanced score card tracking

d. Goal-question metric tracking

e. Milestone tracking

f. Issue and “red flag” tracking

g. Quality and defect tracking

h. Change request tracking

i. Requirements creep tracking

j. Requirements churn tracking

6

k. Complexity measurement

l. Benchmark productivity data

m. Benchmark quality data

n. Coordination with other managers

o. Coordination with supply chain managers

10) Software Technology Acquisition

a. Project planning tools

b. Project estimating tools

c. Project tracking tools

d. Defect tracking tools

e. Complexity analysis tools

f. Development methodologies

g. Maintenance and enhancement methodologies

h. Quality control methodologies

i. Methodology management tools

j. Development workbenches

k. Maintenance workbenches

l. Reverse engineering tools

m. Reengineering tools

n. Programming languages

11) Software Mergers and Acquisitions

a. Due diligence for purchaser

b. Analysis of tools in company to be acquired

c. Technology conversion after merger

7

d. Organization planning

e. Reorganization after mergers

12) Software Litigation Support

a. Breach of contract litigation

b. Intellectual property and patent litigation

c. Employee confidentiality litigation

d. Preparation of materials on plaintiff side

e. Preparation of materials on defendant side

f. Depositions if required

g. Testimony if required

As of 2008 automated tools are available for many but not all of the tasks that software project

managers are likely to perform. For common tasks such as schedule planning and cost

estimating there are dozens of available tools. For less common tasks such as litigation support

or due diligence prior to a merger or acquisition, few tools are available.

Project Management Knowledge Acquisition

Academic training in the area of software project management is not as sophisticated as

academic training in basic software engineering or even in other forms of engineering

management. For example there is a shortage of academic training even for important topics

such as quality estimation, risk analysis, and milestone tracking. For more unusual topics, such

as litigation support or commercial off the shelf software (COTS) acquisition academic training

hardly exists at all.

The lack of academic training for software project managers is compensated for in some degree

by training and “bodies of knowledge” or BOK provided by a number of professional

associations. Among the large associations that provide useful information to software project

managers can be found:

Academy of Management (AOM) www.aomonline.org

American Management Association (AMA) www.amanet.org

American Society for Quality (ASQ) www.asq.org

IEEE Computer Society www.computer.org

8

Information Technology Metrics and Productivity Institute (ITMPI) www.itmpi.org

International Function Point Users Group (IFPUG) www.ifpug.org

International Project Management Association (IPMA) www.ipma.org

International Organization for Standards (ISO) www.iso.org

Project Management Association of Japan www.pmaj.or.jp

International Software Benchmark Standards Group (ISBSG) www.isbsg.org

Project Management Institute (PMI) www.pmi.org

Software Engineering Institute (SEI) www.sei.org

Society of Project Management (SPM) www.spm-japan.jp

There are also scores of national and local organizations that also provide information and

seminars on software project management topics. Examples include the Australian Institute of

Project Management & Australian Software Metrics Association & Software Quality Assurance

(ASMA-SQA), the Brazilian Association for Project Management, the Netherlands Software

Metrics Association (NESMA), and dozens of local chapters of the Software Process

Improvement Network (SPIN) in the United States, Europe, South America, and Asia.

Several web sites contain useful links to the project management literature. Among the

commercial links can be found the previously mentioned Information Technology Metrics and

Productivity Institute ITMPI) portal (www.ITMPI.org). Among academic links one of the most

complete is that of Dave W. Farthing of the University of Glamorgan in the United Kingdom

(www.comp.glam.ac.uk). This interesting portal has links to dozens of project management sites

and the publishers of scores of project management books.

From discussions with project managers in large corporations, in-house training of managers in

companies such as IBM, Microsoft, AT&T, Google, and CISCO is actually the most effective

source of information for software project managers. Topics such as quality estimation,

measurement, and risk analysis are often better covered via in-house training than by

universities.

However this form of training is only available for project managers in rather large corporations.

Small companies must depend upon commercial training or upon professional associations, plus

a few universities that do have curricula in software management topics.

Training in specific project management tools is also available from the vendors themselves. For

example fairly sophisticated training in software cost estimating is available from the vendors of

9

commercial estimating tools such as KnowledgePlan, Price-S, SLIM, and SEER. Computer Aid

Inc. and its subsidiary the Information Systems Metrics and Productivity Institute (ITMPI) also

provide training in a number of software management topics.

Overall, training of software project managers is a topic that needs to be improved. Quality

control, economic analysis, estimation, and measurement are all topics where additional training

is needed for entry-level managers just starting their careers.

The History of Software Project Management Tools

Software project management is of course only one of many different forms of project

management, such as home construction projects, electrical engineering projects, or civil

engineering projects such as building roads and bridges. Software project management does

have some distinguishing attributes that seem to differ from other and more established forms of

project management:

1. Because software is abstract and invisible to the eye, sizing software projects is more

difficult than for many other kinds of project.

2. Because software engineering is not highly automated, manual effort by skilled

knowledge workers plays a more important role in software projects than in many other

kinds of project.

3. Because of the large volume of bugs or defects in all software deliverables, quality

estimation and quality control are more important for software projects than for many

other kinds of project.

4. For large software projects in the 10,000 function point range or larger, outright failure,

major cost overruns, and major schedule delays occur more frequently than for many

other kinds of project.

5. Although changing requirements can occur for any kind of project, they occur with

unusual frequency for software projects. Sometimes more than 50% of the delivered

features of software projects are not identified until the project is already in process.

Software requirements changes have been measured to occur at a rate of between 1% and

2% per calendar month.

Long before the computer era and long before software itself, project management was starting

to be studied as a technology that needed formalization. Among the pioneers who dealt with

project management issues include Henry Gantt whose famous Gantt charts circa 1917 facilitated

schedule planning. Later in the 1950’s Willard Fazar’s “program evaluation and review

technique” (PERT) added rigor to isolating and identifying the critical paths which complex

projects needed to follow. On the quality side, W. Edwards Deming revolutionized the statistical

10

analysis of quality. Deming’s work was initially adopted in Japan circa 1950 and later came to

the United States when it became obvious that quality was a major factor in successful

competition. Other pioneers in related topics include Walter Shewart (one of Deming’s

teachers), Phil Crosby of ITT, Joseph Juran, and Frederick Taylor.

In terms of software project management tools as opposed to general-purpose project

management tools, some of the pioneers include A.J. Albrecht and the invention of function

point metrics circa 1973; Dr. Barry Boehm and the development of the famous “constructive

cost model” or COCOMO circa 1981; Watts Humphrey and the development of the capability

maturity model (CMM) circa 1983; and Peter Hill and the foundation of the International

Software Benchmarking Standards Group (ISBSG) circa 1997. A more recent pioneer is Tony

Salvaggio of Computer Aid, Inc. This company is pioneering a flexible form of project

management tool that can add “cartridges” at will for specific project management functions.

The phrase “project management tools” has been applied to a large family of tools whose

primary purpose is sophisticated scheduling for projects with hundreds or even thousands of

overlapping and partially interdependent tasks. These tools are able to drop down to very

detailed task levels, and can even handle the schedules of individual workers. However for

software, there are many other critical areas that also need the support of tools: quality

prediction and measurement; requirements changes, and the earned-value of ongoing projects.

Software projects require additional capabilities to be under full management control.

There are a host of managerial functions that standard project management tools don’t deal with

in depth, such as sizing with function point metrics, contract management, personnel

management, assessments, quality estimating, defect tracking, and the like.

The family of project management tools originated in the 1950’s with the U.S. Navy and Air

Force. Weapons systems and other complex projects were becoming so large that manual

planning and control were no longer feasible. Since computers were starting to become more

powerful, it was natural to want to use computer power as an aid for project managers.

Project management tools are an automated form of several management aids developed by the

Navy for controlling large and complex weapons systems: the “program evaluation and review

technique” (PERT), critical path analysis, resource leveling, and the classic Gantt charts. Such

general-purpose tools can of course be used for software projects, but in addition specialized

tools are needed that support the unique requirements of software projects.

Project management tools did not originate for software, but rather for handling very complex

scheduling situations where hundreds or even thousands of tasks need to be determined and

sequenced, and where dependencies such as the completion of a task might affect the start of

subsequent tasks. Project management tools are general-purpose in nature, and can be applied to

any kind of project: building an aircraft, constructing an office building, or a software project.

11

General-purpose project management tools have no built-in expertise regarding software, as do

specialized software cost estimating tools. For example, if you wish to explore the quality and

cost impact of an object-oriented programming language such as Objective C a standard project

management tool is not the right choice. By contrast, many software cost estimating tools have

built-in tables of programming languages and will automatically adjust the estimate based on

which language is selected for the application.

Since software cost estimating tools originated about 10 years after commercial project

management tools, the developers of software cost estimating tools seldom tried to replicate

project management functions such as construction of detailed PERT diagrams or critical path

analysis. Instead, the cost estimation tools would export data so that the final schedule could be

adjusted via the project management tool. Thus bi-directional interfaces between software cost

estimating tools and generic project management tools are now standard features in the

commercial estimation market.

The latest versions of software project management tools integrate cost and schedule estimation,

quality estimation, risk analysis, and many other features. The Computer Aid “application

insight” tool is an example of a general-purpose tool with the ability to be extended in many

directions via plug-in “cartridges.”

Usage Patterns of Software Project Management Tools

Observations made from depositions and testimony during software litigation for breach of

contract reveals that software projects that end up in court for outright failure or major overruns

usually do not utilize many project management tools. Estimates were often informal and

tracking seriously deficient. By contrast, data gathered while performing assessments and

benchmarks of successful software projects that combined high productivity and high quality

indicate that many specialized project management tools were utilized for both predictions

before the projects started and for tracking and measurement while they were underway.

Table 1 shows the approximate sizes of project management tool features measured in function

points. The table also illustrates the patterns of tool usage noted on “lagging,” “average,” and

“leading” projects. Information on the lagging projects is taken from information discovered

during litigation. Information on the average and leading project is taken from benchmark and

assessment studies by the author.

12

Table 1: Patterns of Project Management Tool Usage

(Tool size expressed in terms of function point metrics)

Predictive Tools Lagging Average Leading

Project sizing (function points) 750 750

Project sizing (lines of code) 250 250 250

Project sizing (requirements changes) 250

Project schedule estimating 1,000 1,500 3,000

Project cost estimating 250 500 3,000

Project quality estimating 2,000

Project risk estimating 2,000

Project value estimating 1,500

Departmental budget planning 2,000 2,000 2,000

SUBTOTAL 3,500 5,000 14,750

Measurement and Tracking Tools

Lagging Average Leading

Earned Value Tracking 2,000 3,000

Project cost tracking 2,000 2,000 2,000

Project milestone tracking 1,000 1,500

Project defect tracking 1,500 2,000

Resource tracking 1,000 1,000 2,000

Cost variance reporting 1,000 2,000 2,000

Productivity measurements 1,000

13

Project benchmark reporting 1,000

Project quality measurement 1,000

SUBTOTAL 4,000 9,500 15,500

TOTAL 7,500 14,500 30,250

Table 1 illustrates two important aspects of software project management tool usage: 1) Leading

or sophisticated project managers use more kinds of tools than do managers on lagging and

average projects; 2) Leading or sophisticated project managers use more of the features or select

more powerful tools than do project managers on lagging or average projects.

The very significant use of project management tools on leading projects results in one

overwhelming advantage: “No surprises.” The number of on-time projects in the leading set is

far greater than in the lagging set, and all measurement attributes (quality, schedules,

productivity, etc.) are also significantly better.

Software project management tools are far from being a panacea, but advanced knowledge of

software project size, potential numbers of defects or bugs, and schedules derived from similar

historical projects can go a long way towards yielding a successful conclusion and eliminating

the rather high odds of failure associated with large software projects.

For large software projects in large corporations, the project will typically be supported by a

“Project Office.” IBM was a pioneer in the utilization of project offices and first deployed them

on versions of the main IBM operating system in the early 1970’s.

Typically a project office will be formed for a major software application in the 100,000 function

point size range. Applications of this large size have hundreds of development personnel and

dozens of subordinate project managers under at least three higher levels of management. A

typical project office for such a large system would contain perhaps half a dozen staff to as many

as 10 personnel equipped with a variety of estimating and planning tools, plus historical data

from similar projects. The project office would assist the individual project managers in cost

estimating, quality estimating, schedule estimating and other predictive tasks. They would also

monitor rates of progress, bug reports, earned value, costs, and other status data as the project

was underway.

Project offices tend to raise the percentage of successful large projects, but failures can still

occur. For example one major cancelled project occurred primarily because the head of the

project office and the head of the development group were political rivals and therefore did not

cooperate well. Incidentally, in large companies and large projects, personal disputes and

dislikes are frequent contributors to software disasters. This topic is difficult to study because

none of the managers involved are likely to be candid.

14

Recent Evolution of Software Project Management Tools

Project management tools, like other forms of commercial software, continue to evolve and add

new features. The following paragraphs summarize some of the recent observations of project

management tool capabilities noted in 2008.

Software Project Sizing: Sizing software projects has been a difficult activity for project

managers. Sizing in terms of “lines of code” primarily involved either comparison to similar

projects or guess work. Adding to the difficulty, there are more than 700 programming

languages in existence and some of these lack any rules for counting code. Further many

software applications utilize several programming languages at the same time. More important,

for large software projects the costs and schedules associated with producing paper documents

and finding and repairing defects costs more than the code itself and takes more time during the

schedule. These non-coding tasks cannot be estimated or measured using lines of code metrics.

Function point metrics have proven to be effective for sizing non-coding work and also

programming, so function points are a good choice for life-cycle economic analysis of software

projects. Sizing in terms of function point metrics can be done with accuracy from requirements,

but by the time requirements are available several iterations of cost estimates are normally

required. Also, function point analysis has been somewhat slow and expensive. Certified

function point analysts count function points only at a rate of perhaps 400 function points per

day. Thus the long schedules and high costs of counting function point metrics have been a

barrier

As of 2008 several new forms of high-speed function point tools have been announced.

Relativity Technologies has announced a tool for sizing legacy applications called “Function

Point Analyzer” or FPA by parsing source code in selected languages. The assertion is that this

new tool provides counts with an accuracy equivalent to normal function point analysis, but is

able to accomplish this in a matter of minutes rather than a matter of days or weeks.

Several other forms of high-speed function point sizing are also under development. Other high-

speed function point methods include: 1) pattern matching against historical projects; 2)

Automated prediction of function points from requirements and specifications; 3) data mining of

legacy source code to recreate synthetic specifications; 4) automated source code analysis to

improve mathematical conversion from source code statements to equivalent function point

metrics.

The last method, of mathematical conversion between source code and function points has

actually existed since 1975. The common name for this method is “backfiring.” The first ratios

of source code to function points were developed by A.J. Albrecht and his colleagues at IBM

during the development of the original function point metric. As of 2008 published ratios are

available for more than 700 programming languages and dialects. However the accuracy of

15

backfiring is low due to variations in code counting methods and variations in individual

programming styles. The backfire method is most effective when based on automated counts of

logical source code statements, as opposed to counts of physical lines.

A problem with function point sizing as of 2008 is the fact that there are several different kinds

of function point metrics. The function point metric defined by the International Function Point

Users Group (IFPUG) is the most widely used. But there are also COSMIC function points,

Mark II function Points, web-object points, Story points, and several national variants from the

Netherlands and Australia, plus usage of mathematical conversion from lines of code to function

points called “backfiring.” All of these variants produce different size predications, and as of

2008 there are no proven conversion rules from one variation to another.

Another new sizing topic getting underway in 2008 deals with the impact of technical and

architectural factors on application size. While user-requested functions are measured with

function point metrics, part of the size of applications is also due to architectural and technical

factors and the “layers” that the application will contain.

Other sizing topics are waiting more research and more empirical data: sizing the volume data

bases and sizing value other than financial value (i.e. customer satisfaction; enterprise prestige)

are two topics that need more research.

Software Project Schedule Planning: There are dozens of commercial tools, in-house tools,

and even open-source freeware tools that can perform critical path analysis of software projects,

as well as other kinds of projects. Examples of such tools include Microsoft Project, Artemis

Views, and TimeLine. However for software projects schedule planning can also be performed

by specialized software estimating tools such as COCOMO, KnowledgePlan, Price-S, SEER,

and SLIM in alphabetical order. There are even newer forms of tools about to be released circa

2008 that can handle schedules. For example the Application Insight tool (AI) by Computer Aid

Inc. integrates schedule planning, risk analysis, cost estimates, and tracking of resources, defects,

and costs. This tool also supports interfaces with other tools, and even allows plugging in

“cartridges” of specialized functions such as quality prediction or function point analysis.

Overall a detailed work-breakdown structure that reaches the level of specific tasks and

individual assignments is the most effective. This level of granularity can only be provided by

automated tools: manual methods are too labor-intensive and also to difficult to update.

As of 2008, schedule estimating remains a difficult challenge for software project managers.

Above 10,000 function points in size, about two thirds of all software projects run late, exceed

their budgets, or are cancelled because the delays and overruns have degraded the return on

investment to a negative number.

16

Software Cost Estimating: Software cost estimation can be traced back to the first IBM

software cost estimating tool in 1973 and also to the first commercial estimating tool which was

the Price-S tool originally marketed by the RCA corporation. These pioneering tools were soon

followed by other commercial software cost estimating tools such as Checkpoint, COCOMO,

Estimacs, ISBSG, SEER, SLIM, SoftCost, SPQR/20 and dozens of others.

These specialized software cost estimating tools were often superior to manual cost estimates,

and they also included schedule and quality estimates as well. Because the software estimating

tools appeared about 10 years after general-purpose project management tools, they support bi-

directional data transfer with general purpose tools such as Microsoft Project or Artemis Views.

Another advantage of the commercial software cost estimating tools is that they contain built-in

knowledge of topics that affect software projects, which are lacking in general-purpose tools.

Examples of these factors include the specific programming languages to be used, the CMM

level of the development organization, the methodology to be used such as Agile development or

Extreme programming, and the structure or complexity of the application. The commercial

software estimating tools also have built-in knowledge of key topics such as the defect removal

efficiency levels of various kinds of inspection and testing. The first commercial estimating tool

that could predict defect quantities and defect removal efficiency levels was SPQR/20, which

entered the market in 1985. This was also the first commercial estimating tool based on function

point metrics, although it predicted lines of code as well.

Large software projects often spend more money on the production of paper documents such as

requirements, specifications, plans, user manuals, etc. than they do on the production of the

source code itself. Estimating these costs lagged for many years because lines of code metrics

could neither measure nor predict paperwork. However modern commercial software estimating

tools can predict the sizes and costs for more than 50 kinds of paper document, which

collectively can accumulate more than 35% of the total development cost of a large software

project. A few software cost estimating tools can even predict the costs of Nationalization, or

converting some documents into foreign languages for international sales.

Software Project Risk Estimating: The major forms of risk for software projects include

outright cancellation due to failure to reach operational stability, cost overruns, schedule

overruns, poor quality, security vulnerabilities, difficult human interfaces, and sluggish

performance. Since software project failures are common, it can be stated that software risk

assessment and risk predictions are either inadequate or not performed with rigor.

As of 2008 there are some risk prediction features in commercial estimating tools such as

KnowledgePlan, SLIM, and SEER. There are also some stand-alone risk models as well as risk

estimation in integrated tools such as the Application Insight tool (AI) by Computer Aid Inc..

Automated risk estimation is becoming very sophisticated circa 2008, and is also very quick.

17

Currently risk analysis by trained risk experts was the traditional and most effective form of risk

abatement through 2008, but automated risk analysis is now about equal in effectiveness and

much faster in terms of speed.. While risks associated with cost overruns and schedule delays

can be predicted early and with good accuracy, risks associated with security flaws are more

difficult. In part this is due to the fact that security vulnerabilities are not completely understood

circa 2008, coupled with the fact that hackers are constantly adding new forms of viruses and

new forms of spyware.

Another aspect of risk management is that of “governance” or ensuring the integrity of software

applications from hacking, security flaws, and possible misuse for fraudulent purposes. The

Sarbanes-Oxley act of 2002 has elevated governance to a very important activity, because it

threatens harsh penalties including criminal charges for software executives and boards of

directors who are lax in governance.

Make no mistake about it, large software projects in the 10,000 function point size range are

among the most risky business endeavors in human history. The major risks for large software

projects include:

1. Outright cancellation due to excessive cost and schedule overruns 2. Inadequate or careless estimating 3. Failure to compare estimates to validated historical data 4. Cost overruns in excess of 50% compared to initial estimates 5. Schedule overruns in excess of 12 months compared to initial estimates 6. Quality control so inept that the software does not work effectively 7. Requirements changes in excess of 2% per calendar month 8. Executive or client interference, so that the project is disrupted 9. Failure of clients to review requirements and plans effectively 10. Security flaws and vulnerabilities 11. Potential misuse or tampering with outputs for fraudulent purposes 12. Performance or speed too slow to be effective 13. Loss of key personnel from the project during development 14. The presence of “error-prone modules” in legacy applications 15. Patent violations or theft of intellectual property 16. External risks (fire, earthquakes, hurricanes, etc.)

From analysis of depositions and court documents in breach of contract litigation, most failing

projects did not even perform a formal risk analysis. Worse, project tracking was so inept that

major problems were concealed rather than being dealt with as they occurred.

Research and prototypes of risk analysis tools are a sign that future kinds of risk analysis may

improve the situation. In particular, security vulnerabilities need better predictive capabilities in

the modern world where viruses, spyware, and denial of service attacks are daily events that

18

affect thousands of companies and millions of software applications. As outsourcing becomes

more and more common, litigation avoidance is also a risk that needs more attention than it has

received to date.

Software Quality Estimating: Quality estimation for software was not well done before the

invention of function point metrics because lines of code could not be used for either measuring

or predicting defects in requirements, design, user documents and other non-code items. Once

function point metrics were applied to the task of quality prediction, it was soon discovered that

errors in requirements and design outnumbered coding errors. Commercial software estimating

tools that could predict the full spectrum of software defects from all major sources

(requirements, design, code, user documents, and “bad fixes” or secondary defects) only became

available in 1985 with the SPQR/20 commercial estimation tool. As of 2008 most of the

commercial software estimation tools can also predict quantities of defects.

Another important aspect of quality estimation is that of defect removal efficiency, or predicting

the percentages of defects that will be found by various kinds of review, inspection, and test

stage. Historical data gathered since the 1970’s indicate that formal design and code inspections

have the highest defect removal efficiency and alone can top 85%. Most forms of testing such as

unit test, new function test, regression test, etc. are less than 35% efficient or only find about one

bug out of three. A sequence of formal inspections followed by six to eight different testing

stages can top 95% in cumulative defect removal efficiency. As of 2008 several commercial

estimating tools can predict defect potentials and defect removal efficiency levels.

Some commercial estimating tools can also predict the specific sequence of inspections and test

stages that should be used to achieve optimal levels of defect removal efficiency. Numbers of

test cases and test coverage can also be predicted for many forms of testing such as unit test

(manual or automated), regression test, new function test, performance test, system test, etc.

Since most forms of testing are less than 40% efficient in finding bugs, it is obvious that a multi-

stage series of inspections and tests will be needed to achieve high levels of defect removal

efficiency. The good news about this fact is that projects that approximate 95% in cumulative

defect removal efficiency have shorter schedules and lower costs than “average” projects with

only about 85% cumulative defect removal efficiency. Maintenance costs are also reduced. Phil

Crosby’s aphorism that “Quality is Free” has not only been proven, but also measured on many

successful software projects. An automated testing company, Agitar, has developed a testing

dashboard that provides interesting data on test results, test coverage, and other relevant topics.

Because the costs and schedules associated with finding and fixing bugs is the most significant

cost element of large software projects and the primary factor leading to cancellation and delay,

quality prediction is a critical element of successful cost and schedule prediction. Poor quality is

a frequent cause of litigation too.

19

Many specific software quality methods such as six-sigma for software, quality function

deployment (QFD), formal design and code inspections, automated testing, error-prone module

removal, and joint application design (JAD) can now be modeled and predicted using automated

tools. The most interesting results come from side-by-side comparisons that show a specific

approach such as six-sigma compared against an identical project that did not use the same

approach.

Software Project Outsource Contract Analysis: Since many outsource contracts are cancelled

and about 5% end up in court for breach of contract, it can be stated that the available tools for

predicting the results of software outsource contracts need improvement. There are some

specialized and proprietary tools available from consulting companies that assist clients in

dealing with outsource vendors. Also, normal commercial software cost estimating tools can be

useful in predicting topics such as schedules, costs, and quality levels for specific projects.

However contract analysis as of 2008 does not have much in the way of sophisticated project

management tools.

What often occurs in litigation involving cancelled projects are side-by-side comparisons of the

project as it was actually performed against an alternate versions that would have used better

methods of quality control and dealing with requirement changes. This feature of producing

side-by-side comparisons of alternate approaches should become a standard part of predictive

tools uses for cost, quality, and schedule planning purposes.

Software contracts often are incomplete or ambiguous in dealing with requirements changes and

also with quality control. All contracts need language and sections that define how changes in

requirements will be funded and what kind of schedule relief will be provided. Contracts also

should deal specifically with quality, such as including clauses that require at least 95% defect

removal efficiency levels on the part of the developing company.

Software Project Commercial Off-the-Shelf (COTS) Acquisition: Most corporations and

government agencies lease or purchase more than half of the software they use on a daily basis.

Operating systems, office suites, enterprise resource planning (ERP) packages, financial and

accounting packages, human resource packages, and a host of others comprise a large portion of

total corporate portfolios. There are very few tools available that can assist in making good

selections of commercial software.

Software vendors are reluctant to publish data on defects in their commercial packages. They are

also reluctant to publish size data in either function point or lines of code form. As a result, it is

hard to ascertain basic information such as after-market defect levels, installation problems, and

other issue. There are some specialized tools available for specific tasks such as estimating the

deployment of ERP packages, but these are usually proprietary tools owned by consulting

20

companies that specialize in ERP deployment. Overall, this is a major gap in project

management tools circa 2008.

Historical data on the numbers of bugs and installation problems in major commercial software

packages such as Microsoft Vista, of the ERP packages from Oracle, SAP, and other vendors is

not available to ordinary consumers and clients, event though the vendors actually have such

data available internally.

As of 2008 there are prototype tools available that can predict defects and deployment costs of

large commercial applications such as Vista and ERP packages. Some of the data comes from

litigation filed against these vendors, so the predictions also include learning curves by users,

installation defects, and even predications of consequential damages or the harm which bugs in

commercial packages can cause to client companies. For example a serious bug in a commercial

financial package caused a client company to restate past year earnings and therefore lose credit

with its bank.

Software Personnel Management: Strictly speaking personnel management is not the same as

project management. However in day to day work, software project managers are also software

personnel managers so they need to be involved in both kinds of activities. As of 2008 there are

dozens of commercial tools for dealing with personnel issues such as hiring, skills inventories,

appraisals, compensation, and other human resource issues. This is one of the most extensively

automated areas of the entire spectrum of management work.

Software Project Measurement, Tracking, and Control: There are scores of commercial and

in-house tools for collecting resource and cost data for software projects. Often these cost and

resource tracking tools are part of corporate-wide tool suites, so they operate for other kinds of

projects besides software projects. However there are some common gaps and omissions in

resource and cost tracking. The most common omission is measuring unpaid overtime.

Software personnel tend to have rather energetic work habits, and often work on weekends or at

home. However accurate measurement of unpaid overtime is rare. Failure to record unpaid

overtime tends to artificially inflate productivity rates.

Another gap in resource tracking concerns the work of part-time specialists such as quality

assurance, data base administration, and technical writers. Since these workers may charge time

to several projects or even be classed as overhead, sometimes their effort on specific projects is

missing or inaccurate.

Yet another kind of gap is the work performed by users when they participate in requirements

analysis or are assigned to project teams as may occur when Agile development is used.

From on-site studies where software project personnel were interviewed and asked to compare

their actual effort to the results of project tracking systems, it was noted that the “leakage” from

21

many corporate resource tracking systems was in excess of 25%. That is more than one fourth of

the actual effort was not recorded. Unpaid overtime and the work of users and specialists were

the most common omissions. However, some companies also failed to track managerial effort

and only reported technical effort by the programming team or software engineers. Leakage of

more than 50% was noted in some cases.

An interesting form of project tracking has been developed by the Shoulders Corporation for

keeping track of object-oriented projects. This method uses a 3-Dimensional model of software

objects and classes using Styrofoam balls of various sizes that are connected by dowels to create

a kind of mobile. The overall structure is kept in a visible location viewable by as many team

members as possible. The mobile makes the status instantly visible to all viewers. Color coded

ribbons indicate status of each component, with different colors indicated design complete, code

complete, documentation complete, and testing complete (gold). There are also ribbons for

possible problems or delays. This method provides almost instantaneous visibility of overall

project status. The same method has been automated using a 3-D modeling package, but the

physical structures are easier to see and have proven more useful on actual projects. The

Shoulders Corporation method condenses a great deal of important information into a single

visual representation that non-technical staff can readily understand.

Software Technology Acquisition: A weak link in the chain of project management tasks is

that of technology acquisition. With more than 700 programming languages available, more

than 40 named design methods, and more than 30 forms of development method is not easy to be

sure that the languages, tools, and methods selected for any given project are in fact the best

available.

There are many important questions that are not easy to answer: when should Agile methods be

used, and when are Agile methods inappropriate? Is level 3 on the capability maturity model

(CMM) sufficient for a specific project, or would level 5 give better results? What kind of

benefits will occur from using the Team Software Process (TSP) and Personal Software Projects

(PSP)? What value does lean Six Sigma provide to specific projects? Will object-oriented

programming languages be suitable for a specific project, or should some other language be

used? Will automated testing be beneficial or not? If formal inspections are used will they

lower testing costs? Are Use Cases more effective than other forms of specification? These are

samples of questions that need accurate empirical data.

There are few if any tools available for technology selection that match specific projects with

the best choices for methodologies, languages, or design techniques. The growing volume of

benchmark data collected by the International Software Benchmark Standards Group (ISBSG) is

starting to provide better data based on historical projects. However more data and more

measured projects are still needed.

22

Some of the commercial software estimation tools provide a side-by-side capability that allows

the same project to be estimated with varying assumptions, such as changing programming

languages, changing design methods, or changing CMM levels. These side-by-side estimates

can be helpful, but not every language or method is supported. However the use of side-by-side

modeling to predict the results of alternate scenarios is definitely a feature that should be added

to all software project predictive tools.

Software Mergers and Acquisitions: Some software managers can spend an entire career

without being involved with mergers and acquisitions. Other managers may be involved as often

as several times a year. Some of the issues with mergers and acquisitions are those of joining

the portfolios of the two companies; with merging the software organizations; and with selecting

the software tools and methods that will be used after the merger is complete, assuming the two

companies use different approaches for development and maintenance. There are no standard

tools available for supporting mergers and acquisitions, although some commercial estimating

tools and personnel management tools can be helpful.

Because the technologies deployed by merging partners are often different, here too side-by-side

comparisons of one set of methods against another would be useful.

Software Litigation Support: Software litigation comes in many varieties such as breach of

contract, patent violations, tax litigation, theft of intellectual property, and violations of non-

competition agreements. For breach of contract litigation expert witnesses are often asked to

predict what a failing project would have looked like if state of the art methods had been used;

i.e. what would the schedule and costs have been had the project been done well instead of done

poorly. Questions about the cost and quality impacts of requirements changes often occurs in

litigation too. For tax litigation, reconstructing the original development cost of software at a

given point in time is a topic that occurs often, as well as questions about what it would cost to

build a new version today.

A combination of software cost estimating tools, software quality estimation tools, and general-

purpose critical path tools are often used by expert witnesses in software litigation. For cases

involving patent violations or theft of intellectual property, specialized tools that analyze the

structure of software applications and produce graphs of that structure may be used. If two

different software applications have common structures, it can be assumed that one version may

have been copied from another version.

For many kinds of litigation it is important to compare what actually took place against an

alternate scenario of what might have taken place had more suitable methods been deployed.

Therefore side-by-side comparisons are an important aspect of litigation support.

23

Because many lawsuits are filed due to poor quality or the presence of major bugs that prevented

the applications from being used successfully, quality prediction and quality measurement are

important aspects of litigation support.

Litigation is a topic that is seldom taught by universities to computer science students, software

engineering students, or even MBA candidates. However litigation is common enough in the

software world so that information on litigation costs, probable duration, and disruption of staff

time needs to be included in software project management tools.

As of 2008 there is a prototype tool that can predict litigation costs and schedules for breach of

contract suits for both the plaintiff and defendant. But the prototype does not deal with patent

violations, theft of intellectual property, criminal hacking, violations of employment agreements,

or other forms of litigation.

The Costs and Value of Software Project Management Tools

The cost of software project management tools ranges from free (for open-source tools or in-

house tools) to more than $5,000 per seat for some commercial software cost estimating tools.

Assuming a normal mix of in-house tools, open-source tools, and commercial tools the

acquisition cost of a suite of software project management tools would cost about $0.50 per

function point using the size of the tools in function points as the base. Thus to equip an

individual software project manager with a suite of tools totaling 30,000 function points would

cost about $15,000 for one manager. Because of discounts and multi-seat licenses, the effective

cost per manager in a large company or a large project would probably be about $5000 for a suite

of project management tools that encompass estimating, planning, risk analysis, quality control,

and tracking of costs and progress.

The average cost to build a successful software application in the 10,000 function point size

range is about $1000 per function point or $10,000,000. However, unsuccessful projects that are

cancelled usually accrue costs of more than $1,500 per function point and are about 12 months

late when they are cancelled. In other words, cancelled software projects in the 10,000 function

point range cost about $15,000,000 and obviously are a write-off without any positive value.

If the failing project becomes subject to breach of contract litigation, the damage costs can be

much higher than the cost of the software itself. But litigation damages are difficult to predict

ahead of time.

A 10,000 function point project would normally have about eight project managers and perhaps

64 development personnel. The costs of fully equipping the software project managers on

10,000 function point projects with state of the art software cost and quality estimating tools is

not very expensive: at an average cost of $5,000 per manager and eight managers, the total cost

would be perhaps $40,000.

24

Thus an investment of $40,000 in state of the art project management tools and quality control

tools has the potential to keep the project within bounds and hence eliminate a $15,000,000

failure. If you subtract the $10,000,000 cost of a successful project from the potential cost of a

cancelled project, the savings due to effective project management with a powerful tool suite

would be about $5,000,000. Thus an investment of $40,000 in software project management

tools could yield savings of $5,000,000 which is a return on investment of $125 for every dollar

expended. This prediction is only an approximation, but the basic principles are probably valid.

The presence of a suite of project management tools is not, by itself, the main differentiating

factor between successful and unsuccessful software projects. The primary reason for the

differences noted between failing and successful projects is that the project managers who utilize

a full suite of management tools are usually better trained and have a firmer grasp of the

intricacies of software development than the managers who lack adequate management tools.

Bringing a large software project to a successful conclusion is a very difficult task which is filled

with complexity. The managers who can deal with this complexity recognize that some of the

cost and resource scheduling calculations exceed the ability of manual methods. They also

recognize that state of the art quality control is a necessary prerequisite for success.

Managers on failing projects, on the other hand, tend to have a naïve belief that project planning

and estimating are simple enough to be done using rough rules of thumb and manual methods.

Further, when projects began to show stress and experience problems, the managers on failing

projects often do not know how to take corrective actions, and may just ignore the problems until

far too late to solve them. Last, managers on failing projects (as noted in litigation) do not

understand the economic value of quality control and hence encounter massive delays and cost

overruns when testing begins.

The Future of Software Project Management Tools

Software project management circa 2008 is facing a number of future challenges even before

successfully solving historical challenges. For the past 50 years the average size of software

projects has been increasing. Since risk of failure and risk of poor quality correlate directly with

size, larger projects require more care in predicting quality and controlling quality than small

projects. This has long been a weakness of both software engineering and software management.

New forms of software such as the service-oriented architecture (SOA) running on new kinds of

platforms such as “cloud computing” are rapidly approaching. New development methods such

as Agile development, Extreme Programming (XP), and Team Software Programming (TSP)

need specialized estimating and planning methods. Security vulnerabilities have long been

troublesome, and in the future will become even more troublesome than in the past. Thus project

planning and estimating are becoming more complex as the industry evolves.

25

Although hundreds of commercial project management tools are available in 2008, many of

these deal only with a small subset of overall management tasks. For example a software project

management tool that can deal with schedule assignments, but which cannot deal with quality, is

not an effective solution because bugs or defects are the primary source of schedule slippage.

Further, current project management tools often do not interface well or share data well with

other tools, so that it is difficult to create a consolidated picture of every important topic. What

would benefit software project management tools for the next generation would be better ease of

sharing data, and an improved ability to link together sets of specialized functions. In other

words, apply the principles of service-oriented architecture to the domain of project management

tools.

From examining scores of cancelled projects and hundreds of costs and schedule overruns, it is

obvious that a few basic topics need to be estimated and controlled in the future much better than

they have been in the past:

Quality control is the prime weakness of software projects. Excessive numbers of bugs and

inadequate methods of defect removal are associated with every project failure and all litigation

involving breach of contract.

Requirements changes are endemic in the software world. It is not possible to stop requirements

from changing, so the solution is to deal with changing requirements in the most effective and

efficient manner. Cancelled projects or those with major overruns tended to be swamped by

unexpected changes for which no schedule or cost relief had been envisioned.

Tracking of progress on software projects has been a weak link. Project tracking omissions or

outright concealment of true status tends to occur in breach of contract lawsuits. When problems

are noted, they should be highlighted and addressed, not concealed and ignored.

Furthermore, gaps in historical cost data are common. Unpaid overtime, managerial costs, and

the work of part-time specialists such as quality assurance and data base administration are often

missing or incomplete in software project cost tracking systems. This is especially true for in-

house projects, but some omissions such as unpaid overtime even occur for outsourced

applications.

Accurate plans and estimates are needed before projects start. But it often happens that even

accurate plans and estimates are rejected by clients or top executives and replaced by impossible

schedule demands. Accurate estimation by itself is not a total solution, but accurate estimation

supported by accurate historical data is likely to be believed. Therefore accurate benchmarks of

historical projects are needed in larger volumes than have been available in the past.

26

The shared goals of the software project management community should be to improve the

professionalism of software project management. We need to be able to develop large software

projects with lower defect potentials, higher levels of defect removal efficiency, lower costs, and

shorter schedules than has been accomplished in the past. The technologies for achieving these

goals are starting to become available, but improvements in academic preparation for software

project managers and improvements in selecting the best methods are still needed.

27

Suggested Readings on Software Project Management

Baird, Linda M. & Brennan, M. Carol; Software Measurement and Estimation: A Practical

Approach; IEEE Computer Society Press, Los Alamitos, CA; John Wiley & Sons, Hoboken NJ;

ISBN 0-471-67622-5; 2006; 257 pages.

Boehm, Barry Dr.; Software Engineering Economics; Prentice Hall, Englewood Cliffs, NJ; 1981;

900 pages.

Brooks, Fred: The Mythical Man-Month, Addison-Wesley, Reading, Mass., 1974, rev. 1995.

Crosby, Phil; Quality is Free; New American Library, Mentor Books; New York, NY; 1979, 270 pages.

DeMarco, Tom; Why Does Software Cost So Much?; Dorset House, New York, NY; ISBN 0-9932633-34-X; 1995; 237 pages.

Fleming, Quentin W. & Koppelman, Joel M.; Earned Value Project Management; 2nd edition; Project Management Institute, NY; ISBN 10 1880410273; 2000; 212 pages.

Galorath, Daniel D. & Evans, Michael W.; Software Sizing, Estimation, and Risk Management: When Performance is Measured Performance Improves; Auerbach, Philadelphia, AP; ISBN 10-0849335930; 2006; 576 pages.

Garmus, David & Herron, David; Function Point Analysis; Addison Wesley, Boston, MA; ISBN 0-201069944-3; 363 pages; 2001.

Garmus, David & Herron, David; Measuring the Software Process: A Practical Guide to

Functional Measurement; Prentice Hall, Englewood Cliffs, NJ; 1995.

Jones, Capers; Program Quality and Programmer Productivity; IBM Technical Report TR

02.764, IBM San Jose, CA; January 1977.

Jones, Capers; Sizing Up Software; Scientific American Magazine; New York NY; Dec. 1998,

Vol. 279 No. 6; December 1998; pp 104-109.

Jones, Capers; Applied Software Measurement; McGraw Hill, 3rd edition 2008; ISBN 978-0-07-

150244-3; 575 pages; 3rd edition (March 2008).

Jones, Capers; Software Assessments, Benchmarks, and Best Practices; Addison Wesley

Longman, Boston, MA, 2000; 659 pages.

Jones, Capers; Conflict and Litigation Between Software Clients and Developers; Version 6;

Software Productivity Research, Burlington, MA; June 2006; 54 pages.

Jones, Capers; Estimating Software Costs; McGraw Hill, New York; 2nd edition, 2007; 644

pages; ISBN13: 978- 0-07-148300-1.

28

Kan, Stephen H.; Metrics and Models in Software Quality Engineering, 2nd edition; Addison

Wesley Longman, Boston, MA; ISBN 0-201-72915-6; 2003; 528 pages.

Kaplan, Robert S & Norton, David B.; The Balanced Scorecard; Harvard University Press,

Boston, MA; ISBN 1591391342; 2004.

Love, Tom; Object Lessons – Lessons Learned in Object-Oriented Development Projects; SIG

Books Inc., New York NY; ISBN 0-9627477-3-4; 1993; 266 pages.

McConnell, Steve; Software Estimation – Demystifying the Black Art; Microsoft Press,

Redmond, Wa; ISBN 10: 0-7356-0535-1; 2006.

Parthasarathy, M.A.; Practical Software Estimation – Function Point Methods for Insourced and

Outsourced Projects; Addison Wesley, Boston, MA; ISBN 0-321-43910-4; 2007; 388

pages.

Putnam, Lawrence H.; Measures for Excellence – Reliable Software On-Time Within Budget;

Yourdon Press, Prentice Hall, Englewood Cliffs, NJ; ISBN 0-13-567694-0; 1992; 336

pages.

Putnam, Lawrence & Myers, Ware; Industrial Strength Software – Effective Management Using

Measurement; IEEE Press, Los Alamitos CA; ISBN 0-8186-7532-2; 1997; 320 pages.

Strassmann, Paul; The Squandered Computer; Information Economics Press, Stamford, CT;

1997.

Stutzke, Richard D.; Estimating Software-Intensive Systems – Projects, Products, and Processes;

Addison Wesley, Boston, MA; ISBN 0-301-70312-2; 2005; 917 pages.

Yourdon, Ed; Outsource – Competing in the Global Productivity Race; Prentice Hall PTR,

Upper Saddle River, NJ; ISBN 0-13-147571-1; 2004; 251 pages.

Yourdon, Ed: Death March—The Complete Software Developer’s Guide to Surviving “Mission

Impossible” Projects, Prentice Hall PTR, Upper Saddle River, N.J., ISBN 0-13-748310-4,

1997.

29