Programming Languages and Lifecycle Cost 30 August 1997 Ada Joint Program Office Center for Computer Systems Engineering Joint Interoperability & Engineering.

Programming Languages and Lifecycle Cost

30 August 1997

Ada Joint Program Office

Center for Computer Systems Engineering

Joint Interoperability & Engineering Organization

Defense Information Systems Agency

Language and Lifecycle Cost 2 mm970822.ppt

PURPOSE

Provide Quantitative Information on Lifecycle Cost Implications of Programming Language Use

(Ada* vs C/C++)

* Includes Ada 83 and Ada 95


OUTLINE

• Introduction

• Cost Factors

• Language Studies

• Project Examples

• Findings


CAVEATS

• Language debate is often highly subjective and inconclusive– Different perspectives produce different positions

– Many have negative reaction to Ada mandate

• Qualitative arguments do not resolve key issues– “Ada is too slow”; “No Ada programmers”; “No Ada tools”

– “C code is cryptic”; “C is unsafe”; “C maintenance cost high”

• Quantitative cost and productivity data difficult to obtain– Lifecycle cost data requires discipline to collect

– Productivity data is often company proprietary

– “Controlled experiment” implementing equivalent large, complex applications in multiple languages is exceedingly expensive

• Technical language characteristics considered only indirectly– Discussed only with respect to impact on lifecycle cost


SOURCES

• Commercial Examples– Dr. John McCormick (SUNY)– Kurt Welker (USAF)– Ulf Olsson (CelsiusTech)– Dr. Stephen Zeigler (Rational)

• Information Sources– Currie Colket (SPAWAR 332)– Les Dupaix (STSC/Hill AFB)– Michael Schrank (MITRE Corp.)– AEGIS Community (Various)– NASA, IEEE, ACM, etc.

• Study conducted by:– Michael Masters (NSWCDD)– James Readan (NSWCDD)– J. Hoppel (CSC/KGO)– W. Wright (CSC/KGO)


BACKGROUND

• C created in 1970s to support system software development– “Portable assembly language” - Dennis Ritchie (C co-developer)

• Ada 83 designed to address issues related to complex systems• C++ added object-oriented features and some type checking to C• Ada 95 revision based on a decade of experience with Ada 83

– Real-time and distribution upgrades

– Object-oriented features

– “Programming in the large” upgrades

• C in widespread use in commercial industry; C++ use increasing• Ada 83 used predominantly in complex/real-time system market


SOFTWARE GROWTH

10000

100

1000

10

1

1960 95908580757065

GEMINI 3

GEMINI 12

APOLLO 7APOLLO 17

SKYLAB 2

MISSION CONTROL: GROUND STATION

MERCURY 3F-111

P-3A

AWACS

APOLLO 7

P-3A

B-1A

B-1BSHUTTLE/OFT

SHUTTLE/OPERATIONAL

F-15EB-2

GEMINI 2

APOLLO 11S-3A

SHUTTLE/OFT MANNED SYSTEMS

C-5A

F-111

F-15

F-16 C/D

A-7D/E

GEMINI 3

PERSHING 1

TITAN

E-2C

SKYLAB 2TITAN 34D (IUS)

PERSHING 11(ED)

VIKING

C-17PROJECTED

GALILEO

PERSHING 11(AD)

MISSILE

TRIDENT C4

VOYAGERTITAN IIIC

PERSHING 1APOSEIDON C3

SURVEYORMARINERVENUS MERCURY

UNMANNED

UNMANNED INTERPLANETARY

MANNED A/C

MANNED SPACE

MANNED SPACE CONTROL

UNMANNED SYSTEMS

ATFPROJECTED

Source: USAF Software Technology Support Center

Pro

gra

m S

ize

(KS

LOC

)

Year

Combat System Element CPU Family CPUs/ Machine Operating System

Equipment Suite

Hull Cut-In Language

ORTS Upgrade HP 770 1 HP-UX TAC-4 DDG 79 AdaC2P Rehost, CDLMS HP 743 1 HP-UX UYQ-70 DDG 84 CMS2*, C

MVME-167 1 VxWorks UYQ-70SGS-AC Mark 162 DDG 68 AdaADS Mark III (Consoles, ADS-1A/ 1B, ADS-2A/ 2B)

HP 743 1 HP-UX UYQ-70 DDG 79, 81 C

ATWCS Phase 1 HP 750 1 TAC-3 DDG 73 Ada, CATWCS Phase 2 HP 750 1 TAC-3 DDG 79 Ada, CASWCS MARK 116 Mod X DDG 84SQQ-89(v) 10: - SIMAS, UYQ-65 SPARC 10 1 HP-UX TAC-4

DDG 79C

- EMSP Maint Console HP 750 1 HP-UX TAC-4 CSQQ-89(V)X tbd tbd tbd tbd DDG 84 tbdGWS FFCS tbd tbd tbd tbd DDG 72, 81 Ada, CMS2

BFTT - Unmanned Rack - BOPC

Motorola 68040 HP 770 1 HP-UX

TAC-4 TAC-4

DDG 79C C

ACTS Rehost HP 743 3 ATOE TAC-4 DDG 79 AdaAN/ SPS-67(V) COTS Motorola 68040 2 DDG 77VLS OCD Motorola 68040

Power PCtbd tbd Mark 162 DDG 79 C

SARTIS Intel 80386 UPX-34 CG Fortran, AssemblerComposite ID DDG 79 COpeval 98 CEC CES Motorola 68040 6+ VxWorks AN/ USG-2(V) CG CACSIS DEC 8400 AXP 6 VMS C, PascalID Upgrade (C&D-1) HP 743 3 HP-RT/ ATOE UYQ-70 DDG 79 AdaESSM (WCS-1) HP 743 3 HP-UX UYQ-70 DDG 79 AdaISMS HP 770 1 HP-UX TAC-4 DDG 79 CFODMS Maint Console HP 770 1 HP-UX TAC-4 DDG 79AOCD (NGP) HP 743 1 HP-UX UYQ-70 DDG 79 CNAVSSI HP 770 1 HP-UX TAC-4 DDG 77 AdaNTCS-A HP 700 Series 2 HP-UX TAC-4 DDG 51Note 1: Blank indicates information not available Note 2: Tbd indicates decisions have note been madeNote 3: MVME-167 is a Motorola 68000 series Note 4: "C" for Language indicates use of C or C++Note *: CMS2 compiler for Motorola 68000 series by UNISYS


• Introduction

• Cost Factors



• Findings

OUTLINE


COST FACTORS

• Commercial viability • Standardization and portability

– Ada 83 and Ada 95 are ISO standards

– C is ANSI standard; “It will probably be years before the C++ standard is finally adopted.” H. Schildt, ANSI C++ committee,1995

• Lifecycle Productivity– Training, programmer productivity, reuse, maintainability

• Process and tools– High cost of Ada 83 compilers; Ada 95 GNAT is freeware

– Added cost of C/C++ “process” to provide features inherent in Ada

• Reliability and safety– Ada compiler/runtime validation

– Ada technical features: type checking, run-time range checks, restricted pointer use, library consistency, determinism, etc


3GL (g.p.) M SLOC

Ada 83 49.70

C 32.50

FORTRAN 19.55

CMS-2Y/M 18.55

Jovial 13.80

C++ 5.15

Pascal 3.91

PL/1 1.00

Other 3GLs 4.22

TOTAL 148.38

Generation M SLOC

First 3.90

Second 26.30

Third (gen.purp.) 148.38

Third (special) 3.70

Fourth 5.00

Fifth 0.29

TOTAL 187.57

DOD WEAPON SYSTEMS

Source: Institute for Defense Analyses survey, P-3054, Jan 95


NON-DOD Ada EXAMPLES

• Non-DOD Government Use– DOE AdaSage (200K SLOC)

– FAA Display System Replacement (DSR) (2.3M SLOC)

– NASA Space Station

• Commercial Use– Boeing 777 (10M SLOC; 2-4M will be inherited/reused)

– “DEC Design” tool

– Motorola Cellular Phone

– Nippon Communication Products (over 4M SLOC)

– Rockwell Beech Starship (375K SLOC)

– Rockwell Global Positioning System

– GM Heavy Truck transmission computer

– Volvo Automated Parts Delivery

– Weirton Steel Automated Rolling Mill (500K SLOC)

– Genesis Prompt Paymaster (250K SLOC)

– Reuters NY Stock Exchange Trader program (250K SLOC)


COMMERCIAL Ada USE

Air Traffic Control

AustraliaBelgiumBrazilCanadaChinaCzech RepublicDenmarkFinlandFranceGermanyGreeceHong KongHungaryIndiaIrelandKenyaNetherlandsNew ZealandPakistanScotlandSingaporeSouth AfricaSpainSwedenUnited KingdomUnited StatesVietnam

Banking and Financial Networks

Reuters news serviceSwiss Postbank Electronic Funds

Transfer system

Commercial Aircraft

Airbus 330Airbus 340Beechjet 400A (US business jet)Beech Starship I (US business

turboprop)Beriev BE-200 (Russian forest fire

patrol)Boeing 737-200, -400, -500, -600, -700,

-800Boeing 747-400Boeing 757Boeing 767Boeing 777Canadair Regional JetEmbraer CBA-123 and CBA-145

(Brazilian-made regional airliners)Fokker F-100 (Dutch DC-9-size airliner -

American Airlines flies these)Ilyushin 96M (Russian jetliner)Saab 2000Tupolev TU-204 (Russian jetliner)

Communication and Navigational Satellites and Receivers

INMARSAT - voice and data communications to ships and mobile communications

Intelsat VIINSTAR (Nippon Telephone and

Telegraph)PanAmSat (South American Intelsat-like

consortium)United States Coast Guard Differential

Global Positioning System (GPS)Rockwell Collins NavCore V GPS

receiverESA/Alcatel-SEL GPS receiverTDRSS Ground Terminals - NASA

Scientific Satellites

Cassini command subsystemENVISAT-1 - European Space Agency

(ESA), Earth observation satelliteXMM - ESAEOS - NASA's Earth Observing SystemGOESRadarSat (Canada)UK Space Technology Research Vehicle,

auxillary payload on Ariane4

Railway Transportation

Cairo MetroCalcutta MetroCaracas MetroChannel TunnelConrail (major U.S. railway

company)French High-Speed Rail

(TGV)French National RailwaysHong Kong Suburban RailLondon UndergroundParis MetroParis Suburban Rail

Television Industry

Canal+ (French pay-per-view TV, remote cable box control software)

Medical Industry

JEOL Nuclear Magnetic Resonance

Source: George Washington University EE & Comp. Science Dept.


LANGUAGE USE

“What are your top three development tools?”

Percentage of

Tool Companies using

PowerBuilder 44

Visual BASIC 42

COBOL 40

C++ 20

Access (DBMS) 18

Oracle (DBMS) 14

Borland Delphi 8

Source: Forrester Research Inc. survey responses from 50 Fortune 1000 companies

Changing Language Use

Percent Change

Language from 1993 to 1994

C / C++ -32

FORTRAN 49

Visual BASIC 60

COBOL 75

Smalltalk 200

Source: Dataquest survey of Microsoft Windows application programmers, Communications Week, Aug 1995Source: IEEE Computer, March 1996


FOUNDATION LANGUAGES

Source: University of Michigan Computer Science Dept. survey

WORLDWIDE USA

Ada

Modula5%

C11%

C++10%

Scheme12%

Pascal40%

Others6%

16%Ada17%

Modula8%

C9%

C++8%

Scheme11%

Others12%

Pascal36%

Pascal

Ada

Modula

C

C++

Scheme

Others


Ada AS A FOUNDATION

• University of Alabama

• Auburn University

• California State Long Beach

• Clemson University

• Gallaudet University

• Georgia State University

• University of Maryland

• University of Missouri

• University of New Orleans

• State University of New York

• Penn State University

• University of Texas Austin

• West Virginia University

Number of institutions teaching Ada as a Computer Science foundation

Year CS1 CS2/7 Total

1991 20 20 40

1992 30 27 57

1993 52 33 85

1994 63 37 100

1995 97 41 138

1996 110 36 146

1997 114 36 150

Source: George Washington University EE & CS Dept. survey


STANDARDIZATION

• Ada 83 is an ISO standard• Ada 95 is an ISO standard• C is an ANSI standard• C++ is not yet a standard

– “It will probably be years before the C++ standard is finally adopted.” H. Schildt, ANSI C++ committee,1995

– Results of recent ISO canvas for C++ standard:• 10 in favor• 6 against• USA abstained• Britain lodged approximately 85 pages of comments


Ada VIABILITY

• Increasing acceptance in USA– 50M SLOC 3GL code base in DOD Weapon Systems

– Increasing non-DOD and commercial use

– Ada increasingly taught in universities

– Ada 95 provides enhanced object oriented features• GNAT Ada95 freeware compiler likely to encourage use• JAVA WWW language has partial C++ syntax and Ada95 semantics

• High degree of acceptance in Europe– NATO requires Ada; European vendors rarely request waivers

• High degree of acceptance in Far East– Compiler sales greater than US and Europe combined

Ada market appears stable or increasing


OUTLINE

• Introduction

• Cost Factors



• Findings


LANGUAGE STUDIES

• USAF Study, “Ada and C++: A Business Case Analysis,” 1991– Sponsored by Lloyd K. Mosemann, SAF/AQK

• USAF Software Technology Support Center (STSC) surveys– Les Dupaix of STSC tracked software business case info since 1990

– Monitored numerous government studies (over 100M SLOC)

– Survey of 11 European aerospace and defense contractors, 1995

• Institute for Defense Analyses survey• MITRE Software Maintenance study

– “Ada in the Maintenance Arena,” Schrank, Boyce & Davis, 1995

• Other sources: NASA, IEEE, ACM, AT&T, Ada Information Clearinghouse, etc.

• AEGIS Experience


AIR FORCE STUDY

• “Ada and C++: A Business Case Analysis,” 1991– Five independent studies sponsored by Lloyd Mosemann, SAF/AQK

• Institute for Defense Analyses study of tools and training– Tools and courses are available

• Software Engineering Institute used IBM / FAA ranking method– Bottom line scores (scale 0-100): C++ = 64, Ada = 79

• CTA Inc. analyzed cost factors from Reifer Consultants database– Productivity (KSLOC/SM): All = 183; C++ = 187; Ada = 210– Error rate (Errors/KSLOC): All = 33; C++ = 31; Ada = 24

• TRW detailed analysis of typical info systems & C3I projects– Info system ranking: C++ = 1324; Ada = 1631 (23% better)– C3I system ranking: C++ = 1401; Ada = 1738 (24% better)

• Naval Postgraduate School addressed policy issues– Recommended continued investment in Ada


STSC EUROPEAN SURVEY

• BT Systems, Sweden• CelsiusTech, Sweden• CRI, Denmark• Dassault Electronique,

France• FMV, Sweden• Heise Software, Germany

Note: Software products from the above companies represent an Ada code base of over 25M SLOC

• SATT Control, Sweden• Sextant Avionique,

France• TERMA, Denmark• Thompson CSF, France• Volvo, Sweden


• Introduction

• Cost Factors

• Language Studies– Productivity

– Program Integration

– Reliability & Maintenance


• Findings

OUTLINE


DOMAIN PRODUCTIVITYP

rodu

ctiv

ity (

SLO

C/M

M)

Avionic/Sonar

Command& Control

DataProcessing

Environ. Comm. AI0

100

200

300

400

500

600

AdaAll Lanugages(including Ada)


Based on over 100M SLOC of military and commerical software


DOMAIN COST

$/SOLC

Avionic/Sonar Cmd & Control Data Processing Environment Telecomm.

0

20

40

60

80

100

120

140

160

180

200

AdaAll Lanugages(including Ada)




Source: “Ada vs C++: A Business Case Analysis,” CTA Substudy,

July 1991

SLO

C /

MM

0

50

100

150

200

250

Standard (all lang.) Ada Long TermFirst Ada Project

PRODUCTIVITY

0

2

4

6

8

10

12

SLOC / Hour

118%

Improvement

627%

Improvement

Historical Ada (First Ship)

Ada & Reuse

Source: CelsiusTech, Sweden


NASA Ada REUSE

Project 1

(86/87)

Project 2

(87/88)

Project 3

(88/89)

Project 4

(88/89)

Project 5

(88/90)

Project 6

(88/90)

0

10

20

30

40

50

60

70

80

90

100

Total Reuse Verbatim Reuse

Source: NASA/Goddard Space Flight Center, Oct 91

% R

eu

sed


DISECONOMY OF SCALE

• Barry Boehm:*– “The more members that are added to a

team, the more . . . time is consumed in communication with other team members . . .”

– “This decrease in [programmer] productivity on larger projects is called a diseconomy of scale in economic terms.”

• Methods with different impacts on productivity/cost will have a relatively greater impact as program size increases

* Source: Software Engineering Economics, Barry Boehm, Prentice-Hall (COCOMO model)

Program Size (KSLOC)

Pro

duct

ivity

(S

LOC

/SM

) *

050

100150200250300350400

2 8 32 128


PRODUCTIVITY IMPROVEMENT

Lessthan500

500to 5k

5k to50k

50kto

350k

350kto

750k

over750k

-5

0

5

10

15

20

25

% Im

pro

vem

en

t

Ada productivity compared to other HOLs




• Introduction

• Cost Factors





• Findings

OUTLINE


PROGRAM ERRORS

10%

30%

50%

RequirementsDefinition

SoftwareDesign

Coding TestingDeployment

$ 5

$ 10

RequirementsDefinition

SoftwareDesign

Coding TestingDeployment

Source of Errors - %'s

Communications of the ACM, Jan. '84

Relative Cost to Correct Errors - $1000'sSource: AT&T Bell Labs Estimates


DEVELOPMENT PHASES

Pe

rcn

et

of

To

tal E

ffo

rt (

%)

Design

Code & Debug

Test

0

5

10

15

20

25

30

35

40

45

50

Non-Ada Ada

Source: NASA/Goddard Space Flight Center, 1982-89


DEVELOPMENT PHASES

RAD Design Code Integration0%

5%

10%

15%

20%

25%

30%

35%

40%

RAD Design Code Integration

Historical Ada

Source: USAF STSC European survey

Perc

ent o

f T

otal

Eff

ort (

%)


PROGRAM INTEGRATION

Analysis, Design, Code

Integration

0%10%

20%

30%

40%

50%

60%

70%

80%

90%

Historical

Ada (First Ship)

Source: CelsiusTech, Sweden

• Errors found at integration are costly to fix– “Integration is the most

costly phase”• Requires the most people

and equipment

– “Integration is the most repeated phase”

• Required after major upgrades

• Tools that reduce integration time also reduce cost


CSC EXPERIENCE

Note: design, code, unit test, and integration & test values are percent of total effort* Baselines 1 through J2.2, 1984-1995

Program Lang. SLOC Design Code U/T I&T

AEGIS CMS-2Y 31M* 38 12 13 37

ADM Ada 75K 39 19 22 20

Australian Sub Ada 100K 42 16 21 21

JCALS Ada 400K 38 21 21 20

ENWGS Ada 511K 16 40 26 18

AN/BSY-2 Ada 700K 42 16 21 21

IPS Ada 718K 16 23 24 37

JCALS C 7K 38 21 21 20

TGC C 43K 35 29 15 22

E-CALS C 91K 40 20 10 30


• Introduction

• Cost Factors





• Findings

OUTLINE


RELIABILITY

• New York, Chicago telephone system crashs

– Crashs caused by C pointer problems

• Baltimore Memorial Hospital X-ray machine

– 6 deaths, 250 injuries– Defect caused by C pointer

problem

• Smog sensor– $1M+ fine by EPA

• Internat’l Energy Commission– “150 features of C language

are unsafe for nuclear use”Integration Formal Qual. Test

0

5

10

15

20

25

30

35

Standard (all lang.) Average Ada

Source: USAF study - “Ada vs C++:

A Business Case Analysis,”

CTA Substudy, July 1991

Error Rates

Err

ors

/ K

SLO

C


AIRBUS BRAKES

Ada C Assembly

12 250

240

400

50

100

150

200

250

Ada C Assembly

Code Walk Through Integration

Data with-held

Source: Thompson, France

Nu

mbe

r of

Err

ors


MAINTENANCE

72 LOC 87 LOC 350 LOC 550 LOC 870 LOC 10,000LOC

100,000 1,000,000

10% Change 30% Change

-10

-5

0

5

10

15

20

25

30

35

40

Pe

rce

nt

Ad

van

tag

e f

or

Ad

ain

Co

st t

o M

ake

Ch

an

ge

Maintenance cost advantage of Ada over other languages

Source: “Ada Technology: Current Status and Cost Impact,” IEEE Vol. 79

No. 1

LOC LOC


MITRE STUDY

• “Ada in the Maintenance Arena,” Schrank, Boyce & Davis, MITRE, 1995– Used Lockheed Martin’s Programmed Review of Information for

Costing and Evaluation (PRICE) Software (PRICE-S) model

– Examined military airborne avionics software

– Used Capers Jones’ “function point” data base• Compares equal functionality vice equal SLOC

– Estimated 10-year maintenance cost for • Ada• C• “Other HOL” (Pascal & FORTRAN)

– Assumed experienced programmers and good tools


ANNUAL MAINTENANCE

Function Points

1994

do

llars

X $

1000

75,000 LOC

112,500 LOC

135,000 LOC

150,000 LOC

225,000 LOC

270,000 LOC

Airborne avionics software0

200

400

600

800

1000

1200

1400

1600

350 700 1050 1400 1750 2100

Ada

other HOL

C

Source: MITRE Study, Schrank, Boyce & Davis, 1995


OUTLINE

• Introduction

• Cost Factors



• Findings


PROJECTS

• Perceived Failures

• Real-time course at the State University of New York (SUNY), Plattsburgh NY

• Air Force Improved Many-On-Many (IMOM) reengineering and Idaho National Engineering Laboratory (INEL) studies

• CelsiusTech, Swedish defense contractor (corvette class)

• Rational Software Corporation experience with Verdix Ada Development Systems (VADS)


PERCEIVED FAILURES

SYSTEM

Federal Aviation Administration Advanced AutomatIon System (AAS) - program was restructured and descoped in 1994

F16 Avionics - system worked but had cost and schedule overruns

SUBACS - Failed to deliver “on schedule”

P3C Update 4 - Cancelled while behind schedule

ASSESSMENT

Lincoln Lab report: Shifting requirements. “[T]he benefits of using Ada for the development and maintenance of large systems are well-known.”

Development team unfamiliar with Ada and F16, had 18 mo. schedule, used new parallel processor w/shared memory, changed runtime and methodology, used rate monotonic scheduling.

Well into the project the sponsor cut one year off the schedule; product delivered on original schedule.

Mission need diminished due to declining Soviet threat. Technology transferred to allies.


FAA AAS

• $4B program to replace FAA air traffic control system hardware and software tower, terminal and enroute facilities; 2M SLOC Ada on distributed IBM RS/6000

– In 1994 the program was restructured: terminal part cancelled; tower part descoped; enroute part became Display Replacement System (DSR)

• “The Role of Ada in the Restructuring of the Advanced Automation System Program”– Abstract of paper by Jonathan Dehn and Michael Glasgow, Loral Air Traffic Control

• “Root problems in the AAS”– “Inadequate requirements baseline control”

– “Extremely high availability requirements that led to high complexity in the implementation”

– “Changing mission needs”

• “[Ada] had little to do with the problems that caused the restructuring of the program.”– Note: beneficial aspects of Ada use not yet addressed in the draft paper

– “Shortage of truly knowledgable Ada programmers. . .”

– “Ada is frequently maligned for being a poor [execution] performer . . . We reject this notion.”

– “Resource intensive in the development environment”

– “Ada suffers from a severe image problem . . . viewed as a dead-end language”


PROJECT “X”

• Project “X” (also perceived by some as an Ada “failure”)– Company and project name withheld by agreement with source

• Large, real-time, distributed system on a COTS UNIX paltform– Command and control system with a high reliability requirement

• Average level of programmer experience• Approximately 1M SLOC of Ada• Approximately 110K SLOC lines of C code

– Handled interface with a COTS network product

• Project development completed– Currently in system test phase


PROJECT “X” RESULTS

• Productivity rates for both Ada and C (SLOC per staff month)– 300: Includes time spent writing code, unit and string testing

– 200: Including design and CDR time

– 160: Including integration time

– 150: Including system test time

• 11 defects observed per KSLOC, regardless of language• C defects took 4 times longer to fix than Ada defects

– C fix rate may have been impacted by low-level nature of services (communication code) implemented in C


SUNY REAL-TIME COURSE

• Real-time Course at the State University of New York (SUNY), Plattsburgh NY– “A Model Railroad for Ada and Software Engineering”,

Communications of the ACM, November 1992

– POC John W. McCormick, Ph.D.

• Teams of 3-4 students design and build real-time model railroad control system– Students have background in Pascal or Modula-2

– 5,000-15,000 lines of code

– 30-60 loosely coupled threads of control

– 27 independently powered blocks of track

– 6 reversing loops, 25 turnouts (switches)

– 150 toggle switches would be needed to manually control layout


SUNY RESULTS

• Used C as the programming language for the first five years– No team ever completed its project, even when the instructor

provided up to 60% of the project code

• Five years ago the programming language was changed to Ada– Initially about 50% of the teams completed their projects without

instructor support code

– With instructor support code, about 75% of the teams now complete their project


Dr. McCORMICK’S COMMENTS

• Students produced working systems significantly faster with Ada• Students were able to focus on software engineering issues

rather than language issues– Elimination of single keystroke errors (e.g., using = instead of ==)– High level of abstraction for multi-tasking– Higher level of abstraction for bit manipulation and data types (far less

pointer manipulation)– Ada program library (team members know when a specification changes)– Exception handlers to separate normal processing from error handling– Modularity and information hiding through packages– Encapsulation through private and limited private types– Code reuse through generic units


IMOM REENGINEERING

• Air Force Improved Many-On-Many (IMOM) reengineering and Idaho National Engineering Laboratory (INEL) studies– “Electronic Combat Model Reengineering”, Ada Information

Clearinghouse, March 1995

– “Improved Many-On-Many (IMOM) Model Research Study”, United States Air Force Electronic Warfare Center, May 1991

– POC Kurt Welker

• Developed in 1984 in FORTRAN for proprietary hardware– 333,400 lines of code

• Translated to C in 1988 for users without proprietary hardware – Used automated translation tool to convert FORTRAN to C

– C required significant manual clean-up

– Maintained original software design and architecture


IMOM MAINTENANCE

• High maintenance costs– Maintaining two systems in different languages

– Original software design deteriorated significantly

• Air Force sponsored study by INEL; the study recommended:– Reengineer and redesign the system

• Translated C harder to maintain than FORTRAN original• Code produced by translator not optimized

– Use object-oriented techniques• Modularity, data abstraction, and process abstraction

– Use the Ada programming language• Ada implementation more efficient, compact (data types and structures)• Ada support environment provides needed tools (e.g., debugger)• Package abstraction improves program maintainability


IMOM Ada REDESIGN

• Redesigned and recoded in Ada 83 – 213,000 SLOC

• FORTRAN to C conversion required 54 MM– “Final C implementation required a significant amount of clean-up”

• Reengineering of IMOM required 72 MM• Reengineering of three other IMOM-based models required a

total of 20 MM– Significant amount of reuse in Ada among IMOM models

– 65% of the code modules were reused in one or more models


MAINTAINABILITY INDEX

• Maintainability Index - a measure of success– Set of polynomial metrics developed at the University of Idaho

– Uses Halstead’s effort/module and McCabe’s cyclomatic complexity/module, plus other factors

– Validated in the field by Hewlett-Packard

– Language independent

– Index is a number between 0 and 100

– Used primarily to determine if modules have a high, medium, or low degree of difficulty to maintain

• HP concluded “modules with a MI less than 65 are considered difficult to maintain”


IMOM MI HISTORY

• FORTRAN MI was low due to poor adaptability of original design to meet changing requirements

• C MI was low due to the fact that the C code was originally generated from the FORTRAN code

• Ada MI was high, which resulted in a very maintainable system

0

FunctionalityAda IMOM

FORTRAN IMOM

C IMOM

1020

30405060

7080

Mai

nta

nab

ilit

y

A IMOM 2.0A IMOM 1.0

F IMOM 4.0F IMOM 4.4

C IMOM 1.0

C IMOM 2.0

F IMOM 5.0

DIFFICULT TO MAINTAIN


CelsiusTech

• CelsiusTech, Swedish defense contractor (corvette class)– On the verge of bankrupcy in 1988; switched to Ada

– $350M profit in 1992; key to profitability is reuse

– POC Ulf Olsson

• Eleven real-time combat system applications delivered in Ada– VME-based shared memory compute nodes (non-UNIX)

– Processing distributed over 20 nodes via Ethernet & TCP/IP

– Designed to minimize network traffic; high frequency operations allocated within individual nodes

– Ada tasks communicate asynchronously

• Total code base: approximately 3M SLOC Ada– Ada required significant initial investment

– Swedish & Danish ships shared only one interface; had 65% reuse

– Ported 500K+ SLOC from 68000 to RS/6000; changed 0.6% of code


REUSE

Source: CelsiusTech Sweden

CelsiusTech approach: design for reuse from the beginning

Built an architecture for shipboard use Delivered to 10

customers Extended to ground-

based intercept system Reuse level depends on

number of common interfaces

NewCode

Reusew/mod

Reuse TotalReuse

0%10%20%30%40%50%60%70%80%90%

NewCode

Reusew/mod

Reuse TotalReuse

Ship 5 Ground-based System


• Comparison of C and Ada at Rational Software Corporation– “Comparing Development Costs of C and Ada”, Rational Software

Corporation, March 1995– POC Stephen F. Zeigler, Ph.D.

• Productivity data on Verdix Ada Development System* (VADS) product line from March 1986 through June 1994– Products include compilers, builders, runtimes, and debug tools written in

a combination of Ada and C– Includes data for development and maintenance activities– Automated data collection

• Product line originally written entirely in C, achieved “approximate” language parity in 1991– 1.51M SLOC in C (as of October 1994)– 1.27M SLOC in Ada (as of October 1994)

*Verdix merged with Rational Corp. in 1994

RATIONAL Ada SYSTEM


RATIONAL (cont.)

• Same developers worked on Ada and C– Same design methods used regardless of language

– Low personnel turnover (5%) throughout life of project• Used DOD rule to decide what to rewrite

– Anything new written in Ada– Anything changed by more than 1/3 rewritten in Ada

• Exercised more careful control of C code during development– Avoided C’s higher optimization levels

– Avoided complex data structuring of nested unions and structures

• Equivalent degrees of difficulty for VADS components– Code generator still written in C

– Runtime, optimizer, cross linker, library services, and support tools were rewritten in Ada


Verdix Ada System Other ProjectsMetric C Ada Factor C++ Ada Factor

SourceKSLOC

Fixes perKSLOC

Fixes per“feature”

Defects * per KSLOC

Cost perSLOC

RATIONAL RESULTS

* Defects: customer-reported problems

9.21 4.59 2.0

0.52 0.25 2.0

$10.52 $6.62 1.7

.676 .096 7.0(!) .978 .222 4.4

1509 1273 883 589


Dr. ZEIGLER’S COMMENTS

• Ada “reduces the effective complexity of the overall developer’s job” – “encourages better program design”

– “encourages its users to spend more time in writing to describe their code”

• Ada provides “better tool support” (i.e., language features)– Particularly in the area of parallel processing (i.e., tasking)

• Ada provides “better locality of errors”– Local in time and space


OUTLINE

• Introduction

• Cost Factors



• Findings


SOURCES

• George Washington University

• State Univ. of New York

• University of Michigan

• AT&T Bell Atlantic

• Barry Boehm (COCOMO)

• Capers Jones

• Celsius Tech, Sweden

• Computer Sciences Corp.

• CTA Inc. / Reifer Consultants

• Lockheed Martin Corp.

• Loral Air Traffic Control

• MITRE / LMC PRICE-S model

• OC Systems

• Rational Corp.

• Thompson, France

• TRW

• George Washington University

• State Univ. of New York

• University of Michigan

• AT&T Bell Atlantic

• Barry Boehm (COCOMO)

• Capers Jones

• Celsius Tech, Sweden

• Computer Sciences Corp.

• CTA Inc. / Reifer Consultants

• Lockheed Martin Corp.

• Loral Air Traffic Control

• MITRE / LMC PRICE-S model

• OC Systems

• Rational Corp.

• Thompson, France

• TRW

• ACM

• Ada Information Clearinghouse

• DISA

• Idaho National Eng. Lab

• IEEE

• Institute for Defense Analyses

• NASA/GSFG

• Naval Postgraduate School

• NSWCDD Cost Group

• NSWCDD LSE CPCRs

• Software Engineering Institute

• SPAWAR

• USAF Business Case Study

• USAF IMOM

• USAF STSC

• ACM

• Ada Information Clearinghouse

• DISA

• Idaho National Eng. Lab

• IEEE

• Institute for Defense Analyses

• NASA/GSFG

• Naval Postgraduate School

• NSWCDD Cost Group

• NSWCDD LSE CPCRs

• Software Engineering Institute

• SPAWAR

• USAF Business Case Study

• USAF IMOM

• USAF STSC


Staff Number Purify Rational Apex SGI C++/GNAT

Size Licenses Tool C++ * Ada C++ * Ada

COMPILER/TOOLS COST

* Includes Purify tool

100 67 155K 561K 1104K 314K 159K

150 100 232K 825K 1672K 456K 224K

250 167 387K 1333K 2776K 702K 315K

200 133 310K 1098K 2224K 585K 275K

50 33 $77K $290K $574K $162K $85K


FINDINGS

• Commercial viability of Ada– C/C++ will continue to occupy a large market share– Numerous and increasing non-DOD use suggests Ada longevity– Ada 95 GNAT freeware compiler likely to increase Ada use– Java impact unknown but may enhance Ada 95 use

• Standardization and portability– Ada 83, Ada 95 and C are standards; C++ is not

• Programmer training– C/C++ programmers widely available but without real-time training– Ada not difficult to learn, may be easier to learn for real-time apps.– Increasing number of universities teaching Ada, esp. as foundation

• Development cost– Some Ada initial development costs may be higher than C


FINDINGS (cont.)

• Lifecycle maintenance and upgrade cost– Ada has long term advantage due to increased maintainability– Ada follow-ons and upgrades less costly due to ease of reuse

• Process and tools– Ada 83 compilers are expensive; Ada 95 GNAT is low cost or free– Lack of C/C++ reliability features suggests additional tool support– Cost of maintaining two language environments does not double

(programmers likely to specialize in one language or the other)• Reliability and safety

– Ada error rate and cost-to-fix are lower than C and C++• Based on available lifecycle cost data . . .

Absent extenuating factors, use of C/C++ cannot be substantiated


IMPORTANT FACTORS

• Programming in the large– Translates to reduced lifecycle cost

• Reliability and safety

• Real-time and multitasking


Inheritance

COMPARISON

Feature Ada C++

Concurrency

Distribution

Abstraction level

Standardization

Library consistency

Encapsulation

Error handling

Low level operations

Feature (cont.) Ada C++

Compiler checking

Determinism

Compiler validation

Run time checks

Real-time support

Polymorphism

Readability

full support partial support little or no support


DEFINITIONS

• Abstraction level: distance from assembly language features

• Compiler checking: type checks• Compiler validation: formal testing

of compiler vs test case• Concurrency: semantic support for

multi-threaded processing• Determinism: guarantees of timing,

ordering, no priority inv.• Distribution: support for multi-

computer platforms• Encapsulation: ability to hide

implementation detail from user• Error handling: formal support for

detecting & handling errors

• Inheritance: composition from previous types and operations

• Low-level operations: bit, byte, address & register operations

• Library consistency: compiler detection of obsolete code units

• Polymorphism: same operation on multiple entities, e.g., data types

• Readability: clarity of written code• Real-time: type time, scheduling,

priorities, synchronization• Run-time checks: out-of-range,

type compatibility checks at dispatch time

• Standardization: formal process of acceptance


SUN COMPARISON

• Source: “The Java Language Environment, A white Paper,” James Gosling and Henry McGilton, October 1995, Sun Microsystems

• Key:

Feature exists

Somewhat exists

Doesn’t exist

Java SmallTalk TCL Perl Shells C C++

High Medium Low Medium Low High High

Simple

Object Orient.

Robust

Secure

Interpreted

Dynamic

Portable

Neutral

Threads

Garbage Col.

Exceptions

Performance


EXPERT COMMENTS

• Jean Ichbiah - inventor of Ada“C was designed to be written; Ada was designed to be read”

• Herbert Schildt - ANSI C++ Standards Committee“C gives the programmer what the programmer wants; few restrictions, few complaints . . . C++ maintains the original spirit of C, that the programmer not the language is in charge.”

• P.G. Plauger - ANSI C Committee“Beyond 100,000 lines of code, you should probably be coding in Ada.”

• Sextant - French aerospace contractor“Lack of experience in Ada causes poor code performance; lack of experience in C produces code errors.”

• CelsiusTech - Swedish defense contractor“We no longer need worry about getting more efficient at producing software because that’s not where our cost is.”


COMPUTING CORNERSTONES

• Capacity– Sufficient computing resources to

perform required tasks• Connectivity

– Sufficient communication bandwidth and latency among system components

• Control– Bounded, deterministic management of

time, computational sequence and reaction to external events

• Continuous Availability– Continuously available despite failures

and damage• Correctness

– Free from errors that impact mission

• Efficacy of Design– Lifecycle cost effectiveness, including

low cost of ownership and ease of change

• In commercial practice, user does final QA• Military systems must be “ready for war.”


CORRECTNESS CORNERSTONE

• Bjarne Stroustrup, inventor of C++

• “C makes it easy to shoot yourself in the foot. C++ makes that harder but when you do, it blows away your whole leg.”

• James Gosling - Inventor of Java

• Some languages, such as Ada, “fit well and are pretty straightforward” ports of the Java virutal machine. Sun will not attempt to run C and C++ code on the virtual machine, however, because it is more difficult to ensure the reliability and, therefore, the security of downloaded code.

• Reference Manual for the Ada Programming Language, ANSI/MIL-STD-1815A-1983

– “Ada was designed with three overriding concerns: program reliability and maintenance, programming as a human activity, and efficiency.”

– “. . . emphasis was placed on program readability over ease of writing.”

– “. . . error-prone notations have been avoided.”

– “. . . compilers can ensure that operations on variables are compatible with the properties intended for objects of the type.”

Presentation Request Cover Sheet

Special Instructions:4/97This is the reworked Masters brief, with script (Carl, John and Don Rosen did script.) The script is called “mmscript.doc.”

8/22/97Derived from Ada_cos3.ppt - PA cleared with notes to make amendments to remove all Policy mentions (slides 1, 3, 6, 7, 21, 22, 23, 25, 32, 38, 44, 46, 64, 68 changed; slides 4 and 10 removed.)

script is mm970822.doc

Both .ppt and .doc will be put on web.

Name: Gary Shupe

Date submitted: 8/15/97

Requested delivery date: ASAP

Presentation title: Formerly known as “Master’s brief”

Presented by:

Presentation date: for web

Presented to:

Acetates: B/W Color

Paper:# of originals B/W Color

# of copies B/W Color

Stapled

Single-sided/no script

Single-sided/script

Double-sided

Double-sided/script

Clear Through Public Affairs

74 mm970822.ppt

PROGRAMMING LANGUAGES AND LIFECYCLE COST

24 August 1997

Ada Joint Program Office

Center for Computer Systems Engineering

Joint Interoperability & Engineering Organization

Defense Information Systems Agency

Programming Languages and Lifecycle Cost 30 August 1997 Ada Joint Program Office Center for Computer Systems Engineering Joint Interoperability & Engineering.

Documents

sloc ada

lifecycle cost slide

ada reliability

compilers ada

portability ada

ada programmers

sloc slide

lifecycle cost data