Factors Affect Software Maintenance

8/3/2019 Factors Affect Software Maintenance

1/16

F A C T O R S A F F E C T IN G S O F T W A R E M A I N T E N A N C EP R O D U C T I V I T Y : A N E X P L O R A T O R Y S T U D Y 1

R a j i v D . B a n k e rS r i k a n t M . D a t a rC a r n e g i e M e l l o n U n i v e r s i t yChri s F . KemererSloan School of ManagementMassachusetts Institute of Technology

A B S T R A C T

Systems developers and researchers have long been interested in the factors that affectsoftware development productivity. Identification of factors as either aiding or hinderingproductivity enables management to take steps to encourage the positive influences and toeliminate the negative ones. This research has explored the possibility of developing anestimable model of software development productivity using a frontier estimation method. Theapproach taken is based upon output metrics for the entire project life-cycle, and includesproject quality metrics. A large number of factors potentially affect ing software maintenanceproductivity were included in this initial investigation. The empirical analysis of a pilot dataset indicated that high project quality did not necessarily reduce project productivity.Significant factors in explaining positive variations in productivity included project teamcapability and good system response (turn around) time. Factors significan tly associated withnegative variations in productivity included lack of team application experience and highproject staff loading. The use of a new structured analysis and design methodology alsoresulted in lower short term productivity. These preliminary results have suggested a numberof new research directions and have prompted the data-site to begin a full scale data collec-tion effort in order to validate a model of software maintenance productivity.

1. INTRODUCTIONProduction of software is generally constrained bywhat is commonly referred to as the "softwarebottleneck" -- the high and growing demand forsoftware that has far outstripped the currentcapacity of software developers. For instance,Zavala (1985) notes that demand for software isgrowing at 20% to 30% per year while the supply oftrained staf f is only growing at 3% to 4%. Oneresult of the failure of software supply to keep upwith demand is the long "applications backlog" atmany large data processing departments. To a largedegree, this backlog is caused by the increasingburden that systems maintenance inflicts on dataprocessing departments (Grammas and Klein 1985).According to a wide variety of sources, maintenanceprogramming, the correction and enhancement ofexisting programs, accounts for anywhere from 50%

to 80% of data processing resources (Elshoff 1976;Freedman 1 986; Kolodziej 1 986 ). Parikh (1986)estimates that more than $30 billion are spentworldwide on software maintenance. The trend isthat the maintenance problem will continue toescalate (Jones 1986) . Research into productiv ity inthe maintenance subset of software development2has been especially lacking (Lientz 1980).This research makes an initial contribution to theunderstanding of software maintenance by providinginsights into factors that affect software mainten-ance productivity. One possible cause of produc-tivity variances is the attention spent in producinga high quality product . A common view is that atradeoff exists between quality and productivity,that to achieve a high level of either requiressacrificing the other (Case 1985, Kriebel 1979). Analternative hypothesis is that both dimensions are

160


2/16

u n d e r t h e c o n t r o l o f t h e p r o j e c t l e a d e r , a n d t h a ts u p e r i o r p r o j e c t l e a d e r s w i l l m a k e p r o d u c t i v e u s e o ft h e i r s t a f f s i n w a y s t h a t d o n o t s a c r i f i c e q u a l i t y( L a m b e r t 1 9 84 ; M o h a n t y 1 98 1 ). A n e x a m p l e o f t h isi s t h e u s e o f s o f t w a r e t o o l s , s u c h a s d a t ad i c t i o n a r i e s o r c o d e g e n e r a t o r s , t h a t b o t h r e l i e v ep r o j e c t t e a m m e m b e r s o f s o m e o f th e m o r e m u n d a n et a s k s w h i l e i m p r o v i n g q u a l i t y b y e n s u r i n g c o n s i s -t e n c y . A s e c o n d r e s e a r c h q u e s t i o n t h a t w e a d d r e s si s t h e r e l a t i o n s h i p b e t w e e n q u a l i t y a n d p r o d u c t i v i t yo n a s o f t w a r e m a i n t e n a n c e p r o j e c t .T h i s p a p e r r e p o r t s o n o u r e f f o r t s t o d e v e l o p a n de s t i m a t e a p r e l i m i n a r y m o d e l o f t h e s o f t w a r ep r o d u c t i o n p r o c e s s u s i n g p i l o t d a t a f r o m 6 5 s o f t w a r em a i n t e n a n c e p r o j e c t s r e c e n t l y c o m p l e t e d b y a l a r ger e g io n a l b a n k ' s d a t a p r o c e ss i n g d e p a r t m e n t . T h e r ea r e f o u r e v e n t u a l g o a l s o f o u r o n g o i n g r e s e a r c h .T h e f i r s t o f t h e s e g o a l s i s t o m e a s u r e f a c t o r s t h a ta f f e c t s o f t w a r e maintenance p r o d u c t i v i t y , a n i s s u et h a t h a s n o t b e e n a d d r e s s e d i n t h e M I S l i t e r a t u r e .T h e s e c o n d g o a l i s t o i n t e g r a t e t h e q u a l i t y a n dp r o d u c t i v i t y d i m e n s i o n s o f s o f t w a r e m e a s u r e m e n t .T h i r d , i n c o n t r a s t to m u c h o t h e r r e s e a r c h i n t h isa r e a , t h e i n t e n t i s t o e x a m i n e t h e p r o d u c t i v i t y o fe n t i r e p r o j e c t s r a t h e r t h a n o n l y t h e p r o g r a m m i n gp h a s e , w h i c h t y p i c a l l y a c c o u n t s f o r l e s s t h a n h a l ft h e e f f o r t o n a s o f t w a r e p r o j e c t ( B o e h m 1 98 1) .F o u r t h , w e w i l l i n c l u d e v a r i a b l e s r e l a t i n g t o t h eq u a l i ty o f l a b o r e m p l o y e d o n t h e p r o j e c ts . M a n ys m a l l s t u d i e s o r c o n t r o l l e d e x p e r i m e n t s h a v e n o t e dt h e i m p o r t a n c e o f t h e s e v a r i a b l e s , b u t r e l a t i v e l y f e wl a r g e e m p i r i c a l s t u d i e s h a v e b e e n a b l e t o e m p i r i c a l l ya s s e s s t h e i m p o r t a n c e o f t h e s e f a c t o r s ( C h r y s l e r1978; Cur t i s 1981; Sackman 1968) .I n o r d e r t o i n v e s t i g a t e t h e s e t o f p o t e n t i a l p r o d u c -t i v i t y f a c t o r s , w e e m p l o y t h e t e c h n i q u e o f D a t aE n v e l o p m e n t A n a l y s i s ( D E A ) t o e s t i m a t e t h er e l a t i o n s h i p b e t w e e n t h e i n p u t s a n d p r o d u c t s o fs o f t w a r e m a i n t e n a n c e . T h e c h o i c e o f D E A ism o t i v a t e d b y t h e n e e d t o s im u l t a n e o u s l y c o n si d e rm u l t i p l e i n p u t s a n d p r o d u c t s a n d t o n o t i m p o s e a na r b i t r a r y p a r a m e t r i c f o r m f o r t h e u n d e r l y i n gp r o d u c ti o n c o r r e sp o n d e n c e . F u r t h e r m o r e , D E Ae s t i m a t e s t h e m i n i m u m a m o u n t o f i n p u t r e q u i r e d ,g i v e n t h e s iz e a n d c o m p l e x i t y o f t h e p r o j e c t , r a t h e rt h a n t h e a v e r a g e a m o u n t o f i n p u t w h i c h w o u l d b ee s t i m a t e d u s in g r e g r e s s i o n - b a s e d m e t h o d s . T h ef o r m e r is m o r e m e a n i n g f u l f o r m a n a g e m e n t c o n tr o la n d e f f i c i e n c y e v a l u a ti o n p u r p o s e s a n d is c o n s is t e n tw i t h a m i c r o e c o n o m i c d e f i n i t i o n o f a p r o d u c t i o nf u n c t i o n .

T h e g e n e r a l a p p r o a c h o f t h i s r e s e a r c h i s t o m o d e ls o f t w a r e d e v e l o p m e n t a s a m i c r o e c o n o m i c p r o d u c t i o np r o c e s s u t i l i z i n g i n p u t s a n d p r o d u c i n g p r o d u c t s .T h i s a p p r o a c h i s s u g g e s t e d b y t h e w o r k o f K r i e b e lan d Ra v i v (1 9 8 0 ; 1 9 8 2 ) an d S t ab e l l ( 1 9 8 2 ). T h i sg e n e r a l m o d e l i s b e s t r e p r e s e n t e d b y t h e s i m p l ed i a g r a m s h o w n i n F i g u r e 1.

. . . . . > P RO D U CI

E N V I R O N M E N T A LC O M P L E X I I Y

P R O D U C T I O N P R O C E S S( S /W D E V E L O P M E N T )NPUT . . . . >

Figure 1 . G e n e r a l P r o d u c t i o n P r oc e s s M o d e l

T h e a m o u n t o f i n p u t ( e . g . , l a b o r h o u r s ) r e q u i r e d b ya s o f t w a r e d e v e l o p m e n t p r o j e c t d e p e n d s o n t h e s i z ea n d c o m p l e x i t y o f t h e r e s u l t in g p r o d u c t a n d t h ee f f e c t s o f a n u m b e r o f e n v i r o n m e n t a l c o m p l e x i tyf a c t o r s , s u c h a s t h e r e s p o n s e t i m e o f t h e d e v e l o p -m e n t h a r d w a r e . S i n c e t h e p r o d u c t is s p e c i f i e d i na d v a n c e t o t h e s o f t w a r e p r o j e c t l e a d e r , h e o r sh em u s t a c t t o m i n i m i z e t h e a m o u n t o f i n p u t r e q u i r e di n o r d e r t o i m p r o v e p r o d u c t i v i ty , s T h e r e f o r e , t h ed e p e n d e n t v a r i a b l e , l a b o r h o u r s , i s a f u n c t i o n o f t h es iz e a n d c o m p l e x i t y o f t h e p r o d u c t ( d e s c r ib e d i nS e c t io n 2 ) a n d t h e e n v i r o n m e n t a l c o m p l e x i t y fa c t o rs( d e s c r i b e d i n S e c t i o n 3 ) .T h e r e m a i n d e r o f t h i s p a p e r h a s t h e f o ll o w i n gs t r u c tu r e . T h e s o f t w a r e m a i n t e n a n c e i n p u ts a n dp r o d u c t s a r e d e s c r i b e d in S e c ti o n 2 . S e c t i o n 3c o m p a r e s t h e e n v i r o n m e n t a l v a r i a b le s s e l e c te d f o rt h i s r e s e a r c h w i t h v a r i a b l e s c h o s e n b y r e s e a r c h e r si n n e w s o ft w a r e d e v e l o p m e n t p r o d u c t i v i t y . S e c ti o n4 d e s c r i b e s t h e m o d e l a n d it s e s t i m a t i o n . T h es o u r c e o f t h e d a t a a n d t h e d a t a c o l l e c t i o n m e t h o d sa r e o u t l i n e d i n S e c t i o n 5. S e c t i o n 6 p r e se n t s th er e s u l t s o f t h e a n a l y s i s a n d S e c t i o n 7 p r o v i d e s s o m ec o n c l u s i o n s a n d s u g g e s t i o n s f o r f u t u r e r e s e a r c h .

161


3/16

2. SOFTWARE DEVELOPMENT INPUTS ANDPRODUCTSThe critical input to software development that wefocus on is the amount of professional work-hoursexpended by the project team. Personnel costsconstitute at least 45% to 50% of a data processingdepartment's budget (Grammas 1985), and 80% of thedepartment's costs at the current data-site. Sinceprofessional data processing staff time is the mostexpensive and scarce input resource in softwaredevelopment, work-hours has been the variable ofinterest in most previous studies. Furthermore, thecost of the other major input, hardware, (i.e., CPUtime, disk storage, etc.) continues to decline,increasing the ratio of personnel cost to machinecost.

The identification of consistent, quantifiableproducts from the software development process isprobably the single biggest challenge in the field ofsoftware metrics. As the final produc t of anysystems development project is a coded program orprograms, the traditional measure has been thecount of the number of written source lines of code(SLOC). 4 SLOC has the advantage of being easilycountable by automated means, in addition to ap-parently representing the amount of work requiredto build a system. The SLOC metric, however, isnot without its weaknesses. Two common problemsare comparing programs written in differentlanguages and comparing the results of studies thathave used different counting rules for countingSLOC (Jones 19 86). These problems were not anissue in the current research, as all projects werewritten in COBOL and consistent counting ruleswere employed.

We address this problem using Albrecht's FunctionPoint metric (Albrecht and Gaffn ey 1983) . TheFunction Point metric first counts the number ofunique input types, output types, logical files,external interface files, and external queries handledby an application. These counts are then weighteddepending upon difficulty and further modified byfourteen "complexity factors" defined by Albrecht.sFunction Points thus capture the magnitude andcomplexity of the analysis and design task ofvarious projects.The use of Function Points as a measure of theproduct of software development has been validatedor suggested by Behrens (1983), Vacca (1985), Jones(1986), Kemerer (1987), Albrecht (1985), Gaf fney(1986), and Lambert (1984). In a recen t Delphi-typesurvey by the Quality Assurance Institute (Perry1986), Function Points per man-month was selectedas the leading productivity measurement by anumber of Fortune 500 level firms. In summary,the inputs of the general model were implementedwith work-hours and the products with FunctionPoints and Source Lines of Code. (See Figure 2.)

C N V I l t O N M E N I A LCX)MP I E X l l Y4 ,

I - - I I U N C I I O N P O I N I SI P p ~ O D t J C T I O N P I ~ I S S ( ( A n a ly s t s & [ ) t t s t 0 n I ' h i lSe )I ( ,c g W D { V [ I O P M E N T ) I ( C o d l n0 & 10Sl in ( J Phaso )t - - - J

SLOC, however, is actually the product of only onephase of the projec t, the programming phase. Fornew development projects SLOC is generallyconsidered to be an accurate surrogate for allproject activities since larger systems typicallyrequire both more analysis and more programmingthan smaller systems. In the case of maintenanceprojects, this assumption will not, in general, hold.It is easy to imagine a project in a maintenanceenvironment with large amounts of effort expendedin analysis and design that result in relatively fewadditions or changes to lines of code. There fore ,while SLOC is an adequate measure of the size ofthe coding and testing phase, it is inadequate withrespect to the size and complexity of analysis anddesign on a maintenance project.

Figure 2. Specific Production Process Model

3. ENVIRONMENTAL VARIABLES SELECTIONPrevious research on new development has identifieda large number of factors which may have animpact on productivity. In addition, detailed discus-sions with managers at the data-site led to theidentification of other factors believed to beimportant sources of productivity variation. Thesefactors are summarized into the following fourcategories: personnel, project management, user,and technical environment . A brief discussion ofeach follows. An additional facto r that mayinfluence productivity is the overall quality of the

162


4/16

T a b l e 1 P e r s o n a l F a c t o r V a r i a b l e s

a b c d e f g h i j SDP Ex p e r i e n X X X X X X 6Ap p l Ex p e r n X X X X X X 6S / W E x p e r n c X X X X 4H / W E x p e r n c X X X 3F a c i l i t y Ex p r X 1C a p a b i l i t y X 1E d u c a t i o n X 1I n h o u s e % X 1Pa r t t i m e % X 1Pr o g Pa r t i c X 1A g e X 1M o r a l e X 1

BDKXX

XXXX

a f f i Gay le (1971) ; b f f i Sco t t (1974) ; c = Wo lver ton (1974) ; d = Wa ls ton (1977) ; e =Ch r y s l e r ( 1 9 7 8 ) ; f f f i Pu t n a m ( 1 9 7 8 ) ; g = A l b r e c h t a n d G a f f n e y ( 1 9 8 3 ) ; h = Bo e h m(1981) ; i f f i Rub in ; j f f i Jones (1986) ; S f f i Summary ; BDK = Banker , Da ta r and Kemerer .

p r o d u c t p r o d u c e d . T h i s v a r i a b l e h a s g e n e r a l l y n o t i n C O B O L f o r I B M m a i n f r a m e s , t h e f a c i l it yb e e n i n c l u d e d in p r e v i o u s e m p i r i c a l s t u d i e s o f e x p e r i e n c e d a t a o b v i a t e d t h e n e e d t o c o ll e c tp r o d u c t i v i t y a n d i s d i s c u s s e d se p a r a t e l y i n S e c t i on s o f t w a r e a n d h a r d w a r e e x p e r i e n c e .3.5 .

3 . 1 P e r s o n n e l V a r i a b l e s

P e r s o n n e l v a r i a b l e s a r e w i d e l y b e l i e v e d t o b ec r i t i c a l i n a f f e c t i n g t h e p r o d u c t i v i t y p e r f o r m a n c e o fa p r o j e c t t e a m . T a b l e 1 i s t h e f i rs t o f f o u r t a b le ss h o w i n g t h e s e l e c t i o n o f p r o d u c t i v i t y v a r i a b l e s b yo t h e r re s e a r c h e rs . E a c h r o w r e p re s e n t s a v a r i a b le ,a n d e a c h c o l u m n a r e s e a r c h e r . A n "X " i n d i c a t e st h a t t h e v a r i a b l e w a s u s e d b y t h e r e s e a r c h e r . As u m m a r y c o l u m n s h o w s t h e n u m b e r o f p re v i o u sr e s e a r c h e r s u s i n g a p a r t i c u l a r v a r i a b l e , a n d t h e l a s tc o l u m n ( " B D K " ) d e s i g n a t e s w h e t h e r i t w a s u s e d int h e c u r r e n t r e s e a r c h . F r o m T a b l e 1 , i t i s a p p a r e n tt h a t t h e e x p e r i e n c e o f p r o j e c t t e a m m e m b e r s ,m e a s u r e d a l o n g o n e o r m o r e d i m e n s i o n s , i s b e l i e v e dt o b e a c r i t i c a l e l e m e n t . F o r th i s s t u d y , e a c hp r o j e c t t e a m m e m b e r ' s t o t a l d a t a p r o c e s si n ge x p e r i e n c e , h i s d a t a p r o c e s s i n g e x p e r i e n c e a t t h i sf a c i l i t y , a n d h i s e x p e r i e n c e w i t h e a c h a p p l i c a t i o nw e r e r e c o r d e d . A s a ll o f th e p r o j e c t s w e r e w r i tt e n

C a p a b i l i t y , o r s o m e m e a s u r e o f t a l e n t, i s o f t e nd i s c u s s e d b u t r a r e l y u s e d i n r e s e a r c h o f t h is t y p ed u e t o d i f f i c u l t ie s in m e a s u r e m e n t . I n t h isr e s e a rc h , s t a f f c a p a b i l i t y w a s c a p t u r e d t h r o u g h t h eu s e o f t h e p e r s o n n e l r e v i e w s y s t e m a t t h e d a t a - s i t e .E a c h s t a f f m e m b e r i s g i v e n a y e a r l y r e v i e w t h a t iss u m m a r i z e d i n a n u m e r i c a l s c o r e r a n g in g f r o m 1( b e s t ) t o 5 ( w o r s t ) . T h e s e d a t a a r e u s e d a s am e a s u r e o f c a p a b i l i t y o r s k i l l .W e a l s o c o l l e c t e d i n f o r m a t i o n o n t h e h i g h e s t l e v e lo f e d u c a t i o n a n d t h e a m o u n t o f i n - h o u s e v e r s u so u t s i d e c o n t r a c t o r s t a f f in g . H o w e v e r , t h e h i r in ga n d p e r s o n n e l p o l i c i e s a t t h e d a t a - s i t e g e n e r a t e d av e r y h o m o g e n e o u s d a t a s e t , a n d t h e r e f o r e e d u c a t i o na n d i n - h o u s e p e r c e n t a g e w e r e d r o p p e d a s p o t e n ti a lv a r ia b l e s. T h e r e m a i n i n g v a r i a b l e s w e r e e a c h u s e db y o n l y o n e o f t h e t e n p r e v i o u s r e s e a r c h e r s , a n dw e r e n o t f e l t to b e e i th e r i m p o r t a n t o r m e a s u r a b l ea t t h e d a t a - s i t e . T h e r e f o r e , t h e v a r ia b l e s i n c l u d e di n t h e m o d e l w e r e c a p a b i l i t y , a p p l i c a t i o n e x p e r i e n c e ,a n d d a t a p r o c e s s i n g e x p e r i e n c e .

16 3


5/16

T a b l e 2 . P r o j e c t M a n a g e m e n t F a c t or V a r i a b l es

a b c d e f g h i j SS c h d C o n s t r X X 2S t a f f L o a d X X 2T r a v e l X X 2C o m m u n i c t n X 2

B D KXX

a ffi G a y l e ( 1 9 7 1 ) ; b = S c o t t ( 1 9 7 4 ) ; c ffi W o l v e r t o n ( 1 9 7 4 ) ; d ffi W a l s t o n ( 1 9 7 7 ) ; e =C h r y s l e r ( 1 9 7 8 ) ; f ffi P u t n a m ( 1 9 7 8 ) ; g = A l b r e c h t a n d G a f f n e y ( 1 9 8 3 ) ; h -- B o e h m( 1 9 8 1 ) ; i = R u b i n ; j ffi J o n e s ( 1 9 8 6 ) ; S = S u m m a r y ; B D K -- B a n k e r , D a t a r a n d K e m e r e r .

3 . 2 P r o j e c t M a n a g e m e n t V a r i a b l e sProject management variables, including scheduleconstraints, staff loading, travel requirements, andproject communication are less well represented inthe literature. (See Table 2.)Schedule variables are among the most critical thatmay be under a project manager's direct control.This research also recorded the calendar duration ofthe project in order that the loading, or work-months per calendar month, could be calculated.None of the projects in the dataset required anytravel, and their small size in terms of the number

of staff (average project size for the 65 projectswas 2.6 people) meant that intra-project communica-tion was net a critical issue. Therefore, thevariables considered were deadline pressure andmanpower loading.

3.3 User VariablesAlthough Lientz and Swanson (1981) have discussedthe potential importance of user variables, Table 3shows that user variables have played only a limitedrole in previous empirical studies.

Table 3. User Factor Variables

a b c d e f g h i j S BD KH i g h R e l i a b X X X 3 XR e q m t V o l a t X X X 3 XU s e r P a r t i c X X 2 X# U s e r O r g n X 1 XU s r D P K n w l X 1 XU s r A p p l K n X 1 X

a -- G a y l e ( 1 9 7 1 ) ; b ffi S c o t t ( 1 9 7 4 ) ; c = W o l v e r t o n ( 1 9 7 4 ) ; d = W a l s t o n ( 1 9 7 7 ) ; e =C h r y s l e r ( 1 9 7 8 ) ; f ffi P u t n a m ( 1 9 7 8 ) ; g = A l b r e c h t a n d G a f f n e y ( 1 9 8 3 ) ; h ffi B o e h m( 1 9 8 1 ) ; i = R u b i n ; j = J o n e s ( 1 9 8 6 ) ; S -- S u m m a r y ; B D K - - B a n k e r , D a t a r a n d K e m e r e r .

164


6/16


7/16

l in e s , i s i n s u f f i c i e n t . T w o i m p o r t a n t d i m e n s i o n s o fp r o g r a m q u a l i t y a r e a d h e r e n c e t o s p e c i f i c a t i o n s a n df r e e d o m f r o m d e f e c t s .A s i g n i f i c a n t b o d y o f li t e r a t u r e e x i s t s o n t h ec o n s t r u c t o f u s e r s a t is f a c ti o n . U n f o r t u n a t e l y , t h ef o c u s o f t h a t r e s e a r c h i s g e n e r a l u s e r s a t i s f a c t i o nw i t h a d a t a p r o c e s s i n g d e p a r t m e n t , w h e r e a s o u rf o c u s i s o n u s e r s a t i s f a c t i o n a c r o s s p r o j e c t s w i t h i nt h e s a m e d e p a r tm e n t . H o w e v e r , a s u r v e y i n st r u m e n tf o r m e a s u r i n g u s e r p r o j e c t s a t i s fa c t i o n h a s b e e nd e v e l o p e d b y P o w e r s ( 1 9 7 1 ) a n d w a s l a t e r v a l i d a t e db y M c K e e n ( 1 9 8 3 ). T h i s i n st r u m e n t w a s u s e d in t hec u r r e n t r e s e a r c h .T h e c o n v e r s e p r o b l e m e x i s t s i n t h e s o f t w a r e q u a l i t yr e s e a r c h l i t e r a t u r e ; t h a t i s , m a n y o f t h e m e t r i c sd e v e l o p e d i n t h i s a r e a h a v e g e n e r a l l y b e e n t o os p e c i f i c , a t t h e l e v e l o f a li n e o f c o d e o r g r o u p s o fl in e s o f c o d e w i t h i n a p r o g r a m , e T h e d a t a r e q u i r e df o r t h i s m i c r o l e v e l o f d e t a i l w e r e n o t a v a i l a b l e a tt h e d a t a - s i te . H o w e v e r , a r e c e n t s u r v e y ( P e r r y1 9 8 6 ) b y t h e Q u a l i t y A s s u r a n c e I n s t i t u t e s u g g e s t st h r e e q u a l i t y m e t r i c s a s t h e m o s t w i d e l y a c c e p t e d i ni n d u s t r y . T h e s e a r e u s e r p e r c e i v e d f u n c t i o n a lq u a l i t y , u s e r s o f t w a r e s a t i s f a c t i o n , a n d p r o d u c t i o nj o b s p r o c e s s e d w i t h o u t i n c i d e n c e . T h e f i rs t t w o o ft h e se a r e c o v e r e d b y t h e P o w e r s i n s t ru m e n t . T h et h i r d i d e a w a s d e v e l o p e d i n t o a s i t e - s p e c i f i c q u a l i t ym e t r i c t h a t r a t e d p r o j e c t s a s a v e r a g e , a b o v ea v e r a g e , o r b e l o w a v e r a g e w i t h r e s p e c t t o t h ep r o b l e m s e n c o u n t e r e d a f t e r t h e p r o j e c t ' s s o f t w a r ew a s t u r n e d o p e r a t i o n a l . T h i s m e t r i c is d e s c r i b e d inS ec t i o n 5 .2 .3 .4 . E X P L O R A T O R Y M O D E L A N D E S T IM A T I O NT h e p r i m a r y p u r p o s e o f t h i s i n i t i a l a n a l y s i s i s t oi n v e s t i g a te t h e p o t e n t i a l i m p a c t o f a n u m b e r o fp o t e n t i a l p r o d u c t i v i t y f a c t o r s . W e m o d e l t h e a c t u a li n p u t r e s o u r c e s ( l a b o r h o u r s ) u s e d a s a m u l t i p l i c a -t i v e f u n c t i o n o f t h e p r i m a r y p r o d u c t i o n c o r r e s p o n -d e n c e a n d t h e e n v i r o n m e n t a l f a c t o r s . T h i s g e n e r a lm o d e l i s s i m i l a r t o o t h e r s d e v e l o p e d i n t h e l i t e r a -t u r e ( A l b r e c h t a n d G a f f n e y 1 9 8 3; B o e h m 1 9 8 1) . I ts h o u l d b e n o t e d t h a t s i n c e t h e p r o d u c t r e q u i r e m e n t sa r e p r e s p e c i f ie d , w e m o d e l t h e p r i m a r y p r o d u c t i o nc o r r e s p o n d e n c e a s t h e m i n i m u m a m o u n t o f i n p u tr e s o u r c e s r e q u i r e d t o p r o d u c e t h e p r e s p e c i f i e dp r o d u c t , w h i c h i s d e s c r i b e d i n t e r m s o f F u n c t i o nP o i n t s a n d S L O C . S i n c e o u r o b j e c t i v e i s t oe s t i m a t e t h e m i n i m u m ( r a t h e r t h a n t h e a v e r a g e )c o n s u m p t i o n o f i n p u t r e s o u r c e s , w e a d o p t ane x t r e m a l o r f ro n t i e r e s t i m a t i o n t e c h n i q u e , D a t a

E n v e l o p m e n t A n a l y si s ( D E A ) . D E A u s e s a l in e a rp r o g r a m m i n g a p p r o a c h t o i d e n t i f y t h e m o s t e f f i c i e n tp r o j e c t s ( B a n k e r , C h a r n e s a n d C o o p e r 1 9 8 4 ; C h a r n e s ,C o o p e r a n d R h o d e s 1 98 1) .D E A i s a n a p p r o p r i a t e t o o l f o r t h i s p u r p o s e f o rs e v e r a l r e a s o n s . F i r s t , s i n c e n o p r e s e t s t a n d a r d se x i s t , p r o d u c t i v i t y n e e d s t o b e e v a l u a t e d r e l a t i v e t oo t h e r p r o j e c t s , w h i c h i s t h e b a s i s o f t h e D E Ae f f i c i e n c y r a ti n g. S e c o n d , s o f t w a r e d e v e l o p m e n tp r o d u c e s m u l t i p l e p r o d u c t s , s o t h a t s i m p l e p a r t ia lp r o d u c t i v i t y r a t i o m e a s u r e s a r e i n s u f f i c i e n t . T h i r d ,D E A d o e s n o t i m p o s e a p a r a m e t r i c f o r m o n t h ep r o d u c t i o n f u n c t i o n a n d o n l y a s s u m e s a m o n o t o n ica n d c o n v e x r e l a t io n s h i p b e t w e e n i n p u t s a n dp r o d u c t s . G i v e n th e l i m i t e d k n o w l e d g e a b o u t t h ep r o d u c t i o n p r o c e s s u n d e r l y i n g s o f t w a r e d e v e l o p m e n t ,s p e c i f y i n g a p a r a m e t r i c f o r m s u c h a s C o b b - D o u g l a s( S t a b e l l 1 9 8 2 ) f o r t h e p r o d u c t i o n c o r r e s p o n d e n c e i sd i f f i c u l t t o s u b s t a n t i a t e t h e o r e t i c a l l y o r v a l i d a tes t a t is t i c a ll y a n d i t is n o t i m m e d i a t e l y a p p a r e n t w h a tr e s t r i c t i o n s t h e s e h y p o t h e s e s , t r e a t e d a s a x i o m s i nt h e e c o n o m e t r i c a p p r o a c h , i m p o s e o n t h e p r o d u c t i o nc o r r e s p o n d e n c e .W e n e x t e x p l o r e t h e a v e r a g e i m p a c t o f d i f f e r e n te n v i r o n m e n t a l f a c t o r s o n t h e D E A e f f i c i e n c y r a ti n g.S i n c e o u r o b j e c t i v e i s t o i d e n t i f y t h e a v e r a g ei m p a c t o f e n v i r o n m e n t a l f a c t o r s o n p r o d u c t i v i t y , w eu s e m u l t i v a r i a t e r e g r e s s i o n a n a l y s is . T h e g e n e r a li d e a i s t h a t t w o p r o j e c t s c o u l d b e i d e n t i c a l i n t e r m so f t h e ir o u t p u t s , y e t o n e m a y h a v e e n v i r o n m e n t a lf a c t o r s ( s u c h a s p o o r h a r d w a r e r e s p o n s e t i m e ) t h a tc a u s e s t h a t p r o j e c t t o c o n s u m e m o r e l a b o r h o u r s .T h e l a t t e r p r o j e c t w i l l b e r a t e d a s i n e f f i c i e n tr e l a t i v e t o t h e f o r m e r , s i n c e b o t h p r o d u c e d t h es a m e o u t p u t s b u t t h e s e c o n d r e q u i r e d m o r e i n p u ts .T h e p u r p o s e o f t h e a n a l y s i s w i l l b e t o i s o l a t e a n dm e a s u r e t h e f a c t o r s t h at m a y h a v e i n f l u e n c e d t h ep r o d u c t i v i t y r a t i n g s .5 . D A T A C O L L E C T I O N5 . 1 D a t a S o u r c e

D a t a f o r t h i s r e s e a r c h w e r e c o l l e c t e d a t a l a r g er e g io n a l b a n k ' s d a t a p r o c e s s in g d e p a r t m e n t . T h et y p e s o f a p p l i c a t i o n s r e p r e s e n t e d a r e t y p i c a lf i n a n c i a l t r a n s a c t i o n p r o c e s s i n g s y s t e m s , a n d a rew r i t t e n in C O B O L to r u n o n I B M h a r d w a r e . C O B O La n d I B M a r e t h e m o s t w i d e l y u s e d s o f t w a r e a n dh a r d w a r e i n c o m m e r c i a l d a t a p r o c e s s i n g a n dt h e r e f o r e t h is s i t e is l i k e l y t o b e r e p r e s e n t a t i v e o fm u c h o f c u r r e n t b u s i n e s s d a t a p r o c e s s i n g . 7 T h e

166


8/16

d a t a p r o c e s s in g d e p a r t m e n t i s d i v i d e d i n t o e ig h t e e n" s e c t i o n s , " w h i c h a r e o r g a n i z e d a r o u n d c o m m o n s e t so f a p p l i c a t i o n s . T h r e e o f t h e s e c t i o n s w e r e s e l e c t e db y t h e B a n k a s r e p r e s e n t a t i v e o f t h e d e p a r t m e n t a sa w h o l e . T w o c r it e r i a w e r e u s e d to s e l e c t p r o j e c t sc o m p l e t e d b y t h e s e t h r e e s e c t io n s : s i ze a n dr e c e n c y . S e l e c t i n g l a r g e r p r o j e c t s a l l o w s t h ee x a m i n a t i o n o f t h e p r o j e c t s th a t c o n s u m e t h e b u l ko f t h e B a n k ' s r e s o u r c e s . P r o j e c t s i ze i s a l s oi m p o r t a n t i n t h a t t h e f a c t o r s a f f e c t i n g p r o d u c t i v i t yo n s h o r t , o n e p e r s o n p r o j e c t s a r e l i k e l y t o b eo v e r w h e l m e d b y i n d i v i d u a l s k il l d i f f e r e n c e s a c r o s sp r o j e c t s t a f f m e m b e r s ( D e M a r c o 1 98 2; S a c k m a n ,E r i k s o n a n d G r a n t 1 9 6 8 ). W e o n l y c o n s i d e r e d" s i g n if i c a nt " p r o j e c t s a t t h e B a n k t h a t c o s t am i n i m u m o f $ 5 , 0 0 0 i n in t e r n a l d o l l a rs .P r o j e c t r e c e n c y is i m p o r t a n t f o r t w o r e a s o n s . S i n c ed a t a w e r e c o l l e c t e d r e t r o s p e c t i v e l y , o l d p r o j e c t sw e r e n o t i n c l u d e d b e c a u s e p e r s o n n e l t u r n o v e r a n dl a c k o f d o c u m e n t a t i o n r e t e n t i o n m a d e d a t a c o l l e c -t i o n i m p o s s i b l e . S e c o n d , u s i n g o n l y r e c e n t p r o j e c t sl e g i t i m i z e s c r o s s p r o j e c t c o m p a r i s o n s i n t h a t t h et e c h n o l o g y a n d p e r s o n n e l i n v o l v e d a r e l i k e l y t o b ev e r y s i m i l ar . A f t e r d i s c u s s i o n s w i t h B a n k s t a f f ,o n l y p r o j e c t s c o m p l e t e d w i t h i n t h e 1 8 m o n t h p e r i o db e t w e e n J a n u a r y 1 , 19 8 5 a n d J u l y 1 , 1 9 86 w e r ei n c l u d e d i n t h e s t u d y . D a t a w e r e c o l l e c t e d d u r i n gt h e s u m m e r o f 1 9 86 . D u e t o a n u m b e r o f f a ct o r s ,i n c l u d i n g r e o r g a n i z a t i o n s , t h e c o n v e r s i o n t o a n e wt i m e r e p o r t i n g s y s t e m , u s e o f c o n t r a c t o r s , p e r s o n n e lt u r n o v e r , a n d t h e e l i m i n a t i o n o f a f e w u n s u i t a b l e( i. e. , n o n - C O B O L ) p r o j e c t s , c o m p l e t e d a ta w e r ea v a i l a b l e f o r o n l y 6 5 o f t h e 8 4 p o t e n t i a l p r o j e c t s .T h e s e 6 5 p r o j e c t s h a v e t h e c h a r a c t e r i s t i c s s h o w n i nT a b l e 5 .

5 . 2 D a t a C o l l e c ti o n M e t h o d s a n d C o n t r o l s

T h i s s e c t i o n d e s c r i b e s t h e m a i n d a t a t y p e s a n d h o wt h e y w e r e c o l l ec t e d . W h e n p o s s ib l e , w e a t t e m p t e dt o u s e d a t a a l r e a d y c o l l e c t e d a n d e m p l o y e d b y t h eB a n k , r a t h e r t h a n d e v e l o p i n g n e w d a t a c o l l e c t i o ni n s t ru m e n t s t h a t w o u l d i m p o s e a d d i t io n a l b u r d e n o nB a n k s t a f f .5 . 2 .1 P r o f e s s i o n a l w o r k - h o u r s

T h e k e y i n p u t v a r i a b l e w a s t h e n u m b e r o f w o r k -h o u r s c h a r g e d b y p r o j e c t b y p e r s on . P r e v i o u sr e s e a r c h h a s g e n e r a l l y b e e n s a t i s f i e d w i t h w o r k - -h o u r s b y p r o j e c t o n l y . T h e l i m i t a t i o n o f t h a ta p p r o a c h i s i m m e d i a t e l y a p p a r e n t i f a 1 0 00 w o r k -h o u r p r o j e c t s t a f f e d b y a t e a m o f v e t e r a n p r o g r a m -m e r / a n a l y s t s w h o w e r e a l s o i n t i m a t e l y f a m i l i a r w i t ht h e a p p l ic a t i o n b e i n g p r o v i d e d i s c o m p a r e d w i t h o n es t a f f e d b y a t e a m o f n o v i c e s . I n t u i t i o n su g g e s t st h a t t h e f o r m e r t e a m i s l i k e l y t o b e m o r e p r o d u c -t i v e , y e t m u c h p r i o r r e s e a r c h h a s t r e a t e d b o t h o ft h e s e s i m p l y a s " t w o 1 0 0 0 w o r k - h o u r " p r o j e c t s .T h i s p a p e r c h a r a c t e r i z e s t h e a c t u a l w o r k - h o u r se x p e n d e d a l o n g a n u m b e r o f d i m e n s i o n s, p a r t i c u la r l ye x p e r i e n c e a n d c a p a b i l i t y .T h e c h a r a c t e ri z a t i o n o f w o r k - h o u r d a t a w asa c c o m p l i s h e d v i a a p e r s o n n e l s u r v e y t h a t r e q u e s t e de a c h p r o j e c t m e m b e r t o f i ll i n d a t a o n h i s o r h e rt o t a l d a t a p r o c e s s i n g e x p e r i e n c e , d a t a p r o c e s s i n ge x p e r i e n c e a t t h e B a n k , a p p l i c a t i o n e x p e r i e n c e , a n de d u c a t i o n . T h e s e f o r m s w e r e m a t c h e d to t h er e c o r d s in t i m e r e p o r t i n g v i a a n e m p l o y e e n u m b e r .F o r m s w e r e n o t r e c e i v e d f r o m a l l p r o j e c t m e m b e r sc h a r g i n g t i m e , c h i e f l y d u e t o t h e f a c t t h a t t h e

T a b l e 5 . P r o j e c t S u m m a r y D a t a

M E A N S T A N. DE V . MIN IMUM MA X IMUMW o r k - h o u r s 9 3 7 7 1 7 1 3 0 3 , 3 4 2F u n c P o i n ts 1 1 8 1 2 6 8 6 1 6S LO C 5 , 4 1 5 7 , 2 3 0 5 0 3 1 , 0 6 0D u r a t i o n 6 4 1 2 2

16 7


9/16

i n d i v i d u a l h a d t r a n s f e r r e d o r l e f t t h e B a n k o r h a db e e n a n o u t s i d e c o n t r a c t o r . I n t h e e v e n t t h a t t h eh o u r s o n a p r o j e c t c o u l d n o t b e c a t e g o r i z e d , t h a tp r o j e c t w a s d r o p p e d f r o m t h e st u d y .

t e a m f o r a s e c o n d r e v i e w f o r c o m p l e t e n e s s a n dr e a s o n a b l e n e s s .5 . 2 . 3 Qu a l i t y d a t a c o l l e c t i o n

5 . 2 . 2 P r o d u c t s i z e a n d e n v i r o n m e n t a l c o m p l e x i t yf a c t o r sP r o d u c t si ze a n d e n v i r o n m e n t a l c o m p l e x i t y d a t aw e r e c o l l e c t e d v i a a s u r v e y o f p r o j e c t le a d e r s. T h es i z e d a t a c o l l e c t i o n f o r m c a p t u r e d d a t a o n

o F u n c t i o n P o i n t so N e w a n d m o d i f i e d s o u r c e li ne s o f c o d ew h i l e th e e n v i r o n m e n t a l c o m p l e x i t y f o r m c a p t u r e dd a t a o n

o F u n c t i o n P o i n t c o m p l e x i t yo P r o j e c t m a n a g e m e n to U s e r f a c t o r so T e c h n i c a l e n v i r o n m e n t

A n i m p o r t a n t i s s u e i n m e a s u r i n g p r o d u c t i v i t y isw h e t h e r t h e p r o d u c t s o f e f f i c i e n t p r o j e c t s a r e o fth e s a m e q u a l i t y a s t h o s e o f l e s s e f f i c i e n t p r o j e c t s .T h i s s t u d y a d d r e s s e d t w o q u e s t i o n s a s a d j u n c t s t ot h e e f f i c i e n c y m e a s u r e g e n e r a t e d . T h e s e m e t r ic ss h o u l d n o t b e c o n f u s e d w i t h g e n e r a l m e a s u r e s o fsys tems e f f e c t i v e n e s s .T h e f i r s t q u a l i ty c o n c e p t i s t h a t o f o p e r a t i o n a lq u a l i t y , w h e t h e r t h e s y s t e m o p e r a t e s s m o o t h l y o n c ei t i s i m p l e m e n t e d . T h i s m e a s u r e w a s g e n e r a t e d b y as t a f f s e c ti o n w i t h i n t h e B a n k f r o m t h r e e e x i s t in gs o u r c e s :

o d a il y a b n o r m a l e n d ( A B E N D ) r e p o r to w e e k l y s e c t i o n s t a tu s r e p o r t so a d h o c u s e r p r o b l e m r e p o r t s

D u e t o th e b r o a d n a t u r e o f th e p h e n o m e n o nm o d e l e d , a l a rg e n u m b e r o f f a c t o r s w e r e i d e n t i f i e da s p o s s i b l e v a r i a b l e s . I n o r d e r to m a k e t h e d a t ac o l l e c t i o n e f f o r t f e a s i b l e a t t h e f i e l d s i t e , m o s t o ft h e f a c t o r s w e r e m e a s u r e d i n o n l y o n e w a y . T h i sr a is e s t h e q u e s t i o n o f w h e t h e r a n y f a c t o r s w e r e n o ts h o w n t o b e s i g n i f i c a n t d u e t o m e t h o d v a r i a n c e .O n e c o n t r o l t h a t w a s u s e d t o m i t i g a t e t h i s w a s tou s e q u e s t i o n s d r a w n f r o m p r e v i o u s r e s e a r c hw h e n e v e r p o s s i b l e .A n u m b e r o f s te p s w e r e t a k e n t o a s su r e t h a t th ed a t a c o l l e c t i o n f o r m s w o u l d b e f i l l e d o u t a sa c c u r a t e l y a s p o s s i b l e . A tr a i n i n g s e s s i o n t o w a l kt h r o u g h t h e d a t a c o l l e c ti o n f o r m s w a s h e l d f o r a llp r o j e c t l e a d e rs a n d t h e i r s e c t i o n h e a d s . I na d d i t i o n , a m e m b e r o f th e r e s e a rc h t e a m w a s o n -s i te d u r i n g t h e e n t i r e d a t a c o l l e c t i o n p r o c e s s a n dp r o v i d e d a d h o c s u p p o r t to p r o j e c t l ea d e r s. A na u t o m a t e d t o o l w a s al s o a v a i l a b l e t o a i d i n th ec o u n t i n g o f s o u r c e l i n e s o f c o d e .T h e f o l l o w i n g c o n t r o l s w e r e e s t a b l is h e d t o a t t e m p tt o p r o v i d e a d d i t i o n a l a s s u r a n c e s o f d a t a v a l i d it y .A f t e r t h e p r o j e c t l e a d e r h a d c o m p l e t e d t h e d a t ac o l l e c t i o n f o r m , i t w a s f i r s t r e v i e w e d b y t h e s e c t i o nh e a d . A s e a c h p r o j e c t w a s c o m p a r e d o n l y to o t h e rp r o j e c t s w i t h i n t h e s a m e s e c t i o n , t h e r e v i e w b y t h es e c t io n h e a d a l so e n s u r e d c o n s i s t e n c y a c r o ssp r o j e c t s . A f t e r r e v i e w b y t h e s e c t io n h e ad , t h ed a t a c o l le c t io n f o r m w a s f o r w a r d e d t o t h e r e s e a rc h

D a t a f r o m t h e t w o m o n t h p e r i o d f o l lo w i n g i m p l e -m e n t a t io n w e r e c o m p a r e d w i t h d a t a fr o m t h ep r e v i o u s t w e l v e m o n t h s ' t r e n d . S i g n i f i c a n t d e v i a -t io n s r e s u l t e d i n a b o v e o r b e l o w a v e r a g e o p e r a t i o n a lq u a l i t y r a ti n g s. T h e c o n t r o l f o r t h is m e a s u r e w a st o f o r w a r d t h e r a t in g s f o r e a c h s e c t i o n t o th ea p p r o p r i a t e s e c t i o n h e a d f o r r e v i e w .T h e s e c o n d q u a l i t y c o n c e p t i s t h a t o f u s e r p r o j e c ts a t is f a c ti o n . A s u r v e y , b a s e d o n t h e P o w e r si n s t r u m e n t , w a s s e n t t o t h e u s e r s w h o h a dr e q u e s t e d t h e i n d i v i d u a l p r o j e c t s . T h e s e f o r m s w e r er e t u r n e d d i r e c t l y t o t h e r e s e a r c h t e a m w i t h o u tr e v i e w b y t h e s e c t i o n s .6 . D A T A A N A L Y S I ST h e r e w e r e t w o b r o a d o b j e c t i v e s t o t he d a t aa n a l y si s : 1 ) t o d e t e r m i n e t h e a p p r o p r i a t e n e s s o f t h eg e n e ra l a p p r o a c h o f u s in g D E A f o r s o f tw a r ed e v e l o p m e n t a n a l y s i s , a n d 2 ) t o i d e n t i f y w h i c hf a c t o r s i n o u r p i l o t d a t a s a m p l e s e e m e d t o b e th em o s t i m p o r t a n t a n d t h e r e f o r e m e r i t fu r t h e ri n v e s t i g a t i o n i n f u t u r e r e s e a r c h . T h e r e s u l ts o ft h e s e a n a l y s e s a r e p r e s e n t e d b e l o w .6 . 1 D EA Ef f i c i e n c y R a t i n g sA D E A e f f i c i e n c y s c o r e fo r e a c h p r o j e c t w asd e v e l o p e d u s i n g m e t r i c s f o r t o t a l w o r k h o u r s ,F u n c t i o n P o i n ts , a n d S L O C . T h r e e s e p a r a t e p r im a r y

168


10/16

T a b l e 6 . S u m m a r y o f D E A R e s u l tsMAXIMUM MINIMUM MEAN ST. DEV. n

Section M 1.00Section D 1.00Section L 1.00

.19 .62 .28 19

.14 .62 .30 27.14 .62 .32 19

p r o d u c t i o n f u n c t i o n s w e r e e s t i m a t e d u s in g D E A ( o n ef o r e a c h se c t i o n ) . T h i s w a s d o n e t o e n s u r e t h a tp r o j e c t s w e r e b e i n g r a t e d o n l y a g a in s t s im i l a ra p p l i c a t i o n s .O n e i n t e r e s t i n g r e s u l t w a s t h a t a l l t h r e e s e c t i o n ss h o w e d w i d e v a r i a t i o n s i n p r o d u c t i v i t y , a s s h o w n i nT a b l e 6 .T h i s i s c o n s i s t e n t w i t h m u c h o f t h e l it e r a t u re o ns o f t w a r e d e v e l o p m e n t p r o d u c t i v i t y , p a r t i c u l a r l y t h a td e a l in g w i t h i n d i v id u a l d i f f e r e n c e s . I n a d d i t i o n , t h ed i s t r ib u t i o n o f e f f i c i e n c y r e s u l ts w i t h i n e a c h s e c t i o ni s c o n s i s t e n t a c r o s s s e c t i o n s , w h i c h s u p p o r t s t h ep o o l i n g o f t h e i n d i v i d u a l s e c t i o n r e s u l t s i n t h em u l t i v a r i a t e r e g r e s s i o n a n a l y s i s .

S L O C , a n d F u n c t i o n s P o i nt s . T h i s t h r e e v a r i a b lem o d e l d i d n o t e x h i b i t a p p r e c i a b l y m o r e e x p l a n a t o r yp o w e r t h a n t h e t w o v a r i a b l e m o d e l ( R 2 o f . 69 8v e r s u s . 68 9 ) . T h e n u l l h y p o t h e s i s o f e q u a l i t y o f t h ee s t i m a t e d c o e f f i c i e n t s f o r N e w S L O C a n d M o d i f i e dS L O C c o u l d n o t b e r e j e c t e d a t t h e 1 0% le v e l( P i n d y c k a n d R u b i n f e l d 1 9 8 1 ). T h e r e f o r e , t h e t h r e ev a r i a b l e m o d e l w a s n o t p u r s u e d i n t h e i n t e r e s t s o fp a r s i m o n y . I n s u m m a r y , w e c o n c l u d e t h a t t h eo r ig i n a l a s s u m p t i o n s r e g a r d i n g t h e c h o i c e o f m e t r i c ss u f f i c i e n t l y r e p r e s e n t t h e p r o d u c t o f t h e s o f t w a r em a i n t e n a n c e p r o c e s s . W e u s e t h e D E A e f f i c i e n c yr a ti n g s t o e x a m i n e t h e e f f e c t s o f t h e e n v i r o n m e n t a lv a r i a b l e s o n p r o d u c t i v i t y .6 . 2 M u l t i v a r i a t e R e g r e s s i o n R e s u l t s

W e a l s o e s t i m a t e d a l in e a r r e g r e ss i o n m o d e l T h e re c i p r o c a l o f t h e D E A e f f i c i e n c y s c o r e i s( c o n s i s t e n t w i t h t h a t o f o t h e r r e s e a r c h e r s ) w i t h r e g r e s s e d a g a i n s t t h e e n v i r o n m e n t a l v a r i a b l e sw o r k - h o u r s a s t h e d e p e n d e n t v a ri a b l e a n d F u n c t i o n d e s c r i b e d i n S e c ti o n 3 i n a m u l t i v a r i a t e r e g r e ss i o nP o i n ts a n d t o ta l S L O C ( n e w S L O C p l u s m o d i f i e d m o d e l . A s u m m a r y o f t h is m o d e l a p p e a rs i nS L O C ) a s t h e i n d e p e n d e n t v a r i ab l e s . T h i s m o d e l T a b l e 7 , a n d w e d is c u s s e a c h o f t h e si g n i fi c a n t( p r e s e n t e d b e l o w ) s h o w e d t h a t t h e s e t w o m e a s u r e s v a r i a bl e s i n t u r n .o f s iz e w e r e e x c e l l e n t p r e d i c t o r s o f t o t al e f f o r t .

A c t u a l w o r k - h o u r s ffi 3 5 5 . 0 + 3 . 4 9 ( F P ) + . 0 3 ( S L O C )( .1 1 ) ( 7 .2 5 ) ( 3 .7 6 )

R ~ ffi 69.9 % (-R2 = 68 .9)O n e c o n c e r n w i t h t h is m o d e l m i g h t b e m u l t i c o l -l i n e ar i ty , a s p r e v i o u s r e s e a r c h o n n e w d e v e l o p m e n tp r o j e c t s h a s s h o w n a c o r r e l a t i o n o f . 9 4 a n d g r e a t e rb e t w e e n F u n c t i o n P o i n ts a n d S L O C ( A l b r e c h t a n dG a f f n e y 1 9 8 3 ). O u r e a r l i e r d i s c u s s i o n s u g g e s t s t h a tF u n c t i o n P o i n t s a n d S L O C a r e n o t l i k e ly t o b e a sh i g h ly c o r r e l a t e d i n t h e c a s e o f m a i n t e n a n c ep r o j e c ts . I n d e e d , th e c o r r e l a ti o n b e t w e e n F u n c t i o nP o i n t s an d S L O C i s . 5 7 i n t h i s d a t a s e t .A n a l t e r n a t i v e m o d e l w a s a ls o e s t i m a t e d , u t i l i z in gt h r e e i n d e p e n d e n t v a ri a b le s : N e w S L O C , M o d i f i e d

T h e d e p e n d e n t v a r i a b le i s t h e r e c i p ro c a l o f t he D E Ae f f i c i e n c y s c o re . T h e r e f o r e , t h e i n t e r p r e ta t i o n o ft h e s i g n s o f t h e c o e f f i c i e n t s i s t h a t p o s i t i v e ( + )s ig n s s h o w r e d u c e d p r o d u c t i v i t y , w h i l e n e g a t iv e ( - )s ig n s s h o w i n c r e a s e d p r o d u c t i v i t y . T h e R 2 f o r t h ee n t i r e s i x t e e n v a r i a b l e m o d e l is .5 3 ( F - v a l u e o f 3 .3 2i s s i g n i f i c a n t a t t h e 1 % l e v e l ). T h e v a l u e o f t h ei n t e r c e p t w a s 1 .9 ( tf fi l. 98 ) . T h e B e l s l e y - K u h - W e l s c h( 1 9 8 0 ) t e s t d i d n o t i n d i c a t e a n y m u l t i c o l l i n e a r i t yp r o b l e m s .I t s h o u l d b e n o t e d t h a t t h e R 2 i n d i c a t e s t h e a m o u n to f v a r i a t io n i n p r o d u c t i v i t y e x p l a i n e d . O t h e rr e s e a r c h e r s h a v e e x p l a i n e d t h e v a r i a t i o n i n h o u r s ,w h e r e t h e i n d e p e n d e n t v a r i a b l e s h a v e i n c l u d e d s i z e .F o r o u r d a t a s e t , a n a n a l o g o u s r e g r e s s i o n m o d e l w i t hh o u r s a s t h e d e p e n d e n t v a r i a b l e a n d s i z e a n de n v i r o n m e n t a l c o m p l e x i t y a s t h e i n d e p e n d e n tv a r i ab l e s p r o d u ces an R 2 ffi . 8 5 ( F = 1 4 .0 2 ).

169


11/16

T a b l e 7 . S u m m a r y o f t h e R e g r e s s i o n M o d e l

V A R I A B L E B e t a t p DESCRIPTION

TOPST AFF - . 0 1LOAPPEXP +.80LODPEXP +.65LOADING +.42TIGHTD EAD - . 79IN TER N AL - .7 9LOUSERDP - . 69S T A F F A P T + . 6 1LOUSERAP +.55HIGHRELY +.34LOAGREE +.28~ R E S P - . 8 6STRCMETH +.83V O L A T L T Y + . 5 2

+ .0 9Q U A L I T Y + . 0 6

- 2 . 4 3 . 0 2+ 2 . 0 8 . 0 4+ 1 . 4 2 . 1 6+ 2 . 3 7 . 0 2- 2 . 2 5 . 0 3- 1 . 5 7 . 1 2- 1 . 3 2 . 1 9+ 1 . 1 8 . 2 4+ 1 . 1 4 . 2 6+0.91 .37+ 0 . 6 2 . 5 4- 2 . 2 8 . 0 3+2 .0 9 .0 4+1 .4 9 .1 4+0 .2 4 .8 1+ 0 . 1 8 . 8 6

% H o u r s ch a r g e d b y b e s t s ta f fH ig h % o f a p p l i ca t i o n n o v i ce sH ig h % o f d p n o v i ce sW o r k - m o n t h s / C a l e n d a r M o n t h s> a v e r a g e d e a d l i n e p r e s s u r eN o e x te r n a l u se r< a ve r a g e u se r d p kn o w le d g eP o o r c o m m u n i c a t i o n s w i t h u s e r< a ve r a g e u se r a p p l kn o w le d g eH i g h r e q u i r e d r e l i a b i l i t yL o w i n i t i a l u s e r a g r e e m e n tG o o d r e s p o n s e / t u r n a ro u n d t im eN e w s t r u c t u r e d m e t h o d u s e dM a n y s / w e n v i r o n m e n t c h a n g e sG o o d d o c u m e n t a t i o n a v a i l a b l e3 p o in t sca le h i=a b o ve a ve r a g e

6 . 2 . 1 P e r s o n n e l v a r i a b l e sT h e p e r s o n n e l v a r i a b l e s a r e o f t e n c i t e d a s b e i n gc r i t ic a l t o s o f t w a r e p r o d u c t i v i t y , a n d t h e s e r e s u lt ss u g g e s t t h a t t h e y a r e i m p o r t a n t i n s o f t w a r e m a i n t e -n a n c e . T h e m o s t s i g n i f i c a n t v a r i a b l e , h o w e v e r , i so n e t h a t i s o f t e n d i s c u s s e d b u t r a r e l y m e a s u r e d :t h e c a p a b i l i t y o f t h e p r o j e c t t e a m , h e r e l i s t e d a sT O P S T A F F . T h e B a n k g i v e s a y e a r l y r e v i e w to a lls t a f f m e m b e r s a n d t h e r e v i e w i s s u m m a r i z e d i n t h ef o r m o f a n u m e r i c a l s c o r e , f r o m o n e t o f i v e , o n eb e i n g t h e b e s t. T h e v a r i a b le T O P S T A F F i s t h ep e r c e n t a g e o f h o u r s c h a r g e d t o p r o j e c t s b yi n d i v i d u a l s r a t e d o n e o r t w o . T h e h i g h e rp e r c e n t a g e o f t h e s e h i g h c a p a b i l i t y s t a f f m e m b e r s ,t h e b e t t e r t h e p r o d u c t i v i t y .S t a f f m e m b e r s w e r e c l a s s i f i e d a s a p p l i c a t i o n n o v i c e si f t h e y h a d l e s s t h a n 2 4 m o n t h s e x p e r i e n c e o n a na p p l i c a t i o n p r i o r t o t h e p r o j e c t . P r e v i o u s r e s e a r c hb y J e f f e r y a n d L a w r e n c e ( 1 98 5 ) ha s s h o w n t h a ta d d i t io n a l e x p e r i e n c e b e y o n d 2 4 m o n t h s d o e s n o ts e e m t o r e s u l t i n i n c r e a s e d p r o d u c t i v i t y . P r o j e c ts t a f f at th e B a n k h a d i n d i c a t e d t h a t a m i n i m u m o fo n e n o n - n o v i c e i n d i v i d u a l w a s n e c e s s a ry i n o r d e r t o" l e v e r a g e " h i s o r h e r s k i l l s o v e r a n y a p p l i c a t i o nn o v i c e t e a m m e m b e r . T h e a b s e n c e o f t h is d e si r a b les i tu a t i on w a s q u a n t i f i e d b y s e tt i ng a d u m m y

v a r ia b le , L O A P P E X P , e q u a l to o n e w h e n 9 0% o rm o r e o f t h e h o u r s w e r e c h a r g e d b y a p p l ic a t i o nn o v i c e s . T h i s v a r i a b l e w a s s i g n i f i c a n t a t t h e 5 %l e ve l . T h e s i g n i f ic a n c e o f p r o j e c t t e a m c a p a b i l it ya n d a p p l i c a t io n e x p e r i e n c e i n e x p la i n i n g s o f t w a r em a i n t e n a n c e p r o d u c t i v i t y i s c o n s i s t e n t w i th m u c hp r i o r r e s e a r c h o n n e w s o f t w a r e d e v e l o p m e n t .S i m i l ar ly , s t a f f m e m b e r s w e r e c a t e g o r i z e d a s d a t ap r o c e s s i n g n o v i c e s i f t h e y h a d l e s s t h a n 2 4 m o n t h so f d a t a p r o c e s si n g e x p e r i e n c e . T h e e x p e r i e n c ec o u l d i n c l u d e n o n - B a n k d a t a p r o c e s s i n g e x p e r i e n c e ,a l t h o u g h , d u e t o t h e B a n k ' s h i r i n g p o l i c i e s , t h e v a s tm a j o r i t y o f s t a f f m e m b e r s i n t h is s t u d y h a d o n l yB a n k e x p e ri e n c e . P r o j e c t s w i t h a m a j o r i t y o f d a t ap r o c e s s i n g n o v i c e s w e r e e x p e c t e d t o b e l e s sp r o d u c t i v e t h a n t h o s e n o t s o b u r d e n e d . A c c o r d i n g -l y , t h e d u m m y v a r i a b l e L O D P E X P e q u a l s o n e w h e n5 0 % o r g re a t e r o f t h e h o u r s c h a r g e d w e r e b y d a tap r o c e s si n g n o v i c e s. L O D P E X P w a s n o t a ss i g n i fi c a n t a s L O A P P E X P s i n e x p l a i n in g v a r ia t io n si n s o f t w a r e m a i n t e n a n c e p r o d u c t i v i t y .6 . 2 . 2 P r o j e c t m a n a g e m e n t v a r i a b l e sT h e f i r s t s i g n i f i c a n t p r o j e c t m a n a g e m e n t v a r i a b l e i nt h is a n a l ys i s is L O A D I N G , t h e r a te a t w h i c h p e o p l ea r e a d d e d t o t h e p ro j e c t . L O A D I N G i s e q u a l t o t h e

170


12/16

t ot al n u m b e r o f w o r k - m o n t h s d i v i d e d b y t h e to ta lp r o j e c t d u r a t i o n i n c a l e n d a r m o n t h s . H i g h e r lo a d i n gi n d i ca t e s a g r e a t e r a m o u n t o f p a r a ll e l is m o n t h ep r o j e c t , p l u s a p o s si b l e i n c r e a s e i n t h e a m o u n t o fp r o j e c t c o m m u n i c a t i o n s . T h i s w a s f o u n d t o h a v e an e g a t i v e i m p a c t o n p r o d u c t i v i t y a t th e 5 % l e v e l, a n di s c o n s i s t e n t w i t h t h e r e s u l t s o f r e s e a r c h e r s o n n e ws o f tw a r e d e v e l o p m e n t .A s e c o n d s i g n i f i c a n t v a r i a b l e i s d e a d l i n e p r e s s u r e .P r o j e c t l e a d e r s a n d t h e i r s e c t i o n h e a d s w e r e a s k e do n t h e ir s u r v e y s w h e t h e r t h e r e w a s g re a t e r t h a na v e r a g e d e a d l i n e p r e s s u r e o n t h e p r o j e c t . T h i sv a r ia b l e, T I G H T D E A D , w a s f o u n d t o be a s i gn i fi c a n tb o o s t to p r o d u c t i v i t y , a t l e a st in t h e s h o r t r u ns e n se i n d i c a t e d b y t h is m e a s u r e . T h e e x p l a n a t io ns e e m s t o b e t h a t i n c r e a s e d d e a d l i n e p r e s s u r er e d u c e s , a t l e as t f o r t h e d u r a t i o n o f t h e p r o j e c t ,s o m e a m o u n t o f t h e s l a c k t h a t i s p r e s e n t i n a n yo r g a n iz a t io n . W h e t h e r a n o r g a n i z a t io n w o u l d w a n tt o p u r s u e t h i s t a c t i c a s a l o n g - t e r m s t r a t e g y i sq u e s t i o n a b l e , h o w e v e r , g i v e n t h e l i k e l y d e l e t e r io u se f f e c t o n m o r a l e a n d t h e r e s u l t i n g i n c r e a s e i nt u r n o v e r . T h i s i s p a r t i c u l a r l y i m p o r t a n t i n l i g h t o ft h e s i g n if i c a n c e o f t h e a p p l i c a t i o n e x p e r i e n c ev a r i a b l e .

6 . 2 . 3 U s e r v a r i a b l e sI n g e n e r a l , t h e s i g n i f i c a n c e o f t h e u s e r v a r i a b l e sw a s l o w . 9 G i v e n t h e B a n k s t a f f ' s a p r i o r i s u g g e s -t i o ns , t hi s w a s a s u r p r i s i n g r e s u l t. I t m a y b e t h a tt h e i m p r e s s i o n t h a t p o o r u s e r r e l a t i o n s h i p s l e a v ew i t h p r o j e c t l e a d e r s i s g r e a t e r t h a n t h e i r a c t u a le f f e c t o n p r o j e c t p r o d u c t i v i t y .T h e m o s t s i g n i f i c a n t o f t h e u s e r v a r i a b l e s w a sI N T E R N A L . A s u r v e y q u e s ti o n c o n c e r n in g t h en u m b e r o f u s e r s i g n o f f s r e q u i r e d w a s d e s i g n e d t oi d e n t i f y t h o s e p r o j e c t s t h a t n e e d e d t o r e a c ha g r e e m e n t a c r o s s m u l t i p l e u s e r s . I n t e r m s o fr e s p o n se s , h o w e v e r , v e r y f e w p r o j e c t s h a d g r e a t e rt h a n o n e u s e r , w h i l e a s i g n i f i c a n t n u m b e r o fp r o j e c t s w e r e i n t e r n a l l y g e n e r a t e d , t y p i c a l l y t oi n c r e a s e e f f i c i e n c y o r t h r o u g h p u t o n a n a p p l i c a t i o n .T h i s v a ri a b l e w a s c o d e d a s a z e r o - o n e d u m m y ,w h e r e I N T E R N A L ffi I m e a n t t h a t n o o u t s i d e u s e rsw e r e i n v o l v e d . T h e f a c t t h a t t h e s e p r o j e c t s m a y b em o r e e f f i c i e n t i s n o t s u r p r i s i n g , s i n c e t h e r e m o v a lo f t h e n e e d t o c o m m u n i c a t e s p e c i f i c a t io n s a c r o s sd e p a r t m e n t s a n d t h e l i k e ly re d u c e d d o c u m e n t a t i o nb u r d e n w o u l d a id i n i n c r e a si n g e f f i c i e n c y o f p r o d u c td e v e l o p m e n t .

6 . 2 . 4 T e c h n i c a l e n v i r o n m e n t v a r i a b le sT h e m o s t s i g n i f i c a n t v a r i a b l e i n t h e l i st o fT e c h n i c al E n v i r o n m e n t v a r ia b l e s w a s G O O D R E S P , ad u m m y v a r i a b l e i n d i c a t i n g e i t h e r a n i n t e r a c t i v ed e v e l o p m e n t e n v i r o n m e n t o r g o o d ( < 4 h o u rs ) b a t c ht u r n a r o u n d . T h i s h a d t h e e f f e c t o f i m p r o v i n gp r o d u c t i v i t y , w h i c h i s i n t u i ti v e , a n d c o n s i s t e n t w i t hs o m e l i m i t e d r e s e a r c h i n th i s a r e a ( B o e h m 1 98 1;L a m b e r t 1 9 8 4 ) .A s e c o n d s i g n i f ic a n t v a r i a b l e w a s th e d u m m yv a r ia b l e S T R C M E T H , w h i c h i n d i c a t ed t h e u se o f as t r u c t u r e d a n a l y si s a n d d e s i g n m e t h o d o l o g y b a s e d o nt h e G a n e / S a r s o n p r i n c i p l e s a n d t o ol s. P r o j e c t su s i n g t h i s m e t h o d o l o g y w e r e l e s s p r o d u c t i v e t h a nt h o s e th a t d i d n o t , a n i n i t i a ll y e y e - o p e n i n g r e s u l tf o r t h e m a n a g e r s a t t h e B a n k , b u t o n e t h a t a c t u a l lym a k e s a g o o d d e a l o f s e n s e u p o n c l o se s c ru t i n y .W h a t i s b e in g m e a s u r e d i s a s n a p s h o t o f s h o r t - t e r mp r o d u c t i v it y , n o t l o n g - t e r m p r o d u c t i v it y . M a n y o ft h e b e n e f i t s o f u s i n g a d e t a il e d m e t h o d o l o g y t h a tr e q u i r e s a l o t o f d o c u m e n t a t i o n a r e n o t o b s e r v e du n t il t h e n e x t p r o j e c t , w h e n e n h a n c e m e n t o r r e p a ir sn e e d t o b e m a d e t o t h e s y s t e m . I n t h e s h o r t t e r m ,t h e e x t r a e f f o r t i s n o t n e c e s s a r i l y g o i n g t o s h o wa n y b e n e f i t , a n d t h e e x t r a h o u r s w i l l s h o w u p a sr e d u c e d p r o d u c t i v i t y . A d d i t i o n a l l y , i t s h o u l d b ea d d e d t h a t u s e o f t h i s m e t h o d o l o g y w a s n e w a t t h eB a n k , a n d w a s , a t l e a s t f o r o n e o f t h e s e c t i o n s ,e x a c t l y c o i n c i d e n t w i t h t h e p r o j e c t s c o l l e c te d i nt h is d a t a s e t . T h e r e f o r e , t h is f a c t o r m a y a l soe x h i b i t a l e a r n i n g c u r v e .V O L A T L T Y , d e f i n e d a s f r e q u e n t c h a ng e s to th eh a r d w a r e / s o f t w a r e e n v i r o n m e n t , e i t h e r e v e r y f e ww e e k s f o r m a j o r c h a n g e s o r e v e r y f e w d a y s f o rm i n o r c h a n g e s , w a s o n l y m a r g i n a l ly s i g n if ic a n t .T h i s v a r i a b l e w a s a l so s h o w n t o r e d u c e p r o d u c t i v i t y( c o n s i s t e n t w i t h o t h e r r e s e a r c h e r s , s e e B o e h m 1 9 8 1 ),a l t h o u g h o n l y a t t h e 1 5 % s i g n i f i c a n c e l e v e l .G o o d d o c u m e n t a t io n ( G O O D D O C ) ha d b e e n s u g g es t edb y m a n a g e r s a t t h e B a n k a s a p o t e n t i a l i m p o r t a n tf a c t o r i n e x p l a i n i n g p r o d u c t i v i t y . A s s h o w n inT a b l e 7 , i t w a s n o t a s i g n i f i c a n t f a c t o r .6 . 2 . 5 Q u a l i t y a s a p r o d u c t i v i t y v a r i a b l eO n e r e m a i n i n g q u e s t i o n a b o u t t h e s e p r o d u c t i v i t ym e a s u r e s i s t h e r e l a ti o n s h i p b e t w e e n t h e m o s tp r o d u c t i v e p r o j e c t s a n d q u a li ty : D o p r o j e c t s w i t hh i g h q u a l i t y e x h i b i t h i g h p r o d u c t i v i t y , o r is h i g hp r o d u c t i v i t y a t t a i n e d o n l y b y s a c r if i c i n g q u a l i t y ?

17 1


13/16

Tab l e g. Operational Quality Versus Productiv ity

L o w P r o d u c t i v it y M e d P r o d u c t i v it y H i g hL o w Q u a l it y 3 1M ed Qu a l i ty 1 5 1 2H igh Qu a l i t y 3 8

P roduc t i v i t y51 62

Table 9. User Project Satisfaction Versus Productiv ity

L o w P r o d u c t i v i t yA b o v e A v e r a g e S a t i s f a c t i o n ?

No 7Yes 6

M ed P rod uc t i v i t y H igh P rod uc t i v i t y

3 81 2 9

A first attempt to answer this question wouldinvolve adding the two quality metrics, operationalquality (QUALITY) and user project satisfaction(CUSTACCP), as independent variables in themultiple regression analysis. Unfort unatel y, only 45of the 65 user surveys were returned, and thereforeCUSTACCP was not employed as an independentvariable. However, QUALITY was added and wasnot significant. (See Table 7.) A second approachwas to cross-tabulate the data as shown in Tables 8and 9.Table 8 shows the operational quality data versus anaggregation of the productivity data. For lowquality projects, there is an approximately equalchance of a low or high productivity rating, andsimilarly for high quality projects. This explainswhy the quality variable is not found to be signifi-cant in the exploratory multivariate regressionanalysis. Therefo re, the data from this data-site donot support the hypothesis that achieving highproductivity or quality requires sacrificing theother, loTable 9 was constructed by converting a five pointscale on which data for user project satisfactionwere converted into a dummy variable, aboveaverage user project satisfaction. As with opera-tional quality, a relatively random spread of quality-productivi ty occurrences is seen. 11

7. CONCLUSIONS AND FUTURE RESEARCH

This paper explored the potential of developing aDEA-based model of software maintenance produc-tivity and sought to identify productivity factorsthat merit fur ther study. The estimation of thismodel using pilot data collected from a largecommercial bank have suggested several interestinginsights. Our analysis indicates that the factorsthat affect new software development (particularlypersonnel experience and capability) also seem toplay an important role in software maintenance.Our analysis suggests that high productivity appearsto be possible in a maintenance environment withoutsacrificing quality, and that the quality/productivityrelationship bears fur the r investigation.This research has raised many questions whichsuggest possible avenues for future research. Aninteresting methodological extension would be thesimultaneous consideration of output and inputvariables as well as the environmental factors in asingle model. Another methodological extensionwould involve estimating a stochastic frontier usingtechniques of Stochastic DEA. This involves acomposed error fo rmulation with a two-sided randomcomponent and a one-sided error caused byinefficiencies.

172


14/16

A n o t h e r a r e a f o r f u r t h e r w o r k s te m s f r o m t h e f a c tt h a t t h e p r o d u c t i v i t y m e a s u r e s u s e d i n t h i s a n a l y s i sa r e c l e a r ly s h o r t - t e r m . T h e lo n g - t e r m i m p a c t o np r o d u c t i v i t y o f s o m e o f t h e s e f a c t o r s ( p a r t i c u l a r l yt h e u s e o f s t r u c t u r e d a n a l y si s a n d d e s i g n m e t h o d -o l o g i e s ) w o u l d b e a n i n t e r e s t i n g e x t e n s i o n . T h en o t i o n o f l o n g - t e r m p r o d u c t i v i t y i s r e l a te d t oq u a l i t y i n t h a t a b e t t e r q u a l i t y p r o d u c t t o d a y s h o u l dr e s u l t i n l e ss m a i n t e n a n c e i n t h e f u t u r e . R e s e a r c hc o u l d b e d i r e c t e d a t m o d e l i n g s o f t w a r e q u a l i t y a s ap r i m a r y g o a l, r a t h e r t h a n a s a n a d j u n c t a s w a s d o n eh e r e . F i n a l l y , a l a r g e r a n d r ic h e r d a t a s e t c o u l da l l o w m o r e d e t a i l e d e x a m i n a t i o n o f t h e f a c t o r s a n dt h e i r p o s s i b l e i n t e r p l a y , a n e x e r c i s e n o t r e a l l yf e a s i b l e w i t h t h e l i m i t e d a m o u n t o f d a t a a v a i la b l e i nt h i s s t u d y .

E N D N O T E S1 T h i s r e s e a r c h w a s f u n d e d i n p a r t b y t h e C e n t e rf o r th e M a n a g e m e n t o f T e c h n o l o g y a n d I n fo r m a t i o ni n O r g a n i z a ti o n s , G r a d u a t e S c h o o l o f I n d u s tr i a lA d m i n i s t ra t i o n , C a r n e g i e - M e l l o n U n i v e r s i t y , a n d t h eI n t e r n a ti o n a l B u s i n e ss M a c h i n e s C o r p o r a t i o n .H e l p f u l c o m m e n t s f r o m f o u r a n o n y m o u s r e f e r e e s ar eg r a t e f u l l y a c k n o w l e d g e d .2 T h e t e r m " d e v e l o p m e n t " i s u s e d h e r e i n i ts m o s tg e n e r a l s e n s e , w h i c h i n c l u d e s m a i n t e n a n c ep r o g r a m m i n g . T h e t e r m " n e w d e v e l o p m e n t " w i l l b eu s e d i n t h i s p a p e r t o d e s c r i b e p r o g r a m m i n g t h a t iss t r i c t l y t h e g e n e r a t i o n o f n e w c o d e .3 N o t e t h a t t h i s is i n c o n t r a s t t o m a n y o t h e rp r o d u c t i o n s e t t i n g s w h e r e t h e m a n a g e r h a s f i x e di n p u t s a n d d e s i r e s t o m a x i m i z e o u t p u t .4 C o m p u t e r s c i e n ti s ts h a v e a l so d e v e l o p e d w h a tm i g h t b e t e r m e d " m i cr o " s o f t w a r e m e t r i c s , t h o s eb e l o w t h e l e v el o f a s o u r c e l in e o f c o d e . E x a m p l e so f t h e s e w o u l d b e t h e s o f t w a r e s c i e n c e m e t r i c s o fH a l s t e a d (1 9 7 7 ) a n d t h e c o m p l e x i t y m e a s u r e o fM c C a b e ( 19 7 6 ). T h e s e m e t r i c s h a v e g e n e ra l l y n o tb e e n a p p l i e d t o l a r g e s c a l e s o f t w a r e p r o d u c t i v i t yd u e t o d i f f i c u l t i e s i n m e a s u r e m e n t .5 T h e s e a r e d a t a c o m m u n i c a t i o n s , d i s t r i b u t e dp r o c e s s i n g , a p p l i c a t i o n p e r f o r m a n c e o b j e c t i v e s ,h e a v i l y u s e d c o n f i g u r a t i o n , h i g h t r a n s a c ti o n r a te ,o n l i ne d a t a e n t r y , e n d u s e r e f f i c i e n c y , o n l in eu p d a t e , c o m p l e x p r o c e s s i n g , r e u s a b i l i t y , i n s t a l l a t i o n

e a s e , o p e r a t i o n a l e a s e , m u l t i p l e s i t e s , a n d f l e x i b i l i t y(see A l b r e c h t 1 9 8 4 ) .e S e e , f o r e x a m p l e , t h e s u r v e y b y M o h a n t y ( 1 97 9 ).7 O f c o u r s e , w h i l e t h e d a t a s e t c o n t a i n s 6 5 p r o j e c t s ,t h e y w e r e a l l g a t h e r e d w i t h i n o n e o r g a n i za t io n .T h e r e f o r e , t h e e x t e r n a l v a l i d i t y o f t h e r e su l tsr e m a i n s t o b e d e m o n s t r a t e d .s T h e c o r r e la t io n c o e f f i c i e n t b e t w e e n L O D P E X P a n dL O A P P E X P i s . 3 0 .9 W e t e s t e d t h e p o s s i b i l i t y o f h i g h c o r r e l a t i o na m o n g th e u s e r v a r i a b l e s . T h e h i g h e s t c o r r e l a t i o n sb e t w e e n u s e r v a r i a b l e s ( a n d t h e h i g h e s t c o r r e l a t i o no f a n y i n d e p e n d e n t v a r i a b l e s) w e r e b e t w e e nS T A F F A P T a n d L O A G R E E ( .4 6) a n d b e t w e e nS T A F F A P T a n d I N T E R N A L (. 43 ). S ix s e p ar a ter e g r e s s i o n s w e r e r u n w i t h o n e u s e r v a r i a b l e a s t h ed e p e n d e n t v a r i a b l e a n d t h e o t h e r f i v e a s i n d e p e n -d e n t s , v a r y i n g t h e d e p e n d e n t v a r i a b l e o n e a c h r u n .T h e b e s t f i t ( R ~ = . 3 6) w a s f o r t h e S T A F F A P Tv a r i a b l e . W h i l e t h i s d e g r e e o f c o r r e l a t i o n w a s n o tb e l i e v e d t o b e la r g e , a ru n o f t h e m a i n p r o d u c t i v i t ym o d e l w a s m a d e , o m i t t in g S T A F F A P T . T h e t -s ta t is ti c f o r L O A G R E E i m p r o v e d , b u t n o t en o u g h tom a k e i t a s i g n i f i c a n t v a r i a b l e a t t h e 1 0 % l e v e l.lO T h e c h i - s q u a r e d t e s t v a l u e i s 7 . 9 3 , s i g n i f i c a n t a tt h e 1 0% l e v e l . T h e e x p l a n a t i o n f o r t h i s r e l a t i v e l yh i g h v a l u e i s t h a t t h e r e s e e m s t o b e s o m e d r i f tt o w a r d s a v e r a g e p r o d u c t i v i t y ( n e i t h e r h i g h n o r l o w )w h e n t h e q u a l i t y is h i g h. N o t e t h a t th e c h i - sq u a r et e s t m a y b e i n a p p r o p r i a t e i n t h i s c a s e , g i v e n t h a t2 / 3 o f t h e c e ll s h a v e e x p e c t e d v a l u e s l e ss t h a n f i v e .11 T h e c h i - s q u a r e d t e s t v a l u e i s 3 .8 9 , s i g n i f i c a n t a tt h e 1 5% l e v e l . T h e e x p l a n a t i o n o f t h e s e r e s u lt sg i v e n i n f o o t n o t e 1 0 a p p l i e s h e r e a s w e l l , m u t a t i sm u t a n d i s .RE F ERENCE SA l b r e c h t , A . J . A D / M P r o d u c ti v it y M e a s u re m e n ta n d E s t i m a t e V a l i d a t i o n . C I S & A G u i d e l i n e 3 1 3 ,I B M C o r p o r a t e I n f o r m a t i o n S y s t e m s a n d A d m i n i s t r a -t i o n , N o v e m b e r 1 , 1 9 8 4 .A l b r e c h t , A . J . " F u n c t i o n P o in t s H e l p M a n a g e r sA s s e ss A p p l i c a t i o n , M a i n t e n a n c e V a l u es ."C o m p u t e r w o r l d S p e c i a l R e p o r t o n S o f t w a r e P r o d u c -t ivi ty , C W C o m m u n i c a t i o n s , 1 9 85 , p p . S R 2 0 - S R 2 1 .

17 3


15/16

Albre ch t , A . J . , and Ga f fne y , J . J r . "Sof twareF u n c t i o n , S o u r c e L i n e s o f C o d e , a n d D e v e l o p m e n tE f fo r t P red ic t ion : A Sof twa re Sc ience Val ida t ion . "IEEE Transactions on Software Engineering, S E - 9 ,No . 6 , Novem ber 1983 , pp . 639-648 .Bank er , R . ; Cha rnes A .; and Cooper , W . "Som eM o d e l s f o r E s t i m a t i n g T e c h n i c a l a n d S c a l eI n e f f i c i e n c i e s i n D E A . " Management Science, Vol.30, No. 9 , September 1984, pp. 1078-1092.Behrens , C . A . "Mea sur ing the P rod uc t iv i ty o fC o m p u t e r S y s t e m s D e v e l o p m e n t A c t i v i t i e s w i t hFunc t ion Po in t s . " IEEE Transactions on SoftwareEngineering, SE -9 , No . 6 , Novem ber 1983 , pp . 648-652.Be ls ley , D . ; Kuh , E . ; and W elsch , R . RegressionDiagnostics. John W i ley and Sons , Ne w York , 1980 .Boehm , B . W . Software Engineering Economics.Pren t ice -H al l , E ng lew ood Cl i f f s , N J , 1981 .C a s e , A . F . " C o m p u t e r - A i d e d S o f tw a r e E n g i n e e r i n g ."Database, Vol. 17, No. 1, Fall 1985, pp. 35-43.Charne s , A . ; Cooper , W . W .; and Rho des , E ." E v a l u at i n g P r o g r a m a n d M a n a g e r i a l E f f i c i e n c y : A nA p p l i c a t i o n o f D a t a E n v e l o p m e n t A n a l y s i s t oP r o g r a m F o l l o w T h r o u g h . " Management Science,Vol. 27, No. 6 , June 1981, pp. 668-697.C h r y s l e r , E . " S om e B a s ic D e t e r m i n a n t s o f C o m p u t e rP r o g r a m m i n g P r o d u c t i v i t y . " Communications of theACM, Vol. 21, No. 6 , June 1978, pp. 472-483.Cur t i s , B . "Subs tan t i a ting P rog ram m e r Var iab i l i ty . "Proceedings of the IEEE Conference, Vol. 69, No. 7 ,July 1981, p. 846.D e M a r c o , T . Controlling Software Projects.Yourdon P res s , New York , 1982 .E lshof f , J . L . "An Analys i s o f Som e Com m erc ia lPL / I P rogram s . " IEEE Transactions on SoftwareEngineering, Vol. SE2, No. 2 , 1976, pp. 113-120.F r e e d m a n , D . H . " P r o g r a m m i n g W i t h o u t T e a rs ."High Technology, Vol. 6 , No. 4 , Apr i l 1986, pp. 38-45.G a f f n e y , J . E . " T h e I m p a c t o n S o f t w a r e D e v e l o p -m en t Cos t s o f Us ing HOL ' s . " IEEE Transactions on

Software Engineering, Vol . SE -12 , No . 3 , March1986, pp. 496-499.G a y l e , J . B . " M u l t i p l e R e g r e s s i o n T e c h n i q u e s f o rE s t i m a t i n g C o m p u t e r P r o g r a m m i n g C o s t s . " Journalof Systems Management, Vol . 22 , No . 2 , February1971, pp. 13-16.Gram m as , G . W . , and K le in , J . R . "Sof twareP roduc t iv i ty as a S t r a teg ic Var iab le . " Interfaces.Vol. 15, No. 3 , May-June 1985, pp. 116-126.Hals tead , M. H . Elements of Software Science.E lsev ie r , New York , 1977 .J e f f e r y , D . R . , a n d L a w r e n c e , M . J . " M a n a g in gP r o g r a m m i n g P r o d u c t i v it y . " Journal of Systems andSoftware, Vol. 5 , 1985, pp. 49-58.Jones , C . Programming Productivity. M c G r a w - H i l lB o o k C o m p a n y , N e w Y o r k , 1 9 8 6 .K e m e r e r , C . F . " A n E m p i r i c a l V a l i d a t io n o fSof tware Cos t E s t im at ion Models . " Communicationsof the ACM, Voi. 30, No. 5 , May 1987, pp. 416-429.Ko lodz ie j , S . "Gain ing Con t ro l o f Main tenan ce . "Computerworld Focus, Vol . 20 , No . 7A , February 19 ,1986, pp. 31-36.K r i e b e l , C . H . " E v a l u a t in g t h e Q u a l i t y o f I n f o r m a -t ion Sys tem s ." In N . Szyper sk i and E . Groc h la(eds . ) , Design and Implementation of ComputerBased Information Systems, S i t j h o f f & N o o r d h o f f ,T he Nether lands , 1979 , Chap te r 2 , pp . 29 -43 .K r i e b e l , C . H . , a n d R a v i v , A . " A n E c o n o m i c sA p p r o a c h t o M o d e l i n g t h e P r o d u c t i v i t y o f C o m p u t e rSys tems." Management Science, Vol. 26, No. 3 ,March 1980 , pp . 297-311 .K r i e b e l , C . H . , a n d R a v i v , A . " A p p l i c a ti o n o f aP r o d u c t i v i t y M o d e l f o r C o m p u t e r S y s te m s ." DecisionSciences, Vol. 13, Apr i l 1982, pp. 266-284.L a m b e r t , G . N . " A C o m p a r a t i v e S t u d y o f S y s t e mR e s p o n s e T i m e o n P r o g r a m D e v e l o p e r P r o d u c t i v i t y . "IBM Systems Journal, Vol. 23, No. I , 1984, pp. 36-43.L ien tz , B . P . , and Swanson , E . B . SoftwareMaintenance Management. A d d i s o n - W e s l e y , R e a d i n g ,MA, 1980.

174


16/16

Lien tz , B . P . , an d Swan so n , E . B. " Pr o b lem s inA p p l i c a t i o n S o f t w a r e M a i n t e n a n c e . " C o m m u n i c a t i o n so f t h e A C M , Vo l . 2 4 , No . 1 1 , No v em b er 1 9 8 1 , p p .7 6 3 - 7 6 9 .M c C a b e , T . " A C o m p l e x i t y M e a s u r e ." I E E ET r a n s a c t i o n s o n S o f t w a r e E n g i n e e r i n g , V o l . S E - 2 ,D e c e m b e r 1 9 7 6 , p p . 3 0 8 - 3 2 0 .M c K e e n , J . D . " S u c c e s sf u l D e v e l o p m e n t S t r a te g i e sf o r B u s i n e s s A p p l i c a t i o n S y s t e m s . " M I S Q u a r t e r l y ,Vo l . 7 , No . 3 , Sep tem b er 1 9 8 3 , p p . 4 7 - 6 5 .M o h a n t y , S. N. " M o d e l s a n d M e a s u r e m e n t f o rQ u a l i t y A s s e s s m e n t o f S o f t w a r e . " A C M C o m p u t i n gSurv e y s , Vol . 11 , 1979 , pp . 251-275 .M o h a n t y , S . " S o f t w a re C o s t E s t i m a t i o n : P r e s e n ta n d F u t u r e . " S o f t w a r e : P r a c t i c e a n d E x p e r i e n c e ,Vol . 11 , 1981 , pp . 103-121 .P a r i k h , G . " R e s t r u c t u r i n g Y o u r C O B O L P r o g r am s . "C o m p u t e r w o r l d F o c u s , Vo l . 2 0 , No . 7 A, Feb r u a r y 1 9 ,1986 , pp . 39-42 .P e r r y , W . E . T h e B e s t M e a s u r e s f o r M e a s u r i n g D a t aP roc e s s ing Q ua l i t y and P roduc t i v i t y . Q u a l i t yAssu r an ce I n s t i t u t e , 1 9 8 6 .P i n d y c k , R . S . , a n d R u b i n f e l d , D . L . E c onom e t r i cM o d e l s a n d E c o n o m i c F o r e c a s t s . M c G r a w - H i l l B o o kC o m p a n y , N e w Y o r k , 1 9 81 .Po wer s , R . F . A n E m p i r i c a l I n v e s t i g a t i o n o fS e l e c t e d H y p o t h e s e s R e l a t e d t o t h e S u c c e s s o fM a n a g e m e n t I n f o r m a t i o n S y s t e m P r o j e c t s .U n p u b l i s h e d P h . D . t h e s i s , U n i v e r s i t y o f M i n n e s o t a ,Apr i l 1971 .

P u t n a m , L . H . " G e n e r a l E m p i r i c a l S o l u t i o n t o t h eM a c r o S o f t w a r e S i z i n g a n d E s t i m a t i n g P r o b l e m . "I E E E T r a n s a c t i o n s o n S o f t w a r e E n g i n e e r i n g , Vol . 4 ,1978 , p p . 3 4 5 - 3 6 1 .R u b i n , H . A . U s in g E S T I M A C S E . M a n a g e m e n t a n dC o m p u t e r S e r v ic e s , In c . , V a l l e y F o r g e , P A .Sack m an , H . ; E r ik so n , W . J . ; an d Gr an t , E . E ." E x p l o r a t o r y E x p e r i m e n t a l S t u d i e s C o m p a r i n g O n l i n ea n d O f f i n e P r o g r a m m i n g P e r f o rm a n c e . " C o m m u n i c a -t i o n s o f t h e A C M , Vol . 11 , No. 1 , January 1968 , pp .3 - 1 1 .S c o t t, R . F . , a n d S i m m o n s , D . " P r o g r a m m e rP r o d u c t i v i t y a n d t h e D e l p h i T e c h n i q u e . "D a t a m a t i o n , Vo l . 2 0 , No . 5 , May 1 9 7 4 , p p . 7 1 - 7 3 .S tab e l l, C . B . " Of f i ce P r o d u c t iv i ty : A M ic r o -e c o n o m i c F r a m e w o r k f o r E m p i r i c a l R e s ea r ch ."O f f i c e T e c h n o l o g y a n d P e o p l e , Vol. 1 , No. 1, 1982,p p . 9 1 - 1 0 6 .V a c c a , J . " F u n c t i o n P o i n ts : T h e N e w M e a s u r e o fSo f twar e . " C o m p u t e r w o r l d X I X , N o . 4 6 , N o v e m b e r18 , 1985 , pp . 99-108 .W als to n , C . E . , an d Fe l ix , C . P . " A M eth o d o fP r o g r a m m i n g M e a s u r e m e n t a n d E s t im a t io n . " I B MS y s t e m s J o u r n a l , Vol . 16 , No. 1 , 1977 , pp . 54-73 .W o lv e r to n , W . R . " C o s t o f Dev e lo p in g La r g e Sca leSo f twar e . " I E E E T r a n s a c t i o n s o n C o m p u t e r s , Vo l .2 3 , Ju n e 1 9 7 4 , p p . 6 1 5 - 6 3 4 .Z a v a l a , A . R e se arc h on F ac tor s t ha t In f l ue nc e t heP r o d u c t i v i t y o f S o f t w a r e D e v e l o p m e n t W o r k e r s .F i n a l R e p o r t 4 6 7 7 - 8 5 - F R - 6 8 , S R I I n te r n a ti o n a l, J u n e ,1985.

17 5

Factors Affect Software Maintenance

Documents