Page 1
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 1
Assessing and Estimating Corrective, Enhancive, and Reductive Maintenance
Tasks: A Controlled Experiment *
Presented by Vu Nguyen
on behalf of
Vu Nguyen, Barry Boehm, Phongphan Danphitsanuphan
(*) paper accepted for Asia-Pacific Software Engineering Conference 2009 (APSEC 2009)
Page 2
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 2
Outline
Motivation and Background
Experiment Design
Results and Explanatory Models
Conclusions
Page 3
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 3
Motivation and Background
Page 4
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 4
Maintenance is crucial in software engineering
• Systems are tightly coupled with their environment
– Environment changes require changing its software systems
• Technologies and requirements are continuously changing
– Software systems are outdated quickly
– Software systems must be updated and upgraded to maintain their values
• Maintenance is important in market competition
– New software has more advantages than the existing one
– Software system must be upgraded to keep its market share
Page 5
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 5
Majority of software costs incur after the first operational release (Boehm ’81)
• Maintenance cost is usually 2x to 100x as much as new development cost (Sommerville, 2006)
K$
Time
100 200 300
System 1
System 2
Release 1 Release 2 Release 3
Release 1 Release 2
Adapted from (Summerville, 2006)New Development
Maintenance
Release N
Page 6
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 6
Software estimation community has paid little attention to software maintenance
• Most estimation models regard maintenance estimation as secondary
– COCOMO, SEER-SEM, SLIM, PRICE-S models were built using mainly data of new development projects
• Use of SLOC metrics for models is inconsistent
– Some models use SLOC added, modified, deleted
– Others use only SLOC added and modified
• Impact of different SLOC metrics on productivity has not been investigated
Projects tend to use experience or expert judgment methods to estimate software effort and cost instead
Page 7
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 7
Research Questions and Hypotheses
• RQ1: Are there any differences in the productivity of enhancive, corrective, and deductive maintenance?
– Hypothesis 1 (H1): no difference
• RQ2: Are there any differences in the effort distribution among the maintenance types?
– Hypothesis 2 (H2): no difference
Explanatory models to estimate participant’s maintenance effort
Page 8
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 8
Software Maintenance
• Software Maintenance
– “Modification of a software product after delivery to correct faults, to improve performance or other attributes, or to adapt the product to a modified environment” [IEEE ‘98]
• Types of Maintenance
– Swanson ’76: Adaptive, Corrective, Perfective
– IEEE ’98: all Swanson’s plus Preventive
– Chapin et al, 2001: 12 types, including three business rules types:• Enhancive• Corrective• Reductive
Page 9
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 9
Experimental Design
Page 10
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 10
Experiment DescriptionTask Group # Tasks # Participants
Enhancive 5 7
Corrective 6 9
Reductive 6 8
• 23 masters’ students and 1 senior, computer science major
• Participants worked on tasks individually in the lab
– Enhancive: add new capabilities
– Corrective: fix the existing capabilities
– Reductive: remove the existing capabilities
• UCC as a target program
– 5K+ source statements (logical SLOC) in 20 C++ classes
• MS Visual Studio 2005 was used for maintenance
Page 11
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 11
Calculating Maintenance SLOC
Equivalent SLOC = TRCF x AAM
1,100
1,])1(1[1( 2
AAFforUNFMxSU
AAF
AAFforUNFMxSUxAAFAAFAAM
TRCF
SAAF
TRCF = the total SLOC of task-relevant code fragments
S = the size in SLOC (added, modified, or deleted)
SU = the software understandability
UNFM = the level of programmer unfamiliarity with the program
Task-relevant code fragment
(TRCF)
Page 12
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 12
Results and Models
Page 13
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 13
Resulted Data
• 24 students participated
• Each task requires four activities
– Task comprehension
– Code isolation
– Editing code
– Unit test
• Timesheet has 490 activity records, totaling 77.02 hours
• Total of 909 SLOC added, modified, and deleted
402 added
216 modified
291 deleted
Page 14
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 14
Effort distribution is different among the groups
• Corrective group spent much time for code isolation
– twice as much as that of the enhancive group
• Enhancive group spent majority of time for editing code
• Effort distribution is statistically different among three groups (p-value = 0.0013)
– H2 is rejectedKruskal-Wallis rank-sum test
Page 15
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 15
Productivity is significantly different among the groups
• Corrective group has lowest productivity
• Reductive group has highest productivity
• Productivity between groups are statistically different (p-value = 0.0004)
– H1 is rejectedEnhancive Reductive Corrective
Pro
du
ctiv
ity
(SL
OC
/Hou
r)
40 30 20 10
Page 16
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 16
Participant Time Explanatory Models
• Two models for participant effort
M1 Effort = 78.1 + 2.2 * S * EAF
M2 Effort = 43.9 + (2.8*Add + 5.3*Mod + 1.3*Del) * EAF
Effort = Time spent by the participant on all maintenance tasks
EAF = Effort adjustment factor, a product of Programmer capability (PCAP), Language experience (LTEX), and Platform Experience (PLEX)
Add, Mod, Del = Equivalent SLOC added, modified, and deleted by the participant, respectively
S = Add + Mod + Del
All the estimates of coefficients are significant (p-value < 0.05)
R2 = 0.5 R2 = 0.75
Page 17
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 17
Model M2 outperforms M1
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Participant #
MR
E
M1 M2
M1 M2
MMRE 33% 20%
PRED(.25) 46% 71%
PRED(.3) 58% 79%
i
iii Actual
EstimateActualMRE
Page 18
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 18
Threats to Validity
• Internal design
– Groups have imbalanced skills and capability
– Imbalanced complexity of tasks on three groups
– Incorrect time recorded
• Generalizability
– Professional programmers are more experienced
– Professional programmers are more familiar with the software maintained
– Maintenance process is different in industry
Page 19
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 19
Conclusions
• Productivity and effort distribution are significantly different among the maintenance types
• SLOC metrics are relevant factors for estimating effort
• Three SLOC metrics (added, modified, and deleted) have different impact on effort
– SLOC deleted is an important factor for estimating effort
– It is more expensive to modify than to add or delete a statement
• Assigning experienced programmers to fixing defect can save up to 40% of effort
Page 20
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 20
References
• Barry W. Boehm, “Software Engineering Economics”, Prentice Hall, 1991
• Ian Sommerville, “Software Engineering,” 8th Ed., Addison-Wesley, 2006
• Ned Chapin, et al., “Types of software evolution and software maintenance,” Journal of Software Maintenance: Research and Practice, v.13 n.1, p.3-30, Jan. 2001
• IEEE, IEEE Standard Glossary of Software Engineering Terminology. Institute of Electrical and Electronics Engineers: New York NY, 1990: 83 pp
• IEEE Std 1219-1998, IEEE Standard for Software Maintenance, IEEE Computer Society, 1998