1 Detecting Defects in Object-Oriented Designs: Using Reading Techniques to Increase Software Quality UFRJ COPPE Department of Computer Science Experimental Software Engineering Group Fraunhofer Center - Maryland Current Research Team: Prof. Victor R. Basili Forrest Shull, Ph.D. Guilherme H. Travassos, D.Sc. (1) Jeffrey Carver, Graduate Research Assistant {basili,travassos,carver}@cs.umd.edu [email protected]This work is partially supported by UMIACS and by NSF grant CCR9706151 (1) On leave from the Federal University of Rio de Janeiro - COPPE/Computer Science and System Engineering Department, partially supported by CAPES - Brazil DCS/ESEG OO Reading Techniques UFRJ COPPE Evolving a Process for Inspecting OO Designs Outline: y Reading Techniques x Families of Reading Techniques y Reading Techniques for OO Design x First Experiment at UMCP • Lessons Learned from the Initial Study y The Road Ahead x Observational Studies • Lessons Learned from the Observational Studies y Open Questions
13
Embed
Detecting Defects in Object-Oriented Designs: Using ...mvz/mswe609/oort1.pdf · 1 Detecting Defects in Object-Oriented Designs: Using Reading Techniques to Increase Software Quality
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Detecting Defects in Object-OrientedDesigns: Using Reading Techniques to
Increase Software Quality
UFRJCOPPE
Department of Computer ScienceExperimental Software Engineering Group Fraunhofer Center - Maryland
Current Research Team:Prof. Victor R. BasiliForrest Shull, Ph.D.
Guilherme H. Travassos, D.Sc. (1)Jeffrey Carver, Graduate Research Assistant
This work is partially supported by UMIACS and by NSF grant CCR9706151
(1) On leave from the Federal University of Rio de Janeiro - COPPE/Computer Science andSystem Engineering Department, partially supported by CAPES - Brazil
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Evolving a Process for Inspecting OODesigns
Outline:y Reading Techniques
x Families of Reading Techniques
y Reading Techniques for OO Designx First Experiment at UMCP
• Lessons Learned from the Initial Study
y The Road Aheadx Observational Studies
• Lessons Learned from the Observational Studies
y Open Questions
2
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques
Why read?
Software practitioners are taught how to write, buttypically not how to read, software documents
Many software processes assume that practitioners knowhow to effectively find the information they need (i.e. howto read) such documents
e.g. inspection process: read a document to find defects
Defect General DescriptionOmission Necessary information about the system
has been omitted from the softwareartifact.
Incorrect Fact Some information in the software artifactcontradicts information in therequirements document or the generaldomain knowledge.
Inconsistency Information within one part of thesoftware artifact is inconsistent with otherinformation in the software artifact.
Ambiguity Information within the software artifact isambiguous, i.e. any of a number ofinterpretations may be derived that shouldnot be the prerogative of the developerdoing the implementation.
ExtraneousInformation
Information is provided that is not neededor used.
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques
z “Software reading techniques” attempt to increase theeffectiveness of inspections by providing proceduralguidelines that can be used by individual reviewers toexamine (or “read”) a given software artifact and identifydefects
z There is empirical evidence that software reading is apromising technique for increasing the effectiveness ofinspections on different types of software artifacts, notjust limited to source code.
monthly report informing payment on time[ payment time <= due time ]receive a monthly report
Default
monthly report informing late payment[ payment time > due time + 10 ]
monthly report informing late payment[ due time < payment time < due time + 10 ]
monthly report informing late payment[ payment time > due time + 10 ]
monthly report informing payment on time[ payment time <= due time ]
Loan State Diagram
Fanny May : Loan Arranger
Borrower : Borrower
A Lender : Specified Lender
Loan : Loan
verify_report()
new_loan(lender, borrowers)
new_
look_for_a_lender(lender)
look_for_a_loan(loan)
look_for_a_
update_loan(lender, borrower)
update_
lender :
new_lender(name,contact, phone_number)
update(lender)
monthly_report(lender, loans, borrowers)
identify_report_format()
Receive Monthly Report
Specified Lender
Investor
Fanny May
Receive Reports
Monthly Report
Investment Request
Request
Generate Reports
Loan Analyst
Loan Arranger Classes Description
Class name: Fixed_Rate Loan Category: Logical View Documentation: A fixed rate loan has the same interest rate over the entire term of the mortgage
External Documents: Export Control: Public Cardinality: n Hierarchy: Superclasses: Loan Public Interface: Operations: risk principal_remaining
State machine: No Concurrency: Sequential Persistence: Persistent
Operation name: risk Public member of: Fixed_Rate Loan Return Class: float Documentation: take the average of the risks' sum of all borrowers related to this loan if the average risk is less than 1 round up to 1 else if the average risk is less than 100 round up to the nearest integer otherwise round down to 100 Concurrency: Sequential
Banks generate income in many ways, often by borrowing money from their depositorsat a low interest rate, and then lending that same money at a higher interest rate in theform of bank loans. However, property loans, such as mortgages, typically have terms of15, 25 or even 30 years. For example, suppose that you purchase a $150,000 house witha $50,000 down payment and borrow a $100,000 mortgage from National Bank forthirty years at 5% interest. That means that National Bank gives you $100,000 to pay thebalance on your house, and you pay National Bank back at a rate of 5% per year over aperiod of thirty years. You must pay back both principal and interest. That is, the initialprincipal, $100,000, is paid back in 360 installments (once a month for 30 years), withinterest on the unpaid balance. In this case the monthly payment is $536.82. Althoughthe income from interest on these loans is lucrative, the loans tie up money for a longtime, preventing the banks from using their money for other transactions. Consequently,the banks often sell their loans to consolidating organizations such as Fannie Mae andFreddie Mac, taking less long-term profit in exchange for freeing the capital for use inother ways.
6
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO Design
Target Artifacts:
RequirementsDescriptions
Use-CasesRequirementsSpecification
High LevelDesign
ClassDiagrams
ClassDescriptions
State MachineDiagrams
InteractionDiagrams
(Sequence)Vertical readingHorizontalreading
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO Design
looking for defects:
Type of Defect DescriptionOmission One or more design diagrams that should contain some concept from
the general requirements or from the requirements document do notcontain a representation for that concept.
Incorrect Fact A design diagram contains a misrepresentation of a concept describedin the general requirements or requirements document.
Inconsistency A representation of a concept in one design diagram disagrees with arepresentation of the same concept in either the same or anotherdesign diagram.
Ambiguity A representation of a concept in the design is unclear, and could causea user of the document (developer, low-level designer, etc.) tomisinterpret or misunderstand the meaning of the concept.
ExtraneousInformation
The design includes information that, while perhaps true, does notapply to this domain and should not be included in the design.
Table 1 – Types of software defects, and their specific definitions for OO designs
7
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO DesignFirst Experiment at UMCP
Students organized in 15 teams (14 (3 students team) and 1 (2 studentsteam))
System: financial database application (a system responsible for organizing theloans held by a financial consolidating organization, and for bundling them for resale to investors)
Problem features: small system with low number of classes but withsome design complexity due to non-functional performancerequirements
Target programming languages: C++ or Java
Goal: to evaluate the first version of the reading techniques
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO DesignFirst Experiment at UMCP
Size Measures for the inspected design:Class Name Attrs. WMC DIT NOC CBO State
number of attributes (Attrs.), Weighted Methods/Class (WMC), Depth of Inheritance (DIT),Number of Children (NOC), and Coupling Between Objects (CBO)
8
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO DesignFirst Experiment at UMCP
Reader 1
Reader 2
Reader 3
looking for consistencyhorizontal reading
looking for consistencyhorizontal reading
looking for traceabilityvertical reading
Meet as a team to discuss acomprehensive defect list.Each reader is an “expert” ina different aspect
Final list of all defects sentto designer for repairing
The inspection process with OO reading techniques:
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO DesignFirst Experiment at UMCP
Subset of metrics collected in the feasibility study
When Collected MetricsBefore the study a) Details on subjects’ amount of experience with requirements, design, and codeAfter individual review b) Time spent on review (in minutes)
c) Opinion of effectiveness of technique (measured by what percentage of thedefects in the document they thought they had found)
d) How closely they followed the techniques (measured on a 3-point scale)e) Number and type of defects reported
After team meeting f) Time spent (in minutes)g) Usefulness of different perspectives (open-ended question)
In post-hoc interviews h) How closely they followed technique (open-ended question, to corroborate d)i) Were the techniques practical, would they use some or all of them again (open-
ended question)
9
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO DesignFirst Experiment at UMCP
Reading Technique for Class diagrams x Class descriptions For each class modeled into the class diagram, do:1)Read the class description to find an associated description to the class.F Underline with a yellow pen the portion of the Class descriptions corresponding to theclassVerify if:1.1)All the attributes are described and with basic types associatedF Underline them with a blue pen1.2)All the behaviors and conditions are describedF Underline them with a green pen1.3)All the class inheritance relationships are describedF Draw a box around them with a yellow pen if there is any1.4) All the class relationships (association, aggregation and composition) are described with multiplicity indicationF Circle each multiplicity indication with a blue pen if it is correct.
First version of a horizontal reading technique. The emphasis is on syntacticchecking, that is, that the OO notation on both diagrams agrees.
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO DesignFirst Experiment at UMCP
Some results from the experiment:y The definition of design defects by extending an already existing
defect taxonomyy It is important to measure the effectiveness (number of defects
found):x Subjects using vertical reading tended, on average, to report slightly
more defects of omission and incorrect fact (i.e. of types of defectsuncovered by comparisons against the requirements) than those usinghorizontal reading (6.8 versus 5.4 defects)
x Subjects using horizontal reading tended to report more defects ofambiguity and inconsistency (i.e. of types of defects uncovered byexamination of the design diagrams themselves) than subjects usingvertical reading (5.3 versus 2.9)
10
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO DesignFirst Experiment at UMCP
Some results from the experiment:y developers agreed that using:
x heuristics to construct OO artifacts is goodx some kind of OO reading technique is worthwhile
y some developers said that they would like to use the sametechniques again but, the mechanisms used to instrument themshould be improved. The study allowed us to identify weaknessesin the first version of the techniques that have led to a secondversion
y It was possible to demonstrate that such reading techniques canbe used as part of design inspections, and do help reviewersdetect defects
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO DesignThe Road Ahead:
1. Improving first version of the reading techniques usingthe subjects’ feedback by:
x inserting semantic concerns into the reading techniques toimprove the detection of defects (incorrect fact, ambiguityand extraneous)
x inserting construction concerns into the reading techniques toallow developers argue about design reasoning
11
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO DesignReading Technique for Class diagrams x Class descriptions
…2) Read the class descriptions to find an associated description to each class on the class diagram.Take a look at the class description. Can this description be used to describe the class that you areconsidering at this time? Is it using an adequate abstraction level?F Box the class name in the class description with a blue pen and write in the same number given tothe class on the class diagram.F Mark found classes with a blue symbol (*) on the class diagramMake good use of this time and verify if:•All the attributes are described along with basic typesCan this class encapsulate all these attributes? Are the attributes associated to feasible/possible basictypes? Does it make sense to have these attributes in the class description? Is some attribute missingbetween the two documents or sounding extraneous?F Write down missing and extraneous attributes with a yellow pen•All the behaviors and constraints are describedCan this class encapsulate all these behaviors? Is some behavior missing between the two documentsor sounding extraneous? Are the behavior descriptions using the same abstraction level? Do thebehaviors use the class attributes to accomplish the procedure? Are the constraints reachable using theattributes and behaviors of the class? Do the constraints make sense for this class? Are the constraintsdepending upon some class attributes? Were they described?F Write down missing/extraneous behaviors with a green penF Write down missing/extraneous constraints with a yellow pen
…
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO Design
The Road Ahead:2. Assessment issues:
x Better metrics for assessing inspections effectiveness bylooking for a way to take account “false positives” (i.e. itemsthat are reported by reviewers but which should really not beconsidered defects)
x Developing qualitative as well as quantitative means ofassessment. Qualitative methods would help us understandthe process as well as the results.
12
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO DesignThe Road Ahead
y Observational Study:x we use the term “observational” to define a setting in which an
experimental subject performs some task while theexperimenter gathers data about what exactly the subjectdoes.
y Purpose:x to collect data about how the particular task is accomplishedx observational data
• Time taken• Problems encountered
x Inquisitive data• Influence of prior knowledge
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO DesignLessons Learned from the Observational Studies
y Observational datax Time taken:
• time for applying the techniques is influenced by prior experienceof the reader, complexity of the information, order of reading stepsand levels of abstraction
x Problems encountered:• organizational and semantic issues, format of software artifacts
y Inquisitive datax Influence of prior knowledge:
• domain knowledge seemed to be necessary to support horizontalreading
• basic OO knowledge seemed to be necessary
13
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO DesignThe Road Ahead:
3. Improving defect taxonomyx making use of expert opinion (where experts include skilled
practitioners and researchers) to understand if the taxonomycaptures the important types of defects in OO designs
4. Empirically validating the reading techniquesx doing experiments with specified projects and real developmentx Experiment: http://www.cs.umd.edu/class/fall1999/cmsc735/
5. Packaging and replicating the experiments in other placesx identifying the applicability of such reading techniques in different
development environmentsx Technical Report: ftp://ftp.cs.umd.edu/pub/papers/papers/ncstrl.umcp/CS-
TR-4070/CS-TR-4070.ps.Z
DCS/ESEGOO Reading Techniques
UFRJCOPPE
Reading Techniques for OO Design
Open Questions:
1. What is the role of domain knowledge for these twosets of reading techniques?
Horizontal x Vertical reading
2. What is the adequate level of automated support thatshould be provided for such techniques?
Some steps are repetitive and mechanical to the readerClerical activities need to be identified