Ginger2: An Environment for Computer- Aided Empirical Software Engineering Ginger2: An Environment for Computer- Aided Empirical Software Engineering Koji Torii, Ken-ichi Masumoto, Kumiyo Nakakoji, Yoshihiro Takada, Shingo Takada, Kazuyuki Shima IEEE TRANSACTIONS ON SOFTWARE ENGINEERING 1999 2006 2006 - - 08 08 - - 16 16 Kim, Su Kim, Su - - hyun hyun
30
Embed
Ginger2: An Environment for Computer- Aided Empirical ...€¦ · Aided Empirical Software Engineering Ginger2: An Environment for Computer-Aided Empirical Software Engineering Koji
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Ginger2: An Environment for Computer-Aided Empirical Software Engineering
Ginger2: An Environment for Computer-Aided Empirical Software Engineering
IntroductionBackgroundCAESE FrameworkGINGER2 System- Data Collection- Data & Control Integration- Data Analysis
Case StudyConclusion & Future WorkDiscussion
Software Engineering Lab, KAIST 3/22
Empirical software engineering
The study of software related artifacts for the purpose of characterization, understanding, evaluation, prediction, control, management, or improvement through qualitative or quantitative analysis [Harrison and Basili]
A series of actions to obtain knowledge and better understanding about some aspects of software development given a set of problem statements in the form of issues, questions or hypotheses.[Koji Torii et al.]
Introduction (1/2)Introduction (1/2)
The study of software related artifactsfor the purpose of characterization, understanding, evaluation, prediction, control, management, or improvementthrough qualitative or quantitative analysis [Harrison and Basili]
In vivo (naturalistic)
A series of actions to obtain knowledge and better understanding about some aspects of software development given a set of problem statements in the form of issues, questions or hypotheses.[Koji Torii et al.]
In vitro (laboratory)
Actual software development project Consist of controlled Experiments
* These two approaches complement each other
Relevance of real, ill-defined problems that emerge in actual S/W development
Develop and test hypothesis about particular aspect of human behavior
Develop generalized model of behaviorProposed techniques and methods
Make “knowledge” available and reusable for other projects and members
Software Engineering Lab, KAIST 4/22
Computer-Aided Empirical Software Engineering (CAESE)- Supporting in vitro study
Involving in empirical activitiesDealing with many types of data
- FeatureEmpirical study process modelData collection modelEmpirical study tools
Introduction (2/2)Introduction (2/2)
Software Engineering Lab, KAIST 5/22
Background (1/2)Background (1/2)
Architecture of GINGER [IST’96]GINGER automatically collects software process and product data, and performs analysis to provide feedback to developers
Software Engineering Lab, KAIST 6/22
Background (2/2)Background (2/2)
History of GINGER [IST’96] (since 1985)PerformanceQuality Software artifacts
Programmer Performance Model
Programmer Performance Model
Evaluation ofProgrammer Performance
Evaluation ofProgrammer Performance
Evaluation ofTeam Performance
Evaluation ofTeam PerformanceEvaluation of Quality
(automated testing)Evaluation of Quality(automated testing)
Evaluation ofData Flow Diagram(from CASE tool)
Evaluation ofData Flow Diagram(from CASE tool)Evaluation of
Programmer debugging Performance
Evaluation of Programmer debugging
Performance
Off-line feedback(size, change)
Off-line feedback(size, change)
On-line feedback(size, change)
On-line feedback(size, change)
Estimation residual faults based on change
Estimation residual faults based on change
Estimation residual faults based on petrinet model
Estimation residual faults based on petrinet model Evaluation of process
Detailed specifications for the experiment. Experimental tools, types of subjects, analysis methods and tools
Experimenters analyze data collected
Experiments are monitored and data are collected
Experimenters compile “knowledge” into a “package” available as an output from the study.
Software Engineering Lab, KAIST 9/22
CAESE Framework (3/4)CAESE Framework (3/4)
Data collection model- Three entities & Five viewpoints
Same collection tool for different aspect of dataData analysis needs to be conducted on the five types of data
SubjectSoftwareArtifact
DevelopmentTool
States ofsubject
Input from subjects to software engineering tools
Output from software engineering tools to subjects
States of software engineering tools
States of software engineering artifact repositories
Software Engineering Lab, KAIST 10/22
Empirical study toolCAESE
CAESE Framework (4/4)CAESE Framework (4/4)
NeedsAnalysis
Tool
NeedsAnalysis
Tool
ExperimentDesignTool
ExperimentDesignTool
Data CollectionTools
Data CollectionTools
Data AnalysisTools
Data AnalysisTools
PackagingTools
PackagingTools
Data Integration ToolsData Integration Tools
Control Integration ToolsControl Integration Tools
Empirical Study Management ToolsEmpirical Study Management Tools
Data CollectionTools
Data CollectionTools
Data AnalysisTools
Data AnalysisTools
Data Integration ToolsData Integration Tools
Control Integration ToolsControl Integration Tools
Managing the empirical study itself(Same way as project management tools in CASE)
Empirical Study Management ToolsSynchronizing the use of different toolsControl Integration ToolsIntegrating the various types of dataData Integration Tools
Software Engineering Lab, KAIST 11/22
Data Collection (1/2)Data Collection (1/2)
Considerations- Developers are engaged in many types of activities
simultaneously- CAESE should support continuous automatic
collection of dataFine-grained dataThe load on the subject is lessenedNecessary data will not be left out by mistakeThe chances of mis-informing would be minimized