1 IV&V Facility FY2002 Initiative: Software Architecture Metrics Hany Ammar, Mark Shereshevsky, Nicholay Gradetsky, Diaa Eldin Nassar, Walid AbdelMoez, J. Shriver, A. Jalan LANE Department of Computer Science and Electrical Engineering West Virginia University Ali Mili, Bo Yu, Yan Wang, Krupa Doshi College of Computing Science New Jersey Institute of Technology The OSMA Software Assurance Symposium Berkley Springs, WV, September, 2002 The more information I give The more information I give the more errors I may pass– the more errors I may pass– Couched by Tim Couched by Tim WVU UI: Software Architecture WVU UI: Software Architecture Metrics Metrics
FY2002 Initiative: Software Architecture Metrics. The more information I give the more errors I may pass– Couched by Tim WVU UI: Software Architecture Metrics. Hany Ammar, Mark Shereshevsky, Nicholay Gradetsky, Diaa Eldin Nassar, Walid AbdelMoez, J. Shriver, A. Jalan - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
IV&V Facility
FY2002 Initiative: Software Architecture Metrics
Hany Ammar, Mark Shereshevsky,Nicholay Gradetsky, Diaa Eldin Nassar, Walid AbdelMoez, J. Shriver, A. Jalan
LANE Department of Computer Science and Electrical EngineeringWest Virginia University
Ali Mili, Bo Yu, Yan Wang, Krupa DoshiCollege of Computing Science
New Jersey Institute of TechnologyThe OSMA Software Assurance Symposium
Berkley Springs, WV, September, 2002
The more information I give The more information I give
the more errors I may pass–the more errors I may pass– Couched by TimCouched by Tim
• What we can do• Benefits• Project Overview • Research Objectives• Quantitative Factors • Error Propagation• Change Propagation• Future Work
3
IV&V Facility
Core ideaThe amount of information Transferred
• If component A talks to components B– It is important how much information can be transferred– In Information theoretic terms this is measured by the entropy of data and
control information• Standard view
– Call graphs – control-flow and data-flow information between components– “This talks to that”
• What is the amount of information transferred?– example1:
• Binary information- entropy is 1-bit– example2:
• 1000 possible commands – entropy is 10-bits
• Intuition:– Error propagation in example2 >
Error propagation in example1
4
IV&V Facility
What We Can Do Develop error propagation matrix (details follow)
• Use metrics to identify and focus on problem areas in the architecture
• There is a need to analyze the quality of reference architectures in product-line architectural development (NASA Reference Architecture – the SEEDS project at GFSC, Workshop was held in June 17 - 19, 2002 in San Diego, CA) http://lennier.gsfc.nasa.gov/seeds/
• The developments of techniques for measuring Error/Change Propagation help analysts identify trouble spots in the architecture (advances the state of the art)
• The development of automated tools help the analyst apply these techniques on large architectures (advances the state of the practice)
6
IV&V Facility
Project Overview
FY01• Defined quantitative factors that are relevant to qualitative
attributes of the architecture – Error Propagation – Change Propagation– Requirements Propagation
• Defined information theoretic (Entropy-based) architecture metrics
7
IV&V Facility
Project Overview (cont.)
FY02
• Establish analytical/and empirical relationships between the Entropy-based metrics and Error/Change propagation factors
• Automate computation of the metrics, and apply to a NASA case study, – Architecture Metrics Tool to be presented in the tools
Controlled Experiments Empirical Validation of Error Propagation Analysis
ExtractModel Info.
.rtmdl DesignExperiments
CorruptModel
Simulateand Log
Normal Log
Error Logs
AnalyzeLogs
Empirical Error Propagation
Components,Connectors
Connector,Message
MATLAB
SummitBASIC
FaultModel
16
IV&V Facility
Error PropagationControlled Experiments Results
The Analytical/Empircal Error Propagation
0.0000
0.2000
0.4000
0.6000
0.8000
1.0000
1.2000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Connetors
EP
Analytical
Empirical
Fault Injection
0
200
400
600
800
1000
1200
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Connector
# Fa
ult I
njec
tion
Correlation results
-1.5000
-1.0000
-0.5000
0.0000
0.5000
1.0000
1.5000
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Connector
Corr
elat
ion
Coef
ficie
nt
•The following correlation results are based on 3562 experiments
•The correlation goes down as the number of injected faults in the experimental side goes down
•The over all correlation coefficient between the two matrices = 0.8335.
17
IV&V Facility
Change Propagation
We define Change Propagation from component A to component B as the probability that a change in A due to corrective/ perfective maintenance requires a change in B to maintain the overall function of the system.
We propose an analytical formula that approximate change propagation, and present conducted empirical study to measure change propagation.
18
IV&V Facility
Change PropagationAnalytical study
1||12||),( ||
)(DIR||||
A
BA
IA
BIIA
OOBACP
•Where |IX| and |OX| stand for the number of inputs and outputs, respectively of component X.•DIR(Deterministic injectivity rate) of X is defined as
|)(|||log
|||)(| DIR(X) 1
1
vXI
IvX X
Ov XX
An upper bound on Change Propagation FactorCP between two components A and B
19
IV&V Facility
Change PropagationEmpirical Study
• In our experiment, we randomly select a large amount of changes within each component, then go into the source code level to see, if the change will propagate to other components.
• By computing for each pair of components (A,B), the number for which change in A cause a change in B, we are able to derive the change propagation matrix.