SEDS Research Group School of EECS, Washington State University
Annual Reliability & Maintainability SymposiumJanuary 30, 2002
Frederick T. Sheldon and Hye Yeon Kim
Software Engineering for Dependable Systems (SEDS) Research Laboratory
School of Electrical Engineering and Computer Science Washington State University
Validation of Guidance Control Software Requirements Specification for Reliability and Fault-Tolerance
SEDS Research Laboratory School of EECS, Washington State University
Overview Goal: Show the feasibility of this analysis approach
using a industrial strength SRS to ensure:Completeness and ConsistencyFault-tolerance
Specification Under StudyA NASA provided Guidance and Control Software (GCS)
development specification for the Viking Mars Lander. Analysis Approach
Using Zed to specify the data Using Statecharts : Statemate for dynamical analysis
Summary and Future study
SEDS Research Laboratory School of EECS, Washington State University
IntroductionWhy Requirements Specification?
Cost, Time, and Effort
Defect detected phase Typical cost of correctionRequirements Specification $100 - $1,000Coding/Unit Testing $1,000 or moreSystem Testing $7,000 - $8,000Acceptance Testing $1,000 - $100,000After Implementation Up to millions of dollars
SEDS Research Laboratory School of EECS, Washington State University
Reliable SpecificationIs Correct
Complete, consistent and robust
Can the specification be trusted while minimizing the risk of costly errors?
How to analyze the specification to prevent the propagation of errors into the downstream activities?
SEDS Research Laboratory School of EECS, Washington State University
Consistency and CompletenessCompleteness: The lack of ambiguity
Incomplete if …… the system behavior is not specified
precisely because the required behavior for some events or conditions is omitted or is subject to more than one interpretation.
ConsistencyThe Specification is free from conflicting
requirements and undesired nondeterminism.
SEDS Research Laboratory School of EECS, Washington State University
Fault ToleranceFaults
A fault is a feature of a system that precludes it from operating according to its specification
– H. Ammar, B. Cukic, C. Fuhrman, and A. Mili, A comparative Analysis of HW and SW fault tolerance: Impact on software reliability engineering, 1999
Fault ToleranceThe ability to respond to unexpected system
failure (detection and mask/recover)
SEDS Research Laboratory School of EECS, Washington State University
Guidance and Control Software Software Requirements – GCS Dev. Spec.
The system was designed to provide software control of the embedded sensors and actuators of the Viking Mars Lander during the terminal decent (landing) phase.
ARSP (Altimeter Radar Sensor Processing)The ARSP module reads data provided by the
altimeter radar sensor to determine the lander’s altitude from the Mars surface.
SEDS Research Laboratory School of EECS, Washington State University
Mars Lander trajectories
SEDS Research Laboratory School of EECS, Washington State University
Velocity – Altitude Contour
SEDS Research Laboratory School of EECS, Washington State University
EXTERNALRUN_PARAMETERS
SENSOR_OUTPUTGUIDANCE_STATE
TDLRSP.3
GSP.4
ARSP.2
ASP.1
TSP.5
TDSP.6
SEDS Research Laboratory School of EECS, Washington State University
SEDS Research Laboratory School of EECS, Washington State University
Zed OverviewClarifying ambiguitiesIdentify assumptionsCorrectness checkingMathematical proofsGiving an in-depth understanding of the SRS
SEDS Research Laboratory School of EECS, Washington State University
StatechartsVisual formalism: Diagrammatic in natureTestability is provided through symbolic
simulationPredevelopment evaluation through
Fault InjectionStatemate consists of …
Activity chartStatecharts
SEDS Research Laboratory School of EECS, Washington State University
Natural Language based SRS
Inherently ambiguous risking the possibility of multiple interpretations
SEDS Research Laboratory School of EECS, Washington State University
Zed Schema
SEDS Research Laboratory School of EECS, Washington State University
From NL to ZedDiscovered Ambiguities
The confusing definition of variable “Rotation”, and direction of the rotation.
The type assigned to the AR_COUNTER variable was unclear.
An undefined 3rd order polynomial.Where the AR_COUNTER should be
modified?
SEDS Research Laboratory School of EECS, Washington State University
Statecharts Model: Activity chart
SEDS Research Laboratory School of EECS, Washington State University
Statecharts Model: Statechart 1
SEDS Research Laboratory School of EECS, Washington State University
Statecharts Model: Statechart 2
SEDS Research Laboratory School of EECS, Washington State University
Some Theory …Set of Inputs
()
Set of Outputs
Unknowns ()
KnownKnown
SafeUnsafe
AssumedSafe
Sources: Normal Operation Hardware Failures Human Intervention Models/Simulators
SEDS Research Laboratory School of EECS, Washington State University
SEDS Research Laboratory School of EECS, Washington State University
Paradigms …Software Failures:
“Software does not fail - it just does not perform as intended” Professor Nancy Leveson, MIT
Design and test for functionality:Also specify what the system should not do. . .. . . then test it!
SEDS Research Laboratory School of EECS, Washington State University
Some Theory… lets take a second look
Set of Inputs ()
Set of Outputs
Unknowns ()
Known
Known SafeUnsafe
AssumedSafe
Sources: Normal Operation Hardware Failures Human Intervention Models/Simulators
Fault Injection(added known)
SEDS Research Laboratory School of EECS, Washington State University
Testing and Fault InjectionBy using symbolic simulation in
StatemateBoundary Testing
Input that is inside of the variable rangeInput that is outside of the variable range
Fault InjectionState variable alternationState transition redirection
SEDS Research Laboratory School of EECS, Washington State University
Testing Results ARSP Specification Test Input and Output
Variable Case 1 Case 2 Case 3 Case 4 Case 5
Input
FRAME_COUNTER 2 2 1 1 3
AR_STATUS - - [0, 0, 0, 0] - [0, 1, 0, 0]
AR_COUNTER -1 19900 -1 20000 -1
Output
AR_STATUS KP KP [1, 0, 0, 0] [0, -, -, -] [1, 0, 1, 0]
K_ALT KP KP [1, 1, 1, 1] [1, -, -, -] [0, 1, -, 1]
AR_ALTITUDE KP KP [*, -, -, -] [2000,-,-,-] KP
SEDS Research Laboratory School of EECS, Washington State University
Detailed Testing Results - Case 1
Initial valuesFinal values Initial variable
values are being calculated based on the given equations.
Variable Case 1
Input
FRAME_COUNTER 2
AR_STATUS -
AR_COUNTER -1
Output
AR_STATUS [1, 0, 0, 0]
K_ALT [1, 1, 1, 1]
AR_ALTITUDE [2000, -, -, -]
[1, 1, 0, 0]
[1, 1, 1, 1]
[2000, 2000, -, -]
SEDS Research Laboratory School of EECS, Washington State University
Fault Injection ResultsAltered state variable
FRAME_COUNTER AR_COUNTER AR_STATUS Fault injected State Case
1 Case
2 Case
3 Case
4 Case
5 Case
1 Case
2 Case
3 Case
4 Case
5 Case
1 Case
2 Case
3 Case
4 Case
5
CURRENT_STATE x x x x x x x x x x x x x x x
KEEP_PREVIOUS_VALUE b b b b b b b b b b b b b b b CALCULATION b b b b b b b x x x b b x b x
ODD b b b b b b b x x x b b x b x ESTIMATE_ALTITUDE b b b b b b b N/A b b b b N/A b b
CALCULATE_ALTITUDE b b b b b b b b x b b b b b b KEEP_PREVIOUS b b b b b b b b b b b b b b b
DONE b b b b b b b b b b b b b b b x incorrect outputs, b no defect
SEDS Research Laboratory School of EECS, Washington State University
Detailed Fault Injection Results
Change FRAME_COUNTER at CURRENT_STATE from 2 to 3
Variable Case 1
Input
FRAME_COUNTER 2
AR_STATUS -
AR_COUNTER -1
Output
AR_STATUS [1, 0, 0, 0]
K_ALT [1, 1, 1, 1]
AR_ALTITUDE [2000, -, -, -]
[1/0, 1, 0, 0]
[1, 1, 1, 1]
[*, 2000, -, -]
SEDS Research Laboratory School of EECS, Washington State University
SummaryInterpretation from NL to Zed
Clarifying ambiguitiesInterpretation from Zed to Statecharts
Clarifying misinterpreted Zed specificationIterative process
Boundary Testing, Fault Injection analysisReveals weak point(s) in the systemFault Tolerance validation
SEDS Research Laboratory School of EECS, Washington State University
ConclusionUsing this combination of FMs we could:
Clarify ambiguitiesValidate Correctness, Completeness, and ConsistencyValidate Fault tolerance features of the SRS
This approach enabled us to:Avoid the problems that result when incorrectly
specified artifacts force corrective rework.Minimize the risk of costly errors being propagated into
downstream activities
SEDS Research Laboratory School of EECS, Washington State University
Future StudyBuild concrete translation rules between the
methods
Find an effective algorithm to automate the process
Validate the algorithm for the different (domain/ application specific) critical software requirements
In depth comparative study with other approaches