Using Instrumentation for Quality Assessment of Resilient Software in Embedded Systems Armin Wellig 2 David Lawrence 1 Didier Buchs 1 1 University of Geneva 2 Honeywell International Sarl 6th International Workshop on Software Engineering for Resilient Systems 16/10/2014 1
35
Embed
SERENE 2014 Workshop: Paper "Using Instrumentation for Quality Assessment of Resilient Software in Embedded Systems"
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Using Instrumentation for Quality Assessment of Resilient Software in Embedded Systems
Armin Wellig2 David Lawrence1 Didier Buchs1
1 University of Geneva
2 Honeywell International Sarl
6th International Workshop on Software
Engineering for Resilient Systems
16/10/2014
1
System
… t += 1 print()
Introduction
2
• Correct internal state OR false positive?
• What about resilient system correctness?
• Ideas/Solutions => Improve observability
• System correctness
System observation
• Automaton specification
o T(spec) are words ∈ (Σin x Σout)*
• Satisfaction relation
o For any program p based on a given specification spec
3
Limited observability
• What if the developer made a fault with the
counter modulo?
4
Improve observability
• Adding new observation points obs to a program p
• For any program p, obs are correct observers iff
5
Observability: example
• Obs1: each increment of cnt
• Obs2: when “cnt mod 5 = 0”
6
Observers expressiveness
7
• For a given program p, the observers obs
expressiveness can be defined as follow
Metric: distance
8
• A distance between instrumented program traces
and specification traces can be stated
• What about the observers relevance?
Metric: quality/relevance
9
H0
…
Hi
Hj
H
…
T0
…
…
T
Tj
Ti
H0
…
Hi
Hj
H
…
T0’
…
…
T’
Tj’
Ti’
P P + obs
T >> T’
Instrumentation process
10
Spec. Model Develop
Compare
Program Traces
Impl.
Embed
Obs.
Observe
Instr. Impl.
Extract
Specification Traces
Instrumentation: example void Thermostat (int* t) { if (t > 21) { *t += -0.1 * t; // Increase temperature } else if (t < 19) { t += 5 - 0.1 * t; // Decrease temperature } }
11
void Thermostat (int t) {
if (t > 21) {
t += -0.1 * t; // Increase temperature
} else if (t < 19) {
t += 5 – 0.1 * t; // Decrease temperature
}
return t;
}
t = 21 t=18.9 t=21.29 t=19.161
Instrumentation: example
12
t = 21 t=18.9 t < 19 t=21.29 t=19.161
void Thermostat (int t) {
if (t > 21) {
t += -0.1 * t; // Increase temperature
} else if_instr (t < 19) {
t += 5 – 0.1 * t; // Decrease temperature
}
return t;
}
Instrumentation: example
13
void Thermostat (int t) {
if_instr (t > 21) {
t += -0.1 * t; // Increase temperature
} else if_instr (t < 19) {
t += 5 – 0.1 * t; // Decrease temperature
}
return t;
}
t = 21 t=18.9 t < 19 t=21.29 t > 21 t=19.161
Resilient system
14
Compensation
Normal behavior
Degradation
Instrumenting resilient systems
• Resilient system designed with defined FSMs
• System crashes considered as degradation
• Instrumentation requirements
• LTL used to discuss properties
15
Instrumenting resilient systems
• Extended system
o Fsys’’ = Fsys + feedback + recovery + obssys
• Compensation
o Fcomp’ = Fcomp + obscomp
• Degradation
o Fdeg’ = Fdeg + obsdeg
16
System with
feedback and
recovery
System with
feedback
High level picture
17
Degradation Compensation
Environment
System
feedback
recovery
crash
Preparing the system
• Preliminary preparation
o Add atomic propositions to distinguish states
18
s1
s2
ν(s2) = 2
ν(s1) = 1
Extending the system
• 1st extension Fsys’ (Feedback)
1) Add crash state
2) Add feedback state
3) Add arcs from every state to crash state
• crash as arc inscription
• Unique state ID stored in crashed_state variable
4) Add arc between crash and feedback
• feedback as arc inscription
5) Add atomic propositions
19
Extending the system
20
crashed_state := 1
crashed_state := 2
feedback
s1
ν(s1) = 1
s2
ν(s2) = 2
crash
ν(crash) = syscrash
feedback
ν(feedback) = feedback
Extending the system
• 2nd extension Fsys’’ (Recovery handling)
1. Add arcs from feedback state to every system state
• recovery as arc inscription
2. Guards
• Avoid non determinism
• Recovery to last consistent state using crashed_state
21
Extending the system
22
[crashed_state = 2]
[crashed_state = 1]
recovery
recovery
…
…
…
s1
ν(s1) = 1
s2
ν(s2) = 2
crash
ν(crash) = syscrash
feedback
ν(feedback) = feedback
Compensation and degradation
23
crash feedback
recovery recovery
Compensation Degradation
idle detection normal inject
ν(idle) = …
ν(detection) = …
ν(normal) = …
ν(inject) = …
System composition
• Complete resilient system
o Fres = Fsys’’ || Fcomp’ || Fdeg’
• Composition
1. Add arc looping on each state with εas inscription
• Resilient system translated as a transition system
• Strategies/rewriting rules defined independently
for each components
• State space computed
Conclusion
• Theoretical basis on instrumentation
• Insights on resilient systems instrumentation
• Methodology to extend a system with resilience
o Even though the model is simple
• Temporal properties enunciated
o Mechanisms and overall resiliency
32
Future works
• Level resiliency
o More complete/complex model [Trivedi et al. 2009]
• Model checking
o LTL with StrataGEM when available
o Other model checkers
• Tests generation
o Model based tests generation [Fraser et al. 2009]
o Timing insights [Braberman et al. 1997]
33
Questions ?
34
Thank you
References
o Edmundo López Bóbeda, Maximilien Colange, and Didier Buchs. Stratagem: A generic petri net verification framework. In Gianfranco Ciardo and Ekkart Kindler, editors, Petri Nets, volume 8489 of Lecture Notes in Computer Science, pages 364– 373. Springer, 2014.
o Kishor S. Trivedi, Dong Seong Kim, and Rahul Ghosh. 2009. Resilience in computer systems and networks. In Proceedings of the 2009 International Conference on Computer-Aided Design (ICCAD '09). ACM, New York, NY, USA, 74-77.
o Fraser, G., Wotawa, F., & Ammann, P. E. (2009). Testing with model checkers: a survey. Software Testing, Verification and Reliability, 19(3), 215-261.
o Victor Braberman, Miguel Felder, and Martina Marr. Testing timing behavior of real-time software. In International Software Quality Week, 1997.