www.unamur.be Towards Statistical Prioritization for Software Product Lines Testing Xavier Devroey * ; Gilles Perrouin ; Maxime Cordy ; Pierre-Yves Schobbens ; Axel Legay ; Patrick Heymans 8th International Workshop on Variability Modelling of Software- intensive Systems, VaMoS ’14 Nice, France
35
Embed
Towards Statistical Prioritization for Software Product Lines Testing (VaMos '14)
Paper may be downloaded at https://pure.fundp.ac.be/ws/files/7911785/VAMOS2014_FTS_statistical_prioritization.pdf
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
www.unamur.be
Towards Statistical Prioritization for Software Product Lines Testing
Xavier Devroey * ; Gilles Perrouin ; Maxime Cordy ; Pierre-Yves Schobbens ; Axel Legay ; Patrick Heymans
8th International Workshop on Variability Modelling of Software-intensive Systems, VaMoS ’14
• Usage Model (DTMC)– 96 states and 2149 transitions– 2 hours computation
• Ubuntu Linux (Intel Core i3, 3.10 GHz, 4GB mem.)
• Feature Diagram (FD)– Built manually by inspecting a Claroline local instance– 44 features
• Lots of optional features
• Featured Transition System (FTS)– Web crawler on local instance to get the pages– (1 page 1 state) + initial state– Every state accessible from anywhere– Transitions tagged with feature expressions based on the knowledge of the
system– 107 states and 11236 transitions
Claroline: Setup and Results
Run 1 Run 2 Run 3 Run 4
Lmax 98 98 98 98
Pr min 1E-4 1E-5 1E-6 1E-7
Pr max 1 1 1 1
#DTMC traces 211 1389 9287 62112
#Valid traces 211 1389 9287 62112
Traces avg. size 4.82 5.51 6.35 7.17
Traces avg. proba 2.06E-3 3.36E-4 5.26E-5 8.10E-6
#Pruned FTS states 16 36 50 69
#Pruned FTS transitions
66 224 442 844
Claroline: Discussion
• Observation: – Even with a “simple” algorithm, computation time is reasonable– Independence of the features and low size of valid traces
• #products associated to each trace too important
• Generate longer traces by coupling probabilistic approach with state/transitions coverage criteria
• Select minimal features set needed to execute a trace– Use knowledge of the application domain– Select features according to their frequency in the feature
expressions of valid traces
Claroline: Discussion
• Multiple usage models: one per role (i.e., student, teacher, admin, visitor)
• Use other selection criteria on the usage model– Least/Most probable traces
• Main threat: Web nature of the considered application
CONCLUSION…
… and Future Works
Conclusion
• Contribution: – A first approach prioritazing behaviours statistically for testing
SPLs in a family-based manner
• Future works:– Improve exploration algorithm in order to support other
“statistical selection” criteria on the usage model• Least/Most probable behaviours
– Combine structural selection criteria with statistical testing in an SPL context• State coverage, transition coverage, transition pairs coverage, path
coverage, etc.
THANK YOU FOR YOUR ATTENTION !
Models and tools available on http://info.fundp.ac.be/~xde/fts-testing/