Automatic Test Case Generation for Train Control Systems Adnan Causevic TOCSYC Industrial Workshop, 2014-11-13, Västerås
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Automatic Test Case Generation for Train Control Systems Adnan Causevic
TOCSYC Industrial Workshop, 2014-11-13, Västerås
2
Today’s talk is about…
● Software Testing Laboratory @ MDH
● Research: ● Test Automation ● Quality of Tests ● Developers Testing
● AGENTS - KKS funded project ● Bombardier Transportation ● Maximatecc
● Involved in the TOCSYC research environment
● Intelligent Embedded Systems Master Program Coordinator
3
About me
● ATAC research project ● 2011-2014 ● Funded by ITEA2/Vinnova ● Advanced Test Automation for Complex Software-
Intensive System
4
Cooperation with Bombardier
“The project has brought test automation on the agenda of European industry.”
- ITEA3 Final Review Report
● Train Control Management System (TCMS) ● a high capacity, infrastructure backbone ● the center of the distributed system
● Functions controlled by TCMS include ● collecting line voltage, ● controlling the train engines, ● opening and closing the train doors, ● upload of diagnostic data.
5
About Bombardier
● Vehicle ● Vehicle Integration testing
● System ● System Integration testing
● Software ● Modules integration testing ● Modules testing
● “Get to the driver’s perspective as soon as possible”
6
About Bombardier … and their testing
Responsibility of a Developer!
● IEC 61131-3 ● Function Block Diagram ● Graphical programming language ● Based on element composition ● Various predefined elements
7
Development Environment
● Coverage ● Strict measurements mandated by safety standards
● Tooling ● No tool support to measure coverage on FBD
● Time ● Creating tests just to reach the coverage leaves very little
time for functional testing
8
Current challenges … with module testing
9
CompleteTest
10
Research approach FBD Program
Transformation 1 Annotation2UPPAAL timed
automata
...
Reachability properties
∃ ♢ β ,∃ ♢ pi ,
∃ ♢ c0 and c1
Test traces Test cases(Step) (1)(Time) (20s)(Inputs) (0 1 23)(Outputs) (1 2 543,9)
UPPAAL model checker
Test Generation3
Logic-‐‑based coverage criteria
DC, CC, MC/DC
”Automated Test Generation using Model-Checking: An Industrial Evaluation”, Eduard Paul Enoiu, Adnan Causevic, Thomas J. Ostrand, Elaine J. Weyuker, Daniel Sundmark, Paul Pettersson. International Journal on Software Tools for Technology Transfer, 2014, Springer.
● Time required to generate tests ● satisfying the DC, CC and MC/DC logic coverage criteria
11
Case Study at BT
● For 34 of the 157 programs, the tool did not terminate after running for a substantial period of time.
● Cut-off time was set to 10 minutes.
● There is an indication that as the number of decisions increases, the performance deteriorates and the cost of using the tool may become prohibitive.
● But, if you have so many decisions, how would you test it manually?
12
Case Study at BT (2)
● Should a developer generate the tests from scratch? ● Should a developer provide manual tests first? ● How many (or how much in terms of coverage)?
● When should a developer use the Validate Test button?
● Should developers use this tool at all or should we automate that as well (nightly build, etc.) ?
13
Best Practices?
● Internal behavior of each function block has to be modeled ● Currently, blocks from the standard language are modeled ● and a few custom made by Bombardier which are often used
● State-space explosion ● a known problem of model-checking ● large number of Boolean or Integer input values ● but, again, how would you do it manually ?
14
Limitations
● Complementing model-checking with: ● Static analysis of FBD’s ● Search-based software testing
● Trying some new approaches: ● Different model-checkers, SMT solvers, …
● Measuring effectiveness of generated tests ● In terms of how good they are in finding faults
● Experimenting with FBD programs from
other organizations
15
On-going and Future work
● Adnan Causevic [email protected]
● Eduard Enoiu [email protected]
● www.completetest.org
16
Questions?
Related Documents
