1 Model Based Conformance Testing for Extensible Internet Protocols Anastasia Tugaenko Scientific Adviser: Nikolay Pakulin, PhD
Dec 30, 2015
1
Model Based Conformance Testing for Extensible Internet Protocols
Anastasia Tugaenko
Scientific Adviser: Nikolay Pakulin, PhD
2/19
Internet Protocols Features
Internet protocols are underspecified
Internet protocols are nondeterministic
Internet protocols requirements differ in the level of obligations (MUST, SHOULD, MAY, …)
Protocol’s architecture is extensible
3/19
Architecture of Extensible Protocols
Protocol Functions
Basic
• are mandatory for all protocol implementations;
• specifies the basic protocol functionality, functions that are the basis for the protocol
Extensions• are optional, may be unsupported by implementations;
• add new functions without changing the basic protocol’s model;
• alter format of the protocol’s messages;
• alter basic protocol model, alter functionality of other extensions;
• alter protocol’s modus operandi
4/19
Conformance Testing
Many implementations of protocols from different developers are functioning in the contemporary Internet
The total reliability of data transfer substantially
depends on implementations compatibility
The basic method of attesting implementations compatibility is conformance testing (testing to conform to the standard)
Protocol’s extensions may alter protocol’s standards
5/19
Traditional Methodology of Conformance Testing
Test suite consists of formal given tests which are not connected to the implementation
Test purposes describe situations to be tested. Test purpose is realized in one or several tests
Connections between requirements and test purposes and between test purposes and their tests are informal
Implementation is considered to conform to the standard if all test purposes have been passed
6/19
A Problem with Test Suites Extensions
Basic Standard+ few extensions Test
Basic Standard+ few extensions
+ one more extension
How can we get a new test
?
assume, that we already got it
• manually change the test suite• constructing new test from a set of test units
7/19
A Problem with Adapting Test Suites to Implementations
IUT1• extension 1• extension 2
IUT2• extension 1• extension 3
IUT3• extension 4
How can we compose tests for each implementation?
Manually adapt the tests
Set a number of options in the configuration fileif (extension1) {…} else {… }
Use specification and test units and automatically compose test suites for target set of extensions from that units
8/19
Requirements for Test Suite for Extensible Protocols Testing
For basic standards: formal relation between requirements and tests automated verdict assignment about the correctness of IUT behavior automated tests generation depending on IUT responses
For extensions: easily changing of a protocol model the ability to combine specifications and tests from separate units
9/19
A Method for Extensible Protocols Testing
Constructing specification from units Specification for implementation is a collection of specifications units (basic specification, extensions specifications).
Developing tests as units The comprehensive test is a composition of units of tests for basic functionality and supported implementations.
Automated construction of model and tests for an arbitrary set of extensions
10/19
A Method for Extensible Protocols Testing. Features
Each specification unit do check the postcondition change the test state assign a verdict concerning the correctness of IUT behavior
Test sequence iterator test sequence generates automatically from test model and contract specifications
Automated coverage calculation states and transitions may be labeled with marks and branches, in this case they will be represented in the test report.
Easily changing of a protocol model
11/19
Case Study. SMTP Protocol
SMTP – Simple Mail Transfer Protocol
SMTP protocol has 7 states:
DISCONNECTED
CONNECTED
AFTER HELLO
AFTER MAIL FROM
AFTER RCPT TO
AFTER DOT
AFTER DATA
Transaction commands are allowed only from these states
12/19
Case Study. Changing SMTP Protocol Model
If AUTH extension is supported SMTP protocol has 8 states:
DISCONNECTED
CONNECTED
AFTER HELLO
AFTER AUTH
AFTER MAIL FROM
AFTER RCPT TO
AFTER DOT
AFTER DATA
Several commands are allowed only after AUTH command was successfully handled
13/19
A Method for Extensible Protocols Testing. Protocol Model
Model is an extended finite state machine
command addition
state addition
a с t i o n sstates
14/19
A Method for Extensible Protocols Testing. Protocol Model Case Study
Define 2 mappings <States → Actions>:
allowed
denied
Got 2 types of testing:
conformance (unknown transitions are considered as denied)
robustness (unknown transitions are considered as allowed)
15/19
A Method for Extensible Protocols Testing. Main Steps
Creation of requirements catalogues Designing of extensible model for the protocol Developing units with specifications and units with tests Formalization of requirements Developing test scenarios Constructing a comprehensive test for target implementation, executing it and analyzing the results
16/19
Test Suite for SMTP Protocol
Basic Specification model ~1250 strings test scenario ~250 strings
DSN extension (Delivery Status Notifications)Adds new functionality
Brings in new parameters into existing commands MAIL and RCPT, doesn’t change the protocol model.
model ~100 strings test scenario ~ 50 strings
AUTH Extension (Authentication) Alter basic functionality
Adds new state into the protocol model, adds new AUTH command or parameter for existing command MAIL, brings in new response codes.
model ~250 strings test scenario ~120 strings
17/19
Future Work
Extend the method to develop tests for other types of protocols extensions
Dynamically construct the model of target implementation taking into account the profile of supported extensions
Extend test traces and improve test report generator
18/19
Conclusion
The method for model based conformance test suite development for extensible protocols is presented
method is model based uses unit approach for modeling and specifying tests for protocols extensions uses pure Java without any extensions
The prototype of test suite for SMTP protocol with extensions was developed using presented method. The developed test suite allows to find the following types of incompliances:
protocol rules violation absence of supporting of required commands wrong reply codes to protocol commands
19/19
Thank You!