1 Experiences in Automating the Testing of SS7 Signaling Transfer Points ISSTA 2002 July 22-24, 2002 Via di Ripetta, Rome - Italy Tim Moors, Malathi Veeraraghavan, Zhifeng Tao, Xuan Zheng, Ramesh Badri Polytechnic University Brooklyn, NY, USA [email protected]http://kunene.poly.edu/~mv This project is co-sponsored by Verizon, NYSTAR and CATT.
22
Embed
1 Experiences in Automating the Testing of SS7 Signaling Transfer Points ISSTA 2002 July 22-24, 2002Via di Ripetta, Rome - Italy Tim Moors, Malathi Veeraraghavan,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Experiences in Automating the Testing of SS7 Signaling Transfer Points
ISSTA 2002 July 22-24, 2002 Via di Ripetta, Rome - Italy
Tim Moors, Malathi Veeraraghavan, Zhifeng Tao, Xuan Zheng, Ramesh Badri
Abstract expected events form a tree of causation.
If (2) {
if (4) {
foreach $t (@set_1) {/*5*/
6$t
}
}
if (9) {
10
}
}Representing the abstract event tree on the left as a series of nested if statements.
16
Outline
• Background
• Problem statement
• Solution approach
• Implementation — Automated SS7 Test Results Analyzer (ASTRA)
• Summary
17
Architecture of ASTRA
Network Configuration Files
TimingFiles
Test Data
Data Formatting
Test Action Record
Actual Event Record
Analyzer
ParameterObservation
List
Matched Event
Record
Event Lists: Hidden, Missing,
Unexpected
ExpectedBehavior
Translator
• Translate EBEL code into a PERL program• Add time values (consult TAR)• Convert to depth-first search
• Adjust variations in format from different test data
• Synchronize all events
• Filter database to remove irrelevant information
• Sort events in the database into chronological order
DataProgram
18
Analyzer - matching operation
NO
Write to Hidden
Event List
Event Found?NO Write to
Missing Event List
Find all messages with Match Flag = 0
Write to Unexpected
Event List
Was the link monitored?
Find Event on link, params, [t1,t2] from EB
Actual Event Record
Search for event in the time period
YES
Matched Event
Record
YESSet Match Flag to 1
20
ExpectedBehavior
Analyzer
Actual Event Record
Architecture of ASTRA
Translator
ParameterObservation
List
Matched Event
Record
Event Lists: Hidden, Missing,
Unexpected
Test Results Database
Statistical Analysis &Report Generation
Legend
Data
Program
Network Configuration Files
TimingFiles
Test Data
Data Formatting
Test Action Record
•Calculate the numbers of expected, missing, hidden and unexpected events
• Calculate timer observations
• Calculate load measurements
• Declare failure/pass/inconclusiveness of the tests
21
Outline
• Background
• Problem statement
• Solution approach
• Implementation — Automated SS7 Test Results Analyzer (ASTRA)
• Summary
22
ASTRA
Productivity of ASTRA• The size of the EB code 5909• The number of events in collected test results 29196
EB code 6,000 lines
Test Data 30,000 events
Produced database 3,000 events
23
Summary• Challenging problem
• Non-determinism (e.g., simultaneous link restoral)– If A restores before B, a different type of message will be
sent than if B restores before A; if both options not listed with an “OR” in EB, then either “unexpected” or “missing” event
– Solutions (?)• Cover all possible scenarios by automatically creating EB directly from
SDL specification - state explosion problem• Allow EB program to consult the test results to make further predictions
with “IF” statement. This approach trusts test results; Also found need for a “NOT” statement.
• Asides:– Pay attention to monitors: “more the better” and they miss messages! – Need automation of all steps of testing – not just results analysis– Found 3 implementation bugs in STPs tested!