Page 1
imbus AG
Kleinseebacher Strasse 9D-91096 MoehrendorfGermany
Phone: +49 (0)9131 75180 Fax: +49 (0)9131 751850
Email: [email protected] : www.imbus.de
© imbus AG 2001
Automated Generation and Execution of Test Suites
Bernd Matternimbus AG
Kleinseebacher Strasse 991096 Moehrendorf
The
Project
Page 2
© imbus AG, Bernd Mattern The AGEDIS Project 2
AGEDIS
What is AGEDIS?
A utomated
G eneration and
E xecution of Test Suites for
Dis tributed Component-based Software
Information Society Technologies (IST) Programme
Total budget: 4,3 M EUR EC pays: 2,3 M EUR
Page 3
© imbus AG, Bernd Mattern The AGEDIS Project 3
AGEDIS
Agenda
1. Motivation
2. Goals of AGEDIS
3. Overview of AGEDIS
4. AGEDIS Modelling Language
5. Other Approaches
6. Summary and Outlook
Page 4
© imbus AG, Bernd Mattern The AGEDIS Project 4
AGEDIS
1. Motivation
Design Specification
Review
FunctionalSpecification
Test Specification
Page 5
© imbus AG, Bernd Mattern The AGEDIS Project 5
AGEDIS
1. Motivation
Design Specification
Review
FunctionalSpecification
Test Specification
AGEDIS Model
auto
mat
icde
rivat
ion
Page 6
© imbus AG, Bernd Mattern The AGEDIS Project 6
AGEDIS
2. Goals of AGEDIS
Creation of a new testing methodology
Cheaper software development costs
Continuous approach to testing from modelling the software to the evaluation of the test run results
Adoption of standards
Page 7
© imbus AG, Bernd Mattern The AGEDIS Project 7
AGEDIS
3. Partners (Institutions)
IBM Haifa, Israel: Alan Hartman, Kenneth Nagin, Sergey Olvovsky, Andrei Kirshin,...
Oxford University, UK: Jim Davies, Alessandra Cavarra
France Telecom R&D, France: Yves-Marie Quemener
Intrasoft International, Luxembourg, Greece: Kyriakos Bechrakis, Antonis Ramfos
Page 8
© imbus AG, Bernd Mattern The AGEDIS Project 8
AGEDIS
IBM Hursley, UK: Ian Craggs
VERIMAG - Université Joseph Fourier, France: Laurent Mounier, Yassine Lakhnech
Irisa, France: Thierry Jeron
3. Partners (Institutions)
imbus AG, Germany: Manuela Heigl, Johannes Trost, Bernd Nossem, Klaudia Dussa-Zieger, ...
Page 9
© imbus AG, Bernd Mattern The AGEDIS Project 9
AGEDIS
User
Interface
Software ModelIntermediate Format
EncodingGenerator
Abstract
Test Suite
Execution
Engine
Suite Execution
Trace
Analyzer
Visualizer
Test Goals
Compiler
Test ExecutionDirectives
3. Architecture
Page 10
© imbus AG, Bernd Mattern The AGEDIS Project 10
AGEDIS
3. Work Package Structure
Work Package (WP) total effort
WP1 - Project Management 33 PM
WP2 - Software Modelling Language 25 PM
WP3 - Modelling Language Compiler 31 PM
WP4 - Test Generation Tool 32 PM
WP5 - Test Execution Tool 32 PM
WP6 - Test Productivity Tool 37 PM
WP7 - Test Automation Environment 36 PM
WP8 - Application Experiments 56 PM
WP9 - Technology Transfer 54 PM
Sum 336 PM (28 PY)
Page 11
© imbus AG, Bernd Mattern The AGEDIS Project 11
AGEDIS
3. Roles of Partners
Coordinator, leader of WP3, WP5, WP6, moderator exp 2
Leader WP2 and educational expertise
tool user, delivers test subjects
tool user, leader of WP7, integration, delivers test subjects
tool user, supplier of test subjects
leader WP4, moderator exp 1
tool user, testing expertise, leader of WP9, moderator exp 3, 4, 5
Page 12
© imbus AG, Bernd Mattern The AGEDIS Project 12
AGEDIS
Survey Language Spec
First Experiments
Prototypes:CompilerGeneratorExecutor
2ndExperiments
FinalTools
ExploitationPublication
4 months
12 months
requirements
design
implementation
test
12 months
3 months
Today
3. Flow of Events
Page 13
© imbus AG, Bernd Mattern The AGEDIS Project 13
AGEDIS
UML based
Subset of UML diagrams
Definition of special semantics
Definition of software design
Definition of test goals / test directives
4. Modelling Language
Page 14
© imbus AG, Bernd Mattern The AGEDIS Project 14
AGEDIS
4. From Software DesignClass Diagrams:
Class1+attr1+attr2+operation1+operation2
Class2-attr3#attr4-operation1+operation2
Class3-attr3-attr4#operation1+operation2
Class4#attr1#attr2+operation1#operation2
0..*
Page 15
© imbus AG, Bernd Mattern The AGEDIS Project 15
AGEDIS
4. From Software DesignClass1+attr1+attr2+operation1+operation2
state 1
state 3 state 4
state 2
every state has:
- entry & exit actions
- internal transitions
- NEW in AGEDIS: deferred eventstimer events
decision
final
initial
trigger[gard]/action
For each class:State transition diagrams:
Page 16
© imbus AG, Bernd Mattern The AGEDIS Project 16
AGEDIS
4. AGEDIS Semantic extension
<<signal>>Signal1
{controllable, observable}+attr1+attr2
<<signal>>Signal2
{controllable}+attr1+attr2
<<signal>>Signal3
{observable}+attr1+attr2
<<signal>>Signal4
+attr1+attr2
Controllable:
Signals, that can be sent to the system by the tester
Observable:
Signals from the system, that can be validated by the tester
Stereotype <<signal>> used to specify events and actions in state diagrams:
Page 17
© imbus AG, Bernd Mattern The AGEDIS Project 17
AGEDIS
Object Diagrams:
- Define states of the system during test. Attributes are defined.
- AGEDIS Semantic extension: Special stereotypes are given to diagrams: <<start>>, <<finish>>, <<include>>, <exclude>>
<<start>>
4. Test Directives
obj1:Class2-attr3=10#attr4=0
obj2:Class3-attr3=“Hi“-attr4=“Bye“
Page 18
© imbus AG, Bernd Mattern The AGEDIS Project 18
AGEDIS
<<exclude>>
<<include>>
<<finish>>
<<start>>
<<exclude>>
<<exclude>>
<<include>>
<<include>>
<<include>>
4. Test Directives
What does the AGEDIS test generator do?
Searches for test cases:
„Good case“!
Page 19
© imbus AG, Bernd Mattern The AGEDIS Project 19
AGEDIS
<<exclude>>
<<include>>
<<finish>>
<<start>>
<<exclude>>
<<exclude>>
<<include>>
<<include>>
<<include>>
4. Test Directives
another „Good case“!
What does the AGEDIS test generator do?
Searches for test cases:
Page 20
© imbus AG, Bernd Mattern The AGEDIS Project 20
AGEDIS
<<exclude>>
<<include>>
<<finish>>
<<start>>
<<exclude>>
<<exclude>>
<<include>>
<<include>>
<<include>>
4. Test Directives
„Bad cases“!
What does the AGEDIS test generator do?
Searches for test cases:
Page 21
© imbus AG, Bernd Mattern The AGEDIS Project 21
AGEDIS
4. Test Directives
state 1
Reject
Accept
state 2
Alternative approach:
- Definition of a system-wide state transition machine
- AGEDIS semantic extension: „Accept“ and „Reject“ state
signal1
signal2
signal2
signal1
Page 22
© imbus AG, Bernd Mattern The AGEDIS Project 22
AGEDIS
Lots of sequences including: Variations in work flows
Variations in input data
Variations in output data
Summed up: An abstract test suite
Output format: XML
Details under discussion
4. Output from AGEDISobj1:Class1 obj2:Class2env:Env
get(1):Signal3
put(1):Signal1
put(0):Signal2
get(0):Signal3
send(1):Signal4
receive(1):Signal4
Page 23
© imbus AG, Bernd Mattern The AGEDIS Project 23
AGEDIS
Testbeans GTC (Gotcha - TCBeans from IBM)
Meta-Words (WinRunner Script based method from imbus)
Your collection of black or white box test functions
4. Bringing the test suite to life
Page 24
© imbus AG, Bernd Mattern The AGEDIS Project 24
AGEDIS
Arbitrary GUI Test tools e.g. WinRunner (Mercury Interactive)
4. Test Execution Engine
Protocol interface driver customer tools
Page 25
© imbus AG, Bernd Mattern The AGEDIS Project 25
AGEDIS
StP/UML (Software through pictures, Aonix) template technology
> 50% source code generation
j-UML (Extension for Java, VTT, Finland) Symbols for special Java features
no tool
xUML (Kennedy Carter) complete Action Language for source code generation
Tool bases validation of design through test cases
5. Other approaches
Page 26
© imbus AG, Bernd Mattern The AGEDIS Project 26
AGEDIS
AGEDIS will reduce test specification and review effort
The AGEDIS software derives test cases from UML diagram via test directives
AGEDIS brings test documentation closer to development departments
AGEDIS makes testing more complete
AGEDIS makes testing more fun
6. Conclusion
Page 27
© imbus AG, Bernd Mattern The AGEDIS Project 27
AGEDIS
ContactHeadquarter Erlangen/Moehrendorf:
imbus AG Tel: 09131/7518-0Kleinseebacher Strasse 9 Fax: 09131/7518-50 91096 Moehrendorf Email: [email protected]
Office Munich:
imbus AG Tel: 089/3219909-0Unter der Linde 16 Fax: 089/3219909-50
80939 München Email: [email protected] Germany
Office Frankfurt:
imbus Rhein-Main GmbH Tel: 06192/92192-0Zanggasse 6 Fax: 06192/92192-50 65719 Hofheim Email: [email protected]
Page 28
© imbus AG, Bernd Mattern The AGEDIS Project 28
AGEDIS
www.agedis.de
Page 29
© imbus AG, Bernd Mattern The AGEDIS Project 29
AGEDIS
Download further information
Page 30
© imbus AG, Bernd Mattern The AGEDIS Project 30
AGEDIS
Discussion and Newsletter