An Automated Test Strategy Based on UML Diagrams F. Basanieri *, A. Bertolino *, E. Marchetti *, A. Ribolini *, G. Lombardi **, G. Nucera ** *PISATEL LAB,
Post on 27-Dec-2015
216 Views
Preview:
Transcript
An Automated Test Strategy Based An Automated Test Strategy Based on UML Diagramson UML Diagrams
F. Basanieri *, A. Bertolino *, E. Marchetti *, A. Ribolini *, F. Basanieri *, A. Bertolino *, E. Marchetti *, A. Ribolini *, G. Lombardi **, G. Nucera **G. Lombardi **, G. Nucera **
*PISATEL LAB, IEI-CNR, Pisa, Italy*PISATEL LAB, IEI-CNR, Pisa, Italy
** ERICSSON LAB ITALY** ERICSSON LAB ITALY
PISATEL
Page 2
Agenda
• Cow_Suite tool • New features • Traceability
• Cow_Suite vs RUP• Guidelines
• Discussion and future work
Page 3
Objectives• Use for test planning the same UML diagrams developed
(e.g., by Rational Rose) for design and analysis• Tool-supported generation and maintenance of an
updated and strategic test plan, as the design evolves.
PISATEL
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics To Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
DecisionModel
_decision : String
DecisionModel()GetDecisions()
(from Logical View)GoalModel
_goals : String
GoalModel()getGoals()
(from Logical View)
Decision
_name : String_priority : Integer = 0
Decision()getName()getPriority()
(from Logical View)
+contains
Designer
(from Logical Vi...
Goal
_name : String = "a"_priority : Integer = 3
Goal()getName()getPriority()
(from Logical View)
+contains
: Designer
: Decision...
: Decision : GoalModel
: Goal
1. DecisionModel( ) 1.1. Decision(name, priority)
2. GoalModel( ) 2.1. Goal(name, priority)
3. GetDecisions( ) 3.1. getName( )
3.2. getPriority( )
4. getGoals( ) 4.1. getName( )
4.2. getPriority( )
Test Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
NamingTest Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
Naming Test Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
NamingTest Case
getDecisions()getName()/getPriority()
Opening a saved file_decisions
Naming
Test Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
Naming
Test Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
Naming
Test Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
Naming
Page 4
Cow_Suite approach
• The processing of the Cow_Suite approach starts by analysing the project description*, and proceeds in parallel with the design and analysis refinement steps.
• The reference documentation is derived by collecting together and automatically organizing the diagrams developed during the design phase without additional manual effort of formalization
[*: the internal description of the project model, given by Rational Rose in a .mdl file]
PISATEL
Cow_Suite consists of two main components: UIT (Use Interaction Test) and CoWTeSt (Cost Weighted Test Strategy)
Cow_Suite =Cowtest pluS UIT Environment
Page 5
Cow_Suite Scheme
Test caseTest case
Test case
Test case
Test caseTest case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test caseTest Test
Procedures Procedures
listlist
Strategy
StrategySele
ction
Selecti
on
Obj 1 Obj 2 Obj 3
Method1()
Method3()
Method2()
U I TU I T
SDs selection according to the strategy chosen
Automatic UIT application for Test
cases derivation
Test specific
ation
Interaction with the user for deriving executable test procedures
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics To Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Critics To Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
0.250.25 0.50
0.500.50
0.7 0.3
0.30.35 0.15
Cow_SuiteCow_Suite
PISATEL
Page 6
Tree Derivation
Use Cases (UCs) and Sequence Diagrams (SDs) of the .mdl project design realized with Rational Rose are automatically organized by Cow_Suite tool in a sort of hierarchical (pseudo-)tree
Hierarchical TreeHierarchical Tree
The case study is Course Registration System from RUP documentation
PISATEL
Page 7
Tree Derivation
The users can get a complete view of the status of the functionalities specification and a detailed documentation:
•UCs and their realization•SDs associated to each specification level
•The links between Use Case and Design Model
•Reused nodes•Not linked model elements
Hierarchical TreeHierarchical Tree
PISATEL
Page 8
Use Case View vs Logical View
The Main Tree can be analyzed from two different points of view which correspond to two different typologies of test.
•Using the information of the Use Case View System test Subsystem test
•Using the information of the Logical View Subsystem test Component test
PISATEL
Page 9
Use Case View
describes the set of UCs (and scenarios) that represents the significant functionalities and the interaction of the system with the external world.
PISATEL
The Main Tree derived represents a detailed documentation of how features/system requirement are realized in the Use Case View.
•The path from a high level Use Case to a leaf (UCs or SDs) describes the refinement of the associated system feature
The Test Cases derived from the SDs of the Use Case View are specifically designed for
•System integration Test•Subsystem integration Test
Page 10
Logical ViewDescribes how the system functionalities are provided in term of static structure and dynamic collaborations of the objects.
PISATEL
•Cow_Suite tool analyses the Logical View information about:–Use Cases realization –Sequence diagrams, classes and their organization in packages and subsystems
•The tool automatically derives:The UC traceability by analyzing the package Use Case Realizations.The Design Tree: derived by analyzing the Design Model packages
Page 11
Use Case Diagram “Traceabilities”
• The “Use Case Realizations” package links the information of the Use Case View and the Logical View .
• The Use Case Diagram “Traceabilities” traces the evolution of the Use Cases in the Design Model
PISATEL
•Each Traceability Use CaseRepresents the evolution of a UC of the Use Case View into the Design ModelDetails the sub-flow of a UC with Sequence (Collaboration) DiagramsSpecifies the Classes and the relationship that participate in the realization of the UC (Class Diagrams)Optionally holds the package Design Link
Page 12
Use Case Realization analysis The Main Tree derived from the Use Case View is automatically
extended with the information of the Use Case Realizations Package
The path from a high level Use Case, through its Use Case Realization, to a leaf documents and traces the implementation of a system feature
Considering the subtree rooted in a Traceability Use Case, its SDs: contain low level information with respect to those of the Use Case
View describe in a detailed manner the collaboration of the involved
objects The Test Cases derived from these SDs are specifically designed
for• Subsystem integration Test
• Component integration Test
PISATEL
Page 13
Design Link Package• Design Link package: for every use case realization it collects
the package/s that represent the implementation of the relative UC.
PISATEL
It is an explicit link between the Main Tree and the Design tree
Page 14
Design Tree
Analyzing this tree it is possible to:
Observe the dependence relationship between packages
Know which are the reused packages and where the reuse happens
Connect to the main tree the low level SDs not explicitly linked in the .mdl file
PISATEL
Design Tree: is derived directly from the Design Model and represents the organization of the packages in a hierarchy.
By selecting the SDs of the Design Tree and applying to them the Cow_Suite test derivation strategies, a list of low level Test Cases can be derived
Page 15
Deduce the Critical Profile The effort devoted to the test of different functionalities must be
planned and scheduled considering their relative “importance” (risk-based testing, reliability guided testing, etc…).
The importance, represented by number (weight) belonging to [0,1] interval, can be assigned in two different manner:
• Interacting with Cow_Suite Tool: Following level by level the tree structure of the UCs and SDs, the user annotates the corresponding node in the tree with its weights
• Directly in the .mdl file (UCs only): During the development of the Use Case Model architects/designers assign a value, belonging to [1,9] interval, to each Use Case.
The value is memorized in the use case documentation tab with the following structure: WGH = value (For SDs or when no value is expressed for UCs the default value is 5)
PISATEL
Page 16
Weighted tree derivationThe tool : reads the assigned
values the .mdl file (if they exist)
sums the values of the node at the same level in the tree
normalizes the weights among the nodes at the same level so that their sum is 1.
PISATEL
The user can always modify the assigned weights
Page 17
Integration Stage selectionThe Cow_Suite user selects the integration stage at which the test is conducted.
The nodes that are considered by Cow_Suite tool for the subsequent evaluations are all and only those belonging to the selected integration stage plus all the tree leaves existing at higher levels.
PISATEL
Page 18
Cow_Test StrategyTwo different situations are considered in test cases planning:
1: Cowtest_ing with fixed number of tests
• A certain budget is available, which is translated in terms of number of tests, NT, to execute.
• In this case the tool select the most adequate selection of NT tests from among the (many) various test cases that could be potentially conceived.
PISATEL
Test Case
Test Case Test Case
Tes
t Cas
e
Test Case
Tes
t Cas
e
Test Case
Test C
ase
Test Case Test Case
Test C
ase
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case
Test Case
Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
NTNT
Page 19
Cow_Test Strategy
2: Cowtest_ing with fixed functional coverage• A certain percentage of functionalities must be covered.
• In this case by the tool is possible to define in advance which are the functionalities to be covered and the minimum number of tests to
execute.
PISATEL
Functional Coverage = 90%
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Critics To Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
0.250.25 0.50
0.500.50
0.7 0.3
0.30.35 0.15
10
2 2
11
Minimum Number of test to execute = 16
Page 20
Select the Test Strategy
PISATEL
Cowtest_ing with fixed functional coverage.
The most critical system functionalities are
selected for reaching the fixed coverage.
Test Procedures are distributed accordingly.
Cowtest_ing with fixed number of tests.
The selection of the most suitable distribution of test procedures is developed on the basis of leaves weights.
Page 21
For each fixed integration stage, a weighted subtree is derived according to the chosen test criterion. Only the relative SDs are considered for Test Case generation.
SDs selectionSDs selection
Sequence Diagrams Selection
PISATEL
N. Test Procedures
Final weight
Page 22
Test Case Derivation
The tool uses the UIT method, to derive the Test Cases.
The method is mainly based on the SDs (particularly objects, messages and parameters)
SDs SelectionSDs Selection
PISATEL
Page 23
Sequence Diagram analysis
: Designer
: Decision...
: Decision : GoalModel
: Goal
1. DecisionModel( ) 1.1. Decision(name, priority)
2. GoalModel( ) 2.1. Goal(name, priority)
3. GetDecisions( ) 3.1. getName( )
3.2. getPriority( )
4. getGoals( ) 4.1. getName( )
4.2. getPriority( )
•Test Units: horizontal axis analysis.
•Interactions Categories and Settings Categories: vertical axis analysis.
• MessageSequences: set of Messages (in temporal order), involved in a SD, used by objects to define and elaborate specific functionalities.
PISATEL
Page 24
Test Case DerivationFor each traceable Message Sequence a Test Case is generated. It contains the list of all its Settings and Interactions Categories and their values.
The final weight of every SDs is used to automatically derive the number of Test Procedures associated to each Test Case.
Detailed Test Case Description
PISATEL
Number of associated Test Procedures
Page 25
Test Procedures Specification and derivation
Test Specification
For each identified category, we consider all its possible values and constraints (“choices”).
PISATEL
The user interacts with the tool for inserting Choices values.
A Test Procedure is automatically generated from a Test Specification, for each possible combination of choices of every category involved in a Messages Sequence.
Page 26
Cow_Suite vs RUP
• RUP is based on an iterative process characterized by four phases divided in workflows.
• A workflow is a set of activities that produces a observable result (artifacts). The involved workflows are:
– Project Management
– Requirement
– Analysis and Design
– Test
PISATEL
• Guidelines for using Cow_Suite inside RUP
• Best Practices for Cow_Suite application
25
60
15
0
0
10
20
30
40
50
60
70
80
Inception Elaboration Construction Transition
RUP Phases
Cow_Suite Integration
Page 27
Inception PhaseDuring the inception phase:• The scope of the project and the external entities with which the system
will interact (actors) are defined.
• The use cases describing the most significant interaction are also
identified
PISATEL
Workflow Requirement
1. Analyse the Problem
2. Understand Stakeholder Needs
3. Define the System
4. Manage the scope of the System
5. Refine the System Definition
6. Manage Changing Requirements
7. Classify Requirements
Workflow Analysis and Design
• Perform Architectural Synthesis
• Evaluate Use Case
• Derive Cow_tree
Workflow Test• Test Plan• Testing Cost Estimation
Page 28
Elaboration PhaseDuring the inception phase:• The analysis of the problem domain and the basic architecture are established• The project plan is developed and the possible risk elements are evaluated• A prototype is built, and the use case model is almost complete
PISATEL
Workflow Project Management• Plan for Next Iteration• Manage Iteration• Monitor ad Control Project• Analyse Cow_tree • Reevaluate Project Scope and Risk
Workflow Requirement1. Analyse the Problem2. Understand Stakeholder Needs3. Define the System4. Manage the scope of the System5. Refine the System Definition6. Manage Changing Requirements7. Classify Requirements
Workflow Analysis and Design1. Define a Candidate Architecture
2. Process UCs and SDs
3. Derive Cow_tree3. Analyse Behaviour4. Implement Traceability 5. Evaluate Traceability6. Design component/design for Real-Time7. Cow design tree8. Design databaseWorkflow Test• Testing Cost Estimation• Plan test• Derive UIT Test • Test Traceability• Design test• Implement test• Execute Test in Integration Test Stage• Execute Test in System Test Stage• Evaluate Test
Page 29
Construction Phase• Includes the development of the system remaining components and features and
the test of the entire system.
PISATEL
Workflow Project Management• Plan for Next Iteration• Manage Iteration• Monitor ad Control Project• Analyse Cow_tree • Reevaluate Project Scope and Risk
Workflow Requirement1. Manage Changing Requirements2. Classify Requirements
Workflow Analysis and Design1. Define a Candidate Architecture
2. Process UCs and SDs
3. Derive Cow_tree
4. Analyse Behaviour5. Implement Traceability 6. Evaluate Traceability7. Design component/design for Real-Time8. Cow design tree9. Design database
Workflow Test• Testing Cost Estimation• Plan test• Derive UIT Test • Test Traceability• Design test• Implement test• Execute Test in Integration Test Stage• Execute Test in System Test Stage• Evaluate Test
Page 30
Inception Phase – Workflow Requirement Classify Requirements
The requirements must be classified and grouped depending on:
criticality system importance difficulty in the development
Optional: a number in [1,9] interval representing this importance value should be assigned to each requirement/requirement set.
PISATEL
Page 31
Inception Phase – Workflow Analysis and Design Evaluate Use Case
Basing on the requirement classification an importance value ( [1,9] interval) must be assigned to each developed UC.
The importance is an absolute value characterizing the UC The value is memorized in the use case documentation
tab with the following structure:
WGH = value
PISATEL
Page 32
Inception Phase – Workflow Analysis and Design Derive Cow_Tree
Apply the Cow_Suite and derive the main tree.
Possible design weaknesses and incompleteness are
displayed.
The user profile is derived
The most important UCs, i.e. system functionalities, are
highlighted
PISATEL
Page 33
Inception Phase – Workflow Test Testing Cost Estimation
Applying Cow_Suite test strategies it is possible to : Distribute the Test Cases among the system
functionalities Know how many Test Cases are necessary to cover
the developed system functionalities Derive the completion time or “cost” (in terms of man
hours/required budget) of the Test Phase by assigning a value to each typology of Test Case
PISATEL
Page 34
Elaboration Phase – Workflow Project Management Analyse Cow_tree
Apply the Cow_Suite and derive the main tree that: Evidences the degree of detail reached in the requirement
development. Highlights the design weaknesses in the Use Case
Model/Design Model developed. The main tree integrates the information derived from the Use Case
View with those of the Logical view and can be used for the completion of the Sw Architecture Document.
Marks the most important functionalities of the developed system The final weight associated to each node represent the importance of
the implemented functionalities for the system. The correctness of the values assigned must be checked and
eventually modified
PISATEL
Page 35
Elaboration Phase – Workflow Requirement Classifying Requirements
Software architects and Program managers:
highlight the most critical UCs
select the UCs that will be refined
They assign or modify the importance value associated to
the UCs during the Inception Phase.
PISATEL
Page 36
Elaboration Phase – Workflow Analysis and Design Process UCs and SDs
• Basing on the requirement classification an importance value ( [1,9] interval) must be assigned to each developed UC. – The value is memorized in the use case documentation tab
with the following structure:
WGH = value • Optional: Insert the default values in the SDs. They
represent the values of the Interaction Categories of the UIT method. – The values are memorized in the message/class method
documentation tab with the following structure:
DEF (value1, value2…)
PISATEL
Page 37
Elaboration Phase – Workflow Analysis and Design Derive Cow_tree
Apply the Cow_Suite and derive the Main Tree.
An extension/modification of the main tree corresponds to every refinement of the Use Case/Design Model.
The evolution of the system requirement (Use case View), into diagrams of the Logical View is highlighted
The missing links between the two views and the diagram
not properly connected are shown and evidenced.
PISATEL
Page 38
Elaboration Phase – Workflow Analysis and Design Implement Traceability
o Associate to every Traceability Use Case a value representing the criticality in the realization of the associate UC, and annotate in the UC documentation tab this number.
o Link to every Traceability Use Case the SDs representing the most important scenarios and the interaction between the involved classes. In this manner every requirement will be automatically associated to the proper set of Test Cases
o Optional: create a class diagram, called the Design Link, with the list of the packages in the Design Model that represent the implementation of the UC
PISATEL
Page 39
Elaboration Phase – Workflow Analysis and Design Evaluate Traceability
Apply the Cow_Suite and derive the Main tree and the Design Tree
The Design Link evidences the evolution of the UCs in the Design model:
The Design Tree represents the degree of detail reached in the requirements (UCs) implementation
The objects or packages involved in the requirement implementation are organized in a hierarchical tree using packages dependencies.
The weights associated to the Traceability Use Cases are analyzed and possibly modified
PISATEL
Page 40
Elaboration Phase – Workflow Analysis and Design Cow Design Tree
The Design Tree: shows the hierarchy between the different system
components, the subcomponents, the reused components and the associated SDs
Represents a detailed documentation which associates to each requirement the relative system components.
following the path in the main Tree from an high UC, throw Use
Case realization, to the Design Link node. Then automatically
jump to the proper position in the Design tree and cover it down till
the leaves.
PISATEL
Page 41
Elaboration Phase– Workflow Test Testing Cost Estimation
o Considering the main tree:• Evaluate the correctness of the assigned weights
• Possibly modify the values
o Applying Cow_Suite test strategies it is possible to : • Distribute the Test Cases among the system functionalities
• Know how many test are necessary to cover the developed system functionalities
• Derive the completion time or “cost” (in terms of man hours/required budget) of the Test Phase by assigning a value to each typology of Test Case
PISATEL
Page 42
Elaboration Phase– Workflow Test Derive UIT Test
Apply the UIT method. The Test Cases derived from the main tree can be used
for system or subsystem or component test depending on the integration level chosen.
The Test Cases derived from the Design tree can be used for component/unit test.
The Test Procedures, as derived by the tool, are not directly executable ( Test Scripts). The derived Test Procedures have to be executed and logged by a test driver.
The degree of detail of the Test Set depends directly on, the design detail level and granularity of the inserted values (choices of the UIT categories).
PISATEL
Page 43
Elaboration Phase– Workflow Test Test Traceability
Considering the main tree and the Design tree:
the SDs on the path from an high level UC to a leaf (of the main tree or Design tree) produces Test Cases specialized for the testing and the verification of the requirement (system functionalities) associated to the UC.
The degree of detail of the derived Test Cases depend on hierarchical position of the SDs in the tree (the higher is the level, the lower is the degree of detail)
PISATEL
Page 44
Discussion and Future Work
PISATEL
Tool integrated with the design tool (e.g. Rational Rose)
Systematic and automated support for test planning and test case generation, based on the same UML diagrams developed for design and analysis
Test case derivation (UIT) to be validated on ERI case studies
Page 45
PISATEL
top related