Model Based Test Case Generation

8/6/2019 Model Based Test Case Generation

1/24

MODEL BASED TEST

GENERATION


2/24

WHATS A MODEL?

A model:

Is an abstraction or simplified representation ofthe

system from a particular perspective

Supportsinvestigation, discovery, explanation, prediction,or construction

May be expressed as a description, table, graphical

diagram, or quantitative mathematical model

Is not necessarilycomprehensive

2


3/24

MODELS IN EVERYDAY LIFE

3


4/24

EXAMPLES OF MODELS

Flow Charts

Data Flow Diagrams

Entity- Relationship Diagrams

State Diagrams

Deployment Diagrams

ClassDiagrams

Use Ca

se

s ActivityDiagrams

State Transition Tables

4


5/24

MODEL BASED TEST GENERATION

5

set of

Test Cases

sequence of stimuli

expected responses

Formal Model

Test GenerationDirectives

Generator


6/24

MODELING

6

Modeling Validation

Refinement Verification

SpecificationAsmL

ImplementationC/C++/C#

Product Idea/ Specification

Are you buildingthe product right ?

Are you

building theright product?

What productare you

building?


7/24

FSM: START STATE & ACTION

7

Start state (Entry Point)

Files Shown: 0

File Select: Not Selected

Exit Point: False


8/24

OTHER

STATES

8

start


9/24

SIMPLE FSM MODEL (STATE MODEL)

9


10/24

WALK EVERY STATE-CHANGE

10

Create

Invert Selection

Invert Selection

Delete


11/24

TEST EVERY WALK < 5 STEPS

11

A: Invert Selection

B: Create; Delete

C: Create; Invert Selection; Invert Selection; Delete


12/24

MODEL BASED TEST GENERATION

TOOLS

T-VEC(Test Vector Generation System)

TestMaster

UniTesk (Unified Testingand spec toolkit)

GOCHA Techbeans

AS

MLEtc..

12


13/24

THE ASML MODELING TOOLKIT

Abstract state machine Language is a powerfulmodelinglanguage

AsmL provides Authoringvia MS Word and XML

Simulation and interoperabilityvia COM and .NET

Automatic test oracle

Semi-automatic test case generation

Integration with Microsofts test platform

13


14/24

ASML: GETTING STARTED, SORTEXAMPLE

14

var A as Seq ofInteger = []

Swap()choose i in indices(A),

j in indices(A)where i < j and A(i) > A(j)

A(i) := A(j)A(j) := A(i)

sort()step until fixpoint

swap()

A = [2,3,1]

A = [1,2,3]

A = [2,1,3]

Non-determinism

A = [1,3,2]

Parallelism


15/24

ASML: PREDEFINED DATA

TYPES

Sets

menu = {"ham", "cheese","blt"}

15

SequencesSequencesrandomrandom == [[3,3, 6,6, 7,7, 1,1, 55]] ////asas SeqSeq ofof IntegerInteger

MapsMapsdirectorydirectoryas Map ofas Map of StringString toto Integer =Integer =

{{policepolice -->>112, info112, info -->>118118}}

TuplesTuplesnameAndAgenameAndAge == ((John,39John,39)) ////((StringString,,IntegerInteger))


16/24

USING MODELS FOR TEST

16

Model

Implementation

Test OracleTest Cases

Are run byProvides

actual results for

Pass

No pass

Provides expected

results forGenerates

User

Info


17/24

ASML-BASED TEST HARNESS

Side-by-side runs

Discrepancies

automaticallyflagged

17

AsmLmodel

Test harness

Implementationunder test

Anyclient


18/24

ASML TEST CASE GENERATION - 1

Solves two problems

How to find interesting parameter values

and combinations How to find interestingcall sequences

Approach Use information derived from the model to

optimize the number oftest cases

18


19/24

ASML TEST CASE GENERATION -

2

Divide state space accordingto state predicates

System covers the state predicates Driven byinput generator (parameter selection) or

Driven by enabledness (call sequences)

Tester providesinputs, givesinterestingstate

predicates, directs the search

19


20/24

THE DUAL ROLE OF ASM MODELS

20

AsmL

ModelM

Test cases

Guided

generation(Param

Select

withFeedback)

Reduce

FSM

tool

IUT

Call next action

Get state

AsmL

ModelM

Testing harness

Is S

valid according

to M?

Fail: witness

Pass S

view

view

view

FSM


21/24

ASML TEST CASE GENERATION - 3

Generate only as many test cases as needed to havefull coverage ofthe specification

21

Generator ModelInput

Coverage criteriaCoverage criteria

Branch: Every branch of the spec must beBranch: Every branch of the spec must betakentaken

Modified condition: independent atomicModified condition: independent atomicconditionsconditions

OffOff--byby--One: offOne: off--byby--one errorsone errors

HyperHyperstate coverage: Everystate coverage: Every abstractabstractstatestatemust be visitedmust be visited

Feedback


22/24

ASML TEST (ASMLT)

asmlT can be used to:

1. Find interestingsequences ofmethod calls finds the shortest route in a generated FSM that uses every edge to

traverse the states

2. Find interestingparametersfor each method call

Access Driven Filtering(ADF) you write a predicate (boolean function) to decide ifa given

parameter isinteresting

3. Perform a conformance test

Compare Model with Implementation

22


23/24

REFERENCES

Model-Based Test Generation Tools, Alan Hartman,AGEDIS CONSORTIUMhttp://www.agedis.de/documents/ModelBasedTestGenerationTools_cs.pdf

Finite State Model-Based Testingon a Shoestring, HarryRobinsonhttp://www.model-based-testing.org/shoestring.htm

Model Based Testing, L. Apfelbaum, et al, Software Quality Week Conference inMay, 1997.http://www.model-based-testing.org/sqw97.pdf

Abstract State Machine Language, Microsoft Researchhttp://research.microsoft.com/fse/asml/

The AsmL Test Generator tool, Microsoft Researchhttp://research.microsoft.com/fse/AsmL/doc/AsmLTester.html

Intelligent Test Automation, HarryRobinsonSoftware Testing& QualityEngineering(STQE) magazine Sept/Oct 2000http://www.model-based-testing.org/robinson.pdf

23


24/24

24

Queries..??

Model Based Test Case Generation

Documents