Data Flow Testing

Csci 565Spring 2009

ObjectivesDefine/Use TestingDU-Path Test Coverage MetricsExample Commission problem

Test coverage criteriaThe main focus is structural testing

Data flowControl flowMutation testing

How much testing is enough?need to define test coverage criteria (or yard

stick) Test coverage criteria?

A minimal set of test requirements (e.g. p0, p1, etc.) that must be collectively exercised by the test cases when we execute the program

Test Coverage criteriaControl flow analysis

Uses flow graphControl flow Coverage Criteria

All-nodesAll-edgesAll-paths

Informal Definitions of The DF TestingRefers to the form of structural testing (or

path testing)used for compiler optimizationAttempts to “bridge the gap” between branch

testing and path testingSuffers from un-executable paths (infeasible

paths)The focus is on the locations (i.e., statements)

where variables are defined and used A definition (value is stored in MM) A Use (a value is fetched from MM)

Data Flow diagrams (or charts) are modeling notations used at different abstraction levels

Data flow analysis (DFA)’s anomaliesDetects Defines/reference anomalies, such

asA variable that is defined but never used

(unused variable)A variable that is used but never defined

(undefined variable)A variable that is defined twice before it is used

(redundant operations)Data reads from location not previously written to

(uninitialized variables)

The anomalies can be discovered by “static analysis”

Define/Use TestingEarly work was done in 1980s by [Rapps 85] The research adopts the formalization of

program graph in whichNodes correspond to program statementsEdges correspond to the flow of information

Single entry and single exist pointsNo self-looping nodes

Defining node?Node n G(P) is a defining node of the

variable v V, written as DEF(v, n), iff the value of the variable v is defined at the

statement fragment corresponding to node n G(P) refers to Flow Graph of Program P

Example of defining statementsDefining Statements include

Input statements (e.g., read)Assignments statementsLoop control statementsProcedure calls

When the code corresponding to the above statements executes, the contents of the memory locations related to the variables are changed

Usage node?Node n G(P) is a usage node of the variable

v V, written as USE(v, n), iff the value of the variable v is used at the

statement fragment corresponding to node n

Example of Usage statementsUsage Statements include

Output statements (e.g., write)Assignments statements (r.h.s.)Conditional statementsLoop control statementsProcedure calls

When the code corresponding to the above statements executes, the contents of the memory locations related to the variables are unchanged

Predicate use: (P-use)A usage node USE(v, n) is either

a predicate use (denoted by P-use)Or, a computation use (denoted by C-use)

The nodes corresponding to P-use have out-degree 2

The nodes corresponding to C-use always have outdegree 1

Definition-use (sub)pathA definition-use (sub)path w.r.t. a variable v

(denoted du-path) is a (sub)path in PATHS(P) such that for some vV there are defined and usage nodes

DEF(v, m) and USE(v,n) Where

m is initial node n is final nodes

Definition-clear (or dc) pathA defintion-clear(sub)path w.r.t. a variable v

(denoted dc-path) is a DU(sub)path in PATHS(P) with initial and final nodes DEF(v, m) and USE (v, n) such that no other node in the (sub)path is a defining node of v

DF testingRequires that the test data execute dc-path from

each node containing a global definition of a variable to specified nodes containing global c-uses and edge containing p-uses of that variable

The significant of DU-Paths and DC-PathsDu-paths and dc-paths capture the essence of

computing with stored data values Du-Paths and dc-Paths describe the flow of

data across source statements from the locations where the values are defined to the locations where the values are used

Du-paths that are not Dc-paths are considered as problematic spots

About VAR and CONSTVAR and CONST declaration

Should they be considered as defining nodes? Yes and no

Example: The Commission problem discussed in [Jorgenson95]The Commission Problem Statement

Rifle salesperson in the Arizona Territory sold rifle locks, stocks, and barrels made by a gunsmith in Missouri. Locks cost $45.00, stocks cost $30.00, and barrels cost $25.00. Salespersons had to sell at least one complete rifle per month, and production limits are such that the most one salesperson could sell in one month is 70 locks, 80 stocks, 90 barrels. Each rifle salesperson sent a telegram to the Missouri company with the total order for each town s/he visits; salespersons visit at least on town per month, but travel difficulties made ten towns the upper limit. At the end of each month, the company computed commission as follows: 10% on sales up to $1000 15% on the next $800 20% on any sales in excess of $1800.

The company had four salespersons. The telegram from each salesperson were sorted into piles (by person) and at the end of each month a data file is prepared, containing the salesperson's name, followed by one line for each telegram order, showing the number of locks, stocks, and barrels in that order. At the end of the sales data lines, there is an entry of “-1” in the position where the number of locks would be to signal the end of input for that salesperson. The program produces a monthly sales report that gives the salesperson’s name, the total number of locks, stocks, and barrels sold, the salesperson's total dollar sales, and finally his/her commission.

Some Du-paths Stocks variable

DEF(stocks, 25) USE(stocks, 27) DuPath

P0=<25, 27> Dc-Path

<25,27> Locks variable

DEF(locks,22) DEF(locks, 29) USE(locks,23) Use(locks,26) Du-path

p1=<22,23> p2=<22,23,24,25,26> (begin the

loop) p3<29,30,23> p4=<29,30,23,24,25,26> (repeat the

loop) p1’=<22,23,31> (by pass the loop) p3’=<29,30,23,31> (exist the loop)

Dc-paths p1,p2,p3,p4,p1’,p3’

Complete set of test cases for the WHILE-loop: p1’, p2, p3’, p4

DU-paths w.r.t. num_locks DU-Paths w.r.t. num_locks

Used in computational uses (c-uses) Defining nodes

DEF(num_locks,19) DEF(num_locks,26)

Usage nodes USE(num_locks,26) USE(num_locks,33) USE(num_locks,36)

DU-paths P5=<19,20,21,22,23,24,25,26> (dc-path) P6=<19,20,21,22,23,24,25,26, 26, 27,28,29,30,31,32,33> (NOT dc-path) Compacted p6:

P6=<19,20,21,22,23,24,25,26, 27,28,29,30,31,32,33> p7 = <19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36) p7 can also be rewritten as

P7= <p6,34,35,36> (NOT dc-path because it contains node 26) P8=<26,27,28,29,30,31,32,33> (subpath of p7; dc-path) P9=<26,27,28,29,30,31,32,33,34,35,36> (subpath of p7; dc-path)

DU-paths w.r.t. salesDu-paths w.r.t. sales

ONLY one defining node for sales (i.e. the paths are dc-paths) p10=<36,37> p11=<36,37,38,39> p12=<36,37,38,39,40,41,42,43> (dc-use)

Look at IF, ELSE IF (statements 39-50) p13=<36,37,38,39,45,46,47,48> p14=<36,37,38,39,45,50>

DU-paths w.r.t. CommissionDu-paths w.r.t. Commission

<43,44,45,48,50,51> (no cp-path)Consider only du-paths that begin with three

“real” defining nodes for commissionp15 =<43,51>p16=<48,51>p17=<50,51>

Table 4 shows the full set of du-paths

DU-PATH TEST Coverage Metrics (or Rapps-Weyuker dataflow criteria)Idea behind analyzing a program is to

define a set of test coverage (AKA the Rapps-Weyuker DF criteria) ALL-pathsAll-edgesAll-nodesAll-DefsAll-UsesAll P-Uses/Some C-UsesALL-C-Uses/Some P-UsesAll-DU-paths

ALL-DefsThe set T Satisfies the All-Defs criterion for

the program P iff for every variable vV, T contains dc-paths

from every defining node of v to a use of v where V: a set of variables T: a set of (sub)paths in G(P) of a program P

ALL-UsesThe set T Satisfies the All-Uses criterion for

the program P iff for every variable vV, T contains dc-paths

from every defining node of v to every use of v , and to the successor node of each USE(v,n)

where V: a set of variables T: a set of (sub)paths in G(P) of a program P

ALL-P-Uses/Some C-UsesThe set T Satisfies the All-P-Uses/Some C-

Uses criterion for the program P iff for every variable vV, T contains dc

(sub)paths from every defining node of v to every predicate use of v , and if a definition of v has no P-uses, there is a definition-clear path to at least one computation use

where V: a set of variables T: a set of (sub)paths in G(P) of a program P

ALL-C-Uses/Some P-UsesThe set T Satisfies the All-P-Uses/Some C-

Uses criterion for the program P iff for every variable vV, T contains definition clear (sub)paths from every defining node of v to every use of v , and if a definition of v has no C-uses, there is a definition-clear path to at least one predicate use

whereV: a set of variablesT: a set of (sub)paths in G(P) of a program P

All-DU PathsThe set T Satisfies the All-DU paths criterion

for the program P iff for every variable vV, T contains definition clear (sub)paths from every defining node of v to every use of v, and to the successor node of each USE(v,n), and that these paths are either single loop traversals, or they are cycle free

Summary of the relevant terms

Summary: Du-paths

Summary of the test coverage criteria

Simple Example:2

References [Rapps 85] S. Rapps and E. Weyuker. Selecting software

test data using data flow information. IEEE Transaction on Software Eng., vol.se-11, no.4, pp:367-375, April, 1985.

[ Frankl 88] P. Frankl and E. Weyuker. An Applicable Family of Data Flow Testing Criteria. IEE Transaction on software eng., vol.14, no.10, October 1988.

[Weyuker 88] E. Weyuker. The evaluation of Program-based software test data adequacy criteria. Communication of the ACM, vol.31, no.6, June 1988.

[Jorgenson 02]. Software Testing: A Craftsman’s Approach.2nd CRC publication, 2002.