.ppt

Software EngineeringResearch paper presentation

Ali Ahmad

•Formal Approaches to Software Testing

•Hierarchal GUI Test Case Generation Using Automated Planning

Formal Approaches to Software Testing

P. DasiewiczIEEE Canadian Conference on Electrical and Computer Engineering2002

Introduction Software Testing: checking and validating the correctness

of software Time consuming Difficult Error prune

Formal methods Specifying and verifying software systems using mathematical and

logical approaches Target: proof of correctness

Some verification techniques Model Checking State space exploration

Introduction (cont’d)

Current methods are not feasible Too much manual effort Models that are very complex for analysis

It is needed to overcome the following The state explosion problem Cost of creating models

Applicability: Design level Less complexity May reduce maintenance costs

Introduction (cont’d)

UML Simple, general purpose, visual modeling Specify, visualize, construct and document UML-statechart basis for dynamic behavioral

description

Objectives: Overview model checking to the validation and test

case generation (using UML statecharts and interaction diagrams)

Applicability of formal methods to component integration and interaction testing

Integration/Component Level Testing

Offutt Changes in component states due to change

events UMLTEST, integrated with Rational Rose

Highly effective test cases can be generated for system level specification

Test prefix: ensure that the system is @ a certain pre state before performing the test

Issues:• Interaction is not considered• Some states may never been entered

Incorporated UML collaboration diagrams

Integration/Component Level Testing (cont’d)

Yoon Test case generation based on UML sequence and

collaboration diagrams A node that represents both integration target and

message flow Testing technique

• Extract sequence diagram• If concurrency, extract collaboration diagram• Divide into ASF (Atomic System Function)• Extract nodes and message flows• Obtain test cases by applying test criteria (e.g. all-edge)

Limitations• May require a large number of sequence diagrams• Test case coverage (only normal or abnormal flow)

Integration/Component Level Testing (cont’d)

Kim Divide statecharts into extended finite state

machines Control flows are represented as paths Coverage criteria

• Path coverage• State coverage• Transition coverage

Test cases • breadth or depth first searches• can be generated using the data flow on the UML statechart• Determine whether class implements correct control and data

flow

System/Integration Testing Hartmann

Testing the interfaces among several components UML statecharts Mealy Finite State Machines

with restricted point to point synchronous communication model

Incremental composition and reduction algorithm Test cases: category partition method Limitations

• Interaction are modeled as synchronous communications• Event exchanges contain no parameters• No support for nested machines

Source Code Based Testing Major problem: State space explosion

problem Reducing the state space entails

eliminating irrelevant code segments Hatcliff

Program slicing techniques Used by Bandera for translating Java code to Jimple, e.g. SPIN,

SMV Generates a finite state model for the reduced code

Java Path Finder Integrates model checking, program analysis and Testing Goal: Apply formal methods at source level Initially, check for safety properties like deadlocks (LockTree)

Translation of Statecharts Promela: input modeling language for

SPIN C-like, extended with non-deterministic, and

loop guarded constructs Latella presents a translation from UML

statecharts into Promela, where statecharts are first converted to hierarchal automata. A proof of correctness is also given

Component Interaction Testing Major issue: Are the components developed

separately work properly together Formal methods of component interaction

Define testing requirements Automatically generate the test cases using model checking

Focus of author’s research: Component interaction of software systems Detect subtle interaction errors without duplicating the work @

unit level Underlying model: Labeled transition system ObjectState: modeling language to show the feasibility of

creating formal models, it relies on hierarchal FSM.

Hierarchal GUI Test Case Generation Using Automated Planning

Memon A., Pollack M., Lou Soffa M.IEEE Transactions on Software Engineering 2000

Introduction GUI testing is difficult

The interaction space is enormous Determining the coverage of test cases Regression testing is a major challenge

Automation is necessary for generating GUI test cases

PATHS: Planning Assisted Tester for grapHical user interface Systems Input: Possible goals for GUI user Generates sequences of events to satisfy the user goals, these

become test cases

Introduction (cont’d)

PATHS performs automated analysis of the hierarchal structure of the GUI to create hierarchal operators that will be used during plan generation

Contribution to Research Make use of an AI technique (Automated Planning) Exploits GUI structural features Makes Regression testing easier The test suite is portable Allow the reuse of operator definitions that commonly

appear across GUIs

Overview GUI consists of components like labels, buttons,

menus, and pop-up lists Example: Microsoft WordPad GUI has 2 types of windows

GUI windows Object windows,

doesn’t contain any window components

Overview (cont’d)

PATHS- Plan Generation Inputs

• Initial state• Goal state• A set of operators applied to a set of objects

• Preconditions and effects Solution to a Planning Problem: sequence of instantiated

operators which result in the goal state when executed in the initial state

2 Phases• Setup

• Create a hierarchal model of the GUI• Plan Generation

• Specify scenarios (initial and goal states)• PATHS generate a test suit for the scenarios

Overview (cont’d)

Role of Test Designer and PATHS during test generation

GUI Events and Planning Operators

Overview (cont’d)

Operator Event Mapping

Example operator: Edit_Cut

Abstract: Invokes a window that monopolizes the user interaction

Overview (cont’d)

High level plan

Expanded plan

Plan Generation

GUI test case generation can be modeled by hierarchal plans that doesn’t require conflict resolution

Planning GUI Test Cases Developing a representation for the GUI and its operators

Operator Derivation Process• Traverse the GUI and press on its components, the label is read off the

component label GUI events

• Menu open (File, Edit)• Unrestricted focus (Basic shapes in PowerPoint)• Restricted focus (Edit-Preferences in PowerPoint)• System-interaction (Cut and paste)

Planning Operators• System-Interaction• Abstract

Modeling Initial and Goal States and Generating Test Cases using algorithm

Generating multiple plans in PATHS Creating multiple linearizations of the partial plans Repeating the planning process, thus generating a different test case

Planning GUI Test Cases (Cont’d)

Planning GUI Test Cases (Cont’d)

Planning GUI Test Cases (Cont’d)

Planning GUI Test Cases (Cont’d)

Some Experimental Results

Multiple Tasks

Hierarchal vs. Single Layer

Questions