GUI Tetsing

Csci 565Spring 2009H.Reza

Event-based systemsGUI Testing

Concepts, strategies, etcThe Currency Conversion Program

To show unit testing for GUI To show integration testing for GUI To show system testing for GUI

What is GUI? Universal client alternative to traditional

forms/commands based user interfaces that accepts as input user-generated and/or system-generated events from a set of finite events and produces predicable graphical outputs

GUIs are organized in hierarchical fashion and Composed of objects: Objects has a set of fixed properties having discrete

values▪ Menu

▪ provide almost continuous availability of the various features of the systems;

▪ Buttons /keyboard shortcuts ▪ enable the user to navigate and access the various functions of their

application. ▪ User interacts and manipulate with objects using events (or

sequence of events)▪ Windows allow multiple windows to be displayed at the same

time ▪ Windows provides functionality using fields in which text or numeric

data can be entered ▪ Windows operate in two mode:

Modal (restrict the user's foucs and interactions to one window) Modalless ( does not restrict monopolozes the GUI interactions)

The GUI events can be classified as Restricted-focus events: open modal windows

▪ E.g. Set Language

Unrestricted-focus events open modeless windows▪ E.g., Replace in MS World

Termination Events : close modal windows▪ E.g., OK and Cancel

System- interaction events: interact with underlying software to perform some action▪ E.g., Copy event used for coping objects to the clipboard

GUI testing issues The event-driven nature of GUIs

▪ presents the first serious testing difficulty because users can click on any pixel on the screen and there so many of them

Unsolicited events▪ E.g., when the printer is out of papers

The infinite sequence of events (or permutations of events) GUI can be represented by Object oriented paradigm Hidden synchronization and dependencies (objects in the same

windows and different windows)▪ E.g., if a check box is set to true, a text box intended to accept a

numeric value elsewhere in the window may be made inactive or invisible.

Many events in/Many events out Testability (difficulty to specify, to execute and to analyze test cases)

▪ event-based system +hidden interactions▪ Coverage criteria??? (memon et al uses events, and events

sequences) Suffers from some of problems related to COTs (or precompiled

components)

There are serious legal issues related to both design and testing of GUI Legal issues

▪ Privacy/security ▪ E.g., access to the privileged/stored data

▪ Safety/ reliability▪ E.g., GUI for Safety critical systems can affect life-or-

death decisions (functionality and ease of use)

▪ Accessibility▪ Equal access for disabled users

Focus on errors to reduce the scope of tests

Separation of concerns (divide and conquer)

Use Test design techniques when it is applicable (e.g., blackbox)

GUI errors include Data validation Incorrect field defaults Mishandling of server process failures Mandatory fields, not mandatory Wrong fields retrieved by queries Incorrect search criteria Field order Multiple database rows returned, single row expected Currency of data on screens Window object/DB field correspondence Correct window modality? Window system commands not available/don’t work Control state alignment with state of data in window? Focus on objects needing it? Menu options align with state of data or application mode? Action of menu commands aligns with state of data in window Synchronization of window object content State of controls aligns with state of data in window?

Testing Strategies include Functionality

Unit testing (Buttons) Integrations Testing (synchronizations) System Testing (functional behaviors according

to requirements or functional testing) Non-Functionality(Quality)

Usability testing Security testing …

Model-Based Approach to test GUI requires: Identify components and Events Select a hierarchal formal model (e.g., EFSM)

to represent GUI elements (components) Intra vs. inter components Use model to generate threats or event

sequences of various lengths corresponding to visible & observable system outputs▪ Total number of events sequences increases when

the length increase▪ It might impractical to test for ALL possible event

sequences▪ be selective?

Unit and Integration testing minimally needed

Main focus is on System Testing Model?

▪ Event-driven Petri nets (EDPNs)▪ Testing threads

Event drivenUnit “button level” testingNot much integration testingSystem level testing: Models

UML Behavioral models StateCharts Finite State Machines

Starts testing (functional and structural) of user-supplied code for buttons Compute

▪ Valid and invalid data (e.g., non-numerical, etc) Clear Quit

Two approaches Using Drivers (good) Using GUI (system level for input/output

events)

Not too much of Integration testing Three possible options:

Focus on all logic in one place and use the value of the option buttons as conditions in IF tests

Use OO approach using methods invocation to exchange the rate value

Use visual Basic▪ Event procedures for each option button uses constant

value for specific exchange rate▪ (e.g. Private Sub OptEU_click () exchange =1.35 End

sub)

Heavily usage of System testingNeed event-driven model

Event-Driven PN (EDPN) to specify and to generate scenarios/threads

Tables 19.1 and Table 19.2

The process to generate threads (positive/negative scenarios) include Map the elements of CCP to EDPN

▪ Map input/out events to input /output events of EDPN▪ Map Data to EDPN data places▪ Map each Action to the transition in EDPN

Create EDPN for each ASF (tasks) Compose the EDPNs

▪ Use ASF graph

Generate a subset (or suitable?) of threads

How many threads are needed? Coverage criteria?

Reasonable coverage includes Test ALL port inputs Test All port outputs Test All ASF (tasks or requirements)

▪ Difficult to test each ASF Test Threats

▪ There are too many▪ Need to be selective

Definition: EDPN is tripartite-directed graph (P, D, S, In, Out) P: a set of port events D: a set of data places S: a set of transitions In: a set of ordered Paris from (P D) S Out: a set of ordered pairs from S (P D)

P, D, S correspond to Data, Input/output Events, Action,

Tuple (p4, d2, p12, p17,p21)

description

m1 (1,0,0,0,0) S4 enabled; s4 fired

m2 (0,1,1,1,1) Nothing enabled

m3

..

mn

Represent the snap shot of the simulating EDPN

Port Input Events P1: Enter US Dollar Amount P2: Click on Brazil P3: Click on Canada P4: Click on European Union P5: Click on Japan P6: Click on Compute Button P7: Click on Clear Button P8: Click on Quit Button

Output Events P9: Display U.S. dollar amount P10: Display Brazilian Reals P11: Display Canadian dollars P12 :Display European Community Euros P13: Display Japanese Yen P14: Display ellipsis P15: Indicate Brazil P16: Indicate Canada P17: Indicate European Community P18: Indicate Japan

Output Event P19: Reset selected country P20: Reset Brazil P21: Reset Canada P22: Reset European Community P23: Reset Japan P24: Reset equivalent currency amount P25: End application

Atomic System Functions (or actions) maps to transition S1: Store US Dollar amount S2: Sense Click on Brazil S3: Sense click on Canada S4: Sense click on EU S5: Sense click on Japan S6: Sense click on clear button S7: sense click on the Clear button S8: Sense click on Quit button

D1: US dollar amount enteredD2: country selected

The composition is not defined Place fusions?

▪ Looks like it Transition fusions?

Consider a set of threads T = {T1, T2, T3, T4} Scenario 1: 1) enter US amount, 2) select EU, 3) press

compute button, 4) press quit T1 = {s1, s4, s6, s8}

Scenerio2: 1) User enters US$, 2) Click on Brazil, 3) Clicks on Compute, Clicks on Clear button) T2 = {s1, s2, s6, s7}

Scenario 3: T3 = {s3, s1, s6, s7}

Scenario 4: T4 = {s5, s1, s7, s8}

In general the set of T covers (minim set of usages for CCP) Every action (or S) Every port input (every input data and input event) Every port output (every output data and output event)

Next-level user behaviorCommon usage scenario

Converts amount to all 4 currencies, clears and quits

T5 = [s1, s2, s6, s3, s6, s4, s6, s5, s6, s7, s8]

Unexpected user behavior (negative scenario) Neg-Sencario1: User changes his/her mind

about currency T6 = [s1, s2, s3, s4, s5, s6, s7, s8]

unexpected behavior (negative scenario) Neg-Scenario2:User toggles between 2

countries, then quits T7 = [s1, s2, s3, s2, s3, s2, s3, s8]

…

Unit testing & integration testing not important

System testing most complicated test.

ASFs and EDPNs can help identify and validate test cases Valid usages (positive scenarios) Invalid usages (negative scenarios)

How to use MBT to test GUI? How to measure the benefit and cost of using MBT

in GUI? Usability testing/evaluation: requires specified

requirements or formal models of GUI Expert reviews

▪ Heuristic evaluation▪ Guidelines review▪ Consistency inspection▪ Cognitive walkthrough▪ Formal usability inspection

Usability Testing and Laboratories Surveys Instruments Acceptance tests Evaluation during Active Use

Software Testing by P. JorgensenModel-Based Testing by Utting