Stories from the Trenches Seven Productivity Improvements for Test Automation Frameworks Presented By Chris Petrov Leverage Point Corporation SCQAA Inland.

Stories from the Trenches

Seven Productivity Improvements for Test Automation Frameworks

Presented By

Chris PetrovLeverage Point Corporation

SCQAA Inland Empire Chapter May 8, 2008

Leverage Point Corporation417B W. Foothill Blvd. #164Glendora, CA 91741626-857-9446www.levpoint.com

Leverage Point Corporation

• Founded in 2006• Specialize In Software Quality Assurance

Consulting and Staffing• Outsourced Testing• Test Automation Framework Development• Testing Project Support

"We will provide the best people, both technically and in customer focus, to deliver specialized high-value and indispensable IT services. Each of our employees will maintain a singular focus on using all of their experience, skill and creativity to ensure the quality, performance and reliability of our customer’s critical business applications.”

Productivity Improvements

• Faster Learning • Faster Test Creation• Reduced Maintenance • Faster Analysis• Flow

Productivity Improvements• Give the Users a Familiar Test Creation

Environment• Supply List of Available Test Actions

• Lightweight Interface• Intuitive vs. Usable

• Static Test Case Validation• Build Compartments• Improve user experience

Project Profile

• Clients Background • 2 Independent Organizations• Similar Applications• Compressed Test Schedules • Limited Application Availability• Different Framework Users

– Testers with no automation experience and no application experience in one

– Business analysts with application experience, but no testing experience in

the other

Project GoalsCreate a Keyword Driven Framework • Fast and Easy Test Case Creation• Require Minimal Training for Users with no

Automation Experience• Tests Cases Creation Before the Application is

Available• Frequent Incremental Deliverables

Automation Challenges• Complexity• Maintenance• Extensibility• Robustness• Learning Curve• ROI

Best Framework• What are your goals?• What are your products?• What are your users?• What is the culture?• What lies in the Future?• Trade offs

Framework Design• Repository• Engine• Documentation• Front End

Keywords Selection• Keyword Selection Options– Many (200+) Keywords Based on Business

Scenarios– Few (20-) Low Level Keywords– Combined Approach• Low Level Keywords• High Level Action available as executable scripts• Script Areas (Libraries)

Keywords Selection• Selecting Limited Amounts of Actions Available to the User

– 11 Frequently Used Actions• Enter• Click• Select• Run Script• Validate

• Hide Control Knowledge– It looks like a button, but it is a link – Click vs. Click Button

• Action Overloading -Using the same action for different controls• Click Link, Button, Checkbox• Select Radio Button, Dropdown, Checkbox

• Hiding Complex Flows Behind Scripts– Identifying the common business processes– Getting the best of both worlds

Keywords Selection

GUI Map Organization• Create Maintainable and User Friendly GUI Map• Map Interface for Application that is not yet

Available• Make the GUI Map customizable for the intended

product• Building Object Hierarchy from Usability Point of

View– “Menu -> Screen -> Frame -> Field” Approach– “Menu -> Screen -> Field” Approach

• GUI Map that allows user friendly naming convention• Removes the confusion for the tester

GUI Map Organization

GUI Map Organization• Object Naming Conventions

– Name the object exactly as it is on the screen– Screens – how do we name the screens to be recognizable even by

users that don’t know the application– Objects with the Same Label – how do we differentiate

• Change the name of both objects, to avoid misidentification• Add the Frame name after the label• Add the Action name after the label

– Objects without label• Use the Help file descriptions• Use Purpose + Default Values

– Grids• Column Name: Grid Name

Give the Users a Familiar Test Creation Environment

• No Testers interaction with the Test Tool• Easy to use• Familiar To your users• Domain Experts could write automated test cases• Fast Test Creation• Minimal Effort• Copy and Paste

Supply List of Available Test Actions• Problems when typing names

– Spelling (Disbursments - Disbursements)– Misidentifying Controls ( Address 1 - Address)– Using Wrong Duplicate Controls ( Zip Address - Zip Information)– Abbreviations (Addr – Address)– Ambiguous Entries – (Name - Last Name or First Name)– Partial Entries – (Last – Last Name)– Single/Plural– Remembering the control Name

• Dropdowns– Based on Menu, have only the Screens available for this menu as a choice– Based on Screen, have only the Fields children of that menu available in the

dropdown– Update the dropdown values dynamically– Demo or Movie

Framework Front End Demo

Lightweight Interface• No wizards• Playing Music• Performance Measurements

– Initial Prototype with 1000 screens – 45 sec– Redesigned Prototype with 1000 screens – 2 sec

• Identify Time Required for Specific Actions– No delays between actions

• Screen Frame Field or Screen Field Measurements

Intuitive vs. Usable• Difference• Frequency of Use• Design that Reduces Test Case Creation Time

– Screens by Tab Order – because the application already is measured for usability

– Fields by Alphabetical order• Tab Order Changes• Duplicate Controls are next to each other• Easier to find the control that you barely remember

– Keywords by frequency used

Dropdown Usability Design• Performance Measurements

– Initial Prototype with 1000 screens – 45 sec– Redesigned Prototype with 1000 screens – 2 sec

• Identify Time Required for Specific Actions– No delays between actions

• Screen Frame Field or Screen Field Measurements• Design that Reduces Test Case Creation Time

– Screens by Tab Order – because the application already is measured for usability

– Fields by Alphabetical order• Tab Order Changes• Duplicate Controls are next to each other• Easier to find the control that you barely remember

Static Test Case Validation • Static Test Case Validation• Correct many errors before the test case is executed

– If possible fix the problems before, not during the test• Identifying Frequent Errors

– During the Initial Design– Root Cause Analysis of Errors in Existing Tests

• Adding Validation for Frequent Errors• Visual Indication of the Defect with Correction Advice• Formatting the report with the corresponding Font styles and colors• Error Report with identification of the file, sheet and the row the

error occurs• Single feature that reduced the Automation Debug Time by 40%

Static Test Case Validation

Compartments • Hide Complex Flows• Isolate Volatile Areas• Isolate Frequently Used Actions• Specify the Essential• Keep default values

Improve user experience • Continuously improve productivity– Watch the users– Improve the experience– Actively Eliminate problems– Flow

• Build the needed tools

Productivity Tools • GUI Map Builder– Build Complete GUI Map with Single Click

• “Where Used” Maintenance Tool– Update Test Cases when the GUI Changes

• Traceability Matrix with Integrated Progress Reports and Trends– One-Click Reporting

• UAT Builder– Build “Bare Bones” Test Cases from a List of

Productivity Tools

Framework Updates• Designed for Unobtrusive Updates• Server Location• Single Update File• Notification that New Version is Available

Data Driven Add-in • Benefits and Limitations of the Data Driven Tests– Fast Design– Great for Discreet logic when equivalence partitioning is

used– Require Advanced User to write and parameterize them

• Getting the Best of Both Worlds– Build the test using the keywords– Parameterize them on the fly with the build in variable

functionality

• Running Tests from the Design Sheet– Demo or Movie

Data Driven Add-in

GUI Unit Test Add-in • Testing GUI Screens from the Specification• Learning Curve• Integration with the Keyword Driven

Approach– Changing the Paradigm

GUI Unit Test Add-in

DB Query Engine Add-in • Challenges

– Data Control of the Test Artifacts– Testing Converted Data– No Application Database Refresh Available– Ever Changing Data– Complex Database Structure– Different logic based on record data

• User Friendly Implementation– Hide the Complex SQL Statements behind user friendly names– Change the where clause on the fly– Mark the record as used, so it is not locked by other test case

Accomplishments• Established Sustainable, Scalable Automation

Solutions• Reduced Test Creation Time by 50%• Reduced the Test Automation Debug Time by

40%• Learning Curve of 2 to 8 Hours

Stories from the Trenches Seven Productivity Improvements for Test Automation Frameworks Presented By Chris Petrov Leverage Point Corporation SCQAA Inland.

point corporationfounded

leverage point corporation417b

usability point of viewmenu

application experience

test automation frameworks

testing experience

business scenariosfew

business analysts

Documents

Rajko Petrov Nogo

SCQAA-SF Meeting on May 21 2014

AUTHORS FR: PETROV, A.D. TO: PETROV, B.F. · Title: AUTHORS...

The Petrov affair

ALAM+PETROV 2014 BLUMEN

UControl Stoyan Petrov Ivan Petrov Sajib Saha John Kenyon 1....

Meshless Local Petrov-Galerkin Method for Bending...

Understanding and managing perception workshop scqaa valley...

SCIENTIFIC ABSTRACT PETROV, N.N. - PETROV, N.P. · Title:.....

Petrov 2804

SCIENTIFIC ABSTRACT PETROV, G.P. - PETROV, I. · title:...

Vasil petrov-2014-1

GBE - ::: The Petrov Lab

Vassil Petrov AIA (abridged)

Petrov Maksym

Petrov Charity Match Programme