Testers Desk Presentation

A Technical Presentation on

TestersDesk.com

The One-source Online Toolkit for Software Test Design and Test Data Generation.

What is TestersDesk.com?

• You have 10000+ possibilities of executing a test with different combinations of values from its Test Parameters

• You want 500 e-mail IDs to test something• You want to generate a 6 MB file with Alphanumeric

characters as your test input• and such…

TestersDesk.com provides Computer Aided Software Testing through an innovative Toolkit of System-independent tools and utilities that improve Test Productivity in small and big ways

It targets "Test Design Automation" and is complementary to "Test Execution Automation" tools currently in use

Some Questions

• Let us say there are 3 test inputs A, B and C. Let us further say that A has 2 critical classes of input values, B has 3, and C has 4 - How do you build/visualize the table of decisions that show all the combinations (2*3*4 = 24) in multiple rows? Manually prepare it in Excel?

• Let us say your test can be configured with 5 test parameters that can be any of 10 different values each, and you have a total of 105 i.e., 1 million ways to run the test – while you use experience to identify the key tests, how do you get Test Coverage with reasonable acceptance? How do you reduce 1 million to a manageable number?

Some more questions

• You want to generate a file of size 10mb with binary data in it, how do you create it?

• Say you want some test data like 1000 real person names, 2000 fake e-mail ids in valid format, a few hundred timestamps in the date format and timezone you choose…what would you do?

Certain testing activities are imposed on us by the technicalities

TestersDesk.com tries to off-load a lot of them and enable testers to focus on the real testing activities

The online toolkits

• Test Design Toolkit with tools to help software functionality test design activities

• Test Data Generation Toolkit with tools to help software testers to help quickly prepare test inputs

• Miscellaneous Toolkit with a collection of uncategorized tools

Technical Features

Rich Internet Application – uses AJAX and interoperable data formats such as CSV/XML

Completely online – Web based tools for easier access (keeping the test machines free from maintaining several tools)

Unicode support (input can be in any language, and so will the output be)

Requests can be made through screens as well as file-uploads

Output displayed inside the browser (which can be copied) and through downloadable files

Description on each tool/feature in the toolkits

TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ Pairwise TestCase Generator

The SituationMany test cases are influenced by multiple test parameters, each with several significantly different inputs (or equivalence classes). Typical problem arises when they have to be executed covering all possibilities/combinations.

An ExampleThink that a registration page has to be tested which has four fields- Username, Password, E-mail, Receive_notifcations. Let us say that Username can have two types of valid classes on inputs and three classes of invalid inputs. Similarly let us say that Password can have four valid classes three invalid classes of inputs. E-mail, let us say, can have four valid classes and two invalid classes on inputs. Receive_notifications can be Yes or No. Let us say that all these fields have to be tested in three browsers IE6, IE7 and FF3. Here we are talking about each input having some significance in the system behavior.

To summarize, following is what we are looking at as the Test Parameters that drive the registration tests.

Username: Valid1, Valid2, Invalid1, Invalid2, Invalid3 Password: Valid1, Valid2, Valid3, Valid4, Invalid1, Invalid2, Invalid3 E-mail: Valid1, Valid2, Valid3, Valid4, Invalid1, Invalid2 Browsers: IE6, IE7, FF3 Receive_notifications: Yes, No

If we were to test all the combinations of values of the above Test Parameters, we are talking about 5 * 7 * 6 * 3 * 2 = 1260 ways to configure test inputs. There is a clear need to reduce this number without compromising Test Coverage.

Benefit from TestersDesk.com

Welcome to Pairwise Testing (if you are not already enjoying its virtues). We generate a very reduced number of Test Cases that cover all the two-way interactions between all the values of all the test parameters.

Each value of each test parameter talks to each value of all other parameters at least once.

Now, that's intelligent test design. The list produced by our tool, and the additional tests identified by your judgment/experience yield the maximum number of defects in the shortest possible time.

For the example presented in the previous slide, we provide as little as 42 Test Cases that guarantee Pairwise Coverage. Check it out!

Pairwise TestCase Generatortest combinations can be reduced

TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ T-way TestCase Generator

The Situation

Extending the previous problem, in certain rare cases, one might want to not only cover 2-way interactions but also target 3-way interactions to gain more confidence, i.e., all combinations of three values of all the test parameters interacting at least once in the produced list of Test Cases. Research has shown that 50% to 90% interaction errors are found by Pairwise TestCase Generator itself, but in cases where there is an absolute need, the coverage strength can be increased to 3 (or 4 or 5) -way combinations of the Test Parameters.

An Example

In testing of a mission-critical Marine Control System that controls crew on the ship, communication devices etc, it may be desired to increase the Interaction Coverage from 2 to 3. This, of course, is in addition to judgment/experience based tests.


Much like the Pairwise TestCase Generator, the Test Parameters and their respective values are collected to generate a reduced set of Test Cases that guarantee Interaction Coverage of 3, 4 or 5 -way combinations.

T-way TestCase Generatortest combinations can be reduced with

increased coverage

TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ N-way/Random TestCase Generator

The Situation

Test Coverage is a serious problem with the decisions to be made in executing a test. This has been conceptually solved by the Decision Table matrix formation technique that covers all the possibilities and lets testers mark the appropriate action for each decision set. The problem with this approach is that it tends to grow horizontally as the decisions and their options grow, making it infeasible to use it as part of the Test Case Documentation. Further more, manually prepared decision tables are tough to be fed into Test Automation tools like QTP, for example, and call for re-work.

An Example

Let us there is a business function "Calculate Monthly Premium" in a Health Insurance Application where there are three decisions that influence the outcome, namely, Person Age Group, Gender, and Health Index. Let us also say that Person Age Group can be one of G1, G2, G3 and G4. Gender can be male, female. And Health Index can be 1 to 10 (and each of them has a difference influence on the calculated premium).

Now let us say tests have to be prepared covering all the possibilities of sets of options chosen for the respective decisions. We are looking at 4 values of Person Age Group * 2 values of Gender * 10 values of Health Index, a total of 80 possibilities. It may not be very desirable to have 80 columns in a decision table.


Let us say that the following is given as input to TestersDesk.com's N-way TestCase Generator.

Person Age Group: G1, G2, G3, G4 Gender: male, female Health Index: 1,2,3,4,5,6,7,8,9,10

A vertical matrix is created, each row representing a set of possibilities. So we have just 3 columns and 80 rows. This data can be pasted in a spreadsheet, and a 4th column can be added for the "Expected Behavior". This way you have the Test Case Documentation done in an accelerated way. If a particular row has no significance, you can delete it. Try it out!

Further more, such a table can be easily integrated into test automation scripts developed using data-driven approaches.

N-way/Random TestCase Generatordecision tables for test coverage can

easily be generated

TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ N-way/Random TestCase Generator (with random option)

The Situation

Occasionally, there is a need to randomly pick test combinations without following any specific scheme. This is to test an application completely based on a random data set to study its behavior.

An Example

In game testing, while making the choices, most of them are based on individual gaming styles. So, even after testing in all known ways, there still exists a need to perform certain random tests.


TestersDesk.com collects the Test Parameters, their values and produces randomly picked combinations of values (about 10% of the total).

N-way/Random TestCase Generator (with random option)

test cases with random combinations can be selected

TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ Subset TestCase Generator

The Situation

Often times, while choosing multi-input test values, the problem of selecting a subset of test items from a larger list arises. There is a need to see all possible subsets of a given size from the list, and then select the needed subsets from that.

An Example

A 7-day week company's payroll application works differently based on which of the 5 days in the calendar week a given employee has attended the office. So, we need to select all possible ways of taking 5 days from 7 days.


TestersDesk.com generates all the subsets from the larger set by taking two simple inputs- the complete list of possible test items, and the size of the subset (i.e., "how many at a time"). For the example discussed here, there are 21 ways how 5 items can be picked out of the 7 test items.

Subset TestCase Generatorall subsets of test inputs from a larger

set can be generated

TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ Boundary TestCase Generator (BVA++)

The Situation

General theory of Boundary Value Analysis talks about testing the boundaries of a single test field, and when there are multiple range/list based fields, there is a need for a method to cover the interactions between their boundaries (instead of testing boundaries in isolation).

An Example

There are three fields on a screen each of which take a numeric range. Each individual field has been tested with the seven tuples (values) viz. min-1, min, min+1, mid, max-1, max and max+1. The expected behavior was achieved, however there is no guarantee that the "interactions" between the seven tuples of all the fields together work as expected.

What if there is an error that surfaces only when the min+1 value of field #1 and max - 1 of field #2 occur together in a single test - does our test suite has test cases to catch such defects?


By collecting range/list information about test fields, this tool applies pairwise coverage and a few other techniques to generate a list of test cases that cover the interactions between the boundaries (for example, the minimum value of field #1 interacts with maximum value of field #3 at least in one of the generated tests and so on).

Boundary TestCase Generator (BVA++)

multi-field boundary value analysis tests can be generated

TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ TestCase Permutations Generator

The Situation

Change in order of actions can lead to change in behavior. It is a usually agreed fact in the way how software applications work. Testers have the need to change the order of key test actions to see if they cohesively work well-- some call them as Scenario Tests, Test Stories, Act-like-a-customer tests and so on... But in what ways can the order of 4 tests be changed – can we see a list of possibilities and select what we want?

An Example

Let us say that a banking application works perfectly in showing the balance money when the test actions are done in the following sequence-

Change Password-->Check Balance--->Transfer Funds

Let us hypothetically say there is a conflict on the data updating process when balance is checked after changing the password, then the following sequence may not work-

Transfer Funds--->Change Password-->Check Balance


You can provide the 3 or 4 critical test actions and get a list of all possible permutations (changes in order) of test execution. This does not necessarily formulate the actual Scenario Tests but at least helps you in visualizing what are the possible orders that the key test actions can take, and lets you pickup the interesting rows.

TestCase Permutations Generatorpermutations (changed orders) of

critical tests can be generated

TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Common Test Data Generators |____ Language String Generator

The Situation

Modern day applications are required to accept inputs in multiple languages. Testers face the problem of quickly preparing strings in different languages, with the specified lengths.

An Example

An application needs UTF-8 based input consisting characters in all European languages.


By letting the user choose from a list of different languages, TestersDesk.com generates strings in that language (in the specified length and quantity).

Language String Generatortest input strings in multiple languages

can be generated

TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Common Test Data Generators |____ Custom Syntax String Generator

The Situation

Not all Test input fields are insensitive to the format of the data provided. It is a common need for Testers to prepare data in the format/syntax as needed in field's specific context.

An Example

1. The US Phone number fields need data in the format 999-999-9999 (where each '9' can be a number 0 to 9).

2. An application has in ID field that needs 4 alphabets, followed by 6 numbers, and than one more alphabet.


By letting you choose from a symbol table, TestersDesk.com allows a Custom Syntax to be defined, and then generates as many test values as needed according to that format.

Custom Syntax String Generatortest input strings with a custom syntax/format can be generated

TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Common Test Data Generators |____ Quantified String Generator

The Situation

In the daily life of a typical Software Tester, different test fields are needed be tested for length requirements. Preparing these test strings with the specified lengths, determining whether they are accepted completely or got truncated at a wrong position, and other such related work is, by large done manually, and eats away a lot of during test execution (especially since there is no repeatable technique of preparing strings with specified lengths).

Further more, the strings we use to test the length of a field are not self-descriptive and need to be again pasted back in a text editor to analyze where it got broke (if I type a 100 A’s and only 99 got pasted in my field, I cannot know that unless I again copy the value from the field into an editor).

An Example

A field should not take 30 characters. The tester types 5 times the letter 'a' in a text editor and pastes it six times. It is then pasted into the field, and now copied back into the editor to see all the characters are pasted or not. Then the same procedure is repeated for an invalid value like 31 characters and so on. Some testers use marker characters in end but the preparation process still remains big.


TestersDesk.com generates what it calls as Quantified Strings - Strings that can indicate the position of a character anywhere within. For the previous example, such a string will look like "$3$5$7$9$12$15$18$21$24$27$30$" - each number in the string specifies the position of the next $ character. Suppose let us say when this is pasted, we see only "$3$5$7$9$12$15$18$21$24$27$30", it is of immediate observation that it got truncated at 29, a typical boundary bug. 30 is saying that the $ that follows it is the 30th $, but we do not see it- concluding that it broke at 29.

You can change the $ character to any other character you want.

Quantified String Generatorquantified strings can be generated to

simplify testing field lengths

TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Template-based File Generator

The Situation

The Application Under Test takes a file as test input. This file has several fields in a specified format. There is a need to prepare multiple files each having different test data.

In the end you have, let us say, a large number of files with the same format and only data varies in each of them. In the next new build, let us say, the format has been changed or is added with place for a new field. This calls for individual opening and editing of each such file.

Note: EDI, WebServices, XML, Configuration files... there are many cases where the test input is a file with a specified format.

An Example

1. An EDI file has to be sent to a partner system and needs multiple files with varied test inputs

2. A Properties file has to be prepared as a test input. While the names remain common in all files, values can be changed in different ways

3. A Web Service needs to be tested with several XML files. The format of each file is same and only data changes.


By taking two input files- One a Template file that shows the format of your test input file that has placeholders where data needs to be embedded, and the other with several rows of test data, a single ZIP file is generated that has as many files as the number of rows in the CSV Test Data file provided. Each individual file represents each individual row of test data. The advantage is that many files can be generated in one-go, and editing and other maintenance work is done on just two files.

Template-based File Generatormultiple test input files can be created with just one template and one data file

TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Size-based File Generator

The Situation

All of us know that test inputs are not always just field level values, and it is a common test need to have files of the specified size and type.

An Example

1. An E-learning application is being tested and its course assets module needs files of different sizes

2. A CD-Rom burning feature of a Publishing Application is being tested, and needs a 700 MB file with binary data

3. An Upload resume feature of a Job Search Application is being tested, and needs two files of sizes 5 MB and 10 MB.


By providing three inputs- File size, Measurement unit (KB, MB, GB) and Type of data (Alphabets, Binary etc) a tiny batch file can be downloaded. This batch file creates the required file when executed on the test machine, in the same directory where it is executed from. After test execution if completed the generated file can be deleted to save space, and can be re-created by re-executing the batch file whenever needed. Currently this feature works on Windows.

Size-based File Generatorfiles with the specified data and size

can be generated

TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Date/Time (Timestamp) Generator

The Situation

It is a common test input requirement to provide a date/time value to an Application Under Test. Sometimes, several values may be needed instead of just one. It becomes tedious to prepare them manually.

An Example

An E-greeting cards application is being tested, and needs several test dates in a particular month or a particular range; and in a specified format.


By providing Timestamp format, and optionally timezone information, the specified number of timestamps are generated.

Date/Time (Timestamp) Generatordifferent timestamps can be generated

TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Password Test Generator

The Situation

A password field needs to be tested against a set of test requirements, and there is a need for both valid and invalid values.

An Example

There is a password field in a web application's registration form and it is specified that it:

should have the minimum length as 6 and maximum length as 20 should not have spaces special characters are allowed


By collecting some inclusion/exclusion options and length requirements, a set of valid and a set of values are generated that can be used as test inputs in testing the requirements of a password field. We are adding more options soon.

Password Test Generatortest inputs to test password fields can

be generated

TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Luhn (Mod-10) Number Generator

The Situation

A variety of identification numbers, such as credit card numbers and Canadian Social Insurance numbers need to comply with Luhn formula, also known as mod-10 check. Testers need test inputs covering both valid as well as invalid Luhn (mod-10) values to exercise the Application Under Test.

An Example

A shopping cart is being tested, and the checkout functionality needs sample Credit-card numbers as part of the test input.


By configuring an optional prefix (like 4111 for visa) and length of the number, the specified number of values can be generated that adhere to (or violate) the Luhn (Mod-10) check.

Luhn (Mod-10) Number Generatortest credit card numbers can be

generated

TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Person Name Generator

The Situation

Many applications need person names as test inputs. While strings like xyz3 and abcd9 also qualify technically, the output may not look good, and you may want to see real person names.

An Example

A payroll application is being tested, and its reporting functionality has to be validated. There is a test requirement to have real names instead of random strings.


By collecting options like gender, format and quantity, TestersDesk.com generates real person names.

Person Name Generatorreal person names with the specified

quantity can be generated

TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Universal Unique ID Generator

The Situation

Let us say that a test system needs certain test inputs in a screen, and one of the field has to be globally unique, meaning it should have never been used before any time on any machine, you would need a tool that helps generate such values.

An Example

A licensing module of an application needs to be tested, and one-time unique values are needed to execute several test cases, in each test cycle.


TestersDesk.com provides a simple to use feature that just asks for the quantity needed, and generates IDs that are universally unique.

Universal Unique ID Generatorunique IDs can be generated

TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ E-Mail Address Generator

The Situation

There usually are many cases where a particular test may need a lot of dummy test e-mail IDs in different sizes and types.

An Example

an Address-book functionality is being tested and 100 E-mail IDs are needed to be loaded into it to test Display, Sorting and Pagination features

a Mailing-list functionality is being tested and you need a 1000 E-mail IDs to test its bulk insert feature

a performance test needs to bombard a huge number of E-mail IDs into the Application Under Test


By taking a few options like whether you want only E-mail IDs ending with "dot com" or "all types", whether the generated random values should be new or from a previous seed, different download-ready E-mail IDs are generated.

E-Mail Address Generatortest e-mail addresses can be

generated

TestersDesk.com Online Community Edition |_ Miscellaneous Testing Toolkit |____ SQL Injection Test Generator

The Situation

Malicious test inputs can be crafted to trick the SQL logic running in applications and compromise Application Security (especially when the user supplied inputs are consumed in building SQL). SQL Injection has stood out as the most popular vulnerability, generally in multi-user applications like Web Applications. It has become mandatory to test the applications for such vulnerabilities, at least at the entry-level pre-login screens. But to perform such tests you either need expensive tools or really tech-savvy testers on board.

An Example

A web application provides activation licenses to a game by taking an order ID. Traditional tests include testing it with valid order IDs, empty fields, lengthy order IDs, and so on... All such tests, let us say, have yielded expected results. There is a chance that it has one major test uncovered - that is whether or not SQL can be injected into the field to trick the server-side logic and compromise Application Security.


TestersDesk.com collects the basic information about a Web Form (its action URL, field names with some dummy values, and the target_field on which you want to perform the SQL injection) and then crafts automated forms with different types of SQL injections, and presents a single downloadable HTML file. Once downloaded, it can be executed on your application and the content within the page may reveal potential security holes in the Application Under Test.

SQL Injection Test GeneratorSQL injection security tests on web

forms can be automated

TestersDesk.com Online Community Edition |_ Miscellaneous Testing Toolkit |____ OS Information Scripts

The Situation

In several instances, it is required to gather OS level information before or after test execution on a particular test machine. This information could be related to the filesystem, software, processes, hardware and so on.

An Example•A bug report needs to include information about the drives (harddisk drives, cd drives etc) present on the test machine

•The tester wanted to execute a test case and see what are the processes currently running on the machine, and send that list somewhere

•There is some problem found in an installation folder on the test machine and information with all attributes was asked for

•An unknown conflict has occurred on a test machine, and the developer asks the tester to send the list of the software programs installed on that machine


TestersDesk.com provides several simple VB Script files that can be downloaded and executed from the command line. All of them produce readable output that can be copied into outgoing test reports, bug reports etc. In future scripts for other Operating Systems will be available too but currently they are available for Windows.

OS Information Scriptssystem-dependent information scripts

can be downloaded (for windows)

More to come!

Testers Desk Presentation

Technology

test coverage

test productivity

software test design

test design automation

multiple test parameters

test machines free

reduced number of test

thats intelligent test