Test Design Best Practices By Gabriel Rodriguez. Copyright 2006-2007. MSQAA Federation Chapter. Agenda Test Design Test Data Test Matrix Test Cases Test.

Test Design

Best Practices

By

Gabriel Rodriguez

Copyright 2006-2007. MSQAA Federation Chapter.

Agenda

Test Design Test Data Test Matrix Test Cases Test Scripts Test Scenarios E2E Scenarios Q&A Reference


Test Design


Test Design

Test design is the major function of a testing team. This activity begins not when a software build is placed

into the hands of the testing team, but as soon as the first requirements have been approved and baselined.

As requirements and system design are refined over time and through system-development iterations, so are the test procedures and test cases refined to cover the new or modified requirements and system functions.


Test Data


Test Data

The concept of test data is a simple one:– Creating representative processing conditions using

test transactions.– The complex part of creating test data is

determining which transactions to make test transactions.

– Experience shows that it is uneconomical to test all conditions in an application system.


Test Data

Test File Design:– To design an adequate test file, the tester must be

familiar with system flowcharts, operating instructions and other documentation.

– A test file should use transactions having a wide range of valid and invalid input data:

Valid data for testing normal processing operations.

Invalid data for testing programmed controls.


Test Data

Test of normally occurring transactions– For Example:

Entering Alphabetic characters in text fields. Entering Numeric Characters in Calendar Fields.


Test Data

Test using invalid data– For example:

Entering Alphabetic characters when numeric characters are expected and vice versa.

Using invalid account or identification numbersEntering negative amounts when only positive

amounts are valid and vice versa.Entering transactions or conditions that will violate

limits established by law or by standard operating procedures.


Test Matrix


Test Matrix

A Test Matrix is used to track a series of test cases that are essentially the same.

A Test Matrix is a tool to control the number of test cases created for the application, usually this document is used as an evidence of historical data.

A Test Matrix is a very effective way of displaying and tracking the progress of test.


Test Matrix

The Structure of a general Test Matrix:– The Rows display the Functionality or (Objects)

under test.– The Columns display the test to be run against the

objects.


Test Matrix

The process of defining the Test Matrix:– Enter the Functionality in the horizontal axis of the

test matrix.– Enter the test to be run on the vertical axis of the

test matrix.– Match up the test to be run with the functionality in

the intersection cells enter notes that briefly. describe how the test to be run will be met

– Review the test matrix at a high level to determine redundancy and coverage.


Test Matrix

The next slide will show an example of a test matrix, please have the following points in mind while reviewing the test matrix in the next slide:– Every organization has a way to structure a test

matrix.– Due to the lack of knowledge of what a test matrix

is, some organizations design this artifact not in a very effective way, causing rework for the Software testers.


Test Matrix


Test Matrix

Description of the Test Matrix Example (Previous slide):– Top Columns:

Test To Be Run – This refers to the total of number of test planned to be run. The screenshot shows:

– PC001 - This was the name given to the future test case, in other words our nomenclature was established in that way.

– Control testing - This is the testing type to be implemented


Test Matrix

Description of the Test Matrix Example (Previous slide):– Left side

Functionality - These are the features to be tested, usually these features are gathered from Requirements doc or Functional specs. There is some test data specified under the features to be tested such as ‘Special Chars’, “Alphanumeric Chars’ this data is used to create different combinations of test cases.

– Center side of the spreadsheet These are all the combinations of test cases that can be

designed, the ‘X’ means that the test case will have to be designed with the features on the left columns.


Test Matrix

Granularity of the Test Matrix:– The granularity of the test matrix needs to be

sufficient such that the QA team can answer the following questions:

Where did we test this piece of functionality?What do we need to run in order to get the

coverage we want?What coverage did we get with this particular

scenario?


Test Cases


Test Cases

What is a Test Case?– IEEE Standard 610 (1990) defines test case as

follows:“(1) A set of test inputs, execution conditions,

and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.


Test Cases

– Quality Assurance Institute (QAI) defines test case as follows:

Documents the actual values used for input along with the anticipated outputs.

– Definition of Test Case from www.pcmagA set of test data and test programs (test scripts)

and their expected results. A test case validates one or more system requirements and generates a pass or fail.


Test Cases

What are we trying to learn or achieve when we run test cases?

Find DefectsMaximize bug countHelp Managers make ship/no-ship decisionsAssess conformance to specificationVerify correctness of the productAssure Quality


Test Cases

Characteristics of a good test case:– It has a reasonable probability of catching an error.– It is not redundant.– It’s the best of its breed.– It is neither too simple nor too complex.


Test Cases

Blackbox Testing Technique– When creating test cases, the input data used is

critical. Three successful techniques for managing the amount of input data required include:

– Equivalence Partitioning– Boundary Analysis– Error Guessing


Test Cases

Equivalence Partitioning– An equivalence class is a subset of data that is

representative of a larger class. Equivalence partitioning is a technique for testing equivalence classes rather than undertaking exhaustive testing of each value of the larger class.

For Example: A program which edits credit limits within a given range ($10,000 - $15,000) would have three equivalence classes:

– Less than $10,000 (invalid)– Between $10,000 and $15,000 (valid)– Greater than $15,000 (invalid)


Test Cases

Boundary Analysis– A technique that consist of developing a test cases

and data that focus on the input and output boundaries of a given function. In same credit limit example, boundary analysis would test:

Low boundary plus or minus one ($9,999 and $10,001)

On the boundary ($10,000 and $15,000)Upper boundary plus or minus one ($14,999 and

$15,001)


Test Cases

Error Guessing– Based on the theory that test cases can be

developed based upon the intuition and experience of the Test Engineer.

For example , in an example where one of the inputs is the date, a test engineer may try February 30, 2000 or 9/9/99


Test Cases

The following documents are needed in order to design test cases:– Requirements Document– Use Cases (When applicable)– Functional Specification Document– Design Specification Document– Test Plan– Test Matrix

This document is created to have better control on the combinations of test cases.


Test Cases

The following slide shows an example of a Test Case Template. Please have the following points in mind:

– Every organization designs its test case template to meet its needs.

– There are test case templates that are more effective than others because some of them have a more structure way to organize the test cases.


Test Cases


Test Cases

Tips to write test cases:– For Action Steps:

Use the following short words:– Click on (Buttons, Links)– Select (Radio Buttons, Options from lists)– Move to (Next Page, Next Window)– Navigate to (Next Page, Next Window)– Check ( Check Boxes)– Type in ( Text Fields, Message Boxes)


Test Cases

Tips to write test cases:– For Expected Results

Use the following statements:– Verify the following…– The following page will be displayed…– The following error message will be

displayed…


Test Scripts


Test Scripts

QAI Definition of a Test Script:– Note: Generally there is a confusion between a test

script and a test case. A Test script are step-by-step instructions that

describe how a test case is to be executed.Scripts should specify an order of actions that

should be performed during a test session The scripts should also contain expected results. Test scripts may be manually prepared using paper

forms, or may be automated using capture/playback tools or other kinds of automated scripting tools.


Test Scripts

Definition of a test script from www.pcmag– The instructions in a test program. It defines the

actions and pass/fail criteria. For example, if the action is "to enter a valid account number," the expected result is that the data are accepted. Entering an invalid number should yield a particular error message.


Test Scripts

Test Script Example


Test Scenarios


Test Scenarios

A set of test cases that ensure that the business process flows are tested from end to end.

They may be independent tests or a series of tests that follow each other, each dependent on the output of the previous one.

The terms "test scenario" and "test case" are often used synonymously, however there is a difference between them.


E2E Scenarios


E2E Scenarios

E2E stands for End To End Goal of E2E Scenarios:

– End-To-End (E2E) Testing is a QA activity whose primary goal is to verify interaction points between multiple features and/or systems using real-world user processes and data workflow.

– By simulating these real-world scenarios, E2E Tests increase the quality of the applications in the areas most critical to our customers.


E2E Scenarios

Goal of E2E Scenarios:– A secondary goal of E2E Testing is to find important

usability flaws that affect the ability of the application to deliver its intended value.

– By exercising the software in a manner that is similar to how customers will use the software, macro-level usability issues will surface and can be evaluated.


E2E Scenarios

The idea behind adopting E2E Testing is to help close gaps in the testing model.

Standard UI test cases focus on testing functionality of a particular feature area, but tend to overlook the interaction points between multiple features and/or systems. E2E Testing focuses more on a use-case based or data flow approach.


E2E Scenarios

When is E2E a good fit:– E2E Testing can be a good fit when:

You have business use cases at the outset of the project.

It makes sense to focus on how customers will use the software.

You are taking a managed-risk approach and the focus is on making sure that the things customers will do will work appropriately.


E2E Scenarios

When is E2E a good fit:– E2E Testing can be a good fit when:

The area under test cuts across multiple functional areas and it makes sense to test them according to a workflow.


E2E Scenarios

Example of an E2E Scenario


Q&A

Any questions…


Reference

CSTE Study Guide 2002 by QAI CSTE Study Guide 2006 by QAI Effective Software Testing: 50 Specific Ways to

Improve Your Testing by Elfriede Dustin QA Websites: www.pcmag, www.wikipedia.com


Thank you…

Test Design Best Practices By Gabriel Rodriguez. Copyright 2006-2007. MSQAA Federation Chapter. Agenda Test Design Test Data Test Matrix Test Cases Test.

Documents

test design test design

test transactions

test data slide

test procedures

test matrix slide

progress of test

test data test file

test design slide