Software Testing Technique and Tools

UNIT III

Determining software testing technique and test tools

•Selecting technique / tools•Structural testing technique•Functional testing technique•Unit testing technique•Test factors/ test technique matrix•Test tools•Selecting and using test tools

Functional testing technique

• Evaluation of programs based on functional requirements to ensure the program works as it was specified to do.

Definition

• A functional test case validates the solution as defined by the requirements specification or detailed technical specification of the component.

• A functional test case design is based on the analysis of the functional specification of the specific functional component without reference to its internal workings.

Definition

• Functional Testing primarily focuses on – the solution components work as designed, and

that the design sufficiently meets the business requirements.

• Functional Testing is often the most complex area of Testing as it involves staggering the test cycle inline with development as it occurs.

• Typically, it is also where there highest degree of changes occur during the testing process.

Types of Functional Testing Criteria

Types of testing criteria for functional Requirements

Demonstration: Demonstrate the operation of the function to the system owner to show that the requirement has been met.

Inspection: Sometimes used in combination with demonstration, it requires visual examination of the code documentation, table dumps, scans, etc.

Analysis/Special Qualification: The processing of accumulated data obtained from other methods, or any special tools, techniques, procedures, facilities, and acceptable limits. This type of testing usually affects critical parts of the system and/or interfaces.

Demonstration: Demonstrate the operation of the function to the system owner to show that the requirement has been met.

Examples:

Test that a screen navigates to the correct next screenTest that a field is displayedTest that an inquiry can be made on a database

Inspection: Sometimes used in combination with demonstration, it requires visual examination of the code documentation, table dumps, scans, etc.

Examples:Test that an update is made to a databaseTest that a report is in the right formatTest that a database table is created

Analysis/Special Qualification: The processing of accumulated data obtained from other methods, or any special tools, techniques, procedures, facilities, and acceptable limits.

This is the typical type of testing we think of when we do functional testing. The types of these tests are enumerated in the next slide.

Functional Testing ItemsFunctional Testing will often contain the highest overall number of test cases, as it is focused on testing the solution against the following:

Business Rules Data Mapping Key Mapping Validation Data Validation Filtering Data Re-Engineering Derived calculations

Examples

Functional Requirements

The system shall calculate interest on a daily basis For a loan given to a producer, the system will calculate interest daily based on the prime rate of +1.2

–Derived calculations

Functional Testing–Techniques

• Black-box testing – Equivalence Partitioning– Boundary Value Analysis– Extreme input testing – State Transition Testing– Cause – effect graphing

• Equivalence Partitioning

• Equivalence Partitioning - evaluating the input and/or output data that falls into categories where the system responds in a similar manner.

• Equivalence Partitioning divides all possible inputs into classes such that there are finite no of equivalence classes.

• Equivalence Partitioning Process– Look at functions that require input– Select input and output data - invoke the same

process for the particular condition. • These should include – All valid input/output data for a given condition– All invalid input/output data

• Equivalence Partitioning Example/Design

• If a business rule stated that discounts were given to senior citizens (over 60) of 5%.

• We would have test cases such as• One person over 62

• One person under 62

Effectiveness• Reduces the number of test cases that must be run,

thus reduces the cost.• Helps identify the different classes for which the

program is not working properly.• Partition analysis can detect missing path errors,

incorrect relational operators in conditional sts.,, computation errors,…

• Can be applied during unit, integration and system testing.

• Boundary Value Analysis

• The purpose of boundary value testing is to evaluated a functions based on the boundaries of conditions.

• Boundary Value Process– Look at functions that require selection based on

specific values. – Select input data that is representative of the

boundary values.

• Boundary Value Example– If a business rule stated that discounts were

given to senior citizens (over 60) of 5%. – We would have a test cases for – One person who is 62

Extreme input testing • this determines if the system can work in

extreme conditions.• Examples– Only max values– Combination of these mim and max values.

• State Transition Testing

• State transition testing focuses on the testing of transitions from one state (e.g., open, closed) of an object (e.g., an account) to another state.

• A good technique to test screen navigation.

• Cause – effect graphing• It uses a logical relationship between cause

and effect, which can be expressed as a Boolean graph.

• Each cause becomes a condition with values –T ot F – on an input.

• Each effect becomes an effect of the outcome.

Structural testing technique

• Statement testing• Branch / Decision testing

Statement testing

• It is a white box testing technique to execute every possible st.

Design: test cases shall be designed to exercise executable sts.

For each test case, • the input(s) to the module• st(s) to be executed as a test case• expected outcome.

• Effectiveness:– Relatively weak criterion– Weakest white box testing

Branch / Decision testing• Analysis– This technique traces every possible path a

program can take (every possible sequence of sts, and decision in the sts.)

– Each possible transfer of control is a decision.Effectiveness:• This technique for a large module would have infinite

paths and is not a realistic option.• Sometimes there are infeasible paths. (infeasible

combinations and conditions).

Unit testing

• Unit testing is a method by which individual units of source code are tested to determine if they are fit for use.

• Unit tests are basically written and executed by software developers to make sure that code meets its design and requirements and behaves as expected.

• The goal of unit testing is to segregate each part of the program and test that the individual parts are working correctly.

• Method Used for unit testing: White Box Testing method is used for executing the unit test.

• When Unit testing should be done?• Unit testing should be done before Integration testing.• By whom unit testing should be done?• Unit testing should be done by the developers.• Advantages of Unit testing:• Issues are found at early stage• Unit testing helps in maintaining and changing the

code• Since the bugs are found early in unit testing hence it

also helps in reducing the cost of bug fixes.• Unit testing helps in simplifying the debugging

process

Component testing

• Component testing is a method where testing of each component in an application is done separately.

• Suppose, in an application there are 5 components. Testing of each 5 components separately and efficiently is called as component testing.

• Component testing is also known as module and program testing. It finds the defects in the module and verifies the functioning of software.

• Component testing is done by the tester.

• Missing software is replaced by Stubs and Drivers and simulate the interface between the software components in a simple manner.

• Stub: A stub is called from the software component to be tested. – ‘Stub’ is called by ‘component A’.

• Driver: A driver calls the component to be tested. – ‘component B’ is called by the ‘Driver’.

Test Factors

• Factors affecting test effort• Factors to consider when selecting test

procedures • Factors to consider when designing Test

Cases….

Factors affecting test effort

• Increasing the size of the product leads to increases in the size of the project and the project team

• Good project documentation is a positive factor - to maintain such detailed documentation requires lots of effort, as does working with test data that must be maintained or restored frequently during testing.

• life cycle itself is an significant process factor• Process maturity, including test process maturity, is

another factor• Time pressure

Contd…

• People factors - People execute the process - skills of the individuals and the team as a whole, and the alignment of those skills with the project’s needs

• Project team - solid relationships, reliable execution of agreed-upon commitments and responsibilities and a determination to work together towards a common goal

• Test results - the total amount of test effort during test execution. The delivery of good-quality software at the start of test execution and quick, solid defect fixes during test execution prevents delays in the test execution process

Factors to consider when selecting test procedures

• objectivity• validity• Reliability• Economy

Factors to consider when designing Test Case

• Correctness• Negative• User Interface• Usability• Performance• Security• Integration• Reliability• Compatibility

• Correctness : Correctness is the minimum requirement of software, the essential purpose of testing. The tester may or may not know the inside details of the software module under test e.g. control flow, data flow etc.

• Negative : In this factor we can check what the product it is not supposed to do.

User Interface : In UI testing we check the user interfaces. For example in a web page we may check for a button. In this we check for button size and shape. We can also check the navigation links.

• Usability : Usability testing measures the suitability of the software for its users, and is directed at measuring the following factors with which specified users can achieve specified goals in particular environments.

– Effectiveness : The capability of the software product to enable users to achieve specified goals with the accuracy and completeness in a specified context of use.

– Efficiency : The capability of the product to enable users to expend appropriate amounts of resources in relation to the effectiveness achieved in a specified context of use.

• Performance : In software engineering, performance testing is testing that is performed from one perspective to determine how fast some aspect of a system performs under a particular workload.

performance criteria.– Load Testing: This is the simplest form of performance testing. A load test is

usually conducted to understand the behavior of the application under a specific expected load.

– Stress Testing: Stress testing focuses on the ability of a system to handle loads beyond maximum capacity. System performance should degrade slowly and predictably without failure as stress levels are increased.

– Volume Testing: Volume testing belongs to the group of non-functional values tests. Volume testing refers to testing a software application for a certain data volume. This volume can in generic terms be the database size or it could also be the size of an interface file that is the subject of volume testing.

• Security : Process to determine that an Information System protects data and maintains functionality as intended. – Confidentiality : A security measure which protects against the

disclosure of information to parties other than the intended recipient that is by no means the only way of ensuring

– Integrity: A measure intended to allow the receiver to determine that the information which it receives has not been altered in transit other than by the originator of the information.

– Authentication: A measure designed to establish the validity of a transmission, message or originator. Allows a receiver to have confidence that the information it receives originated from a specific known source.

– Authorization: The process of determining that a requester is allowed to receive a service/perform an operation.

• Integration : Integration testing is a logical extension of unit testing. In its simplest form, two units that have already been tested are combined into a component and the interface between them is tested.

• Reliability : Reliability testing is to monitor a statistical measure of software maturity over time and compare this to a desired reliability goal.

• Compatibility : Compatibility testing of a part of software's non-functional tests. This testing is conducted on the application to evaluate the application's compatibility with the computing environment.

factors for the software testing tool selection

• Assessment of the organization’s maturity (e.g. readiness for change);

• Identification of the areas within the organization where tool support will help to improve testing processes;

• Evaluation of tools against clear requirements and objective criteria;

• Proof-of-concept to see whether the product works as desired and meets the requirements and objectives defined for it;

• Evaluation of the vendor (training, support and other commercial aspects) or open-source network of support;

• Identifying and planning internal implementation (including coaching and mentoring for those new to the use of the tool).

Software Test Matrix

• It is used to convey progress of our project status report. A test matrix is simply a spreadsheet that suggests test and captures test results by laying them out in the form of a table. In other words it is a measurement of the work we have done in the project to improve our process and check where we are lagging.

use of Software Test Matrix

• A test matrix can be used for a number of purposes:– to record a consistent set of tests;– to document a desired level of test coverage based on an

intersection of two criteria and, when filled out, to be used as evidence that the desired amount of testing has been done.

– as a quick visual indication of how much testing has been done on those criteria.

– to help to identify and refine the specific environments in which a defect occurs.

• When it is completely filled out, as a checklist to assure that some exhaustive task has been completed.

Test Process and Test Product Metrics

• QA recognize two subsets of Software Testing Metrics:

– Test Process Metrics– Test Product Metrics

• Test Process Metrics– These measures provide information about

preparation for testing, test execution and test progress. They don’t provide information about the test state of the product and are primarily of use in measuring progress of the Test Phase.

• Test Product Metrics– These measures provide information about the

test state of the product and are generated by test execution and code fixes or deferment. Using these metrics we can gauge the products test state and indicative level of quality, useful for product release decisions.

Test factors to be considered• Test Matrix is also called as Test responsibility matrix. It consists of Test Factors like--• · correctness• · compliance• · coupling• · continuity of process• · authorization• · access control• · audit check• · performance• · portability• · Ease of use• · Ease of operation• · Maintainability• · Reliability• · File integrity• · secure levels•

• To create a test matrix – we have to:• · Put the objects that you’re testing on the

rows.• · Show the tests on the columns.• · Check off the tests that you actually

completed in the cells.

Terminology used in Software Test Metrix:

• Test Procedure coverage: No. of test procedures Vs total number of test procedures execution

• Error Density: Total number of errors Vs no. of test procedures execution.

• Test Case coverage: Total number of test cases Vs no. of test cases execution and no. of test cases passed.

• Defects: Severity and Priority, time to find defect, time to fix defect , turnaround time to defect.

What is Traceability Matrix?

• The Requirements Traceability Matrix (RTM) captures the complete user and system requirements for the system, or a portion of the system. The RTM captures all requirements and their traceability in a single document, and is a mandatory deliverable at the conclusion of the lifecycle.

• The RTM is used to record the relationship of the requirements to the design, development, testing and release of the software as the requirements are allocated to a specific release of the software. Changes to the requirements are also recorded and tracked in the RTM. The RTM is maintained throughout the lifecycle of the release, and is reviewed and baselined at the end of the release.

• It is very useful document to track Time, Change Management and Risk Management in the Software Development.

• The RTM Template shows the Mapping between the actual Requirement and User Requirement/System Requirement.

Instructions for Requirements Traceability Matrix Template.

• Requirements traceability matrix (RTM) provides traceability between the [workspace/workgroup] approved requirements, design specifications, and test scripts.

• 1. Requirement ID2. Risks3. Requirement Type (User or System)4. Requirement Description5. Trace to User Requirement/Trace From System Requirement6. Trace to Design Specification7. UT * Unit Test Cases8. IT * Integration Test Cases9. ST * System Test Cases10. UAT * User Acceptance Test Cases11. Trace to Test Script

Software Testing Tools List

• Test Management tools• Functional Testing Tools• Load Testing Tools

1) Open Source Tools

• a) Test Management tools– TET (Test Environment Toolkit)– RTH is called as “Requirements and Testing Hub”.– The Test Manager is an automated software

testing tool– The TETware is the Test Execution Management

Systems

b) Functional Testing Tools

• Selenium• Soapui• Watir• HTTP::Recorder• WatiN• Canoo WebTest• Webcorder• Solex• Imprimatur• SAMIE• Swete• ITP• WET• WebInject

• c) Load Testing Tools• Jmeter• FunkLoad

2) Proprietary/Commercial tools

• a) Test Management tools• HP Quality Center/ALM• QA Complete• T-Plan Professional• Automated Test Designer (ATD)• Testuff• SMARTS• QAS.TCS (Test Case Studio)• PractiTest• Test Manager Adaptors• SpiraTest• TestLog• ApTest Manager• DevTest

b) Functional Testing Tools

• QuickTest Pro• Rational Robot• Sahi• SoapTest• Badboy• Test Complete• QA Wizard• Netvantage Functional Tester• PesterCat• AppsWatch• Squish• actiWATE• liSA• vTest• Internet Macros• Ranorex

c) Load Testing Tools

• WebLOAD Professional• HP LoadRunner• LoadStorm• NeoLoad• Loadtracer• Forecast• ANTS – Advanced .NET Testing System• vPerformer• Webserver Stress Tool• preVue-ASCII• Load Impact