Test Life Cycle

An An Introduction Introduction to Testing to Testing Life Cycle Life Cycle ProcessProcess

Contents

Testing Life Cycle Overview Test Strategy Test Planning Test Case Design Test Execution Testing Types Testing Tools Overview


Testing Life Cycle Overview

Testing Life Cycle Overview

•Normally, testing of any Large Systems will be in TWO parts. The functional verification and validation against

the Requirement Spec and Performance evaluation against the indicated requirements

Testing activity is involved right from the beginning of the project

V&V Process model to successfully deliver a project on time

This model is followed for a complete project life cycle model

Testing Life Cycle Overview Test Strategy Test Planning Test Case Designing Test Execution Testing Types Testing Tools Overview

Testing Life Cycle - V&V Process Model

HLD/LLD for each application

involvedInterface design

Application Integration

IntegratedSolution

Release

Project Planning

Master Test Strategy

Test Planning

Strategy for individual

applications, Test case design

Performance Testing,

Business Cycle testing

Acceptance Tests&

Certification

Incremental Integration Testing

System Testing (Application level

Testing)

Solution Mapping/

Development/Customization

Business requirements,

SolutionArchitecture


Testing Life Cycle - V&V Process Concept

The "V &V " concept relates the build components of development to the test components that occur during

that build phase.Testing related activities during Requirement phase

Creation and finalization of testing templates Creation of over-all Test Plan and Test Strategy

Capturing Acceptance criteria and preparation of Acceptance Test Plan

Capturing Performance criteria of the software requirements


Testing Life Cycle - V&V Process Concept …

• Testing activities in System Testing phase– System test is done for validating the product with respect to client

requirements– Testing can be in multiple rounds– Defect found during system test should be logged into Defect

Tracking System for the purpose of tracking. – Test logs and defects are captured and maintained.– Review of all the test documents


Testing Life Cycle - V&V Process Concept…

Testing activities in Design phaseDevelop Test cases to ensure that product is on par with

Requirement Specification document.Verify Test Cases & test scripts by peer reviews.

Preparation of traceability matrix from system requirements



• Testing activities in Integration Testing Phase – This testing is conducted in parallel with integration of various

applications (or components)– Testing the product with its external and internal interfaces

without using drivers and stubs.– Incremental approach while integrating the interfaces.



• Performance Testing – This is done to validate the performance criteria of the product / application. This is non-functional testing

• Business Cycle testing – this refers to end to end testing of real life-like business scenarios.

• Testing activities during Release phase– Acceptance testing is conducted at the customer location.– Resolves all defects reported by the customer during

Acceptance Testing– Conduct Root Cause Analysis (RCA) for those defects

reported by customer during acceptance testing


Testing Life Cycle - Team Structure

• An effective testing team includes a mixture of members who has– Testing expertise – Tools expertise– Database expertise– Domain/Technology expertise Testing

Life Cycle Overview Test Strategy Test Planning Test Case Designing Test Execution Testing Types Testing Tools Overview

Testing Life Cycle - Team Structure (Contd…)

• The testing team must be properly structured, with defined roles and responsibilities that allow the testers to perform their functions with minimal overlap.

• There should not be any uncertainty regarding which team member should perform which duties.

• The test manager will be facilitating any resources required for the testing team.


Testing Life Cycle - Roles and Responsibilities

• Clear Communication protocol should be defined with in the testing team to ensure proper understanding of roles and responsibilities.

• The roles chart should contain both on-site and off-shore team members.



• Test Manager – Single point contact between onsite and

offshore team– Prepare the project plan – Test Management– Test Planning– Interact with onsite lead, Client QA manager – Team management– Work allocation to the team– Test coverage analysis



• Test Manager cont..– Co-ordination with onsite for issue resolution. – Monitoring the deliverables– Verify readiness of the product for release through release

review– Obtain customer acceptance on the deliverables – Performing risk analysis when required– Reviews and status reporting– Authorize intermediate deliverables and patch releases to

customer.



• Test Lead– Resolves technical issues for the product group– Provides direction to the team members– Performs activities for the respective product group– Review and Approve of Test Plan / Test cases– Review Test Script / Code– Approve completion of Integration testing– Conduct System / Regression tests– Ensure tests are conducted as per plan– Reports status to the Offshore Test Manager



• Test Engineer– Development of Test cases and Scripts– Test Execution– Result capturing and analysing– Defect Reporting and Status reporting


Testing Life Cycle - Term Definitions

• Testing: It is a process of executing software in a controlled manner, in order to answer the question “Does the software behave as specified?”

• Verification: Says “Are we building the product right?” All QC activities throughout the life cycle that ensure that interim deliverables meet their input specification.

• Validation: Says “Are we building the right product?” The test phase of the life cycle which assures that the end product meets the user’s needs.

• Test Case: “An input operation and the corresponding expected output” in order to test a small unit of work.

• Test Script: “A logical group of test cases which, when taken together, test a particular function or unit of a system”


Testing Life Cycle - Term Definitions (Contd.)

• Test Bed: “A group of test scripts which, when taken together, test all functions of an entire system”

• Audit: An inspection/assessment activity that verifies compliance with plans, policies and procedures

• Baseline: A quantitative measure of the current level of performance

• Bug: A catch all term for all software defects or errors• Certification: Acceptance of a software by an

authorized agent or its validity been demonstrated to an authorized agent

• Cost of Quality (COQ): Money spent above and beyond expected production costs, this involves prevention, appraisal, internal & external failure costs

• Cyclomatic Complexity: The number of decision statements plus one.



• Defect: It is “A deviation from specification or standard” or “Any thing that causes customer dissatisfaction”.

• Desk Check: Verification technique conducted by the author of the artifact to verify the completeness

• Dynamic Assertion: A dynamic analysis technique that inserts into the program code assertions about the relationship between program variables

• Force field Analysis: A group technique used to identify both driving and restraining forces that influence a current situation

• Inspection: A formal assessment of a work product conducted by one or more qualified independent reviewers to detect defects, violation of development standards, etc. Inspection identifies defects but does not attempt to correct them.



• Instrumentation: The insertion of the additional code into a program to collect information about a program behavior during program execution.

• Life Cycle Testing: The process of verifying the consistency, completeness and correctness of software at each phases of the development life cycle.

• Phase Containment: Method of control put in place within each stage of the development process to promote error identification and resolution so that defects are not propagated downstream to subsequent stages of the development process

• Productivity: The ratio of the output of a process to the input



• Quality Assurance: The set of activities (including facilitation, training, measurement, and analysis) needed to provide adequate confidence that process are established continuously improved to produce products that meet specifications and are fit for use.

• Quality Control: The process by which product quality is compared and detected w.r.t requirements and other relevant specifications, focus is in detection and removal

• Test Driver: A program that directs the execution of another program against a collection of test data sets

• Test item: A software item that is an object of testing• User: The customer that actually uses the product

received


Test Strategy Testing Life Cycle Overview Test Strategy Test Planning Test Case Designing Test Execution Testing Types Testing Tools Overview

Test Strategy• Test strategy is statement of overall approach of

testing to meet the business and test objectives.• It is a plan level document and has to be prepared

in the requirement stage of the project.• It identifies the methods, techniques and tools to be

used for testing .• It can be a project or an organization specific.• Developing a test strategy which effectively meets

the needs of the organization/project is critical to the success of the software development

• An effective strategy has to meet the project and business objectives

• Defining the strategy upfront before the actual testing helps in planning the test activities


Elements of Test Strategy• A test strategy will typically cover the following

aspects– Definition of test objective– Strategy to meet the specified objective– Overall testing approach– Test Environment – Test Automation requirements– Metric Plan– Risk Identification, Mitigation and Contingency plan– Details of Tools usage– Specific Document templates used in testing


Test Strategy - Project/Business Objectives

• All the business and test objectives are identified and test strategy shall be defined in order to meet those objectives.

• As far as possible all the test objectives to be met are defined in quantitative terms in the test strategy.

• Examples of test objectives can be like 90 % of code coverage, finding at least a minimum of 100 defects etc.


Test Strategy - Test Approach• Test approach will be based on the objectives

set for testing• Test approach will detail the way the testing to

be carried out• Types of testing to be done viz Unit,

Integration and system testing• The method of testing viz Black–box, White-

box etc.,• Details of any automated testing to be done


Test Strategy - Test Environment• All the Hardware and Software requirements

for carrying out testing shall be identified in detail.

• Any specific tools required for testing will also be identified

• If the testing is going to be done remotely, then it has to be considered during estimation Testing

Life Cycle Overview Test Strategy Test Planning Test Case Designing Test Execution Testing Types Testing Tools Overview

Test Strategy - Metric Plan• Metric plan will contain the measures that will

indicate the effectiveness of testing .• All the metrics to be tracked for testing like

defect density ,residual defect density ,code coverage etc are identified in test strategy .

• The objectives set for testing are also measured in the metric plan.

Note: More details are covered in Test Execution section


Test Strategy - Risks• Risk analysis should carried out for testing phase• The risk identification will be accomplished by

identifying causes-and-effects or effects-and-causes • The identified Risks are classified into to Internal and

External Risks. The internal risks are things that the test team can control or influence. The external risks are things beyond the control or influence of the test team

• Once Risks are identified and classified, the following activities will be carried out– Identify the probability of occurrence– Identify the impact areas – if the risk were to occur– Risk mitigation plan – how avoid this risk?– Risk contingency plan – if the risk were to occur what do we

do?


Test Strategy - Tools Usage• Tools required in the project and rationale to use

that particular tool(s) should be documented.• Automation requirements should be identified

and analyzed– Which phases need to be automated– How much of testing needs to be automated

• Based on the automation requirements, tools strategy should be prepared.


Test Strategy - Templates • Any specific templates to be used for testing apart

from the normal templates based on client or project requirements are identified in the test strategy.

• In some case client provides the templates and we’ll use that.

Sample Strategy doc


Test Strategy - Acceptance criteria• Acceptance test plan can be prepared either by

us or by the customer, as agreed.• Customer will be involved in preparation, review

and approval of acceptance test cases• This plan specifies the criteria for client

acceptance of the final tested product including Features/functionalities to be tested and traceability information.


Test Strategy - Summary• Test strategy is a high level document which

states in detail way testing is to be carried out.• It is done in line to meet the test and business

objectives• Tools ,Methods ,and techniques to be used are

stated in the test strategy. Testing Life Cycle Overview Test Strategy Test Planning Test Case Designing Test Execution Testing Types Testing Tools Overview

Test Planning Testing Life Cycle Overview Test Strategy Test Planning Test Case Designing Test Execution Testing Types Testing Tools Overview

Test Planning - Documenting assumptions

• If any assumptions are there, it needs to be documented to avoid any confusions– Testers has understanding of V&V-process model or

any SDLC model– Testers are aware of test phases of the project– A good understanding of the Domain – Standard naming conventions are followed in

documentation Testing Life Cycle Overview Test Strategy Test Planning Test Case Designing Test Execution Testing Types Testing Tools Overview

Test Planning -Traceability • Requirements tracing is the process of

documenting the links between the user requirements for the system you are building and the work products developed to implement and verify those requirements.

• Work products include software requirements, design specifications, software code, test plans and other artifacts of the systems development process

Traceability Matrix


Test Planning -Traceability • Traceability is documenting of

– the dependencies and logical links between individual requirements and other system elements

– traceability to greatly facilitate change impact analysis.

– traceability is that it can help identify the propagation of change that can result when a specific requirement is deleted or modified.

– specific requirements mapped into tasks in a work-breakdown structure, you know those tasks that will be affected if and when a requirement is changed or deleted.


Test Planning - Schedule/ Milestones

• Test plan captures the details about different milestones of the project and the schedule details.

• Define the schedule for each event for offshore and onsite. Events may include familiarization, training, test case preparation, test scripting, test execution etc.,

• Completion criteria for each milestone should be identified and documented


Test Planning - Resources • Staffing:

– Check what types of personnel should be considered for inclusion on the test team

– Check for the skills of the tester– Ensure a good ratio of testers to developers.

• Steps to form the best team:– Identify potential test team members– Recruit test team members and develop

tentative test assignments– Define individual work assignments


Test Planning - Test run plan • The execution order of test

cases depends on many factors and is shown as below:

TC priority Execution Order

ComplexityExecution

rate

Impacted Functionality

Unstable functionality

Customer Priority

Test Execution Run Plan

Release Plan 4

0200400600800

100012001400

7/4 7/5 7/6 7/7 7/8 7/9 7/10 7/11 7/12 7/13 7/14 7/15 7/16 7/17

# of Days

Cum

ulat

ive

TC

s E

xecu

ted

Plan 106 257 575 669 767 877 983 1065 1119 1119

Actual 297 483 632 737 855 939 1015 1079 1140 1147

4-Jul 5-Jul 6-Jul 9-Jul 10-Jul 11-Jul 12-Jul 13-Jul 16-Jul 17-Jul

Functionality Dependency

Execution Order

TC Inter- dependency


Test Planning - Change management

• Version control - Versioning test cases before and after release to customer

• Change control - Authority to approve and prioritize changes

• Configuration auditing - Ensure changes made properly

• Reporting• Addition/Updating of test cases/plans because of

bugs which are identified by the customer and that are not available in test plans


Test Planning – Suspension and Resumption criteria

• Deadlines (release/testing deadlines etc.) are met and quality of the product is as per the agreed norms.

• Test cases completed with certain percentage passed

• Test budget depleted• Coverage of code/functionality/requirements

reaches specified point• Beta or Alpha testing period ends


Determination of Test Completion criteria • Most popular method of establishing a test

phase completion criteria is by detecting a pre-defined number of bugs or when a pre-defined time elapses

• E.g.: Unit testing of a unit is not deemed complete unless at least 5 errors are detected

• Testing completion criteria could be detection and fixing of 30 bugs, or elapsed time of 30 days, whichever is later.



Estimation of number of errors

• How to determine the number of errors to be detected? – An estimate of total number of errors in the

program.

– An estimate of what percentage of these errors can be feasibly found through testing.

– Estimates of what fraction of errors originated in a particular phase, during what testing phases these are likely to be detected.



• Methods of estimation– Rough estimate of total number of errors can be obtained

by previous experience on similar programs.– Error-seeding techniques – involves intention introduction

(seeding) of error in the code. After some period of testing time, the product of the number on non-seeded errors found during testing and the ratio of total seeded errors to the number of seeded errors detected during testing, gives an estimate of the number of non-seeded errors remaining in the program.



• Estimation of errors in a particular phase - Example– Existing data reveals that in large programs, 40% of errors

are coding and logic-design mistakes and the rest are generated in earlier phases.

– Example to illustrate estimation of total number of errors –– We have a 10,000 statement program, and the number of

errors remaining in the program after code review is 5 per 100 statements. Our objective is to find 98% of coding and logic design errors and 95% design errors. Therefore-

• Total number of estimated errors = 500• Coding and Logic design errors = 200 (40% of 500)• Design errors = 300.• We need to find at least:• 196 Coding and Logic Design errors• 285 Design errors.



Coding & Logic Design errors

Design Errors

Unit Test 65% 0%

Module Integration Test

30% 60%

System Test 3% 35%

Total 98% 95%

Estimation of error distribution in different phases



Establishment of completion criteria• Exit Criteria for Unit testing would be detection and

correction of 130 errors (65% of estimated 130 coding and design logic errors)

• Exit Criteria for Module Integration testing would be detection and correction of 240 errors (30% of 200 plus 60% of 300) or elapse of one month; which ever is later.

• Exit Criteria for System Testing is when 111 errors are detected or when 2 months elapses; which ever is later.

Assumption: We have established one month for Integration Testing and 2 months for System Testing.



• Resolution in case of over estimation of defects• In the previous example, let us say that 240

errors could not be detected even after one month of integration testing is completed. Then the project manager should take the following action – With the help of the appropriate tool, he/she should

analyze the test cases and decide whether • There are inadequate Test cases, or• Excellent Test cases, but lack of errors to detect



• Determination of completion criteria through a graphical representation• A graph is plotted using number of errors found per unit time during a

testing phase. By examining the shape of the curve, one can determine whether to continue the test phase or end it.

• Assume a program is being Integration tested, and the number of errors per day is plotted. Lets say on the 7h day the graph looks as follows:

No. of days 7

Num

ber o

f err

ors AT the end of 7th day, even

if the number of errors detected has crossed the criteria mark, it would be wise to continue Integration Testing. This is because, the error rate is still very high.



• Determination of completion criteria through a graphical representation

• On the hand if the graph looks as shown, then this shows that error rate has decreased, and that testing can be stopped.

No. of days 7

Num

ber o

f err

ors



Test Planning – Document templates

• Typical Test Plan– Test Plan identifier– Introduction– Test items– Features to be tested– Features not to be tested– Approach– Testing strategy – Item pass/fail criteria– Suspension criteria and resumption criteria– Test deliverable– Automation scope – Testing tasks– Effort estimation– Environmental needs– Roles and Responsibilities

testplan template


Test Case Design Testing Life Cycle Overview Test Strategy Test Planning Test Case Design Test Execution Testing Types Testing Tools Overview

Test Case Design (Contd…)Development Of Test Objectives:• At very First stage of testing , you need to set the

goals and objectives of what you need to achieve.• This Simplifies the limits or borders of testing

process.• Test Objectives will be contain the list of tasks you

are going to do and the achievements.Test Case Writing:• A specific set of steps and data along with expected

results for a particular test objective. A test case should only test one limited subset of a feature or functionality.


Test Case Design (Contd…)A typical test case sheet contains these fields…• Test Case ID: It is unique number given to test case in

order to be identified.• Test description: The description if test case you are

going to test.• Revision history: Each test case has to have its revision

history in order to know when and by whom it is created or modified.

• Function to be tested: The name of function to be tested.• Priority/Build/time: The priority of the Test Case,

Build version of the test case and the Minimum time-to-execute the test case is also mentioned.


Test Case Design (Contd…)• Environment: It tells in which environment you are

testing.• Test Setup: Anything you need to set up outside of

your application for example printers, network etc.,• Test Execution: It is detailed description of every step

of execution.• Expected Results: The description of what you expect

the function to do.• Actual Results: Pass /Failed.

If pass – What actually happens when you run the testIf failed - Description of what you've observed.


Test Case Design (Contd…)Characteristics of a Good Test: • Tests are likely to catch bugs• No redundancy• Not too simple or too complex.

TIP: Test case is going to be complex if you have more than one expected results.


Test Case Design (Contd…)Test Data Inputs:• Before testing, the tester should plan what

kind of data he is giving for test.• Give data inputs as functional, boundary,

stress, performance, usability values etc.


Test Case Design (Contd…)Test Scripts:• A test script is the executable form of a test. • It defines the set of actions to carry out in order to

conduct a test and it defines the expected outcomes and results that are used to identify any deviance in the actual behavior of the program from the logical behavior in the script (errors during the course of that test).

• In essence it is a program written for a human computer (tester) to execute.

TestCase Template TestCase Status Template


Test Execution Testing Life Cycle Overview Test Strategy Test Planning Test Case Design Test Execution Testing Types Testing Tools Overview

Test Execution• Test execution is culmination of testing

activities which involves executing the planned test cases and conducting of the tests.

• Test execution phase broadly involves execution and reporting.


Test Execution - Activities • Test execution consists of following activities

to be performed – Creation of test setup or Test bed– Execution of test cases on the setup – Test Methodology used – Collection of Metrics – Defect Tracking and Reporting– Regression Testing


Test Execution Cycle


Test Execution - Testing• Only exhaustive testing can show a program that it is

defect free ,but exhaustive testing is impossible• Tests should exercise the systems capabilities fully • Testing typical situations is more important than

boundary value conditions


Test Execution -Test Data and Test Cases

• Test Cases: Inputs to test the system and predicted output from the system are detailed, further will contain the expected result and test topology

• Test Data: Inputs which are to exercise the system under test.


Test Execution -Types of Testing and Methodologies

• There are three major types of testing and methods used are based on the testing type– Unit testing– Integration testing – System testing


Test Execution - Unit Testing• Unit testing is typically done as white box testing.• Unit testing is done at module level.• Derivation of Test cases are based on the program

structure .Knowledge of the program is used to identify the test cases .

• Objective is to exercise all parts of the program .• Test coverage measurements are done to verify that

all program parts are exercised• Drivers are used to feed the data to the program to

test it, wherever required.• Test stubs are used to stub for the functions which

are outside the module


Test Execution - Integration Testing

• Tests partial systems composed of integrated components

• Integration testing is black box testing• Main difficulty in integration testing is

localizing errors • There two approaches to integration testing

– Top-down approach– Bottom - up approach


Integration testing- Top Down approach

• Start from the top level module and integrate individual components and stub the rest of the system


Integration testing - Bottom –Up approach

• Integrate individual components from lower level module until whole system is integrated .


Test Execution - System Testing

• System testing is typically “Black Box “ testing where the system is taken as black –box and tested based only on its functionality

• Test cases are based on the system specification• It is critical part of the testing process as the

system is verified as a whole Testing Life Cycle Overview Test Strategy Test Planning Test Case Design Test Execution Testing Types Testing Tools Overview

Test Execution - Regression Testing

• Regression testing is done after fixing all the defects.

• It is re-testing after fixing the bugs found during testing and also when there is change in the test environment.

• The impacted portions of the code after fixing the bug is re-tested.

• Automation of testing helps in faster regression testing.


Test Execution - Defect Tracking and Reporting

• All the defects/bugs found during testing is logged and they are classified based on the severity and priority.

• They are assigned to developers for fixing.• Each defect found is logged and tracked to

closure.• Test summary report shall be prepared

containing the details like numbers of test cases passed/failed ,number of cycles of testing.

• Generally test defect logging and tracking tools are used for this purpose.

Test report template


Test Execution - Start / Stop Criteria

• Testing is done in rounds. Software passes though several rounds before it gets released.

• Testing is stopped when there are many major bugs and critical system errors, when further testing is not feasible

• Testing is stopped when the system becomes un stable during test.

• When those critical errors are fixed and system is stable, the next round of testing is started .

• The test cases are executed all over again in each round.


Test Execution - Metrics• Metrics are important to collect for any project as it

helps in monitoring the health of project /product /application under test.

• Metrics are collected during testing depending on the objective of testing.

• There are different kinds of metrics which can be used in testing.– For example LOC (Lines of Code) is a metric for Unit

testing. i.e., Code related metrics comes in the category of Unit testing.

• Test case related metrics can be used for all kinds of testing types, since for every type of test, we will write test cases.

• Normally source of errors/defects is requirements, design or Implementation. Testing can not be the source of errors/defects.


Test Execution - Metrics• Typical testing metrics

– No of LOC tested per day – No of test cases executed per day– Number of test cases failed– Number of bugs reported – Number of rejects of bugs reported– Number of persons working on the test plan during

the month– Total Effort spent on test execution in person hours– Number of simulations made during the month– Number of out of norms per month– Average time taken to find a problem categorized by

priority– Defect density– Review efficiency


Test Execution - Metrics• Having a complete Metrics data helps in

building the “Trustworthy, Capable, Reliable and Predictable” Product or Application.– Productivity Increase– Field error rate Decrease– Customer Overhead reduces– Improvement release on release– On time delivery– Risks in Program identified and proactively intimated– Decrease in Cost of Quality– Decrease in Re-work– Helps in identifying the area which needs focus in the

project/organization for improvement– Helps in arriving at better estimates for the project


Test Execution – Good test engineer?

• What makes a good test engineer?– A good test engineer has a 'test to break' attitude.– An ability to take the point of view of the customer– A strong desire for quality, and an attention to detail.– Tact and diplomacy for maintaining a cooperative relationship with

developers.– An ability to communicate with both technical (developers) and non-

technical (customers, management) people.– Previous software development experience can be helpful, gives the

tester an appreciation from the developers' point of view, and reduce the learning curve in automated test tool programming.

– Judgment skills are needed to assess high-risk areas of an application on which to focus testing efforts when time is limited.

– Notice little things that others miss/ignore (See symptom not bug)– Good testers use files, Databases and all the other accoutrements of

an organized mind – Testers are fundamentally honest and incorruptible – Last but not the least is “Patience”


Test Execution – Testing Rules?• Software testing 10 rules?

– Test early and test often.– Integrate the application development and testing life

cycles.– Formalize a testing methodology; you'll test everything the

same way and you'll get uniform results.– Develop a comprehensive test plan; it forms the basis for

the testing methodology.– Use both static and dynamic testing.– Define your expected results.– Understand the business reason behind the application.

You'll write a better application and better testing scripts.– Use multiple levels and types of testing (regression,

systems, integration, stress and load).– Review and inspect the work, it will lower costs.– Don't let your programmers check their own work; they'll

miss their own errors.Source: SQAtester.com


Test Execution - Summary• Test execution consists of preparing of test

bed/setup ,execution of planned test cases ,verification of results ,Test tracking and reporting and metrics collection.

• Apart from that ,it involves applying different test methods like black box to system testing ,white box at unit testing level and top –down and bottom –up in case of integration testing .


Test Execution – Summary Contd..

• Testing will involve several rounds based on the stability and bugs found on the software and start /stop criteria identified .

• Regression testing will be carried out when ever the new bugs are found and fixed in the tested software to verify the impacted items.

• Test execution will involve preparation of test reports ,collection of different metrics and meeting the test objectives .


Testing Tools Overview Testing Life Cycle Overview Test Strategy Test Planning Test Case Design Test Execution Testing Types Testing Tools Overview

Testing Tools Overview (contd…)

• Testing tools are used to automate the testing process.

• These tools will have features like– Record and play back– Test Script generation– Virtual users

And so on… Testing Life Cycle Overview Test Strategy Test Planning Test Case Design Test Execution Testing Types Testing Tools Overview


• Different vendors released different testing tools in the market. Some of them are

Functional / GUI Testing Tools:• Win Runner• Rational Robot • Visual Test



Performance Testing: • Rational Quantify• True time

Web Testing: • Silk Test• Astra Quick Test• eTest Suite



Java Testing Tools: • Jtest• JProbe• QStudio• Optimizeit• JStyle Testing

Life Cycle Overview Test Strategy Test Planning Test Case Design Test Execution Testing Types Testing Tools Overview

Appendix 1: Example for test life cycle

• To understand the different stages of testing, consider an example in which you need to test a Webpage, say HRWEB->resume part.

• To test this Webpage following activities needs to be done– Prepare Test Strategy for testing HRWEB->Resume section– Prepare Test Plan for testing HRWEB->Resume section– Develop Test Cases for testing HRWEB->Resume section– Execute the test cases and report the results, including bug

reporting, re-test and tracking.

Appendix 1: Example for test life cycle cont..

• Test strategy to test HRWEB->Resume page includes overview of– Document the test requirements– User Interface testing– Test for associated links– Error Handling and Exceptions– Functionality testing– Boundary conditions – Test for Load, Performance, Security issues– Deciding of tools and test set-up to test

• Test Planning to test HRWEB->Resume includes – Resume page and all associated links will be tested– Specify how it will be tested - Test Cases– Identify resources for testing and finalize timeline– Specify Hardware, Software configuration requirement


• Test Case Development– Several Test Cases will be generated as per the test strategy and plan

for User Interface, Functionality, Error Handling of Resume Section in HRWEB

– Integration and System Test cases will be generation to test for the whole application including Resume Section.

• Test report:– Test Cases will be executed and Test Results will be recorded and

tracked.– After bugs are fixed, the system is tested again to verify the same– The whole system is also tested to verify that new bugs have not been

introduced.


Sample User Interface Test Case for Resume Section• Perform the necessary steps to view the Resume Section on

TEDWEB• Verify that the menu TEDWEB Menu displays at the top and

the Resume Menu displays at the left of the screen• Verify the presence of the following buttons - Add Resume,

Modify Resume, Add project, Modify Project etc..Sample Functionality Test Case for Resume Section• Click on ‘Add Resume’ link• Verify a new screen appears to the right that allows the user

to add his resume details• Put the cursor in Educational Qualifications field and verify it

is enabled for user input• Put the cursor in Summary field and verify it is enabled for

user input.


Sample Error Handling Test Case for Resume Section• Click on ‘Add Resume’ when the resume already exists.

Verify the message displayed is ‘Profile Already Exists’• Click on ‘Modify Resume’ when no resume has been added.

Verify that a proper error message is displayed that no resume exists.

Appendix 2: Testing Standards• IEEE Standard for Software Test Documentation

(ANSI/IEEE Standard 829-1983). This is a summary of the ANSI/IEEE Standard 829-1983.

• It describes a test plan as: “A document describing the scope, approach, resources, and schedule of intended testing activities. It identifies test items, the features to be tested, the testing tasks, who will do each task, and any risks requiring contingency planning.”

IEEE testplan template

Appendix 3: Additional readings• Please go through the following documents, which are

available in Interops KM Page.– Inter101_002-Intro Software Testing– Inter101_003-Test Plan & Test Case– Inter101_004-SDLC&TLC– Inter101_005-Unit & Integration Testing– Inter101_006-Functional & Regression Testing– Inter101_007-General Testing Types– Inter101_008-Test Report Automation Tools– Inter101_009-Review & Walk thru– Inter101_010-Reliability terminology

Test Life Cycle

Economy & Finance

resumption

resumption

planned test

resume section

good test

life cycle

resume section

completion