Test process

Test Process

Includes: Test Planning, Monitoring, ControlTest AnalysisTest DesignTest ImplementationTest ExecutionTest Closure ActivitiesTest ManagementTest Documentation and Work productsTest StrategyProject Rick ManagementTest EstimationTest MetricsReviewsBusiness Value of TestingDefect Management

Test Process

Test Planning includes: identifying activities for meeting objectives defined in test strategy identifying methods for gathering and tracking metrics determine adherence to plan and assess achievement of objectives selection of tools scheduling of trainings establishing documentation guidelines

Test Monitoring: defining targets measuring progress based on test conditions and groups of test conditions

Test Control: is an ongoing activity involves comparing actual progress against the plan implementing corrective actions when needed revisiting test planning activities if needed

Test Planning, Monitoring and Control:

Defines “what” is to be tested in the form of test conditions

Test Conditions: identified by analysis of test basis, test objectives, and product risks. identified using formal test techniques and general analytical techniques

include: Level of testing Level of detail and Quality of test basis System/Software Complexity Project/Product Risk Relationship between Test basis – What is to be tested ? How it is to be tested ? SDLC in Use Test Management tool in Use Level at which test design and test work products are to be specified Knowledge of Test Analysts Level of maturity of test process and Organization Availability of project stakeholders

Test Analysis:

Test Implementation:

Defines “how” something is to be testedIdentification of test casesStepwise elaboration of identified test conditions or test basis using test techniques

Test Design:

Tests are organized and prioritized by Test AnalystsTest designs are implemented as concrete test cases, test procedures, and test dataInvolves final checks to ensure test team is ready for test execution. These checks include:

Availability of required test environment Availability of test data Delivery of code All test cases have been written, reviewed and ready for run Test execution schedule Order in which manual and automated test cases are to be run Check for constraints, risks and priorities

Test Closure Activities:

Efficiency is the KEYbegins once test object is delivered and entry criteria to test execution are satisfiedTools – for test management, defect tracking, test execution and automation – need to be in placeStandards for test logging and defect reporting are publishedTracking of test resultsTracking of test metricsMetrics to be understood by the teamEvaluating exit criteria and reporting

Test Execution:

Fall into four main groups: Test completion checkTest artifacts handoverLessons learnedArchiving results, logs, reports, and work products in Configuration Management Systems

Test Strategy:

Secure and utilize resources-people, software, hardware, infrastructure etc.Carry out value-adding processPlan and control test processUnderstanding testing stakeholdersAlignment of Test activities and Lifecycle activitiesManaging Non-functional testingManaging Experience-based testing

Test Management:

Integration proceduresTest specification techniquesIndependence of testingMandatory and optional standardsTest environmentsTest automation & Test toolsReusability of software work products and test work productsConfirmation testing and Regression testingTest control and reportingTest measurement and MetricsDefect Management, Roles and Responsibilities & Configuration manager

Risk based testing:Risk Identification:

Expert Interviews Independent Assessments Use of risk templates Project retrospectives Risk workshops Brainstorming Checklists Calling on past experience

Risk Assessment: Complexity of technology and

teams Personnel and training issues Conflict within the team Contractual problems with

suppliers Geographically distributed team Legacy versus new approaches Tools and technology Weak managerial or technical

leadership Time, resource, budget and

management pressure Lack of earlier quality assurance

activities High change rates High earlier defect rates Interfacing and Integration issues

Factors impacting project/product risks: Frequency of use of affected feature Criticality of feature Damage to reputation Loss of business Potential financial, ecological or

social losses or liability Civil or criminal legal sanctions Loss of license Lack of reasonable workarounds Visibility of failure leading to

negative publicity Safety

Risk MitigationRisk MitigationGoalGoal

Risk ManagementRisk Management

Master Test Plan:Items to be tested and not to be testedQuality characteristics to be tested and not to be testedTesting schedule and budgetTest execution cycle and their relationship to the software planRelationships and deliverables among departmentsDefinition of test items – in-scope and out-of-scopeSpecific entry criteria, continuation criteria and exit criteriaRelationships among levelsTest project risksOverall governance of the testing effortResponsibilities for executing each of the test levelsInputs and Outputs from each of the test levels

Test Estimation:Considers all factors that can influence cost, effort and duration of testing activities. Factors include below:Required level of quality of the systemSize of system to be testedHistorical data, Industry data and Benchmark dataProcess factors:

Test strategy Development and Maintenance cycle Process maturity Accuracy of project estimate

Material factors Test automation and tools Test environment Test data Development environment Project documentation

People factorsComplexity of process, technology, organization etc.

Product(quality) risks Defects Tests Coverage Confidence

Estimation Techniques: (can be top-down or bottom-up) Intuition, guesses or past experience Work breakdown structures (WBS) Team estimation sessions (e.g. Wide Band Delphi) Company standards and Norms Percentage of the overall project effort or staffing levels Organizational history and metrics

Test Metrics-Definition and Usage: Testing Metrics Project Metrics Product Metrics Process Metrics People Metrics

Test progress is monitored by:

Definition

Reporting

ValidityTest Manager

Tracking

Reviews: Contractual reviews Requirement reviews Top-level design reviews Detailed design reviews Code reviews Test work product reviews Test entry reviews Acceptance reviews

Types of reviews: Informal review Walkthrough Technical review Inspection Management reviews Audits

Role of review leader: Collect review metrics Use review metrics to define ROI Provide feedback to review participants and stake holders

Business Value of testing:Delivers value to the organization, project, operation in both quantitative and qualitative waysDistributed, outsourced and insourced testingHelps manage the application of industry standardsCost of Quality

Costs of preventionCosts of detectionCosts of internal failureCosts of external failure