Software Test Estimation

8/3/2019 Software Test Estimation

1/26

Nga Huynh

What is the best approach tosoftware test estimation?


2/26

Agenda

Introduction

Why good estimates are important and reasons for poorestimates?

Test Estimation vs Estimation (of any other activities)

Confidence rating

Estimation Methods ways to estimate test effort

Considerations

Questions?


3/26

Introduction

Successful test estimation is a challenge

Difficulty to estimate the software projectdevelopment accurately even harder to estimate fortesting effort

Lack of detailed information about, example Detailed requirements

Organisation's experience with the similar projects in the past

Understanding of what to should be included in a testing effort


4/26

Why good estimates are important and reasonsfor poor estimates?

Why good estimates are important?

Testing is often blamed for late delivery

Testing time is squeezed

It promotes early risk assessment

Reasons for poor estimates? Incomplete/ambiguous requirements

New technology

Skills (Testers & developers)

Project delays

Poor environments


5/26

Estimation vs Test Estimation

Any estimation involves the following: Identify/estimate/assign tasks

Start and finish for each task

Resources and skills

Dependencies (task precedencies)

Test Estimation

No. of iterations/cycles

Availability of the resources (test environment, people etc)

Quality (Software, test environment, test cases etc)

Delivery of the software to be tested Reviews of test artefacts


6/26

Confidence rating

System under test Test Effort Confidence Rating

System A 400 hrs 20 %

System A 350 hrs 80%

Start of the project

More detailed info


7/26

Estimation Methods ways to estimate testeffort

What is the best approach to software test estimation?

Ways to estimate test effort

Guessing

Formula Based

Parkinsons Law versus Pricing to Win

work expands to fill the time allocated for it

Minimum amount that we can get away with Consensus Of Experts

Past Project Knowledge

Work Breakdown Structure

MITs Model (MITs = Most Important Tests)


8/26

What is the best approach to software testestimation?

Highly dependent on Organisation

Project

Experience of the resources involved

Criticality of the system Life-critical equipment software

Low-cost computer game Complexity and size of the system

Allocate time for estimation and re-estimation

~ 5% - 7.5% of the testing budget on test estimation(Ross Collard,)

Overhead

More than one method to check each other

Involve the team


9/26

Estimation Methods

Guessing - Finger In Air (FIA) Pure guess/gut feel

Based on experience

past project experiences

expertise of the estimator

Bad method to base our final estimate on

Easily questioned and challenged

Try to delay

Past Project Knowledge

Historical estimation

Estimation based on previous test efforts

e.g. 40 hours, 80 tests (0,5 hrs/test ) 100 tests, 50 hours

Best predictor only if the historical data is available applicable andaccurate


10/26

Estimation Methods

Formula Based 40% of Development Effort

Quick method

Dependent of accurate development effort estimates

System and acceptance test

Function Point Analysis (FPA) Formal technique to estimate size of the system under test

Based on five elements plus adjustment factors (inputs,outputs, queries, files, interfaces etc)

Test Point Analysis (TPA)

Test estimation technique based on FPA

Described in TMAP

TPA takes risk into account


11/26

Consensus of Experts

Useful when unsure ofsystem, e.g. new technologyor application

3-4 knowledgeable peoplefor independent judgement

If the estimates are similarthen take the average

If the estimates are differentthen a consensus must beproduced


12/26

Work Breakdown Structure(micro-estimating)

Identify tasks & activities

Identify dependencies

What tasks must be start/finish before others

Effort & resources, start & end times

Who should do what & when? Schedule

What is the critical path?

Involve the team


13/26

Work Breakdown Structure(micro-estimating)

Phase Historical Value % of Project Preliminary Estimate Adjusted Estimate

1 Project Startup 140 2.6 179 179

2 Early Project Support (requirementsanalysis, etc.)

120 2.2 152 152

3 Decision to Automate Testing 90 1.7 117 -

4 Test Tool Selection and Evaluation 160 3 207 -

5 Test Tool Introduction 260 5 345 345

6 Test Planning 530 10 690 690

7 Test Design 540 10 690 690

8 Test Development 1,980 37 2,553 2,553

9 Test Execution 870 17 1,173 1,173

10 Test Management and Support 470 9 621 621

11 Test Process Improvement 140 2.5 173 -

PROJECT TOTAL 5,300 100% 6,900 6,403


14/26

The Most Important Tests (MITs) Method

Sizing test efforts based on the risk of failure in the

system

Criticality

Complexity

Objective Weighting

Planned Test Effort


15/26


Function Complexity Criticality Objective

Weighting

Planned Test

Effort

Function A 50 1.75 40% 35

Function B 150 2.25 5% 17

Function C 60 2.00 20% 24

Total Units 76

Complexity determined by countinge.g.

Business decisions

Number of screens involved

Number of data values entered

etc


16/26


Function Complexity Criticality Objective

Weighting

Planned Test

Effort

Function A 50 1.75 40% 35

Function B 150 2.25 5% 17

Function C 60 2.00 20% 24

Total Units 76

Criticality consider the following:

e.g.

Customer Affected

User Affected

Business Affected

Frequency


17/26

Determining Criticality

Customer

Affected

User Affected Business

Affected

Frequency

High (3) Directly Cannot work Loss of business

Everyday

Medium (2) Could be Work around Loss of money Occasional

Low (1) Not at all User doesntknow

No effect onbusiness

Once only

Total: 7 Average: 7/4 = 1.75


18/26

2420%2.0060Function C

76Total Units

175%2.25150Function B

3540%1.7550Function A

PlannedTest Effort

ObjectiveWeighting

CriticalityComplexityFunction


19/26

Objective Weighting

Purpose:

Allows testers to specify an appropriate amount oftest effort.

Example of Objective Weighting5% - 25% minimal regression testing

30% - 50% thorough regression testing

55% - 75% minimal testing

80% - 100%thorough testing

> 100% rigorous testing


20/26

Planned Test Effort

Planned Test Effort (units) = Complexity x Criticalityx Objective Weighting

Planned Test Effort figures give the proportion oftest effort that will spent in each function/area

Specify what a unit means Hour

Test case


21/26

Determining Criticality Other Criteria

Visibility Number of people who will see a failure

Reliability

Number of faults that have occurred in the past

Tolerability Rating how forgiving end users are to failure


22/26

Considerations... 1(2)

The testing target

Level of testing

Type of test

The quality of the system under test

Test environment setups (test tools)

Test maturity of the organisation

Scope of test requirement

Test engineer skill level


23/26

Considerations... 2(2)

Domain knowledge

Involve the team

Test iterations/Cycles (typical 3-4 iterations)

Defects found in previous testing/to be found

Time to report defects the more time spent onreporting the less testing you will actually do!


24/26

Finally...

Follow up your estimations

A combination of methods to get amore comprehensive and accurate

estimate


25/26

Questions?


26/26

Thank You!

Software Test Estimation

Documents