Software Estimation

1

ESTIMATIONAuthor: Nguyen Phuc Hai

Created date: 1/9/2008

Agenda2

� What is estimation?

� Reasons to make wrong estimations

� Fundamental estimation Techniques

� Estimation process of Agile Mobile� Estimation process of Agile Mobile

� Politics, Negotiation, and Problem Solving

Estimates, Target and CommitmentsGood estimation

What is estimation3

Good estimation

Estimation4

� Estimation is a prediction of how long a project will take or how much it will cost

Target5

� A Target is a statement of a desirable business objective.

� Examples:� "We need to have Version 2.1 ready to demonstrate at a trade show in May."trade show in May."

� "We need to have this release stabilized in time for the holiday sales cycle.“

� Businesses have important reasons to establish targets independent of software estimates. But the fact that a target is desirable or even mandatory does not necessarily mean that it is achievable.

Commitment6

� A commitment is a promise to deliver defined functionality at a specific level of quality by a certain date. A commitment can be the same as the estimate, or it can be more aggressive or more conservative than the estimateconservative than the estimate

Distinguish between estimates, targets, and commitments.Distinguish between estimates, targets, and commitments.

7

Estimate, Target and Commitments

Good estimation

What is an estimation8

Good estimation

Good estimation9

� A good estimation approach should provide estimates that are within 25% of the actual results 75% of the time.

� However, accurate estimation results cannot be � However, accurate estimation results cannot be accomplished through estimation practices alone. They must also be supported by effective project

control.

Estimation and Project Control10

Estimation Track Record11

Project Outcomes by Project Size12

Estimation and Project Control13

� The criteria for a "good" estimate cannot be based on its predictive capability, which is impossible to assess, but on the estimate's ability to support project success

Reasons to make incorrect estimation

14

Source of estimation Uncertainly15

Cone of Uncertainly16

The cone doesn’t narrow itself17

Don't assume that the Cone of Uncertainty will narrow itself. You must force the Cone to narrow by removing sources of variability from your project.

The cone doesn’t narrow itself18

Estimation Error by Software-Development Activity

The cone doesn’t narrow itself19

� Relationship Between the Cone of Uncertainty and Commitment

�Meaningful commitments are not possible in the early, wide part of the Cone. Effective organizations delay their commitments until they have done the work to their commitments until they have done the work to force the Cone to narrow.

�Meaningful commitments in the early-middle part of the project (about 30% of the way in) are possible and appropriate.

Chaotic Development Process20

Common examples of project

chaotic21

� Requirements that weren't investigated very well in the first place

� Lack of end-user involvement in requirements validation

Poor designs that lead to numerous errors in the � Poor designs that lead to numerous errors in the code

� Poor coding practices that give rise to extensive bug fixing

� Inexperienced personnel

� Incomplete or unskilled project planning

� Abandoning planning under pressure

� Developer gold-plating

Unstable requirements22

Unstable requirements23

� If requirements cannot be stabilized, the Cone of Uncertainty can't be narrowed, and estimation variability will remain high through the end of the project.

To deal with unstable requirements,

consider project control strategies

instead of or in addition to estimation

strategies.

Omitted activities24

Software-Development Activities

Commonly Missing25

� Ramp-up time for new team members

� Mentoring of new team members

� Management coordination/manager meetings

� Requirements clarifications� Requirements clarifications

� Maintaining the revision control system

� Review of technical documentation

� Coordination with team members

Software-Development Activities

Commonly Missing26

� Technical support of existing systems during the project

� Maintenance work on previous systems during the projectproject

� Integration work

� Reviewing plans, estimates, architecture, detailed designs, stage plans, code, test cases, and so on

� Input to user documentation and review of user documentation

Non-Software-Development

Activities Commonly Missing 27

� Vacations and Holidays

� Sick days

� Company/Department meetings

� Hardware and Software problems� Hardware and Software problems

� Setting up new Workstations

Unfounded optimism28

Common optimistic issues29

� We'll be more productive on this project than we were on the last project.

� A lot of things went wrong on the last project. Not so many things will go wrong on this project.so many things will go wrong on this project.

� We started the project slowly and were climbing a steep learning curve. We learned a lot of lessons the hard way, but all the lessons we learned will allow us to finish the project much faster than we started it.

Common optimistic issues30

� We will recruit/staff the talents or experience to team and timeline could be reduced

� We have experience with that kind of project beforebefore

Subjectivity and Bias31

Off-The-Cuff Estimates32

Avoid Off-The-Cuff Estimation33

Average error of estimates in these 24 groups of estimators for off-the-cuff estimates versus estimates that go through a group review process.

Unwarranted Precision34

Accuracy PrecisionVS

Accuracy refers to how close to the real value a number is. Precision refers merely to how exact a number is. In software estimation, this amounts to how many significant digits an estimate has. A measurement can be precise without being accurate, and it can be accuratewithout being precise.

Unwarranted Precision35

VSHigh accuracy but low precision High precision but low accuracyVSExample Comments

This project will take 395.7 days, ± 2 months Highly precise, but not accurate to the precision stated

This project will take 1 year Not very precise, but could be accurate

This project will require 7,214 staff hours Highly precise, but probably not accurate to the precision stated

This project will require 4 staff years Not very precise, but could be accurate

Other sources of error36

� Unfamiliar business area

� Unfamiliar technology area

� Incorrect conversion from estimated time to project time (for example, assuming the project team will focus on the project eight hours per day, five days focus on the project eight hours per day, five days per week)

� Misunderstanding of statistical concepts (especially adding together a set of "best case" estimates or a set of "worst case" estimates)

� Budgeting processes that undermine effective estimation (especially those that require final budget approval in the wide part of the Cone of Uncertainty)

Other sources of error (cont.)37

� Having an accurate size estimate, but introducing errors when converting the size estimate to an effort estimate

� Having accurate size and effort estimates, but � Having accurate size and effort estimates, but introducing errors when converting those to a schedule estimate

� Overstated savings from new development tools or methods

� Simplification of the estimate as it's reported up layers of management, fed into the budgeting process, and so on

Estimate Influences38

Determinate factors to estimate39

� Project size

� Don't assume that effort scales up linearly as project size does. Effort scales up exponentially.

� The kind of software� The kind of software

� Personnel factors

� The programming language

Fundamental Estimation Techniques40

Choosing Estimation Techniques41

� Project size

� Software Development Style

� Development Stage

� Accuracy Possible� Accuracy Possible

• Count, Compute, Judge

Calibration and Historical Data

Fundamental Estimation Techniques42

• Calibration and Historical Data

Applicability43

Count Compute

What's estimated Size, Features Size, Effort, Schedule, Features

Size of project S M L S M L

Development Stage Early-Late Early-Middle

Iterative or sequential Both Both

Accuracy possible High High

Count First44

� If you can count the answer directly, you should do that first. That approach produced the most accurate answer in the story.

� If you can't count the answer directly, you should � If you can't count the answer directly, you should count something else and then compute the answer by using some sort of calibration data

Count if at all possible. Compute when you

can't count. Use judgment alone only as a last

resort.

What to count45

� Find something to count that's highly correlated with the size of the software you're estimating

� Find something to count that's available sooner rather than later in the development cyclethan later in the development cycle

� Understand what you're counting

� Find something you can count with minimal effort

Convert Counts to Estimates

Quantity to Count Historical Data Needed to Convert the Count to an Estimate

Marketing requirements • Average effort hours per requirement for development• Average effort hours per requirement for independent testing• Average effort hours per requirement for documentation• Average effort hours per requirement to create engineering requirements from marketing requirements

Features • Average effort hours per feature for development and/or testing

Use cases • Average total effort hours per use case• Average number of use cases that can be delivered in a particular amount of calendar time

46

• Average number of use cases that can be delivered in a particular amount of calendar time

Stories • Average total effort hours per story• Average number of stories that can be delivered in a particular amount of calendar time

Change requests • Average development/test/documentation effort per change request (depending on variability of the change requests, the data might be decomposed into average effort per small, medium, and large change request)

Function Points • Average development/test/documentation effort per Function Point• Average lines of code in the target language per Function Point

Defects found • Average effort hours per defect to fix• Average effort hours per defect to regression test• Average number of defects that can be corrected in a particular amount of calendar time

Test cases already

written

• Average amount of release-stage effort per test case

Use Judgment Only as a Last Resort47

� Expert judgment is the least accurate means of estimation. Estimates seem to be the most accurate if they can be tied to something concrete

• Count, Compute, Judge

Calibration and Historical Data

Fundamental Estimation Techniques48

• Calibration and Historical Data

Applicability49

Calibration with

Industry-Average Data

Calibration with

Organizational Data

Calibration with Project-Specific

Data

What's

estimated

Size, Effort, Schedule, Features

Size, Effort, Schedule, Features Size, Effort, Schedule, Features

Size of project S M L S M L S M L

Development

Stage

Early-Middle Early-Middle Middle-Late

Iterative or

sequential

Both Both Both

Accuracy

possible

Low-Medium Medium-High High

Data to collect50

� Size (lines of code or something else you can count after the software has been released)

� Effort (staff months)

� Time (calendar months)� Time (calendar months)

� Defects (classified by severity)

Improved Accuracy and Other

Benefits of Historical Data51

� Accounts for Organizational Influences

� How complex is the software, how much documentation is required, how precedented is the application

� Is the project manager free to remove a problem team member from the project, or do the organization's Human Resources policies make it difficult or impossible to remove a problem employee?

� Is the team free to concentrate on the current project, or are team members frequently interrupted with calls to support production releases of previous projects?

Improved Accuracy and Other

Benefits of Historical Data (Cont.)52

� Can the organization add team members to the new project as planned, or does it refuse to pull people off other projects before those projects have been completed?

� Does the organization support the use of effective � Does the organization support the use of effective design, construction, quality assurance, and testing practices?

� Can the project manager depend on team members staying until the project is complete, or does the organization have high turnover?

Improved Accuracy and Other

Benefits of Historical Data (Cont.)53

� Avoids Subjectivity and Unfounded Optimism

� Use historical data as the basis for your productivity assumptions. Unlike mutual fund disclosures, your organization's past performance really is your best indicator of future performance.indicator of future performance.

� Reduces Estimation Politics

� Use historical data to help avoid politically charged estimation discussions arising from assumptions like "My team is below average."

How to calibrate54

� The ultimate goal of collecting data is to convert the data to a model that you can use for estimation. Example:� Our developers average X lines of code per staff month.� Our team is averaging X staff hours per use case to create the use case, and Y hours per use case to construct and the use case, and Y hours per use case to construct and deliver the use case.

� Our testers create test cases at a rate of X hours per test case.

� In our environment, we average X lines of code per function point in C# and Y lines of code per function point in Python.

� On this project so far, defect correction work has averaged X hours per defect.

Using Project Data to Refine Your

Estimates55

� Even if you don't have historical data from past projects, you can collect data from your current project and use that as a basis for estimating the remainder of your project.

� Your goal should be to switch from using organizational data or industry-average data to project data as soon as you can. The more iterative your project is, the sooner you'll be able to do this.

Estimation process of Agile Mobile56

What is story point?57

� Story point is a random measure used by Scrum teams. This is used to measure the effort required to implement a story.

� Story points do give some indication of how much time was spent in a sprint.

� Example:� Example:

At the end of a 10 day sprint a team of 4 covers 40 story points.

10 days = 10 * 8 working hours = 80 hours. = 320 total hours as there are 4 developers. That equated to 160 pairing hours.

160 divided by 40 = 4 hours pair hours per story point.

Agile Mobile estimation method58

� We use the story point as the unit to measure the size of features

� Maintain the project/product historical data (size/effort/schedule/defects) to improve the (size/effort/schedule/defects) to improve the estimate precision

Politics, Negotiation, and Problem Solving59

Attributes of Executives60

1 Executives will always ask for what they want.

2 Executives will always probe to get what they want if they don't get it initially.

3 Executives will tend to probe until they discover your point of discomfort.

4 Executives won't always know what's possible, but they will know what would be good for the business if it were possible.

5 Executives will be assertive. That's how they got to be executives in the first place.

6 Executives will respect you when you are being assertive. In fact, they assume you will be assertive if you need to be.

7 Executives want you to operate with the organization's best interests at heart.

8 Executives will want to explore lots of variations to maximize business value.

9 Executives know things about the business, the market, and the company that you don't know, and they may prioritize your project's goals differently than you would.

10 Executives will always want visibility and commitment early (which would indeed have great business value, if it were possible).

Political Influences on Estimates61

� External Constraints

� Be aware of external influences on the target. Communicate that you understand the business requirements and their importance.

� Negotiating an Estimate vs. Negotiating a

Commitment

� You can negotiate the commitment, but don't negotiate the estimate.

Political Influences on Estimates62

� What to Do if Your Estimate Doesn't Get Accepted

� Let it be

� Responsibility of Technical Staff to Educate

Nontechnical StakeholdersNontechnical Stakeholders

� Educate nontechnical stakeholders about effective software estimation practices.

Problem Solving and Principled

Negotiation63

� Treat estimation discussions as problem solving, not negotiation.

� Recognize that all project stakeholders are on the same side of the table. same side of the table.

� Everyone wins, or everyone loses.

References64

References65

� Steve McConnell, Software Estimation: Demystifying the Black Art – Microsoft Press

� Mike Cohn, Agile Estimating and Planning – Prentice HallHall