This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Why Can’t People Estimate: Estimation Bias and Strategic Mis-Estimation
Divide the problem into the lowest items. Estimate each item… sum the parts and add a factor
Complete WBS can be verified.
The whole is bigger than the sum of the parts.
Costs occur in items that are not considered in the WBS.
Design To Cost
Uses expert judgment to determine how much functionality can be provided for given budget.
Easy to get under stakeholder number.
Little or no engineering basis. Alwaysover original cost
Simple CER’s
Equation with one or more unknowns that provides cost / schedule estimate.
Some basis in data.
Simple relationships may not tell the whole story.Historical data may not tell the whole story.
Comprehensive Parametric Models
Perform overall estimate using design parameters and mathematical algorithms.
Models are usually fast and easy to use, and useful early in a program; they are also objective and repeatable.
Models can be inaccurate if notproperly calibrated and validated; historical data may not be relevant to new programs; optimism in parameters will lead to underestimation.
• Routinely exaggerate benefits and discount costsDelusions of Success: How Optimism Undermines
Executives' Decisions (Source: HBR Articles | Dan Lovallo, Daniel Kahneman | Jul 01, 2003)
7
Solution - Temper with “outside view”:Past Measurement Results, traditional forecasting, risk
analysis and statistical parametrics can help
Don’t remove optimism, but balance optimism withrealism
Cognitive Bias: How Fair Are We (Source BeingHuman.org)
• Cognitive bias: Tendency to make systematic decisions based on cognitive factors rather than evidence
• Human beings exhibit inherent errors in thinking
• Researchers theorize in the past, biases helped survival• Our brains using shortcuts (heuristics) that sometimes
provide irrational conclusions"We usually think of ourselves as sitting the driver's seat, with ultimate
control over the decisions we made and the direction our life takes; but, alas, this perception has more to do with our desires—with how we want to view ourselves—than with reality." Behavioral economist Dan Ariely
• Bias affects everything: from deciding how to handle our money, to relating to other people, to how we form memories
You would think this would help ensure viable estimates but… Its what we believe, not
necessarily what is reality
Negativity Bias (Being Human.org)
• Unconsciously pay give more weight to negative experiences than positive ones
• Brains react powerfully to negative information than they do to positive information
• Daniel Kahneman explained:
• “The brains of humans and other animals contain a mechanism that is designed to give priority to bad news. By shaving a few hundredths of a second from the time needed to detect a predator, this circuit improves the animal’s odds of living”
• More important for our ancestors to be able to avoid a threat quickly than to gain a reward
Business Case results were eliminated because of over-optimism in costs and over-optimism in benefit
10
Reference Class Forecasting (adapted from http://www.slideshare.net/assocpm/a-masterclass-in-risk)
• Best predictor of performance is actual performance of implemented comparable projects (Nobel Prize Economics 2002)
• Provide an “outside view” focus on outcomes of analogous projects
• “Reference Class Forecasting” attempts to force the outside view and eliminate optimism and misrepresentation• Choose relevant “reference class” completed analogous
projects
• Compute probability distribution
• Compare range of new projects to completed projects
Comparison of Parametric & Bottoms Up Methods (Source Hamaker)
Parametric Estimates
Benefits Top down Less detail Based on performance metrics Less labor intensive Quicker Ease of trade-offs analyses Generally more disciplined
• Standard methodology• Independent• Done by trained analysts• Captures totality of past
programs Issues Parametric database Not always
accepted “Black Magic” aura
Detailed Build-Up Estimates*
Benefits Bottoms up More detail Accepted method Generally understood Based on time and material Issues Labor intensive Time consuming Trade offs need details Performance standards More susceptible to distortions
• Optimism/Pessimism• Special interest/buy-in• Done by managers/engineers• Missing
• Be careful of red herring arguments against models• “We cannot model that…it is too complex.”
• “Models will have error and therefore we should not attempt it.”
• “We don’t have sufficient data to use for a model.”
• “It works but we cant see all data so we should not use it”
• Build on George E. P. Box: “Essentially, all models are wrong, but some are useful.”
• Some models are more useful than others
• Everyone uses a model – even if it is intuition or “common sense”
• So the question is not whether a model is “right” or whether to use a model at all
• Question is whether one model measurably outperforms another
• A proposed model (quantitative or otherwise) should be preferred if the error reduction compared to the current model (expert judgment, perhaps) is enough to justify the cost of the new model