How do we classify uncertainties? What are their sources? Lack of knowledge vs. variability.

Slide 1

How do we classify uncertainties? What are their sources?Lack of knowledge vs. variability.

What type of safety measures do we take?Design, manufacturing, operations & post-mortemsLiving with uncertainties vs. changing them

How do we represent random variables?Probability distributions and moments Uncertainty and Safety Measures

Reading assignmentS-K Choi, RV Grandhi, and RA Canfield, Reliability-based structural design, Springer 2007. Available on-line from UF library http://www.springerlink.com/content/w62672/#section=320007&page=1

Source: www.library.veryhelpful.co.uk/ Page11.htm

Classification of uncertaintiesAleatory uncertainty: Inherent variabilityExample: What does regular unleaded cost in Gainesville today?

Epistemic uncertainty Lack of knowledgeExample: What will be the average cost of regular unleaded January 1, 2014?

Distinction is not absoluteKnowledge often reduces variabilityExample: Gas station A averages 5 cents more than city average while Gas station B 2 cents less. Scatter reduced when measured from station average!

Source: http://www.ucan.org/News/UnionTrib/

The main classification of uncertainties is into aleatory and epistemic uncertainties. Aleatory uncertainties are due to inherent variability. The example here is of gas prices on a given day in the city of Gainesville. Somebody doing a survey of gas stations may summarize it by noting that the average is $2.25 a gallon for regular, with a standard deviation of 8 cents. When dealing with engineering systems, aleatory uncertainty is due to variability in manufacturing and material properties of nominally identical parts, or variability in operating conditions. It is sometimes called irreducible uncertainty because additional knowledge does not reduce it, but manufacturing variability, for example, can be reduced by investment in more stringent tolerances. Even for the gas price example, we may reduce the variability by learning which stations are the cheapest.Epistemic uncertainty reflects lack of knowledge. An example is predicting the average price of gas a year ahead. In engineering applications we use analytical and numerical models to estimate response and performance, and the epistemic uncertainty mostly reflects the limitations of the models.3A slightly differentuncertainty classification.

British Airways 737-400Distinction between Acknowledged and Unacknowledged errorsType of uncertaintyDefinitionCausesReduction measuresErrorDeparture of average from modelSimulation errors, construction errorsTesting and model refinementVariabilityDeparture of individual sample from averageVariability in material properties, construction tolerancesTighter tolerances, quality control

4For this course, we assume that our key epistemic uncertainty is due to errors, and so the main distinction is between error and variability. Furthermore, since we calculate uncertainty for engineering systems, we will usually deal with a population of nominally identical parts or vehicles. In such setting it is useful to make the distinction between error and variability based on which apply to the entire population and which apply to the individuals.Specifically we consider errors the departure of the average from the model. For example, the population may include 500 737-400 airliners. Our model predicts that they have a range of 3,000 miles, but they average only 2,900. The 100 mile departure is due to errors in our model that predicts range, or errors in the input to the model (for example, we have the wrong aircraft weight).Variability is the departure of the population from the average. In the case of the airliners it may reflect variability in the efficiency of their engines due to manufacturing or operating conditions.Errors are often divided into acknowledged and unacknowledged errors. Acknowledged errors are often reduced by refining a model. Unacknowledged ones may be uncovered an corrected by tests. Variability may be reduced by tighter manufacturing tolerance or quality control.Modeling and Simulation.

The epistemic uncertainties that we are mostly interested in are due to our attempts to simulate reality by a computerized model. This figure, originally from the Society of Computer simulation) but borrowed directly from Oberkampf et al., illustrates the processes responsible for simulation errors.The first step is modeling reality with a conceptual model, such as Newton laws or Einstein laws of motion to describe the trajectory of a rocket. The trajectory of the rocket will then require analysis leading to a set of differential equations that describe the motion. The process of model qualification requires checks to ensure that the model is appropriate. For example, if we model the rocket as a point mass, we will need some checks that its finite size is not likely to have a large effects on the solution. The next step is to write or select software to solve the conceptual model, e.g., the differential equations. The process of checking the fidelity of the software for providing the solution is called verification. It is typically done by comparing with analytical solutions or solutions obtained by other software.Verification only ensures that the software can provide an accurate solution to the conceptual model. Comparison with reality, via physical tests, checks the entire process and is called model validation.

Oberkmapf et al. Error and uncertainty in modeling and simulation, Reliability Engineering and System Safety, 75, 333-357, 2002

5Error modeling

Having errors in simulations is acceptable when the magnitude of the errors can be estimated, so that decisions based on the simulations take them into account. For example, if our errors in predicting the trajectory of a rocket are excessive, we may compensate by installing a control system that will measure errors and correct the trajectory accordingly.The processes of qualification, verification and validation often provide error estimates. For example, the validation test may have revealed 3% difference, and that may have been judged acceptable. Personal or group experience may provide additional information on error magnitude. For example, an analyst with experience in trajectory calculation, may have an idea how well a given model may approximate reality. However, both sources of error estimation most often are expressed as bounds on the error rather than more detailed assessment.We often settle on numerical models that are coarse or simplified because we need to analyze them many times, for example for optimization. 6Safety measures Design: Conservative loads and material properties, building block and certification tests.

Manufacture: Quality control.

Operation: Licensing of operators, maintenance and inspections

Post-mortem: Accident investigations

To counter the uncertainties we have a host of safety measures. At the design stage we add safety factors. For example in aircraft design, structures need to be designed to withstand loads that are 50% higher than the highest loads expected in service. Variability in material properties, are compensated for by taking conservative properties. For example, aircraft component that are not protected by redundancy are designed for by using A-basis properties. These have to be lower than 99% of the distrribution of material strength, with 95% confidence.

Manufacturing errors are compensated in part by quality control processes, and once a product goes into service manufacturing glitches and ageing effects are caught by inspections and maintenance. Operators and maintenance workers are often licensed by governmental agencies in order to reduce errors.

Finally, failures do happen, and when they are fatal they are often followed by accident investigations. These help limit the effect of systematic design or operation errors to a small number of cases, by leading to recalls or other preventive measures.7 Airlines invest in maintenance and inspections. FAA invests in certification of aircraft & pilots.

NTSB, FAA and NASA fund accident investigations.

Boeing invests in higher fidelity simulations and high accuracy manufacturing and testing. The federal government (e.g. NASA) invests in developing more accurate models and measurement techniques.

Many players reduce uncertainty in aircraft.

In general, safety measures often act by reducing uncertainty about the performance of a product. For aircraft, this uncertainty reduction is undertaken by a large number of players.

Federal agencies like NASA and the department of defense conduct or support research into developing more accurate models and measurement techniques. This are often implemented and refined further by engineering software developers, like NASTRAN.Aircraft manufacturers invest in high fidelity simulations, high accuracy manufacturing, and in rigorous testing of aircraft parts. This reduces both epistemic and aleatory uncertainties. The FAA oversees this process via certification.Airlines invest in maintenance and inspections to reduce uncertainties about the health of their airplanes, and in pilot training to reduce uncertainty about operations. The FAA helps airlines resist temptation to cut corners by overseeing and regulating the process.Finally, accident investigations reveal the cause of failure and often expose errors in simulations, design, maintenance and operation. These errors are then eliminated or reduced by recommended correctives.8Representation of uncertaintyRandom variables: Variables that can takemultiple values with probability assigned toeach value

Representation of random variablesProbability distribution function (PDF)Cumulative distribution function (CDF)Moments: Mean, variance, standard deviation, coefficient of variance (COV)

Probability density function (PDF)If the variable is discrete, the probabilities of each value is the probability mass function. For example, with a single die, toss, the probability of getting 6 is 1/6.If you toss a pair dice the probability of getting twelve (two sixes) is 1/36, while the probability of getting 3 is 1/18.The PDF is for continuous variables. Its integral over a range is the probability of being in that range.

One common way of describing a random variable is by giving the probability for each value it can take. For example, with a single die toss, the random variable may be the number on the top face of the die; the probability of taking 1,2,3,4,5. or 6 is 1/6. If we toss two dice, and the random variable is the sum of the two, then the probabilities of different outcomes are different. For example the probability of 12 is 1/36 (only one combination out of 36) while the probability of getting 11 is 1/18 (two ways 5,6 and 6,5). The function that gives the probability for each possible value is called probability mass function.For continuous variables we have instead a probability density function (PDF), whose integral over any range gives the probability of falling in that range. For example, the figure shows the PDF of a normal (Gaussian) distribution. The integral of the PDF over the center region (darker blue) is 0.5 for the top figure and 0.6827 for the bottom one (Figure from Wikipedia). Since the integral over a single point is zero, the probability of taking any given value is zero. For example, If the random variable is the top sustained wind speed of a hurricane hitting Gainesville Florida, the probability of that speed being exactly 85 MPH is zero, while the probability of its being between 84.5 and 85.5 is the integral of the PDF from 84.5 to 85.5.10HistogramsProbability density functions have to be inferred from finite samples. First step is histogram.Histogram divide samples to finite number of ranges and show how many samples in each range (box)Histograms below generated from normal distribution with 50 and 500,000 samples.

If the PDF is not known in advance we can get an idea of its shape from a sample by plotting a histogram. The histogram is obtained by dividing the overall range of the sample into a finite number of intervals (usually of equal size) and showing the number of samples in each interval.For example, the Matlab sequence z=randn(1,50)+10; hist(z,8); generates 50 samples from the normal distribution (see lecture on random variable distributions) and then divides the range to eight intervals. The resulting plot on the left, only vaguely resembles a normal distribution. With z=randn(1,500000)+10 (500,00 samples) we get a better resemblance, but we may have benefitted from more intervals (boxes).It is also worth noting that random variables can infinite range, and many common distributions, like the normal distribution do. For those distributions the probability of getting very large or very small values is usually small, so the larger the sample the larger the range of the sample. This is shown in the two figures in that the range of values for 500,000 samples is almost twice as large as the range for 50 samples.

11Number of boxes

12Histograms and PDFHow do you estimate the PDF from a histogram?Only need to scale.

13

Cumulative distribution functionIntegral of PDF

Experimental CDF from 500 samples shown in blue, compares well to exact CDF for normal distribution.

14Probability plotA more powerful way to compare data to a possible CDF is via a probability plot

15

MomentsMean

Variance

Standard deviation

Coefficient of variationSkewness

A compact way to give information about samples and distributions is to provide some of their moments. The first moment is the mean, and it is commonly denoted by the letter . The operation of integrating a function of a random variable times its PDF is also called calculating the expectation of the function. So the mean is also the expected value of the random variable.The square deviation of the random variable from its mean, or its second moment is called the variance. We normally use the square root of the variance, known as the standard deviation. Alternatively, we use the coefficient of variation, which is the ratio of the standard deviation to the mean.Higher order moments are also used, especially normalized central moments. They are centralized by taking them around the mean, and normalized by the standard deviation. The third normalized central moment is called skewness and it measures the asymmetry in the distribution. 16

problemsList at least six safety measures or uncertainty reduction mechanisms used to reduce highway fatalities of automobile drivers.

Give examples of aleatory and epistemic uncertainty faced by car designers who want to ensure the safety of drivers.

3. Let x be a standard normal variable N(0,1). Calculate the mean and standard deviation of sin(x)

Source: Smithsonian InstitutionNumber: 2004-57325

17