257ln04

7/29/2019 257ln04

1/36

ECE257ECE257 NumericalNumerical MethodsMethods andandScientificScientific ComputingComputing

Error AnalysisError Analysis

7/29/2019 257ln04

2/36

ECE 257 Numerical Methods and Scientific Computing

Fall 2004

Lecture 4

John A. Chandy

Dept. of Electrical and Computer Engineering

University of Connecticut

TodayTodays class:s class:

Introduction to error analysisIntroduction to error analysis

ApproximationsApproximations Round-Off ErrorsRound-Off Errors

7/29/2019 257ln04

3/36


Fall 2004

Lecture 4

John A. Chandy



IntroductionIntroduction Error is the difference between the exact solutionError is the difference between the exact solution

and a numerical method solution.and a numerical method solution.

In most cases, you donIn most cases, you dont know what the exactt know what the exact

solution is, so how do you calculate the error.solution is, so how do you calculate the error. Error analysis is the process of predicting what theError analysis is the process of predicting what the

error will be even if you donerror will be even if you dont know what the exactt know what the exactsolutionsolution

Errors can also be introduced by the fact that theErrors can also be introduced by the fact that thenumerical algorithm has been implemented on anumerical algorithm has been implemented on acomputercomputer

7/29/2019 257ln04

4/36


Fall 2004

Lecture 4

John A. Chandy



Falling Object VelocityFalling Object Velocity

From Numerical Methods forEngineers, Chapra and Canale, Copyright The McGraw-Hill Companies, Inc.

7/29/2019 257ln04

5/36


Fall 2004

Lecture 4

John A. Chandy



Significant digitsSignificant digits

Can a number be used with confidence?Can a number be used with confidence?

How accurate is the number?How accurate is the number? How many digits of the number do we trust?How many digits of the number do we trust?

7/29/2019 257ln04

6/36


Fall 2004

Lecture 4

John A. Chandy





7/29/2019 257ln04

7/36


Fall 2004

Lecture 4

John A. Chandy




TheThe significant digitssignificant digits of a number are those that canof a number are those that can

be used with confidencebe used with confidence

The digits that are known with certainty plus oneThe digits that are known with certainty plus oneestimated digitestimated digit

Exact numbers vs. measured numbersExact numbers vs. measured numbers

Exact numbers have an infinite number of significantExact numbers have an infinite number of significant

digitsdigits is an exact number but usually subject to round-offis an exact number but usually subject to round-off

7/29/2019 257ln04

8/36


Fall 2004

Lecture 4

John A. Chandy





49.0 mph - 3 significant digits

87324.45 miles

7 significant digits

7/29/2019 257ln04

9/36


Fall 2004

Lecture 4

John A. Chandy



Accuracy and PrecisionAccuracy and Precision

AccuracyAccuracyis how close a computed oris how close a computed or

measured value is to the true valuemeasured value is to the true value

PrecisionPrecision is how close individual computedis how close individual computed

or measured values agree with each otheror measured values agree with each other

ReproducabilityReproducability

Inaccuracy/Bias vs. Imprecision/UncertaintyInaccuracy/Bias vs. Imprecision/Uncertainty

7/29/2019 257ln04

10/36


Fall 2004

Lecture 4

John A. Chandy





7/29/2019 257ln04

11/36


Fall 2004

Lecture 4

John A. Chandy




The level of accuracy and precision requiredThe level of accuracy and precision required

depend on the problemdepend on the problem

7/29/2019 257ln04

12/36


Fall 2004

Lecture 4

John A. Chandy



Error DefinitionsError Definitions

Two general types of errorsTwo general types of errors

Truncation errorsTruncation errors - due to approximations of- due to approximations of

exact mathematical functionsexact mathematical functions

EEtt = True value - approximation= True value - approximation

Round-off errorsRound-off errors - due to limited significant- due to limited significant

digit representation of exact numbersdigit representation of exact numbers

EEtt = True value - representation= True value - representation

7/29/2019 257ln04

13/36


Fall 2004

Lecture 4

John A. Chandy




EEtt does not capture the order of magnitudedoes not capture the order of magnitudeof errorof error

1 V error probably doesn1 V error probably doesnt matter if yout matter if youreremeasuring line voltage, but it does matter ifmeasuring line voltage, but it does matter ifyouyoure measuring the voltage supply to are measuring the voltage supply to aVLSI chip.VLSI chip.

Therefore, its better to normalize the errorTherefore, its better to normalize the errorrelative to the valuerelative to the value

t=

true error

true value100%

7/29/2019 257ln04

14/36


Fall 2004

Lecture 4

John A. Chandy



Error definitionsError definitions

Example:Example:

Line voltageLine voltage

Chip supply voltageChip supply voltage

Et=120V 119V =1V

Et = 3.3V 2.3V =1V

t=

1V

120V100% = 8.33%

t=

1V

3.3V100% = 30.3%

7/29/2019 257ln04

15/36


Fall 2004

Lecture 4

John A. Chandy



Error definitionsError definitions What if we donWhat if we dont know the true value?t know the true value?

Use an approximation of the true valueUse an approximation of the true value

How do we calculate the approximate error?How do we calculate the approximate error?

Use an iterative approachUse an iterative approach

Approximate error = current approximation - previousApproximate error = current approximation - previous

approximationapproximation

Assumes that the iteration will convergeAssumes that the iteration will converge

t= approximate error

approximate value100%

7/29/2019 257ln04

16/36


Fall 2004

Lecture 4

John A. Chandy




For most problems, we are interested in keepingFor most problems, we are interested in keeping

the error less than specified error tolerancethe error less than specified error tolerance

a<

s

How many iterations do you do, before youHow many iterations do you do, before yourere

satisfied that the result is correct to at least nsatisfied that the result is correct to at least n

significant digits?significant digits?

s= 0.5 x 10

2n( )%

7/29/2019 257ln04

17/36


Fall 2004

Lecture 4

John A. Chandy



ExampleExample

Infinite series expansion of eInfinite series expansion of exx

As we add terms to the expansion, theAs we add terms to the expansion, the

expression becomes more exactexpression becomes more exact

Using this series expansion, can weUsing this series expansion, can wecalculate ecalculate e0.50.5 to three significant digits?to three significant digits?

ex

=1+ x+x

2

2!

+

x3

3!

+L+

xn

n!

7/29/2019 257ln04

18/36


Fall 2004

Lecture 4

John A. Chandy



ExampleExample

Calculate the error toleranceCalculate the error tolerance

s= 0.5 x 10

23( )% = 0.05%

First approximationFirst approximation

e0.5

=1

e

0.5

=1+ 0.5 =1.5

Second approximationSecond approximation

a=

1.5 0.5

1.5= 33.3% > 0.05%

Error approximationError approximation

7/29/2019 257ln04

19/36


Fall 2004

Lecture 4

John A. Chandy



ExampleExample

e0.5

=1+ 0.5+0.5

2

2=1.625

Third approximationThird approximation

a=

1.6251.5

1.625= 7.69% > 0.05%

Error approximationError approximation

7/29/2019 257ln04

20/36


Fall 2004

Lecture 4

John A. Chandy



ExampleExample

0.0158%0.0158%0.00142%0.00142%1.6486971661.64869716666

0.158%0.158%0.0172%0.0172%1.64843751.648437555

1.27%1.27%0.175%0.175%1.6458333331.64583333344

7.69%7.69%1.44%1.44%1.6251.62533

33.3%33.3%9.02%9.02%1.51.522

39.3%39.3%1111

aattResultResultTermsTerms

True value is 1.6487212707

7/29/2019 257ln04

21/36


Fall 2004

Lecture 4

John A. Chandy



Round-Off ErrorsRound-Off Errors

Round-offRound-offerrors are caused because exacterrors are caused because exact

numbers can not be expressed in a fixed number ofnumbers can not be expressed in a fixed number of

digits as with computer representationsdigits as with computer representations

For example, a 32-bit number only has a rangeFor example, a 32-bit number only has a range

fromfrom2147483648 to 21474836472147483648 to 2147483647

For larger numbers or fractional quantities, we canFor larger numbers or fractional quantities, we can

use floating point numbers, but we will encounteruse floating point numbers, but we will encounterround-off errorsround-off errors

7/29/2019 257ln04

22/36


Fall 2004

Lecture 4

John A. Chandy



Floating-PointFloating-Point What can be represented in N bits?What can be represented in N bits?

UnsignedUnsigned 00 toto 22NN

2s Complement2s Complement - 2- 2N-1N-1 toto 22N-1N-1 - 1- 1

1s Complement1s Complement -2-2N-1N-1+1+1 to 2to 2N-1N-1-1-1

BCDBCD 00 toto 1010N/4N/4 - 1- 1

But, what about?But, what about?

very large numbers?very large numbers?

9,349,398,989,787,762,244,859,087,6789,349,398,989,787,762,244,859,087,678

very small number?very small number?0.00000000000000000000000456910.0000000000000000000000045691

rationalsrationals 2/32/3

irrationalsirrationals

transcendentalstranscendentals ee

N

2

7/29/2019 257ln04

23/36


Fall 2004

Lecture 4

John A. Chandy



Recall Scientific NotationRecall Scientific Notation

6.02 x 10 1.673 x 1023 -24

exponent

radix (base)Mantissa

decimal point

Sign, magnitude

Sign, magnitude

IEEE F.P. 1.M x 2e - 127

Issues:Issues:

Arithmetic (+, -, *, / )Arithmetic (+, -, *, / )

Representation, Normal formRepresentation, Normal form

Range and PrecisionRange and Precision

RoundingRounding

Exceptions (e.g., divide by zero, overflow, underflow)Exceptions (e.g., divide by zero, overflow, underflow)

ErrorsErrors

Properties ( negation, inversion, if AProperties ( negation, inversion, if A B then A - BB then A - B 0 )0 )

7/29/2019 257ln04

24/36


Fall 2004

Lecture 4

John A. Chandy



Floating-Point NumbersFloating-Point Numbers

Representation of floating point numbers in IEEE 754 standard:

single precision1 8 23

sign

exponent:excess 127binary integer

mantissa:sign + magnitude, normalizedbinary significand w/ hiddeninteger bit: 1.M

actual exponent ise = E - 127

S E M

N = (-1) 2 (1.M)S E-127

1 < E < 254 (E=0,255 are used for special values)

0 = 0 00000000 0 . . . 0 -1.5 = 1 01111111 10 . . . 0

7/29/2019 257ln04

25/36


Fall 2004

Lecture 4

John A. Chandy



Floating point numbersFloating point numbers

Problems:Problems:

Limited range of representable numbersLimited range of representable numbers

Overflow and underflowOverflow and underflow

Finite number of representable numbersFinite number of representable numbers

within rangewithin range

Interval between numbers increases asInterval between numbers increases asnumbers grow in magnitudenumbers grow in magnitude

7/29/2019 257ln04

26/36


Fall 2004

Lecture 4

John A. Chandy



Floating point numbersFloating point numbers

From Numerical Methods forEngineers, Chapra and Canale, The McGraw-Hill Companies, Inc.

7/29/2019 257ln04

27/36


Fall 2004

Lecture 4

John A. Chandy



Overflow and underflow errorsOverflow and underflow errors

Range of numbers for IEEE single-precisionRange of numbers for IEEE single-precision

is from 2is from 2-126-126 to (2-2to (2-2-23-23))22127127 or 1.18x10or 1.18x10-38-38 toto

3.40x103.40x103838

Overflow becomes infinity and underflowOverflow becomes infinity and underflow

becomes zero.becomes zero.

Double precision extends range to (2-2Double precision extends range to (2-2--5252))2210231023

7/29/2019 257ln04

28/36


Fall 2004

Lecture 4

John A. Chandy



Infinity andInfinity and NaNsNaNsresult of operation overflows, i.e., is larger than the largest number that

can be represented

overflow is not the same as divide by zero (raises a different exception)

+/- infinity S 1 . . . 1 0 . . . 0

It may make sense to do further computations with infinitye.g., X/0 > Y may be a valid comparison

Not a number, but not infinity (e.q. sqrt(-4))invalid operation exception (unless operation is = or =)

NaN S 1 . . . 1 non-zero

NaNs propagate: f(NaN) = NaN

HW decides what goes here

7/29/2019 257ln04

29/36


Fall 2004

Lecture 4

John A. Chandy



QuantizationQuantization

Finite number of representable numbers due toFinite number of representable numbers due toround-off orround-off orchoppingchopping

becomes 3.1416 or 3.1415 instead ofbecomes 3.1416 or 3.1415 instead of3.14159265353.1415926535

Due to base-conversion, rational numbers mayDue to base-conversion, rational numbers maybecome unrepresentable as well.become unrepresentable as well.

0.10.11010 becomes 0becomes 0.00011001100110011001100110.0001100110011001100110011022

Round-off is better than chopping. Upper boundRound-off is better than chopping. Upper bounderror oferror ofx/2 instead ofx/2 instead ofx.x.

7/29/2019 257ln04

30/36


Fall 2004

Lecture 4

John A. Chandy



Increasing IntervalIncreasing Interval

xx Allows floating point numbers to preserveAllows floating point numbers to preserve

significant digitssignificant digits

Quantization errors will be proportional toQuantization errors will be proportional to

magnitude of numbermagnitude of number

is the machine epsilon - measure of the precisionis the machine epsilon - measure of the precisionof the mantissaof the mantissa

x

x

for chopping

x

x

2for rounding

= b1 t with no implicit 1

= bt with implicit 1

7/29/2019 257ln04

31/36


Fall 2004

Lecture 4

John A. Chandy



Round-off errorsRound-off errors

Large numbers of interdependentLarge numbers of interdependent

computationscomputations

Adding a large and small numberAdding a large and small number

Subtractive calculationSubtractive calculation

Subtracting two nearly equal numbersSubtracting two nearly equal numbers

7/29/2019 257ln04

32/36


Fall 2004

Lecture 4

John A. Chandy



Real world examplesReal world examples

Why does round-off error matter?Why does round-off error matter?

Real world examplesReal world examples (from(fromhttp://mathworld.wolfram.com/RoundoffError.html)http://mathworld.wolfram.com/RoundoffError.html)

Vancouver Stock ExchangeVancouver Stock Exchange

Ariane rocketAriane rocket

Patriot missilePatriot missile

7/29/2019 257ln04

33/36


Fall 2004

Lecture 4

John A. Chandy



Vancouver Stock ExchangeVancouver Stock Exchange

In 1982, the index was initiated with a starting valueIn 1982, the index was initiated with a starting valueof 1000.000 with three digits of precision andof 1000.000 with three digits of precision andtruncationtruncation

After 22 months, the index was at 524.881After 22 months, the index was at 524.881

The index should have been at 1009.811.The index should have been at 1009.811.

Successive additions and subtractions introducedSuccessive additions and subtractions introduced

truncation error that caused the index to be off sotruncation error that caused the index to be off somuch.much.

7/29/2019 257ln04

34/36


Fall 2004

Lecture 4

John A. Chandy



Ariane rocketAriane rocket

Ariane 5 rocket was launched in June 1996.Ariane 5 rocket was launched in June 1996.

The rocket was on course for 36 seconds and thenThe rocket was on course for 36 seconds and then

veered off and crashedveered off and crashed The internal reference system was trying to convertThe internal reference system was trying to convert

a 64-bit floating point number to a 16-bit integer.a 64-bit floating point number to a 16-bit integer.This caused an overflow which was sent to the on-This caused an overflow which was sent to the on-board computerboard computer

The on-board computer interpreted the overflow asThe on-board computer interpreted the overflow asreal flight data and bad things happened.real flight data and bad things happened.

7/29/2019 257ln04

35/36


Fall 2004

Lecture 4

John A. Chandy



Patriot missilePatriot missile

The Patriot missile had an on-board timer thatThe Patriot missile had an on-board timer thatincremented every tenth of a secondincremented every tenth of a second

Software accumulated a floating point time value bySoftware accumulated a floating point time value byadding 0.1 secondsadding 0.1 seconds

Problem is that 0.1 in floating point is not exactlyProblem is that 0.1 in floating point is not exactly0.1. With a 23 bit representation it is really only0.1. With a 23 bit representation it is really only0.0999999046326.0.0999999046326.

Thus, after 100 hours (3,600,000 ticks), theThus, after 100 hours (3,600,000 ticks), thesoftware timer was off by .3433 seconds.software timer was off by .3433 seconds.

Scud missile travels at 1676 m/s. In .3433Scud missile travels at 1676 m/s. In .3433seconds, the Scud was 573 meters away fromseconds, the Scud was 573 meters away fromwhere the Patriot thought it was.where the Patriot thought it was.

7/29/2019 257ln04

36/36


Fall 2004

Lecture 4

John A. Chandy



Next LectureNext Lecture

Taylor SeriesTaylor Series

Truncation ErrorTruncation Error

Read Chapter 4Read Chapter 4

257ln04

Documents