NuMeths[unit1]

Numerical Methods

Introduction to Numerical Methods Page 1

INTRODUCTION TO NUMERICAL METHODS

1.0 Introduction

Numerical methods are techniques by which mathematical problems are formulated so that they can be

solved with arithmetic operations, however, they involve large numbers of tedious arithmetic calculations.

With the availability of fast, digital computers, numerical methods of solving engineering problems became

more popular. In pre-computer era, engineers solved problems in the following manner:

• Solutions were derived for some problems using analytical, or exact methods.

• Graphical solutions were used to characterize the behavior of systems.

• Calculators and slide rules were used to implement numerical methods manually.

With this, significant amount of energy were expended on the solution technique itself, rather than on the

problem definition and interpretation. With fast and efficient computers, numerical methods can be

implemented and solutions obtained within a short span of time.

Figure 1. 1 Process of problem solving with (a) depicting the process pre-computer era and (b) when computers are made

available. The size of the box is proportional to the time spent for each process.

Numerical Methods

Introduction to Numerical Methods Page 2

Reasons why an engineer should study numerical methods:

• Numerical methods are extremely powerful problem-solving tools.

• In the field, there are occasions which an engineer will use an available prepackaged or “canned”

computer programs that involve numerical methods. However, not all problems can be solved by

these “canned” programs. Studying numerical methods can increase the capabilities of these

“canned” programs by customizing it according to the specifications of the problem.

• Numerical methods are an efficient vehicle for learning to use computers.

• Numerical methods provide a vehicle for you to reinforce your understanding of mathematics.

The following subject areas are covered in the study of numerical methods:

• Roots of equations – these problems are concerned with the value of a variable or a parameter that

satisfies a single nonlinear equation. These problems are especially valuable in engineering design

contexts where it is often impossible to explicitly solve design equations for parameters.

Figure 1. 2 Roots of equation.

• Systems of linear algebraic equations – these problems are similar in spirit to roots of equations in

the sense that they are concerned with values that satisfy equations. However, in contrast to

satisfying a single equation, a set of values is sought that simultaneously satisfies a set of linear

algebraic equations.

Figure 1. 3. Systems of linear algebraic equations.

Numerical Methods

Introduction to Numerical Methods Page 3

• Optimization – these problems involve determining a value or values of an independent variable

that correspond to a “best” or optimal value of a function. Thus, optimization involves identifying

maxima and minima.

Figure 1. 4. Optimization

• Curve fitting – the techniques developed for this purpose can be divided into two general

categories: regression and interpolation. Regression is employed where there is a significant

degree of error associated with the data, and the strategy is to derive a single curve that represents

the general trend. Interpolation is used when the objective is to determine intermediate values

between relatively error-free data, and the strategy is to fit a curve directly through the data points

and use the curve to predict the intermediate values.

Figure 1. 5. Curve fitting.

• Integration and differentiation - numerical integration is the determination of area under the curve,

while differentiation is finding the slope of the curve at various points.

Figure 1. 6. Integration.

Numerical Methods

Introduction to Numerical Methods Page 4

Figure 1. 7. Differentiation.

• Ordinary and partial differential equations – many physical laws are couched in terms of the rate of

change of the quantity rather than the quantity itself. If the quantity is one-dimensional, an ordinary

differential equation arises; a partial differential equation is obtained when the quantity is

multidimensional.

1.1 Mathematical Modeling and Engineering Problem Solving

A mathematical model can be broadly defined as the formulation or equation that expresses the essential

features of a physical system or process in mathematical terms. In a very general sense, it can be

represented as a functional relationship of the form

Dependentvariable f �independent

variables , parameters, forcingfunctions� (1.1)

where the dependent variable is a characteristic that usually reflects the behavior or state of the system;

the independent variables are usually dimensions, such as space and time, along which the system’s

behavior is being determined; the parameters are reflective of the system’s properties or composition; and

the forcing functions are external influences acting upon the system. The actual mathematical expression of

Equation 1.1 can range from simple algebraic relationship to large complicated sets of differential

equations.

Mathematical models of the physical world are characterized by the following:

• It describes a natural process or system in mathematical terms.

• It represents an idealization and simplification of reality.

• It yields reproducible results and consequently can be used for predictive purposes.

Numerical Methods

Introduction to Numerical Methods Page 5

Example 1.1 Construct a mathematical model for a falling parachutist and solve it analytically. The schematic is shown below. Assume that the mass of the parachutist is 68.1 kg and the drag coefficient is 12.5 kg/s and that the parachutist is initially at rest.

Answer: A free-falling body is acted upon by gravity, and by the Newton’s second law of motion, its acceleration, or

the rate of change of its velocityv, is given as dv

dt Fm

where F is the net force acting on the body and m is its mass. The net force acting on the body consists of

the pull of gravity FD taken positive and the drag force FUacting opposite the direction of the motion, or

F FD $ FU The pull of gravity or the weight is the force exerted due to the mass of the body, or

FD mg

where g 9.8 m/s&. The drag force FU is assumed to be directly proportional to the velocity of the body, or

FU cv where c is the drag coefficient in kg/s. Relating these to the acceleration of the body gives dv

dt mg $ cvm

Numerical Methods

Introduction to Numerical Methods Page 6

or dv

dt g $ cm v

which is a first-order linear differential equation. Its general analytical solution, assuming the body is initially at rest is given as

v't( gmc )1 $ exp )$ ct

m++

Inserting the given parameters, the analytical model for the falling parachutist is

v't( 53.39'1 $ exp'$0.18355t(( Solving for values of the velocity for different times generates the following table.

Plotting the values gives the following graph

Numerical Methods

Introduction to Numerical Methods Page 7

In the case of Example 1.1, the resulting differential equation can be solved analytically using elementary

methods. However, there are mathematical models which cannot be solved analytically. For these cases, a

numerical solution can be formulated.

Example 1.2 Formulate a numerical solution for the mathematical model of Example 1.1. Answer: To recall, the differential equation that describes the motion of the falling parachutist is dv

dt g $ cm v

The derivative dv/dt can be approximated as (see the figure below the equation)

dvdt . Δv

Δt v't012( $ v't0(t012 $ t0

where v't0( is the value of the veolocity at time t0 and v't012( is the value of the velocity at some late time

t012.

This equation is called the finite divided difference approximation of the derivative. This is substituted to the mathematical model derived for the falling parachutist. Thus,

v't012( $ v't0(t012 $ t0

g $ cm v't0(

Numerical Methods

Introduction to Numerical Methods Page 8

Rearranging,

v't012( v't0( 3 4g $ cm v't0(5 't012 $ t0(

which gives an idea on how to (numerically) compute the value of the velocity v't012( at time t012 using the previous value of the velocity v't0( at time t0. If one is given an initial value for velocity at some time t0, the value of the velocity for the later time t012 can be computed. Using this, the following table can be generated.

The plot of the numerical solution, together with the exact solution is shown below.

Numerical Methods

Introduction to Numerical Methods Page 9

From Example 1.2, the following ideas can be deduced:

• The approach used to compute for subsequent values of the velocity was

New value Old value 3 slope 9 step size

This approach is formally called the Euler’s method.

• The numerical solution of the differential equation is the approximation of the exact solution. This is

because a finite value for the time interval or the step size is used. The approximation can be

improved if the step size is made small; however this results to a increasing the number of

computations.

• In numerical methods, a trade-off between accuracy of the solution and the computational effort

exists. Thus, one has to compute more values to achieve more accurate solution.

Drill Problems 1.1 1. Rather than the linear relationship for the drag force employed in modeling the motion of a falling

parachutist of Example 1.1, use the second-order relationship

FU cv&

where c here is the second-order drag coefficient. (a) Obtain an analytical solution for the velocity of the body. Assume that the jumper has zero velocity

at t 0. Assuming that the mass of the jumper is 68.1 kg and the drag coefficient as 0.225 kg/m, compute the velocity for the first ten seconds starting from t 0 with two seconds interval.

(b) Formulate a numerical solution to the differential equation and compute for the velocity of the body for the same instances as that of (a).

(c) Recompute the numerical solution for (b), reducing the time intervals to one seconds. Compare the results with that of (b) in terms of their differences with the solution of (a).

2. The amount of a uniformly distributed radioactive contaminant contained in a closed reactor is

measured by its concentration c (in becquerel per liter or Bq/L). The contaminant decreases at a decay

rate proportional to its concentration. Use Euler’s method to numerically solve the model for t 0 up to t 1d (d days(, with the decay constant k 0.2 d<2, step size Δt 0.1 and at t 0, the

concentration of the contaminant is 10 Bq/L. 3. A storage tank as shown in the figure below contains a liquid at depth y where y 0 when the tank is

half full. Liquid is withdrawn at a constant flow rate Q to meet demands. The contents are resupplied at

a sinusoidal rate 3Q sin& t. A conservation model for this system is written as

d'Ay(dx 3Q sin& t $ Q

Numerical Methods

Introduction to Numerical Methods Page 10

or, since the surface area A is constant, dy

dx 3 QA sin& t $ Q

A

Use Euler’s method to solve for the depth y from t 0 to 10 d with a step size of 0.5 d . The

parameter values are A 1200 m& and Q 500 mB/d. Assume that the initial condition is y 0.

4. Suppose that a spherical droplet of liquid evaporates at a rate that is proportional to its surface area, dV

dt $kA

where V = volume (mmB), t = time (min), k = evaporation rate (mm/min), and A = surface area

(mm&). Use Euler’s method to compute the volume of the droplet from t 0 to 10 min using a step size of 0.25 min. Assume that k 0.1 mm/min and that the droplet initially has a radius of 3 mm. What is the radius of the droplet after ten minutes?

5. Newton’s law of cooling says that the temperature of a body changes at a rate proportional to the difference between its temperature and that of the surrounding medium (the ambient temperature) dT

dt $k'T $ TE(

where T = temperature of the body (F), t = time (min), k = proportionality constant (per min), and TE

= the ambient temperature (F). Suppose that a cup of coffee originally has a temperature of 68F. Use Euler’s method to compute the temperature from t 0 to 10 min using a step size of 1 min if

TE 21F and k 0.1/min.

Numerical Methods

Introduction to Numerical Methods Page 11

1.2 Programming and Software

In the previous article, a model for the free-falling motion of a parachutist was developed. It took the form of

the differential equation

dvdt g $ c

m v (1.2)

Also, a solution to this equation was developed by using a simple numerical approach called the Euler’s

method,

v012 v0 3 dv0dt Δt (1.3)

Given an initial condition, this equation can be repeatedly used to compute the velocity as a function of

time. But accurate solutions require very small step sizes and hence a lot of computational effort. With the

aid of a computer, the computations can be performed easily.

Computer programs are set of instructions that direct the computer to perform a certain task. Computer

programming can be narrowed down into the following topics:

• Simple information representation (contants, variables, and type declarations)

• Advanced information representation (data structure, arrays, and records)

• Mathematical formulas (assignment, priority rules, and intrinsic functions)

• Input and output

• Logical representation (sequence, selection, and repetition)

• Modular programming (functions and subroutines)

In the early days of computers, programmers usually did not pay much attention to whether their programs

were clear and easy to understand. Today, it is recognized that there are many benefits to writing

organized, well-structured code. Thus, structured programming came into place. Structured programming is

a set of rules that prescribe good style habits for the programmer. A key idea behind structured

programming is that any numerical algorithm can be composed using the three fundamental control

structures: sequence, selection, and repetition.

To demonstrate the idea of structured programming, flowcharts and pseudocodes are used. A flowchart is

a visual or graphical representation of an algorithm. The flowchart employs a series of blocks and arrows,

each of which represents a particular operation or step in the algorithm. The arrows represent the

sequence in which the operations are implemented. See Figure 1.8.

Numerical Methods

Introduction to Numerical Methods Page 12

Figure 1. 8. Symbols used in flowcharts.

Another approach used to express algorithms is called the pseudocode. This technique uses code-like

statements in place of the graphical symbols of flowcharts.

As stated before, structured programming requires effective use of the following control structures:

• Sequence. The sequence structure expresses the trivial idea that unless you direct it otherwise, the

computer code is to be implemented one instruction at a time.

Figure 1. 9. Sequence structure of computer programs.

Numerical Methods

Introduction to Numerical Methods Page 13

• Selection. The selection structure provides a means to split the program’s flow into branches based

on the outcome of a logical condition.

Figure 1. 10. Selection structure.

If there are multiple alternatives, the following selection structures can be used.

Numerical Methods

Introduction to Numerical Methods Page 14

Figure 1. 11.Supplementary selection or branches construct.

• Repetition. Repetition provides a means to implement instructions repeatedly. The resulting

constructs are called loops. There are two types of loops, depending on how they are terminated.

The first one is called a decision loop because it terminates based on the result of a logical

condition. The second one is called a count-controlled loopperforms a specified number of

repetitions or iterations.

Numerical Methods

Introduction to Numerical Methods Page 15

Figure 1. 12. A decision loop structure.

Figure 1. 13. A count-controlled structure.

Aside from structured programming, modular programming is also being practiced in the development of

computer programs. Modular programming refers to the practice of dividing the computer program into

modules, called procedures, designed to specifically perform a certain task. There are two types of

procedures commonly employed:

• Functions – which return a single result;

• Subroutines – which return several results.

Numerical Methods

Introduction to Numerical Methods Page 16

Modular programming has a number of advantages:

• The use of small, self-contained units makes the underlying logic easier to device and to

understand for both the developer and the user.

• Development, as well as maintenance and modification of the program is facilitated.

• Increases the ease with which a program can be debugged and tested.

Example 1.3 Develop a pseudocode for a computer program that will compute for the roots of a quadratic equation

ax& 3 bx 3 c 0. The algorithm is as follows: 1. Prompts the user for coefficients a, b and c. 2. Implements the quadratic formula, guarding against all eventualities. 3. Displays the solution. 4. Allows the user the option to return to step 1 and repeat the process.

Example 1.4 Develop a pseudocode for a computer program that will implement the Euler’s algorithm as applied to solving the differential equation describing the motion of the parachutist of Example 1.1.

In this course, Microsoft Excel VBA and the Mathscript mode of LabVIEW will be used to implement the

algorithms developed. See Figures 1.14 and 1.15 for implementation of pseudocodes in Excel and

Mathscript, respectively.

Drill Problems 1.2 1. Write a well-structured pseudocode to implement the flowchart depicted in the figure below.

Numerical Methods

Introduction to Numerical Methods Page 17

2. Develop a well-structured function procedure that is passed a numeric grade from 0 to 100 and returns a letter grade according to the following scheme:

3. The cosine function can be evaluated by the following infinite series:

cos x 1 $ x&2! 3 xH

4! $ xJ6! 3 K

Write an algorithm to implement this formula so that it computes and prints out the values of cos x as each term in the series is added. In other words, compute and print in sequence the values for

cos x 1

cos x 1 $ x&2

cos x 1 $ x&2! 3 xH

4!

up to the order term n of your choosing. Write the algorithm as a well-structured pseudocode.

4. The “divide and average” method, an old-time method for approximating the square root of any positive

number a can be formulated as

x012 x0 3 E

LM2

Write a well-structured pseudocode for the implementation of this algorithm. It should do the following:

• Allow for a user input a.Check whether the user input is valid. Display 0 when the input is 0 or not valid.

• Set a value of tolerance ( 10<N).

• Implement the formula. Repeat the calculation until the answer is correct to six decimal places.

• Display the answer. 5. Write a well-structured pseudocode that will compute the factorial of an input integer. Make sure that

the algorithm includes checks on valid values, as well as a correct output for 0!.

Numerical Methods

Introduction to Numerical Methods Page 18

Figure 1. 14. Pseudocode implementation in Excel VBA.

Numerical Methods

Introduction to Numerical Methods Page 19

Figure 1. 15. Pseudocode implementation in Mathscript.

Numerical Methods

Introduction to Numerical Methods Page 20

1.3. Approximations and Round-Off Errors

This section covers basic topics related to the identification, quantification, and minimization of these errors.

Two major forms of errors are discussed in this section and the next: round-off errors is due to the fact that

computers can represent only quantities with a finite number of digits; and truncation error is the

discrepancy introduced by the fact that numerical methods may employ approximations to represent exact

mathematical operations and quantities.

Whenever numbers are employed in computations, there should be an assurance that it can be used with

confidence. The concept of significant figure, or digit, has been developed to formally designate the

reliability of a numerical value. The significant digits correspond to the number of certain digits plus one

estimated digit.

The concept of significant figures has two important implications for the study of numerical methods:

• Numerical methods yield approximate results. Therefore, there should a criteria to specify the

confidence in the approximate result.

• Computers can only represent finite number of digits for a given number. Quantities such as e, π,

and √7 has infinite number of significant digits. Therefore, the computer will omit most of these

significant digits which will result in round-off errors.

The errors associated with both calculations and measurements can be characterized with regard to their

accuracy and precision. Accuracy refers to how closely a computed or measured value agrees with the true

value. Precision refers to how closely individual computed or measured values agree with each other.

Figure 1.16 explains the point.

Figure 1. 16. Illustration of accuracy and precision.

Numerical Methods

Introduction to Numerical Methods Page 21

Inaccuracy (also called bias) is defined as the systematic deviation from the true value. Imprecision (also

called uncertainty) refers to the magnitude of the scatter of the measurements or computed values.

Numerical methods should ensure that there is no bias nor uncertainty with the results for it to be

considered a valid solution.

Numerical errors arise from the use of approximations to represent exact mathematical operations and

quantities. These include round-off and truncation errors. For both types, the following relationships hold

true:

True value approximation 3 error

Thus, error is defined as the discrepancy between the true and approximated values, or

ES True $ approximate (1.4)

where ES is called the true or exact error. Errors are normalize to take into account the scale or magnitude

of the problem being considered. Thus, the true fractional relative error as a percentage is given as

eS ESTrue value 9 100% (1.5)

Example 1.5

The measured values of a bridge and a rivet are 9999 cm and 9 cm, respectively. If the true values are 10000 cm and 10 cm respectively, compute the true error and the true percent relative error for each case. Answers: For the bridge, ES 1 cm, eS 0.01% For the rivet, ES 1 cm, eS 10%

In most cases, the true value will never be known, unless there is an analytic solution to the problem. An

alternative for this is to normalize the error using the best available estimate of the true value, that is

eE approximate errorapproximate value 9 100% (1.6)

One of the challenges of numerical methods is to determine error estimates in the absence of knowledge

regarding the true value. That is why numerical methods employ different ways of determining the

approximate error depending on the algorithms used.

Numerical Methods

Introduction to Numerical Methods Page 22

Errors can be positive or negative, however, signs of the error may not play a role in the computation.

Therefore it is often useful to employ the absolute values of the errors and compare it with a certain

tolerance, or

|eE| V eW (1.7)

If this relationship holds, the results are assumed to be within the prespecified acceptable level eW. It is also

convenient to relate these errors to the number of significant figures in the approximation. Thus, it can be

shown that

eW '0.5 9 10&<X(% (1.8)

where n is the number of significant figures (at least).

In the following example, the Taylor series expansion for eL will be used to demonstrate errors.

Example 1.5 The Taylor series expansion for eL is

eL 1 3 x 3 x&2 3 xB

3! 3 K 3 xXn!

Using this expansion, determine the value of eY.N correct to three significant figures. Answer: Adding successive terms of the Taylor series expansion, the following table can be generated.

After six terms, eY.N 1.648697917 which is correct to five significant figures.

Numerical Methods

Introduction to Numerical Methods Page 23

Round-off Errors. Round-off errors result from the following situations:

• Irrational numbers such as π, e, and √7 cannot be expressed by a fixed number of significant

figures; hence, some of these significant figures are dropped.

• Computers use base-2 number representation; they cannot precisely represent certain base-10

numbers.

The discrepancy introduced by this omission of significant figures is called round-off error.

Computer Representation of Numbers. The base-10 number system is the number system that humans

are most familiar with. However, since computers are built on devices that acquires only two states (on /

off), computers represent numbers using a binary, or base-2 system.

Numbers in computers are represented in two forms: integer and floating-point representation. Integers

may be in signed or unsigned magnitude format. If the format is unsigned, all the bits representing the

number determine the magnitude of the number. If the format is signed, the most significant bit (left-most

bit) represents the sign of the number, with a 0 for a positive number and 1 for a negative. The remaining

bits determine the magnitude of the number.

Figure 1. 17. Weights of each binary bit position for an unsigned 8-bit binary number.

Figure 1. 18. Weights of each binary bit position for a signed 8-bit binary number.

Example 1.6

What is the number 10000101 (base-2) in decimal if it is written as (a) unsigned integer; (b) signed integer? Answers: (a) 3133; (b) $123

Numerical Methods

Introduction to Numerical Methods Page 24

On the other hand, fractional quantities are typically expressed in computers using floating-point form. In

this approach, the number is expressed as a fractional part, called a mantissa or significand, and an integer

part, called an exponent or characteristic, as in m 9 bZ, where m is the mantissa, b is the base of the

number system being used, and c is the exponent. The mantissa holds a value that is

1b [ m [ 1

Thus, for a base-10 number, the mantissa is between 0.1 and 1.

Figure 1. 19. Number representation using floating-point.

For computer systems, IEEE Standard 754 defines the standard for representing floating-point numbers. In

this standard,

• Single precision floating-point numbers, which uses 32 bits, has one sign bit, 8-bit biased exponent

and 24-bit mantissa (but only 23 bits are presented);

• Double precision floating-point numbers, which uses 64 bits, has one sign bit, 11-bit biased

exponent and a 53-bit mantissa (but only 52 bits are presented).

Example 1.7 Represent the decimal numbers 312 and $0.75 into single precision floating-point numbers. Answers:

Example 1.8 Create a hypothetical floating-point number set for a machine that stores information using 7-bit words. Employ the first bit for the sign of the number, the next three for the sign and magnitude of the exponent, and the last three for the magnitude of the mantissa.

Numerical Methods

Introduction to Numerical Methods Page 25

The following are some important points about floating-point numbers that are significant in relation to

computer round-off errors as observed from Example 1.8:

1. There is a limited range of quantities that may be represented.

2. There are only a finite number of quantities that can be represented within the range.

3. The interval between numbers increases as the numbers grow in magnitude.

Figure 1. 20. Problems associated with floating-point numbers.

Arithmetic Manipulations of Computer Numbers. Aside from the limitations of a computer’s number

system, the actual arithmetic manipulations involving these numbers can also result in round-off error.

• Addition and subtraction – when two floating-point numbers are added or subtracted, the mantissa

of the number with the smaller exponent is modified so that the exponents are the same. For

subtraction, the sign of the subtrahend is complemented. The results are then added and the

results are chopped off to the specified number of mantissa digits.

• Multiplication and division – when two floating-point numbers are multiplied (or divided), the

mantissas of the numbers are multiplied (divided) and the exponents added (or subtracted). Most

computers hold intermediate results in a double-length register (since multiplication will yield

results double the number of digits). The results are then chopped.

• Large computations – Certain methods require extremely large numbers of arithmetic

manipulations to arrive at their final results. In addition, these computations are often

Numerical Methods

Introduction to Numerical Methods Page 26

interdependent. Consequently, even though an individual round-off error could be small, the

cumulative effect over the course of a large computation can be significant.

Drill Problems 1.3 1. If |x| V 1 it is known that 1

1 $ x 1 3 x 3 x& 3 xB 3 K

For x 0.1, evaluate this series correct to seven significant figures. How many terms are required to achieve such accuracy?

2. Evaluate e<N using two approaches

e<L 1 $ x 3 x&2 $ xB

3! 3 K

and

e<L 1eL 1

1 3 x 3 L\& 3 L]

B! 3 K

and compare with the true value of 6.737947 9 10<B. Use 20 terms to evaluate the series and compute the true and relative errors as terms are added. 3. The infinite series

f'n( ^ 1kH

X

_`2

converges on a value of f'n( abcY as n approaches infinity. Evaluate this infinite series in single and

double precision to calculate f'n( for n 10000 by computing the sum from k 1 to 10000 and in reverse order. Also evaluate the true and approximate relative errors for each case. 4. Determine the roots of a quadratic equation having a 1, b 3000.001, and c 3. Use single and double precision arithmetic. Evaluate the true relative error and determine the cause of the discrepancy between the solutions. The true roots for the equation are x2 $0.001 and x& $3000.

5. For 100 equally spaced values of x between 10<c to 10<d.H, evaluate the expressions

y √x 3 4 $ √x 3 3 and

y 1√x 3 4 3 √x 3 3

Note that both expressions are mathematically equivalent. Using plotting tools, graph the function in the given interval. Which function is better in terms of resisting loss of significance?

Numerical Methods

Introduction to Numerical Methods Page 27

1.4. Truncation Errors and the Taylor Series

Truncation errors are those that result from using an approximation in place of an exact mathematical

procedure. In order to gain insight into the properties of such errors, the Taylor series will be used.

Taylor Series. The Taylor series provides a means to predict a function value at one point in terms of the

function value and its derivatives at another point. In particular, Taylor’s theorem states that any smooth

function can be approximated by a polynomial.

The Taylor’s theorem states that, if the function f and its first n 3 1 derivatives are continuous on an

interval containing a and x, then the value of the function at x is given by

f'x( f'a( 3 f e'a('x $ a( 3 f ee'a(2! 'x $ a(& 3 f 'B('a(

3! 'x $ a(B 3 K 3 f 'X('a(n! 'x $ a(X 3 RX (1.9)

where the remainder RX is defined as

RX g 'x $ t(Xn!

L

Yf 'X12('t( dt (1.10)

Equation 1.9 is called the Taylor series expansion of f'x(. If the remainder RX is omitted, Equation 1.9 is

the Taylor polynomial approximation to f'x(. The remainder RX has another form called the derivative or

the Lagrange form of the remainder, which can be derived from the integral mean-value theorem.

RX f 'X12('ξ('n 3 1(! 'x $ a(X12 (1.11)

where ξ is a point between the interval a and x such that the function f'x(, takes the average value f'ξ(.

Figure 1.21 depicts how Taylor series approximates the function. The zero-order approximation, involves

the first term of the series,

f'x012( . f'x0( (1.12)

Indicates that the value of the f at the new point is the same as its value at the old point. However, better

approximation will result if more terms of the Taylor series are added, that is, the first-order approximation,

f'x012( . f'x0( 3 f e'x('x012 $ x0( (1.13)

takes the value of f closer to the true values. Subsequent addition of terms,

Numerical Methods

Introduction to Numerical Methods Page 28

f'x012( . f'x0( 3 f e'x('x012 $ x0( 3 f ee'x0(2! 'x012 $ x0(& (1.14)

f'x012( . f'x0( 3 f e'x('x012 $ x0( 3 f ee'x0(2! 'x012 $ x0(& 3 f eee'x0(

3! 'x012 $ x0(B (1.15)

and so on results in better approximations, resulting in the complete Taylor series

f'x012( f'x0( 3 f e'x('x012 $ x0( 3 f ee'x0(2! 'x012 $ x0(& 3 f eee'x0(

3! 'x012 $ x0(B 3 K3 f 'X('x0(

n! 'x012 $ x0(X 3 RX

(1.16)

where RX is the remainder term that accounts for all the terms above n up to infinity.

Figure 1. 21. Taylor series approximation.

It is often convenient to simplify the Taylor series by defining a step size h x012 $ x0 and expressing

1.16 as

f'x012( f'x0( 3 f e'x(h 3 f ee'x0(2! h& 3 f eee'x0(

3! hB 3 K 3 f 'X('x0(n! hX 3 RX (1.17)

Numerical Methods

Introduction to Numerical Methods Page 29

where the remainder term is now expressed as

RX f 'X12('ξ('n 3 1(! hX12 (1.18)

Example 1.9 Use zero- through fourth-order Taylor series expansions to approximate the function

f'x( $0.1xH $ 0.15xB $ 0.5x& $ 0.25x 3 1.2

from x0 0 with h 1. Answers: Zero-order approximation : f'1( . 1.2

First-order approximation f'1( . 0.95 Second-order approximation: f'1( . 0.45

Third-order approximation: f'1( . 0.3 Fourth-order approximation: f'1( . 0.2 Note that the true value of the function at x 1 is 0.2, that is, the fourth-order approximation is the true value.

For an nth degree polynomial, the nth-order Taylor series approximation yields the exact value of the

polynomial at the point. For other differentiable and continuous functions, such as exponentials and

sinusoids, a finite number of term will not yield an exact result. Thus, truncation errors are incurred, in the

form of the remainder term RX.

Although the above is true, the practical value of the Taylor series expansions is that, in most cases, the

inclusion of only a few terms will result in an approximation that is close enough to the true value for

practical purposes. The assessment of how many terms are required to get “close enough” is based on the

remainder term of the expansion. However, the relationship of 1.18 has two major drawbacks:

• The number ξ is not known exactly but merely lies between x0 and x012.

• The 'n 3 1(th derivative of f'x( must be known, but f'x( is basically unknown.

Despite this, the remainder term is still useful for gaining insight into truncation errors, because h can be

chosen and hence the number of terms to be included in the expansion can be controlled. It can be shown

that the error is proportional to the step size h raised to the 'n 3 1(th power, or

RX O'hX12( (1.19)

Numerical Methods

Introduction to Numerical Methods Page 30

Example 1.10

Use Taylor series expansions with n 0 to 6 to approximate f'x( cos x at x012 aB, on the basis of

the value of f'x( and its derivatives at x0 aH.

Answer: The table below gives subsequent results of computations.

The Remainder Term For the Taylor Series Expansion. If the Taylor series expansion was truncated

after the zero-order approximation, the remainder term RX is

RY f e'x0(h 3 f ee'x0(2 h& 3 f eee'x0(

3! hB 3 K (1.20)

A graphical depiction of such truncation is shown in Figure 1.22.

Figure 1. 22. Graphical interpretation of zero-order truncation.

Numerical Methods

Introduction to Numerical Methods Page 31

Note that in this truncation, the lower-order derivatives usually account for the greater share in the

remainder term than the higher-order derivatives, although the results are still inexact because the second-

and higher-order terms are neglected.

According to the derivative mean-value theorem, if a function f'x( and its first derivative are continuous

over an interval from x0 to x012, then there exists a point on the function that has a slope, f e'ξ(, that is

parallel to the line joining f'x0( and f'x012(. The point ξ marks the x value where this slope occurs.

Figure 1. 23. Graphical interpretation of the derivative mean-value theorem.

Thus the zero-order remainder term, with this theorem can be written as

RY f e'ξ(h (1.21)

The higher-order remainder term can be derived in the same manner, for example, the first-order remainder

term as

R2 f ee'ξ(2 h& (1.22)

Using the Taylor Series To Estimate Truncation Errors. The Taylor series approach can be used to

numerically solve the problem of Example 1.1 (and hence reinforce the idea of the Euler’s method

presented in Example 1.2). The Taylor series expansion of the velocity as a function of time gives

v't012( v't0( 3 ve't0( j 't012 $ t0( 3 vee't0(2 j 't012 $ t0(& 3 K 3 RX (1.23)

Numerical Methods

Introduction to Numerical Methods Page 32

Truncating the series after the first derivative,

v't012( v't0( 3 ve't0( j 't012 $ t0( 3 R2 (1.24)

with which

ve't0( v't012( $ v't0(t012 $ t0

$ R2t012 $ t0

(1.25)

where the first term on the right of 1.25 depicts the first-order approximation term while the second one is

the truncation error. Thus, the algorithm used to estimate the derivative of ve't0( before now has an

estimate of the truncation error associated with this approximation, which, according to 1.22. is

R2t012 $ t0

vee'ξ(2 't012 $ t0( (1.26)

or

R2t012 $ t0

O't012 $ t0( (1.27)

which implies that the error is a function of the step size h t012 $ t0.

Example 1.11 Investigate the effect of increasing the value of m for the function f'x( xk to the truncation error when

Taylor series approximation is used to estimate the function. Note that as m increases, the function becomes more nonlinear. Also, for a particular value of m, say m 4, observe how the first-order truncation error changes as the step size is varied.

Note that Equation 1.25 is called the finite divided difference and can be generally represented as

f e'x0( f'x012( $ f'x0(x012 $ x0

3 O'x012 $ x0( (1.28)

or

f e'x0( Δf0h 3 O'h( (1.29)

where Δf0 is called the first forward difference and h is called the step size. This is one of many numerical

methods developed from the Taylor series to approximate derivatives numerically. The others are called

backward and centered difference approximations. Moreover, more accurate approximations of the first

Numerical Methods

Introduction to Numerical Methods Page 33

derivative can be developed by using higher-order terms of the Taylor series. Finally, all the above versions

can also be developed for second, third, and higher derivatives.

Figure 1. 24. Evaluation of first derivative using (a) forward; (b) backward; and (c) centered finite difference.

The Taylor series can be expanded backward to calculate a previous value on the basis of the present

value, as in

f'x0<2( f'x0( $ f e'x0( j h 3 f ee'x0(2 j h& $ K (1.30)

Truncating this equation after the first derivative and rearranging yields

f e'x0( f'x0( $ f'x0<2(h 3 O'h( lf2

h 3 O'h( (1.31)

where the error O'h( and lf2 is called the first backward difference

Numerical Methods

Introduction to Numerical Methods Page 34

A third way to approximate the first derivative is to subtract 1.31 from the forward Taylor series expansion

which will give the centered difference representation of the first derivative

f e'x0( f'x012( $ f'x0<2(2h $ O'h&( (1.32)

Example 1.12 Use forward and backward difference approximation of O'h( and a centered difference approximation of

O'h&( to estimate the first derivative of

f'x( $0.1xH $ 0.15xB $ 0.5x& $ 0.25x 3 1.2

at x 0.5 using a step size (a) h 0.5 and (b) h 0.25. Compare the true error in both cases. Answers: (a) For h 0.5

True error

Forward difference 58.9%

Backward difference 39.7%

Centered difference 9.6%

(b) For h 0.25

True error

Forward difference 26.5%

Backward difference 21.7%

Centered difference 2.4%

Total Numerical Error. The total numerical error is the summation of the truncation and round-off errors.

From the previous discussions, the following are observed:

• the only way to minimize round-off error is to increase the number of significant figures of the

computer.

• truncation error can be reduced by decreasing the step size.