Métodos Numéricos En Ing. Química Para programar

Contents

1 On models 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Classification of chemical process models . . . . . . . . . 41.3 Lumped parameter, steady state models . . . . . . . . . . 5

1.3.1 Example of a stagewise separation process . . . . 51.3.2 Process flow sheet simulation . . . . . . . . . . . . 91.3.3 Example of a multicomponent flash . . . . . . . . . 111.3.4 Example of a phenomenalogical model . . . . . . . 131.3.5 Example of reactors in series . . . . . . . . . . . . . 14

1.4 Lumped parameter, dynamic models . . . . . . . . . . . . 151.4.1 Example of cooling a molten metal . . . . . . . . . 151.4.2 Ozone decomposition . . . . . . . . . . . . . . . . . 16

1.5 Distributed parameter, steady state models . . . . . . . . 171.5.1 Heat transfer through a tappered fin . . . . . . . . 17

1.6 Distributed parameter, dynamic models . . . . . . . . . . 201.6.1 Heat transfer through a tappered fin . . . . . . . . 20

2 Single nonlinear algebraic equation 222.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.2 Bisection method . . . . . . . . . . . . . . . . . . . . . . . . 242.3 Regula-falsi method . . . . . . . . . . . . . . . . . . . . . . 292.4 Secant method . . . . . . . . . . . . . . . . . . . . . . . . . . 292.5 Newton’s method . . . . . . . . . . . . . . . . . . . . . . . . 312.6 Muller’s method . . . . . . . . . . . . . . . . . . . . . . . . . 322.7 Fixed point iteration . . . . . . . . . . . . . . . . . . . . . . 342.8 Error analysis and convergence acceleration . . . . . . . . 37

2.8.1 Convergence of Newton scheme . . . . . . . . . . . 402.9 Deflation technique . . . . . . . . . . . . . . . . . . . . . . . 412.10 Parameter continuation . . . . . . . . . . . . . . . . . . . . 43

2.10.1 Euler-Newton continuation . . . . . . . . . . . . . . 432.10.2 Homotopy continuation . . . . . . . . . . . . . . . . 43

i

CONTENTS ii

2.11 Software tools . . . . . . . . . . . . . . . . . . . . . . . . . . 432.11.1 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . 432.11.2 Mathematika . . . . . . . . . . . . . . . . . . . . . . . 44

2.12 Exercise problems . . . . . . . . . . . . . . . . . . . . . . . 452.12.1 Multicomponent, isothermal flash model . . . . . . 452.12.2 Compressible flow in a pipe . . . . . . . . . . . . . 462.12.3 A model for separation processes . . . . . . . . . . 472.12.4 Peng-Robinson Equation of State . . . . . . . . . . . 492.12.5 Transient heat conduction in semi-infinite slab . . 502.12.6 Turbulent flow in a parallel pipeline system . . . . 51

3 Review of Linear Algebra 533.1 Matrix notation . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.1.1 Basic operations . . . . . . . . . . . . . . . . . . . . . 543.2 Matrices with special structure . . . . . . . . . . . . . . . . 573.3 Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

3.3.1 Laplace expansion of the determinant . . . . . . . 593.4 Direct methods . . . . . . . . . . . . . . . . . . . . . . . . . 60

3.4.1 Cramers rule . . . . . . . . . . . . . . . . . . . . . . . 603.4.2 Matrix inverse . . . . . . . . . . . . . . . . . . . . . . 613.4.3 Gaussian elimination . . . . . . . . . . . . . . . . . . 643.4.4 Thomas algorithm . . . . . . . . . . . . . . . . . . . 703.4.5 Gaussian elimination - Symbolic representaion . . 713.4.6 LU decomposition . . . . . . . . . . . . . . . . . . . 75

3.5 Iterative methods . . . . . . . . . . . . . . . . . . . . . . . . 783.5.1 Jacobi iteration . . . . . . . . . . . . . . . . . . . . . 803.5.2 Gauss-Seidel iteration . . . . . . . . . . . . . . . . . 813.5.3 Successive over-relaxation (SOR) scheme . . . . . . 823.5.4 Iterative refinement of direct solutions . . . . . . . 84

3.6 Gram-Schmidt orthogonalization procedure . . . . . . . . 853.7 The eigenvalue problem . . . . . . . . . . . . . . . . . . . . 86

3.7.1 Wei-Prater analysis of a reaction system . . . . . . 863.8 Singular value decomposition . . . . . . . . . . . . . . . . 863.9 Genaralized inverse . . . . . . . . . . . . . . . . . . . . . . . 863.10 Power iteration . . . . . . . . . . . . . . . . . . . . . . . . . 863.11 Software tools . . . . . . . . . . . . . . . . . . . . . . . . . . 86

3.11.1 Lapack, Eispack library . . . . . . . . . . . . . . . . . 863.11.2 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . 863.11.3 Mathematica . . . . . . . . . . . . . . . . . . . . . . . 86

3.12 Exercise problems . . . . . . . . . . . . . . . . . . . . . . . 863.12.1 Laminar flow through a pipeline network . . . . . 86

CONTENTS iii

4 Nonlinear equations 894.1 Newton’s method . . . . . . . . . . . . . . . . . . . . . . . . 904.2 Euler-Newton continuation . . . . . . . . . . . . . . . . . . 964.3 Arc-length continuation . . . . . . . . . . . . . . . . . . . . 964.4 Quasi-Newton methods . . . . . . . . . . . . . . . . . . . . 96

4.4.1 Levenberg-Marquadt method . . . . . . . . . . . . . 964.4.2 Steepest descent method . . . . . . . . . . . . . . . 964.4.3 Broyden’s method . . . . . . . . . . . . . . . . . . . 96

4.5 Exercise problems . . . . . . . . . . . . . . . . . . . . . . . 964.5.1 Turbulent flow through a pipeline network . . . . 96

5 Functional approximations 985.1 Approximate representation of functions . . . . . . . . . 99

5.1.1 Series expansion . . . . . . . . . . . . . . . . . . . . 995.1.2 Polynomial approximation . . . . . . . . . . . . . . 99

5.2 Approximate representation of data . . . . . . . . . . . . . 1045.2.1 Least squares approximation . . . . . . . . . . . . . 105

5.3 Difference operators . . . . . . . . . . . . . . . . . . . . . . 1075.3.1 Operator algebra . . . . . . . . . . . . . . . . . . . . 1085.3.2 Newton forward difference approximation . . . . . 1095.3.3 Newton backward difference approximation . . . . 112

5.4 Inverse interpolation . . . . . . . . . . . . . . . . . . . . . . 1165.5 Lagrange polynomials . . . . . . . . . . . . . . . . . . . . . 1185.6 Numerical differentiation . . . . . . . . . . . . . . . . . . . 122

5.6.1 Approximations for first order derivatives . . . . . 1225.6.2 Approximations for second order derivatives . . . 1245.6.3 Taylor series approach . . . . . . . . . . . . . . . . . 126

5.7 Numerical integration . . . . . . . . . . . . . . . . . . . . . 1265.7.1 Romberg Extrapolation . . . . . . . . . . . . . . . . 1295.7.2 Gaussian quadratures . . . . . . . . . . . . . . . . . 1325.7.3 Multiple integrals . . . . . . . . . . . . . . . . . . . . 132

5.8 Orthogonal functions . . . . . . . . . . . . . . . . . . . . . 1325.9 Piecewise continuous functions - splines . . . . . . . . . . 132

6 Ordinary differential equations - Initial value problems 1336.1 Model equations and initial conditions . . . . . . . . . . . 133

6.1.1 Higher order differential equations . . . . . . . . . 1346.2 Taylor series expansion . . . . . . . . . . . . . . . . . . . . 135

6.2.1 Alternate derivation using interpolation polynomials 1356.2.2 Stability limits . . . . . . . . . . . . . . . . . . . . . . 1376.2.3 Stiff differential equations . . . . . . . . . . . . . . 139

6.3 Multistep methods . . . . . . . . . . . . . . . . . . . . . . . 141

CONTENTS iv

6.3.1 Explicit schemes . . . . . . . . . . . . . . . . . . . . 1416.3.2 Implicit schemes . . . . . . . . . . . . . . . . . . . . 1426.3.3 Automatic stepsize control . . . . . . . . . . . . . . 143

6.4 Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . 1456.4.1 Explicit schemes . . . . . . . . . . . . . . . . . . . . 1466.4.2 Euler formula revisited . . . . . . . . . . . . . . . . 1476.4.3 A two-stage (v = 2) Runge-Kutta scheme . . . . . . 1476.4.4 Semi-implicit & implicit schemes . . . . . . . . . . . 1506.4.5 Semi-Implicit forms of Rosenbrock . . . . . . . . . 151

6.5 Dynamical systems theory . . . . . . . . . . . . . . . . . . 152

7 Ordinary differential equations - Boundary value problems 1537.1 Model equations and boundary conditions . . . . . . . . . 1537.2 Finite difference method . . . . . . . . . . . . . . . . . . . . 156

7.2.1 Linear problem with constant coefficients . . . . . 1567.2.2 Linear problem with variable coefficients . . . . . . 1577.2.3 Nonlinear problem . . . . . . . . . . . . . . . . . . . 159

7.3 Quasilinearization of nonlinear equations . . . . . . . . . 1617.4 Control volume method . . . . . . . . . . . . . . . . . . . . 1637.5 Shooting method . . . . . . . . . . . . . . . . . . . . . . . . 1637.6 Collocation methods . . . . . . . . . . . . . . . . . . . . . . 1637.7 Method of weighted residuals . . . . . . . . . . . . . . . . 163

A ChE Network A.1A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . A.1A.2 Userid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.5A.3 Overview of the network . . . . . . . . . . . . . . . . . . . . A.5A.4 The client-server model . . . . . . . . . . . . . . . . . . . . A.8

A.4.1 File servers . . . . . . . . . . . . . . . . . . . . . . . . A.8A.4.2 License servers . . . . . . . . . . . . . . . . . . . . . A.9A.4.3 X servers . . . . . . . . . . . . . . . . . . . . . . . . . A.11A.4.4 News server . . . . . . . . . . . . . . . . . . . . . . . A.13A.4.5 Gopher service . . . . . . . . . . . . . . . . . . . . . A.15A.4.6 World wide web service . . . . . . . . . . . . . . . . A.16

A.5 Communication . . . . . . . . . . . . . . . . . . . . . . . . . A.16A.5.1 Software tools . . . . . . . . . . . . . . . . . . . . . . A.16A.5.2 Connection to the AIX machines from OS/2 machines A.17A.5.3 Connection to the AIX machines from a home com-

puter . . . . . . . . . . . . . . . . . . . . . . . . . . . A.19A.5.4 Connection to the AIX machine from a DOS machine

using tn3270 . . . . . . . . . . . . . . . . . . . . . . . A.20A.5.5 File transfer with Kermit . . . . . . . . . . . . . . . A.21

CONTENTS v

A.5.6 File transfer with ftp . . . . . . . . . . . . . . . . . . A.22A.5.7 File transfer from AIX to OS/2 machines . . . . . . A.23

A.6 Operating systems . . . . . . . . . . . . . . . . . . . . . . . A.23A.6.1 How to signon and logout . . . . . . . . . . . . . . . A.23A.6.2 Customizing your environment - the .profile file . A.24A.6.3 File management . . . . . . . . . . . . . . . . . . . . A.24A.6.4 How to get online help . . . . . . . . . . . . . . . . . A.25

A.7 Editors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.26A.7.1 Emacs - the ultimate in editors . . . . . . . . . . . . A.26

A.8 Fortran compilers . . . . . . . . . . . . . . . . . . . . . . . . A.30A.9 Debuggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.31A.10Application programs . . . . . . . . . . . . . . . . . . . . . A.32

A.10.1ASPEN PLUS . . . . . . . . . . . . . . . . . . . . . . . A.32A.10.2Xmgr . . . . . . . . . . . . . . . . . . . . . . . . . . . A.32A.10.3TEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.33A.10.4pine - the mail reader . . . . . . . . . . . . . . . . . A.33A.10.5tin - the news reader . . . . . . . . . . . . . . . . . . A.34

A.11Distributed Queueing System . . . . . . . . . . . . . . . . . A.34A.11.1Example of a CPU intensive MATLAB job . . . . . . A.35A.11.2Example of a FLOW3D job . . . . . . . . . . . . . . . A.36

A.12Printing reports, graphs etc. . . . . . . . . . . . . . . . . . A.36A.12.1Using the network printer from AIX machines . . A.38A.12.2Using the network printer from OS/2 machines . . A.39

A.13Anonymous ftp service . . . . . . . . . . . . . . . . . . . . A.39

B MATLAB primer B.1B.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . B.1B.2 Starting a MATLAB session . . . . . . . . . . . . . . . . . . B.2

B.2.1 Direct access on AIX machines . . . . . . . . . . . . B.2B.2.2 Remote access from OS/2 machines or home com-

puter . . . . . . . . . . . . . . . . . . . . . . . . . . . B.3B.3 MATLAB basics . . . . . . . . . . . . . . . . . . . . . . . . . B.4

B.3.1 Using built in HELP, DEMO features . . . . . . . . . B.5B.3.2 Data entry, line editing features of MATLAB . . . . B.8B.3.3 Linear algebra related functions in MATLAB . . . . B.13B.3.4 Root finding . . . . . . . . . . . . . . . . . . . . . . . B.15B.3.5 Curve fitting . . . . . . . . . . . . . . . . . . . . . . . B.16B.3.6 Numerical integration, ordinary differential equations B.16B.3.7 Basic graphics capabilities . . . . . . . . . . . . . . B.17B.3.8 Control System Toolbox . . . . . . . . . . . . . . . . B.19B.3.9 Producing printed output of a MATLAB session . . B.21B.3.10 What are m-files? . . . . . . . . . . . . . . . . . . . . B.23

CONTENTS vi

B.3.11 Programming features . . . . . . . . . . . . . . . . . B.25B.3.12 Debugging tools . . . . . . . . . . . . . . . . . . . . . B.26

C Unix primer C.1C.1 Introduction to the shell and the desktop . . . . . . . . . C.1

C.1.1 The ".profile" file . . . . . . . . . . . . . . . . . . . . C.2C.2 Managing files . . . . . . . . . . . . . . . . . . . . . . . . . . C.4

C.2.1 Making sense of the directory listing - the "ls" commandC.4

C.2.2 Changing permission on files - the "chmod" command C.6C.2.3 Moving files . . . . . . . . . . . . . . . . . . . . . . . C.6C.2.4 Copying files . . . . . . . . . . . . . . . . . . . . . . C.7C.2.5 Changing ownership of files - the "chown" command C.8C.2.6 Compressing files - the "compress" command . . C.8C.2.7 Removing files - the "rm" command . . . . . . . . C.8

C.3 Managing processes . . . . . . . . . . . . . . . . . . . . . . C.9C.3.1 Examining jobs or processes - the "ps" command C.9C.3.2 Suspending jobs or processes . . . . . . . . . . . . C.10C.3.3 Terminating jobs or processes - the "kill" command C.10C.3.4 Initiating background jobs - the "nohup" command C.10C.3.5 Script files & scheduling jobs - the "at" command C.11

C.4 List of other useful AIX command . . . . . . . . . . . . . . C.12

Bibliography R.1

List of Tables

2.1 Example of fixed point iteration . . . . . . . . . . . . . . . 372.2 Feed composition & equilibrium ratio of a natural gas mix-

ture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.1 Loss of precision and need for pivoting . . . . . . . . . . . 71

5.1 Convergence of P2n+1(x) to erf(x) at selected values of x 1005.2 Saturation temperature vs. pressure from steam tables . 1055.3 Inverse interpolation . . . . . . . . . . . . . . . . . . . . . . 1185.4 Summary of difference approximations for derivatives . 124

A.1 List of machines in the chemical and materials engineeringnetwork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.7

A.2 List of software on the chemical and materials engineeringnetwork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.12

A.3 List of ftp commands . . . . . . . . . . . . . . . . . . . . . A.22A.4 List of frequently used emacs functions and their key bind-

ings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.29

B.1 List of MATLAB Ver 4.0 help topics . . . . . . . . . . . . . B.6B.2 General purpose MATLAB Ver 4.0 commands . . . . . . . B.7B.3 Linear algebra related functions in MATLAB . . . . . . . . B.14B.4 Graphics related function in MATLAB . . . . . . . . . . . . B.18B.5 List of functions from control system tool box . . . . . . B.22B.6 Program control related help topics . . . . . . . . . . . . . B.27

C.1 List of other useful AIX command . . . . . . . . . . . . . . C.13

vii

List of Figures

1.1 Scheme of things - a personal view . . . . . . . . . . . . . 21.2 Three stage separation process . . . . . . . . . . . . . . . . 51.3 Linear and nonlinear equilibrium model . . . . . . . . . . 81.4 Example of material balance equations in a process flow

sheet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.5 Multicomponent, isothermal flash process . . . . . . . . . 121.6 Reactors in series . . . . . . . . . . . . . . . . . . . . . . . . 141.7 Heat transfer from a molten metal . . . . . . . . . . . . . 161.8 Heat transfer through a fin . . . . . . . . . . . . . . . . . . 18

2.1 Graphs of some simple functions . . . . . . . . . . . . . . 232.2 Graphical representation of some simple root finding algo-

rithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.3 MATLAB implementation of the bisection algorithm . . . 272.4 MATLAB implementation of the secant algorithm . . . . 302.5 Graphical representation of Muller’s scheme . . . . . . . 332.6 Graphical representation of fixed point scheme . . . . . . 362.7 Graphical illustration of mean value theorem . . . . . . . 382.8 Graphical illustration deflation technique . . . . . . . . . 422.9 Multicomponent flash function . . . . . . . . . . . . . . . . 452.10 Turbulent flow in a parallel pipe . . . . . . . . . . . . . . . 51

3.1 MATLAB implementation of inverse of an upper triangularmatrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.2 Naive Gaussian elimination shceme . . . . . . . . . . . . . 673.3 MATLAB implementation of naive Gaussian elimination . 683.4 Thomas algorithm . . . . . . . . . . . . . . . . . . . . . . . 723.5 MATLAB implementation of Thomas algorithm . . . . . . 733.6 MATLAB implementation of LU decomposition algorithm 773.7 MATLAB implementation of Gauss-Seidel algorithm . . . 833.8 Laminar flow in a pipe network . . . . . . . . . . . . . . . 86

viii

LIST OF FIGURES 1

4.1 MATLAB implementation of Newton scheme . . . . . . . 934.2 CSTR in series example - function & Jacobian evaluation 95

5.1 Error distribution of ε(x,n) := |erf(x) − P2n+1(x)| fordifferent levels of truncation . . . . . . . . . . . . . . . . . 100

5.2 MATLAB implementation illustrating steps of functional ap-proximation . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

5.3 Structure of Newton forward difference table form equallyspaced data . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

5.4 Example of a Newton forward difference table . . . . . . 1125.5 Structure of Newton backward difference table for 5 equally

spaced data . . . . . . . . . . . . . . . . . . . . . . . . . . . 1145.6 Example of a Newton backward difference table . . . . . 1155.7 Example of a inverse interpolation . . . . . . . . . . . . . . 1165.8 Structure of divided difference table for 4 unequally spaced

data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1205.9 MATLAB implementation of Lagrange interpolation polyno-

mial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1215.10 Illustration of Romberg extrapolation . . . . . . . . . . . . 1315.11 MATLAB implementation of quadrature evaluation . . . . 132

6.1 Spring and dash pot model . . . . . . . . . . . . . . . . . . 1396.2 Stepsize control strategies for multistep methods . . . . 1446.3 MATLAB implementation of ozone decomposition model 1456.4 Results of ozone decomposition model shows a stiff system

behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

7.1 One dimensional finite difference grid of equally spaceddata points . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

7.2 One dimensional control volume discretization . . . . . . 163

A.1 Anatomy of a Motif window . . . . . . . . . . . . . . . . . . A.4A.2 Logical dependencies between various machines in the net-

work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.6A.3 Network File System server and clients . . . . . . . . . . . A.9A.4 Network Information Service . . . . . . . . . . . . . . . . . A.10A.5 X-Windows client-server model . . . . . . . . . . . . . . . . A.14A.6 Anatomy of an emacs window . . . . . . . . . . . . . . . . A.27

C.1 Output of the "ls" command . . . . . . . . . . . . . . . . . C.5

To see a World in a Grain of Sand,And a Heaven in a Wild Flower,Hold Infinity in the palm of your hand,And Eternity in an hour

— WILLIAM BLAKE

Chapter 1

The physical, mathematical andcomputational models

1.1 Introduction

In the Trekie language, the prime directive of a chemical engineer shouldbe explore and understand physical and chemical processes involved inconverting a raw material into a useful product. Use this knowledge indesigning, constructing and operating chemical process plants. This def-inition is as arbitrary as anything else that one might propose. In fact ifone substitutes the word chemical in the above definition with any other(such a mechanical or electrical) it would remain equally valid. This isbecause the basic principles and the scientific methodology we use touncover such principles remain the same in any field of engineering orscience. A broader, although highly personal, view of the our attemptto understand the physical world, describe it in the language of math-ematics, and finally investigate its consequence by means of analytical,graphics or numerical methods is shown in Figure 1.1.

A mathematical model is at best an approximation to the physicalworld. Such models are constructed based on certain conservation prin-ciples and/or empirical observations. Those curious about the nature ofthe physical laws should read a delightful little book by Feynman (1967)on the character of physical laws. As a matter of convenience mathemat-ical models can be classified as linear or non-linear, steady-state or dy-

1

1.1. INTRODUCTION 2

Physical world

Empirical observationsbased on

(physcical experiments)(need a good eye)

Mathematical model(an approximation to the physical world)

Numerical model(an approximation to the mathematical model)

Conceptual modelling based on

thought experimentsIntroduce concepts like force, energy etc.

(need a good mind)

For Chemical engineers this world consists of process

plants involving transport, reaction & equilibrium

processes

examplesASPEN, HYSIM,

PROCESS, FLOW3D, HCOMP, SPEEDUP,

DREM

Linear modelsIn light of ignorance we tend tend

to assume linearity(dating back to flat earth model!)

Non-linear modelsIn chemical process models nonlinearity arises primarily in the description of equilibrium models (phase behavior), reaction rate models or fluid flow models. Before the computer era (<1950) graphical methods were often used to solve such problems. They have been replaced largely by computer simulation tools.

Steady state models Lumped - algebraic equations

Distributed - ordinary differential equations(BVP) - partial differential equations (elliptic)

Dynamic modelsLumped - ordinary differential equations(IVP)

Distributed - partial differential equations (parabolic)

Classification II

Construct Conservation laws(e.g. mass, energy)

orPhenomenological models

(e.g. equation of state)

Construct empirical relationships like y=f(x)

between observed (y) and manipulated (x) variables

(curve fitting)

Input Output

Classification I

Figure 1.1: Scheme of things - a personal view

1.1. INTRODUCTION 3

namic, lumped or distributed. Examples to illustrate each type of modelare provided later in this chapter. In general, non-linearity is found tooccur quite naturally and frequently in nature; it is also very difficult toanalyse non-linear models without the aid of a computer.

A numerical model (or a computer simulation tool) is an approxima-tion to the mathematical model. Although the importance of mathemat-ical modelling in chemical engineering was recognized since the early1920’s, it was the text by Bird et al. (1960) on Transport Phenomenathat has proved to be the major inspiration in exploring the link betweenthe physical world and the mathematical one for transport processes in-volving momentum, heat and mass transfer. Since then a number ofoutstanding texts have appeared that explore this relationship for reac-tion and equilibrium processes as well. While such studies form the coreof chemical engineering curriculum, the importance of sharpening ourmathematical abilities, and the need to incorporate these as part of thecurriculum was recognized and aided by the appearance of early textbooks by Amundson (1966) and Jenson & Jeffreys (1963). These dealtspecifically with mathematical applications of chemical engineering. Thetexts by Lapidus (1962) and Rosenbrock (1966) served a similar purposein introducing digital computational methods in the analysis of chemicalprocesses.

We are now at a new threshold; computers have become quite per-vasive. Significant advances have been made in our ability to analysenon-linear systems. The advances in both the hardware and softwaretechnology have been revolutionary. As a result of these advances, com-puter aided design and analysis has become a standard tool as evidencedby the success of several commercial packages such as ASPEN PLUS, PRO-CESS, HYSIM (steady state process simulators), FLOW3D, FLUENT (fluiddynamics simulators), HCOMP, DREM (multiphase and transient pipelineflow simulators) etc. In addition to such simulators that are specific forcertain classes of problems, general purpose mathematical tools such asMATLAB (for matrix linear algebra functions), Mathematika and MAPLE(for symbolic computation) provide easy access to a vast array of mathe-matical functions and the ability to process them both symbolically andnumerically. Such packaged tools tend to accomplish the following: (i)codify the most advanced algorithms, (ii) assemble a vast database (inthe case of physical properties) and knowledge base in the case of math-ematical functions (in MAPLE and Mathematika) and (iii) make these ac-cessible to the end user through an intuitive interface. While this puts alot of power at the hands of the end user, in order to use them wisely andinterpret the results correctly, the users are expected to have a soundknowledge of the relationship between the physical world and the mathe-

1.2. CLASSIFICATION OF CHEMICAL PROCESS MODELS 4

matical model and that between the mathematical model and the numer-ical approximation. One is well served to remember the cliche garbagein, garbage out!

In this course we examine the link between the mathematical andthe numerical model. In so doing modern computational tools are usedquite liberally. Concepts of a networked computer environment are dis-cussed in Appendix A. A basic introduction to MATLAB can be foundin Appendix B. MATLAB is used throughout in illustrating various algo-rithms.

1.2 Classification of chemical process models

In modelling chemical processes, one is interested in tracking materialand energy of process streams from the raw material stage to the finishedproduct state. The state of a stream is characterized by the concentra-tion of the various species that it carries and its temperature, pressureand flow rates. Applying the laws of conservation of mass, energy andmomentum allows us to track changes in the state of the system. Typi-cally we subject the raw material streams to either physical treatment toadd or remove chemical species exploiting such property differences asdensity, solubility, volatility, diffusivity etc. (transport and equilibriumprocesses) or, chemical treatment to alter the chemical structure (reac-tion processes).

If the state variables are assumed to be independent of time and spa-tial position, then we often have a lumped parameter, steady state modelresulting in a set of coupled algebraic equations. If they are assumed tohave no spatial variation, but are time dependent, then we have lumpedparameter, dynamic models which result in ordinary differential equa-tions of the initial value type. If there is no time dependence, but there isa spatial variation and that too restricted to one dimension (for reasonsof symmetry or scale), then we have ordinary differential equations ofthe boundary value type. If both spatial and time dependence are im-portant, then we end up with partial differential equations, which arefurther classified into parabolic, elliptic and hyperbolic equations. Theclassification outlined in this paragraph are illustrated with specific ex-amples in the next sections. The examples are drawn from transport,equilibrium and reaction processes. The objective is to sensitize you tothe model building process in the hope that you would begin to appre-ciate the relationship between the physical world and the mathematicalmodel that represents it.

1.3. LUMPED PARAMETER, STEADY STATE MODELS 5

1 2 3

L, x0

V, y1

L, x3

V, y4

x1 x2

y2 y3

Figure 1.2: Three stage separation process

1.3 Lumped parameter, steady state models

1.3.1 Example of a stagewise separation process

Consider a stagewise separation process shown in Figure 1.2. Supposewe wish to process a gas stream at a rate of V kmole/s containing apollutant at a mole fraction of y4. We wish to remove the pollutantby scrubbing it with a solvent in a counter-current 3-stage separationdevice. The liquid rate is, say, L kmole/s and it contains a pollutant con-centration of x0 (which may be zero for a pure solvent stream). Only thepollutant transfers from the gas phase to the liquid phase and we makeuse of the solubility differences between the inert carrier gas and thepollutant. So far we have made an attempt to describe a physical world.Is the description adequate to formulate a mathematical model? Howdo we know that such a model should result in a steady state, lumpedparameter model? The answer is no, we don’t! We need to further de-fine and refine the problem statement. For a process engineer this is themost important step - viz. understand the objective of the task and thenature of the process (the physical world) to formulate a mathematicalmodel. Let us continue with the description of the problem.

The state variables in this problem are (L, V ,x0, x1, x2, x3, y1, y2, y3, y4).By focusing only the steady state operation, we remove the dependenceof state variables on time. Such a model cannot clearly answer any ques-tions concerning start up or shutdown of this process. Next, we assumethat in each stage the gas and liquid are mixed thoroughly so that thereis no spatial variation of concentration within the equipment. This is theso-called lumped parameter approximation.

We further assume that the streams leaving a stage are in thermody-namic equilibrium. This implies that for a given inlet streams, no matterwhat we do inside the process equipment, the exit concentrations cannotbe changed as they have reached an invariant state. To state it anotherway, there is a unique relationship, y = f(x), between the exit concen-


trations of each stage and this relationship could be determined in alaboratory and entered into a database. Often this relation is expressedas, y = Kx where K is called the equilibrium ratio; at extremely low con-centration range K may be assumed to be a constant (results in a linearmodel), while at higher concentrations the equilibrium ratio may itself bea function of concentration, K(x) (results in a non-linear model). Whileexperience and experimentation suggest that such relationships do ex-ist, study of equilibrium thermodynamics takes this one step further inattempting to construct predictive models for the function, y = f(x) byexamining the equilibrium process at a molecular level. Let us continuewith the assumption that the equilibrium model is given by

yn = Kxn n = 1,2,3 (1.1)

where we have introduced the subscript n to indicate the equilibriumrelationship is valid for each stage of the separation process. This yieldsus three equations, but recall that the state of this 3-stage separationtrain is described by 10 variables: (L, V ,x0, x1, x2, x3, y1, y2, y3, y4). Atthis stage we ask ourselves if there are other laws or principles that thissystem should obey. Conservation laws such as mass, momentum andenergy conservation should come to mind. In the present case our ob-jective has been narrowly focused on tracking the concentration of thepollutant in each of the three stages. In particular we have not concernedourselves with flow and heat effects. Let us speculate briefly what theseeffects might be! Heat transfer effects might include heat of absorption,while flow effects will include imperfect mixing in a stage. The laterin fact has serious consequence in negating two of our earlier assump-tions: viz. lumped parameter system implying concentration is spatiallyuniform in each stage and the exit streams are in thermodynamic equi-librium. Nevertheless, we still proceed with the assumption of perfectmixing; a model description that takes into accounts such realities oftenbecomes intractable. Neglecting heat and flow effects, we have only massconservation principle. Applying this for the pollutant species aroundeach of the three stages, we obtain,

Stage 1: V(y2 −y1) = L(x1 − x0)Stage 2: V(y3 −y2) = L(x2 − x1) (1.2)

Stage 3: V(y4 −y3) = L(x3 − x2)

Note that in each of these equations, the left hand side represents theamount of pollutant that has been removed from the gas phase and theright hand side represents the same amount of material absorbed into


the liquid phase. Now we have a total of six equations, but still tenvariables. Hence we conclude that we have four degrees of freedom.This implies that we can choose four of the variables and the remainingsix variables must be determined by satisfying the six equations.

One can also write an overall balance, taking all three stages as onegroup:

Overall: V(y4 −y1) = L(x3 − x0) (1.3)

This, however, is not an independent equation since summing equations(1.2) produces equation (1.3). This will be used later in introducing con-cepts of linear independence and rank of matrices.

Specifications: a 3-stage linear system

Let us assume that we pick the four variables associated with the inletstreams to be specified, viz. (L, V ,x0, y4). Defining S = L/KV (a knownvalue) and eliminating variables (y1, y2, y3) from equations (1.2) we getthe following system of three equations

(1+ S) −1 0

−S (1+ S) −10 −S (1+ S)

x1

x2

x3

=

Sx0

0y4/K

(1.4)

in the unknowns (x1, x2, x3). This can be represented in compact matrixform as

Tx = b (1.5)

where T represents the tridiagonal matrix

T = d1 c1 0a1 d2 c2

0 a2 d3

=

(1+ S) −1 0

−S (1+ S) −10 −S (1+ S)

and

x = x1

x2

x3

b =

b1

b2

b3

=

Sx0

0y4/K

First variation: n-stage separation sequence

Once we have expressed the mathematical model in a symbolic, matrixform as in equation (1.5), we can generalize the model to any number of


x x

y

y =

f(x)

(a) linear model

y 1=

K(x 1

) x 1

(b) nonlinear model

x1

y= K

xK=constant

Figure 1.3: Linear and nonlinear equilibrium model

stages. Suppose there are n stages, then we merely have,

T =

d1 c1 0 · · · 0a1 d2 c2 · · · 0

. . .0 0 an−2 dn−1 cn−1

0 · · · 0 an−1 dn

x =

x1

x2...

xn−1

xn

b =

b1

b2...

bn−1

bn

where ai = −S, di = (1 + S), ci = −1. Efficient algorithms for solvingsuch system will be developed in Chapter 3.

Second variation: nonlinear equilibrium model

Instead of assuming the equilibrium ratios, K in equation (1.1) to beconstant as in figure 1.3a, if they are found to be dependent on concen-trations x, then we have a nonlinear system of equations. One can theninterpret the K(x) values to be the slopes of the chord as shown in fig-ure 1.3b, which is no longer a constant, but depends on x. This impliesthat, S(x) = L/K(x)V and hence T becomes a function of x. Thus theelements in T cannot be determined without knowing the solution x.An intutive approach to resolving this dilemma in solving such systems,T(x)x = bmight be to make an initial guess for xold and use this guessto evaluate, K(xold), S(xold) and hence T(xold) and obtain a new solu-tion for xnew by solving the linearized system, T(xold)xnew = b. Onecan repeat this procedure until the difference between the new and theold values of x becomes vanishingly small. Although there are numer-ous variations on this scheme, a large class of non-linear problems are


solved within the conceptual frame work (i) estimating an initial guess(ii) devising an algorithm to improve the estimate and (iii) checking forconvergence of the result.

Third variation: alternate specification

In all of the previous cases we considered the inlet streams to be spec-ified viz. (L, V ,x0, y4). This would be typical for performance analysisproblems where the output of an existing process is desired, given itsinlet conditions. A design engineer, who gets into this game at an ear-lier stage, might face an alternate problem. For example, environmentalregulations might dictate that the exit concentration of the pollutant y1

be below a certain acceptable level. Thus the four degrees of freedommight be used up in specifying (V ,x0, y4, y1). Assuming once again alinear equilibrium model (K constant), the system of equations (1.2) inthe unknown set (L,x2, x3) can be written as:

f1(L,x2;V,y1, x0) := V(Kx2 −y1)− L(y1/K − x0) = 0

f2(L,x2, x3;V,y1, x0) := KV(x3 − x2)− L(x2 −y1/K) = 0 (1.6)

f3(L,x2, x3;V,y4) := V(y4 −Kx3)− L(x3 − x2) = 0

In spite of assuming a linear equilibrium model, the above set of equa-tions are non-linear! Why? Although the mathematical model has re-mained the same for various specifications, we have nice tridiagonal ma-trix structure for some specifications while no such structure exists forothers.

1.3.2 Process flow sheet simulation

Consider the flow sheet shown in figure 1.4. It is an extremely simpleunit consisting of a reactor and a separator. We are given the mole frac-tions of components in the recycle stream and the exit stream from thereactor. We are asked to determine the molar rates of CO and H2 in theinlet stream, the recyle rate R and the product rate, P . In analysing thisproblem we setup a series of material balance equations. Focusing onthe reactor loop (loop 1) shown in dashed line in figure 1.4, we can writethe following three component material balance equations:

C-balance: x + R(0.302+ 0.004) = (0.274+ 0.095)× 275

or

x + 0.306R = 101.475


x: COy: H2

Reactor CondenserP (rate): CH3OH

R (recycle rate)

Find x, y, R, PF = 275 mol/minmole fr of H2 = 0.631mole fr of CO= 0.274

mole fr of CH3OH= 0.095

mole fr of H2 = 0.694mole fr of CO= 0.302

mole fr of CH3OH= 0.004

loop 1

loop 2

Figure 1.4: Example of material balance equations in a process flowsheet.

H2-balance: y + R(0.694+ 0.008) = (0.631+ 2× 0.095)× 275

or

y + 0.702R = 225.775

O-balance: x + R(0.302+ 0.004) = (0.274+ 0.095)× 275

or

x + 0.306R = 101.475

Note that the O balance equation is redundant and in the language oflinear algebra, these three equations do not form a linearly independentset of equations. So we proceed to construct additional equations byexamining material balance around the full flow sheet (loop 2). Thesegive rise to:

C-balance: x = P

H2-balance: y = 2P


These five equations can be arranged in a matrix form as,

1 0 0.306 00 1 0.702 01 0 0.306 01 0 0 −10 1 0 −2

xyRP

=

101.475225.775101.475

00

(1.7)

Recognizing the redundancy between the first and third equations andalso combining equations four and five to eliminate P , we can write theabove set in an alternate form as

1 0 0.3060 1 0.702−2 1 0

xyR

=

101.475

225.7750

(1.8)

1.3.3 Example of a multicomponent flash

Next, we examine a model for a multicomponent, isothermal flash pro-cess. This also results in a lumped, steady state model description. Itis also an example of how a potentially large system system of algebraicequations can be reduced to a single equation in one unknown throughclever manipulations. Thus root finding algorithms could be used effi-ciently to solve this system. A sketch of the process is shown in figure1.5. A feed stream of known flow rate, F , composition (mole fractions),zi|i = 1 · · ·N, temperature, TF and pressure, PF is flashed into a drummaintained at a temperature and pressure of (T , P), respectively. Underright conditions, the feed will split into a vapor phase and a liquid phase.The objective is to predict the flow rate and compositions of the vapor,(V ,yi) and the liquid (L,xi) phases. Each exit stream contains (N + 1)unknowns. The assumptions are that the process is operating understeady conditions, perfect mixing takes place inside the drum (lumpedapproximation) and the exit streams are in thermodynamic equilibrium.The model equations are as follows:

Thermodynamic equilibrium - (empirical model)

yi = Ki(T , P)xi i = 1 · · ·N (1.9)

Component material balance - (mass conservation)Observe thenonlinear terms inthis equation: viz.product ofunknowns V & yiand L & xi

Fzi = Vyi + Lxi i = 1 · · ·N (1.10)


FZi i=1 .. NTFPF

Vyi i=1 .. N

Lxi i=1 .. N

ψ

f(ψ)

3 2 1 0 1 2 330

20

10

0

10

20

30

40Non-physical solution

Real solution

Figure 1.5: Multicomponent, isothermal flash process

Overall material balance

F = V + L (1.11)

Mole fraction constraints

N∑iyi =

N∑ixi = 1 (1.12)

A simple count indicates that we have written down (2N + 3) equationsfor the (2N + 2) unknowns; but it is easy to verify that summing equa-tions (1.10) over all components and using the mole fraction constraint,results in equation (1.11). Thus, equation (1.11) is not an independentone. Although these equations could be solved as a system of nonlinearalgebraic equations, a much more efficient scheme is to eliminate all,except one variable and reduce the system into a single equation. Firsteliminate yi from equation (1.10) using (1.9) to obtain

Fzi = (KiV + L)xi or xi = Fzi/(KiV + L)

Rearrange equation (1.12) as,

N∑i(xi −yi) = 0 or

N∑i(1−Ki)xi = 0


Combine the last two equations as,

N∑i

(1−Ki)FziKiV + L = 0

Eliminate L from above equation using (1.11) and define ψ = V/F to getthe final form of the flash equation as.

N∑i

(1−Ki)zi(Ki − 1)ψ+ 1

= 0 (1.13)

This is a single equation in one unknown, viz. ψ. In general the numberof roots that a nonlinear equation posses cannot be known a priori. Apossible sketch of the function is shown in figure 1.5b. Sinceψ is definedas the fraction of feed that appears as vapor, (V/F), the physical worlddictates that it must lie between (0,1) and it is sufficient if the searchfor the root is limited to this range. The flash equation (1.13)may possesother roots outside the range of interest (0,1). Such roots are validmathematical solutions of the problem, they are not physically relevant.

1.3.4 Example of a phenomenalogical model

In the previous two examples, models were built based on conservationlaws. Models based on empirical observations are also quit common.The Pressure-Volume-Temperature (PVT) behavior of gases, for exam-ple, could be modeled by the ideal gas law viz. PV = nRT . A morerefined model, called the Peng-Robinson equation of state is used widelyin chemical engineering literature. It is given by the following equations:

P = RT(V − b) −

a(T)V(V + b)+ b(V − b) (1.14)

where

a(T) = 0.45724R2T 2

cPc

α(Tr ,ω)

b = 0.0778RTcPc

α1/2 = 1+m(1− √Tr )m = 0.37464+ 1.54226ω− 0.26992ω2

Here Tc, Pc are the critical temperature and pressure of the componentand ω is the accentric factor. These are properties of a component.


1

a0

i NF

V V V

a1 ai-1 ai aN-1 aN

Figure 1.6: Reactors in series

We define Tr = T/Tc as the reduced temperature, and Z = PV/RT asthe compressibility factor. Equation (1.14) can be rearranged as a cubicequation in Z as follows:

Z3 − (1− B)Z2 + (A− 3B2 − 2B)Z − (AB − B2 − B3) = 0 (1.15)

where A = aP/R2T 2 and B = bP/RT . For the class of problems wherethe pressure and temperature (P, T) are given and the material is iden-tified ( i.e., Tc, Pc,ω are known), the coefficients (A, B) in equation (1.15)can be calculated and hence the cubic equation can be solved to find theroots, Z . This allows the determination of the volume (or density) fromZ = PV/RT .

1.3.5 Example of reactors in series

An example from chemical reaction engineering process that gives riseto a system of nonlinear equations is that of a continuously stirred tankreactor in series. A sketch is shown in figure 1.6 Consider an isother-mal, irreversible second order reaction. The composition in each reactoris assumed to be spatially homogeneous due to thorough mixing. Thereaction rate expression is given by,

r = kVa2i

whereai is the exit concentration of component A from the i−th reactor,k is the reaction rate constant and V is the volume of the reactor. Amaterial balance under steady state conditions on the i − th reactorresults in,

kVa2i = F(ai−1 − ai) (1.16)

Letting β = kV/F , we have the following n-simultaneous nonlinear equa-tions.

1.4. LUMPED PARAMETER, DYNAMIC MODELS 15

fi := βa2i + ai − ai−1 = 0 i = 1 · · ·N (1.17)

While we have constructed N equations, there are (N + 2) variables intotal. They are [a0 · · ·aN β]. Hence we have two degrees of freedom. Inanalysing an existing reactor train, for example, one might regard (β,a0)to be known and solve for the remaining N variables including the exitconcentration aN (and hence the conversion). In a design situation onemight wish to achieve a specific conversion and hence regards (a0, aN)as knowns and solve for remaining N variables including β (and hencethe volume V ).

1.4 Lumped parameter, dynamic models

Lumped parameter, dynamic models arise typically when the spatial vari-ation of the state variables can be ignored for some reason, but time vari-ation cannot be ignored. Let us consider an example from heat transfer.

1.4.1 Example of cooling a molten metal

A sample of molten metal at an inital temperature of Ti is placed in acrucible (at an initial temperature of T∞) and allowed to cool by convec-tion. A sketch is shown in figure 1.7. Let T1(t) be the temperature ofthe molten metal at any time t and T2(t) be the temperature of the cru-cible. The argument used to justify neglecting spatial variation is thatthe thermal conductivity of the two materials are sufficiently large tokeep the temperature of each material uniform within its boundaries.The conservation law statement is:

rate of accumulation = rate in - rate out+ rate of generationApplying this first to the molten metal,

ddt(m1Cp1T1) = −h1A1(T1 − T2)︸︷︷︸

heat loss from 1 to 2

(1.18)

Energy balance on the crucible results in,

ddt(m2Cp2T2) = h1A1(T1 − T2)︸︷︷︸

heat gain by 2 from 1

− h2A2(T2 − T∞)︸︷︷︸heat loss from 2 to ∞

(1.19)

These two equations can be presented using matrix notation as follows:

dθdt

= Aθ + b

1.4. LUMPED PARAMETER, DYNAMIC MODELS 16

Insulation

Molten metal T1

T∞

T2

Figure 1.7: Heat transfer from a molten metal

where

θ =[T1

T2

]A =

− h1A1

m1Cp1

h1A1m1Cp1

+ h1A1m2Cp2

−h1A1+h2A2m2Cp2

b =

0h2A2T∞m2Cp2

The initial condition is

θ(t = 0) =[TiT∞

]

This problem depends on several parameters which are assumed to beknown. A1 is the heat trasfer area at the metal-crucible interface, A2

is the area at the crucible-air interface. (h1, h2) are the correspondingheat transfer coefficients, (m1,m2) are the corresponding mass of thematerials, (Cp1 , Cp2) are the specific heats of the two materials. Since allof these are assumed to be known constants, the problem is linear.

1.4.2 Ozone decomposition

A number of mechanisms (some involving 40 steps and 40 equations)have been proposed to model the decomposition of ozone in the atmo-sphere. Let us consider a simple two-step model.

O3 +O2 z O + 2O2

O3 +O → 2O2

1.5. DISTRIBUTED PARAMETER, STEADY STATE MODELS 17

In the early stages of research, we were mainly concerned with identi-fying the mechanisms of ozone depletion in the atmosphere. For lackof better data, the compositions were assumed to be spatially homoge-neous in the atmosphere, although we know now that there can be spatialvariations. For the present purpose we will assume the compositions tobe spatially uniform. Let y1 be the composition of O3 and y2 be that ofO. The model equations are,

dy1

dt= f1(y1, y2) = −y1 −y1y2 + εκy2 (1.20)

εdy2

dt= f2(y1, y2) = y1 −y1y2 − εκy2 (1.21)

The initial compositions are y(t = 0) = [1.0,0.0]. The parameters areε = 1/98 and κ = 3.0. This is a system of two non-linear ordinary dif-ferential equations. It is an interesting problem in the limit of ε → 0. Inthe reaction analysis literature, the consequence of this limit is knownas the quasi-steady-state-approximation. The physical interpretation isthat the second reaction is much faster than the first one so that it canbe assumed to have reached the equilibirum state at every instant oftime. The second equation becomes an albegraic one. In the appliedmathematics literatture it is called the singular perturbation problem.From the computational point of view this limit gives rise to a phenom-ena called stiff systems. We will explore these features further in laterchapters.

1.5 Distributed parameter, steady state models

1.5.1 Heat transfer through a tappered fin

Let us examine an example from transport processes. Consider the useof a fin to enhance the rate of heat transfer. Basically, a fin provides alarge heat transfer surface in a compact design. In the design and perfor-mance analysis of fins one might be interested in a variety of questionssuch as what is the efficiency of the fin? (as a corollary what is a usefuldefinition of fin efficiency?), How many fins are required to dissipate acertain heat load?, What is the optimal shape of the fin that maximizesthe heat dissipation for minimum weight of fin material? How long doesit take for the fin to reach a steady state? etc. You will learn to developanswers to these questions in a heat transfer course. Our interest at thisstage is to develop a feel for the model building process. A sketch of afin is shown in figure 1.8. Let us first examine the steady state behavior


T∞

x

T0

z

hP δx

T(x)δx

y

LW

t0

qA|x qA|x+δx

hP δx

(b) Cylindrical fin(a) Planar fin

Figure 1.8: Heat transfer through a fin

of a planar fin shown in figure 1.8a. The base of the fin is maintainedat a uniform temperature of T0 and the ambient temperature is T∞. Thestate variable that we are interested in predicting is the temperature ofthe fin, T . In general in might be a function of all three spatial positionsi.e., T(x,y, z). (Note time, t is eliminated by assuming steady state). Ifwe know something about the length scales of the fin and the materialproperty of the fin, we make further assumptions that will reduce thecomplexity of the problem. Let us also assume that the fin is made of a What types of

materials mightviolate thisassumption?

homogeneous material i.e., its thermal conductivity, k is independent ofposition. If the length, L, of the fin is much larger than the thickness, t0,then we might argue that the temperature variation in the y directionwill be smaller than that in the x direction. Thus we can assume T tobe uniform in the y direction. Next, we examine what happens in thez direction? This argument is somewhat subtle as it is based on sym-metries in the system. The basic premise here is that symmetric causesproduce symmetric effects. An excellent and easily accessible exposi-tion on this topic can be found in Golubiksky and Stewart (1993). Firstwe assume that the ends of the fin in the z direction are at infinity (orW >> L) so that the end effects can be neglected. Since the temperaturegradient within the fin is caused by the driving force T0 and T∞ whichare independent of z direction, we can expect the fin to respond in a


similar way - viz. T to be independent of z. Note that the end effect,however small it may be, is always present in a planar fin. By making Wlarge compared to L we reduce the error caused by the two-dimensionaleffect near the ends. On the other hand the azimuthal symmetry in thecircular fin (figure 1.8b) make the problem truly one dimensional withtemperature varying only in the radial direction. Now that we have a bet-ter feel for what kinds of arguments or assumptions make this problemone-dimensional, let us proceed with constructing the model equation.

Since the temperature variation is present only in the x direction, wetake an elemental control volume of thickness δx and identify the inputand output sources of energy into this control volume. See figure 1.8.Energy enters by conduction mechanism at a rate of (qA)|x through theleft boundary at x and leaves at a rate of (qA)|x+δx through the rightboundary at x + δx. Heat is also lost by convection through the upperand lower boundaries, which is represented by hPδx(T − T∞). Here qis the heat flux (J/s ·m2) by conduction. This is given by another phe-nomenological model called the Fourier law: q = −kdTdx . k in the Fourierlaw defines a material property called thermal conductivity (J/m·s ·oC).A is the cross-sectional area which is allowed to be a function of x (ta-pered fin), h is the heat transfer coefficient (J/s ·m2 ·o C). P = 2W isthe perimeter (m). The conservation law statement is:

rate of accumulation = rate in - rate out+ rate of generationIn symbolic terms, it is given by,

0 = (qA)|x − (qA)|x+δx − hPδx(T − T∞)Dividing by δx and taking the limit of δx → 0, we obtain,

0 = −d(qA)dx

− hP(T − T∞)Using the Fourier law to replace q in terms of T ,

ddx

[kAdTdx

]− hP(T − T∞) = 0 (1.22)

Equation (1.22) is a second order ordinary differential equation. Thephysical description dictates that two conditions be specified at the twoends of the fin, viz.

T(x = 0) = T0

T(x = L) = T∞ (1.23)dTdx

∣∣∣∣x=L

= 0

1.6. DISTRIBUTED PARAMETER, DYNAMIC MODELS 20

This problem can be solved to obtain T(x) provided the geometrical pa-rameters A(x), P, the material property, k and the heat transfer envi-ronment h,T∞, To are known. The problem is nonlinear if the thermalconductivity is a fucntion of temperature, k(T). In order to determinethe effectiveness of the fin, one is interested in the total rate of heattransfer, Q through the fin. This is can be computed in one of two waysas given by,

Q =∫ LohP[T(x)− T∞] dx = −k dTdx

∣∣∣∣x=0

·A(x = 0)

1.6 Distributed parameter, dynamic models

1.6.1 Heat transfer through a tappered fin

As an example of a distributed, dynamic model let us re-examine the finproblem, but during the early transient phase. Let the fin be initially atthe uniform ambient temperature of T∞. At time t = 0 suppose the baseof the fin at x = 0 is brought to a temperature of To. One migth askquestions like, how long will it take for the fin to reach a steady state?what will be the temperature at the tip of the fin at a given time? etc.Now we have the temperature as a function of both position and time,i.e., T(x, t). We can represent the rate of accumulation within a control

volume symbolically as,d(ρAδxCpT)

dt where the term in paranthesis is theenergy (J) at any time within the control volume, Cp is the specific heatof the fin material (J/kg ·o C) and ρ is the density of the material. Thusthe transient energy balance becomes,

∂(ρAδxCpT)∂t

= (qA)|x − (qA)|x+δx − hPδx(T − T∞)

Dividing by δx and taking the limit of δx → 0, we obtain,

∂(ρACpT)∂t

= −∂(qA)∂x

− hP(T − T∞)

Finally using the Fourier’s law to replace q in terms of T ,

∂(ρACpT)∂t

= ∂∂x

[kA∂T∂x

]− hP(T − T∞) (1.24)

Note that we have switched to partial derivatives as T is a function ofboth (x, t) and equation (1.24) is a partial differential equation. In ad-dition to the boundary conditions specified in equation (1.23), we need

1.6. DISTRIBUTED PARAMETER, DYNAMIC MODELS 21

an initial condition at t = 0 to complete the problem specification. Theconditions are

IC : T(x, t = 0) = T∞,BC1 : T(x = 0, t) = T0, (1.25)

BC2 : T(x = L, t) = T∞

Detection is, or ought to be, an exact science, andshould be treated in the same cold and unemotionalmanner. You have attempted to tinge it with ro-manticism, which produces much the same effectas if you worked a love-story or an elopement intothe fifth proposition of Euclid.

— SIR ARTHUR CONAN DOYLE

Chapter 2

Single nonlinear algebraicequation

2.1 Introduction

In this chapter we consider the problem of finding the root of a singlenonlinear algebraic equation of the form,

f(x) = 0 (2.1)

where f(x) is a continuous function of x and the equation is satisfiedonly at selected values of x = r , called the roots. The equation can be asimple polynomial as we saw with the Peng-Robinson equation of state insection 1.3.4 or a more complicated function as in the multicomponentflash example discussed in section 1.3.3. If the equation depends onother parameters, as is often the case, we will represent them as

f(x;p) = 0 (2.2)

where p represents a set of known parameter values. Clearly, we cangraph the function f vs. x for a range of values of x. The objectiveof such an exercise is to graphically locate the values of x = r wherethe function crosses the x axis. While such a graphical approach hasan intuitive appeal, it is difficult to generalize such methods to higher

22

2.1. INTRODUCTION 23

-3 -2 -1 0 1 2 3

-15

-10

-5

0

5

10

-3 -2 -1 0 1 2 3

-5

-2.5

0

2.5

5

7.5

10

-3 -2 -1 0 1 2 3-4

-2

0

2

4

6

-7.5 -5 -2.5 0 2.5 5 7.5-1

-0.5

0

0.5

1(a) f(x) := sin(x), f(r)=0 at r=nπ, n=0,1,.. (b) f(x):= x3 + 4 x2 + x -6, f(r)=0 at r=-3,-2,1

(c) f(x):= x3 + 3 x2 - 4, f(r)=0 at r=-2,-2,1 (d) f(x):= x3 - x2 + x - 1, f(r)=0 at r=1

Figure 2.1: Graphs of some simple functions

dimensional systems. Hence we seek to construct computational algo-rithms that can be generalized and refined successively to handle a largeclass of nonlinear problems.

Before we embark on such a task, some of the potential problems areillustrated with specific examples. In the general nonlinear case, thereis no way to know a priori, how many values of r can be found that willsatisfy the equation, particularly if the entire range of x in (−∞,∞) isconsidered. For example, the simple equation

f(x) := sin(x) = 0

has infinitely many solutions given by r = nπ where n is any integer.The graph of the function shown in figure 2.1a illustrates this clearly.Often, the physical description of the problem that gave rise to the math-ematical function, will also provide information on the range of valuesof x that are of interest. For example, in the multicomponent flash equa-tion, the problem was so formulated that the dependent variable had thephysical interpretation of fraction of feed in vapor; hence this fractionmust be between (0,1). Although the mathematical equation may havemany other roots, outside of this range, they would lack any physicalmeaning and hence would not be of interest.

Algebraic theory tells us that the total number of roots of a polyno-mial is equal to the degree of the polynomial, but not all of them may bereal roots. Furthermore, if the coefficients of the polynomial are all real,

2.2. BISECTION METHOD 24

then any complex root must occur in pairs. Consider the three cubicequations given below:

f(x) := x3 + 4x2 + x − 6 = 0 r = −3,−2,1

f(x) := x3 + 3x2 − 4 = 0, r = −2,−2,1

f(x) := x3 − x2 + x − 1 = 0 r = 1

These are graphed in figures 2.1b,c,d respectively. In the first case thereare three distinct roots. The function has a non-zero slope at each valueof the root and such roots are called simple roots. In the second casewe have a degeneracy or a non-simple root at r = −2. This problemmanifests itself in a graphical representation with a zero slope of thefunction at the multiple root, r = −2. If the coefficients of the polyno-mial were slightly different, the curve could have moved slightly upwardgiving rise to two distinct roots or downwards yielding no roots in thisregion. Algebraically, we can see that the root is a multiple one with amultiplicity of 2 by factoring the function into (x + 2)(x + 2)(x − 1). Inthe third case there is only a single real root.

We begin by constructing some simple algorithms that have an in-tuitive appeal. They are easy to represent graphically and symbolicallyso that one can appreciate the connection between the two represen-tations. Subsequently we can refine the computational algorithms tomeet the challenges posed by more difficult problems, while keeping thegraphical representation as a visual aid.

There are essentially three key steps in any root finding algorithm.They are:

step 1: Guess one or more initial values for x.

step 2: Iterate using a scheme to improve the initial guess.

step 3: Check convergence - i.e., has the improvement scheme ofstep 2 produced a result of desired accuracy?

The crux of the algorithm is often in the second step and the objectivein devising various clever schemes is to get from the initial guess to thefinal result as quickly as possible.

2.2 Bisection method

The bisection algorithm is quite intuitive. A graphical illustration of thisalgorithm is shown in figure 2.2a. In step 1, we make two guesses x1 and


x1

f2

f1

x3 x2

f3

x1

f2

f1

x2

f3

x3=(x1+x2)/2

r

rx3

Discard x2

Discard x1

(a) Bisection

(c) Secant method

(keep the two most recent values)

(keep the two values that bracket the root, r)

x1

f2

f1

x2

f3

rx3

Discard x1

(b) Regula-Falsi

(keep the two values that bracket the root, r)

x1

f1

f2

rx2

(d) Newton method

slope = f’ 1

( )( )ff

xxfxx −

12

12113 −

−=

( )( )ff

xxfxx −

12

12113 −

−=f

fxx

1

112 −= ’

δ Figure 2.2: Graphical representation of some simple root finding algo-rithms

x2 and calculate the functions values f1 = f(x1) and f2 = f(x2). If thefunction values have opposite signs, it implies that it passes throughzero somewhere between x1 and x2 and hence we can proceed to thesecond step of producing a better estimate of the root, x3. If there isno sign change, it might imply that there is no root between (x1, x2). So What else might it

imply?we have to make a set of alternate guesses. The scheme for producing abetter estimate is also an extremely simple one of using the average ofthe two initial guesses, viz.

x3 = x1 + x2

2(2.3)

In figure 2.2a, (x1, x3) bracket the root r ; hence we discard x2, or betterstill, store the value of x3 in the variable x2 so that we are poised torepeat step 2. If the situation were such as the one shown in figure 2.2b,


then we would discard x1 or better still, store the value of x3 in thevariable x1 and repeat step 2. In either case, (x1, x2) will have betterguesses than the original values.

The final step is to check if we are close enough to the desired rootr so that we can terminate the repeated application of step 2. One testmight be to check if the absolute difference between two successive val-ues of x is smaller than a specified tolerance, i.e.,

|xi+1 − xi| ≤ ε

Another test might be to check if the absolute value of the function fiat the end of every iteration is below a certain tolerance, i.e.,

|f(xi)| ≤ ε

In addition, we might wish to place a limit on the number of times step2 is repeated. A MATLAB function, constructed in the form of a m-file,is shown in figure 2.3.

Example - Multicomponent flash equation

Let us consider the multicomponent, isothermal flash equation devel-oped in section §1.3.3. The equation is

N∑i

(1−Ki)zi(Ki − 1)ψ+ 1

= 0 (2.4)

where (Ki, zi,N) are treated as known parameters andψ is the unknownvariable. Implementation of this function in MATLAB is shown below asan m-file. See Appendix B for

an introduction toMATLAB.


function r=bisect(Fun,x,tol,trace)%BISECT find the root of "Fun" using bisection scheme% Fun - the name of the external function% x - vector of length 2, (initial guesses)% tol - error criterion% trace - print intermediate results%% Usage bisect(’flash’,[0,1])% flash is the name of the external function.% [0,1] is the initial guess

%Check inputsif nargin < 4, trace=0; endif nargin < 3, tol=eps; trace=0; endif (length(x) ˜= 2)error(’Please provide two initial guesses’)end

f = feval(Fun,x); %Fun is assumed to accept a vector

if (prod(sign(f))) > 0, %Check if roots are bracketederror(’No sign change - no roots’)end;

for i = 1:100 %Set max limit on iterationsx3 = (x(1) + x(2))/2; %Update the guessf3 = feval(Fun,x3); %Cal. f(x3)

%Check if x2 or x1 should be discardedif sign(f(1)*f3) < 0, x(2)=x3; else x(1)=x3; end;

if abs(f3) < tol, r=x3; return; end %Check convergenceif trace, fprintf(1,’%3i %12.5f %12.5f\n’, i,x3,f3); endenderror(’Exceeded maximum number of iterations’)

Figure 2.3: MATLAB implementation of the bisection algorithm


function f=flash(psi)% K is a vector of any length of equil ratios.% z is the feed composition (same length as K)% K, z are defined as global in main% psi is the vapor fraction.

global K zif ( length(K) ˜= length(z) )error(’Number of K values & compositions do not match’)endn=length(psi);for i = 1:nf(i)=sum( ((K-1).*z) ./ (1+(K-1)*psi(i)) );end

Observe that this function, while being concise, is fairly general to han-dle any number of components N and a vector of guesses psi of anylength and return a vector of function values, one corresponding to eachelement in the guessed variable psi. Assuming that you have such afunction defined in a file named flash.m, you are encouraged to workthrough the following exercise using MATLAB.

» global K z» z=[.25 .25 .25 .25] %define a 4-component system» K=[2 1.5 0.5 0.1] %define equilibrium values» bisect(’flash’,[0,.1]) %find the root using bisect ans=0.0434» x=0:0.05:1; %create a vector of equally spaced data» y=flash(x); %evaluate the function» plot(x,y) %plot the function

In this section, we have developed and implemented the bisection al-gorithm as a function in MATLAB and used it to solve an example prob-lem from multicomponent flash. The function bisect can be used tosolve any other root finding problem as long as you define the problemyou want to solve as another MATLAB function along the lines of theexample function flash.m.

While we managed to solve the problem, we did not concern our-selves with questions such as, (i) how many iterations did it take to con- Is it always

possible to findsuch a guess?Consider thepathological caseof r = −2 infigure 2.1c!

verge? and, (ii) can we improve the iteration scheme in step 2 to reducethe number of iterations? By design, the bisection scheme will always

2.3. REGULA-FALSI METHOD 29

converge provided an acceptable set of initial guesses have been chosen.This method, however, converges rather slowly and we attempt to devisealgorithms that improve the rate of convergence.

2.3 Regula-falsi method

Instead of using the average of the two initial guesses as we did withthe bisection scheme, we can attempt to approximate the function f(x)by straight line (a linear approximation) since we know two points onthe function f(x). This is illustrated graphically in figure 2.2b with thedashed line approximating the function. We can then determine the root,x3 of this linear function, f (x). The equation for the dashed straightline in figure 2.2b is

f (x) := x − x1

f − f1= x2 − x1

f2 − f1

Now we can determine the value of x = x3 where the linear functionf = 0 in the above equation. This results in,

x3 = x1 − f1x2 − x1

f2 − f1(2.5)

which is used as the iterative equation in step 2. When we evaluate theoriginal function at x3, f(x3) clearly will not be zero (unless the schemehas converged) as shown in figure 2.2b; but x3 will be closer to r thaneither x1 or x2. We can then retain x3 and one of x1 or x2 in such amanner that the root r remains bracketed. This is achieved by followingthe same logic as in the bisection algorithm to discard the x value thatdoes not bracket the root.

The MATLAB function bisect can be easily adapted to implementRegula-Falsi method by merely replacing equation (2.3) with equation(2.5) for step 2.

2.4 Secant method

The secant method retains the linear approximation procedure of theregula-falsi method, but differs by retaining the two most recent valuesof x viz. x2 and x3 and always discarding the oldest value x1. Thissimple change from the Regula-Falsi scheme produces a dramatic dif-ference in the rate of convergence. A MATLAB implementation of the

2.4. SECANT METHOD 30

function r=secant(Fun,x,tol,trace)%SECANT find the root of a function "Fun" using secant scheme% Fun - the name of the external function% x - vector of length 2, (initial guesses)% tol - error criterion% trace - print intermediate results%% Usage secant(’flash’,[0,1])% Here flash is the name of the external function.% [0,1] is the initial guess

%Check inputsif nargin < 4, trace=0; endif nargin < 3, tol=eps; trace=0; endif (length(x) ˜= 2)error(’Please provide two initial guesses’)end

f = feval(Fun,x); %Fun is assumed to accept a vector

for i = 1:100 %Set max limit on iterationsx3 = x(1) - f(1)*(x(2)-x(1))/(f(2)-f(1)) ; %Update(step 2)f3 = feval(Fun,x3); %Cal. f(x3)

%Keep the last two valuesx(1) = x(2);f(1) = f(2); x(2) = x3; f(2) = f3;

if abs(f3) < tol, r=x3; return; end %Check for convergenceif trace, fprintf(1,’%3i %12.5f %12.5f\n’, i,x3,f3); endenderror(’Exceeded maximum number of iterations’)

Figure 2.4: MATLAB implementation of the secant algorithm

2.5. NEWTON’S METHOD 31

secant method is shown in figure 2.4. The convergence rate of the se-cant method was analyzed by Jeeves (1958).

2.5 Newton’s method

The Newton method is by far the most powerful and widely used algo-rithm for finding the roots of nonlinear equations. A graphical repre-sentation of the algorithm is shown in figure 2.2d. This algorithm alsorelies on constructing a linear approximation of the function; But this isachieved by taking the tangent to the function at a given point. Hencethis scheme requires only one initial guess, x1. The linear function f (x)shown by dashed line in figure 2.2d is,

f (x) := f − f1

x − x1

The root of this linear equation, is at (x = x2, f = 0). Under theseconditions one can solve the above equation for x2 as,

x2 = x1 − f1

f ′1(2.6)

which forms the iterative process (step 2). Note that this algorithm re-quires that the derivative of the function be evaluated at every iteration,which can be a computationally expensive operation.

While we have relied on the geometrical interpretation so far in con-structing the algorithms, we can also derive Newton’s scheme from aTaylor series expansion of a function. This is an instructive exercise,for it will enable us to generalize the Newton’s scheme to higher dimen-sional ( i.e., more than two equations) systems as well as provide someinformation on the rate of convergence of the iterative scheme.

The Taylor series representation of a function around a referencepoint, xi is,

f(xi + δ) =∞∑k=0

f (k)(xi)k!

δk (2.7)

where f (k)(xi) is the k−th derivative of the function atxi andδ is a smalldisplacement from xi. While the infinite series expansion is an exactrepresentation of the function, it requires all the higher order derivativeof the function at the reference point. We can construct various levels of

2.6. MULLER’S METHOD 32

approximate representations of the function, f by truncating the seriesat finite terms. For example a three term expansion (k = 0,1,2) is

f (xi + δ) = f(xi)+ f ′(xi)δ+ f ′′(xi)δ2

2!+O(δ3)

where the symbol O(δ3) stands as a reminder of the higher order terms(three and above in this case) that have been neglected. The error in-troduced by such omission of higher order terms is called truncationerror. In fact to derive the Newton scheme, we neglect the quadraticterm O(δ2) also. In figure 2.2d, taking the reference point to be xi = x1,the displacement to be δ = xi+1−xi, and recognising that f (xi+δ) = 0we can rewrite the truncted two-term series as,

0 = f(xi)+ f ′(xi)(xi+1 − xi)+O(δ2)

which can be rearranged as

xi+1 = xi − f(xi)f ′(xi)

This is the same as equation (2.6).

2.6 Muller’s method

This scheme can be thought of as an attempt to generalize secant methodand is important at least from the point of illustrating such generaliza-tions. Instead of making two guesses and constructing an approximatelinear function as we did with the secant method, we can choose threeinital guesses and construct a quadratic approximation to the originalfunction and find the roots of the quadratic. A graphical representationof this is shown in figure 2.5. The three initial guesses are (x0, x1, x2)and the corresponding function values are represented by (f1, f2, f3)respectively. We construct a second degree polynomial as,

p2(v) = av2 + bv + c

wherev = (x−x0). Note that the polynomial is represented as a functionof a new independent variable v , which is merely a translation of theoriginal independent variable x by x0. An alternate view is to regardv as the distances measured from reference point x0, so that v = 0at this new origin. (a, b, c) are the coefficients of the quadratic thatmust be determined in such a way that p2(v) passes through the three

2.6. MULLER’S METHOD 33

x2

f1

f2

x0 x1

f0

r

(a) Muller’s scheme

h1 = x1 - x0h2 = x0 - x2

f(x)

p2(v)

Figure 2.5: Graphical representation of Muller’s scheme

data points (x0, f0), (x1, f1) and (x2, f2). Defining h1 = (x1 − x0) andh2 = (x0−x2) and requiring that the polynomial pass through the threepoints, we get,

p2(0) = a(0)2 + b(0)+ c = f0 ⇒ c = f0

p2(h1) = ah21 + bh1 + c = f1

p2(−h2) = ah22 − bh2 + c = f2

The reason for coordinate shift should be clear by now. This enables cto be found directly. Solving the remaining two equations we obtain aand b as follows:

a = γf1 − f0(1+ γ)+ f2

γh21(1+ γ)

b = f1 − f0 − ah21

h1

where γ = h2/h1. So far we have only constructed an approximate rep-resentation of the original function, f(x) ≈ p2(v). The next step is tofind the roots of this approximate function, p2(v) = 0. These are givenby,

v = r − x0 = −b ±√b2 − 4aca2

2.7. FIXED POINT ITERATION 34

This can be rearranged as,

r = x0 − 2cb ±√b2 − 4ac

(2.8)

Since p2(v) is a quadratic, there are clearly two roots r in equation (2.8).In order the take the root closest tox0 we choose the largest denominatorin equation (2.8). In summary, the sequential procedure for impliment-ing Muller’s scheme is as follows:

• Guess (x0, x1, x2)

• Compute (f0 = f(x0), f1 = f(x1), f2 = f(x2)).

• Compute h1 = (x1 − x0), h2 = (x0 − x2), γ = h2/h1

• Compute c = f(x0).

• Compute a = (γf1 − f0(1+ γ)+ f2)/(γh21(1+ γ))

• Compute b = (f1 − f0 − ah21)/(h1).

• Compute the roots from equation (2.8).

• From (x0, x1, x2) discard the point farthest from r and substituethe new root in its place and repeat.

Note that Muller’s method converges almost quadratically (as does New-ton’s scheme), but requires only one additional function evaluation atevery iteration which is comparable to the computational load of the se-cant method. In particular derivative evaluation is not required, which isa major advantage as compared to Newton’s method. Also, this schemecan converge to complex roots even while starting with real initial guessesas long as provision is made for handling complex arithmetic in the com-puter program. MATLAB handles complex arithmetic quite naturally.

2.7 Fixed point iteration

Another approach to construct an update scheme (for step 2) is to rear-range the given equation f(x) = 0 into a form,

x = g(x)


Then, starting with a guess xi, we can evaluate g(xi) from the right handside of the above equation and the result itself is regarded as a betterestimate of the root, i.e.,

xi+1 = g(xi) i = 0,1 · · · (2.9)

Given, f(x) = 0 it is not difficult rewrite it in the form x = g(x); nor isthis process unique. For example, we can always let g(x) = x + f(x).Such an iterative scheme need not always converge. Let us examine thepossible behavior of the iterates with a specific example. In particular wewill illustrate that different choices of g(x) lead to different behavior.Consider the function

f(x) = x2 − x − 6 = 0 (2.10)

which has roots at r = −2 and r = 3. In the first case let us rewrite it as

x =√x + 6

A geometrical interpretation is that we are finding the intersection of twocurves, y = x (the left hand side) and y = √x + 6 (the right hand side).See figure 2.6 for a graphical illustration. Starting with an initial guess,say x0 = 4, we compute x1 = g(x0) =

√x0 + 6. This is tantamount to

stepping between the y = x and y = g(x) curves as shown in figure2.6a. It is clear that the sequence will converge monotonically to theroot r = 3. The table 2.1 shows the first ten iterates, starting with aninital guess of x0 = 5.

Observe that the slope of the function at the root is g′(r = 3) <1. We will show shortly that the condition for convergence is indeed|g′(r)| < 1. As an alternate formulation consider rewritting equation(2.10) as x = g(x) = 6/(x − 1). Now, g(x) has a singularity at x = 1. Agraphical illustration is shown in figure 2.6b. Using this new g(x), butstarting at the same initial guess x0 = 4 the sequence diverges initially inan oscillatory fashion around the root r = 3, but eventually is attractedto the other root at r = −2, also in an oscillatory fashion. Observethat the slopes at the two roots are: g′(3) = −3/2 and g′(−2) = −2/3.Both are negative and hence the oscillatory behavior. The one with abso-lute magnitude greater than unity diverges and the other with absolutemagnitude less than unity converges. Finally consider the formulationx = g(x) = (x2 − 6). The behavior for this case is shown in figure 2.6c.For reasons indicated above, the sequence will not converge to eitherroot! The following shows the MATLAB implementation for generatingthe iterative sequence for the first case. Enter this into a file called g.m.


r

r

r

y=x

y=x

y=x

r g’(r=3) = 6 > 1

y=g(x)y=g(x)

|g’(r=-2) = -4| > 1

y=g(x)

x0

x0

x0

g’(r=3) = 1/6 < 1

( )

xxg += 6 ( )( )x

xg−

=1

6

( ) ( )xxg −= 2 6

|g’(r=-2) = -2/3| < 1

|g’(r=3) = -3/2 | > 1

x1

(a) (b)

(c)

Figure 2.6: Graphical representation of fixed point scheme

2.8. ERROR ANALYSIS AND CONVERGENCE ACCELERATION 37

Iteration IterateNumberx1 5.0000000x2 3.3166248x3 3.0523147x4 3.0087065x5 3.0014507x6 3.0002418x7 3.0000403x8 3.0000067x9 3.0000011x10 3.0000002

Table 2.1: The first ten iterates of xi+1 =√xi + 6 starting with x0 = 5

function x=g(x)for i=1:10fprintf(1,’%2i %12.5e\n’,i,x); %print the iteratesx=sqrt(x+6); %also try x=6/(x-1) and x=(xˆ2-6) here

end

Invoke this function from within MATLAB with various initial guesses,e.g., try initial guess of 5 by entering,

» g(5)

2.8 Error analysis and convergence acceleration

A simple error analysis can be developed for the fixed point iterativescheme which will provide not only a criterion for convergence, but alsoclues for accelerating convergence with very little additional computa-tional effort. We are clearly moving away from the realm of intuition tothe realm of analysis! Consider the fixed point iteration xi+1 = g(xi).After convergence to the root r we will have r = g(r). Subtracting thetwo equations we get,

(xi+1 − r) = g(xi)− g(r)


r

xi

ξ

g’(ξ)g(xi) - g(r)

xi - r

r < ξ <xi

Figure 2.7: Graphical illustration of mean value theorem

Multiplying and diving the right hand side by (xi − r) ,

(xi+1 − r) = g(xi)− g(r)(xi − r) (xi − r)

Now the difference ei = (xi−r) can be interpreted as the error at iterativestep i and hence the above equation can be written as,

ei+1 = g′(ξ) ei (2.11)

where we have used the mean value theorem to replace slope of thechord by the tangent to the curve at some suitable value of x = ξ, i.e.,

(g(xi)− g(r))(xi − r) = g′(ξ)

A geometrical interpretation of the mean value theorem is shown in fig-ure 2.7. From equation (2.11), it is clear the error will decrease with everyiteration if the slope |g′(ξ)| < 1; otherwise the error will be amplifiedat every iteration. Since the error in the current step is proportional tothat of the previous step, we conclude that the rate of convergence ofthe fixed point iteration is linear. The development has been reasonablyrigorous so far. We now take a more pragmatic step and assume thatg′(ξ) = K is a constant in the neighbourhodd of the root r . Then wehave the sequence, Will K be the same

constant at everyiteration?e2 = Ke1, e3 = Ke2 = K2e1, e4 = Ke3 = K3e1 · · ·


and hence we can write a general error propagation solution as,

en = Kn−1e1 or xn − r = Kn−1e1 (2.12)

It should be clear now that en → 0 as n → ∞ only if |K| < 1. . We refer Well, if we knowthe error in thefirst step, none ofthis analysis wouldbe necessary!r = x1 − e1 woulddo it!

to equation (2.12) as the error propagation solution since it provides asolution of estimating the error at any step n, provided the error at thefirst step e1 and K are known.

We can develop a convergence acceleration scheme using the error so-lution (2.12) to estimate the three unknowns (r ,K, e1) in the second formof equation (2.12). Once we have generated three iterates, (xn,xn+1, xn+2),we can use equation (2.12) to write down,

xn = r +Kn−1e1

xn+1 = r +Kne1 (2.13)

xn+2 = r +Kn+1e1

Now we have three equations in three unkowns which can be solved toestimate r (and K, e1 as well). If K were to remain a true constant withevery iteration, r would be the correct root; since K is not a constantin general, r is only an estimate of the root, hopefully a better estimatethan any of (xn,xn+1, xn+2). Now let us proceed to construct a solutionfor r from the above three equations. We will define a first order forwarddifference operator ∆ as,

∆xn = xn+1 − xnThink of the symbol ∆ as defining a new rule of operation just like aderivative operator d

dx defines a rule. When ∆ operates on xn it is com-puted using the rule shown on the right hand side. Now, if we apply theoperator ∆ to xn+1 we should have,

∆xn+1 = xn+2 − xn+1.

If we apply the ∆ operator twice (which is equivalent to defining higherorder derivatives), we should get,

∆(∆xn) = ∆2xn = (∆xn+1)− (∆xn) = xn+2 − 2xn+1 + xn.

You can verify that using equation (2.13) in the above definitions, we get,

(∆xn)2∆2xn= Kn−1e1 = xn − r


and hence r is given by,

r = xn − (∆xn)2∆2xn= xn − x

2n+1 − 2xnxn+1 + x2

nxn+2 − 2xn+1 + xn

Thus the three iterates (xn,xn+1, xn+2) can be plugged into the righthand side of the above equation to get a better estimate of r .

Example of convergence acceleration

Let us apply this convergence acceleration procedure to the first threeiterates of table 2.1.

x1 = 5.0000000

x2 = 3.3166248

x3 = 3.0523147

∆x1 = (x2 − x1) = −1.6833752

∆x2 = (x3 − x2) = −0.26431010

∆2x1 = (∆x2 −∆x1) = 1.41906510

r = x1 − (∆x1)2∆2x1= 5.000000− (−1.6833752)2

1.41906510= 3.0030852

Compare this with the fourth iterate produce in the original sequencex4 = 3.0087065.

2.8.1 Convergence of Newton scheme

The Newton scheme given by equation (2.6) can be thought of as a fixedpoint iteration scheme whereg(x) has been specified in a special manneras,

xn+1 = xn − f(xn)f ′(xn)= g(xn)

Hence,

g′(x) = 1− (f′)2 − ff ′′(f ′)2

=∣∣∣∣∣ ff

′′

(f ′)2

∣∣∣∣∣ < 1

Since f(r) = 0 by definition, g′(r) = 0 (barring any pathological situa-tion such as f ′(r) = 0) and the inequality should hold near the root r .Thus the Newton method is guaranteed to converge as long as we havea good initial guess. Having progressed this far, we can take the next

2.9. DEFLATION TECHNIQUE 41

step and ask the question about the rate of convergence of the Newtonmethod. A Taylor series expansion of g(x) around r is,

g(xn) = g(r)+ g′(r)(xn − r)+ g′′(r)2

(xn − r)2 + · · ·

Recognizing that en+1 = xn+1 − r = g(xn) − r , en = (xn − r) andg′(r) = 0, the truncated Taylor series expansion can be rearranged as,

en+1 = g′′(r)2

e2n

which shows that the error at any step goes down as the square of theprevious step - i.e., quadratically! This manifests itself in the form ofdoubling accuracy at every iteration.

2.9 Deflation technique

Having found a root, r , of f(x) = 0, if we are interested in finding addi-tional roots of f(x) = 0, we can start with a different initial guess andhope that the new initial guess lies within the region of attraction of aroot different from r . Choosing a different initial guess does not guar-antee that the iteration scheme will not be attracted to the root alreadydiscovered. In order to ensure that we stay away from the known root, r ,we can choose to deflate the original function by constructing a modifiedfunction,

g(x) = f(x)/(x − r)which does not have r as a root. For a single equation the concepts arebest illustrated with a graphical example. Consider the illustration infigure 2.8a where the original function,

f(x) := (x − 2) sin(2x)e−0.8x

can be seen to have several roots including one at r = 2. A sketch of thedeflated function, g(x) = f(x)/(x − 2) is shown in figure 2.8b. Sincer = 2 turns out to be a simple root of f(x) = 0, the deflated functiong(x) = 0 does not contain the already discovered root at x = 2. Hencestarting with a different initial guess and applying an iterative methodlike the secant or Newton scheme on the function g(x) will result inconvergence to another root. This process can obviously be repeated bydeflating successively found roots. For example if we know two roots r1

and r2 then a new function can be constructed as

h(x) = f(x)(x − r1)(x − r2)

.

2.9. DEFLATION TECHNIQUE 42

0 2 4 6 8 10

-0.1

-0.05

0

0.05

0.1

0 2 4 6 8 10

-0.1

-0.05

0

0.05

0.1

de

flate

d r

oo

td

efla

ted

ro

ot

( ) ( ) ( )exnisxxf −= 22 − 80 x.

( ) ( ) ( )xxfxg −= 2

0 2 4 6 8 10-0.2

-0.1

0

0.1

0.2

0 2 4 6 8 10-0.2

-0.1

0

0.1

0.2

de

flate

d r

oo

td

efla

ted

ro

ot

(a)

(b)

(c)

(d)

( ) ( ) ( )exnisxxf −= 22 802 − x.

( ) ( ) ( )xxfxg −= 2

Figure 2.8: Graphical illustration deflation technique

The successive application of this approach is of course susceptible topropagation of round off errors. For example if the roots r1, r2 are known Can you think of a

way to alleviatethis problem?

to only a few significant digits, then the definition of the deflated func-tion h(x) will inherit these errors and hence the roots of h(x) = 0 willnot be as accurate as those of the original equation f(x) = 0.

Another problem of the deflation technique pertains to non-simpleroots. A sketch of the function,

f(x) := (x − 2)2 sin(2x)e−0.8x

is shown in figure 2.8c. It is immediately clear that r = 2 is a doubleroot - i.e., occurs with a multiplicity of two. Hence the deflated function What

computationalproblem might thispose?

g(x) = f(x)/(x − 2) still has r = 2 as a simple root as seen in figure2.8d.

2.10. PARAMETER CONTINUATION 43

2.10 Parameter continuation

2.10.1 Euler-Newton continuation

2.10.2 Homotopy continuation

2.11 Software tools

2.11.1 MATLAB

The MATLAB function for determining roots of a polynomial is calledroots. You can invoke it by entering,

» roots(c)

where c is a vector containing the coefficients of the polynomial in theform,

pn(x) = c1xn + c2xn−1 + · · · + cnx + cn+1.

Let us consider the factored form of the polynomial p3(x) = (x+2)(x+i)(x − i) so that we know the roots are at (−2,±i). To check whetherMATLAB can find the roots of this polynomial we need to construct thecoefficients of the expanded polynomial. This can be done with the con-volve fucntion conv(f1,f2) as follows.

» f1 = [1 2] %Here we define coeff of (x+2) as [1 2]» f2 = [1 i] %Here we define coeff of (x+i) as [1 i]» f3 = [1 -i] %Here we define coeff of (x-i) as [1 -i]» c=conv(conv(f1,f2),f3) % c contains coeff of polynomial» r=roots(c) %returns roots of polynomial defined by c

Note that the function roots finds all of the roots of a polynomial,including complex ones.

The MATLAB function for finding a real root of any real, single non-linear algebraic equation (not necessarily a polynomial) is called fzero.You can invoke it by entering,

» fzero(’fn’,x)

where fn is the name of a m-file that defines the function, x is the initialguess for the root. This fzero is not based on a very robust algorithm.If the function you want to solve has singularities, or multiple roots, thescheme fails to converge, often without producing any appropriate er-ror or warning messages. Hence use with caution. After it produces an

2.11. SOFTWARE TOOLS 44

answer check that it is the correct result by evaluating the function atthe root. As an example try the multicomponent flash problem consid-ered previously. You are encouraged to try the following steps during aMATLAB session.

Note that thedesired result isroot=0.0949203.But starting withdifferent initialguesses, MATLABproduces differentresults! Why? Tryplotting thefunction over therange ψ ∈ [−5,5]in MATLAB and seeif you canunderstandMATLAB behavior!(Clue: sign change)

»global K z; % define K,z to be global»K=[2 1.5 0.5 0.2]; % define K values»z=[.25 .25 .25 .25]; % define z values»root=fzero(’flash’,.5) % solve x0=0.5, ⇒ ans=0.0949»flash(root) % check solution»root=fzero(’flash’,-.85)% solve x0=-0.85, ⇒ ans=-1.0000»flash(root) % check solution»root=fzero(’flash’,1.85)% solve x0=1.85, ⇒ ans=1.6698»flash(root) % check solution»root=fzero(’flash’,1.90)% solve x0=1.90, ⇒ ans=2.0»flash(root) % check solution

2.11.2 Mathematika

Mathematika is another powerful software package for mathematicalanalysis including symbolic processing. It also has an inteactive envi-ronment; i.e., commands, functions are executed as soon as you enterthem. If you invoke Mathematica using a Graphical User Interface (GUI)then, the ploting functions will display the graphs. Otherwise you arelimited to the use of computational features of Mathematica. A completereference to Mathematica can be found in the book by Wolfram (1988).

Example of solving the flash equation with Mathematica

The Mathematica function for finding the roots of an equation is calledSolve. In MATLAB we constructed a m-file to define a function. InMathematica this is done in one line within the workspace. Anythingdelimited by (* *) is treated as comments and ignored by Mathematica.You may wish to work through the following exercise.

Observe that firstletter of allMathematicafunctions is inuppercase.

(* Define the multicomponent flash function f[psi]for a 4-component system.K[[i]], z[[i]] are called lists in Mathematica.

Treat them as arrays.

2.12. EXERCISE PROBLEMS 45

-4 -2 0 2 4-10

-5

0

5

10

0 0.1 0.2 0.3 0.4 0.5

-0.2

-0.15

-0.1

-0.05

0

0.05

ψ ψ

f(ψ)

Figure 2.9: Multicomponent flash function

Sum[f(i), i,N] is a function that sums f(i)over the index range 1,N *)

f[psi_]:=Sum[(K[[i]]-1)*z[[i]]/((K[[i]]-1)*psi+1), i,4]

K=2., 1.5, .5, .2 (* Define K values *)z=.25, .25, .25, .25 (* Define z values *)Solve[f[x] == 0, x ] (* Solve f(x)=0 *)Plot[f[x],x,-5,5, Frame -> True] (* Plot f[x] over 5 to 5 *)Plot[f[x],x,0,.5, Frame -> True] (* Plot f[x] over 0 to 0.5 *)

Note that no initial guess is needed. Mathematica finds all the threeroots. They are:

x -> -1.57727, x -> 0.0949203, x -> 1.66984

Ofcourse, only the second root is of interest. Others do not have anyphysical relevance. Note also that while ploting functions, Mathematicasamples the function at sufficient number of data points ( i.e., x-values)to provide a smooth function. Graphs of the flash equation produced byMathematica are shown in figure 2.9.

2.12 Exercise problems

2.12.1 Multicomponent, isothermal flash model

A sketch of a multicomponent flash process is shown in figure 1.5. Thefollowing equation, which was derived in Chapter 1, models the multi-component flash process. This is a single non-linear algebraic equation


Component i zi KiCarbon dioxide 1 0.0046 1.650Methane 2 0.8345 3.090Ethane 3 0.0381 0.720Propane 4 0.0163 0.390Isobutane 5 0.0050 0.210n-Butane 6 0.0074 0.175Pentanes 7 0.0287 0.093Hexanes 8 0.0220 0.065Heptanes+ 9 0.0434 0.036

Table 2.2: Feed composition & equilibrium ratio of a natural gas mixture

in the unknown ψ, which represents the fraction of feed that goes intothe vapor phase.

N∑i=1

(1−Ki)zi(Ki − 1)ψ+ 1

= 0 (2.14)

This equation has several roots, but not all of them have any physical For a bonus pointyou may choose tofind the number ofroots for the testdata above!

meaning. Only the root for ψ ∈ [0,1] is of interest.The test data in Table 2.2, (taken from Katz etal) relate to the flashing

of a natural gas stream at 1600 psia and 120oF . Determine the fractionψ using the secant algorithm given in figure 2.4 and another root findingfunction that is provided in MATLAB named fzero.

2.12.2 Compressible flow in a pipe

In a fluid mechanics course you might come across the Weymouth equa-tion, which is used for relating the pressure drop vs. flow rate in apipeline carrying compressible gases. It is given by,

Qo = 433.54ToPo

[(P2

1 − P22 )

L σ T

]0.5

d2.667η (2.15)

where

Qo is the gas flow rate = 2000000 SCFDTo is the standard temperature = 520oR


Po is the standard pressure = 14.7 psiaP1 is the upstream pressure, (?), psiaP2 is the downstream pressure, (21.7), psiaL is the length of pipe = 0.1894 milesσ is the specific gravity of gas (air=1) = 0.7T is the actual gas temperature = 530oRd is the diameter of the pipe, (?) inchesη is the efficiency = 0.7 (a fudge factor!)

1. If the diameter of the pipe is 4.026 inches, determine the upstreampressure using the secant (initial guess of [5, 45]) and fzero (initialguess of 25) functions. Compare the flops which stands for thefloating point operations.

2. Suppose the maximum pressure the pipeline can withstand is only24.7 psia. Other conditions remaining the same as in previous part,determine the diameter of the pipe that should be used using thesecant (initial guess of [4, 8]) and fzero (initial guess of 6) functions.Compare the flops.

2.12.3 A model for separation processes

Consider a stagewise separation process shown in Figure 1.2. A modelfor this process was developed in Chapter 1. The variables of interestare (L, V ,x0, x1, x2, · · · , xn,y1, y2, y3, · · · , yn+1). Under the assump-tion of linear equilibrium model, yi = Kxi it is possible to successivelyeliminate all of the variables and obtain the following single, analyticalexpression relating the input, (x0, yn+1), the output, xn, the separationfactor S = L/KV and the number of stages n.

[x0 − xn][x0 −yn+1/K

] =[(1/S)n+1 − (1/S)

][(1/S)n+1 − 1

] (2.16)

The equation is called the Kremser-Brown-Souders equation. We have asingle equation relating six variables, viz. (x0, xn,yn+1, K, S,n). Givenany five of these variables, we can solve for the 6th one. Your task is toformulate this problem as a root finding problem of the type

f(x;p) = 0

where the unknown variable is associated with x and p is the parameterset consisting of the remaining five known values. Write an m-file to


represent the function in the form

f(x;p) = (x0 − xn)(x0 −yn+1/K)

−[(1/S)n+1 − (1/S)

][(1/S)n+1 − 1

] = 0 (2.17)

1. In a typical design problem you might be given the flow rates, (sayL = 10, V = 10), the inlet compositions (say, x0 = 0.8, yn+1 = 0)and a specified recovery (xn = 0.1615). Your task is to determinethe number of stages (n) required to meet the specifications. Takethe equilibrium ratio to be K = 0.8. Here the unknown variable xis associated with n and the others form the parameter set. Solvefor n using secant and bisection methods using initial guesses of[10,30]. Report the number of iterations required for convergenceto the MATLAB built-in convergence tolerance of eps = 10−16. Youcan use the secant.m and bisect.m algorithms outlined in figures2.4,2.3. You must construct a m-file to define the function rep-resented by equation (2.17) in terms of the unknown. Here is asample function for the first case.

function f=KBS1(x)% Kremser-Brown-Souders equation% number of stages is unknown i.e. solve for x=n

K=0.8; L=10; V=10;x0 = 0.8; ynp1= 0; xn=0.1615; S=L/K/V; %Known values

m=length(x);for i=1:mn=x(i);f(i) = (x0-xn)/(x0-ynp1/K) - ( (1/S)ˆ(n+1) - (1/S)) ...

/ ( (1/S)ˆ(n+1) - 1);end

Make sure that you understand what the above function does! Inthe next two parts you will have to modify this function to solvefor a different unknown! Create a file named KBS1.m and enter theabove function. Then to solve the problem from within MATLABenter

» secant(’KBS1’,[10,30],eps,1)


You may wish to plot the function to graphically locate the rootusing

» x=10:1:30;» y=KBS1(x);» plot(x,y)

or, you can do the same in just one line using

» plot([10:1:30],KBS1(10:1:30))

2. In a performance analysis problem, you will be analyzing an exist-ing process with a known number of stages (say, n = 10). Supposex0 = 0.8, yn+1 = 0, L = 10, xn = 0.01488. Find the amount of gasV that can be processed. Use an initial guess of [5,20] and [5,30]with both secant and bisect algorithms. Record and comment onyour observations.

3. In another variation of the performance analysis problem, the amountof gas to be processed (V = 10) may be given. You will have to de-termine the exit composition xn. Take n = 20, x0 = 0.8, yn+1 =0, L = 10. Try initial guesses of [0, .2] and [0,1] on both bisectionand secant algorithms. Record and comment on your observations.

2.12.4 Peng-Robinson Equation of State

The phase behavior of fluids can be predicted with the help of equationsof state. The one developed by Peng & Robinson is particularly welltuned, accurate and hence is widely used. The equation is given below.

P = RT(V − b) −

a(T)V(V + b)+ b(V − b) (2.18)

where

a(T) = 0.45724R2T 2

cPc

α(Tr ,ω), b = 0.0778RTcPc,

√α = 1+m(1− √Tr )

m = 0.37464+ 1.54226ω− 0.26992ω2, Tr = T/Tc, and Z = PV/RT .

Whenever (P, T) are given it is convenient to write equation (2.18) as acubic equation in Z


Z3 − (1− B)Z2 + (A− 3B2 − 2B)Z − (AB − B2 − B3) = 0 (2.19)

where A = aP/(R2T 2), B = bP/(RT).Use equation (2.19) to compute the density of CO2 in gmole/lit at

P = 20.684MPa and T = 299.82oK. The critical properties requiredfor CO2 are Tc = 304.2oK, Pc = 7.3862MPa and ω = 0.225, R =8314Pa m3/kmol oK.

a) Use the function roots(c) in MATLAB to find all the roots of the cubicequation (2.19) in terms of Z . In MATLAB, how does the functionroots differ from the function fzero?

b) Use the secant method to find the real roots of the above equation.

c) After finding the compressibility Z from each of the above meth-ods, convert it into molar density and compare with the experimen-tal value of 20.814gmole/lit

d) Consider the case where you are given (P, V) and you are asked tofind T . Develop and implement the Newton iteration to solve forthis case. Use the above equation to compute the temperature ofCO2 in oK at P = 20.684× 106Pa and V = .04783lit/gmole.

Compare the number of iterations required to obtain a solution toa tolerance of ||f || < 10−15 using an initial guess of T = 250 byNewton method with that required by the secant method with aninitial guess of [200,310].

e) Suppose that you are given (T , V) and you are asked to find P ,which form of equation will you choose? Eqn. (2.18) or Eqn.(2.19)?What method of solution would you recommend?

2.12.5 Transient heat conduction in semi-infinite slab

Many engineering problems can be cast in the form of determining theroots of a nonlinear algebraic equation. One such example arises in de-termining the time required to cool a solid body at a given point to apredetermined temperature level.

Consider a semi-infinite solid, initially at a temperature of Ti = 200oCand one side of it is suddenly exposed to an ambient temperature ofTa = 70oC . The heat transfer coefficient between the solid and sur-roundings is h = 525W/(m2oC). The thermal conductivity of the solidis k = 215W/moC and the thermal diffusivity of the solid is α = 8.4 ×


L1=1000 m D1=0.1m ε1=0.000046m

L2=1100 m D2=0.05m ε2=0.0

QT=0.045 m3/sA B

Figure 2.10: Turbulent flow in a parallel pipe

10−5m2/s. Determine the time required to cool the solid at a distanceof x = 4cm measured from the exposed surface, to T = 120oC . Thetemperature profile as a function of time and distance is given by thefollowing expression.

θ = 1− erf(ξ)−[e(hx/k+τ)

][1− erf(ξ +√τ)]

where the dimensionless temperature, θ = (T−Ti)(Ta−Ti) , and ξ = x

2√αt and

τ = h2αtk2 , t is the time, x is the distance and erf is the error function.

2.12.6 Turbulent flow in a parallel pipeline system

Consider the flow of an incompressible (ρ = 1000kg/m3), Newtonianfluid (µ = 0.001Pa · s) in a parallel pipe system shown in figure 2.10.The lengths, diameters, roughness for the pipes as well as the total flowrate are as shown in figure 2.10. Your task is to determine the individualflow rates in each of the pipe segments 1 and 2. The equation to besatisfied is obtained based on the fact that the pressure drop betweenpoints A and B is the same. The equation is

f1(v1)L1

D1

v21

2= f2(v2)

L2

D2

v22

2(2.20)

where v1, v2 are the velocities in the two pipes and f1, f2 are the frictionfactors given by the Churchill equation.

fi(vi) = 8∗[(

8Rei

)12

+ 1(A+ B)1.5

]1/12

where

A =[

2.457 ln(

1(1/Rei)0.9 + 0.27(εi/Di)

)]16

B =[

37530Rei

]16

and Rei = Diviρµ


Finally the mass balance equation provides another constraint as,

π4(D2

1v1 +D22v2) = QT

This problem can be formulated as two equation in two unknowns (v1, v2),but your task is to pose this as a single equation in one unknown, v1, byrearranging equation 2.20 as,

F(v1) = f1(v1)L1

D1

v21

2− f2(v2)

L2

D2

v22

2= 0

i.e., for a given guess of v1, write a m-file that will calculate F(v1). Thencarryout the following calculations.

• Solve the problem using secant algorithm with initial guess of [4.5,5.5].[Ans:v1 = 4.8703]

• Suppose the total flow rate is increased to 0.09m3/s, what will bethe new velocities in the pipes.

• Consider the case where L1 = 1000, L2 = 900 and D1 = 0.1,D2 =0.09, other values being the same. Is there is flow rate QT forwhich the velocities in both pipes will be the same? If so what isit? [Ans:QT = 0.0017]

• Plot v1 vsQT and v2 vsQT for the case above over a suitable rangeof QT .

• Discuss the pros and cons of implementing the Newton method forthis problem.

Mathematics, rightly viewed, possesses not onlytruth, but supreme beauty – a beauty cold and aus-tere, like that of sculpture.

— BERTRAND RUSSELL

Chapter 3

System of linear algebraicequation

Topics from linear algebra form the core of numerical analysis. Almostevery conceivable problem, be it curve fitting, optimization, simulationof flow sheets or simulation of distributed parameter systems requiringsolution of differential equations, require at some stage the solution of asystem (often a large system!) of algebraic equations. MATLAB (acronymfor MATrix LABoratory) was in fact conceived as a collection of tools toaid in the interactive learning and analysis of linear systems and wasderived from a well known core of linear algebra routines written inFORTRAN called LINPACK.

In this chapter we provide a quick review of concepts from linearalgebra. We make frequent reference to MATLAB implimentation of var-ious concepts throughout this chapter. The reader is encouraged to tryout these interactively during a MATLAB session. For a more completetreatment of topics in linear algebra see Hager (1985) and Barnett (1990).The text by Amundson (1966) is also an excellent source with specificexamples drawn from Chemical Engineering. For a more rigorous, ax-iomatic introduction within the frame work of linear opeartor theorysee Ramakrishna and Amundson (1985).

53

3.1. MATRIX NOTATION 54

3.1 Matrix notation

We have already used the matrix notation to write a system of linearalgebraic equations in a compact form in sections §1.3.1 and §1.3.2.While a matrix, as an object, is represented in bold face, its constituentelements are represented in index notation or as subscripted arrays inprogramming languages. For example the following are equivalent.

A = [aij], i = 1, · · · ,m; j = 1, · · · , nwhere A is an m×n matrix. aij represents an element of the matrix Ain row i and column j position. A vector can be thought of as an objectwith a single row or column. A row vector is represented by,

x = [x1 x2 · · ·xn]while a column vector can be represented by,

y =

y1

y2...ym

These elements can be real or complex.Having defined objects like vectors and matrices, we can extend the

notions of basic arithmetic opeartions between scalar numbers to higherdimensional objects like vectors and matrices. The reasons for doing soare many. It not only allows us to express a large system of equationsin a compact symbolic form, but a study of the properties of such ob-jects allows us to develop and codify very efficient ways of solving andanalysing large linear systems. Packages like MATLAB and Mathematicapresent to us a vast array of such codified algorithms. As an engineeryou should develop a conceptual understanding of the underlying prin-ciples and the skills to use such packages. But the most important taskis to indentify each element of a vector or a matrix, which is tied closelyto the physical description of the problem.

3.1.1 Basic operations

The arithmetic opeartions are defined both in symbolic form and usingindex notation. The later actually provides the algorithm for implement-ing the rules of operation using any programing language. The additionoperation between two matrices is defined as,

addition: A = B + C ⇒ aij = bij + cij


This implies an element-by-element addition of the matrices B and C.Clearly all the matrices involved must have the same dimension. Notethat the addition operation is commutative as with its scalar counterpart. i.e.,

A + B = B + A

Matrix addition is also associative, i.e., independent of the order in whichit is carried out, e.g.,

A + B + C = (A + B) + C = A + (B + C)The scalar multiplication of a matrix involves multiplying each elementof the matrix by the scalar, i.e.,

scalar multiplication: kA = B ⇒ k aij = bijSubtraction operation can be handled by combining addition and sccalarmultiplication rules as follwos:

subtraction: C = A+ (−1)B = A − B ⇒ cij = aij − bijThe product between two matrices B (of dimension n ×m) and C (ofdimension m× r ) is defined as, MATLAB syntax for

the productoperator betweenmatrices isA=B*C

multiplication: A=BC ⇒ aij =m∑k=1

bikckj

and the resultant matrix has the dimension n × r . The operation in-dicated in the index notation is carried out for each value of the freeindices i = 1 · · · n and j = 1 · · · r . The product is defined only if thedimensions of B, C are compatible - i.e., number of columns in B shouldequal the number of rows in C. This implies that while the product BC may be defined, the product C B may not even be defined! Even whenthey are dimensionally compatible, in general

BC 6= BCi.e., matrix multiplication is not commutative.

Multiplying a scalar number by unity leaves it unchanged. Extensionof this notion to matrices resutls in the definition of identity matrix, MATLAB function

for producing anidentity matrix ofsize N isI=eye(N)I =

1 0 · · · 00 1 · · · 0... 0 1 00 0 · · · 1

⇒ δij =

1 i = j0 i 6= j


Multiplying any matrixA with an identity matrix I of appropriate dimen-sion leaves the original matrix unchanged, i.e.,

AI = A

This allows us to generalize the notion of division with scalar numbersto matrices. Division operation can be thought of as the inverse of the MATLAB function

for finding theinverse of a matrixA isB=inv(A)

multiplciation operation. For example, given a number, say 2, we candefine its inverse, x in such a way that the product of the two numbersproduce unity. i.e., 2×x = 1 or x = 2−1. In a similar way, given a matrixA, can we define the inverse matrix B such that

AB = I or B = A−1

The task of developing an algorithm for finding the inverse of a matrixwill be addressed late in this chapter.

For a square matrix, powers of a matrix A can be defined as,

A2 = AA A3 = AAA = A2A = AA2

Note that ApAq = Ap+q for positve integers p and q. Having extended MATLAB operatorfor producing then-th power of amatrix A is,Aˆnwhile the syntaxfor producingelement-by-element poweris,A.ˆn.Make sure that youunderstand thedifference betweenthese twooperations!

the definition of powers, we can extend the definition of exponentialfrom scalars to square matrices as follows. For a scalar α it is,

MATLAB functionexp(A)evaluates theexponentialelement-by-elementwhileexpm(A)evaluates the truematrix exponential.

eα = 1+α+ α2

2+ · · · =

∞∑k=0

αk

k!

For a matrix A the exponential matrix can be defined as,

eA = I +A+ A2

2+ · · · =

∞∑k=0

Ak

k!

One operation that does not have a direct counter part in the scalar worldis the transpose of a matrix. It is defined the result of exchanging therows and columns of a matrix, i.e.,

B = A′ ⇒ bij = ajiIt is easy to verify that

(A+ B)′ = A′ + B′

Something that is not so easy to verify, nevertheless true, is

(AB)′ = B′A′

3.2. MATRICES WITH SPECIAL STRUCTURE 57

3.2 Matrices with special structure

A diagonal matirx D has non-zero elements only along the diagonal.

D =

d11 0 · · · 00 d22 · · · 0... 0

. . . 00 0 · · · dnn

A lower triangular matrix L has non-zero elements on or below the di-agonal,

L =

l11 0 · · · 0l21 l22 · · · 0...

. . . 0ln1 ln2 · · · lnn

A upper triangular matrix U has non-zero elements on or above thediagonal,

U =

u11 u12 · · · u1n0 u22 · · · u2n

0 0. . .

...0 0 · · · unn

A tridiagonal matrix T has non-zero elements on the diagonal and oneoff diagonal row on each side of the diagonal

T =

t11 t12 0 · · · 0t21 t22 t23 0 0

0. . . . . . . . . 0

... 0 tn−1,n−2 tn−1,n−1 tn−1,n0 · · · 0 tn,n−1 tn,n

A sparse matrix is a generic term to indicate those matrices without anyspecific strucutre such as above, but with a small number (typically 10to 15 %) of non-zero elements.

3.3 Determinant

A determinant of a square matrix is defined in such a way that a scalarvalue is associated with the matrix that does not change with certain rowor column operations on the matrix - i.e., it is one of the scalar invariantsof the matrix. In the context of solving a system of linear equations the

3.3. DETERMINANT 58

determinant is also useful is knowing whether the system of equations issolvable uniquely. It is formed by summing all possible products formedby choosing one and only one element from each row and column of thematrix. The precise definition, taken from Amundson (1966), is

det(A) = |A| =∑(−1)h(a1l1a2l2 · · ·anln) (3.1)

Each term in the summation consists of a product ofn elements selectedsuch that only one element appears from each row and column. Thesummation involves a total of n! terms accounted for as follows: for thefirst element l1 in the product there are n choices, followed by (n − 1)choices for the second element l2, (n− 2) choices for the third elementl3 etc. resulting in a total of n! choices for a particular product. Notethat in this way of counting, the set of second subscripts l1, l2, · · · lnwill contain all of the numbers in the range 1 to n, but they will not be intheir natural order 1,2, · · ·n. hence, h is the number of permutationsrequired to arrange l1, l2, · · · ln in their natural order. MATLAB function

for computing thedeterminant of asquare matrix isdet(A)

This definition is neither intutive nor computationaly efficient. But itis instructive in understanding the following properties of determinants.

1. The determinant of a diagonal matrix D, is simply the product ofall the diagonal elements, i.e.,

det(D) =n∏k=1

dkk

This is the only product term that is non-zero in equation (3.1).

2. A little thought should convince you that it is the same for loweror upper triangular matrices as well, viz.

det(L) =n∏k=1

lkk

3. It should also be clear that if all the elements of any row or columnare zero, then the determinant is zero.

4. If every element of any row or column of a matrix is multipliedby a scalar, it is equivalent to multiplying the determinant of theoriginal matrix by the same scalar, i.e.,∣∣∣∣∣∣∣∣∣∣

ka11 ka12 · · · ka1na21 a22 · · · a2n

.... . .

...an1 an2 · · · ann

∣∣∣∣∣∣∣∣∣∣=

∣∣∣∣∣∣∣∣∣∣

a11 a12 · · · ka1na21 a22 · · · ka2n

.... . .

...an1 an2 · · · kann

∣∣∣∣∣∣∣∣∣∣= kdet(A)

3.3. DETERMINANT 59

5. Replacing any row (or column) of a matrix with a linear combinationof that row (or column) and another row (or column) leaves thedeterminant unchanged.

6. A consequence of rules 3 and 5 is that if two rows (or columns) ofa matrix are indentical the determinant is zero.

7. If any two rows (or columns) are interchanged, it results in a signchange of the determinant.

3.3.1 Laplace expansion of the determinant

A definition of determinant that you might have seen in an earlier linearalgebra course is

det(A) = |A| = ∑n

k=1 aikAik for any i∑nk=1 akjAkj for any j (3.2)

where Aik, called the cofactor, is given by,

Aik = (−1)i+kMik

andMik, called the minor, is the determinant of (n−1)×(n−1) submatrixof A obtained by deleting ith row and kth column of A. Note that theexpansion in equation (3.2) can be carried out along any row i or columnj of the original matrix A.

Example

Consider the matrix derived in Chapter 1 for the recycle example, viz.equation (1.8). Let us calculate the determinant of the matrix using theLaplace expansion algorithm around the first row.

det(A) =∣∣∣∣∣∣∣

1 0 0.3060 1 0.702−2 1 0

∣∣∣∣∣∣∣

= 1

∣∣∣∣∣ 1 0.7021 0

∣∣∣∣∣+ (−1)1+2 × 0×∣∣∣∣∣ 0 0.702−2 0

∣∣∣∣∣+ (−1)1+3 × 0.306×∣∣∣∣∣∣

0 1

−2 1

∣∣∣∣∣∣= 1× (−0.702)+ 0+ 0.306× 2 = −0.09

A MATLAB implementation of this will be done as follows:

3.4. DIRECT METHODS 60

»A=[1 0 0.306; 0 1 0.702; -2 1 0] % Define matrix A»det(A) % calculate the determinant

3.4 Solving a system of linear equations

3.4.1 Cramers rule

Consider a 2× 2 system of equations,[a11 a12

a21 a22

][x1

x2

]=[b1

b2

]

Direct elimination of the variable x2 results in

(a11a22 − a12a22) x1 = a22b1 − a12b2

which can be written in an laternate form as,

det(A) x1 = det(A(1))where the matrix A(1) is obtained fromA after replacing the first columnwith the vector b. i.e.,

A(1) =∣∣∣∣∣ b1 a12

b2 a22

∣∣∣∣∣This generalizees to n×n system as follows,

x1 = det(A(1))det(A), · · · xk = det(A(k))det(A)

, · · · xn = det(A(n))det(A).

where A(k) is an n × n matrix obtained from A by replacing the kthcolumn with the vector b. It should be clear from the above that, in orderto have a unique solution, the determinant of A should be non-zero. Ifthe determinant is zero, then such matrices are called singular.

Example

Continuing with the recycle problem (equation (1.8) of Chapter 1), solu-tion using Cramer’s rule can be implemented with MATLAB as follows:

A x = b ⇒ 1 0 0.306

0 1 0.702−2 1 0

x1

x2

x3

=

101.48

225.780


»A=[1 0 0.306; 0 1 0.702; -2 1 0]; % Define matrix A»b=[101.48 225.78 0]’ % Define right hand side vector b»A1=[b, A(:,[2 3])] % Define A(1)»A2=[A(:,1),b, A(:, 3)] % Define A(2)»A3=[A(:,[1 2]), b ] % Define A(3)»x(1) = det(A1)/det(A) % solve for coponent x(1)»x(2) = det(A2)/det(A) % solve for coponent x(2)»x(3) = det(A3)/det(A) % solve for coponent x(3)»norm(A*x’-b) % Check residual

3.4.2 Matrix inverse

We defined the inverse of a matrix A as that matrix B which, when mul-tiplied by A produces the identity matrix - i.e., AB = I; but we did notdevelop a scheme for findingB. We can do so now by combining Cramer’srule and Laplace expansion for a determinant as follows. Using Laplaceexpansion of the determinant of A(k) around column k,

detA(k) = b1A1k + b2A2k + · · · + bnAnk k = 1,2, · · · , n

whereAik are the cofactors ofA. The components of the solution vector,x are,

x1 = (b1A11 + b2A21 + · · · + bnAn1)/det(A)

xj = (b1A1j + b2A2j + · · · + bnAnj)/det(A)xn = (b1A1n + b2A2n + · · · + bnAnn)/det(A)

The right hand side of this system of equations can be written as a vectormatrix product as follows,

x1

x2...xn

=

1det(A)

A11 A21 · · · An1

A12 A22 · · · An2...

. . ....

A1n A2n · · · Ann

b1

b2...bn

orx = B b

Premultiplying the original equation A x = b by A−1 we get

A−1Ax = A−1b or x = A−1b


Comparing the last two equations, it is clear that,

B = A−1 = 1det(A)

A11 A21 · · · An1

A12 A22 · · · An2...

. . ....

A1n A2n · · · Ann

=

adj(A)det(A)

The above equation can be thought of as the definition for the adjointof a matrix. It is obtained by simply replacing each element with itscofactor and then transposing the resulting matrix.

Inverse of a diagonal matrix

Inverse of a diagonal matrix, D,

D =

d11 0 · · · 00 d22 · · · 0... 0

. . . 00 0 · · · dnn

is given by,

D−1 =

1d11

0 · · · 0

0 1d22

· · · 0... 0

. . . 00 0 · · · 1

dnn

It is quite easy to verify using the definition of matrix multiplication thatDD−1 = I.

Inverse of a triangular matrix

Inverse of a triangular matrix is also triangular. Suppose U is a givenupper triangular matrix, then the elements of V = U−1, can be foundsequentially in an efficient manner by simply using the definition UV =I. This equation, in expanded form, isu11 u12 · · · u1n0 u22 · · · u2n

0 0. . .

...0 0 · · · unn

v11 v12 · · · v1nv21 v22 · · · v2n

. . ....

vn1 vn2 · · · vnn

=

1 0 · · · 00 1 · · · 0

. . ....

0 0 · · · 1

We can develop the algorithm ( i.e., find out the rules) by simply carry-ing out the matrix multiplication on the left hand side and equating it


element-by-element to the right hand side. First let us convince ourselfthat V is also upper triangular, i.e.,

vij = 0 i > j (3.3)

Consider the element (n,1) which is obtianed by summing the productof each element of n-th row of U (consisting mostly of zeros!) with thecorresponding element of the 1-st column of V . The only non-zero termin this product is

unnvn1 = 0

Since unn 6= 0 it is clear that vn1 = 0. Carrying out similar arguments ina sequential manner it is easy to verify equation 3.3 and thus establishthat V is also upper triangular.

The non-zero elements of V can also be found in a sequential manneras follows. For each of the diagonal elements (i, i) summing the productof each element of i-th row of U with the corresponding element of thei-th column of V , the only non-zero term is,

vii = 1uii

i = 1, · · · , n (3.4)

Next, for each of the upper elements (i, j) summing the product ofeach element of i-th row ofU with the corresponding element of the j-thcolumn of V , we get,

uiivij +j∑

r=i+1

uirvrj = 0

and hence we get,

vij = − 1uii

j∑r=i+1

uirvrj j = 2, · · · , n; j > i; i = j − 1,1 (3.5)

Note that equation (3.5) should be applied in a specific order, as oth-erwise, it may involve unknown elements vrj on the right hand side.First, all of the diagonal elements of V ( viz. vii) must be calcualted fromequation (3.4) as they are needed on the right hand side of equation (3.5).Next the order indicated in equation (3.5), viz. increasing j from 2 to nand for each j decreasing i from (j − 1) to 1, sould be obeyed to avoidhaving unknowns values appearing on the right hand side of (3.5).


A MATLAB implementation of this algorithm is shown in figure 3.1to illustrate precisely the order of the calculations. Note that the built-in, general purpose MATLAB inverse function ( viz. inv(U) ) does nottake into account the special structure of a triangular matrix and henceis computationally more expensive than the invu funcion of figure 3.1.This is illustrated with the following example.

Example

Consider the upper triangular matrix,

U =

1 2 3 40 2 3 10 0 1 20 0 0 4

Let us find its inverse using both the built-in MATLAB function inv(U)and the function invu(U) of figure 3.1 that is applicable sepcifically foran upper triangular matrix. You can also compare the floating pointopeartion count for each of the algorithm. Work through the followingexample using MATLAB. Make sure that the function invu of figure 3.1in the search path of MATLAB.

»U=[1 2 3 4; % matrix entry continues on next three lines»0 2 3 1;»0 0 1 2;»0 0 0 4]; % Definition of matrix A complete»flops(0); % initialize flop count»inv(U) % MATLAB built-in function»flops % examine flops for MATLAB built-in function (ans: 131)»flops(0); % initialize flop count»invu(U) % inverse of upper triangular matrix»flops % examine flops for gauss solver (ans: 51)

3.4.3 Gaussian elimination

Gaussian elimination is one of the most efficient algorithms for solvinga large system of linear algebraic equations. It is based on a systematicgeneralization of a rather intuitive elimination process that we routinelyapply to a small, say, (2× 2) systems. e.g.,

10x1 + 2x2 = 4


function v=invu(u)% Invert upper triangular matrix% u - (nxn) matrix% v - inv(a)

n=size(u,1); % get number of rows in matrix a

for i=2:nfor j=1:i-1v(i,j)=0;

endend

for i=1:nv(i,i)=1/u(i,i);end

for j=2:nfor i=j-1:-1:1v(i,j) = -1/u(i,i)*sum(u(i,i+1:j)*v(i+1:j,j));

endend

Figure 3.1: MATLAB implementation of inverse of an upper triangularmatrix


x1 + 4x2 = 3

From the first equation we have x1 = (4 − 2x2)/10 which is used toeliminate the variable x1 from the second equation, viz. (4− 2x2)/10+4x2 = 3 which is solved to get x2 = 0.6842. In the second phase, thevalue of x2 is back substituted into the first equation and we get x1 =0.2632. We could have reversed the order and eliminated x1 from thefirst equation after rearranging the second equation as x1 = (3 − 4x2).Thus there are two phases to the algorithm: (a) forward elimination ofone variable at a time until the last equation contains only one unknown;(b) back substitution of variables. Also, note that we have used two rulesduring the elimination process: (i) two equations (or two rows) can beinterchanged as it is merely a matter of book keeping and it does not inany way alter the problem formulation, (ii) we can replace any equationwith a linear combination of itself with another equation. A conceptualdescription of a naive Gaussian elimination algorithm is shown in figure3.2. All of the arithmetic operations needed to eliminate one variable ata time are identified in the illustration. Study that carefully.

We call it a naive scheme as we have assumed that none of the diag-onal elements are zero, although this is not a requirement for existenceof a solution. The reason for avoiding zeros on the diagoanls is to avoiddivision by zeros in step 2 of the illustration 3.2. If there are zeros onthe diagonal, we can interchange two equations in such a way the di-agonals do not contain zeros. This process is called pivoting. Even ifwe organize the equations in such a way that there are no zeros on thediagonal, we may end up with a zero on the diagonal during the elimina-tion process (likely to occur in step 3 of illustration 3.2). If that situationarises, we can continue to exchange that particular row with another oneto avoid division by zero. If the matrix is singular we wil eventually endup with an unavoidable zero on the diagonal. This situation will arise ifthe original set of equations is not linearly independent; in other wordsthe rank of the matrix A is less than n. Due to the finite precision ofcomputers, the floating point operation in step 3 of illustration 3.2 willnot result usually in an exact zero, but rather a very small number. Lossof precision due to round off errors is a common problem with directmethods involving large system of equations since any error introducedat one step corrupts all subsequent calcualtions.

A MATLAB implementation of the algorithm is given in figure 3.3through the function gauss.m. Note that it is

merely forillustrating theconcepts involvedin the eliminationprocess; MATLABbackslash, \operator providesa much moreelegant solution tosolve Ax = b in


ċ

ċ

? ?

ċ

100

10

1

a

aa

aaa

+×

+××

+×××

1,

1,22

1,1121

nn

nn

nn

ċ

ċ

? ?

ċ

−−−

−−−

0

0

1

aaaaaaaaa

aaaaaaaaa

aaa

+×

+××

+×

+××

+×××

1,111,112112

1,1121,21122211222

1,1121

nnnnnnnnnn

nnnn

nn

for j=i+1:n+1; a(i,j)=a(i,j)/a(i,i);end

for k=i+1:n+1; a(j,k)=a(j,k)- a(j,i)*a(i,k);end

STEP 2: Make diagonal elements a(i,i) into 1.0

STEP 1: Arrange A and b into an (n x n+1) matrix

STEP 3 : Make all elements in column i below diagonal into 0

for i=1:n

for j=i+1:n

end

End of forward elimination. Resulting matrix structure is:end

ċ

ċ

? ?

ċ

baaa

baaabaaa

21

222212

112111

nnnnn

n

nċ

ċ

? ? ċ

aaaa

aaaaaaaa

+

++

1,21

1,2222121,112111

nnnnnn

nnnn

ċ

ċ

? ?

ċ

100

10

1

a

aa

aaa

+×

+××

+×××

1,

1,22

1,1121

nn

nn

nn

STEP 4: Back substitution

ċ

ċ

? ? ċ

1

nnnnnn

nna

aaa

aa +

11

1,1

11

1

11

21 nn

aaaa

aaaa

+

+

1,21

1,222212

a(n,n+1) = a(n,n+1)

for j=n-1:-1:1; a(j,n+1) = a(j,n+1) - a(j,j+1:n)*a(j+1:n,n+1);end

Solution is returned in the last column a(1:n,n+1)

Figure 3.2: Naive Gaussian elimination shceme


function x=gauss(a,b)% Naive Gaussian elimination. Cannot have zeros on diagonal% a - (nxn) matrix% b - column vector of length n

m=size(a,1); % get number of rows in matrix an=length(b); % get length of bif (m ˜= n)error(’a and b do not have the same number of rows’)

end

%Step 1: form (n,n+1) augmented matrixa(:,n+1)=b;

for i=1:n%Step 2: make diagonal elements into 1.0a(i,i+1:n+1) = a(i,i+1:n+1)/a(i,i);

%Step 3: make all elements below diagonal into 0for j=i+1:na(j,i+1:n+1) = a(j,i+1:n+1) - a(j,i)*a(i,i+1:n+1);

endend%Step 4: begin back substitutionfor j=n-1:-1:1a(j,n+1) = a(j,n+1) - a(j,j+1:n)*a(j+1:n,n+1);end

%return solutionx=a(:,n+1)

Figure 3.3: MATLAB implementation of naive Gaussian elimination


Example

Let us continue with the recycle problem (equation (1.8) of Chapter 1).First we obtain solution using the built-in MATLAB linear equation solver( viz. x = A\b and record the floating point operations (flops). Thenwe solve with the Gaussian elimination function gauss and compare theflops. Note that in order to use the naive Gaussian elimination function,we need to switch the 2nd and 3rd equations to avoid division by zero.

»A=[1 0 0.306; % matrix entry continues on next two lines»-2 1 0;»0 1 0.702]; % Definition of matrix A complete»b=[101.48 0 225.78]’ ; % Define right hand side column vector b»flops(0) ; % initialize flop count»A\b % solution is : [23.8920 47.7840 253.5556]»flops % examine flops for MATLAB internal solver (ans: 71)»flops(0) ; % initialize flop count»gauss(A,b) % solution is, of course : [23.8920 47.7840 253.5556]»flops % examine flops for gauss solver (ans: 75)»flops(0) ; % initialize flop count»inv(A)*b % obtain solution using matrix inverse»flops % examine flops for MATLAB internal solver (ans: 102)

Example - loss of accuracy & need for pivoting

The need for pivoting can be illustrated with the following simple exam-ple.

εx1 + x2 = 1

x1 + x2 = 2

where ε is a small number. In matrix form it will be,[ε 11 1

][x1

x2

]=[

12

]

In using naive Gaussian elimination without rearranging the equations,we first make the diagonal into unity, which results in

x1 + 1εx2 = 1

ε


Next we eliminate the variable x1 from the 2nd equation which resutlsin, (

1− 1ε

)x2 = 2− 1

εRearranging this and using back substitution we finally get x2 and x1 as,

x2 =2− 1

ε

1− 1ε

x1 = 1ε− x2

εThe problem in computing x1 as ε → 0 should be clear now. As ε crossesthe threshold of finite precision of the computation (hardware or soft-ware), taking the difference of two large numbers of comparable magni-tude, can result in significant loss of precision. Let us solve the problemonce again after rearranging the equations as,

x1 + x2 = 2

εx1 + x2 = 1

and apply Gaussian elimination once again. Since the diagonal elementin the first equation is already unity, we can eliminate x1 from the 2ndequation to obtain,

(1− ε)x2 = 1− 2ε or x2 = 1− 2ε1− ε

Back substitution yields,x1 = 2− x2

Both these computations are well behaved as ε → 0.We can actually demonstrate this using the MATLAB function gauss

shown in figure 3.3 and compare it with the MATLAB built-in functionA\b which does use pivoting to rearrange the equations and minimizethe loss of precision. The results are compared in table 3.1 for ε inthe range of 10−15 to 10−17. Since MATLAB uses double precision, thisrange of ε is the threshold for loss of precision. Observe that the naiveGaussian elimination produces incorrect results for ε < 10−16.

3.4.4 Thomas algorithm

Many problems such as the stagewise separation problem we saw in sec-tion §1.3.1 or the solution of differential equations that we will see in


Naive elimination Built-inε without pivoting MATLAB

gauss(A,b) A\b1× 10−15 [1 1] [1, 1]1× 10−16 [2 1] [1, 1]1× 10−17 [0 1] [1, 1]

Table 3.1: Loss of precision and need for pivoting

later chapters involve solving a system of linear equations Tx = b witha tridiagonal matrix structure.

T =

d1 c1 0 · · · 0a1 d2 c2 · · · 0

. . .0 0 an−2 dn−1 cn−1

0 · · · 0 an−1 dn

x =

x1

x2...

xn−1

xn

b =

b1

b2...

bn−1

bn

Since we know where the zero elements are, we do not have to carry outthe elimination steps on those entries of the matrix T ; but the essentialsteps in the algorithm remain the same as in the Gaussian eliminationscheme and are illustrated in figure 3.4. MATLAB implementation isshown in figure 3.5.

3.4.5 Gaussian elimination - Symbolic representaion

Given a square matrix A of dimension n × n it is possible to write it isas the product of two matrices B and C, i.e., A = BC. This process iscalled factorization and is in fact not at all unique - i.e., there are inifnitelymany possiblilities for B and C. This is clear with a simple counting ofthe unknowns - viz. there are 2×n2 unknown elements in B and C whileonly n2 equations can be obtained by equating each element of A withthe corresponding element from the product BC.

The extra degrees of freedom can be used to specify any specificstructure for B and C. For example we can require B = L be a lowertriangular matrix and C = U be an upper triangular matrix.This processis called LU factorization or decomposition. Since each triangular matrixhas n× (n+ 1)/2 unknowns, we still have a total of n2 + n unknowns.The extra n degrees of freedom is often used in one of three ways:

• Doolitle method assigns the diagonal elements of L to be unity.


??

?

? ? ?

bdbcd

bcdbcd

∗∗−

∗−−

∗

∗∗

111

222

111

0000

000000

nn

nnn

d(j) = d(j) - a(j-1)/d(j-1)*c(j-1) b(j) = b(j) - a(j-1)/d(j-1)*b(j-1)

STEP 1: Eliminate lower diagonal elements

Given

STEP 2: Back substitution

for i=n-1:-1:1 b(i) = b(i) - c(i)*b(i+1)/d(i);end

b(n) = b(n)/d(n)

Solution is stored in b

??

?

? ? ?

bdabcda

bcdabcd

−−−−−

1

1112

2221

111

0000

000

nnn

nnnn

for j=2:n

end

??

?

? ? ?

bdbcd

bcdbcd

∗∗−

∗−−

∗

∗∗

111

222

111

0000

000000

nn

nnn

Figure 3.4: Thomas algorithm


function x=thomas(a,b,c,d)% Thomas algorithm for tridiagonal systems% d - diagonal elements, n% b - right hand side forcing term, n% a - lower diagonal elements, (n-1)% c - upper diagonal elements, (n-1)

na=length(a); % get length of anb=length(b); % get length of bnc=length(c); % get length of cnd=length(d); % get length of dif (nd ˜= nb | na ˜= nc | (nd-1) ˜= na)error(’array dimensions not consistent’)

endn=length(d);%Step 1: forward elimination

for i=2:nfctr=a(i-1)/d(i-1);d(i) = d(i) - fctr*c(i-1);b(i) = b(i) - fctr*b(i-1);

end

%Step 2: back substitutionb(n) = b(n)/d(n);for j=n-1:-1:1b(j) = (b(j) - c(j)*b(j+1))/d(j);end

%return solutionx=b;

Figure 3.5: MATLAB implementation of Thomas algorithm


• Crout method assigns the diagonal elements of U to be unity.

• Cholesky method assigns the diagonal elements of L to be equal tothat ofU - i.e., lii = uii.

While a simple degree of freedom analysis, indicates that it is possible tofactorize a matrix into a product of lower and upper trinagular matrices,it does not tell us how to find out the unknown elements.

Revisiting the Gaussian elimination method from a different perspec-tive, will show the connection between LU factorization and Gaussianelimination. Note that the algorithm outlined in section §3.4.3 is themost computationally efficient scheme for implimenting Guassian elimi-nation. The method to be outlined below is not computationally efficient,but it is a useful conceptual aid in showing the connection between Guas-sian elimination and LU factorization. Steps 2 and 3 of figure 3.2 thatinvolve making the diagonal into unity and all the elements below the di-agonal into zero is equivalent to pre-multuplying A by L1 - i.e., L1A = U1or,

1a11

0 · · · 0−a21a11

1 · · · 0... 0

. . . 0−an1a11

0 0 1

a11 a12 · · · a1na21 a22 · · · a2n

. . ....

an1 an2 · · · ann

=

1 a(1)12 · · · a(1)1n0 a(1)22 · · · a(1)2n

. . ....

0 a(1)n2 · · · a(1)nn

Repeating this process for the 2nd row, we pre-multiply U1 by L2 - i.e.,L2U1 = U2 or, in expanded form,

1 0 0 · · · 00 1

a(1)220 · · · 0

0 −a(1)32

a(1)221 · · · 0

...... 0

. . . 0

0 −a(1)n2

a(1)220 0 1

1 a(1)12 · · · a(1)1n0 a(1)22 · · · a(1)2n

. . ....

0 a(1)n2 · · · a(1)nn

=

1 a(1)12 a(1)13 · · · a(1)1n0 1 a(2)23 · · · a(2)2n

0 0 a(2)33 · · · ...

0 0... · · · ...

0 0 a(2)n3 · · · a(2)nn

Continuing this process, we obtain in succession,

L1A = U1L2U1 = U2L3U2 = U3L4U3 = U4


Ln−1Un−2 = Un−1Note that each Lj is a lower triangular matrix with non-zero elements onthe j-th column and unity on other diagonal elements. Eliminating all ofthe intermediate Uj we obtain,

(Ln−1Ln−2 · · · L1)A = Un−1Since the product of all lower triangular matrices is yet another lowertriangular matrix, we can write the above equation as,

LA = UAlso, the inverse of a lower triangular matrix is also lower triangular- i.e., L = L−1. Hence a given square matrix A can be factored into aproduct of a lower and upper triangular matrix as,

A = L−1U = LUAlthough the development in this section provides us with an algorithmfor constructing both L and U , it is quite inefficient. A more direct andefficient algorithm is developed next in section §3.4.6.

3.4.6 LU decomposition

Consider the product of L and U as shown in the expanded form be-low. All of the elements of L and U are unkonwn. By carrying out thematrix product on the left hand side and equating element-by-elementto the right hand side, we can develop sufficient number of equationsto find out all of the unkown elements on the left hand side. The trick,however is, (as we did with inverting a triangular matrix) to carry out thecalculations in a particualr sequence so that no more than one unknownappears in each equation.

l11 0 0 · · · 0l21 l22 0 · · · 0l31 l32 l33 · · · 0...

......

. . . 0ln1 ln2 ln3 · · · lnn

1 u12 u13 · · · u1n0 1 u23 · · · u2n0 0 1 · · · u3n...

......

. . ....

0 0 0 · · · 1

=

a11 a12 · · · a1na21 a22 · · · a2na31 a32 · · · a3n

...... · · · ...

an1 an2 · · · ann

Let us first consider elements in column 1 of L. Caryying out themultiplication and equating we obtain,

li1 = ai1 i = 1, · · · , n (3.6)


Next focusing on the elements in the first row of U we get, It would beinefficient toproceed to the 2ndcolumn of L. Why?u1j = a1j/l11 j = 2, · · · , n (3.7)

Next we alternate between a column of L and a row of U . The generalexpression for any element i in column j of L is,

lij = aij −j−1∑k=1

likukj j = 2, · · · , n i = j, · · · , n (3.8)

Similarly the general expression for any element i in row j of U is,

uji =[aji −

∑j−1k=1 ljkuki

]ljj

j = 2, · · · , n i = j + 1, · · · , n (3.9)

Equations (3.6-3.9) form the basic algorithm for LU decomposition. Inorder to illustrate the implementation of equations (3.6-3.9) as an algo-rithm, a MATLAB function called LU.m is shown in figure 3.6. Note thatMATLAB provides a built-in function for LU decomposition called lu(A).

Recognizing that A can be factored into the product LU , one can im-plement an efficient scheme for solving a system of linear algebraic equa-tions Ax = b repeatedly, particularly when the matrix A remains un-changed, but different solutions are required for different forcing termson the right hand side, b. The equation

Ax = b

can be written as

LUx = b ⇒ Ux = L−1b = b′

and hence

x = U−1b′

The operations required for forward elimination and back substitutionare stored in the LU factored matrix and as we saw earlier it is rela-tively efficient to invert triangular matrices. Hence two additional vector-matrix products provide a solution for each new value of b.


function [L,U]=LU(a)% Naive LU decomposition% a - (nxn) matrix% L,U - are (nxn) factored matrices% Usage [L,U]=LU(A)

n=size(a,1); % get number of rows in matrix a

%Step 1: first column of LL(:,1)=a(:,1);

%Step 2: first row of UU(1,:)=a(1,:)/L(1,1);

%Step 3: Alternate between column of L and row of Ufor j=2:nfor i = j:nL(i,j) = a(i,j) - sum(L(i,1:j-1)’.*U(1:j-1,j));

endU(j,j) = 1;for i=j+1:nU(j,i)=(a(j,i) - sum(L(j,1:j-1)’.*U(1:j-1,i) ) )/L(j,j);

endend

Figure 3.6: MATLAB implementation of LU decomposition algorithm

3.5. ITERATIVE METHODS 78

Example

Work through the following exercise in MATLAB to get a feel for thebuilt-in MATLAB implementation of LU factorization with that given infigure 3.6. Before you work through the exercise make sure that the fileLU.m that contains the function illustrated in figure 3.6 is in the MATLABpath. Also, be aware of the upper case function LU of figure 3.6 and thelower case lu which is the built-in function.

»A=[1 0 0.306; 0 1 0.702; -2 1 0]; % Define matrix A»b=[101.48 225.78 0]’ % Define right hand vector b»flops(0) % initialize flop count»x=A\b % solve using built-in solver»flops % flops = 74»flops(0) % re-initialize flop count»[l,u]=LU(A) %Use algorithm in figure 3.6»flops % flops = 24»x=inv(u)*(inv(l)*b) % Solve linear system»flops % Cumulative flops = 213»flops(0) % re-initialize flop count»[L,U]=lu(A) %use built-in function»flops % flops = 9»x=inv(U)*(inv(L)*b) % Solve linear system»flops % Cumulative flops = 183

3.5 Iterative algorithms for systems of linear equations

The direct methods discussed in section §3.4 have the advantage of pro-ducing the solution in a finite number of calculations. They suffer, how-ever, from loss of precision due to accumulated round off errors. Thisproblem is particulalry sever in large dimensional systems (more than10,000 equations). Iterative methods, on the other hand, produce theresult in an asymptotic manner by repeated application of a simple al-gorithm. Hence the number of floating point operations required to pro-duce the final result cannot be known a priori. But they have the naturalability to eliminate errors at every step of the iteration. For an author-itative account of iterative methods for large linear systems see Young(1971).

Iterative methods rely on the concepts developed in Chapter 2. Theyare extended naturally from a single equation (one-dimensional system)


to a system of equations (n-dimensional system). The development par-allels that of section §2.7 on fixed point iterations schemes. Given anequation of the form, A x = b we can rearrange it into a form,

x(p+1) = G(x(p)) p = 0,1, · · · (3.10)

Here we can view the vector x as a point in a n-dimensional vector spaceand the above equation as an iterative map that maps a point x(p) intoanother point x(p+1) in the n-dimensional vector space. Starting with aninitial guess x(0) we calculate successive iterates x(1), x(2) · · · until thesequence converges. The only difference from chapter 2 is that the aboveiteration is applied to a higher dimensional system of (n) equations. Notethat G(x) is also vector. Since we are dealing with a linear system, G willbe a linear function of x which is constructed from the given A matrix.G can typically be represented as

x(p+1) = G(x(p)) = Tx(p) + c. (3.11)

In section §2.7 we saw that a given equation f(x) = 0 can be rearrangedinto the form x = g(x) in several different ways. In a similar manner,a given equation Ax = b can be rearranged into the form x(p+1) =G(x(p)) in more than one way. Different choices of G results in differentiterative methods. In section §2.7 we also saw that the condition forconvergence of the seuqence xi+1 = g(xi) is g′(r) < 1. Recognizingthat the derivative of G(x(p)) with respect to x(p) is a matrix, G′ = Ta convergence condition similar to that found for the scalar case mustdepend on the properties of the matrix T . Another way to demonstratethis is as follows. Once the sequence x(1), x(2) · · · converges to, say, requation (3.11) becomes,

r = Tr + c.Subtracting equation (3.11) from the above,

(x(p+1) − r) = T(x(p) − r).

Now, recognizing that (x(p) − r) = ε(p) is a measure of the error at it-eration level p, we have

ε(p+1) = Tε(p).Thus, the error at step (p + 1) depend on the error at step (p). If thematrix T has the property of amplifying the error at any step, then theiterative sequence will diverge. The property of the matrix T that de-termines this feature is called the spectral radius. The spectral radius is


defined as the largest eigenvalue in magnitude of T . For converence ofthe iterative sequence the spectral radius of T should be less than unity,

ρ(T ) < 1 (3.12)

3.5.1 Jacobi iteration

The Jacobi iteration rearranges the given equations in the form,

x(p+1)1 = (b1 − a12x

(p)2 − a13x

(p)3 − · · · − a1nx

(p)n )/a11

x(p+1)j =

bj −

j−1∑k=1

ajkx(p)k −

n∑k=j+1

ajkx(p)k

/ajj (3.13)

x(p+1)n = (bn − an1x

(p)1 − an2x

(p)2 − · · · − an,n−1x

(p)n−1)/ann

where the variable xj has been extracted form the j − th equation andexpressed as a function of the remaining variables. The above set ofequations can be applied repetitively to update each component of theunknown vector x=(x1, x2, · · · , xn) provided an inital guess is knownfor x. The above equation can be written in matrix form as, Note that MATLAB

functionsdiagtriltriuare useful inextracting parts ofa given matrix A

Lx(p) + Dx(p+1) + Ux(p) = b

where the matrices D, L, U are defined in term of components of A asfollows.

D =

a11 0 · · · 00 a22 · · · 0... 0

. . . 00 0 · · · ann

L =

0 0 · · · 0a21 0 · · · 0

.... . . 0

an1 an2 · · · 0

U =

0 a12 · · · a1n0 0 · · · a2n

0 0. . .

...0 0 · · · 0

which can be rearranged as,

x(p+1) = D−1(b − (L + U)x(p)) (3.14)

and henceG(x(p)) = −D−1(L+U)x(p)+D−1b andG′ = T = −D−1(L+U). This method has been shown to be convergent as long as the originalmatrix A is diagonally dominant, i.e.,


|aii| ≥n∑

j=1,j 6=i

∣∣∣aij∣∣∣ ı = 1, · · · , n

An examination of equation (3.13) reveals that none of the diagonalelements can be zero. If any is found to be zero, one can easily exchangethe positions of any two equations to avoid this problem. Equation (3.13)is used in actual computational implementation, while the matrix formof the equation (3.14) is useful for conceptual description and conver-gence analysis. Note that each element in the equation set (3.13) can beupdated independent of the others in any order because the right handside of equation (3.13) is evaluated at the p-th level of iteration. Thismethod requires that x(p) and x(p+1) be stored as two separate vectorsuntil all elements of x(p+1) have been updated using equation (3.13). Aminor variation of the algorithm which uses a new value of the elementin x(p+1) as soon as it is available is called the Gauss-Seidel method. Ithas the dual advantage of faster convergence than the Jacobi iterationas well as reduced storage requirement for only one array x.

3.5.2 Gauss-Seidel iteration

In the Gauss-Seidel iteration we rearrange the given equations in theform,

x(p+1)1 = (b1 − a12x

(p)2 − a13x

(p)3 − · · · − a1nx

(p)n )/a11

x(p+1)j =

bj −

j−1∑k=1

ajkx(p+1)k −

n∑k=j+1

ajkx(p)k

/ajj (3.15)

x(p+1)n = (bn − an1x

(p+1)1 − an2x

(p+1)2 − · · · − an,n−1x

(p+1)n−1 )/ann

Observe that known values of the elements in x(p+1) are used on theright hand side of the above equations (3.15) as soon as they are availablewithin the same iteration. We have used the superscripts p and (p + 1)explicitly in equation (3.15) to indicate where the newest values occur.In a computer program there is no need to assign separate arrays forp and (p + 1) levels of iteration. Using just a single array for x willautomatically propagate the newest values as soon as they are updated.

The above equation can be written symbolically in matrix form as,

Lx(p+1) + Dx(p+1) + Ux(p) = b


where the matrices D, L, U are defined as before. Factoring x(p+1) weget,

x(p+1) = (L + D)−1(b − Ux(p)) (3.16)

and hence G(x(p)) = −(L + D)−1Ux(p) + (L + D)−1b and G′ = T =−(L + D)−1U . Thus the convergence of this scheme depends on thespectral radius of the matrix, T = −(L + D)−1U . This method hasalso been shown to be convergent as long as the original matrix A isdiagonally dominant.

MATLAB implementation of the Gauss-Seidel algorithm is shown infigure 3.7.

3.5.3 Successive over-relaxation (SOR) scheme

The relaxation scheme can be thought of as a convergence accelerationscheme that can be applied to any of the basic iterative methods likeJacobi or Gauss-Seidel schemes. We introduce an extra parameter, ωoften called the relaxation parameter and choose its value in such a waythat we can either speed up convergence by using ω > 1 (called over-relaxation ) or in some difficult problems with poor initial guess we canattempt to enlarge the region of convergence usingω < 1 (called under-relaxation). Let us illustrate the implementation with the Gauss-Seidelscheme. The basic Gauss-Seidel scheme is:

t := x(p+1)j =

bj −

j−1∑k=1

ajkx(p+1)k −

n∑k=j+1

ajkx(p)k

/ajj (3.17)

Instead of accepting the value of x(p+1)j computed from the above for-

mula as the current value, we store it in a temporary variable t and forma better (or accelerated) estimate of x(p+1)

j from,

x(p+1)j = x(p)j +ω [t − x(p)j ]

Observe that if ω = 1, the method remains the same as Gauss-Seidelscheme. For ω > 1, then the difference between two successive iterates(the term in the square bracketts) is amplified and added to the currentvalue x(p)j .

The above opeartions can be written in symbollic matrix form as,

x(p+1) = x(p) +ω[D−1(b − Lx(p+1) − Ux(p)) − x(p)]where the term in braces represent the Gauss-Seidel scheme. After ex-tracting x(p+1) from the above equation, it can be cast in the standard


function x=GS(a,b,x,tol,max)% Gauss-Seidel iteration% a - (nxn) matrix% b - column vector of length n% x - initial guess vector x% tol - convergence tolerance% max - maximum number of iterations% Usage x=GS(A,b,x)

m=size(a,1); % get number of rows in matrix an=length(b); % get length of bif (m ˜= n)error(’a and b do not have the same number of rows’)

endif nargin < 5, max=100; endif nargin < 4, max=100; tol=eps; endif nargin == 2error(’Initial guess is required’)

endcount=0;

while (norm(a*x-b) > tol & count < max),x(1) = ( b(1) - a(1,2:n)*x(2:n) )/a(1,1);for i=2:n-1x(i) = (b(i) - a(i,1:i-1)*x(1:i-1) - ...

a(i,i+1:n)*x(i+1:n) )/a(i,i);endx(n) = ( b(n) - a(n,1:n-1)*x(1:n-1) )/a(n,n);count=count+1;

end

if (count >= max)fprintf(1,’Maximum iteration %3i exceeded\n’,count)fprintf(1,’Residual is %12.5e\n ’,norm(a*x-b) )end

Figure 3.7: MATLAB implementation of Gauss-Seidel algorithm


iterative from of equation (3.11) as,

x(p+1) = (D+ωL)−1[(1−ω)D−ωU]x(p) +ω(D+ωL)−1b (3.18)

Thus the convergence of the relaxation method depends on the spectralradius of the matrix T (ω) := (D +ωL)−1[(1 −ω)D −ωU]. Since thismatrix is a function of ω we have gained a measure of control over theconvergence of the iterative scheme. It has been shown (Young, 1971)that the SOR method is convergent for 0 < ω < 2 and that there is anoptimum value ofω which results in the maximum rate of convergence.The optimum value of ω is very problem dependent and often difficultto determine precisely. For linear problems, typical values in the rangeof ω ≈ 1.7 ∼ 1.8 are used.

3.5.4 Iterative refinement of direct solutions

We have seen that solutions obtained with direct methods are proneto accumulation of round-off errors, while iterative methods have thenatural ability to remove errors. In an attempt to combine the best ofboth worlds, one might construct an algorithm that takes the error-pronesolution from a direct method as an initial guess to an iterative methodand thus improve the accuracy of the solution.

Let us illustrate this concept as applied to improving the accuracy ofa matrix inverse. Suppose B is an approximate (error-prone) inverse of agiven matrixA obtained by one of the direct methods outlined in section§3.4. If Bε is the error in B then

A(B + Bε) = I or ABε = (I − AB)We do not, of course, know Bε and our task is to attempt to estimateit approximately. Premultiplying above equation by B, and recognizingBA ≈ I, we have

Bε = B(I − AB)Observe carefully that we have used the approximation BA ≈ I on theleft hand side where products of order unity are involved and not on theright hand side where difference between numbers of order unity areinvolved. Now we have an estimate of the error Bε which can be addedto the approximate result B to obtain,

B + Bε = B + B(I − AB) = B(2I − AB)Hence the iterative sequence should be,

B(p+1) = B(p)(2I − AB(p)) p = 0,1 · · · (3.19)

3.6. GRAM-SCHMIDT ORTHOGONALIZATION PROCEDURE 85

3.6 Gram-Schmidt orthogonalization procedure

Given a set of n linearly independent vectors, xi | i = 1, · · · , n, theobjective is to produce an orthonormal set of vectors ui | i = 1, · · · , n.

We begin by normalizing thex1 vector using the norm ||x1|| = √x1 · x1

and call it u1.

u1 = x1

||x1||

Subsequently we construct other vectors orthogonal tou1 and normalizeeach one. For example we constructu′2 by subtractingu1 fromx2 in sucha way that u′2 contains no components of u1 - i.e.,

u′2 = x2 − c0 u1

In the above c0 is to be found in such a way that u′2 is orthogonal to u1.

uT1 · u′2 = 0 = uT1 · x2 − c0 or c0 = uT1 · x2

Similarly we have,

u′3 = x3 − c1 u1 − c2 u2

Requiring orthogonality with respect to both u1 and u2

uT1 · u′3 = 0 = uT1 · x3 − c1 or c1 = uT1 · x3

uT2 · u′3 = 0 = uT2 · x3 − c2 or c2 = uT2 · x3

u′3 = x3 − (uT1 · x3) u1 − (uT2 · x3) u2

In general we have,

u′s = xs −s−1∑j=1

(uTj · xs) uj us = u′s||u′s|| s = 2, · · ·n; (3.20)

3.7. THE EIGENVALUE PROBLEM 86

1

3

4

5

6

2v12

v35

v46

v34p1

p5

p6

p2

Figure 3.8: Laminar flow in a pipe network

3.7 The eigenvalue problem

3.7.1 Wei-Prater analysis of a reaction system

3.8 Singular value decomposition

3.9 Genaralized inverse

3.10 Power iteration

3.11 Software tools

3.11.1 Lapack, Eispack library

3.11.2 MATLAB

3.11.3 Mathematica


3.12.1 Laminar flow through a pipeline network

Consider laminar flow through the network shown in figure 3.8. The gov-erning equations are the pressure drop equations for each pipe elementi− j and the mass balance equation at each node.The pressure drop between nodes i and j is given by,


pi − pj = αijvij where αij =32µlijd2ij

(3.21)

The mass balance at node 2 is given, for example by,

d212v12 = d2

23v23 + d224v24 (3.22)

Similar equations apply at nodes 3 and 4. Let the unknown vector be

x = [p2 p3 p4 v12 v23 v24 v34 v35 v46]

There will be six momentum balance equations, one for each pipe ele-ment, and three mass balance (for incompressible fluids volume balance)equations, one at each node. Arrange them as a system of nine equationsin nine unknowns and solve the resulting set of equations. Take the vis-cosity of the fluid, µ = 0.1Pa · s. The dimensions of the pipes are givenbelow.

Table 1

Element no 12 23 24 34 35 46

dij (m) 0.1 0.08 0.08 0.10 0.09 0.09lij (m) 1000 800 800 900 1000 1000

a) Use MATLAB to solve this problem for the specified pressures ofp1 = 300kPa and p5 = p6 = 100kPa. You need to assemble thesystem of equations in the form A x = b. Report flops. Whenreporting flops, report only for that particular operation - i.e., ini-tialize the counter using flops(0) before every operation.

• Compute the determinant of A. Report Flops.

• Compute the LU factor ofA using built-in function lu. Reportflops. What is the structure of L? Explain. The function LUprovided in the lecture notes will fail on this matrix. Why?

• Compute the solution using inv(A)*b. Report flops.

• Compute the rank of A. Report Flops.

• Since A is sparse ( i.e., mostly zeros) we can avoid unnecessaryoperations, by using sparse matrix solvers. MATLAB Ver4.0(not 3.5) provides such a facility. Sparse matrices are stored


using triplets (i, j, s) where (i, j) identifies the non-zero en-try in the matrix and s its corresponding value. The MATLABfunction find(A) examines A and returns the triplets. Use,

» [ii,jj,s]=find(A)

Then construct the sparse matrix and store it in S using

» S=sparse(ii,jj,s)

Then to solve using the sparse solver and keep flops count,use

» flops(0); x = S\b; flops

Compare flops for solution by full and sparse matrix solution.To graphically view the structure of the sparse matrix, use

» spy(S)

Remember that you should have started MATLAB under X-windows for any graphics display of results!

• Compute the determinant of the sparse matrix, S (should bethe same as the full matrix!). Report and compare flops.

b) Find out the new velocity and pressure distributions when p6 ischanged to 150kPa.

c) Suppose the forcing (column) vector in part (a) is b1 and that inpart (b) is b2, report and explain the difference in flops for the fol-lowing two ways of obtaining the two solutions using the sparsematrix. Note that in the first case both solutions are obtained si-multaneously.

» b = [b1, b2];flops(0);x = S\b,flops» flops(0);x1 = S\b1, x2 = S\b2,flops

Repeat the above experiment with the full matrix, A and reportflops.

d) Comment on how you would adopt the above problem formulationif a valve on line 34 is shut so that there is no flow in that line 34.

Let knowledge grow from more to more,But more of reverence in us dwell;That mind and soul, according well,May make one music as before.

— ALFRED TENNYSON

Chapter 4

Systems of nonlinear algebraicequations

In this chapter we extend the concepts developed in Chapter 2 - viz.finding the roots of a system of nonlinear algebraic equations of theform,

f(x) = 0 (4.1)

where f(x) is a vector function of x - i.e., there are n equations whichcan be written in expanded or component form as,

f1(x1, x2, · · · , xn) = 0

f2(x1, x2, · · · , xn) = 0

· · ·fn(x1, x2, · · · , xn) = 0

As with the scalar case, the equation is satisfied only at selected values ofx = r = [r1, r2, · · · , rn], called the roots. The separation process modeldiscussed in section §1.3.1 (variations 2 and 3, in particular) and thereaction sequence model of section §1.3.5 are two of the many exam-ples in chemical engineering that give rise to such non-linear system ofequations. As with the scalar case, the equations often depend on otherparameters, and we will represent them as

f(x; p) = 0 (4.2)

89


where p represents a set of known parameter values. In such cases itmay be required to construct solution families for ranges of values ofp - i.e., x(p). This task is most efficiently achieved using continuationmethods. For some of the recent developments on algorithms for non-linear equations see Ortega and Rheinboldt (1970), Rabinowitz (1970),Scales (1985) and Drazin (1992).

4.1 Newton’s method

For a scalar equation a geometrical interpretation of the Newton schemeis easy to develop as shown in figure 2.2d. This is difficult to visualizefor higher dimensional systems. The algorithm developed in section§2.5 can, however, be generalized easily to higher dimensional systems.The basic concept of linearizing a nonlinear function remains the sameas with a scalar case. We need to make use of the multivariate formof the Taylor series expansion. We will illustrate the concepts with atwo-dimensional system of equations written in component form as,

f1(x1, x2) = 0

f2(x1, x2) = 0

Thus the vectors f(x) = [f1(x1, x2), f2(x1, x2)] and x = [x1, x2] con-tain two elements. Let the roots be represented by r = [r1, r2] - i.e.,f(r) = 0.

Suppose x(0) be some known initial guess for the solution vector xand let the root be at a small displacement δ from x(0) - i.e.,

r = x(0) + δ

If we can device a scheme to estimate δ then we can apply such a schemerepeatedly to get closer to the root r. Variations in the function valuef1(x1, x2) can be caused by variations in either components x1 or x2.Recognizing this, a bi-variate Taylor series expansion around x(0) can bewritten as,

f1(x(0)1 + δ1, x

(0)2 + δ2) = f1(x

(0)1 , x(0)2 )+

∂f1

∂x1

∣∣∣∣[x(0)1 ,x(0)2 ]

δ1︸︷︷︸variation due to x1

+ ∂f1

∂x2

∣∣∣∣[x(0)1 ,x(0)2 ]


+O(δ2)


f2(x(0)1 + δ1, x

(0)2 + δ2) = f2(x

(0)1 , x(0)2 )+

∂f2

∂x1

∣∣∣∣[x(0)1 ,x(0)2 ]


+ ∂f2

∂x2

∣∣∣∣[x(0)1 ,x(0)2 ]


+O(δ2)

Since δ is supposed to be small, we can neglect higher order termsO(δ2)in the above equations and this step is the essence of the linearizationprocess. Since x(0) + δ = r and f(r) = 0, the left hand sides of theabove equations are zero. Thus we get,

0 = f1(x(0)1 , x(0)2 )+ ∂f1

∂x1

∣∣∣∣[x(0)1 ,x(0)2 ]

δ1 + ∂f1

∂x2

∣∣∣∣[x(0)1 ,x(0)2 ]

δ2

0 = f2(x(0)1 , x(0)2 )+ ∂f2

∂x1

∣∣∣∣[x(0)1 ,x(0)2 ]

δ1 + ∂f2

∂x2

∣∣∣∣[x(0)1 ,x(0)2 ]

δ2

These are two linear equations in two unknowns [δ1, δ2]. Note that thetwo functions [f1, f2] and their four partial derivatives are required tobe evaluated at the guess value of [x(0)1 , x(0)2 ]. The above equations canbe arranged into matrix form as,

[00

]=[f1

f2

]+ ∂f1∂x1

∂f1∂x2

∂f2∂x1

∂f2∂x2

[ δ1

δ2

]

or in symbolic form

0 = f (0) + J(0) δwhere J(0) is called the Jacobian matrix and the superscript is a reminderthat quantities are evaluated using the current guess value of x(0). Thus,the displacement vector δ is obtained by solving the linear system,

δ = −J−1f

In general then, given x(0), the algorithm consists of (i) evaluating thefunction and the Jacobian at the current iterate x(k), (ii) solving the linearsystem for the displacement vector δ(k) and (iii) finding the new estimatefor the iterate x(k+1) from the equations

δ(k) = −[J(k)]−1f (k) x(k+1) = x(k) + δ(k) k = 0,1, · · · (4.3)


Convergence check

The final step is to check if we are close enough to the desired root rso that we can terminate the iteration. One test might be to check if theabsolute difference between two successive values of x is smaller thana specified tolerance. This can be done by computing the norm of δ

|δ| ≤ εAnother test might be to check if the absolute value of the function fat the end of every iteration is below a certain tolerance. Since we aredealing with vectors, once again a norm of f must be calculated.

|f | ≤ εThe norm can be computed as

|f | =√∑n

i=1 f2i

n

where n is the dimension of the system.In addition to the above convergence tests, we might wish to place a

limit on the number of times the iteration is repeated. A MATLAB func-tion, constructed in the form of a m-file, is shown in figure 4.1. Note thatthis MATLAB function requires an initial guess as well as two externalfucntions for computing the function values and the Jacobian.

Example of reactors in series

This example is from section §1.3.5 and consists of a system of nonlinearequations that model a series of continuously stirred tank reactors. Asketch is shown in figure 1.6 Recall that the model equations are givenby,

fi := βa2i + ai − ai−1 = 0 i = 1 · · ·n (4.4)

As discussed in section §1.3.5, there are n equations and (n + 2)variables in total. Hence we have two degrees of freedom. We considera design situation where inlet and outlet concentrations are specifiedas, say, a0 = 5.0, an = 0.5 mol/lit. The unknown vector consists of nvariable elements,

x = a1, a2 · · ·an−1, βWe are required to determine the volume of each reactor for a givennumber n. The volume is given by the expression β = kV/F . The rateconstant is k = 0.125 lit/(mol min) and the feed rate is F = 25 lit/min.


function x=newton(Fun,Jac,x,tol,trace)% Newton method for a system of nonlinear equations% Fun - name of the external function to compute f% Jac - name of the externan function to compute J% x - vector of initial guesses% tol - error criterion% trace - print intermediate results%% Usage newton(’Fun’,’Jac’,x)

%Check inputsif nargin < 5, trace=0; endif nargin < 4, tol=eps; trace=0; end

max=25;n=length(x);count=0;f=1;

while (norm(f) > tol & count < max), %check convergencef = feval(Fun,x); %evaluate the functionJ = feval(Jac,x); %evaluate Jacobianx = x -J\f; %update the guesscount=count+1;if trace,fprintf(1,’Iter.# = %3i Resid = %12.5e\n’, count,norm(f));end

end

if (count >= max)fprintf(1,’Maximum iteration %3i exceeded\n’,count)fprintf(1,’Residual is %12.5e\n ’,norm(f) )end

Figure 4.1: MATLAB implementation of Newton scheme


In solving the above equations we are primarily interested in β, but wealso get all of the intermediate concentrations. Before we can invoke thealgorithm of figure 4.1 we need to write two functions for evaluation thefunction values and the Jacobian. The Jacobian is given by,

J =

∂f1∂x1

0 0 · · · ∂f1∂xn

∂f2∂x1

∂f2∂x2

0 · · · ∂f2∂xn

0 ∂f3∂x2

∂f3∂x3

· · · ∂f3∂xn

... 0. . . . . .

...

0 · · · 0 ∂fn∂xn−1

∂fn∂xn

=

2xnx1 + 1 0 0 · · · x21

−1 2xnx2 + 1 0 · · · x22

0 −1 2xnx3 + 1 · · · x23

... 0. . . . . .

...0 · · · 0 −1 a2

n

(4.5)

The MATLAB functions to compute the function in equation (4.4) andthe Jacobian in equation (4.5) are shown in figure 4.2. Work throughthe following example after ensuring that the three m-files newton.m,cstrF.m and cstrJ.m are the MATLAB search path.

»x=[1 .5 .2 .1 0]’ % Initial guess for n=5»r=newton(’cstrF’,’cstrJ’,x,1.e-10,1) % call Newton schemeIter.# = 1 Resid = 4.06325e+00Iter.# = 2 Resid = 1.25795e+01Iter.# = 3 Resid = 2.79982e+00Iter.# = 4 Resid = 4.69658e-01Iter.# = 5 Resid = 2.41737e-01Iter.# = 6 Resid = 4.74318e-03Iter.# = 7 Resid = 1.61759e-06Iter.# = 8 Resid = 1.25103e-12

r =

2.22621.29190.86910.63990.5597


function f=cstrF(x)% Reactor in series model, the function% x=[a(1),a(2), .., a(n-1),beta]% f(i) = beta a(i)ˆ2 + a(i) - a(i-1)

n=length(x);a0=5.0; an=0.5; %define parameters in equation

f(1) = x(n)*x(1)ˆ2 + x(1) - a0;for i = 2:n-1f(i)= x(n)*x(i)ˆ2 + x(i) - x(i-1);endf(n) = x(n)*anˆ2 + an - x(n-1);f=f’;

function J=cstrJ(x)% Reactor in series model, the Jacobian% x=[a(1),a(2), .., a(n-1),beta]

n=length(x);a0=5.0; an=0.5; %define parameters in equation

J(1,1) = x(n)*2*x(1) + 1;J(1,n) = x(1)ˆ2;for i = 2:n-1J(i,i) = x(n)*2*x(i) + 1;J(i,i-1) = -1;J(i,n) = x(i)ˆ2;endJ(n,n) = anˆ2;J(n,n-1) = -1;

Figure 4.2: CSTR in series example - function & Jacobian evaluation

4.2. EULER-NEWTON CONTINUATION 96

»V=r(5)*25/.125 % compute V (ans:111.9427)»x=[1:-.1:.1]’ % repeat solution for n=10»r=newton(’cstrF’,’cstrJ’,x,1.e-10,1) % call Newton scheme»V=r(10)*25/.125 % compute V (ans:44.9859)

In the above example, observe first that the number of reactors is definedimplicitly by the length of the vector x. Secondly observe the quadraticconvergence of the iteration sequence - viz. the residual goes down from10−3 in iteration number 6 to 10−6 in iteration number 7 and 10−12 initeration number 8. In other words the number of significant digits ofaccuracy doubles with every iteration once the iteration reaches close tothe root.

4.2 Euler-Newton continuation

4.3 Arc-length continuation

4.4 Quasi-Newton methods

4.4.1 Levenberg-Marquadt method

4.4.2 Steepest descent method

4.4.3 Broyden’s method


4.5.1 Turbulent flow through a pipeline network

Consider turbulent flow through the network shown in figure 3.8. Thegoverning equations are the pressure drop equations for each pipe ele-ment i − j and the mass balance equation at each node. The pressuredrop between nodes i and j is given by,

pi − pj = αijv2ij where αij =

2fρlijd2ij

(4.6)

In general the friction factor f is given by the Moody chart or its equiv-alent Churchill correlation. In fully developed turbulent flow it is rel-atively insensitive to changes in Re. Hence take it to be a constantf = 0.005.


The unknown vector is,

x = [p2 p3 p4 v12 v23 v24 v34 v35 v46]

There will be six momentum balance equations, one for each pipe ele-ment, and three mass balance (for incompressible fluids volume balance)equations, one at each node. Arrange them as a system of nine equationsin nine unknowns and solve the resulting set of equations. Take the vis-cosity of the fluid, µ = 0.1Pa·s and the density as ρ = 1000kg/m3. Thedimensions of the pipes are given below.

Table 1

Element no 12 23 24 34 35 46

dij (m) 0.1 0.08 0.08 0.10 0.09 0.09lij (m) 1000 800 800 900 1000 1000

a) Use MATLAB to solve this problem using Newton method for thespecified pressures of p1 = 300kPa and p5 = p6 = 100kPa. Re-port the number of iterations and the flops.

b) Implement the Jacobi iteration to solve this problem. Report thenumber of iterations and the flops.

c) Implement the Gauss-Seidel iteration to solve this problem. Reportthe number of iterations and the flops.

d) Implement the Successive-relaxation iteration to solve this prob-lem. Report the number of iterations and the flops.

e) Find out the new velocity and pressure distributions when p6 ischanged to 150kPa.

f) Comment on how you would adopt the above problem formulationif a valve on line 34 is shut so that there is no flow in that line 34.

The chess-board is the world; the pieces are the phe-nomena of the universe; the rules of the game arewhat we call the laws of Nature. The player on theother side is hidden from us. We know that his playis always fair, just, and patient. But also we know,to our cost, that he never overlooks a mistake, ormakes the smallest allowance for ignorance.

— T.H. HUXLEY

Chapter 5

Functional approximations

In previous chapters we have developed algorithms for solving systemsof linear and nonlinear algebraic equations. Before we undertake thedevelopment of algorithms for differential equations, we need to developsome basic concepts of functional approximations. In this respect thepresent chapter is a bridge between the realms of lumped parametermodels and distributed and/or dynamic models.

There are at least two kinds of functional approximation problemsthat we encounter frequently. In the first class of problem, a knownfunction f(x) is approximated by another function, Pn(x) for reasonsof computational necessity or expediency. As modelers of physical phe-nomena, we often encounter a second class of problem in which thereis a need to represent an experimentally observed, discrete set of dataof the form xi, fi|i = 1, · · ·n as a function of the form f(x) over thedomain of the independent variable x.

98

5.1. APPROXIMATE REPRESENTATION OF FUNCTIONS 99

5.1 Approximate representation of functions

5.1.1 Series expansion

As an example of the first class of problem, consider the evaluation ofthe error function given by,

erf(x) = 2√π

∫ x0e−ξ

2dξ

Since the integral does not have a closed form expression, we have touse a series expansion for,

e−ξ2 =

∞∑k=0

(−1)kξ2k

k!

Note that this expansion is around ξ = 0. We can integrate the seriesexpansion term-by-term to obtain,

erf(x) = 2√π

∞∑k=0

(−1)kx2k+1

(2k+ 1)k!

We can now choose to approximate this function as,

erf(x) ≈ P2n+1(x) = 2√π

n∑k=0

(−1)kx2k+1

(2k+ 1)k!+ R(x)

by truncating the infinite series to n terms. The error introduced bytruncating such a series is called the truncation error and the magnitudeof the residual function, R(x) represents the magnitude of the truncationerror. Forx close to zero a few terms of the series (smalln) are adequate.The convergence of the series is demonstrated in Table 5.1. It is clearthat as we go farther away from x = 0, more terms are required forP2n+1(x) to represent erf(x) accurately. The error distribution, definedas ε(x,n) := |erf(x) − P2n+1(x)|, is shown in figure 5.1. It is clearfrom figure 5.1a, that for a fixed number of terms, say n = 8, the errorincreases with increasing values of x. For larger values of x, more termsare required to keep the error small. For x = 2.0, more than 10 termsare required to get the error under control.

5.1.2 Polynomial approximation

In the above example we chose to construct an approximate function torepresent f(x) = erf(x) by expanding f(x) in Taylor series aroundx =


n P2n+1(x = 0.5) P2n+1(x = 1.0) P2n+1(x = 2.0)2 0.5207 0.865091 2.858564 0.5205 0.843449 2.094376 0.5205 0.842714 1.331248 0.5205 0.842701 1.05793

10 0.5205 0.842701 1.0031820 0.5205 0.842701 0.995322

Exact 0.5205 0.842701 0.995322

Table 5.1: Convergence of P2n+1(x) to erf(x) at selected values of x

0 0.5 1 1.5 20

0.0001

0.0002

0.0003

0.0004

0.0005

0.0006

0.0007

2 4 6 8 100

-65. 10

0.00001

0.000015

0.00002

0.000025

2 4 6 8 100

0.002

0.004

0.006

0.008

0.01

2 4 6 8 100

0.25

0.5

0.75

1

1.25

1.5

1.75

x n

n=2 4 6 8

(b) x=0.5

(d) x=2.0(c) x=1.0

(a)

nn

erro

rer

ror

Figure 5.1: Error distribution of ε(x,n) := |erf(x)− P2n+1(x)| for dif-ferent levels of truncation


0. This required that all the higher order derivative be available at x = 0.Also, since the expansion was around x = 0, the approximation failsincreasingly as x moves away from zero. In another kind of functionalapproximation we can attempt to get a good representation of a givenfunction f(x) over a range x ∈ [a, b]. We do this by choosing a set ofn basis functions, φi(x)|i = 1 · · ·n that are linearly independent andrepresenting the approximation as,

f(x) ≈ Pn(x) =n∑i=1

aiφi(x)

Here the basis functions φi(x) are known functions, chosen with careto form a linearly independent set and ai are unknown constants thatare to be determined in such a way that we can make Pn(x) as good anapproximation to f(x) as possible - i.e., we can define an error as thedifference between the exact function and the approximate representa-tion,

ε(x;ai) = |f(x)− Pn(x)|and device a scheme to select ai such that the error is minimized.

Example

So far we have outlined certain general concepts, but left open the choiceof a specific basis functions φi(x), the definition of the norm |.| in theerror or the minimization procedure to get ai.

Let the basis functions be

φi(x) = xi−1 i = 1, · · ·n

which, incidentally is a poor choice, but one that is easy to understand.Hence the approximate function will be a polynomial of degree (n − 1)of the form,

Pn−1(x) =n∑i=1

aixi−1

Next, let us introduce the idea of collocation to evaluate the error at nselected points in the range of interest x ∈ [a, b]. We choose n pointsxk|k = 1, · · ·n because we have introduced n degrees of freedom(unknowns) in ai. A naive choice would be to space these collocationpoints equally in the interval [a, b] - i.e.,

xk = a+ (k− 1)(b − a)(n− 1)

k = 1, · · · , n


Finally we can require the error at these points to be exactly equal tozero - i.e.,

ε(xk;ai) = f(xk)− Pn−1(xk) = 0

orn∑i=1

aixi−1k = f(xk) k = 1, · · · , n (5.1)

which yields n linear equations in n unknowns ai. This can be writtenin matrix form

Pa = fwhere the elements of matrix P are given by, Pk,i = xi−1

k and the vectorsare a = [a1, · · · , an] and f = [f (x1), · · · , f (xn)]. Thus we have re-duced the functional approximation problem to one of solving a systemof linear algebraic equations and tools of chapter 3 become useful!

Let us be even more specific now and focus on approximating theerror function f(x) = erf(x) over the interval x ∈ [0.1,0.5]. Let usalso choose n = 5 - i.e., a quartic polynomial. This will allow us to writeout the final steps of the approximation problem explicitly. The equallyspaced collocation points are,

xk = 0.1,0.2,0.3,0.4,0.5

and the error function values at the collocation points are

f = f(xk) = [0.1125,0.2227,0.3286,0.4284,0.5205]

Thus, equation (5.1) yields the following system

P =

1 x1 x21 x3

1 x41

1 x2 x22 x3

2 x42

1 x3 x23 x3

3 x43

1 x4 x24 x3

4 x44

1 x5 x25 x3

5 x45

=

1.0 0.10 0.010 0.0010 0.00011.0 0.20 0.040 0.0080 0.00161.0 0.30 0.090 0.0270 0.00811.0 0.40 0.160 0.0640 0.02561.0 0.50 0.250 0.1250 0.0625

Solution of the linear system yields the unknown coefficients as

a = 0.0001,1.1262,0.0186,−0.4503,0.1432

A MATLAB function that shows the implementation of the above pro-cedure for a specified degree of polynomialn is given in figure 5.2. Recallthat we had made a comment earlier that the basis function φi(x) =xi−1 i = 1, · · ·n is a poor choice. We can understand why this is so,


function a=erf_apprx(n)% Illustration functional (polynomial) approximation% fits error function in (0.1, 0.5) to a% polynomial of degree n

%define intervala = 0.1; b=0.5;

%pick collocation pointsx=a + [0:(n-1)] *(b-a)/(n-1);

%Calculate the error function at collocation pointsf=erf(x); %Note that erf is a MATLAB function

%Calculate the matrixfor k=1:nP(k,:) = x(k).ˆ[0:n-1];end

%Print the determinant of Pfprintf(1,’Det. of P for deg. %2i is = %12.5e\n’, n,det(P) );

%Determine the unknown coefficients a_ia=P\f’;

Figure 5.2: MATLAB implementation illustrating steps of functional ap-proximation

5.2. APPROXIMATE REPRESENTATION OF DATA 104

by using the function shown in figure 5.2 for increasing degree of poly-nomials. The matrix P becomes poorly scaled and nearly singular withincreasing degree of polynomial as evidence by computing the determi-nant of P. For example the determinant of P is 1.60000×10−2 for n = 3and it goes down rapidly to 1.21597 × 10−12 for n = 6. Selecting cer-tain orthogonal polynomials such as Chebyshev polynomials and usingthe roots of such polynomials as the collocation points results in wellconditioned matrices and improved accuracy. More on this in section§5.8.

Note that MATLAB has a function called polyfit(x,y,n) that willaccept a set of pairwise data xk,yk = f(xk) | k = 1, · · · ,m and pro-duce a polynomial fit of degreen (which can be different fromm) using aleast-squares minimization. Try using the function polyfit(x,y,n) forthe above example and compare the polynomial coefficients a producedby the two approaches.

»x=[0.1:0.1:0.5] % Define Collocation Points»y=erf(x) % Calculate the function at Collocation Points»a=polyfit(x,y,4)% Fit 4th degree polynomial. Coefficients in a»polyval(a,x) % Evaluate the polynomial at collocation pts.»erf(x) % Compare with exact values at the same pts.

5.2 Approximate representation of data

The concepts of polynomial approximation were discussed in section§5.1.2 in the context of constructing approximate representations ofcomplicated functions (such as the error function). We will develop andapply these ideas further in later chapters for solving differential equa-tions. Let us briefly explore the problem of constructing approximatefunctions for representing a discrete set of m pairs of data points

(xk, fk) | k = 1, · · · ,m

gathered typically from experiments. As an example, let us consider thesaturation temperature vs. pressure data taken from steam tables andshown in Table 5.2. Here the functional form that we wish to constructis to represent pressure as a function of temperature, P(T) over the tem-perature range T ∈ [220,232]. A number of choices present themselves.


T(oF) P(psia)220.0000 17.1860224.0000 18.5560228.0000 20.0150232.0000 21.5670

Table 5.2: Saturation temperature vs. pressure from steam tables

• We can choose to fit a cubic polynomial, P3(T) that will pass througheach of the four data points over the temperature range T ∈ [220,232].This will be considered as a global polynomial as it covers the entirerange of interest in T .

• Alternately we can choose to construct piecewise polynomials of alower degree with a limited range of applicability. For example, wecan take the first three data points and fit a quadratic polynomial,and the last three points and fit a different quadratic polynomial.

• As a third alternative, we can choose to fit a global polynomial ofdegree less than three, that will not pass through any of the givendata points, but will produce a function that minimizes the errorover the entire range of T ∈ [220,232].

The procedures developed in section §5.1.2 are directly applicableto the first two choices and hence they warrant no further discussion.Hence we develop the algorithm only the third choice dealing with theleast-squares minimization concept.

5.2.1 Least squares approximation

Suppose there are m independent experimental observations (m = 4 inthe above example) and we wish to fit a global polynomial of degree n(n <m) we define the error at every observation point as,

εk = (Pn−1(xk)− fk) k = 1, · · · ,m

The basis functions are still the set, xi−1 | i = 1, · · ·n and the poly-nomial is

Pn−1(x) =n∑i=1

aixi−1


Here ai are the unknowns that we wish to determine. Next we constructan objective function which is the sum of squares of the error at everyobservation point - viz.

J(a) =∑mk=1 ε

2k

m=∑mk=1(Pn−1(xk)− fk)2

m=∑mk=1(

∑ni=1 aix

i−1k − fk)2

m

The scalar objective function J(a) is a function of n unknowns ai. Fromelemetary calculus, the condition for the function J(a) to have a mini-mum is,

∂J(a)∂a

= 0

This condition provides n linear equations of the form Pa = b that canbe solved to obtain a. The expanded form of the equations are,

∑mk=1 1

∑mk=1 xk

∑mk=1 x

2k · · · ∑m

k=1 xn−1k∑m

k=1 xk∑mk=1 x

2k

∑mk=1 x

3k · · · ∑m

k=1 xnk∑m

k=1 x2k

∑mk=1 x

3k

∑mk=1 x

4k

∑mk=1 x

n+1k

......

. . ....∑m

k=1 xn−1k

∑mk=1 x

nk

∑mk=1 x

n+1k · · · ∑m

k=1 x2(n−1)k

a1

a2

a3...an

=

∑mk=1 fk∑mk=1 fkxk∑mk=1 fkx

2k

...∑mk=1 fkx

n−1k

Observe that the equations are not only linear, but the matrix is alsosymmetric. Work through the following example using MATLAB to gen-erate a quadratic, least-squares fit for the data shown in Table 5.2. Makesure that you understand what is being done at each stage of the cal-cualtion. This example illustrates a cubic fit that passes through each ofthe four data points, followed by use of the cubic fit to interpolate dataat intermediate temperatures of T = [222,226,230]. In the last part theleast squares solution is obtained using the procedure developed in thissection. Finally MATLAB’s polyfit is used to generate the same leastsquares solution!

»x=[220,224,228,232] % Define temperatures»f=[17.186,18.556,20.015,21.567] % Define pressures»a3=polyfit(x,f,3) % Fit a cubic. Coefficients in a3»polyval(a3,x) % Check cubic passes through pts.»xi=[222,226,230] % Define interpolation points»polyval(a3,xi) % Evaluate at interpolation pts.»%get ready for least square solution!»x2=x.ˆ2 % Evaluate x2

»x3=x.ˆ3 % Evaluate x3

»x4=x.ˆ4 % Evaluate x4

5.3. DIFFERENCE OPERATORS 107

»P=[4,sum(x),sum(x2); ... % Define matrix P over next 3 lines»sum(x), sum(x2), sum(x3); ...»sum(x2), sum(x3), sum(x4) ]»b=[sum(f), f*x’, f*x2’] % Define right hand side»a = P\b′ % ans: (82.0202,-0.9203,0.0028)»c=polyfit(x,f,2) % Let MATLAB do it! compare c & a»norm(f-polyval(a3,x)) % error in cubic fit 3.3516× 10−14

»norm(f-polyval(c,x)) % error in least squares fit 8.9443× 10−4

5.3 Difference operators

In the previous sections we developed polynomial approximation schemesin such a way that they required a solution of a system of linear alge-braic equation. For uniformly spaced data, introduction of differenceoperators and difference tables, allows us to solve the same polynomialapproximation problem in a more elegant manner without the need forsolving s system of algebraic equations. This difference operator ap-proach also lends itself naturally to recursive construction of higherdegree polynomials with very little additional computation as well asextension to numerical differentiation and integration of discrete set ofdata.

Consider the set of data (xi, fi) | i = 1, · · · ,m where the indepen-dent variable, x is varied uniformly generating an equally spaced data -i.e.,

xi+1 = xi + h, i = 1, · · ·m or xi = x1 + (i− 1)h

The forward difference operator, as introduced already in section §2.8,is defined by,

Forward difference operator

∆fi = fi+1 − fi (5.2)

In a similar manner we can define a backward difference, central differ-ence and shift operators as shown below.

Backward difference operator

∇fi = fi − fi−1 (5.3)


Central difference operator

δfi = fi+1/2 − fi−1/2 (5.4)

Shift operator

Efi = fi+1 (5.5)

We can also add the differential operator to the above list.

Differential operator

Df(x) = df(x)dx

= f ′(x) (5.6)

The difference operators are nothing but rules of calculations, justlike a differential operator defines a rule for differentiation. Clearly theserules can be applied repeatedly to obtaind higher order differences. Forexample a second order forward difference with respect to referencepoint i is,

∆2fi = ∆(∆fi) = ∆(fi+1 − fi) = fi+2 − 2fi+1 + fi

5.3.1 Operator algebra

Having introduced some new definitions of operators, we can discoversome interesting relationships between various operators such as thefollowing. ∆fi = fi+1 − fi and Efi = fi+1

Combining these two we can write,

∆fi = Efi − fi = (E − 1)fi

Since the operand fi is the same on both sides of the equation, the op-erators (which define certain rules and hence have certain effects on theoperand fi) have an equivalent effect given by,

∆ = (E − 1) or E = (1+∆) (5.7)

Equation (5.7) can then be applied on any other operand like fi+k. Allof the operators satisfy the distributive, commutative and associativerules of algebra. Also, repeated application of the operation can be rep-resented by,

Eα = (1+∆)α


Note that Eαf(x) simply implies that the function f is evaluated aftershifting the independent variable by α - i.e.,

Eαf(x) = f(x +αh)Hence α can be an integer or any real number. Similarly, we have

∇fi = fi − fi−1 and Efi−1 = fi and fi−1 = E−1fi

where we have introduce the inverse of the shift operator E to shift back-wards. Combining these we can write,

∇fi = fi − E−1fi = (1− E−1)fi

Once again recognizing that the operand fi is the same on both sides ofthe equation, the operators are related by,

∇ = (1− E−1) or E−1 = (1−∇) or E = (1−∇)−1 (5.8)

Yet another relation between the shift operator E and the differentialoperator D can be developed by considering the Taylor series expansionof f(x + h),

f(x + h) = f(x)+ hf ′(x)+ h2

2!f ′′(x)+ · · ·

which can be written in operator notation as,

Ef(x) =[

1+ hD + h2D2

2!+ · · ·

]f(x)

The term in square brackets is the exponential function and hence

E = ehD (5.9)

While such a game of discovering relationships between various oper-ators can be played indefinitely, let us turn to developing some usefulalgorithms from these.

5.3.2 Newton forward difference approximation

Our objective is to construct a polynomial representation for the discreteset of data (xi, fi) | i = 1, · · · ,m using an alternate approach fromthat of section §5.1.2. Assuming that there is a function f(x) represent- Is such an

assumption alwaysvalid?


ing the given data, we can express such a function as,

f(x) = Pn(x)+ R(x)where Pn(x) is the polynomial approximation to f(x) and R(x) is theresidual error. Given a set of m data points we know at least one way(section §5.1.2) to can construct a polynomial of degree (m − 1). Nowlet us use the power of operator algebra to develop an alternate way toconstruct such a polynomial and in the process, also learn somethingabout the residual function R(x). Applying equation (5.7) repeatedly αtime on f(x) we get,

Eαf(x) = (1+∆)αf(x)Now for integer values of α the right hand side is the binomial expansionwhile for any real number, it yields an infinite series. Using such anexpansion the above equation can be written as,

f(x +αh) =[

1+α∆+ α(α− 1)2!

∆2 + α(α− 1)(α− 2)3!

∆3 + · · ·α(α− 1)(α− 2) · · · (α−n+ 1)

n!∆n + · · ·]f(x)(5.10)

Up to this point in our development we have merely used tricks ofoperator algebra. We will now make the direct connection to the given,discrete set of data (xi, fi) | i = 1, · · · ,m. Taking x1 as the referencepoint, the transformation

x = x1 +αhmakes α the new independent variable and for integer values of α =0,1, · · · (m−1) we retrieve the equally spaced data set x1, x2, · · ·xmand for non-integer (real) values of α we can reach the other values ofx ∈ (x1, xm). Splitting equation (5.10) into two parts,

f(x1 +αh) =[

1+α∆+ α(α− 1)2!

∆2 + α(α− 1)(α− 2)3!

∆3 + · · ·α(α− 1)(α− 2) · · · (α−m+ 2)

(m− 1)!∆m−1

]f(x1)+ R(x)

we can recognize the terms in the square brackets as a polynomial of de-gree (m − 1) in the transformed variable α. We still need to determinethe numbers ∆f(x1),∆2f(x1), · · · ,∆(m−1)f (x1). These can be com-puted and organized as a forward difference table shown in figure 5.3.Since forward differences are needed for constructing the polynomial, it


x1 f1x2 f2x3 f3x4 f4

xm fm

xm-1 fm-1

∆f1∆f2∆f3

∆fm-1

∆2f1∆2f2∆2f3

∆2fm-2

∆f4

∆3f1∆3f2

∆3fm-3

x5 f5

∆4f1

∆4fm-4

∆m-1f1

Figure 5.3: Structure of Newton forward difference table for m equallyspaced data

is called the Newton forward difference polynomial and it is given by,

Pm−1(x1 +αh) =[

1+α∆+ α(α− 1)2!

∆2 + α(α− 1)(α− 2)3!

∆3 + · · ·α(α− 1)(α− 2) · · · (α−m+ 2)

(m− 1)!∆m−1

]f(x1)+O(hm) (5.11)

The polynomial in equation (5.11) will pass through the given data set(xi, fi) | i = 1, · · · ,m - i.e., for integer values of α = 0,1, · · · (m− 1)it will return values of f1, f2, · · ·fm. This implies that the residualfunction R(x) will have roots at the data points xi | i = 1, · · · ,m. Fora polynomial of degree (m−1), shown in equation (5.11), the residual atother values of x is typically represented as R(x) ≈ O(hm) to suggestthat the leading term in the truncated part of the series is of order m.

Example

A set if five (m = 5) equally spaced data points and the forward differ-ence table for the data are shown in figure 5.4. For this example, clearlyh = 1 and x = x1 + α. We can take the reference point as x1 = 2 andconstruct the following linear, quadratic and cubic polynomials, respec-tively. Note that

P4(2+α) =P3(2+α) for thiscase! Why?


fxf

ffx

ff

fffx

ff

ffxf

fx

55

4

32

44

23

3

14

22

33

13

2

12

22

1

11

===∆

=∆===∆=∆

=∆=∆===∆=∆

=∆===∆

==

612619

035215616

042464673

8172391

82

Figure 5.4: Example of a Newton forward difference table

P1(2+α) = [1+α∆] f (x1) = (8)+α(19)+O(h2)

P2(2+α) = (8)+α(19)+ α(α− 1)2!

(18)+O(h3)

P3(2+α) = (8)+α(19)+ α(α− 1)2!

(18)+ α(α− 1)(α− 2)3!

(6)+O(h4)

You can verify easily that P1(2 + α) passes through x1, x2, P2(2 + α)passes through x1, x2, x3 and P3(2+α) passes through x1, x2, x3, x4.For finding the interpolated value of f(x = 3.5) for example, first deter-mine the values of α at x = 3.5 from the equation x = x1 + αh. It isα = (3.5− 2)/1 = 1.5. Using this value in the cubic polynomial,

P3(2+1.5) = (8)+1.5(19)+ 1.5(0.5)2!

(18)+ 1.5(0.5)(−0.5)3!

(6) = 42.875

As another example, by taking x3 = 4 as the reference point we canconstruct the following quadratic polynomial

P2(4+α) = (64)+α(61)+ α(α− 1)2!

(30)

which will pass through the data set x3, x4, x5. This illustration shouldshow that once the difference talbe is constructed, a variety of polyno-mials of varying degrees can be constructed quite easily.

5.3.3 Newton backward difference approximation

An equivalent class of polynomials using the backward difference oper-ator based on equation (5.8) can be developed. Applying equation (5.8)


repeatedly α times on f(x) we get,

Eαf(x) = (1−∇)−αf(x)which can be expanded as before to yield,

f(x +αh) =[

1+α∇+ α(α+ 1)2!

∇2 + α(α+ 1)(α+ 2)3!

∇3 + · · ·α(α+ 1)(α+ 2) · · · (α+n− 1)

n!∇n + · · ·

]f(x)(5.12)

As with the Newton forward formula, the above equation (5.12) termi-nates at a finite number of terms for integer values of α and for non-integer values, it will always be an infinite series which must be tran-cated, thus sustaining a trunctaion error.

In making the precise connection to a given discrete data set (xi, fi) | i =0,−1 · · · ,−n, typically the largest value of x (say, x0) is taken as thereference point. The transformation

x = x0 +αhmakes α the new independent variable and for negative integer valuesof α = −1, · · ·−n we retrieve the equally spaced data set x−1, · · ·x−nand for non-integer (real) values of α we can reach the other values ofx ∈ (x−n,x0). Splitting equation (5.12) into two parts,

f(x0 +αh) =[

1+α∇+ α(α+ 1)2!

∇2 + α(α+ 1)(α+ 2)3!

∇3 + · · ·α(α+ 1)(α+ 2) · · · (α+n− 1)

n!∇n + · · ·

]f(x0)+ R(x)

we can recognize the terms in the square brackets as a polynomial ofdegree n in the transformed variable α. We still need to determinethe numbers ∇f(x0),∇2f(x0), · · · ,∇nf(x0). These can be computedand organized as a backward difference table shown in figure 5.5. Sincebackward differences are needed for constructing the polynomial, it iscalled the Newton backward difference polynomial and it is given by,

Pn(x0 +αh) =[

1+α∇+ α(α+ 1)2!

∇2 + α(α+ 1)(α+ 2)3!

∇3 + · · ·α(α+ 1)(α+ 2) · · · (α+n− 1)

n!∇n]f(x0)+O(hn+1) (5.13)

The polynomial in equation (5.13) will pass through the given data set(xi, fi) | i = 0, · · · ,−n - i.e., for integer values of α = 0,−1, · · ·−n itwill return values of f0, f−1, · · ·f−n. At other values of x the residualwill be of order O(hn+1).


fxf

ffx

ff

fffx

ff

ffxf

fx

−−

−

−−−

−−

−−−

−−−

00

0

02

11

03

1

04

12

22

13

2

22

33

3

44

∇∇

∇∇∇∇

∇∇∇

∇

Figure 5.5: Structure of Newton backward difference table for 5 equallyspaced data


? ? ? ? ?? ? ? ? ? ?

? ? ? ?? ? ? ? ?

? ? ?? ? ? ? ?

? ? ? ?? ? ? ? ? ?

? ? ? ? ?fxf

ffx

ff

fffx

ff

ffxf

fx

−−

−

−−−

−−

−−−

−−−

00

0

02

11

03

1

04

12

22

13

2

22

33

3

44

===∇

=∇===∇=∇

=∇=∇===∇=∇

=∇===∇

==

612619

035215

616

042464673

8172391

82

NBF around x0

NBF around x -2

NFF around x-2

Figure 5.6: Example of a Newton backward difference table

Example

A set if five (n = 4) equally spaced data points and the backward differ-ence table for the data are shown in figure 5.6. This is the same exampleas used in the previous section! It is clear that h = 1 and x = x0 + α.In the previous case we constructed a linear, quadratic and cubic poly-nomials, with x3 = 4 as the reference point. In the present case let ususe the same reference point, but it is labelled as x−2 = 4. A quadraticbackward difference polynomial in α is,

P2(4+α) = (64)+α(37)+ α(α+ 1)2!

(18)+O(h3)

which passes through the points (x−2, f−2), (x−3, f−3) and (x−4, f−4)for α = 0,−1,−2, respectively. Recall that the forward difference poly-nomial around the same point was, Calculate the

interpolated valueof f(4.5) fromthese twopolynomials

P2(4+α) = (64)+α(61)+ α(α− 1)2!

(30)

which passes through the three forward point for α = 0,1,2. Althoughthey are based on the same reference point, these are two different poly-nomials passing through a different set of data points.

As a final example, let us construct a quadratic backward differencepolynomial around x0 = 6. It is, Is this polynomial

different from theNFF, P2(4+α)constructed above?

P2(6+α) = (216)+α(91)+ α(α+ 1)2!

(30)

5.4. INVERSE INTERPOLATION 116

f(x=?)=100

-2 0 2 4 6

0

25

50

75

100

125

150

f(x)

P2 (4+α)

x

f(x) = x3

x=0

.28

69

62

(sp

urio

us

roo

t)

x=4

.64

63

7 (

de

sire

d r

oo

t)

Figure 5.7: Example of a inverse interpolation

5.4 Inverse interpolation

One of the objectives in constructing an interpolating polynomial is tobe able to evaluate the function f(x) at values of x other than the onesin the discrete set of given data points (xi, fi). The objective of inverseinterpolation is to determine the independent variable x for a given valueof f using a given discrete data set (xi, fi). If xi are equally spaced, wecan combined two of the tools (polynomial curve fitting and root finding)to meet this objective, although this must be done with caution.

We illustrate this with the example data shown in figure 5.4. Supposewe wish to find the value of x where f = 100. Using the three data pointsin the neighbourhood of f = 100 in figure 5.4 viz. (x3, x4, x5), and usinga quadratic polynomial fit, we have,

P2(4+α) = (64)+α(61)+ α(α− 1)2!

(30)

A graph of this polynomial approximation P2(4+α) and the actual func-tion f(x) = x3 used to generate the data given in figure 5.4 are shownin figure 5.7. It is clear that the polynomial approximation is quite goodin the range of x ∈ (4,6), but becomes a poor approximation for lowervalues of x. Note, in particular, that if we solve the inverse interpolationproblem by setting

P2(4+α)− 100 = 0 or (64)+α(61)+ α(α− 1)2!

(30)− 100 = 0

5.4. INVERSE INTERPOLATION 117

we will find two roots. One of them at α = 0.64637 or x = 4.64637 isthe desired root while the other at α = −3.71304 or x = 0.286962 is aspurious one. This problem can become compounded as we use higherdegree polynomial in an effort to improve accuracy.

In order to achieve high accuracy, but stay close to the desired root,we can generate an initial guess from a linear interpolation, followed byconstructing a fixed point iteration scheme on the polynomial approxi-mation of the desired accuracy. Convergence is generally fast as shownin Dahlquist and Bjorck (1974). Suppose we wish to findx correspondingto f(x) = d, the desired function value. We first construct a polynomialof degree (m− 1) to represent the tabular data.

Pm−1(x1 +αh) =[

1+α∆+ α(α− 1)2!

∆2 + α(α− 1)(α− 2)3!

∆3 + · · ·α(α− 1)(α− 2) · · · (α−m+ 2)

(m− 1)!∆m−1

]f(x1)

Then we let f(x1 +αh) = d and rearrange the polynomial in the form

αi+1 = g(αi) i = 0,1,2 · · ·

where g(α) is obtained by rearranging the polynomial,

g(α) = 1∆f1

[d− f1 − α(α− 1)

2!∆2f1 − α(α− 1)(α− 2)

3!∆3f1 + · · ·

]

and the initial guess obtained by truncating the polynomial after thelinear term,

α0 = d− f1∆f1

Example

Continuing with the task of finding x where f(x) = 100 for the datashown in figure 5.4, the fixed point iterate is,

αi+1 = [100− 64− 15αi(αi − 1)] /61

and the initial guess is

α0 = d− f1∆f1= 100− 64

61= 0.5902

The first ten iterates, produced from the m-file given below,

5.5. LAGRANGE POLYNOMIALS 118

i αi1 .590163932 .649640283 .646133064 .646388155 .646369806 .646371127 .646371028 .646371039 .64637103

10 .64637103

Table 5.3: Inverse interpolation

function a=g(a)for i=1:10fprintf(1,’%2i %12.7e\n’,i,a);a=(100 - 64 - 15*a*(a-1))/61;end

are shown in Table 5.3.

5.5 Lagrange polynomials

So far we have examined ways to construct polynomial approximationsusing equally spaced data in x. For a data set xi, fi|i = 0, · · ·n, thatcontains unequally spaced data in the independent variable x, we canconstruct Lagrange interpolation formula as follows.

Pn(x) =n∑i=0

fiδi(x) (5.14)

where

δi(x) =n∏

j=0,j 6=i

x − xjxi − xj

Note that

δi(xj) =

0 j 6= i1 j = i


and each δi(x) is a polynomial of degree n. It is also clear from equation(5.14) that Pn(xj) = fj - i.e., the polynomial passes through the datapoints (xj, fj).

An alternate way to construct the Lagrange polynomial is based onintroducing the divided difference and constructing a divided differencetable. The polynomial itself is written in the form

Pn(x) =n∑i=0

aii∏j=0

(x − xj−1) (5.15)

= a0 + a1(x − x0)+ a2(x − x0)(x − x1)+ · · ·+an(x − x0) · · · (x − xn−1)

The advantage of writing it the form shown in equation (5.15) is that theunknown coefficients ai can be constructed recursively or found directlyfrom the divided difference table. The first divided difference is definedby the equation,

f[x0, x1] = f1 − f0

x1 − x0

Similarly the second divided difference is defined as,

f[x0, x1, x2] = f[x1, x2]− f[x0, x1]x2 − x0

With these definitions, we return to the task of finding the coefficientsai in equation (5.15) For example, the first coefficient a0 is,

Pn(x0) = a0 = f[x0] = f0

The second coefficient, a1, is obtained from,

Pn(x1) = a0 + a1(x1 − x0) = f1


a1 = f1 − f0

x1 − x0= f[x0, x1]

The third coefficient is obtained from,

Pn(x2) = a0 + a1(x2 − x0)+ a2(x2 − x0)(x2 − x1) = f2

The only unknown here is a2, which after some rearrangement becomes,

a2 = f[x1, x2]− f[x0, x1]x2 − x0

= f[x0, x1, x2]

In general the n-th coefficient is the n-th divided difference.

an = f[x0, x1, · · · , xn]


[ ][ ]

[ ] [ ][ ]

[ ].f.x

.xxf.xxxf.f.x

.xxxxf.xxf.xxxf.f.x

.xxf.f.x

33

32

32122

321021

21011

10

00

===

=====

====

==

69046100127,

0034,,5733510001,,,00945,

0073,,82712100463,

000101

Figure 5.8: Structure of divided difference table for 4 unequally spaceddata

Example

Consider the example data and the divided difference table shown in fig-ure 5.8. If we wish to construct a quadratic polynomial passing through(x0, f0), (x1, f1), (x2, f2) for example using equation (5.14), it will be

P2(x) = f0(x − x1)(x − x2)(x0 − x1)(x0 − x2)

+ f1(x − x0)(x − x2)(x1 − x0)(x1 − x2)

+ f2(x − x0)(x − x1)(x2 − x0)(x2 − x1)

= 1.00(x − 1.2)(x − 1.5)(1− 1.2)(1− 1.5)

+ 1.728(x − 1)(x − 1.5)(1.2− 1)(1.2− 1.5)

+ 3.375(x − 1)(x − 1.2)(1.5− 1)(1.5− 1.2)

The same polynomial using equation (5.15) and the difference table shownin figure 5.8 will be written as,

P2(x) = f0 + f[x0, x1](x − x0)+ f[x0, x1, x2](x − x0)(x − x1)= 1.000+ 3.64(x − 1)+ 3.70(x − 1)(x − 1.2)

Observe that in order to construct a cubic polynomial by adding the addi-tional data point (x3, f3) Lagrange polynomial based on equation (5.14)requires a complete reconstruction of the equation, while that based onequation (5.15) is simply,

P2(x) = f0 + f[x0, x1](x − x0)+ f[x0, x1, x2](x − x0)(x − x1)+f[x0, x1, x2, x3](x − x0)(x − x1)(x − x2)

= 1.000+ 3.64(x − 1)+ 3.70(x − 1)(x − 1.2)+1(x − 1)(x − 1.2)(x − 1.5)

A MATLAB function that implements that Lagrange interpolation for-mula shown in equation (5.14) is given in figure 5.9. This function ac-


function f=LagrangeP(xt,ft,x)% (xt,ft) are the table of unequally spaced values% x is where interpolated values are required% f the interpolated values are returned

m=length(x);nx=length(xt);ny=length(ft);

if (nx ˜= ny),error(’ (xt,ft) do not have the same # values’)

end

for k=1:msum = 0;for i=1:nxdelt(i)=1;for j=1:nxif (j ˜= i),delt(i) = delt(i)*(x(k)-xt(j))/(xt(i)-xt(j));

endendsum = sum + ft(i) * delt(i) ;endf(k)=sum;end

Figure 5.9: MATLAB implementation of Lagrange interpolation polyno-mial

5.6. NUMERICAL DIFFERENTIATION 122

cepts a table of values (xt, ft), constructs the highest degree Lagrangepolynomial that is possible and finally evaluates and returns the inter-polated values of the function y at specified values of x.

»xt=[1 1.2 1.5 1.6] % Define xt, unequally spaced»ft=[1 1.728 3.375 4.096] % Define ft, the function values»x=[1.0:0.1:1.6] % x locations for interpolation»f=LagrangeP(xt,ft,x) % interpolated f values.

5.6 Numerical differentiation

Having obtained approximate functional representations as outlined insections §5.3.2 or §5.5, we can proceed to construct algorithms for ap-proximate representations of derivatives.

5.6.1 Approximations for first order derivatives

Consider the Newton forward formula given in equation (5.11)

f(x) ≈ Pm−1(x1 +αh) =[

1+α∆+ α(α− 1)2!

∆2+α(α− 1)(α− 2)

3!∆3 + · · · + α(α− 1) · · · (α−m+ 2)

(m− 1)!∆m−1

]f(x1)+O(hm)

that passes through the given data set (xi, fi) | i = 1, · · · ,m. Notethat the independent variable x has been transformed into α using x =x1 +αh, hence dx/dα = h. Now, the first derivative is obtained as,

f ′(x) = dfdx

≈ dPm−1

dx= dPm−1

dαdαdx

= 1h

[∆+ α+ (α− 1)2

∆2+α(α− 1)+ (α− 1)(α− 2)+α(α− 2)

6∆3 + · · ·

]f(x1)(5.16)

Equation (5.16) forms the basis of deriving a class of approximations forfirst derivatives from a tabular set of data. Note that the equation (5.16)is still a function in α and hence it can be used to evaluate the derivativeat any value of x = x1 + αh. Also, the series can be truncated afterany number of terms. Thus, a whole class of successively more accuraterepresentations for the first derivative can be constructed from equa-tion (5.16) by truncating the series at higher order terms. For example


evaluating the derivative at the reference point x1, ( i.e., α = 0) equation(5.16) reduces to,

f ′(x1) = 1h

[∆− 12∆2 + 1

3∆3 − 1

4∆4 · · · ± 1

m− 1∆m−1

]f(x1)+O(hm−1)

This equation can also be obtained directly using equation (5.9) as,

E = ehD or hD = lnE = ln (1+∆)Expanding the logarithmic term we obtain,

hD = ∆− ∆2

2+ ∆3

3− ∆4

4+ · · ·

Operating both sides with f(x1) ( i.e., using x1 as the reference point),we get,

Df(x1) = f ′(x1) = 1h

[∆− ∆2

2+ ∆3

3− ∆4

4+ · · ·

]f(x1) (5.17)

Now, truncating the series after the first term (m = 2),

f ′(x1) = 1h[∆f(x1)]+O(h)

= 1h[f2 − f1]+O(h)

which is a 2-point, first order accurate, forward difference approximationfor first derivative at x1. Truncating the series after the first two terms(m = 3),

f ′(x1) = 1h

[∆f(x1)− 12∆2f(x1)

]+O(h2)

= 1h

[(f2 − f1)− 1

2(f1 − 2f2 + f3)

]+O(h2)

= 12h[−3f1 + 4f2 − f3]+O(h2)

which is the 3-point, second order accurate, forward difference approx-imation for the first derivative at x1. Clearly both are approximate rep-resentations of the first derivative at x1, but the second one is moreaccurate since the truncation error is of the order h2.

Note that while, equation (5.17) is evaluated at the reference pointon both sides of the equation, the earlier equation (5.16) is a polynomialthat is constructed around the reference pointx1, but can be evaluated at


Derivative Difference approximation truncationat xi errorf ′(xi) (fi+1 − fi)/h O(h)f ′(xi) (fi − fi−1)/h O(h)f ′(xi) (−3fi + 4fi+1 − fi+2)/2h O(h2)f ′(xi) (+3fi − 4fi−1 + fi−2)/2h O(h2)f ′(xi) (fi+1 − fi−1)/2h O(h2)f ′(xi) (fi−2 − 8fi−1 + 8fi+1 − fi+2)/12h O(h4)f ′′(xi) (fi+1 − 2fi + fi−1)/h2 O(h2)f ′′(xi) (fi+2 − 2fi+1 + fi)/h2 O(h)f ′′(xi) (−fi−3 + 4fi−2 − 5fi−1 + 2fi)/h2 O(h2)f ′′(xi) (−fi+3 + 4fi+2 − 5fi+1 + 2fi)/h2 O(h2)

Table 5.4: Summary of difference approximations for derivatives

any other point by choosing appropriateα values. For example, considerthe first derivative at x = x2 or α = 1. Two term truncation of equation(5.16) yields,

f ′(x2) = 1h

[∆+ 12∆2]f(x2)+O(h2)

or

f ′(x2) = 12h[f3 − f1]+O(h2)

which is a 3-point, second order accurate, central difference approxima-tion for the first derivative at x2.

Going through a similar exercise as above with the Newton backwarddifference formula (5.13), truncating the series at various levels and us-ing diffrent reference points, one can easily develop a whole class ofapproximations for first order derivatives. Some of the useful ones aresummarized in Table 5.4.

5.6.2 Approximations for second order derivatives

The second derivative of the polynomial approximation is obtained bytaking the derivative of equation (5.16) one more time - viz.

f ′′(x) ≈ ddα

[dPm−1

dαdαdx

]dαdx

= 1h2

[∆2+α+ (α− 1)+ (α− 1)+ (α− 2)+α+ (α− 2)

6∆3

+· · ·] f (x1)+O(hm−2) (5.18)


Evaluating at α = 0 ( i.e., x = x1), we obtain,

f ′′(x1) = 1h2

[∆2 −∆3 + 1112∆4 −−5

6∆5 + 137

180∆6 · · ·

]f(x1) (5.19)

This equation can also be obtained directly using equation (5.9) as,

(hD)2 = (lnE)2 = (ln (1+∆))2Expanding the logarithmic term we obtain,

(hD)2 =[∆− ∆2

2+ ∆3

3− ∆4

4+ ∆5

5· · ·

]2

=[∆2 −∆3 + 11

12∆4 − 5

6∆5 + 137

180∆6 − 7

10∆7 + 363

560∆8 · · ·

]

Operating both sides on f(x1) ( i.e., using x1 as the reference point), weget,

D2f(x1) = f ′′(x1) = 1h2

[∆2 −∆3 + 1112∆4 − · · ·

]f(x1)

Truncating after one term,

f ′′(x1) = 1h2

[∆2f(x1)]= 1h2 (f1 − 2f2 + f3)+O(h)

Truncating after two terms,

f ′′(x1) = 1h2

[∆2f(x1)−∆3f(x1)]

= 1h2 (2f1 − 5f2 + 4f3 − f4)+O(h2)

Evaluating equation(5.18) at x2 (or α = 1), we get,

f ′′(x2) = 1h2

[∆2 − 0 · δ

3

6

]f(x1)+O(h2)

Note that the third order term turns out to be zero and hence this for-mula turns out to be more accurate. This is a 3-point, second order ac-curate central difference approximation for the second derivative givenas,

f ′′(x2) = 1h2

[∆2f(x1)]= 1h2 (f1 − 2f2 + f3)+O(h2)

5.7. NUMERICAL INTEGRATION 126

5.6.3 Taylor series approach

One can derive finite difference approximations from Taylor series ex-pansion also. Consider the following expansions around xi.

f(xi + h) = f(xi)+ hf ′(xi)+ h2

2f ′′(xi)+ h

3

3!f ′′′(xi)+ · · ·

f(xi − h) = f(xi)− hf ′(xi)+ h2

2f ′′(xi)− h

3

3!f ′′′(xi)+ · · ·

Subtracting the second from the first equation, and extracting f ′(xi), weget,

f ′(xi) = f(xi + h)− f(xi − h)2h− h

2

6f ′′(xi)+ · · ·

or,

f ′(xi) = fi+1 − fi−1

2h+O(h2)

which is a central difference formula for the first derivative that we de-rived in the last section §5.6. Adding the two Taylor series expansionsabove, we get,

fi+1 + fi−1 = 2fi + h2f ′′(xi)+ h4

12f ′′′′(xi)+ · · ·

or,

f ′′(xi) = 1h2 [fi+1 + fi−1 − 2fi]+O(h2)

which is a central difference formula for the second derivative that wederived in the last section §5.6.

5.7 Numerical integration

The ability to evaluate definite integrals numerically is useful either (i)when the function is complicated and hence is not easy to integrate ana-lytically or (ii) the data is given in equally spaced, tabular from. In eithercase the starting point is to use the functional approximation methodsseen in earlier sections followed by the integration of the approximatefunction. By doing this formally, with the Newton forward polynomials,we can develop a class of integration formulas. Consider the integral,

∫ baf(x)dx (5.20)


Since the function can be represented by a polynomial of degree n as,

f(x) = Pn(x0 +αh)+O(hn+1)

with an error of order hn+1, we can use this approximation to carry outthe integration. We first divide the intervalx ∈ [a, b] inton subdivisionsas shown in the sketch below; hence there will be

i=0 i=n

x=-a

x=b

1 2 n-1i

f(x)

fif2f1

(n+ 1) data points labelled as x0, x1, · · ·xn and we have

h = (b − a)/n, x = x0 +αh dx = hdαAs an illustration let us take a first degree polynomial between x0 andx1. We have∫ x1

x0

f(x)dx ≈∫ 1

0P1(x0 +αh)hdα+

∫ 1

0O(h2)hdα

or, ∫ x1

x0

f(x)dx ≈∫ 1

0[1+α∆] f0 h dα+O(h3)

which upon, completing the integration becomes,∫ x1

x0

f(x)dx ≈ h2[f0 + f1]+ O(h3)︸︷︷︸

local error

(5.21)

This formula is the well known trapezoidal rule for numerical integra-tion. The geometrical interpretation is that it represents the shaded areaunder the curve. Note that while numerical differentiation, as developedin equation (5.16), lowers the order of the truncation error by one due tothe term dα/dx = 1/h numerical integration increases the order of the


truncation error by one due to the term dx = hdα. In the above formulathe truncation error is of order O(h3). It is called the local truncationerror since it is the error in integrating over one interval x ∈ (x0, x1).To obtain the complete integral over the interval x ∈ [a, b] we applyequation (5.21) repeatedly over each of the subdivisions as,

∫ baf(x)dx =

n∑i=1

∫ xixi−1

f(x)dx =n∑i=1

h2[fi−1 + fi]+

n∑i=1

O(h3)

︸︷︷︸global error

Recalling that n = (b − a)/h the global or accumulated error becomesof order O(h2). Thus the trapezoidal rule has a local truncation errorof order O(h3) and a global truncation error of order O(h2) and theequation is,

∫ baf(x)dx = h

2

n∑i=1

[fi−1 + fi]+ O(h2)︸︷︷︸global error

(5.22)

By taking an quadratic functional approximation and integrating overthe range of x ∈ [x0, x2] we obtain the Simpson’s rule.

∫ x2

x0

f(x) dx ≈∫ 2

0P2(x0 +αh) h dα+

∫ 2

0O(h3)hdα

or,

∫ x2

x0

f(x) dx ≈∫ 2

0

[1+α∆+ α(α− 1)

2∆2]f0 h dα+O(h4)

which upon, completing the integration becomes,∫ x2

x0

f(x) dx ≈ h3[f0 + 4f1 + f2]+ O(h4)︸︷︷︸

local error

(5.23)

Note that the next neglected term in the polynomial P2(x0 + αh) thatcorresponds to order O(h3) term viz.

∫ 2

0

α(α− 1)(α− 2))3!

∆3f0 h dα

turns out to be exactly zero, thus making the local truncation error inthe Simpson’s rule to be actually of order O(h5) Repeated application of


the Simpson’s rule results in,

∫ baf(x)dx = h

3[f0 + 4f1 + 2f2 + 4f3 + 2f4 + · · · + fn]+ O(h4)︸︷︷︸

global error

(5.24)Note that in applying Simpson’s rule repeatedly over the interval x ∈[a, b], we must have an even number of intervals (n even) or equivalentlyan odd number of points.

5.7.1 Romberg Extrapolation

An idea similar to that used in section §2.8 to accelerate convergenceis the notion of extrapolation to improve accuracy of numerical integra-tion. The basic idea is to estimate the truncation error by evaluating theintegral on two different grid sizes, h1 and h2. Let us apply this idea onthe trapezoidal rule which has a global truncation error of O(h2). Letthe exact integral be represented as,

I = I(h1)+ E(h1)

where I(h1) is the approximate estimate of the integral using grid sizeof h1 and E(h1) is the error. Similarly we have,

I = I(h2)+ E(h2)

But, for trapezoidal rule we have E(hi)∝ h2i . Hence

E(h1)E(h2)

= h21

h22

We can combine these equations as,

I = I(h1)+ E(h1) = I(h2)+ E(h2)

or,

I(h1)+ E(h2)h2

1

h22= I(h2)+ E(h2)

which can be solved to obtain E(h2) as,

E(h2) = I(h1)− I(h2)[1− (h1/h2)2

]


Hence a better estimate for the integral is,

I = I(h2)+ 1[(h1/h2)2 − 1

][I(h2)− I(h1)]

If h2 = h1/2 then we have,

I = I(h2)+ 1[22 − 1

][I(h2)− I(h1)]

Since we have estimated and eliminated the truncation error of orderO(h2) term, the above equation will have an error of order O(h4) whichis the next leading term in the Taylor series expansion. By repeatedapplication of the above approach to estimate and eliminate successivelyhigher order terms, we can arrive at the following general formula forRomberg extrapolation.

Ij,k =4k−1Ij+1,k−1 − Ij,k−1

4k−1 − 1(5.25)

Example of Romberg extrapolation

Consider the following integral,

∫ 0.8

0(0.2+ 25x − 200x2 + 675x3 − 900x4 + 400x5) dx = 1.64053334

(5.26)which has the exact value as shown. A sketch of the function f(x) andthe Romberg extrapolation results are shown in figure 5.10. It is clearfrom this example that by combining three rather poor estimates of theintegral on grids of h = 0.8,0.4 and 0.2, a result accurate to eight sig-nificant digits has been obtained! For example, I2,2 is obtained by usingj = 2 and k = 2 which results in,

I2,2 = 4× I3,1 − I2,14− 1

= 4× 1.4848− 1.06883

= 1.6234667

Similarly, I1,3 is obtained by using j = 1 and k = 3 which results in,

I1,3 = 42 × I2,2 − I1,242 − 1

= 42 × 1.6234667− 1.367466715

= 1.64053334


0 0.2 0.4 0.6 0.80

0.5

1

1.5

2

2.5

3

3.5

( ) ( ) ( )( ) ( ) ( )

..hj

...hj

....hj

....hjkkk

hOhOhO

========

===

642

80061104

7664936184841203

4333504617664326188601402

4333504617664763182710801321

f(x)

x

Figure 5.10: Illustration of Romberg extrapolation

5.8. ORTHOGONAL FUNCTIONS 132

function f=int_ex(x)%defines a 5th degree polynomial

m=length(x);for i=1:mf(i) = 0.2 + 25*x(i) - 200*x(i)ˆ2 + ...

675*x(i)ˆ3 - 900*x(i)ˆ4 + 400*x(i)ˆ5;end

Figure 5.11: MATLAB implementation of quadrature evaluation

MATLAB example

MATLAB provides a m-file to evaluate definite integrals using adaptive,recursive Simpson’s quadrature. You must of course, define the func-tion through a m-file which should accept a vector of input argumentsand return the corresponding function values. A m-file that implementsequation (5.26) is shown in figure 5.11. After creating such a file workthrough the following example during an interactive session.

»quad(’int ex’,0,0.8,1e-5) % evaluate integral over (0,0.8)»quad(’int ex’,0,0.8,1e-5,1) % Display results graphically»quad(’int ex’,0,0.8,1e-8) % Note the warning messges!

5.7.2 Gaussian quadratures

5.7.3 Multiple integrals

5.8 Orthogonal functions

5.9 Piecewise continuous functions - splines

Education has produced a vast population able toread but unable to distinguish what is worth read-ing.

— G.M. TREVELYAN

Chapter 6

Ordinary differential equations- Initial value problems

In this chapter we develop algorithms for solving systems of linear andnonlinear ordinary differential equations of the initial value type. Suchmodels arise in describing lumped parameter, dynamic models. Entirebooks (Lapidus & Seinfeld, 1971; Lambert, 1973) are devoted to the de-velopment of algorithms for such problems. We will develop only ele-mentary concepts of single and multistep methods, implicit and explicitmethods, and introduce concepts of numerical stability and stiffness.

General purpose routines such as LSODE that implement several ofadvanced features such as automatic step size control, error control etc.are available from NETLIB.

6.1 Model equations and initial conditions

Ordinary differential equations of the initial type are represented typi-cally as a system of first order equations of the form,

dydt= f (y, t) (6.1)

y(t = t0) = y0 (6.2)

wherey(t) is a vector containing elementsy = y1(t),y2(t), · · · , yn(t)and our objective is to construct an approximate representation of thefunction y(t) over some interval of interest t ∈ [t0, tf ] that satisfy the

133

6.1. MODEL EQUATIONS AND INITIAL CONDITIONS 134

initial conditions, given by another vector y0 = y1,0, y2,0, · · · , yn,0.If the functions f (y, t) depend on t explicitly, then the equations arecalled non-autonomous; otherwise they are called an autonomous sys-tem of equations.

6.1.1 Higher order differential equations

A higher order differential equation (say of ordern) can be converted intoan equivalent system of (n) first order equations. Consider the equation,

andnθdtn

+ an−1dn−1θdtn−1 + · · ·a1

dθdt+ a0θ = b (6.3)

subject to a set of n initial conditions at t0 of the form,

dn−1θdtn−1

∣∣∣∣∣t0= cn−1

dn−2θdtn−2

∣∣∣∣∣t0= cn−2

... = ... (6.4)dθdt

∣∣∣∣t0= c1

θ|t0 = c0

Since all of these conditions are given at t0, this remains an initial valueproblem. Equation (6.3) can be recast into a system of n first order equa-tions of the form (6.1) as follows. Let us define θ and all of its (n − 1)successive higher derivatives as

y1(t) = θ(t), y2(t) = dθdt , y3(t) = d2θdt2 , · · · yn(t) = d

n−1θdtn−1

Then we have,

dy1

dt= y2, y1(t0) = c0

dy2

dt= y3, y2(t0) = c1

... = ... (6.5)dyn−1

dt= yn, yn−1(t0) = cn−2

dyndt

= 1an[b − a0y1 − a1y2 − · · · − an−1yn], , yn(t0) = cn−1

6.2. TAYLOR SERIES EXPANSION 135

where the last equation has been obtained from the n-th order equation(6.3). Also shown in equations (6.5), are the transformed initial condi-tions from equation (6.4) in terms of the new variable set y.

Note that the coefficients a0, a1, · · ·an,b in equation (6.3) can ingeneral be nonlinear functions of θ and its derivatives. This nonlinearitywill reflect in equations (6.5),since the coefficients a0, a1, · · ·an,b willbe functions of the transformed variables y1, y2 · · ·yn.

6.2 Taylor series expansion

Consider the differential equation,

dydt= f(y), y(t0) = y0 (6.6)

Our task is to construct a sequence yn|n = 0,1, · · · that representsan approximate solution to y(t) at a discrete set of points tn|n =0,1, · · ·. We can achieve this by constructing a Taylor series expansionfor y(t) around tn with a step size of h as,

y(tn + h) = y(tn)+y′(tn)h+y′′(tn)h2

2+ · · ·

Truncating after the linear term and recognizing that y′(tn) = f(yn),we have the Euler scheme for generating the solution sequence,

yn+1 = yn + hf(yn)+ O(h2)︸︷︷︸local error

n = 0,1, · · · (6.7)

which is single-step, explicit scheme with a local truncation error of orderO(h2). It is called a single-step method because it requires only the valueatyn to predict the value at next stepyn+1. It is explicit because the righthand side terms [yn+hf(yn)] can be computed explicitly using knownvalue of yn.

6.2.1 Alternate derivation using interpolation polynomials

Rewriting the differential equation (6.6) as,

∫ yn+1

yndy =

∫ tn+1

tnf (y)dt (6.8)


and using Newton forward and backward interpolating polynomials toapproximate the function f(y)we can recover, not only the Euler scheme,but develop a mechanism for obtaining a whole class of implicit and mul-tistep methods. First let us use the m-th degree Newton forward poly-nomial from equation (5.11), viz.

f(y) ≈ Pm(tn +αh) =[

1+α∆+ α(α− 1)2!

∆2 + · · ·α(α− 1)(α− 2) · · · (α−m+ 1)

(m)!∆m]fn +O(hm+1)

where tn has been used as the reference point, fn means f(yn) and his the step size. Since t = tn +αh we have dt = hdα. Using a one termexpansion in equation (6.8), ( i.e., m = 0) results in,

yn+1 −yn =∫ tn+1

tnP0(tn +αh)dt +

∫ tn+1

tnO(h)dt︸︷︷︸

local truncation error

=∫ 1

0P0(tn +αh) hdα+

∫ 1

0O(h)hdα

=∫ 1

0fn hdα+O(h2)

= fn h+O(h2)

which is the same equation as (6.7). This approach, however, lends itselfnaturally to further development of higher order methods. For examplea two-term expansion ( i.e., m = 1) results in,

yn+1 −yn =∫ tn+1

tnP1(tn +αh)dt +

∫ tn+1

tnO(h2)dt︸︷︷︸


=∫ 1

0P1(tn +αh) hdα+

∫ 1

0O(h2)hdα

=∫ 1

0[fn +α∆fn] hdα+O(h3)

= h[α fn +∆fnα2

2

]1

0+O(h3)

= h[fn + (fn+1 − fn)12

]+O(h3).


Hence we have the final form of the modified Euler scheme as,

yn+1 = yn + h2 [fn + fn+1]+ O(h3)︸︷︷︸local error

n = 0,1, · · · (6.9)

Both the Euler method given in equation (6.7) and the modified Eulerscheme given by equation (6.9) are single-step methods since only ynis required to predict yn+1. The modified Euler method is an implicitscheme since we need to compute fn+1 which depends on yn+1. Notethat implicit schemes requires the solution of a nonlinear algebraic equa-tion at every time step. Thus to calculate yn+1 from equation (6.9) weneed to use an iterative method that involves providing an initial guessfor yn+1 and using equation (6.9) as a fixed point iteration scheme untilyn+1 converges to desired accuracy. At a first glance, this might appearto be a disadvantage of the implicit schemes. However, implicit schemeshave the ability to anticipate sharp changes in the solution between ynand yn+1 and hence are suitable (in fact required) for solving the socalled stiff differential equations.

This initial guess could be provided by the Euler method ( viz. equa-tion (6.7)). When an explicit scheme is combined with an implicit schemein this manner, we have the so called predictor-corrector scheme. TheEuler and modified Euler predictor-corrector pair is,

yPn+1 = yn + hf(yn) and yCn+1 = yn +h2

[f(yn)+ f(yPn+1)

](6.10)

where the superscript P represents the predicted value from an explicitscheme and C represents the corrected value from an implicit scheme.

It should be clear that extending the Newton forward polynomial toa three-term expansion will not be fruitful, since that would involve notonly fn+1, but also fn+2. We can, however, use Newton backward poly-nomials to develop higher order methods as will be done in section §6.3.But, let us explore first the reason for and the circumstances under whichimplicit schemes are useful.

6.2.2 Stability limits

Let us consider a model, linear equation,

dydt= λ y, y(t = 0) = 1


which has the analytical solution,

y(t) = eλt

For λ < 0 the exact solution decreases monotonically to zero as t → ∞.Let us examine the sequence yn|n = 0,1, · · · generated by the explicit,Euler scheme and the implicit, modified Euler scheme. Note that in thismodel problem the function f(y) = λy . The Euler equation is,

yn+1 = yn + hf(yn) = yn + hλyn = [1+ hλ]ynThus the sequence is,

y1 = [1+ hλ]y0

y2 = [1+ hλ]y1 = [1+ hλ]2y0

y3 = [1+ hλ]y2 = [1+ hλ]3y0

... = ...

leading to the general solution,

yn = [1+ hλ]ny0

When the step size h is chosen to be too large (more specifically |hλ| > 2in this case), the sequence will diverge, while the exact solution remainsbounded. This phenomenon is called numerical instability caused by thediscretization. Explicit methods in general have such a stability boundon the step size h.

Let us examine the behavior of an implicit scheme - viz. the modifiedEuler scheme.

yn+1 = yn + h2 [fn + fn+1] = yn + h2[λyn + λyn+1

]Note that yn+1 appears on both sides. Solving for yn+1 we get,

yn+1 =[

1+ hλ/21− hλ/2

]yn

Thus the sequence is,

y1 =[

1+ hλ/21− hλ/2

]y0

y2 =[

1+ hλ/21− hλ/2

]y1 =

[1+ hλ/21− hλ/2

]2

y0

y3 =[

1+ hλ/21− hλ/2

]y2 =

[1+ hλ/21− hλ/2

]3

y0

... = ...


x

m

k

c

x

t

slow time scale solution: e-ct/k

fast time scale solution: e-kt/m

Figure 6.1: Spring and dash pot model

leading to the general solution,

yn =[

1+ hλ/21− hλ/2

]ny0

It is clear that for λ < 0, the ratio[

1+hλ/21−hλ/2

]< 1 for any choice of step

size h. Thus the implicit scheme is absolutely stable. Hence, for explicitschemes, the choice of h is governed by both stability and truncation er-ror considerations while for implicit schemes only truncation error con-siderations dictate the choice of step size, h.

6.2.3 Stiff differential equations

The physical interpretation for λ in the above model problem is that itrepresents the characteristic time scale of the problem. For a secondorder equation (or equivalently a system of two first-order equations),there will be two such time scales λ1 and λ2. If the time scales are widelyseparated in magnitude then we have a stiff system of differential equa-tions. Consider the spring and dash pot model shown in figure 6.1. Thedisplacement x is modelled by the force balance equation,

md2xdt2 + k

dxdt+ cx = 0

where c is the spring constant, k is the damping factor, m is the massand x is the displacement. Let us assume that it is subject to the initialconditions x(t = 0) = 0 and x′(t = 0) = constant. We can write thecharacteristic equation as,

mkλ2 + λ+ c

k= 0


and hence the two roots are given by,

λ = −1± √1− 4mc/k2

(2m/k)

In the limit of m → 0 we can approximate these as,

λ1 = − km and λ2 = −ck

where L’Hopitals rule is used to obtain the second root. Clearly asm → 0,we have λ1 >> λ2 and this limit corresponds to the stiff behavior of thesolution. In general the stiffness ratio is defined as the ratio of the largestto the smallest eigenvalues. In this example the stiffness ratio is (k2/mc)and it becomes large asm is made small. The solution satisfying the firstinitial condition is,

x(t) = A1[e−kt/m︸︷︷︸fast

− ect/k︸︷︷︸slow

]

where the fast and slow response terms are as shown. The sketch infigure 6.1 also shows the fast and slow response solutions. Note that ifm = 0, the order of the differential equation drops by one and λ1 is theonly time scale for the problem. This kind of phenomena also occursin a number of chemical reaction systems, where some of the reactionscan occur on a rapid time scale while others take place on a longer timescale. The ozone decomposition model discussed in section §1.4.2 isanother example of stiff differential equations.

It should now be clear that stiffness phenomena corresponds to largeeigenvalues and fast response regions where the solution changes rapidly.In a system of n first order equations there will be n characteristic rootsor eigenvalues. If λmax is the largest eigenvalue, then explicit schemeswill typically have a numerical stability limit of the form |hλmax| <constant. Hence explicit schemes require that extremely small stepsize h be used in regions where the system responds very rapidly; oth-erwise the integration sequence will diverge. Implicit schemes that areabsolutely stable have no such restrictions. The integration sequenceusing implicit schemes will remain bounded. The choice of step size isdetermined only by the desired accuracy of the solution. Stability analy-sis for a variety of explicit and implicit methods are discussed in greaterdetail by Lapidus and Seinfeld (1971).

6.3. MULTISTEP METHODS 141

6.3 Multistep methods

6.3.1 Explicit schemes

Consider approximating the function f(y) in equation (6.8) by the fol-lowing m-th degree Newton backward polynomial from equation (5.13),

f(y) ≈ Pm(tn +αh) =[

1+α∇+ α(α+ 1)2!

∇2 + · · ·α(α+ 1)(α+ 2) · · · (α+m− 1)

m!∇m

]fn +O(hm+1)

Here, tn has been used as the reference point. Since this polynomial in-volves only points at earlier times such as fn, fn−1, fn−2 · · ·, we can de-velop a class of explicit schemes of high orders. These are called Adams-Bashforth schemes. Consider a three-term expansion ( i.e.,m = 2). Equa-tion (6.8) becomes,

yn+1 −yn =∫ tn+1

tnP2(tn +αh)dt +

∫ tn+1



=∫ 1

0P2(tn +αh) hdα+

∫ 1

0O(h3)hdα

=∫ 1

0

[fn +α∇fn + α(α+ 1)

2!∇2fn

]hdα+O(h4)

= h[α fn + α

2

2∇fn + 1

2!

α3

3+ α

2

2

∇2fn

]1

0+O(h4)

= h[fn + 1

2(fn − fn−1)+ 5

12(fn − 2fn−1 + fn−2)

]+O(h4).

which can be rearranged into the form,

yn+1 = yn + h12[23 fn − 16 fn−1 + 5 fn−2]+ O(h4)︸︷︷︸

local error

n = 2,3,4, · · ·

(6.11)The following points should be observed on the above equation.

• This is a multistep scheme since it requires (yn,yn−1, yn−2) to pre-dict yn+1.

• Hence it is not a self-starter! Typically, in a well posed initial valueproblem, we know only y0. Hence y1 and y2 must be generated


from other single-step methods before we can switch to the abovemultistep scheme.

• It is an explicit method as yn+1 does not appear on the right handside.

• As a consequence it cannot anticipate sharp changes in y(t) - i.e.,not suitable for stiff differential equations.

• Makes good use of previous calculations to give low truncation er-ror.

• requires only one function evaluation per step.

The next higher order scheme can be developed from a four-term ex-pansion ( i.e.,m = 3) . This is called 5-th order Adams-Bashforth scheme.viz.

yn+1 −yn =∫ tn+1

tnP3(tn +αh)dt +

∫ tn+1



=∫ 1

0P3(tn +αh) hdα+

∫ 1

0O(h4)hdα

=∫ 1

0

[fn +α∇fn + α(α+ 1)

2!∇2fn + α(α+ 1)(α+ 2)

3!∇3fn

]hdα+O(h5)

= h[fn + 1

2∇fn + 5

12∇2fn + 3

8∇3fn

]+O(h5)

which can be rearranged into the form,

yn+1 = yn + h24[55 fn − 59 fn−1 + 37 fn−2 − 9fn−3]+ O(h5)︸︷︷︸

local error

n = 3,4, · · ·

(6.12)

6.3.2 Implicit schemes

In order to construct implicit schemes we need to construct backwardpolynomial approximations with tn+1 as the reference point. viz.

f(y) ≈ Pm(tn+1 +αh) =[

1+α∇+ α(α+ 1)2!

∇2 + · · ·α(α+ 1)(α+ 2) · · · (α+m− 1)

m!∇m

]fn+1 +O(hm+1)


In this manner fn+1 is introduced on the right hand side. This classof implicit schemes are called Adams-Moulton schemes. We are stillintegrating one step from tn to tn+1. Since t = tn+1 +αh and the limitsof integration in α become (−1,0). A four-term expansion results in,

yn+1 −yn =∫ tn+1

tnP3(tn+1 +αh)dt +

∫ tn+1



=∫ 0

−1P3(tn +αh) hdα+

∫ 0

−1O(h4)hdα

=∫ 0

−1

[fn+1 +α∇fn+1 + α(α+ 1)

2!∇2fn+1 + α(α+ 1)(α+ 2)

3!∇3fn+1

]hdα+O(h5

= h[fn+1 − 1

2∇fn+1 − 1

12∇2fn+1 − 1

24∇3fn+1

]+O(h5)

which can be expanded and rearranged into the form,

yn+1 = yn + h24[9 fn+1 + 19 fn − 5 fn−1 + fn−2]+ O(h5)︸︷︷︸

local error

n = 2,4, · · ·

(6.13)The pair of explicit-implicit schemes given by (6.12,6.13) respectively canbe used as a predictor-corrector pair.

6.3.3 Automatic stepsize control

Some of the start up and step size control issues are illustrated in figure6.2 for the 5-th order Adams schemes developed in the last section. Notethat only y0 is given and hence (y1, y2, y3) must be generated usingsome other single step methods with a step size of h before the 5-thorder Adams scheme given by equations (6.12,6.13) can be used. In doingso, it is important to realize that any error introduced in these three stepsare likely to be propagated during the transient phase of the simulation.Hence if lower order schemes are used to generate (y1, y2, y3), thensmaller step sizes must be used. The difference between the predictedand corrected values could be used as a measure of the truncation error.If this error is below an acceptable tolerance, then we can choose todouble the next step size. But this can begin only after y6 has beencomputed, because we need four previous values at equal intervals of(2h) - i.e., (y0, y2, y4, y6). If at any time during the intergration process,the difference between the predicted and corrected values is above the


y0

giv

en

y1 y2 y3 y4

generate usingother methods

begin usingAdams scheme

possible to double step size

if you need to halve the step sizegenerate y4.5 and y5.5

y5 y6

Figure 6.2: Stepsize control strategies for multistep methods

tolerance, then we must halve the step size and repeat the calculationfor that step. In so doing, we need to generate intermediate values atintervals of (h/2). For example if the result for y7 does not meet thetolerance, then we repeat the calculation from y6 with a step size ofh/2. We need to generate intermediate values at y4.5 and y5.5. This canbe done using the Newton backward interpolation polynomials; but thetruncation errors in the interpolating polynomials should be of the sameorder and the Adams scheme. Specifically the interpolation rules are:

yn− 12= 1

128[35yn + 140yn−1 − 70yn−2 + 28yn−3 − 5yn−4]

yn− 32= 1

64[−yn + 24yn−1 + 54yn−2 − 16yn−3 + 3yn−4]

Example

MATLAB has several built-in functions for solving initial value problems.The functions named ADAMS and GEAR, use multistep methods. All ofthe ODE solvers in MATLAB are part of the SIMULINK toolbox. Hencethe m-files that define the problem must have a special structure. Inthis section we illustrate how to use these functions to solve the ozonedecomposition model. Recall that the equations are,

dy1

dt= f1(y1, y2) = −y1 −y1y2 + εκy2

dy2

dt= f2(y1, y2) = (y1 −y1y2 − εκy2)/ε

The initial compositions are y(t = 0) = [1.0,0.0]. The parameters areε = 1/98 and κ = 3.0. The equations are clearly nonlinear. The m-file

6.4. RUNGE-KUTTA METHODS 145

function [ydot,y0]=ozone(t,y,u,flag)k=3.0;epsilon=1/98;

if abs(flag) == 1ydot(1) = -y(1) - y(2)*y(1) + k*epsilon*y(2);ydot(2) = (y(1)-y(1)*y(2)-epsilon*k*y(2))/epsilon;

elseif flag == 0ydot=[2,0,0,0,0,0]; %first element=number of equationsy0=[1 0]; %initial conditions

elseydot=[];end

Figure 6.3: MATLAB implementation of ozone decomposition model

named ozone.m is shown in figure 6.3. This function should be writtenin such a way that it should return the derivatives y′(t) in the variableydot when iflag==1 and it should return the number of equations andthe initial condition when iflag==0 as shown in the figure 6.3.

To use the GEAR and ADAMS functions and integrate the ozone modelto a final time of 3.0 do the following during an interactive MATLAB ses-sion.

»tf=3.0; % Define final time»type ozone % test that ozone.m exists»[t,y]=gear(’ozone’,tf) % Integrate using GEAR»[t,y]=adams(’ozone’,tf) % Integrate using ADAMS

The independent variable t and the solution y at the same time valuesare returned in the corresponding varaibles. These results are showngraphically in figure 6.4. Observe that y2(t) increases rapidly from aninitial condition of zero and hence the system is very stiff during theearly times.

6.4 Runge-Kutta Methods

While multi-step methods achieve high order accuracy by making effi-cient use of previously generated results, Runge-Kutta methods achieve


time

y1 o

r y2

0 1 2 30

0.25

0.50

0.75

1.00

y1y2

Figure 6.4: Results of ozone decomposition model shows a stiff systembehavior

the same goal in a single step, but at the expense of requiring manyfunction evaluations per step. Being single-step schemes, they are self-starters. They are also classified as explicit, semi-implicit and implicitschemes. Implicit schemes require solution of a set on non-linear alge-braic equations at every time step, but they are suitable for stiff differ-ential equations.

6.4.1 Explicit schemes

Explicit schemes have the general form,

yn+1 = yn +v∑i=1

wi ki (6.14)

ki = h f

tn + ci, yn + i−1∑

j=1

aij kj

, c1 = 0, i = 1,2, · · ·v

In these equations, ci,wi, aij are all parameters. The development ofa specific scheme entails determining the best possibile values for theseconstants by matching the expansion of this formula with a Taylor seriesexpansion. Often these parameter values are given in tabular form as,


0c2 a21

c3 a31 a32

c4 a41 a42 a43

w1 w2 w3 w4

or

c Aw

For explicit methods, A is a lower triangular matrix.

6.4.2 Euler formula revisited

Let us consider v = 1 in equation (6.14) and write the equations explicitlyas,

yn+1 = yn +w1 k1 (6.15)

k1 = h f(tn,yn)

oryn+1 = yn +w1 h f(tn,yn)

The procedure to determine w1 is to match the above equaiton with theTaylor series expansion,

yn+1 = yn + hy′n +h2

2y′′n +

h3

3!y′′′n + · · · (6.16)

The first term on the right hand side is the same in both equations.Recongnizing that y′ = f the second term will also match if we makew1 = 1 and this results in recovering the Euler scheme developed inequation (6.7). We cannot match with any higher order terms and hencethe local truncation error is of order O(h2).

6.4.3 A two-stage (v = 2) Runge-Kutta scheme

Let us consider v = 2 in equation (6.14) and write the equations explicitlyas,

yn+1 = yn +w1 k1 +w2 k2 (6.17)

k1 = h f(tn,yn)k2 = h f(tn + c2h,yn + a21k1)


This scheme has four unknown paramters w1,w2, c2, a21 which mustbe determined by matching the Taylor series expansion of equation (6.17)with the equation (6.16). Expanding equation (6.17) we get,

yn+1 = yn +w1 h f(tn,yn)+w2 h f(tn + c2h,yn + a21k1)

or,

yn+1 = yn +w1 h fn +w2 h[f + ∂f

∂t(c2h)+ ∂f∂y a21(hf)

]n+ · · ·

(6.18)Substituting for y′ and its higher derivatives in terms of f in equation(6.16) and expanding, we get,

yn+1 = yn + hfn + h2

2dfdt+O(h3)

or

yn+1 = yn + hfn + h2

2

[∂f∂t+ ∂f∂y

∂y∂t

]+O(h3) (6.19)

Now comparing the fn terms between equations (6.18) and (6.19) werequire that

w1 +w2 = 1

for the two equations to match. Next comparing ∂f∂t terms, we require,

w2c2 = 1/2

Finally comparing ∂f∂y

∂y∂t we require,

w2a21 = 1/2

Thus, we have matched all terms of order O(h2) leaving a truncation er-ror or orderO(h3). In that process we have developed 3 constraint equa-tions on the four unknonws w1,w2, c2, a21 appearing in the 2-stageRUnge-Kutta scheme (6.17). Any choice of values for w1,w2, c2, a21that satisfies the above three constraints will result in a 3-rd order, 2-stage Runge-Kutta scheme. Since there are four variables and only threeequations, we have one extra degree of freedom. Hence the solution isnot unique. Two sets of results are:

w1 = 2/3, w2 = 1/3, c2 = 3/2, a21 = 3/2

andw1 = 1/2, w2 = 1/2, c2 = 1, a21 = 1


The later is the equivalent of the predictor-corrector pair using Euler andmodified Euler schemes developed in equations (6.10). In summary thisscheme is a 2-stage RK method since it requires two function evaluationsper step. It is explicit and has a local truncation error of O(h3).

Using the first set of parameter values in equation (6.17), we have,

yn+1 = yn + 23k1 + 2

3k2

k1 = h f(tn,yn)

k2 = h f(tn + 32h,yn + 3

2k1)

or in tabular form,

03/2 3/2

2/3 2/3

A fourth order Runge-Kutta scheme

Higher order Runge-Kutta methods can be developed by carrying outthe matching process with the Taylor series expansion to higher orderterms. An explicit fourth-order form that matches with the Taylor seriesto h4 terms (and hence has a truncation error of O(h5) is,

01/2 1/21/2 0 1/2

1 0 0 11/6 2/6 2/6 1/6

Which in expanded form,

yn+1 = yn + 16[k1 + 2k2 + 2k3 + k4] (6.20)

k1 = h f(tn,yn)

k2 = h f(tn + h2 , yn +k1

2)

k3 = h f(tn + h2 , yn +k2

2)

k4 = h f(tn + h,yn + k3)

As with all explicit schemes, it is good for non-stiff differential equations.


Embedded forms

The embedded forms of Runge-Kutta algorithms provide a pair of schemesthat use a common set of function evaluations to predict two estimates ofthe solution at yn+1. Typically a lower order scheme is embedded withina higher order scheme. The motivation for developing such schemes isto have a convenient estimate of the local truncation error at every timestep, which can then be used to develop a step size control stragegy. Apopular scheme, called RK45, is given below.

k1 0k2

14

14

k338

332

932

k41213

19322197 −7200

219772962197

k5 1 439216 −8 3680

513−845410

25216 0 1408

256521974104 −1

5 ← y(4)n+1

k612 − 8

27 2 −35442565

18594104 −11

4016135 0 6656

128252856156430 − 9

50255 ← y(5)n+1

MATLAB has an implimentation of this scheme both as a built-in func-tion called rk45.m and a m-file called ode45.m.

6.4.4 Semi-implicit & implicit schemes

In the general form given in equation (6.14), we fixed c1 = 0 and A to belower triangular. These constraints on the paramters ensure that eachof the ki could be computed explicitly without the need for iterativesolution. Let us now relax these constraints and write the general formas,

yn+1 = yn +v∑i=1

wi ki (6.21)

ki = h f

tn + ci, yn + v∑

j=1

aij kj

, i = 1,2, · · ·v

Thus, for a two-stage process (v = 2), we will have,

yn+1 = yn +w1 k1 +w2 k2 (6.22)


k1 = h f(tn + c1h,yn + a11k1 + a12k2)k2 = h f(tn + c2h,yn + a21k1 + a22k2)

or in compact tabular form,

c1 a11 a12

c2 a21 a22

w1 w2

Note that in order to compute k1 and k2 in the above form, we needto solve two sets of nonlinear algebraic equations simulataneously. Onthe positive side of such schemes, fully implicit nature of the algorithmresults in numerically stable schemes making them suitable for stiff dif-ferential equations. Also, a two-stage scheme (v = 2) has eight param-eters, and hence we can match these equations with the Taylor seriesexpansion to higher order terms. Hence more accurate formulas can beconstructed. An example of a fully implicit, 2-stage, 4-th order accuratescheme is the Gauss form given by,

(3−√3)/6 1/4 (3− 2√

3)/12(3+√3)/6 (3− 2

√3)/12 1/4

1/2 1/2

For an extensive catalogue of such schemes see Lapidus and Seinfeld(1971).

6.4.5 Semi-Implicit forms of Rosenbrock

While the fully-implicit forms of the last section §6.4.4, have desirablestability properties, they are computationally demanding since a systemof non-linear algebraic equations must be solved iteratively at every timestep. In an effort to reduce the computational demand while retainingthe stability characteristics, Rosenbrock (1963) proposed a special formof the algorithm. These are suitable for autonomous system of equationsof the form,

dydt= f (y) y(t = t0) = y0

A 2-stage, 3-rd order scheme is shown below.

yn+1 = yn +w1 k1 +w2 k2 (6.23)

k1 = h [I − ha1J(yn)]−1f (yn)k2 = h [I − ha2J(yn + c1k1)]−1f (yn + b1k1)

6.5. DYNAMICAL SYSTEMS THEORY 152

The parameters are,

a1 = 1+√

6/6, a2 = 1−√

6/6

w1 = −0.41315432, w2 = 1.41315432

b1 = c1 = −6−√6+√

58+ 20√

66+ 2

√6

Here J = ∂f∂y is the Jacobian, which must be evaluated at every time

step. Note that the main advantage in using equation (6.23) is that k1,k2

could be computed without the need for iteration, although it requirestwo matrix inverse computations per step.

6.5 Dynamical systems theory

He who can, does.He who cannot teaches.

— GEORGE BERNARD SHAW

Chapter 7

Ordinary differential equations- Boundary value problems

In the present chapter we develop algorithms for solving systems of (lin-ear or nonlinear) ordinary differential equations of the boundary valuetype. Such equations arise in describing distributed, steady state mod-els in one spatial dimension. The differential equations are transformedinto systems of (linear and nonlinear) algebraic equations through a dis-cretization process. In doing so, we use the tools and concepts devel-oped in Chapter 5. In particular we will develop (i) finite difference meth-ods using the difference approximations given in Table 5.4, (ii) shootingmethods based on methods for initial value problems seen in chapter 6and (iii) the method of weighted residuals using notions of functionalapproximation developed in Chapter 5.

We will conclude this chapter with an illustration of a powerful soft-ware package called COLSYS that solves a system of multi-point bound-ary value problems using the collocation method, cubic splines and adap-tive mesh refinement. It is available from NETLIB.

7.1 Model equations and boundary conditions

Consider the model for heat transfer through a fin developed in section§1.5.1. We will consider three specific variations on this model equation(1.22) and the associated boundary conditions. First let us scale theproblem by introducing the following dimensionless temperature and

153

7.1. MODEL EQUATIONS AND BOUNDARY CONDITIONS 154

distance variables,

θ = T − T∞T0 − T∞ ξ = x

L

Using these definitions, equation (1.22) can be rewritten as,

ddξ

[kAdθdξ

]− hPL2θ = 0 (7.1)

θ(ξ = 0) = 1 θ(ξ = 1) = 0

where the objective is to find the continuous function θ(ξ) over thedomain of interest, viz. ξ ∈ [0,1] for a prescribed set of parametersk,A,h, P, L. All of these parameters can be constants, or some of themmight be dependent on the position ξ ( e.g., A(ξ)) or on the unknowntemperature itself ( e.g., k(θ)). We examine each case next.

For constant area, A and thermal conductivity, k, equation (7.1) re-sults in a second order linear differential equation with constant coeffi-cients for which an analytical solution is possible. But we focus only ondeveloping methodologies for obtaining a numerical solutions.

d2θdξ2 −

hPL2

kAθ = 0 (7.2)

θ(ξ = 0) = 1, θ(ξ = 1) = 0

When the values of the dimensionless temperature, θ on the boundaryare specified, the boundary condition is called the Dirichlet boundaryconditions.

In a variation of the above model, if we let the area be a variableA(ξ) ( i.e., tapered fin), but keep the thermal conductivity, k, constant,we obtain a variable coefficient, linear boundary value problem, still ofsecond order.

A(ξ)d2θdξ2 +

dA(ξ)dξ

dθdξ− hPL

2

kθ = 0 (7.3)

θ(ξ = 0) = 1,dθdξ

∣∣∣∣∣ξ=1

= 0

7.1. MODEL EQUATIONS AND BOUNDARY CONDITIONS 155

In the above model, we have also introduced a variation on the bound-ary condition at ξ = 1. Such boundary conditions, where the derivativesare specified, are called Newman boundary conditions. The temperaturevalue at ξ = 1 is an unknown and must be found as part of the solutionprocedure.

In yet another variation, consider the case where the thermal conduc-tivity is a function of temperature, viz. k(θ) = α + βθ2 where α and βare experimentally determined constants. Let the area, A be a constant.The equation is nonlinear and must be solved numerically.

k(θ)d2θdξ2 +

dk(θ)dθ

[dθdξ

]2

− hPL2

Aθ = 0 (7.4)

θ(ξ = 0) = 1,[kdθdξ+ hθ

]ξ=1

= 0

At this opportunity a third variation on the boundary condition, calledthe mixed or Robin boundary condition has been used. Once again, thetemperature value at x = L is an unknown and must be found as part ofthe solution procedure.

All of these problems can be represented symbolically as,

Dθ = f on ΩBθ = g on ∂Ω

where D and B are differential operators, Ω is the domain of interestand ∂Ω represents its boundary. Our task is to obtain an approximatesolution, θ to the above problem. The approximation consists in con-structing a discrete version of the differential equations which results ina system algebraic equations. If the differential equations are linear (asin equations (7.2,7.3), then the resulting discrete, algebraic equations willalso be linear of the typeAθ = b and methods of Chapter 3 can be usedto obtain the final approximate solution. If the differential equationsare nonlinear (as in equation (7.4)) then the resulting discrete, algebraicequations will also be nonlinear of the type F(θ) = 0 and methods ofChapter 4 can be used to obtain the final approximate solution.

In the following sections we develop various schemes for construct-ing approximate solutions.

7.2. FINITE DIFFERENCE METHOD 156

i=0 i=n+1

ξ=0

1 2 ni

θ(ξ)

θi

θ2

θ1

ξ=1∆ξ

Figure 7.1: One dimensional finite difference grid of equally spaced datapoints

7.2 Finite difference method

7.2.1 Linear problem with constant coefficients

Let us consider equation (7.2) which is a linear problem subject to Dirich-let boundary conditions. In solving equation (7.2) by the finite differencemethod, we divide the domain Ω = ξ ∈ [0,1] into (n+1) equally spacedsubdivisions as shown in figure 7.1. The distance between the two gridpoints is denoted by ∆ξ. The grid spacing ∆ξ and the value of the inde-pendent variable ξ at the nodal point i are given by,

∆ξ = 1− 0n+ 1

ξi = i ∆ξ i = 0,1, · · · (n+ 1)

Next, instead of attempting to find a solution θ(ξ) as a continuous func-tion of ξ that satisfies the differential equation (7.2) exactly at every ξ, wecontent ourselves with finding an approximate solution at the selectednodal points shown in the figure - i.e., θ(ξi) = θi|i = 1,2, · · ·n, wheren is the number of interior grid points. Note that for the Dirichlet typeof boundary conditions θ0 and θn+1 are known. Hence, it remains todetermine only n unknowns at the interior points. We obtain n equa-tions by evaluating the differential equation (7.2) at the interior nodalpoints. In doing this we replace all the derivatives by the correspondingfinite difference approximation from Table 5.4. Clearly, we have severalchoices; but it is important to match the truncation error in every term tobe of the same order. We illustrate this process using central differenceapproximations.


Using the central difference approximation for the second derivativein equation (7.2), we obtain,[

θi−1 − 2θi + θi+1

(∆ξ)2 +O((∆ξ)2)]− hPL

2

kAθi = 0 i = 1,2, · · ·n

The term in the square brackets is the central difference approximationfor the second derivative and O((∆ξ)2) is included merely to remind usof the order of the truncation error. We now have a system of n linearalgebraic equations. Let us write these out explicitly for n = 4.

θ0 − 2θ1 + θ2 −[hPL2

kA(∆ξ)2

]θ1 = 0

θ1 − 2θ2 + θ3 −[hPL2

kA(∆ξ)2

]θ2 = 0

θ2 − 2θ3 + θ4 −[hPL2

kA(∆ξ)2

]θ3 = 0

θ3 − 2θ4 + θ5 −[hPL2

kA(∆ξ)2

]θ4 = 0

θ0 in the first equation and θ5 in the last equation are known from theDirichlet boundary conditions. The above equations can be expressed inmatrix notation as, T θ = b,−(2+α) 1 0 0

1 −(2+α) 1 00 1 −(2+α) 10 0 1 −(2+α)

θ1

θ2

θ3

θ4

=

−θ0

00−θ5

(7.5)

where α = hPL2

kA (∆ξ)2. Note that the boundary values θ0 and θ5 appearas forcing terms on the right hand side. Equation (7.5) is the discreteversion of equation (7.2). Once the structure is apparent, we can increasen to reduce (∆ξ) and hence reduce the truncation error. In the limit of∆ξ → 0 the solution to equations (7.5) will approach that of equation(7.2). The matrix size will increase with decreasing ∆ξ and increasing n.The matrix T is tridiagonal and hence the Thomas algorithm developedin section §3.4.4 can be used to get the solution.

7.2.2 Linear problem with variable coefficients

Next we consider equation (7.3) which is also a linear problem subjectto Dirichlet condition at ξ = 0 and Neuman boundary condition at ξ =


1. It also has variable coefficients, A(ξ) and A′(ξ) - i.e., A is a knownfunction of ξ. The discretization procedure remains the same as seen inthe previous section §7.2.1, with the exception that θn+1 is now includedin the unknown set

θ = θ1, θ2, · · · , θn+1.But we have the Neuman boundary condition as an extra condition thatwill provide the additional equation. Using the central difference ap-proximations for both the second and first derivatives in equation (7.3),we obtain,

A(ξi)[θi−1 − 2θi + θi+1

(∆ξ)2 +O((∆ξ)2)]

+

A′(ξi)[θi+1 − θi−1

2(∆ξ) +O((∆ξ)2)]− hPL

2

kθi = 0 i = 1,2, · · ·n+ 1

Once again the truncation error term has been retained at this stage, onlyto emphasize that it should preferably be of the same order for everyderivative that has been replaced by a difference approximation; other-wise the effective error is of the same order as the term with the lowestorder truncation error. Multiplying throughout by (∆ξ)2 and collectinglike terms together, we get,

[A(ξi)−A′(ξi)(∆ξ)2

]θi−1 +

[−2A(ξi)− hPL

2(∆ξ)2k

]θi+[

A(ξi)+A′(ξi)(∆ξ)2

]θi+1 = 0, i = 1,2, · · ·n+ 1

Letting

ai =[A(ξi)−A′(ξi)(∆ξ)2

]

di =[−2A(ξi)− hPL

2(∆ξ)2k

]i = 1, · · ·n+ 1

ci =[A(ξi)+A′(ξi)(∆ξ)2

]

we can rewrite the equation as,

aiθi−1 + diθi + ciθi+1 = 0, i = 1,2, · · ·n+ 1

Observe that the coefficients ai, di, ci in the above equations are known.However, unlike in equation (7.5), they vary with the grid point locationi. Also, for i = 1, θ0 on the left boundary is known through the Dirichlet


boundary condition. The last equation for i = n+1 needs special atten-tion since it contains the unknown θn+2 which lies outside the domainof interest. So far we have not used the Neuman boundary condition atthe right boundary ξn+1 = 1. Using the central difference approximationfor the first derivative to discretize the Neuman boundary condition weget,

dθdξ

∣∣∣∣∣ξn+1=1

≈[θn+2 − θn

2∆ξ]= 0

which implies θn+2 = θn. This can be used to eliminate θn+2 from thelast equation, which becomes,

(ai + ci)θn + diθn+1 = 0, for i = n+ 1

Thus we obtain a tridiagonal system of linear equation of the formT θ =b. For n = 4, as an example, we get the follwoing five equations.

d1 c1 0 0 0a2 d2 c2 0 00 a3 d3 c3 00 0 a4 d4 c4

0 0 0 a5 + c5 d5

θ1

θ2

θ3

θ4

θ5

=

−a1θ0

0000

(7.6)

Equation (7.6) is the discrete version of equation (7.3). Once the structureis apparent, we can increase n to reduce (∆ξ) and hence reduce thetruncation error. In the limit of ∆ξ → 0 the solution to equations (7.6)will approach that of equation (7.3). The matrix size will increase withdecreasing ∆ξ and increasing n. The matrix T is tridiagonal and hencethe Thomas algorithm developed in section §3.4.4 can be used to get thesolution.

7.2.3 Nonlinear problem

Conceptually there is no difference in discretizing a linear or a non-lineardifferential equation. The process of constructing a grid and replacingthe differential equations with the difference equations at each grid pointis the same. The main difference lies in the choice of solution techniqueavailable for solving the nonlinear algebraic equations. Let us considerthe nonlinear model represented by equation (7.4). In this case we havea Robin condition at ξ = 1 and hence θn+1 is unknown. Thus the un-knowns on the discrete grid consist of

θ = θ1, θ2, · · · , θn+1.


and we need n+ 1 equations.Discretizing equation (7.4) at a typical grid point i, we obtain the

following (n+ 1) nonlinear algebraic equations.

fi(θ) := k(θi)[θi−1 − 2θi + θi+1

(∆ξ)2]+

k′(θi)[θi+1 − θi−1

2(∆ξ)]2

− hPL2

Aθi = 0 i = 1,2, · · ·n+ 1

In this set of equations i = 1 and i = (n+1) require special considerationto incorporate the boundary conditions. Thus, making use of the leftboundary condition, θ0 = 1, f1(θ) becomes,

f1(θ1, θ2) := k(θ1)[

1− 2θ1 + θ2

(∆ξ)2]+ k′(θ1)

[θ2 − 12(∆ξ)

]2

− hPL2

Aθ1 = 0

At the right boundary, we discretize the Robin boundary condition as,

k(θn+1)[θn+2 − θn

2(∆ξ)]+ hθn+1 = 0


θn+2 =[θn −

(2(∆ξ)hk(θn+1)

)θn+1

]=[θn − βθn+1

]This can be used in the equation fn+1(θ) to eliminate θn+2.

fn+1(θn, θn+1) := k(θn+1)[θn − 2θn+1 + [θn − βθn+1]

(∆ξ)2]+

k′(θn+1)[[θn − βθn+1]− θn

2(∆ξ)]2

− hPL2

Aθn+1 = 0

The above equations f1 = 0, f2 = 0, · · ·fn+1 = 0 can be representedsymbolically as a system of (n + 1) nonlinear equations of the form,F(θ) = 0. These can be solved most effectively by the Newton methodfor the unknowns θ = θ1, θ2, · · · , θn+1.

θp+1 = θp − J−1f (θ

p) p = 0,1, · · ·

The Jacobian matrix, J = ∂F∂θ

has the following tridiagonal structure.

J =

∂f1

∂θ1

∂f1

∂θ20 0

∂f2

∂θ1

∂f2

∂θ2

∂f2

∂θ30

0 ∂f3

∂θ2

∂f3

∂θ3

∂f3

∂θ4

0 0 ∂f4

∂θ3

∂f4

∂θ4

7.3. QUASILINEARIZATION OF NONLINEAR EQUATIONS 161

7.3 Quasilinearization of nonlinear equations

Recall the Newton method applied to a system of nonlinear algebraicequations:

f (x) = 0 (7.7)

A solution is sought by linearizing the equations using

x = xa + δ (7.8)

where xa is an approximate solution and δ is the correction vector.Hence the original equation becomes,

f (xa + δ) = f (xa)+ ∂f∂x

∣∣∣∣xaδ +O(δ2) (7.9)

Neglecting second and higher order terms, one gets the linearized equa-tion,

∂f∂x

∣∣∣∣xaδ = −f (xa) (7.10)

Similar concepts can be applied to a (system of) differential equationsas well. Consider for example the equation given below

ddξ

[k(θ)ξ

dθdξ

]= 0 (7.11)

with the boundary conditions

θ(ξ = 1) = 1 inner wall; θ(ξ = ξ0) = 0 outer wall

The thermal conductivity is given by k(θ) = α + γθ2 The equation canbe written as,

f(θ) := k(θ)ξd2θdξ2 + k(θ)

dθdξ+ ξ dk

dθ

[dθdξ

]2

= 0 (7.12)

The solution to this nonlinear problem can be obtained by successiveiteration from an initial guess θa(ξ). Let the actual solution be writtenas

θ(ξ) = θa(ξ)+ δ(ξ) (7.13)

where δ(ξ) is a small correction to be obtained by solving a linearizedform of equation (7.12). The linearized equation is derived by substitut-ing equation (7.13) in equation (7.12), expanding the resulting expressionand neglecting quadratic and higher order terms in δ(ξ) in the expan-sion.

7.3. QUASILINEARIZATION OF NONLINEAR EQUATIONS 162

k(θa+δ)ξd2(θa + δ)dξ2 +k(θa+δ)d(θa + δ)dξ

+ξ dkdθ

∣∣∣∣θa+δ

[d(θa + δ)

dξ

]2

= 0

(7.14)Note that each nonlinear term must now be expanded in Taylor series,e.g.

k(θa + δ) ≈ k(θa)+ dkdθ

∣∣∣∣θaδ+ · · ·

dkdθ

∣∣∣∣θa+δ

≈ dkdθ

∣∣∣∣θa+ d2kdθ2

∣∣∣∣∣θaδ+ · · · (7.15)

etc. Hence the expanded form of the equation is,

ξ[k(θa)

d2θadξ2 + k(θa)

d2δdξ2 +

dkdθ

∣∣∣∣θaδd2θadξ2 +

dkdθ

∣∣∣∣θaδd2δdξ2 + · · ·

]

k(θa)dθadξ

+ k(θa)dδdξ +dkdθ

∣∣∣∣θaδdθadξ

+ dkdθ

∣∣∣∣θaδdδdξ+ · · ·

ξ

dkdθ

∣∣∣∣θa+ d2kdθ2

∣∣∣∣∣θaδ+ · · ·

dθa

dξ

2

+dδdξ

2

+ 2

dθadξ

dδdξ

= 0(7.16)

Now, terms of order δ2 and higher are neglected since δ is small. Theterms that are evaluated at the current guess θa are moved to the righthand side. Hence we get,

k(θa)ξd2δdξ2 + ξ

dkdθ

∣∣∣∣θaδd2θadξ2 + k(θa)

dδdξ+ dkdθ

∣∣∣∣θaδdθadξ

+

ξd2kdθ2

∣∣∣∣∣θaδdθadξ

2

+ 2ξdkdθ

∣∣∣∣θa

dθadξ

dδdξ

= −k(θa)ξd2θa

dξ2 + k(θa)dθadξ

+ ξ dkdθ

∣∣∣∣θa

dθadξ

2 (7.17)

Note that the right hand side of equation (7.17) is the same as equation(7.12) evaluated at θa and when it is zero the problem is solved! Theleft hand side of equation (7.17) is linear in δ subject to homogeneousboundary condition of δ(1) = 0 and δ(ξ0) = 0. Hence the solution (orcorrection) δ(ξ) will be zero when the iteration is converged. Now onecan discretize equation (7.17) and solve for the correction iteratively.Equation (7.17) can also be written in operator form as,

Dθδ = −f(θ)

7.4. CONTROL VOLUME METHOD 163

i=0 i=n+1

ξ=0

1 2 ni

θ(ξ)

θi

θ2

θ1

ξ=1∆ξ

Figure 7.2: One dimensional control volume discretization

where the linear operator Dθ, called the Frechet derivative is given by,

k(θa)ξd2·dξ2 +

[k(θa)+ 2ξ

dkdθ

∣∣∣∣θa

dθadξ

]d·dξ+

ξ dkdθ

∣∣∣∣θa

d2θadξ2 +

dkdθ

∣∣∣∣θa

dθadξ

+ ξ d2kdθ2

∣∣∣∣∣θa

dθadξ

2 ·

7.4 Control volume method

7.5 Shooting method

7.6 Collocation methods

7.7 Method of weighted residuals

Where observation is concerned, chance favoursonly the prepared mind.

— LOUIS PASTEUR

Appendix A

Networked computingenvironment within thechemical and materialsengineering department

A.1 Introduction

In these notes you will be introduced to the networked computing envi-ronment within the department of chemical and materials engineering atthe University of Alberta. You can choose to work with a variety of work-stations such as the IBM RS/6000 which use the AIX operating system(a version of Unix), SUN workstations using SUNOS operating system, orthe personal computers (Prospec 486) which use OS/2 and/or Win95 withDOS emulation. Hence forth the computers will be identified merely bythe operating system used to run them viz. AIX, SunOS, OS/2 or WIN95machines. Computers that are available for student use are distributedas follows in the Chemical and Mineral Engineering (CME) building.

Computers for teaching and research

AIX machines Nine IBM RS 6000 - model 220 in room CME 244 +One IBM RS 6000 - model 550 as server

SUNOS machines Three SUN workstations in room CME 244.

A.1

A.1. INTRODUCTION A.2

OS/2 machines Twenty Prospec 486 computers in room CME 244

WIN95 machines a variety of PC’s in room CME 474.

Computers for graduate research

AIX machines A variety of IBM RS 6000 machines in room CME475. These machines belong to individual researchers;but all staff and graduate students can have accessprovided the priority based on ownership is hon-ored.

You are expected to be familiar with the basic concepts of operating sys-tems, file systems, editors and structured programming and debuggingprinciples. I will focus on the concepts that are unique to the networkedenvironment and discuss briefly the software tools that are available forvarious tasks. This document is intended merely as an introduction toget you started and to identify the hardware and software resources thatare available on the network. Once you are aware of the art of the possi-ble, you are on your own to make the proper choice of machine, softwaretools and implement solutions strategies for solving a variety of prob-lems. Advanced users should consult the original documentation oneach software. Online help is available for most of the software.

In a heterogeneous, networked environment, such as the one we have,the range of services such as file space, printer, plotter capabilities, avail-ability of specialized software etc. are distributed on a variety of ma-chines, often matching the software needs with hardware capabilities.For example, graphics packages and word processors on PC’s are ade-quate for most document processing needs. Powerful editors are avail-able on all of the machines; you may already have your own favoriteeditor on one of these machines. Use it! Data processing, visualization,and simulation of flow and chemical processes are numerically inten-sive tasks and are best carried out on powerful workstations. MAPLE(a symbolic computation package) and APSEN (a process simulator) aremade available only on one IBM RS 6000 machine for licensing reasons.Hence it is essential to develop the skills to navigate through the network,transfer data between computers and select the machine best suited fora task.

Workstations using AIX or SUNOS operating system support multi-tasking and multiple users. Multi-tasking implies that the computer canhandle several tasks (or processes) at the same time using time sharingprinciples. In addition, AIX (and all other flavors of UNIX) can handle


multiple users by keeping track of user directories and accounting in-formation. Thus, you need to get a userid and some disk space to storeyour own files in your home directory.

On the other hand OS/2 supports multi-tasking, but not multipleusers. This implies that whenever you use an OS/2 machine, you can edita program in one window and compose a letter in another ( i.e., multi-tasking feature), but you are responsible for keeping your personal filesseparately in a floppy. If you leave them in a hard drive, there is noguarantee that they will be available to you the next day! But you do notneed a userid to use OS/2 machines.

All of the machines (AIX, SunOS and OS/2) provide a friendly Graph-ical User Interface (GUI) to interact with the operating system. The basicelement of a GUI is a window and a desktop. AIX uses Motif window man-ager (a standard that is getting wide acceptance) and OS/2 uses the Pre-sentation Manager (or PM). The figure A.1 illustrates the basic anatomyof a window and how to manipulate its size and location. A typical AIXwindow on IBM RS 6000 and an OS/2 window on Prospec 486, providea shell through which you can enter commands to the operating system.Well designed applications avoid command line based interaction withthe computer; instead you have to merely (double) click on the icon tostart an application. These icons, of course, have to be made availableto the user and the concept of a desktop is used in both OS/2 and AIXto organize and present file systems and groups of applications to theuser. Since AIX is a multiuser environment, each user can customize thedesktop. This information is saved for each user and during subsequentlogins you are presented with your own customized desktop. UnderOS/2 there is only one standard desktop administered by the computersupport staff - do not mess with it as it will make it difficult for subse-quent users to access programs in a standard manner.

While illustrating a dialogue with the computer, following conventionsare adopted throughout this document.

courier font will indicate the prompt from the computer

bold font will indicate a command that you should enterexactly as shown

italics font will indicate a parameter like a file name, directoryname etc. that should be substituted with the real name.

Unlike DOS, UNIX is case sensitive and most of the UNIX commands arein lower case.


Click the left mouse button here to minimize the window (make into icon).

[Click on the icon to recover the window]

Click the left mouse button here for a pull down

menu.

Click the left mouse button on a side

bar and drag to resize the window

Click the left mouse button on a

corner and drag to resize the window

Click anywhere on the window to make it the current window.

This window will then have the focus of the keyboard. i.e. anything that you enter from

keyboard goes into this window.

Click the left mouse button on the

title bar and drag to move the window

Click the left mouse button here to maximize the

window size.[Click again to recover the

original size]

X-windows background screen.

When the cursor is in this region, clicking the left or right mouse button offers

choice of additional menus

Figure A.1: Anatomy of a Motif window

A.2. USERID A.5

A.2 Userid

All students get a userid on the General Purpose Unix servers (or GPU)maintained by the Computing and Networking Services (CNS). If you alsopurchase the Netsurf 97 CD-ROM from CNS (for about $10), you get apowerful set of software for connecting your home computer to the IN-TERNET through the University. For more information about computingand network services see their Web site at http:\\www.ualberta.ca\CNS.Once you get the userid on the machines gpu.srv.ualberta.ca you can en-able access to the machines maintained by the department of chemicaland materials engineering by (a) signing on to gpu.srv.ualberta.ca and(b) running the following script:

[userid@gpu]> /afs/ualberta.ca/labs/cmel244/bin/register-244

You need to do this only once at the begining of the year. If youencounter any problems see one of the DACS center staff (Mr. Bob Bartonor Mr. Jack Gibeau).

A.3 Overview of the network

Figure A.2 provides a conceptual frame work for introducing the net-work structure. A variety of workstations and personal computers areconnected by ethernet. This local area network (LAN) is a subnet of thelarger university wide ethernet network as well as the world wide INTER-NET network. The underlying communication protocol is called TCP/IP(Transmission Control Protocol/Internet Protocol) and has been widelyaccepted as the standard.

In addition to providing some basic connectivity between machines,a network enables sharing of hardware and software resources as wellas sharing of information and communication on a world wide basis.We will focus on the departmental subnet to illustrate various conceptswhich can then be easily extended to national and international levelnetworks. The figure A.2 illustrates the logical dependencies for thepurpose of sharing resources between various machines and not thephysical connections. As a user, we need not concern ourselves withthe network hardware connections. It is sufficient to realize that anymachine on the network can address any other machine, much like tele-phone connections. This immediately requires that each machine onthe INTERNET have a unique IP number and a host name. For exam-

A.3. OVERVIEW OF THE NETWORK A.6

ugrads.eche.ualberta.caIBM RS/6000 550

AIX 3.2.3NIS server, NFS server,

X-server/clientsMATLAB, MAPLE, ASPEN, FORTRAN, emacs, gnuplot

ravana.eche.ualberta.caIBM RS/6000 320

AIX 3.2.3NIS server, NFS server,

X-server/clientsFORTRAN, C++, emacs,

gnuplot, xmgr, khoros, tex, FLOW3D

cmel31.ucs.ualberta.cato

cmel39.ucs.ualberta.caIBM RS/6000 220

AIX 3.2.3NIS client to ugrads,

X-server/clients

MATLAB, FORTRAN, emacs, gnuplot, xmgr, khoros, tex

cmel11.ucs.ualberta.cato

cmel28.ucs.ualberta.ca

Prospec 486OS/2

X-clients

DOS/Windows emulation, Lotus 123, Wordperfect

telne

t, ftp

,

x-wind

ows telnet, ftp,

x-windows

U of A modem pool dial 492-0024DIAL>telnet machine.eche.ualberta.ca

Any machine on INTERNET .Staff offices,

rooms CME 473, 475 various labs in CME

building

HOMEU of A servers provide

NEWS, ftp, Gopher, e-mailservices

Room: CME 475

Room: CME 244

Room: CME 244

telne

t, ftp

,

x-wind

ows

sixIBM RS/6000

AIX 3.2.3NIS client to ravana,

X-server/clientsFORTRAN, C++, emacs,

gnuplot, xmgr, khoros, tex, FLOW3D

telne

t, ftp

,

x-wind

ows

Room: CME 475

Figure A.2: Logical dependencies between various machines in the net-work

A.3. OVERVIEW OF THE NETWORK A.7

hostname IP number Machine type Operatingsystem

for undergraduate teaching and course related workugrads.labs.ualberta.ca 129.128.44.50 IBM RS 6000 - 550 AIX 4.1ugrads1.labs.ualberta.ca 129.128.44.51 Sun Sparc Ultra2 SunOS 5.5.1ugrads2.labs.ualberta.ca 129.128.44.52 Sun Sparc Ultra2 SunOS 5.5.1cmel31 .ucs.ualberta.ca 129.128.44.31 IBM RS 6000 - 220 AIX 4.1· · · · · · · · · · · ·cmel39 .ucs.ualberta.ca 129.128.44.39 IBM RS 6000 - 220 AIX 4.1cmel11 .ucs.ualberta.ca 129.128.44.11 Prospec 486 OS 2· · · · · · · · · · · ·cmel28 .ucs.ualberta.ca 129.128.44.29 Prospec 486 OS 2

for graduate research and faculty useravana.eche.ualberta.ca 129.128.56.102 IBM RS 6000 - 320 AIX 4.1prancer.eche.ualberta.ca 129.128.56.23 IBM RS 6000 - 320 AIX 4.1comet.eche.ualberta.ca 129.128.56.19 IBM RS 6000 - 375 AIX 4.1dancer.eche.ualberta.ca 129.128.56.11 IBM RS 6000 - 375 AIX 4.1cupid.eche.ualberta.ca 129.128.56.82 IBM RS 6000 - 355 AIX 4.1dina.eche.ualberta.ca 129.128.56.70 IBM RS 6000 - 320 AIX 4.1jhm2.eche.ualberta.ca 129.128.56.74 IBM RS 6000 - 375 AIX 4.1chopin.eche.ualberta.ca 129.128.56.11 IBM RS 6000 - 320 AIX 4.1poincare.eche.ualberta.ca 129.128.56.15 IBM RS 6000 - ??? AIX 4.1handel.eche.ualberta.ca 129.128.56.54 IBM RS 6000 - 3AT AIX 4.1brahms.eche.ualberta.ca 129.128.56.13 IBM RS 6000 - 320 AIX 4.1

Table A.1: List of machines in the chemical and materials engineeringnetwork

ple the server within the department of chemical and materials engi-neering for undergraduate student use has the IP number 129.128.44.50and the host name ugrads.labs.ualberta.ca. The part of the IP number"129.128.56." represents the department of chemical and materials engi-neering subnet and we can connect up to 256 computers to this subnet.Similarly the last part of the host name, viz. eche.ualberta.ca, also calleddomain name represents the chemical and materials engineering do-main. Different machines on this domain will have different names like,prancer.eche.ualberta.ca, ravana.eche.ualberta.ca. Table A.1 lists com-puters that are generally available for chemical and materials engineer-ing students and staff. To determine your eligibility to have access toany of these resources see one of the DACS center staff. Typically under-graduate students can expect to have access to ugrads.eche.ualberta.caand the associated machines, while graduate students and staff will have

A.4. THE CLIENT-SERVER MODEL A.8

access to ravana.eche.ualberta.ca and its associated machines. You cansignon to these machines through a variety of means including remoteconnections from home. These procedures will be outlined later in thisdocument.

A.4 The client-server model

In a networked, distributed computing environment, the client-servermodel serves a very useful role in both providing a variety of services onthe network and in sharing the limited resources with maximum flexibil-ity. From a users point of view understanding the client-server concepthelps in (i) navigating through the network and using the resources ef-ficiently and (ii) diagnosing likely sources of problems when things donot work as anticipated.

In simple terms a server provides a service to a client authorized torequest such a service. Typically, both the server and the client are pro-grams that communicate over the network. The server is always running,listening for requests over the network. Such programs that are runningall the time are also called daemons. When the client program initiates arequest, the server program checks for authentication and provides theservice. Some of the common services are explained below.

A.4.1 File servers

Let us take an example of Network File System, often called NFS. It is a setof protocols developed by Sun Microsystems and it has been acceptedwidely as a standard. It uses TCP/IP for communication, but provides ahigher level function in terms of sharing files between various machinesin a heterogeneous environment.

In figure A.3, the computer chopin.eche.ualberta.ca has a local filesystem called /usr/local which contains a variety of very useful programslike emacs, gnuplot, TeX, xmgr, pine etc. This file system is quite largewith about 450 Megabytes of programs and data that can be used byother similar machines. So this file system is "exported" to group ofmachines within the network. Thus chopin.eche.ualberta.ca acts as theNFS server to this group of machines. The client machines then "mount"this file system and make it appear as a local file system. In this waythe user sees the same file structure on each of the machine and theadministrator has to install and maintain only one copy of the software.

In a similar fashion, the authorization to signon to a group of ma-chines ( i.e., user ids and passwords) is also maintained in a central loca-


The file system /usr/localresides physically in chopinbut has been exported to a group of machines

/usr/local

chopin.eche.ualberta.caprancer

ethernet

ravana comet

NFS clients

NFS SERVERThe file system chopin:/usr/localis mounted as /usr/local on each ofthese machines. Logically it appears as a localfile system. But any directories and files under/usr/local are actually fetched from chopin.

Some useful Unix commandsdf - Reports information about space on file systemsshowmount - Displays a list of clients that have remotely mounted file systems exportfs - Exports and unexports directories to NFS clientsmount - Makes a file system available for usedu - Summarizes disk usage

Figure A.3: Network File System server and clients

tion and served by the NIS (Network Information Service) server machine.Figure A.4 shows a such a setup.

The Andrew File System or AFS

The University of Alberta supports sharing of the same file space from avariety of machines on campus. There is a single home directory foreach userid from any UNIX machine on campus. CNS provides lim-ited amount of disk space for each user (about 10Meg). In additionthe department of chemical and materials engineering provides addi-tional space to undergraduate students under a directory called lab-disk and for graduate students under a directory called chemeng. Therelevant directories will be found as a subdirectory under the homedirectory of each user. To learn more about AFS go to the web sitehttp://www.ualberta.ca/HELP/afs/afs1.html.

A.4.2 License servers

The client-server model is also used to manage software licenses. Dueto financial constraints, unrestricted licenses are rarely purchased. Typ-


The file system /homecontains home directories of users.It has been exported to other machines.

/home

ravana.eche.ualberta.ca

prancer

ethernet

chopin

comet

NIS clientsNIS SERVER

serves information onuser ids and passwords

to other clients.When you attempt to signon to a client, it verifieswith the NIS server that your user id is a valid one.Your home directory is also made available on each client.

Some useful Unix commandspasswd - Changes a user’s password [it may take a while to propagate to other clients]rlogin - Connects the local host with a remote host. rsh - Executes the specified command at the remote host

brahms

jhm2 dina

Figure A.4: Network Information Service


ically each software is licensed to run only on a specific group of ma-chines. We also rely heavily on a large collection of freely available soft-ware. Two of the common licensing arrangements are floating networklicense and node locked license. The former allows a fixed number ofconcurrent users of a software on any machine in the network, while thelater allows any number of users on a single machine.

Typically a license server (which is a small program) runs on one ofthe machines in the network as a daemon. Let us use FLOW3D and ASPENPLUS as examples. FLOW3D is a commercial software package for usein fluid dynamics simulations and ASPEN PLUS is a very powerful steadystate process flow sheet simulator. For managing FLOW3D, the licenseserver (two programs named lmgrd and CFDSd) runs all the time on blue-jay.ucs.ualberta.ca. Any time FLOW3D is started from any of the clientswithin the network, the client contacts the server to check whether ithas the permission to start a new copy of the program. The server caneither grant the permission or deny it because that particular client isnot allowed to run that software or all of the available licenses are in useat that time. For FLOW3D we have license for 100 concurrent users.

For ASPEN PLUS, the department of chemical and materials engineer-ing has a multiuser, node locked license. This implies that any numberof users can use the program at the same time, but the software canrun on only one machine. In our case ASPEN PLUS runs on ugrads1 orugrads2.

For MATLAB the university has a site license for up to 150 concurrentusers. The most recent version of MATLAB 5.0 is available on ugrads1or ugrads2.

Table A.2 lists all of the software available on the undergraduate andgraduate research network and the licensing status of each software.

A.4.3 X servers

X-windows is a sophisticated communication/window management soft-ware developed at MIT. It uses TCP/IP for communication. It allows youto interact with the local workstation through windows, menus, dialogboxes, icons etc., in an intuitive way and minimizes the need for interac-tion through command lines. One can view this as the next logical stepin the historical progression of the way humans have interacted withcomputers. Originally it was accomplished through batch processing,followed by interactive computing using commands which is being re-placed currently by more intuitive interaction through a Graphical UserInterface (GUI) using menus and icons. One can expect this to be fol-lowed by voice level interaction, leading ultimately to interaction with


Software ugrad-net grad-net Windows CommentsASPEN PLUS ugrads1 X process simulator

floating, multiuser licenseMAPLE num.srv X symbolic computation package

floating, multiuser licenseMATLAB ugrads1 prancer X powerful linear algebra package

cmel31 - comet with numerous tool boxescmel39 floating, multiuser license

FORTRAN all AIX machines IBM HESC license agreementxde all AIX machines X powerful debugger

IBM HESC license agreementemacs all AIX machines a powerful editor

GNU public licensegnuplot all AIX machines a simple 2-D graphics package

GNU public licensetgif all AIX machines X a powerful drawing package

free to use licensepine all AIX machines a standard e-mail program

free to use licensetin all AIX machines a standard news reader

free to use licenseTeX all AIX machines a powerful typesetter

free to use license123 OS/2 Windows 3.1 spread sheetFreelance OS/2 Windows 3.1 GraphicsWordperfect OS/2 Windows 3.1 word processorHYSIM OS/2 DOS process simulator

Table A.2: List of software on the chemical and materials engineeringnetwork


machines using natural language!While Microsoft (Windows, Windows NT), Apple (MacIntosh), NeXT

(NeXTSTEP), IBM (OS2-PM) and others have their own Graphical User In-terface to interact with their operating system and application programs,X-Windows goes one step further, in making the GUI software indepen-dent of the underlying hardware and operating system. How this is ac-complished is of interest only to programmers. From a users point ofview we should understand the concepts and be able to reap the benefitsof these features. In particular, X-windows makes the following possible:

• Signon from a local machine to a remote machine. The two ma-chines can be either of the same make or completely different, e.g.,a 486 running under OS/2 can be the local machine and IBM RS 6000can be the remote machine.

• Run an application software on the remote machine e.g., MATLAB,ASPEN PLUS, FLOW3D etc. , but have the X-windows display routedto your local machine. In other words you can interact with a re-mote computer, located as far away as in another building or an-other continent, with full window capabilities! (One needs a highspeed link to interact with a computer in another continent, ofcourse)

A conceptual description of the client-server interaction and the stepsrequired to establish this interaction between a local and a remote com-puter using X-windows are illustrated in figure A.5. Note that underAIX the server program is called xinit and the client program is calledaixterm. Under SunOS, the client program is called xterm.

A.4.4 News server

The INTERNET network provides a news service. Discussions and ex-change of ideas take place through organized and moderated news groups.It is more like a conference on thousands of topics. You can choose toparticipate in topics of your choice. The topics are as wide ranging asthe ones on classical music (rec.music.classical), MATLAB (comp.soft-sys.matlab), the AIX operating system (comp.unix.aix), dynamical sys-tems theory (comp.theory.dynamic-sys) etc. The server is maintainedand administered by Computing and Network Services. The client pro-gram is called a news reader and several versions of news readers exist.On the AIX machines, the news reader can be invoked with the command

NOTE: The stringuser@machine:diris a prompt fromthe computer


localX -server (xinit)

This machine must have a monitor, keyboard and mouse.

Command sequence1. login2. xinit 3. xhost remote4. rlogin remote5. aixterm -display local:0 &

ethernet

Some useful Unix commandsxinit - starts the X-server on the local machinexhost - permit remote host to send display to local machineaixterm - start a X-terminal (or X-client) and send the display to the machine local.

NOTE: Substitute the actual name of the machine for local and remote. Use fully qualified domain names if the machines are on different subnet.

remoteX client (aixterm)

Note: The client machine need not even have a monitor attached to it. It merely runs the applications and sends the display to the local machine.

Some useful Unix files to exploreIf these files are found in your home directory, they are used in controling rlogin between machines and in customizing your X-window environment..rhosts - controls rlogin between machines.xinitrc - customizes the X-server..Xdefaults - customizes the Window appearance and functionality.mwmrc - customizes the Motif window manager

What they dosignon to the local machine with user id and passwordstart the X-serverpermit remote machine to send display to localsignon to the remote machine from localstart X-client on remote and redirect the display to local.

Figure A.5: X-Windows client-server model


user@machine:dir> tin

This program maintains a local database of the news groups that youhave subscribed. It fetches the latest articles from the server and allowsyou to read and, optionally respond to articles. If you post a response,it goes to the whole internet community. This service is a privilege andshould be used responsibly. Any abuse of the system will result in denialof access to the entire network services.

A.4.5 Gopher service

While the News service allows two-way communication and thus activeparticipation, Gopher is a worldwide one-way information retrieval ser-vice. A number of Universities and organizations participate in provid-ing this service. The University of Alberta is one of the participants. TheGopher server is maintained by Computing and Network Services on oneof their workstations. Any client on the campus ethernet (or even froma home computer with a modem) can connect to this server and browsethrough a wealth of information on such topics as

1. What is CWIS?/2. What’s New on CWIS?/3. Student Information and Services/4. Libraries on the Network/5. University Faculties and Departments/6. Administrative Policies and Procedures (MAPPS)/7. Campus Phone Directory <CSO>8. Computing Resources/9. International CWIS Systems/10. Magazines and Publications/11. University Services Directory/12. Weather Report.

The Student Information and Services includes job posting, andthe International CWIS Systems gives access to other Gopher ser-vices around the world. There are also nodes that link you to the Libraryof Congress and other university libraries. To connect to Gopher fromany computer on campus with telnet feature enter,

user@machine:dir> telnet gopher.srv.ualberta.ca

Use as your userid gopherc; no password is required. When you connect

A.5. COMMUNICATION A.16

to the campus network from home through a modem by dialing 492-0024, CWIS will be provided as a menu option and selecting it will con-nect you to the Gopher server. When you are logged on to ugrads.ucs.ualberta.ca,you can also access gopher by entering

user@machine:dir> gopher

When you are logged on to gpu.srv.ualberta.ca, you can also access go-pher or world wide web by entering

user@machine:dir> lynx

and selecting the appropriate menu.

A.4.6 World wide web service

This is a high end version of information sharing tool (like gopher) us-ing windows and hypertext links established between computers on theinternet on a global level. It also operates on a client-server model.Netscape is available on all of the AIX machines. Anyone with someinformation to share can operate a server on one of their workstationsand there are thousands of such servers on the INTERNET. To accessinformation on WWW, start the client on your local workstation by en-tering the following after you have started X-windows.

user@machine:dir> netscape

Your client should automatically know where the nearest server is lo-cated. Typically there will be home page from where you can begin yourexploration. Have fun! Watch the time you spend on this one!! It is atime sink!!!

A.5 Communication

A.5.1 Software tools

There are many communication software packages available to connectfrom one computer to another. The following are available for use onthe Chemical And Materials Engineering network.

kermit Preferred for communication from home computer to Chemi-cal And Materials Engineering network via a modem. It allows


for VT100 emulation ( i.e., full screen use of editors like emacs,news reader tin, e-mail program pine and gopher) and tek-tronix emulation ( i.e., plotting programs like gnuplot can dis-play graphs on your home computer in color!) and file trans-fer between home computer and the university computer. Youcan get a copy of MS-KERMIT Ver 3.10 for your PC from DACScenter staff. It can be distributed freely without any licensingproblems. Additional details can be found in Gianone (1991).

telnet Preferred for communication between UNIX machines or fromOS/2 to UNIX machines over ethernet. It supports VT100 em-ulation, but has no graphic or file transfer support. Only com-mand line interaction with the remote machine is possible.

ftp Preferred for file transfer over ethernet. Both AIX and OS/2provide support for ftp.

tn3270 Preferred for ethernet communication from DOS based PC’s(in room CME 473, staff offices etc. ) to workstations. It pro-vides VT100 and tektronix emulation as well as ftp support.

aixterm Preferred for remote login between AIX machines or fromOS/2 machines to AIX machines. It supports full GUI featureswith the remote machine.

A.5.2 Connection to the AIX machines from OS/2 machines

If you are using a 486 PC under OS/2 in room CME 244, the followingsteps will establish a X-windows connection to a remote IBM RS 6000running AIX.

• Make sure that the X-server is running under OS/2. This server iscalled presentation manager X or PMX server and, under normaloperating conditions, it should be active and running. To verifythat it is running, look for the PMX icon on the screen. If you donot find one, the key sequence ctrl Esc will give a list of windows.Check if PMX is listed. If you do not find it, you can start a X-serveras follows:

> Open a OS/2 Full Screen Window by double clicking on theappropriate icon.

> To start the PMX server, enter If PMX server isalready running,you will beinformed of thatfact.

[C:\] xinit


• Open a OS/2 Full Screen Window, if one is not open already.

• Telnet to one of the IBM RS 6000 machines - e.g., cmel34.ucs.ualberta.ca

[C:\] telnet cmel34.ucs.ualberta.ca

Remember that you can choose any one of the 9 AIX machines inthe network. You will see the same home directory and other soft-ware resources from each of the nine machines. Hence choose arandom number between 31 and 39, so that no one machine is overloaded all the time.

• Signon to the remote AIX machine.

login: useridPassword: password

• Start the aixterm client on the remote machine, redirecting the dis-play to the local machine by entering

user@machine:dir> open-Xpc nn

where nn is the local station number of the Prospec 486 runningOS/2. This effectively tells the AIX machine (in this example cmel34)to open a X-Window and display the window on the local worksta-tion (i.e. cmelnn). Alternatively, you can enter,

user@machine:dir> nohup aixterm -display cmelnn.ucs:0 & Observe that thecomputer promptuser@machine:dir>identifies youruserid, themachine name andthe presentdirectory.

NOTE: open-Xpc is nothing but a script that executes the abovecommand for you!

• Exit from the AIX machine and from the OS/2 Full Screen Window.i.e. exit twice!

user@machine:dir> exit[C:\] exit

• After some delay, an AIX window will open on your local 486 sta-tion! Now you are connected to the AIX machine and typically youwill be in a shell called “ksh”. You can start any application likeMATLAB, xmgr etc. by simply entering the name of the programe.g.,


user@machine:dir> matlab

If an application, such as ASPEN PLUS, uses X-windows effectively,it might open up other windows, dialogue boxes etc. and also pro-vide online help windows. To start ASPEN PLUS enter

user@machine:dir> mmg

which is the graphical version of the model manager.

• Before you leave the OS/2 station, either logout from the remotemachine with the logout command as follows

user@machine:dir> logout You can also usectrl-D key to logout

or close the aixterm window.Caution: If you leave the OS/2 station without completing this laststep, the next person using that OS/2 station will have access toyour account on the AIX machine!

A.5.3 Connection to the AIX machines from a home computer

Use of Kermit is recommended. You also need a modem. Kermit has aninitialization file which can be setup in such a way that it automaticallydials the telephone number. The available phone numbers to connectto the University of Alberta network server are : 492-4811 or 492-0024(2400 Baud) , 492-0096 (9600 Baud) or 492-3214 (high speed modem).Sample initialization files can be obtained from DACS center staff. Youstart the kermit program by entering

C:> kermit -finitfile.ini

where initfile.ini is the name of the initialization file. By default it ismskermit.ini. If no file by name mskermit.ini exists in your directory andyou do not specify explicitly the name of an initialization file as shownabove with the -f flag, then the Kermit program will start on the PC, butyou will be left with the prompt

MS-KERMIT>

At this stage you can ask for additional help on kermit by entering thekermit command help or ?. But the connection to the University networkmust be done manually with the following commands


MSKERMIT>set speed 2400 ;this sets the modem speedMSKERMIT>set port com2 ;this sets the portMSKERMIT>OUTPUT ATDT4924811 ;this dials the numberMSKERMIT>connect ;this connects you to the Univ. network

These steps can be automated by putting the above kermit commandsin the initialization file. Note that anything following a semi-colon istreated as a comment by MS-KERMIT. Whichever procedure you use, youshould get the following prompt if everything has gone well up to thisstage.

DIAL>

At this stage enter

telnet, 129.128.44.50

or

telnet, ugrads.eche.ualberta.ca

to get connected to the AIX machine or any other valid internet number(or name) for which you have a valid userid.

A.5.4 Connection to the AIX machine from a DOS machine usingtn3270

All the PCs in room CME 473 and those in staff offices that have ethernetcards are connected to INTERNET. The communication program tn3270is also available in each of the machine. On these machines you can startthe connection by entering,

[C:\] tn3270 remote.machine Substitute theactual machinename.You should be connected directly to the remote AIX machine. This com-

munication program is quite powerful. It provides full VT 100 emulationand hence allows full screen use of editors (emacs, vi), news reader (tin)etc. It also supports tektronix emulation and has facilities for capturinggraphics screen images as Postscript files on the local PC. Use the Alt-Hkey sequence to get brief online help.


A.5.5 File transfer with Kermit

A client-server concept is used once again to transfer files between theremote IBM RS 6000 and the local home computer. The MS-KERMIT onyour PC is the client. You must start a server on the IBM RS 6000, calledthe C-KERMIT. Once you are logged in to the IBM RS 6000, enter

user@machine:dir> kermit

to start the C-KERMIT program. The prompt string should change to

C-Kermit>

Enter,

C-Kermit> server This command isfor C-Kermit

to put C-KERMIT in server mode. Then escape back to the local MS-KERMIT (Typically ctrl-[C is the escape sequence. The prompt shouldbe,

MS-Kermit>

Any command you enter now is acted up on by the local PC. To fetch afile from the AIX machine (in your home directory, of course) to local PC,enter,

MS-Kermit> get filename This command isfor MS-Kermit

To send a file from your local PC to the AIX machine (in your homedirectory, of course) enter,

MS-Kermit> send filename

Once the file transfer is completed, enter Although thiscommand is forMS-Kermit, it tellsC-Kermit toterminate servermode. The nextcommand, C,re-establishesdirect connectionto AIX

MS-Kermit> fin

to signal the server that file transfer session is finished. Then enter,

MS-Kermit> C

to re-connect to the AIX machine. Note that the C-Kermit is still runningon the AIX; it has only terminated its server mode. Finally enter


FTP subcommand its functionhelp ask for help on FTP subcommandsget remotefname localfname get a file from the remote machinemget remote-pattern get multiple files matching the patternput localfname remotefname put a file on the remote machinemput local-pattern send multiple files matching the patternbin to enable file transfer in binary modeascii to enable file transfer in ascii mode (this is the default)ls list files in the current directory on the remote machinedir also list files in the current directory on the remote machinecd change the directory on the remote machinelcd change the directory on the local machinepwd display present working directory on remote machinequit to terminate the FTP session

Table A.3: List of ftp commands

C-Kermit> quit The quit commandis acted on by C-Kermitto terminate the C-KERMIT program on the AIX machine and to return

to the shell level.

A.5.6 File transfer with ftp

The file transfer program called ftp allows transfer of files between ma-chines on ethernet. This is also implemented on the client-server model.From a local machine connect to the remote machine as follows:

user@machine:dir> ftp 129.128.44.50

On the remote machine the ftp daemon called ftpd acts as the server.You will be prompted for userid and password. Once the connection isestablished you can use the commands shown in Table A.3 to transferfile.

The FTP procedure summarized above is essentially the same onmost of the machines. Be bold and try them out and observe how fastthe file transfer is compared to KERMIT through a serial line.

A.6. OPERATING SYSTEMS A.23

A.5.7 File transfer from AIX to OS/2 machines

ftp feature is also available under OS/2 machine. If you want to make acopy of your personal files on the AIX machine to a floppy disk, do thefollowing.

• Make sure that you have a writable, formatted floppy disk in driveA: of an OS/2 machine.

• Open a OS/2 Full Screen Window, if one is not open already.

• Enter,

[C:\] ftp remote.aix.machine Substitute theactual machinename.

• Signon as usual.

• Change the local directory to A: using

[C:\] lcd A:

• Transfer the file using,

[C:\] get remote.filename Substitute theactual file name.

A.6 Operating systems

A.6.1 How to signon and logout

The first time you signon to an AIX machine you will be forced to changeyour password. Subsequent times, you will be given informational mes-sage regarding your previous singon time and date and any unsuccess-ful attempts to singon to your id. Change your passwords periodically.Make sure that you always logout before you leave your workstation.

The following commands summarize the syntax for loging in and outof AIX systems and for changing passwords.

Login procedure

login: userid Substitute theactual userid andpassword.

password: password


Logout procedure

user@machine:dir> logout You can also usectrl-D key to logout

Changing password

user@machine:dir> passwd Respond to theprompts for theold and newpasswords

A.6.2 Customizing your environment - the .profile file

Once you login successfully to an AIX machine, you will always start thesession in your home directory. There should be a file in your homedirectory named .profile. Every time you login, the contents of this fileare executed once. Hence one can use this file to customize the work-ing environment on an AIX machine with the help of this file. There isalso a system wide profile file, which is used to control such things asthe default search path for finding executable programs, controlling theprompt string, identifying the terminal type, and to define a number ofenvironment variables that other application programs might need.

You can feel free to copy my version of this file in /u/kumar/.profileand adopt it to your needs. My version of this file enables the commandline editing features - i.e., all the commands that you enter during a ses-sion are stored in a buffer and you can scroll back and forth to retrieveprevious commands with Ctrl-p for previous and Ctrl-n for next. Afterretrieving a previous command you can edit it with the cursor movementkeys Ctrl-b for backward, Ctrl-f forward and Ctrl-d for deleting the cur-rent character. In effect it supports the same editing capabilities on thecommand line that the editor emacs supports for a file. More on emacslater. You can also set alias for the most frequently used commands.For example you can set

alias dir=’ls -al’so that when you enter dir you will get the directory listing. Anotheruseful alias is

alias rm=’rm -i’which prompts you for confirmation before removing (deleting) files.

A.6.3 File management

If you are familiar with the DOS directory and file structure, it should beequally easy to work with the file systems of OS/2 and AIX. While DOS file


names are restricted to 12 characters, AIX allows very long file names.But the mechanisms for creating directories and navigating up and downthe directory tree structure are essentially the same. Frequently usedcommands that relate to file management are tabulated below.

Task UNIX OS/2 DOS 5.0 or aboveseek online help man command help command help commandlist the directory ls -al dir dirlist contents of a file cat fname type fname type fnamelist one page at a time more fname more fname more < fnamecreate a file touch fnameerase a file rm fname erase fname erase fnamecopy a file cp fn1 fn2 copy fn1 fn2 copy fn1 fn2append fn1 to fn2 cat fn1 >> fn2 copy fn2+fn1rename a file mv fn1 fn2 rename fn1 fn2 rename fn1 fn2create a directory mkdir dirname mkdir dirname mkdir dirnamechange directory cd dirname cd dirname cd dirnamedelete an empty directory rmdir dirname rmdir dirname rmdir dirnamepresent working directory pwd cd prompt pgdisk usage summary du -s chkdskstatus of system ps -l pstat

A.6.4 How to get online help

Each of these commands, particularly in UNIX, can take several optionalparameters, flags etc. that further identify any specific features of thecommand that you want to enable. This command list is by no meanscomplete. They are the more frequently used commands. On UNIX addi-tional details on each of the command can be found with the command

user@machine:dir> man command

These are called manual pages or man pages for short. Try

user@machine:dir> man man

to get started! Although the man pages provide only limited help, theyare always available from any type of terminal. Much more exhaustiveonline help using hypertext is available when you are connected to anAIX machine via X-windows. To access this enter

user@machine:dir> info

and follow the instructions. Extensive help based on hypertext is also

A.7. EDITORS A.26

available on OS/2 machines.

A.7 Editors

The functions that one expects from a good editor have several commonfeatures. These can be broadly grouped into the following.

cursor movements by characters, words, lines, blocks, etc.delete text by characters, words, lines, blocks, etc.insert text by characters, words, lines, blocks, etc.move text by characters, words, lines, blocks, etc.copy text by characters, words, lines, blocks, etc.locate text by strings perhaps with regular expressionssearch & replace text1 by text2

A good editor must have a set of commands or keyboard sequencesthat invoke functions to carry out the above tasks. On AIX there areseveral editors. I prefer emacs, as I find it to be extremely powerful; italso has built in online help and hence you can learn at your own paceand grow with it. OS/2 also has a full screen visual editor. I am sure thatyou will have your own favorite editor. If you have one use it and ignorethis section.

A.7.1 Emacs - the ultimate in editors

If you want to learn the ultimate in editors (my biased view of course!)try emacs. To start this editor in AIX use the command,

user@machine:dir> emacs

The anatomy of the emacs editor screen is shown in Figure A.6. Thisis also a full screen, visual editor. It works under both X-windows andVT100 emulation.

Buffers

Emacs uses the concept of a buffer to keep a temporary copy of the filethat you are editing. You can edit any number of files at a time and each

A.7. EDITORS A.27

Buffer name emacs mode relative file position

Mini buffer where commands to emacs are entered.

%%

Indicates that the file is read only.

** in this position indicates that the file has been modified.

cursor movementsC-b

C-p

C-n

C-f

Figure A.6: Anatomy of an emacs window

A.7. EDITORS A.28

file is kept in a separate buffer. There is a one line command buffer calledminibuffer at the bottom of the screen and a text buffer at the top. Astatus line in reverse video separates the minibuffer from the text buffer.The key sequence C-x b allows you to cycle through the buffers.

Key sequences

Being the most powerful editor, emacs provides a large number of edit-ing functions. These functions are accessed either by entering the func-tion name in the command minibuffer or by directly entering a key se-quence. This process of attaching key sequences to functions is calledkey bindings. Actually when you enter a key sequence like C-f, it invokesa function called forward-char. Some of the frequently used functionsand their key bindings are listed in Table A.7.1. In these C-x means thekeeping the control key pressed, enter the character x, while M-x impliesa two-character key sequence with the Esc key as the first one followedby the character x. If you get into trouble or become confused with thiseditor at any time enter the key sequence C-g C-g to discontinue whatyou started. This means that while keeping the control key down, enterthe letter g couple of times.

Command completion

If you find the use of key sequences difficult to remember,but wouldrather remember only the function names (and that too only vaguely!),then you can enter M-x followed by the first few letters of the functionname and the space-bar key. A list of all possible functions beginningwith those few letters will be displayed in a lower window. This processis called function completion. Try

M-x sa space-barYou will see that the function name will be completed till save- andstop as an ambiguity exists at the point. Another space-bar will show allfunctions that begin with save- and you can continue to identify the oneyou want - e.g., b space-bar will identify the function as save-buffer.Recall that this function is bound to the key C-x C-s.

This concept of completion is also used in selecting a file. For exam-ple, the key sequence C-x C-f will get a new file into a buffer for editing.Once you execute the key sequence, the current directory will be dis-played in the minibuffer. Entering the first few characters of a file namefollowed by space-bar will complete the file name till the next point ofambiguity.

A.7. EDITORS A.29

task key sequence function-nameask for help C-h C-h help-for-helpask for tutorial C-h t help-with-tutorialask for information C-h i infoto end emacs session C-x C-c save-buffers-kill-emacsabort if you get into trouble C-g keyboard-quit

edit a new file in a new buffer C-x C-f find-fileinsert a new file at cursor position C-x i insert-fileswitch to a different buffer C-x b switch-to-buffersave the current buffer into the file C-x C-s save-buffersave the current buffer as a new file C-x C-w write-file

move one character backward C-b backward-charmove one character forward C-f forward-charmove to previous line C-p previous-linemove to next line C-n next-linedelete the current character C-d delete-charundo C-x u advertised-undo

move to beginning of line C-a beginning-of-linemove to end of line C-e end-of-lineopen a new line for typing C-o open-linekill from cursor to end of line C-k kill-lineyank it back (explore kill ring!) C-y yankmove to beginning of buffer C-x [ backward-pagemove to end of buffer C-x ] forward-page

page forward to next screen C-v scroll-uppage backward to previous screen M-v scroll-downredisplay current screen C-l recenter

incremental search forward C-s string isearch-forwardincremental search backward C-r isearch-backwardsearch & replace Mx % query-replaceto cancel search C-g C-g

start recording keystrokes C-x ( start-kbd-macroend recording keystrokes C-x ) end-kbd-macroreplay/execute recorded keystrokes C-x e call-last-kbd-macro

NOTE: C-h means keeping the control key down, enter the key “h”.M-v means press and release the Meta key or the Esc key, then press thekey “v”.C-x [ means keeping the control key down, enter the key “x”, then pressthe key “[”

Table A.4: List of frequently used emacs functions and their key bindings

A.8. FORTRAN COMPILERS A.30

Emacs contains many more functions than listed in Table A.7.1 andnot all functions have key bindings. You are encouraged to go throughthe online tutorial which can be invoked with the keystrokes C-h t. Onlinedocumentation is available via C-h C-h i. The key sequence C-h C-h bwill show all the key bindings within emacs in a buffer called *Help*after splitting the screen into two windows. You can switch to the lowerwindow with the key sequence C-x o which is the same as invoking thefunction other-window. Repeating the key sequence C-x o will cycle youthrough the various windows. To expand the current buffer into fullscreen use the key sequence C-x 1.

This is only a minimal introduction to emacs. This editor is not forthe uninitiated user. You need patience to master this editor, but if youpersist the rewards in terms of increased productivity are great. Withthis introduction you should be able to explore emacs deeper and deeper!If you are bored while using emacs, chat with the doctor - use M-x doctor- have fun.

A.8 Fortran compilers

Fortran compilers and X-window based debuggers are available on all ofthe AIX machines. The FORTRAN compiler on the AIX server is namedxlf. To compile your code use the command

user@machine:dir> xlf srcfile.f -O

This is adequate for any self contained FORTRAN program. The -O op-tion invokes optimization of the code. The executable code from thecompiler will automatically be stored in a file named a.out. To executeyour program use,

user@machine:dir> a.out

If you read data from unit 5 and write data to unit 6 within your FOR-TRAN program, these I/O will be redirected, by default, to your terminal.If you want to use files connected to these units, then you can use

user@machine:dir> a.out <indata >outdata

Another way to use these files is to OPEN then explicitly within yourFORTRAN program.

If your program calls any library routines like the nswc math library,then you must specify the location of the library as follows in the com-

A.9. DEBUGGERS A.31

pilation step.

user@machine:dir> xlf srcfile.f -O -lnswc -L/usr/local/lib

Here -l parameter identifies the name of the file containing the libraryand the -L parameter identifies the directory where the library file re-sides. Note that the library file will be named libnswc.a. By convention,however, the prefix lib and the postfix .a need not be specified in theparameter -l.

Use the man xlf to find about other parameters the xlf compiler canaccept.

A.9 Debuggers

A source level debugger allows one to step through the program, exe-cuting one line at a time, set break points at pre-selected lines, displayvalues of variables and change values of variables. There is an excellentX-Windows based front end called xde to the debugger (dbx) on the RISCserver that is quite easy to use. Find out more about it using man xde. Ifyou want to use this debugger, however, you should access the computerthrough X-Windows.

In order to enable the debugging features you should compile theprogram with the -g option as follows.

user@machine:dir> xlf srcfile.f -g

The a.out file so produced will contain all the symbol table informationwhich is needed by the debugger. To invoke the debugger use,

user@machine:dir> xde a.out

This will open up several windows, one of which will show the sourcelisting. Other windows provide menus and buttons that you can useto set break points, begin execution, display variables etc. There is alsoonline help explaining the various features of xde.

For the real computer hacks, a command line version of the debuggercalled dbx is available. You can use this to debug FORTRAN, C andPASCAL programs. Venturing into this is recommended only if you knowthe language C quite well. This can be accessed, however, without X-windows.

A.10. APPLICATION PROGRAMS A.32

A.10 Application programs

There are a large number of application programs on the AIX machines.They are all located under the directory "/usr/local". The procedure forstarting several useful applications is given in the next few sections.Online help or online documentation is often adequate to learn moreabout the software.

A.10.1 ASPEN PLUS

APSEN is a powerful steady state process flow sheet simulator. It isuseful for carrying out rigorous, steady state mass and energy balancecalculations. It has a fairly extensive thermodynamic data base. The X-window based front end, called the model manager allows you to definethe problem interactively. The flow sheet can be constructed graphi-cally by grabbing modules and connecting them up. Operating condi-tions can be defined interactively by filling out forms. The expert sys-tem interface ensures that all required input parameters are specifiedbefore the simulation is started. ASPEN PLUS is licensed to run only onugrads.eche.ualberta.ca.

• Establish a X-window connection to ugrads from any OS/2 machine(follow section A.5.2) or from other AIX machines (see figure A.5).

• From the command line enter,

user@machine:dir> mmg

to start the model manager. Explore more on your own!

Advanced users of ASPEN PLUS can use the simulator directly from thecommand line using

user@machine:dir> aspen

The input file containing the commands to ASPEN PLUS must be pre-pared by the user using any standard editor. One can think of the ModelManager as a front end that enables you to build the command file forASPEN PLUS in a GUI environment.

A.10.2 Xmgr

This is a powerful 2-D plotting and data analysis package. You can con-trol every facet of the graph with this program. It runs only under X-

A.10. APPLICATION PROGRAMS A.33

windows. This program is available on most of the AIX machines. Tostart the program

• Establish a X-window connection to ugrads from any OS/2 machine(follow section A.5.2) or from other AIX machines (see figure A.5).

• From the command line enter,

user@machine:dir> xmgr

to open up a main window. Explore more on your own!

A.10.3 TEX

TEX is a powerful typesetting package that is widely available on severalplatforms and it has very few licensing restrictions. This document, infact, was typeset using TEX. It is available on most of the AIX machines.The authoritative document on TEX is by Knuth (Knuth, 1984) and onLATEX is by Lamport (Lamport, 1986). The steps for compiling, previewingand printing a TEX document are outlined here. See the above referenceson how to create a TEX document. To compile a TEX file use,

user@machine:dir> tex file.tex

To compile a LATEX file use,

user@machine:dir> latex file.tex

Either of these steps should produce a file named file.dvi, which is adevice independent file. The contents of this "dvi" file can then be usedto either preview the document on the screen or send it to a printer. Topreview use,

user@machine:dir> xdvi.sh file.dvi

Note that on the AIX machine, you must be using X-windows to use thepreviewer. A program called dvips takes the "dvi" file and sends it toa postscript printer. Since printer configurations vary, see one of thesupport staff to find out the exact procedure for printing a document.

A.10.4 pine - the mail reader

Pine is the standard mail reader on our AIX network. To start it simplyenter

A.11. DISTRIBUTED QUEUEING SYSTEM A.34

user@machine:dir> pine

This program will work under VT100 emulation and offer full screensupport. Online help can be accessed with the question mark, ?. Notethat e-mail addresses are formed as [email protected] userid is your signon name or id on the AIX machine, machine is thename of the machine. The rest of the e-mail address, eche.ualberta.carefers to the chemical and materials engineering subnet domain name.If you have accounts on several machines it is recommended that youselect and consistently use one machine as your primary e-mail system.

A.10.5 tin - the news reader

Tin is the standard news reader on our AIX network. To start it simplyenter

user@machine:dir> tin

This program will work under VT100 emulation. Online help can beaccessed with the "h" key.

A.11 Distributed Queueing System

The Distributed Queueing System (DQS) allows CPU intensive tasks tobe executed on a machine that is relatively free within the graduate net-work, while at the same time allowing faster keyboard/terminal responseto interactive users. Any task that takes longer that 5 min of CPU timeshould be submitted through the DQS. Otherwise the jobs will be termi-nated automatically. The commands that you should be familiar withare

• qsub - for submitting batch jobs.

• qstat - to monitor the status of jobs in the DQS

• qmon - under X-windows to monitor status of jobs.

• qdel - to delete your own (and only your) jobs.

For the experts, online documentation via man pages is available on eachof the above commands. The following examples illustrate how to con-struct a script file that you would submit using qsub.

A.11. DISTRIBUTED QUEUEING SYSTEM A.35

A.11.1 Example of a CPU intensive MATLAB job

Construct a script file named, say test.bat, containing the followinglines.

#!/bin/csh# make the current directory the CWD#$ -cwd# lets put STDOUT/STDERR in the file "gaga"#$ -eo gaga# i’d like to know when she fires#$ -mu [email protected]#$ -mb# and when she finishes#$ -mematlab >out « ’eof’ #the shell starts matlab; output goes to "out"secant(’ass3a’,[0,1],eps,1) %matlab acts on this line and runs secantfzero(’ass3a’,0.5,eps,1) %matlab acts on this line and runs fzeroquit %matlab acts on this line and quitseofps -ael #back in the shell; output goes to "gaga"ls -al #The shell acts on this too!

In the above file any line beginning with "#$" is a command for thequeueing system and any line that begins with "#" is a comment line.The command

#$ -eo gagasend all of the output that normally appears on the screen during an in-teractive session to a file named "gaga". [Change the file name to some-thing different from "gaga"!] The command

#$ -mu [email protected] mail to the user when the job starts. Use your correct e-mail ad-dress here. Similarly the command

#$ -mesends mail when the job finishes.

After the preamble you can put any sequence of commands that youwould normally enter during an interactive session and these will be ex-ecuted in sequential order. In the above example the command

matlab >out « ’eof’starts MATLAB, redirects MATLAB output to a file named out and takesthe input to MATLAB from the following lines until "eof", the end-of-filemarker. The lines following "eof" should make sense to the shell, as itinterprets these lines. If you understand these principles you can con-struct any complicated script using the full programming capabilities of

A.12. PRINTING REPORTS, GRAPHS ETC. A.36

the Kron Shell! To submit the job to DQS, use,

user@machine:dir> qsub -G Medium 1 test.job

The machines are grouped into "High", "Medium" and "Low" groups basedon the hardware capabilities. Here "-G" identifies the group as "Medium"and only one machine is requested for the job with "1". Note that DQSsupports "parallel virtual machine" for jobs that can be executed in par-allel on more than a single machine. In such cases you should requestthe number of machines by replacing "1" with "n" where "n" can be be-tween 1 and 9 since we have only nine machines. The default group is"Medium" and the default number of machines is "1". So you could havesimply entered,

user@machine:dir> qsub test.job

The command qstat shows the current status of your job.

user@machine:dir> qstat

A.11.2 Example of a FLOW3D job

Construct a script file named, say m01.bat, containing the following lines.

#!/bin/csh#$ -cwd#$ -eo gaga#$ -mu [email protected]#$ -mb#$ -merunf3d -fort m02.f -command m02.fc -release 3.2.1 -geom m01.geo

A.12 Printing reports, graphs etc.

In room CME 244 there are several old Epson printers connected directlyto the OS/2 machines. Each printer serves two OS/2 machines. In roomCME 473 each of the DOS machine is connected directly to a HP Laser Jetprinter.

Since AIX and OS/2 have network support, printing on these ma-chines can be done using network printers. Hence there are no printersconnected directly to each of these machines. There are two network


LaserJet printers, one located in room CME 244 and the other in CME473. The network printers also operate on the client-server model. Itis important that you understand how the print servers operate anduse this facility in a responsible manner. If the facility is abused thisservice will be discontinued and you will have to take your print jobs tothe Micro DEMO center at the book store!

• You are responsible for providing your own paper.

• Do not send a print request to a remote printer in another location.

• If you send a print job to a remote location, the job can be deletedby the system administrator if it interferes with other queued printjobs.

• Be present in the room where the printer is located and be preparedto attend to the printer immediately after you send a print job tothe printer - i.e., feed paper, watch for paper jams, collect youroutput etc. .

• The printers are to be used only for course/research related work.For personal needs take your print job to the Micro DEMO center.

Most applications running under OS/2, AIX or DOS can generate printedoutput for a number of different types of printers. Each type of printer/plotterunderstands a particular set of instructions. For example HP LaserJetuses PCL (Print Control Language) format, while HP plotters use HPGL(HP Graphics Language). Another widely used page description languageis called Postscript. Application programs use something called a printerdriver to generate the output suitable for a particular output device.

If the output is generated on a computer that is connected directlyto a printer (like most DOS machines to HP LaserJet in CME 473 or OS/2machines to Epson in CME 244) then you can send the print job directlyto the printer, typically through the printer port LPT1:

If the output is generated on a computer that does not have a printerconnected directly to it or you want a LaerJet output from an OS/2 ma-chine, then you must use one of the network printers. In this case youmust first save the output from the application program into a file. Mostwell designed application programs will give you the option to select theoutput device and to save the printed output into a file. Use the follow-ing convention in naming the output files:


filename.ps postscriptfilename.eps Encapsulated postscript

(Useful for merging graphics with text)filename.pcl HP Laser Jetfilename.hgl HP plotters using HPGL instruction setfilename.dot Epson dot matrix printers

Note that such files contain specific instructions that are understood byspecific printers/plotters. Hence they must be sent to the appropriateprinters. If you send a PCL file to an Epson printer you will get garbledoutput that will make no sense!

A.12.1 Using the network printer from AIX machines

The simplest approach is to use the script named prnt on AIX machinesavailable within the chemical and materials engineering department. Itis a script written by Bob Barton. Hence it is available only on machinesmaintained by Bob Barton.

user@machine:dir> prnt options file name

Unix experts may want to try the basic set of unix commands lpr, lpq,lprm to submit, monitor and manage a print task. On the AIX machines,the command to submit a PCL file to a network printer is

user@machine:dir> lpr -Pprinter name file name.pcl

where the printer name is either LJ 244 (LaerJet printer in room CME244), LJ 473 (LaserJet printer in room CME 473) or PS 475 (the postscriptprinter in room CME 475). Your job is then queued and you can examine Note that PS 475 is

available only onthe graduatenetwork, whileLJ 244 and LJ 473are available on theundergraduatenetwork.

the status of the queue with the command,

user@machine:dir> lpq -Pprinter name

You can remove your print jobs from the queue with the command,

user@machine:dir> lprm -Pprinter name job number

where job number is the number returned by the lpq command.If your application program does not support the HP Laser Jet ( i.e.,

PCL format) output device, you can select the postcript output deviceand save the file as fname.ps. A public domain utility program called

A.13. ANONYMOUS FTP SERVICE A.39

"ghostscript" allows such postscript files to be either previewed on themonitor of an AIX machine under X-windows or convert the file to PCLformat for printing on a Laser Jet printer. To use this conversion pro-gram enter,

user@machine:dir> gs -sDEVICE=laserjet -sOutputFile=fname.pcl fname.ps

If you are using X-windows and want to preview the contents of the filefname.ps enter,

user@machine:dir> DISPLAY=hostname:0user@machine:dir> gs fname.ps

where hostname is the name of the X-client. To find out more about"ghostscript" options use one of

user@machine:dir> gs -?user@machine:dir> man gs

Important Note: Once you have printed the document delete the outputfiles from your home directory.

A.12.2 Using the network printer from OS/2 machines

The "lpr" command is also available on OS/2 machines. From an applica-tion program you should select the output device to be HP LaserJet andsave the output into a file as discussed in the previous section. This filewill be on the local computer in the directory that you select. You canprint such an output file as follows:

On HP LaserJet in CME 473

[C:\] lpr -ssprint.ucs.ualbeta.ca -pHPLaserJ fname

A.13 Anonymous ftp service - Netlib and other archives

A number of computer sites on the INTERNET provide a useful softwaredistribution service called anonymous-ftp-service. You can signon totheir machines as an anonymous user and retrieve (or download) soft-ware. You should use anonymous as the userid and your e-mail address

A.13. ANONYMOUS FTP SERVICE A.40

as the password. Recall from section §A.10.4, that the e-mail addresseshave the general form [email protected]. You must becourteous in using such services. Normally you are requested to down-load software only during off-peak hours. You must also conform to allthe software licensing conditions. A sample ftp session is given below.

Sample ftp session

user@machine:dir> ftp ftp-os2.nmsu.eduConnected to hobbes.NMSU.Edu.220 hobbes FTP server (Version 5.1 (NeXT 1.0) Tue Jul 21, 1992)ready.Name (ftp-os2.nmsu.edu:kumar): anonymous331 Guest login ok, send ident as password.Password: [email protected] Guest login ok, access restrictions apply.ftp>

Note that it is not possible to connect to INTERNET from the subnet129.128.44.nnn ( i.e., from room CME 244). Some well known sites andthe type of software that they serve are listed below.

Type of Software internet address domain nameMS-DOS related software 192.88.110.20 WSMR-SIMTEL20.ARMY.MILMS-DOS related software 128.252.135.4 wuarchive.wustl.eduAIX related software 128.97.2.211 Harpo.SEAS.UCLA.EDUdynamical systems theory 132.239.86.10 lyapunov.ucsd.eduTEXrelated software 192.92.115.8 Niord.SHSU.eduTEXrelated software 134.173.4.23 ymir.claremont.eduNeXT related software 128.210.15.30 sonata.cc.purdue.eduOS/2 related software 128.123.35.151 ftp-os2.nmsu.eduU of A ftp server 129.128.76.12 ftp.srv.ualberta.canumerical analysis software 192.20.225.2 research.att.com

The reasonable man adapts himself to the world:the unreasonable one persists in trying to adapt theworld to himself. Therefore all progress depends onthe unreasonable man.

— GEORGE BERNARD SHAW

Appendix B

An introduction to MATLAB

B.1 Introduction

MATLAB is a powerful, interactive software tool for numerical compu-tations normally encountered in engineering and science. It is availableon several platforms including personal computers using DOS, worksta-tions, mainframes and supercomputers using UNIX. It brings together alarge collection of powerful numerical algorithms (from LINPACK, EIS-PACK etc) for solving a variety of linear algebra problems and makesthem available to the user through an interactive and easy-to-use inter-face. Very little programming effort is required to use many of its stan-dard functions. Yet, an experienced programmer can write advancedfunctions and even develop entire tool boxes for specific applicationslike control system design and signal processing. In fact several suchtool boxes already exist.

MATLAB is available in the micro computer lab in room CME 244 onnine of the IBM RS/6000-220 workstations and on the SunOS serversugrads1.labs and ugrads2.labs running under SunOS operating system.These machines are on the internet and hence are accessible through avariety of means. Note that a student edition of MATLAB is availablefrom the Book store. If you have a PC at home the software and themanual is a great buy.

Since MATLAB is interactive, you are encouraged to try out the ex-amples as you read this manual. After each step, observe the outcomecarefully. Since computers are programmed to respond in predictable

B.1

B.2. STARTING A MATLAB SESSION B.2

manner, the key to mastering them is to be very observant.Familiarity with the basic concepts of the operating system and the

networked environment are assumed. In this notes you will be intro-duced to some of the basic numerical and graphical capabilities of theMATLAB. In particular the following will be explored.

• Starting a MATLAB session

• Using built in HELP, DEMO features

• data entry, line editing features of MATLAB.

• Summary of some of the built in functions in MATLAB for solvingproblems in

> linear algebra

> root finding

> curve fitting

> numerical integration

> integration of initial value problems

> nonlinear equations and optimization

> basic plotting capabilities

> Writing MATLAB functions and scripts - the m-file

For the adventurous, here are some of its advanced features. Explorethem on your own! The package provides features (or tool boxes) for sig-nal processing, control system design, identification and optimizationthrough what are called m-files. The graphic features support include3D and contour plotting as well as device drivers for a variety of out-put devices including Postscript and meta file capabilities for producinghigh quality plots (not just screen dumps!). It also provides facilitiesfor developing ones own tool boxes as well as facilities for interfacingwith other high level languages such as FORTRAN or C and invoke suchroutines from within MATLAB.

B.2 Starting a MATLAB session

B.2.1 Direct access on AIX machines

Find a free station in room CME 244 the range of numbers 31 to 39.These are the AIX machines. Signon by entering your userid and passs-word. i.e.,

B.2. STARTING A MATLAB SESSION B.3

Console login: useridpassword: password

Next start the X-Window interface by entering

user@machine:dir> xinit

You may copy the files ".mwmrc" and ".Xdefaults" from the directory"/afs/ualberta.ca/home/k/u/kumar/". These files customize the X-windowsenvironment when the X-server is started with the command "xinit". Sev-eral windows will be started up and one of them will be named "Aixterm".Normally this would be the shell "ksh" and all the paths to applicationprogram will be setup correctly for you to start running application pro-grams. To start MATLAB simply enter


If MATLAB does not start, seek help from the system administrator!

B.2.2 Remote access from OS/2 machines or home computer

The procedure for connecting from an OS/2 machine to AIX machinevia X-windows was outlined in section A.5.2. This will provide a full X-window based access to MATLAB. If you want to use MATLAB from ahome computer, only VT100 based emulation support is available, un-less you have X-windows client on your home computer. Follow the stepsoutlined in section A.5.3 to connect to an AIX machine from home usingkermit. In either case, after a successful connection has been estab-lished, enter,


to start a MATLAB session. The current version of MATLAB is 4.0a. Ithas advanced 2-D and 3-D graphics capabilities. The graphics features,however, rely heavily on X-windows. Hence you must invoke MATLABunder X-windows in order to see the graphs and images on the screen.If you start MATLAB from a home computer, or a VT100 terminal oncampus. you are limited to seeing the textual output only on the screen.You can still generate graphs with appropriate MATLAB commands, savethem on to a file or print them, but you cannot see them on the screen.

B.3. MATLAB BASICS B.4

B.3 MATLAB basics

Once you start MATLAB successfully, you should see the following prompton your screen.

< M A T L A B (tm) >(c) Copyright 1984-92 The MathWorks, Inc.

All Rights ReservedVersion 4.0aDec 11 1992

Commands to get started: intro, demo, help helpCommands for more information: help, whatsnew, info, subscribe

»

This provides you with an interactive workspace in which you can defineany number of variables and invoke any function. To exit MATLAB atany time enter

» quit

The commands that you enter within MATLAB are acted upon immedi-ately. As soon as you enter a line like,

» fname

MATLAB checks if "fname" is a valid MATLAB command or a built in func-tion. If so it will be executed immediately. If not MATLAB searches thepath to look for a external function or a file by the name "fname.m". Sucha file is called a m-file, as its file extension is "m". If such a file is foundit will execute the contents of that file. If not, MATLAB will generate anappropriate error message. m-files can be either scripts (i.e. a series ofvalid MATLAB commands that are executed often and hence stored ina file) or they can be used to define entirely new MATLAB functions ofyour own. More on m-files later.

While in MATLAB, if you have the need to execute a UNIX shell com-mand, you can do so with the escape character ! - e.g., try When you exit the

editor usingctrl-x ctrl-c,you will return toMATLAB

» !emacs

to invoke the emacs editor, or


» !ls -al

to look at a list of all the files in your directory.

B.3.1 Using built in HELP, DEMO features

MATLAB provides extensive online help using commands like help, demo,type, lookfor, whatsnew. They are not only useful for checking the syn-tax of a particular function, but also for exploring and learning about newtopics. Since the help command often generates lots of text that tend toscroll by very quickly, it is useful to enable a feature called “more” withthe command,

» more on

When this is enabled, you will be shown one screen full of informationat a time. Note that this is also UNIX feature that you can use with anyprogram that generates lots of scrolling text. To get started with theonline help, first get a list of help topics using

» help

Table B.1 provides a list of help topics which should give you some ideaabout the broad scope of MATLAB. You can obtain a list of functionsunder each topic (or directory) by entering help topic. For example toget a listing of general purpose commands (the first item in the abovetable) enter,

» help general

The list so produced is given in Table B.2 to serve as a reference mate-rial. Many of the functions that will be useful in a numerical methodscourse are listed in subsequent sections of this chapter. One way tobecome proficient in MATLAB is to use this HELP feature liberally - i.e.,when ever you are in doubt call on the HELP!

There is also a built in DEMO feature. To invoke this feature simplyenter Try some graphics

demos. In Version4.0, this works onlyunder X-windows

» demo

It will provide you with a menu of items. Select the ones that interestyou most. You can also search by keywords using the command lookfor.Try,


directory/topic Brief descriptionmatlab/general General purpose commandsmatlab/ops Operators and special charactersmatlab/lang Language constructs and debuggingmatlab/elmat Elementary matrices and matrix manipulationmatlab/specmat Specialized matricesmatlab/elfun Elementary math functionsmatlab/specfun Specialized math functionsmatlab/matfun Matrix functions & numerical linear algebramatlab/datafun Data analysis and Fourier transform functionsmatlab/polyfun Polynomial and interpolation functionsmatlab/funfun Function functions & nonlinear numerical methodsmatlab/sparfun Sparse matrix functionsmatlab/plotxy Two dimensional graphicsmatlab/plotxyz Three dimensional graphicsmatlab/graphics General purpose graphics functionsmatlab/color Color control and lighting model functionsmatlab/sounds Sound processing functionsmatlab/strfun Character string functionsmatlab/iofun Low-level file I/O functionsmatlab/demos Demonstrations and samplestoolbox/control Control System Toolboxtoolbox/ident System Identification Toolboxtoolbox/local Local function librarytoolbox/optim Optimization Toolboxtoolbox/signal Signal Processing Toolboxsimulink/simulink SIMULINK model analysis and construction functionssimulink/blocks SIMULINK block librarysimulink/simdemos SIMULINK demonstrations and samples

Table B.1: List of MATLAB Ver 4.0 help topics


Function Brief descriptionManaging commands and functions

help On-line documentationwhat Directory listing of M-, MAT- and MEX-filestype List M-filelookfor Keyword search through the HELP entrieswhich Locate functions and filesdemo Run demospath Control MATLAB’s search path

Managing variables and the workspacewho List current variableswhos List current variables, long formload Retrieve variables from disksave Save workspace variables to diskclear Clear variables and functions from memorypack Consolidate workspace memorysize Size of matrixlength Length of vectordisp Display matrix or text

Working with files and the operating systemcd Change current working directorydir Directory listingdelete Delete filegetenv Get environment value! Execute operating system commandunix Execute operating system command & return resultdiary Save text of MATLAB session

Controlling the command windowcedit Set command line edit/recall facility parametersclc Clear command windowhome Send cursor homeformat Set output formatecho Echo commands inside script filesmore Control paged output in command window

Starting and quitting from MATLABquit Terminate MATLABstartup M-file executed when MATLAB is invokedmatlabrc Master startup M-file

Table B.2: General purpose MATLAB Ver 4.0 commands


» lookfor inverse

which will scan for and print out the names of functions which have thekeyword "inverse" in their help information. The result is reproducedbelow.

INVHILB Inverse Hilbert matrix.ACOS Inverse cosine.ACOSH Inverse hyperbolic cosine.ASIN Inverse sine.ASINH Inverse hyperbolic sine.ATAN Inverse tangent.ATAN2 Four quadrant inverse tangent.ATANH Inverse hyperbolic tangent.ERFINV Inverse of the error function.INVERF Inverse Error function.INV Matrix inverse.PINV Pseudoinverse.IFFT Inverse discrete Fourier transform.IFFT2 Two-dimensional inverse discrete Fourier transform.UPDHESS Performs the Inverse Hessian Update.

B.3.2 Data entry, line editing features of MATLAB

The basic variables in MATLAB are treated as matrices. Vectors andscalar are special cases of a general matrix data structure. SimilarlyMATLAB handles complex variables and numbers in a natural way. Realvariables, then are, special cases. Note that MATLAB is case sensitive.

• MATLAB remembers the previous command lines that you have en-tered. You can recall them by simply using the up and down arrowkeys (or ctrl-p and ctrl-n key combinations) and then edit themand reenter the edited command as a new command. Basically,it supports the following emacs key definitions for command lineediting.


Function Key sequencePrevious line ctrl-pNext line ctrl-nOne character left ctrl-bOne character right ctrl-fOne word left esc b, ctrl-lOne word right esc f, ctrl-rCursor to beginning of line ctrl-aCursor to end of line ctrl-eCancel line ctrl-uDelete character ctrl-dInsert toggle ctrl-tDelete to end of line ctrl-k

• To assign a value to a variable use the assignment operator "=". Forexample,

» A = [1 2 3; 4 5 6; 7 8 9]

will result in a 3x3 matrix. Note that there is no need to explicitlydeclare the dimension of an array. Since MATLAB is case sensitiveyou have defined only "A " and "a" remains undefined. Similarly

» x=[2+4*i, 3+5*i]

will generate a complex vector with two elements. If you want toadd another element enter what would the

value of x(3) be?» x(4)=5+6*i

Note that the dimension of the vector x is now automatically in-creased to 4. Observe that the square brackets are used in formingvectors and matrices. Semicolon is used to separate rows in a ma-trix. Comma is used to separate individual elements of a vector (ormatrix). Parentheses are used to identify individual array elements.(Try help punct and help paren)

• After you have defined the variables A and x as above, go throughthe following exercise and make sure you understand the result.

» A(2:3,1:2)

Observe the use of () and : to select a sub block of A. Next, try What mighthappen if the sizeof sub-blocks aredifferent?


» B(4:5,2:3)=A(2:3,1:2)

This demonstrates how to extract a sub-block matrix of A and as-sign it to another sub-block of B. Next, try, What would be the

value of x after youexecute thiscommand? Why?

» x(4:-1:1)

which reverses the order of elements of x. Next, try the command,

» p=[1 3]; x(p)

Observe that there are two commands, separated by semicolon.This example also demonstrates a powerful way of selecting spe-cific elements of a vector. This is easily extended to matrices also.Well, try,

» q=[2 3]; A(p,q)

I hope you get the idea.

• To examine the value of a variable simply enter the name of thevariable. All the variables that you define during a MATLAB sessionare stored in the workspace ( i.e., in computer memory) and theyremain available for all subsequent calculations during the entireMATLAB session i.e., until you “quit” MATLAB.

• You can declare any variable to be global in nature using,

» global A

If the same variable is also declared as global in several functions,then all those functions share the same value. To check if a variableis global use,

» isglobal(A)

A value of 1 is returned if it is global.

• To examine the list of variables currently defined in your workspaceand the attributes of those variables, use one of the two commands"who" and "whos".

» whos


• To generate a set of equally spaced values in a simple manner fol-low the example below:

» x = 0 : 0.05 : 1.0

will generate x = [0 0.05 0.1 0.15 0.2 · · · 1.0]. (Try help colon).

• To suppress the automatic echoing of any line that you enter fromkeyboard, terminate such a line with a semi-colon ";". For example

» x = 0 : 0.05 : 1.0;

will define x as before, but will not echo its value. (Try help punct).

• To continue the entry of a long statement onto the next line use anellipsis consisting of three or more dots at the end of a line to becontinued. For example

» s = 1− 1/2+ 1/3− 1/4+ 1/5− 1/6+ 1/7 · · ·» −1/8+ 1/9− 1/10+ 1/11

• Anything that follows a % sign is treated as a comment. For exam-ple the following is a valid command line.

» I = 1 : 1 : 20 %generating a set of integers from 1 to 20

• The numeric display format is controlled by the "format" command.Use

» format long

for 14 digits display. (Try help format )

• You can save the contents of a workspace with the "save" command.Try, Try the command

!ls jnk*Observe that theextension .mat hasbeen added

» save jnk

In the next few statements examine the currently defined variables,clear the workspace and load a previously saved workspace.

» whos» clear


» whos» load jnk» whos

• The following matrix operations are available in MATLAB. You canuse help on each of them to find out more precise information onthem.

+ addition, e.g., C = A+ B ⇒ Cij = Aij + Bij− subtraction, e.g., e.g., C = A− B ⇒ Cij = Aij − Bij∗ matrix multiplication, e.g., C = A∗ B ⇒ Cij =

∑k AikBkj

ˆ Matrix power. Z = Xˆy is X to the y power if y is a scalar andX is square. If y is an integer greater than one, the poweris computed by repeated multiplication. For other values ofy the calculation involves eigenvalues and eigenvectors. (tryhelp arith).

′ Matrix transpose. X′ is the complex conjugate transpose of X.X.′ is the non-conjugate transpose. (try help punct).

\ left division. A\B is the matrix division of A into B, which isroughly the same as inv(A)*B , except it is computed in a dif-ferent way. If A is an N-by-N matrix and B is a column vectorwith N components, or a matrix with several such columns,then X = A\B is the solution to the equation A∗ X = B com-puted by Gaussian elimination. (try help slash)

/ right division. B/A is the matrix division of A into B, which isroughly the same as B*inv(A).

Note that the dimensions of the matrices must be compatible forthe above operations to be valid; if you attempt matrix operationsbetween incompatible matrices an appropriate error message isgenerated.

• The following relational operators are available in MATLAB. Tryhelp relop for additional details.

< Less than relational operator

> Greater than relational operator

<= Less than or equal

>= Greater than or equal


== equal

˜= not equal

They are applied element-by-element between matrices of the samesize, producing a resultant matrix consisting of 0’s and 1’s.

• Element-by-element multiplicative operations are obtained as fol-lows:

operator example index notation.∗ C = A.*B Cij = AijBij.ˆ C = A.ˆB Cij = ABijij./ C = A./B Cij = Aij/Bij.\ C = A.\B Cij = Bij/Aij

B.3.3 Linear algebra related functions in MATLAB

A list of all advanced matrix related functions in MATLAB is given inTable B.3 Use the help command on each of these functions to find outmore about the function and its exact syntax.

Work through the following exercise to become familiar with the us-age of some of the linear algebra functions and refresh some of the re-sults from a first year linear algebra course.

Exercise - review of 1st year linear algebra

• Define the matrix, A and a vector, b as

» A = [1 0 0.307; 0 1 0.702; -2 1 0] Why was "A"echoed on thescreen, while "b"was not? Is "b" arow or a columnvector?

» b = [0.369*275;0.821*275;0];

Observe the two different ways semicolon has been used here. Whatare they?

• Solve the equation Ax = b using,

» x = A\b

• Verify that x satisfies the equation by calculating,

» A*x - b


Function name ActionMatrix analysis

cond Matrix condition numbernorm Matrix or vector normrcond LINPACK reciprocal condition estimatorrank Number of linearly independent rows or columnsdet Determinanttrace Sum of diagonal elementsnull Null spaceorth Orthogonalizationrref Reduced row echelon form

Linear equations\ and / Linear equation solution; use "help slash"chol Cholesky factorizationlu Factors from Gaussian eliminationinv Matrix inverseqr Orthogonal-triangular decompositionqrdelete Delete a column from the QR factorizationqrinsert Insert a column in the QR factorizationnnls Non-negative least-squarespinv Pseudoinverselscov Least squares in the presence of known covariance

Eigenvalues and singular valueseig Eigenvalues and eigenvectorspoly Characteristic polynomialhess Hessenberg formqz Generalized eigenvaluesrsf2csf Real block diagonal form to complex diagonal formcdf2rdf Complex diagonal form to real block diagonal formschur Schur decompositionbalance Diagonal scaling to improve eigenvalue accuracysvd Singular value decomposition

Matrix functionsexpm Matrix exponentialexpm1 M-file implementation of expmexpm2 Matrix exponential via Taylor seriesexpm3 Matrix exponential via eigenvalues and eigenvectorslogm Matrix logarithmsqrtm Matrix square rootfunm Evaluate general matrix function

Table B.3: Linear algebra related functions in MATLAB


• Next calculate the norm of the residual,

» norm(A*x - b)

• Determine the rank of A using

» rank(A)

• Carry out the LU decomposition using

» [L,U]=lu(A) Why does "L" notappear to be lowertriangular?

• Calculate the determinant of A using

» det(A)» det(L)*det(U)

• Calculate the eigenvalues and eigenvectors of A.

» [v,d]=eig(A)» prod(diag(d)) Can you explain

this result?

• Find out the characteristic polynomial of A and its roots.

» c1=poly(A)» roots(c1) Can you explain

these results?» prod(ans)

B.3.4 Root finding

x=fsolve(’fun’,x0) solution to a system of nonlinear equations (orzeros of a multivariable function).fun(x) is a function that you should write toevaluate f(x) - i.e., you define your problem inan m-file.x0 is the initial guess for the root. [There isobviously more to it than I can describe here!Read the manual or try help fsolve].


fzero(’fun’,x0,tol) finds the root of a single nonlinear algebraicequation.fun(x) is the external function describing yourproblem that you should write in a m-file.x0 is the initial guess.

poly(V) if V is a vector then it returns the coefficientsof the polynomial with roots determined by V.i.e., roots and poly are inverse functions of eachother.

roots(c) computes all the roots of a polynomial whosecoefficients are in c. i.e., Pn(x) = (c1xn +c2xn−1 + · · · + cn+1).

B.3.5 Curve fitting

c=polyfit(x,y,n) least-squares curve fitting of degree n. The co-efficients in descending powers of x are returnedin c.

polyval(c,s) evaluates the polynomial whose coefficients arein c at locations determined by s.

yi=spline(x,y,xi) Generates a cubic spline through the data vec-tors (x,y) and then computes a vector of in-terpolated values of yi at xi.

diff(x,n) computes the n forward differences from thevector x.

The other functions of possible interest are fmin, fmins, residue, conv, table1.

B.3.6 Numerical integration, ordinary differential equations

quad(’fun’,a,b,tol,trace) computes the definite integral over the limit(a,b) using adaptive recursive Simpson’s rule.fun(x) is an external function that you mustprovide in a m-file.tol is the acceptable global error. trace is anoptional flag to monitor the integration pro-cess.


[t,y]=ode45(’fun’,t0,tf,y0,tol,trace) integrates a system of nonstiff differ-ential equations of the form dy/dt = f(y) us-ing 4 and 5 order Runge-Kutta methods.fun(y)is the external function which defines yourproblem. You must provide this via a m-file.(t0,y0) is the initial condition.tf is the final point at which you want to stopthe integration.tol is the acceptable global error in the solu-tion. trace trace is the optional flag to printintermediate results.

The other functions of possible interest are ode23, quad8

B.3.7 Basic graphics capabilities

MATLAB ver 4.0 maintains separate graphics windows and a text win-dow. Your interactive dialogue takes place on the text window. Whenyou enter any graphics command, MATLAB plots that graph immedi-ately on the graphics window. It can open several graphics windows.So, clearly commands are needed to select a specific window to be thecurrent one. The list of graphics related commands are given in TableB.4. Work through the following exercise interactively and observe thecomputer response in order to understand the basic graphic capabilitiesof MATLAB. Make sure that you are running MATLAB under X-windows.Text following the percent sign (%) are explanatory comments. You neednot enter them.

Exercise - producing a simple graph

»x=0:0.1:2*pi; % create a vector x in the range (0,2 Pi)»figure(1) % open a graphics window labeled Figure 1»figure(2) % open a graphics window labeled Figure 2»plot(x,sin(x)) % plot sin(x)»hold % keep the graph of sin(x)»plot(x,cos(x),’go’) % add graph of cos(x) with line type ’go’»title(’My first plot’) % put some title»xlabel(’x-axis’) % label the x-axis»ylabel(’y-axis’) % label the x-axis»print -deps fig1.eps% produce a postscript copy in file fig1.eps»!ls -al fig1.eps % verify that the figure is saved in a file»!xpreview fig1.eps % use the postscript previewer of AIX (optional)


Function name ActionFigure window creation and control

figure Create Figure (graph window)gcf Get handle to current figureclf Clear current figureclose Close figure

Axis creation and controlsubplot Create axes in tiled positionsaxes Create axes in arbitrary positionsgca Get handle to current axescla Clear current axesaxis Control axis scaling and appearancecaxis Control pseudocolor axis scalinghold Hold current graph

Handle Graphics objectsfigure Create figure windowaxes Create axesline Create linetext Create textpatch Create patchsurface Create surfaceimage Create imageuicontrol Create user interface controluimenu Create user interface menu

Handle Graphics operationsset Set object propertiesget Get object propertiesreset Reset object propertiesdelete Delete objectdrawnow Flush pending graphics eventsnewplot M-file preamble for NextPlot property

Hardcopy and storageprint Print graph or save graph to fileprintopt Configure local printer defaultsorient Set paper orientationcapture Screen capture of current figure

Movies and animationmoviein Initialize movie frame memorygetframe Get movie framemovie Play recorded movie frames

Miscellaneousginput Graphical input from mouseishold Return hold statewhitebg Set graphics window defaults for white backgroundgraymon Set graphics window defaults for gray-scale monitors

Table B.4: Graphics related function in MATLAB


»gcf % get current figure (should be 2)»figure(1) % make figure 1 the current figure»close(1) % close window 1»gcf % get current figure (should be 2)»close(2) % close window 1

In this exercise you produced the data from within MATLAB. If you havecolumns of data in a file, you can read them into MATLAB and plot themas above. The postscript file produced in the above example can bemerged with other documents or printed on a postscript printer. Usehelp print to find out about support for other type of printers and plot-ters.

B.3.8 Control System Toolbox

The Control system toolbox, which uses MATLAB matrix functions, wasbuilt to provide specialized functions in control engineering. The Con-trol system toolbox is a collection of algorithms, expressed in m-files,that implement common control system design, analysis, and modelingtechniques.

Dynamic systems can be modeled as transfer functions or in state-space form. Both continuous-time and discrete-time system are han-dled. Conversions between various model representations are possible.Time responses, frequency responses, and root-locus measures can becomputed and plotted. Other functions allow pole-placement, optimalcontrol, and estimation.

The following example shows the use of some of the control systemdesign and analysis tools available in MATLAB.

Example

The process transfer function, G is defined as:

G = 1(s + 1)(s + 2)(s + 3)

The transfer function is entered into Matlab by entering the numeratorand the denominator coefficients separately as follows:

» num = 1;» den1 = [1 1];» den2 = [1 2];


» den3 = [1 3];

The denominator polynomial is the product of the three terms. Convo-lution, conv, is used to obtain the polynomial product:

» den = conv(den1,conv(den2,den3));

To get an open-loop process response to a unit step change, the functionstep can be used:

» t = 0:0.5:5; Define time in therange 0-5.Generate stepresponse and plot.

» y = step(num,den,t);» plot(t,y,’*’);

The Bode plot can be obtained by first defining a vector of frequencies,and then using the function bode:

» w = logspace(-1,1); Generate equallyspaced data in therange 10−1 and 101

» [mag,phase] = bode(num,den,w);

The bode plots for amplitude ratio and phase can be obtained by typing:

» loglog(w,mag)» semilogx(w,phase)

The root-locus can be obtained by defining a vector of desired gains, andthen using the function rlocus:

» k = 20:5:70; Define gains in therange 20-70.Generate and plotthe root-locus.

» y = rlocus(num,den,k);» plot(y,’*’)

The closed-loop transfer function can be represented by:

YYsp

= GcGp1+GcGp

The closed-loop transfer function is calculated and entered into Matlabfor analysis using the same functions used in the open-loop system.Discretization can only be done through the state-space model represen-tation. Therefore, it is necessary to transform transfer function modelsto state-space models. The transfer function model can easily be trans-formed into the state-space model by using the function tf2ss:


» [A,B,C,D] = tf2ss(num,den);where A,B,C,D are matrices in the differential equations dxdt := Ax+Buand y = Cx + Du. To obtain a discretized model, the function c2d isused:

» [ad,bd] = c2d(A,B,Ts); % Ts is the sample time

This function converts state-space models from continuous time to discrete-time assuming a zero-order hold on the input. To obtain a step responseon the discretized model, the function dstep can be used:

» y = dstep(ad,bd,C,D,1,100);» plot(y),title(’step response’);

Several additional control functions that are available in Matlab arelisted in Table B.5 The online help screen should be referred to forinformation on how to use these tools. The function what can be usedto find out what other functions are available.

B.3.9 Producing printed output of a MATLAB session

If you want to produce a hard copy of your interactive MATLAB session,you can log a record of the entire session in a file with the diary com-mand. The command

» diary file

will start recording every keyboard entry and most of the computers tex-tual response (not graphics) in file. To suspend the recording, use

» diary off

and to resume recording you can use,

» diary on

The file contains simple text (ASCII) and can be printed on the networkprinter.


Function name PurposeFunctions for model conversion

[num,den] = ss2tf (a, b, c, d, iu) State-space to transfer function[z,p, k] = sstzp(a, b, c, d, iu) State-space to zero-pole[a, b, c, d] = tf2ss(num,den) Transfer function to state-space[z,p, k] = tf2zp(num,den) Transfer function to zero-pole[a, b, c, d] = zp2ss(z,p, k) Zero-pole to state-space[num,den] = zp2tf (z,p, k) Zero-pole to transfer function[ad,bd] = c2d(a,b, Ts) Continuous to discrete[a, b] = d2c(ad,bd, Ts) Discrete to continuous

Functions for modelingappend Append system dynamicsconnect System interconnectionparallel Parallel system connectionseries Series system connectionord2 Generate A,B,C,D for a second order system

Continuous time and frequency domain analysisimpulse impulse responsestep Step responselsim Simulation with arbitrary inputsbode Bode and Nichols plotsnyquist Nyquist plots

Discrete time and frequency domain analysisdimpulse Unit sample responsedstep Step responsedlsim Simulation with arbitrary inputsdbode Discrete Bode plots

Table B.5: List of functions from control system tool box


B.3.10 What are m-files?

MATLAB derives its strength and wide popularity from being extensi-ble through the m-file facility. Extensibility means that using a core setof built-in functions, users can extend the capabilities of MATLAB bywriting their own functions. The functions are stored in files with theextension “.m”. Any file with the extension “.m” in the search path ofMATLAB is treated as a MATLAB m-file. To find out the current path ofMATLAB enter,

» path

You can list the contents of a m-file with the type command. While thehelp command produces only documentation on the function, the typecommand produces a complete listing of the function. Try,

» type sin» help sin» type erf» help erf

Note the “sin” is a built-in function and hence no code is listed. On theother hand “erf” is the error function implemented as a m-file and hencea complete listing is produced.

The m-files can take two forms - viz. (i) a script file and (ii) files thatdefine entirely new functions. Such files should be in the MATLAB searchpath.

Example of a script file

In a script file, you can put any MATLAB commands that you wouldnormally enter in an interactive session. Simply entering the name ofthe file would then execute the contents of that file. For example toenter a large matrix, create a file called "A.m" in your home directoryusing your favorite editor. This file should contain the following text.

B = [ 1 2 3 4 5 6 7 8 9;2 3 4 5 6 7 8 9 0;3 4 5 6 7 8 9 0 1;4 5 6 7 8 9 0 1 2;5 6 7 8 9 0 1 2 3]

b=sin(B)


To execute the contents of this file from within MATLAB enter,

» A

Note that a matrix variable "B" of size (5 × 9) has been defined in yourworkspace and the variable "b" contains the values of sin(B).

In a script file you can include any such sequence of valid MATLABcommands, including program flow control commands like for, if,while loops etc. However a script file is not a function file and henceyou cannot pass any arguments to the script. Also, when you executea script file from the workspace, all of the variables defined in a scriptfile become global variables. In contrast any variable defined within afunction file is local to that function and only the results returned by thefunction become global in nature.

Example of a function file

Let us take the example of an isothermal, multicomponent flash equa-tion, given by,

f(ψ) :=N∑i=1

zi(1−Ki)1+ψ(Ki − 1)

= 0

In this equation, (zi, Ki) are known vectors of length N and ψ is theunknown scalar variable. So we should like to write a function, say,flash(psi) that would return the value of f(ψ). This function should besaved in a file named "flash.m" in your home directory. Such a functionmight look as follows:

function f=flash(psi)% Calculates the flash equation f=flash(psi,K,z)% K is a vector of any length of equilibrium ratios% z is the feed composition (same length as K)% K, z are known.% psi is the vapor fraction% This is the last line of help. Notice the blank line below!

global K zf=((1-K).*z) ./ (1+(K-1)*psi);f=sum(f);

Let us understand the anatomy of this function. The first line shouldalways contain the keyword "function" in order to identify it as a func-


tion definition and not a script file. Then a list of values computed andreturned by the function should appear - in the present case only "f" isbeing returned. If you have more variables being returned you wouldlist them as "[f1, f2, f3] etc. Next, the equal sign is followed by thename of the function. Then the list of input variables are given in paren- Note that the file

name isconstructed byappending ".m" tothe function name.In the aboveexample the filename will beflash.m

thesis. The next several lines begin with the percent sign (%) and henceare treated as comments. Here is the place to put the documentationon what the function does and how to use it. This is also the part thatis printed out when a user asks for help on this function. A blank linesignifies the end of the help message. The actual code follows the blankline. Notice the use of element-by-element multiplication of two vectorswhich avoids the use of do loops. How elegant!

Assuming that you have created a file called "flash.m" containingthe above lines, work through the following steps.

» help flash» type flash» global K z» z=[.25 .25 .25 .25]» K=[1 .5 .4 .1]» whos» flash(0.1)

As a challenge, take up the task of modifying the function flash suchthat it will take in a vector of ψ values and return the correspondingfunction values!

B.3.11 Programming features

If you know any one high level programming language such a FORTRAN,C or even BASIC, you should have no difficulty in understanding the ele-mentary program flow control features of MATLAB. A list of help topicsis given in Table B.6. Let us take the example of "flash.m" and illustratethe use of "if" and "for" constructs. First we check if the length of vec-tors K, z are the same; if not we generate an error message. Note thelength and error are built-in MATLAB functions. In the next sectionwe determine the length of input vector "x" and build a loop to calculatethe function for each element of "x" and store it in the correspondingelement of "f". Use "help relop" and "help lang" to find out more aboutrelational operators and programming language features.


Example

function f=flash(x)% K is a vector of any length of equil ratios.% z is the feed composition (same length as K)% K, z are defined as global in main% x is the vapor fraction% The following is the isothermal flash eqn.

global K z

if ( length(K) ˜= length(z) )error(’Number of K values & compositions do not match’)end

n=length(x); %Find the length of the input vectorfor i = 1:n %setup a loop for each element of xt=((K-1).*z) ./ (1+(K-1)*x(i));t=sum(t);f(i) = t;end

B.3.12 Debugging tools

Version 4.0 of MATLAB provides for the first time some debugging tools.If you are familiar with the debugging concepts, use of this facility shouldbe straight forward. Basically it is a tool for debugging new functionsthat a user develops. It provides tools for the following:

• to setup break points, where the execution of a function will besuspended and the control returned to the user, (dbstop function-name)

• examine values of variable,

• execute one line at a time. (dbstep)

• trace the sequence of function calls (dbstack),

• resume execution. (dbcont)

• list a function with line numbers. (dbtype function-name)


Function name ActionMATLAB as a programming language

script About MATLAB scripts and M-filesfunction Add new functioneval Execute string with MATLAB expressionfeval Execute function specified by stringglobal Define global variablenargchk Validate number of input arguments

Control flowif Conditionally execute statementselse Used with IFelseif Used with IFend Terminate the scope of FOR, WHILE and IF statementsfor Repeat statements a specific number of timeswhile Repeat statements an indefinite number of timesbreak Terminate execution of loopreturn Return to invoking functionerror Display message and abort function

Interactive inputinput Prompt for user inputkeyboard Invoke keyboard as if it were a Script-filemenu Generate menu of choices for user inputpause Wait for user responseuimenu Create user interface menuuicontrol Create user interface control

Debugging commandsdbstop Set breakpointdbclear Remove breakpointdbcont Resume executiondbdown Change local workspace contextdbstack List who called whomdbstatus List all breakpointsdbstep Execute one or more linesdbtype List M-file with line numbersdbup Change local workspace contextdbquit Quit debug mode

Table B.6: Program control related help topics


• quit debugging session. (dbquit)

Since all the variables with in a function are treated as local variables,their values are not available in the workspace. To examine their valueswithin a function, you have to be able to stop the execution at a spec-ified line within a function and examine the values of local variables.The following exercise illustrates the debugging process using the flashfunction developed earlier. So we assume that a "flash.m" file exists inyour current directory.

Exercise - debugging a function

»K=[2 1 .5 .3]; % define K values»z=[.25 .25 .25 .25]; % define z»global K z % define K z to be global variables»whos % examine current variables»dbstop flash % set up break point in flash»psi=[.2:.2:.8] % define psi (a vector)»flash(psi) % begin execution of flash, it will stop at8 global K z the first executable line (line 8 here)K»dbtype flash % list the function with line numbers1 function f=flash(x)2 % K is a vector of any length of equil ratios.3 % z is the feed composition (same length as K)4 % K, z are defnied as global in main5 % x is the vapor fraction6 % The following is the isothermal flash eqn.78 global K z9 if ( length(K) = length(z) )10 error(’Number of K values & compositions do not match’)11 end12 n=length(x)13 for i = 1:n14 t=((K-1).*z) ./ (1+(K-1)*x(i));15 t=sum(t);16 f(i) = t;17 endK»dbstop 12 % set up a new break point at line 12K»dbcont % resume execution from current line 812 n=length(x) stops before executing line 12K»x % examine the value of x


K»n % n - should be undefined at this stageK»dbstep % execute one line and stop at line 1313 for i = 1:nK»n % now display value of n (should be 4)K»dbstatus flash % display the break points.K»K % display the value of KK»z % display the value of zK»dbcont % resume execution till the end of function.K»dbquit % terminate debugging of flash

This simple example illustrates how to set up break points, examinevalues of variables and step through execution one line at a time.

Angling may be said to be so like the mathematics,that it can never be fully learnt.

— IZAAK WALTON

So is UNIX.

— K. Nandakumar

Appendix C

Some basic unix commands

C.1 Introduction to the shell and the desktop

In a command line oriented, interactive environment, a command shell(which is a program or a process) accepts a command from the keyboard,passes it to the operating system for execution, prints out any error or in-formational messages generated by the command and displays a promptstring indicating the readiness to accept another command. There areseveral shells available under AIX. The Kron shell or ksh is one of themost powerful shells and is the default shell on the AIX machines main-tained by the department of chemical engineering.

In the GUI oriented environment, the equivalent of a command shellis the desktop which organizes various tools and application programssuch as file manager, program manager, printer manager, etc. as objectsaccessible via icons. The interaction takes place through dialogue boxesand forms that must be completed. Program execution begins simply bydouble clicking on the appropriate icons.

If you have a good reason to change your default shell to somethingother that ksh, you can do so with the chsh command,

user@machine:dir> chsh

This command will display your current shell, and prompt you for the

C.1

C.1. INTRODUCTION TO THE SHELL AND THE DESKTOP C.2

name of the new shell. The change takes effect when you login the nexttime. At any time you can invoke a new shell, different from the loginshell, e.g.,

user@machine:dir> csh

invokes a C-shell.You can invoke a desktop at any time on AIX machines by entering

user@machine:dir> xdt3

Since the use of desktop is supposed to be rather intuitive, you are en-couraged to explore its features on your own!

C.1.1 The ".profile" file

The ".profile" file is used to customize the shell environment for ksh.The following is a typical example of a ".profile" file.

PATH=$PATH:$HOME/bin:$KHOROS_HOME/binHOSTNAME=‘hostname‘PS1=’ $LOGNAME@$HOSTNAME:$PWD>’EDITOR=emacs

#defines alias for commonly used commandsalias ls=’ls -al’alias rm=’rm -i’

#export environment variables to other processes ...export PATH HOSTNAME PS1 EDITOR TERM

Here, several environment variables such as PATH, HOSTNAME, PS1 etc.have been defined. The concept of the environment is like a bulletinboard. You can post definitions of any number of variables there. Ap-plication programs that expect specific variables can look for them anduse their values, if they are defined. Note that when you set the value ofa variable ( i.e., left hand side of an equal sign), the name is used withoutany prefix. When you want to retrieve the value the $ prefix is used. Forexample try,

user@machine:dir> echo $PATH

to look at the value of the current path.

C.1. INTRODUCTION TO THE SHELL AND THE DESKTOP C.3

In the first line of the example above, a system wide environmentvariable $PATH, which is already defined, is redefined to add additionalpaths, such as $HOME/bin separated by colon. Observe the ’$’ prefix tothe name of the environment variable. $HOME itself is an environmentvariable, containing the value of the home directory. In the 2nd line thevariable HOSTNAME is defined to contain the name of the workstation.This name is actually retrieved by the program ‘hostname‘. The 3rd lineredefines the prompt string PS1 using other variables such as LOGNAME,HOSTNAME and PWD. These variables contain, respectively, the values ofthe userid, machine name and present working directory. If the variableEDITOR is set to emacs, then command line editing features using emacskeys are enabled.

You can also define aliases for certain commands. In the exampleabove, the "ls" string is defined to be ’ls -al’ - so when you enter "ls" atthe prompt, the command ’ls -al’ is executed. To examine all the cur-rently defined aliases, enter,

user@machine:dir> alias

By default, the "rm" command removes files without prompting you forconfirmation which could result in accidental deletion of files. The aliasdefined above, assigns ’rm -i’ to "rm". The keyword "-i" stands for inter-active mode and hence you will always be prompted before removing afile.

The variables defined in a shell environment are available only to thatshell environment and not to other shells that you may start from thecurrent one. The export command is used to export the variables to allsubsequent shells. The last line in the above example exports severalenvironment variables.

To look at all of the environment variables defined in the current kshshell, enter,

user@machine:dir> set

To examine the value of an environment variable, enter,

user@machine:dir> echo $PS1

To set a new environment variable, use

user@machine:dir> DUMMY=junk

C.2. MANAGING FILES C.4

In addition to assigning values to environment variables, the shellallows programming flow control features. Thus one can write quitepowerful scripts to execute a complex sequence of commands. A scriptis nothing but a set of AIX instructions placed in a file. By enabling theexecute permission for this file, and entering its name from the commandline you can cause the instructions in that file to be executed.

C.2 Managing files

In managing your files and directories, you need to be able to list thecontents of a directory or file, copy and move files, compress and uncom-press files, create and delete files and directories, control the ownershipand access to files etc. Commands to carryout these tasks are illustratedbelow with specific examples. Try them out at a terminal. To get a com-plete description of each command use the man pages i.e.,


C.2.1 Making sense of the directory listing - the "ls" command

The ls command produces a listing of all the files in the current direc-tory. In its most useful form, you will use the “-al” keywords, i.e.,

user@machine:dir> ls -al dir

Typically, files that begin with the “.” ( e.g., .profile) are treated as hiddenfiles. They keyword “-a” however lists all of the files including the hid-den ones. The keyword “-l” produces the long listing, a sample of whichis shown in figure C.1. This listing provides information on file accesscontrol, ownership, size, time stamp etc. Each line contains informationfor a file or directory. The first character identifies whether it is a file (-),a directory (d) or a symbolic link (l). A symbolic link is a pointer to someother file (think of it as an alias). The next set of nine characters iden-tify the file access control, in groups of three. Since AIX is a multiuserenvironment, users can control ownership and access of their files toothers. The possible access modes are: read (r), write (w) execute (x) ornone(-). These modes apply to (user, group, others). The groups are es-tablished by the system administrator. The owner and group names arelisted next, followed by file size in bytes, the time stamp for last changeand the file name.


drwxr-sr-x 27 kumar sys 1536 May 24 23:14 .drwxr-sr-x 59 sys sys 1536 May 13 08:52 ..-rw-r--r-- 1 kumar others 1937 Jan 07 11:47 .Xdefaultsdrwx------ 2 kumar others 512 Jul 21 1992 .elm-rw-r--r-- 1 kumar sys 2504 May 19 12:08 .mwmrc-rwxr-xr-x 1 kumar sys 610 May 04 12:36 .profile-rw------- 1 kumar sys 348 May 14 12:22 .rhostsdrwxr-xr-x 3 kumar others 512 Jul 21 1992 .tin-rw-r--r-- 1 kumar sys 136 May 11 14:11 .xdt3-rw-r----- 1 kumar others 1222 Jan 19 1992 Ass1.mdrwxr-xr-x 2 kumar others 512 May 19 13:12 CHEM2drwx------ 2 kumar others 512 May 27 1992 Mail

time stamp

File name Owner Group File sizepermission

control

d indicates a directoryl indicates a symbolic link

r read permissionw write permissionx execute permission- no permission

Other related Unix commandsls -al - detailed listing of directory such as the abovechmod - change permission on files and directorieschown - change ownership of files and directoriesrm - remove or delete a filermdir - remove or delete a directorymkdir - create a new directory

1st set applies to owner2nd set applies to group3rd set applies to all

Examples: The file .profile has (read,write,execute) permission for owner (kumar in this case) and (read,execute) permission for both the group (sys in this case) and everyone.

The directory Mail has (read,write,execute) permission for owner only

The command chmod g+r filewill give read access to group for file, while chmod o-w filetakes away write access to all for file

Figure C.1: Output of the "ls" command


C.2.2 Changing permission on files - the "chmod" command

The chmod command allows you to modify the access control of files anddirectories.

Examples

• To give read permission to group for file use,

user@machine:dir> chmod g+r file

• To give write permission to everyone for all the files in a directoryuse,

user@machine:dir> chmod -R a+w dir

Note the the "-R" flag stands for recursive use of the command forall files in all subdirectories.

• Note that, in order to give read permission to a directory, the exe-cute permission at the directory level must be set.

user@machine:dir> chmod a+x dir

C.2.3 Moving files

The mv (move) command moves files and directories from one directoryto another, or renames a file or directory. You cannot move a file ontoitself.Warning: The mv command can overwrite many existing files unless youspecify the -i flag. The -i flag prompts you to confirm before it overwritesa file.

Examples

• To rename a file, enter:

user@machine:dir> mv oldname newname

This renames file oldname to newname. If a file named newnamealready exists, its contents are replaced with those of oldname.


• To move a directory, enter:

user@machine:dir> mv olddir newdir

This moves all files and directories under olddir to the directorynamed newdir, if newdir exists. Otherwise, the directory olddir isrenamed to newdir.

• To move several files into another directory, enter:

user@machine:dir> mv file1 dir1/file2 newdir

This moves file1 to newdir and dir1/file2 to newdir/file2.

C.2.4 Copying files

The cp command creates a copy of the contents of the file or directoryfrom a source to a target. If the file specified as the target exists, thecopy writes over the original contents of the file. If you are coping morethan one source file, the target must be a directory.

Examples

• To make a copy of a file in the current directory, enter:

user@machine:dir> cp file.old file.new

If file.new does not already exist, then the cp command creates it.If it does exist, then the cp command replaces it with a copy of thefile.old file.

• To copy a file in your current directory into another directory, enter:

user@machine:dir> cp file.old dir/sub.dir/

This copies file.old to dir/sub.dir/file.old.

• To copy all the files in a directory to a new directory, enter:

user@machine:dir> cp /home/user/dir1/* /home/user/dir2

This copies all the files in the directory /home/user/dir1/ to thedirectory /home/user/dir2/. As a variant, explore the "-R" flag tocopy not only all of the files, but also all of the subdirectories.


C.2.5 Changing ownership of files - the "chown" command

The chown command changes the owner of the file specified by the Fileparameter to the user specified by the Owner parameter. The Ownerparameter can be specified either as a user ID or as a login name foundin the /etc/passwd file. Optionally, a group can also be specified. Thegroup can be specified either as a group ID or as a group name found inthe /etc/group file. The syntax is,

user@machine:dir> chown -R owner:group file

Only the root user can change the owner of a file.

C.2.6 Compressing files - the "compress" command

The compress command reduces the size of files using adaptive Lempel-Zev coding. Each original file specified by the file parameter is replacedby a compressed file with a ".Z" appended to its name. The compressedfile retains the same ownership, modes, and access and modificationtimes of the original file. If compression does not reduce the size of afile, a message is written to standard error and the original file is notreplaced. The syntax is,

user@machine:dir> compress file

To restore the file to its original state use the command,

user@machine:dir> uncompress file

Also try the GNU version of compress utility called gzip and gunzip -they are more efficient in both speed and size.

C.2.7 Removing files - the "rm" command

The rm command removes the entries for the specified file or files from adirectory. If an entry is the last link to a file, the file is then deleted. Youdo not need read or write permission for the file you want to remove.However, you must have write permission for the directory containingthat file.

Examples

• To delete a file, enter:

C.3. MANAGING PROCESSES C.9

user@machine:dir> rm myfile

If there is another link to this file, then the file remains under thatname, but the name myfile is removed. If myfile is the only link,the file itself is deleted. Caution: You are not asked for confir-mation before deleting the file. It is useful to set an alias in your".profile" file to redefine "rm" as

alias rm=’rm -i’

After each file name is displayed, enter "y" to delete the file, orpress the Enter key to keep it.

C.3 Managing processes

Since AIX is a multi-tasking operating system, several tasks (or processes)can be running at the same time. So we need a set of tools to monitor thecurrently running processes and the resources they consume, suspendor terminate specific processes, set priority for certain tasks or schedulesome tasks for execution at specified times. Commands to accomplishthese tasks are illustrated next.

C.3.1 Examining jobs or processes - the "ps" command

The ps command displays a set of currently running tasks. In its sim-plest and most useful form, the command is,

user@machine:dir> ps -ael

This provides a long listing of all the currently running processes in-cluding all of the daemons started by the root at the time of booting thecomputer. A typical sample output might look like,

F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD200001 R 21 15101 17095 13 66 20 196d 116 pts/0 0:00 ps240801 S 21 17095 16070 3 61 20 1dce 108 pts/0 0:00 ksh260801 S 0 3637 3112 0 60 20 b25 260 - 0:00 sendmail260801 S 0 12169 1 0 60 20 8a5 152 hft/0 0:00 lmgrd222801 S 0 12938 12169 0 60 20 16aa 352 hft/0 0:05 CFDSd40801 S 0 10342 8542 0 60 20 357a 196 - 0:11 nfsd

The process name (or the command name) is shown in the last col-umn. Other useful parameters are the process identification number(PID), the nice value (NI) which determines the priority of the process,


and the cpu time (TIME) used up by the task. In the above examplelisting, sendmail is the mail program, lmgrd is the license manager dae-mon, CFDSd is the license server for FLOW3D program, nfsd is the NFSdaemon; all of these tasks are run by root with a user identification num-ber (UID) of 0. Note that the ps command itself is a task.

C.3.2 Suspending jobs or processes

If you started a process like "emacs" or "matlab" and you want to sus-pend that task and return to the shell you can do so with the key se-quence

user@machine:dir> ctrl-z

The PID number is displayed at that time. Even if you did not note itdown, you can find a list of all suspended jobs with the command

user@machine:dir> jobs

To resume the job, enter

user@machine:dir> fg %n

where n is the job number produced by the jobs command (and not thePID number!). The "fg" command brings a job to the foreground.

C.3.3 Terminating jobs or processes - the "kill" command

If you started a process in error and want to terminate it, you can use the"kill" command. You need to find out the PID number of the processusing "ps command.

user@machine:dir> kill -9 PID

Except for the super user (or root), one can terminate only those pro-cesses that belong to (or initiated by) individual users.

C.3.4 Initiating background jobs - the "nohup" command

The nohup command runs a command or a script ignoring all hangupsand QUIT signals. Use this command to run programs in the backgroundafter logging off. To run a nohup command in the background, add an& (ampersand) to the end of the command. Its syntax is:


user@machine:dir> nohup command or script &

When used in its simplest form as above, any output that would normallyappear on the screen will be saved in a file named nohup.out in thecurrent directory. Wait before logging off because the nohup commandtakes a moment to start the command or script you specified. If youlog off too quickly, your command or script may not run at all. Onceyour command or script starts, logging off does not affect it. Note thatin order to run a script, the script file must have execute permission.

C.3.5 Script files & scheduling jobs - the "at" command

If you wish to schedule a job to begin at a specified time (typically latenight), use the at command. The job should be constructed in the formof a script file. For example a file named test.bat contains the followinglines and has execute permission enabled with the chmod command.

matlab >out « ’eof’secant(’ass3a’,[0,1],eps,1)fzero(’ass3a’,0.5,eps,1)quiteof

In the above script we start MATLAB in the first line and redirect anyoutput generated by MATLAB for the standard output ( i.e., screen duringan interactive session) to a file named out. During an interactive session,MATLAB expects commands from the standard input ( i.e., the keyboard).Such inputs are now taken from the script file itself as seen in the nextfew lines where we execute some MATLAB functions and finally quitMATLAB.

The contents of such a script file can be executed interactively whilelogged in to a machine by simply entering the file name as

user@machine:dir> test.bat

The above example illustrates script programming in its simplest form.It is possible to write very sophisticated scripts in the Kron shell or anyother shell. When you invoke MATLAB, for example, with the commandmatlab, you are actually executing a powerful script. Browse throughthe matlab script file using

user@machine:dir> pg /usr/local/matlab/bin/matlab

C.4. LIST OF OTHER USEFUL AIX COMMAND C.12

to appreciate the power of script programming.Once a script file has been constructed, you can schedule it to be ex-

ecuted at a specified time using the at command as follows

user@machine:dir> at 21:00 scrit file

which will begin executing the script file at 21:00 hours. To examine alisting of all the jobs scheduled use,

user@machine:dir> at -l

To remove a job that you have accidentally submitted, you can use,

user@machine:dir> at -r job number

C.4 List of other useful AIX command

A list of less frequently used AIX commands is given in Table C.1. Youcan use either the man page feature with


or the

user@machine:dir> info

command which starts the InfoExplorer to find out about the syntax andusage of these and other commands. The directory /usr/bin containsall of the Unix commands.

C.4. LIST OF OTHER USEFUL AIX COMMAND C.13

command Functionat to schedule a task to start at a given timecat to list a filecd to change directorydiff compare two filesdosformat formats a floppy diskette using MS-DOS standardsdosread copies a DOS file from a floppydoswrite copies a unix file to a DOS formatted floppyfind find a fileinfo InfoExplorer - online documentationksh start a Kron Shellmake a powerful UNIX make facilitymail read mailmkdir create a directoryman display online manual pageslogout logout of current AIX sessionlpq list the queue of print jobslpr send a print job to a network printerlprm remove a print job from a queuenice control job prioritynohup Don’t kill a process upon logoutpg display a file, one page at a timeping to check if another machine is alivepwd display present working directoryrlogin remote login to another machinercp remote copy files from one host to other

need to have ".rhosts" file setuprm remove (delete) filesrmdir remove directoriesrsh execute a command on a remote machine

need to have ".rhosts" file setuprusers list remote users in the local area networkscript logs a terminal session to a filetalk talk to another user currently signed ontar archive filestelnet connect to remote hostswhoami find out the current userxinit start X-serverxlc c-compilerxlC c++ compilerxlf Fortran compiler

Table C.1: List of other useful AIX command

Métodos Numéricos En Ing. Química Para programar

Documents