DIRECT METHODS TO THE SOLUTION OF LINEAR EQUATIONS SYSTEMS Lizeth Paola Barrero Riaño Numerical Methods – Industrial Univesity of Santander
DIRECT METHODS TO THE SOLUTION OF LINEAR EQUATIONS SYSTEMS
Lizeth Paola Barrero Riaño
Numerical Methods – Industrial Univesity of Santander
•Symmetric Matrix•Transposes Matrix•Determinant•Upper Triangular Matrix•Lower Triangular Matrix•Banded Matrix•Augmented Matrix•Matrix Multiplication
BASIC FUNDAMENTALS
Matrix
A matrix consists of a rectangular
array of elements represented by a single symbol. As depicted in Figure
[A] is the shorthand
notation for the matrix and
designates an individual
element of the matrix.
A horizontal set of elements is called a row (i) and a vertical
set is called a column (j).
11 12 13 1
21 22 23 2
1 2 3
...
...
...
m
m
n n n nm
a a a a
a a a aA
a a a a
Column 3
Row 2
Symmetric Matrix
It is a square
matrix in which the
elements are
symmetric about the main diagonal
is a Symmetric Matrix if , nm ij ij jiA a a a ij
Example:
If A is a symmetric matrix, then:
a. The product is defined and is a symmetric matrix.
b. The sum of symmetric matrices is a symmetric matrix.
c. The product of two symmetric matrices is a symmetric matrix if the matrices commute
tA A
Scalar, diagonal and
identity matrices, are
symmetric matrices.
Transposes Matrix
Let any matrix
A=(aij) of mxn order, the matrix B=(bij) de
order nxm is the A
transpose if the A rows are the B columns .
This operation is
usually denoted by At = A' = B
t A such that , i,jm n ij n m ij ji ijA a b b a
Example:
2 3 3 2
1 2 2 1
1 01 2 2
2 40 4 3
2 3
22 9 B
9
t
t
A A
B
a. ( )t tA Ab. ( )t t tA B A B c. ( )t t tA B B A d. ( ) , t tk A k A k
Properties:
Determinant
Given a square matrix A of n size,
its determinant is defined as the sum of
the any matrix line elements
product (row or column) chosen, by
their correspondin
g attachments.
Example:
2 4 5
To the matrix 6 7 3
3 0 2
A
4 5 2 5 2 4det( ) 3 +0 +2
7 3 6 3 6 7
=3 (-12-35)+0 (-(6-30))+2 (-14-24)=-141+0-76
=-217
A
applying the definition, if we choose the third row is:
Determinant Properties
a. If a matrix has a line (row or column) of zeros, the determinant is zero.
b. If a matrix has two equal rows or proportional, the determinant is null
c. If we permute two parallels lines of a square matrix, its determinant changes sign.
d. If we multiply all elements of a determinant line by a number, the determinant is multiplied by that number.
e. If a matrix line is added another line multiplied by a number, the determinant does not change.
f. The determinant of a matrix is equal to its transpose,g. If A has inverse matrix, A-1, it is verified that:
1 1det( )
det( )A
A
Upper and Lower Triangular Matrix
Upper Triangular Matrix Lower Triangular Matrix
is Upper Triangular if 0 , ij ijA a a i j
2 0 2
0 1 0
0 0 3
A
Example:
It is a square matrix in which all the items under the main
diagonal are zero.
is Lower Triangular if 0 , n n ij ijA a a i j
Example:
22 0 0
2 1 0
1 1 0
D
It is a square matrix in which all the elements above the
main diagonal are zero.
Banded Matrix
A band matrix is a sparse matrix whose non-zero entries are confined to a diagonal band,
comprising the main diagonal and zero or more diagonals on either side.
4 0 0 0 0 8 7 6 0 02 0 0 0
7 8 1 0 0 9 3 0 2 00 1 0 0
D= M=0 0 5 2 0 3 1 8 9 100 0 5 0
0 0 1 3 5 0 0 3 5 80 0 0 7
0 0 0 3 4 0 0 7 4 0
C
DiagonalTriadiagonal Pentadiagonal
Example:
Augmented Matrix
It is called extended or augmented
matrix that is formed by
the coefficient matrix and
the vector of independent terms, which are usually separated
with a dotted line
1 3 2 4
2 0 1 , B= 3
5 2 2 8
The augmented matrix
is represented as follows:
1 3 2 4
2 0 1 3
5 2 2 1
A
A B
A B
Example:
Matrix Multiplication
To define A ⋅ B is necessary that the number of columns in the first matrix coincides with the number of rows in the second matrix. The product order
is given by the number of rows in the first matrix per
the number of columns in the second matrix.
That is, if A is mxn order and B is nxp order, then C = A ⋅
B is mxp order.
11 1 11 1
1 1
11 11 1 1 11 1 1 1
1 11 1 1 1
y B=
then
n p
m mn n np
n n p n p
m mn n m p mn np
a a b b
A
a a b b
a b a b a b a b
AB
a b a b a b a b
.
y ,m n n p ijA aij B b
m pA
1
.n
ij ik kjk
c a b
Given the product A.B is
another matrix in which each Cij is the nth
row product of A per the nth column of B, namely the
element
Matrix Multiplication
1 3 5 1 3 4 0
0 0 1 B= 0 8 9 1
4 1 2 7 5 5 1
36 4 56 8
7 5 5 1
18 14 35 3
A
C A B
Graphically:
Example:
Solution of Linear Algebraic Equation
Linear algebra is one of the corner stones of
modern computational mathematics. Almost all numerical schemes
such as the finite element method and
finite difference method are in act techniques
that transform, assemble, reduce, rearrange, and/or approximate the
differential, integral, or other types of
equations to systems of linear algebraic
equations.
A system of linear algebraic equations can be expressed as
11 12 1 1 1
21 22 2 2 2
1 2
where a and b are constants,
i=1,2,..., , 1,2,..., .
n
n
m m nm m m
ij i
a a a x b
a a a x b
a a a x b
m j n
Solution of Linear Algebraic Equation
Or:AX=B
Solving a system with a coefficient
matrix is equivalent to finding
the intersection point(s) of all m
surfaces (lines) in an n dimensional space.
If all m surfaces happen to pass through a single point then the
solution is unique
• If the intersected part is a line or a surface, there are an infinite number of solutions, usually expressed by a particular solution added to a linear combination of typically n-m vectors. Otherwise, the solution does not exist.
• In this part, we deal with the case of determining the values x1, x2,…,xn that simultaneously satisfy a set of equations.
mxnA
Small Systems of Linear Equations
1. Graphical Method2. Cramer’s Rule3. The Elimination of Unknows
1. Graphical Method
When solving a system with two
linear equations in two variables, we are looking for the
point where the two lines cross.
This can be determined by
graphing each line on the same
coordinate system and estimating the
point of intersection.
When two straight lines are
graphed, one of three possibilities may result:
When two lines cross in exactly one point, the
system is consistent and independent
and the solution is the one
ordered pair where the two lines cross. The coordinates of
this ordered pair can be
estimated from the graph of the
two lines:
Case 1
Independent system:
one solution point
Graphical Method
This graph shows two distinct lines that are parallel.
Since parallel lines never cross, then there can be no
intersection; that is, for a system of
equations that graphs as parallel lines, there can be no solution. This is
called an "inconsistent"
system of equations, and it has no solution.
Case 2
Inconsistent system:
no solution andno
intersection point
Graphical Method
This graph appears to show
only one line. Actually, it is the same line drawn
twice. These "two" lines,
really being the same line,
"intersect" at every point along their
length. This is called a
"dependent" system, and the "solution" is the
whole line.
Case 3
Dependent system:
the solution is the
whole line
Graphical Method
ADVANTAGES:The graphical method is good because it clearly illustrates the principle involved.
DISADVANTAGES:•It does not always give us an exact solution. •It cannot be used when we have more than two variables in the equations.
Graphical Method
For instance, if the lines cross at a shallow angle it can be just about impossible to tell where the lines cross:
Example
Solve the following system by graphing.
2x – 3y = –24x + y = 24
First, we must solve each equation for "y=", so we can graph easily:
2x – 3y = –2 4x + y = 242x + 2 = 3y y = –4x + 24(2/3)x + (2/3) = y
Graphical Method
x y = (2/3)x + (2/3) y = –4x + 24
–4–8/3 + 2/3 = –6/3 = –
216 + 24 = 40
–1 –2/3 + 2/3 = 0 4 + 24 = 28
2 4/3 + 2/3 = 6/3 = 2 –8 + 24 = 16
510/3 + 2/3 = 12/3 =
4–20 + 24 = 4
816/3 + 2/3 = 18/3 =
6–32 + 24 = –
8
Graphical Method
Solution: (x, y) = (5, 4)
The second line will be easy to graph using just the slope and intercept, but it is necessary a T-chart for the first line.
Cramer’s Rule
Cramer’s rule is another technique that is best
suited to small numbers of equations. This rule
states that each unknown in a system of
linear algebraic equations may be
expressed as a fraction of two determinants
with denominator D and with the numerator obtained from D by
replacing the column of coefficients of the
unknown in question by the constants b1, b2,
… ,bn.
1 12 13
2 22 23
3 32 331
b a a
b a a
b a ax
D
For example, x1 would be computed as
Example
Solution:
We begin by setting
up and evaluating the three determina
nts:
Use Cramer’s Rule to solve the system:
5x – 4y = 26x – 5y = 1
1 1
2 2
5 4 5 5 6 4 25 4 1
6 5
a bD
a b
1 1
2 2
2 4 2 5 1 4 10 4 6
1 5
c bDx
c b
1 1
2 2
5 4 5 1 6 2 5 12 7
6 1
a cDy
a c
From Cramer’s Rule, we have:
6 6
1
Dxx
D
and
7 7
1
Dyx
D
The solution is (6,7)
Cramer’s Rule does not apply if D=0. When D=0 , the system is either inconsistent or dependent.
Another method must be used to solve it.
Example
The Elimination of Unknows
The basic strategy is to multiply the equations by
constants so that of the unknowns will
be eliminated when the two equations are combined. The result is a single
equation that can be solved for the
remaining unknown. This value can then be substituted into
either of the original equations to
compute the other variable.
The elimination of unknowns by combing equations is an algebraic approach that can be illustrated for a set of two equations:
11 1 12 2 1
21 1 22 2 2
1
2
a x a x b
a x a x b
For example, these equations might be multiplied by a21 and a11 to give
11 21 1 12 21 2 1 21
21 11 1 22 11 2 2 11
3
4
a a x a a x ba
a a x a a x b a
Subtracting Eq. 3 from 4 will, therefore, eliminate the xt term from the equations to yield
The Elimination of Unknows
22 11 2 12 21 2 2 11 1 21a a x a a x b a ba
Which can be solve for
11 2 21 1
211 22 12 21
a b a bx
a b a a
This equation can then be substituted into Eq. 1, which can be solved for
22 1 12 2
111 22 12 21
a b a bx
a b a a
The Elimination of UnknowsNotice that these equations follow directly from Cramer’s rule, which states
EXAMPLEUse the elimination of unknown to solve,
1 12
2 22 1 22 12 21
11 12 11 22 12 21
21 22
11 1
21 2 11 2 1 212
11 12 11 22 12 21
21 22
b a
b a ba a bx
a a a a a aa a
a b
a b a b bax
a a a a a aa a
1 2
1 2
1
2
3 2 18
2 2
Solution
2(18) 2(2)4
3(2) 2( 1)
3(2) ( 1)183
3(2) 2( 1)
x x
x x
x
x
Gaussian Elimination
Gaussian Elimination is
considered the workhorse of
computational science for the
solution of a system of linear equations.
Karl Friedrich Gauss, a great 19th
century mathematician, suggested this
elimination method as a part of his
proof of a particular theorem.
Computational scientists use this “proof” as a direct
computational method.
Gaussian Elimination is a systematic application of elementary row operations to a system of linear equations in order to convert the system to upper triangular form.
Once the coefficient matrix is in upper triangular form, we use back substitution to find a solution.
Gaussian Elimination
The general procedure for Gaussian Elimination can be summarized in the
following steps: 1. Write the augmented matrix for the system of
linear equations.
2. Use elementary row operations on the augmented matrix [A|b] to transform A into upper triangular form. If a zero is located on the diagonal, switch the rows until a nonzero is in that place. If you are unable to do so, stop; the system has either infinite or no solutions.
3. Use back substitution to find the solution of the problem.
Gaussian Elimination
1. Write the augmented matrix for the system of linear equations.
2 4
2 6
2 7
y z
x y z
x y z
0 2 14
1 1 2 6
2 1 17
2.Use elementary row operations on the augmented matrix [A|b] to transform A into upper triangular form.
2
1
0 2 14 ( )
1 1 2 6 ( )
2 1 17
r
r
3 1
1 1 2 6
0 2 14
2 1 17 ( ) ( 2 )r r
Change row 1 to row 2
and vice versa
Example
Gaussian Elimination
Notice that the original coefficient matrix had a “0”
on the diagonal in row 1. Since we needed to use
multiples of that diagonal element to eliminate the
elements below it, we switched two rows in order to move a nonzero element into that position. We can use the same technique
when a “0” appears on the diagonal as a result of calculation. If it is not
possible to move a nonzero onto the diagonal by
interchanging rows, then the system has either
infinitely many solutions or no solution, and the
coefficient matrix is said to be singular.
3 2
1 1 2 6 1 1 2 6
0 2 1 4 0 2 1 4
0 1 3 5 1 5 30 0( ) ( ) 22
r r
Since all of the nonzero elements are now located in the “upper triangle” of the matrix, we have completed the first phase of solving a system of linear equations using Gaussian Elimination.
Gaussian EliminationThe second and final phase of Gaussian Elimination is back substitution. During this phase, we solve for the values of the unknowns, working our way up from the bottom row.
3. Use back substitution to find the solution of the problem
1 1 2 6
0 2 1 4
5 30 0 2
The last row in the augmented matrix represents the equation:
5 63 z=
2 5z
Gaussian Elimination
4 4 6 5 72 4
2 2 5z
y z y y
117 62 6 x=6-y-2z=6- 2 5 5 5x y z x
The second row of the augmented matrix represents the equation:
Finally, the first row of the augmented matrix represents the equation
Gaussian-Jordan EliminationAs in Gaussian
Elimination, again we are transforming
the coefficient matrix into
another matrix that is much
easier to solve, and the system represented by
the new augmented
matrix has the same solution
set as the original system
of linear equations.
In Gauss-Jordan Elimination, the goal is to transform the coefficient matrix into a diagonal matrix, and the zeros are introduced into the matrix one column at a time. We work to eliminate the elements both above and below the diagonal element of a given column in one pass through the matrix.
Gaussian-Jordan Elimination
The general procedure for Gauss-Jordan Elimination can be summarized in the following steps:
1. Write the augmented matrix for the system of linear equations.
2. Use elementary row operations on the augmented matrix [A|b] to transform A into diagonal form. If a zero is located on the diagonal, switch the rows until a nonzero is in that place. If you are unable to do so, stop; the system has either infinite or no solutions.
3. By dividing the diagonal element and the right-hand-side element in each row by the diagonal element in that row, make each diagonal element equal to one.
Gaussian-Jordan Elimination
We will apply Gauss-Jordan Elimination to the same example that was used to demonstrate Gaussian Elimination
1-Write the augmented matrix for the system of linear equations.
Example
2 4
2 6
2 7
y z
x y z
x y z
0 2 14
1 1 2 6
2 1 17
2
1
0 2 14 ( )
1 1 2 6 ( )
2 1 17
r
r
3 1
1 1 2 6
0 2 14
2 1 17 ( ) ( 2 )r r
2. Use elementary row operations on the augmented matrix [A|b] to transform A into diagonal form.
Gaussian-Jordan Elimination
Notice that the coefficient matrix is now a diagonal matrix with ones on the diagonal. This is a special matrix called the identity matrix.
3-By dividing the diagonal element and the right-hand-side element in each row by the diagonal element in that row, make each diagonal element equal to one.
1 31 2
2 3
3 2
3 111 3 ( ) ( )( ) ( ) 1 0 5 51 1 2 6 4 1 0 02 22 14 0 2 1 4 0 2 1 4 ( ) ( ) 0 2 05 5
0 1 3 5 1 5 3 5( ) ( ) 0 0 0 0 32 2 2
r rr r
r r
r r
2
3
1111551 0 0 1 0 0
714 1 0 2 0 ( ) ( ) 0 1 05 2 55 0 0 10 0 23 6( ) ( )2 5 5
Entonces,
11 7 6x= , y= , and z=
5 5 5
r
r
LU DecompositionJust as was the case with Gauss elimination,
Lu decomposition requires pivoting to
avoid division by zero. However, to simplify
the following description, we will defer the issue of
pivoting until after the fundamental approach
is elaborated. In addition, the following explanation is limited
to a set of three simultaneous
equations. The results can be directly extended to n-
dimensional systems.method.
Linear algebraic notation can be rearranged to give
0 1A X B
Suppose that this equation could be expressed as an upper triangular system:
11 12 13 1 1
22 23 2 2
33 3 3
0 2
0 0
u u u x d
u u x d
u x d
Elimination is used to reduce the system to upper triangular form. The above equation can also be expressed in matrix notation and rearranged to give 0 3U X D
LU Decomposition
Now, assume that there is a lower diagonal matrix with 1’s on the diagonal,
21
31 32
1 0 0
1 0 4
1
L l
l l
That has the property that when Eq. 3 is premultiplied by it, Eq. 1 is the result. That is,
5L U X D A X B
If this equation holds, it follows from the rules for matrix multiplication that
6
and
7
L U A
L D B
LU DecompositionA two-step strategy for obtaining solutions can be based on Eqs. 3, 6 y 7.
• LU decomposition step. [A] is factored or “decomposed” into lower [L] and upper [U] triangular matrices.
• Substitution step. [L] and [U] are used to determine a solution {X} for a right-hand side {B}. This step itself consists of two steps. First, Eq. 7 is used to generate an intermediate vector {D} by forward substitution. Then, the result is substituted into Eq. 3, which can solved by back substitution for [X].
In the other hand, Gauss Elimination can be implemented in this way.
Bibliography
CHAPRA, Steven. Numerical Methods for engineers. Editorial McGraw-Hill. 2000.http://www.efunda.com/mathhttp://www.purplemath.comhttp://ceee.rice.edu/Books