LU Decomposition 1. Introduction Another way of solving a system of equations is by using a factorization technique for matrices called LU decomposition.

LU Decomposition

1

Introduction• Another way of solving a system of equations is by using a

factorization technique for matrices called LU decomposition. • This factorization is involves two matrices, one lower triangular

matrix and one upper triangular matrix.• LU factorization methods separate the time-consuming

elimination of the matrix [A] from the manipulations of the right-hand-side [b].

• Once [A] has been factored (or decomposed), multiple right-hand-side vectors can be evaluated in an efficient manner.

2

LU Decomposition

and

Where,

3

mm

m

m

u

uu

uuu

U

00

0 222

11211

mmmm lll

ll

l

L

21

2221

11

0

00

LUA

LU decomposition was originally derived as a decomposition of quadratic and bilinear forms. Lagrange, in the very first paper in his collected works( 1759) derives the algorithm we call Gaussian elimination. Later Turing introduced the LU decomposition of a matrix in 1948 that is used to solve the system of linear equation.

Let A be a m × m with nonsingular square matrix. Then there exists two matrices L and U such that, where L is a lower

triangular matrix and U is an upper triangular matrix.

J-L Lagrange

(1736 –1813) A. M. Turing

(1912-1954)

LU Factorization• LU factorization involves two

steps:– Factorization to decompose the

[A] matrix into a product of a lower triangular matrix [L] and an upper triangular matrix [U]. [L] has 1 for each entry on the diagonal.

– Substitution to solve for {x}

• Gauss elimination can be implemented using LU

factorization

LU Decomposition

5

LU Decomposition is another method to solve a set of simultaneous linear equations

Which is better, Gauss Elimination or LU Decomposition?

To answer this, a closer look at LU decomposition is needed.

MethodFor most non-singular matrix [A] that one could conduct Naïve Gauss Elimination forward elimination steps, one can always write it as

[A] = [L][U]

where

[L] = lower triangular matrix

[U] = upper triangular matrix

LU Decomposition

6

How does LU Decomposition work?

7

LU Decomposition

8

How can this be used?

Given [A][X] = [C]

1. Decompose [A] into [L] and [U]

2. Solve [L][Z] = [C] for [Z]

3. Solve [U][X] = [Z] for [X]

When is LU Decomposition better than Gaussian Elimination?

To solve [A][X] = [B]

Table. Time taken by methods

where T = clock cycle time and n = size of the matrix

So both methods are equally efficient.

9

Gaussian Elimination LU Decomposition

3

412

3

8 23 n

nn

T

3

412

3

8 23 n

nn

T

To find inverse of [A]

10

Time taken by Gaussian Elimination Time taken by LU Decomposition

3

412

3

8

||2

34 n

nn

T

CTCTn BSFE

3

2012

3

32

|||

23 n

nn

T

CTnCTnCT BSFSLU

n 10 100 1000 10000

CT|inverse GE / CT|inverse LU 3.28 25.83 250.8 2501

Table 1 Comparing computational times of finding inverse of a matrix using LU decomposition and Gaussian elimination.

Method: [A] Decompose to [L] and [U]

11

33

2322

131211

3231

21

00

0

1

01

001

u

uu

uuu

ULA

[U] is the same as the coefficient matrix at the end of the forward elimination step.

[L] is obtained using the multipliers that were used in the forward elimination process

Finding the [U] matrix

12

Using the Forward Elimination Procedure of Gauss Elimination

112144

1864

1525

112144

56.18.40

1525

56.212;56.225

64

RowRow

76.48.160

56.18.40

1525

76.513;76.525

144

RowRow

Step 1:

Finding the [U] Matrix

13

Step 2:

76.48.160

56.18.40

1525

7.000

56.18.40

1525

5.323;5.38.4

8.16

RowRow

7.000

56.18.40

1525

U

Matrix after Step 1:

Finding the [L] matrix

14

Using the multipliers used during the Forward Elimination Procedure

1

01

001

3231

21

56.225

64

11

2121

a

a

76.525

144

11

3131

a

a

From the first step of forward elimination

112144

1864

1525

Finding the [L] Matrix

15

15.376.5

0156.2

001

L

From the second step of forward elimination

76.48.160

56.18.40

15255.3

8.4

8.16

22

3232

a

a

Does [L][U] = [A]?

16

7.000

56.18.40

1525

15.376.5

0156.2

001

UL ?

Using LU Decomposition to solve SLEs

17

Solve the following set of linear equations using LU Decomposition

2279

2177

8106

112144

1864

1525

3

2

1

.

.

.

x

x

x

Using the procedure for finding the [L] and [U] matrices

7.000

56.18.40

1525

15.376.5

0156.2

001

ULA

Example

18

Set [L][Z] = [C]

Solve for [Z]

2.279

2.177

8.106

15.376.5

0156.2

001

3

2

1

z

z

z

2.2795.376.5

2.17756.2

10

321

21

1

zzz

zz

z

Example

19

Complete the forward substitution to solve for [Z]

735.0

21.965.38.10676.52.279

5.376.52.279

2.96

8.10656.22.177

56.22.177

8.106

213

12

1

zzz

zz

z

735.0

21.96

8.106

3

2

1

z

z

z

Z

Example

735.07.0

21.9656.18.4

8.106525

3

32

321

a

aa

aaa

20

Set [U][X] = [Z]

Solve for [X] The 3 equations become

7350

2196

8106

7.000

56.18.40

1525

3

2

1

.

.

.

x

x

x

Example

21

From the 3rd equation

050170

7350

735070

3

3

3

.a.

.a

.a.

Substituting in a3 and using the second equation

219656184 32 .a.a.

701984

0501561219684

5612196

2

2

32

.a.

...a

.

a..a

Example

22

Substituting in a3 and a2 using the first equation

8106525 321 .aaa

Hence the Solution Vector is:

050.1

70.19

2900.0

3

2

1

a

a

a

2900025

050170195810625

58106 321

.

...

aa.a

Finding the inverse of a square matrix

23

The inverse [B] of a square matrix [A] is defined as

[A][B] = [I] = [B][A]

Finding the inverse of a square matrix

24

How can LU Decomposition be used to find the inverse?

Assume the first column of [B] to be [b11 b12 … bn1]T

Using this and the definition of matrix multiplication

First column of [B] Second column of [B]

0

0

1

1

21

11

nb

b

b

A

0

1

0

2

22

12

nb

b

b

A

The remaining columns in [B] can be found in the same manner

Example: Inverse of a Matrix

25

Find the inverse of a square matrix [A]

112144

1864

1525

A

7000

561840

1525

153765

01562

001

.

..

..

.ULA

Using the decomposition procedure, the [L] and [U] matrices are found to be


26

Solving for the each column of [B] requires two steps

1)Solve [L] [Z] = [C] for [Z]

2)Solve [U] [X] = [Z] for [X]

Step 1:

0

0

1

15.376.5

0156.2

001

3

2

1

z

z

z

CZL

This generates the equations:

05.376.5

056.2

1

321

21

1

zzz

zz

z


27

Solving for [Z]

23

5625317650

537650

562

15620

5620

1

213

12

1

.

...

z.z.z

.

.

z. z

z

23

562

1

3

2

1

.

.

z

z

z

Z


28

Solving [U][X] = [Z] for [X]

3.2

2.56

1

7.000

56.18.40

1525

31

21

11

b

b

b

2.37.0

56.256.18.4

1525

31

3121

312111

b

bb

bbb


29

Using Backward Substitution

04762.0

25

571.49524.05125

51

9524.08.4

571.4560.156.28.4

560.156.2

571.47.0

2.3

312111

3121

31

bbb

bb

b So the first column of the inverse of [A] is:

571.4

9524.0

04762.0

31

21

11

b

b

b


30

Repeating for the second and third columns of the inverse

Second Column Third Column

0

1

0

112144

1864

1525

32

22

12

b

b

b

000.5

417.1

08333.0

32

22

12

b

b

b

1

0

0

112144

1864

1525

33

23

13

b

b

b

429.1

4643.0

03571.0

33

23

13

b

b

b


31

The inverse of [A] is

429.1000.5571.4

4643.0417.19524.0

03571.008333.004762.01A

To check your work do the following operation

[A][A]-1 = [I] = [A]-1[A]

32

Cholesky Decomposition

Cholesky died from wounds received on the battle field on 31 August 1918 at 5 o'clock in the morning in the North of France. After his death one of his fellow officers, Commandant Benoit, published Cholesky's method of computing solutions to the normal equations for some least squares data fitting problems published in the Bulletin géodesique in 1924. Which is known as Cholesky Decomposition

Cholesky Decomposition: If A is a real, symmetric and

positive definite matrix then there exists a unique lower triangular matrix L with positive diagonal element such that .

TLLA

Andre-Louis Cholesky

1875-1918

Cholesky Factorization

• Symmetric systems occur commonly in both mathematical and engineering/science problem contexts, and there are special solution techniques available for such systems.

• The Cholesky factorization is one of the most popular of these techniques, and is based on the fact that a symmetric matrix can be decomposed as [A]= [U]T[U], where T stands for transpose.

• The rest of the process is similar to LU decomposition and Gauss elimination, except only one matrix, [U], needs to be stored.

33

LU Decomposition 1. Introduction Another way of solving a system of equations is by using a factorization technique for matrices called LU decomposition.

Documents