4.1.2 Compressed Sparse Row Format: CSR

5

4.1.2 Compressed Sparse Row

Format: CSR

AA 2 1 5 3 4 6 7 8 9 10 11 12

JA 4 1 4 1 2 1 3 4 5 3 4 5

Values

Column indices

row 1 row 2 row 3 row 4 row 5

IA: 1 3 6 10 12 13 pointer to row i

Storage:

n and nnz,

n+nnz+1 integer,

nnz float.

6

AA 2 1 5 3 4 6 7 8 9 10 11 12

JA 4 1 4 1 2 1 3 4 5 3 4 5

Values

Column indices

row 1 row 2 row 3 row 4 row 5

IA: 1 3 6 10 12 13 pointer to row i

Code for computing c = A*b:

c = 0;for i = 1 : n

for j = IA(i) : IA(i+1)-1ci = ci + AA(j)*bJA(j) ;

endend

Indirect addressing only in b.

Columnwise compressed sparse column format

7

CSR with extracted main diagonal

AA 1 4 7 11 12 * 2 3 5 6 8 9 10

JA 7 8 10 13 14 14 4 2 4 1 4 5 3

main diagonal entries | nondiagonal entries in CSR

Pointer to begin of i-th row:

Storage: n and nnz, nnz + 1 integer, nnz + 1 float

8

AA 1 4 7 11 12 * 2 3 5 6 8 9 10

JA 7 8 10 13 14 14 4 2 4 1 4 5 3

main diagonal entries | nondiagonal entries in CSR

for i = 1 : n ci = AAi * bi ; for j = JA(i) : JA(i+1)-1 ci = ci + AAj*bJA(j) ; endend

Code for c=A*b

9

4.1.4 Diagonalwise storage

=

121100000900807600504300201

A

Diagonal number -1 0 2

Values in: ( )201,

*1211*10987654321*

−=

= IOFFDIAG

Storage: n, nd = number of diagonals, nd integers for IOFF and n*nd float

10

4.1.5 Rectangular Storage Scheme by Pressing from the Right

=

012110109876543021

1211000010900807600504300201

COEFgives

=

*54*43532421*31

JCOEF

Storage: n, n*nl integer and float

nl := nnz of longest row.

11

Code for c = A b:

c = 0;for i = 1 : n for j = 1 : nl ci = ci + COEFF(i,j) * b(JCOEFF(i,j)); endend

This format was used in ELLPACK (package of subroutines for elliptic PDE).

Coordinate form is used by MATLAB.

12

4.1.6 Jagged Diagonal FormPrestep: Sort rows after their length. Long rows first.

=⇒

=

1211000010900002018076005043

1211000010900807600504300201

PAA

Length 3

Length 2

Pointer to beginning of j-th diagonal: ( )131161=IDIAG

( )545433243121=JDIAGColumn indices:

( )851210274119163=DJValues of PA:

First jagged diagonal second jagged diag.

13

NDIAG = number of jagged diagonals

Storage: n, NDIAG, nnz float, nnz + NDIAG integer

Code for c = A b:

c = 0;for j = 1 : NDIAG for i = 1 : IDIAG(j+1) - IDIAG(j) k = IDIAG(j) + i - 1; ci = ci + DJ(k) * b(JDIAG(k)); endend

Advantages: - Always start with row 1.- More operations on neighboring data.- Less indirect addressing.- Pre-permutation changes only rows. Can be done implicitly.

Length of j-th jagged diag.

14

4.2 Sparse Matrices and Graphs4.2.1 Graph G(A) for symmetric positive definite spd A=AT >0

n x n –matrix: vertices e1, … , en with edges (ei,ek) for aik ≠ 0 , undirected Graph

=

**0****00****0**

A G(A): e1 e2 e3 e4

G(A) as directed graph: e1 e2 e3 e4

15

Adjacency Matrix for G(A) or A:

=

1101111001111011

))(( AGA can be obtained directly by replacing in A each nonzero entry by 1.

Symmetric permutations of A in the form P A PT change the ordering of the rows and columns of A simultaneously.

Therefore, the graph of P A PT can be obtained by the graph of A by renumbering the vertices:

16

Matrix A with graph G(A)

Symmetric permutation P A PT with graph G(P A PT)

G(PAPT): e1 e2 e4 e3

G(A): e1 e2 e3 e4

Example: P permutation that changes 3 4:

e1 e2 e3 e4

17

4.2.2 Matrix A nonsymmetric, G(A) directed

=⇒

=

1001111001100011

))((

*00****00**000**

AGAA

G(A): e1 e2 e3 e4

How can we characterize „good“ sparsity patterns?

„good“: Gaussian Elimination can be reduced to smaller subproblems or produces no (or small) fill-in.

18

Block Diagonal Pattern

=⇒

=

1010010110100101

))((

*0*00*0**0*00*0*

AGAA

G(A): e1 e2 e3 e4

=

=2

1

00

**00**0000**00**

AA

PAPTe1 e3 e2 e42 3:

By this permutation, A can be transformed into block diagonalform easy to solve!

=

−

−−

12

11

1

2

1

00

00

AA

AA

19

Banded Pattern

=

+−

+−

nnqnn

npn

q

p

aa

a

a

aa

A

1,

,1

1

111

=

+− nnqnn

q

ll

l

l

L

1,

1

11

=+−

nn

npn

p

u

u

uu

U

,1

111

Gauss Elimination without pivoting preserves the sparsity pattern

With pivoting the bandwidth in U grows, but remains <= p+q.

20

Overlapping Block Diagonal

=A

Pattern is preserved by Gaussian Elimination (in case of restricted pivoting).

21

Dissection Form

0

0

0

0

0

0

0

0

0

0 0

00

0

0 0

Nested (recursive) dissection:

Pattern are preservedduring GE withoutpivoting. No fill in

22

Schur Complement Reduction

=

++

=

⋅

−−

−

−

−

II

SBDBBBSBDBI

SDB

BBBB

*0

0

!

143

113

121

1

11

43

21

Write matrix B in terms of smaller submatrices:

To satisfy this equation we have to set:

12

112

1134

14

12

113

143

12

11

121 0

−−−

−−−−

−−−

−=−=⇒

+−=⇒=+

−=⇒=+

SBBDandBBBBS

SBSBBBIISBDB

SBBDSBDB

S Schur Complement

23

⋅

=

= − S

BBIBB

IBBBB

B0

0 2111343

21

Therefore, solving linear system in B is reduced to solving two smaller linear systems, one in B1 and the other in the Schur complement S.

B sparse B1 also sparse, but S usually dense!

Example: Schur complement and dissection form:

=

321

22

11

00

AGGFAFA

ASchur complement:

( )

21221

1113

2

112

11

213 00

FAGFAGA

FF

AA

GGAS

−−

−

−

−−=

=

⋅

⋅−=

24

Direct derivation of Schur complement:

3332211

23222

13111

3

2

1

3

2

1

321

22

11

00

bxAxGxGbxFxAbxFxA

bbb

xxx

AGGFAFA

=++=+=+

⇒

=

⋅

32122

122

311

111

11

xFAbAx

xFAbAx

−−

−−

−=

−=⇒

33

21221

111332

1221

1113

333321222

12231

1111

111

~

)(

)()(

bSx

bAGbAGbxFAGFAGA

bxAxFAGbAGxFAGbAG

=⇒

−−=−−⇒

=+−+−⇒

−−−−

−−−−

25

Algorithm for solving Ax=b based on Schur complement:

1. Compute S by using inv(A1) and inv(A2)

2. Solve Sx3 =b~3

3. Compute x1 and x2 by using inv(A1) and inv(A2)

The explicit computation of S can be avoided by solving the linearSystem in S iteratively, e.g. Jacobi, pcg, ….Then we need only part of S and in every iteration step we have to compute S * intermediate vector.

To achieve fast convergence, a preconditioner (approximation) for S has to be used!

Iterative methods and preconditioning will be subject of later chapters.