Dependence Testing Optimizing Compilers for Modern Architectures, Chapter 3 Allen and Kennedy Presented by Rachel Tzoref and Rotem Oshman
Mar 23, 2016
Dependence TestingOptimizing Compilers for Modern Architectures, Chapter 3Allen and Kennedy
Presented by Rachel Tzoref and Rotem Oshman
DO i1 = L1, U1
DO i2 = L2, U2
...DO in = Ln, Un
S1 A(f1(i1,...,in),...,fm(i1,...,in)) = ...S2 ... = A(g1(i1,...,in),...,gm(i1,...,in))ENDDO...ENDDOENDDO
The General Problem
Dependence exists ,9 iteration vectors such that
fj()=gj() for all j,1· j · m
Basic Definitions
Index: A loop variable Subscript: The pair of expressions that appear in
a certain coordinate in a pair of array references A(i,j,m) = A(i,k,j) + C‹i,i› is the first subscript‹j,k› is the second subscript‹m,j› is the third subscript
Distance and Direction Vectors - ReminderDO I=1, NDO J=1, M
DO K=1, LA(I+1,J,K-1) = A(I,J,K)+10
ENDDOENDDO
ENDDODistance Vector: (1, 0, -1)
Direction Vector(> ,= ,<) :
What can we do?INPUT N;DO i = 1, 100DO j = 1, 100
A(N+(i+j)3) = A(i(j+N)+(ji)+(j2+i2))ENDDO
ENDDO
The general case – the problem is undecidable during compile time
What can we do? Most subscripts are polynomialsDO i = 1, 100
DO j = 1, 100 DO k = 1, 100
A(i2+j2,4¢k3) = A(k2,i4) ENDDOENDDO
ENDDO
Problem reduced to solving a system of Diophantine equations
Too complex: consider only linear subscript expressions
What can we do?DO i = 1, 100DO j = 1, 100 DO k = 1, 100
A(i+j,4¢k) = A(k,i)ENDDO
ENDDOENDDO
Finding integer solutions to a system of linearDiophantine Equations is NP-Complete
Testing may have to be imprecise
Our Goal - Conservative Testing Result = “no dependence” ) no dependence Result = “dependence” + direction vector(s) )
dependence may or may not exist
Code is always correct, but may be less than optimal
Directions output by testing
Directions where dependenceexists
What About Nonlinear Subscripts?
Can’t analyze something ) assume dependence
Example:DO i = 1, 50
DO j = 51, 100A(i3+j2,i) = A(j4,j)
ENDDOENDDO
May still prove independence or limit possible direction vectors
A(5,i+1,j) = A(1,i,k) + C
ZIV Subscript: no indices SIV Subscript: one index MIV Subscript: more than one index
Definition: Subscript Complexity
Definition: Separability A subscript is separable if its indices do not occur
in other subscripts
A(i+1,j) = A(k,j) + CBoth subscripts are separable
If two different subscripts contain the same index they are coupled
A(i,j,j) = A(i+1,j,k) + CSecond and third subscripts are coupled
Coupled Subscript Groups
Why are they important?DO i = 1, 100
S1 A(i+1,i) = B(i) + CS2 D(i) = A(i,i) ¢ E
ENDDO
Coupling can cause imprecision in dependence testing
> , <
A(j-1,i+1,i+3,k) = A(j+2,j,i,k+1)
Building Coupled Groups
Test each group separately
ki+1 jj-1 i+3 ij+2 k+1> , <> , < > , <
Subscript Tests Input:
Separable subscript OR coupled group Loop bounds
Output: “No dependence”, OR A set of direction vectors in which dependence may exist
Merging Direction VectorsDO i = 1, 100DO j = 1, 99
A(i+1,j) = B(i)+CD(j) = A(i,100-j)
ENDDOENDDO
Cartesian product:
Direction Vectors:
General Algorithm OverviewInput: a pair of array references and loop bounds.1. Partition subscripts into separable and coupled groups.2. Classify each subscript as ZIV, SIV or MIV.3. For each separable subscript: apply single subscript
test according to its complexity. If independence established: output “no dependence” and halt.
4. For each coupled group: apply multiple subscript test. If independence established: output “no dependence” and halt.
5. Merge all direction vectors computed in the previous steps into a single set of direction vectors.
Roadmap: Part I Single Subscript Tests:
ZIV Strong SIV Weak SIV
Weak-Zero SIV Weak-Crossing SIV
SIV Tests in Complex Iteration Spaces
Roadmap: Part II MIV Tests:
GCD Test Banerjee Inequality Trapezoidal Banerjee Inequality
Coupled Subscripts: the Delta Test Empirical Results
Roadmap: Part I Single Subscript Tests:
ZIV Strong SIV Weak SIV
Weak-Zero SIV Weak-Crossing SIV
SIV Tests in Complex Iteration Spaces
ZIV TestDO j = 1, 100A(e1) = A(e2) + B(j)
ENDDO
e1,e2 are loop invariant expressions e1=5, e2=4 ) no dependence e1=L, e2=L+1 ) no dependence e1=L, e2=K ) dependence
If (e1-e2) is a non-zero constant then no dependence exists
INPUT L,KDO i = 1, N
S1B(i+1) = A(L+K) + X(i)S2A(L) = B(i) + CENDDO
The values of L and K are unknown If K0 there is no dependence from S2 to S1 K0 is called the Breaking Condition
ZIV Test & Breaking Conditions
IF (K0) THENDO i = 1, N B(i+1) = A(L+K) + X(i)ENDDO DO i = 1, N A(L) = B(i) + CENDDO
ELSEDO i = 1, N B(i+1) = A(L+K) + X(i)A(L) = B(i) + CENDDO
ENDIF
ZIV Test & Breaking Conditions
IF (K0) THENB(2:N+1) = A(L+K) + X(1:N)DO i = 1, N
A(L) = B(i) + CENDDO
ELSEDO i = 1, N
B(i+1) = A(L+K) + X(i)A(L) = B(i) + C
ENDDOENDIF
ZIV Test & Breaking Conditions
Roadmap: Part I Single Subscript Tests:
ZIV Strong SIV Weak SIV
Weak-Zero SIV Weak-Crossing SIV
SIV Tests in Complex Iteration Spaces
Strong SIV
Examples:
Strong SIV Test
Dependence exists if d is an integer and
A(m1)
Strong SIV Test
DO i = 1, N A(i+2¢N) = A(i+N) + C
ENDDO
N < N-1 ) No dependence
Strong SIV & Breaking ConditionsDO i = 1, L
A(i+N) = A(i) + BENDDO N<L-1 is the breaking condition
IF (N¸L) THENA(N+1:N+L) = A(1:L) + B
ELSEDO i = 1, L
A(i+N) = A(i) + BENDDO
ENDIF
Roadmap: Part I Single Subscript Tests:
ZIV Strong SIV Weak SIV
Weak-Zero SIV Weak-Crossing SIV
SIV Tests in Complex Iteration Spaces
Weak SIV
Examples:
Dependence exists for all integer solutions to the following linear Diophantine equation:
Roadmap: Part I Single Subscript Tests:
ZIV Strong SIV Weak SIV
Weak-Zero SIV Weak-Crossing SIV
SIV Tests in Complex Iteration Spaces
Weak-Zero SIV Test
Dependence equation (assume a2=0):
Dependence exists if is an integer and is within the loop bounds
Weak-Zero SIV Test
Dependence always caused by a particular iteration
DO i = 1, NA(i,N) = A(1,N) + A(N,N)
ENDDO Dependence caused by iterations 1 and N. These iterations can be peeled from the loop:A(1,N) = A(1,N) + A(N,N)DO i = 2, N-1
A(i,N) = A(1,N) + A(N,N)ENDDOA(N,N) = A(1,N) + A(N,N)
Weak-Zero SIV & Loop Peeling
DO i = 1, NA(i,N) = A(1,N) + A(N,N)
ENDDO Dependence caused by iterations 1 and N. These iterations can be peeled from the loop:A(1,N) = A(1,N) + A(N,N)
A(N,N) = A(1,N) + A(N,N)
Weak-Zero SIV & Loop Peeling
A(2:N-1) = A(1,N) + A(N,N)
Roadmap: Part I Single Subscript Tests:
ZIV Strong SIV Weak SIV
Weak-Zero SIV Weak-Crossing SIV
SIV Tests in Complex Iteration Spaces
Weak-Crossing SIV
Line of symmetry
A(m1)A(m2)
A(m3)
When does dependence exist?
Weak-Crossing SIV TestLine of symmetry
Dependence exists if the line of symmetry is: • within the loop bounds ) L · (c2-c1)/2a · U
A(m1)
Weak-Crossing SIV TestLine of symmetry
A(m1)
Dependence exists if the line of symmetry is: • within the loop bounds ) L · (c2-c1)/2a · U• an integer or has a non-integer part equal to ½ )
The line of symmetry is halfway between two integers
Weak-Crossing SIV & Loop SplittingDO i = 1, N
A(i) = A(N-i+1) + CENDDO
Line of symmetry: i=(N+1)/2
DO i = 1, (N+1)/2A(i) = A(N-i+1) + C
ENDDODO i = (N+1)/2+1, N
A(i) = A(N-i+1) + CENDDO
DO i = 1, NA(i) = A(N-i+1) + C
ENDDO Line of symmetry: i=(N+1)/2
Weak-Crossing SIV & Loop Splitting
A(1:(N+1)/2) = A(N:(N+1)/2)+C
A((N+1)/2+1:N) = A((N+1)/2-1:1)+C
Roadmap: Part I Single Subscript Tests:
ZIV Strong SIV Weak SIV
Weak-Zero SIV Weak-Crossing SIV
SIV Tests in Complex Iteration Spaces
Complex Iteration Spaces Triangular: One of the loop bounds is a function of at
least one other loop index
1
1
2
2
5
DO i = 1, 5 DO j = 1, i
3 4 5
4
3
DO i = 1, 5DO j = i-1, 2¢i-1
Complex Iteration Spaces Trapezoidal: Both loop bounds are functions of at
least one other loop index
1
1
2
2
5
7
3 4
543
6
8
Complex Iteration Spaces SIV tests can be extended to such loops, with
some loss of precision
Strong SIV in Complex Iteration Spaces
DO i = 1, NDO j = L0+L1¢i, U0+U1¢iS1 A(j+d) = …S2 … = A(j) + BENDDOENDDO
No dependence if )
Unless inequality holds for all values of i in the loop ) assume dependence
Index Set SplittingDO i = 1, 100DO j = 1, i
A(j+20) = A(j) + BENDDO
ENDDO
For
there is no dependence
Use i>21 as a breaking conditionDO i = 1, 20DO j = 1, i
A(j+20) = A(j) + BENDDO
ENDDODO i = 21, 100DO j = 1, i
A(j+20) = A(j) + BENDDO
ENDDO
Index Set Splitting
Use i>21 as a breaking condition
DO i = 21, 100DO j = 1, i
A(j+20) = A(j) + BENDDO
ENDDO
Index Set Splitting
PARALLEL DO i = 1, 20A(21:20+i) = A(1:i) + B
ENDDO
Weak-Zero SIV in Complex Iteration SpacesDO i = 1, NDO j = L0+L1¢i, U0+U1¢iS1 A(c) = …S2 … = A(j) + BENDDOENDDO
No dependence if )
Unless inequality holds for all values of i in the loop ) assume dependence
Coupling in Complex Iteration SpacesDO i = 1, 100DO j = 1, i
A(j+20,i) = A(j,19) + BENDDO
ENDDO
Testing each subscript separately causes imprecision: First subscript: dependence only for i ¸ 21 Second subscript: dependence only for i=19
Subscripts are actually coupled