Branch Detection and Sparsity Estimation in Matlab Marina Menshikova & Shaun Forth Engineering Systems Department Cranfield University (DCMT Shrivenham) Shrivenham, Swindon SN6 8LA, U.K. email: [email protected]/[email protected]6th European Workshop on Automatic Differentiation INRIA Sophia-Antipolis, France, November 15-16 2007 1/ 28 Branch Detection and Sparsity Estimation in Matlab
29
Embed
Branch Detection and Sparsity Estimation in Matlab EuroAd Workshop - Marina... · Branch Detection and Sparsity Estimation in ... Sparsity estimation for Jacobian function y = y(x(1),x
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Branch Detection and Sparsity Estimation in Matlab
Marina Menshikova & Shaun Forth
Engineering Systems DepartmentCranfield University (DCMT Shrivenham)
8/ 28 Branch Detection and Sparsity Estimation in Matlab
function MADStoreBranches: part 2
% Check if this location has been seen before.if change
i = i + 1;% Record the information about new locationMADBRANCHES.info{i,1} = ST(3).file;MADBRANCHES.info{i,2} = ST(3).name;MADBRANCHES.info{i,3} = ST(3).line;MADBRANCHES.info{i,4} = op;MADBRANCHES.info{i,5} = 1;MADBRANCHES.i = i;
end
9/ 28 Branch Detection and Sparsity Estimation in Matlab
Example
1 function y = func(a,b,c)
2 v = a + b;3 if v > 24 y = 2*c;5 else6 y = v.*c;7 end
>> a = fmad(2, [1 0 0]); b = fmad(1, [0 1 0]); c =fmad(-3, [0 0 1]);>> MADSetBranches(’BranchDetection’,’ON’);The branch detection is switched on.>> func(a,b,c);>> branch = MADGetBranches(’Branches’)branch =
1
10/ 28 Branch Detection and Sparsity Estimation in Matlab
Example
1 function y = func(a,b,c)
2 v = a + b;3 if v > 24 y = 2*c;5 else6 y = v.*c;7 end
>> MADSetBranches(’BranchDetection’,’AGAIN’);The result of previous branch detection is recorded.>> func(c,a,b);>> branch = MADGetBranches(’AllBranches’)branch =
[1] [0]
11/ 28 Branch Detection and Sparsity Estimation in Matlab
12/ 28 Branch Detection and Sparsity Estimation in Matlab
Example
1 function y = func2(a,b,c)
2 y = 0;3 while abs(a) <= 104 y = y + 1;5 a = a + 1;6 end
>> MADSetBranches(’BranchDetection’,’ON’);The branch detection is switched on.>> func2(a,b,c);>> MADGetBranches(’HTML’);
13/ 28 Branch Detection and Sparsity Estimation in Matlab
Example
1 function y = func2(a,b,c)
2 y = 0;3 while abs(a) <= 104 y = y + 1;5 a = a + 1;6 end
14/ 28 Branch Detection and Sparsity Estimation in Matlab
Sparsity Estimation
Compression Techniques [Gri00, Chap. 7] can greatly improve efficiency forcalculating sparse Jacobians or Hessians.
15/ 28 Branch Detection and Sparsity Estimation in Matlab
Example (Tridiagonal Jacobian)
1 Suppose Jf(x) is tridiagonalJf(x) =
b1 c1
a2 b2 c2
a3 b3 c3
a4 b4 c4
a5 b5
.2 Choose 5× 3 seed matrix
S =
1 0 00 1 00 0 11 0 00 1 0
.
3 Then calculating Jf(x) · S byforward mode AD (or FDapprox.) gives,
Jf(x) ·S =
b1 c1 0a2 b2 c2
c3 a3 b3
b4 c4 a4
a5 b5 0
,i.e., all the Jacobian entries.
4 Tridiagonal Jacobianevaluated using 3Jacobian-vector productsnot n.
16/ 28 Branch Detection and Sparsity Estimation in Matlab
Compression Process
1 Obtain the sparsity pattern - the (i , j) locations of all possiblynon-zero entries in the Jacobian.
2 Construct a seed matrix (or matrices) to initiate forward (and/orreverse) mode [Gri00, Chap. 7], [CV98], [GMP05].
3 Calculate Jacobian products with seed(s) and extract all Jacobianentries.
17/ 28 Branch Detection and Sparsity Estimation in Matlab
Compression Process
1 Obtain the sparsity pattern - the (i , j) locations of all possiblynon-zero entries in the Jacobian.
2 Construct a seed matrix (or matrices) to initiate forward (and/orreverse) mode [Gri00, Chap. 7], [CV98], [GMP05].
3 Calculate Jacobian products with seed(s) and extract all Jacobianentries.
Here we concentrate on step 1 - sparsity estimation.
17/ 28 Branch Detection and Sparsity Estimation in Matlab
AD and Sparsity Estimation
ADIFOR’s SparselinC package [BKBC96] allows for sparse storage ofderivatives - can be used to give Jacobian sparsity for a givenindependent x.
Griewank and Mitev have considered using Bayesian update strategiesto make use of Jacobian-vector and vector-Jacobian products toiteratively deduce sparsity patterns - probing.
Giering and Kaminski [GK06] - source-transformation implementationusing bit-vectors to indicate dependence of active variable on eachindependent (or dependent) in a pure (no values) calculation.
ADOL-C features overloaded bit-vector propagation.
18/ 28 Branch Detection and Sparsity Estimation in Matlab
Sparsity Detection in Matlab
More recent versions of Matlab do support some bit operations(bitand, bitor and bitxor) on unsigned integers - not investigatedto date since MAD tool [For06] relies on use of high-level (sparse)matrix operations.
Matlab also offers sparse logical matrices - can perform all thehigh-level matrix operations of the MAD package on them.
19/ 28 Branch Detection and Sparsity Estimation in Matlab
The sparsemad Class
Three components:
value - stores the numerical value of the object.
entries - logical array identifying the (nonzero) entries inthe value component.
sparsity - a derivvec object holding the sparsity patternfor each component of the value.
20/ 28 Branch Detection and Sparsity Estimation in Matlab
Linearity of f (t, y) detected - a global MADsparsemadNonlinear set totrue if nonlinear function encountered by sparsemad class.
26/ 28 Branch Detection and Sparsity Estimation in Matlab
Conclusions
Branch detection allows us to check if sparsity pattern, tape or PDFTaylor series is safe to use for all inputs x.
Sparsity Detection allows us to determine Jacobian sparsity patterns.
Future Work
Discontinuous derivative testing to ensure validity of Taylor series.
Can we estimate how much of a pdf passes along currentcontrol-flow?
Adaptation of Taylor order for accurate statistical moments.
Sparsity detection based on bit-wise operations - can it beat thecurrent sparse matrix implementation?
27/ 28 Branch Detection and Sparsity Estimation in Matlab
References
Christian H. Bischof, Peyvand M. Khademi, Ali Bouaricha, and Alan Carle.
Efficient computations of gradients and Jacobians by dynamic exploitation of sparsity in automatic differentiation.Optimization Methods and Software, 7:1–39, 1996.
Thomas F. Coleman and Arun Verma.
The efficient computation of sparse Jacobian matrices using automatic differentiation.SIAM J. Sci. Comput., 19(4):1210–1233, 1998.
Shaun A. Forth.
An efficient overloaded implementation of forward mode automatic differentiation in MATLAB.ACM Trans. Math. Softw., 32(2):195–222, June 2006.DOI: http://doi.acm.org/10.1145/1141885.1141888.
Ralf Giering and Thomas Kaminski.
Automatic sparsity detection implemented as a source-to-source transformation.In Vassil N. Alexandrov, Geert Dick van Albada, Peter M. A. Sloot, and Jack Dongarra, editors, Computational Science –ICCS 2006, volume 3994 of Lecture Notes in Computer Science, pages 591–598, Heidelberg, 2006. Springer.dx.doi.org/10.1007/11758549_81.
Assefaw Hadish Gebremedhin, Fredrik Manne, and Alex Pothen.
What color is your Jacobian? Graph coloring for computing derivatives.SIAM Review, 47(4):629–705, 2005.
Andreas Griewank.
Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation.Number 19 in Frontiers in Applied Mathematics. SIAM, Philadelphia, PA, 2000.
28/ 28 Branch Detection and Sparsity Estimation in Matlab