Deterministic and stochastic acceleration Deterministic and stochastic acceleration Deterministic and stochastic acceleration Deterministic and stochastic acceleration Massimiliano Lupo Pasini 1 , Michele Benzi 1 , Thomas M. Evans 2 , Steve Massimiliano Lupo Pasini 1 , Michele Benzi 1 , Thomas M. Evans 2 , Steve Massimiliano Lupo Pasini , Michele Benzi , Thomas M. Evans , Steve 1 Emory College, Department of Mathematics and Com 1 Emory College, Department of Mathematics and Com 1 Emory College, Department of Mathematics and Com 2 Oak Ridge National Laboratory, 1 Be 2 Oak Ridge National Laboratory, 1 Be Oak Ridge National Laboratory, 1 Be 3 Georgia Institute of Technology, School of Civil and Environm 3 Georgia Institute of Technology, School of Civil and Environm Introduction Introduction Alternating Anderso Introduction Alternating Anderso Introduction Alternating Anderso AAR uses approximate solutions co • Scientific computing is moving to exascale AAR uses approximate solutions co • Scientific computing is moving to exascale AAR uses approximate solutions co Richardson’s steps to build the ma • Applications require high level of concurrency Richardson’s steps to build the ma • Applications require high level of concurrency Richardson’s steps to build the ma • Next generation computers will exhibit more hardware failures • Next generation computers will exhibit more hardware failures • Next generation computers will exhibit more hardware failures – applications must be resilient – applications must be resilient The Anderson mixing is defined a – applications must be resilient The Anderson mixing is defined a • Standard Krylov subspace methods struggle to • Standard Krylov subspace methods struggle to simultaneously obtain efficiency and concurrency simultaneously obtain efficiency and concurrency • Standard Krylov methods struggle to achieve resilience The vector is chosen so as to m • Standard Krylov methods struggle to achieve resilience The vector is chosen so as to m • Standard Krylov methods struggle to achieve resilience • Richardson’s schemes benefit computational and data locality • Multiple Richardson’s steps with • Richardson’s schemes benefit computational and data locality • Multiple Richardson’s steps with • Richardson’s schemes benefit computational and data locality computational and data locality computational and data locality • Convergence on positive defini • Convergence on positive defini Mathematical framework Mathematical framework Mathematical framework • Reformulate the sparse linear system of interest • Reformulate the sparse linear system of interest • Reformulate the sparse linear system of interest as a fixed point scheme as a fixed point scheme as a fixed point scheme Conclusions and fu such that the Neumann series recasts the solution as Conclusions and fu such that the Neumann series recasts the solution as Conclusions and fu • Identified classes of matrices for (1) • Identified classes of matrices for (1) guaranteed (1) guaranteed • Competitive performance compa • Competitive performance compa • One level fixed point schemes are renown for their deteriorated different choices of preconditione • One level fixed point schemes are renown for their deteriorated different choices of preconditione asymptotic convergence rate different choices of preconditione asymptotic convergence rate Convergence analysis on specifi • Multilevel schemes can accelerate convergence Convergence analysis on specifi • Multilevel schemes can accelerate convergence Performance assessment at extr • Multilevel schemes can accelerate convergence Performance assessment at extr • Purely deterministic algorithms struggle to deal with inherently Use of GPU accelerations • Purely deterministic algorithms struggle to deal with inherently Use of GPU accelerations random faulty phenomena Refere random faulty phenomena Refere Refere Refere • “Anderson acceleration of the Jacobi ite One level relaxation scheme • “Anderson acceleration of the Jacobi ite One level relaxation scheme alternative to Krylov methods for large, s alternative to Krylov methods for large, s P. P. Pratapa, P. Suryanarayana, and J. P. P. Pratapa, P. Suryanarayana, and J. J. Comput. Phys., 306, pp. 4354, 2016. J. Comput. Phys., 306, pp. 4354, 2016. • “Converge analysis of Anderson-type a Deterministic accelerations Stochastic accelerations • “Converge analysis of Anderson-type a Deterministic accelerations improve convergence Stochastic accelerations enhance resilience • “Converge analysis of Anderson-type a iteration”, M. Lupo Pasini – In preparatio improve convergence enhance resilience iteration”, M. Lupo Pasini – In preparatio techniques for Richardson-type iterations techniques for Richardson-type iterations techniques for Richardson-type iterations techniques for Richardson-type iterations en P. Hamilton 2 , Stuart R. Slattery 2 , Phanish Suryanarayana 3 en P. Hamilton 2 , Stuart R. Slattery 2 , Phanish Suryanarayana 3 en P. Hamilton , Stuart R. Slattery , Phanish Suryanarayana mputer Science, 400 Dowman Drive, Atlanta, GA 30322 mputer Science, 400 Dowman Drive, Atlanta, GA 30322 mputer Science, 400 Dowman Drive, Atlanta, GA 30322 ethel Valley Rd, Oak Ridge, TN 37830 ethel Valley Rd, Oak Ridge, TN 37830 ethel Valley Rd, Oak Ridge, TN 37830 mental Engineering, 790 Atlantic Drive NW, Atlanta, GA 30332 mental Engineering, 790 Atlantic Drive NW, Atlanta, GA 30332 on-Richardson (AAR) Monte Carlo Linear Solvers (MCLS) on-Richardson (AAR) Monte Carlo Linear Solvers (MCLS) on-Richardson (AAR) Monte Carlo Linear Solvers (MCLS) MCLS use random walks defined on a transition matrix P and a omputed by successive MCLS use random walks defined on a transition matrix P and a omputed by successive MCLS use random walks defined on a transition matrix P and a sequence of weights w = H / P , the statistical estimator omputed by successive atrix sequence of weights w i,j = H i,j / P i,j , the statistical estimator atrix sequence of weights w i,j = H i,j / P i,j , the statistical estimator redefines (1) as atrix redefines (1) as redefines (1) as as as • Convergence guaranteed for strictly diagonally dominant, M • Convergence guaranteed for strictly diagonally dominant, M matrices, generalized diagonally dominant matrices minimize the residual norm. matrices, generalized diagonally dominant matrices minimize the residual norm. • Choice of preconditioners affects convergence hout optimization benefit • Choice of preconditioners affects convergence hout optimization benefit • Choice of preconditioners affects convergence • We use adaptive methods to select the number of histories • We use adaptive methods to select the number of histories ite matrices guaranteed Reference solution Tolerance = 0.5 ite matrices guaranteed Reference solution Tolerance = 0.5 Tolerance = 0.1 Tolerance = 0.01 Tolerance = 0.1 Tolerance = 0.01 Conclusions and future developments uture developments Conclusions and future developments uture developments Conclusions and future developments uture developments • Identified classes of matrices and preconditioners for which r which convergence is • Identified classes of matrices and preconditioners for which r which convergence is MCLS are guaranteed to converge a priori MCLS are guaranteed to converge a priori • Difficulty in ensuring a priori convergence of MCLS for a general ared to restarted GMRES for • Difficulty in ensuring a priori convergence of MCLS for a general ared to restarted GMRES for problem er problem er problem er Algorithm scalability has not yet been analyzed ic classes of problems Algorithm scalability has not yet been analyzed ic classes of problems Testing MCLS on large parallel architectures and evaluating reme scale Testing MCLS on large parallel architectures and evaluating reme scale their resilience in presence of faults still to be addressed their resilience in presence of faults still to be addressed References ences References ences References ences References ences • “A Monte Carlo synthetic-acceleration method for solving the thermal erative method: an efficient • “A Monte Carlo synthetic-acceleration method for solving the thermal erative method: an efficient radiation diffusion equation“ . T. M Evans, S. W. Mosher , S. R. Slattery , S. sparse linear systems”, radiation diffusion equation“ . T. M Evans, S. W. Mosher , S. R. Slattery , S. P. Hamilton – J. Comp. Phys., Vol. 258, 2014, pp. 338-358. sparse linear systems”, E. Pask – P. Hamilton – J. Comp. Phys., Vol. 258, 2014, pp. 338-358. E. Pask – • “Analysis of Monte Carlo Accelerated Iterative Methods for Sparse Linear • “Analysis of Monte Carlo Accelerated Iterative Methods for Sparse Linear Systems”. M. Benzi, T. M. Evans, S. P. Hamilton, M. Lupo Pasini, S. R. acceleration for Richardson’s Systems”. M. Benzi, T. M. Evans, S. P. Hamilton, M. Lupo Pasini, S. R. acceleration for Richardson’s Systems”. M. Benzi, T. M. Evans, S. P. Hamilton, M. Lupo Pasini, S. R. Slattery – Numer . Linear Algebra Appl. Vol. 24, Issue 3, 2017. acceleration for Richardson’s on. Slattery – Numer . Linear Algebra Appl. Vol. 24, Issue 3, 2017. on.