References [1] P. Neumann and N. Tchipev, A Coupling Tool for Parallel Molecular Dynamics–Continuum Simulations, Proceedings of the International Symposium on Parallel and Distributed Computing, 2012 [2] R. Delgado-Buscalioni, P.V. Coveney. USHER: An algorithm for particle insertion in dense fluids. J. Chem. Phys. 119 (2): 978–987 (2003). [3] A. Dupuis, E.M. Kotsalis, P. Koumoutsakos. Coupling lattice Boltzmann and molecular dynamics models for dense fluids. Phys. Rev. E 75, 046704 (2007). 1 Department of Computer Science, Technische Universität München, Boltzmannstr. 3, D-85748 Garching, Germany, [email protected] 2 Department of Applied Physics, Technische Universiteit Eindhoven, P.O. Box 513, 5600 MB Eindhoven, The Netherlands, [email protected] ✬ ✫ ✩ ✪ Outlook: Nanofilters • Flow between two reservoirs through a molecular filter • Flow simulation in reservoirs: LB • Simulation of flow through filter: MD • Fully three-dimensional steady-state cou- pling based on [3] ✬ ✫ ✩ ✪ MaMiCo: Macro-Micro-Coupling Tool ParticleInsertion MomentumInsertion MacroscopicCellService MacroscopicCellService MomentumController KineticEnergyController MoleculeWrapper MoleculeIterator MDSolverInterface MacroscopicSolverInterface MDSimulation ContinuumSimulation TransferStrategy implements utilises utilises sends data to receives data from/ sends data to implements utilises utilises MaMiCo User/ Programmer uses/ provides new implementation of • C++-development for spatiotemporal hybrid molecular–continuum simulations [1] • Support of 2D/ 3D hybrid scenarios • Macroscopic cells within MaMiCo are used to – map macroscopic quantities between the macroscopic solver and MaMiCo – map macro-/microscopic quantities between the MD solver and MaMiCo • Particle insertion/ deletion [2] encapsulated within MaMiCo Macroscopic Solver MaMiCo Molecular Dynamics Particle Insertion/ Deletion Momentum Mass Momentum Mass Momentum Mass Momentum ✬ ✫ ✩ ✪ Macroscopic Cell Exchange • MPI-based exchange of macroscopic cell information between MD and LB • Exchange (mostly) hidden from user • Example: LB on one process, MD on four processes LB MD LB MD MacroscopicCellService → Send to MD Start: traverse LB cells. Step 1: track cells that should be sent to MD LB MD MacroscopicCellService → Receive from MD LB MD MacroscopicCellService → Setup Send-Recv Step 2: track cells that should be re- ceived from MD Step 3: setup send-recv topology within MacroscopicCellService ✬ ✫ ✩ ✪ Parallel Results MD on rank 0 LB domain and embedded MD • Channel flow simulation with MD region embedded in the center • LB solver: sequential, 54 × 54 × 54 cells, BGK • MD solver: parallel, 1 000 000 single-centered LJ molecules • LB→ MD: Velocity relaxation in (MD) boundary cells, retaining mass in outermost cells using USHER • MD→ LB: Only map velocities and incorporate them into particle distribution functions Proc. Shaheen Huygens MD-LB MD MD-LB MD 1 1.0 1.0 1.0 1.0 8 7.2 7.3 7.3 6.9 64 46.0 49.6 45.7 45.3 512 244.0 321.0 235.5 249.7 1728 484.4 814.5 456.7 494.7 Strong scaling for channel flow scenario. MD-LB: coupled simulation. MD: reference MD simulation for same MD domain without coupling. 1 10 100 1000 10000 0.7 0.78 0.86 0.94 1.02 Huygens t0 Huygens t1 Huygens t2 Shaheen t0 Shaheen t1 Shaheen t2 # cores Efficiency Weak scaling for parallel USHER-based particle insertion [1] ✬ ✫ ✩ ✪ Overview Efficient implementations of hybrid molecular-continuum flow solvers are required to allow for fast and massively parallel simulations of large complex systems. Several cou- pling strategies have been proposed over the last years for 2D/ 3D, time-dependent/ steady-state or compressible/ incompressible scenarios. Despite their different applica- tion areas, most of these schemes comprise the same or similar building blocks. Still, to the authors’ knowledge, no common implementation of these building blocks is available yet. In this contribution, the Macro-Micro-Coupling Tool is presented which is meant to support developers in coupling mesh-based methods with molecular dynamics. It is written in C++ and supports two- and three-dimensional scenarios. Its design is re- viewed, and aspects for massively parallel coupled scenarios are addressed. Scaling results are presented for a hybrid simulation which couples a molecular dynamics code to the Lattice Boltzmann application of the Peano framework. Philipp Neumann 1 Jens Harting 2 Massively Parallel Molecular-Continuum Simulations with the Macro-Micro-Coupling Tool