FRG: Multiscale Simulation of Atomistic Processes in ... · FRG: Multiscale Simulation of Atomistic Processes in Nanostructured Materials Rajiv K. Kalia, Aiichiro Nakano & Priya Vashishta

FRG: Multiscale Simulation of Atomistic Processes in Nanostructured Materials

Rajiv K. Kalia, Aiichiro Nakano & Priya VashishtaConcurrent Computing Laboratory for Materials Simulations

Dept. of Physics, Dept. of Computer Science, Louisiana State Univ.

Dept. of Materials Science, Dept. of Physics, Dept. of Computer Science, Dept. of Biomedical Engineering

Univ. of Southern California (after September 2002)Email: {kalia, nakano, priyav}@bit.csc.lsu.edu

URL: www.cclms.lsu.edu

NSF Division of Materials ResearchComputational Materials Theory Program Review

Program Managers: Dr. Bruce Taggart & Dr. Daryl HessOrganizers: Dr. Duane Johnson & Dr. Jeongnim Kim

June 20, 2002, Urbana, ILCCLMSCCLMSCCLMS

Outline

1. Multiscale finite-element/molecular-dynamics /quantum-mechanical simulation on a computational Grid

2. Immersive & interactive visualization of billion-atom systems

3. Multimillion atom MD simulations of fracture & nanoindentation in crystalline, amorphous & nanophase materials & ceramic nanocomposites

4. New educational and outreach programs

Multiscale FE/MD/QM Simulation

10-6 - 10-10 m> 10-6 m

Multiscale simulationto seamlessly couple:• Finite-element (FE)

calculation• Atomistic molecular-

dynamics (MD) simulation

• Quantum-mechanical(QM) calculationbased on the densityfunctional theory(DFT)

• Newton’s equations of motion

• Interatomic potential

• N-body problemlong-range electrostatic interaction—O(N2)

• Efficient O(N) solution—space-time multiresolution algorithm1. Fast Multipole Method (FMM) (Greengard & Rokhlin, 87)

2. Symplectic Multiple Time-Scale (MTS) method (Tuckerman et al., 92)

Molecular Dynamics Simulation

mi

d2r r i

dt2= −

∂Vr r N( )

∂r r i(i = 1,...,N )

V = uij rij( )

i< j∑ + v jik

r r ij ,r r ik( )

i, j<k∑

Evaluate at x = xj (j = 1,...,N)Ves(x) =qi

| x − xi |i=1

N∑

Density functional theory (DFT)

O(CN ) O(N3 )Constrained minimization problem:• Minimize E[{ψn}] with orthonormal constraints,

Efficient parallelization: real-space approaches• Finite difference (Chelikowsky, Troullier, Saad, 94) & multigrid acceleration (Bernholc, 96)

O(N) algorithm (Kim, Mauri & Galli, 95)

• Asymptotic decay of density matrix:• Localized functions:

• Unconstrained minimization—Lagrange multiplier

Quantum Mechanical Calculation

ψ (r1,r2 ,K,rNel) ψn (r) | n = 1,K,N el{ }

dr∫ ψm* (r)ψ n (r) =δmn

ρ(r, ′ r ) ≡ ψn* (r )ψ n ( ′ r )

n=1

Nel

∑ ∝ exp −C | r − ′ r |( )

φm (r) = ψ n (r)Unmn∑

ψn(r) φm(r)

Rcut

On 1,024 IBM SP3 processors:• 6.44-billion-atom MD of SiO2• 444,000-electron DFT of GaAs

Scalable Scientific Algorithm Suite1,024 IBM SP3 & Cray T3E processors at NAVO

IEEE/ACM Supercomputing 2001Best Paper Award

Hybrid MD/QM Algorithm

QM

MD

Handshakeatoms

Additive hybridizationReuse of existing MD & QM codes(Morokuma et al., ‘96)

Scaled-position link atomsSeamless coupling ofMD & QM systems

MD simulation embeds a QM cluster described by a real-space multigrid-based density functional theory

E ≅ EMDsystem + EQM

cluster − EMDcluster

QM

MD≅ + −MD QM MD

Hybrid FE/MD Algorithm• FE nodes & MD atoms coincide in the handshake region• Additive hybridization

MD

FE

[0 1 1]

[1 1 1]_

HS

_[1 1 1]

[2 1 1]

Si/Si3N4 nanopixel

FE/MD/QM Simulation: Oxidation on Si Surface

Dissociation energy of O2 on a Si (111) surface dissipated seamlessly from the QM cluster through the MD regionto the FE region

MD FE

Stress Corrosion in Si

Yellow: QM-HRed: QM-OGreen: QM-SiBlue: HS-SiGray: MD-Si

Reaction of H2O molecules at a Si(110) crack tip

237 QM atoms

Stress Corrosion in SiSignificant effects of stress intensity factor on the reaction

Yellow: HRed: OGreen: Si

Distributed Cluster Computing

• Globus/MPICH-G2 — Ian Foster (ANL)• S. Ogata (Yamaguchi), F. Shimojo (Hiroshima),

K. Tsuruta (Okayama), H. Iyetomi (Niigata)

MD

QM1

QM3

QM2

Additive hybridization; multiple QM clustering

Multiscale Simulation on a Grid

• Scaled speedup, P = 1 + 8n (n = number of clusters)• Efficiency = 94.0% on 25 processors in the US & Japan

Immersive & Interactive Visualization

• Octree-based fast view-frustum culling• Parallel/distributed processing

Billion-atom walkthrough

PC cluster

WAND User

positionGraphics

server

ImmersaDeskthrough DURIP

Reduced data

Parallel & Distributed Visualization

Nearly real-time walkthrough for 1 billion atoms on an SGI Onyx2 (2 MIPS R10K, 4GB RAM) connected to a PC cluster (4 800MHz P3)

Outline

1. Multiscale finite-element/molecular-dynamics /quantum-mechanical simulation on a computational Grid

2. Immersive & interactive visualization of billion-atom systems

3. Multimillion atom MD simulations of fracture and nanoindentation in crystalline, amorphous & nanophase materials and ceramic nanocomposites

4. New educational and outreach programs

Fracture in Glasses, NanophaseCeramics & Nanocomposites

Systems– Nanophase Si3N4, SiC, Al2O3

– SiO2 glass – Nanocomposite - SiC fibers in a Si3N4 matrix

Issues– Atomistics of crack propagation– Mechanisms of energy dissipation– Scaling properties of fracture surfaces

Nanophase ceramics

Si3N4

Amorphous SiO2

0

0.5

1

1.5

2

0 5 10 15 20

Expt.

MDS

N(q

)

q (Å -1)

Validation of Interatomic Potential

MD results for elastic moduli also agree well with experiments

Fracture in Amorphous Silica: 15 Million Atom MD Simulations

Fracture ToughnessKIC =1 MPa.m 1/2 (MD)

0.8 MPa.m 1/2 < KIC < 1.2 MPa.m 1/2 (Experiment)

Experiment by Bouchaud et al.

Cavity formation & coalescence with crack in a glass

(Top) AFM study in a glass:(a) nanometric cavities before the crack advances; (b) growth of cavities; (c) crack propagation via the coalescence of cavities.(Bottom) MD results showing the same phenomenon in a-SiO2

Fracture Simulation & Experiment

Dynamic Fracture in Nanophase Si3N4

Pore CoalescenceCrack Deflection

Toughening Mechanism: Crack deflection, branching & coalescence with nanopores

Nanophase Si3N4 is much tougher than Si3N4 crystal

Morphology & Scaling Behavior of Fracture Surfaces

Fracture surfaces are anisotropic & statistically invariant under an affine transformation

),,(),,( bzbyxbzyx ζ→

z z+r

∆h(r)

x

ζ is called the roughness exponent

( ) 2/12)()()(z

zxrzxrh −+=∆

ζrrh ∝∆ )(

P. Daguier, B. Nghiem, E. Bouchaud & F. Creuzet (1997)

Scaling Behavior of Cracks

Two regimes with roughness exponents 0.5 & 0.8

0

0.5

1.0

1.5

2.0

2.5

2 3 4 5 6ln(z)

ln∆h

(x)

ζ = 0.58 ± 0.14

ζ = 0.84 ± 0.12

MD results for roughness exponents agree with experiments

MD results reveal that the smaller roughness exponent is due to intrapore correlations and the larger one due to coalescence of pores and crack

NanophaseSi3N4

Color code: Si3N4; SiC; SiO2

Fracture in a Nanocomposite

Silicon nitride matrix-silicon carbide fiber nanocomposite

1.5-billion-atom MD on 1,024 IBM SP3 processors

0.3 mm

Nanoindentation in Si3N4

Multi-million atom MD simulations

Nanohardness yields important informationabout elastic and plastic deformation

Indentation Fracture & Amorphization

<1210> Indentation fractureat indenter diagonals

Amorphous pile-upat indenter edges

Anisotropicfracture toughness

<1010>

<0001>

Hardness of Silicon Nitride

Load-displacement curve

α-crystal (0001)50GPa

Amorphous32GPa

Microindentation experiments on α-crystal (0001): 31-48GPa

Summary

• Parallel multiscale MD/QM simulation methodology

• Interactive visualization of billion atoms

• Multimillion atom MD simulations> fracture in silica glass, nanophase

ceramics & nanocomposite> nanoindentation in crystalline &

amorphous Si3N4

Graduate Education

Ph.D. in the physical sciences & M.S. in computer science

Dual-Degree Program

International course on computational physics

• Synthesis

• Characterization

• Property measurements

Experimental training

• LSU/USC

• TU Delft, The Netherlands

• Niigata Univ., Japan

Undergraduate Outreach Activities

Computational Science Workshop for Underrepresented Groups

• 19 participants from 11 institutions — Hampton, Clark-Atlanta, Morehouse, Jackson State, Mississippi State, Texas Southern, Univ. of Texas –– Pan American, Xavier, Grambling, Southern & Univ. of Louisiana in Monroe

• Activities: Construction of a PC cluster from off-the-shelf components & using this parallel machine for algorithmic and simulation exercises.