HOPR - High Order Preprocessor
Florian Hindenlang1,
Thomas Bolemann2, Malte Hoffmann2, Claus-Dieter Munz2
1Max-Planck Institute for Plasma Physics, Garching2Institute for Aero- and Gasdynamics, University of Stuttgart
High-Order Computational Fluid Dynamics Technologies WebinarSeptember 21, 2015
1 / 22
Curved Meshes: HOPR
I DG is high order accurate on unstructured grids
I Curved elements to maintain accuracy at curved wall boundaries
I DG for industrial applications:I Wall-bounded flowsI 3D curved geometries, intersecting surfaces, sharp cornersI Unstructured and coarse grids
⇒ HOPR (High Order Preprocessor)released Open-Source: www.hopr-project.org
2 / 22
Facts on HOPR
I Open-source under GPL v2: www.hopr-project.org
I Mesh preprocessor for parallel high order simulations(DG, FR, SEM, pFEM...)
I Written in FORTRAN
I Generate simple meshes for testing, directly in HOPR
I HOPR Input:I Unstructured hybrid meshes from any grid generatorI Several formats supported (CGNS mostly used)
⇒ Different strategies to produce high order meshes from the initiallinear mesh
I HOPR Output: HDF5 mesh file
3 / 22
HOPR output: HDF5 mesh file
I Idea: Direct parallel readinI HOPR sorts elements along a Space-Filling Curve
⇒ 1D element list, can be split into arbitrary number of domainsI Element-per-element datablocks:
high order node coordinates, side info (BC, connectivity)
⇒ Split global arrays into non-overlapping blocks during parallel readin!
HDF5 mesh file
globalattributes
BCNames
BCTypes
ElemInfo SideInfoNodeCoordselem1elem2elem3
elem4elem5elem6
globalattributes
BCNames
BCTypes
globalattributes
BCNames
BCTypes
globalattributes
BCNames
BCTypes
computenLocElemsoffsetElem
computenLocElemsoffsetElem
computenLocElemsoffsetElem
elem1elem2elem3
elem4elem5elem6
elem7elem8
elem7elem8
offset/sizeSidesNodes
offset/SizeSidesNodes
offset/SizeSidesNodes
buildelement
connectivity
buildelement
connectivity
buildelement
connectivity
rank
#0
rank
#1
rank
#2
4 / 22
HOPR Mesh Curving Flowchart
commercial mesher: ICEM, ANSA,... (cgns unstruct. format) linear unstructured hybrid mesh
curved boundary faces
Normal vector(s) at surface nodes
point-normalsICEM
spectral elementssurface
subdivision
ICEM Cheb.-Lobattonodes on edges
curved boundary edges
blend curved edges to face
blend curved edgesto inner faces
blend curved facesto volumes
subdividedsurface mesh
curved high order mesh
block-structured mesh(cgns struct. format)
Agglomerationof subblocks to
curved hexahedra
5 / 22
HOPR Mesh Curving Flowchart
commercial mesher: ICEM, ANSA,... (cgns unstruct. format) linear unstructured hybrid mesh
curved boundary faces
Normal vector(s) at surface nodes
point-normalsICEM
spectral elementssurface
subdivision
ICEM Cheb.-Lobattonodes on edges
curved boundary edges
blend curved edges to face
blend curved edgesto inner faces
blend curved facesto volumes
subdividedsurface mesh
curved high order mesh
block-structured mesh(cgns struct. format)
Agglomerationof subblocks to
curved hexahedra
5 / 22
Curved edges from normal vectors
Get normal vectors at linear surface mesh points:
1. In HOPR without additional data:a) Reconstruction via linear surface meshb) Analytic functions (sphere ,cylinder ...)
2. Additional point-normal file(from CAD,...)
⇒ Generate curved edges
6 / 22
Curved edges from normal vectors
Get normal vectors at linear surface mesh points:
1. In HOPR without additional data:a) Reconstruction via linear surface meshb) Analytic functions (sphere ,cylinder ...)
2. Additional point-normal file(from CAD,...)
⇒ Generate curved edges
6 / 22
Multiple normal vectors to recover sharp edges & corners
p1
p2
n1
t 1
e p1
p2
n1
t 1
en2
(a) (b)
(a) 1 normal vector, tangential vector by projection
(b) 2 normal vectors, tangential vector by cross product
7 / 22
Blending curved edges to curved faces
=12 { X
a X
b X
c− X
t }
= X a X b − X t
Superposition of linear blended pairs of curved edges (Coons patches)
8 / 22
HOPR Mesh Curving Flowchart
commercial mesher: ICEM, ANSA,... (cgns unstruct. format) linear unstructured hybrid mesh
curved boundary faces
Normal vector(s) at surface nodes
point-normalsICEM
spectral elementssurface
subdivision
ICEM Cheb.-Lobattonodes on edges
curved boundary edges
blend curved edges to face
blend curved edgesto inner faces
blend curved facesto volumes
subdividedsurface mesh
curved high order mesh
block-structured mesh(cgns struct. format)
Agglomerationof subblocks to
curved hexahedra
9 / 22
High Order Surface by Subdivision
linear surface mesh
I Mesh generator: subdivision of the initial mesh while keeping newpoints on the CAD surfaces
⇒ Polynomial surface patches by interpolation
10 / 22
High Order Surface by Subdivision
after one subdivision
I Mesh generator: subdivision of the initial mesh while keeping newpoints on the CAD surfaces
⇒ Polynomial surface patches by interpolation
10 / 22
High Order Surface by Subdivision
after two subdivisions
I Mesh generator: subdivision of the initial mesh while keeping newpoints on the CAD surfaces
⇒ Polynomial surface patches by interpolation
10 / 22
Blending curved faces to curved volume
1. Curved face on boundary
2. Blend adjacent faces with linear edges
3. Blending of all element faces → volume mapping
4. Generalized for all element types (tets,pyramids,prisms,hexas)
11 / 22
Blending curved faces to curved volume
1. Curved face on boundary
2. Blend adjacent faces with linear edges
3. Blending of all element faces → volume mapping
4. Generalized for all element types (tets,pyramids,prisms,hexas)
11 / 22
Blending curved faces to curved volume
1. Curved face on boundary
2. Blend adjacent faces with linear edges
3. Blending of all element faces → volume mapping
4. Generalized for all element types (tets,pyramids,prisms,hexas)
11 / 22
Blending curved faces to curved volume
1. Curved face on boundary
2. Blend adjacent faces with linear edges
3. Blending of all element faces → volume mapping
4. Generalized for all element types (tets,pyramids,prisms,hexas)
11 / 22
Example: DLR-F6 Wing-Body-Nacelle Configuration
I Test case of an idealized passenger jetliner
I CAD geometry provided as STEP file
I Element layer on geometry ∼ 3600 hexa∼ 400 prism
I Volume grid : ∼ 225000 tetrahedra∼ 2900 pyramids
12 / 22
Example: DLR-F6 Wing-Body-Nacelle Configuration
13 / 22
Example: DLR-F6 Wing-Body-Nacelle Configuration
⇒ Visualization of polynomial patches shows smooth boundary surfaceelements
14 / 22
HOPR Mesh Curving Flowchart
commercial mesher: ICEM, ANSA,... (cgns unstruct. format) linear unstructured hybrid mesh
curved boundary faces
Normal vector(s) at surface nodes
point-normalsICEM
spectral elementssurface
subdivision
ICEM Cheb.-Lobattonodes on edges
curved boundary edges
blend curved edges to face
blend curved edgesto inner faces
blend curved facesto volumes
subdividedsurface mesh
curved high order mesh
block-structured mesh(cgns struct. format)
Agglomerationof subblocks to
curved hexahedra
15 / 22
Hexa-meshes: Agglomeration
I Start from fine block-structured meshes
⇒ Coarsen by a fixed factor
⇒ Use inner points to generate high order element mapping
16 / 22
Agglomeration Example: Serrated nozzle I
I Test case provided by Andrei Cimpoeru, Imperial College
I 85 blocks, 20.6M linear elements, 21.8M points, 0.5 GB CGNS file
I Agglomeration of 4x4x4 elements ⇒ 322,110 P4-elements
I HOPR used ≈ 6 GB RAM, run for ≈ 2 minutes(1 min eliminate duplicate nodes, 40s Jacobian checker)
17 / 22
Agglomeration Example: Serrated nozzle II
all elements curvedinvalid elements inside the volume
only curved at wall BCinvalid elements at the boundary
#elems with Jscaled < 0 0− 0.1 0.1− 0.2 0.2− 0.3 ≥ 0.3
all linear 0 0 65 204 321,841all curved 10 737 896 1,684 318,783curved BC 61 6 70 216 321,757
18 / 22
Agglomeration Example: Serrated nozzle II
1st layer curvedinvalid elements
2 layers curved⇒ all elements valid
#elems with Jscaled < 0 0− 0.1 0.1− 0.2 0.2− 0.3 ≥ 0.3
all linear 0 0 65 204 321,841all curved 10 737 896 1,684 318,783curved BC 61 6 70 216 321,757
1st layer 10 50 104 376 321,5702 layers 0 0 197 520 321,393
18 / 22
Agglomeration Example: Serrated nozzle II
3 layers curved⇒ all elements valid
4 layers curved⇒ all elements valid
#elems with Jscaled < 0 0− 0.1 0.1− 0.2 0.2− 0.3 ≥ 0.3
all linear 0 0 65 204 321,841all curved 10 737 896 1,684 318,783curved BC 61 6 70 216 321,757
1st layer 10 50 104 376 321,5702 layers 0 0 197 520 321,393
3 layers 0 0 178 667 321,2654 layers 0 0 160 746 321,204
18 / 22
Ongoing Work on Boundary Layer Meshes
I Structured grids:
⇒ Agglomeration (works in current HOPR release)
I Unstructured grids: Only curving the first element at the surfaceleads to inverted elements
⇒ For prismatic layers of linear elements:Blending of element stack with curved boundary face (not released)
⇒ Insert boundary layers afterwards (not released)
19 / 22
Boundary layer Insertion
I Valid curved mesh with prismatic elements (hex, prism)I Split the first element layer
⇒ into a given number boundary layer elements (input parameter)
⇒ with a given stretching (input parameter)
20 / 22
Boundary layer Insertion
I Valid curved mesh with prismatic elements (hex, prism)I Split the first element layer
⇒ into a given number boundary layer elements (input parameter)
⇒ with a given stretching (input parameter)
20 / 22
Conclusion
I HOPR provides unstructured high order meshes
I Different curving strategies, depending on provided data
I HDF5 mesh file, designed for parallel IO
I Software is open-source
⇒ Announcement for next ICOSAHOM:David Kopriva wants to organize a mini-symposium todiscuss a standard for a high order mesh format
21 / 22
Thanks to E. Sonnendrucker, C.-D. Munz, G. Gassner, T. Bolemann, H.Frank, and
Thank you for your attention!
www.hopr-project.org
HOPR Homepage (with Mailing List, Tutorials, Documentation...)
Contact: [email protected], [email protected]
22 / 22
Appendix:
23 / 22
Mesh Format: Element Node Connectivity
I All element mappings in the mesh have the same order (watertight)I Element node connectivity defined for all orders (i , j , k loops)I Full order elements only
XI, i
ETA, j
ZETA, k
1 3
6
13 15
18
2
14
4
16
5
17
7 9
12
8
10 11
1 3
6
10
2
4 5
7 8
9
1 3
97
19 21
2725
2
8
20
26
4 6
22 24
5
23
10 12
1816
11
17
13 1514
1 3
97
14
2
8
4 65
10 11
12 13
24 / 22
SFC Domain Decomposition
I 3D unstructured simulation (N = 4) of a turbulent flow past asphere (Re = 1000)
I Domain decomposition based on a space filling curve element sorting
25 / 22
SFC Domain Decomposition
I SFC-DD allows arbitrary number of domains, given a-posteriori
#Domains #Elements/core #MPIneighbors (mean)128 165-166 5-23 (11.75)
25 / 22
SFC Domain Decomposition
I SFC-DD allows arbitrary number of domains, given a-posteriori
#Domains #Elements/core #MPIneighbors (mean)128 165-166 5-23 (11.75)
1024 20-21 5-21 (11.99)
25 / 22
SFC Domain Decomposition
I SFC-DD allows arbitrary number of domains, given a-posteriori
#Domains #Elements/core #MPIneighbors (mean)128 165-166 5-23 (11.75)
1024 20-21 5-21 (11.99)4096 5-6 5-20 (10.37)
25 / 22