Page 1
HOPR - High Order Preprocessor
Florian Hindenlang1,
Thomas Bolemann2, Malte Hoffmann2, Claus-Dieter Munz2
1Max-Planck Institute for Plasma Physics, Garching2Institute for Aero- and Gasdynamics, University of Stuttgart
High-Order Computational Fluid Dynamics Technologies WebinarSeptember 21, 2015
1 / 22
Page 2
Curved Meshes: HOPR
I DG is high order accurate on unstructured grids
I Curved elements to maintain accuracy at curved wall boundaries
I DG for industrial applications:I Wall-bounded flowsI 3D curved geometries, intersecting surfaces, sharp cornersI Unstructured and coarse grids
⇒ HOPR (High Order Preprocessor)released Open-Source: www.hopr-project.org
2 / 22
Page 3
Facts on HOPR
I Open-source under GPL v2: www.hopr-project.org
I Mesh preprocessor for parallel high order simulations(DG, FR, SEM, pFEM...)
I Written in FORTRAN
I Generate simple meshes for testing, directly in HOPR
I HOPR Input:I Unstructured hybrid meshes from any grid generatorI Several formats supported (CGNS mostly used)
⇒ Different strategies to produce high order meshes from the initiallinear mesh
I HOPR Output: HDF5 mesh file
3 / 22
Page 4
HOPR output: HDF5 mesh file
I Idea: Direct parallel readinI HOPR sorts elements along a Space-Filling Curve
⇒ 1D element list, can be split into arbitrary number of domainsI Element-per-element datablocks:
high order node coordinates, side info (BC, connectivity)
⇒ Split global arrays into non-overlapping blocks during parallel readin!
HDF5 mesh file
globalattributes
BCNames
BCTypes
ElemInfo SideInfoNodeCoordselem1elem2elem3
elem4elem5elem6
globalattributes
BCNames
BCTypes
globalattributes
BCNames
BCTypes
globalattributes
BCNames
BCTypes
computenLocElemsoffsetElem
computenLocElemsoffsetElem
computenLocElemsoffsetElem
elem1elem2elem3
elem4elem5elem6
elem7elem8
elem7elem8
offset/sizeSidesNodes
offset/SizeSidesNodes
offset/SizeSidesNodes
buildelement
connectivity
buildelement
connectivity
buildelement
connectivity
rank
#0
rank
#1
rank
#2
4 / 22
Page 5
HOPR Mesh Curving Flowchart
commercial mesher: ICEM, ANSA,... (cgns unstruct. format) linear unstructured hybrid mesh
curved boundary faces
Normal vector(s) at surface nodes
point-normalsICEM
spectral elementssurface
subdivision
ICEM Cheb.-Lobattonodes on edges
curved boundary edges
blend curved edges to face
blend curved edgesto inner faces
blend curved facesto volumes
subdividedsurface mesh
curved high order mesh
block-structured mesh(cgns struct. format)
Agglomerationof subblocks to
curved hexahedra
5 / 22
Page 6
HOPR Mesh Curving Flowchart
commercial mesher: ICEM, ANSA,... (cgns unstruct. format) linear unstructured hybrid mesh
curved boundary faces
Normal vector(s) at surface nodes
point-normalsICEM
spectral elementssurface
subdivision
ICEM Cheb.-Lobattonodes on edges
curved boundary edges
blend curved edges to face
blend curved edgesto inner faces
blend curved facesto volumes
subdividedsurface mesh
curved high order mesh
block-structured mesh(cgns struct. format)
Agglomerationof subblocks to
curved hexahedra
5 / 22
Page 7
Curved edges from normal vectors
Get normal vectors at linear surface mesh points:
1. In HOPR without additional data:a) Reconstruction via linear surface meshb) Analytic functions (sphere ,cylinder ...)
2. Additional point-normal file(from CAD,...)
⇒ Generate curved edges
6 / 22
Page 8
Curved edges from normal vectors
Get normal vectors at linear surface mesh points:
1. In HOPR without additional data:a) Reconstruction via linear surface meshb) Analytic functions (sphere ,cylinder ...)
2. Additional point-normal file(from CAD,...)
⇒ Generate curved edges
6 / 22
Page 9
Multiple normal vectors to recover sharp edges & corners
p1
p2
n1
t 1
e p1
p2
n1
t 1
en2
(a) (b)
(a) 1 normal vector, tangential vector by projection
(b) 2 normal vectors, tangential vector by cross product
7 / 22
Page 10
Blending curved edges to curved faces
=12 { X
a X
b X
c− X
t }
= X a X b − X t
Superposition of linear blended pairs of curved edges (Coons patches)
8 / 22
Page 11
HOPR Mesh Curving Flowchart
commercial mesher: ICEM, ANSA,... (cgns unstruct. format) linear unstructured hybrid mesh
curved boundary faces
Normal vector(s) at surface nodes
point-normalsICEM
spectral elementssurface
subdivision
ICEM Cheb.-Lobattonodes on edges
curved boundary edges
blend curved edges to face
blend curved edgesto inner faces
blend curved facesto volumes
subdividedsurface mesh
curved high order mesh
block-structured mesh(cgns struct. format)
Agglomerationof subblocks to
curved hexahedra
9 / 22
Page 12
High Order Surface by Subdivision
linear surface mesh
I Mesh generator: subdivision of the initial mesh while keeping newpoints on the CAD surfaces
⇒ Polynomial surface patches by interpolation
10 / 22
Page 13
High Order Surface by Subdivision
after one subdivision
I Mesh generator: subdivision of the initial mesh while keeping newpoints on the CAD surfaces
⇒ Polynomial surface patches by interpolation
10 / 22
Page 14
High Order Surface by Subdivision
after two subdivisions
I Mesh generator: subdivision of the initial mesh while keeping newpoints on the CAD surfaces
⇒ Polynomial surface patches by interpolation
10 / 22
Page 15
Blending curved faces to curved volume
1. Curved face on boundary
2. Blend adjacent faces with linear edges
3. Blending of all element faces → volume mapping
4. Generalized for all element types (tets,pyramids,prisms,hexas)
11 / 22
Page 16
Blending curved faces to curved volume
1. Curved face on boundary
2. Blend adjacent faces with linear edges
3. Blending of all element faces → volume mapping
4. Generalized for all element types (tets,pyramids,prisms,hexas)
11 / 22
Page 17
Blending curved faces to curved volume
1. Curved face on boundary
2. Blend adjacent faces with linear edges
3. Blending of all element faces → volume mapping
4. Generalized for all element types (tets,pyramids,prisms,hexas)
11 / 22
Page 18
Blending curved faces to curved volume
1. Curved face on boundary
2. Blend adjacent faces with linear edges
3. Blending of all element faces → volume mapping
4. Generalized for all element types (tets,pyramids,prisms,hexas)
11 / 22
Page 19
Example: DLR-F6 Wing-Body-Nacelle Configuration
I Test case of an idealized passenger jetliner
I CAD geometry provided as STEP file
I Element layer on geometry ∼ 3600 hexa∼ 400 prism
I Volume grid : ∼ 225000 tetrahedra∼ 2900 pyramids
12 / 22
Page 20
Example: DLR-F6 Wing-Body-Nacelle Configuration
13 / 22
Page 21
Example: DLR-F6 Wing-Body-Nacelle Configuration
⇒ Visualization of polynomial patches shows smooth boundary surfaceelements
14 / 22
Page 22
HOPR Mesh Curving Flowchart
commercial mesher: ICEM, ANSA,... (cgns unstruct. format) linear unstructured hybrid mesh
curved boundary faces
Normal vector(s) at surface nodes
point-normalsICEM
spectral elementssurface
subdivision
ICEM Cheb.-Lobattonodes on edges
curved boundary edges
blend curved edges to face
blend curved edgesto inner faces
blend curved facesto volumes
subdividedsurface mesh
curved high order mesh
block-structured mesh(cgns struct. format)
Agglomerationof subblocks to
curved hexahedra
15 / 22
Page 23
Hexa-meshes: Agglomeration
I Start from fine block-structured meshes
⇒ Coarsen by a fixed factor
⇒ Use inner points to generate high order element mapping
16 / 22
Page 24
Agglomeration Example: Serrated nozzle I
I Test case provided by Andrei Cimpoeru, Imperial College
I 85 blocks, 20.6M linear elements, 21.8M points, 0.5 GB CGNS file
I Agglomeration of 4x4x4 elements ⇒ 322,110 P4-elements
I HOPR used ≈ 6 GB RAM, run for ≈ 2 minutes(1 min eliminate duplicate nodes, 40s Jacobian checker)
17 / 22
Page 25
Agglomeration Example: Serrated nozzle II
all elements curvedinvalid elements inside the volume
only curved at wall BCinvalid elements at the boundary
#elems with Jscaled < 0 0− 0.1 0.1− 0.2 0.2− 0.3 ≥ 0.3
all linear 0 0 65 204 321,841all curved 10 737 896 1,684 318,783curved BC 61 6 70 216 321,757
18 / 22
Page 26
Agglomeration Example: Serrated nozzle II
1st layer curvedinvalid elements
2 layers curved⇒ all elements valid
#elems with Jscaled < 0 0− 0.1 0.1− 0.2 0.2− 0.3 ≥ 0.3
all linear 0 0 65 204 321,841all curved 10 737 896 1,684 318,783curved BC 61 6 70 216 321,757
1st layer 10 50 104 376 321,5702 layers 0 0 197 520 321,393
18 / 22
Page 27
Agglomeration Example: Serrated nozzle II
3 layers curved⇒ all elements valid
4 layers curved⇒ all elements valid
#elems with Jscaled < 0 0− 0.1 0.1− 0.2 0.2− 0.3 ≥ 0.3
all linear 0 0 65 204 321,841all curved 10 737 896 1,684 318,783curved BC 61 6 70 216 321,757
1st layer 10 50 104 376 321,5702 layers 0 0 197 520 321,393
3 layers 0 0 178 667 321,2654 layers 0 0 160 746 321,204
18 / 22
Page 28
Ongoing Work on Boundary Layer Meshes
I Structured grids:
⇒ Agglomeration (works in current HOPR release)
I Unstructured grids: Only curving the first element at the surfaceleads to inverted elements
⇒ For prismatic layers of linear elements:Blending of element stack with curved boundary face (not released)
⇒ Insert boundary layers afterwards (not released)
19 / 22
Page 29
Boundary layer Insertion
I Valid curved mesh with prismatic elements (hex, prism)I Split the first element layer
⇒ into a given number boundary layer elements (input parameter)
⇒ with a given stretching (input parameter)
20 / 22
Page 30
Boundary layer Insertion
I Valid curved mesh with prismatic elements (hex, prism)I Split the first element layer
⇒ into a given number boundary layer elements (input parameter)
⇒ with a given stretching (input parameter)
20 / 22
Page 31
Conclusion
I HOPR provides unstructured high order meshes
I Different curving strategies, depending on provided data
I HDF5 mesh file, designed for parallel IO
I Software is open-source
⇒ Announcement for next ICOSAHOM:David Kopriva wants to organize a mini-symposium todiscuss a standard for a high order mesh format
21 / 22
Page 32
Thanks to E. Sonnendrucker, C.-D. Munz, G. Gassner, T. Bolemann, H.Frank, and
Thank you for your attention!
www.hopr-project.org
HOPR Homepage (with Mailing List, Tutorials, Documentation...)
Contact: [email protected] , [email protected]
22 / 22
Page 33
Appendix:
23 / 22
Page 34
Mesh Format: Element Node Connectivity
I All element mappings in the mesh have the same order (watertight)I Element node connectivity defined for all orders (i , j , k loops)I Full order elements only
XI, i
ETA, j
ZETA, k
1 3
6
13 15
18
2
14
4
16
5
17
7 9
12
8
10 11
1 3
6
10
2
4 5
7 8
9
1 3
97
19 21
2725
2
8
20
26
4 6
22 24
5
23
10 12
1816
11
17
13 1514
1 3
97
14
2
8
4 65
10 11
12 13
24 / 22
Page 35
SFC Domain Decomposition
I 3D unstructured simulation (N = 4) of a turbulent flow past asphere (Re = 1000)
I Domain decomposition based on a space filling curve element sorting
25 / 22
Page 36
SFC Domain Decomposition
I SFC-DD allows arbitrary number of domains, given a-posteriori
#Domains #Elements/core #MPIneighbors (mean)128 165-166 5-23 (11.75)
25 / 22
Page 37
SFC Domain Decomposition
I SFC-DD allows arbitrary number of domains, given a-posteriori
#Domains #Elements/core #MPIneighbors (mean)128 165-166 5-23 (11.75)
1024 20-21 5-21 (11.99)
25 / 22
Page 38
SFC Domain Decomposition
I SFC-DD allows arbitrary number of domains, given a-posteriori
#Domains #Elements/core #MPIneighbors (mean)128 165-166 5-23 (11.75)
1024 20-21 5-21 (11.99)4096 5-6 5-20 (10.37)
25 / 22