OpenFOAM: Introduction, Capabilities and HPC Needs

Cyprus Advanced HPC Workshop Winter 2012. Tracking: FH6190763 Feb/2012

OpenFOAM: Introduction, Capabilitiesand HPC Needs

Hrvoje Jasak

[email protected]

Faculty of Mechanical Engineering and Naval Architecture

University of Zagreb, Croatia

OpenFOAM: Introduction, Capabilities and HPC Needs – p. 1


Background

Objective

• Provide basic information about OpenFOAM software from User’s viewpoint

◦ What is it? How do I run it? What capabilities does it have?

• Examine mode of use and HPC requirements in CFD

• Present examples of capabilities relevant to your needs

Topics

• OpenFOAM: Executive Overview

• Running OpenFOAM

• Points of interest: why consider OpenFOAM

• HPC requirements in CFD

• Examples of capabilities1. External aerodynamics

2. Free surface flows3. CFD and geometric shape optimisation

4. OpenFOAM Turbo Tools

5. Vehicle soiling: Lagrangian tracking, finite area method and liquid film

6. Coupled simulations and Fluid-Structure coupling (FSI)



OpenFOAM: Executive Overview

What is OpenFOAM?

• OpenFOAM is a free-to-use Open Source numerical simulation software withextensive CFD and multi-physics capabilities

• Free-to-use means using the software without paying for license and support,including massively parallel computers : free multi-1000-CPU CFD license!

• Software under active development, capabilities mirror those of commercial CFD

• Substantial installed user base in industry, academia and research labs

• Possibility of extension to non-traditional, complex or coupled physics:Fluid-Structure Interaction, complex heat/mass transfer, internal combustionengines, nuclear

Main Components

• Discretisation: Polyhedral Finite Volume Method, second order in space and time

• Lagrangian particle tracking, Finite Area Method (2-D FVM on a curved surface)

• Massive parallelism in domain decomposition mode

• Automatic mesh motion (FEM), support for topological changes

• All components implemented in library form for easy re-use

• Physics model implementation through equation mimicking



Implementing Continuum Models

Equation Mimicking

• Natural language of continuum mechanics: partial differential equations

• Example: turbulence kinetic energy equation

∂k

∂t+∇•(uk)−∇•[(ν + νt)∇k] = νt

[

1

2(∇u+∇uT )

]

2

−ǫo

kok

• Objective: represent differential equations in their natural languag e

solve(

fvm::ddt(k)+ fvm::div(phi, k)- fvm::laplacian(nu() + nut, k)== nut*magSqr(symm(fvc::grad(U)))- fvm::Sp(epsilon/k, k));

• Correspondence between the implementation and the original equation is clear

• Library-based implementation of custom physical models with full freedom insolution algorithms. New: support for multi-equation implicit coupling



OpenFOAM: Executive Overview

Physical Modelling Capability Highlights

• Basic: Laplace, potential flow, passive scalar/vector/tensor transport

• Incompressible and compressible flow: segregated pressure-based algorithms

• Heat transfer: buoyancy-driven flows, conjugate heat transfer

• Multiphase: Euler-Euler, VOF free surface capturing and surface tracking

• RANS for turbulent flows: 2-equation, RSTM; full LES capability

• Pre-mixed and Diesel combustion, spray and in-cylinder flows

• Stress analysis, fluid-structure interaction, electromagnetics, MHD, etc.



Running OpenFOAM

Structure of OpenFOAM

• OpenFOAM is assembled from components◦ Foundation libraries, containing discretisation, mesh handling etc. in re-usable

form. Functionality shared across many application

◦ Physical modelling libraries: thermo-physical models (liquids and gasses),viscosity models, turbulence models, chemical reactions interface

◦ Utilities: mesh import and manipulation, parallel processing, post processorhook-up (reader module) and data manipulation

◦ Customised, purpose-written and optimised executables for each physicssegment. All are founded on common components

◦ Linkage for user extensions and on-the-fly data analysis

OpenFOAM Executable

• Custom executable for specific physics segment: few 100s of lines of code

• Easy to read, understand, modify or add further capability

• Existing code is used as a basis for own development: simulation environment

• Low-level functions, eg. mesh handling, parallelisation, data I/O handledtransparently: no need for special coding at top level



Point of Interest

Why Consider OpenFOAM

• Open architecture◦ Access to complete source: no secret modelling tricks, no cutting corners

◦ Both community-based and professional support available

◦ Common platform for new R&D projects: shipping results of research into thehands of a customer with no delay

• Low-cost CFD◦ No license cost, portable to any computing platform (IBM Blue Gene)

◦ Easily scriptable and embeddable: “automated CFD” and optimisation

◦ Efficient on massively parallel computers, portable to new comms protocols

• Problem-independent numerics and discretisation◦ Tackle non-standard continuum mechanics problem: looking beyond the

capabilities of commercial CFD

• Efficient environment for complex physics problems

◦ Tackling difficult physics is made easier through equation mimicking

◦ Utility-level tools readily available: parallelism, moving mesh◦ Track record in non-linear and strongly coupled problems

◦ Excellent piece of C++ and software engineering! Decent piece of CFD



Massively Parallel Scaling

• Mesh size and resolution environment in 21st century are revolutionary differentfrom (the comfortable) 1990s: thousands of processors, a billion cell mesh

• Complex hardware architecture without proper programming support complicatesthe programming and roll-out into real-world applications

• Still learning GPU lessons: potentially changing the way we write CFD codes backtowards structured mesh codes and fixed internal infrastructure

• What about parallel CFD clouds: do we need custom infrastructure?



Examples of Simulation

Example of Capabilities of OpenFOAM Relevant to Your CFD Needs

• This is only a part of the OpenFOAM capabilities!

• Chosen for relevance and illustration of the range of capabilities rather thanexhaustive illustration of range of capabilities

• In some cases, simplified geometry is used due to confidentiality

• Regularly, the work resulted in a new solver; in many cases, it is developed as anextension or combination of existing capabilities

Description of Simulation and Setup

• Physics and numerical method setup

• Standard or customised solver; details of mesh resolution and customisation



External Aerodynamics with LES

Detached Eddy Simulation for External Aerodynamics

• Pushing state-of-the-art by applying Detached Eddy Simulation (DES) to full carbody external aerodynamics simulations: native solver and mesher, no change

• Increase in simulation cost over transient RANS is over 1 order of magnitude!

• Controlling the Cost of Full Car DES :

◦ Automated meshing and simulation environment, from STL surface of the carbody to averaged DES results and forces

◦ Hex-core mesher with near-wall layers and local refinement: mesh isdesigned to make it good for second-order LES numerics with minimal cost

◦ No parallel license cost of CFD solver: simulations run on approx. 200 CPUs

• Improvement in CD, CL and force-per-component predictions due to bettercapturing of turbulence and transient flow features

Reproduced with permission SAE 2009-01-0333, Islam et.al.



Free Surface Flows

Free Surface Flow Simulations

• Efficient handling of interface breakup

• Accurate handling of dominant surface tension: no parasitic velocity

• OpenFOAM solver: interFoam , no modifications

Ink-Jet Printer Nozzle, d = 20µm: Breakup Under Dominant Surface Tension

Complex Surface Breakup Phenomena: Sloshing and wet wall impact



Free Surface Flow

Three-Phase Free Surface Flow in a Tundish

• 3 phases with extreme density ratio: liquid steel, liquid slag, air (7000:2500:1)

• Note the presence of multiple phase-to-phase interfaces: using consistentdiscretisation for multiple phase α equations

• Temperature-dependent properties of slag and steel

• Simultaneous filling and pouring with large outlet velocity

• Examples

◦ Simulation of a Turbo-Stop device for initial fill

◦ Simulated break-down of a slag layer: detail of flow at exit nozzle



Free Surface Flow

Naval Hydrodynamics Examples

• Naval hydrodynamics simulations require a special formulation of the free surfaceflow algorithm for robustness and accuracy

◦ “Traditional” steady-state VOF solver with under-relaxation

◦ Modified formulation for ship flows

• Incoming wave boundary conditions

• Numerical beach: damped boundary regionremoves wave reflection

• Support for dynamic mesh features

• Accurate reconstruction of the pressure fieldfor moving bodies

• Post-processing and data extraction



Complex Mesh Motion

6-DOF Floating Body in Free Surface Flow

• Flow solver : turbulent VOF free surface, with moving mesh support

• Mesh motion depends on the forces on the hull: 6-DOF solver

• 6-DOF solver : ODE + ODESolver energy-conserving numerics implementedusing quaternions, with optional elastic/damped support

• Variable diffusivity Laplacian motion solver with 6-DOF boundary motion as theboundary condition for the mesh motion equation

• Topological changes to preserve mesh quality on capsize

• Coupled transient solution of flow equations and 6-DOF motion, force calculationand automatic mesh motion: custom solver is built from library components



Complex Mesh Motion

Dynamic Mesh Examples of Complex Combination of Motion and Sliding



Geometric Shape Optimisation

Geometric Shape Optimisation with Parametrised Geometry

• Specify a desired object of optimisation and use the parametrisation of geometryto explore the allowed solution space in order to find the minimum of theoptimisation objective

objective = f(shape)

1. Parametrisation of Geometry• Computational geometry is complex and usually available as the

computational mesh: a large amount of data

• Parametrisation tool: RBF mesh morphing , defining deformation at a smallnumber of mesh-independent points in space

2. CFD Flow Solver is used to provide the flow solution on the current geometry, inpreparation for objective evaluation

3. Evaluation of Objective : usually a derived property of the flow solution

4. Optimiser Algorithm : explores the solution space by providing sets of shapecoordinates and receiving the value of objective. The search algorithm iterativelylimits the space of solutions in search of a minimum value of objective



Geometric Shape Optimisation

Example: HVAC 90 deg Bend: Flow Uniformity at Outlet

• Flow solver: incompressible steady-turbulent flow, RANS k − ǫ model; coarsemesh: 40 000 cells; 87 evaluations of objective with CFD restart

• RBF morphing: 3 control points in motion, symmetry constraints; 34 in total

• Objective: flow uniformity at outlet plane

iter = 0 pos = (0.9 0.1 0.1) v = 22.914 size = 0.69282iter = 5 pos = (0.1 0.1 0.1) v = 23.0088 size = 0.584096iter = 61 pos = ((0.990164 0.992598 0.996147) v = 13.5433 size = 0.000957122



OpenFOAM Turbo Tools

General Grid Interface

• Turbomachinery CFD requires additional features: implemented in library form

• General Grid Interface (GGI) and its derived forms

◦ Cyclic GGI

◦ Partial overlap GGI

◦ Mixing plane interface (under testing)

• Implementation and parallelisation is complete: currently running validation casesin collaboration with commercial clients and Turbomachinery Working Group

• Other turbo-related components in pipeline: harmonic balance solver solver

• Library-level implementation allows re-use of GGI beyond turbomachinery



Multi-Phase Compressibility

Simulation of Under-Water Explosions

• This is ongoing collaboration with Johns Hopkins APL, Penn State University andWikki: working hard for almost a year

• Dominating effects of compressibility in air and water: massive change in density,with propagating pressure waves

• Pressure ranges from 500 bar to 20 Pa

• Stiff numerics: collapse of over-expanded bubble due to combined compressibilityof both phases are the basis of the phenomenon

• Test cases: Rayleigh-Plesset oscillation, undex under a plate, explosion

• Eric Paterson, Scott Miller, David Boger, Penn State; Ashish Nedungadi, JHU-APL



Multi-Phase Compressibility

First Simulations of Under-Water Explosions: Eric Paterson, Penn State

• Bubble of high initial pressure expands after explosion

• Initial pressure pulse is very fast - with little effect

• Bubble collapse creates re-entrant jer which pierces the free surface

• Stability problems resolved in segregated solver

• Next phase: block-coupled p− α solution algorithm



Lagrangian Phase Model

Lagrangian Particle Modelling

• Particles are modelled in Lagrangian framework: tracked through fluid domain

• Equation of motion for a Lagrangian particle:

mpdvp

dt=

1

2ρ|v − vP |(v − vP )

d2pπ

4CD +mpg

ρl − ρ

ρl+ F∇p

where◦ mp is the particle mass

◦ vp is the particle velocity

◦ v is fluid velocity◦ dp is particle diameter

◦ CD is the drag coefficient◦ g is the gravitational acceleration

◦ ρl is the particle (liquid) density

◦ ρ is the fluid density

◦ F∇p is the force on the particle due to the fluid pressure gradient




Integration of Motion Equation

• Motion equation for a particle is an Ordinary Differential Equation (ODE)

• During integration, particle traverses a number of cells and continuous phaseproperties are updated based on its position

• For accurate integration, motion equation is formulated as an ODE and solverusing an ODE solver

Particle Tracking

• Cell-to-face-to-cell tracking: no inverted Jacobians, no lost particles

• If more accurate integration is necessary, sub-cycling is used

Integration stencil U




Solid Particles with Wall Interaction

• Using particle tracking code with physics equations attached to a particles

• Particle-to-particle interaction: proximity model with correction

• Wall interaction model: bounce with energy loss

Diesel Spray Model

• Consists of a parcel, injector, spray and a set of sub-models

• parcel class: a particle represents a population of droplets: mean diameter andstatistical distribution of particle size and other properties

• Cloud of parcels = Diesel spray: spray

• Example: Junwei Su, Imperial College and Xi’an Jiaotong University, China



Finite Area Method

Finite Area Discretisation

• Finite Area Method discretised equations on a curved surface in 3-D

• Surface is discretised using polygonal faces. Discretisation takes into accountsurface curvature . A level of smoothness is assumed in calculation of curvatureterms

• Surface motion is allowed: decomposed into normal and tangential motion

• Nomenclature for a surface element P and its neighbour N

nP

Nne

PNd e

eL

j

i

ee’ m

n f

SP



Thin Liquid Film Model

Modelling Assumptions: 2-D Approximation of a Free Surface Flow

• Isothermal incompressible laminar flow

• Boundary layer approximation:◦ Tangential derivatives are negligible compared to normal

◦ Normal velocity component is negligible compared to tangential

◦ Pressure is constant across the film depth

• Similitude of the flow variables in the direction normal to the substrate◦ Prescribed cubic velocity profile

n

Sw

v

vfs

Sfs

pgvg

md,i

Dependent variables: h and v̄σ

vd,iv̄

h

v̄ = 1h

h∫

0

v dh

η = nh, 0 ≤ n ≤ h

v(η) = vfs• diag (aη + bη2 + cη3)



Thin Liquid Film Model

Validation: Droplet Spreading Under Surface Tension

• Liquid film equations governing the flow; self-similar velocity profile

• Droplet spread driven by gravity and counteracted by surface tension

• Equation set possesses an (axi-symmetric) analytical solution: validation ofimplementation and numerics

0

5e-06

1e-05

1.5e-05

2e-05

2.5e-05

3e-05

3.5e-05

4e-05

0 0.0005 0.001 0.0015 0.002

h, m

r, m

t = 0.000 st = 0.015 s

Analytical solution

Implementation of Liquid Film Model Volume-Surface-Lagrangian Coupling

• Full second order in space and time on curved moving surfaces

• Parallelisation bound to volumetric FVM

• Shared matrix and discretisation: possibility of close coupling with FVM



Coupling Wall Film with Spray

Volume-Surface-Lagrangian Coupling

• Coupling a volumetric flow model with Lagrangian particles for a dispersed phaseto a thin liquid film model on the solid wall Custom solver

• In terms of numerics, coupling of volumetric, surface and Lagrangian models iseasy to handle: different modelling paradigms

• Main coupling challenge is to implement all components side-by-side and controltheir interaction: volumetric-to-particle-to-surface data exchange

• Close coupling is achieved by sub-cycling or iterations over the system for eachtime-step



Close Coupling: Matrix Level

Assembling a Matrix for Conjugate Heat Transfer Problems

• OpenFOAM supports multi-region simulations, with possibility of separateaddressing and physics for each mesh: multiple meshes, with local fields

• Some equations present only locally, while others span multiple meshes

coupledFvScalarMatrix TEqns(2);

TEqns.hook(

fvm::ddt(T) + fvm::div(phi, T)- fvm::laplacian(DT, T)

);

TEqns.hook(

fvm::ddt(Tsolid) - fvm::laplacian(DTsolid, Tsolid));

TEqns.solve();

• Coupled solver handles multiple matrices together in internal solver sweeps:arbitrary matrix-to-matrix and domain-to-domain coupling



Linear Algebra and Solver Support

Example: Conjugate Heat Transfer

• Coupling may be established geometrically: adjacent surface pairs

• Each variable is stored only on a mesh where it is active: (U, p, T)

• Choice of conjugate variables is completely arbitrary: e.g. catalytic reactions

• Coupling is established only per-variable: handling a general coupled complexphysics problem rather than conjugate heat transfer problem specifically

• Allows additional models to be solved on each region without overhead: structuralstress analysis, turbulence or LES



Fluid-Structure Interaction

Fluid-Structure Coupling Capabilities in OpenFOAM

• As a Continuum Mechanics solver, OpenFOAM can deal with both fluid andstructure components: easier setup of coupling

• (Parallelised) surface coupling tools implemented in library form: facilitate couplingto external solvers without “coupling libraries” using proxy surface mesh

• Structural mechanics in OpenFOAM targeted to non-linear phenomena: considerbest combination of tools◦ Large deformation formulation in absolute Lagrangian formulation

◦ Independent parallelisation in the fluid and solid domain

◦ Parallelised data transfer in FSI coupling

• Dynamic mesh tools and boundary handling used to manipulate the fluid mesh



Deployment and Support

Deployment and Support

• OpenFOAM developed mainly on Linux/Unix: Windows and Mac OS X ports alsoexist but are not widely used. This needs to be improved: binary distribution forMac OS X and native Microsoft Windows port

• Data input/output is in file form: there is no global graphical user interface that canbe tailored to sufficient level of usability (work in progress)

• No built-in 3-D graphics; 2-D, graphing and sampling tools present

• External post-processing with integrated readers: paraFoam (open source)



Summary

Project Status Summary

• OpenFOAM is a free software, available to all at no charge: GNU Public License

• Object-oriented approach facilitates model implementation

• Equation mimicking opens new grounds in Computational Continuum Mechanics

• Extensive capabilities already implemented; open design for easy customisation

• Solvers are validated in detail and match the efficiency of commercial codes

• Open Source model dramatically drops the cost of industrial CFD

OpenFOAM in Research and Industry

• Technical development driven by Special Interest Groups & Birds-of-a-Feather

Turbomachinery Ship Hydrodynamics Simulation of Engines

Turbulence Fluid-Structure Interaction Multiphase Flows

Aeroacoustics Combustion and explosions Solid Mechanics

Documentation High Performance Computing OpenFOAM in Teaching

• Sixth OpenFOAM Workshop : Penn State University 13-16 June 2011http://www.openfoamworkshop.org


OpenFOAM: Introduction, Capabilities and HPC Needs

Documents