Compiling basics for HPC - National Institute for ......User can link application dynamically by either setting the environment variable CRAYPE_LINK_TYPE to dynamic or by adding -dynamic

Compiling basics for HPC

Dr. Shiquan Su NICS

from source code to executable

• edit source code: hello_world.cpp • compile object file: hello_world.o • link to executable: hello_world (a.out)

edit source code

• main program (driver) *.cpp • header files ( required at compile time) *.h • function / subroutine files (libraries) (required

at link time) *.cpp *.h

compile object file

• main program driver.cpp à driver.o • header files ( required at compile time) *.h • function / subroutine files (required at link

time) *.cpp à *.o à lib.a

link to executable

• driver, libraries: driver.o , lib.a à executable (a.out) driver.cpp à driver.o

simple example: using command line shiquan1@darter2:~/testcompiler> g++ hw.cpp shiquan1@darter2:~/testcompiler> ./a.out Hello World! CPP program. shiquan1@darter2:~/testcompiler> cat hw.cpp #include <iostream> using namespace std; int main (){ cout << "Hello World! CPP program.\n"; return 0; }

MPI Compiler

• mpicxx • check the mpi package for: mpi header file: mpi.h mpi library: libmpi.a mpi compiler: mpicxx mpi launcher: mpirun (see the openmpi library on Keeneland as example)

compiler wrapper (Cray) Cray provides a convenient set of compiler wrapper commands that should be used for compiling and linking programs to be run on the compute nodes. The wrapper invokes the corresponding back-end compiler in the currently loaded programming environment. Invoking the wrappers will automatically: •  Import the path for key header files like 'mpi.h' and other

libraries •  Link in the MPI and other fundamental header files and libraries

needed to run on Cray system •  Automatically include and link in header files and libraries

loaded via module •  Specify the correct target processor arguments •  Provides a common interface to all compilers •  mpi launcher: aprun.

compiler wrapper Cray GNU Intel PGI C cc craycc gcc icc pgcc C++ CC crayCC g++ icpc pgCC Fortran ftn crayftn gfortran ifort pgfortran

9

The following table lists the compiler wrappers and the corresponding back-end compilers available on the system

example: Darter

Compilers on Darter • The default compiler is the Cray compiler

(CCE) • The available compilers on Darter are : Cray

(default), GNU and Intel. • The PGI and Pathscale compilers are no

longer available

11

PGI Cray Intel GNU Description -fast optimized

by default -fast –no-ipo -O3 –ffast-math standard optimization

-mp -homp -openmp -fopenmp enable OpenMP support

-Mfixed -Mfree

-ffixed -ffree

-fixed -free

-ffixed-form -ffree-form

process Fortran source as fixed or free format

Common Compiler Options

Using Dynamic Linking on Darter

Some application requires the use of dynamic linking and shared libraries on the compute nodes. Example of these include software from third-party vendor and popular applications such as Python and OpenFoam. Darter fully supports the use shared libraries and dynamic linking on the compute nodes. All Cray's provided software on Darter were also built with shared libraries support and can be used by dynamically linked application.

User can link application dynamically by either setting the environment variable CRAYPE_LINK_TYPE to dynamic or by adding -dynamic flag during linking. The following show some examples on building dynamically linked application and shared libraries. Building Hello World program with dynamic linking: export CRAYPE_LINK_TYPE=dynamic cc -o hello hello.c Same example as above, but using linker flag: cc -dynamic -o hello hello.c The same procedures apply to ftn and CC to build Fortran and C++ code with dynamic linking, respectively.

13

Using Dynamic Linking on Darter Example

You may find more info for Darter machine here: https://www.nics.tennessee.edu/computing-resources/darter/compiling

Use Makefile to manage dependence

target [target ...]: [component ...] Tab ↹[command 1] . . . Tab ↹[command n]

Use flags make.inc FC = mpif90 CC = mpicc NOOPT = -O0 FCFLAGS = -O3 CCFLAGS = -O3 FCLOADER = $(FC) CCLOADER = $(CC) FCLOADFLAGS = $(FCFLAGS) CCLOADFLAGS = $(CCFLAGS) BLASLIB = -L$MKL_DIR -mkl -lpthread -lm LAPACKLIB = LIBS = $(LAPACKLIB) $(BLASLIB)

realistic example: compiling Scalapack and test driver using makefile

• edit driver • the reference for the library can be found at: http://www.netlib.org/scalapack/explore-html/ • edit make.inc • Scalapack depends on blas and lapack

libraries