CÉCI/CISM HPC training sessions Use of Matlab on … · Calcul intensif et Stockage de Masse CÉCI/CISM HPC training sessions Use of Matlab on the clusters

Calcul intensif et Stockage de Masse

CÉCI/CISM HPC training sessionsUse of Matlab on the clusters

CISM Typical usage...

InteractiveType in and get an answer

SequentialPerform tasks one after theother

BatchSubmit job and fetch results

ParallelPerform multiple tasks at thesame time

CISM One more obstacle: Matlab Licensing

Using Matlab in batch modeWith Matlab (e.g. your computer or Lagrange)Without Matlab (e.g. the clusters)

Using Matlab in parallelWith no effortWith little effortWith a lot of effort

November 2015 damien.francois@uclouvain.be

Parallel Matlab on the cluster

CISM Using Matlab in Batch mode

You might be used to ...

... but no GUI in batch mode !

Two methods for launching a script:

You have a script myscript.m:

You have a function "function a = myfun(x,y) ..."

You need to adapt your Matlab script

Prompts

Figures

Make sure to automate any setup that you usually do byhand: adding paths, loading data, saving results, etc.

No more clicks!

Prompts

Figures

Put all 'configuration' values in a file and load it

Prompts

Figures

GUIUse 'print' to put your

graphics to a file.

Or avoid plotting at all and dothat interactively atpostprocess time

Prompts

Figures

Simplest solution: avoid GUI'sand use command line version

And then launch it !

Options

Launch

-nodisplay: do not try to display plots-nodesktop: do not launch full GUI-nojvm: do not launch Java support

(do not use in recent versions of Matlab)-nosplash: do not display splashscreen

And then launch it !

Use 'screen' for unattended execution

As soon as you pressENTER, the screen is

cleared and you are in a 'screen' session

We are now in a 'screen' session

We launch Matlaband we 'detach' with

CTRL-a d

We are now detachedand disconnected.

We connect backto lm9 and 'reattach'

to the 'screen' session

And we find our result

CISM Your turn..1. Connect to lm9 “ssh -X lm9.cism.ucl.ac.be”

Login: tutoXX where XX is a number between 01 and 19Password: xxxxxxxx

2. Copy directory ~dfr/matlab/batch to your directory and “cd” there

3. Load module matlab/R2010a3. Launch Matlab

4. Run 'go' to see what it does (you will have to set the paths correctly: File > Set Paths myTools)

5. Edit go.m so as to be able to run it in batch6. Quit Matlab

7. Test your Matlab script in 'batch mode'

8. Make a longer test with “screen”

CISM Your turn..

Step 2 : create the SGE script

Step 3 & 4 : submit & wait

Step 5 : get the results

Tell SGE you need Matlabuse -nosplash -nodesktop -nodisplay -nojvmuse redirection < or option -r

CISM Your turn..

Using Matlab in batch modeWith Matlab (e.g. your computer or lagrange)Without Matlab (e.g. the clusters)

Parallel matlab on the cluster

CISM Dealing with the license

Number of licenses is limited!

Option 1 : Compile Matlab to C...

Issues

IssuesUse -a to add resources (additional code or mat files)

Addpath are forbidden in compiled code. Protect them with if ~isdeployed addpath(...)end

Within Matlab:

myfunction must be a function, not a script

Issues

Option 1 : ... and deploy with MCR

Issues

CISM Your turn..

1. Connect to lm9 “ssh -X lm9.cism.ucl.ac.be”Login: tutoXX where XX is a number between 01 and 19Password: *******

2. Copy directory ~dfr/matlab/compile to your directory and “cd” there

3. Load module matlab/R2014a and launch Matlab 4. Compile go_f.m (note if ~isdeployed ) mcc -a myTools/ -m go_f.m

5. Connect to Hmem with your CÉCI login

6. Copy your 'compile' directory from lm97. Load MCR module version 713 (no need to copy/install MCR)

9. Run go_f (no need for run_go_f.sh)

Matlab is not installed on Manneback but the MCR is

Option 2 : Develop with Matlab, run with Octave

“a language that is mostly compatible with Matlab”GPL license, free

Issues

You have a script myscript.m. Rather than:

Simply write

The other option '-r' becomes '--eval '

Issues

Toolboxes

Multithreading

Not as good as Matlab's

CISM Your turn..

1. Connect to lm9 “ssh -X lm9.cism.ucl.ac.be”Login: tutoXX where XX is a number between 01 and 19Password: ???????

2. Copy your 'batch' directory into 'octave' : cp -r batch octave

3. Go to your octave directory 4. Load the octave module 5. Launch octave

6. Run go_f

7. exit octave

8 Launch “octave –eval go_f”

Using Matlab in batch modeLaunch a script, get resultsDeal with licenses

Parallel matlab on the cluster

CISM No effort: Multithreading

CISM No effort: MultithreadingmaxNumCompThreads

exists since R2007b, defaults to one core

defaults to available number of cores from R2008a

marked deprecated since R2009b warning off MATLAB:maxNumCompThreads:Deprecated to suppress warning

CISM No effort: Multithreading

Ask for several coresin the submission script

Alone on the node With 2 Matlab running concurrently

CISM No effort: MultithreadingElement wise operations and linear algebra

CISM No effort: MultithreadingMore with the Parallel Computing Toolbox

Matlab on the cluster

CISM Little effort: embarrassingly parallel

Embarrassingly parallel programs (fork-join without communication / synchronization)

e.g. Parameter sweep, Monte carlo, Multiple data sets

Outer-loop splitting:

e.g. Loop over parameter

Program-level parallelism

Inner-loop splitting :

e.g. Loop over a vector index

Data-level parallelism

Outer-loop splitting with Slurm's srun

Requires one license per task !

But we know what to do, don't we ?

Try it yourself! : ~dfr/matlab/embarrassingly*

Parallel Computing Toolbox

Parfor, spmd and distributed arrays

Can be compiled !

Compilation can fail :(

Matlab 3rd party peval : jPar

On the solvers: >> jpar_solver(['hostname']);

On the server: >> a = rand(100,100,10) + i*rand(100,100,10);

>> [V,D] = jpar_client('eig', a)In

Matlab 3rd party peval : Multicore

On the slaves: >> startmulticoreslave

On the master: >> for i=1:10; a{i} = rand(100,100) ;end

>> cellRes = multicoremaster(@eig, a)

Compil

Compile (e.g. lm9):

matlab -r "mcc -a myTools/ -a multicore/ -m go_adapted_function_compile_peval;"

matlab -r "cd multicore/;mcc -a ../myTools/ -a ../ -m startmulticoreslave;exit"

Submit (e.g. Manneback):

Try it yourself! : ~dfr/matlab/multicore_compiled

Octave peval: parcellfun, pararrayfun

A = {rand(100,100), rand(100,100), rand(100,100)}

Res = parcellfun(2, @eig, A)Mult

srun --ntasks=1 --cpus-per-task=4 octave < go_f_parcellfun.mTry it ! ~dfr/matlab/parcellfun_octave

A = rand(100,100,3)

Res = pararrayfun(2, @eig, A)

Octave peval: multicore

On the slaves: >> startmulticoreslaves

On the master: >> for i=1:10; a{i} = rand(100,100) ;end

>> cellRes = multicoremaster(@eig, a)m

ultino

Try it ! ~dfr/matlab/multicore_octave

Matlab on the cluster

CISM A lot of effort: explicitly parallel

Explicitly parallel programs

SPMD & MPMD

Communications handled explicitly by the user

Matlab not specifically good at it..

OpenMP-like construct based on MPI for distributed memory

Lincoln Laboratory MatlabMPI

Lincoln Laboratory pMatlab

http://www.walkingrandomly.com/?p=1795

OpenMP Mex files

Octave: reval (parallel package)

3rd party toolboxes

Compile

Octave

Octave and MPI

CISM Batch Matlab/Octave

● Scripts need adjustment● Batch processing with Matlab

● Use 'screen'

● Batch processing without Matlab● Compile with mcc● Dev. with Matlab, Prod. with Octave

CISM Parallel Matlab/Octave

● No effort: Matlab Multithreading● Some effort: embarrassingly parallel

● Matlab: Jpar, multicore● Octave: parcelleval, multicore

● More (too much?) effort● Matlab: SPMD, MPI toolboxes● Octave: parallel, openmpi_ext

Coming

Octave

----------------------------------------------MATLAB | MATLAB | MATLAB | Release | Component | Compiler | | Runtime (MCR) | Version | ----------------------------------------------R14 (7.0) | 7.0 | 4.0 | R14SP1 (7.0.1) | 7.1 | 4.1 | R14SP2 (7.0.4) | 7.2 | 4.2 | R14SP3 (7.1) | 7.3 | 4.3 | R2006a (7.2) | 7.4 | 4.4 | R2006b (7.3) | 7.5 | 4.5 | R2007a (7.4) | 7.6 | 4.6 | R2007b (7.5) | 7.7 | 4.7 | R2008a (7.6) | 7.8 | 4.8 | R2008b (7.7) | 7.9 | 4.9 | R2009a (7.8) | 7.10 | 4.10 | R2009b (7.9) | 7.11 | 4.11 | R2009bSP1 (7.9.1) | 7.12 | 4.12 | R2010a (7.10) | 7.13 | 4.13 | R2010b(7.11) | 7.14 | 4.14 |R2010bSP1 (7.11.1)| 7.14.1 | 4.14.1 |R2011a(7.12) | 7.15 | 4.15 |R2011b(7.13) | 7.16 | 4.16 |R2012a(7.14) | 7.17 | 4.17 |R2012b(8.0) | 8.0 | 4.18 |----------------------------------------------

CÉCI/CISM HPC training sessions Use of Matlab on … · Calcul intensif et Stockage de Masse CÉCI/CISM HPC training sessions Use of Matlab on the clusters

Documents

INTERNATIONAL MILITARY SPORT COUNCIL (CISM) · FRIENDSHIP.....

CÉCI/CISM HPC training sessions Use of Matlab on the...

Introduction to HPC at UCL - UCLouvain. 16th 2014...

CISM Courseware

Connecting with Secure SHell - UCLouvain · 2016. 10....

Presentation CISM

Critical Incident Stress Management: CISM Implementation...

CISM sertifikacija

CISM Middle Distance

Meticulous CISM Study Guide

CRITICAL INCIDENT STRESS MANAGEMENT (CISM) … and... ·...

An introduction to checkpointing - UCLouvain · An...

2014 CISM WORLD MILITARY WRESTLING...

CISM Course 2004Minus Mayo · Critical Incident Stress...

CISM - HUKL

CISM Candidate's Guide