Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M. HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE INVERSE PROBLEMS OF SEISMIC.

Biryaltsev E.V., Galimov M.R., Demidov D.E., Elizarov A.M.

HPC CLUSTER DEVELOPMENT AND OPERATION EXPERIENCE FOR SOLVING THE

INVERSE PROBLEMS OF SEISMIC PROSPECTING

HPC - INDUSTRIAL DEPLOYMENT2

MASS MARKET READY

OIL & GAS INDUSTRY AND TECHNOLOGY REPLACEMENT3

NEW SOLUTIONS REQUIRED

PRICE DROP

PRODUCTION & EXPLORATION COMPLEXITY

NUMERICAL SIMULATION4

GRID SIMULATION METHODS

FULL WAVE INVERSION5

LOW-FREQUENCY SEISMIC SOUNDING (LFS) 6

APPLICATIONS 7

1. Oil & gas prospecting

2. Fracturing control

3. Borehole environment monitoring

REQUIREMENTS8

ACCESSIBILITY

BIG DATA ARRAYS

PERFORMANCE

VISUALIZATION

30 000 models per cycle

109 – 1012 cells

~1 Tb per model

RESEARCH & DEVELOPMENT9

1. The best performance for the massive parallelism tasks.2. High computation density per equipment units.3. Perspective technology.

GPU — EFFICIENT CHOICE

GPU CLUSTER10

UPS Eaton 9155 Power Rating 8000 / 7200 (VA/Watts)0.7 Power factor

Server>80 Tb Data storage space + SSD cache

Virtualization serverCitirix XenServer platform2 x NVIDIA GK104GL with NVIDIA GRID K2 technology

Mellanox InfiniBand network switch 56 Gbit/s

6 GPU cluster nodes

4 x AMD FirePro W9100 (16Gb GDDR5, 5.24 Tflops peak)

2 x Intel Xeon E5-2643 v2 (25M cache, 3.50 GHz, 12 threads)

Performance35,6/64,9 Tflops

38th position in

Russian top 50 supercomputers rating

RAM –

320 GbOverall disk space

volumes

86 Тb

HARDWARE & SOFTWARE ENVIRONMENT11

VDI

GPU

3D

1. GPU clusters ОC Gentoo & Open Grid Engine

2. VexCL (vector expression template library) OpenCL/CUDA

3. Simulation & analyze software systemС++, MPI, Java

4. VDI serverCitix XenDesktop & NVidia GRID

5. Data warehousePostgreSQL & NFS (RAID)

WORKFLOW12

Sensors data

Simulation data

Processing

3D full wave simulation

GPU cluster

Interpretation

SUCCESSFUL SOLUTIONS13

Cloud solutionAmazon

SOFTWARE – HPC ENVIRONMENTOpenCL, CUDA, OpenMP, MPI

HARDWARE – GPU CLUSTER

Local solutionT-platforms, Meijin, ARBYTE SC, STSS

PROBLEMS14

DISTRIBUTED RENDERING

REMOTE & SHARED ACCESS

BIG DATA MANAGMENT

INTEGRATION

HPC 2D&3D

RD Store

INTEGRATED FRAMEWORKS 15

SCHLUMBERGER OCEAN

HUE SPACE

NVIDIA INDEX

COMMERCIAL AND

PROPRIETARY

NVIDIA TECHNOLOGY

C++, .NET, Java

SPECIALIZED FRAMEWORKS 16

Open Inventor, Nvidia Engines (SceniX, OptiX…), Equalizer, Chromium ...

HPC

Hadoop, HPCC, Array-based DBMS (Rasdaman, SciDB) ...

NVidia CUDA Toolkit, AMD OpenCL SDK ...

Big Data

Визуализация

CONCLUSION17

HPC HARWARE & SOFTWARE EXISTS

INDUSTRIAL SPECIALIZATION

COMPLETE DATA & PROCESSES INTEGRATION METHODOLOGY

COMMON HPC SOFTWARE ECOSYSTEM

SOFTWARE FRAMEWORKS HARD CHOICE

18

THANK YOU FOR YOUR ATTENTION!