自動チューニングとビックデータ：機械学習の適用の可能性

自動チューニングとビックデータ

：機械学習の適用の可能性

片桐孝洋東京大学情報基盤センター

1

Sapporo Summer HPC Seminar2015年7月23日（木）17:00～17:40北海道大学情報基盤センター北館 4階会議室

話の流れATとビックデータと機械学習

ATの機械学習適用に関する国内外の研究

機械学習を適用したい事例

◦①疎行列反復解法の前処理の（実行時）自動選択

◦②疎行列-ベクトル積(SpMV)の実装の自動選択

◦③自動チューニング言語2







自動チューニング（AT）機構とは

4

•計算機アーキテクチャ•計算機システム

•プログラム•アルゴリズム

性能調整つまみ（性能パラメタ）

自動チューニング機構

調整機構•最適化•パラメタ探索•学習／自己適応性能モニタ

機構つまみ自動生成機構

•プログラム•アルゴリズム

AT性能データベース

ビックデータとAT対象社会データ◦ ビジネスデータ金融市場、企業取引

◦ 公共システム◦ 交通

データサイエンス◦ ゲノム、物理実験施設（加速器）、衛星データ

データ同化◦ 気象データ（天気予測）◦ 地象データ（地震、津波、災害予測）

5

計算機ログシステムログ（ネットワークログ）機械学習による異常検知

性能プロファイルデータ

性能プロファイルデータはビックデータか？ →ＹＥＳ（数値計算分野においても！）密行列系（行列分解処理）◦ 前提：

事前（インストール時）に行うため、可能な実行条件のデータを静的に取る行列サイズ：～１００００（ノード内）キャッシュブロッキング幅：～１０２４アンローリング段数：～１２８スレッド数：～３００カーネル数：～１０以上のデータ量：~３０TB／１ライブラリ機能

◦ ユーザ全体で１０種機能 →３００ＴＢ級（＋通信最適化のログは別）

疎行列系（陽解法、陰解法）◦ 前提：疎行列形状を得るため実行時（ライブラリコール時）にデータを取る

演算カーネル種類：～１００ループ変換種類：～１００通信（ＭＰＩ）種類：～１０疎行列ソルバの前処理：～１０数値計算上のパラメタ：～１００以上のデータ量：～１ＧＢ／シミュレーション起動あたりＡＴ部分起動回数：１００回／月 →１００ＧＢ／ユーザ／月

◦ センタ全体ユーザ数：100人→１０ＴＢ級／月以上は、前提知識が無い場合。実際はアドホック手法＆性能モデル化で組合わせ数は減る 6







ATシステムと機械学習（数値計算） AutoPilot [2001, D. Reed, U. Illinois]◦ Fuzzy Rule

Sans and Salsa [2002, J. Dongarra, U. Tennessee]◦ Self-Adapting Numerical Software (SANS)◦ Self-Adapting Large-scale Solver Architecture

(SALSA) ◦ Alternating Decision Trees (AdaBoost)

ATMathCoreLib [2011, R. Suda, U. Tokyo]◦ Bayesian Inference

Compiler Optimization [2012, J. Cavazos, U. Delaware]◦ Artificial Neural Network◦ Neuro-Evolution for Augmenting Topologies

Nitro [2015, M. Hall, U. Utah]◦ Support Vector Machine,◦ Radial-Basics Function 8

国内の研究動向根拠に基づく性能チューニングの支援◦ 理研AICS 橋本政朋研究員

◦ 戦略アプリのチューニングログを利用

◦ 機械学習で、チューニングパターンを予測「このカーネルコード」の「ここをループ分割」すると、「京コンピュータ」で速くなる確率は60％

◦ 以下の課題があるが注目される研究の１つ：1. チューニングデータベースの充実通常、失敗したチューニングのコード履歴は残さない

網羅的なチューニングパターンがいる（自動化が必要）

2. 計算機環境データの充実 CPU種別、コア数、問題サイズなど、多種の実行条件の

データが必要

9

AT言語で複数のAT候補の実行時間を自動測定可能。自動でデータベース化し、機械学習への展開に期待

数値計算分野で求められる機械学習（疎行列ソルバーの場合）

10

(Matrices Source: https://www.cise.ufl.edu/research/sparse/matrices/)

数値解法 GMRESGMRESBiCGStabIDR GCR BiCGStab

実装／データ構造 DIAG CRS ELL Hyb. CRS CRSCOO

前処理 None Jacobi SSOR ILU(0) ILUT ILUT

疎行列形状

学習セット

疎行列形状

前処理

数値解法

実装／データ構造

予測最適候補

未知の行列

疎行列ソルバ選択の機械学習は何が困難か？適切な学習データがそろえられるか

行列特性は、分野限定しないと多種多様高速となる組合わせだけでなく、収束しない組合わせがある

パラメタが多い◦ 前処理の手法には、フィルイン・カットオフ値など、「実数」のパラメタがある離散データ化しにくい→離散データ化すると最適化空間爆発

分野限定すると意味が無い◦ 利用分野を限定（例えば、流体の～問題）に限定すると、数値解法が限定し、学習しやすくなる疎行列の構造、数値特性、が固定となるため

◦ その反面、ベストな解法の組合わせが自明となる対称行列、AMG前処理で限定、など

◦ それでも、実装選択には「機械学習」が有効かも問題サイズの影響、CPU実行かGPU実行かその両方か、など

11

汎用ライブラリに対する機械学習の適用が問題







実例③：

自動チューニング言語

75

平成２３年度～平成２７年度：科学技術振興機構(JST)、CREST, 研究領域「ポストペタスケール高性能計算に資するシステムソフトウェア技術の創出」、研究課題：「自動チューニング機構を有するアプリケーション開発・実行環境」、代表者：中島研吾（研究協力者（東大中島グループ））

Directive‐based Auto‐tuning for the Finite Difference Method

on the Xeon PhiTakahiro Katagiri, Satoshi Ohshima,

Masaharu Matsumoto(Information Technology Center, The University of Tokyo)

76

iWAPT2015, Hyderabad International Convention Centre, Hyderabad, INDIASession 3, 13:30 ‐14:00May 29th, 2015

ppOpen‐AT SystemppOpen‐APPL /*

ppOpen‐ATDirectives

User KnowledgeLibrary

Developer

① Before Release‐time

Candidate1

Candidate2

Candidate3

Candidaten

ppOpen‐ATAuto‐Tuner

ppOpen‐APPL / *

AutomaticCodeGeneration②

:Target Computers

Execution Time④

Library User

③

Library Call

Selection

⑤

⑥

Auto‐tunedKernelExecution

Run‐timeThis user

benefited from AT.

Tuning Profile Log

Scenario of AT for ppOpen‐APPL/FDM

84Execution with optimized

kernels without AT process.

Library User

Set AT parameter, and execute the library

(OAT_AT_EXEC=1)

■Execute auto-tuner: With fixed loop lengths (by specifying problem size and number of MPI processes and OpenMP threads)Measurement of timings for target kernels.Store the best candidate information.

Set AT parameters, and execute the library(OAT_AT_EXEC=0)

Store the fastest kernelinformation

Using the fastest kernel without AT (except for varying problem size and number of MPI processes and OpenMP threads.)

Specify problem size, number of MPI processes

and OpenMP threads.

Target Application• Seism_3D: Simulation for seismic wave analysis.

• Developed by Professor Furumura at the University of Tokyo.– The code is re‐constructed as ppOpen‐APPL/FDM.

• Finite Differential Method (FDM) • 3D simulation

–3D arrays are allocated.• Data type: Single Precision (real*4) 89

Flow Diagram of ppOpen‐APPL/FDM

90

),,,(

}],,,)21({},,)

21({[1

),,(

2/

1

zyxqp

zyxmxzyxmxcx

zyxdxd

pqpq

M

mm

pq

Space difference by FDM.

),,(,121

21

zyxptfzyx

uu np

nzp

nyp

nxpn

p

n

p

Explicit time expansion by central difference.

Initialization

Velocity Derivative (def_vel)

Velocity Update (update_vel)

Stress Derivative (def_stress)

Stress Update (update_stress)

Stop Iteration?NO

YES

End

Velocity PML condition (update_vel_sponge)Velocity Passing (MPI) (passing_vel)

Stress PML condition (update_stress_sponge)Stress Passing (MPI) (passing_stress)

Target Loop Characteristics• Triple‐nested loops

!$omp parallel do do k = NZ00, NZ01do j = NY00, NY01do i = NX00, NX01<Codes from FDM>

end doend do

end do!$omp end parallel do

OpenMP directive to the outer loop (Z‐axis)

Loop lengths are varied according to problem size, the number of MPI processes and OpenMP threads.

The codes can be separable by loop split.

What is separable codes?Variable definitions and references are separated.There is a flow‐dependency, but no data dependency between each other.

d1 = …d2 = ……dk = ……

… = … d1 …… = … d2 ……… = … dk …

Variable definitions

Variable references

d1 = …d2 = …… = … d1 …… = … d2 …

……dk = ……… = … dk …

Split toTwo Parts

ppOpen‐AT Directives: Loop Split & Fusion with data‐flow dependence

93

!oat$ install LoopFusionSplit region start!$omp parallel do private(k,j,i,STMP1,STMP2,STMP3,STMP4,RL,RM,RM2,RMAXY,RMAXZ,RMAYZ,RLTHETA,QG)

DO K = 1, NZDO J = 1, NYDO I = 1, NX

RL = LAM (I,J,K); RM = RIG (I,J,K); RM2 = RM + RMRLTHETA = (DXVX(I,J,K)+DYVY(I,J,K)+DZVZ(I,J,K))*RL

!oat$ SplitPointCopyDef region start QG = ABSX(I)*ABSY(J)*ABSZ(K)*Q(I,J,K)

!oat$ SplitPointCopyDef region endSXX (I,J,K) = ( SXX (I,J,K) + (RLTHETA + RM2*DXVX(I,J,K))*DT )*QGSYY (I,J,K) = ( SYY (I,J,K) + (RLTHETA + RM2*DYVY(I,J,K))*DT )*QGSZZ (I,J,K) = ( SZZ (I,J,K) + (RLTHETA + RM2*DZVZ(I,J,K))*DT )*QG

!oat$ SplitPoint (K, J, I)STMP1 = 1.0/RIG(I,J,K); STMP2 = 1.0/RIG(I+1,J,K); STMP4 = 1.0/RIG(I,J,K+1)STMP3 = STMP1 + STMP2RMAXY = 4.0/(STMP3 + 1.0/RIG(I,J+1,K) + 1.0/RIG(I+1,J+1,K))RMAXZ = 4.0/(STMP3 + STMP4 + 1.0/RIG(I+1,J,K+1))RMAYZ = 4.0/(STMP3 + STMP4 + 1.0/RIG(I,J+1,K+1))

!oat$ SplitPointCopyInsertSXY (I,J,K) = ( SXY (I,J,K) + (RMAXY*(DXVY(I,J,K)+DYVX(I,J,K)))*DT )*QGSXZ (I,J,K) = ( SXZ (I,J,K) + (RMAXZ*(DXVZ(I,J,K)+DZVX(I,J,K)))*DT )*QGSYZ (I,J,K) = ( SYZ (I,J,K) + (RMAYZ*(DYVZ(I,J,K)+DZVY(I,J,K)))*DT )*QG

END DO; END DO; END DO!$omp end parallel do!oat$ install LoopFusionSplit region end

Re‐calculation is defined.

Using the re‐calculation is defined.

Loop Split Point

Specify Loop Split and Loop Fusion

Re‐ordering of Statements: increase a chance to optimize register allocation by compiler

94

!OAT$ RotationOrder sub region startSentence iSentence ii

!OAT$ RotationOrder sub region end!OAT$ RotationOrder sub region start

Sentence 1Sentence 2

!OAT$ RotationOrder sub region end

Sentence iSentence 1Sentence iiSentence 2

Generated Code

The Kernel 1 (update_stress)• m_stress.f90（ppohFDM_update_stress）!OAT$ call OAT_BPset("NZ01")!OAT$ install LoopFusionSplit region start!OAT$ name ppohFDMupdate_stress!OAT$ debug (pp)!$omp parallel do private(k,j,i,RL1,RM1,RM2,RLRM2,DXVX1,DYVY1,DZVZ1,D3V3,DXVYDYVX1,

DXVZDZVX1,DYVZDZV1)do k = NZ00, NZ01do j = NY00, NY01do i = NX00, NX01RL1 = LAM (I,J,K)

!OAT$ SplitPointCopyDef sub region startRM1 = RIG (I,J,K)

!OAT$ SplitPointCopyDef sub region endRM2 = RM1 + RM1; RLRM2 = RL1+RM2; DXVX1 = DXVX(I,J,K); DYVY1 = DYVY(I,J,K); DZVZ1 = DZVZ(I,J,K)D3V3 = DXVX1 + DYVY1 + DZVZ1SXX (I,J,K) = SXX (I,J,K) + (RLRM2*(D3V3)‐RM2*(DZVZ1+DYVY1) ) * DTSYY (I,J,K) = SYY (I,J,K) + (RLRM2*(D3V3)‐RM2*(DXVX1+DZVZ1) ) * DTSZZ (I,J,K) = SZZ (I,J,K) + (RLRM2*(D3V3)‐RM2*(DXVX1+DYVY1) ) * DT

Measured B/F=3.2

The Kernel 1 (update_stress)• m_stress.f90（ppohFDM_update_stress）!OAT$ SplitPoint (K,J,I)!OAT$ SplitPointCopyInsert

DXVYDYVX1 = DXVY(I,J,K)+DYVX(I,J,K)DXVZDZVX1 = DXVZ(I,J,K)+DZVX(I,J,K)DYVZDZVY1 = DYVZ(I,J,K)+DZVY(I,J,K)SXY (I,J,K) = SXY (I,J,K) + RM1 * DXVYDYVX1 * DTSXZ (I,J,K) = SXZ (I,J,K) + RM1 * DXVZDZVX1 * DTSYZ (I,J,K) = SYZ (I,J,K) + RM1 * DYVZDZVY1 * DT

end doend do

end do!$omp end parallel do!OAT$ install LoopFusionSplit region end

Automatic Generated Codes for the kernel 1ppohFDM_update_stress #1 [Baseline]: Original 3-nested Loop #2 [Split]: Loop Splitting with K-loop

(Separated, two 3-nested loops) #3 [Split]: Loop Splitting with J-loop #4 [Split]: Loop Splitting with I-loop #5 [Split&Fusion]: Loop Fusion to #1 for K and J-loops

(2-nested loop) #6 [Split&Fusion]: Loop Fusion to #2 for K and J-Loops

(2-nested loop) #7 [Fusion]: Loop Fusion to #1

(loop collapse) #8 [Split&Fusion]: Loop Fusion to #2

(loop collapse, two one-nest loop)

The Kernel 2 (update_vel)• m_velocity.f90（ppohFDM_update_vel）!OAT$ install LoopFusion region start!OAT$ name ppohFDMupdate_vel!OAT$ debug (pp)!$omp parallel do private(k,j,i,ROX,ROY,ROZ)do k = NZ00, NZ01do j = NY00, NY01do i = NX00, NX01! Effective Density

!OAT$ RotationOrder sub region startROX = 2.0_PN/( DEN(I,J,K) + DEN(I+1,J,K) )ROY = 2.0_PN/( DEN(I,J,K) + DEN(I,J+1,K) )ROZ = 2.0_PN/( DEN(I,J,K) + DEN(I,J,K+1) )

!OAT$ RotationOrder sub region end!OAT$ RotationOrder sub region start

VX(I,J,K) = VX(I,J,K) + ( DXSXX(I,J,K)+DYSXY(I,J,K)+DZSXZ(I,J,K) )*ROX*DTVY(I,J,K) = VY(I,J,K) + ( DXSXY(I,J,K)+DYSYY(I,J,K)+DZSYZ(I,J,K) )*ROY*DTVZ(I,J,K) = VZ(I,J,K) + ( DXSXZ(I,J,K)+DYSYZ(I,J,K)+DZSZZ(I,J,K) )*ROZ*DT

!OAT$ RotationOrder sub region endend do; end do; end do

!$omp end parallel do!OAT$ install LoopFusion region end

Measured B/F=1.7

Automatic Generated Codes for the kernel 2ppohFDM_update_vel• #1 [Baseline]: Original 3‐nested Loop.• #2 [Fusion]: Loop Fusion for K and J‐Loops.

(2‐nested loop)• #3 [Fusion]: Loop Split for K, J, and I‐Loops.

(Loop Collapse)• #4 [Fusion&Re‐order]:

Re‐ordering of sentences to #1. • #5 [Fusion&Re‐order]:

Re‐ordering of sentences to #2. • #6 [Fusion&Re‐order]:

Re‐ordering of sentences to #3.

An Example of Seism_3D Simulation West part earthquake in Tottori prefecture in Japan

at year 2000. ([1], pp.14) The region of 820km x 410km x 128 km is discretized with 0.4km. NX x NY x NZ = 2050 x 1025 x 320 ≒ 6.4 : 3.2 : 1.

[1] T. Furumura, “Large‐scale Parallel FDM Simulation for Seismic Waves and Strong Shaking”, Supercomputing News, Information Technology Center, The University of Tokyo, Vol.11, Special Edition 1, 2009. In Japanese.

Figure : Seismic wave translations in west part earthquake in Tottori prefecture in Japan. (a) Measured waves; (b) Simulation results; (Reference : [1] in pp.13)

AT Candidates in This Experiment1. Kernel update_stress

– 8 Kinds of Candidates with Loop Collapse and Loop Split.2. Kernel update_vel

– 6 Kinds of Candidates with Loop Collapse and Re‐ordering of Statements. 3 Kinds of Candidates with Loop Collapse.3. Kernel update_stress_sponge4. Kernel update_vel_sponge5. Kernel ppohFDM_pdiffx3_p46. Kernel ppohFDM_pdiffx3_m47. Kernel ppohFDM_pdiffy3_p48. Kernel ppohFDM_pdiffy3_m49. Kernel ppohFDM_pdiffz3_p410. Kernel ppohFDM_pdiffz3_m4 Kinds of Candidates with Loop Collapse for Data Packing and Data

Unpacking.11. Kernel ppohFDM_ps_pack12. Kernel ppohFDM_ps_unpack13. Kernel ppohFDM_pv_pack14. Kernel ppohFDM_pv_unpack 104

Total Number of Kernel Candidates: 47

Machine Environment (8 nodes of the Xeon Phi) The Intel Xeon Phi Xeon Phi 5110P (1.053 GHz), 60 cores Memory Amount：8 GB (GDDR5) Theoretical Peak Performance：1.01 TFLOPS One board per node of the Xeon phi cluster InfiniBand FDR x 2 Ports

Mellanox Connect‐IB PCI‐E Gen3 x16 56Gbps x 2 Theoretical Peak bandwidth 13.6GB/s Full‐Bisection

Intel MPI Based on MPICH2, MVAPICH2 4.1 Update 3 (build 048)

Compiler：Intel Fortran version 14.0.0.080 Build 20130728 Compiler Options：

‐ipo20 ‐O3 ‐warn all ‐openmp ‐mcmodel=medium ‐shared‐intel –mmic‐align array64byte

KMP_AFFINITY=granularity=fine, balanced (Uniform Distribution of threads between sockets)

Execution Details• ppOpen‐APPL/FDM ver.0.2• ppOpen‐AT ver.0.2• The number of time step: 2000 steps• The number of nodes: 8 node• Native Mode Execution• Target Problem Size (Almost maximum size with 8 GB/node)– NX * NY * NZ = 1536 x 768 x 240 / 8 Node– NX * NY * NZ = 768 * 384 * 120 / node(!= per MPI Process)

• The number of iterations for kernels to do auto‐tuning: 100

0

200

400

600

800

1000

1200

P8T240 P16T120 P32T60 P64T30 P128T15 P240T8 P480T4

Others

IO

passing_stress

passing_velocity

update_vel_spongeupdate_vel

update_stress_spongeupdate_stress

def_stress

def_vel

Whole Time （without AT）[Seconds]

Comp.Time

Comm.Time

NX*NY*NZ = 1536x768x240/ 8 Node

Hybrid MPI/OpenMPPhi : 8 Nodes (1920 Threads)

0

200

400

600

800

1000

1200

P8T240 P16T120 P32T60 P64T30 P128T15 P240T8 P480T4

Others

IO

passing_stress

passing_velocity

update_vel_spongeupdate_vel

update_stress_spongeupdate_stress

def_stress

def_vel

51.5% Speedups

Whole Time（with AT）[Seconds]

Comp.Time

Comm.Time

12.1% Speedups 3.3% Speedups



NX*NY*NZ = 1536x768x240/ 8 Node

Hybrid MPI/OpenMPPhi : 8 Nodes (1920 Threads)

Maximum Speedups by AT(Xeon Phi, 8 Nodes)

558

200 17130 20 51

Speedup [%]

Kinds of Kernels

Speedup = max ( Execution time of original code / Execution time with AT )

for all combination of Hybrid MPI/OpenMP Executions (PXTY)

NX*NY*NZ = 1536x768x240/ 8 Node

The Fastest Code (update_stress) Xeon Phi (P240T8)#5 [Split&Fusion]: Loop Fusion to #1 for K and J‐loops (2‐nested loop)!$omp parallel do private (k,j,i,RL1,RM1,RM2,RLRM2,DXVX1,DYVY1,DZVZ1,D3V3,DXVYDYVX1,DXVZDZVX1,DYVZDZV1)DO k_j = 1 , (NZ01‐NZ00+1)*(NY01‐NY00+1)k = (k_j‐1)/(NY01‐NY00+1) + NZ00; j = mod((k_j‐1),(NY01‐NY00+1)) + NY00; DO i = NX00, NX01RL1 = LAM (I,J,K); RM1 = RIG (I,J,K); RM2 = RM1 + RM1; RLRM2 = RL1+RM2;DXVX1 = DXVX(I,J,K); DYVY1 = DYVY(I,J,K); DZVZ1 = DZVZ(I,J,K); D3V3 = DXVX1 + DYVY1 + DZVZ1;SXX (I,J,K) = SXX (I,J,K) + (RLRM2*(D3V3)‐RM2*(DZVZ1+DYVY1) ) * DTSYY (I,J,K) = SYY (I,J,K) + (RLRM2*(D3V3)‐RM2*(DXVX1+DZVZ1) ) * DTSZZ (I,J,K) = SZZ (I,J,K) + (RLRM2*(D3V3)‐RM2*(DXVX1+DYVY1) ) * DTDXVYDYVX1 = DXVY(I,J,K)+DYVX(I,J,K); DXVZDZVX1 = DXVZ(I,J,K)+DZVX(I,J,K); DYVZDZVY1 = DYVZ(I,J,K)+DZVY(I,J,K)SXY (I,J,K) = SXY (I,J,K) + RM1 * DXVYDYVX1 * DTSXZ (I,J,K) = SXZ (I,J,K) + RM1 * DXVZDZVX1 * DTSYZ (I,J,K) = SYZ (I,J,K) + RM1 * DYVZDZVY1 * DT

END DOEND DO

!$omp end parallel do

Ivy Bridge(P80T2) FX10 (P128T1)#1 [Baseline]: Original Loop#4 [Split]: Loop Splitting

with I‐loop

!$omp parallel do private (k,j,i,RL1,RM1,RM2,RLRM2,DXVX1,DYVY1,DZVZ1,D3V3,DXVYDYVX1,DXVZDZVX1,DYVZDZV1)

do k = NZ00, NZ01do j = NY00, NY01do i = NX00, NX01RL1 = LAM (I,J,K); RM1 = RIG (I,J,K);RM2 = RM1 + RM1; RLRM2 = RL1+RM2DXVX1 = DXVX(I,J,K); DYVY1 = DYVY(I,J,K)DZVZ1 = DZVZ(I,J,K)D3V3 = DXVX1 + DYVY1 + DZVZ1SXX (I,J,K) = SXX (I,J,K)

+ (RLRM2*(D3V3)‐RM2*(DZVZ1+DYVY1) ) * DTSYY (I,J,K) = SYY (I,J,K)

+ (RLRM2*(D3V3)‐RM2*(DXVX1+DZVZ1) ) * DTSZZ (I,J,K) = SZZ (I,J,K)

+ (RLRM2*(D3V3)‐RM2*(DXVX1+DYVY1) ) * DTend dodo i = NX00, NX01

RM1 = RIG (I,J,K)DXVYDYVX1 = DXVY(I,J,K)+DYVX(I,J,K)DXVZDZVX1 = DXVZ(I,J,K)+DZVX(I,J,K)DYVZDZVY1 = DYVZ(I,J,K)+DZVY(I,J,K)

SXY (I,J,K) = SXY (I,J,K) + RM1 * DXVYDYVX1 * DTSXZ (I,J,K) = SXZ (I,J,K) + RM1 * DXVZDZVX1 * DTSYZ (I,J,K) = SYZ (I,J,K) + RM1 * DYVZDZVY1 * DTend do

end doend do


!$omp parallel do private (k,j,i,RL1,RM1,RM2,RLRM2,DXVX1,DYVY1,DZVZ1,D3V3,DXVYDYVX1,DXVZDZVX1,DYVZDZV1)do k = NZ00, NZ01do j = NY00, NY01do i = NX00, NX01RL1 = LAM (I,J,K); RL1 = LAM (I,J,K)RM1 = RIG (I,J,K); RM2 = RM1 + RM1RLRM2 = RL1+RM2; DXVX1 = DXVX(I,J,K); DYVY1 = DYVY(I,J,K)DZVZ1 = DZVZ(I,J,K)D3V3 = DXVX1 + DYVY1 + DZVZ1SXX (I,J,K) = SXX (I,J,K)

+ (RLRM2*(D3V3)‐RM2*(DZVZ1+DYVY1) ) * DTSYY (I,J,K) = SYY (I,J,K)

+ (RLRM2*(D3V3)‐RM2*(DXVX1+DZVZ1) ) * DTSZZ (I,J,K) = SZZ (I,J,K)

+ (RLRM2*(D3V3)‐RM2*(DXVX1+DYVY1) ) * DTDXVYDYVX1 = DXVY(I,J,K)+DYVX(I,J,K)DXVZDZVX1 = DXVZ(I,J,K)+DZVX(I,J,K)DYVZDZVY1 = DYVZ(I,J,K)+DZVY(I,J,K)

SXY (I,J,K) = SXY (I,J,K) + RM1 * DXVYDYVX1 * DTSXZ (I,J,K) = SXZ (I,J,K) + RM1 * DXVZDZVX1 * DTSYZ (I,J,K) = SYZ (I,J,K) + RM1 * DYVZDZVY1 * DTend do

end doend do


The Fastest Code (update_vel)

Xeon Phi (P240T8)#5 [Fusion&Re‐order]: Re‐ordering of sentences to #2.

!$omp parallel do private (i,j,k,ROX,ROY,ROZ)DO k_j = 1 , (NZ01‐NZ00+1)*(NY01‐NY00+1)

k = (k_j‐1)/(NY01‐NY00+1) + NZ00j = mod((k_j‐1),(NY01‐NY00+1)) + NY00do i = NX00, NX01

ROX = 2.0_PN/( DEN(I,J,K) + DEN(I+1,J,K) )VX(I,J,K) = VX(I,J,K) &+ ( DXSXX(I,J,K)+DYSXY(I,J,K)+DZSXZ(I,J,K) )*ROX*DT

ROY = 2.0_PN/( DEN(I,J,K) + DEN(I,J+1,K) )VY(I,J,K) = VY(I,J,K) &+ ( DXSXY(I,J,K)+DYSYY(I,J,K)+DZSYZ(I,J,K) )*ROY*DT

ROZ = 2.0_PN/( DEN(I,J,K) + DEN(I,J,K+1) )VZ(I,J,K) = VZ(I,J,K) &+ ( DXSXZ(I,J,K)+DYSYZ(I,J,K)+DZSZZ(I,J,K) )*ROZ*DT

end doend do!$omp end parallel do

Ivy Bridge(P80T2) FX10 (P128T1)#1 [Baseline]: Original Loop#5 [Fusion&Re‐order]:

Re‐ordering of sentences to #2.

!$omp parallel do private (i,j,k,ROX,ROY,ROZ)do k = NZ00, NZ01do j = NY00, NY01do i = NX00, NX01ROX = 2.0_PN/( DEN(I,J,K) + DEN(I+1,J,K) )ROY = 2.0_PN/( DEN(I,J,K) + DEN(I,J+1,K) )ROZ = 2.0_PN/( DEN(I,J,K) + DEN(I,J,K+1) )VX(I,J,K) = VX(I,J,K) +

( DXSXX(I,J,K)+DYSXY(I,J,K)+DZSXZ(I,J,K) )*ROX*DTVY(I,J,K) = VY(I,J,K) +

( DXSXY(I,J,K)+DYSYY(I,J,K)+DZSYZ(I,J,K) )*ROY*DTVZ(I,J,K) = VZ(I,J,K) +

( DXSXZ(I,J,K)+DYSYZ(I,J,K)+DZSZZ(I,J,K) )*ROZ*DTend doend do

end do!$omp end parallel do

Select節によるコード選択と階層的なAT指定

Program main….!OAT$ install select region start!OAT$ name ppohFDMupdate_vel_select!OAT$ select sub region startcall ppohFDM_pdiffx3_p4( SXX,DXSXX,NXP,NYP,NZP,….)call ppohFDM_pdiffy3_p4( SYY,DYSYY, NXP,NYP,NZP,…..)…if( is_fs .or. is_nearfs ) thencall ppohFDM_bc_stress_deriv( KFSZ,NIFS,NJFS,IFSX,….)end ifcall ppohFDM_update_vel ( 1, NXP, 1, NYP, 1, NZP )!OAT$ select sub region start

!OAT$ select sub region startcall ppohFDM_update_vel_Intel ( 1, NXP, 1, NYP, 1, NZP )!OAT$ select sub region start

!OAT$ install select region end

上位のAT指定

subroutine ppohFDM_update_vel(….)….!OAT$ install LoopFusion region start!OAT$ name ppohFDMupdate_vel!OAT$ debug (pp)!$omp parallel do private(i,j,k,ROX,ROY,ROZ)do k = NZ00, NZ01do j = NY00, NY01do i = NX00, NX01

…..….

下位のAT指定

subroutine ppohFDM_pdiffx3_p4(….)….!OAT$ install LoopFusion region start….

SWoPP2015@別府で発表(2015/8/6)

おわりに「性能チューニングログ」はビックデータになりうる

AT分野で機械学習は適用されているが、まだ決定的な方式が無い

汎用ソルバのアルゴリズム選択へ機械学習の適用が期待される。課題は：1. パラメタ空間が膨大（実数パラメタの考慮）2. チューニング履歴など学習データ収集が困難3. 多様な実行環境データをどう集めるか

AT言語を利用し、以下を推進：1. ＡＴで自動生成されるチューニング候補の

「性能プロファイルログ」の自動取得2. 学習データとして、機械学習へ連結3. 学習結果をＡＴライブラリへ自動適用

143

参考文献（著者に関連するもの）

1. H. Kuroda, T. Katagiri, M. Kudoh, Y. Kanada, “ILIB_GMRES: An auto‐tuning parallel iterative solver for linear equations,” SC2001, 2001. (Poster.)

2. T. Katagiri, K. Kise, H. Honda, T. Yuba, “ABCLib_DRSSED: A parallel eigensolver with an auto‐tuning facility,” Parallel Computing, Vol. 32, Issue 3, pp. 231–250, 2006.

3. T. Sakurai, T. Katagiri, K. Naono, H. Kuroda, K. Nakajima, M. Igai, S. Ohshima, S. Itoh, “Evaluation of auto‐tuning function on OpenATLib,” IPSJ SIG Technical Reports, Vol. 2011‐HPC‐130, No. 43, pp. 1–6, 2011. (in Japanese)

4. T. Katagiri, K. Kise, H. Honda, T. Yuba, “FIBER: A general framework for auto‐tuning software,” The Fifth International Symposium on High Performance Computing (ISHPC‐V), Springer LNCS 2858, pp. 146–159, 2003.

5. T. Katagiri, S. Ito, S. Ohshima, “Early experiences for adaptation of auto‐tuning by ppOpen‐AT to an explicit method,” Special Session: Auto‐Tuning for Multicore and GPU (ATMG) (In Conjunction with the IEEE MCSoC‐13), Proceedings of MCSoC‐13, 2013.

6. T. Katagiri, S. Ohshima, M. Matsumoto, “Auto‐tuning of computation kernels from an FDM Code with ppOpen‐AT,” Special Session: Auto‐Tuning for Multicore and GPU (ATMG) (In Conjunction with the IEEE MCSoC‐14), Proceedings of MCSoC‐14, 2014.

7. T.Katagiri, S.Ohshima, M. Matsumoto, "Directive‐based Auto‐tuning for the Finite Difference Method on the Xeon Phi," The Tenth International Workshop on Automatic Performance Tuning (iWAPT2015) (In Conjunction with the IEEE IPDPS2015 ), Proceedings of IPDPSW2015, pp.1221‐1230,2015.