Progress Report of JMA Chiashi Muroi, JMA 18-22 Oct. 2010, Tokyo WGNE-26.

Progress Report of JMA

Chiashi Muroi, JMA18-22 Oct. 2010, Tokyo

WGNE-26

Contents

• Outline of our NWP system• Recent changes and developments• Install plan of next supercomputer and model

upgrade plan (since Mar. 2012)• Very short range Forecasting of Precipitation

( from Dr. Kato)

2

OUTLINE OF NWP SYSTEM

3

Current NWP models of NPD/JMA Global Spectral

Model(GSM)

MesoScale Model(MSM)

Local Forecast Model (LFM)

One-weekEnsemble(WEPS)

TyphoonEnsemble

(TEPS)

ObjectivesShort- and Medium-

rangeforecast

Disaster reduction,Short-range

forecast

Disaster preventingAviation forecast

One-week forecast Typhoon forecast

Forecast domain Global

Japan and its surroundings

(3600km x 2880km)

Japan center regions

(1600km x 1100km)

Global

Horizontal resolution TL959(0.1875 deg) 5km 2km TL319(0.5625 deg)

Verticallevels / Top

600.1 hPa

5021800 m

6060

0.1 hPa

ForecastHours

(Initial time)

84 hours(00, 06, 18 UTC)

216 hours(12 UTC)

15 hours(00, 06, 12, 18

UTC)33 hours

(03, 09, 15, 21 UTC)

9 hours216 hours(12 UTC)

51 members

132 hours(00, 06, 12, 18

UTC)11 members

Initial ConditionGlobal Analysis

(4D-Var)Mesoscale Analysis

(4D-Var)Local Analysis

(3D-Var)

Global Analysiswith ensemble perturbations

Perturbations are produced by SV-method

New!

Data assimilation systems of NPD/JMA Global Analysis

(GA)

Mesoscale Analysis

(MA)

Local Analysis

(LA)

Analysis scheme 4DVAR 3DVAR

Analysis time 00, 06, 12, 18 UTC00, 03, 06, 09, 12, 15,

18, 21 UTC00, 03, 06, 09, 12, 15,

18, 21 UTC

Data cut-off time

2 hours 20 minutes [Early Analysis]

11 hours 35 minutes (00, 12 UTC)

5 hours 35 minutes (06, 18 UTC)[Cycle Analysis]

50 minutes 30 minutes

Horizontal resolution

(inner-model resolution)

TL959 / 0.1875 deg

(T159 / 0.75 deg)

5 km

(15 km)5km

Vertical levels 60 levels up to 0.1 hPa50 levels up to

21800m50 levels

Assimilation window-3 hours to +3 hours of analysis

time-3 hours to analysis

time-

New!

Centre region1600x1100km2

Objectives : Disaster prevention / Aviation forecast

Resolution : 2kmForecast Domain : 1600kmx1100kmInitial time : 3 hourly ( 8/day )Forecast Period : 9 hour

3-ice microphysics scheme(Cumulus parameterization is not used)

Local Forecast Model (LFM)- Trial Run will be started from Nov. 2010 -

• Resolution 5km, 50 vertical levels• Domain ： 2000kmx 1500km （ Grid size: 400 x 300 ）• 3 hourly• Cut off time: 30 minutes• 3DVAR, Rapid Update Cycle• Observation

Surface, AMeDASAircraftWind ProfilerDoppler Velocity Ground based GPS TPW

7

18 21 22 23 00

5km MSM

LA

MSM

LA LA LA

2km LFM

5km MSM

MSM MSM

03 06 09 ＵＴＣ

Local Analysis

Lateral Boundary

Lateral Boundary

Rapid Update Cycle

Quantitative precipitation forecast, Typhoon MalouLFM(2km): 107mm/3hr OBS(2km): 240mm/3hr

MSM(5km): 69mm/3hr GSM(20km): 35mm/3hr

9

Rainfall Verification of LFM• Winter (DJF)

Against G

rid M

eanA

gainst Grid

Max

False Alarm RatioETSBias Score

RECENT CHANGES AND DEVELOPMENT

10

Development– physics and dynamics-

• Under development– Cumulus parameterization– Coupling of an ocean mixed-layer model – Increasing the number of vertical levels– Conservative semi-Lagrangian scheme– Non-orographic gravity wave drag– Yin-Yang Grid nonhydrostatic model

Blue: GSM L100v2

Red: LBL

Lack of cooling?

Yin-Yang Grid- Williamson et al. (1992) case 5 -

LAT-LON grid Spectral Conformal Cubic grid

Development – assimilation, data-• Recent changes

– Nov. 2009: Assimilation of Metop/GRAS and GRACE has been started

– Nov. 2009: Assimilation of AIREP temperature data has been started

– Apr. 2010: Revise of Typhoon bugussing– Mar. 2010 Assimilation of Brazil RARS has been started– Oct.? 2010: Assimilation of COSMIC will be started

• Under development– Update of Global Analysis inner model of 4DVAR– LETKF– AIRS and IASI– Reevaluate thinning parameters and observation errors– Rapid scan winds of MTSAT

Development – EPS -

• Recent changes– Nov.? 2010: Stochastic Physics

• Under development– Dec.? 2010: Revise Initial perturbations– Reforecast– LETKF

Initial Perturbation

• Too large spread in NH, in Winter

• Revised perturbation amplitude

• Add perturbation in SH

数値予報課コロキウム 18

Revised

Operational

Revised

Development – regional -

• Recent changes– Sep. 2010: VarQC, Revise of Typhoon Bogussing– Nov. 2010: New 2-km NWP system : Local Forecast Model– Nov. 2010: Revise of the cumulus convection

• Under development– Assimilation of radar reflectivity data– Development of a new dynamical core for the non-

hydrostatic model– Development of a new dynamical core on CUDA/GPU– Development of a mesoscale ensemble system

19

Mixing ratio in Kain- Fritsch scheme

2010/04/08

メソ数値予報モデルにおける Kain-Fritsch 対流ス

キームの混合率の変更 － 数値予報課コロキウム

20

:

:

:

:

:

0

0

R

p

M

MR

pMM

u

e

ue

Mixing ratio

Upward mass flux at LCL

Constant

Interval of vertical levels

Radius of convection at LCL

Constant– 0.2 (starting plume)– 0.3 (McCarthy, 1974;

Simpson 1983)– 0.75 (Malkus, 1960)– 1.0 (Barnes et al., 1996)

In finner mesh simulation as 5km-mesh,the constant is not adequate for small scale convection

メソ数値予報モデルにおける Kain-Fritsch 対流スキームの混合率の変更 － 数値予報

課コロキウム212010/04/08

28 June 2009 15UTC33hour forecast by 5km-MSM

Observed Rainfall [mm/3h]

Operational

Revised

Development of New Dynamical Core on GPGPU

• Joint research between Tokyo-tech and JMA

• Development of “ASUCA” on TSUBAME– Original : Fortran90– Rewrite to C/C++, CUDA– All dynamics and physics

are calculated on GPUs.

• “Best Student Award” in SC10

22

ASUCAGoverning equations Flux form

Fully compressible equations

Prognostic variables ρu, ρv, ρw, ρθ, ρ

Spatial discretization Finite volume method

Time integration Runge-Kutta 3rd

Treatment of sound Split explicit

Advection Flux limiter function by Koren

15 TFLOPS on 520 GPUs

INSTALL PLAN OF NEXT SUPERCOMPUTER

23

System Procurement in 2010

• Next System Procurement ~Jun 2010– Demand 8.2x faster sustained computational speed– Become operational in Mar 2012– Benchmark Tests

• Global: GSM TL959, EPS TL479 & TL319, 4DVAR • Meso: JMA-NHM 5 km & 2 km, 4DVAR, 3DVAR, ASUCA

– HITACHI won the procurement again!!• HITACHI supplies JMA HPC systems for 50 years.

Next JMA Supercomputer• HITACHI next SR series

– Peak Performance : 829.4TFLOPS• 2 Subsystems : 2x 414.7 TFLOPS

– Operation + Backup

– Memory : 108 TB– High-speed Storage : 348 TB– Data Storage : 3.7 PB + Tape Library– Benchmark result of TL959L100

• 9 days run with 40 nodes (1280 cores) = 35 minutes

?

Mar. 2006- Feb. 2012 Mar. 2012 –

HITACHI SR11000

Successor of HITACHI’s current super computer SR16000

Total Peak Performance 27.584TFlops 829.4TFlops

Total number of nodes 210 nodes （ 16CPU/1node ）

864 nodes （ 32CPU/1node ）

Memory 64GB/node 128GB/node

Memory Bandwidth 67.2-134.4GB/s/node 612GB/s/node

Network Bandwidth 8GB/s (one-way) 96GB/s (one-way)

System configuration 80nodes x 2 +50nodes x 1

432nodes x 2

JMA’s super computer system will be upgraded in 2012.

A tentative NWP model upgrade plan for the next computer system (2012-)

• Local Forecast Model (2km, hourly, 9 hour forecast, 3D-Var data assimilation) will be operational.

• Mesoscale Ensemble Prediction System (10km, 5 members: pre-operational test) (TBD)

• Extension of forecast period of One-week ensemble forecast to 18-days

• Enhancement of vertical resolution (GSM, MSM and One-week and Typhoon ensemble)

• Enhancement of horizontal resolution (WEPS and TEPS)

• Increase of ensemble members (TEPS)

New!

New!

New!

Current and Planned NWP models 1/2Model Current Planned

GSMGlobalSpectralModel

Resolution TL959L60 TL959L100

Initial time 00,06,12,18UTC ->

Forecast period 216hour for 12utc84hour for else

->

Assimilation 4DVAR 4DVAR

WEPSOne-WeekEnsemblePredictionSystem

Resolution TL319L60 TL479L100

Initial time 12UTC 00,12UTC

Member 51 27 x 2

Forecast period 216hour 432hour(13member)216hour(14member)

TEPSTyphoonEnsemblePredictionSystem

Resolution TL319L60 TL479L100

Initial time 00,06,12,18UTC ->

Member 11 25

Forecast period 132hour ->

28

Current and Planned NWP models 2/2Model Current Planned

MSMMesoScaleModel

Resolution 5km(721x577x50) 5km(817x661x75)

Initial time 00,03,06,09,12,15,18,21UTC

->

Forecast period 15hour for 00,06,12,18UTC33hour for 03,09,15,21UTC

36hour

Analysis 4DVAR 4DVAR

LFMLocalForecastModel

Resolution 2km(800x550x60) 2km(1581x1301x60)

Initial time 00,03,06,09,12,15,18,21UTC

Hourly

Forecast period 9hour ->

Assimilation 3DVAR 3DVAR

MEPSMesoEnsemblePredictionSystem

Resolution - 10km(409x331x60) (TBD)

Initial time - 00,06,12,18UTC

Member - 5 (TBD)

Forecast period - 39hour29

Future Plan ofMedium range and one month ensemble prediction system

W1 W2 W3 W4

1-month EPS 　 (TL319L100)

1w EPS 　 (TL479L100)

2012-

1w EPS （ TL479L100 ） 2w EPS

2014?-

1w EPS （ TL479L100 ） Ext-1w EPS

1-month EPS 　 (TL319L100)

2018?- 1-month EPS

Separeted system

Unified system

Ext-1w EPS

2w EPS 　 (TL319L100)

HindcastHindcast

HindcastHindcast

ReforecastReforecast

Reforecast/HindcastReforecast/Hindcast

•Meso Scale Model (MSM)•MSM has been operating since March 2001.•The objectives are disaster prevention and short range forecast.• We plan to expand the forecast domain to reduce the influence of lateral boundary condition on the next super computer system.

•Local Forecast Model (LFM)•The trial run will start in December 2010•The objectives are disaster prevention and aviation forecast.•We plan to expand the forecast domain to cover whole Japan.

Current

Plan

Forecast Domain and Topography of MSM Forecast Domain and Topography of LFM

Current

Plan