Slide 1 ET-CTS - GISC Network Bandwidth RequirementsSlide 1 GISC network bandwidth requirements A proposed method for an educated guess… Hiroyuki Ichijo.

ET-CTS - GISC Network Bandwidth Requirements

Slide 1

Slide 1

GISC network bandwidth requirements

A proposed method for an educated guess…

Hiroyuki Ichijo – JMA

Rémy Giraud - ECMWF

ET-CTS Co-Chairs


Slide 2

Slide 2

Introduction As part of the requirements of the WIS, each GISC must:

host a Discovery, Access and Retrieval (DAR) catalogue hold and distribute at least 24 hours of current WMO data and

products intended for global distribution.

Each GISC is responsible for coordinating with the Centres within its area of responsibility a WIS telecommunications infrastructure that can meet the WIS requirements for information exchange within the area and that can exchange agreed WIS time critical and operational critical information.

The WIS real-time network would be composed of a small number of Area Meteorological Data Communication Networks (AMDCNs) and a WIS core network interconnecting the GISCs and AMDCNs together. GISCs would be responsible for ensuring that their AMDCNs are developed and maintained and, where the appropriate agreements are in place, the traffic between WIS centres in different AMDCNs is managed. [extract from CBS-XIV Abridged final report ]


Slide 3

Slide 3

Main taskRequired GISC bandwidth on the core network should be considered carefully considering not only bulk but also peak traffic. In addition, it is important to study smooth evolution process in gradual participation of operational GISCs.

Background for consideration (1) The Improved MTN (IMTN) project started at ET-IMTN meeting in

Sep/Oct 1999. Its implementation has progressed in technical evolution process as the configuration of two clouds (i.e. Network I & Network II) until Sep 2009. By the end of 2009, all links via Network I have migrated to via Network II. Currently the IMTN is operating on a single coordinated MPLS cloud.

Since a WIS core network will be established on the IMTN, it is easy to realize the full-mesh GISC topology. However it is expected that traffic on the WIS core network handled by GISCs will considerably increase. To support future-oriented WIS operation, network capacity of each GISC will have to be expanded in collaboration with others.


Slide 4

Slide 4

Background for consideration (2) Appropriate maximum number of GISCs has come up for discussion

repeatedly since the initial stage called as FWIS:[Extraction from the final report of CBS-ext.02, Annex IV]Several (perhaps four to 10) centres would serve as GISCs. Each GISC would have a defined area of responsibility. GISCs would usually be located within or closely associated with a centre running a global data assimilation system or having some other global commitment, such as a WMC.

[Extraction from ET-CTS outcome reported to ICG-WIS-3 in 2006]Correlation between the number of GISCs and reasonableness of full-mesh topology of a WIS core network: From the practical and relative evaluation, the full-mesh can be appropriate on the assumption that the number of GISCs would be less than 7 inclusive. In case of more GISCs, the full-mesh should be avoided.

However 13 GISCs candidates have been identified as of the end of November 2009.

Sense of a possible basisFurther study is desired to clarify the appropriate maximum number of GISCs from the practical view of network bandwidth requirements.


Slide 5

Slide 5

Hypothesis (1) Flows

Each GISC will receive data from its area of responsibility on an AMDCN

A backup GISC should be available to receive data from Centres from another AMDCN

All GISCs will be linked directly each other (i.e. mesh-topology)

Each GISC will push all data received to all other GISCs

Not all AMDCN will provide the same amount of data. (i.e. unbalanced model)


Slide 6

Slide 6

Hypothesis (2) Technical

A WIS core network to link all GISCs will not be based on multi-cast but uni-cast mechanism for, at least, initial implementation

All network protocol have an overhead. A 20% overhead is a typical value for FTP

Despite control, there will be duplicated data over the network.

In order to define the bandwidth requirements, it will be necessary to estimate:

Daily data volume to be sent and received

Estimate of peak rates if the data transfers are not evenly

spread over the 24 hours.


Slide 7

Slide 7

Points to study (1) The required bandwidth for each GISC will depend on:

The total number of GISCs

The estimated amount of data to be transferred daily by each GISC

The potential peaks of traffic within the 24 hours period

The traffic repartition between AMDCN:

An unbalanced traffic pattern between the GISCs will have a lot of impact

The protocols used on the network and their overhead

Thus we should: i) estimate data volume for global exchange more practically; ii) develop a matrix sheet to indicate bandwidth requirements in each case; iii) recommend an appropriate maximum number of GISCs if possible.


Slide 8

Slide 8

Points to study (2) An online spreadsheet available at

http://spreadsheets.google.com/ccc?key=0AsDEZG6XKnasdG9MNWlRYXUzeVJJRkJsWFppWlI0UEE&hl=fr

Is a tool to be used to automate some calculation for the bandwidth. Based

on values to be defined online by the user, it is then possible to estimate

the required bandwidth.

To access this document a freely available Google account is required

Please use Sheet “Comments and History” to add comments and

suggestions to improve the spreadsheet.

Slide 9

References and hints


Slide 10

Slide 10

IMTN evolution

Exeter

SofiaMelbourne

Buenos Aires

TokyoBeijin

g

Nairobi

Washington

Network INetwork I

Jeddah

Prague

Toulouse

Dakar Algiers

Offenbach

Brasilia

Cairo

New Delhi

Moscow

Network IINetwork II

Exeter

Sofia

Melbourne

Buenos Aires

TokyoBeijing

Nairobi

Washington

Jeddah

Prague

Toulouse

Dakar Algiers

Offenbach

Brasilia

Cairo

New Delhi

IMTN cloud

Moscow

RA I

RA II

RA III

RA IV

RA V

RA VI

Exeter

Sofia

Melbourne

Buenos Aires

TokyoBeijing

Nairobi

Washington

Jeddah

Prague

Toulouse

Dakar Algiers

Offenbach

Brasilia

Cairo

New Delhi

IMTN cloud

Moscow

RA I

RA II

RA III

RA IV

RA V

RA VI

Exeter

Sofia

Melbourne

Buenos Aires

TokyoBeijing

Nairobi

Washington

Jeddah

Prague

Toulouse

Dakar Algiers

Offenbach

Brasilia

Cairo

New Delhi

IMTN cloud

Moscow

RA I

RA II

RA III

RA IV

RA V

RA VI


Slide 11

Slide 11

WIS multicast network InternetInternet

IGDDS,etIGDDS,etcc

Full-mesh topology of WIS core network Full-mesh topology will bring two benefits: (1) maximum redundancy in the configuration of GISC backup channels;(2) operational simplicity in data synchronization

GTSGTSIMTNIMTN

WIS core network

More than 180 centers

Max 18 centers

Less than 13 centers

AMDCN

AMDCN

AMDCN

Conceptual interrelation between GTS and WIS networks

DARDAR

AMDCN


Slide 12

Slide 12

AMDCN

Priorities of study for bandwidth requirements

WIS core

networkAMDC

NGISC

InternetInternetIGDDS,etIGDDS,etcc DARDAR

First prioritySecond priority

Third priority

Target in this period


Slide 13

Slide 13

Architecture and flows


Slide 14

Slide 14

Balanced model (4 GISCs case)

WIS core network

(Full-meshed topology)

Responsible area #1

AMDCN #1

GISC #1

2GB Responsible area #2

AMDCN #2

GISC #2

2GB

Responsible area #3

AMDCN #3

GISC #3


AMDCN #4

GISC #4

2GB

2GB2GB

2GB2GB

2GB2GB

Incoming

daily volume

Outgoing

daily volume

Necessary

port speed

GISC #1

6 GB 6 GBBandwidth

appropriate

for 6GB/day

GISC #2

ditto ditto ditto

GISC #3

ditto ditto ditto

GISC #4

ditto ditto ditto

Example case of total volume of 8GB


Slide 15

Slide 15

WIS core network

(Full-meshed topology)

Responsible area #1

AMDCN #1

GISC #1


AMDCN #2

GISC #2

1GB

Responsible area #3

AMDCN #3

GISC #3

Responsible area #4

AMDCN #4

GISC #4

1GB1GB

1GB1GB

1GB5GB5GB

5GB

1GB1GB1GB5GB

1GB

1GB

Unbalanced model (4 GISCs case)

Incoming

daily volume

Outgoing

daily volume

Necessary

port speed

GISC #1

3 GB 15 GB

Bandwidth

appropriate

for 15GB/day

GISC #2

7 GB 3 GBBandwidth

appropriate

for 7GB/day

GISC #3

ditto ditto ditto

GISC #4

ditto ditto ditto

Example case of total volume of 8GB


Slide 16

Slide 16

Internal 77.81%

FTP 18.01% Sockets 3.81%X.25 0.35%

Async 0.02%

HTTP 0.00%

Email 0.00%

Reference to volume required (1)

Average daily volume received at RTH Washington: 26.66 GB (as of May 2008)

Volume received by Protocol


Slide 17

Slide 17

Internal 8.86%FTP 8.50%

Sockets 81.97%

X.25 0.36%

Async 0.31%


Volume transmitted by Protocol

Average daily volume transmitted from RTH Washington: 30.81 GB (as of May 2008)


Slide 18

Slide 18


Results of RTH data survey 1 August 2007

RTH-A RTH-B RTH-C RTH-D Average Min Max Daily volume (messages) 855 MB 2 GB Daily volume (files) 986 MB 0.55 GB Total daily volume (GB) 1.8 2.6 4.52 2.92 2.96 1.8 4.52

Daily number of messages 202,200 289,784 369,065 397,021 314517.5 202200 397021

Incoming (including duplicated ones)

Daily number of files 425 567 1,540 30,727 8314.75 425 30727

Daily volume (messages) 545 MB 2.4 GB

Daily volume (files) 571 MB 1.6 GB

Total daily volume (GB) 1 4 0.753 6.89 3.18575 0.753 6.89

Daily number of messages 292500 855,318 238,747 1,135,909 630618.5 238747 1135909

GTS

Outgoing (after duplicated check and

elimination)

Daily number of files 526 951 539 47,098 12278.5 526 47098

Total daily incoming traffic (GB) 146 2.6 41.57 8.47 49.66 2.6 146 Download volume

Daily obtaining volume on operational basis 1.8 GB 13 MB

Total daily outgoing traffic (GB) 137 13.9 148.43 19.18 79.6275 13.9 148.43 Internet

Upload volume Daily providing volume on operational basis 2.3 GB 1.1 GB

Incoming Total daily volume (GB) 0.2 24.8 8.1 0.01 8.3 0.01 24.8 Other connections Outgoing Total daily volume (GB) 0.1 62.5 8.3 0.3 17.8 0.1 62.5

Growth Rate Per Annum in volumes 12% 32-50% 35-50%

Extract from ICG-WIS IV (31.VIII.2007) /Doc. 3.1(1)http://www.wmo.int/pages/prog/www/ISS/Meetings/ICG-WIS_ECMWF2007/documents/RRR-Doc3-1-1.doc


Slide 19

Slide 19


Past and projected data volumes (RTH-C)

Extract from ICG-WIS IV (31.VIII.2007) /Doc. 3.1(1)http://www.wmo.int/pages/prog/www/ISS/Meetings/ICG-WIS_ECMWF2007/documents/RRR-Doc3-1-1.doc

Slide 1 ET-CTS - GISC Network Bandwidth RequirementsSlide 1 GISC network bandwidth requirements A proposed method for an educated guess… Hiroyuki Ichijo.

Documents

gisc bandwidth

wis core network

wis realtime network

network capacity

network protocol

backup gisc

wis requirements

giscs candidates