- 1 - uMeteo-K Principle Investigator : Jai-Ho Oh (Pukyong National Univ, KOREA) Co-researchers: In-Sik Kang (Seoul Nation al Uuiv, KOREA) Byung-Lyol Le e (NCAM/KMA, KOREA) 17 17 th th APAN Meetings APAN Meetings arch Collaborations Among Institut arch Collaborations Among Institut Employing Grid Technology Employing Grid Technology January 29, 2004 Hawaii Imin International Conference Center University of Hawaii, Honolulu, Hawai i, USA
57
Embed
UMeteo-K - 1 - Principle Investigator : Jai-Ho Oh (Pukyong National Univ, KOREA) Co-researchers: In-Sik Kang (Seoul National Uuiv, KOREA) Byung-Lyol Lee.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
- 1 -
uMeteo-KuMeteo-K
Principle Investigator : Jai-Ho Oh (Pukyong National Univ, KOREA)
Co-researchers: In-Sik Kang (Seoul National Uuiv, KOREA)
Byung-Lyol Lee (NCAM/KMA, KOREA)
1717thth APAN Meetings APAN Meetings
Research Collaborations Among InstitutionsResearch Collaborations Among InstitutionsEmploying Grid TechnologyEmploying Grid Technology
January 29, 2004
Hawaii Imin International Conference CenterUniversity of Hawaii, Honolulu, Hawaii, USA
- 2 -
uMeteo-KuMeteo-K
Main GoalsMain Goals
Establishment of uMeteo-K, ubiquitous Korea meteorological research cooperation system
- 3 -
uMeteo-KuMeteo-K
About uMeteo-KAbout uMeteo-K
The concept of virtual lab. for interdisciplinary meteorological researches
- Cooperation meteorological research environment (Access Grid)- Parallelized numerical weather/climate prediction modeling (Computational Grid)- Virtual server for large meteorological data (Data Grid)
5 2003. 7. 13. SNU & Tokyo Uni. 세미나 국내전문가ⅹ 8 인 + 국외전문가ⅹ 8 인 = 8,800,000 +α
6 2003. 7. 31. 중간발표 대비 점검 ( 국내전문가ⅹ 1 인 + 수행연구원ⅹ 2 인 ) 2ⅹ 일 = 300,000 +α
7 2003. 8. 26. GFK 중간보고 및 발표 ( 국내전문가ⅹ 1 인 + 수행연구원ⅹ 2 인 ) 2ⅹ 일 = 300,000 +α
8 2003. 8. 27. 16th APAN 국제회의 ( 국내전문가ⅹ 5 인 ) 6ⅹ 일 + 국외전문가ⅹ 5 인 = 8,000,000 +α
9 2003. 9. 3. KISTI 홍보자료 제작 ( 국내전문가ⅹ 5 인 + 수행연구원ⅹ 3 인 ) 2ⅹ 일 = 800,000 +α
10 2003. 9. 25. CAgM/ETSIDN 회의 국외전문가ⅹ 5 인 + 국내전문가ⅹ 3 인 = 7,800,000 +α
11 2003. 10. 23. 부산외국어대학교 강의 국내전문가ⅹ 1 인 + 수행연구원ⅹ 2 인 = 200,000 +α
12 2003. 11. 9. uMeteo-K 8th Workshop ( 국내전문가ⅹ 6 인 + 수행연구원ⅹ 4 인 ) 3ⅹ 일 = 2,700,000 +α
13 2003. 11. 21. 2th 레이더 전문가 협의회 ( 국내전문가ⅹ 4 인 + 수행연구원ⅹ 4 인 ) 2ⅹ 일 = 1,200,000 +α
14 2003. 12. 1. Winter G.F.K workshop ( 국내전문가ⅹ 1 인 + 수행연구원ⅹ 3 인 ) 3ⅹ 일 = 750,000 +α
15 2003. 12. 18. GLOBEC Workshop ( 국내전문가ⅹ 1 인 + 수행연구원ⅹ 2 인 ) 2ⅹ 일 = 400,000 +α
16 2003. 12. 23. 슈퍼컴퓨팅 성공사례발표 ( 국내전문가ⅹ 1 인 + 수행연구원ⅹ 2 인 ) 2ⅹ 일 = 400,000 +α
17 2004. 1. 6. 초청 세미나 국외전문가ⅹ 1 인 =1,000,000 +α
18 2004. 1. 9. Workshop on Grid-tech.( 국내전문가ⅹ 6 인 + 수행연구원ⅹ 6 인 ) 3ⅹ 일 =2,700,0000 +α
Benefits/Costs USD 32k +α USD 27k국내 전문가 1 인 =100,000 원 , 국외전문가 1 인 =1,000,000 원 , 수행연구원 1 인 =5 만원
- 24 -
uMeteo-KuMeteo-K
Differences between AG 1.2 and AG 2.0Differences between AG 1.2 and AG 2.01. Easy Venue Server configuration and usage2. Instead of PIG, using individual Venue Server 3. Complexity about Certification Authority4. Sharing Data, Sharing Application, Sharing Service enforced5. GUI6. Service total Management about Access Grid utility
Data sharing
Remote controll
bridge direct access
Manage total applicationsthrough one program
Manage total applicationsthrough different programs
AG 1.2 AG 2.0
client server server server
client client
- 25 -
uMeteo-KuMeteo-K
Operate AG 2.0 – (1) Video ConferencingOperate AG 2.0 – (1) Video Conferencing
- 26 -
uMeteo-KuMeteo-K
Operate AG 2.0 – (2) Sharing DataOperate AG 2.0 – (2) Sharing Data
- 27 -
uMeteo-KuMeteo-K
Operate AG 2.0 – (3) Data DownloadOperate AG 2.0 – (3) Data Download
- 28 -
uMeteo-KuMeteo-K
Operate AG 2.0 – (4) Shared BrowerOperate AG 2.0 – (4) Shared Brower
- 29 -
uMeteo-KuMeteo-K
Operate AG 2.0 – (5) Shared PresentationOperate AG 2.0 – (5) Shared Presentation
- 30 -
uMeteo-KuMeteo-K
• Enable face-to-face meeting activities
• What can be done:– Sharing Data– Shared Applications– Text chat
- File transfer to venus.gridcenter.or.kr using gsincftpput and gsincftpget
- 40 -
uMeteo-KuMeteo-K
• It is successful to compile MM5 ( Mesoscale Model version 5 ) and CReSS ( Cloud resolving Storm Simulator)• It is successful to run CReSS, but fail to run MM5 because testbed is not supported some library and routine.
Running weather numerical modelRunning weather numerical model
Research & development in Nagoya Univ., JapanResearch & development in Nagoya Univ., JapanResearch & development in Nagoya Univ., JapanResearch & development in Nagoya Univ., Japan
Meso-to-cloud scale simulation systemMeso-to-cloud scale simulation system Inclusion of a detailed cloud Inclusion of a detailed cloud .. microphysical processes microphysical processes
It takes many times for simulation.
Meso-to-cloud scale simulation systemMeso-to-cloud scale simulation system Inclusion of a detailed cloud Inclusion of a detailed cloud .. microphysical processes microphysical processes
It takes many times for simulation.
Performance of parallel processingPerformance of parallel processingPerformance of parallel processingPerformance of parallel processing
- 42 -
uMeteo-KuMeteo-K
Downburst Simulation for 21000 sec.
< Wind vector in y section [m s-1] >
- 43 -
uMeteo-KuMeteo-K
Parallel CReSS Benchmarks with GLOBUS
The required time for 21000 sec (1 hour) model integration in uMeteoK Testbed
The required time for 21000 sec (1 hour) model integration in KISTI Testbed
Jupiter 8 nodes 16 min 39 sec
Venus 8 nodes 15 min 35 sec
Jupiter 4 nodes + Venus 4 nodes 715min 13 sec
Jupiter 8 nodes + Venus 8 nodes
PKNU cluster 8 nodes (MPICH) 15 min 49 sec
PKNU A 4 nodes + PKNU B 4 nodes 16 min 39 sec
- 44 -
uMeteo-KuMeteo-K
GASS
- Simple, Multi-protocol file transfer Tools, integrated with GRAM
GridFTP
- Provides high-performance, reliable data transfer for modern WANS
Data Grid in Globus ToolsData Grid in Globus ToolsData Transport & Access
Data Replica Catalog
- Provides a catalog service for keeping track of replicated datasets
Replica Management
- Provides services for creating and managing replicated datasets
Data Replication and Management
- 45 -
uMeteo-KuMeteo-K
Data grid for climate prediction Data grid for climate prediction Data transportation
SNU
KISTI
Supercom
PKNU
Observation data
Forecast output
Forecast output
NASA
NCEP KMA
Model output
Model output
Data input Data input
Wu-Ftp Connecting
KMA
Supercom
NASA
Model output
Atmospheric e-science Data Grid
Forecast output
Model output
- 46 -
uMeteo-KuMeteo-K
Hardware structure of Data-GRIDHardware structure of Data-GRID
Linux serverIntel Dual CPU
Linux serverIntel Dual CPU
Linux serverIntel Dual CPU
cdldata
cpsdata
climate
Disk Raid2.3TB
Disk Raid1.8TB
Disk Raid500G
Linux serverIntel Dual CPU
neosky15
Linux serverIntel Dual CPU
pknuGB01
KORENNetwork
KORENNetwork
KORENNetwork
Disk Raid1.2TB
Disk Raid500G
Seoul National University
Korea MeteorologyAdministration
Pukyong NationalUniversity
- 47 -
uMeteo-KuMeteo-K
Overview Data gridOverview Data gridldapadd Adding the uMeteoK-DataGrid to LDAP server
Adding the Test-catalog to LDAP server
Creating the Test-Collection in the test-catalog
Register location for test-collection (snu-kma-pknu)
Create & Register & list file-list for all location
Copy file snu->kma or snu->pknu
List all location where some files can be found
Delete the files & location
Delete the Test-collection & Test-catalog
ldapadd
Globus-replica-management
Globus-replica-management
Globus-replica-Management& catalog
Globus-replica-management
Globus-replica-catalog
Globus-replica-management
Globus-replica-management
- 48 -
uMeteo-KuMeteo-K
Grid FTPGrid FTP
Using ‘globus-url-copy’Secured ftpIncluding Globus Tool kit
Grid-ftp
Using ‘NCFTP’ Programsreliable and re-startable data transferparallel and Striped Data transferAutomatic negotiation of TCP buffer/window sizes
GSINCFTP
Globus Tool kit
Grid-Proxy-Init
Data transfer &Access
Data grid middleware
User certification
- 49 -
uMeteo-KuMeteo-K
Data Grid in Globus ToolsData Grid in Globus ToolsGlobus Toolkit install-process
It appear that Test Result of Globus Software(globusrun) is successful
Grid Software : Globus 2.4 Toolkit
CA Software : SimpleCA
Result of run /bin/data
Grid-proxy-init
Globus-personal-gatekeeper
Globusrun
- 50 -
uMeteo-KuMeteo-K
Data Grid in Globus ToolsData Grid in Globus ToolsGSINCFTP Testing - Successful
SNU-SNU- Cdldata climate
SNU-KMA- Cdldata Neosky15
SNU-PKNU- Cdldata PknuGB01
- 51 -
uMeteo-KuMeteo-K
Replica management Replica management
- Layer on other Grid services GSI, transport, information service
- Use LDAP as catalog form at and protocol, for consistency
- Use as a building block for other tools
Characteristic of Replica management
- LDAP object classes for representing logical-to physical mapping in an LDAP catalog- Low-level replica catalog * globus_replica_catalog library * manipulate replica catalog-High-level replica catalog * globus_replica_manager library * Combines calls to file transfer operation calls to low-level API functions
Data Replication Management TestData Replication Management TestLocation A
host name : climate.snu.ac.krSlapd & Grid-FTP running
Logical Name : climate prediction ALocation : /home/globus/dataList of file : climatePrediction 1
climatePrediction 2climatePrediction 3
Location B host name : cdldata.snu.ac.kr
Slapd & Grid-FTP runningLogical Name : climate prediction B
Location : /home/ymyang/dataList of file : climatePrediction 4
climatePrediction 5climatePrediction 6
Copy Delete Copy Delete
- 54 -
uMeteo-KuMeteo-K
Data Grid in Globus ToolsData Grid in Globus ToolsScenario for globus replica service – between ‘cdldata’ of SNU and ‘pknuGB01’ of PKNU
Running Shell Script for globus replica services
In the Copy command, there is someProblem that it is available to copy
a file from SNU to PKNU, but it Occurred error when doing opposite.We suppose that it result in from
Firewall policy of PKNU Computing Agency
- 55 -
uMeteo-KuMeteo-K
In the futureIn the future•Multicasting network might be necessary for duplex Access Grid. => Currently, ICSYM/PKNU and NCAM/KMA within uMeteo-K used unicast quick bridge network (KISTI supported)
•Enhancement of display panel and improvement of sound quality might be necessary. =>Update to AG version 2.0
•An urgent repair might be necessary for the failure on the common DNS identification among members of uMeteo-K
•Addition to uMeteo-K CG testbed nodes
• Between SNU and KMA, we is about to testing Globus replica service
• Application of realistic scenarios for uMeteo-K
- 56 -
uMeteo-KuMeteo-K
Future plan of uMeteo-KFuture plan of uMeteo-K
Ocean Grid
NDP Grid
APCN Grid
Environ Grid
Energy Grid
Hydrologic Grid
N*Grid
CAgM Grid
uMeteo-KuMeteo-K
- 57 -
uMeteo-KuMeteo-K
Thank you for your Thank you for your attention!attention!