OSG All Hands, 3/5/2007 Cindy Zheng Peter Arzberger Philip Papadopoulos Mason Katz P acific R im A pplication and G rid M iddleware A ssembly University.

OSG All Hands, 3/5/2007

Cindy ZhengPeter Arzberger

Philip PapadopoulosMason Katz

Pacific Rim Application and Grid Middleware AssemblyUniversity of California, San Diego

http://www.pragma-grid.nethttp://goc.pragma-grid.net

PRAGMA: A Framework For Collaborations

OSG All Hands, 3/5/2007

Overview

• PRAGMA– Goal and approach

• Collaborations– In testbed

• Applications and middleware• With science and technology teams

– With other Grids• GIN-OPS, Peer-grids

– In education and training• PRIME, PRIUS, NBCR institute

OSG All Hands, 3/5/2007http://www.pragma-grid.net

Overarching GoalsPRAGMA

“A Practical Collaborative Framework”.

Strengthen Existing and Establish New Collaborations

Work with Science Teams to Advance Grid Technologies and Improve the

Underlying Infrastructure

In the Pacific Rim and Globally

OSG All Hands, 3/5/2007

Overview and ApproachProcess to Promote Routine Use Team Science

Application-Driven CollaborationsApplications Middleware

Routine Use Lab/TestbedTesting Applications

Building Grid and GOC

Multiway DisseminationKey Middleware

Workshops and Organization

Information Exchange

Planning and Review

New Collaborations

New Members

Expand Users

Expand Impact

Outcomes

Improved middlewareBroader Use

New CollaborationsTransfer Tech.

StandardsPublications

New KnowledgeData AccessEducation

OSG All Hands, 3/5/2007

PRAGMA Grid TestbedPRAGMA Grid Testbed

31 Clusters from 27 institutions in 14 countries/regions (+8 in preparation)

UZurichSwitzerland

NECTECThaiGridThailand

UoHydIndia

MIMOSUSMMalaysia

CUHKHongKong

ASGCNCHCTaiwan

IOIT-HCMVietnam

AISTOsakaUUTsukubaTITechJapan

BIIIHPCNGOSingapore MU

Australia

APACQUTAustralia

KISTIKorea

JLUChina

SDSCUSA

CICESEMexico

UNAMMexico

UCNChile

UChileChile

UMCUSA

UUtahUSA

NCSAUSA BU

USA

ASURCCosta Rica

BESTGridNew Zealand

CNICGUCASChina

AIST

SDSC

NGO

NECTECThaiGrid

5 gfarm sites

OSG All Hands, 3/5/2007

Applications and Middleware http://goc.pragma-grid.net/applications/default.html

• Real science applications pair and drive middleware development

• Achieve long-run and scientific results• Open to applications of all scientific disciplines

– Climate simulation• Savannah/Nimrod (MU, Australia)• MM5/Mpich-Gx (CICESE, Mexico; KISTI, Korea)

– Quantum-mechanics, quantum-chemistry:• TDDFT, QM-MD, FMO/Ninf-G (AIST, Japan)

– Genomics• iGAP/Gfarm/CSF (UCSD, USA; AIST, Japan; JLU,

China)• HPM: genomics (IOIT-HCM, Vietnam)• mpiBlast/Mpich-G2 (ASGC, Taiwan)

– Organic chemistry• Gamess-APBS/Nimrod (UZurich, Switzerland)

– Molecular simulation• Siesta/Nimrod (UZurich, Switzerland; MU,

Australia)• Amber/Rsh ( USM, Malaysia)

– Compute Science• Load Balancer (VAST-HCM, Vietnam)• GriddLeS (MU, Australia)

OSG All Hands, 3/5/2007

Mutually beneficial• Nimrod based Gamess-APBS

– Gamess-APBS: UZurich, Switzerland– Nimrod: MU, Australia

• Mpich-Gx based MM5 and WRF– MM5 and WRF: CICESE, Mexico– Mpich-Gx: KISTI, Korea

• Test/develop Gfarm and CSF with iGAP– Gfarm: AIST, Japan– CSF: JLU, China– iGAP: UCSD, USA

Application/Middleware Collaborationshttp://wiki.pragma-grid.net http://goc.pragma-grid.net

OSG All Hands, 3/5/2007

• Grid security– Naregi (Japan), APGrid, GAMA (SDSC, USA)

• Grid infrastructure– Monitoring - SCMSWeb (ThaiGrid, Thailand)– Accounting - MOGAS (NTU Singapore)– Metascheduling - Community Scheduling

Forum (JLU, China)– Cyber-environment - CSE-Online (UUtah,

USA)– Rocks and middleware (SDSC, USA; …)

• Ninf-G, SCE, Gfarm, Bio, K*Rocks, Condor, …• Datagrid, sensor, network

– Gfarm-fuse (AIST, Japan)– GEON data network– GLEON sensor network– OptIPuter

• High performance networked TDW• Telescience

Collaborations With Science and Technology Teams

OSG All Hands, 3/5/2007

Controlling @ CNIC Viewing @ UCSD

TDW/OptIPuter ExampleSource: Elaine Liu, PRIME 2006

OSG All Hands, 3/5/2007

Collaborations in BioScience

• Avian Flu Research Project– UCSD, CNIC, JLU, Uni. Of Tskuba/AIST,

University of Hawaii and Konkuk University– EGEE – Academia Sinica

• Biosciences Portal– Collaboration between Osaka University,

University of Queensland, JLU, UCSD.– Involves PRIME, PRIUS and summer students

from UCSD, Osaka and JLU.• Metagenomics Research and Genome

Informatics– Bioinformatics, computational biology software– Gfarm/CSF4, TskubaU/JLU, Japan/China– With University of Wisconsin, UCSD, PRIME

project

OSG All Hands, 3/5/2007

GEON-iGEON-PRAGMA

GEON is a coalition among IT and Earth Science researchers with the goal of developing advanced information technologies to enable new modes of geosciences research GEON is developing technologies for information integration and knowledge discovery

www.geongrid.org

Internationalization of GEON – iGEON PRAGMA

Some sites already working on geoscience projects Building a datagrid Welcome geoscience applications Seeking collaboration in grid interoperation

OSG All Hands, 3/5/2007

PRAGMA and GEON/iGEON collaboration

Sharing data, software know-how, interoperate grids• Geoscience workgroup established at

PRAGMA11, joint conferences with PRAGMA• GUCAS (China)

– Setup a GEON node in PRAGMA testbed

• UMC (USA) and GUCAS– Preparing a GEON application to run in PRAGMA

testbed

• CNIC (China) – Is setting up a GEON data node in PRAGMA

testbed

• UoHyd (India) – has setup a GEON node, will join PRAGMA testbed

• AIST Geogrid (Japan) plan– Share data with GEON– Interoperate with PRAGMA testbed

OSG All Hands, 3/5/2007

Collaboration in Environmental Science• GLEON

– A grassroots network of• People: lake scientists,

engineers, information technology experts

• Institutions: universities, national laboratories, agencies

• Programs: PRAGMA, AS-Forest Biogeochemistry,US-LTER, TERN, KING, EcoGrid, etc.

• Instruments • Data

– Linked by a common purpose and cyberinfrastructure

– With a goal of understanding lake dynamics at local, regional, continental, and global scales

OSG All Hands, 3/5/2007

Collaborate in Publishing Research Results

Some published papers in 2006:• Arzberger P, Papadopoulos P. PRAGMA: Example of Grass-Roots Grid Promoting

Collaborative EScience Teams. CTWatch. Vol 2, No. 1 Feb 2006. www.ctwatch.org/quarterly/articles/2006/02/pragmaexample-of-grass-roots-grid-promoting-collaborativee-science-teams

• Abramson D, Lynch A, Takemiya H, Tanimura Y, Date S, Nakamura H, Jeong K, Hwang S, Zhu J, Lu Z, Amoreira C, Baldridge K, Lee H, Wang C, Shih HL, Molina T, Li, W, Arzberger P. Deploying Scientific Applications on the PRAGMA Grid testbed: Ways, Means and Lessons. IEEE/CCGRID International Workshop on Grid Computing, 2006, Singapore.

• Lee B-S, Tang M, Zhang J, Soon O Y, Zheng C, Arzberger P. Analysis of Jobs on a Multi-Organizational Grid Testbed. IEEE/CCGRID Int’l Workshop on Grid Computing, 2006, Singapore.

• Zheng C, Abramson D, Arzberger P, Ayuub S, Enticott C, Garic S, Katz M, Kwak J, Lee B S, Papadopoulos P, Phatanapherom S, Sriprayoonsakul S, Tanaka Y, Tanimura Y, Tatebe O, Uthayopas P. The PRAGMA Testbed: Building a Multi-Application International Grid. 2005 IEEE/CCGRID International Workshop on Grid Computing, 2006, Singapore.

• Li WW, Arzberger PW, Yeo CL, Ang L, Tatebe O, Sekiguchi S, Jeong K, Wuang S, Date S, Kwak JH. Proteome Analysis Using iGAP in Gfarm. The Second International Life Science Grid Workshop 2005, Grid Asia 2005, Singapore 2005.

• Wei X, Ding Z, Li W W, Tatebe O, Jiang J, et al. GDIA: A Scalable Grid Infrastructure for Data Intensive Applications. IEEE Int’l Conference on Hybrid Information Technology, ICHIT 2006, Cheju Island, Korea.

• Krishnan S, Baldridge K K, Greenberg J. P, Stearn B, Bhatia K. An End-to-End Web Services-Based Infrastructure for Biomedical Applications. Proceedings of Grid 2005, 6th IEEE/ACM Int’l Workshop on Grid Computing, November 13-14, 2005, Seattle, WA, U.S.

• …

OSG All Hands, 3/5/2007

Grid Interoperation Now (GIN)http://forge.gridforum.org/sf/wiki/do/viewPage/projects.gin/wiki/GinOps

OGF – GIN-OPS• GIN testbed (February, 2006 – on-going)• TDDFT/Ninf-G (PRAGMA - AIST, Japan)

– PRAGMA, TeraGrid, OSG, NorduGrid; EGEE• Savanah fire simulation (PRAGMA - MU, Australia)

– PRAGMA, TeraGrid, OSG• Multi-Grid monitoring

– SCMSWeb probe matrix (PRAGMA - ThaiGrid, Thailand)– Common schema (PRAGMA, TeraGrid, EGEE,

NorduGrid)

OSG All Hands, 3/5/2007

• PRAGMA->TeraGrid (October, 2006 – on-going)– PRAGMA member runs application across both grids– QM/MD/Ninf-G (AIST, Japan)– Manual reservation, 7 sites in PRAGMA, 3 sites in TeraGrid

• OSG<->PRAGMA (January, 2007 – on-going)– Members from both grids run applications across both grids– Applications:

• OSG– GISolve– spatial Interpolation– UIowa, USA

• PRAGMA– FMO/Ninf-G– quantum Chemistry– AIST, Japan

Peer-grid Interoperation Experimentshttp://goc.pragma-grid.net/wiki/index.php/Main_Page#Grid_Inter-operations

OSG All Hands, 3/5/2007

OSG-PRAGMA Interoperation Experimenthttp://goc.pragma-grid.net/wiki/index.php/OSG-PRAGMA_Grid_Interoperation_Experiments

Resources• OSG

– FermilabGrid• PRAGMA

– SDSC, USA – NECTEC, Thailand – NGO, Singapore– ThaiGrid, Thailand

Process• Jan 4, 2007, identify who and how

– Coordinator, application drivers, resource supporters

– Wiki, application, requirements• Jan 25, 2007, initial VTC

– Goals, Q/A exchange• Start working

– Email immediately issues, solutions– Weekly status, email, wiki

Status• Feb 26 - March 1, started application

runs successfully

A set of jobs are processed on the VO, collected, and updated for the next iteration.

Learned• Differences in grid environment effect users

and applications– Job submission

• Change RSL parameter• Remove inter-node parallel execution

– Security policy• Take work to setup access• Account on gatekeeper node

• Large scale calculations are possible• Middleware fault tolerance and flexible

resource management is important

OSG All Hands, 3/5/2007

PRIME http://prime.ucsd.edu

• Providing UCSD undergraduate students international interdisciplinary research internships and Cultural experiences

• Preparing the global work place of the 21st century• Built on top of PRAGMA R&D network and activities for

Undergraduate Research• A Pilot Project for Global Engagement

PRIME Class 2006

• Summer research abroad• Mentors from UCSD and

host institutions• Real science and practical

projects• Learning science,

technology and culture• Demo’s and talks• Continuing research

involvement thereafter

PRIME 2006 (Thrid Year)Host Institutions and Projects at a Glance

• CNIC, China– Virtual Screening of Avian Influenza H5N1

Neuraminidase and Hemagglutinin proteins against two libraries of ligands

– High-resolution astronomial visualization and visual collaboration environment

• MU, Australia– Computational analysis of biological structures

using GAMESS-APBS/Nimrod programs– Develop efficient method for large scale virtual

screening of ligands– More stable ionic model of ventricular myocytes

• OsakaU, Japan– Homology modeling of the protein kinases and

binding site prediction– Docking simulations of protein kinases and virtual

screening of ligands for inhibitors– Established networked Tile Display Wall

• NCHC, Taiwan– implementing image-processing software for

neural networks– Develop software tools for earthquake

experiments, monitoring and analysisPrime Students 2006Prime Students 2006Hosted by CNIC, China Hosted by CNIC, China At Lao She Tea HouseAt Lao She Tea House

OSG All Hands, 3/5/2007

PRIUS - Pacific Rim International UniverSity Based at Osaka University

• Establish a consistent educational program for graduate students.

• On-the-Job-training and education with PRAGMA

Lecture course:Stimulate students’ ambitious

Short Abroad Internship:Offer trial opportunities

Long Abroad Internship:Provide skill building chances

M1

M2

Doc

P.D Exchange prgm: produces liquidity in human resource

P.D

Educational Part

Research Part

Source: Susumu Date, Osaka University

http://prius.ics.es.osaka-u.ac.jp/en/index.html

OSG All Hands, 3/5/2007

PRIUS (First Year)

• 10 International instructors– All from PRAGMA members

• UCSD, USA• NCHC, Taiwan• NTU, Singapore• QUT, MU, Australia

– Grid computing/applications• Grid and web services• Clustering tools• Optical network computing• Grid accounting systems• Biomedical grid applications• Economic and social impact of

cyberinfrastructure– Interaction with students

• Internship abroad– UCSD, USA

• Develop QM/MM hybrid simulation program for protein research using OPAL

– NTU, Singapore• Design/prototype a grid security

monitoring system as an add-on to MOGAS

– QUT, Australia• Develop a bioscience portal for

PRAGMA– NCHC, Taiwan

• Implement high-definition-quality streaming video over a high-speed network with various TDW technologies

OSG All Hands, 3/5/2007

Future Meeting

• 20 – 22 March 2007, Bangkok Thailand– PRAGMA 12 Hosted by NECTEC and Thai

National Grid Center– 18 – 20 March 2007: GEOGrid Workshop

• Fall 2007, Urbana-Champaign USA– PRAGMA 13 Hosted by NCSA

• Spring 2008, Hsinchu Taiwan– PRAGMA 14 Hosted by NCHC

• Fall 2008, Penang Malaysia– PRAGMA 15 Hosted by USM

OSG All Hands, 3/5/2007

http://www.pragma-grid.nethttp://goc.pragma-grid.net

Thank You

• PRAGMA is supported by the National Science Foundation (Grant No. INT-0216895, INT-0314015, OCI -0627026), and member institutions

• PRIME is supported by the National Science Foundation under NSF INT 04007508