Particle Physics Data Grid - From Fabric to Physics Computing Services Fermilab Computing www.ppdg.net the VDT www.griphys.org/vdt Steering Committee (PI’s in bold) Richard Mount (SLAC), Miron Livny (Wisc.), Harvey Newman (Caltech), Ruth Pordes (FNAL), Lothar Bauerdick (FNAL), Roger Barlow (Man.), Ian Foster (ANL), Bruce Gibbard (BNL), John Huth (Harvard), Andy Kowalski (JLab), Bill Kramer (LBNL), Jerome Lauret (BNL), Wyatt Merritt (FNAL), Reagan Moore (SDSC), Dave Morrison (BNL), Greg Nawrocki (ANL), Doug Olson (LBNL), Don Petravick (FNAL) , Arie Shoshani (LBNL) Authentication and Authorization Protocols (e.g., GSI) Compute Resource Management Data Movement (Brokering) Collective 2: Services specific to application domain or Virtual org. Data Federation Services Data Transport Services Resource Monitoring / Auditing Data Discovery Services Data Filtering or Transformation Services Compute Scheduling (Brokering) Monitoring / Auditing Services Collective 1: General services for coordinating multiple resources Collective Database Management Services Data Filtering or Transformation Services Storage Resource Managment File Transfer Service (GridFTP) Resource: Sharing Single Resources Other Storage Systems Compute Systems Mass Storage System (e.g., HPSS) Networks Communication Protocols (e.g. TCP/IP stack) Site Authen. and Authorization Systems Connectivity Fabric Request Interpretation and Planning Svcs Workflow of Request Mgmt Svcs Application Specific Data Discovery Svcs Community Authorization Services Troubleshooting Previous Proposed Legend - “Proposed” identifies new work areas and “Previous” Identifies areas with significant accomplishments. Diverse global services Core services Local OS A p p l i c a t i o n s As PPDG progresses from initial integration of grid middleware into the experiments end-to-end applications to deployment of these applications running across a broad shared grid fabric the program of work is being adjusted accordingly. The diagram above shows the various elements of a grid software architecture in the different layers as well as the “hourglass” illustrating the advantage of using a standard suite of middleware for core services that enables creativity and decoupling of fabric layer services from application functionality. PPDG endorses the use of the VDT packaging of the grid middleware as standard. VDT is increasingly used PPDG experiments and envisioned to be universally adopted as a kernal for Grid deployment. The architecture is annotated to show those elements where significant effort was devoted in the past and those where significant effort is being devoted for the future. The goal and focus of PPDG is to achieve grid-enabled end-to-end applications of the experiments running on a distributed and shared grid fabric to enable frontier science for international virtual organizations. Beginning in 2004 as the next phase, PPDG is partnering with the major DOE scientific computing facilities for high-energy and nuclear physics to form a shared grid fabric supporting the data analysis and simulations applications of all the experiments participating in PPDG. This effort and shared grid deployment is aligned with the Open Science Grid roadmap (www.opensciencegrid.org) and we expect this to be part of and help encourage an expanding shared grid for scientific computing. PPDG is benefiting GLOBUS, CONDOR, SRB, SRM through contributions to: • Robustness • Scalability • Research directions • Usability PPDG is benefiting the Experiments: • Robust, sustained, hands-off, production data transfer of terabytes of data using GridFTP and SRM implementations. • Grid-based job scheduling and execution based on Condor-G, DAGMan and GRAM • Deployment and use of end-to-end Grid systems for experiment applications. In particular the recent Grid2003 shared by U.S. ATLAS and U.S. CMS (among others) • Procedures, integration and support for ESnet’s DOEGrids certificate-based authentication at all the facilities • Demonstrations of interoperability and compatibility across the US and European Grid infrastructures of the experiments, in particular BaBar, D0 and CDF, US ATLAS and US CMS. PPDG End-to-End Applications over the Grid and End-to-End Grid Systems Collaborating Communities of Scientists Physics from Simulated ATLAS Events BaBar DataSets Distributed Worldwide Physics at STAR! Triggered Correlation Studies from Datasets on the Grid Simulation Production for DZero - No. of Files by Remote Site US-CMS simulation production on the grid is growing and surpasses 25% of the total simulation.