OSG Production Report OSG Area Coordinator’s Meeting Nov 17, 2010 Dan Fraser.

OSG Production ReportOSG Production Report

OSG Area Coordinator’s MeetingOSG Area Coordinator’s MeetingNov 17, 2010Nov 17, 2010

Dan FraserDan Fraser

Overall ProductionOverall Production

OSG DisplayOSG DisplayOSG delivered across 80 sitesIn the last 24 Hours:732,000 Jobs1,114,000 CPU Hours121,000 Transfers805 TB Transferred

In the last 30 Days:13,088,000 Jobs36,099,000 CPU Hours50,860,000 Transfers29,726 TB Transferred

In the last Year:152,016,000 Jobs332,227,000 CPU Hours493,663,000 Transfers204,653TB Transferred

Status at 11:50 AM http://display.grid.iu.edu

Some Production Examples…Some Production Examples…Effort from the entire teamEffort from the entire team Tracking and solving BDII IssuesTracking and solving BDII Issues

One BDII can no longer handle the entire loadOne BDII can no longer handle the entire load Plan to add more BDIIs into round-robinPlan to add more BDIIs into round-robin Plan to upgrade to BDII v5Plan to upgrade to BDII v5 Stress testing; consistency verification; VM testing (problems)Stress testing; consistency verification; VM testing (problems) Patched by GOCPatched by GOC

General CERN BDII Instabilities (often detected by OSG)General CERN BDII Instabilities (often detected by OSG)

BNL dropping out of CERN BDIIBNL dropping out of CERN BDII Turned out to be a GigaPOP issue at IndianaTurned out to be a GigaPOP issue at Indiana

SL5 Kernel vulnerability patchSL5 Kernel vulnerability patch GOC now using Puppet for s/w managementGOC now using Puppet for s/w management Atlas critical alarm against T1 dCache systemAtlas critical alarm against T1 dCache system

Verification of alarm processVerification of alarm process

Updated View from ProductionUpdated View from ProductionNew VOs can quickly come up to speed (with handholding)New VOs can quickly come up to speed (with handholding)

LSST capable of getting >60k hours/dayLSST capable of getting >60k hours/day

Pilot factory running at SDSC now supporting multiple VOsPilot factory running at SDSC now supporting multiple VOs CMS, HCC, SBGrid/NEBioGridCMS, HCC, SBGrid/NEBioGrid GlueX, IceCube, Glow (setup but not really used yet)GlueX, IceCube, Glow (setup but not really used yet)

Opportunistic storage is the #1 problemOpportunistic storage is the #1 problem A very difficult problemA very difficult problem No OSG solution on the horizon. No OSG solution on the horizon. CMS and Atlas experimenting with an Xrootd based data access CMS and Atlas experimenting with an Xrootd based data access

strategy using “transparent remote streaming and data caching” to strategy using “transparent remote streaming and data caching” to create a more “global” system (Brian)create a more “global” system (Brian)

It could eventually have some implications for OSG …It could eventually have some implications for OSG …