Preparation of KIPT (Kharkov) computing facilities for CMS data analysis L. Levchuk Kharkov Institute of Physics and Technology (KIPT), Kharkov, Ukraine
Jan 21, 2016
Preparation of KIPT (Kharkov) computing facilities for CMS data
analysis
L. Levchuk
Kharkov Institute of Physics and Technology (KIPT), Kharkov, Ukraine
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 2
• WLCG infrastructure of the CMS experiment (CMS computing model)
• KIPT T2 center (T2_UA_KIPT): status and prospects
• Summary
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 3
CERN: acquisition, first-pass reconstruction, storage & distribution
1.25 GB/sec (heavy ions)
3
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 4
CMS Physics GoalsCMS is expected to shed light on • origin of particle masses (search for Higgs boson for all
possible MH: does it really exist?)
• origin of “dark matter” in the Universe (search for SUSY signals)
• origin of asymmetry matter/antimatter in the Universe (study of CP violation in B decays)
• whether new states of hadronic matter exist (study of heavy ion collisions)
• if any “exotica” can emerge at the TeV (1012 eV)) scale
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 5
T2_ * _ *
T2_UA_KIPTT2_ * _ *T2_RU_JINR
T2_ * _ *
50 sites
Storage of RECO and AOD data(~0.1-0.3 MB/evt)
Data analysis & MC production
Connectivity: ~1 Gbit/s; SE: ~200 TB WN: ~4 kHSP06 (LHC startup)
CMS trigger T0_CH_CERN
T1_CH_CERN
T1_DE_FZK
T1_ES_PIC
T1_FR_CCIN2P3
T1_IT_CNAF
T1_TW_ASGC
T1_UK_RAL
T1_US_FNAL
8 sites
event reconstruction; storage of data of all types (RAW, RECO, AOD, skim)
T3_ * _*
currently 50 sites
WLCG User Interface (UI); no special requirements for resources but reasonable connectivity with T2
RAW
~1-3 MB/evt ~100 evt/s
CMS Computing ModelCMS Computing Model
CMS Dashboard
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 7
KIPT CMS Computing FacilityKIPT CMS Computing Facility
HEPSPEC06 Benchmark Results:system OS kernel 32/64 mem gcc benchmark total per core
Dual Intel Xeon EM64T 3.4 GHz SLC5 2.6.18-128.7.1.el5 64 4 GB 4.1.2 S2k6 all_cpp 64bit 13.8 6.9
Dual CPU, Quad Core,
Intel Xeon E5420 2.5GHz
SLC5 2.6.18-128.7.1.el5 64 16 GB4.1.2 S2k6 all_cpp 64bit 63.8 8.0
Storage Element (SE, SRM):
12 nodes (24 CPU’s Intel Xeon EM64T: 203.2 + 42.8 GHz)11 SATA RAID5 (400 GB disks) +1 SATA RAID6 (1.0 TB disks)
SRM space: ~ 50 TB (currently), upgrade to ~120 TB by LHC startup (end of 2009)
Type: DPM (SLC4)
Worker Nodes (WN):
68 x86-64 CPU cores (Intel Xeon EM64T 3.4 GHz & E5420 2.5GHz), 0.5 kHEPSPEC06
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 8
KIPT CMS Computing KIPT CMS Computing FacilityFacility
Year
2001 – RH Linux cluster of a few dual PIII nodes
from 2002 on – participation in CMS Monte Carlo production
2003 – Grid middleware (LCG2) deployed for the first time
from 2005 on – the site registered in WLCG (Kharkov-KIPT-LCG2)
from 2007 on – SLC cluster built on x86-64 CPU platform
2008 – CMS software (CMSSW & PhEDEx) deployment; site registration in CMS SiteDB (T2_UA_KIPT)
2009 (June) – successful commissioning of T2_UA_KIPT
2009 (July) – T2_UA_KIPT is in “Production” (“Ready” > 80% of time) !!!
Up to now, the KIPT computing facility remains to be
- the only active WLCG/EGEE site of Ukraine;
- the only CMS RDMS T2 site, which is non-”_RU_”
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 9
Conditions for site ‘Readiness’Conditions for site ‘Readiness’The CMS site readiness takes into account several sources of information to define whether a site is in a good status:
site availability according to the CMS Site Availability Monitor (SAM) tests16 CMS-specific critical tests of CE, SRM (SE) and CMS software in addition to “routine” WLCG SAM tests; daily SAM availability must be 80%
fraction of successful jobs submitted by the Job Robot automatic submission and management of “fake” CMS analysis jobs via WLCG (using CRAB) (typically, 6×100 jobs/day); daily JR efficiency must be 80%
number of operational links connecting the site with other sites number of commissioned links TO Tier-1 sites ≥ 2(to commission one such link, site has to sustain ≥ 5 MB/s of LoadTest during 24 hours)number of commissioned links FROM Tier-1 sites ≥ 4 (to commission one such link, site has to sustain ≥ 20 MB/s of LoadTest during 24 hours)
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 10
04 Sep 2009 12:41 http://dashb-ssb.cern.ch/dashboard/request.py/siteview?
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 11
Commissioning of first 6 T2_UA_KIPT Commissioning of first 6 T2_UA_KIPT uplinksuplinks
LoadTest from: CH_CERN ES_PIC DE_FZK IT_CNAF US_FNAL FR_CCIN2P3
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 12
Commissioning of first 6 T2_UA_KIPT Commissioning of first 6 T2_UA_KIPT uplinks (II)uplinks (II)
33.02 TB
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 13
Internet Connectivity Internet Connectivity Network backbone for academic institutes of UkraineNetwork backbone for academic institutes of Ukraine
Project has been completed in 2007 – 2008 (UARNET)
KIPT
10 Gbit/s 10 Gbit/s
L’vov Kiev Kharkov
Frankfurt
DE
Moscow
RU
Connectivity of NSC KIPT -- 2 Gbit/sConnectivity of Kharkov-KIPT-LCG2 (T2_UA_KIPT) -- up to 1 Gbit/s
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 14
T2_UA_KIPT
T2_RU_ITEP
T2_RU_SINP
T2_RU_JINR
http://lhcweb.pic.es/cms/SiteReadinessPlots
01 September 2009
T2 site q
uality ran
king
in A
ug
ust
2009
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 15
ITEP JINR SINP
KIPT IHEP RC KI
INR
Activity of CMS RDMS T2 sites in Activity of CMS RDMS T2 sites in August 09August 09
-- MC Production
-- Data Analysis
Thanks to Peter Kreuzer
Units: average number of CMS jobs at a given moment of the day
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 16
T2_UA_KIPT
pledge for 1st half of 2009
CMS Т2 site in 2010 (CMS Comp. ТDR)
SRM Storage (TB) 50 200
WN (kHEPSPEC06) 0.5 4
Connectivity (Gbit/s) 0.4 1.0
Resources: available vs. neededResources: available vs. needed
For 2-nd half of 2009: storage upgrade up to 120 TB
Major upgrade of CPU capacity is planned for 2010
No problem to have connectivity of 1 Gbit/s right now
11 September 09L.Levchuk, NEC'09, Varna, Bulgaria 17
SUMMARYSUMMARY
Preparation of KIPT WLCG infrastructure for CMS data analysis is in good progress
Site T2_UA_KIPT is commissioned (‘Ready’)
Site T2_UA_KIPT is stable (‘Ready’ >80% of time)
Upgrade of hardware to fit CMS Computing TDR requirements is still needed