DOE/NSF Review May 8-10, 2001
1
US CMS US CMS
US CMS M&O Planning
Dan Green
US CMS Project Manager
DOE/NSF Lehman Review, May 8-10, 2001
DOE/NSF Review May 8-10, 2001
2
OutlineOutline
• CMS and CERN Planning
• Resource Loaded Schedules– Category A
– Category B
• US CMS - Planning for the Physics - the research program.
DOE/NSF Review May 8-10, 2001
3
Planning for the PhysicsPlanning for the Physics• CMS has put “CPT” in place to prepare for the research
program. This is a coherent sum of Core SWC, Physics reconstruction and selection, and high level Triggers.
• CMS will present draft MOU for M&O and SWC at the next RRB meeting at CERN.
• Draft cost profiles for M&O, SWC have been shown to the JOG.
• Upgrade R&D and some fabrication is part of the research plan.
• US CMS has created a draft Management Plan for M&O.
• A resource loaded schedule (WBS) was made in preparation for the Lehman review. FNAL had a PMG review April 11-12, 2001.
• The Lehman review (May 8-10) has requested a complete plan for all costs of the US CMS Research Program - M&O + SWC + Upgrades.
DOE/NSF Review May 8-10, 2001
4
US CMS Research Program - M&OUS CMS Research Program - M&O
• Letter of August 17, 2000 from the Joint Oversight Group ( JOG - DOE/NSF) to Fermilab to manage Maintenance and Operations (M&O)– Detector operations and data monitoring
– M&O of US CMS supplied subsystems
– Virtual control room at FNAL for US based physics analysis
– R&D for and fabrication of upgrades to the experiment
• First cost estimate shown to JOG on December 6, 2000.
DOE/NSF Review May 8-10, 2001
5
CMS Steering CommitteeCMS Steering CommitteeChanges have been made in CMS to plan for SWC + Physics. CMS is planning for the research program.
M&O + SWC at the next RRB.
US physicists are an integral part of the CMS management.
US CMS
DOE/NSF Review May 8-10, 2001
6
““CPT”CPT”
CSC (SWC MOU)
Pimia
Stickland
TRIDAS(Detector MOU)
Cittolin
Smith
PRS
Sphicas
Common Technical Board
CSC IB TRIDAS IB
Tier 1 Tier 1 L. Bauerdick - US CMS SWC
DOE/NSF Review May 8-10, 2001
7
PRS and Detector L2PRS and Detector L2Tasks: detector simulation
detector reconstruction
detector calibration
monitoring
physics objects, reconstruction and selection
test beam analysis e - Seez J, - Eno
- Gasparini, Acosta
,b - Mannelli
ECAL - Faure
HCAL - Green
MU - Gasparini
Mitselmakher
Tracker - Rolandi
DOE/NSF Review May 8-10, 2001
8
CMS L2 and PRSCMS L2 and PRS
DOE/NSF Review May 8-10, 2001
9
Resource Loaded Schedule - M&OResource Loaded Schedule - M&O
• WBS 1-7 are subsystem costs for US built and/or managed systems (Category B)
• WBS 8 contains the Category A (common CMS costs), the Remote Control Room, and US CMS Outreach. Estimates for Cat. A are from CMS/CERN
• WBS 9 gives detail on the estimated costs for Project offices at FNAL and at CERN.
• WBS 10 is the management reserve.
DOE/NSF Review May 8-10, 2001
10
WBS Dictionary - NotesWBS Dictionary - Notes
US CMS has begun to create a resource loaded schedule for M&O. The “notes” field is used for the dictionary and BOE.
DOE/NSF Review May 8-10, 2001
11
Resource SheetResource Sheet
The base program resources must be called out explicitly, even if there is no explicit M&O cost. The US CMS research program cannot succeed without strong base support.
DOE/NSF Review May 8-10, 2001
12
WBS 8.1 - M&O CERN and CMSWBS 8.1 - M&O CERN and CMS
DOE+NSF/CERN MOU is umbrella document - with “general conditions applicable to experiments performed at CERN”. Now work on “MOU for Collaboration in the Exploitation of the CMS Detector” and an IMOU on SWC.
Category A error is large +- 400 k$/yr. US is pro rata ~ 20%. NO costs to US CMS before FY03.
CERN RRB Proposal
0
500
1000
1500
2000
2500
2001 2002 2003 2004 2005 2006
FY
AY
(M$
)
Outreach
Consultancy andmanagementServices
Shops and Labs
Test Beams andCalibrationData Management
On-line computing
Communications
Publications
Detector Services
DOE/NSF Review May 8-10, 2001
13
8.1 Category A Subtasks8.1 Category A Subtasks
WBS 8.1.1 is the largest item in WBS 8.1 Explicitly call out the subtasks of 8.1
DOE/NSF Review May 8-10, 2001
14
US CMS - at PresentUS CMS - at Present387 Members from 38 Institutions387 Members from 38 Institutions
US CMS - at PresentUS CMS - at Present387 Members from 38 Institutions387 Members from 38 Institutions
DOE/NSF Review May 8-10, 2001
15
How Will US CMS Grow?How Will US CMS Grow?
D0 History
0
100
200
300
400
500
600
700
1987
1989
1991
1993
1995
1997
1999
2001
CY
To
tal
Ph
ysic
ists
Grad Students
Physicists
D0 doubled in size around the time of first data taking. US CMS is imagined to do likewise. CDF and D0 are “natural” future additional collaborators. It is expected that US LHC physics will engage a substantial fraction of the US HEP experimental community by the time of first LHC data.
DOE/NSF Review May 8-10, 2001
16
8.2 Remote Control Room8.2 Remote Control Room
• Expect (D0 experience) US CMS will grow dramatically in FY05, 06 by a factor ~ 2. Taking US CMS ~ CDF now, we estimate 200 students and 120 university postdocs “in residence” if we adopt this model.
• If all are at CERN, just the incremental costs to the “base program” are ~ 5.4 M$/year. This would also be a major change in the sociology of U.S. HEP.
• An alternative is to create a “critical mass” stationed in the US using the Tier1 UF (SWC) and Tier2 (NSF) as the primary computing resources for US physics analysis. The Remote Control Room (M&O) would be used to debug equipment, teleconference hold physics analysis meetings and take data shifts.
• We will attempt to create a coherent national US CMS physics effort with a real identity. Upper Atmospheric Research Collaboratory (UARC) is a data point (NSF).
DOE/NSF Review May 8-10, 2001
17
US CMS - UF + RCRUS CMS - UF + RCR
Raw Data: 1000 Gbit/sRaw Data: 1000 Gbit/s
5 TeraIPS5 TeraIPSEvents:
10 Gbit/sEvents:
10 Gbit/s
10 TeraIPS10 TeraIPS
Controls: 1 Gbit/s
Controls: 1 Gbit/s
To regional centers 622 Mbit/s
To regional centers 622 Mbit/s
CMS networks, farms and data flows
Remote control rooms
Remote control rooms
Controls: 1 Gbit/sControls: 1 Gbit/s
US CMS
DOE/NSF Review May 8-10, 2001
18
US CMS Remote ControlUS CMS Remote Control
Reality check: US CMS is using the CDF/KEK remote control room for Run II as a starting point. However, we want to expand the scope to encompass a US based physics group and US LHC accelerator tasks.
DOE/NSF Review May 8-10, 2001
19
8.3 US CMS - Outreach8.3 US CMS - OutreachThe US CMS Construction PO now supports proposals for educational initiatives. The US CMS Education Coordinator is responsible for prioritizing the proposals.
The US CMS Project Office for the detector project then funds the agreed upon proposals. This effort would continue smoothly into the research phase of US CMS.
QuarkNet is a powerful example of the educational possibilities inherent in the LHC. We plan to continue to exploit the outreach opportunities of US CMS e.g. CMS web cam in SX5
DOE/NSF Review May 8-10, 2001
20
WBS 1-6, US CMS Category BWBS 1-6, US CMS Category B
• The US CMS L2 Managers were asked to estimate their M&O tasks for;
– M&S costs to maintain and operate systems provided by US CMS (Category B).
– Resources needed to maintain technical personnel at CERN.
– Additional costs associated with the major annual shutdown.
– NOTE: Post-doc and graduate student salaries and incremental costs (e.g. per diem) are assumed to be supported by the university base programs. Use annual SOW to define that commitment?
DOE/NSF Review May 8-10, 2001
21
WBS 1-6, US CMS - ScalingWBS 1-6, US CMS - Scaling
Reality Check:• For CDF Operations there are 2 FTE engineers
and 14 FTE technicians.• We plan for a similar economy of scale by
putting a “US CMS pool” together for general engineering and technician tasks.
• This team would have 4 FTE engineers and 20 FTE technicians.
• This team in residence costs ~ 1.6 M$/yr.
DOE/NSF Review May 8-10, 2001
22
WBS 7 - US CMS POWBS 7 - US CMS PO
Reality Check:• US CMS has experience from the construction project
and we plan a smooth “handoff”.
• There will need to be both a CERN and a FNAL/NEU “branch” of the PO.
• There are ~ 6-8 FTE in the PPD/CDF Ops Dept. We expect ~ 4 FTE at CERN, ~ 4 FTE at Fermilab.
• The M&O PO costs ~ 1 M$/year and scales well with the current PO.
DOE/NSF Review May 8-10, 2001
23
US CMS - L2 AccountsUS CMS - L2 Accounts
• US CMS - CERN Branch Project Office (opened on May 1, 2000)
• US CMS L2 CERN Team Accounts
– Funds sent to FNAL
– Allocations by L2 managers
– Transfer to Team Accounts at CERN
– Authorization of expenses by L2 managers
– Invoices sent to FNAL for financial tracking
– PO, EMU, HCAL, and SiTrkr have started L2 accounts
• Serves as a model for M&O (Cat A+B) funding and tracking. FNAL has accepted a “pass through” overhead rate, treating CERN as a collaborating institution. This decision allowed US CMS to plan for the installation and commissioning phase of the detector.
DOE/NSF Review May 8-10, 2001
24
US CMS - FY01 RequestsUS CMS - FY01 RequestsUS CMS - FY01 University Request
Texas Tech
UC Davis
Illinois Chicago
Rice
UC San Diego (DOE)
UC Davis
Florida
Caltech
The P.O. already has input to DOE/NSF in regards to a project perspective on base support increments for postdocs. This recommendation refers ONLY to project related activities and not research activities covered by the peer review process. Adequate base support wil, however, be critical to the success of the research program.
DOE/NSF Review May 8-10, 2001
25
Management ReserveManagement Reserve• Much of the research program costs will be salaries.
• A full contingency analysis at the lowest task level seems inappropriate.
• A management reserve of 25 % is proposed to be held in the Project Office.
• This level is similar to the R&D reserve held at the start of the detector project. This mechanism worked well in that instance.
• Problems will arise that must be solved, e.g. currency fluctuations. Other possibilities would be an accident at the detector or more frequent than expected repairs in this hostile accelerator environment. We must plan to cover these contingencies.
DOE/NSF Review May 8-10, 2001
26
WBS - Cost and ManpowerWBS - Cost and Manpower
Reality Check: The T+E level is close to the scaling estimate. These cost estimates do not contain salaries of research personnel - postdocs and graduate students. However, the tasks are called out and there are ~ 48 “FTE” PD working on M&O. Note also Cat B does not start until ~ FY05
US CMS M&O Cost Estimate
$0
$1,000,000
$2,000,000
$3,000,000
$4,000,000
$5,000,000
$6,000,000
$7,000,000
$8,000,000
$9,000,000
$10,000,000
FY03 FY04 FY05 FY06 FY07 FY08
AY
$
Mgt Reserve
OpsOffice
Common
SiTrk
Fpix
ECAL
DAQ
Trigger
HCAL
EMU
US CMS M&O Resource Usage
0
10
20
30
40
50
60
70
80
FY
03
FY
04
FY
05
FY
06
FY
07
FY
08
Fiscal Year
FT
Es
Post-docs
Engineers
Technicians
DOE/NSF Review May 8-10, 2001
27
US CMS Research Program - SWCUS CMS Research Program - SWC
11/98: DOE and NSF Joint Review of ATLAS and CMS Computing/Software
5/99: DOE and NSF Review of US ATLAS and US CMS Computing and Software
1/00: DOE/NSF Review of US LHC Software and Computing Projects
10/00: Hoffmann Review Report (CERN SWC Review)
11/00: US LHC Software and Computing ReviewThere have been several US reviews of SoftWare and Computing needs over more than 2 years. A large investment in SWC is needed to realize the investment in the construction of the CMS detector. Note SUSY can come in the first month at 10% luminosity.
DOE/NSF Review May 8-10, 2001
28
US CMS Research ProgramUS CMS Research Program
• Upgrades are estimated - both R&D and fabrication. They should be proposal driven and reviewed in the context of the full US HEP program. Note that SWC also has upgrades as a part of the plan (Moore’s Law).
• Contingency is explicitly held in an annual management reserve. The proposals for allocation of reserve should be reviewed as they are in the construction project.
• Postdocs and students are estimated from CDF and D0 scaling. They are identified as necessary to the success of US CMS research, but their support is assumed to reside in the “base program”.
DOE/NSF Review May 8-10, 2001
29
The Cost of US CMS ResearchThe Cost of US CMS Research
The components of the research program are SWC, M&O, Upgrades and scaled estimates for the dislocation costs associated with postdocs and students - base program cost.
US CMS - Construction and Research
0
5
10
15
20
25
30
35
40
19
96
19
98
20
00
20
02
20
04
20
06
20
08
FY
AY
(M$
)
ResearchPersonnel
Upgrades
M&O CostEstimate
SWC CostEstimate
ConstructionProject Funding
DOE/NSF Review May 8-10, 2001
30
US CMS - SummaryUS CMS - Summary
• The physics available at the LHC is compelling.• The transition from US CMS construction to
research is being planned.• There are now resource loaded schedules in
place for all the components the US CMS research program.
• Our aim is to enable the US physicists to be a coherent group making a major impact on physics analysis within CMS.
DOE/NSF Review May 8-10, 2001
31
Detector Upgrades - IDetector Upgrades - I
• US CMS Scope restoration (8.6 M$)– ME4/2 mechanics– ME4/2 electronics– ME4/1 electronics– Installation and Commissioning the above– Fpix third layer
DOE/NSF Review May 8-10, 2001
32
Detector Upgrades - IIDetector Upgrades - II
• CMS Scope Restoration - US Management (4.0 M$) of EMU, HCAL– ME1/1 electronics and cables– HF shielding– HF tooling and supports
• Repairs (5.5 M$)– HE |y|=3 scintillator repair– Fpix replacement for radiation damage
DOE/NSF Review May 8-10, 2001
33
Detector Upgrades - IIIDetector Upgrades - III
• R&D EDIA - ( 0.2 M$/year)
• “TRIDAS” Upgrades ( ~1.0 M$/year)– increase in C.M. energy ~ transparent– increase in luminosity --> “smarter” L1
trigger and faster L2, L3 event building– DAQ improvements with commodity
purchases– r.f. changes in bunch crossing time ?
DOE/NSF Review May 8-10, 2001
34
JOG and FermilabJOG and Fermilab
• Letter of August 17, 2000 from JOG to Fermilab to manage M&O– Detector operations and data monitoring (Category A, WBS 8)
– M&O of US CMS supplied subsystems (Category B)
– Virtual control room at FNAL for US based physics analysis (WBS 8)
– R&D for and fabrication of upgrades to the experiment (WBS 8)
• First cost estimate shown to JOG on December 6, 2000.
DOE/NSF Review May 8-10, 2001
35
M&O PlanningM&O Planning
• CMS has put “CPT” in place to prepare for Physics.
• Draft cost profiles for M&O, SWC have been shown to the JOG, most recently Dec. 6, 2000.
• Upgrade R&D and some fabrication is part of the plan.
• We DO NOT assume that the Project can use M&O funding to complete installation.
• Planning and resource loaded schedules are being created for SWC and M&O which will both be needed to realize the investment in the CMS detector. These efforts need to be globally optimized.
• US CMS has made a draft OMP for M&O.
• The Lehman review has requested a complete plan for all costs of the US CMS Research Program - M&O + SWC.
DOE/NSF Review May 8-10, 2001
36
D0 - Demographic GrowthD0 - Demographic Growth
CDF has ~ 406 physicists with 154 students. D0 has ~ 493 physicists with 108 students.
The are now ~ 160 students and university postdocs in residence at D0. There will be an increase when Run IIa is in full operation.
D0 doubled in size just before data taking. US CMS is imagined to do likewise.
D0 History
0
100
200
300
400
500
600
700
1987
1989
1991
1993
1995
1997
1999
2001
CY
To
tal
Ph
ysic
ists
Grad Students
Physicists