Executive Session Director’s CD-2/3a Review of the MicroBooNE Project July 12-14, 2011 Ed Temple
Feb 24, 2016
Executive Session Director’s CD-2/3a Review of the MicroBooNE Project
July 12-14, 2011Ed Temple
Director's CD-2/3a Review of the MicroBooNE Project
212-Jul-2011
Agenda for Exec Session• Charge to Review Team• Review Agenda• Reviewer Writing Assignments• DOE O 413.3 Critical Decision Tables• Document Requirements• Breakout Review Assignments and Guidance• Project Cost – Management Reserve/Contingency
Table• Cost/Schedule Review Guidance• Reporting Structure• Discussion
3
Charge
12-Jul-2011 Director's CD-2/3a Review of the MicroBooNE Project
This charge is for the Committee to conduct a Director’s CD-2/3a Review of the Large Liquid Argon Detector for Neutrino Physics (MicroBooNE) Project. MicroBooNE is a liquid Argon Time Projection Chamber (LArTPC) that will be exposed to the Booster neutrino beam and the NuMI neutrino beam. The experiment will address the low energy excess observed by the MiniBooNE experiment, measure low energy neutrino cross sections, and serve as the necessary next step in a phased program towards massive Liquid Argon TPC detectors. MicroBooNE received CD-1 approval “Approve Alternative Selection and Cost Range” from DOE on July 9, 2010. This review is to ensure that all the requirements will be met DOE’s requirements for CD-2/3a. CD-2 is approval of the Performance Baseline. The Performance Baseline is developed based on a design document (Preliminary Design or a Technical Design Report), a well-defined and documented scope, a resource-loaded detailed schedule, a definitive cost estimate, defined Key Performance Parameters and other project management documents. CD-3a is approval of long lead procurements. Approval of CD-2/3a authorizes submission of a budget request for the Total Project Cost (TPC) and detailed engineering design and procurement of the identified long lead items.
Director's CD-2/3a Review of the MicroBooNE Project
4
Charge (Continued)
12-Jul-2011
The technical part of the review is to assess the preliminary design as documented in MicroBooNE’s Technical Design Report (TDR). This should include the base design and any current alternatives. The cost and schedule baselines are based on a detailed WBS – Work Breakdown Structure, WBS Dictionary, BOE – Basis of Estimate documentation, risk and contingency analyses, RLS – Resource Loaded Schedule, and time phased funding and cost profiles. The committee is asked to review and comment on the formal project management documentation provided in support of CD-2/3a. The committee is also asked to review and comment on the project’s plan to get to CD-3b and that the organization is in place to achieve it. The committee is to assess the project’s scope of work presented during this review and compare it to the project’s scientific objectives “Mission Need” to ensure it can be met. Then evaluate if the estimated cost presented, including management reserve/contingency, is reasonable for the project’s scope of work and without undue risk. The scope of work that is to be assessed includes the base design and current alternatives.
Director's CD-2/3a Review of the MicroBooNE Project
5
Charge (Continued)
12-Jul-2011
Additionally, the review team is to assess the projects monthly process for statusing progress of cost/schedule against plan and reporting to DOE and Lab Management. The review committee is asked to address the following questions to assess the Project’s progress:
1. Is the Project’s preliminary design appropriately developed and well documented in their Technical Design Report (TDR)? Does the design satisfy the project’s performance requirements to carry out the scientific mission?
2. Has a list of long lead procurement items been identified and are those items at a state of final design/specifications such that the Procurement Department can go out for a Request for Proposal (RFP) or a Request for Quote (RFQ) meeting the requirements for CD-3a?
3. Has the Project developed a quality fully resource loaded schedule that includes the entire project’s scope of work and is achievable?
4. Has the Project thoroughly documented the Basis of Estimate (BOEs) for the scope of the project that supports the baseline cost and schedule presented? Are the estimated effort and M&S costs reasonable?
5. Has the Project implemented Risk Management by identifying risks, performing a risk assessment (qualitative and quantitative) and developing mitigation plans?
Director's CD-2/3a Review of the MicroBooNE Project
6
Charge (Continued)
12-Jul-2011
FINAL 05-Jul-2011
1. Has the Project updated or created the required project management documents per DOE Order 413.3B for CD-2/3a?
2. Are the Project organization and staffing levels, for both current and planned, adequate to manage the work to get to CD-3b?
3. Are ES&H issues being properly addressed given the project’s current stage of development?
4. Does the project have the processes in place to meet CD-2 reporting requirements and have they demonstrated their ability to do so.
5. Does it appear that the MicroBooNE Project will be ready for a DOE CD-2/3a review in August?
Additionally the committee is to review and comment on the Project’s response and actions taken on the recommendations from the prior Director’s and DOE Reviews. Finally, the committee should present answers to the above questions and present findings, comments, and recommendations at a closeout meeting with MicroBooNE’s and Fermilab’s management. A written report will be provided soon after the review.
Director's CD-2/3a Review of the MicroBooNE Project
7
Typical CD-2/3a Questions
12-Jul-2011
Technical Are the requirements that form the basis for the design and engineering phase of the project clearly
documented? Does the preliminary design satisfy the performance requirements? Has the preliminary design been
document in a Technical Design Report (TDR) been developed that includes a clear and concise description of the alternatives analyzed, the basis for the alternative selected, how the alternative meets the approved mission need?
Has the Project employed value engineering/management as early as possible in the project development and design process so recommendations can be included in the planning and implemented without delaying the progress of the project or causing significant rework of completed designs?
Has the Project identified specific standards to which components, subsystems and systems will be procured, fabricated, constructed, inspected, and tested?
Does the baseline design meet the project’s objectives (mission need) and can it be built? Cost
Does the Technical Design Report and supporting documentation adequately justify the stated baseline cost?
Do the cost estimates for each WBS (or cost) element have a sound documented Basis of Estimate (BOEs)?
Has the amount of Management Reserve (MR) for known risk and Contingency for unknown risks been developed with a sound bases and adequate to successfully complete the project?
Does an obligation profile exist and is it within funding guidance?
Director's CD-2/3a Review of the MicroBooNE Project
8
Typical CD-2/3a Questions (continued)
12-Jul-2011
Schedule Does the Project’s Work Breakdown Structure (WBS) define the total scope of the project as a product-
oriented family tree composed of hardware, software, services, data, facilities and other components? Is there a well documented WBS Dictionary?
Is the schedule well developed and appropriately structured by specifying relationships, predecessors, successors, critical path, resource loaded, etc?
Are the durations for the activities and overall schedule reasonable and achievable with the assumed resources?
Does the schedule contain appropriate levels of milestones, sufficient quantity of milestones for tracking progress and do they appear to be achievable?
Does the schedule include activities for design reviews, which include assessment of the designs readiness for procuring prototypes and preproduction materials?
Has the schedule been developed to support Earned Value Management statusing and reporting to meet the requirements of the FRA Earned Value Management System?
Director's CD-2/3a Review of the MicroBooNE Project
9
Typical CD-2/3a Questions (continued)
12-Jul-2011
20-Mar-2011
Management Is there an appropriate management organization in place with the responsibilities defined and
documented for the scope of work? Is there adequate resources assigned to the project to complete the work to be ready for CD-2 and to be
ready for CD-3? Are ES&H aspects being properly addressed and are future plans sufficient given the projects current
stage of development? Is the documentation required by DOE O 413.3 in order and at the required for CD-2 Approval? Is there a funding plan available or proposed to meet the resource requirements to realize the project? Has Risk Management been performed which includes risks assessments on each potential design
alternative as a factor in selecting which alternative is to be pursued? Procurement
Have the critical procurements been identified and are they included in the schedule with adequate lead time built in?
Have critical make vs. buy decisions been evaluated in conjunction with the scope and is that reflected in the baseline cost estimate, schedule and technical risk plan?
Are the Project designs final and procurement packages prepared to the degree appropriate to order materials and initiate construction as scheduled?
Director's CD-2/3a Review of the MicroBooNE Project
10
Agenda
12-Jul-2011
Tuesday, July 12, 2011 EXECUTIVE SESSION – Comitium (WH2SE) 8:00 – 8:45 AM 45 Executive Session OVERVIEW PLENARY SESSION – One West (WH1W) 8:45 – 8:50 AM 05 Welcome Pier Oddone 8:50 – 9:20 AM 30 Project Overview Regina Rameika 9:20 – 9:40 AM 20 Plan for Operations Bonnie Fleming 9:40 – 10:10 AM 30 Detector Systems Overview Craig Thorn 10:10 – 10:25 AM 15 BREAK (Reviewers – outside Comitium) 10:25 – 10:55 AM 30 Infrastructure Overview Cat James 10:55 – 11:05 AM 10 Intro to BOEs Regina Rameika FIRST PARALLEL BREAKOUT SESSIONS 11:05 – 12:35 PM 90
Session 1: Cryostat and Cryogenics – EAG (WH7W) Cat James Session 2: Active Detectors (TPC & PMTs) – Dark Side (WH6W) Bonnie Fleming Session 3: Electronics, Readout DAQ – Snake Pit (WH2NE) Hucheng Chen Session 4: Management – Quarium (WH8SW) Regina Rameika
Director's CD-2/3a Review of the MicroBooNE Project
11
Agenda continued
12-Jul-2011
12:35 – 1:30 PM 55 LUNCH FIRST PARALLEL BREAKOUT SESSIONS - continued 1:30 – 3:00 PM 90
Session 1: Cryostat and Cryogenics – EAG (WH7W) Cat James Session 2: Active Detectors (TPC & PMTs) – Dark Side (WH6W) Bonnie Fleming Session 3: Electronics, Readout DAQ – Snake Pit (WH2NE) Hucheng Chen Session 4: Management – Quarium (WH8SW) Regina Rameika 3:00 – 3:20 PM 20 BREAK (Reviewers – outside Comitium) SECOND PARALLEL BREAKOUT SESSIONS 3:20 – 5:00 PM 100
Session 1: Detector Assembly – Snake Pit (WH2NE) Craig Thorn Session 2: Installation in Building – Dark Side (WH6W) Cat James Session 3: Management – Quarium (WH8SW) Regina Rameika 5:00 – 6:00 PM 60 Executive Session - Comitium (WH2SE)
Director's CD-2/3a Review of the MicroBooNE Project
12
Agenda continued
12-Jul-2011
Wednesday, July 13, 2011 THIRD PARALLEL BREAKOUT SESSIONS 8:00 – 9:30 AM 90
Session 1: Cryostat and Cryogenics – Dark Side (WH6W) Cat James Session 2: Active Detectors and Electronics/RO/DAQ – West Wing (WH10NW) Bonnie Fleming Hucheng Chen
9:30 – 9:45 AM 15 BREAK (Reviewers – One East) PLENARY SESSION – One East (WH1E) 9:45 – 11:00 AM 75 Answers to Day 1 Questions 11:00 – 12:00 PM 60 Executive Session - One East (WH1E) 12:00 – 1:00 PM 60 LUNCH 1:00 – 2:30 PM 2:40 Executive Session continued and Report Writing - One East (WH1E) 2:30 – 2:45 AM 15 BREAK (Reviewers – One East)
Director's CD-2/3a Review of the MicroBooNE Project
13
Agenda continued
12-Jul-2011
2:45 – 5:00 PM 2:40 Executive Session continued and Report Writing - One East (WH1E) If the committee members need to have additional follow-up discussion with the project, the following rooms are available for those breakout meetings: (5 Rooms Available from 1:00 – 6:00 PM)
West Wing Video Conference Room (WH10NW) Sun Rise Video conference Room (WH11NE) Theory (WH3NW) Quarium (WH8SW) LBNE Room (WH12SW) Thursday, July 14, 2011 8:30 AM Full Committee Executive Session Dry Run – Comitium (WH2SE) 10:30 AM Closeout Presentation – Curia II (WH2SW) 11:30 AM Adjourn
Director's CD-2/3a Review of the MicroBooNE Project
14
Reviewer Writing Assignments
12-Jul-2011
*Underlined names are the primary writer.
Executive Summary Ed Temple 1.0 Introduction 2.0 Cryostat/Cryogenics Mike Sarychev
Alex Martinez Barry Norris
3.0 Active Detector (TPC & PMTs) Bob Wagner Mike Crisler
4.0 Electronics/Readout/DAQ Peter Shanahan Gustavo Cancelo
5.0 Detector Assembly Barry Norris/ Gustavo Cancelo Bob Wagner Mike Sarychev
6.0 Installation in Building Joe Howell Mike Crisler Alex Martinez Peter Shanahan
Director's CD-2/3a Review of the MicroBooNE Project
15
Reviewer Writing Assignments (Continued)
12-Jul-2011
7.0 Project Management 7.1 Cost Wyatt Merritt/Nancy
Grossman All
7.2 Schedule Sherese Humphrey All
7.3 Management Nancy Grossman Wyatt Merrit
7.4 ES&H Mike Andrews 8.0 Charge Questions
8.1 Is the Project’s preliminary design appropriately developed and well documented in their Technical Design Report (TDR)? Does the design satisfy the project’s performance requirements to carry out the scientific mission?
Peter Shanahan All
8.2 Has a list of long lead procurement items been identified and are those items at a state of final design/specifications such that the Procurement Department can go out for a Request for Proposal (RFP) or a Request for Quote (RFQ) meeting the requirements for CD-3a?
Bob Wagner All
*Underlined names are the primary writer.
Director's CD-2/3a Review of the MicroBooNE Project
16
Reviewer Writing Assignments (Continued)
12-Jul-2011
8.3 Has the Project developed a quality fully resource loaded schedule that includes the entire project’s scope of work and is achievable?
Sherese Humphrey All
8.4 Has the Project thoroughly documented the Basis of Estimate (BOEs) for the scope of the project that supports the baseline cost and schedule presented? Are the estimated effort and M&S costs reasonable?
Sherese Humphrey All
8.5 Has the Project implemented Risk Management by identifying risks, performing a risk assessment (qualitative and quantitative) and developing mitigation plans?
Nancy Grossman All
8.6 Has the Project updated or created the required project management documents per DOE Order 413.3B for CD-2/3a?
Nancy Grossman All
8.7 Are the Project organization and staffing levels, for both current and planned, adequate to manage the work to get to CD-3b?
Wyatt Merritt All
8.8 Are ES&H issues being properly addressed given the project’s current stage of development?
Mike Andrews All
*Underlined names are the primary writer.
Director's CD-2/3a Review of the MicroBooNE Project
17
Reviewer Writing Assignments (Continued)
12-Jul-2011
8.9 Does the project have the processes in place to meet CD-2 reporting requirements and have they demonstrated their ability to do so?
Sherese Humphrey All
8.10 Does it appear that the MicroBooNE Project will be ready for a DOE CD-2/3a review in August?
Ed Temple All
*Underlined names are the primary writer.
Director's CD-2/3a Review of the MicroBooNE Project
18
Summary of Major Requirements Table
12-Jul-2011
Delegation AllowedReviewed by SC-28 (OPA)
Approved by SC-1Reviewed by SC-28 Approved by SC-1
Reviewed by SC-28 Approved by SC-1
Reviewed by SC-28 Approved by SC-1
Reviewed by SC-28 Approved by SC-1
Reviewed by SC-28 Approved by SC-AD
Reviewed by SC-28 Approved by SC-1
Reviewed by SC-28 Approved by SC-1
Reviewed by SC-28 Approved by SC-2
Reviewed by SC-28 Approved by SC-AD
Reviewed by SC-28 Approved by SC-AD
Reviewed by SC-28 Approved by SC-AD
CD-0--APPROVE MISSION NEED S-2 (CD-1 to 4 delegated, see
below)
SC-1 SC-1 SC-AD SC-AD SC-AD
CD-1--APPROVE ALTERNATIVE SELECTION AND COST RANGE S-4 SC-1 SC-1 SC-AD SC-AD SC-AD
CD-2 --APPROVE PERFORMANCE BASELINE S-4 SC-1 SC-2 SC-AD SC-AD SC-AD
CD-3--APPROVE START OF CONSTRUCTION SC-1 SC-1 SC-2 SC-AD SC-AD SC-AD
CD-4--APPROVE START OF OPERATIONS OR PROJECT COMPLETION SC-1 SC-1 SC-2 SC-AD SC-AD SC-AD
Deviations
New Performance Baseline Approval S-4 SC-1 SC-2 SC-2 SC-2 SC-2
Directed Change
Program SC-AD SC-AD SC-AD SC AD SC AD SC AD
Project Prog. Mgr., SOM or FPD Prog. Mgr., SOM or FPD Prog. Mgr., SOM or FPD Prog. Mgr., SOM or FPD Prog. Mgr., SOM or FPD Prog. Mgr., SOM or FPD
Contractor Contractor Contractor Contractor Contractor Contractor Contractor
Mission Validation Independent Review SC-AD SC-AD SC-AD SC-AD SC-AD SC-AD
Design Review Prior to CD-1, CD-2, and CD-3 Team External to Project Team External to Project Team External to Project Team External to Project Team External to Project Team External to Project
Conduct Independent Project Review or External Independent Review prior to CD-0 to 3
ICE or ICR by OECM with OPA
ICE or ICR by OECM with OPA
ICE or ICR by OECM with OPA
Prior to CD-1 to CD-4 by SC-28
Prior to CD-2 & CD-3 Tailored by SC-28
Prior to CD-2 & CD-3 Tailored by SC-28
SC-AD Request Annual Peer Reviews by PMSO Post CD-2 SC-28 SC-28 SC-28 SC-28 SC-28
TailoredSC-28
Tailored
Performance Baseline Deviation Reviews after CD-2 SC-28 SC-28 SC-28 SC-28 SC-28 SC-28
EVMS Review--Certification Prior to CD-3 and Bi-annual Surveillance (annual by contractor) SC-28 SC-28 SC-28 SC-28 SC-28 N/A
ORR/RA--Operational Readiness Review/Readiness Assessment Prior to CD-4 Team External to Project Team External to Project Team External to Project Team External to Project Team External to Project Team External to Project
Technology Readiness Assessment (TRA) prior to CD-2 and 3 by SAE or AE N/A N/A N/A N/A N/A N/A
Project Definition Rating Index (PDRI) by OECM N/A N/A N/A N/A N/A N/A
Hazard Cat. 1,2,3 Nuclear Facility--Technical IPR Prior to CD-2 PSO PSO PSO PSO PSO PSO
Monthly PARS II Reporting (EVMS for Projects>$20 M) Monthly Project Status After CD-0
Monthly or Quarterly Project Reporting (QPR) or Meeting after CD-0
SC-AD Invite SC-1 and SC-28
SC-AD Invite SC-1 and SC-28
SC-AD Invite SC-2 and SC-28
SC-AD to invite SC-28 SC-AD to invite SC-28 SC-AD to invite SC-28
TOTAL PROJECT COST (TPC) $750M or more Less than $750M to $400M Less than $400M to $100M Less than $100M to $50M* Less than $50M* to $20M Less than $20M to $10M**
DECISION / REQUIREMENTS1 / APPROVAL2
Prior to CD0, Approve Mission Need Statement
Prior to CD-1, Approve Acquisition Strategy
CR
ITIC
AL
DEC
ISIO
NS
BA
SELI
NE
MA
NA
GEM
ENT
If performance, scope, schedule, or cost baseline at CD-2 cannot be met, then SC-1 and SC-2 must be notified & a determination made to terminate the project or establish a new performance baseline.
Project changes caused by Policy Directives that have the force and affect of law and regulation, or Regulatory, or Statutory action and are initiated by entities external to the Department.
REV
IEW
SR
EPO
RT
ING
Project Status After CD-0 and EV After CD-2
Rout
ine
Proj
ect
Chan
ges/
Co
ntro
l
No T
PC
Chan
geCh
ange
s to
TP
C
Director's CD-2/3a Review of the MicroBooNE Project
19
CD-2 Requirements
12-Jul-2011
Delegation Allowed
S-4 SC-1 SC-2 SC-AD SC-AD SC-AD
Approve updated Acquisition Strategy if changes are majorS-2
(CD-1 to 4 delegated, see below)
SC-1 with SC-28 concurrence
SC-1 with SC-28 concurrence
SC-1 with SC-28 concurrence
SC-AD with SC-28 concurrence
SC-AD with SC-28 concurrence
Establish a Performance Baseline (PB) FPD FPD FPD FPD FPD FPD
Approve updated PEP S-4 SC-1 SC-2 SC-AD SC-AD SC-AD
Prepare a Baseline Fund. Profile & reflect in budget docs. & PEP. Consider full funding if TPC < $50M S-4 SC-1 SC-2 SC-AD SC-AD SC-AD
Approval of Long-Lead Procurement S-4 SC-1 SC-2 SC-AD SC-AD SC-AD
Develop Project Management Plan, if applicable N/A N/A N/A N/A N/A N/A
Complete Preliminary Design Project Project Project Project Project Project
Incorporate High Perf. & Sustainable Bldg. & Sustainable Environmental Stewardship Project Project Project Project Project Project
Conduct a Preliminary Design Review Team external to project Team external to project Team external to project Team external to project Team external to project Team external to project
Complete Preliminary Design Report Project Project Project Project Project Project
Perform Baseline Validation ReviewICE or ICR by OECM
with OPAICE or ICR by OECM
with OPAICE or ICR by OECM
with OPA SC-28 SC-28 SC-28
Conduct a Project Definition Rating Index analysis as part of an EIR N/A N/A N/A N/A N/A N/A
Conduct a Technical Readiness Assessment & develop a Technical Maturation Plan N/A N/A N/A N/A N/A N/A
Employ an EVMS compliant with ANSI/EIA-748A, or as defined in the contract Contractor Contractor Contractor Contractor Contractor N/A
Prepare a Hazard Analysis ReportField Organization (Site
Office) or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab
Continue with Quality Assurance Program Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab
Conduct Preliminary Security Vulnerability Assessment, if necessary Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab
Issue Final NEPA determination (i.e., FONSI) SC-1 or Site Office SC-1 or Site Office SC-1 or Site Office SC-1 or Site Office SC-1 or Site Office SC-1 or Site Office
Update budget documents and Exhibit 300 if applicable SC-AD SC-AD SC-AD SC-AD SC-AD SC-AD
Hazard Cat. 1,2,3 Nuclear Facility--Update Safety Design Strategy (SDS)
SBAA & FPD, w/CNS or CDNS concurrence, as
appropriate
SBAA & FPD, w/CNS or CDNS concurrence, as
appropriate
SBAA & FPD, w/CNS or CDNS concurrence, as
appropriate
SBAA & FPD, w/CNS or CDNS concurrence, as
appropriate
SBAA & FPD, w/CNS or CDNS concurrence, as
appropriate
SBAA & FPD, w/CNS or CDNS concurrence, as
appropriateHazard Cat. 1,2,3 Nuclear Facility--Prepare a Preliminary Safety Design Report updating the CSDR SBAA via the PSVR SBAA via the PSVR SBAA via the PSVR SBAA via the PSVR SBAA via the PSVR SBAA via the PSVR
Hazard Cat. 1,2,3 Nuclear Facility--Prepare a Preliminary Safety Validation Report (PSVR) SBAA SBAA SBAA SBAA SBAA SBAA
Hazard Cat. 1,2,3 Nuclear Facility--Conduct a Technical Independent Project Review PSO PSO PSO PSO PSO PSO
Hazard Cat. 1,2,3 Nuclear Facility--Place Code of Record under Configuration Control Project Project Project Project Project Project
Submit approved CD or equivalent documents to OECM. If applicable, any PB BCP to OECM SC-28 SC-28 SC-28 SC-28 SC-28 SC-28
Submit budget request for the remainder of TPC SC-AD SC-AD SC-AD SC-AD SC-AD SC-AD
Update PARS II with monthly statusProg. Mgr., FPD, and
Contractor Prog. Mgr., FPD, and
Contractor Prog. Mgr., FPD, and
Contractor Prog. Mgr., FPD, and
Contractor Prog. Mgr., FPD, and
Contractor Prog. Mgr. & FPD
No Earned Value (EV)
Continue with Monthly or Quarterly Project Reporting/Meeting
SC-AD Invite SC-1 and SC-28
SC-AD Invite SC-1 and SC-28
SC-AD Invite SC-2 and SC-28
SC-AD to invite SC-28 SC-AD to invite SC-28 SC-AD to invite SC-28
SC-AD Request Annual Project Peer Review by PMSO SC-28 SC-28 SC-28 SC-28SC-28
TailoredSC-28
Tailored
TOTAL PROJECT COST (TPC) $750M or more Less than $750M to $400M Less than $400M to $100M Less than $100M to $50M* Less than $50M* to $20M Less than $20M to $10M**
DECISION / REQUIREMENTS1 / APPROVAL2
CD-2--APPROVE PERFORMANCE BASELINE
PRIO
R T
O C
D-2
--PR
ELIM
INA
RY
DES
IGN
POST
CD
-2
Nucl
ear
Faci
lity
Director's CD-2/3a Review of the MicroBooNE Project
20
CD-3 Requirements
12-Jul-2011
Delegation Allowed
SC-1 SC-1 SC-2 SC-AD SC-AD SC-AD
Approve updated CD-2 Project Documentation (PEP, AS, PDS, etc) if major changes
Reviewed by SC-28 Approved by SC-1
Reviewed by SC-28 Approved by SC-1
Reviewed by SC-28 Approved by SC-2
Reviewed by SC-28 Approved by SC-AD
Reviewed by SC-28 Approved by SC-AD
Reviewed by SC-28 Approved by SC-AD
Complete Final Design S-2
(CD-1 to 4 delegated, see below)
Project Project Project Project Project
Incorporate High Performance & Sustainable Bldg. & Sustainable Env. Stewardship
Project Project Project Project Project Project
Conduct a Final Design Review Team external to project Team external to project Team external to project Team external to project Team external to project Team external to project
Complete Final Design Report Project Project Project Project Project Project
Employ a certified EVMS compliant with ANSI/EIA-748A, or as defined in the contract
Certified by SC-28 Certified by SC-28 Certified by SC-28 Certified by SC-28 Contractor N/A
Execution Readiness Review ICE by OECM if warranted or IPR by OPA
ICE by OECM if warranted or IPR by OPA
ICE by OECM if warranted or IPR by OPA
SC-28 SC-28 SC-28
Conduct a Technology Readiness Assessment, where significant CTE modification occurs
N/A N/A N/A N/A N/A N/A
Update the Hazard Analysis Report Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab
Prepare Construction Project Safety and Health Plan Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab
Update the Quality Assurance Program Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab
Finalize the Security Vulnerability Assessment Report, if necessary
Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab Site Office or Lab
Hazard Cat. 1,2,3 Nuclear Facility--Update Safety Design Strategy (SDS)
SBAA & FPD, w/ CNS or CDNS concurrence, as
appropriate
SBAA & FPD, w/ CNS or CDNS concurrence, as
appropriate
SBAA & FPD, w/ CNS or CDNS concurrence, as
appropriate
SBAA & FPD, w/ CNS or CDNS concurrence, as
appropriate
SBAA & FPD, w/ CNS or CDNS concurrence, as
appropriate
SBAA & FPD, w/ CNS or CDNS concurrence, as
appropriateHazard Cat. 1,2,3 Nuclear Facility--Prepare a Preliminary Documented Safety Analysis 4 that updates the PSDR
SBA Authority via the SER SBA Authority via the SER SBA Authority via the SER SBA Authority via the SER SBA Authority via the SER SBA Authority via the SER
Hazard Cat. 1,2,3 Nuclear Facility--Prepare a Safety Evaluation Report (SER) SBAA w/ FPD concurrence SBAA w/ FPD concurrence SBAA w/ FPD concurrence SBAA w/ FPD concurrence SBAA w/ FPD concurrence SBAA w/ FPD concurrence
Hazard Cat. 1,2,3 Nuclear Facility--Revise the Code of Record Project Project Project Project Project Project
Submit approved CD or equivalent documents to OECM. If applicable, any PB BCP to OECM
SC-28 SC-28 SC-28 SC-28 SC-28 SC-28
Allow expenditure of TPC funds. Update budget document and OMB 300s if applicable.
SC-AD SC-AD SC-AD SC-AD SC-AD SC-AD
Update PARS II with monthly status Prog. Mgr., FPD, and Contractor
Prog. Mgr., FPD, and Contractor
Prog. Mgr., FPD, and Contractor
Prog. Mgr., FPD, and Contractor
Prog. Mgr., FPD, and Contractor
Prog. Mgr. & FPD No Earned Value (EV)
Continue with Monthly or Quarterly Project Reporting/Meeting
SC-AD Invite SC-1 and SC-28
SC-AD Invite SC-1 and SC-28
SC-AD Invite SC-2 and SC-28
SC-AD to invite SC-28 SC-AD to invite SC-28 SC-AD to invite SC-28
Perform EVMS surveillance review Bi-annually by SC-28 Annually by Contractor
Bi-annually by SC-28 Annually by Contractor
Bi-annually by SC-28 Annually by Contractor
Bi-annually by SC-28 Annually by Contractor
Annually by Contractor N/A
Submit Lessons Learned regarding up-front planning and design 90 days after CD-3
FPD FPD FPD FPD FPD FPD
SC-AD Request Annual Project Peer Review by PMSO SC-28 SC-28 SC-28 SC-28 SC-28 Tailored
SC-28 Tailored
TOTAL PROJECT COST (TPC) $750M or more Less than $750M to $400M Less than $400M to $100M Less than $100M to $50M* Less than $50M* to $20M Less than $20M to $10M**
DECISION / REQUIREMENTS1 / APPROVAL2
CD-3--APPROVE START OF CONSTRUCTION
PRIO
R T
O C
D-3
--FIN
AL
DES
IGN
POST
CD
-3
Nucl
ear
Faci
lity
Director's CD-2/3a Review of the MicroBooNE Project
21
CD-2/3a Documentation• Acquisition Strategy• Final Project Execution Plan (PEP)• Final Project Management Plan (PMP)• Updated Assumptions Document• Technical Design Report (TDR)• Final Design Requirements for Long Lead Procurements (CD-3a Items)• Baseline Cost & Schedule and • Resource Loaded Schedule and Bases of Estimate (BOEs)• Configuration Management Plan• Preliminary Hazard Analysis Report• Risk Management Plan, Risk Register, Risk Assessment, Risk Mitigation
Plans• Value Management Documentation• Quality Assurance Plan
12-Jul-2011
Director's CD-2/3a Review of the MicroBooNE Project
22
Reviewer Breakout Assignments
12-Jul-2011
Tuesday, July 12 (11:05 – 12:35 AM and continued 1:30 – 3:00 PM) Session 1: Cryostat and Cryogenics – EAG (WH7W) Alex Martinez
Barry Norris Mike Sarychev Joe Howell
Session 2: Active Detectors (TPC & PMTs) – Dark Side (WH6W) Michael Crisler Bob Wagner
Session 3: Electronics, Readout DAQ – Snake Pit (WH2NE) Gustavo Cancelo Peter Shanahan
Session 4: Management – Quarium (WH8SW) Mike Andrews Sherese Humphrey Nancy Grossman Wyatt Merritt Ed Temple
Director's CD-2/3a Review of the MicroBooNE Project
23
Reviewer Breakout Assignments (continued)
12-Jul-2011
Tuesday , July 12 (3:20 – 5:00 PM) Session 1: Detector Assembly – Snake Pit (WH2NE) Gustavo Cancelo
Barry Norris Bob Wagner Mike Sarychev (Mike Andrews)
Session 2: Installation in Building– Dark Side (WH6W) Mike Crisler Joe Howell Alex Martinez Peter Shanahan (Mike Andrews)
Session 3: Management – Quarium (WH8SW) Sherese Humphrey Nancy Grossman Wyatt Merritt Ed Temple
Director's CD-2/3a Review of the MicroBooNE Project
24
Reviewer Breakout Assignments (continued)
12-Jul-2011
Wednesday, March 30 (8:00 – 9:30 AM) Session 1: Cryostat and Cryogenics – Dark Side Video Conference Room (WH6W)
Alex Martinez Mike Sarychev Joe Howell
Session 2: Active Detectors and Electronics/RO/DAQ – West Wing Video Conference Room (WH10NW)
Gustavo Cancelo Mike Crisler Barry Norris Peter Shanahan Bob Wagner
Director's CD-2/3a Review of the MicroBooNE Project
25
Breakout Session Focus
• Subproject Risks– Have the risks been identified, analyzed, and mitigation plans &
costs been established– Has the L2 been involved in the process and own their risks
• Subproject Schedule– Do the L2s know and own their part of the schedule and own it– Is the schedule realistic for the scope of work– Do the L2s know the critical path for their work and where they
are at related to the project’s critical path
12-Jul-2011
Director's CD-2/3a Review of the MicroBooNE Project
26
Breakout Session Focus (continued)
• Subproject Costs– Are there BOEs for the entire scope of the subproject and do the
estimates appear to be realistic and achievable– Has cost estimate uncertainty been developed and appropriate for
the scope of work and phase of the project• Long Lead Procurements (CD-3a list) – Sessions 1
– Do final design specifications/requirements/drawings exist– Are packages prepared to the degree appropriate to order materials
and initiate construction as scheduled
12-Jul-2011
Director's CD-2/3a Review of the MicroBooNE Project
27
Breakout Session Focus (continued)
• Progressing /Reporting /Change Control– Is monthly progress reporting performed and what is the process– Does the process include schedule updates and a narrative
description of progress– Does the L2s understand their role in the change control process
and what is currently being used
12-Jul-2011
Director's CD-2/3a Review of the MicroBooNE Project
28
Project’s Cost Estimate Details
12-Jul-2011
TotalMaterials All Labor Total Materials Labor Total Matl Labor Total Cost
1.1 Project Management 119.0$ 2,183.0$ 2,302.0$ 6.0$ 218.0$ 224.0$ 5% 10% 10% 2,526.0$ 1.2 Cryogenic System 1,814.0$ 1,154.0$ 2,968.0$ 519.0$ 321.0$ 840.0$ 29% 28% 28% 3,808.0$ 1.3 Cryostat 754.0$ 194.0$ 948.0$ 294.0$ 44.0$ 338.0$ 39% 23% 36% 1,286.0$ 1.4 TPC -$ -$ -$ -$ -$ -$ 0% 0% 0% -$ 1.5 Electronics and Readout 1,227.0$ 680.0$ 1,907.0$ 307.0$ 118.0$ 425.0$ 25% 17% 22% 2,332.0$ 1.6 Exp Infrastructue 115.0$ 20.0$ 135.0$ -$ -$ -$ 0% 0% 0% 135.0$ 1.7 Assembly & Installation 385.0$ 733.0$ 1,118.0$ 72.7$ 222.0$ 294.7$ 19% 30% 26% 1,412.7$ 1.8 PMT -$ -$ -$ -$ -$ -$ 0% 0% 0% -$ 1.9 DAQ 180.0$ 684.0$ 864.0$ 66.0$ 190.2$ 256.2$ 37% 28% 30% 1,120.2$
1.10 Assembly Integeration 135.0$ 194.0$ 329.0$ 37.2$ 51.4$ 88.6$ 28% 26% 27% 417.6$ Total TEC: 4,729.0$ 5,842.0$ 10,571.0$ 1,301.9$ 1,164.6$ 2,466.5$ 28% 20% 23% 13,037.5$
1.1 Project Management 133.0$ 1,410.0$ 1,543.0$ 2.9$ 50.8$ 53.7$ 2% 4% 3% 1,596.7$ 1.2 Cryogenic System 99.0$ 620.0$ 719.0$ -$ 7.7$ 7.7$ 0% 1% 1% 726.7$ 1.3 Cryostat -$ 424.0$ 424.0$ -$ 11.4$ 11.4$ 0% 3% 3% 435.4$ 1.4 TPC -$ -$ -$ -$ -$ -$ 0% 0% 0% -$ 1.5 Electronics and Readout 78.3$ 997.0$ 1,075.3$ 4.4$ 56.4$ 60.8$ 6% 6% 6% 1,136.1$ 1.6 Exp Infrastructue 424.0$ 473.0$ 897.0$ 19.3$ 0.8$ 20.1$ 5% 0% 2% 917.1$ 1.7 Assembly & Installation 16.0$ 133.0$ 149.0$ 3.3$ 17.9$ 21.2$ 21% 13% 14% 170.2$ 1.8 PMT -$ -$ -$ -$ -$ -$ 0% -$ 1.9 DAQ 5.3$ 47.4$ 52.7$ -$ -$ -$ 0% 52.7$
1.10 Assembly Integeration 232.0$ 232.0$ -$ 7.9$ 7.9$ Total OPC: 755.6$ 4,336.4$ 5,092.0$ 29.9$ 152.9$ 182.8$ 4% 4% 4% 5,274.8$
TPC: 5,484.6$ 10,178.4$ 15,663.0$ 1,331.8$ 1,317.5$ 2,649.3$ 24% 13% 17% 18,312.3$
MIE
OPC
Items
MicroBooNE's Cost Estimate AY $K
WBSEstimated Cost (with indirects) Contingency/MR %Contingency/MR Estimate
Director's CD-2/3a Review of the MicroBooNE Project
2912-Jul-2011
Cost/Schedule Review Guidance Reviewable
Estimate must “roll-up” from the lowest level to the total and reviewers must be able to drill down from the top to the lowest level
Credible Basis of estimate must be specified Catalog prices Similar work, where cost is documented Engineering estimates WAG – wild ass guess
Director's CD-2/3a Review of the MicroBooNE Project
3012-Jul-2011
Cost/Schedule Review Guidance (Continued)
Cost Choose >~5 top level WBS elements from your system
Drill down to successively lower levels of the WBS; while at each step Understanding the scope of the element Understanding the schedule for that element Understanding the basis of estimate (BOE) for both M&S and Labor for that element Choose a few elements next lowest level of the WBS And repeat this procedure until you get to the bottom level. I.e., the lowest level of the WBS
Choose >~5 items in the system for which you have personal experience Interact with the responsible managers to determine if the Estimate is complete, documented, reviewable, and credible
Director's CD-2/3a Review of the MicroBooNE Project
3112-Jul-2011
Cost/Schedule Review Guidance (Continued)
Check that there is a BOE for all work elements in your system Check whether the estimate for your system “rolls-up” from the lowest level WBS element to the total for your system
Assess the “bottoms up” contingency that the WBS level managers would assign their components. Assess the “top down” contingency analysis assignments by the Project Manager Schedule Is there a detailed schedule, including a critical path, for completing the project? Are milestones appropriate in number and type identified so that the project teams, Fermilab management, and DOE can effectively track and manage progress? Based on past experience, can the proposed schedules be met? Are appropriate schedule contingencies provided? Is there a “resource loaded schedule” and plan for providing the needed resources (M&S and technical support staff and physicists)?
Director's CD-2/3a Review of the MicroBooNE Project
3212-Jul-2011
Cost/Schedule Review Guidance (Continued)
Funding Have techniques such as forward funding by collaborators and phased funding of large contracts been appropriately incorporated into the planning? Does the anticipated funding profile support the resource requirements?
Management Is an appropriate / adequate project organizational structure in place and staffed (or are plans in place) to do the job. Has the appropriate project management documentation been prepared. Is it of a quality adequate for this stage of the project? Are appropriate / adequate management systems (Cost and Schedule Control System / Earned Value Reporting, Critical Path Management, Risk Management, etc.) in place or planned for use during project execution?
Director's CD-2/3a Review of the MicroBooNE Project
33
Reporting Structure• Results of the review are to be documented as findings, comments, and recommendations.
• Answer the charge questions. Answers should be short and precise.
• Any additional actions required to be completed by the project team to acceptably address the review charge are to be documented as Recommendations
• Answers to the questions and any recommendations should be presented in writing at a closeout with MicroBooNE’s and Fermilab’s management.
• .12-Jul-2011
Director's CD-2/3a Review of the MicroBooNE Project
34
Findings, Comments, and Recommendations• Findings
• Comments
• Recommendations
• Findings are statements of fact that summarize noteworthy information presented during the review.
• Comments are judgment statements about the facts presented during the review. The reviewers' comments are based on their experiences and expertise.
• The comments are to be evaluated by the project team and actions taken as deemed appropriate.
• Recommendations are statements of actions that should be addressed by the project team.
• A response to the recommendation is expected and that the actions taken would be reported on during future reviews.
12-Jul-2011
Director's CD-2/3a Review of the MicroBooNE Project
35
Examples of Findings, Comments, and Recommendations
Finding– A plan for the MI upgrades was presented. The major elements of
this plan consist of an upgrade of a MI quad power supply, which is nearly complete, and the addition of two more RF stations. The cavities to be installed currently exist as spares so there is no design and prototyping required.
Comment– The project has decided to build the DCCT in-house. The
committee supports this effort since the technology and design of this device is well developed and well known.
Recommendation– Work with Fermilab management to acquire resources needed to
complete the accelerator and beamline modifications.
12-Jul-2011
Director's CD-2/3a Review of the MicroBooNE Project
36
Reviewer Write-ups• Write-up template (Review Closeout Presentation Format) is
posted on Director’s Review Webpage. http://www.fnal.gov/directorate/OPMO/Projects/MBooNE/DirRev/2011/03_29/ Template-Closeout_Presentation_DR_MicroBooNE_2011-03-29.docx
• Write-ups are to be sent to Terry Erickson at [email protected] prior to 7:30 AM on Thursday, July 14 for the Closeout Dry Run starting at 8:30 AM
• A final report will be issued within 1 weeks after the closeout.
12-Jul-2011
Director's CD-2/3a Review of the MicroBooNE Project
37
Discussion
12-Jul-2011
• Questions and Answers