DKIST Overview J. McMullin DKIST SOC 2015-Oct
DKIST Overview
J. McMullin
DKIST SOC 2015-Oct
Outline
• Main – Overview – Schedule – Budget – Scope – Issues/Risks
• Backup Slides – 2015 NSF Baseline Review – Annual Safety Report – NSF Large Facilities Manual – Email Auditing Process – Subsystem Status Details
DKIST SOC 2015-Oct
Overview of Observatory
DKIST SOC 2015-Oct
(Last SOC) 20 March 2015
DKIST SOC 2015-Oct
Building interior – offices!
DKIST SOC 2015-Oct
Solar Physicist Containment System
(Last SOC) October 2015
DKIST SOC 2015-Oct
East-West Section Through Site (work completed @ Oct 2015 shown in color)
Also completed but not shown: 1) site infrastructure & electrical transformers; 2) back-feed electrical, comm., and fiber to IfA facilities; 3) retaining wall & roadwork
Enclosure cable wraps
DKIST SOC 2015-Oct
Overview: Schedule
Project Schedule • Initial funding/start of construction 2010 Jan • Permits in place/start of site construction 2012 Dec • Site Prep, Excavation, Foundations 2013-2014 • Lower Enclosure, Enclosure 2014-
2015 • Telescope Mount Assembly 2016-
2017 • Optics systems installation/Integration 2017-
2018 • Instrument systems commissioning 2018-2019 • Start of Operations
2019 Sep (80 day slip from 2012 baseline; within planned schedule contingency).
DKIST SOC 2015-Oct
Schedule
• Status
– 81 days behind schedule • Relative to October 2012 baseline
• 5.75 years of construction
• approx. 2.75 years of site construction
• 19 Sep 2019
– Roughly 1 month/year
– Reviewing recovery plans
• Schedule Projections
- Baseline allocated 12 months of schedule contingency
- Composed of:
- 7 months funded contingency (in contract budgets for these critical path efforts)
- 6 months of liens on contingency
• Schedule Performance - Site work has exceeded
planned schedule contingency needs
- Weather (!) - Fit-up of non-
preassembled components
- Weather exposure ends after Enclosure SAT, S&O BOD
Best Site for Solar Physics
DKIST SOC 2015-Oct
Maui Unions
Protests Petrels
Schedule status
• Big picture: Schedule within projections over 5.75 years of construction/2.75 years of site construction
• Not for lack of challenges: – Ongoing/permitting restrictions:
• NPS HALE road restrictions (sunrise+30 minutes to sunset-30 minutes and can’t have traffic from 1100-1400).
• Inspection of all vehicles prior to summit use (coordinate with NPS) • Site Logistics (small footprint, limited laydown availability, other HO activities/construction) • Tropical storms/anomalous (?) rain cause work slowdowns (sometimes with NPS shutdowns) • Protests
– Other (some examples) • Crack in initial M1 Blank • DMS Vendor transition at CDR • HI issues with union/PLA, restricted bidding, hiring difficulties, PLA lanaguage • Permitting challenges/protests/etc • Petrel Mortalities • FAA finding of hazard • ARRA spendout • M1 Polishing • VBI Filters
DKIST SOC 2015-Oct
Overview: Budget
DKIST SOC 2015-Oct
Budget
• Funding – $268M (to date)
• $146M ARRA
• $122M MREFC
• Profile
- 2016: $20M
- 2017: $20M
- 2018: $20M
- 2019: $16.1M
DKIST SOC 2015-Oct
• Projections • EAC (spending as planned):
$297M • EAC (spending as past
performance): $297M • BAC: $307M
Component Company Location Status
Telescope Mount Assembly Ingersoll Tools Rockford, IL Coude Rotator: Accepted July 2014 Mount: Accepted May 2015
Enclosure AEC IDOM Bilbao, Spain Accepted Feb 2014; SAT ongoing
M1 Schott AG (blank) UA COS (polish)
Mainz, Germany Tucson, AZ
Accepted @60nm rms ; FAT ongoing
M2 (TEOA) L3 Pittsburgh, PA Factory Testing (heat stop)
M3, M4, M6 L3 Pittsburgh, PA Factory Testing
M5 Physik Instrumente Auburn, MA Accepted
M7, M8, M9 Arizona Optical Systems, LLC Tucson, AZ In Fabrication
M10 Xinetics Devens, MA Accepted May 2015
M1 Cell Assembly AMOS Liege, Belgium Factory Testing
HLS NSO, OSL * FDR Sep 2015
Site – Excavation Site – Concrete Site – Steel Erection Site – Electrical Site – FTS/Plumbing
Tom’s Backhoe & Exc Reef, Rojac Trucking Inc SteelTech Inc Du-Watts Electric Inc NSO*
Kula, HI Wailuku, HI Lahaina, HI Wailuku, HI Pukalani, Maui, HI
Complete Complete Ongoing (Enclosure, S&O Building) Ongoing FTS ongoing
Instruments – WFC Instruments – PA&C Instruments – Cameras* Instruments – VBI Instruments – ViSP Instruments – VTF Instruments – Cryo-NIRSP Instruments – DL-NIRSP
NSO NSO UK Consortium (QUB lead) NSO HAO KIS UH/IfA UH/IfA
Boulder, CO* Boulder CO* Belfast, N. Ireland UK Boulder, CO Boulder, CO Freiburg, Germany Pukalani, Maui, HI Pukalani, Maui, HI
CDR 2015 July PDR 2016 Fabrication Lab Testing & Integration Fabrication Contract Approvals pDR June 2015 In Fabrication In Fabrication
DKIST SOC 2015-Oct
March 2015 SOC
• Identified Issues – M1 Mirror Polishing
• Delayed but now deterministic
– TEOA • FAT went well; Heat Stop still an issue (residual risk to operations)
– FTS • Catching up (Union impact)
– PA&C • Team assembled and planning for 2016 PDR
• Long lead time optics in fabrication/testing
– Visible Camera • Moving forward (DKIST supporting management of the effort)
DKIST SOC 2015-Oct
Risks • Risk Summary
– 95 (High Level) Risks – Exposure at $28.4M – 80% probability at $31.7M
• Highest exposure risks (>$1M): – Schedule delays – Permitting/Legal – Instrument system
Fabrication/Integration – Incomplete System Design
• TMA Optical/Mechanical configuration interface problems
– Staffing Levels
• Changes in Risk Exposure – FTS (contract->self perform->PLA) – PA&C Reboot – Instrument Fabrication – Expanded storage/security
requirements in Maui • Protests, water damage
– Enclosure Site Assembly
• Contingency - $37.8M - % contingency: 31% (ETC1) - % contingency: 32% (ETC2) - Identified Risk/Contingency:
- 83% - (80% prob/available)
• Legal • 2 outstanding cases both
under review by the HI Supreme Court
DKIST SOC 2015-Oct
Summary
• Project is roughly on schedule after 5.75 years of construction (2.75 years of site construction); 2019 start of operations
• Project is within budget projections
• Accepted components/designs have met the challenging performance specifications.
• Successful (ongoing) site/project safety program.
• IT&C challenges ahead; working to ameliorate these in advance
Long way to go… DKIST SOC 2015-Oct
Backup Slides
• 2015 NSF Baseline Review
• Annual Safety Report
• NSF Large Facilities Manual Impact
• Email Auditing Process
• Subsystem Status Details
DKIST SOC 2015-Oct
NSF Baseline Review
• Tucson, AZ
• December 7-9
• Panel
– Tony Beasley, NRAO Director
– Jonathan Cirtain, NASA MSFC, Heliophysics Team Lead
– Brenna Flaugher, Fermilab, Department Head, Astrophysics
– Alistair McPherson, SKA, Head of Project
– Gary Sanders, TMT, Project Manager
DKIST SOC 2015-Oct
NSF Baseline Review
Charge is to Assess: • Project Performance (scope, cost, schedule, risk) • Project Processes for Project Management and
Control (change control, risk management, EVM, Requirement & Compliance tracking, contracting/procurement)
• Cost Estimation History/Performance • Safety • Engagement with Stakeholders • +?
DKIST SOC 2015-Oct
2015 Annual Safety Review
• August 2015 (composed of multi-day field/site inspection and one day face to face meeting)
• Panel
– Richard Hislop (Chair; Hislop & Associates, Inc)
– Shawn Bradfield (NREL; Bradfield Safety Consultants)
– Chuck Gessner (LSSR Head of Safety)
– Mark Grushka (MJGrushka Consulting)
DKIST SOC 2015-Oct
2015 Annual Safety Review
• Findings
– Many positives (leader in safety by Design approach, hazard analysis process, hurricane incident handling, etc)
– Areas of concern:
• Construction incident rate
– Repeat offenders
• Medical evac
• IT&C Electrical system handling (energization/de-energization; arc flash analysis), formal site management responsibility transitions, Plan-of-the-Day coordination, etc
DKIST SOC 2015-Oct
NSF Large Facilities Manual Impact
DKIST SOC 2015-Oct
Large Facilities Manual Summary Changes
• January 4, 2013 – 72 pages – Outline
• Intro
• Planning and Managing for the MREFC Account (4 pages)
• Planning and Managing LF Projects not MREFC
• Detailed Considerations and Requirements (17 pages)
• Special Topics and Supplementary Materials (4 pages)
• Appendices, References, List of Acronyms, etc
• March 13, 2015 – 229 pages (!) – Outline
• Intro • LF Life Cycle and the MREFC
Process (60 pages) • Life Cycle Management Plans
for Large Facilities (16 pages)
• Key Management Principles and Reqs (39 pages)
• Special Topics and Supplementary Materials (75 pages)
• Appendices, References, List of Acronyms, etc
LFM Section Changes Life Cycle Management
Plans for LF • 2015 Draft Guidance
– Project Execution Plan Contents
• Current Project Practice
– Status: Non-compliant • Project follows the PMI
framework
Action/Impact: Project will adapt/update the PEP to incorporate this draft’s requirements (4 FTE weeks)
LFM Section Changes Key Mgmt Principles and Reqs for LF
• 2015 Draft Guidance
– Cost Estimating/Budget – Management Fees – Budget Contingency Plan – Reporting Requirements
• Change Control
– System Integration, Commissioning, Testing, etc
– Documentation Reqs – Requirements for
Performance Oversight
• Current Project Practice – Status: Compliant – Status: In Discussion – Status: Compliant – Status: Compliant – Status: Analogous – Status: Analogous Definition
• Note: Need Transition Plan (1 year prior to start of commissioning)
– Status: Compliant – Status: NSF Analysis/Impact: None(?)
LFM Section Changes Special Topics and Supp
Materials - 1 • 2015 Draft Guidance
– Risk Management Plan
– Risk Register
– Determination of Risk Exp
– Risk Definition
– Risk Exposure Definition
– Allowable Contingency
– Contingency BOE • 70-90% Confidence Level
• Process for use of Contingency
– Schedule Contingency
• Current Project Practice
– Status: Compliant
– Status: Compliant
– Status: Compliant
– Status: Compliant
– Status: Compliant
– Status: Compliant
– Status: Compliant • Status: Compliant
• Status: Compliant
– Status: Analogous (translated to budget)
LFM Section Changes Special Topics and Supp Materials - 2
• 2015 Draft Guidance
Risk Management Steps/Methodology
– Risk Management Planning
– Risk Management Plan
– Risk Identification • Risk Description
– Risk Breakdown Structure • (Tech, Ext, Org, PM)
– OMB Risk Categories – Advisory
• Current Project Practice
– Status: Compliant
– Status: Non-compliant
– Status: Compliant • Status: Analogous
– Status: Analogous • (Perf, Bud, Sched, PM)
– Status: Analogous
LFM Section Changes Special Topics and Supp Materials - 3
• 2015 Draft Guidance
Quantitative Risk Analaysis
– Maxwell Assessment – rec
– Risk Level Matrix
– Butterfly/Mirror
– Trend Analysis
Quantitative Analysis
– Schedule
• Current Project Practice Status: Compliant* (more description needed for external)
– Status: Analogous (GAO)
– Status: Analogous
– Status: Not used • opportunities tracked as
mitigations
– Status: Analogous
- Status: Compliant* - Matches what was done at re-
baseline; now non-compliant.
LFM Section Changes Special Topics and Supp Materials - 4
• 2015 Draft Guidance
– Risk Inflation – Risk Response Planning – Risk Monitoring and Control
• Contingency Profile • Change Control • Liens List • Updates of EAC/Risk Exposure • NSF Oversight • Documentation & Reporting
– Cyber Security – External Reviews – Environmental Considerations – Reporting Requirements – BSRs, OMB Inflators
• Current Project Practice – Status: Tracked; not reported – Status: Compliant – Status: Compliant
• Status: Tracked; not reported • Status: Compliant • Status: Analogous • Status: Defiantly non-compliant • Status: Compliant • Status: Compliant
– Status: Evolving – Status: Compliant – Status: Compliant (BR) – Status: Compliant – Status: Compliant
Summary Impact (LF Updates)
• Life Cycle Management Plans for Large Facilities – 4 FTE weeks
• Key Management Principles and Requirements for Large Facilities – Standard management
practice; no impact
• Special Topics and Supplemental Materials – 4 FTE weeks to
homogenize/conform to recommended risk content descriptions
– Assume analogous is compliant for: • RBS • Qualitative Risk Analysis • Quantitative Risk Analysis
– If Schedule regarded as non-compliant – significant effort required.
– Case study is different from project; if this gets religious=big (not useful) effort
• Liens list • RAC is bad math; probability
incorporated into MC modeling
– Unknown impact for Cyber security (being tracked) DKIST SOC 2015-Oct
Meta Summary
• Overall the revised guidelines represent best practices that we are following, though there are differences in the details.
• I believe our processes and practices are rigorously defensible. – My concerns are in the areas where we have
analogous systems; the guidelines are stated as guidelines but if they (or an individual) insisted upon strict adherence, we could waste significant time with little project benefit.
– We will need to discuss with our PO
DKIST SOC 2015-Oct
Email Auditing
• Audits to date – E-mail inquiries – No site presence/interaction
• Case study – Audit of the Project’s Response to the DCAA Audit
• Recurring misunderstandings: – WBS versus Work Package – ARRA and MREFC funding – Cost books as snapshots in Time
» Change Requests » Contract Amendments
– https://www.dropbox.com/s/f1rwywzmxdwtc8q/S-WMEN3-004%20-%20Enclosure%20Fabrication%20and%20Factory%20Assembly%20and%20Testing.mp4?dl=0
DKIST SOC 2015-Oct
Project Response to DCAA
DKIST SOC 2015-Oct
Site & S&O Building
DKIST SOC 2015-Oct
S&O Building • Cladding & Roofing: Now • Platform Lift: Now-10/15 • Electrical: Now-12/15 • Mechanical: ~Now-5/16 • Plumbing: ~Now-5/16 • Fit & Finish: Now-5/16 • Site Closeout: 4/16-4/17
Enclosure
• Site Assembly & Construction In-Progress – Started Site Assembly
Feb 17, 2015
– Structural, Mechanical, Electrical Install ongoing
– Handing Equipment, Cladding, Ventilation Gates Install upcoming
– Site Acceptance Testing planned early 2016.
DKIST SOC 2015-Oct
Telescope Mount Assembly
• Coudé Rotator – Factory Acceptance Complete July
2014 – Shipped to Maui complete Nov
2014 and in storage. – Site Install starts Oct 2015. – Site Acceptance Testing complete
Jan 2017
• Telescope Mount – Factory Acceptance Testing
Complete April 2015 – Shipment arrivals started Jul 2015
and will be complete by Oct 2015. – Site Install starts April 2016 – Site Acceptance Testing complete
Nov 2017
DKIST SOC 2015-Oct
M1 Mirror
• M1 Mirror polishing is near completion.
• We expect:
– Factory Acceptance: November 2015
– Long term storage in Tucson area.
– Ship date: ~Q1 2017
– Site Integration start: Q2 2017 (coating)
DKIST SOC 2015-Oct
M1 Assembly
• Initial integration and testing underway, using the M1 prototype mirror blank.
• Factory Acceptance February 2016
• Shipment to site Q2 2016
DKIST SOC 2015-Oct
TEOA
• The TEOA is fully constructed, assembled and tested. We are at >90% completion of FAT.
• We expect: – Factory Acceptance:
September 2015 – Possible long term storage
at L3 facility – Ship date: Q4 2016 – Site Integration start: Q1
2017
DKIST SOC 2015-Oct
Feed Optics
• Transfer Optics factory acceptance complete.
• Shipment to Tucson Q3 2015
• Shipment to site 2017
• Coudé Optics in preliminary design stage.
• PDR October 2015
• FDR Q2 2016
• FAT Q4 2016
• On site Q1-Q2 2017
DKIST SOC 2015-Oct
2014 2015 2016 2017 2013
Fluid Cooler & CES AHUs Factory Acceptance Test
Facility Plant Equipment, tunnel piping, S&O Bldg HVAC and mechanical equipment room
Finish Coudé Environmental Systems, TEOA closed loop & TMA Thermal Systems
Fluid Cooler & CES AHUs rough install
Complete Enclosure Thermal Systems & Start Coudé Environmental Systems
Site close-out support, deliver as-builts, maintenance and operation manuals and provide IT&C support
2018 → IT&C
Enclosure Thermal Systems (ETS)
Facility Thermal Systems
DKIST SOC 2015-Oct
Instruments
ViSP • Fab Contract
Negotiations Underway
• LAT: 9/17-3/18 • Ship: 3/8-6/18 • Site IT&C: 6/18+
VBI • Fab 90% Complete • LAT: Now-2017 • Ship: 10/17 • Site IT&C: 11/17+
Cryo-NIRSP • Fab Underway • LAT: 10/15-2/17 • Ship: 3/17 • Site IT&C: 6/17+
VTF • In Design • LAT: 6/18-12/18 • Ship: 4/19-6/19 • Site IT&C: 7/19+
DL-NIRSP • Fab Started • LAT: 2/18-2/19 • Ship: 2/19 • Site IT&C: 3/19+
DKIST SOC 2015-Oct
Wavefront Correction System Status
• Current status & timeline: – Lab I&T, Boulder, CO: Oct 2015
– Critical Design Review: Nov 2015
– Summit IT&C: late 2017 – early 2018
• Lab I&T Safety Issues – Eye safety, Class IIIa/3R HeNe Laser, 543 nm,
2 mW
– Electrical safety
– Soldering safety
– Pinch hazards, motorized optical stages
– Heavy lifting
– Solvent use for optical cleaning
– Lab safety briefings for WFC team
Deformable Mirror
Coudé Optical Bench Layout
DKIST SOC 2015-Oct