Systems Engineering: Its the Law! -1 INCOSE Space Coast Chapter April 2010 Systems Engineering: Its the Law! An Overview of the Defense Acquisition Management.

Systems Engineering: It’s the Law! -1INCOSE Space Coast Chapter

April 2010

Systems Engineering: It’s the Law!

An Overview of the

Defense Acquisition Management System and Weapon Systems Acquisition Reform Act

(WSARA)

Geoff Draper

Harris Government Communications Systems Division


Apr 10, 2023

Systems Engineering: It’s the Law!

Topics:• Performance issues in Major

Defense Acquisition Programs (MDAPs)

– Summary of studies and reports

• Overview of the Defense Acquisition Management System

– DoD policies and guidance– DODI 5000.2

• Weapon Systems Acquisition Reform Act of 2009 (WSARA)

• Potential Implications to the Defense Industry

References:P. L. 111-23, Weapon Systems Acquisition Reform Act of 2009. May 22, 2009. http://thomas.loc.gov/cgi-bin/query/z?c111:S.454.enr:

Defense Technical Memorandum, DTM-09-27.“Implementation of the Weapon Systems Acquisition Reform Act of 2009”. Dec 2009. http://www.dtic.mil/whs/directives/corres/pdf/DTM-09-027.pdf

DoDI 5000.02, The Defense Acquisition Management System. Dec 2008.http://www.dtic.mil/whs/directives/corres/pdf/500002p.pdf

DoD Strategic Management Plan. July 2009.http://dcmo.defense.gov/documents/2009SMP.pdf

Defense Acquisition Guidebook (DAG), Chapter 4: Systems Engineering.https://acc.dau.mil/CommunityBrowser.aspx?id=332951&lang=en-US

Implementation of Weapon Systems Acquisition Reform Act (WSARA) of 2009. Mona Lush, OUSD (AT&L). Oct 2009.http://www.acq.osd.mil/damir/2009%20Conference/Future%20of%20DoD%20Acquisition%20Plenary/WSARA%20Implementation%20Lush.pdf DODI 5000.02 and WSARA Impacts on Early System s Engineering.Sharon Vannucci, OSD (AT&L) / DDRE. NDIA SE Conference, Oct ’09.http://www.dtic.mil/ndia/2009systemengr/8925WednesdayTrack2Vannucci.pdf

Weapon System Acquisition Reform Act (WSARA) 2009.Mr. Nicholas Torelli, OUSD (AT&L) / DDRE. NDIA SE Div mtg, July 2009.http://www.ndia.org/Divisions/Divisions/SystemsEngineering/Documents/Past%20Projects/August%2009%20Division%20Meeting/2009_08_06_SE-WSARA%20Brief%20to%20NDIA%20SED-final.pdf A Multi-Level Approach to Addressing SE Issues in Defense Programs.NDIA SE Division, June 2009. http://www.ndia.org/Divisions/Divisions/SystemsEngineering/Documents/Past%20Projects/

(Version 1.0)

Delete this box (from the slide master) after creating all of your slides. Stay inside this “safe area” for correct display.

Addressing SE Issues in the Defense IndustryJuly 2009

3

The Problem –Improving Program Performance through SE

GAO-09-362T - Actions Needed to Overcome Long-standing Challenges with Weapon Systems Acquisition and Service Contract Management

• “DOD’s major weapon systems programs continue to take longer to develop, cost more, and deliver fewer quantities and capabilities than originally planned.”

• “costs … of major defense acquisition programs increased 26 percent and development costs increased by 40 percent from first estimates”

• “programs … failed to deliver capabilities when promised—often forcing warfighters to spend additional funds on maintaining legacy systems”

• “current programs experienced, on average, a 21-month delay in delivering initial capabilities to the warfighter”

NRC/USAF Study - Pre-Milestone A and Early-Phase Systems Engineering: A Retrospective Review and Benefits for Future Air Force Acquisitionhttp://www.nap.edu/catalog/12065.html

• “Attention to a few critical systems engineering processes and functions particularly during preparation for Milestones A and B is essential…”

• “…critically dependent on having experienced systems engineers with adequate knowledge of the domain…”

• “Decisions made prior to Milestone A should be supported by a rigorous systems analysis and systems engineering process involving teams of users, acquirers, and industry representatives.”

Numerous studies and reports document program performance issues and the role of effective

systems engineering

(Version 1.0)



4

Addressing SE Issues - Key Studies and Reports

Study / Report Summary Issues / Findings

Pre-Milestone A and Early-Phase Systems Engineering[report] (NRC/USAF study)

•Inexperienced leadership•External interface complexity•System complexity

•Incomplete/unstable reqts at MS B•Reliance on immature technology•Reliance on large amts of new SW

NDIA Top 5 SE Issues (2006)[report, briefing]

•Inconsistent SE practices•Insufficient SE early in life cycle•Lack SE resources (qty, quality)

•Requirements not well-defined and managed•Inadequate SE tools/environments

NDIA Top SW Issues (2006)[report, briefing]

•SE decisions without SW involvement•Ineffective SW life cycle plans•Lack SW resources (qty, quality)•SW verification: costly, ineffective

•Requirements impact on SW•SW assurance (predictable, secure)•COTS/NDI impacts

Systemic Root Cause Analysis (SRCA) (2008)(publication 2009, pending)

•Unrealistic acq. strategies and plans•Milestone decision gates, criteria

•Staffing shortfalls (quantity, skills, experience)

Systems Engineering of Tactical Air Launched Systems: An Industry Examination (2008) [briefing](Air Armament Center, NDIA Gulf Coast Chapter)

•Incomplete design-to requirements•Requirements creep, not stable•Unplanned reqts verification effort•Unplanned performance/design tests•Configuration variation issues

•Component qualification test•Weapon level integration tests•Weapon level qualification tests•Weapon flight testing•Redesign for producibility/cost•Field reliability, sustainability

House Armed Services Panel on Defense Acq. Reform (final report, March 2010)

•Manage performance of acq. system•Lacking good reqts process (JCIDS)•Ad hoc acquisition of services•Develop/incentivize acq. workforce

•Ineffective financial mgmt system•Protect the industrial base (all sizes)•Responsiveness of core acq system

Details

5

Systemic Issues of Big “A” Acquisition

Small “a” Acquisition

Big “A” Acquisition*

Resources(PPBE)

DefenseAcquisition

System(DAS)

Requirements(JCIDS)

Synchronize JCIDS, DAS, and PPBE to deliver capabilities to Warfighters.

•Funding instability•Insufficient resource trade space •Budget not properly phased/magnitude to

support planned development

•Immature technologies•Inadequate systems engineering•Inadequate requirements flow-down/

traceability/ decomposition•Insufficient schedule trade space•Inadequate implementation of Earned

Value Management System•Lack of time and assets for testing

•Lack of JROC-validated requirements document for basic program (ORD, CDD, CPD)•Inadequate requirements for basic program and any increments•Critical dependence on external programs with developmental issues

•Lack of inter- and intra-departmental stakeholder coordination and support

Joint Capabilities Integration and Development System (CJCSI-3170)Program, Planning, Budgeting, and Execution (DoD-7000, FMRs)

Defense Acquisition System (DoDI-5000)*Systemic Issues of Nunn-McCurdy “Class of 2007” Programs

(Version 1.0)



6

Summary of Key SE Initiatives and Reports

2003

•NDIA Top 5 SE Issues [1/03]

•DODI 5000.2 [5/03]

2006 2008 2009

•NRC/USAF Study: Pre-MS A and Early

SE [1/08]•DODI 5000.2

updates [12/08]•Systemic Root

Cause Analysis (SRCA) [draft]

•PL 111-23 (WSARA) [5/09]

•DTM-09-027 Implementation

of WSARA [12/09]

•Defense Acq Guide (DAG);Chap 4: SE [7/06]

•OSD policy, guidance, reviews: SEP; IMS;

training; risk mgmt; PSRs•NDIA Top SE Issues [7/06]•NDIA Top SW Issues [9/06]

2010

•QDR [2/10]•House Armed

Services Committee

report [3/10]•OSD WSARA

report to Congress [3/10]

Common program issues:-Poor program planning -Unrealistic estimates-Unstable requirements -Immature technology-Not following SE processes -Insufficient SE expertise-Ineffective reviews -Poor system reliability

There is no shortage of sources confirming we have issues –How do government and industry collaborate to act upon solutions?

NDIA Reports DoD policy and actions

Other (Congress, studies, reviews)

Other sources:•Program Support Reviews (PSRs)•Nunn-McCurdy breaches•Congressional oversight•Defense Science Board•Other studies, reviews, reports

7

Chief among institutional challenges facing the Department is acquisition.”

Secretary of Defense Direction

8

“The key to successful acquisition programs isgetting things right from the start with soundsystems engineering, cost estimating, anddevelopmental testing early in the program cycle.The bill that we are introducing today will require theDepartment of Defense to take the steps needed toput major defense acquisition programs on a soundfooting from the outset. If these changes aresuccessfully implemented, they should help ouracquisition programs avoid future cost overruns,schedule delays, and performance problems.”–Senator Carl Levin, Chairman, Senate Armed Services Committee

“The Weapon System Acquisition Reform Act of2009 is an important step in efforts to reform thedefense acquisition process. This legislation isneeded to focus acquisition and procurement onemphasizing systems engineering; more effectiveupfront planning and management of technologyrisk; and growing the acquisition workforce to meetprogram objectives.”–Senator John McCain, Ranking Member, Senate Armed Services Committee

Weapons Systems Acquisition Reform Act

(Version 1.0)



9

P. L. 111-23: Weapon Systems Acquisition Reform Act (WSARA) – May 2009

Key Elements of Legislation:

Organizational• Establishes Directorates for SE and DT&E as principal advisors• Joint tracking of component and MDAP progress against plans and

measurable criteria with annual reporting to Congress• Independent cost estimation and cost analysis (Director, CAPE)• Provide adequate trained staff for SE and development planning• Conduct MDAP performance assessments and root cause analysis

Role of SE across program lifecycle• Developmental planning, lifecycle management, sustainability• Ensure reliability, availability, maintainability (RAM)

Mandates• Measurable performance criteria in SE/DT&E plans• Competitive prototypes for MDAPs; prime make/buy analyses• System PDR before MS B, with formal MDA assessment• Assessment of technical maturity and integration risk of critical

technologies during Technology Development (TD)Technical and cost oversight

• Independent estimates, Problem Assessment Root Cause Analysis• Technical analysis of cost/schedule breaches; presumed termination

Apr 10, 2023

http://thomas.loc.gov/cgi-bin/query/z?c111:S.454.enr http://thomas.loc.gov/cgi-bin/query/z?c111:S.454.enr

Systems Engineering: It’s the Law! -10Engineering Week 2010

Apr 10, 2023

11

Competitive Prototyping

Production & Deployment

MS CMS B

FRP DR

CPDCDDOperations &

Support

MS A

PDR PCDRATechnology Development

Materiel Solution Analysis

Key Acquisition Business Process Changes

Engineering & Manufacturing Development

Enhanced Emphasis on:• Technology Maturity• Systems Engineering• Integrated Testing and Test Planning• Manufacturing and Producibility• Logistics and Sustainment Planning

Re-structured “EMD” Phase

Effective Contracting via Pre-Award Peer ReviewsEffective Contracting via Pre-Award Peer Reviews

Increased Emphasis on Milestone A• Mandatory for MDAPs with Technology

Development Programs• Likely for Most Programs

Preliminary Design Review (PDR) before Milestone B to Enhance Understanding of Derived Requirements and Improve Cost Estimation

When PDR is Conducted after Milestone B an MDA Post-PDR Assessment is Required

PDRP-PDRA

Materiel Development Decision – Mandatory Process Entry Point

Post-Critical Design Review Assessment –A Mandatory Decision Point to Review Progress

Configuration Steering BoardsEstablished to Stabilize Requirements

ICD

MDD

Competitive Prototyping

X

X


Apr 10, 2023


Apr 10, 2023

DoD DSS, vg 14Version 1.1, 9-14-09

Technology and Manufacturing ReadinessTechnology and Manufacturing Readiness

MaterielSolutionAnalysis

Engineering &ManufacturingDevelopment

PRODUCTION &

DEPLOYMENT

B CAOPERATIONS &

SUPPORT

MaterielDevelopmentDecision

Post CDRAssessment

FRPDecisionReview

TECHNOLOGY

DEVELOPMENT

Analytical/Experimental

CriticalFunction/

Characteristic

Proof ofConcept

ComponentAnd/or

BreadboardValidation

In aLaboratory

Environment

SystemPrototype

Demonstrated

In anOperational

Environment

ActualSystem

CompletedQualifiedThroughTest and

Demonstration

ActualSystem

“MissionProven”Through

SuccessfulOperations

Capability to ProduceSystems, SubsystemsOr Components in a

Production RepresentativeEnvironment

Full RateProduction

Demonstrated.Lean Production

Practices In Place

Low RateProduction

Demonstrated.Capability In

Place for FRP

Pilot LineCapability

Demonstrated.Ready for LRIP

Cost Model UpdatedTo System Level

Unit Cost Reduction Efforts Underway

Engineering Cost Model Validated

FRP UnitCost Goals

Met

LRIP CostGoals Met

Learning CurveValidated

Section 2366b of Title 10, United States Code, requires certification that: the technology in the program has been demonstrated in a relevant environment to enter Milestone B. [TRL 6]

Technology Readiness

LevelsDefense Acquisition

Guidebook para. 10.5.2

Manufacturing Readiness

Levels

Draft MRA DeskbookMay 2008

IOC FOC

Capability to produceTechnology In Lab

Environment.Manufacturing Risks

Identified

Manufacturing Cost Drivers

Identified

ManufacturingFeasibility Assessed.Conceptsdefined/

developed

TRLs 1-3

MRL 4 MRL 7 MRL 8 MRL 9 MRL 10MRLs 1-3

TRL 4 TRL 7 TRL 8 TRL 9

ComponentAnd/or

BreadboardValidation

In aRelevant

Environment

Cost ModelConstructed

System/Subsystem

Model orPrototype

Demonstrated

In a RelevantEnvironment

Capability toProduce System/

SubsystemPrototypes

Detailed Cost Analysis

Complete

Capability toProduce Prototype

Components

MRL 5 MRL 6

TRL 5 TRL 6

(Version 1.0)



15

DoD Strategic Management Plan (7/09)Key acquisition-related excerpts

Goals Measures Key Initiatives

Improve acquisition processes and execution to support warfighter reqts

• # of months to IOC • Acquire systems through evolutionary acquisition

Increase use of fixed price contracts • % of contracts at MS B that are FP• % of contracts at MS C that are FP

• Milestone decision reviews

Increase % of MDAPs initiated with low technical risk

• # of MDAPs initiated in GFY• % of MDAPs with technologies

demonstrated in a relevant environment (i.e., TRL 6)

• Competitive prototyping• PDR before MS B• Independent Technical Readiness

Assessments (TRA)

Assess programs as directed in WSARA

• % of Problem Assessment Root Cause Analysis (PARCA) offices

• % of programs assessed

• Root cause analysis

Ensure supportability, maintainability and costs are considered in lifecycle

• % programs meeting DoD sustainment metrics

• Establish sustainment metrics reporting (availability, reliability, TOC)

• Implement recommended next-gen sustainment strategies

Focus R&D to address warfighting requirements

• % of completing demonstration programs transitioned per year

• Track programs transitioning with CY• Review RDT&E funding for transition

Sponsor technology leadership strategy outreach to industry, academia

• # of 2-day dialogues with academia technology leaders

• Understand investment drivers and strategies to sustain tech leadership

Right-shape and re-balance the acquisition workforce - goal to grow by 20,000 positions by FY2015

• % certification positions filled (annual)

• Annual increase in end-strength• # contractor work-yr equiv in-sourced

• Grow the acquisition workforce• Achieve 10,000 acquisition positions

through in-sourcinghttp://dcmo.defense.gov/documents/2009SMP.pdf

(Version 1.0)



16

What’s Next?

Areas of emphasis:• Defense Strategy

- Defense objectives, emerging threats• Rebalancing the Force

- Counterinsurgency, counterterrorism, WMD, cyber- U.S. force structure: sizing, shaping, evolution

• Defense workforce- Supporting troops, deployment- Recruiting, retention, development

• Strengthening relationships- U.S. defense posture, interagency, abroad

• Reforming How We Do Business- Rapid acquisition, security, how we buy, exports- Strengthening the industrial base- Strategic approach to climate and energy

• Defense Risk Management Framework- Operational, force management, institutional,

future challenges, strategic, military, politicalhttp://www.defense.gov/QDR/http://www.defense.gov/QDR/

A U.S. force “prepared to conduct a wide variety of missions under a range of different circumstances.”


Summary

• DODI 5000.2 and WSARA are changing the game – Early life cycle planning (SEP)– Adherence to effective SE practices– Independent cost estimates– Competitive prototyping– Managed technology risks (TRL)– Increased emphasis on reliability and supportability (RAM)– Congressional reporting and oversight– More fixed priced contracts and evolutionary acquisition likely– Revitalization of DoD SE and acquisition workforce

• Questions?– Geoff Draper ([email protected], 321-727-5617)


April 2010

Backup

Apr 10, 2023

(Version 1.0)



19

(Version 1.0)



20

Addressing SE Issues – Key NDIA Task Groups and Milestones

2003

•NDIA Top 5 SE Issues

2006

•NDIA Top 5 SE Issues•NDIA Top SW Issues•Defense Software Strategy Summit

2007

•SE Effectiveness Survey

2008

•Systemic Root Cause Analysis (SRCA) [draft]•Industrial Committee on Program Mgmt (ICPM)

•SE of Tactical Air Launched Systems (Gulf Coast Chapter) (Air Armament Center)

2009

•Top SE/SW Issues Update(planned)•Industrial Committee on SE (ICSE) (planned)

•Government Studies/Reports (GAO, NRC, DCMA, etc.)•Reviews (QDR, etc.)

•Program Support Reviews (PSRs)•Congressional oversight (Nunn-McCurdy, McCain-Levin, …)

•Working Groups•Conferences•Forums

There is no shortage of sources confirming we have issues –How do government and industry collaborate to act upon solutions?

Common program issues:-Poor program planning -Unrealistic estimates-Unstable requirements -Immature technology-Not following SE processes -Insufficient SE expertise-Ineffective reviews -Poor system reliability

(Version 1.0)



21

NRC Study –Pre-MS A and Early-Phase SE

Findings Recommendations

Attention to a few critical systems engineering processes and functions particularly during preparation for Milestones A and B is essential to ensuring that Air Force acquisition programs deliver products on time and on budget.

Require that Milestones A and B be treated as critical milestones in every acquisition program and that … the “Pre-Milestone A/B Checklist” … be used to judge successful completion.

The creation of a robust systems engineering process is critically dependent on having experienced systems engineers with adequate knowledge of the domain relevant to a contemplated program.

Assess needs for officers and civilians in the systems engineering field and evaluate whether either internal training programs … or external organizations are able to produce the required quality and quantity of systems engineers and systems engineering skills. …

The government, FFRDCs, and industry all have important roles to play throughout the acquisition life cycle.… Source selection for system development and demonstration should not be made until after the work associated with Milestones A and B is complete.

Decisions made prior to Milestone A should be supported by a rigorous systems analysis and systems engineering process involving teams of users, acquirers, and industry representatives.

The Air Force used to have a development planning organization that applied pre-Milestone A systems engineering processes to a number of successful programs, but that organization was allowed to lapse.

A development planning function should be established in the military departments to coordinate the concept development and refinement phaseof all acquisition programs to ensure that the capabilities … as a whole are considered and that unifying strategies such as … interoperability are addressed.

Reference: Pre-Milestone A and Early-Phase Systems Engineering: A Retrospective Review and Benefits for Future Air Force Acquisition. http://www.nap.edu/catalog/12065.html

(Version 1.0)



22

NDIA Top 5 SE IssuesNDIA SE Division (July 2006)

Issues Recommendations

1. Key systems engineering practices known to be effective are not consistently applied across all phases of the program life cycle.

Ensure institutionalization of effective SE practices into program planning and execution

2. Insufficient systems engineering is applied early in the program life cycle, compromising the foundation for initial requirements and architecture development.

Integrate engineering planning within the acquisition life cycle to ensure adequate time and effort for SE early in the program life cycle

3. Requirements are not always well-managed, including the effective translation from capabilities statements into executable requirements to achieve successful acquisition programs

Emphasize the application of SE practices and resources to the capability definition process to address warfighter needs and translation into executable programs.

4. The quantity and quality of systems engineering expertise is insufficient to meet the demands of the government and the defense industry.

Grow SE expertise through training, career incentives, and broadening “systems thinking” into other disciplines.

5. Collaborative environments, including SE tools, are inadequate to effectively execute SE at the joint capability, system of systems (SoS), and system levels.

Strengthen and clarify policy and guidance regarding use of collaborative environments, models, simulations, and other automated tools.

Reference: NDIA Top 5 Systems Engineering Issues Report. July 2006. NDIA Systems Engineering Division. [report, briefing]http://www.ndia.org/Divisions/Divisions/SystemsEngineering/Pages/Past_Projects.aspx

Provides an update and status from a previous task group report conducted in 2003.

(Version 1.0)



23

Reference: NDIA Top Software Engineering Issues Report. Sep 2006. NDIA Systems Engineering Division. [report, briefing]http://www.ndia.org/Divisions/Divisions/SystemsEngineering/Pages/Past_Projects.aspx

NDIA Top Software IssuesNDIA SE Division (Sept 2006)

Issues Recommendations

1. The impact of requirements upon software is not consistently quantified and managed in development or sustainment.

Enforce effective software requirements development and management practices, including assessment of change impacts, for both the acquirer and the supplier organizations.

2. Fundamental system engineering decisions are made without full participation of software engineering

Institutionalize the integration and participation of software engineering in all system engineering activities.

3. Software life-cycle planning and management by acquirers and suppliers is ineffective.

Establish a culture of quantitative planning and management, using proven processes with collaborative decision-making across the software life cycle.

4. The quantity and quality of software engineering expertise is insufficient to meet the demands of government and the defense industry.

Collaborate on innovative strategies to staff to appropriate levels, and to attract, develop, and retain qualified talent to meet current and future software engineering needs in government and industry.

5. Traditional software verification techniques are costly and ineffective for dealing with the scale and complexity of modern systems.

Study current software verification practices in industry, and develop guidance and training to improve effectiveness in assuring product quality across the life cycle.

6. There is a failure to assure correct, predictable, safe, secure execution of complex software in distributed environments.

Collaborate with industry to develop approaches, standards, and tools addressing system assurance issues throughout the acquisition life cycle and supply chain.

7. Inadequate attention is given to total lifecycle issues for COTS/NDI impacts on lifecycle cost and risk.

Improve and expand guidelines for addressing total lifecycle COTS/NDI issues.

(Version 1.0)



24

•SRCA workshops and task group•Based on PSR findings (44 programs)•Tagged to core/systemic root causes

Recommendation Area

Problem Statement Industry Actions*

Implement Achievable Acquisition Strategy and Planning

Acquisition strategies and plans are incomplete, ineffective and unrealistic, resulting in unachievable program expectations

•Require the government, prior to RFP release, to provide industry with govt. expectations and common understanding for IMP/IMS/risk management/business rhythms

Enhance Gate Review Process

Lack of timely process and adequately defined and enforceable criteria to assess program maturity at milestones and linkage to technical reviews

•Define criteria for trigger conditions

Enhance Staff Capabilities

Staffing shortfalls (numbers, skills, and experience) lead to adverse acquisition consequences specifically in the areas of requirements, planning, execution and expectations

•Develop and validate a representative staffing model for DoD based on industry that can be applied to the govt. given the current OSD acquisition guidance•Develop a workload analysis to estimate the numbers and expertise needed in the acquisition workforce•Broaden expertise to enhance cross-functional and domain knowledge and skills

•95 preliminary recommendations•48 systemic issues•3 Recommendation Areas

* Reference SRCA briefing for additional details and recommended government actions.

Systemic Root Cause Analysis (SRCA)- Conducted 2007-2008, publication pending

(Version 1.0)



25

Reference: “Systems Engineering of Tactical Air-Launched Weapons: An Industry Examination.”Air Armament Symposium, 2008. [briefing]http://www.ndiagulfcoast.com/events/archive/34th_Symposium/34_Day1/11_SysEngNDIASymposiumBriefOct208.pdf

SE of Tactical Air Launched WeaponsNDIA Gulf Coast Chapter (2008)

SE Deficiencies Root Causes

Program Structure and Control

•Insufficient Maturity Of Design At Critical Decision Points•Insufficient Testing And Analysis Planned To Achieve Maturity•Late Integration Of Production Critical Processes And Controls •Program Funding Profiles Not Structured For Improved Practices

Requirements And Verification

•Lack Of Service Use Profile Leaves Interpretive Requirements•Insufficient Mapping Of Requirements To Design•Ineffective Maturation And Verification Planning

Design Best Practices •Inadequate Design Analysis- Fault Tree Analysis On All Subsystems During Design- Single Point Failure Analysis During Design

•Inadequate Maturation Analysis And Testing- COTS Integration- Design Margin And Sensitivity Development- Critical Manufacture And Assembly Process ID / Control

•Use Of “Production Representative” Configuration For Verification

Risk Identification And Management

•Inadequate Relationship To “Knowledge” Of Design

Supply Chain Practices •Inconsistent Approaches To Design Characterization

(Version 1.0)



26

A Survey of Systems Engineering Effectiveness (2007)Joint NDIA/SEI survey of 46 programs correlating the effectiveness of SE processes with program performance.

SE processes most strongly correlated with better program performance:

• Architecture• Trade Studies• Technical Solution• IPT Capability• Requirements Development and

Management

Report: http://www.sei.cmu.edu/publications/documents/08.reports/08sr034.html

Projects with better Systems Engineering Capabilities deliver

better Project Performance (cost, schedule, functionality)

Relationship of SE Processes to Program Performance

-13%

13%

13%

21%

25%

28%

28%

33%

34%

36%

37%

40%

-20% -10% 0% 10% 20% 30% 40% 50%

Project Monitor/Control

Project Planning

Config Mgmt

Product Integration

Verification

Risk Mgmt

Validation

Reqts Devel & Mgmt

IPT Capability

Technical Solution

Trade Studies

Architecture

SE

Cap

abili

ty

Gamma (strength of relationship)

27

MaterielSolution Analysis

Engineering and ManufacturingDevelopment

Production &Deployment

Systems Acquisition

FR5P DecisionReviewLRIP/IOT&E

Post-CDRAssessment

TechnologyDevelopment

(ProgramInitiation)

MaterielDevelopmentDecision

Pre-Systems Acquisition

Operations & Support

Sustainment

A B C

Warfighter and Sustainment Organization Perspective

Defense Acquisition System Weighted Expenditures

IOC

Technology Development

Engineering and Manufacturing Development

Production & Deployment

Systems Acquisition

Operations & Support

Sustainment

FRP DecisionReview

FOC

Post-CDRAssessment

Pre-Systems Acquisition

MaterielSolutionAnalysis

Materiel Development Decision

BA CProgramInitiation

LRIP/IOT&E

DoDI 5000.02 Perspective

20-35% 65-80%30+ YEARS

Nominal Life Cycle Cost Distribution

Systems Engineering: Its the Law! -1 INCOSE Space Coast Chapter April 2010 Systems Engineering: Its the Law! An Overview of the Defense Acquisition Management.

Documents

sw issues

defense programs

rigorous systems analysis

experienced systems

se gao

ndia se division

se issues key studies

ndia se conference