Briefing for INCOSE Learn. Perform. Succeed. Rev. Oct 07.

Briefing for INCOSE

Learn. Perform.

Succeed.

#1#1Rev. Oct 07

http://www.council.org/

DAU Mission

Provide practitioner training, career management, and services to enable the AT&L community to make smart business decisions and

deliver timely and affordable capabilities to the warfighter.

1. We train the AT&L Workforce through certification and assignment-specific courses

2. We promote career-long learning through our Continuous Learning Center

3. We offer performance support to the AT&L Workforce through consulting, Rapid Deployment Training, and targeted training

4. We facilitate knowledge sharing through online resources and communities of practice

5. Provide Human Capital Strategic Planning for the AT&L Workforce

DAU Regional Orientation

We are part of the community… not just a place to go to take classes.

Capital & Northeast and DSMC

(Fort Belvoir)AT&L WF ~33,357

Mid-Atlantic(Pax River)

AT&L WF ~22,476

Midwest(WP AFB)

AT&L WF ~18,945South(Huntsville)

AT&L WF ~28,056

West(San Diego)

AT&L WF ~25,408

•CL Modules - Online modules to help you earn continuous learning points

•Conferences•PEO / SYSCOM •Business Manager•DAU Acquisition Community Conference

•Consulting - We come to your workplace to assist you

•Targeted Training - Tailored learning for your organization

•RDT - On-site and online training on the latest AT&L policies

Training Courses - DAWIA Certification, assignment-specific, and executive & leadership courses – in the classroom and online

•AKSS - Online gateway to AT&L information & tools

•ACC - Online collaboration communities tailored to your needs

•Virtual Library - Keeping you connected to research tools when you are not on campus

24/7 Learning Assets for the Classroom and the Workplace

AT&L Performance Learning Model

Core Plus is Here!

Core Plus is an enhancement to the existing AT&L certification framework. It is not a wholesale replacement!

Core Plus is designed to guide workforce

members to additional training beyond what’s

required for certification.

“Plus” or job competencypoint-of-need

training (frequently CLMs) (tailored to your specific

job or assignment)

Career Field foundation knowledge and skills

(all career field members)

Common acquisition foundation knowledge &

skills (all DoD AT&L workforce members)

FY08 T&E Level IICore Plus Framework

Acquisition & Functional Core DAWIA Certification Standards – applies to all

career field members

Tailored Cross-Functional “Plus” Training, Education & Experience Opportunities –

based on the job or assignment

Three Level Certification Framework

• Case/scenario based• GS 13-14 & E7-O5

Level II Certification Level IIICertification

SYS 202SYS 202Intermediate Intermediate

Systems Systems Planning, Planning, Research, Research,

Development Development and Engineering, and Engineering,

Part IPart I




Part IPart I

Level I Certification

25 hrs, online

ACQ 101ACQ 101Fundamentals Fundamentals

of Systems of Systems Acquisition Acquisition

ManagementManagement




• Knowledge based• GS 5-9 & E7-O3

ACQ 201ACQ 201Intermediate Intermediate

SystemsSystemsAcquisition Acquisition



35 hours, online1 week classroom

TST 203TST 203Intermediate Intermediate

Test & Test & EvaluationEvaluation

TST 203TST 203Intermediate Intermediate

Test & Test & EvaluationEvaluation

35 hrs, online

5 days classroom

P = Prerequisite

TST 102TST 102Fundamentals Fundamentals

of Test and of Test and EvaluationEvaluation

TST 102TST 102Fundamentals Fundamentals

of Test and of Test and EvaluationEvaluation

Level I “Core Plus”Courses & CL Modules

(See DAU catalog for details)



Level III “Core Plus”Courses & CL Modules




Level II “Core Plus”Courses & CL Modules




P

30 hrs, online

TST 302TST 302Advanced Test Advanced Test and Evaluationand Evaluation

TST 302TST 302Advanced Test Advanced Test and Evaluationand Evaluation

5 days classroom

Vers 8.1

P

CLM 029CLM 029Net-Ready Net-Ready

Key Key Performance Performance

ParameterParameter

CLM 029CLM 029Net-Ready Net-Ready

Key Key Performance Performance

ParameterParameter

P

FY08 Test & Evaluation Field Training

CLE 023CLE 023Modeling and Modeling and Simulation for Simulation for

Test and Test and EvaluationEvaluation

CLE 023CLE 023Modeling and Modeling and Simulation for Simulation for

Test and Test and EvaluationEvaluation

P

35 hrs, online

SYS 101SYS 101Fundamentals Fundamentals

of Systems of Systems Planning, Planning, Research, Research,

Development Development and and

EngineeringEngineering





P

P


• Knowledge and Application/case based• GS 9-12 & E7-O4



25 hrs, online







STM 201STM 201Intermediate Intermediate Science and Science and Technology Technology ManagementManagement

STM 201STM 201Intermediate Intermediate Science and Science and Technology Technology ManagementManagement

3 days classroom

P = Prerequisite









STM 302STM 302Advanced Advanced

Science and Science and Technology Technology ManagementManagement

STM 302STM 302Advanced Advanced

Science and Science and Technology Technology ManagementManagement

5 days classroom

Vers 8.1

P

FY08 SPRDE: Science & Technology Manager

NONECLE 021CLE 021Technology Technology Readiness Readiness

AssessmentAssessment

CLE 021CLE 021Technology Technology Readiness Readiness

AssessmentAssessment

P

P


• Application/case based• GS 9-12 & E7-O4





Part IIPart II




Part IIPart II


25 hrs, online















Development and Development and Engineering, Part Engineering, Part

II



Development and Development and Engineering, Part Engineering, Part

II

35 hrs, online

5 days classroom

P = Prerequisite





















P30 hrs, online

SYS 302SYS 302Technical Technical

Leadership in Leadership in Systems Systems





10 days classroom

Vers 8.1

P

P

FY08 SPRDE: Systems Engineering Training

CLL 008CLL 008Designing for Designing for Supportability Supportability

in DoD in DoD SystemsSystems



CLE 003CLE 003Technical Technical ReviewsReviews


P

P P

Two 100 level DAU Two 100 level DAU courses from among courses from among PM, T&E, PQM, LCL, PM, T&E, PQM, LCL,

BCFM, IT or CON BCFM, IT or CON career fieldscareer fields




• Application/case based• GS 9-12 & E7-O4


SYS 203SYS 203Intermediate Systems Intermediate Systems Planning, Research, Planning, Research,

Development and Development and Engineering, Part IIEngineering, Part II


Development and Development and Engineering, Part IIEngineering, Part II


25 hrs, online














Development and Development and Engineering, Part IEngineering, Part I


Development and Development and Engineering, Part IEngineering, Part I

35 hrs, online

5 days classroom

P = Prerequisite





















P 30 hrs, online







10 days classroom

Vers 8.1

P

PCLL 008CLL 008

Designing for Designing for Supportability Supportability






P

P

FY08 SPRDE: Program Systems Engineer Training (new career field)





One 100 or 200 level One 100 or 200 level DAU course from DAU course from

among PM, T&E, PQM, among PM, T&E, PQM, LCL, BCFM, IT or CON LCL, BCFM, IT or CON

career fieldscareer fields

One 100 or 200 level One 100 or 200 level DAU course from DAU course from



LOG 204LOG 204Configuration Configuration ManagementManagement

LOG 204LOG 204Configuration Configuration ManagementManagement

15 hrs, online

Two 200 or 300 level Two 200 or 300 level DAU courses from DAU courses from



Two 200 or 300 level Two 200 or 300 level DAU courses from DAU courses from



P

DAU Strategic Partnerships

Credit for DAU courses toward degrees & certificates at colleges and universities

– Shorter time to get your degree– Saves tuition assistance and out-of-pocket expenses

Bachelor’s DegreeBachelor’s Degree

ADL Co-Lab (Partner)Academic Co -Lab (U of WI)

Joint Co-Lab (UCF)

ADL Co-Lab (Partner)Academic Co -Lab (U of WI)

Joint Co-Lab (UCF)

Universi ty College (UM UC) andUniversi ty of M aryland - Baltimore

Universi ty College (UM UC) andUniversi ty of M aryland - Baltimore

Florida Institute of TechnologyFlorida Institute of Technology AFIT AFIT AFIT

NPSNPS

“Excelerate” Your Master’s Degree…Excelerate is an exciting new program where you

can earn your master’s degree in less time. Through this program, several universities are

offering the AT&L workforce credit toward masters degrees for DAWIA Level II and III certification.*

*DAWIA course and masters programs vary

More information on DAU’s partnership program can be found at: http://www.dau.mil/about-dau/partnerships.aspx

DAU Learning Assets at: www.dau.mil

https://learn.dau.mil/html/clc/Clc.jsp

http://akss.dau.mil

http://www.dau.mil/registrar/enroll.asp

Continuous Learning Center

AT&L Knowledge Sharing System (AKSS)

DAU Training Courses Performance Support

http://www.dau.mil/performance_support/

Webcast/Podcast Center

http://www.dau.mil/webcast_podcast/webcast_podcast.asp

DAU Acker Library

http://www.dau.mil/library/


http://akss.dau.mil/jsp/default.jsp



http://acc.dau.mil/


http://www.dau.mil/performance_support/PSS_main.asp

http://www.dau.mil/performance_support/

http://www.dau.mil/equivalency


DAU’s Commitment to the DoD AT&L Workforce

Shaping a culture of engagement and career-long learning

•Training - 24 / 7 learning assets: in the classroom and in the workplace

•Continuous Learning - Helping you learn in the workplace: what you need to know , when you need to know it

•Performance Support - Supporting your organization

•Knowledge Sharing - Connecting you – the Engaged Learner - with the experts, resources and materials you need to do your job

Current Events

ELO # 3.1 and 3.2

Honorable John J. Young, Jr.

LtGen Henry Obering III

Dr John Foulkes

Vacant

Director, Systems &Software Engineering

Kristen Baldwin (Acting) SES

Deputy DirectorEnterprise Development

Nic Torelli SES

Deputy DirectorDevelopmental Test

& Evaluation

Chris DiPetto SES

Deputy DirectorSoftware Engineering &

System Assurance

Bruce Amato (Acting) SES

Deputy DirectorAssessments & Support

Glyn James (Acting) SES

CORE COMPETENCIES

• SE Policy• SE Guidance

• SE in Defense Acquisition Guidebook

• Technical Planning• Risk Management• Reliability &

Maintainability• Contracting for SE• SoS SE Guide

• SE Education and Training• DAU SE Curriculum• SPRDE Certification Reqt

• Special Initiatives• Corrosion• RTOC• VE

CORE COMPETENCIES

• DT&E Policy• DT&E Guidance

• T&E in Defense Acquisition Guidebook

• TEMP Development Process

• DT&E Education and Training

• DAU DT&E Curriculum• DT&E Certification Reqt

• Joint Testing, Capabilities & Infrastructure

• Targets Oversight• Modeling & Simulation• Acquisition System Safety

CORE COMPETENCIES

• SWE and SA Policy• Guidance

• SoS, SA Guides• Education and Training

• DAU SW Acq Curriculum• Continuous Learning

Modules for SW, SoS, SA• Software Engineering

• Acquisition Support• Software Engineering

Institute (SEI)• Process Improvement

• CMMI• DoD/National Software

Investment Strategy• Industrial Base capability

CORE COMPETENCIES

• Support of ACAT I and other special interest programs (MDAP, MAIS)

• Assessment Methodology (Defense Acquisition Program Support – DAPS)

• T&E Oversight and Assessment of Operational Test Readiness (AOTR)

• SE/T&E Review of Defense Acquisition Executive Summary Assessments (DAES)

• Lean/6-Sigma Training/Cert

Acquisition program excellence through sound systems and software engineering

Systems & Software EngineeringOrganizational Core Competencies

DoD AT&L Workforce

Career Fields (13) ARMY NAVY/USMC AIR FORCE OTHER TOTAL

Auditing 2 3,485 3,487

Business, Cost Estimating, & Financial Management

4,171 1,716 1,503 218 7,608

Contracting 10,042 5,018 7,371 5,314 27,745

Facilities Engineering 441 3,477 9 3,927

Industrial/Contract Property Management125 56 27 321 529

Information Technology 2,733 744 1,116 248 4,841

Life Cycle Logistics 6,319 4,154 1,781 76 12,330

Production, Quality, and Manufacturing2,193 2,000 334 4,439 8,966

Program Management 4,475 3,624 3,958 717 12,774

Purchasing 332 555 123 667 1,677

SPRDE - S & T Manager 171 19 100 290

SPRDE - Systems Engineering 11,964 16,690 6,239 253 35,146

Test and Evaluation (5.6%) 2,140 2,446 2,598 94 7,278

Unknown 506 6 1,132 1,644

Total45,443 40,651 25,075 17,073 128,242

Less Guidance and Oversight:Performance SpecsSOOPrime Contractor is the Integrator (LSI)

More Guidance and Oversight:MILSPECSSOWGovernment is the Integrator

SE

Emphasis

2008

Auth. Bill

Government

Oversight Pendulum

Government

Oversight Pendulum

Bill Would Bar Contractors from Running Defense Programs

• Main Provisions from the 2008 Defense Authorization Bill Passed by the HASC– No more Lead System Integrators (LSIs) after 2011– DoD directed to study its acquisition workforce and

identify and fill gaps in skills needed to effectively manage programs

– Current LSI managed programs not effected unless opened up to competitive bid

Current Programs Managed byLead System Integrators

• Ground Based Midcourse Defense – Boeing

• Future Combat System– Boeing/SAIC

• Littoral Combat Ship– Lockheed Martin

• Coast Guard Deepwater Modernization– Lockheed Martin/Northrop Grumman

JCIDS PublicationsMay 1, 2007

• CJCSI 3170.01F (JCIDS)• CJCSM 3170.01C (Documents)• Key Performance Parameters

– Force Protection & Survivability (Req)– Sustainment (Materiel Availability)(Req)– NR-KPP (Req)– System Training (Selectively Applied)– Energy Efficiency (Selectively Applied)

Key System Attribute(KSA)

• Those system attributes considered most critical or essential for an effective military capability but not selected as a KPP– KSAs provide decision makers with an

additional level of capability prioritization below the KPP

• But with senior sponsor leadership control– Generally 4-star, Defense Agency Commander or

Principal Staff Assistant

A View From the Top

• We tend to chase technology

• 90% of the work is in the first 10% of System Development

• The Dir, DOT&E wants DOT&E more involved with DT where he feels we have a lack of

government oversight.

Defense Science Board Task Force on DT&E

• AT&L memo, April 30, 2007, request– Examine…

• OSD organization, roles & repsonsibilities for T&E oversight (compare DT&E to OT&E; changes?)

• Changes required to establish statutory authority for OSD DT&E oversight

• Recommend improvements in DT&E process to discover suitability problems early

• Mr Charles Adolph – Chair the Task Force – Mr Chris DiPetto (ODUSD (A&T) and Dr Ernest

Seglie (ODOT&E), Executive Secretaries

Reliability Availability and Maintainability (RAM) Findings

• Acquisition Reform implementation had a detrimental impact on RAM practices

– With some exceptions, the practice of reliability growth methodologies was discontinued during System Design and Development (SDD) and deferred until production

– Relevant military specifications and standards were cancelled and not, in all cases, replaced with industry standards

– The technical/managerial workforce was reduced in most government program offices and test organizations

• RAM shortfalls are frequently identified during DT, but program constraints (schedule and funding) often preclude incorporating fixes and delaying IOT&E

• In some instances, programs had such serious RAM concerns that they were precluded from proceeding to production until the problems could be corrected. Improved RAM decreases life cycle costs and reduces demand on the logistics system

Impact of Low Reliability on Life Cycle Cost (LCC)

• Recent Logistics Management Institute (LMI) and Army studiesinvestigated relationship between reliability and Life Cycle Cost (LCC). Both studies concluded that reliability investment during design and development reduces support cost

– LMI study concludes that:• 30-60% reduction in support cost is achievable• Reacting to lower reliability after IOC (vice during development) results in significant

support cost increases

• OSD CAIG study shows that support costs are two-thirds or more of total system cost

– DoD should address acquisition and LCC strategies concurrently • Support cost bill payers should participate in requirements definition and trade studies

during development

Upfront Investment in Robust RAM Effort EmphasizingUpfront Investment in Robust RAM Effort EmphasizingReliability Growth Reduces Life Cycle CostsReliability Growth Reduces Life Cycle Costs

Upfront Investment in Robust RAM Effort EmphasizingUpfront Investment in Robust RAM Effort EmphasizingReliability Growth Reduces Life Cycle CostsReliability Growth Reduces Life Cycle Costs

RAM Recommendations

The single most important step necessary to correct high suitability failure rates is to ensure programs are formulated to execute a viable systems engineeringstrategy from the beginning, including a robust RAM program, as an integral partof design and development. No amount of testing will compensate fordeficiencies in RAM program formulation. To this end, the following RAM-relatedactions are required as a minimum:

• Identify and define RAM requirements during the Joint Capabilities Integration Development System (JCIDS) process and incorporate them into the Request for Proposal (RFP) and as a mandatory contractual requirement

• During source selection, evaluate the bidders’ approaches to satisfying RAM requirements

– Ensure flow-down of RAM requirements to subcontractors• Make RAM, to include a robust reliability growth program, a mandatory contractual

requirement and document progress as a part of every major program review • Develop a military standard for consistent RAM development and testing that can

be readily referenced in future DoD contracts• Ensure an adequate cadre of experienced RAM personnel are part of the Service

acquisition and engineering office staffs

Roles and Responsibilities of Government T&E Organizations Findings

The aggregate lack of process guidance due to the elimination of specificationsand standards, massive workforce reductions in acquisition and test personnel,and the high retirement rate of the most experienced technical and managerialpersonnel in government and industry has a major negative impact on DoD'sability to successfully execute increasingly complex acquisition programs.

• Major personnel reductions have strained the pool of experiencedgovernment test personnel

• A significant amount of developmental testing is currently performedwithout the needed degree of government involvement or oversight and in some cases, with limited government access to contractor data

• The current trend away from governmental involvement in DT makes operationally oriented testing in some areas during development even more difficult to accomplish

Roles and Responsibilities of Government T&E Organizations Recommendations

As a minimum, government test organizations should retaina cadre of T&E personnel to perform the followingfunctions:

• Participate in the translation of operational requirements into contract specifications, and in the source selection process, including RFP preparation

• Participate in developmental test and evaluation planning including TEMP preparation and approval

• Participate in the technical review process including SRR, PDR,CDR, TRR and FRR

• Participate in test conduct, data analysis, and evaluation and reporting, with emphasis on analysis and reporting

• Capture and re-apply best practices and lessons learned

Integrated Test and EvaluationFindings

• Service acquisition programs are incorporating integrated testing to a limited degree through varying approaches – Army has integrated DT and OT organizations into one

command – Navy utilizes a full-spectrum RDT&E approach to conducting

Test & Evaluation– Air Force employs Combined Test Force concept which

consolidates test execution• Additional emphasis on integrated testing can result in greater

T&E process efficiency and program cost reductions

Integrated Test and EvaluationRecommendations

• Implement OSD policy change mandating integrated DT&E and OT&E planning and execution throughout the program

– Require sharing and access to all appropriate system-level and selected component-level test and model data by government DT and OT organizations as well as the prime contractor, where appropriate

– Incorporate data access requirements in contract– Integrate test events, where practical, to satisfy OT and DT requirements– Define which testing will be accomplished by the prime contractor, government

DT lead, and OT as the lead agency prior to award of contract– Require an operational evaluation framework as a part of the Milestone B TEMP

• Make available a cadre of operational personnel to support DT for Acquisition Category (ACAT I) and special interest programs, as a minimum

• Better integrate Operational Test Agencies (OTAs) into the DR process to include participation on Joint Reliability Maintainability Evaluation Team (JRMET) or Corrective Action Review Board throughout DT

Operational Test Readiness Review (OTRR)Findings

• DoDI 5000.2 requires that “the Service Acquisition Executive (SAE) shall evaluate and determine materiel system readiness for IOT&E" – For all Services, the OTRR process is an in-depth review of readiness – Decision authority is frequently delegated to the Program Executive

Officer (PEO)

• Shortcomings in system performance, suitability, and RAM are usually identified during the OTRR

• In most cases, the operational test readiness certifying authority is aware of

the risk of not meeting OT criteria when major shortcomings exist

• Because of the funding constraints, the low priority given to sustainment, as well as the urgency in recent years to get new capabilities to the Warfighter, major suitability shortcomings have rarely delayed the commencement of dedicated IOT&E

OTRR Recommendations

• Conduct periodic operational assessments to evaluate progress potential for achieving pre-determined entrance criteria for operational test events

• Conduct an independent AOTR prior to the OTRR (included in latest draft DODI 5000.2) for oversight programs

– Results of the AOTR should be briefed at the OTRR• Include a detailed RAM template in preparation for the OTRR• CAE submit a report to OSD that provides the rationale for the readiness decision

– Include an evaluation of weapon system’s capabilities against Critical Operational Issues (COIs)

– Certify that the DT evaluation is complete and indicates acceptable risk of passing OT

– Include an explanation for recommending go-ahead when there is a reasonable

expectation that the system is not effective and/or suitable

DT&E Task Force Strongly Endorses Recent AT&L/DOT&E DT&E Task Force Strongly Endorses Recent AT&L/DOT&E

Initiative to Strengthen OSD OTRR ProcessInitiative to Strengthen OSD OTRR ProcessDT&E Task Force Strongly Endorses Recent AT&L/DOT&E DT&E Task Force Strongly Endorses Recent AT&L/DOT&E

Initiative to Strengthen OSD OTRR ProcessInitiative to Strengthen OSD OTRR Process

OSD DT&E Organization Recommendations

• The most important change required for improved OSD DT oversight is to is to reestablish a recognizable, focused, integrated, and robust DT organization within AT&L

• Consolidate DT-related functions in AT&L *– Program oversight and policy, and Foreign Comparative Test– Have Director, DT&E directly report to DUSD(A&T)– Restore TEMP approval authority to Director, DT&E

• Integrate Test Resource Management Center (TRMC) activities early into DT program planning

*Three Task Force members out of fourteen voted against consolidation

• DUSD(A&T) and DOT&E memo signed February 15, 2008 requests CAE support to establish a working group to implement recommendations to improve RAM on DoD weapon systems during system development

• Specific Tasks: Ensure execution of a viable SE strategy ,

including RAM growth program, as an integral part of design and development

Ensure government orgs reconstitute cadre of experienced T&E and RAM personnel

Implement mandated Integrated DT and OT, ensure access to all appropriate data (contractor and government) and conduct T&E in an operationally representative environment as early as possible

Reliability Improvement Working Group

The entire DSB report is available at:

http://www.acq.osd.mil/ dsb/reports/2008-05-DTE.pdf

Continuous Process Improvement (CPI)

• Establishment of AT&L Continuous Process Improvement (CPI) Program

– To help optimize our organizational performance

• To establish a basic understanding of various CPI principles and roles, all AT&L personnel had to complete CLE 015 (CPI Continuous Learning Module) before March 30, 2007.

Prototyping & Competition

• Many programs initiated with inadequate technology maturity– Program decisions based on paper proposals

• There is a need for & benefits of quality prototyping– Issues need to be discovered before SD&D phase

• Acquisition strategies requiring AT&L approval must be formulated to include competitive, technically mature prototyping through Milestone B

Resources

• OSD SSE: http://www.acq.osd.mil/sse/• OSD SE Pubs:

http://www.acq.osd.mil/sse/guidance.html• ASSIST: http://assist.daps.dla.mil/online/start/• NAVAIR Tech Reviews:

http://www.navair.navy.mil/kms/41g/• Army LSS:

http://www.amc.army.mil/pa/LeanSixSigma.asp• USAF SE: http://www.afit.edu/cse/

Resources (cont)

• DAU: www.dau.mil• DAU SE: https://acc.dau.mil/se• DAG Chapter 4:

https://akss.dau.mil/dag/Guidebook/IG_c4.0.asp• DAU Cont Learning: http://clc.dau.mil/• DAU Training: http://training.dau.mil/• AMRDEC:

http://www.redstone.army.mil/amrdec/

Questions?

Wallace J. “Wally” Tubell

Professor of Engineering

DAU South

256-722-1081

[email protected]

Briefing for INCOSE Learn. Perform. Succeed. Rev. Oct 07.

Documents

certification level

details level

evaluation level

modules online modules

dau catalog

online training

prerequisite level

details p