Contract Price & Schedule Database Briefing Sponsors: Mr. Scott Adamson, AFCAA Mr. Duncan Thomas, NCCA Brian Octeau, Technomics, Inc. DODCAS 2012.

Contract Price & Schedule DatabaseBriefing

Sponsors:Mr. Scott Adamson, AFCAAMr. Duncan Thomas, NCCA

Brian Octeau, Technomics, Inc.DODCAS 2012

Agenda

• Overview/Background• Process• Database Composition/Functionality• Example High Level Growth Factors• CLIN Level Database• Future Efforts

Slide 2

Why Collect Contract Data?1. Contractor Cost Data Reports (CCDRs) and

Contract Performance Reports (CPRs) are limited to major programs and/or large contracts.

– Most programs consist of multiple contracts, not all of which have cost reporting.

2. Contracts provide additional information not found in other data reports.

– Examples: period of performance start date/end dates by item; SOWs; traceability to funding origin; finer price and quantity detail (depending on CLIN structure).

3. Provides insight to reasons for changes over time – valuable for conducting root cause analysis.

4. Provides means for measuring growth over time.5. Provides a cross-check with other data sources.

Slide 3

Data – Quality of Data Types

• Actuals for completed programs/contracts

are better than• Estimates at Complete (EACs) for

contracts greater than 90% completeare better than

Contract line item prices from Schedule [sic] B

are better than• Historical budget data

Slide 4

Source:SCEA CEBoK, Unit II -

Module 4, Data Collection and Normalization

Good Estimates Consider All Data Sources

Slide 5

CFSRs

CCDRs

CPRs

Contracts

Budget Docs

Accounting Data

Comprehensiv

e Cost

Estimate

Example

Slide 6

DO

LLAR

S

DATE

PROGRAM TIMELINE

CONTRACT VALUE CCDR AT COMPLETION CUM BUDGET (SAR)

Example Graph of Cumulative Cost/EAC

Slide 7

DO

LLAR

S

DATE

PROGRAM TIMELINE

CONTRACT VALUE CCDR AT COMPLETION CUM BUDGET (SAR)

Difference due to: government

costs

Difference due to:1. Contract Value (Blue Line)

includes costs that are incurred below threshold value

2. Systems Engineering and Technical Assistance (SETA)

3. Indefinite Delivery/Indefinite Quantity (IDIQ)

4. Waivers

CCDRs not reported on all contracts and data lags contract data

Contract Data – Potential Applications

1. Development of Cost Estimates Cost growth metrics Data Completeness Data Fidelity Cross checks

2. Program assessments / Root Cause Analyses What occurred when and for how much Traceability to CPR and/or CCDR if available (aids in

data verification/validation & augmenting of data for below-threshold (non)-reporting)

3. Support special studies e.g., Platform Integration, qualification testing, learning

curves, award fees, etc.

Slide 8

Slide 9

Contract Price & Schedule Database

• Outlines CONTRACTS by program and phase

• Provides final Contract Line Item Number (CLIN) PRICES AND CLIN descriptions for each contract

• Provides modifications, modification DESCRIPTIONS, and price changes by CLIN for each modification

• Provides QUANTITY information by CLIN and quantity changes by modification

• Provides SCHEDULE period of performance information by CLIN, and period of performance changes by modification

A comprehensive repository of contract data

Provides valuable insight into programs

Process

Contracts

EDA Database

Contract Price & Schedule Database

Track Cost, Quantity and

Schedule data Across CLINs And

Modifications

Filter “Value Change” ModificationsFrom “Admin” Modifications

Categorize “Value Change” Modifications

Contract Data Loader

(Raw Data)

(Nor

mal

ized

Dat

a)

Identify programs and contracts(Contract Announcements, etc.)

Slide 10

Normalized CLIN Database

Data Analyses• Growth Rate

Analysis• CLIN

relationships• “Root Cause”

analyses

Data Analysis Tools & Queries

Ground Rules/Assumptions/Terminology

• Delivery Orders are treated as separate contracts• Contract values “updated with actuals” for completed contracts

(criterion: >1 year without contract activity)• Criteria for categorization of modifications:

Slide 11

Modification Category

Criteria for Categorization

Initial Award Always the Basic contract

Administrative No change to the overall contract price; typically a revision.Ex: Changes to SOW, Schedule, CLIN Descriptions, Funding information

Baseline Anticipated scope changes and exercising of anticipated options; FMS.Ex: New lot buys, Definitizations of initial award, Congressionally-mandated changes

Technical Unanticipated changes through design or scope.Ex: ECPs, Definitizations of technical scope; Exercising of unanticipated options

Schedule Value changes attributable to schedule compression/extension

Cost Unanticipated changes without a change in scope or design.Ex: Cost overruns/underruns, unanticipated rate changes (EPAs), de-obligation or obligation of funds, ceiling increases, unexplained price changes

Unknown Used when a value changed is inferred but the modification is missing

Data Validation Checks• More than 100 slides, 2 training videos, and

regular meetings among analyst team to ensure comprehensive and consistent data entry; documentation of business rules

• Access-based relational data structure, complying with 3rd normal form of relational database management system (RDBMS), ensures data integrity

• Random spot-checks/audits of high profile contracts

• Complete “A-Z” independent reviews conducted of key contracts (capturing 92% of development/production value and all contracts identified in Selected Acquisition Reports (SARs) for F/A-18E/F) Slide 12

DATABASE

Slide 13

Database Composition

Slide 14

Database Update – Air Force

Slide 15

Commodity Air Force Program(s) in Database*

Aircraft Galaxy (C-5), Globemaster III (C-17), Hercules (C-130), Raptor (F-22), Fighting Falcon (F-16), Osprey (V-22), Sentry (E-3A), Spirit (B-2) Stratolifter (KC-135)

Decoys MALD (ADM-160)

Electronics ASIP, LAIRCM

Engine F-119 Engine

Laser Advanced Tactical Laser (ATL)

Missiles ACM (AGM-129), AGM-130, ALCM/CALCM (AGM-86A-D), AMRAAM (AIM-120), CHAMP, DRADM, HAVE NAP (AGM-142), JASSM (AGM-158), Maverick (AGM-65)

Munitions Bunker Buster (GBU-28), GBU-15, JDAM, Paveway II (GBU-12), Paveway III (GBU-24), SDB (GVU-39), SFW (CBU-97), WCMD

Non-Lethal Active Denial System (ADS)

Space AEHF

Targets/Drones AST (QF-16), Firebee (BQM-34), FSAT (QF-4), Skeeter (BQM-167)

UAV Global Hawk (RQ-4), Predator (MQ-1), Reaper (MQ-9)*Primary data source from EDA: 1995 - 2010**Bold indicates program with significant amount of data in the contracts database

Database Update – Navy & Army

Slide 16

Commodity Navy Program(s) in Database*Aircraft Super Hornet (F/A-18E/F), Lightning II (F-35), Poseidon (P-8A),

Seahawk (SH-60/HH-60H/MH-60)Clipper (C-40), Goshawk (T-45), Growler (EA-18G), Harrier II (AV-8B), Hawkeye (E-2), Hornet (F/A-18A-D), Mercury (E-6), Sea Stallion (CH-53E),

Decoys ITALD (ADM-141C)

Engine F-135 Engine, F-136 Engine

Gun CIWS

Missiles AARGM (AGM-88E), AWS, ESSM (RIM-162), HARM (AGM-88), Harpoon (AGM-84)/Slammer, JSOW (AGM – 154A, A/C, C), RAM BLK 0,1,2 (RIM-116/A/B), RATTLRS, Sea Sparrow, Sidewinder (AIM-9), SLAM ER, SM-1,2,3,4,6, Sparrow (AIM-7), Standard Missile, Tomahawk Block III & IV (BGM-109), VLA Missile (RUR-5 ASROC)

Munitions BTERM, ERGM (EX-171)

Targets/Drones CHUKAR (BQM-74), COYOTE (GQM-163), Jayhawk (AQM-37), MSST (GQM-173)

UAV BAMS (MQ-4C)Commodity Army Program(s) in Database*

Aircraft Blackhawk (UH-60L/UH-60M/HH-60M), Cobra (AH-1)

Missiles ATACMS (MGM-140), JAGM, JCM (AGM-169), PAC-3 (MIM-104F), Patriot (MIM-104A-D), SLAMRAAM*Primary data source from EDA: 1995 – 2011**Bold indicates program with significant amount of data in the contracts database

Database Update

Slide 17

Commodity Number of Programs Per Service in the Database*

Aircraft

Decoys

Electronics

Engine

Laser

Gun

Missiles

Munitions

Non-Lethal

Space

Targets/Drones

UAV

Air Force Navy Army

9 13 2

1 1

2

1 2

1

1

9 28 6

8 2

1

1

4 4

3 1

*Primary data source from EDA: 1995 – 2011

Interactive Database and Analysis Tools Overview

• Growth Factors & related statistics– By contract– By program

• Histograms and time series (cumulative value) charts

• Root cause information• Allows user to filter data sets and perform

exploratory data analysis

Slide 18

Database Functionality: Growth Factors

Slide 19

Database Functionality: Growth Factors

Slide 20

Database Functionality: Annotated Plots

Slide 21

Key Database Functionality

• Contract Info: Refers to individual contracts

• Contract Growth Factors– Refers to groups of contracts (e.g. Air Force missiles)– Raw and weighted growth factors and associated statistics– Data filters (Service, Phase, Commodity, Contract Type, Contractor)

• Program Growth Factors– Refers to all contracts associated with a particular program– Raw growth factors– Data tables (CLINs, Modifications, Quantities, Period of Performance)– Charts (CLIN, Contract value over time)– Charts (growth category histograms, contract bar chart)

• By-Program Analysis– Accumulates all contract data across the selected program– Total growth statistics by category– Program growth chart (milestone and data point label capability)

• Search: Searchable data warehouse: search by keyword, program, contract, etc.

Slide 22

Calculating Growth Factors

• Averaging growth across multiple contracts can result in skewed results because each data point is weighted equally.– e.g., very high growth experienced on a very

small contract is treated the same as moderate growth on a very large contract.

• To counter this effect, dollar weighted factors are also calculated.

Slide 23

Global Growth Factors

Slide 24

Development Production O&S0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

14.00%

16.00%

18.00%

20.00%

Global Growth Factors of AircraftWeighted Mean – Total Baseline

CostScheduleTechnical

Global Growth Factors

Slide 25

Development Production O&S-0.50%

1.50%

3.50%

5.50%

7.50%

9.50%

11.50%

Global Growth Factors of MissilesWeighted Mean – Total Baseline

CostScheduleTechnical

CLIN LEVEL DATABASE

Slide 26

Objective

• Compile all CLINs across all contracts by ACAT I program

• Normalize CLINs– Map CLINs to a standard WBS structure– Develop logical/common groupings of

CLINs• Analyze the normalized data within a

program and across programs to develop:– Growth factors– Estimating factors– Learning curves

Slide 27

Background• CLIN analysis can result in more

homogeneous datasets than analysis at the contract level.– Contract scope can often be a mix of mismatched,

yet related efforts

– Mapping issues can arise when CLIN effort spans multiple WBS elements

Slide 28

Contract A Contract B

CLIN 1 AUR Missile Production

CATM Production

CLIN 2 Seeker Spares AUR Missile Production

CLIN 3 Logistics Support Platform Integration

CLIN 4 Missile Test Set Production

Control Section Repairs

CLIN 5 Platform Integration Improved laser guidance study

Approach – Compile CLIN Data• CLIN Database Fields

– CLIN Number, CLIN Description– Program– Contract Number, Contract Description– Phase (Development, Production, O&S)– Production Lot (Lot 1, 2, 3, etc.)– CLIN Type (FFP, CPFF, T&M, etc.)– Primary CLIN Funding Appropriation (e.g. 3600,

3020, 3400, etc.)

Slide 29

Approach - Normalization• CLINs are mapped to program-specific, logical groupings that are based

on the commodity-specific MIL-HDBK-881 WBS (Missile System example shown)

Slide 30

CLIN Groupings

Slide 31

EXAMPLE: Missile Program

TOTAL CLINs: 245TOTAL LEVEL 2 CATEGORIES: 47

LEVEL 1 LEVEL 2 COUNTPME Air Vehicle (AUR) 32

Command and Launch Software 14Control Unit 8GPS 9Guidance Unit 2Containers 2Safe Arming Fuze 2Telemetry 2Airframe 3Engine 4Fuze 2Payload 1Weapon Data Link 2Wing 3Instrumentation 15

System Dev. EMD 4Risk Reduction 5P3I 1PDRR 3

ST&E Simulator 6Test Support 19Test Vehicle 3Flight Test Missile 2Ground Testing 4Instrumented Test Vehicle 1Reliability Test Vehicles 3Separation Test Vehicle 3Simulation Vehicle 1

LEVEL 1 LEVEL 2 COUNTIntegration Aircraft Integration Support 9Mfg Eng Production Support 3

Production Verification 2SE/PM Proposal Preparation 12

Systems Engineering 12Technical Support 7Support 3Ops Safety, Suitability & Effectiveness (OSS&E) Program6Program Management 1Reliability Program 11Weapon System Eval. Program 2

Training Dummy Air Training Missile (DATM) 4Training 1

Data Data 3Spares Obsolete Parts 3Other Fee 2

Asset Management Program 1? 5Other - Contractor Costs 2

CLIN Groupings (continued)

Slide 32

Common

Unique

Common

Unique

MISSILES

MUNITIONS

Munitions (AUR) 18 Munition (AUR) 45Payload 3 Payload 9Command and Launch Software 4 Command and Launch Software 1Fuze 1 Fuze 4Telemetry 5 Altimeter 7Wings 14 Bomblets 16

Control Unit 1Release mechanism 1Propulsion 1Canister 3Cable 3Target Detecting Device 3Upper Housings 3

TOTAL CLINs 45 TOTAL CLINs 97

PROGRAM C PROGRAM DSummary of Common CLIN Mapping (Prime Mission Equipment)

Air Vehicle (AUR) 32 Air Vehicle (AUR) 96Command and Launch Software 14 Command and Launch Software 19Control Unit 8 Control Unit 1GPS 9 GPS 1Guidance Unit 2 Guidance Unit 12Containers 2 Containers 2Safe Arming Fuze 2 Safe Arming Fuze 2Telemetry 2 Telemetry 3Airframe 3 Processor 4Engine 4 Rocket Motor 17Fuze 2 Electronic Counter Measures 3Payload 1 Booster 3Weapon Data Link 2 Instrumentation Kit 95Wing 3 Captive Equipment 2Telemetry 15

TOTAL CLINs 101 TOTAL CLINs 260

Summary of Common CLIN Mapping (Prime Mission Equipment)PROGRAM A PROGRAM B

CLIN Database Composition

Slide 33

USN3

USAF7

11 Programs (ACAT 1)

7,756 Data Points

175 Contracts(71 Contract Vehicles)

1,527 Contracts(142 Contract Vehicles)

- 734 Development CLINs - 409 Production CLINs - 47 Sustainment CLINs

- 244 Development CLINs - 7,120 Production CLINs - 392 Sustainment CLINs

PRODUCTION DATA

4,352 Data Points

1,421 Contracts(59 Contract Vehicles)

- 26 Development CLINs - 46 Production CLINs - 4,280 Sustainment CLINs

O&S DATA

1,190 Data Points

USA1

DEVELOPMENT DATA

Future Efforts• Add additional programs based on prioritized listings• CLIN-level database expansion

– Allows for assignment of programs at the CLIN level, rather than relying upon plurality of dollars

• Extraction of Section G funding data– Provides insight into time phasing

• Separate treatment of Foreign Military Sales (FMS)• Dynamic definition of various criteria (e.g.,

“completed” contracts)• Develop multivariate relationships for estimating

contract growth• Migration to .NET environment

– Eliminates configuration control issues– Provides ‘real-time’ access to new data– Facilitates access to associated pdf file (mod or contract) via

hyperlink– Facilitates user control/management (e.g., full vs. limited

access privileges)

Slide 34