NATO UNCLASSIFIED Releasable to PFP 22 May 2018 DOCUMENT NIAG-D(2018)0013 AC/327-D(2018)0005 NATO UNCLASSIFIED -1- NATO INDUSTRIAL ADVISORY GROUP (NIAG) LIFE CYCLE MANAGEMENT GROUP (LCMG) (AC/327) FINAL REPORT OF NIAG STUDY GROUP 218 - NIAG STUDY ON CONCEPTS AND RATIONALE FOR CONTRACTING FOR LOGISTICS CAPABILITY IN NATO ARMAMENTS SUPPORT PROGRAMMES Note by the NIAG Secretary 1. Enclosed is the Final Report on Concepts and Rationale for Contracting for Logistics Capability in NATO Armaments Support Programmes, as conducted by NIAG SG.218, which is now published to the sponsor. (signed) Nathalie Van Donghen 1 Enclosure Original: English NHQD99273
161
Embed
NATO INDUSTRIAL ADVISORY GROUP (NIAG) LIFE CYCLE ...2018)0… · nato industrial advisory group (niag) life cycle management group (lcmg) (ac/327) final report of niag study group
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
NATO UNCLASSIFIED Releasable to PFP
22 May 2018 DOCUMENT NIAG-D(2018)0013
AC/327-D(2018)0005
NATO UNCLASSIFIED -1-
NATO INDUSTRIAL ADVISORY GROUP (NIAG)
LIFE CYCLE MANAGEMENT GROUP (LCMG) (AC/327)
FINAL REPORT OF NIAG STUDY GROUP 218 -
NIAG STUDY ON CONCEPTS AND RATIONALE FOR CONTRACTING FOR
LOGISTICS CAPABILITY IN NATO ARMAMENTS SUPPORT PROGRAMMES
Note by the NIAG Secretary
1. Enclosed is the Final Report on Concepts and Rationale for Contracting for Logistics Capability in NATO Armaments Support Programmes, as conducted by NIAG SG.218, which is now published to the sponsor.
(signed) Nathalie Van Donghen
1 Enclosure Original: English
NHQD99273
NATO UNCLASSIFIED
Releasable to PFP
ENCLOSURE TO
NIAG-D(2018)0013, MULTI REF
NATO UNCLASSIFIED
Releasable to PFP
NIAG SG218
ON
CONCEPTS AND RATIONALE FOR CONTRACTING FOR
LOGISTICS CAPABILITY ON NATO ARMAMENTS AND
SUPPORT PROGRAMS
(Main Document)
The work described in this report was carried out under the provisions of the NIAG
Study Order for Study Group 218.
Disclosure, utilization, publication or reproduction of this report by industry is subject
to pre-approval by NATO until such time as NATO may have released such work to the
public
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-2
1 EXECUTIVE SUMMARY
This report responds to the requirements of the NATO Study Order to “provide analysis, recommendations, and tools for NATO to effectively exploit outcome-based contracting”.
Commercial entities such as airlines and various Nations’ military organizations have employed outcome-based contracting with varying degrees of success. The beginnings of outcome-based sustainment have been attributed to the commercial aviation industry. The Rolls Royce company pioneered the concept of selling propulsion to aircraft, rather than selling engines, spare parts and repairs – Power by the Hour™ and has since expanded its concept to a number of Nation’s military forces. As well, other similar arrangements have been made between industry and Nations to provide logistics as an outcome-based service. Some examples include aerial refuelling in the U.K., and pilot training in Singapore. In both cases, the government has avoided the cost of investing in infrastructure, training and operations, by having industry provide the desired outcome (refuelled aircraft or trained pilots), at a cost less than the government could have done organically.
The evidence observed during the Study demonstrates that well-structured and defined OBCs will reduce total costs, increase system availability for the war-fighter, optimize industry performance (investments and costs) to deliver an improved outcome, i.e. a win-win for government and industry.
The most important findings of the Study are, that OBC processes have the same value for NATO as traditional contracting processes and the overarching recommendation is, that NATO fully adopts and implements OBC processes.
To implement OBC, it is recommended:
• NATO should develop a high level policy for contracting for systems and logistics services. This policy should cover all kinds of contracts (traditional contracting and outcome-based contracting e.g. for capability). This policy has to be in line with the NATO Policy for Systems Life Cycle Management (SLCM) approved by the North Atlantic Council and signed by the Secretary General in 2006.
• NATO should enter into OBC arrangements starting with two pilot programs, utilizing the Decision Support Tool developed as part of this Study. This trial should be performed by NSPA and should be limited to a defined period of time (e.g. 2 years).
• NATO should review and update the NATO publications on contracting including all aspects of contracting – traditional and OBC- based ones. Furthermore it is recommended to specify in the relevant NATO documentation at which point in the life-cycle of the system the responsible manager should evaluate if OBC contracting is preferred to traditional contracting
The Study provides 15 more recommendations e.g. for modification of NATO contracting regulations, doctrine, and sample contract language for NATO publications. An evaluation method and a decision support tool were developed to support new and legacy
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-3
NATO programmes and programme managers in determining the optimal combination of organic military and industry supported life cycle support for use at any stage of a programme.
6.1 Appendix A – Programmes Reviewed / Other Studies Leveraged ........................................... 35
6.2 Appendix B – OBC Arrangement General Characteristics ....................................................... 37
6.3 Appendix C – Case Studies Supporting Results / Recommendations ....................................... 38
6.4 Appendix D – Sample language ............................................................................................. 42 6.4.1 Definitions ................................................................................................................................................. 42 6.4.2 Found Sources and Suggestions for Sample Language ............................................................................. 43
6.5 Appendix E – Decision Support Tool ...................................................................................... 62 6.5.1 Parameters List ......................................................................................................................................... 62 6.5.2 Definitions of the parameters ................................................................................................................... 63 6.5.3 Parameter list development ..................................................................................................................... 65
6.6 Appendix F – Reference Documents ...................................................................................... 67
ANNEX I GLOSSARY ............................................................................................................. 69
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-5
ANNEX II Development Tool and User Manual ...................................................................... 88
General logic for development of the tool ......................................................................................... 88
Considerations for implementation .................................................................................................. 89 Consideration for a Decision Tree ........................................................................................................................... 90 Consideration for a decision support tool ............................................................................................................... 90
General Process when to identify parameter values ........................................................................ 103 Identification of the key players............................................................................................................................ 104 Identification of the Key documents ..................................................................................................................... 105 Identification of the Key set of parameters .......................................................................................................... 106 Key set of parameters ........................................................................................................................................... 107 Set of Parameters 1 (SoP1), ambition/objective ................................................................................................... 108 Set of Parameters 2 (SoP2), ambition/objective ................................................................................................... 109 Set of Parameters 3 (SoP3), ambition/objective ................................................................................................... 110 Set of Parameters 4 (SoP4), ambition/objective ................................................................................................... 111 General exclusions ................................................................................................................................................ 111
Tool Description / Handling of the tool ........................................................................................... 112 Register Main ........................................................................................................................................................ 113 Register Parameters .............................................................................................................................................. 115 Register Evaluation: .............................................................................................................................................. 116
Tests / Test cases and results .......................................................................................................... 118 Testcase UK Sentry Project: .................................................................................................................................. 118 Testcase UK Aerial Refuelling: ............................................................................................................................... 118
Description of single parameter values: .......................................................................................... 119
ANNEX III DECISION SUPPORT TOOL .................................................................................... 121
ANNEX IV FINAL PROOF POINT............................................................................................. 123
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-6
2 LIST OF PARTICIPANTS
Country Name Role Company
USA Mark Harnitchek Chairman
– till 29/06/2017
The Boeing Company
Germany Peter Janatschek Chairman
– from 29/06/2017
Peter Janatschek Projekte &
Logistik
Spain Juan Carlos Santos
Santos
Vice-Chairman AIRBUS DEFENCE & SPACE
Belgium Gustavo Scotti di Uccio Rapporteur AOS
Country Name Company
BEL Elena Manuela Tudosia PoleCM
BEL Nickola Carlomagno Vitrociset Belgium
CHZ David Hác STV Group a.s.
DEN Per Tony Frederiksen A/S Hydrema Export
DEN Christian Brochmann Jorgensen Nordic Camp Supply Fuel
DEN Jens Praestegaard Nordic Camp Supply Fuel
FR Jean-Charles Boulat Naval Group (ex DCNS)
FR Simon Payet Naval Group (ex DCNS)
FR Lionel Bourlard MBDA France
FR Louis Vergniolle de Chantal THALES International
FR Véronique Robineau THALES SA
GER Kurt Haeusler Aviation Consulting
GER Gabriel Jelodin ESG Elektroniksystem und Logistik GmbH
GER Wolfgang Mack MBDA Deutschland GmbH
GER Reinhold Ziegler MBDA Deutschland GmbH
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-7
GER Andreas Fenkner T-Systems International GmbH
GER Andreas Kirchhofer T-Systems International GmbH
ITA Giovanni Murettino Aerea SpA
ITA Luca Rancati Aerea SpA
ITA Gianluca Bove SIMAV
ITA Icilio Di Luzio SIMAV
ITA Attilio Perego UEE Italia S.r.l (EXPAL Systems Group)
ITA Luigi Ciolli Vitrociset
TUR Cengiz Oflaz ALTAY Information Tech. Def.Industrial Inc
TUR Alper Sueri TUBITAK SAGE
TUR Banu Tekşen TUBITAK SAGE
TUR Remzi Yüksekbulgu Turkish Aerospace Industries (TAI)
TUR Ahmet Kavcar Turkish Aerospace Industries (TAI)
UK Nigel Robert Hunter LEONARDO UK
UKR Ganna Pavlova SE KhMP "FED"
USA Joseph Spruill Lockheed Martin Corporation
LIST OF SG218 Quick Reaction Team (QRT)
Name Organization
Giuseppe Rampini ACO/SHAPE – LCMG-WG3
Steven Larcher Canadian Department of National Defence
Pierre Demers Canadian Department of National Defence
Andreas Keller Federal Office of Bundeswehr Equipment, Information
Technology and In-Service Support (BAAINBw)
Michail Bozoudis MOD GRE – LCMG-WG3
Alain Courtois NCIA
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-8
Stefan Limburg Federal Office of Bundeswehr Equipment, Information
Technology and In-Service Support (BAAINBw)
Oliver Brungs Federal Office of Bundeswehr Equipment, Information
Technology and In-Service Support (BAAINBw)
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-9
3 ORGANIZATION
3.1 Introduction
This report responds to the requirements of the NATO Study Order to “provide
analysis, recommendations and tools for NATO to effectively exploit outcome-based
contracting”.
Commercial entities such as airlines and various Nations’ military organizations
have employed outcome-based contracting with varying degrees of success. The
beginnings of outcome-based sustainment have been attributed to the commercial aviation
industry. Southwest Airlines champions outcome-based methods to reduce “turn-around
time” and increase profits. The Rolls Royce company pioneered the concept of selling
propulsion to aircraft, rather than selling engines, spare parts and repairs – Power by the
Hour™. Rolls Royce has since expanded its concept to a number of Nation’s military forces.
As well, other similar arrangements have been made between industry and Nations to
provide logistics as an outcome-based service. Some examples include aerial refuelling in
the U.K., and pilot training in Singapore. In both cases, the government has avoided the cost
of investing in infrastructure, training and operations, by having industry provide the desired
outcome (refuelled aircraft or trained pilots), at a cost less than the government could have
done organically.
This study explored documentation of many of these outcome-based arrangements
to determine commonality and opportunity for leveraging their successes in NATO.
The Study Order asserted: “Contracting for capability (readiness, availability, etc.)
has been proven in some situations to reduce cost and increase mission effectiveness. This
method of contracting is contrasted to traditional transactional contracting where payment
to a supplier is based on number of parts supplied or number of repairs.”
This Study provides NATO with the rationale and tools to support the determination
of whether, when, and to what extent outcome-based contracting best supports a specific
programme.
This Study addresses, describes and documents the various types of contracting
for capability implementations conducted in NATO and NATO Nations. Benefits, risks, and
lessons learned are addressed.
The Study provides recommendations for modification of NATO policy, doctrine, and
sample contract language for NATO publications. An evaluation method and a decision
support tool were developed to support new and legacy NATO programmes and programme
managers in determining the optimal combination of organic military and industry supported
life cycle support for use at any stage of a programme.
Many of the terms used in this report come from sources outside of NATO writing
convention. As well, there are many different phrases that describe outcome-based
arrangements, that are similar, but with nuanced differences (e.g. “outcome-based services
contracting”, and “performance-based logistics”). To assist the reader, a glossary of terms,
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-10
that is published as a separate document, has been prepared to provide context for those
terms used extensively in this Study. For consistency, in this document, we will use the term
“Outcome-Based Contracting” (OBC).
At the request of the sponsor organization, the scope of the Study is limited to
logistics support capabilities including any of the elements of logistics.
3.2 Background
Contracting for capability includes bringing outcome-based, commercially-provided
solutions to logistics and sustainment to increase user-defined availability and capability, for
the same or reduced costs. Examples maybe described as outcome-based services
contracting, performance-based logistics, fleet management, turn-key solutions, or total life
cycle management. These are methods of acquiring operational armament support that
acquires “performance” (outcomes) rather than materiel, maintenance and/or repair services
(outputs). OBC generally expresses requirements in terms of a desired end-state, rather
than providing for prescriptive or specification-based work statements. Simply put, OBC is
more about the “what”, than the “how” in delivering the desired product.
The approach is aimed at decreasing overall programme risk, improving reliability
and maintainability, and driving down life cycle costs.
Contracting for capability includes a wide spectrum of support solutions ranging from
minimum contractor involvement – the traditional model – where equipment is supported by
transactional repair contracts, to true contracting for capability where the contractor provides
a complete support package. Most solutions fall somewhere between the two extremes of
the continuum and include elements of both traditional and non-traditional support
arrangements. Identifying the options and the conditions under which each option optimizes
the support environment is a critical consideration during programme planning.
Increasingly, NATO Nation military organisations (e.g., UK, GER, NL, US) are
leveraging the defence industrial base to provide operational support, and are contracting
for that support, based on the delivery of performance outcomes.
The Study concentrates on three focus areas:
1. It contains a broad-spectrum analysis of programmes which have been conducted
using a Contracting for Capability model to achieve logistics support or sustainment.
The programme conditions, results, and lessons learned are documented. The
Quick Reaction Team (QRT) has given additional input on which programmes are
to be included in the study to ensure that a representative blend of conditions and
outcomes is addressed. The evaluation of the programmes includes a comparative
cost benefit analysis between the results of the programme and expected results
using traditional contracting methods.
2. Relevant NATO policy and doctrine are identified for updates, to include the
concepts of OBC. The Study provides sample language for consideration in NATO
publications.
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-11
3. The Study provides a decision support tool that can be used by NATO / Nations to
determine whether, and to what extent, OBC might be appropriate, given the
programme parameters.
3.3 Output of the Study
The Study considers operational scenarios that require a component of life-cycle
product support. The operational scenarios include situations where it is necessary to
maintain the readiness and operational capability of a system or subsystem. The specific
conditions range from ensuring that parts are available for repair activities, to ensuring
operational availability of a weapon system. In general, the Study considers operational
scenarios where there is a mission availability or readiness requirement that can be fulfilled
by industry support, and where innovation can increase efficiency and reduce cost.
3.4 Study Organization
The Concepts and Rationale for Contracting Capability in NATO Support
Programmes (“Contracting for Capability Study”) has been conducted in two phases:
• 1st Phase: The Study Team has been organized in three Study Sub-Groups (SSG)
to:
o execute the analysis of programmes that have been conducted using a
Contracting for Capability model to achieve logistics or sustainment. This activity
included an analysis of the programme conditions, results, and lessons learned.
The QRT was consulted for the list of programmes analysed;
o identify relevant NATO policy and doctrine that may require updates to
implement contracting for capability; and
o develop a decision support tool that could be used by NATO programme
managers to determine whether, and to what extent, contracting for capability
might be appropriate given the programme parameters.
• 2nd Phase: the Study Team worked cooperatively to:
o perform a global analysis of the benefits, risks, and lessons learned from
National experience and extrapolate that into the multi-national environment;
o prepare recommendations on how NATO doctrine and policy might be
enhanced to enable to effective consideration of different support solutions; and
o set up a decision support tool to guide NATO programme development with best
practices of contracting for capabilities.
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-12
3.5 Programme
Six Plenary Meetings of the Study Team took place, allowing the specialists to
develop this document (figure 1).
During such Plenary Meetings, the three SSGs collaborated on their respective
inputs. The proposals were shared with the QRT representatives, taking their feedback into
consideration.
Prior to Plenary Meeting #4, an initial draft of the Study Report was generated.
Several iterations of the draft document were circulated after each meeting for further
improvement and alignment.
All Plenary Meetings were conducted with the presence, and assistance of a QRT
Representative.
A final presentation to the QRT was performed to assure the consistency and
alignment with the Study Order.
The Final Report was presented to the Sponsor of the study in May 2018.
Figure 1 – Programme planning
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-13
3.6 Sponsor Liaison
The Study Team has been in continuous liaison with the QRT.
1st Name Last
Name
Organization Notes
Giuseppe Rampini ACO/SHAPE – LCMG-WG3 Met in Mons on Sept 26/2017
Steven Larcher Canadian Department of National
Defence (DND)
By Phone on the 4th and 5th Plenary
Meeting
Pierre Demers Canadian Department of National
Defence (DND)
Attended the Kick-Off (1st Plenary)
Meeting in Capellen
Oliver Brungs Federal Office of Bundeswehr
Equipment, Information Technology and
In-Service Support – LCMG-WG1
Attended the 2nd Plenary Meeting
Michail Bozoudis MOD GRE – LCMG-WG3 No show
Alain Courtois NCIA Interested to receive information, but
apologized for not attending.
Stefan Limburg Federal Office of Bundeswehr
Equipment, Information Technology and
In-Service Support
Attended the EG meeting and made the
initial sponsor presentation – plus
commenting on Final Report
Andreas Keller Federal Office of Bundeswehr
Equipment, Information Technology and
In-Service Support – LCMG-WG1
Attended the 3rd, 4th, 5th and the 6th
Plenary Meetings.
Stefan Limburg Federal Office of Bundeswehr
Equipment, Information Technology and
In-Service Support
On Apr 10/2018 meeting with the
Chairman discussing the findings and
recommendations.
3.7 Final Report structure
• Executive Summary (EXECUTIVE SUMMARY).
• Main Analysis (LIST OF PARTICIPANTS, ORGANIZATION, FOCUS AREAS
FINDINGS)
• Findings and Overall Study Recommendations (STUDY RECOMMENDATIONS).
• Appendices (APPENDICES)
• Annexes (separate documents are available on the DI/Web Sharepoint).
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-14
4 FOCUS AREAS FINDINGS
This chapter summarizes the work done within the 3 Study Subgroups:
• broad spectrum analysis of programmes;
• identification of relevant NATO Policy and Doctrine; and
• development of a decision support tool.
4.1 Broad Spectrum Analysis of Programmes
4.1.1 Introduction and Approach
Decision support tool of the Study Order directed the study group to “execute a
broad-spectrum analysis of programmes that have been conducted using a Contracting for
Capability model to achieve logistics or sustainment”.
The analysis was conducted to support the group’s overall recommendations, as
well as provide a basis for the other two tasks of the Study Order – recommend changes to
relevant NATO policy and doctrine (NATO policy and doctrine), and develop a decision
support tool for NATO / Nations decision makers to facilitate the selection of OBC to achieve
desired effects on cost and performance (Decision support tool).
Data sources included: available reports, personal knowledge of team members,
and findings from reports of previously conducted analysis.
The analysis considered the results from the perspectives of the end-user (i.e.
military forces), the contracting officials (i.e. NSPA/NCIA/MoDs), and the provider (i.e.
industry).
Characteristics considered included: capability, system supported, and the desired
outcome. It is important to point out that for many of the programmes reviewed, specific
performance and contractual data was withheld by Nation’s governing agencies for security
or business sensitivity reasons. In these cases, assessments were made using reasonable
estimates of similar situations, or anecdotal data.
This section addresses three areas:
1. discussion of programmes reviewed, or other studies leveraged for analysis;
2. observations of analysis to include characteristics and attributes of the contract,
results achieved, methods of delivery, and a summary of benefits;
3. summary of the analysis with recommendations; and
4. in addition to the glossary, annexes include examples of outcome-based
agreements and copies of source material reviewed.
4.1.2 Programmes Reviewed / Other Studies Leveraged
The Study Group assessed 23 programmes. These included 21 programmes
supporting NATO, and 2 commercial contracts supporting the private sector. The Group also
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-15
leveraged previous, similar analytical work, including 4 reports from academia and the U.S.
Government. A detailed list of programmes and previous studies is included in Appendix A.
4.1.3 Observations and Findings
4.1.3.1 General Observations
Many nations engage in outcome-based contracting for weapons systems support
and other logistics requirements. Many different terms are used to describe these
arrangements (e.g.: the U.S. uses the term “Performance-Based Logistics” (PBL)). This
Study collectively calls these arrangements “Outcome Based Contracts” (OBC).
In traditional methods, the government customer procures the required support and
service through transactional contractual arrangements. In OBC, the government procures
an outcome, where industry takes on the responsibility, and risk, to deliver those required
outcomes. Industry prime contractors include original equipment manufacturers (OEM),
non-OEM competitors, or joint ventures of interested parties.
OBC arrangements have common characteristics and generally seek to achieve
similar results: optimized performance and cost. By moving to OBC arrangements,
governments recognized that by shifting the responsibility and risk, contractors were now
incentivized to make improvements, thus providing better outcomes, at lower cost (see
figure 2).
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-16
Figure 2 – Traditional vs. OBC Support Strategies
Time
RELIABILITY
AVAILABILITY
COST
Implement a business model that inherently incentivises:
• Improving processes and products to improve reliability
and availability
• Resulting in O&S cost reduction & improved Performance
Co
st
an
d P
erf
orm
an
ce
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-17
4.1.3.2 Characteristics of Outcome Based Contracting
4.1.3.2.1 General
This section provides characteristics and attributes observed in the reviewed
programmes and other studies. There are universal characteristics of outcome-based
contracts found in practice. The list below is more fully described in Appendix B:
• specifies “what vs. how”;
• contains defined outcome metrics;
• explicit/implicit financial incentives tied to metrics;
• real financial risk/rewards;
• long-term arrangements; and
• authority to manage and optimize outcomes
4.1.3.2.2 Level and Complexity of the Supported System / Service Provided
Generally, the analysis showed that OBC arrangements could be grouped in three
categories: System Level, Sub-System Level and Component Level, Figure 3.
Figure 3 – OBC Levels of Complexity
System level refers to an arrangement that covers all the Integrated Logistics
Support (ILS) for an entire system (e.g. U.S. MH-60 helicopters) or provides system-level
support service (e.g. U.K. Aerial refuelling). These “Tip-to-Tail” programmes often include
support equipment, training, and infrastructure, providing a turnkey support solution.
System Level
Sub-System
Level
Component
Level
An entire Weapon System
or Major Service (e.g. UK
SENTRY, Aerial
Refueling)
A major subsystem
(e.g., Landing Gear)
A commodity item or service
(e.g., A/C tires, circuit cards,
garrison infrastructure
support)
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-18
Sub-system level refers to a group of components providing a major function within
a larger system. Examples include radars, sensors, communications systems, and aircraft
landing gear. The number of ILS elements engaged depends on the complexity and scope
of the subsystem.
Component level arrangements typically support smaller assemblies, subsystems
or systems (e.g. cockpit displays, H60 GE T700 engines), or provide supply chain services
for a single commodity, or class of commodities (e.g. tires, depot repair consumables). The
scope of ILS elements engaged in supporting component level requirements is generally
limited.
4.1.3.2.3 Metrics
Well defined, customer-focused, outcome metrics are the key to successful OBC
arrangements. Industry avoids unnecessary risks by accepting only those metrics that they
can control and deliver.
The use of a small number of outcome metrics (optimally 6 or less) empower
industry to take the necessary actions to provide requisite support. Contracts with too many
metrics risk becoming specification type contracts. In this case, the metrics measure
outputs, rather than outcomes.
Industry may employ these more specific output metrics to evaluate internal
processes, and control cost of delivery.
4.1.3.2.4 Contract Characteristics
Successful OBC contracts were generally observed to be long-term (5 years or
more), had stable funding, had incentives reflecting assumed risk, small numbers of top-
level outcome metrics, and based on a strategy of performance rather than product. See
figure 4.
The contract types used typically relate to the existence of system operation data.
Newer programmes may be supported with shorter, cost-type contracts, where programmes
with more operational experience use longer, fixed price-type contracts, with various
incentives tied to performance metrics.
While incentives are set to target performance, some OBC arrangements include
disincentives for both under and over-performance.
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-19
Figure 4 – OBC Characteristics
4.1.3.2.5 Prime Contractors and Subcontractor/Partner Relationships
All OBC programmes had a single, prime contractor accountable for meeting overall
performance outcome metrics. These primes usually engage several subcontractors, or
partner companies, and in some cases specified government agencies, to deliver the
required performance. In this role they act as an integrator, performing trades between the
providers to optimize outcomes.
Depending on the complexity of the support provided, some integrators flow down
the outcome metric requirements, as well as the risk and rewards, to their
subcontractors/partners. Adequately gauging and managing subcontractor contributions to
overall performance are critical to overall system and financial performance.
4.1.3.3 Case Studies Supporting Results/Recommendations
To illustrate the observations and characteristics, this section highlights two
significant examples of the 23 reviewed programmes: the first is focused on a Life Cycle
Support Programme, while the second is focused on a Support Services Programme.
Details about all reviewed programmes are provided in Appendix C.
• UK SENTRY is a whole life-cycle support programme for the E-3D Airborne Warning
and Control System (AWACS) – including both the air vehicle and mission
packages, provided by a non-OEM prime contractor. The 20-year contract covers
support to seven mission aircraft. This arrangement has a single outcome metric of
OBC CONTRACTS CHARACTERISTICS
Contract Characteristics
Te
ne
ts o
f S
uc
ce
ss
Contract
TermLong Term; Five Years Plus
Funding Stable; Funding Floor
RiskProvider Delivers Top-Level
Performance-Support Outcomes
MetricsBased on a Few Top Level
Performance-Support Outcome
Metrics
Strategy
Focus
Aligned with Top-Level End Item
Performance
Results
High Incentive to Invest
ROI Business Case for Investment
Higher Risk Offset by Opportunity
for Higher Reward
Provider Has Broad Flexibility in
How to Deliver Outcomes
Clear System-Level Provider
Responsibility
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-20
aircraft availability (requiring 6 of 7 aircraft). The provider (Northrop Grumman
Corporation) is fully empowered to conduct requisite product support activities (e.g.
maintenance, supply support, etc.) to meet required outcomes.
• UK Aerial Refuelling is a contract between UK MOD and a joint venture (Air Tanker
Holdings, Ltd.) to procure and support tanker and transport aircraft. This 27-year
arrangement is a Private Finance Initiative (PFI), with all capabilities funded by
industry. The customer pays for on-demand air transportation of fuel and personnel,
as a service, on a flying hour basis. UK Aerial Refuelling has two outcome metrics
– aircraft availability and flying hours.
Although most of the outcome-based arrangements identified for review were US
DoD contracts, the two programmes reviewed as case studies were selected as they
provided a more established development of outcome-based principles and details for
analysis.
The analysis of programmes revealed a limited body of specific acquisition
regulations for outcome-based arrangements by the individual NATO nations. The
predominant body of national regulatory documents were seen with those nations who have
longer experience engaging in OBC (e.g. US DoD, UK MoD).
4.1.3.4 Other Studies Leveraged – US DoD Proof Point
This section highlights the findings of the document “Proof Point Project – A Study
to Determine the Impact of Performance Based Logistics (PBL) on Life Cycle Costs”, Office
of the Deputy Assistant Secretary of Defense (Materiel Readiness), Washington, DC,
November 2011. The executive summary states the purpose of this study: “The U.S.
Assistant Secretary of Defense (Logistics & Materiel Readiness) chartered this initiative to
provide conclusive evidence regarding the impact of Performance Based Logistics (PBL) on
the life cycle cost of sustaining Department of Defense (DoD) weapon systems, subsystems,
and major components when compared to non-PBL sustainment arrangements.”
The study reviewed 21 OBCs (PBL programmes), and concluded that PBL
arrangements, which adhere to generally recognized PBL tenets, reduce cost while
simultaneously improve system performance. Sample of findings are seen in Figure 5, and
key findings, stratified by level of evidence supporting the conclusions, are seen in Figure 6.
The entirety of the report is included in ANNEX IV FINAL PROOF POINT.
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-21
Figure 5 – DoD Proof Point Summary Findings
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-22
Figure 6 – DoD Proof Point Key Findings
4.1.3.5 Benefit to doing Outcome Based Contracting
Key requirements for successful OBC are:
• Required services and responsibilities of all parties involved must be clearly defined.
• Industry party (prime contractor) must have full responsibility for delivery of the
service – Industry should set up and manage the arrangements between supplier
parties (sub-contractors)
• In systems with multiple providers (e.g. aircraft with different providers for the air
vehicle and the propulsion plant), the customer must designate a single integrator
to avoid ambiguity in responsibilities between parties for overall performance of the
system.
• Desired outcomes must be defined from the perspective of the end-user (e.g. the
operator).
• Specific authorities and responsibilities must be defined for changes in system
configuration (e.g. modifications, upgrades, service bulletin incorporation,
obsolescence management, etc.) prior to contract.
If these key requirements are met, OBCs will deliver the primary benefits of
improved system performance at the same, or reduced total cost (win-win). (Total cost is
defined as the cost incurred in a traditional support environment from both government
sources (organic support) and contracted services (industry)). Effects include:
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-23
• OBC recognizes the shifting of responsibility, and consequently risk, from the
government to industry in pricing and incentive structures;
• increases in revenue, cash flow and profit reflect the additional responsibilities and
risk assumed by industry;
• natural industry behaviour, with increased risk, is to affect continuous process and
product improvements to reduce costs;
• long term contracts are designed to give Industry a reasonable expectation to
achieve a return on invested capital. Profit levels at the end of a period of
performance may appear excessive, but customers must consider periods where
industry made investments to improve service delivery and performance. A suitable
long-term costing and performance model should be established;
• occasionally windfall profits are realized when results from improvements in process
and product exceed expectations. Well-structured OBC arrangements account for
this in the incentives structures, and through gain-sharing schemes, where a
predetermined percentage of profit above expected gains are returned to the
government;
• industry-standard performance levels may exceed what is desired by the
government (e.g. system availability at 90%, where government readiness
requirements or available funding support lesser needs). OBC arrangements should
reflect customer needs and related payment terms to incentivize industry to deliver
what the customer needs; and
• other benefits will be a reduction in total inventory of spares and repairs in the
system, as industry will optimise holdings against lead times, repair and overhaul
turn-around-times, against proven MTBFs to achieve the required system
availability metrics. Considerations, such as customer surge requirements and the
consequent emergency stock levels, need to be included within the contract.
4.1.3.6 Parameters to be Considered for Decision Support Tool
Based on the characteristics of successful OBC observed as part of the broad-
spectrum analysis, a list of parameters to be considered for the proposed Decision Support
Tool was developed. The table of parameters is described in Appendix E.
4.1.4 Summary and Recommendations
The evidence observed from the programmes and other studies reviewed,
demonstrates that well-structured and defined OBCs will reduce total costs, increase system
availability for the war-fighter, optimise industry performance (investments and costs) to
deliver an improved outcome, i.e. a win-win for government and industry, figure 7.
These results suggest that NATO, and the Nations supported by NATO contracting
agencies (e.g. NSPA, NCIA), would similarly benefit by employing OBC.
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-24
The results also suggest that successful OBC requires proper planning, and due
diligence to understand the customer’s desired outcomes, set the appropriate performance
outcome metrics, and negotiate effective incentives.
Joint programmes, like the F-35 Lightning II Joint Strike Fighter, where many of the
participating nations are also NATO nations, present a unique opportunity to engage in OBC
and provide improved system availability with greater efficiencies – optimised outcomes.
The Study recommends NATO enter into OBC arrangements starting with two pilot
programmes, utilizing the Decision Support Tool developed as part of this Study, to identify
the target programmes.
Figure 7 – Win-Win: OBC Benefits
4.2 NATO policy and doctrine
4.2.1 Introduction
This section focuses on identifying “NATO policy and doctrine that would be relevant
to be updated to include the concepts of contracting for capability and provide sample
language for consideration by the document custodian”. Recommendations on how NATO
doctrine and policy might be enhanced to enable effective consideration of different support
solutions, and sample language for inclusion in NATO documents are included.
Contractor
Government
Other Programmes
Opportunity to:
• improve product reliability
• improve process efficiency
• decrease costs
• increase profit
Opportunity to:
• increase availability of system
• Increase reliability of system
• decrease programme costs
• achieve desired outcomes
Opportunity to:
• free up capacity
• gain lessons learned
WIN
WIN
WIN
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-25
The analysis also provided input to support the development of a decision support
tool.
A list of 52 documents was identified, and reviewed by the QRT. The documents
were selected according to levels of importance, and relevance with respect to the NATO
Policy for Systems Life Cycle Management
The 52 documents were selected to reflect:
1. The estimated impact on the object of the study;
2. the hierarchy of NATO documents, from high-level policy on systems life-cycle
management, to main standards and guidance on life-cycle and logistics support
(e.g. AAP-20; AAP-48; ALP-10), to more detailed standards and guidance related
to specific aspects (e.g. quality; risk management; life-cycle costs; acquisition
practices);
3. the majority of life-cycle stages;
4. the key procurement regulations (e.g. NSPA; NCIA); and
5. examples from NATO member nations.
Of the original 52, 20 documents were selected for in-depth analysis. These
documents are selected samples that well represent the points described above. The full list
of documents can be found in Appendix F – Reference Documents.
For the in-depth analysis of the 20 documents, reports have been produced which
include:
1. Name and version of the document;
2. short summary of document, followed by remarks concerning need for update and
recommendations for improvement, as well as connection with other NATO
documents;
3. sample language (when considered appropriate);
4. remarks concerning collaborative relationship (when applicable); and
5. identified need for new definitions or adaptation of old definitions.
These reports can be found in Appendix F – Reference Documents.
In order to further address the request for sample language, a separate section was
created where examples for such language are given in the form of “amendments” to the
current text. This document is to be found at Appendix D – Sample language.
4.2.2 Findings & Conclusions
No specific NATO policy, doctrine, or agreed definition on OBC was identified.
Therefore, the recommendations and sample language suggested are based on the Study
Group experience and analysis of existing OBC-type programmes. The group assumes that
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-26
the same characteristics would be considered acceptable for use by NATO and NATO
Nations.
Although certain elements, similar to OBC, could be found throughout the
documents, the concept of OBC itself does not appear to be integrated in a consistent and
coherent way in existing policies, doctrines and procurement regulations.
For example, guidelines like AACP-2 Vol. 2, suggest a preference, whenever
possible, for firm, or fixed price contracts. However, reference is not made to the possible
positive implications (e.g. on cost, incentives; risk premiums) that firm, or fixed price, has
when OBC is applied.
Other rules and procurement principles, for example those applied in the current
NSPO procurement regulations (e.g. “pre-award contact with suppliers”; “the most
economical price”; “balancing of production”) may not leave enough scope for applying OBC
if they are not adapted accordingly.
Implementation of OBC requires complete understanding of the requirement (“the
what”) by both the Contracting Officer and Industry. Close collaboration between Contracting
Officer and Industry must occur as early as possible in the definition of needs, to ensure
industry adequately addresses the delivery of the outcome (“the how”).
This applies equally for new capabilities, the improvement of an already existing
capacity / service, new contracts and the transition from traditional type of contracting to
OBC.
Some industry consultation instruments were found throughout the analysed
documents, which may be sufficient for the traditional transactional type of contracting but
not sufficient for OBC (e.g. “pre-award contact with suppliers”, “pre-award survey” within
NSPO Procurement Regulations).
Consultation of prospective contractors as early as possible (during the definition of
needs for services or capabilities) is a prerequisite for selecting the right type of contract (i.e.
traditional vs. OBC). If OBC is chosen, enough time is needed for defining the details (e.g.
exact scope of the contract, service level, performance metrics) and, most importantly, for
building trust between contractors and the customer. As such, if OBC is chosen, industry
needs to assess its capacity to properly deliver the requirements defined in contracts, its
readiness, its availability, as well as to estimate risks and benefits for the customer and the
prospective contractor.
Interdependence also exists between the implementation of OBC and the
Government Quality Assurance process. For example, the question can be raised regarding
the extent to which Quality Assurance should take into account “measurable outcomes” or
“performance metrics” identified as part of OBC.
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-27
4.2.3 Recommendations
It is recommended:
• NATO should develop a policy on contracting for systems and logistics services
including “traditional contracting and OBC in order to secure common understanding
of the concept and of its underlying principles. Such policy and common
understanding are likely to facilitate decision-making in a multinational environment.
This policy has to be in line with the NATO Policy for Systems Life Cycle
Management (SLCM) approved by the North Atlantic Council and signed by the
Secretary General in 2006.
• The Study recommends NATO to enter into OBC arrangements starting with two
pilot programs, utilizing the Decision Support Tool developed as part of this Study.
This trial should be performed by NSPA and should be limited to a to a defined
period of time(e.g. 2 years.
• OBC training requires a Training Need Analysis (TNA), it is recommended to
consider, whether the aspect of “Contracting” can be included in the current Life
Cycle Management Training Analysis.
• The Study recommends to review and to update the NATO publications on
contracting including all aspects of contracting – traditional and OBS- based on
and in line with the NATO Policy for Systems Life-Cycle Management.
• Furthermore it is recommended to specify in the relevant NATO documentation at
which point in the life-cycle of the system in the decision-making process the
responsible manager should evaluate if OBS contracting is preferred to traditional
contracting.
• This work should be performed by a new AC/327 Working Group with members
both from Nations and the NIAG IIG and co-chaired by AC/327 and NIIG.
• The basis of this work are the results of the work of the former Group on
Acquisition Practices – AC/313 which was created to advise the CNAD on
government defence procurement practices and procedures relating to defence
trade, armaments cooperative programme arrangements and industrial
collaboration within the Alliance.
• In this context the Study recommends to modify several existing definitions
(Appendix D – Sample language) to recognize the concept of OBC. Additionally it
is recommended that NATO uses the term Outcome Based Contract (OBC)
exclusively.
• To specify in the highest-level NATO documentation (e.g. NATO Policy for Life
Cyle Management, AAP-20, AAP-48, ALP-10, and NSPA/NCIA procurement
regulations) at which point in the decision-making process the customer /
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-28
procurement officer should evaluate if OBC is preferred to traditional contracting.
This assessment should be done by applying the decision support tool;
• The choice between traditional contracting and OBC should be done as early as
possible in its life-cycle, both for the entire System of Interest (SoI) and its sub-
systems, via mutual consultation between industry and contracting authorities;
new instruments for industry consultation should be created to accommodate the
specificities of OBC.
• Any changes to high-level documents must also be reflected in subordinate
Implementing guidance and standards documents. In addition to general OBC
principles, guidance should include specific areas of OBC (e.g.: metrics/indicators,
requirements definition, risk vs. incentive determination);
• The inclusion of the metrics could be investigated in the framework of a follow-on
NIAG Study.
• The implementation of OBC within the NATO community should be promoted from
the highest to the lowest decision-making levels. To ensure its implementation,
specific instructions and the development of a training concept and dedicated
training modules are needed. The contracting officers throughout the NATO
organisations and agencies must be trained on the same basis to ensure
commonality;
• It is recommended to develop processes for Government Quality Assurance
Organisation to evaluate outcome metrics and determine if adequate OBC results
are delivered. The relation between Government Quality Assurance Organisation
and performance metrics specific to OBC implementation can be evaluated in the
frame of a follow-on NIAG study.
4.3 Decision support tool
4.3.1 Description / rationale of the tool
This part of the Study determined a methodology leading to a set of processes
gathered in a decision support tool to support the choice of the applicable contract type. The
result coming from the support tool is to be taken into consideration to select the type of
contract.
This activity was conducted through close discussions within the Study Group and
with some external contribution from relevant stakeholders. (e.g. NSPA/NCIA/MoDs).
The task performed at this stage was a preliminary reflection on how to best
determine a “Logical Choice”. Since it is difficult to consider all the relevant parameters for
a new project, the Study Group limited, for this preliminary thinking, the number of input
parameters for the model proposed. During the use of the tool, the input parameters should
be evaluated and weighted by contract specialists to adapt the tool for final use.
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-29
The final decision for a support contract type could take into consideration further
applicable parameters.
The decision support tool has been benchmarked with two of the contracts analysed
in section 4.1 to check its efficacy.
The tool is expected to support the user in deciding a contract type (e.g. outcome-
based (“OBC preferred”) or activity based (“Non-OBC preferred”)).
The process used to develop the tool included:
• identification and refinement of a set of parameters from existing contracts
examined in Section 4.1;
• establishing a logic, based on the study members experience and taking into
account current NATO contracting regulations as investigated in Section 4.2;
• testing the tool with two test cases. Input for these test cases came from the
examined contracts in Section 4.1; and
• providing instructions on how to use the tool.
More details are described in Annex II
For the implementation of the tool, the Study Group completed the following:
• A sequence for a decision tree was developed. The challenge was to highlight a
logic that enables the user to take sound decisions, as values are applied for the
different parameters inside the decision tree. This logic is generic enough to cover
both OBC and Non-OBC.
• For the development of the decision support tool, the defined logic was turned into
a process, which was split into seven phases:
o Phase 1 – Preparation
o Phase 2 – Parameters valuation
o Phase 3 – Initial scoring
o Phase 4 – Selection of best suitable “Main contract type”
o Phase 5 – Pre-selection of best suitable “Sub contract type”
o Phase 6 – Verification of contracts types compatibility & optimization
o Phase 7 – Final selection of contracts
More details are described in ANNEX II Development Tool and User Manual.
The Tool itself is a protected Excel File and is posted together with a list of passwords on
The Study Group identified and refined different types of parameters, including:
1. Requirement
2. Cost Analysis
3. Metrics – Type
4. Metrics – Number used
5. Complexity of System Supported
6. Performance History / Experience
7. Contract starting point in life cycle
8. Availability of Data and Technology
9. Contractor Freedom of Solution Choice
10. Provision for Performance Improvement
11. Period of Performance: Return on Investment Opportunity
12. Scope – Extent of service provided
13. Contract Type
14. Obsolescence Management / Modernization and Upgrade
15. Number of Nations
16. Contractor OBC Experience
ANNEX II Development Tool and User Manual describes in detail the different
parameters used for the tool.
The Study Group reviewed NATO documents to identify policies and doctrines
relevant to OBC, further including regulations that could be considered for the decision
support tool. The findings included three suggested actions with nine specific
recommendations. The conclusions drawn from this are shown in ANNEX II
Development Tool and User Manual.
The group took into consideration the overall process of the OBC decision support
tool requirement in order to identify the stakeholders, and their expectations were provided
through discussions, review of documents and specifications, criteria, and “input
parameters”.
To ensure the requirement are fully identified, the following list stakeholders were
identified:
• End-Users (e.g. armed forces)
• Customer contracting organisation (e.g. national MoDs, …)
• NATO (e.g. NSPA, NCIA, …)
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-31
• Participating industrial partners
The requirement evolution and the values assigned to the parameters depend on
the level of comprehension obtained during the Study including the identification of the
parameters that will influence the decision making.
Annex II contains flowcharts, and it describes the process to reach a good level of
understanding and to be able to size the parameter level.
Annex II contains the description/handling (user manual) of the tool.
The tool itself is an Excel- file which consists of three worksheets:
• Main (user interface)
• Parameters (setting of parameters and values for the Main worksheet, setting of
digital values in order to enable comparisons)
• Evaluation (performing evaluations of selected parameter values. The result of the
evaluations is shown in the Main worksheet)
It contains the 16 parameters as listed above plus a provision for 3 additional
parameters to enable simple further development of the tool.
It is important to bear in mind that this tool was not planned to be the sole
discriminator for the user to make his decision on which contract type to use. The user must
take into account other circumstances, e.g. regulations and pre-conditions. It is a support
tool!
Being the first release of this tool, the Study Team strongly suggest following the
instructions of Annex II, in order to avoid misinterpretation.
Two test cases have been applied to the tool:
• The first for the UK Sentry contract, covers the Study Group order requirement to
run a test with data of a weapon system contract (result: OBC preferred strongly).
• The requirement for a test case with large/complex capability services is covered by
the test case UK Aerial Refuelling (result: OBC preferred strongly).
Detailed information about the test results is provided in ANNEX II
Development Tool and User Manual.
4.3.2 Recommendations
The task of the Study Group was not to develop a fully comprehensive tool based
on fully validated software, but to show that the processing of a broad set of key parameters
can be of great support for NATO in defining the best type of contract in answer to an RFP.
The Study has shown that the present process and decision support tool can be
adapted in different ways, which are suggested below:
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-32
At process level:
➔ Further develop the process, scope and capabilities
o extend to more contract types
o further test of cross parameters interaction to avoid unforeseen results
➔ Improve process through results from Lessons Learned (LL)
o an LL committee –including NATO personnel- should review the parameters
list. Detailed value changes should be suggested for the parameters based on
the related results
o potentially develop other applications to meet NATO’s personnel expectations
o improve the actual decision support tool, as the considered test cases are
somewhat subjective and have to be taken with caution. E.g. test cases for
Non-OBC have not been taken into account
At tool level:
➔ further develop the tool by continuing to consider the parameters
o pursue identifying and integrating all impacting parameters
o other parameters that could be considered for inclusion are:
▪ Budget
▪ Time frame
▪ Quantity of systems,
▪ Cyber security factors
▪ Intellectual property rights
▪ Detailed contract types
o the current prototype tool is based on Excel®. Consideration should be given
to implementation on dedicated COTS SW once mature.
➔ Improve the tool
o through additional functions
o through technology (e.g. parameter database, integration with Big Data and
Artificial Intelligence
o through innovation
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-33
5 STUDY RECOMMENDATIONS
The most important findings of the Study are that OBC processes have the same value for NATO as traditional contracting processes and the overarching recommendation is, that NATO fully adopts and implements OBC processes.
To implement OBC, it is recommended:
1. NATO should develop a high level policy for contracting for systems and logistics
services. This policy should cover all kinds of contracts (traditional contracting and
outcome-based contracting e.g. for capability).This policy has to be in line with the
NATO Policy for Systems Life Cycle Management (SLCM) approved by the North
Atlantic Council and signed by the Secretary General in 2006.
2. NATO should enter into OBC arrangements starting with two pilot programs,
utilizing the Decision Support Tool developed as part of this Study. This trial
should be performed by NSPA and should be limited to a defined period of time
(e.g. 2 years).
During this time it is recommended to perform a Training Needs Analysis (TNA) for contracting including the use of the Decision Support Tool to be included in the current LCMG TNA.
3. NATO should review and update the NATO publications on contracting including
all aspects of contracting – traditional and OBC- based ones. The updated
publications have to be in line with the NATO Policy for Systems Life-Cycle
Management.
Furthermore it is recommended to specify in the relevant NATO documentation at
which point in the life-cycle of the system the responsible manager should evaluate if OBC
contracting is preferred to traditional contracting.
The basis of this work are the results of the work of the former Group on Acquisition Practices (AC/313) which was created to advise the CNAD on government defense procurement practices and procedures relating to defense trade, armaments cooperative programme arrangements and-industrial collaboration within the Alliance. AC/313 was disbanded.
This work should be performed by a new AC/327 Working Group with members both from Nations and the NIAG IIG and co-chaired by AC/327 and NIIG.
The evidence observed during the Study demonstrates that well-structured and defined OBCs will reduce total costs, increase system availability for the war-fighter, optimize industry performance (investments and costs) to deliver an improved outcome, i.e. a win-win for government and industry.
The Study Group further recommends:
1. NATO should develop a policy on OBC in order to secure common understanding of the concept and of its underlying principles. Such policy and common understanding will facilitate decision-making in a multinational environment.
2. Responsibilities for all parties involved have to be defined clearly.
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-34
3. Industry should have the full responsibility for delivery of the service, to include set
up and managing of the arrangements between supplier parties.
4. In systems with multiple providers, designate a single integrator to avoid ambiguity
in responsibilities.
5. Desired outcomes must be defined from the perspective of the end-user
6. Specific authorities and responsibilities must be defined for changes in system
configuration (e.g. modifications, upgrades, service bulletin incorporation,
obsolescence management, etc.) prior to contract.
7. Proper planning and due diligence are required to understand the customer’s
desired outcomes, set the appropriate performance outcome metrics, and
negotiate effective incentives.
8. NATO (e.g. NSPA/NCIA/) should start OBC with two pilot programmes using the Decision Support Tool developed as part of this Study.
9. The term Outcome Based Contracting (OBC) should be selected to cover all contracting functions that provide outcome-based or performance-based results.
10. The contracting method (OBC or traditional) should be selected as early as possible in the life-cycle of the system/subsystem, in consultation with industry starting with the Concept Stage of a programme.
11. The use of an analytical tool-set for contracting type selection at the highest-level NATO life-cycle policy and contracting regulations is required.
12. Conduct a follow-on study to determine the appropriate use of metrics in contracting. Define processes for Government Quality Assurance Organisations to evaluate outcome metrics and determine if adequate OBC results are delivered.
13. It is recommended to further develop the process, scope and capabilities of the decision support tool, to extend it to more contract types, and to enhance cross parameters interaction to better balance parameter ratings.
14. Initiate and conduct training in contracting processes, including the use of a decision support tool.
15. Improve the use of the tool through return of experience.
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-35
6 APPENDICES
6.1 Appendix A – Programmes Reviewed / Other Studies Leveraged
Members of the study group represented companies of a wide range of industries
and Nations. They leveraged their individual backgrounds and experiences to analyse
programmes supported by outcome-based arrangements, and reviewed related studies
conducted by others. These arrangements, and studies, covered a full-spectrum of
scenarios – both in peace and wartime – providing services, material, and/or maintenance
support to specific end items, from components, to subsystems, to entire weapon systems.
The analysis group was aware of many programmes that are supported by outcome-
based arrangements. The table on page 36 provides a list of programmes reviewed.
Obtaining analysis data for some of these programmes proved difficult, for reasons
including national security, business sensitive, and proprietary nature. In cases where data
was unavailable or inadequate, the group elected to use anecdotal information or results of
previously completed studies. The table on page 37 provides a list of studies that were
leveraged to inform the analysis. The list of these reports is included in Appendix F –
Reference Documents.
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-36
Programmes Reviewed
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-37
Study Title Source Date
Proof Point Project, A Study to Determine the Impact of
Performance Based Logistics (PBL) on Life Cycle Costs
US Department of
Defense
November 2011
Impact of Performance Based Contracting on Product Reliability:
An Empirical Analysis
Management
Science, Vol 58/No.
5
May 2012
Power by the Hour: Can Paying Only for Performance Redefine
How Products are Sold and Serviced?
University of
Pennsylvania,
Wharton School of
Business
January 2018
Performance Based Logistics, Conclusive Evidence Supporting
the Impact of PBLs on Life Cycle Costs, Public Presentation
US Department of
Defense
January 2013
Additional Studies Assessed
6.2 Appendix B – OBC Arrangement General Characteristics
This section provides characteristics and attributes observed in the reviewed
programmes and other studies. There are universal characteristics of outcome-based
arrangements found in practice. The list below is more fully described in Annex I:
• “What vs. How”: Requirements expressed as “what” is desired, not specifics on
“how” to achieve the results. This is the essence of OBC. “How” type contracts are
more typically defined as transactional. To achieve an outcome, the contractor is
given leeway to define how they will perform and deliver the desired outcome as
expressed by the government.
• Defined metrics: To determine if the desired outcome has been met, specific
metrics are defined to measure outcomes. Typically, the more successful
programmes had less than 6 outcome metrics for overall performance, e.g.: time-
on-wing, engineering response time, availability, and perfect order fulfilment.
• Financial incentives tied to metrics: These incentives include awards for
improving system performance or reducing overall cost to support the system. Some
incentives have come in the form of “award term”, providing additional contractual
period of performance without recompeting/renegotiating the contract. Incentive
awards, or conversely, withholding, are tightly associated with the contractor’s ability
to meet the defined metrics.
• Risk/Rewards and Long-term arrangements: Taking on the responsibility for
system performance increases contractor risk. Before, contractors sold spares and
repairs to the government on demand, with no incentive or capability to optimize
system performance and cost. With OBC arrangements, contractors are now
responsible for system performance and cost, raising risk to profit and reputation.
To mitigate this increased risk, contractors will typically make internal investments
to improve the system, or the reliability of spares, with the expectation of reduced
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-38
costs and improved performance. OBC arrangements are typically of a longer length
than transactional contracts – a period of performance long enough to allow
contractors a reasonable expectation for a return on invested capital. Most OBC
arrangements are between 3-5 years. Some arrangements exceed 25 years, made
up of 5-year contracts reviewed in the final year, and if metrics are met additional
periods of performance awarded.
• Authority to manage: To achieve optimal system performance and cost often
requires the integration of many product support, or logistics elements. The
successful OBC arrangements are those where the contractor was provided
authority to integrate, and make trade-offs, between all those supporting activities.
Some nations have legal requirements to control support activities and direct the
use of government-owned, or organic, support. In those cases, successful
programmes engaged in “public-private partnering” between the contractor and
other government support activities.
6.3 Appendix C – Case Studies Supporting Results /
Recommendations
UK Sentry Whole Life Support Program
Contract between the UK MoD Defence Equipment & Support (DE&S) and Northrop
Grumman for the support to the seven UK Airborne Warning and Control System (AWACS)
E-3D aircraft.
Aircraft are based at RAF Waddington, Lincolnshire. Aircraft were delivered in 1991
& 1992 with little consideration of support contracting at time of delivery. Initially there were
up to 12 supply contracts for repairs and spares for the total fleet of 7 aircraft, with the RAF
performing all front-line maintenance. Each aircraft required deep maintenance every 6
years, returned to industry for a period of time up to 9 months. This methodology of
sustainment resulted in only 5 of 7 aircraft being operational for significant periods.
The MOD Integrated Project Team (IPT) looked at creative and innovative methods
for contracting to achieve greater availability. It looked at combining:
• Aircraft maintenance,
• The provision of spares and repairs,
• Design Engineering Support Services (DESS),
• Technical Publications
• Maintenance training
The IPT adopted a Single Track Maintenance (STM) approach whereby only one
aircraft would be in maintenance at any time increasing the available aircraft from 5 to 6.
A competitive bid process was held between Boeing (aircraft OEM) and Northrop
Grumman (radar supplier – primary mission sensor), with an internal UK MOD maintenance
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-39
option as a comparator. Due to high complexity and the IPT looking for innovation and
creative solutions to the problem, a second round of bidding was necessary to refine
assumptions and reduce risk. Northrop Grumman was subsequently selected on a full
through life cost analysis basis.
To mitigate the additional time taken to choose a contractor, a risk reduction
preliminary contract was let to ensure work in critical areas. Requirements were tailored so
that if the full contract was not let as planned, then the work would still benefit reversion to
the UK MoD maintenance option.
The contract runs until 2025 and has firm prices agreed for the first five years. The
remaining period is subject to a Target Cost Incentive Fee (TCIF) arrangement under which
cost savings are split three ways; 50 per cent is retained by the contractor, 20 per cent is
retained by the Authority and 30 per cent is retained in a savings pool.
Money held in the savings pool is used to ‘reward’ the contractor if he meets certain
performance criteria. If there is no money in the saving pool then, irrespective of the
contractor’s performance, no ‘reward’ is paid. The incentive arrangements under the
contract are, therefore, ‘self-funding’.
The contract requires the contractor to make six of the seven Sentry aircraft
available at any time. The contractor is responsible for arranging maintenance, including
spares and repairs, to meet this requirement. Payments are made on the basis of aircraft
availability as set out in the contract.
The IPT over time plans to increase the scope of the contract to cover Whole Life
Support, with the ultimate goal of being to have all Sentry support activities under one
contract. Initially there were some elements of support that were still contracted separately,
leading to dependencies that fell to the Authority to manage under the project.
The contract length helps to mitigate the risks associated with the issue of
obsolescence. For example, smaller parts suppliers are more likely to keep their
manufacturing lines open if there is a long-term contract for aircraft maintenance.
The contract makes use of a partnering approach. This recognises that the IPT
cannot deliver the outputs that they require alone, and neither can industry. Both sides must
work together as partners to deliver the requirement. The difference with partnering is that
both parties are working closely together with greater visibility of the desired outcome. For
Sentry, a Partnering Principles document was generated after the final bidder was selected.
This was an aspirational code of behaviour rather than a legally binding contract.
The method used to draw up the Sentry contract was totally different from previous
methods. In the past, industry would be directed to the proposed solution, rather than
agreeing the most appropriate solution, as in this case. The Team took an innovative
approach to letting the contract, summarised as follows “There are no rules – only
authoritative guidance. So long as we don’t reduce the safety level of the aircraft, and we
are not doing anything illegally, then anything is possible.”
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-40
If the aircraft is required to go to war, the contractor will deploy key employees and
stock to the aircraft’s new operating base. Selected employees of the contractor are
sponsored reservists and, when required, will put on uniform and support the aircraft in
theatre.
UK Aerial Refuelling – RAF Voyager Force Contract
General:
Contract between UK MoD Defence Equipment & Support (DE&S) and Air Tanker
Holdings Ltd for the provision of 14 Voyager Aircraft and full Support for 27 years.
Aircraft are used for four roles:
• Aerial Refuelling (2 or 3 hose and drogue systems)
• Air Transport
• Medivac (40 stretchers and full medical support)
• VVIP transport
PFI Contract signed March 2008 with support to run until 2035 – includes
procurement and aircraft certification and set up of support infrastructure.
Air Tanker Holdings Ltd consortium is a joint venture between the following parties
and their responsibilities:
• Airbus – Basic aircraft Airbus A330-200 and modifications to tanker configuration
• Rolls Royce – Rolls Royce Trent 772 engines
• Thales – avionics supplier and simulator provision
• Cobham – Under-wing and Belly Flight Refuelling Units (FRU)
• Babcock – provision of labour
Infrastructure (provided under contract) located at RAF Brize Norton
• All maintenance facilities
• Training facility (aircrew & maintenance) including full flight simulator
• Aircrews and maintenance personnel all based at Brize Norton
• Aircraft and crews (air and ground) deployed to a number of operating locations in
accordance with MoD tasking demands
Crew (aircrew & maintenance) are a mix of Full-time RAF personnel, Sponsored
RAF Reserves and Civilians. Crew working on MAR aircraft are RAF personnel, crew on
CAR aircraft are generally civilian although military crew with civil certification can fly. The
contract also covers support aircrew for air transport duties (flight attendants).
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-41
Voyager fleet split into ‘Core’ and ‘Surge’ fleet as follows:
• Core fleet – 9 a/c (8 a/c on Military Air Register (MAR) providing Air Refuelling and
1 a/c on Civil Air Register (CAR) providing Transport duties)
• Surge fleet – 5 a/c, 2 a/c on MAR also used for reserves to Core fleet and 3 a/c on
CAR leased out to commercial operators, generates 3rd party revenue for Air
Tanker
• All aircraft have fixed fittings to enable Air Refuelling but when on CAR there are
significant role fit equipment removals and changes to the cabin required to
convert from MAR to CAR and vice-versa
• All aircraft can carry two underwing refuelling pods (hose & drogue) and a limited
number (≈6) also fitted to carry belly refuelling (hose & drogue)
Contract Key Features:
1. A contract planning baseline number of flight hours per annum used (scopes size
of support required) – actual hours over first few years approx. 10% below
planning baseline.
2. Guaranteed Minimum Usage (GMU) payment for flight hours on both MAR and
CAR fleet used to ensure guaranteed payment to Air Tanker (coverage of aircraft
Reserved for further extension (1) unknown / not applicable 3 No preference
Reserved for further extension (2) unknown / not applicable 3 No preference
Reserved for further extension (3) unknown / not applicable 3 No preference
Overall Prefered Contract PBL prefered slightly
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-120
Provision for Performance
Improvement The higher the freedom to handle improvements, the better the overall result
Period of Performance:
Return on Investment
Opportunity
The longer the contract, the better the possibility to achieve ROI. Long-term
contracts enables PBL a good change for a return of investments
Scope – Extent of service
provided
Handling a single ILS-segment is easier than handling several of them due to
dependencies between each other. If the complete ILS-spectrum is covered,
harmonisation with other contracts is not needed.
Contract Type Fixed fees are not attractive for a PBL contract, but penalties/incentives and
gainsharing are interesting for PBL contracts.
Obsolescence
Management /
Modernization and
Upgrade
The lower the possibilities for technical investments in the future, the more a
Non-PBL becomes attractive
Number of Nations Number of involved nations (NSPA-customers)
Contractor PBL
Experience
The better the PBL experience of a contractor, the better for a new PBL
contract with this contractor
NATO UNCLASSIFIED
Releasable to PFP
NATO UNCLASSIFIED
Releasable to PFP
1-121
ANNEX III DECISION SUPPORT TOOL
The Tool itself is a protected Excel File and is posted together with a list of
passwords on the NATO DI Portal: https://diweb.hq.nato.int
(NIAG / Study Groups / SG-218 / Shared Documents):
Parameter Value Weight Preferred Contract
Requirement unknown / not applicable 5 No preference
Cost analysis unknown / not applicable 1 No preference
Metrics - Type unknown / not applicable 5 No preference
Metrics - Number used unknown / not applicable 3 No preference
Complexity of System Supported unknown / not applicable 3 No preference
Performance History / Experience unknown / not applicable 3 No preference
Contract starting point in life cycle unknown / not applicable 3 No preference
Availability of Data and Technology unknown / not applicable 4 No preference
Contractor Freedom of Solution Choice unknown / not applicable 5 No preference
Provision for Performance Improvement unknown / not applicable 4 No preference
Period of Performance: Return on Investment Opportunity unknown / not applicable 4 No preference
Scope - Extent of service provided unknown / not applicable 3 No preference
Contract Type unknown / not applicable 4 No preference
Obsolescence Management / Modernization and Upgrade unknown / not applicable 3 No preference
Number of Nations unknown / not applicable 1 No preference
Contractor OBC Experience unknown / not applicable 3 No preference
Reserved for further extension (1) unknown / not applicable 1 No preference
Reserved for further extension (2) unknown / not applicable 1 No preference
Reserved for further extension (3) unknown / not applicable 1 No preference
Overall Prefered Contract No preference
Weight 1 2 3 4 5
Parameter Value1 Value2 Value3 Value4 Value5 Value6 Value7 Value 8
Requirement unknown / not applicable Nebulous Partially defined Fully defined
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 1 5 10
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 3 5 10
Cost analysis unknown / not applicable Cost value 1 Cost value 2 Cost value 3
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 5 5 5
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 5 5 5
Metrics - Type unknown / not applicable Output - activity measure Mix - Outcome/Output Outcome - availablilty measure
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 1 4 10
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 10 6 2
Metrics - Number used unknown / not applicable 1-2 3-6 7-10 11+
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 10 8 3 1
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 4 6 8 8
Complexity of System Supported unknown / not applicable Major System / tip-to-tail Collection of Sub-systems/Minor System Sub-system Component / Commodity
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 8 4 4 10
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 2 5 7 10
Performance History / Experience unknown / not applicable New market/low-no experience Medium level / few suppliers High level /competitive market/OEM(SS)
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 1 5 10
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 3 5 10
Contract starting point in life cycle unknown / not applicable Prototype Development Production Support Sundowning/End-of-life
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 1 3 6 10 3
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 8 7 6 5 5
Availability of Data and Technology unknown / not applicable Low Medium High
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 1 5 10
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 3 5 8
Contractor Freedom of Solution Choice unknown / not applicable Workshare (gov't directed) Hybrid (PPP - best value decision) Complete Contractor Freedom
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 1 5 10
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 2 5 8
Provision for Performance Improvement unknown / not applicable No authority Restricted authorities Tacit improvement authorized Complete Improvement Freedom for Contractor
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 1 4 7 10
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 3 5 5 6
Period of Performance: Return on Investment Opportunity unknown / not applicable Extremely short (1 yr) Short (2-4 yr) Medium (5-9 yr) Long PoP (10+ year)
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 1 3 7 10
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 5 6 7 8
Scope - Extent of service provided unknown / not applicable Single ILS Element Low Hybrid - 2-5 High Hybrid - 6+ ILS elements Full Spectrum Support / Turn-Key Solution
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 10 6 5 8
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 10 7 5 7
Contract Type unknown / not applicable Cost Plus - Fixed Fee (Not Outcome based) Cost Plus Award Fee - subjective metrics Cost Plus Incentive Fee - (quantitative metrics) Fixed Price Incentive (cost/performance) Fixed Price Award Fee/Term (subj met) Firm Fixed Price (penalties) Firm Fixed Price (pen + gainsharing)
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 1 4 5 6 8 9 10
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 7 5 4 6 6 9 8
Obsolescence Management / Modernization and Upgrade unknown / not applicable No consideration Basic Obsolescence Mgmt Adv Obs Mgt - Tech Refresh Mods/Upgrades
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 1 4 8 10
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 3 4 5 6
Number of Nations unknown / not applicable 1 2 3 4+
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 5 5 5 5
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 5 5 5 5
Contractor OBC Experience unknown / not applicable Low Medium High
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 2 5 8
Digital value for Non-OBC (10 best, 1 worst, 0 means disregard) 0 5 5 5
Reserved for further extension (1) unknown / not applicable Dummy value 1 Dummy value 2 Dummy value 3 Dummy value 4 Dummy value 5
Digital value for OBC (10 best, 1 worst, 0 means disregard) 0 5 5 5 5 5