An Alternative Approach to DO-178B

Trusted to deliver excellence

© 2016 Rolls-Royce plc and/or its subsidiaries

The information in this document is the property of Rolls-Royce plc and/or its subsidiaries and may not be copied or communicated to a third party, or

used for any purpose other than that for which it is supplied without the express written consent of Rolls-Royce plc and/or its subsidiaries.

This information is given in good faith based upon the latest information available to Rolls-Royce plc and/or its subsidiaries, no warranty or representation

is given concerning such information, which must not be taken as establishing any contractual or other commitment binding upon Rolls-Royce plc and/or

its subsidiaries.

An Alternative Approach to DO-178B

Duncan Brown

Rolls-Royce Engineering Fellow – Safety Critical Software

What is DO-178? 2

• Guidance (or guidelines) for software in airborne systems

• A standard??

• A set of objectives and activities to achieve them

• A collection of best practices from the late 1970’s for safety

critical software

• A somewhat arbitrary relaxation of these practices by Design

Assurance Level

• A tried and trusted acceptable means of compliance for

airborne software which has saved thousands of lives!

Why DO-178C?

• Best practice has moved on from 1980 and often now

includes:-

- Model Based Development

- Formal Methods

- Object Oriented Design

- Extensive support tooling (Static analysis, auto-code

generation, simulation etc.)

• Because of the success of DO-178B the terms of reference

included minimal change to the core

• So the concept of “Supplements” was developed with an

idea that future supplements could be created without

change to the core document

3

DO-178C – Rationale, Change and TearsTiers

• A sub-group was created on DO-178C to try to recall the

rationale behind the objectives but this never materialised

• A proposal for a goal based or safety case approach to

the DO-178 objectives

• A proposal by the Formal Methods group to remove the

testing “means” from the core document and to have a

supplement for the traditional approach

• A proposal (IP217) to abstract DO-178 into a process that

iterated around multiple tiers of requirements/design until

code emerged

Note that this concept re-appeared in 2014 in a paper by Mike

Dewalt called “Technology Independent Assurance Method or TIAM”

4

Perceived Problems (As presented by the FAA to GAMA in 2015)

• Product based certification leads to multiple products being

separately scrutinised

• Prescriptive domain specific detailed objectives in standards

such as DO-178C, DO-254 and ARP4754 preclude or make

difficult alternative approaches

• Separation of System, Software and Complex Hardware

disciplines within the authorities and the applicants causes

wasted effort

• No need for four DAL levels, A is very close to B and C to D

5

What is GAMA?

• The General Aviation Manufacturers Association

• Founded in 1970 to “foster and advance the general welfare,

safety, interests and activities of general aviation”

• Worldwide committee based organisation with head quarters

in Washington and Brussels

• Scope includes general aviation aircraft (Part 23) and more

recently rotorcraft (Part 29)

6

Why GAMA? 7

An example review of DO-178C from a “GA” perspective

• Activities should not be there, standard should be objectives only

• Process standards rather than product standards would be better

• Have one set of objectives for all levels of requirements and design

• Remove Parameter Data Item objectives

• Remove structural coverage objectives

• Remove data and control coupling objectives

• Eliminate the requirement for traceability data

• Remove configuration index documents

• Have QA audit against company standards

8

What is Streamlining?

• Harmonization – The FAA and EASA should employ

harmonized approaches to certification

• Move to process based audits where a company can be

shown to have a good, mature process and that it is being

re-used on a number of projects

• Create some domain independent goals that all certifications

must satisfy to allow alternative approaches with appropriate

justification

• Audit for systems, software and complex hardware in

parallel

• Look at revising the number of DALs to two

9

The FAA Initiative – Streamlining Workshop(s)

• October 2015 FAA (Software CSTA Mike Dewalt) sent out

work shop invites to a number of people with a structured

distribution (Countries, industry sectors etc.)

• First workshop (and the only one planned originally) was in

December 2015

• The plan given by Mike Dewalt at the first meeting was to

conclude on the number of DAL levels and a set of less than

10 “meta-objectives” by the close of the meeting and to take

these to an open FAA conference in April 2016

• The idea was to issue an Advisory Circular in the autumn of

2016!

10

Mike Dewalt’s Vision 11

More FAA workshops….

• At the end of the December meeting it was decided that:-

- No real advantage in reducing the number of DALs

- We needed more time on meta-objectives

• Another meeting was arranged for April 2016

• At the end of the April meeting:-

- A plan for an FAA conference in September 2016 had

been firmed up

- It was decided that we needed another meeting on meta-

objectives (and they probably weren’t really meta-objectives)

• Another meeting was arranged for July 2016

12

Final FAA workshop

• In July 2016 we held the last workshop which:-

- Concluded with a set of three “Overarching Properties”

- Prepared material to disseminate the information at the

September conference

• The “open” FAA conference was held in September 2016 with ~225

attendees (Workshop members, cert authorities and technical

representatives from industry such as DERs)

• The meeting conclusions were not clear, however general consensus

that

- The three OPs are logically correct

- They are probably too abstract to be useful without additional

information/training etc.

- They do not help much in solving “variation” across FAA and

EASA (in fact they may make it worse)

13

The European Initiative - RESSAC

• The FAA initiative was international however it was decided

that a separate European approach would be sensible

• IRT St. Exupery in Toulouse launched a research project in

early 2016 involving representatives from industries in

Europe

• The original proposal was a two year project to come up with

- A set of “meta-objectives” (or Overarching Properties)

- Criteria for how the evidence against these could be

judged

- A worked case study against these OPs

14

RESSAC and AeroSpace and Defence Industries Association of Europe (ASD)

15

Overarching Properties (aka Meta Objectives) 16

Desired System Behaviour

Defined Intended Function

Implementation

Requirements Capture

Development

Inten

t N

ecessity Co

rrec

tnes

s

Progress

• The FAA workshops have:-

- Made a decision to continue with four DALs

- Refined three Overarching Properties in a standardised

form

17

Overarching Property Statement:

Correctness: The implementation is correct with respect to its defined intended functions, under foreseeable

operating conditions.

Definitions: words / phrases in the Overarching Property description

a. Implementation: Item or collection of items contributing to system realization, for which acceptance or

approval is being sought; item (from ARP 4754A) is a hardware or software element having bounded and

well-defined interfaces.

b. Defined intended functions: The record of the system needs and constraints as expressed by stakeholders.

c. Foreseeable operating conditions: External and internal conditions in which the system is used,

encompassing all known normal and abnormal conditions.

Pre-requisites: which must exist to allow Overarching Property satisfaction to be shown

a. Defined intended functions exists.

b. The implementation of the functions exists.

c. The record of the foreseeable operating conditions exists.

Constraints: on how Overarching Property satisfaction must be demonstrated

a. The process to satisfy this Overarching Property must be defined and conducted as defined.

b. When tiers of decomposition are used, the means of showing correctness among the tiers and to the

defined intended functions must be defined and conducted as defined.

c. The implementation must be correct when functioning as part of the integrated system or in

environment(s) representative of the integrated system.

d. Criteria for evaluating the artifacts are defined and shown to be satisfied individually and collectively.

e. All artifacts required to establish the Overarching Property are under configuration management and

change control.

f. All design and manufacturing data to support consistent replication of the type design and instructions for

continued airworthiness must be established.

Assumptions: which need only be stated, not justified (if any)

None.


Intent: The defined intended functions are correct and complete with respect to the desired system behavior.


a. Desired system behavior: System needs and constraints expressed by the stakeholders.

b. Defined intended functions: The record of the system needs and constraints as expressed by stakeholders.

c. Failure Condition(s): A condition having an effect on the aircraft and/or its occupants, either direct or

consequential, which is caused or contributed to by one or more failures or errors, considering flight

phase and relevant adverse operational or environmental conditions or external events. (From ARP

4754A)



b. Failure conditions are defined for the aircraft systems.

c. Design Assurance Levels (DALs) are assigned using the failure condition classifications.



b. The defined intended functions must address the failure conditions.

c. Criteria for evaluating the artifacts are defined and shown to be satisfied individually and collectively.

d. All artifacts required to establish the Overarching Property are under configuration management and

change control.


a. Stakeholders have the system knowledge to express the desired system behavior.

b. Performing system safety assessments is not covered by these Overarching Properties.


Necessity: All of the implementation is either required by the defined intended functions or is without

unacceptable safety impact.


a. Unacceptable Safety Impact: An impact which compromises the system safety assessment.

b. Defined intended functions: The record of the system needs and constraints as expressed by

stakeholders.

c. Implementation: Item or collection of items contributing to system realization, for which acceptance or

approval is being sought; item (from ARP 4754A) is a hardware or software element having bounded

and well-defined interfaces.



b. The implementation or a representation of the implementation exists.

c. The system safety assessment exists.



b. The system safety assessment must address all of the implementation.

c. Criteria for evaluating the artifacts are defined and shown to be satisfied individually and collectively.

d. All artifacts required to establish the Overarching Property are under configuration management and

change control.


a. For a TSOA appliance there may not be a complete system safety assessment for the final installation

at the appliance level.

Next Steps

• RESSAC will continue as a research project until mid 2018

and publish the deliverables at that point

• FAA has decided to continue the workshops both virtually

(as a telecom with collaboration on a web site) and as face

to face workshops possibly US and Europe

• The FAA has also re-defined the team working this to allow

all members of RESSAC to be involved as well as a wider

invite to GAMA / AIA / ASD and others

• The aim for the FAA still seems to ultimately be an Advisory

Circular

• It is not yet clear what ASD and EASA might do with the

RESSAC outputs

18

Summary

• It is generally accepted that there is still work to do in harmonizing the

certification authorities both internally and globally

• It is felt that existing guidance for compliance to regulations in some

circumstances can incur disproportionate effort

AND

could inhibit or even preclude innovative approaches in systems,

software and complex electronics which could improve safety

• There is broad agreement about three “Overarching Properties” to be

met to comply with regulation

BUT

It is still a challenge to understand how these can be applied

harmoniously! Key to this are clear criteria to judge the approaches

and the evidence.

19

Timeline 20

1992 2000 2005 2010 2015 2020

https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19990070314.pdf

Questions 21