DO-178 (B, C) and Model Based Development · Page 3 Defence Electronics What is Model Based Development and what is the benefit? • The development processes that provide the …

Defence Electronics

DO-178 (B, C) and Model BasedDevelopment

Cheryl DorseyEADS DE FN

Page 2

Defence Electronics

Agenda

• What is Model Based Development (MBD)?

• Can MBD be used to meet civil airbornecertification requirements?

• Ways in which models are used today– System and Safety

– Software

– Equipment

• Certification concerns to be addressed

• DO-178C and MBD

Page 3

Defence Electronics

What is Model Based Development and what is the benefit?

• The development processes that provide the ability to graphically represent requirements, specifications and designs using domain-specific notations and simulate the resultant behavior for validation purposes.

• Used correctly it can yield substantial benefit in lead time reduction and “right first time”developments. Coupled with increasingly sophisticated code generation technology, model-based development can dramatically shorten software development time and cost

Page 4

Defence Electronics

Strengths of Model Based Development

• Can support validation activity (are these the right requirements) provided it is not the requirements –validation of specification early.

• If used to design, a model can implement requirements in a deterministic manner

• If implementing a state machine, it can help assure the states are mutually exclusive.

• Model diagrams can be easier to understand conceptually (big picture)

• Coding is fast

Page 5

Defence Electronics

Can MBD Be Used to Meet Civil AirborneCertification Requirements?

• To date models have been used to meet certification requirements.

• However the criteria, process, and data required are dependent on:– What the model is used for - system, software, modeling

architecture (HW, SW), producing test cases, requirements validation, algorithm correctness

– Which certification requirements the model will be used for

– Which DO-178B objectives are met using the model

Page 6

Defence Electronics

OEM Problems Found In Model Based Development • Software developers or airframe companies used model-based

development methods and tools to produce model-based designs and embedded flight code, but they failed to document the functional, performance and safety requirements of the system

• Outputs from this approach were large and complex model based designs covering dozens of pages for which– there were no higher-level requirements against which the designs could be reviewed,

tested or traced.

– reviews of the designs were ineffective because they failed to reveal whether the design complied with higher level requirements (since there were none).

– reviews and tests failed to show whether any requirements were unimplemented, or whether the design contained any unintended functionality.

Based on an Issue Paper IP-4 dated 16 Marz 2006 by Andrew Bridge, EADS DE Ulm (formerly of Bombardier Aerospace).

Page 7

Defence Electronics

OEM Problems Found In Model Based Development

• Tests failed to show whether the code met the requirements of the system, or whether the code had the correct functional behaviour

• It was found that model-based systems developed without higher level requirements had suffered from more software problems over a longer period than conventionally designed systems

– many software changes were needed to correct the functionally of the systems and to add functionality that was missing from the original designs.

– this appears to confirm that the development and verification processes used in these cases were deficient at capturing errors and ineffective at revealing requirement or design errors.

Based on an Issue Paper IP-4 dated 16 Marz 2006 by Andrew Bridge, EADS DE Ulm (formerly of Bombardier Aerospace).

Page 8

Defence Electronics

EASA Certification Review Item (CRI)

• Verification and Validation of formalized requirements.

- Cites ARP4754- Completeness and correctness are at issue- Structural Coverage - Independence- Must be requirements at a higher level than the model

for what the model is supposed to do functionally

Page 9

Defence Electronics

CRI Cont‘d

• Completeness and correctness

– compliance

– testability,

– conformity to standard rules the objective of which being to ensure that the rules defined for the specification language use have been

followed

• checks made using methods such as

– Reviews

– Analysis

– Simulations

– tests.

• Traceability to higher specifications

• This process and results will be documented.

Page 10

Defence Electronics

Implementation Allocated

Functions and Requirements

Functional System

System Design

Function, Failure &

Safety Information

Safety Assessment Process

(ARP4761)

Software Development (DO-178B/ED-12B)

Equipment and Hardware

Development (ED-80/DO-254)

System Development Processes(ARP4754/ED-79)

Intended Aircraft Function

ABD 200

ABD 100

System Requirements Analysis System Requirements ValidationSystem Requirement VerificationSystem Algorithm DesignSystem Architecture DesignSystem WCET Analysis

Software Requirements ElicitationSoftware Requirement Verification

• functional test case generation• Structural test case generation

Architecture Analysis and Design• Meeting Design Standard

WCET Analysis (Software)Automated Source Code Generation

• Code review• Meeting Code standard

Page 11

Defence Electronics The Systems Process

Customer Spec (PTS)

System Spec (SRD,

ICD)

HardwareHWCI

System DesignArchitecture

System and Equipment Specification (SES)

SoftwareCSCI

Safety - HW/SWTradeoffs

System Analysis

Propose System

SoftwareCSCI

HardwareHWCI

SES - Validation of Requirements

Produce System Level Plans

Produce EquipmentLevel Plans

Done Through Modeling

Page 12

Defence Electronics

Models Used For System Development

• Models used at the System Level (ARP 4754)

– Here the models are used to elicit the correct requirements

– Model complex algorithms

– Used to validate the system requirements

– Can be used to generate tests at the system level, can also be used

for some of the software requirements based tests (if traceable)

• Avoid incestuous use of the model (CRI)

– Can not use the same model for system activities and softwareactivities (outside of some test case generation)

– Currently, system models can not be used to meet DO-178B, as a

structured development process is required for software but notfor systems development

Page 13

Defence Electronics The Software Process

Requirements Allocated to

Software

Document DesignArchitecture

Generate Code

ModelRequirements

Link OS and Libraries

Software MBD Plans and Stds

Done Through Modeling

Review Mode (LLR or HLR)?lTrace to Higher Level Functional ReqAssure Req Std is met

Review Code or Qualify ToolAssure Code Std is met

Review Code or Qualify Tool

Model Based Testing

Review Tests or Qualify ToolConduct Data and Conrol CouplingConduct Structural Coverage Analysis

Page 14

Defence Electronics

Models Used for Software Development

• When models are used for software development

– They need a set of functionally specified requirements from which themodel was derived.

– There must be traceability from the model elements to thoserequirements

– Reviews of the models that fullfill the objectives of DO-178B tables A-3,

A-4, and A-5 must be conducted (or the tool qualified for the objectivesthat the tool assures)

– Code generated from the models must be traceable to the model – if

manual code is added then this affects the tool qualification (additional work required.)

– Testing and verification that meet DO-178B tables A-6 and A-7 still

apply

– CM, QA, and Certification Liaison do not go away

Page 15

Defence Electronics

Structural Coverage Using Models (DO-178C draft paper)

• Must be based on requirements-based verification test cases (test cases must be based upon the requirements.)

• Test cases must meet all of the standards of requirements-based verification

– Coverage

– Test Cases and Procedures Reviewed for correctness

– Expected Results are required

– Actual results must meet expected results

• There must be independence:

– Between the tools providing the model structural coverage of the model and the

generation of the code, and the tool providing the direct link between the model

constructs and the source code constructs and the generation of the code.

In order to meet the DO-178B criteria the following must be true:

Page 16

Defence Electronics

Structural Coverage Using Models (DO-178C draft paper)

• The model contains both the data and the control flow of the software.

– A direct link from the model constructs, from which the structural coverage

was performed to the source code constructs should be established.

– Data and control flows of the code must accurately represent the data and

control flows of the model.

– There must be a direct one to one correlation between the source code

generated from the model and the constructs of the model.

• That the tool supplied structural coverage data of the model is correct, accurate and complete. This requirement is identical to requiring that structural coverage on source code be correct, accurate and complete.

Page 17

Defence Electronics

Structural Coverage Using Models (DO-178C draft paper)

• Additional verification of compiler generated object code not directly traceable to the model (as with non-model based development.)

• DO-178B Section 6.4.4.3 Structural coverage analysis required. Structural coverage analysis reveals structure not exercised during the requirements-based verification. Resolution is required.

– Need to assess why coverage was not achieved by requirements based tests.

– Dead paths in the model is identical to “Dead Code”. The constructions within the

model should be removed or modified achieving the required coverage..

– Deactivated model constructs: Assure inadvertent execution is prevented,

isolated, or eliminated. There must be a requirement(s) for deactivation,

Page 18

Defence Electronics

Issues and Concerns• What is a requirement, and can a model be used for requirements?

• Can the models and the code and tests they produce, be configuration managed as specified by DO-178B?

• How is traceability maintained and can it be used for regression analysis and testing if a change to the model is a change to the whole source?

• Does the code produced by a model allow for all the range checking (boundary testing) required by DO-178B?

• When testing a model are you really testing requirements or are you testing the implementation?

– If a tool provides code that meets the picture/model when you test the model are you

testing if the functionality is what is really wanted, or are you testing what you know?

Page 19

Defence Electronics

Requirements

• Can requirement specified solely as a model address the following:- Functionality? - Performance?- Safety?

• Are the requirements uniquely identified?

• Can derived requirements be identified, so as to be analyzed for safety?

Page 20

Defence Electronics

Configuration Management

• Not all modelling tools provide the CM capabilities required by DO-178B

• If it is a pictures to code modelling tool, how do you show and manage changes in the source, if generally you just get a whole new source image?

• Can you manage changes to just pieces?

• How do you assure what was supposed to change changed – and nothing else?

Page 21

Defence Electronics

Traceability

• If no functional requirements exist, how do you trace an implementation to an implementation?

• How do you show the implementation meets the desired functionality specified by the OEM and intended by the system?

• Modelling tools generally do not provide requirements/design to code traceability

- If done, when the code is regenerated from the model, how do you know the original requirements are correct?

- If it is retraced each time, can you use the trace data for regression analysis and testing during the change process?

Page 22

Defence Electronics

Functional Testing

• Given the CM and Traceability issues, will a change to the model require total re-testing?

• Some models provide test cases as well – where is the independence then?

• Will the testing test the model or the function?Answer – only if the model is correct – so then why are we testing?

• How is structural coverage assured?

• How is robustness testing assured? (Out of range testing) – the code produced by the models may not support this.

• The models and testing are often done on a desktop, independent of the real-time target environment.

Page 23

Defence Electronics

Goal of DO-178C Committee

• The committee Terms of Reference stress minimizing the changes to the current DO-178B document.

• Proposal to provide technology-based supplements.

• Model-based development Working group consists of airframers, equipment manufacturers, academics, modelling tool manufacturers

Page 24

Defence Electronics

Working Group Issues – What is a Model?

• Many different types of model used for differing purposes – the same model often used differently (e.g. UML, State Machine, Mathematical.

• Some model system requirements, some software requirements, some design, some algorithms, some implement state machines.

• Can a single set of guidance be written that covers all DO-178B aspects for all these types of model, and ways in which they wiil be used?

Page 25

Defence Electronics

What DO-178B Objectives Does the Model Address?

• Committee must come up with objectives that the models must address- Additional objectives or guidance may be required by the

committee

• Committee must a development process is followed if models are used for certification credit- Plans may need to address all model based issues and provide a

lifecycle with all the integral activities and there outputs defined

Page 26

Defence Electronics

Overlap of Three DO-178C Subgroups

MBD Object OrientedDevelopment

Tools

Page 27

Defence Electronics

Where Do We Go?

• Committee will be working on these issues and overall guidance for the next two years. It may be a difficult task to come to agreement with so many competing interests in the group.