Janvier 2010 Software aspects of aeronautical certification and static analysis tools Gérard LADIER Airbus / Aerospace Valley [email protected]@aerospace-valley.com09/2010.

Janvier 2010

Software aspects of aeronautical certification and static analysis

tools

Gérard LADIER

Airbus / Aerospace Valley

[email protected] 09/2010

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 2

Equipment Rules (JAR/FAR 25-1309)

• “Essential” equipment must be designed to perform its intended functions

• The airplane systems and associated components, considered separately and in relation to other systems, must be designed so that :

– The occurrence of any failure condition which would prevent the continued safe flight and landing of the airplane is extremely improbable, and

– The occurrence of any other failure conditions which would reduce the capability of the airplane or the ability of the crew to cope with adverse operating conditions is improbable

• ...

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 3

Means of conformance

Software

It is in general not feasible to assess the number or kinds of software errors, if any, that may remain after the completion of system design, development, and test. DO-178B/ED-12B, provides acceptable means for assessing and controlling the software used to program digital -computer-based systems ”

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 4

Define HLR's

Process T(q)

Define HLR(1)

Process T(q+1)

Define HLR(2)

Process T(q+2)

Produce Object Code

Process T(x)

HLR

HLR(1)+ Implementation detail

HLR(2)+ Implementation detail

Object Code

First principle

We can’t get clean water from a dirty pipe

Evidences are requested on the pipe …

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 5

DO-178/ED-12 : evidences on the pipe …

• “DO-178B/ED-12B is primarily a process-oriented document” => Set of requirements on the processes (dev., verif, etc.) and

their outputs

• “The occurrence of any failure condition which would prevent the continued safe flight and landing of the airplane is extremely improbable”

=> Evidences on the fulfilment of these requirements vary depending on the software “criticality” level

ObjectifApplicabilité par niveau

logicielProduit

Catégorie de contrôle par

niveau logicieldescription Réf A B C D Description Réf. A B C D

3 La couverture de test des exigences de haut niveau est assurée

6.4.4.1 Résultats de Vérification du Logiciel

11.14

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 6

Life cycle and processes

• Definition of separate processes that will be combined for a given project to describe its life cycle:

– Planning process (organization/plans rather than scheduling)

– Development process (specification, design, coding, integration)

– Integral processes (verification, configuration management, quality assurance, certification liaison process).

• Define for each process:– The Assurance objectives

– The means of achieving those objectives

– The process input data

– The process activities

– The process products

– The transition criteria, which must be met in order to proceed

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 7

Main common requirements on the developement processes• Standards must be written and evidences of compliance with the rules should be

provided• Rules ?

Dozen of documentsDefine precisely how to perform an activity (methods, means, constraints,

expected outputs, etc.

Each requirement

or design item should be

precise

verifiabledeveloped

tracable

consistent

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 8

Configuration management & Quality Assurance

To assist in satisfying general objectives to:

Control configuration of the software throughout the software life cycle

Be able to replicate the executable object code

Control process inputs and outputs during the software life cycle

Provide baselines for review, assessment and change control

Ensure problems management and change control

Ensure archiving and recovery.

To provide evidences that:What’s done is what’s described in plansTransition criteria are reachedA conformity review of the software is conducted

Main characteristics :IndependenceActive role during the life cycle process

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 9

Tools qualification -1

Necessary when processes required by the rest of

the document are eliminated, reduced

or automated by the use of

a deterministic software

tool whose outputs are not verified.

3 qualification criteria depending on the

risk associated to the tool usage :

Criteria 1 : Development Tool

Criteria 3 : verification tool

Criteria 2 : Verification tool which could have an impact on the resulting software :

used to justify the elimination or reduction of:Verification process other than that automated by the tool,or Development process which could have an impact on the

resulting software

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 10

Tools qualification - 2

– Combination of the qualification criteria with the software level to give the Tool Qualification Level:

Two distinct roles are defined :– The user

– The developper

• He defines his needs (Tool Operational Requirements-TOR) and validate the tool in its usage context

• He defines his development specification (Tool Requirement-TR), develops the tool and provides the life cycle data

Sharing of activities between these two roles are defined for COTS tools(11.3)

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 11

TQL 1, 2 et 3 :– TQL 4 ones + « implementation » requirements :

• Depending on the level of the final product software developed with the tool

TQL 4 : – TQL 5 ones + «project management» requirements :

• TOR reviews (complete, accurate, and consistent)

• Definition of processes in plans

• Definition of the TR and verification / TOR

• Verification of the tool / TR et requirements coverage

• Configuration and change managementsTQL 5 :

– «contracting authority» requirements, mainly concentrated in the TOR :• Table T-0 defining all the “user” requirements including the validation of the

tool versus the need expressed in the “avionics” PSAC

• Plus one requirement : «Impact of known problems on the TOR »

Tools qualification - 3

Qualification requirements :

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 12

Second principe

A clean pipe may not deliver clean water

Filters are piled to detect and eliminate impurities

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 13

Verification

• The most important section of DO-178/ED-12, in term of : volume : 13 pages of description ( ~ 5 pages for other processes )Workload incurred (A380 : 4 lines of test for line of embedded code …)

• Basic principles:

– “Integral” process => applies to all the development processes

– Combination of :

• Reviews ,

• Analysis

• Tests

to detect and identify errors introduced during development

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 14

Reviews ? Analysis ? Tests ?

Review : inspection of a product by an independant (level A) person; qualitative evaluation

Analysis : detailed examination of a process, potentially done by a tool quantitative evaluation

Test : running the software and comparison of actual outputs to expected ones

– Functional test

– NO TEST BASED ON CODE STRUCTURE

– Functional & Structural coverage analysis

• Three major tools for bug-busters :

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 15

DO-178/ED-12 - The verification process

Compliance: with requirements Conformance: with standards

A-3.2 Accuracy & Consistency A-3.3 HW Compatibility A-3.4 VerifiabilityA-3.5 Conformance A-3.7 Algorithm Accuracy

A-5.3 Verifiability A-5.4 ConformanceA-5.6 Accuracy & Consistency

A-5. 7 Complete & Correct

A-3.1 Compliance A-3.6 Traceability

A-4.1 Compliance A-4.6 Traceability

A-4. 8 Architecture Compatibility

SystemRequirements

High-LevelRequirements

SoftwareArchitecture

Source Code

ExecutableObject Code

(A-2: 3, 4, 5)

(A-2: 7)

(A-2: 6)

(A-2: 1, 2)

Low-LevelRequirements

A-5.1 Compliance A-5.5 Traceability

A-5.2 Compliance

A-6.5 Compatible With Target

A-6.3 Compliance A-6.4 Robustness

A-6.1 Compliance A-6.2 Robustness

A7 Verification of verification (Functional & Structural coverage)

A-4.9 ConsistencyA-4.10 HW Compatibility A-4.11 Verifiability A-4.12 Conformance A-4.13 Partition Integrity

A-4.2 Accuracy & ConsistencyA-4.3 HW Compatibility A-4.4 Verifiability A-4.5 Conformance A-4.7 Algorithm Accuracy

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 16

Third principe

Potentially opposite interests are at stake

Independant authorities assess the process and product

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 17

The ED-12B/DO-178B - certification liaison

• Objective: ensure effective communication/understanding between the applicant and the certification authorities

• Means:– The Plan for Software Aspects of Certification, given to the

Authorities as early as possible– Reviews carried out by the certification authorities

“software” specialists as much as they want

– Software Accomplishment Summary and Software Configuration Index.

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 18

Fourth principe

All the interests but must be taken into account to build a recognized set of requirements

DO-178/ED-12 is built and updated by all the concerned specialists and actors

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 19

Consensus on requirements

• Joint meetings between the RTCA SC-205 EUROCAE WG-71• Openness, consensus :

– More than 1200 people registered on the WEB site– about 120 attendees in each meeting : aircraft manufacturers, engine

makers, equipment suppliers, authorities, scientists and consultants – The final text has to be agreed by each of the attendees

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 20

DO-178C/ED-12C

DO-178C

ED-12C

Supplement A

Supplement B

Supplement …

...Supplement N

Interface Spec

DO-248C/ED-94C FAQ/DP/RATIONALE

DO-278A

/

ED-109A

• The « core document » will be completed by supplements :

Supplement –guidance used in conjunction with DO-178C/ED-12C that addresses the unique nature of a specific technology or a specific method. A supplement adds, deletes or otherwise modifies: objectives, activities, explanatory text, and software life cycle data in DO-178C/ED-12C.

• This structuring principle enables incremental future development of software requirements

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 21

The DO-178C/ED-12C : evolution of the content

• Major improvements are in the supplements : – Object Oriented technology – Model Based Development– Formal Method Technology– Tools Qualification

• In the core document : – Basically clarification and improvement in consistency and

accuracy. – Except for the tool qualification part :

• Separation of the «why ?» part (>in the core doc) from the «how ?» part (>supplement)

• Significant evolution of the « why ? »

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 22

Formal Methods Technology supplement

• The FM introductory section insists on the interest of using FM for verification

• The FA (Formal Analysis) may completely replace :

– Reviews & analysis (except for validation of « derived requirements »)

– Conformance test versus /HLR et /LLR

– Software integration tests

– Robustness tests

• FA may help for the verification of compatibility with the hardware

• FA cannot replace HW/SW integration test

• The structural coverage objectives are achieved if it can be demonstrated that :

– Each requirement is completely covered

– The set of requirements is complete in regards of the attended function

– There is no non expected dependences between output and input data

– There is no dead code

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 23

Fitfh principe

Only requirements are mandatory, not the means

Building the « pipe » is to be dealt with by the suppliers

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Which industrial use for Static Analysis tools ?

Today

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 25

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Not so frequent …

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

How to convince certification Authorities ?

Example of the “unit level proof of LLR”• Discussion with Cert. Auth. software specialists much before the

actual use of the tool, to get a general feedback (go/no go)

• Demonstration of the soundness of the approach

• Definition of specific rules and standards

• Demonstration of the completeness of the properties

• Definition of LLR as pseudo-code + properties

• Smooth integration in the overall verif./traceability processes

• Detect and eliminate dead code

• Verify the executable code

• Qualify the tool (verification ++ tool)

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Which industrial use for Static Analysis tools ?

Tomorrow …

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Tomorrow … 1

• Proof of absence of Real Time Execution error :– ASTREE (ENS/ABSINT)

• For effective certification credit

• Precision of Floating-point calculus : – Fluctuat (CEA) :

• Abstract Interpretation based; analysis of C or assembly code

• Safe computation of the numerical (rounding) errors introduced by basic operators or input filtering code

• In use by Airbus for evaluation :– Research prototype for the C language– Pre-industrial prototype for the TMS320C33

assembly language

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Tomorrow … 2

• Validation of the compilation   – Lcertify (ENS) :

• Research activity in Airbus

– Compcert (INRIA) : • Industrial prototype available• Efficient compilation of a complete subset of the A380 fly-by-

wire software

• Other “translation validation tools” would be highly desirable (e.g. Scade -> C)

• Various analysis of C code : – FRAMA-C (CEA) :

• Plugin WP to succeed CAVEAT • Plugin TASTER for syntaxic control (coding rules enforcement)• Plugin for data flow/control flow verification, coming soon

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Special thanks …

• To the co-chairs of the fantastic formal methods SG of the DO-178C joint committee: – Kelly Hayhurst (NASA) – Duncan Brown (Rolls Royce)

• To the Airbus Formal Methods dream team : – Famantanantsoa Randimbivololona– Jean Souyris– David Delmas

• And to Hervé Delseny from Airbus, who liaised both …

Page 32Made on http://www.wordle.net

© A

IRB

US

FR

AN

CE

S.A

.S.

Tou

s dr

oits

rés

ervé

s. D

ocum

ent

conf

iden

tiel.

Page 33Made on http://www.wordle.net