Validation & Verification of Safety Critical Systems in ... · Validation & Verification of Safety Critical Systems in the Aerospace Domain. ... •Modular Electronics ... Vehicle

1

Validation & Verification of Safety CriticalSystems in the Aerospace Domain.

Workshop: Teststrategien und -techniken für Onboardsysteme in der Luft- und Raumfahrt 07.10.2008Dipl. Ing. Jörg Hofmann

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 2

Structure.

Challenges in Aerospace and Automotive Fields of Activities V&V Strategy – Theory Requirements Elements of the Strategy Optimization

V&V Strategy – Experience Effort – Bad Case – Good Case

Example „Early Verification“ Example „End-to-End Test“ 2 Automation Concepts

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 3

Challenges Aerospace.

•Digital Fire Control/NAV•PT-PT Wiring•Mechanically Controlled Sensors/FLT Controls/ Displays•Crew-Dominated Operation

•Functionally Integrated Data Processing-NAV/WD/Air Data Sensors-Flight Control

•Beam Steering Sensors•Fly By Wire•Dedicated Digital Processing•Crew-Assisted Operations- Weapon Delivery- Automated TF/TA- EW Response

•Aircraft-Wide Information Integration- Sensors/Stores/ Vehicle/Propulsion

•Modular Electronics•Massive Data Bases- Terrain, Threat

•Digital Sensor Processing- Sensor Fusion- Hyperspectral Imaging

•Integrated Diagnostics/System Fault Tolerance

•System Data Security•Limited UAV Autonomy

•Platform Exploitation of Global Information- Information Mining- At-A-DistanceReconfiguration

•Autonomous Vehicle Emphasis- Air & Space

•Air Crew/ Ground CrewMonitoring & Management•Automated Functions- ATR (Multi-Sensor)- Failure Prognostics- Route/ Sensor/ Weapon/ Vehicle Coordination

- Bistatic Sensing(Air/ Space)

- Threat Evasion

DEDICATED SUBSYSTEMS

FEDERATED SUBSYSTEMS

INTEGRATED SYSTEMS

SYSTEM of SYSTEMS

1958 1950’s - 60’s

1990’s - 00’s

2000

1970’s - 80’s

64 KB

1 MB

1 GB

100 MB

Radar

Comm

EW

Integrated AvionicsMission

RadarComm

NAV

Mission

Federated Avionics

Radar

Comm

NAV

Independent Avionics

Source: AFRL

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 4

ACC Stop&GoBFDALCKSGInternet PortalGPRS, UMTSTelematicsOnline ServicesBlue-ToothCar OfficeLocal Hazard WarningIntegrated Safety SystemSteer/Brake-By-WireI-DriveSpurhalteunterstützungPersonalisierungForce Feedback Pedal

Elektronische EinspritzungCheck ControlGeschwindigkeitsreglerZentralverriegelung...

Elektronische GetriebesteuerungElektronische KlimaregelungASC Anti Slip ControlABS Anti Blocking SystemTelefonSitzheizungssteuerungAutom. Spiegelabblendung

NavigationssystemCD-WechslerACC Active Cruise ControlAirbagsDSC Dynamic Stability ControlAdaptive GetriebesteuerungRollstabilisierungXenon LichtBMW AssistRDS/TMCSpracheingabeNotruf

1970 1980 1990 2000

Challenges Automotive.

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 5

System LevelSystem Definition

System Decomposition

Qualificat. Test

System Test

Integration Test

SW DomainTest B

ed Domain

System-SW-Implementation

System-SW-Requirements

System-SW-Design

System-SW-integration

System-SW-Test

Test BedRequirements

Test Bed Design

Test BedImplementation

Test BedIntegration

Test BedQualification

HW ComponentImplementation

HW ComponentRequirements

HW ComponentDesign

HW ComponentTest

HW ComponentIntegration

Effort.

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 6

Structure.

Challenges in Aerospace and Automotive Fields of Activities V&V Strategy – Theory Requirements Elements of the Strategy Optimization

V&V Strategy – Experience Effort – Bad Case – Good Case

Example „Early Verification“ Example „End-to-End Test“ 2 Automation Concepts

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 7

Stakeholder reqs.definition

Requirementsanalysis

Architecturaldesign SW

design

Implementation

Integration

Verification

Validation

Projectplanning

Projectassessment

Projectcontrol

Riskmanagement

Decision makingprocess

Informationmanagement

Modeling andsimulation

Trade studies

Configurationmanagement

Safety andcertification

Support

Technical

Project

Standardization

Tooling

Fields of Activities.

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 8

Stakeholder reqs.definition

Requirementsanalysis

Architecturaldesign SW

design

Implementation

Integration

Verification

Validation

Projectplanning

Projectassessment

Projectcontrol

Riskmanagement

Decision makingprocess

Informationmanagement

Modeling andsimulation

Trade studies

Configurationmanagement

Safety andcertification

Support

Technical

Project

Standardization

Tooling

Fields of Activities.

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 9

Contract

System

Function

Item

System Design&Implementation System Test&Integration

•Val

•Ver

•Spec •Int

•Spec•Ver

•Ver

•Spec

•Int

•Int

•Val

•Val •Ver

•Int•Spec•Val

•Code •Int

Process.

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 10

What to Deal With - Methods of Validation and Verification.

Simulation Analysis, engineering judgement Similarity of requirements or design Demonstration, prototyping or mock-up Reviews or audits Inspection Test Operational trials

Flight test Aircraft ground test System integration test HW-SW integration test

(Bench) SW-SW integration test Coding unit test

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 11

Structure.

Challenges in Aerospace and Automotive Fields of Activities V&V Strategy – Theory Requirements Elements of the Strategy Optimization

V&V Strategy – Experience Effort – Bad Case – Good Case

Example „Early Verification“ Example „End-to-End Test“ 2 Automation Concepts

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 12

Let‘s Find an „Optimized V&V Strategy“.

From Theory……… ……. To Experience

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 13

Requirements on an Optimized V&V Concept.

“Sufficient” Test Coverage

Sufficient Evidence of the System Safety

Limitation of the Effort to Reasonable Budgets

Consideration of the Project Development Phase

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 14

Elements of a Good V&V Strategy.

Verify requirements

&functions early

Coordinate all test stages

Realize the coherence of

functions and test

Establish automated tests early

Use data bases and generators

Realize end to end tests

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 15

Essential Columns of the V&V Strategy – Focusing.

Use the specific advantages of each test stage

Automate TestsAdequate to the

Project

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 16

Verify requirements&functions early Safe Money

Realize end to end tests Assure user's needs

Coordinate all test stages Integrated test concept

Realize the coherence offunctions and test Coverage and traceability

Optimized Strategy (1).Use the specific advantages of each test

stage

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 17

Find out the project adequate Reduction of initial and automation method repetitive effort

Establish automated tests early Reduction of repetitive effort

Use data bases and generators Reduction of document effort

Automate TestsAdequate to the Project

Optimized Strategy (2).

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 18

Structure.

Challenges in Aerospace and Automotive Fields of Activities V&V Strategy – Theory Requirements Elements of the Strategy Optimization

V&V Strategy – Experience Effort – Bad Case – Good Case

Example „Early Verification“ Example „End-to-End Test“ 2 Automation Concepts

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 19

……. To Experience

Experience.

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 20

System LevelSystem Definition

System Decomposition

Qualificat. Test

System Test

Integration Test

SW DomainTest B

ed Domain

System-SW-Implementation

System-SW-Requirements

System-SW-Design

System-SW-integration

System-SW-Test

Test BedRequirements

Test Bed Design

Test BedImplementation

Test BedIntegration

Test BedQualification

HW ComponentImplementation

HW ComponentRequirements

HW ComponentDesign

HW ComponentTest

HW ComponentIntegration

Effort.

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 21

System Level

Poor validation of requirements

&design

„Bad Case“.

All test stages perfo

rm„m

oding and

display tests“ (redundantly!)

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 22

HW Domain

System Level

SW DomainEarly validation of

requirements&functions

All stages perfo

rmprim

arilytests adequate to

their specificcapabiliti

es (withcontro

lled

redundancy)Early

validation

End to end test

Testautomation

„Good Case“.

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 23

Structure.

Challenges in Aerospace and Automotive Fields of Activities V&V Strategy – Theory

Requirements Elements of the Strategy Optimization

V&V Strategy – Experience Effort – Bad Case – Good Case

Example „Early Verification“ Example „End-to-End Test“ 2 Automation Concepts

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 24

Example Early Verification - Terrain Referenced Navigation.

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 25

Emitter Locator

Sensor

Core

Computer

„EWTactical

Display“

Flight Control

Subsystem

Mission

Computer

Navigation

Subsystem

Missile Control

Computer

Example End-to-End Test – EW Chain.

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 26

Requirements

Model basedtests

e.g. UML

Code generation

Manualtests

„Captureand

replay“

Pool of automatic

tests

Two Automation Concepts.

07.10.2008Jörg Hofmann – Aerospace Information Technologies – München 27

Vielen Dank für Ihre Aufmerksamkeit!