Integra Consult A/S Safety Assessment Karachi, January 2006.

Integra Consult A/SIntegra Consult A/S

Safety AssessmentSafety Assessment

Karachi, January 2006Karachi, January 2006


SAFETY ASSESSMENTSAFETY ASSESSMENT

• A Safety Assessment is essentially a process for finding answers to three fundamental questions:– What could go wrong?What could go wrong?– What would be the consequences?What would be the consequences?– How often is it likely to occur?How often is it likely to occur?

• Once we know the answers this Once we know the answers this automatically raises the next question:automatically raises the next question:– Is this acceptable?Is this acceptable?– What can we do if not?What can we do if not?



• Consequently, the objective of Safety Consequently, the objective of Safety Assessments is to:Assessments is to:

– ensure that the system operates normally and without ensure that the system operates normally and without exposing unacceptable risks to anyone; exposing unacceptable risks to anyone;

– reduce and prevent incidents and accidents and; reduce and prevent incidents and accidents and; – limit the consequences of any occurrence that might occur.limit the consequences of any occurrence that might occur.

• The Scope of the Safety Assessments includes:The Scope of the Safety Assessments includes:– Safety Assessment on Air Navigation Systems covering Safety Assessment on Air Navigation Systems covering

people, procedures and equipment;people, procedures and equipment;– … … does not address Air Navigation System “certification” does not address Air Navigation System “certification”

issues;issues;– … … does not address organisational and management does not address organisational and management

aspects related to safety assessment.aspects related to safety assessment.



• SafetySafety– A condition in which the risk of harm or A condition in which the risk of harm or

damages is limited to an acceptable leveldamages is limited to an acceptable level

• RiskRisk– The probable rate of occurrence of a The probable rate of occurrence of a

hazard causing harm and the degree of hazard causing harm and the degree of severity of the harmseverity of the harm

– Risk = Severity * likelihoodRisk = Severity * likelihood

– Need to define severity and likelihoodNeed to define severity and likelihood– Need to define acceptabilityNeed to define acceptability


SEVERITY CLASSIFICATIONSEVERITY CLASSIFICATION

Severity Classification SchemeSeverity Classification Scheme

11 Accident • One or more catastrophic accidentOne or more catastrophic accident

• One or more mid-air collisionOne or more mid-air collision

• One of more collisions on ground between two aircraftOne of more collisions on ground between two aircraft

No independent source of recovery mechanism, such as surveillance or ATC / No independent source of recovery mechanism, such as surveillance or ATC / Flight Crew procedure, can reasonably be expected to prevent the Flight Crew procedure, can reasonably be expected to prevent the accident(s)accident(s)

22 Serious Incident • large reduction in separation (e.g. a separation of less than half the large reduction in separation (e.g. a separation of less than half the separation minima), without crew or ATC fully controlling the situation or separation minima), without crew or ATC fully controlling the situation or able to recover from the situation.able to recover from the situation.

• one or more aircraft deviating from their intended clearance, so that abrupt one or more aircraft deviating from their intended clearance, so that abrupt manoeuvre is required to avoid collision with another aircraft or with terrain manoeuvre is required to avoid collision with another aircraft or with terrain (or when an avoidance action would be appropriate).(or when an avoidance action would be appropriate).

33 Major Incident • large reduction in separation (e.g. a separation of less than half the large reduction in separation (e.g. a separation of less than half the separation minima), without crew or ATC fully controlling the situation or separation minima), without crew or ATC fully controlling the situation or able to recover from the situation.able to recover from the situation.

• Minor reduction in separation (e.g. a separation of more than half the Minor reduction in separation (e.g. a separation of more than half the separation minima), without crew or ATC fully controlling the situation, or separation minima), without crew or ATC fully controlling the situation, or able to recover from the situation, jeopardising the ability to recover without able to recover from the situation, jeopardising the ability to recover without use of collision or terrain avoidance manoeuvresuse of collision or terrain avoidance manoeuvres

44 Significant Incident

• Increased workload on ATCO or Flight Crew or slightly degrading capability of Increased workload on ATCO or Flight Crew or slightly degrading capability of the CSN systemthe CSN system

• Minor reduction in separation (e.g. a separation of more than half the Minor reduction in separation (e.g. a separation of more than half the separation minima), without crew or ATC fully controlling the situation, or separation minima), without crew or ATC fully controlling the situation, or able to recover from the situation and fully able to recover the situationable to recover from the situation and fully able to recover the situation

55 No immediate effect on safety

• No immediate direct or indirect impact on operationsNo immediate direct or indirect impact on operations


LIKELIHOOD CLASSIFICATIONLIKELIHOOD CLASSIFICATION

Likelihood Classification SchemeLikelihood Classification Scheme

11 Frequently Likely to occur frequently (Likely to occur frequently (oftenoften))

22 Probable Likely to occur several times during the life-time of the system (2-5 occurrences per year)

33 Occasional Occurs sometimes during the life-time of the system (1 occurrence per year)

44 Remote Unlikely to occur sometimes during the life-time of the system (1 occurrence per 5 years)

55 Improbable Very unlikely to occur (1 occurrence per 20 years)

66 Extremely Improbable

Extremely unlikely to occur (1 occurrence per 100 years)


RISK CLASSIFICATIONRISK CLASSIFICATION

Risk Classification

Probability Severity

Probability Qualitative Definition Quantitative

Definition 1 2 3 4 5

Frequently Likely to occur frequently. > 5*10-4 A A A A C

Probable Likely to occur several times during system life.

< 5*10-4 A A A B D

Occasional Occurs sometime during system life. < 1*10-5 A A B C D

Remote Unlikely to occur sometimes during system life.

< 1*10-6 A B C D D

Improbable Very unlikely to occur. < 1*10-7 B C D D D

Extremely Improbable

Extremely unlikely to occur. < 1*10-8 C D D D D

Likelihood

Likelihood


AS LOW AS REASONABLE AS LOW AS REASONABLE PRACTICABLEPRACTICABLE

• The risk is less than the pre-The risk is less than the pre-determined unacceptable limit,determined unacceptable limit,

• the risk has been reduced to a level the risk has been reduced to a level which is as low as reasonable which is as low as reasonable practicable (ALARP)practicable (ALARP) andand

• the benefits of the proposed system the benefits of the proposed system or changes are sufficient to justify or changes are sufficient to justify accepting the riskaccepting the risk

All three of the above criteria should be satisfied before a risk is classed as tolerableAll three of the above criteria should be satisfied before a risk is classed as tolerable



ICAO SEVEN STEP APPROACHICAO SEVEN STEP APPROACH• Hazard Identification and Estimation stepsHazard Identification and Estimation steps

– Step 1 – System and Environment DescriptionStep 1 – System and Environment Description– Step 2 – Hazard IdentificationStep 2 – Hazard Identification– Step 3 – Hazard SeverityStep 3 – Hazard Severity– Step 4 – Hazard LikelihoodStep 4 – Hazard Likelihood

• Mitigation stepsMitigation steps– Step 5 – Risk EvaluationStep 5 – Risk Evaluation– Step 6 – Risk MitigationStep 6 – Risk Mitigation

• DocumentationDocumentation– Step 7 – Safety Assessment Documentation Step 7 – Safety Assessment Documentation


STEP 1 - DESCRIPTIONSTEP 1 - DESCRIPTION

• Before a safety assessment can be Before a safety assessment can be performed, we need to describe the performed, we need to describe the ATM system being assessed. For that ATM system being assessed. For that purpose we need (as a minimum):purpose we need (as a minimum):

– System Description;System Description;

– Operational Environment Description.Operational Environment Description.



• A detailed system description should A detailed system description should include:include:– the purpose of the system;the purpose of the system;– how the system will be used;how the system will be used;– a description of system functions;a description of system functions;– the system boundaries and the external interfaces;the system boundaries and the external interfaces;– where appropriate, the transition procedures from the previous where appropriate, the transition procedures from the previous

system to the new system, including any hazards associated with system to the new system, including any hazards associated with the decommissioning of the previous system;the decommissioning of the previous system;

– description of contingency procedures and other procedures for description of contingency procedures and other procedures for non-normal operations;non-normal operations;

– other input such as other safety assessment results, occurrence other input such as other safety assessment results, occurrence and investigation reports, lessons learnt etc.;and investigation reports, lessons learnt etc.;

– regulatory framework and applicable standards.regulatory framework and applicable standards.



• A detailed operational environment description should A detailed operational environment description should include:include:– traffic characteristics;traffic characteristics;– weather characteristics & weather-related factors (e.g. weather characteristics & weather-related factors (e.g.

average frequency of diversions due to severe weather);average frequency of diversions due to severe weather);– topography;topography;– aircraft performance and equipment;aircraft performance and equipment;– infrastructure modes and limitations including e.g. runway in infrastructure modes and limitations including e.g. runway in

use, closed taxiways etc;use, closed taxiways etc;– environmental constraints;environmental constraints;– characteristics of the users of the system;characteristics of the users of the system;– adjacent centre capabilities;adjacent centre capabilities;– ……and other input concerning the environment in which the and other input concerning the environment in which the

system is to be operated.system is to be operated.


HAZARD IDENTIFICATION AND ESTIMATION PROCESS

hazard

hazard

hazardhazard

hazardhazard

Brainstorming – Hazard Identification

hazard

hazard

hazardhazard

hazardhazard

hazard

hazard

hazardhazard

hazardhazard

hazard

hazard

hazardhazard

hazardhazard

Brainstorming – Hazard Identification

hazard1 can lead to?hazard2 can lead to?hazard3 can lead to?

--

hazard1 can lead to?hazard2 can lead to?hazard3 can lead to?

--

Identification of Hazard Consequences

1. Introduction

2. Methodology

3. Operational Environment

4. Scenario

5. Classification Schemes

6. Example

Briefings what are the potentialwhat are the potentialconsequences?consequences?

what can go wrong?what can go wrong?

Catastrophic ?Major Incident ?

Negligible?--

Catastrophic ?Major Incident ?

Negligible?--

Identification of Severities

How severe can it become?How severe can it become?

Frequently ?Occasionally ?

Negligible ?--

How often can it occur?How often can it occur?

List of 10 most safety-critical hazardsIdentification of Likelihood of Occurrence

1

2 3

4

6

5


STEP 2 – HAZARD STEP 2 – HAZARD IDENTIFICATIONIDENTIFICATION

• PurposePurpose

– ……to identify what could go wrong!to identify what could go wrong!(- or anticipate problems before they occur…)(- or anticipate problems before they occur…)

– …….to identify the consequences (on safety) .to identify the consequences (on safety) of the hazardsof the hazards

A hazard is defined as any condition, event or

circumstances which could induce an accident

or incident (ICAO DOC 9422)

The equipment (hardware and software);

The operating environment; The human operators; The human machine interface (HMI); Operational procedures; Maintenance procedures; External services.


STEP 2 – HAZARD IDENTIFICATIONSTEP 2 – HAZARD IDENTIFICATION

• ……to identify the consequences of the hazards to identify the consequences of the hazards on operation!on operation!

– A hazard consequence is defined as the potential effects on operation that a hazard may create

• The The operational consequencesoperational consequences list the effects the hazard list the effects the hazard will have on the operation and emphasise the impact / will have on the operation and emphasise the impact / changes the hazard will introduce compared with “normal changes the hazard will introduce compared with “normal operation”.operation”.

• The The safety consequencessafety consequences are derived from the operational are derived from the operational consequences by deciding the impact on the safe consequences by deciding the impact on the safe provision of ATS. E.g. potential loss of separation.provision of ATS. E.g. potential loss of separation.

- increased receive/transmit- increased co-ordination

- increased receive/transmit- increased co-ordination

- potential loss of separation- potential loss of separation





• The hazard identification step should consider all The hazard identification step should consider all the possible sources of system failure. Depending the possible sources of system failure. Depending on the nature and size of the system under on the nature and size of the system under consideration these could include:consideration these could include:– The equipment (hardware and software);The equipment (hardware and software);– The operating environment (including physical The operating environment (including physical

conditions, airspace and air route design);conditions, airspace and air route design);– The human operators;The human operators;– The human machine interface (HMI);The human machine interface (HMI);– Operational procedures;Operational procedures;– Maintenance procedures;Maintenance procedures;– External services.External services.



• MethodologiesMethodologies

– Brainstorming;Brainstorming;

– Vision Conferences;Vision Conferences;

– Historical Records of Incidents;Historical Records of Incidents;

– Checklists;Checklists;

– Other systematic methods.Other systematic methods.



• Preferred MethodologyPreferred Methodology

– Brainstorming because:Brainstorming because:• Easy and straightforward process. No need to Easy and straightforward process. No need to

complicate or make too academic!complicate or make too academic!

• Such group sessions are usually good at generating Such group sessions are usually good at generating ideas and identifying issues – mutual inspiration;ideas and identifying issues – mutual inspiration;

• The interactions between participants with varying The interactions between participants with varying experience and knowledge tend to lead to broader, experience and knowledge tend to lead to broader, more comprehensive and more balanced more comprehensive and more balanced consideration of safety issues.consideration of safety issues.



WHAT IF?MODERATOR

ATCO

SYSTEMEXPERT

SAFETYEXPERT

• Brainstorming ProcessBrainstorming Process– interactive sessioninteractive session– facilitated by a moderatorfacilitated by a moderator– experts encouraged to bring experts encouraged to bring

forward any safety-related forward any safety-related issue they can think ofissue they can think of

– based upon pre-developed based upon pre-developed scenariosscenarios

– first step: identify hazardsfirst step: identify hazards– second step: identify second step: identify

consequences of the hazardsconsequences of the hazards



• ParticipantsParticipants– participants should be chosen for their expertise in fields participants should be chosen for their expertise in fields

relevant to the project being assessed.relevant to the project being assessed.

• Such experts usually includeSuch experts usually include– System users/operational expertsSystem users/operational experts: ATCOs and Flight : ATCOs and Flight

Crew (where necessary), to assess the consequences of Crew (where necessary), to assess the consequences of hazard(s) from an operational perspective;hazard(s) from an operational perspective;

– System technical expertsSystem technical experts, to explain the system , to explain the system purpose, interfaces and functions;purpose, interfaces and functions;

– Safety and human factors expertsSafety and human factors experts, to guide in the , to guide in the application of the FHA methodology itself and to bring application of the FHA methodology itself and to bring wider experience of the consequences of hazards.wider experience of the consequences of hazards.



EXAMPLEEXAMPLE


STEP 3 – SEVERITY ASSESSMENT

• The severity expresses the impact on The severity expresses the impact on operation or the harm an individual may operation or the harm an individual may suffer.suffer.

• Severity Classification is a gradation, ranging Severity Classification is a gradation, ranging from "worst case/accident" to "no safety from "worst case/accident" to "no safety impact" – expressing the magnitude of the impact" – expressing the magnitude of the consequence of the hazard.consequence of the hazard.

• Thus, a severity is allocated each hazard Thus, a severity is allocated each hazard consequence in accordance with the agreed consequence in accordance with the agreed severity classification scheme.severity classification scheme.


STEP 3 – SEVERITY ASSESSMENT

Severity Classification SchemeSeverity Classification Scheme

11 Accident • One or more catastrophic accidentOne or more catastrophic accident

• One or more mid-air collisionOne or more mid-air collision

• One of more collisions on ground between two aircraftOne of more collisions on ground between two aircraft

No independent source of recovery mechanism, such as surveillance or ATC / No independent source of recovery mechanism, such as surveillance or ATC / Flight Crew procedure, can reasonably be expected to prevent the Flight Crew procedure, can reasonably be expected to prevent the accident(s)accident(s)

22 Serious Incident • large reduction in separation (e.g. a separation of less than half the large reduction in separation (e.g. a separation of less than half the separation minima), without crew or ATC fully controlling the situation or separation minima), without crew or ATC fully controlling the situation or able to recover from the situation.able to recover from the situation.

• one or more aircraft deviating from their intended clearance, so that abrupt one or more aircraft deviating from their intended clearance, so that abrupt manoeuvre is required to avoid collision with another aircraft or with terrain manoeuvre is required to avoid collision with another aircraft or with terrain (or when an avoidance action would be appropriate).(or when an avoidance action would be appropriate).

33 Major Incident • large reduction in separation (e.g. a separation of less than half the large reduction in separation (e.g. a separation of less than half the separation minima), without crew or ATC fully controlling the situation or separation minima), without crew or ATC fully controlling the situation or able to recover from the situation.able to recover from the situation.

• Minor reduction in separation (e.g. a separation of more than half the Minor reduction in separation (e.g. a separation of more than half the separation minima), without crew or ATC fully controlling the situation, or separation minima), without crew or ATC fully controlling the situation, or able to recover from the situation, jeopardising the ability to recover without able to recover from the situation, jeopardising the ability to recover without use of collision or terrain avoidance manoeuvresuse of collision or terrain avoidance manoeuvres

44 Significant Incident

• Increased workload on ATCO or Flight Crew or slightly degrading capability of Increased workload on ATCO or Flight Crew or slightly degrading capability of the CSN systemthe CSN system

• Minor reduction in separation (e.g. a separation of more than half the Minor reduction in separation (e.g. a separation of more than half the separation minima), without crew or ATC fully controlling the situation, or separation minima), without crew or ATC fully controlling the situation, or able to recover from the situation and fully able to recover the situationable to recover from the situation and fully able to recover the situation

55 No immediate effect on safety

• No immediate direct or indirect impact on operationsNo immediate direct or indirect impact on operations


STEP 4 – LIKELIHOOD ASSESSMENT

• The likelihood of occurrence expresses how often the consequence of a hazard is likely to occur.

• Likelihood Classification is a gradation, ranging from "frequently" to “extremely improbable".

• Thus, a likelihood is allocated each hazard consequence in accordance with the agreed likelihood classification scheme.


STEP 4 – LIKELIHOOD ASSESSMENT

Likelihood Classification SchemeLikelihood Classification Scheme

11 Frequently Likely to occur frequently (Likely to occur frequently (oftenoften))

22 Probable Likely to occur several times during the life-time of the system (2-5 occurrences per year)

33 Occasional Occurs sometimes during the life-time of the system (1 occurrence per year)

44 Remote Unlikely to occur sometimes during the life-time of the system (1 occurrence per 5 years)

55 Improbable Very unlikely to occur (1 occurrence per 20 years)

66 Extremely Improbable

Extremely unlikely to occur (1 occurrence per 100 years)


STEP 3 & 4 – SEVERITY AND STEP 3 & 4 – SEVERITY AND LIKELIHOODLIKELIHOOD

EXAMPLEEXAMPLE


STEP 5 & 6 – RISK EVALUATION AND MITIGATION

Is this risk acceptable?

We have a risk

with a defined likelihood

and severity

Acceptablerisks

No

Yes

Notacceptable

risks

One of the causes

training of

Discussion of causes and failures

What are the potential causes

could be insufficientThis consequence

prevented if

How can we resolve it?

Discussion of Risk Mitigation

could be reduced or

Risk Mitigation Plan

Mitigation willremove risk

Mitigation willnot remove risk

Residualrisk

acceptable?

Riskmitigation

impracticable?

Mitigation impracticable

Openrisks

Discussion of acceptability


STEP 5 – RISK EVALUATION

• Determine what is / is not acceptableDetermine what is / is not acceptable – Acceptable level of SafetyAcceptable level of Safety

• Determine acceptability of identified Determine acceptability of identified risksrisks– Clearly unacceptableClearly unacceptable– Clearly acceptableClearly acceptable– May be / may be not acceptableMay be / may be not acceptableRisk Classification






< 5*10-4 A A A B D



< 1*10-6 A B C D D




likelihood

likelihood


STEP 5 – RISK EVALUATION

• Performed by a small groupPerformed by a small group– System users/operational expertsSystem users/operational experts: ATCOs and Flight : ATCOs and Flight

Crew (where necessary), to assess the consequences of Crew (where necessary), to assess the consequences of hazard(s) from an operational perspective;hazard(s) from an operational perspective;

– System technical expertsSystem technical experts, to explain the system , to explain the system purpose, interfaces and functions;purpose, interfaces and functions;

– Safety and human factors expertsSafety and human factors experts, to guide in the , to guide in the application of the FHA methodology itself and to bring application of the FHA methodology itself and to bring wider experience of the consequences of hazards.wider experience of the consequences of hazards.

• May need to be extended with specialists May need to be extended with specialists in areas relevant for the ALARP assessmentin areas relevant for the ALARP assessment


STEP 5 – RISK EVALUATIONSTEP 5 – RISK EVALUATION

EXAMPLEEXAMPLE


STEP 6 – RISK MITIGATIONSTEP 6 – RISK MITIGATION

• Identify potential causes for a risk to occurIdentify potential causes for a risk to occur– Some causes are identified during the hazard Some causes are identified during the hazard

identificationidentification– Ensure that we have identified all causesEnsure that we have identified all causes

• Identify potential mitigationIdentify potential mitigation– Remove the risk (remove the cause of the risk)Remove the risk (remove the cause of the risk)– Reduce the riskReduce the risk

• Reduce severity and/or probabilityReduce severity and/or probability

• Identify preferred mitigation approachIdentify preferred mitigation approach


Risk Classification






< 5*10-4 A A A B D



< 1*10-6 A B C D D




likelihood

likelihood




• Risk mitigation should be sought in any of Risk mitigation should be sought in any of the three components of a system:the three components of a system:– PeoplePeople– ProceduresProcedures– EquipmentEquipment

• The possible approaches to risk mitigation The possible approaches to risk mitigation include:include:– revision of the system (or airport) design;revision of the system (or airport) design;– modification of operational procedures;modification of operational procedures;– changes to staffing arrangements; andchanges to staffing arrangements; and– training of personnel to deal with the hazard.training of personnel to deal with the hazard.



• To identify causes a number of techniques To identify causes a number of techniques may be requiredmay be required– Brainstorming sessionsBrainstorming sessions– Fault tree analysis - Effect tree analysisFault tree analysis - Effect tree analysis– Common cause failure identification (Single point failure)Common cause failure identification (Single point failure)– Task, Fail-Safe & Error Tolerance AnalysisTask, Fail-Safe & Error Tolerance Analysis– Failure Mode and Criticality AnalysisFailure Mode and Criticality Analysis– Reliability, Availability and Maintainability AnalysisReliability, Availability and Maintainability Analysis

• Focus on components giving:Focus on components giving:– Highest likelihoodHighest likelihood– Highest degree of severityHighest degree of severity



• Performed by a small groupPerformed by a small group– System users/operational expertsSystem users/operational experts– System technical expertsSystem technical experts– Safety and human factors expertsSafety and human factors experts

• Different experts may be required to:Different experts may be required to:– Performed detailed studies of the causes of a riskPerformed detailed studies of the causes of a risk

• Study system design to determine component Study system design to determine component potentially causing, e.g. loss of air situation displaypotentially causing, e.g. loss of air situation display

• Study procedures to determine where e.g. Study procedures to determine where e.g. misunderstandings can arisemisunderstandings can arise

• Ways to remove those causesWays to remove those causes



SW

Hazard

S

F

S

S

F

F

Effect 1

Effect 2

Effect 3

Effect 4

P=Likelihood

E = Severity

PR

P=Likelihood

Failure Recovery

Fault Tree and Effect Tree Analysis



• Procedure Assurance LevelProcedure Assurance Level– Procedure development effort should be proportional Procedure development effort should be proportional

to the potential to the potential RiskRisk associated with the associated with the ProcedureProcedure. . To achieve this, objective PAL should be determined To achieve this, objective PAL should be determined and satisfied.and satisfied.

– PAL is setting some objectives to be met during the PAL is setting some objectives to be met during the different phases of the procedure life cycle – Table 1.different phases of the procedure life cycle – Table 1.

– PAL objectives are applicable to the entire Procedure, PAL objectives are applicable to the entire Procedure, not only to some part of it.not only to some part of it.



LevelLevel DefinitionDefinition Design and Design and validationvalidation

ImplementatioImplementationn

Transfer in Transfer in operationsoperations

OperationsOperations

33 •Other/own Other/own experience experience benchmarkinbenchmarkingg

•Specification Specification quality quality assuranceassurance

•Fast time Fast time simulationsimulation

•Qualitative Qualitative risk risk assessmentassessment

•Pre-Pre-implementatioimplementation trialsn trials

•Dedicated Dedicated trainingtraining

•Staff Staff acceptance acceptance argumentationargumentation

•Quality Quality assurance of assurance of implementationimplementation

•Competency Competency argument argument for the staff for the staff to perform to perform transfertransfer

•Contingency Contingency planplan

•Regular Regular proficiency proficiency checkschecks

44 •Other/own Other/own experience experience benchmarkinbenchmarkingg

•Specification Specification quality quality assuranceassurance

•Fast time Fast time simulationsimulation

•Qualitative Qualitative risk risk assessmentassessment

•Pre-Pre-implementatioimplementation trialsn trials

•Quality Quality assurance of assurance of implementationimplementation

•Contingency Contingency planplan

•Regular Regular proficiency proficiency checkschecks

Procedure Assurance Level



• Software Assurance LevelSoftware Assurance Level– Software development effort should be proportional to Software development effort should be proportional to

the potential the potential RiskRisk associated with the associated with the SoftwareSoftware. To . To achieve this, objective SWAL should be determined achieve this, objective SWAL should be determined and satisfied.and satisfied.

– SWAL is setting some objectives to be met during the SWAL is setting some objectives to be met during the different phases of the software life cycle.different phases of the software life cycle.

– SWAL objectives are applicable to the software SWAL objectives are applicable to the software component is question (only some part of of the total component is question (only some part of of the total software).software).



LevelLevel

RequirementRequirement11 22 33 44

37.337.3 Unit, integration and system testingUnit, integration and system testing

37.3.137.3.1 Unit and integration tests shall be Unit and integration tests shall be conducted on individual units and on conducted on individual units and on partially integrated units to partially integrated units to demonstrate that the software is demonstrate that the software is executable and that it produces the executable and that it produces the expected results for the specified test expected results for the specified test cases.cases.

MM MM MM MM

37.3.337.3.3 Integration tests shall as a minimum Integration tests shall as a minimum demonstrate the correctness of all demonstrate the correctness of all interfaces. interfaces.

J1J1 J2J2 MM MM

M Mandatory requirement to the development processJ1 Justification is to be provided if the clause or part of the clause is not followedJ2 Justification for the omission or non-compliance is to be provided

Extract from DEF-STAN-55



• Mitigation actions (safety requirements) Mitigation actions (safety requirements) should be carefully analysed:should be carefully analysed:– Will the mitigation remove the risk or reduce Will the mitigation remove the risk or reduce

the risk (what will be remaining risk be)the risk (what will be remaining risk be)– Will the implementation introduce any new Will the implementation introduce any new

hazards (repeat step 3, 4 and 5)hazards (repeat step 3, 4 and 5)

• Mitigation actions shall be documentedMitigation actions shall be documented– Risk Mitigation PlanRisk Mitigation Plan



EXAMPLEEXAMPLE


STEP 7 - SAFETY ASSESSMENT DOCUMENTATION

• The purpose:The purpose:– To provide a permanent record of the To provide a permanent record of the

final result of the safety assessmentfinal result of the safety assessment– To provide the arguments and evidence To provide the arguments and evidence

demonstrating that the risks associated demonstrating that the risks associated with the implementation of the proposed with the implementation of the proposed system or change:system or change:•have been eliminated, or have been eliminated, or •have been adequately controlled and have been adequately controlled and

reduced to a tolerable level.reduced to a tolerable level.


STEP 7 - SAFETY ASSESSMENT DOCUMENTATION

• Should contain a summary of:Should contain a summary of:– Methods usedMethods used– Safety criteria (the agreed safety levels)Safety criteria (the agreed safety levels)– Results of the hazard identification process (including Results of the hazard identification process (including

Hazard Logs)Hazard Logs)– Risk mitigation required (safety requirements)Risk mitigation required (safety requirements)– Follow-up actionsFollow-up actions– Evidence of compliance with safety requirementsEvidence of compliance with safety requirements

• References should be includedReferences should be included

– Evidence of validity of assumptionsEvidence of validity of assumptions


DIFFICULTIES – DIFFICULTIES – SAFETY ASSESSMENTSAFETY ASSESSMENT

• GeneralGeneral– Complex, resource-demanding activityComplex, resource-demanding activity

• Target Levels of Safety (Severity and Target Levels of Safety (Severity and Likelihood)Likelihood)– ComplexityComplexity– No guidelines or recommendation – in most cases No guidelines or recommendation – in most cases

not even statisticsnot even statistics– No guidelines to apportioning Safety Targets to No guidelines to apportioning Safety Targets to

lower levelslower levels– No guidelines to who does what (Regulator No guidelines to who does what (Regulator

Provider Provider Supplier) Supplier)


DIFFICULTIES – DIFFICULTIES – SAFETY ASSESSMENTSAFETY ASSESSMENT

• Risk MitigationRisk Mitigation– Very demanding concepts (software Very demanding concepts (software

assurance levels, procedure assurance assurance levels, procedure assurance levels)levels)

– Very demanding activities for risk Very demanding activities for risk mitigationmitigation

– Analyses required beyond reach for Analyses required beyond reach for many organisationmany organisation


RECOMMENDATIONSRECOMMENDATIONS

• Start with low level of ambitionStart with low level of ambition– Even simple Safety Assessment provides Even simple Safety Assessment provides

quite efficient risk mitigationquite efficient risk mitigation– Introduce more advanced features once Introduce more advanced features once

the simple version worksthe simple version works– Start with quantitative likelihood Start with quantitative likelihood

classification while data are collected to classification while data are collected to establish qualitative figuresestablish qualitative figures

• Make sure assumptions are well-Make sure assumptions are well-defined and traceddefined and traced


RECOMMENDATIONSRECOMMENDATIONS

• Don’t forget to design a follow-up Don’t forget to design a follow-up system for (ICAO 2.26.5)system for (ICAO 2.26.5)– Hazards (likelihood for different causes)Hazards (likelihood for different causes)– Assumptions, e.g.:Assumptions, e.g.:

•Capacity figuresCapacity figures

•Reliability figuresReliability figures

– Should be extracted from the reporting Should be extracted from the reporting systemsystem


SUPPORTING SLIDESSUPPORTING SLIDES


Target Level of SafetyTarget Level of Safety

METMET NAV/EnrNAV/Enr NAV/NAV/TermTerm

GroundGround TWRTWR APPAPP ACCACC

Safety factor for Accidents (1,55 10Safety factor for Accidents (1,55 10-8-8 per Flight hour) per Flight hour)

Mid-air collisionMid-air collision ÷÷

Controlled flight Controlled flight into terraininto terrain

÷÷

Accident on Accident on ground with ground with

fatalitiesfatalities

÷÷ ÷÷ ÷÷

…………

Safety Factors for Serious IncidentsSafety Factors for Serious Incidents

Separation minima Separation minima infringement (less infringement (less

than 50%)than 50%)

÷÷

Runway incursion Runway incursion with avoiding with avoiding

actionaction

÷÷ ÷÷ ÷÷

…………

Integra Consult A/S Safety Assessment Karachi, January 2006.

Documents

separation minima

situation minor reduction

safety assessment karachi

atc flight crew procedure

integra consult

csn system minor reduction

midair collision

major incident large