Burdun AUVSI-2007

INTELONICSINTELONICS

AUVSI's Unmanned Systems Europe 20078-9 May 2007, Hilton Cologne, Köln, Germany

UAV "Built-in" Safety Protection: A Knowledge-Centered Approach

Ivan Y. Burdun, Ph.D Chief ScientistIntelonics Ltd.

[email protected]

UAV "BuiltUAV "Built--in" Safety Protection: in" Safety Protection: A KnowledgeA Knowledge--Centered ApproachCentered Approach

Ivan Y. Burdun, Ph.D Chief ScientistIntelonics Ltd.

[email protected]

INTELONICS

Copyright © 2007 Intelonics Ltd.

Presentation Slides Presentation Slides Version #2 of 30.04.Version #2 of 30.04.20200707


© 2007 Intelonics Ltd. 2

Presentation PlanPresentation Plan

Problem: UAV flight safety performance prediction Problem: UAV flight safety performance prediction and protection in complex (multifactor) situationsand protection in complex (multifactor) situations

Solution approach: Solution approach: ‘‘Knowledge is PowerKnowledge is Power’’Methodology conceptual framework (introduction):Methodology conceptual framework (introduction):

-- micromicro-- and macroand macro--structural knowledge models of flightstructural knowledge models of flight-- flight situation scenarioflight situation scenario-- operational factoroperational factor-- operational hypothesisoperational hypothesis-- situational treesituational tree-- safety spectrumsafety spectrum-- flight safety [performance] windowflight safety [performance] window-- ‘‘last chance for recoverylast chance for recovery’’ point, selfpoint, self--preservation decision makingpreservation decision making-- safety chances distribution timesafety chances distribution time--historyhistory-- dynamic safety window treedynamic safety window tree

Case study: Case study: ‘‘Notional UAV LowNotional UAV Low--Altitude Flight in the Altitude Flight in the Presence of Urban InfraPresence of Urban Infra--Structure ObstaclesStructure Obstacles’’

ConclusionsConclusions



Natural Tree Analogy of Natural Tree Analogy of PilotPilot’’s Situational s Situational ‘‘Knowledge BaseKnowledge Base’’

Lack of theoretical and practical training (design and testing) – especially under complex (multifactor) conditions – may result in structural disparity of a human pilot’s (automaton’s) internal ‘situational tree’ of flight.

A

AA

AA

A

A

D

B

B

B

C

CD

A

D

D

E

C

E

C 02

1

F F

EB

B

BΩ

A sub-domain where branching is impossible

Physically unattainable flight situation scenariosF

Optimally dense branching

Systematic, yet economically developed and stored, knowledgeE

Insufficient, sparse branching

Fragmentary, incomplete knowledgeD

Excessive, chaotic branching

Non-systematic , occasionally developed knowledgeC

Dry or broken branchesForgotten or shadowed knowledgeB

Absent but possible branchingMissing knowledgeA

Second-order derivative branch

Two-factor non-standard flight situation scenario2

First-order derivative branch

One-factor non-standard flight situation scenario1

Tree’s trunkBasic (standard/non-standard) flight situation scenario0

Space available for tree growth

Space of possible complex flight situation scenariosΩ

Natural tree analogyCharacteristic zone of a pilot’s situational knowledge base

Legend:

A, B,C, D – main defect types of a human pilot’s situational knowledge. Defects to back up by means of AI in UAVs

Strengths to model by means of AI in UAVs



Legend: Characteristic levels of piloting expertise: k∈{1, 2, 3} – experience of a student pilot, k∈{8, 9, 10} –experience of a professional pilot, ace, or test pilot, k∈{4, …, 7} – interim (immature) states of experience.

9 8 7 610

1 2 3 4 5

Fractal Tree Model Of PilotFractal Tree Model Of Pilot’’s Situational s Situational Expertise Growth In LongExpertise Growth In Long--Term MemoryTerm Memory

The most valuable asset of an expert pilot (a perfect automaton) is the reliability and comprehensiveness of his/her (its) knowledge of the system behavior under complex (multifactor, non-standard) operational conditions. This expertise is of critical importance for reliable prediction, timely avoidance or/ and safe resolution of ‘chain reaction’ type emergencies in UAV flight.

desirable maturity levels of AI knowledge for flight safety protdesirable maturity levels of AI knowledge for flight safety protection in UAVsection in UAVs



MicroMicro-- and Macroand Macro-- Structural ModelsStructural ModelsOf Complex Flight Situation DomainOf Complex Flight Situation Domain

ProcessП

Event

E

Elementary situation(Ei, Пj, Ek)

Пj

Ei

Ek

Legend:Ei - flight event; Пj - flight process;

Cm – fuzzy constraint of flight;- reference state; - “bud” type

state; - target state (“leaf”); -source state (“root”); B-1 – parent

branch; B0 - main branch (“trunk”) –basic flight scenario; Bn – n-th order

derivative branch (non-standard scenario with n factors, n = 1, 2, …)

Micro- and macro- structures of flightare two interconnected components of the developed generalized knowledge model of a complex flight situation domain.

Macro-structure of flight

Situational tree-network of flight-

C1

B0C2

C4

C3

B2

B-1

B1

C1

B0C2

C4

C3

B2

B-1

B1

Micro-structure of flight

Flight situation scenario

11

E7

...E5

E6

E8

П7

П8

П15

...

...

E1

П3П4

П1

П6

П9

...

П2

П14

...

П12...П13

...

E2

E4

E4

П5П10

11

E7

...E5

E6

E8

П7

П8

П15

...П



Basic Flight Situation Scenario Basic Flight Situation Scenario (Examples)(Examples)

Basic (Baseline) Scenario Si is a plan of some ‘central’ or reference flight situation – be it standard or non-standard one. It represents the situational tree’s trunk. Variations of the basic scenario – derivative cases – constitute the situational tree’s crown. The vehicle’s flight safety knowledge base is in fact a collection (a ‘forest’) of the situational trees, which are constructed for various basic scenarios and exemplify a complex (multi-factor) flight situation domain.

Si Content Description

S1 Normal takeoff, maintaining commanded flight path and bank angles during initial climb

S2 Normal takeoff under crosswind and given runway’s surface conditions, maintaining commanded flight path and bank angles during initial climb

S3 Continued takeoff (left-hand engine out at given VEF), maintaining commanded flight path and bank angles during initial climb

S4 Normal takeoff under wind shear conditions, maintaining commanded flight path and bank angles during initial climb

S5 Continued takeoff (left-hand engine out at VEF), under crosswind conditions, maintaining commanded flight path and bank angles during initial climb

S6 Low-altitude level flight Scenario #6 will be used in the notional case study



A flight situation scenario is depicted as a directed graph. It defines logic and content of flight. Scenario graph is clear and concise formal description of a flight situation. Basic scenario examples S1, …, S6 are structurally close. They can be modified by adding new events/processes or by modifying existing ones.

E44: engine out speed

44

event

F1: left-hand engine out process

Legend:

basic scenario Si

S4

F1: left-hand engine out

… S3S5

W1: cross-wind-10 m/s

S1

190

E5: pitch 8о5

T3: maintain commanded flight path angle ΘG1(1st phase of climb)

E3: VR achieved 3 P2: elevator –

up for rotation …

E6: altitude 10.7 m

6 P3: wheels - up…

E7: altitude 120 m

P4: flaps - up…

W3: cross-wind 10 m/s

S2

E1: situation start

1

…

T1: maintainground-roll path along

runway’s centerlineE55: in airborne

55

W2: wind-shear

∨

T2: maintain commanded bank (γG) and sideslip(βG) angles

T2: maintain commanded bank

(γG) and sideslip (βG) angles

12

7

E12: flaps retracted

E190: endof situation

E88: altitude 200 m

88

E44: engineout speed44

T4: maintain commanded flight path angle θG2 (2nd phase of climb, level flight, or

descent)

P5: maintain commanded

IAS

T5: maintain commanded bank (γG) and sideslip

(βG) anglesP1: set engine leversto commandedpower rating S6

Joint Graph of Basic Scenarios (Example)Joint Graph of Basic Scenarios (Example)



Design Field of Operational HypothesesDesign Field of Operational Hypotheses

independentdependent

- link between factors in ГWyg Ф5

Cross wind velocity

- operational factor

Г13 - operational hypothesis

Legend:Normally, a single operational factor

is not critically dangerous. More important and much more difficult to learn the effects of multi-factor combinations on flight safety. These multi-factor combinations are called operational hypotheses.

Left-hand engine failure at VEF Commanded flight

path angle (initial climb)

Commanded flight path angle (2nd phase of climb, level flight or descent)

Intensity of wind shear

Cross wind velocity

Flaps-up start altitude

‘Wheels - runway surface’adhesion factor

Rotationairspeed

LongitudinalC.G.

‘Engine out’indicated airspeed

Engines power rating at takeoff

Elevator deflectionfor rotation

Commanded bank angle

Г1

Г2

Г10

Г3

Г5Г6

Г7

Г8

Г9

Г11Г12

Г13

Г4

θG1 Ф7

θG2 Ф8

kW Ф9

Wyg

Ф5

HFLФ6

μФ4

VRФ2

ζLHEФ13

CGxФ1

VEF

Ф12kP

Ф10

Δδe

Ф3

γG

Ф11

Г14



Composition of Situation Scenario (S) and Composition of Situation Scenario (S) and Operational Hypothesis (Operational Hypothesis (ГГ) is ) is

A Situational Tree (SA Situational Tree (S⋅⋅ГГ))

Situational tree’s branches (flight paths) stand for ‘what-if’ derivative (non-standard) situation scenarios. All branches are color coded using integral safety spectra colors.



Safety PaletteSafety Palette. . Fuzzy ConstraintFuzzy Constraint

Operational constraints under multi-factor flight conditions are not known precisely. They are inherently ‘fuzzy’. The notion of fuzzy constraint (by L.A. Zadeh) and the notion of safety palette are employed for approximate measurement of the compatibility of current (i.e. measured at time instants t) system states with operational constraints for key system variables (monitored flight parameters).

Color is natural and, perhaps, the most effective and economic medium for communicating safety-related information to/ from an operator (a pilot or automaton).

Legend: c, d – characteristic points of the carrier of fuzzy set-constraint C, μC(x) – L.A. Zadeh membership function

Fuzzy Constraint (Example)

‘red’‘green’ ‘black’‘yellow’

μC(VFL.D.)1

C: ‘permitted flaps-down flying IAS’

dс0

410390 VFL.D.[km/h]

…470

…

…

green (‘norm’), ξG

yellow/ amber (‘attention’), ξY

black (‘catastrophe’), ξB

grey/white (‘uncertainty’), ξW

red (‘danger’), ξRSafety Palette



Partial and Integral Safety SpectraPartial and Integral Safety Spectra

(∀t) (t∈[t*;t*]) (∃ξ(xk(t)) (ξ(xk(t))∈{ξW, ξG, ξY, ξR, ξB, …} ∧ (ξW < ξG < ξY < ξR < ξB))(ξ(t) = max ξ(xk(t)), k = 1, …, p) ⇒ (ξ(t)∈Σ ∧ Σ = ξ(t*) || ξ(t*+Δ) || ξ(t*+2Δ) || … || ξ(t*))

Integral Safety Spectrum Calculation Algorithm:

- yellow (‘attention’), ξY

- black (‘catastrophe’), ξB

- gray/white (‘uncertainty’), ξW

- red (‘danger’), ξR

- green (‘norm’), ξG

Legend: Σk – partial safety spectrum for variable xk, k = 1, …, p; p – total number of monitored constraints/ variables, p = 20. Σ – integral safety spectrum; t – flight time; ξi – color from safety palette, i ∈ {B (black), R(red), Y (yellow), G (green),…}; < –‘colder than’ operation for comparing two safety colors; max – operation of selecting the ‘hottest’ color at time instant t; || - operation of safety colors concatenation in Σ; [t*; t*] –examined flight time interval; Δ –spectrum construction time step.

For each flight situation from the situational tree, safety levels are measured for all monitored variables xk at all recorded time instants. As a result, for each situation from the tree, a family of Partial Safety Spectra Σk, k = 1, …, p, and an Integral Safety Spectrum Σ are obtained. The integral safety spectrum is a color-coded time-history of violation and restoration of monitored fuzzy constrains during a flight situation.

Mon

itore

d va

riabl

es/ c

onst

rain

ts

Σ1Σ2

...

Σ20

...

Σk

Par

tial f

light

saf

ety

spec

tra

Σ

IAS (δF = 0, airborne)IAS (δF > 0, airborne)

SideslipLoad_factor

East_rate (groundroll)East (groundroll)

North (groundroll)Bank (airborne)

Bank (groundroll)Pitch (airborne)

Pitch (groundroll)Vert_rate (airborne)

AoA (δF = 0)AoA (δF > 0)

Wheels (airborne)Wheels (groundroll)

Elevator (airborne)Elevator (groundroll)

AileronRudder

Integral spectrum

time, s



Safety Classification CategoriesSafety Classification Categories

Flight Safety Category Color Code Name

Situation Classification Criterion

I Safe

The system state resides mainly inside the 'green' zone. As a maximum, the system state may stay, for a short time, in close proximity to operational constraints, i.e. inside the ‘yellow’ zone, but must leave it by the end of the flight situation

II-a Conditionally

Safe – a

As a maximum, the system state may stay for a medium time in close proximity to operational constraints, i.e. inside the ‘yellow’ zone

II-b Conditionally

Safe – b As a maximum, the system state may stay for a long time in close proximity to operational constraints, i.e. inside the ‘yellow’ zone

III Potentially

Unsafe As a maximum, the system state may violate operational constraints, i.e. enter the ‘red’ zone, for a short or medium time, but must leave it by the end of the situation

IV Dangerous

(Prohibited) As a maximum, the system state may stay beyond operational constraints, i.e. inside the ‘red’ zone, for a long time or till the end of the flight situation

V Catastrophic

(‘Chain Reaction’) There is at least one (for a short time) occurrence of a ‘black’ violation of any operational constraint

One more level of flight safety knowledge generalization is introduced. The goal is to measure the vehicle’s safety performance in a flight situation as a whole. With this aim, a generalized ‘safety ruler’consisting of five Safety Classification Categories I, …, V is employed. Why five? – It is because experts cannot reliably recognize and use more than 5-10 gradations of a complex, difficult-to-formalize system-level property (e.g.: Cooper-Harper scale). New ‘light green’ (‘salad green’) and ‘orange’ colors have been added to the existing Safety Palette in order to denote interim Categories II-a and III, respectively.


2007 Intelonics Ltd. 13

Safety Window for Situational Tree SSafety Window for Situational Tree S11⋅⋅ГГ1111: : Takeoff.Takeoff. Errors of SelectingErrors of Selecting CommandedCommanded

FlightFlight PathPath ааnd Bank Angles in Climbnd Bank Angles in Climb

100130Σnj, Σχj | S1⋅Γ11

00V4355IV11III

2229II-b68II-a

2837Iχj, %njξjCategory

3

Flight Safety Window (FSW)Safety Chances

DistributionPie Chart

Let us map safety classification levels (categories) obtained for all situations for tree S1⋅Г11 onto a two-factor plane. This gives a Flight Safety [Performance] Window (FSW). In FSW above, cell CC is located at ‘column AA - row BB’ crossing. This cell depicts safety status of one flight path-branch from the tree. It is a non-standard situation with values of 3030oo and1414oo of factors Φ7 and Φ11 in S1. This cell is painted using the situation’s Flight Safety Category color (‘orange’). The FSW has a dangerous ‘corner’ (upper-left). Rapid transition (3) from safe (‘salad green’) to dangerous (‘red’) zone is possible (Cat. II-a → IV), bypassing interim zones (II-b, III). Control at such ‘corners’ therefore requires enhanced attention.

AA

BBCC



Flight Safety Flight Safety ‘‘TopologyTopology’’ MapMap

1 ‘Abyss’ (catastrophe)2 ‘Hill’ (danger) 3 ‘Slope’ (reversible state

transitions)

4 ‘Valley’ (standard safety, norm)5 ‘Lake’ (maximum safety, optimum)6 ‘Precipice’ (abrupt, irreversible

state transitions, ‘chain reaction’)

In general, the following characteristic objects can

be defined within Flight Safety ‘Topology’ Map:

V

IV

III

II-b

II-a

I

Flight Safety

Categories

Ope

ratio

nalf

acto

rФ1

1

Operational factor Ф2

1

2

5

4

3

6 3

3

2

4

2

Transitions 6 must be known and prevented!

Transitions 3 must be known and controlled!

63

2



SS44⋅⋅ГГ1212: Normal Takeoff. : Normal Takeoff. ‘‘StrongStrong’’ WindWindShear. Errors of SelectingShear. Errors of Selecting CommandedCommanded

Flight Path and Bank Angles in ClimbFlight Path and Bank Angles in Climb

100130Σnj, Σχj | S4⋅Γ12

1013V4661IV11III

1519II-b1519II-a1317Iχj, %njξjCategory

6 6

This safety ‘topology’ corresponds to the tree SS44⋅⋅ГГ12 12 obtained under ‘strong’ wind shear conditions. At small flight path angles θG1 and any bank angles γG it reveals a stable catastrophic ‘abyss’ (a black strip in the bottom) and ‘precipice’ type transitions (6). It means that attempts of climbing at small commanded flight path angles (1o … 2o) will inevitably lead the vehicle to a fatal outcome.

Flight Safety WindowSafety Chances

Distribution Pie Chart



RealReal--Time Safety Knowledge Map Time Safety Knowledge Map (Dynamic Safety Window)(Dynamic Safety Window)

The concept of dynamic safety window is based on use of a ‘forest’ of situational trees. Provided that key operational factors are measurable on board the vehicle in real time, a dynamic safety window can be used as a medium for coherent monitoring of tactical goals and constraints of flight under uncertainty.

Safety chances distribution pie charts are expedient to use in UAV safety indicators to monitor current state and predict the system safety chances dynamics under anticipated operational conditions during flight.

Note that in this particular example, the share of ‘red’ and ‘black’ scenario options increases at the expense of reducing the share of safer outcomes.

Presented is a time-history of safety windows and safety chances distribution pie charts that correspond to a hypothetical complex flight situation domain - a union of three compositions S4⋅(Г11+Г12+Г13): ““Normal takeoff. Possible variations of wind-shear intensity, errors/ variations in maintaining commanded flight path and bank angles during initial climb”.t = t1: ‘strong’ wind-shear warning

t = t2: ‘very strong’ wind-shear warning

t = t0: ‘benign weather’ forecast

t0

t1

t2

time



Situational Trees for ShortSituational Trees for Short--Term Term Prediction of Flight SafetyPrediction of Flight Safety

Legend: to – current flight time, t* – prediction start time, t* – prediction stop time, τ = (t* – to) – decision-making delay, Δt = (t* – t*) – prediction time range (depth of tree-based multi-factor domain exploration)

multi-factor situation domain exploration cone (‘future-looking knowledge radar’)

Situational tree construction and tree-based safety prediction (a ‘what-if’ analysis) methodology accounts for both physics and logic of multi-factor flight situation domain.

t

Δt

τ

safety prediction sub-tree

t*t0

t*



SS66⋅⋅ГГ1414: Low: Low--Altitude Level Flight. Errors Altitude Level Flight. Errors Of SelectingOf Selecting Commanded Flight PathCommanded Flight Path

And Bank AnglesAnd Bank Angles

100247Σnj, Σχj | S6⋅Γ14

1537V3382IV24III

1435II-b512II-a

3377Iχj, %njξjCategory

This Safety Window has two catastrophically dangerous ‘corners’ (6) corresponding to (θG1, γG) ≅ (-10o…-12o, |37.5o…45o|). Sharp transition (3) of states from safe (‘salad green’) to dangerous (‘red’) zone is also possible in the left upper corner (Cat. II-a→IV), bypassing interim zones (Cat. II-b, III).

66

3

Flight Safety WindowSafety Chances

Distribution Pie Chart



-45o

0

30o

45o

-30o

-15o 15o

Note: not to scale

S6⋅Γ14 | t13

24201612840-4-8-12

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

tower building type obstacle(side view)

S6⋅Γ14 | t124201612840-4-8-12

IAS ∼ 320÷360 km/h

‘yellow’ and ‘red’ zones of obstacle’s fuzzy constraint

-45 o

0

30o

45 o

-30 o

-15 o

15 o

-45o

0

30o 45

o

-30o

-15o

15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899

161617171818

1919

15151414

SS66⋅⋅ГГ1414: Low: Low--Altitude Level FlightAltitude Level FlightIn the Presence of Urban ObstaclesIn the Presence of Urban Obstacles

S6⋅Γ14 | t19

S6⋅Γ14 | t13

S6⋅Γ14 | t1S0 ∪ S↓ - terrorist-/ fool-type control, S0 ∪ S↑ -

AI-based self-preservation control.

Legend: S0, S↓, S↑ - scenario segments, S0 -obstacle approach, S↓ - imminent collision, S↑ -collision avoidance, S6⋅Γ14 | ti - tree location at ti

S↑

t0 t1 t2 t3 t4 t5 t6S↓t7 t8 t9 t10 t11 t12 t13

t14 t15 t16 t17 t18 t19

S0

Scenario time lines

tower building type obstacle (top view)

H ∼ 200 ÷ 400 m



-45o

0

30o

45o

-30o

-15o 15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899

SS00: Obstacle Approach (: Obstacle Approach (tt00))Safety Chances Distribution

0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %

The vehicle is approaching the obstacle – a tower building at θG2 = 0 (commanded flight path angle) and γG= -15o

(commanded bank angle). No threat is observed in the safety window at t0.

-15

0

Safety Window

current tactical goal-cell24201612840-4-8-12

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

i

Note: not to scale



-45o

0

30o

45o

-30o

-15o 15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899

Safety Chances Distribution

0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %

SS00: Obstacle Approach (: Obstacle Approach (tt11))

Safety Window A fuzzified safety window state at t1 is shown. The white rectangular in the window is a current tactical goal-cell (θG2/ γG) = (0/ -15o). Still no threat is observed in the safety window.

current tactical goal-cell

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12

i

Note: not to scale



-45o

0

30o

45o

-30o

-15o15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %


Safety Window Some branches of the prediction sub-tree hit the ‘yellow’ zone of the obstacle’s fuzzy constraint.

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12

i



-45o

0

30o

45o

-30o

-15o

15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %


Safety Window The ‘yellow’ zone in the safety window is expanding at the expense of the ‘green’ zone, which is shrinking respectively …

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12

i



-45o

0

30o

45o-30o

-15o

15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899


Safety WindowSafety Chances Distribution

0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %

Now the ‘yellow’ zone occupies almost all the left-hand half of the safety window.

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12

i



-45o

030o

45o

-30o

-15o

15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899



0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %

Some branches of the prediction sub-tree hit the ‘red’zone of the obstacle’s fuzzy constraint. The ‘yellow’ and now ‘red’ zones in the safety window are expanding.

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12

i



-45o

030o

45o

-30o

-15o

15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899



0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %

The ‘red’ zone is spreading through the left-hand part of the safety window.

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12

i



-45o

0 30o

45o

-30o

-15o

15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899



0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %

A ‘black’ (‘imminent collision’) zone appears in the safety window. NB: It overlaps with the current tactical goal-cell.

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12


i

0

-15



-45o

0 30o

45o

-30o

-15o

15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899

SS↓↓: Imminent Collision (: Imminent Collision (tt88))


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %

The ‘black’ zone is expanding – now at the expense of shrinking both the ‘green’ and ‘yellow’zones.

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12

The fact that the goal-cell still remains in the ‘black’ zone says that the aircraft is on a collision course.

i



-45o

0

30o

45o

-30o

-15o

15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %


Safety Window

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12

No safe (‘green’) flightpath-branch alternatives are available. The share of ‘black’ scenarios increases. The share of ‘red’ scenarios remains the same.

i



-45o

0

30o

45o

-30o

-15o

15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %


Safety Window

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12

Almost no ‘yellow’(conditionally safe) branch options are left in the safety window to use for recovery. A catastrophic trend in the situation continues to build-up steadily.

i



-45o

0

30o

45o

-30o

-15o

15o

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %


Safety Window

The ‘black’ zone covers more than 60% of the safety window area, and the rest represents ‘red’(dangerous) scenarios, i.e. the flight paths in a close vicinity of the obstacle…

i



-45o

0

30o

45o

-30o

-15o

15o

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %


The ‘black’ zone occupies more than 80% of the safety window area. A catastrophic trend accelerates, and the chances of collision are very high.

i



-45o

0

30o

45o

-30o

-15o

15o

1 2 3 4 5 6 7 8 9 10 11 12 1313-1 0 i

24201612840-4-8-12

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %


The ‘black’ zone now occupies the entire safety window’s area (100%). This means that the collision is unavoidable…

Safety Window

i



Kazimir MalevichKazimir Malevich’’s s ‘‘The Black SquareThe Black Square’’Painting and Painting and ‘‘9/119/11’’

The safety window state just before collision point (S↓ | t13), perhaps, helps better understand the meaning of Kazimir Malevich’s painting ‘The Black Square’ - The fatal end is imminent. And there is no chance left to remedy the situation …

K. Malevich. ‘The Black Square’(1913)



-45o

0 30o

45o

-30o

-15o

15o

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 88 99 1010 1111 1212 1313

χj, %

Safety Window

‘‘Last Chance for RecoveryLast Chance for Recovery’’ Point (Point (tt↑↑ ≡≡ tt77))

However, the ‘last chance for recovery’point (t↑) does exist, and it must be assigned to t7. This is marked by the system state when the new ‘black’ zone (induced by the obstacle) in the safety window first time overlaps with the current tactical goal-cell of the operator’sflight control.


Note: not to scale

i0

-15



7

Safety Window State at Safety Window State at ‘‘Last Chance Last Chance For RecoveryFor Recovery’’ PointPoint ((tt↑↑ ≡≡ tt77)):: SS00 →→ SS↑↑

Legend: 1 – zone of ΔΦ-secured non-catastrophic scenarios; 2, 3 – zones (‘islands’) of remaining safe/conditionally safe scenarios; 5, 8 – ‘C.G.’ locations for left- and right-hand ‘islands’ of remaining safe/conditionally safe scenarios; 4, 7 – old (catastrophe-prone) and new (safety restoring) cells of the commanded flight path and bank angles, 6 – required shift of the tactical flight goal-cell in the safety window.

1

5

3

2

8

Commanded bank angle, γG

Com

man

ded

fligh

t pat

h an

gle,

θG

2

64

ΔΦ ≡(ΔθG2, ΔγG) =

(2o, 7.5o)



SS0 0 →→ SS↑↑:: SelfSelf--Preservation Automatic Preservation Automatic Decision Making at Decision Making at tt77 ≡≡ tt↑↑

1010

12121313

1111

11

22

33

44

55

66

00

--11

778899

-45o

0 30o

45o

-30o

-15o

15o


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 1414 1515 1616 1717 1818 1919

χj, %

Based on results of safety ‘topology’ analysis at t7, a self-preservation decision must be made - the current tactical control goal is shifted from the old (‘black’, collision-prone) cell, (θG2/γG) = (0/-15o), to a new (‘green’, safe) cell, (θG2/γG) = (6o/+30o), located in the right-hand ‘safety island’ of the window.

30

6

0

-15

old tactical goal-cell new tactical goal-cell

Safety Window

i



1010

12121313

1111

11

22

33

44

55

66

00

--11

778899

Safety Window

SS↑↑: Collision Avoidance (: Collision Avoidance (tt1414))

161617171818

1919

15151414-45o

0

30o

45o-30o

-15o

15o


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 1414 1515 1616 1717 1818 1919

χj, %

The ‘black’ zone in the safety window is still expanding (due to vehicle dynamics lag). However, the ‘red’ zone begins to shrink, and the ‘yellow’ zone size remains unchanged. The commanded (tactical goal) cell is now located outside the danger and catastrophe-prone zones.

i



Safety Window


1010

12121313

1111

11

22

33

44

55

66

00

--11

778899

161617171818

1919

15151414

-45o

0

30o

45o-30o

-15o

15o


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 1414 1515 1616 1717 1818 1919

χj, %

A positive (recovery) safety trend begins to develop. The ‘yellow’ zone is expanding, and the ‘red’ zone is shrinking in the safety window.

i



Safety Window


1010

12121313

1111

11

22

33

44

55

66

00

--11

778899

161617171818

1919

15151414

-45o

0

30o

45o

-30o

-15o15o


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 1414 1515 1616 1717 1818 1919

χj, %

Positive safety trend remains steady. The ‘yellow’ zone is expanding, and the ‘red’ zone is shrinking. The ‘black’ zone begins to shrink as well …

i



Safety Window


1010

12121313

1111

11

22

33

44

55

66

00

--11

778899

161617171818

1919

15151414

-45o

0

30o

45o

-30o

-15o 15o


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 1414 1515 1616 1717 1818 1919

χj, %

‘Black’ zone induced by the obstacle is about to disappear. Positive safety trend is now irreversible. The ‘yellow’ zone continues to expand.

i



Safety Window


1010

12121313

1111

11

22

33

44

55

66

00

--11

778899

161617171818

1919

15151414

-45o

0

30o

45o

-30o

-15o

15o


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 1414 1515 1616 1717 1818 1919

χj, %

The safety window is about to resume its initial state (vehicle’s performance only) as the obstacle (a tower type building) has been safely avoided.

i



Safety Window


1010

12121313

1111

11

22

33

44

55

66

00

--11

778899

161617171818

1919

15151414

-45 o

0

30 o

45o

-30 o

-15 o

15 o


0

20

40

60

80

100

--11 00 11 22 33 44 55 66 77 1414 1515 1616 1717 1818 1919

χj, %

The knowledge-centered process of flight safety restoration is complete by now.

i



‘‘BirdBird’’s Eyes Eye’’ View View Of Dynamic Safety Window Tree: Of Dynamic Safety Window Tree:

Catastrophic and Recovery ScenariosCatastrophic and Recovery Scenarios

Legend:

Scenario segments:S0 – obstacle approachS↓ – imminent collisionS↑ – AI based collision avoidance

Scenario time lines: {t0, t1, ..., t7} – S0{t8, ..., t13} – S↓{t14, ..., t19} – S↑

Key time instants: t7 – ‘last chance for recovery’t13 – ‘just before impact’t19 – ‘safety restoration complete’

S0

t0

t1

t2

t3

t4

t5

t6

t7

S↓

t8

t9

t10

t11

t12

t13

S↑

t14

t15

t16

t17

t18

t19This is a safety window time-history tree.

It provides a systematic – ‘bird’s eye’ view level – picture of two alternative scenarios of aircraft flight control in the presence of an urban type obstacle. Such obstacles can be a part of a multi-factor flight situation domain-‘neighborhood’ of the current situation.

S0

S↓ S↑

‘last chance for recovery’ point (‘fate switch’)



Safety Chances Distribution Safety Chances Distribution TimeTime--History for Two Control TacticsHistory for Two Control Tactics

Legend: A, B, …, L - characteristic states of the aircraft safety dynamics; χj – flight safety chances at ξj level, j∈{I, II-a, II-b, III, IV, V}; ti – time instants, i∈{-1, 0, 1, …, 13} ∨ i∈{-1, 0, 1, …, 7, 14, 15, .., 19}.

VIVIIIII-bII-aI – safety classification categories and colors

(2) AI based self-preservation control(1) terrorist-/ fool-type control

Characteristic states {A, B, C, …, L} of the vehicle’s safety dynamics and their recognition criteria are expedient to use in the automatic or manual recovery decision-making process in emergency situations under uncertainty. In accordance with the self-preservation imperative for a civil aircraft, flight control authority in a life-threatening situation must be dynamically assigned/transferred to a most competent agent.

0

20

40

60

80

100

-1 0 1 2 3 4 5 6 7 14 15 16 17 18 19

χj, %

A B C D H J LI K

0

20

40

60

80

100

-1 0 1 2 3 4 5 6 7 8 9 10 11 12 13

χj, %

A B C D E F G

i i

‘last chance for recovery’ point

S0 S↓ S↑S0



1.1. Generalized knowledgeGeneralized knowledge--centered methodology has been developed for UAV flight centered methodology has been developed for UAV flight safety prediction and protection in multifactor situations near safety prediction and protection in multifactor situations near operational operational constraints. constraints.

2.2. MethodMethod’’s advantages are: use of integrated conceptual framework, simples advantages are: use of integrated conceptual framework, simple realreal--time calculations, open memorytime calculations, open memory--based knowledge system, situationbased knowledge system, situation--independent independent decisiondecision--making algorithm, exploration of situation making algorithm, exploration of situation ‘‘whatwhat--if neighborhoodif neighborhood’’ tree for tree for shortshort--term flight path probing, use of term flight path probing, use of ‘‘birdbird’’s eyes eye’’ view view ‘‘topology mapstopology maps’’ for flight for flight safety status monitoring and automatic recovery in emergencies. safety status monitoring and automatic recovery in emergencies.

3.3. However, prerequisites for successful implementation of developHowever, prerequisites for successful implementation of developed methodology ed methodology are: are:

availability of vehicleavailability of vehicle’’s validated s validated ‘‘parametric definitionparametric definition’’ database, and database, and onboard integrated sensor suit capable of detecting potentially onboard integrated sensor suit capable of detecting potentially dangerous dangerous physical/ virtual obstacles inside vehiclephysical/ virtual obstacles inside vehicle’’s s ‘‘safety ellipsoid/conesafety ellipsoid/cone’’. .

4.4. Potential application areas are as follows: Potential application areas are as follows: design of affordable, yet expert pilot level AI safety protectiodesign of affordable, yet expert pilot level AI safety protection systems based n systems based on selfon self--preservation imperative for unmanned/ manned air vehicles to prepreservation imperative for unmanned/ manned air vehicles to prevent vent key accident/ incident scenarios such as LOC, CFIT, key accident/ incident scenarios such as LOC, CFIT, ‘‘pilot errorpilot error’’, hardware , hardware failure, midfailure, mid--air collision, and air collision, and ‘‘9/119/11’’design of adaptive mission control and autonomous collision avoidesign of adaptive mission control and autonomous collision avoidance dance systems (integrated with C.Reynolds swarming model, ethology prisystems (integrated with C.Reynolds swarming model, ethology principles, nciples, etc.) for heterogeneous multivehicle clusters and freeetc.) for heterogeneous multivehicle clusters and free--flight operations. flight operations.

ConclusionsConclusions



Bernd ChudobaBernd Chudoba, Ph.D., Ph.D.The University of Texas at ArlingtonThe University of Texas at ArlingtonUSAUSA

JeanJean--Pierre CacheletPierre Cachelet, Ph.D., Ph.D.AIRBUSAIRBUSFranceFrance

Till BunseTill Bunse, Ph.D, Ph.DGermanwings GmbHGermanwings GmbHGermanyGermany

AcknowledgementsAcknowledgements

-- for sharing innovative ideas and for for sharing innovative ideas and for cooperation in the area of cooperation in the area of ‘‘virtual flight virtual flight testingtesting’’ of a/c for detecting anomalous of a/c for detecting anomalous flight situations in early design. flight situations in early design.

-- for multifor multi--aspect support, beginning aspect support, beginning from 90from 90’’s during Ph.D studies at s during Ph.D studies at Cranfield University, and up to these Cranfield University, and up to these days. days.

-- for offering excellent air travel for offering excellent air travel package (Moscowpackage (Moscow--KKöölnln--Moscow) to Moscow) to attend the Conference.attend the Conference.

The author wishes to thank the following individuals and companies:



Thank YouThank You

Questions, please …Questions, please Questions, please ……



1.1. Burdun, I.Y., Burdun, I.Y., ““Prediction of Aircraft Safety Performance in Complex Flight SituPrediction of Aircraft Safety Performance in Complex Flight Situationsations”” (Paper 2003(Paper 2003--0101--2988), 2988), Proc. of 2003 Proc. of 2003 Advances in Aviation Safety Conference, September 8 Advances in Aviation Safety Conference, September 8 –– 12, 12, 20032003, Montreal, Montreal, , Canada,Canada, SAE, 2003, 18 pp. SAE, 2003, 18 pp.

2.2. Burdun, I.Y., Burdun, I.Y., ““Studying Physics and Logics of Complex Flight Situation Domains Studying Physics and Logics of Complex Flight Situation Domains by Means of VATES Modeling and by Means of VATES Modeling and Simulation ToolSimulation Tool””, , Proc. of 2Proc. of 2ndnd Science and Technology Conference Science and Technology Conference ““Flight Simulation Technologies and Pilot Training: New Flight Simulation Technologies and Pilot Training: New Approaches and GoalsApproaches and Goals””, TsAGI, 24 , TsAGI, 24 –– 25 April 2003, Zhoukovsky, Moscow Region,25 April 2003, Zhoukovsky, Moscow Region, TsAGI, 2003, 11 pp. (in Russian). TsAGI, 2003, 11 pp. (in Russian).

3.3. Burdun, I.Y., Parfentyev, O.M., Burdun, I.Y., Parfentyev, O.M., ““Fuzzy Situational TreeFuzzy Situational Tree--Networks for Intelligent Flight SupportNetworks for Intelligent Flight Support””, , Int. Journal of Engineering Int. Journal of Engineering Applications of Artificial IntelligenceApplications of Artificial Intelligence ((EAAI)EAAI), 12 (1999), pp. 523 , 12 (1999), pp. 523 –– 541. 541.

4.4. Schrage, D.P.,Schrage, D.P., Calise, A.J.,Calise, A.J., Burdun, I.Y.,Burdun, I.Y., Pritchett, A.,Pritchett, A., and Rysdyk, R.T.,and Rysdyk, R.T., ““An Integrated KnowledgeAn Integrated Knowledge--Based Approach to Based Approach to Improving Aircraft SafetyImproving Aircraft Safety””, , White PaperWhite Paper, School of Aerospace Engineering, Georgia Tech, USA, Oct. 1998,, School of Aerospace Engineering, Georgia Tech, USA, Oct. 1998, 20 pp.20 pp.

5.5. Burdun, I.Y., Burdun, I.Y., ““The Intelligent Situational Awareness And Forecasting EnvironmenThe Intelligent Situational Awareness And Forecasting Environment (The S.A.F.E. Concept): A Case Studyt (The S.A.F.E. Concept): A Case Study””(Paper 981223), (Paper 981223), Proc. of 1998 Advances in Flight Safety Conference and ExhibitioProc. of 1998 Advances in Flight Safety Conference and Exhibition, April 6n, April 6--8, 1998, Daytona Beach, FL, 8, 1998, Daytona Beach, FL, USA USA (P(P--321), SAE, 1998, pp.131 321), SAE, 1998, pp.131 –– 144.144.

6.6. Burdun, I.Y., and Parfentyev, O.M., Burdun, I.Y., and Parfentyev, O.M., ““Analysis of Aerobatic Flight Safety Using Autonomous Modeling anAnalysis of Aerobatic Flight Safety Using Autonomous Modeling and Simulationd Simulation””(Paper 2000(Paper 2000--0101--2100), 2100), Proc. of the 2000 Advances in Aviation Safety Conference, April Proc. of the 2000 Advances in Aviation Safety Conference, April 1111--13, 2000, Daytona Beach, FL, 13, 2000, Daytona Beach, FL, USA USA (P(P--355)355), , SAE, 2000, pp. 75 SAE, 2000, pp. 75 –– 92.92.

7.7. Burdun, I.Y., Parfentyev, O.M., Burdun, I.Y., Parfentyev, O.M., ““AI Knowledge Model for SelfAI Knowledge Model for Self--Organizing Conflict Prevention/Resolution in Close FreeOrganizing Conflict Prevention/Resolution in Close Free--Flight Flight Air SpaceAir Space””, , Proc. Of IEEE Aerospace Applications Conference, Snowmass, ColorProc. Of IEEE Aerospace Applications Conference, Snowmass, Colorado, March 6ado, March 6--13, vol. 2, 199913, vol. 2, 1999, , USAUSA, , IEEE, 1999, pp. 409 IEEE, 1999, pp. 409 –– 428.428.

8.8. Burdun, I.Y., and Burdun, E.I., Burdun, I.Y., and Burdun, E.I., ‘‘VATES VATES –– Virtual Autonomous Test and Evaluation SimulatorVirtual Autonomous Test and Evaluation Simulator’’ (Version 7 (Version 7 –– Professional), Professional), UserUser’’s Manual, 2000, 155 pp.s Manual, 2000, 155 pp.

9.9. Burdun, I.Y.,Burdun, I.Y.,““Prediction of Aircraft Flight Safety Performance in Complex SituPrediction of Aircraft Flight Safety Performance in Complex Situations using Results of Aerodynamics ations using Results of Aerodynamics Research and Flight Modeling and SimulationResearch and Flight Modeling and Simulation””, , Proc.Proc. of the Jubilee Conference of the Jubilee Conference ‘‘6060--th Anniversary of SibNIA Aircraft th Anniversary of SibNIA Aircraft Aerodynamics and Strength Research DivisionsAerodynamics and Strength Research Divisions’’, , 15 15 –– 17June 17June 20042004, , SibNIA, Novosibirsk, SibNIA, Novosibirsk, 2004, 2004, pp. 45 pp. 45 –– 57 (in Russian)57 (in Russian). .

10.10. Burdun, I.Y., Burdun, I.Y., ““Theory, Implementation and ProofTheory, Implementation and Proof--ofof--Concept Study of Flight Safety Concept Study of Flight Safety ‘‘TopologyTopology’’ Knowledge Maps for Accident Knowledge Maps for Accident Prediction and PreventionPrediction and Prevention””, , Proc. ofProc. of EWHSFFEWHSFF--2005 Conference, Chinese Aeronautical Establishment and 2005 Conference, Chinese Aeronautical Establishment and Beihang Beihang University, University, Beijing, P.R. China, 19Beijing, P.R. China, 19--22 October 200522 October 2005, PRC, pp., PRC, pp. 494 494 –– 502.502.

11.11. Burdun, I.Y., Burdun, I.Y., ““C.Reynolds Model of Motion SelfC.Reynolds Model of Motion Self--Organization and Some Issues of Application of HighlyOrganization and Some Issues of Application of Highly--Maneuverable Maneuverable HighlyHighly--Autonomous Unmanned Air VehiclesAutonomous Unmanned Air Vehicles””, , Proc.of Proc.of XVI XVI TsAGI Workshop TsAGI Workshop ““Aerodynamics of AircraftAerodynamics of Aircraft””, , 3 3 –– 4 March 20054 March 2005, , VolodarskyVolodarsky, , Moscow RegionMoscow Region, , TsAGI, TsAGI, 2005, 2005, pppp. 28. 28 –– 2929 (in Russian)(in Russian). .

Selected PublicationsSelected Publications

Burdun AUVSI-2007

Documents

integral safety

safety classification

current tactical

dynamic safety

aviation safety

bypassing

maintain commanded

safety window