Development and Validation of Simulation Metrics FAA-EUROCONTROL Technical Interchange Meeting Performance Metrics in Air Traffic Management Toulouse,

Development and Validation of Simulation Metrics

FAA-EUROCONTROL

Technical Interchange Meeting

Performance Metrics in Air Traffic Management

Toulouse, France

15-17 May 2002

Amanda Stott

Patricia Massimini

2

En Route Congestion

Enroute IntersectionsAt or above FL270

Terminal Area Delays

15,708

5,9005,331 5,246

4,918

3,9573,607

2,8272,462 2,244

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

18,000

ORD IAH ATL SFO PHL DFW EWR TEB STL CVG

Airports

Nu

mb

er

of

De

lay

s

0

500

1000

1500

2000

2500

3000

3500

4000

ORD ATL SFO PHL IAH EWR DFW TEB STL LGA

Airports

Da

ily

Min

ute

s o

f D

ela

y

An

nu

al

4

Airspace Redesign Approach

Define NewAirspace Configuration

Map Out Sectors and Routes

Analyze AlternativeAirspace Configuration

Identify and CharacterizeProblem Areas

Test with Controllers

Tools for Airspace Redesign

IDAT Intersect Density Analysis Toolset

CRCT Collaborative Routing Coordination Tool

DPAT Detailed Policy Assessment Tool

EACM Enhanced Airfield Capacity Model

NIRS Noise Integrated Routing System

MapInfo

NASPAC National Airspace System Performance Analysis Capability

SDAT Sector Design and Analysis Tool

TAAM Total Airspace and Airport Modeller

TRACS Traffic Reroute and Analysis Capability System

TARGETS Terminal Area Route Generation, Evaluation and Traffic Simulation

SIMMOD

RSATS Recombinant Sector Analysis Tool Suite

CAASD

CAASD

ArcView ESRI

CAASD

CAASD

FAA / Metron

MapInfo Corp.

CAASD

FAA

Preston

CAASD

CAASD

ATAC

CAASD

RAMS Plus Reorganised ATC Mathematical Simulator ISA Software

INM Integrated Noise Model FAA

Simulation Tools for Airspace Redesign

• Throughput– Peak and average

runway and airport usage

Metrics from Simulation

Simulated Throughput ofArrivals & Departures by Quarter Hour

0

5

10

15

20

25

30

1:00

2:30

4:00

5:30

7:00

8:30

10:0

0

11:3

0

13:0

0

14:3

0

16:0

0

17:3

0

19:0

0

20:3

0

22:0

0

23:3

0

1:00

Time of Day

Nu

mb

er

of

Op

era

tio

ns

Arrivals

Departures



• Delays– Arrival and

departure, ground and enroute


Cum

ulat

ive





• User impacts– Miles traveled







• Workload– Sector loading


Sector ID

125% of MAT 14 16 18 18 18 23 19 20 18 20 124 124 20 18 23 20 21 19 16 23 23 16 16 18 19 18 19 19 23 13 15 20 23 15 15 15 18 20 23 15 23 23 20 20 23Sector Counts 18:00 1 1 3 3 3 12 6 8 7 6 18 11 19 2 4 8 12 7 8 17 8 13 2 5 8 18 12 10 7 6 8 13 5 8 4 9 13 16 12 4 9 6 9 12 7

18:15 4 7 7 6 7 13 7 6 14 17 23 11 12 8 5 9 11 9 6 15 11 14 3 9 8 17 9 9 7 5 13 15 8 11 14 16 21 23 15 8 17 17 16 11 818:30 5 7 6 5 3 10 8 9 23 7 25 9 13 11 4 27 12 13 11 17 18 19 7 14 10 20 9 12 8 6 25 17 10 12 21 17 19 14 13 11 24 18 13 18 1318:45 8 12 8 6 7 14 11 9 13 9 22 8 15 16 4 25 13 9 14 6 12 22 11 20 10 21 11 13 14 9 24 8 12 13 16 25 26 10 11 9 17 14 12 13 719:00 12 9 10 7 10 13 12 10 7 7 22 9 18 16 9 16 15 8 12 8 11 15 10 21 13 16 17 14 11 5 17 9 10 13 4 10 18 11 14 8 16 14 12 11 319:15 6 7 17 11 13 11 8 8 6 6 26 13 17 24 12 14 19 13 14 17 6 9 9 10 18 14 15 9 7 8 11 21 16 12 5 9 13 11 13 7 8 17 18 15 619:30 12 14 14 10 9 9 6 9 14 5 17 12 22 31 9 9 15 10 15 14 12 18 6 11 19 17 28 7 11 7 18 15 11 8 9 9 20 16 23 11 9 12 17 19 1119:45 8 14 14 12 12 9 4 12 11 7 25 14 25 22 10 9 18 13 9 16 14 14 4 22 16 22 26 6 6 4 12 16 14 10 12 12 27 24 21 8 7 15 14 23 1120:00 10 11 13 6 15 11 6 7 14 14 26 15 27 17 8 17 16 13 16 22 18 13 20 17 15 29 27 8 13 10 12 15 14 13 11 15 30 21 23 7 8 16 16 25 1520:15 10 7 12 10 14 7 6 5 15 11 17 8 19 13 7 18 16 11 16 19 17 19 15 19 13 26 21 11 12 8 12 19 22 10 10 22 28 17 22 12 24 17 16 21 17

Sector ID


18:15 4 7 7 6 7 13 7 6 14 17 23 11 12 8 5 9 11 9 6 15 11 14 3 9 8 17 9 9 7 5 13 15 8 11 14 16 21 23 15 8 17 17 16 11 818:30 5 7 6 5 3 10 8 9 23 7 25 9 13 11 4 27 12 13 11 17 18 19 7 14 10 20 9 12 8 6 25 17 10 12 21 17 19 14 13 11 24 18 13 18 1318:45 8 12 8 6 7 14 11 9 13 9 22 8 15 16 4 25 13 9 14 6 12 22 11 20 10 21 11 13 14 9 24 8 12 13 16 25 26 10 11 9 17 14 12 13 719:00 12 9 10 7 10 13 12 10 7 7 22 9 18 16 9 16 15 8 12 8 11 15 10 21 13 16 17 14 11 5 17 9 10 13 4 10 18 11 14 8 16 14 12 11 319:15 6 7 17 11 13 11 8 8 6 6 26 13 17 24 12 14 19 13 14 17 6 9 9 10 18 14 15 9 7 8 11 21 16 12 5 9 13 11 13 7 8 17 18 15 619:30 12 14 14 10 9 9 6 9 14 5 17 12 22 31 9 9 15 10 15 14 12 18 6 11 19 17 28 7 11 7 18 15 11 8 9 9 20 16 23 11 9 12 17 19 1119:45 8 14 14 12 12 9 4 12 11 7 25 14 25 22 10 9 18 13 9 16 14 14 4 22 16 22 26 6 6 4 12 16 14 10 12 12 27 24 21 8 7 15 14 23 1120:00 10 11 13 6 15 11 6 7 14 14 26 15 27 17 8 17 16 13 16 22 18 13 20 17 15 29 27 8 13 10 12 15 14 13 11 15 30 21 23 7 8 16 16 25 1520:15 10 7 12 10 14 7 6 5 15 11 17 8 19 13 7 18 16 11 16 19 17 19 15 19 13 26 21 11 12 8 12 19 22 10 10 22 28 17 22 12 24 17 16 21 17

Sector ID 10 11 12 14 15 18 19 20 21 24 25 26 27 28 29 30 32 33 35 36 38 40 41 46 47 48 49 52 53 54 55 57 59 61 62 64 66 67 68 70 71 73 74 77 79

11 13 14 14 14 18 15 16 14 16 99 99 16 14 18 16 17 15 13 18 18 13 13 14 15 14 15 15 18 10 12 16 18 12 12 12 14 16 18 12 18 18 16 16 18

125% of MAT

Sector Counts

Sector ID 10 11 12 14 15 18 19 20 21 24 25 26 27 28 29 30 32 33 35 36 38 40 41 46 47 48 49 52 53 54 55 57 59 61 62 64 66 67 68 70 71 73 74 77 79

11 13 14 14 14 18 15 16 14 16 99 99 16 14 18 16 17 15 13 18 18 13 13 14 15 14 15 15 18 10 12 16 18 12 12 12 14 16 18 12 18 18 16 16 18

125% of MAT

Sector Counts

MAT Value







– Conflicts








– Conflicts

– Visualization


• Others as determined by study team

– Communications loading

– Point outs







– Conflicts

– Visualization

12

FAA Operational Data

• Traffic– Current traffic: OAG, ETMS, SAR, ARTS

– Future traffic: airport authority, major carriers, ATA, FAA TAF, EIS, future traffic models

• Operations– OPSNET, ASPM

• Delays, taxi-out and taxi-in times, throughput, interarrival and interdeparture times

– ASPM, ASQP

• Safety– Operational errors / operational deviations

• Traffic flow management actions– Miles in trail– Ground stops, ground delay programs

Degree of Correspondence Between Simulation Results and Operational Data

• Operational errors• Operational deviations• Vectoring• Traffic flow management actions

Operational Data

Arrival delay, taxi out…

• Times for every aircraft and controller action

• Location and time of loss of separation if no controller action

• Altitudes• Taxi-out times• Delays• Interarrival times• Interdeparture times• Runway throughput

Simulation

14

Calibration and Validation

Arrival Rates

Nu

mb

er

of

An

nu

al

Arr

iva

l P

us

he

s a

t S

pe

cif

ied

Ra

te

• Typically, will calibrate baseline operations

– Runway arrival and departure rates

15




– Interarrival or interdeparture times

Time Between Operations (minutes)

Pe

rce

nt

of

To

tal

Tra

ffic ARTS

Simulated results

16





– Departure altitudes0 - 7000 FT 7100 - 17000 FT

17100 - 23000 FT 23100 - 33000 FT




– Departure altitudes

– Taxi-out or taxi-in times

– Delays

• Validation – Visual with controllers

– Analysis of summary statistics

Example: Detroit Terminal and Cleveland En Route Airspace

Example: Detroit Terminal and Cleveland En Route Airspace

CVG

MSP

ORDMDW

STL

PIT

CLE

DFW

MEM

ATL

CLT

BWI IAD DCA

PHLEWR LGA JFK

BOS

DTW

Issues at ZOB and DTW

• Use of new runway for triple arrivals

22L

22R21R

21L



• New arrival and departure routes and relief of MIT

DTW




• Relief of congested enroute airspace



• Relief of congested enroute airspace

• Impact on ground operations

Traffic Schedule

Scheduled Departures and Arrivals per Quarter Hour, 2002 DTW Traffic

05

10152025303540

1:00

2:00

3:00

4:00

5:00

6:00

7:00

8:00

9:00

10:0

0

11:0

0

12:0

0

13:0

0

14:0

0

15:0

0

16:0

0

17:0

0

18:0

0

19:0

0

20:0

0

21:0

0

22:0

0

23:0

0

0:00

Time of Day

Nu

mb

er o

f Op

era

tion

s

22

Departures

Arrivals

Simulated Arrival Throughput

23

Arrivals per Quarter Hour

0

5

10

15

20

25

30

35

40

5:0

0

5:4

5

6:3

0

7:1

5

8:0

0

8:4

5

9:3

0

11:0

0

14:0

0

17:0

0

20:0

0

23:0

0

Time of Day

Arr

ival

s

Dual Arrivals Only

Triple Arrivals AllowedTriple Arrivals

Dual Arrivals

24

$24 million per year with PRM and new procedures

Effect of Triple Arrivals on Delay

0

5,000

10,000

15,000

North Flow South Flow

Dai

ly D

elay

(M

inu

tes)

Dual Arrivals

Triple Arrivals

Applies only to IMC, MIT enforced

16%49%

South restrictions (15-20 MIT)

East restrictions (15-20 MIT)

No restrictions

Miles in Trail Restrictions on DTW Departures

26

Relieving MIT Departure Restrictions

Reroute flows and decrease sector

complexity

Add departure routes

27

$24 million per year with new routes and new sectors

Effect of MIT Restrictions on Departure Delay

0

5000

10000

15000

20000

North South

Min

ute

s o

f D

epar

ture

Del

ay

With MIT restrictions

Without MIT restrictions

En Route Redesign to Support Relief of Departure Restrictions

103 reroute sets defined, more than 1300 flights per day

BeforeAfter

28

Approach for Analysis of En Route Redesign

SDAR--No Reroute

MapInfo to select and visualize reroutes

BeforeAfter

29

Rerouted trajectories

+ Nonrerouted SAR tracks

URET and SDAT to generate metrics

SDAT Viewer to visualize traffic flows

Sector ID 10 11 12 14 15 18 19 20 21 24 25 26 27 28 29 30 32 33 35 36 38 40 41 46 47 48 49 52 53 54 55 57 59 61 62 64 66 67 68 70 71 73 74 77 79

11 13 14 14 14 18 15 16 14 16 99 99 16 14 18 16 17 15 13 18 18 13 13 14 15 14 15 15 18 10 12 16 18 12 12 12 14 16 18 12 18 18 16 16 189 10 11 11 11 14 12 13 11 13 79 79 13 11 14 13 14 12 10 14 14 10 10 11 12 11 12 12 14 8 10 13 14 10 10 10 11 13 14 10 14 14 13 13 14

125% of MAT 14 16 18 18 18 23 19 20 18 20 124 124 20 18 23 20 21 19 16 23 23 16 16 18 19 18 19 19 23 13 15 20 23 15 15 15 18 20 23 15 23 23 20 20 23STATUS CHANGE 18:00 0 0 3 1 -1 1 1 0 2 2 -1 -1 -4 1 -1 1 5 -1 1 -7 1 0 0 0 -3 0 -2 -3 -1 -2 1 -2 0 0 -2 -2 -2 0 -1 2 0 2 -2 -1 0

18:15 1 0 -2 1 -1 4 0 1 0 -2 10 -2 -3 3 -1 0 4 2 -1 -5 4 -2 1 -2 -1 -5 2 -2 0 -1 4 1 4 5 3 -1 -2 -6 -2 -1 -3 -1 -2 -1 618:30 1 1 1 -1 1 2 -1 3 -4 3 -2 -4 -1 6 1 -5 0 -3 0 -1 -1 -2 3 2 1 0 4 0 2 0 0 -2 0 0 1 3 -3 -4 0 2 -2 -4 -2 -2 -218:45 -3 1 1 1 -2 1 3 0 -5 1 2 1 0 1 4 -2 2 -4 -1 1 0 -2 4 -2 0 -3 5 2 3 -2 -3 0 -2 -1 -2 -5 -7 -2 2 -1 -1 3 -1 -1 -119:00 -3 0 0 -2 -2 3 4 1 1 0 1 -2 -6 0 2 -2 3 2 -1 4 -3 1 9 -2 3 -1 -1 -2 -2 2 1 0 1 1 1 -1 2 -3 -1 -1 -6 2 3 -1 2

EXCEL to analyze and

display results

Altitude restrictions

+Adaptation

data

URET restriction builder and

trajectory model

Custom tool to insert rerouted

flight plans

30

Results of Analysis of Redesign:Max 15-Minute Sector Counts

< 80% MAT80% MAT - 100% MAT

> 125% MAT100% MAT - 125% MAT

Sector ID


18:15 4 7 7 6 7 13 7 6 14 17 23 11 12 8 5 9 11 9 6 15 11 14 3 9 8 17 9 9 7 5 13 15 8 11 14 16 21 23 15 8 17 17 16 11 818:30 5 7 6 5 3 10 8 9 23 7 25 9 13 11 4 27 12 13 11 17 18 19 7 14 10 20 9 12 8 6 25 17 10 12 21 17 19 14 13 11 24 18 13 18 1318:45 8 12 8 6 7 14 11 9 13 9 22 8 15 16 4 25 13 9 14 6 12 22 11 20 10 21 11 13 14 9 24 8 12 13 16 25 26 10 11 9 17 14 12 13 719:00 12 9 10 7 10 13 12 10 7 7 22 9 18 16 9 16 15 8 12 8 11 15 10 21 13 16 17 14 11 5 17 9 10 13 4 10 18 11 14 8 16 14 12 11 319:15 6 7 17 11 13 11 8 8 6 6 26 13 17 24 12 14 19 13 14 17 6 9 9 10 18 14 15 9 7 8 11 21 16 12 5 9 13 11 13 7 8 17 18 15 619:30 12 14 14 10 9 9 6 9 14 5 17 12 22 31 9 9 15 10 15 14 12 18 6 11 19 17 28 7 11 7 18 15 11 8 9 9 20 16 23 11 9 12 17 19 1119:45 8 14 14 12 12 9 4 12 11 7 25 14 25 22 10 9 18 13 9 16 14 14 4 22 16 22 26 6 6 4 12 16 14 10 12 12 27 24 21 8 7 15 14 23 1120:00 10 11 13 6 15 11 6 7 14 14 26 15 27 17 8 17 16 13 16 22 18 13 20 17 15 29 27 8 13 10 12 15 14 13 11 15 30 21 23 7 8 16 16 25 1520:15 10 7 12 10 14 7 6 5 15 11 17 8 19 13 7 18 16 11 16 19 17 19 15 19 13 26 21 11 12 8 12 19 22 10 10 22 28 17 22 12 24 17 16 21 1720:30 13 11 6 13 21 11 8 5 11 8 21 7 11 10 7 18 15 10 8 16 10 16 12 18 18 16 19 14 20 12 23 20 14 12 6 18 18 17 19 11 23 15 21 21 920:45 5 6 12 17 15 15 9 7 10 7 22 12 11 16 11 18 13 11 7 12 7 17 9 12 20 19 25 13 16 10 13 17 14 10 6 14 17 17 21 9 12 13 15 14 1121:00 10 10 8 10 11 8 4 12 8 4 20 15 13 15 12 8 11 7 10 18 20 16 3 8 13 14 22 11 20 13 12 12 12 7 10 11 16 16 15 7 11 12 19 14 1321:15 12 20 11 13 18 13 8 13 15 4 23 13 11 19 16 15 19 16 5 21 17 17 11 25 16 15 20 11 17 15 9 15 15 7 8 16 13 18 15 6 14 17 24 15 1821:30 10 18 7 10 18 18 10 19 18 10 26 18 16 15 14 10 16 19 10 16 15 24 11 22 16 28 21 11 12 10 14 22 16 10 4 17 18 21 19 3 14 13 19 15 1821:45 15 16 11 7 13 22 13 14 12 10 30 19 21 10 8 23 18 19 16 16 11 12 7 19 17 25 25 13 6 6 12 14 13 9 14 15 18 15 14 3 18 14 21 20 2122:00 8 12 5 9 14 13 7 5 15 9 22 12 21 14 8 22 18 12 10 17 15 13 12 10 12 23 24 16 11 7 20 22 13 6 16 14 17 26 16 5 20 15 23 20 1822:15 11 13 7 9 10 11 4 11 25 17 22 11 21 14 10 21 22 14 8 22 16 21 13 18 12 23 20 16 12 6 17 20 11 9 14 16 28 27 18 5 14 15 21 26 1022:30 9 10 7 11 13 12 8 13 15 16 25 13 15 18 7 8 14 11 6 26 15 12 14 14 12 24 23 9 4 6 10 14 13 6 16 15 25 21 19 8 5 12 19 26 1322:45 3 8 10 4 10 13 9 11 9 5 28 17 21 20 4 10 9 6 10 24 22 11 10 17 12 21 24 8 8 7 10 16 10 8 11 11 24 21 24 3 6 9 11 22 1423:00 8 12 11 9 14 16 7 6 10 8 27 12 20 16 9 8 9 10 11 20 22 10 7 15 11 21 21 7 11 17 9 11 12 9 8 11 22 16 21 7 14 12 11 17 1223:15 12 15 17 10 20 23 11 4 12 12 28 13 26 24 8 7 12 9 15 17 14 13 7 18 18 21 25 7 14 11 7 12 20 15 5 11 17 18 21 7 13 15 14 12 623:30 7 15 9 8 12 20 10 7 15 12 21 16 27 11 6 8 14 7 16 13 11 10 11 15 17 15 21 10 14 8 8 8 9 9 4 14 15 17 14 6 13 12 13 19 723:45 7 11 10 11 10 16 13 5 17 5 19 13 15 16 3 19 14 11 10 17 10 7 10 18 11 20 19 10 9 8 6 6 14 6 6 17 15 19 13 5 15 14 12 17 13

Sector ID 10 11 12 14 15 18 19 20 21 24 25 26 27 28 29 30 32 33 35 36 38 40 41 46 47 48 49 52 53 54 55 57 59 61 62 64 66 67 68 70 71 73 74 77 79

11 13 14 14 14 18 15 16 14 16 99 99 16 14 18 16 17 15 13 18 18 13 13 14 15 14 15 15 18 10 12 16 18 12 12 12 14 16 18 12 18 18 16 16 18

125% of MAT

Sector Counts

MAT Value

31

Results of Analysis of Redesign:Hourly Sector Counts

< 50/hour51-60/hour

>70/hour61-70/hour

20 29 14 16 14 24 17 23 27 28 21 38 30 18 18 16 1321 26 21 27 42 27 39 38 38 37 32 50 48 34 21 16 1524 18 19 30 21 13 16 18 24 24 26 22 29 29 17 13 1325 72 70 82 53 34 42 56 63 75 72 86 95 70 58 62 4026 18 29 23 8 14 18 24 24 30 30 43 36 38 36 31 2027 27 32 29 31 19 33 56 41 62 50 64 51 59 39 27 2528 25 31 30 8 15 29 21 31 60 43 49 54 51 44 23 2829 10 14 12 2 5 13 16 15 30 25 33 23 20 34 22 1830 46 29 19 32 23 30 26 43 33 44 40 46 29 27 31 2232 54 34 26 31 19 37 33 44 48 45 48 44 39 32 27 3433 42 33 29 33 13 30 32 25 34 40 45 29 34 25 27 2435 31 19 17 9 6 26 24 24 31 30 26 25 30 23 24 936 35 35 22 40 31 37 41 38 48 50 53 68 45 35 30 2738 48 49 36 23 19 30 35 36 35 42 48 56 38 30 29 2840 17 34 32 35 24 35 36 39 34 38 55 39 30 20 22 1741 17 21 24 31 16 34 39 27 34 49 34 31 33 23 11 846 39 44 45 29 22 45 41 59 61 58 57 58 63 46 35 2447 19 28 25 20 18 18 21 26 47 46 42 36 34 41 39 2148 45 46 46 36 33 41 45 57 58 67 70 58 54 46 36 2449 26 39 33 37 32 37 32 36 63 69 73 64 60 52 42 3252 22 31 17 17 12 30 23 28 25 33 30 30 18 22 14 1953 28 32 20 22 13 35 33 30 29 42 36 27 36 24 16 1254 14 30 20 19 13 20 24 15 17 24 24 25 28 16 10 555 31 39 26 17 11 25 33 52 36 40 34 41 21 21 24 1857 40 39 33 21 23 23 28 35 41 46 45 55 27 34 25 4159 36 39 29 26 38 31 25 29 45 54 44 35 44 40 38 3361 34 21 23 18 13 22 25 34 38 32 28 26 25 29 26 1262 33 16 21 12 13 18 12 40 23 22 28 42 16 17 32 964 22 31 40 34 33 42 23 51 28 50 40 44 43 23 15 1166 38 43 33 42 34 32 38 55 67 56 51 73 57 51 49 3867 38 32 37 32 29 27 32 43 50 54 46 63 55 53 43 4268 33 42 39 24 23 32 27 41 50 58 49 59 56 49 35 2870 13 15 19 23 10 17 16 18 23 23 10 12 18 20 6 8

ZO

B S

ecto

r ID

7:00 8:00 9:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00

10 25 19 22 21 14 17 20 19 31 28 36 26 27 18 11 8

EDT

ZO

B S

ecto

r ID

32

Results of Analysis of Redesign: Difference in Sector Counts Before and After Reroutes

No change in sector countSector count decrease

Sector count increase

Sector ID 10 11 12 14 15 18 19 20 21 24 25 26 27 28 29 30 32 33 35 36 38 40 41 46 47 48 49 52 53 54 55 57 59 61 62 64 66 67 68 70 71 73 74 77 79

MAT Value 11 13 14 14 14 18 15 16 14 16 99 99 16 14 18 16 17 15 13 18 18 13 13 14 15 14 15 15 18 10 12 16 18 12 12 12 14 16 18 12 18 18 16 16 1818:00 0 0 3 1 -1 1 1 0 2 2 -1 -1 -4 1 -1 1 5 -1 1 -7 1 0 0 0 -3 0 -2 -3 -1 -2 1 -2 0 0 -2 -2 -2 0 -1 2 0 2 -2 -1 018:15 1 0 -2 1 -1 4 0 1 0 -2 10 -2 -3 3 -1 0 4 2 -1 -5 4 -2 1 -2 -1 -5 2 -2 0 -1 4 1 4 5 3 -1 -2 -6 -2 -1 -3 -1 -2 -1 618:30 1 1 1 -1 1 2 -1 3 -4 3 -2 -4 -1 6 1 -5 0 -3 0 -1 -1 -2 3 2 1 0 4 0 2 0 0 -2 0 0 1 3 -3 -4 0 2 -2 -4 -2 -2 -218:45 -3 1 1 1 -2 1 3 0 -5 1 2 1 0 1 4 -2 2 -4 -1 1 0 -2 4 -2 0 -3 5 2 3 -2 -3 0 -2 -1 -2 -5 -7 -2 2 -1 -1 3 -1 -1 -119:00 -3 0 0 -2 -2 3 4 1 1 0 1 -2 -6 0 2 -2 3 2 -1 4 -3 1 9 -2 3 -1 -1 -2 -2 2 1 0 1 1 1 -1 2 -3 -1 -1 -6 2 3 -1 219:15 -3 -2 2 0 1 -1 0 1 3 0 -4 -2 1 2 -1 -2 2 -3 -6 -1 5 7 1 2 0 -5 7 1 3 1 2 -5 3 -4 -1 0 1 2 5 0 0 2 -2 -1 419:30 3 -1 -1 2 -2 1 -1 3 -2 0 4 -2 5 -5 0 2 3 -2 0 2 7 -2 3 3 0 -5 -6 0 1 -2 -4 -2 6 2 3 -2 0 4 4 0 0 3 0 1 019:45 4 -3 1 3 1 0 2 1 3 1 8 0 -1 -4 1 0 0 -1 0 4 4 -2 4 1 1 0 -3 1 3 -1 -4 1 1 3 0 -1 2 0 2 5 -3 4 3 -3 220:00 2 1 0 0 5 -1 0 0 6 -2 3 -6 -3 -1 0 -1 0 -1 3 -1 2 6 -5 2 -2 -2 -3 0 0 -3 -5 1 4 0 0 0 -4 -1 0 1 3 -1 -2 0 120:15 2 1 1 1 4 1 2 0 3 -1 6 2 0 0 2 -3 1 1 -2 3 0 3 8 3 -1 -8 0 2 1 -2 -1 -1 2 4 -3 -6 -5 1 3 -3 1 2 -3 1 -120:30 -3 -4 3 0 -6 3 4 0 2 -1 3 0 -2 3 1 -1 1 1 -1 3 1 0 3 2 -1 -4 3 0 -1 -2 -1 -5 1 1 -2 2 -1 1 2 -5 -5 1 -3 -4 020:45 0 -5 1 -4 -3 0 0 3 -1 0 3 2 3 1 0 -5 5 2 1 3 6 1 0 1 -1 -4 5 -2 -4 1 1 -3 -1 -1 0 -3 0 -3 -3 -2 -3 5 1 -2 221:00 0 -2 1 -2 0 3 2 1 0 -1 4 -2 3 3 2 -1 8 4 -1 -1 3 -2 6 2 0 -3 2 -4 1 -2 -3 -2 2 1 -1 -1 -1 -4 3 -1 -2 7 -2 1 721:15 0 0 -1 1 -2 2 5 3 1 2 6 1 4 1 0 -8 0 -3 -4 -1 3 2 4 -2 -2 4 0 -1 -1 -5 3 0 5 3 -4 0 0 0 6 -3 -3 0 -6 -2 321:30 -1 -2 3 0 -1 1 6 3 -4 1 1 -1 -1 -2 -4 2 -1 -2 1 3 -1 1 4 -3 1 -4 2 -1 -1 -3 -2 -4 0 -1 1 -4 1 -3 1 0 0 0 -2 1 121:45 4 -3 1 0 6 -2 1 -2 -2 -2 5 -5 1 4 -1 -1 3 -6 -3 0 5 -1 6 -1 1 -10 3 -1 3 0 -2 3 -3 0 0 1 -1 1 2 0 -5 2 1 2 022:00 1 1 -1 -1 2 1 1 -1 1 2 10 -4 -4 2 3 1 0 2 0 4 7 5 4 3 0 -2 -4 0 -2 0 0 5 1 2 -2 0 5 -1 6 -3 -7 1 1 -4 122:15 2 2 2 -1 -1 -2 5 3 -2 -2 12 4 -1 0 0 -1 1 -5 -1 5 3 0 5 2 0 1 -2 -4 -3 -1 -1 0 0 2 1 1 -3 -8 6 -1 -4 1 -3 -2 322:30 1 -3 4 0 -2 3 2 2 2 -3 8 5 0 2 -3 3 -1 -3 -1 -4 5 -1 -5 2 0 -1 1 -3 4 1 -2 2 -2 0 -2 -1 -4 -2 5 0 0 0 -4 -2 322:45 0 -3 0 0 0 4 5 -1 1 0 6 -3 2 4 2 0 0 -1 0 -6 -2 -5 -1 0 1 -2 0 -1 3 3 -1 1 4 2 0 3 -3 -1 3 0 2 -1 0 -3 223:00 -4 -1 1 -3 -1 0 3 1 1 0 -2 4 -2 6 -2 -3 -1 0 1 0 0 4 1 -1 0 0 1 -2 2 -3 -2 0 6 0 0 0 0 -1 3 2 2 0 -2 2 023:15 -1 -3 2 0 -5 1 2 2 0 -2 1 -2 5 -4 -3 2 4 1 0 1 5 -1 2 3 -1 -8 -5 0 1 0 0 -2 -3 -5 0 1 -1 -2 5 0 1 4 -1 5 023:30 5 -3 3 2 -2 1 3 1 -1 -2 1 -2 -3 2 -1 3 4 0 1 4 3 7 5 0 -2 -1 3 -1 -1 -3 -1 0 3 -2 -1 0 1 -2 3 -1 4 2 2 1 523:45 0 0 -1 1 -1 0 2 0 -1 -1 1 -3 -3 2 2 -3 5 1 -3 -1 -3 11 2 0 0 -10 -1 -2 0 -1 0 2 4 -2 -2 -3 3 6 1 2 -2 -6 -5 -4 1

33

Results of Analysis of Redesign: Individual Sector Behavior

Notional results

0

5

10

15

20

25

30

3518

:00

18:1

5

18:3

0

18:4

5

19:0

0

19:1

5

19:3

0

19:4

5

20:0

0

20:1

5

20:3

0

20:4

5

21:0

0

21:1

5

21:3

0

21:4

5

22:0

0

22:1

5

22:3

0

22:4

5

23:0

0

23:1

5

23:3

0

23:4

5

GMT

Nu

mb

er

of

Fli

gh

ts

Baseline

Reroute

MATMonitor Monitor

Alert ParameterAlert Parameter

After Reroute

Before Reroute

Results of Analysis of Redesign: Visualization of Flows

South Flow, Backups under Heavy Future Traffic

Ground Flow Analysis to Support New Runway and Triple Arrivals

35

Identified chokepoints and developed new ground routes

36

Concept Dependencies

• Ability to simulate is concept dependent

• Simulation tools may not be able to deliver all required metrics

Time on Voice Channel

00.5

11.5

22.5

33.5

44.5

5

0 5 10 15 20 25 30 35 40 45 50 55

Time

To

tal M

inu

tes

Use

dp

er 5

-Min

ute

Inte

rval Alt 1

Alt 2Alt 3

Development and Validation of Simulation Metrics FAA-EUROCONTROL Technical Interchange Meeting Performance Metrics in Air Traffic Management Toulouse,

Documents

airspace slide

fl270 slide

airspace redesign slide

simulation cumulative

average runway

airport usage delays

relief of mit slide

departures arrivals