CARE ASAS Validation Framework
Mar 27, 2015
CARE ASAS Validation Framework
Partners
CARE ASAS Board - Francis Casaux EURCONTROL - Mick van Gool, Ulrich
Borkenhagen
Consortium Partners Aena, Isdefe, NATS, NLR, QinetiQ
Currently 6 months into 12 month programme
Validation of ASAS
Validation Does proposed operational concept provide anticipated
performance?
Operational concept
Performance Framework
High Level Validation Process
ASAS Framework Requirements
Operational Concept - Airborne Separation Assurance Systems
Performance Framework Reference Scenario High level objectives Metrics
Links to Previous work Support to Users
Operational Concept
ASAS application categories, from PO-ASAS
Airborne Traffic Situation Awareness• Current ATC rules apply
Airborne Spacing • Aircrew given more tasks
Airborne Separation• Temporary delegation of separation to aircrew
Airborne Self-separation• Full delegation of separation
Performance Framework
ATM 2000+ Strategy Objectives Safety Capacity etc
Framework Scenario Operationally realistic Scenario data repository
Metrics System Performance Metrics Human Performance Metrics
Validation Exercises
Operations
Operationaltrial
Shadow-modetrial
Field test
Large-scalereal-time
simulation
Small-scalereal-time
simulation
Fast-timesimulation
Analyticmodelling
Links to Previous Work
CARE Integra MAEVA EMERALD RTD Plan Other EUROCONTROL work
Support to User
Guidelines - assistance to validation teams and managers
Case Studies (Worked examples) Scenario database Documentation Dissemination Forum
Work Packages
S ce n ario D a ta R e po s ito ry
F ra m ew o rk S ce n a rioW P 1
Isd e fe , A e na & N LR
S ys te m P e rfo rm a n ce M e tricsW P 2
Q in e tiQ & A e na
H u m an P e rfo rm an ce M e tricsW P 3
N L R & N A TS
M e tric s
V a lid a tio n F ram e w o rk
A p p lica tio n G u ide lin esIsde fe
T w o De ta ile d C ase S tu d iesN A T S
E M E R A L D R T D P lanQ in e tiQ
F ra m e wo rk A pp lica tionW P 4
N A T S
O ve ra ll M a n ag e m e n t a n d C o ord ina tionW P 0
N A T S
Scenario Framework
• STANDARDIZE the validation scenarios for ASAS Applications
Scenario Framework based on Template
• Facilitate the TRACEABILITY of the scenarios
• SUPPORT DESIGNERS in the creation of validation scenarios for ASAS applications
• ENSURE THOROUGH AND CONSISTENT validation of ASAS applications
• Why is a scenario template needed ?
• MAIN GOAL: Supporting the designer in the creation of a validation scenario
Objectives of Template
• Structure based on PO-ASAS categories and applications with EMERALD
• One template for each PO-ASAS category: Airborne Traffic Situational Awareness, Airborne Spacing, Airborne Separation, Airborne Self-Separation
• Items for scenario definition clustered in seven groups: Objectives, Airspace, Traffic, ATS Involved, Rules, Tasks, Actors (Performer & Supervisor), Technology
• USER SELECTS the ITEMS that will form the scenario but SOME of them are MANDATORY
• Based on the chosen items, designer develops the scenario
• OBJECTIVES: to be grouped by types (Safety, Capacity…)
• AIRSPACE: Restrictions, Types, Areas, Elements, Geographical Scope
• TRAFFIC: Volume, Complexity, Flight Schedule, Timeframe, Aircraft Type, Equipment Type, Aircraft Performance
• ATS INVOLVED: ATC, FIS, TIS-B, APP, Alert Service, GNSS, ATFM, Performance Management, New Services
• RULES: Flight Rules, Longitudinal Separation, Lateral Separation, Phraseology, RVSM, Aircraft Sequencing, Conflict Resolution Strategy, Co-ordination and Transfer Procedures, Trained Flight Crews
• TASKS AND ACTORS - ‘Perform Separation Assurance’: Performer & Supervisor
• TECHNOLOGY:– GROUND: Communications, Surveillance, ATM– AIRBORNE: Redundancy, Communications, Surveillance, Navigation,
Display
Scenario Definition Groups
Example: Technology GroupGROUND
Communications:VoiceData linkInter-sector information transferSurveillance:Radar coverageNo radar coverageTIS-BADS-BCWP HMI with aircraft equipment, ID and delegation statusController assistance toolSequencing toolSeparation monitoring toolATMPlanningAutomation
AIRBORNERedundancyCommunications:VoiceData linkSurveillance:Mode-STCAS II/ETCASAudio/visual alertsTraffic information processingSurveillance data processing Spacing function processingNavigation:GNSS with augmentationFMSAutomatic management of the separation (connection with FGCS)Display:CDTI/EFIS/MCDU HUD
Enhanced Visual Adquisition, EMERALD
Longitudinal Station Keeping, EMERALD and NATS
Closely Space Parallel Approaches, EMERALD
Station Keeping Approach, EMERTA
Autonomous Aircraft, EMERALD
Baseline En-route Free Flight, NLR
TMA ASAS Autonomous Aircraft, NLR
FREER-EACAC, EEC
Multi agent optimal ASAS operations (COAST), Glasgow University
Optimal FFA/MAS Transition Methodology, Glasgow University
ASAS Crossing Procendure, CENA
Sector capacity assessment for ASAS, CENA
Applied on Historic Data
Scenario Database
Gathered data from sources in CARE ASAS and the VF consortium
Uses EUROCONTROL Validation Data Repository
Agreed data structure for scenarios based on template
Obtained agreement for sharing of data
Access via EUROCONTROL
System Performance Metrics
System Performance Metrics
Aim was not to develop new metrics
Accessible advice to validators
High level objectives Hierarchy mapping objectives to metrics
• Many to many
Taxonomy Perspectives
Objectives Objectives covered, based on ATM 2000+ Strategy
are: Safety Capacity Efficiency Environment Security & Defence
Security & Defence extended to include terrorist threat
Uniformity and Quality not considered further
Hierarchy
OBJECTIVES
PERFORMANCE AREAS
( ASPECTS)
METRICS
Hierarchy
PERFORMANCE AREAS broken down into ASPECTS where appropriate
Example: ACCESS (PERFORMANCE AREA)
• Airports• Sectors (ASPECTS)• Routes
Assists use with scenarios that look at specific airspace
Taxonomy
METRICS INDICATORS CARE-ASAS PRC Validation C/AFT Framework TORCH
Database tool For easy manipulation of data and linkages
Database tool Enabling instant switch from top-down...
Database tool … to bottom-up view
Perspectives
Different views (perspectives) can be applied to the information stored:
Airline perspective as in C/AFT
ATM perspective as in PRC
Scenario
Perspectives
Further perspectives under consideration are:
Study type (FTS, RTS, Analytic or Survey)
ASAS categories (Situational awareness…Self Separation)
Links Human Performance metrics
Links to Previous and Parallel Work
Potential Links
CARE Integra
MAEVA
EMERALD
Ongoing EUROCONTROL work
CARE Integra
EUROCONTROL funded study
Metrics Capacity
Safety
Environment
Economics
QinetiQ involvement in Integra and VF
Use of Integra metrics
MAEVA
A Master ATM European Validation Plan
Supporting EC in management of ATM projects Validation Guideline Handbook - exercise level Validation Master Plan - project validation plans Monitoring of projects
Final Consolidation of Results from Projects
NATS and Isdefe involvement in MAEVA & VF
Alignment of VF with MAEVA VGH
MAEVA Five-Step Validation Methodology
Step 1: Aims
Step 5: Conclusions
Step 2: Preparation
Step 4: Analysis
Step 3: Execution
Others
EMERALD RTD Plan
Plan for RTD for future technology for ATM systems
NATS and QinetiQ involvement in EMERALD & VF
Production of updated EMERALD RTD Plan
On-going EUROCONTROL work Validation Guideline VALSUP
Alignment of VF with VALSUP
Support to the User
Documentation
Final report incorporating work package reports Guidance Material for application of framework Case studies of two applications
In-descent spacing Fully autonomous aircraft
Detailed RTD Plan developed from EMERALD Plan
Data and Tools
Scenario data repository Produced in WP1
Metrics database Produced in WP2 Not a deliverable under contract
Dissemination Forum
Dissemination Forum 10th October at Bretigny
Summary
Validation Framework still under development Exploiting existing work for ASAS validation
exercises Scenario template developed Scenario data collated for VDR Mapping of system performance metrics & human
performance metrics to objectives - in progress Support user - about to start