FY12 TIM Overview

DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution Unlimited (Case Number: 88ABW-2012-3190)

Integrity Service Excellence

Matthew Clark

Control Systems Development & Applications

(AFRL/RBCCZ)

Air Force Research Laboratory

[email protected]

Run Time Assurance


Challenges for

Control Systems Certification

V&V dominant driver

Increases in Test Time, Man-hours & Costs

Unmanageable # of Lines of Code Future UAV Functionality Outdate Current

V&V and Certification Process

On Board Situational Awareness & Contingency

Management

Mixed Initiative: Man-Autonomy

Authority Mgmt: Autonomy-Autonomy

Mixed Criticality: Mission & Flight

Advances in V&V and Certification Enable Intelligent – Autonomous UAV Control Systems

2


Motivations and Challenges • Non-deterministic software cannot be

exhaustively tested or certified offline

• State-space explosion makes new systems too costly for conventional test

Enable certification for unverifiable functionality

through dynamic, predictive bounding

Off-Line Assurance

Run Time Assurance

3

Run Time Assurance


Review of Previous Work

VVIACS

• Identified Gap in VV of Non-deterministic Systems

• Recommended a RTA Follow-on Program

TASS SBIR

I, II

• Established Boundary Algorithm Base

• Inception of Bounded Determinism

• Catalyst for Hybrid Systems Research

CertA

FCS

• Linked Formal Certification Techniques with RTA

• Foundational Demonstration of Capability

• Lacked General approach to RTA problem

RTA UNIV

• Re-Scope the problem

• Academic Exploration Research on the State of the Art

RTA SBIR III

• General Framework Development

• Expansion of SBIR PH I work with Barron Assoc.

• Leverage New Research and Challenge Problems

2002

2012

New

Start

2005

4


Run Time Assurance:

Bounding the problem

• Barron Associates developed a trajectory based approach in the CertA FCS program

• Advanced Auto Land Control Function demonstrated viability

• Looking for hazards and incorrect nominal behavior

5

Current work looks at a more general approach to bounding

non-deterministic behavior of any system


Goal:

•Survey the current research in academia most applicable for a Run Time Assurance framework for flight critical systems

Approach:

• Identify key researchers known in the fields of

•Run Time Verification

•Adaptive Control

•Formal Methods

•Hold two workshops in the public domain, querying the community for solutions bounding a non-deterministic system

FY12 University Study

Not under Contract

6


Barron Assoc vs. CMU Simplex

Wrapper Architecture

7

• Allows Performance beyond baseline Control

• Requires specific conditions of assurance

that may not be applicable in the real world

• Relies on Formal Methods approaches to

offline Boundary verification

• Guaranteed within safe operating region

• Cannot perform better than baseline

control

• Relies on intensive off-line modeling and

simulation to create specific cases

ROR - Region of Recovery RAE - Recoverability achievability envelope RSE - Recovery Safety Envelope BRAE - Baseline Recovery Achievability Envelope


Question 1:

Are there algorithms available to establish a control boundary

with known constraints?

Given: Set of known Inputs / Outputs / Environment

Can vehicle achieve advanced maneuver?

References: Ding, Li, Huang, Tomlin Reachability Based Controller Synthesis for Switched Systems ICRA 2010 http://www.eecs.berkeley.edu/~jding/Presentations/ICRA%20Presentation.ppt

FY12 University Study Run-Time assurance algorithms and challenges

NEED:Control Bounding / Prediction Algorithm

8

http://www.eecs.berkeley.edu/~jding/Presentations/ICRA Presentation.ppt


Algorithms to verify Bound Behavior?

Challenges:

• How do we define envelopes? Model them?

• What states, critical parameters should be monitored?

• How do we leverage current Formal Methods approaches to reduce boundary simulation?

• How far in the future do we predict?

• What guarantees can we make about our prediction?

9


Tools

• SpaceX (VERIMAG) – Reachability using zonotopes

• LTLMOP (Cornell University) – Temporal logic robot planning and control

• PESSOA (U.C.L.A) – Approximate symbolic control of nonlinear systems

• MATISSE (U. Joseph Fourier) – Approximate bisimulation computations

• LTLcon (Boston University) – LTL control of linear systems

• TALIRO (Arizona State, Colorado, NEC Labs) – Verification using robustness

• TuLiP (CalTech) –Model predictive temporal logic control

10


FY12 University Study Monitoring and Checking

Question 2:

How do we create a Run-Time version of the algorithm that enables

safe switching?

NEED: Run-Time implementation of Algorithm

Reference: Kim, M. and Viswanathan, M. and Kannan, S. and Lee, I. and Sokolsky, Java-MaC: A run-time assurance approach for Java programs, 2004

11


Implement Algorithms at Run Time?

Challenges:

• Multi-core real-time monitoring – Resource constraints?

• What instrumentation of the code is needed?

• What properties need to be proven @ run time

• Acceptable false-positive/false-negative rates

• System integration

• Integration with static formal verification

12


FY12 University Study Bounded and Predictable Overhead

Reference: B. Bonakdarpour and S. Kulkarni, Masking Faults While Providing Bounded-Time Phased Recovery, FM’08

Reference: Pike, Goodloe, Morisset, Niller http://code.galois.com/talk/2010/10-11-pike.ppt RTV Conf 2010

Question 3:

How do we ensure real-time sampling constraints are achieved?

NEED: HW Latency / Stability of Algorithm

13

http://code.galois.com/talk/2010/10-11-pike.ppt






What are the associated

timing constraints?

14

Challenges

• Are there classes of programs/ properties that simplify state estimation between samples?

• Where do Adaptive control programs and properties fall?

• How does time-triggered sampling scale? Program size? Property complexity?

• Multi-core for reduced overhead?

• What about monitoring non-functional properties? E.g. timing?

• Can timing properties of RV implementations be verified? E.g. schedule verification?

• Security issues (i.e. what information flow model for the monitor)?


FY12 University Study Bounded and Predictable Overhead

Question 4:

How can model based design / simulation enable

quicker realization of an end product?

NEED: Modeling / Simulation of RTA Framework

Reference: Karsai, Porter, Hemingway, Sztipanovits, Overview of the Model-Integrated Tool Chain for High Confidence Design, AFOSR MURI 2011

15


Design for Certification and

System Modeling

16

Challenges

• What are new features required for RTA?

• e.g., support for modeling of adaptive software

• Are existing languages (AADL, EsMOL,…etc.) sufficient?

• What scheduling methods/tools need to be supported?

• What test-beds are appropriate for evaluation of RTA technologies at the current stage?


• How do we contain a certifiably provable bound given a class of problems? • How far in advance do boundary "checkers" need to have to ensure safety? • What are the constraints to certify Flight Critical Software? • What criteria determines the class of problems with a given RTA method? • Can boundary protection be proven to enable "black box" software? • Are Autonomy and Performance the same? Which is easier to Bound? • How do we instrument systems for RTA without compromise? • How do we create an RTA algorithm that enables safe switching? • How do we ensure real-time sampling constraints are achieved? • How can use model based design enable quicker RTA realization? • How do we create a process to specify RTA input / output contracts? • How do you establish information integrity and trust? • …

BLACK BOX APPROACH

Framing the Questions to be

answered in RTA

17


Run Time Assurance: Summary

• Problem

– There does not exist a general method for Run Time Assurance

– Previous work: VVIACS, SBIR Phase 1, 2, follow-on, CertA FCS

• General approach was not identified – Extensive offline simulation to construct a very

specific solution

– The technology is not mature enough to present a plausible safety case

• FY12 allowed for refocusing the effort

– FY12 University Study launched to explore state of the art

• Identified research and limitations for implementing a Run Time Assurance algorithm

• Focused on the Simplex Architecture developed by Lui Sha and Bruce Krogh at CMU

• Late FY12 new start aims to tackle the general case

– Create a general framework for adaptive control system certification through

bounding

– Adaptive systems or boundaries cannot be fully tested offline

– Both tools and process are required

18


Questions?

DAVID HOMAN, CHIEF - Control Automation Section

Office: (937) 255-4026

[email protected]

JACOB HINCHMAN, Technical Area Lead

Office: (937) 255-8294

[email protected]

MATT CLARK

Office: (937) 255-8439

[email protected]

JON HOFFMAN

Office: (937) 255-2541

[email protected]

DR. LAURA HUMPHREY

Office: (937) 255-6326

[email protected]

CORY SNYDER

Office: (419) 731-3479

[email protected]

Air Force Research Laboratory

Control Systems Development and Applications Branch

AFRL/RBCCZ 2130 Eighth St.

Wright Patterson AFB, OH 45433-7542

FAX: (937) 656-7505

DR. ALAN BURKHARD

Office: (937) 255-8257

[email protected]

SEAN CALHOUN

Office: (614) 754-1141 //

(937) 255-2425

[email protected]

AARON FIFAREK

Office: (937) 904-8250

[email protected]

BRIAN HULBERT

Office: (937) 255-4605

[email protected]

DR. KULDIP RATTAN

Office: (937) 904-8222

[email protected]

mailto:[email protected]















FY12 TIM Overview

Documents