Fahroo - Dynamics and Control - Spring Review 2013

1 DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution 14 February 2013

Integrity Service Excellence

Dr. Fariba Fahroo Program Officer

AFOSR/RTA Air Force Research Laboratory

Dynamics & Control

Date: 4 March 2013

2 DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution

2013 AFOSR SPRING REVIEW

NAME: Fariba Fahroo BRIEF DESCRIPTION OF PORTFOLIO: Developing mathematical theory and algorithms based on the interplay of dynamical systems and control theories with the aim of developing innovative synergistic strategies for the design, analysis, and control of AF systems operating in uncertain, complex, and adversarial environments. LIST SUB-AREAS IN PORTFOLIO:

• General Control Theory: optimal control, adaptive control, stochastic control, hybrid control

• Distributed Multi-Agent Control: path planning, decision making, sensing, task allocation with adversarial and stochastic elements with incomplete information and communication constraints

• Mixed Human-Machine Interface • Control of Distributed Parameter Systems • Emerging Applications: Quantum Control, Vision-based Control


Key Portfolio Research Challenges

Key enabling research areas in the science of autonomy

•Autonomous Dynamic Mission Planning : Hybrid System Formulation (mix of finite, discrete state (decision variables) and continuous Dynamics)

• Human-Machine Interactions and operations: Stochastic Dynamic Programming, Stochastic Control

• Incorporation of Uncertainty in Mission Environment, Model Parameters: Adaptive Control, Stochastic Control

• Adversarial Behavior: Game theoretic Approaches, Stochastic game theory (mean-field theory)

• Incomplete Information: incorporation of learning into games, task allocation, planning

• Computational Issues: Computational Nonlinear Control Theory

• Emergent Applications: Quantum Control, Vision Control


Distributed Control for Networked Systems • Objectives: To address fundamental problems in distributed control of

networked systems with non-traditional constraints and adversarial action.

• Challenges: A comprehensive research program that deals, within a multi-agents context,

– with decentralization, – localization of objectives, – lossy communication and information exchange, – incompleteness of information, – adversarial interference, – multiplicity of objectives, – coordination through multi-level interaction.

• Modeling Framework: deterministic, stochastic and hybrid structures, cast in discrete- as well as continuous-time settings, with a strong mathematical underpinning.

• Multi-Disciplineary Research: Control theory, Information theory, Decision theory, Networks, Coding, Communication, Computing, and Game theory.

Presenter

Presentation Notes

Networked systems have sensors and controllers distributed generally in an ad-hoc manner, which are, however, connected virtually either through communication and information transmission or because of the need to achieve some level of performance driven by individual or common goals, or both. Any effective effort to develop a theoretical foundation for this paradigm necessitates the pooling together of tools (both conceptual and algorithmic) from multiple seemingly disparate disciplines, such as control theory, information theory, coding, communication, computing, and game theory. Research employs a judiciously carved out blend of tools from stochastic control, estimation and filtering, zero-sum dynamic/differential games, dynamic optimization, and information theory.


Multi-Agent Autonomy How (MIT)

• Vision: Team of agents that share experiences and learn to act more intelligently in dynamic, uncertain, & contested environments

– Learn/update models & planners/policies to improve mission performance

• Approach: Intelligent Cooperative Control Architecture (iCCA) that builds/learns effectively from prior knowledge

– Synergistic integration of planning and learning to improve performance

– Bounds risk in learning process

• News: – iCCA with adaptive modeling shown to

out-perform individual methods – Developed new model learning techniques

that greatly reduce sample complexity of iCCA – Performance improvement demonstrated in

experimentally using quadrotors

CBBA

CBBA +

Sarsa

iCCA

Sarsa Sarsa

CBBA conservative

CBBA Aggressive

iCCA iCCA

Adaptive Modeling


Distributed Task Allocation

• Goal: Automate mission planning to improve performance of networked team of heterogeneous agents operating in a contested environment

– Distributed/decentralized planning is required for some communication environments (e.g. slow, high cost, unreliable, ...).

– Planning issues: convergence guarantees/time, mission performance • Approach: Consensus-Based Bundle Algorithm (CBBA) developed as multi-

assignment task allocation algorithm – Recent results highlighted the performance limitations associated with the score

functions that must be used to guarantee convergence • News: Developed bid-warping approach for CBBA

– Expands the set of score functions that can be used

– Can use cost functions that make the most sense

– Provable bound on performance

Presenter

Presentation Notes


Time-Critical Cooperative Missions for Multi-Agent Systems

Naira Hovakimyan (UIUC)

Time-critical applications for multiple vehicles with spatial constraints: • Sequential auto-landing (UAVs) • Coordinated reconnaissance missions • Simultaneous arrival at multiple locations

How to: 1) Efficiently generate trajectories that satisfy desired relative timing constraints? 2) Enforce these constraints with desired performance levels in communication-

limited or communication-deprived environments?

Courtesy: NPS Courtesy: LARSyS, IST


Coordination in Communication-Limited or -Deprived Environments

Connectivity of the communications network satisfies: • Graph connected in an integral

sense, not piecewise in time

Rate of convergence of the coordination dynamics proportional to the connectivity level:

Low connectivity implies slow convergence

Potential directions for research: • Use of onboard estimators:

can improve learning of other agents’ dynamics…

but leads to larger networks (characterized by smaller convergence rates).

• Topology control is needed to decide which estimators to listen to and which not:

‘Edge snapping’ based on link quality and coordination error.

How to modify coordination control algorithms to improve convergence rate?

Presenter

Presentation Notes


Autonomous Optimal Pursuit David Casbeer, Krishna Kalyanam, Phil Chandler, AFRL/RQ

Meir Pachter, AFIT, Swaroop Darbha, TAMU • Problem Formulation: Develop practical and

autonomous decision-making algorithms under uncertainty

• UGS monitors a road network, while a UAV(s) continually visits the UGS for status updates

• When an UGS alerts a UAV of an intruder, the UAV must decide where to go to capture/image the intruder and deliver imagery to operator • Challenging Problem: Stochastic optimization with dual-control aspects

including partial and delayed observations • Very few problems of this type have been addressed • Solution does not require complete information history (implementable) • Developed probabilistic guarantees for autonomous capture in stochastic setting • Transition: Flight test and demonstration


Flight Test (Transition)

– Onboard computation and UAV-to-UGS communication have been designed, tested, and characterized

• An estimation scheme that predicts the location of the intruder given a set of UGS measurements has been integrated with human interface software developed by AFRL/RH

• Flight test algorithm (Spring ‘13) and demonstrate capability at a joint US/AUS military exercise; Summer 2013

• Three flight tests have validated hardware and software architecture to implement intruder capture algorithms

Presenter

Presentation Notes


NETWORKED GUIDANCE AND CONTROL FOR MOBILE MULTI-AGENT SYSTEMS Nuno C. Martins, Univ of Maryland

Objective: Tractable methods to design Markov Decision Processes (MDPs) that

maximize coverage subject to operational constraints


NETWORKED GUIDANCE AND CONTROL FOR MOBILE MULTI-AGENT SYSTEMS

Nuno C. Martins Potential AF impact:

- Characterization of the largest configuration set that can be persistently surveyed by a team of autonomous vehicles

- Determine the minimum number of vehicles required to survey a certain area, subject to safety and performance constraints.

Possible Future directions:

- Extension to continuous state-space.

- Include output feedback.

Scientific impact:

- Defined a new optimization paradigm for MDPs, based on maximum entropy methods.

locations and directions that can be persistently visited

forbidden locations


2010 MURI (PI: Tamer Başar, UIUC)

Multi-Layer and Multi-Resolution Networks of Interacting Agents in Adversarial Environments

Vision and Overarching Goal: To address fundamental issues that arise in networked heterogeneous agents, among which are: complex interactions; uncertainty and adversarial actions; trust, learning; humans-in-the-loop; information and communication; and design of architectures to facilitate generation and transmission of actionable information for performance improvement under different equilibrium solutions. MLMR Games: Games played at different levels, interacting through their outcomes, action spaces, and costs “Games within Games”: Zooming in and out providing game structures with different levels of granularity

14

Multi-Layer and Multi-Resolution Hybrid Stochastic Differential Games in the Large Population Regime and Their Equilibria

• Large-scale interactions happen at different layers of a complex system. Players have inner interactions with members in the group as well as long-range interactions with other groups.

• Coupled decisions = Game theory

• – Players/Actors/Agents

• – Actions/Strategies/Choices

• – Preferences over joint choices (utility functions)

• – Solution concept (e.g., Nash equilibrium)

• Approach: A multi-layer, multi-resolution (MLMR) game framework

DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution

15

Large Population Game Model

• The problem is computationally quite challenging. For the case of large populations, the idea of mean-field representation is used.

• Key Intuition from Statistical Mechanics: Extremely complex large population particle systems may have simple continuum limits with distinctive bulk properties.

• When the number of players is large, the cost of a player does not explicitly depend on the actions of the other players but rather coupled through the distribution of the agent states m.

• Characterization of the Nash equilibrium for this case is a significant result.

• The mean-field equilibrium is obtained by solving coupled HJB backward equations together with the generalized FPK equation

DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution


MURI ‘07: Decision Dynamics in Mixed Teams of Humans and Robots

PI: John Baillieul (BU)

Toward a Control theory of Planning, Perception, and Reaction

• Human decision making – • the effects of peer and other social pressure

• Modeling spatial and topological knowledge and defining knowledge acquisition metrics for mixed teams of humans and automata

• How perceptual dynamics affect human decision makers

• Elements of a formal theory perception-based control


Challenges: An encompassing theoretical basis for joint cognitive systems did not exist. • Need to understand how human decisions differ from those of ideal decisions makers due to: biases, fear of disapproval, and other forms of group pressure. (Psychology addresses such questions.) • Paradigm shift: humans operating on parity with automatons. Approaches and Results: •Design of reward-incentivized games for play by individual subjects and by groups of subjects, with various forms of information sharing. •Trade-offs between exploitation of strategies with known payoff and exploration of alternative strategies in the hope of higher payoff. • Shared information about reward sizes tended to encourage exploration of strategy alternatives in games that were studied. •When people play with awareness of others playing simultaneously, competitiveness emerges. • Knowledge of others’ reward affects play style more than knowledge of others’ play strategies A deeper understanding of decision making in mixed human/robot

teams can be pursued through research on information acquisition, sharing and propagation in complex networks.

Decision Making in Mixed Human/Robot Teams


Task Partitioning in Mixed Human/Robot Teams

•Challenges: •Understand task partitioning between machines and humans in search and reconnaissance. • Use stochastic control principles to develop automated task partitioning and adaptive resource assignment algorithms for teams of agents with finite resources. • Determine advantages/disadvantages of alternative interaction structures between human/automata. •Results: •Automated task partitioning and scheduling algorithms that incorporate risk of vehicle loss; formulated as a stochastic combinatorial problem on graphs. • Dynamic Search/Classify Problems: Agents’ actions generate noisy observations, processed through Bayesian techniques, that are then used by agents in selecting future actions. • Normative solution of such problems requires full stochastic dynamic programming; constraint relaxation allows scalable solution

A fundamental distinction between humans and robots in mixed teams is that robots react rapidly to real-time sensor data, whereas humans react to

more complex perceptions of the environment in which cognitive processes and prior experience play a role.

:


Computational Control Theory McEneaney (UCSD) ,Dower (Uni-Melbourne)

• Challenges: Develop new theory and computational tools to solve previously intractable optimal control and dynamic game problems.

• Approach: 1) Develop and exploit new and existing theory connecting idempotent algebra and dynamic programming to identify or construct fast solution propagators for specific classes of optimal control and dynamic game problem.

• 2) Develop new computational tools to implement these fast solution propagators to solve practical problems.

• The N-body problem stands as a benchmark in astrodynamics. • The focus lies in the (harder) two-point boundary value N-body

problem: – Given a combination of initial and some terminal data, determine the

N-body trajectory meeting these conditions. • Applications: Asteroid danger estimation, Efficient space vehicle trajectory design,

Concept generalizable to other conservative systems and potentials.


Least Action and Game-Based Gravitational Potential Representation

• Max-plus based fundamental solutions for Riccati ODE and PDE are immediately transferable to least-action approach to two-point boundary values problems (TPBVPs) with quadratic potentials (e.g., mass-spring, wave equation dynamics).

• Least-Action Principle: Conservative system evolves so as to minimize (more correctly find stationary point) of the action (kinetic minus potential energy).

• Re-interpretation of the gravitational potential energy. Consider two bodies separated by distance, r. – Traditional potential energy representation: – Competing controller representation:

– Note quadratic form for potential as function of position –

computationally very useful.


Fundamental Solutions => Extremely Rapid Evaluations

• Fundamental solution given as a set of solutions of Riccati equations, S(t). (Least action is obtained as max of linear function over convex set S(t).)

• Having a fundamental solution allows for very rapid solution of large numbers of TPBVPs (fixed set of masses and duration).

• Suppose we are given all N initial positions, x, and N terminal velocities, vf . Want initial velocities (and trajectories).

• Repeat ad nauseum for variety of x, vf .

S(t) depicted as layers indexed by γ, two-body case.

(=initial velocities)


Summary

• Dynamics and Control is the key enabler in the science of autonomy. There have been fruitful collaborations with other disciplines such ad cognitive science, network science, communication, information science.

– Plan is to initiate new MURI and AFOSR Basic Research Initiative (BRI) topics in V&V, Human-Machine interaction, Autonomy in contested environments

• Close collaboration and sharing of information with other agencies: ONR,

ARO and NSF

• New application areas have benefited from control theoretic approaches, and control theory has grown and changed because of these applications=> Examples: Synthetic biology (new BRI topic), Quantum Control (plan for a future BRI)

• Support of fundamental research in Control and Dynamics while considering challenging and relevant application areas is what distinguishes this portfolio.

Fahroo - Dynamics and Control - Spring Review 2013

Technology

adaptive control

hybrid control

vision control

quantum control

optimal control

control theories

distribution statement

general control theory