A Multiple-Shooting Differential Dynamic Programming Algorithm

A Multiple-Shooting Differential Dynamic Programming Algorithm

Etienne PellegriniRyan P. Russell

27th Spaceflight Mechanics Meeting, San Antonio, TX02/06/2017

Summary

• Introduction and Background

• Multiple-Shooting Problem Formulation

• Solving the multiple-shooting problem- Augmented Lagrangian methods- Single-leg expansions- Multi-leg expansions

• Numerical results- Validation of the quadratic expansions and updates- Van Der Pol Oscillator- 2D Spacecraft Orbit Transfer

• Conclusions and future work

2

Introduction and Background

• Modern spacecraft trajectories are increasingly complex (flight times, multiple fly-bys, tour design)

• Need high-fidelity solvers (combined to global search tools for preliminary design)

• Goals:- Improve robustness- Improve computational efficiency

GTOC 1 trajectory, www.esa.int

3

Motivations

• Multi-phase capabilities: - Separation at flybys, interception, rendez-vous, etc… - Each phase can have different dynamics, constraints, etc…

• Multi-shooting framework:- Allows for the decoupling of the legs and phases- Reduces sensitivities and improves robustness- Parallel implementation (solving each leg independently).

4

Hybrid Differential Dynamic Programming

Classic NLP Solvers DDP Methods

5

Summary

• Introduction and Background

• Multiple-Shooting Problem Formulation

• Solving the multiple-shooting problem- Augmented Lagrangian methods- Single-leg expansions- Multi-leg expansions

• Numerical results- Validation of the quadratic expansions and updates- Van Der Pol Oscillator- 2D Spacecraft Orbit Transfer

• Conclusions and future work

6

Problem Statement

7

The Multi-Shooting Subinterval: the Legs

8

Discretization

9

Multi-Shooting Problem Statement

10

Summary

• Introduction and Background

• Multiple-Shooting Problem Formulation

• Solving the multiple-shooting problem- Augmented Lagrangian methods- Single-leg expansions- Multi-leg expansions

• Numerical results- Validation of the quadratic expansions and updates- Van Der Pol Oscillator- 2D Spacecraft Orbit Transfer

• Conclusions and future work

11

• Mixed approach:- Box constraints on the state or controls- “Hard’’ constraints, can not be violated to first order by the

update laws- Accounted for using constrained quadratic programming- In MDDP as it is, using a null-space method in the TR algorithm

- Terminal constraints (intra- and inter-phase) and path constraints

- Accounted for using an augmented Lagrangian technique

Handling the constraints

12

The Augmented Lagrangian Algorithm

• Penalization method for constrained optimization• Transforms a constrained problem into an unconstrained

one by adding a penalty and Lagrange multipliers to the cost function:

13

Single-Leg Problem

14

Single-Leg Quadratic Expansions

15

Multi-leg Quadratic Expansion

16

Multi-Leg Quadratic Expansion

17

Inner Loop of the MS Algorithm

18

Lagrange Multipliers Update

19

The MDDP Algorithm with AL

20

Summary

• Introduction and Background

• Multiple-Shooting Problem Formulation

• Solving the multiple-shooting problem- Augmented Lagrangian methods- Single-leg expansions- Multi-leg expansions

• Numerical results- Validation of the quadratic expansions and updates- Van Der Pol Oscillator- 2D Spacecraft Orbit Transfer

• Conclusions and future work

21

Validation and Numerical Results

• Validation of the quadratic expansions and updates is done using a quadratic problem with linear constraints:

22

Linear Quadratic Problem

Initial States Final States

Controls

23

Van Der Pol Oscillator

24

Classical VDP

Final State Controls

25

Minimum-Time VDP

Final State Controls

26

20-Leg Solution of Min-Time VDP

27

2D Spacecraft Orbit Transfer

28

2DSpacecraft SolutionInitial State

Final State

Controls

29

Single Shooting

30

Multiple-Shooting

31

Conclusions & Future work

• The theoretical developments necessary to the formulation of a multiple-shooting differential dynamic programming algorithm are presented for the first time.

• An algorithm based on augmented Lagrangian methods and multiple-shooting DDP is described and tested, allowing to confirm:- The validity of the quadratic expansions and update equations- The applicability of the multiple-shooting principles to DDP- The resulting reduction in sensitivity for the subproblems

• Future work:- Parallel implementation- Robustness and performance analysis on more complex test problems

32

References (I)• D. Rufer, “Trajectory Optimization by Making Use of the Closed Solution of Constant Thrust-

Acceleration Motion,” Celestial Mechanics, Vol. 14, No. 1, 1976, pp. 91–103.• C. H. Yam, D. Izzo, and F. Biscani, “Towards a High Fidelity Direct Transcription Method for

Optimisation of Low-Thrust Trajectories,” 4th International Conference on Astrodynamics Tools and Techniques, 2010, pp. 1–7.

• G. Lantoine and R. P. Russell, “Complete closed-form solutions of the Stark problem,” Celestial Mechanics and Dynamical Astronomy, Vol. 109, Feb. 2011, pp. 333–366, 10.1007/s10569-010-9331-1.

• R. R. Bate, D. D. Mueller, and J. E. White, Fundamentals of Astrodynamics. New-York, NY. Dover Publications, 1971.

• Lyness, J. N. and Moler, C. B., “Numerical Dierentiation of Analytic Functions," SIAM Journal on Numerical Analysis, Vol. 4, No. 2, 1967, pp. 202-210.

• Squire, W. and Trapp, G., “Using Complex Variables to Estimate Derivatives of Real Functions,” SIAM Review, Vol. 40, No. 1, Jan 1998, pp. 110-112.

• Martins, J. R. R. A., Sturdza, P., and Alonso, J. J., “The Connection Between the Complex-Step Derivative Approximation and Algorithmic Differentiation,” Proceedings of the 39th Aerospace Sciences Meeting and Exhibit, AIAA, Reno, NV, jan 2001.

34

References (II)• Lantoine, G., Russell, R. P., and Dargent, T., “Using Multicomplex Variables for Automatic

Computation of High-Order Derivatives,” ACM Transactions on Mathematical Software, Vol. 38, No. 3, apr 2012, pp. 16:1-16:21.

• G. Lantoine and R. P. Russell, “A Hybrid Differential Dynamic Programming Algorithm for Constrained Optimal Control Problems. Part 1: Theory,” Journal of Optimization Theory and Applications, Vol. 154, apr 2012, doi:10.1007/s10957-012-0039-0.

• Conn, A.R., Gould, N.I.M., Toint, P.L., Trust-Region Methods. MPS - SIAM, Philadelphia, 2000.• Bellman, R., Dynamic Programming, Princeton University Press, Princeton, New Jersey, 1957.• Davidon, W.C., “Variable Metric Method for Minimization”, SIAM Journal on Optimization, Vol. 1,

Issue 1, 1991, pp. 1 - 17.

35