COSTATE ESTIMATION FOR OPTIMAL CONTROL PROBLEMS USING ORTHOGONALCOLLOCATION AT GAUSSIAN QUADRATURE POINTS By CAMILA CLEMENTE FRANC ¸ OLIN A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2013
197
Embed
COSTATE ESTIMATION FOR OPTIMAL CONTROL PROBLEMS …ufdcimages.uflib.ufl.edu/UF/E0/04/58/08/00001/FRANCOLIN_C.pdf · costate estimation for optimal control problems using orthogonal
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
COSTATE ESTIMATION FOR OPTIMAL CONTROL PROBLEMS USINGORTHOGONAL COLLOCATION AT GAUSSIAN QUADRATURE POINTS
By
CAMILA CLEMENTE FRANCOLIN
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOLOF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHY
2.5 Orthogonal Collocation for the Solution of Optimal Control Problems . . . 592.5.1 Global Collocation at LG Points . . . . . . . . . . . . . . . . . . . . 612.5.2 Global Collocation at LGR Points . . . . . . . . . . . . . . . . . . . 632.5.3 Global Collocation at Flipped LGR Points . . . . . . . . . . . . . . 652.5.4 Variable-Order Collocation at LG Points . . . . . . . . . . . . . . . 662.5.5 Variable-Order Collocation at LGR Points . . . . . . . . . . . . . . 682.5.6 Variable-Order Collocation at Flipped LGR Points . . . . . . . . . . 70
3 COSTATE ESTIMATION USING THE INTEGRAL FORMULATION . . . . . . . 72
3.1 Continuous-Time Bolza Optimal Control Problem . . . . . . . . . . . . . . 733.1.1 Differential and Integral Forms of Optimal Control Problem . . . . . 743.1.2 First-Order Optimality Conditions of Differential and Integral Forms 75
6
3.2 Costate Estimation Using Integral Legendre-Gauss Collocation . . . . . . 763.2.1 Differential Form of LG Collocation . . . . . . . . . . . . . . . . . . 763.2.2 KKT Conditions Using Differential LG Collocation . . . . . . . . . . 783.2.3 Integral Form of LG Collocation . . . . . . . . . . . . . . . . . . . . 803.2.4 KKT Conditions Using Integral LG Collocation . . . . . . . . . . . . 823.2.5 A Relationship Between Integral and Differential Costate Estimates 85
3.3 Costate Estimation Using Integral Legendre-Gauss-Radau Collocation . . 863.3.1 Differential Form of LGR Collocation . . . . . . . . . . . . . . . . . 873.3.2 KKT Conditions Using Differential LGR Collocation . . . . . . . . . 883.3.3 Integral Form of LGR Collocation . . . . . . . . . . . . . . . . . . . 913.3.4 KKT Conditions Using Integral LGR Collocation . . . . . . . . . . . 933.3.5 A Relationship Between Integral and Differential Costate Estimates 97
2-6 Error associated with function approximation using uniform, LG, and LGR points 57
2-7 Approximation of integral using Trapezoid rule. . . . . . . . . . . . . . . . . . . 58
2-8 Error in approximation of integral using Trapezoid rule as a function of N . . . . 58
2-9 Error in approximation of integral using Gaussian quadrature as a function of N 60
2-10 Distribution of LG points for global collocation . . . . . . . . . . . . . . . . . . . 62
2-11 Distribution of LGR points for global collocation . . . . . . . . . . . . . . . . . . 64
2-12 Distribution of flipped LGR points for global collocation . . . . . . . . . . . . . . 65
2-13 Distribution of LG points for variable-order collocation . . . . . . . . . . . . . . 67
2-14 Distribution of LGR points for variable-order collocation . . . . . . . . . . . . . 69
2-15 Distribution of flipped LGR points for variable-order collocation . . . . . . . . . 71
4-1 Relationship Between the Direct and Indirect Methods . . . . . . . . . . . . . . 115
5-1 Equivalence of the Direct and Indirect Methods . . . . . . . . . . . . . . . . . . 138
6-1 Primal solution for Example 1 obtained using integral collocation at LG points. . 141
6-2 Costate solutions for Example 1 obtained using collocation at LG points. . . . . 142
6-3 Costate errors for Example 1 obtained using collocation at LG points. . . . . . 143
6-4 Primal solution for Example 1 obtained using integral collocation at LGR points. 145
6-5 Costate solutions for Example 1 obtained using collocation at LGR points. . . . 146
6-6 Costate errors for Example 1 obtained using collocation at LGR points. . . . . 146
6-7 State and control for Example 2 obtained using integral LG collocation. . . . . 148
6-8 Costate solutions for Example 2 obtained using collocation at LG points. . . . . 149
10
6-9 Costate errors for Example 2 obtained using collocation at LG points. . . . . . 150
6-10 State and control for Example 2 obtained using integral LGR. . . . . . . . . . . 152
6-11 Costate solutions for Example 2 obtained using collocation at LGR points. . . . 153
6-12 Costate errors for Example 2 obtained using collocation at LGR points. . . . . 153
6-13 Primal solution for Example 3 obtained using collocation at LG points. . . . . . 155
6-14 Errors in state and control for Example 3 obtained using LG collocation. . . . . 156
6-15 Costate estimate as derived by Ref. [1] for Example 3. . . . . . . . . . . . . . . 158
6-16 Costate errors for estimate derived in Ref.[1] for Example 3. . . . . . . . . . . . 159
6-17 Dual variables for Example 3 obtained using collocation at LG points. . . . . . 161
6-18 Costate errors for Example 3 obtained using collocation at LG points. . . . . . 161
6-19 Primal solution for Example 3 obtained using collocation at LGR points. . . . . 163
6-20 Errors for Example 3 obtained using collocation at LGR points. . . . . . . . . . 164
6-21 Costate Estimate as derived by Ref. [1] for Example 3. . . . . . . . . . . . . . . 165
6-22 Costate errors for estimate derived in Ref.[1] for Example 3. . . . . . . . . . . . 166
6-23 Costate estimate for Example 3 obtained using collocation at LGR points. . . . 167
6-24 Costate errors for Example 3 obtained using collocation at LGR points. . . . . 167
6-25 Primal solution for Example 4 obtained using LG collocation. . . . . . . . . . . 170
6-26 State and control errors for Example 4 using collocation at LG points. . . . . . 171
6-27 Costate Estimate as derived by Ref. [1] for Example 4. . . . . . . . . . . . . . . 173
6-28 Costate errors for estimate derived in Ref.[1] for Example 4. . . . . . . . . . . . 174
6-29 Costate Estimate for Example 4 obtained using collocation at LG points. . . . . 176
6-30 Costate errors for Example 4 obtained using collocation at LG points. . . . . . 177
6-31 Primal solution for Example 4 obtained using LGR collocation. . . . . . . . . . 179
6-32 State and control errors for Example 4 using collocation at LGR points. . . . . 180
6-33 Costate estimate as derived by Ref. [1] for Example 4. . . . . . . . . . . . . . . 182
6-34 Costate Errors using estimate derived in Ref.[1] for Example 4. . . . . . . . . . 183
6-35 Costate Estimate for Example 4 obtained using collocation at LGR points. . . . 185
11
6-36 Costate errors for Example 4 obtained using collocation at LGR points. . . . . 186
12
Abstract of Dissertation Presented to the Graduate Schoolof the University of Florida in Partial Fulfillment of theRequirements for the Degree of Doctor of Philosophy
COSTATE ESTIMATION FOR OPTIMAL CONTROL PROBLEMS USINGORTHOGONAL COLLOCATION AT GAUSSIAN QUADRATURE POINTS
By
Camila Clemente Francolin
August 2013
Chair: Anil V. RaoMajor: Aerospace Engineering
Computing the costate in an optimal control problem is important for verifying
the optimality of the solution and performing sensitivity analysis. This dissertation is
concerned with the problem of estimating the costate in an optimal control problem
using orthogonal collocation at Legendre-Gauss (LG) and Legendre-Gauss-Radau
(LGR) points. First, methods are presented for estimating the costate using orthogonal
collocation at the LG or LGR points when the dynamic constraints of the optimal control
problem are formulated in integral form. A new continuous-time dual variable called
the integral costate is introduced, where the integral costate is the Lagrange multiplier
of the integral dynamic constraint. The first-order optimality conditions of the integral
form of the optimal control problem are derived in terms of the integral costate. The
integral form of the optimal control problem is then discretized using the integral LG
and LGR collocation methods and relationship between the discrete form of the integral
costate and the costate of the original differential optimal control problem are developed.
It is shown that the LGR integration matrix that relates the differential costate to the
integral costate is singular while the corresponding LG integration matrix is full rank. The
approach developed in this research then provides a way to estimate the costate of the
original optimal control problem using the Lagrange multipliers of the integral form of the
LG and LGR collocation methods. Furthermore, the costate estimates presented in this
research result in a set of Karuhn-Kush-Tucker conditions of the nonlinear programming
13
problem which are a discrete approximation of the first-order optimality conditions of the
continuous-time optimal control problem both in differential and integral forms.
The second part of this research focuses on state inequality path constrained
optimal control problems. Problems with active state-inequality path constraints are
difficult to solve due to the high-index differential-algebraic equations (DAE) that result
from the constraint activity. This DAE index fluctuation in the solution domain results in
possible discontinuities in the dual variables which are hard to approximate numerically.
Due to these discontinuities, previous costate estimates for direct transcription methods
using collocation at LG or LGR points resulted in a transformed adjoint system
which was not a discrete approximation to the first-order optimality conditions in the
presence of state inequality path constraints. In this research a different set of costate
estimates are developed which result in a transformed adjoint system that is a discrete
approximation of the first-order optimality conditions of the continuous-time optimal
control problem. Specifically, a costate estimate using the method of indirect adjoining
with continuous multipliers is derived. The equivalence between the first-order optimality
conditions of the finite-dimensional nonlinear program and the first-order optimality
conditions of the continuous-time optimal control problem ensures convergence of the
discrete problem to a local minimum which satisfies the optimality conditions of the
original problem. This costate estimate can thus be used to verify the extremality of the
approximated solution.
14
CHAPTER 1INTRODUCTION
Many problems in engineering, economics, and biology can be modeled as
differential-algebraic systems. In addition, it is often desired to optimize the performance
of such systems. The goal of an optimal control problem is to determine the state and
control that optimize a given performance index subject to a set of differential-algebraic
constraints. In aerospace engineering, optimal control applications include trajectory
optimization, parameter estimation, and vehicle guidance. As alluded to earlier, the
constraints in an optimal control problem include differential equations that describe the
motion of the dynamical system, path constraints that define limits on the process, and
event constraints that define way points that must be met during the motion.
Optimal control problems that involve inequality path constraints are common in
aerospace engineering. Such constraints can be purely a function of the control (for
example, control limits such as maximum allowable thrust), purely a function of the
state (for example, no-fly zone constraints), or more generally a function of both the
control and the state (for example, maximum heating rate constraints). Quoting Ref. [2],
“Solving an optimal control or estimation problem is not easy”. Optimal control problems
with inequality path constraints are particularly challenging to solve because the optimal
trajectory may contain regions where the inequality constraint is active. Even more
challenging are problems with inequality path constraints that are purely a function of
the state, leading to high-index differential-algebraic equation (DAE) constraints [3–6].
Systems with state inequality path constraints of index one or less can generally be
solved numerically using numerical integrators. Systems with state inequality path
constraints of index greater than one, however, pose computational challenges for
numerical integration methods [3]. In the context of an optimal control problem, a state
inequality path constrained high-index differential-algebraic system have a non-smooth
state and possibly a discontinuous costate, while a control inequality constrained
15
problem can have a discontinuous optimal control [7, 8]. Such discontinuities can be
difficult to approximate accurately using numerical methods.
Methods for approximating solutions to optimal control problems fall into two broad
categories: indirect and direct methods. In an indirect method the first-order optimality
conditions are derived using the calculus of variations, resulting in a Hamiltonian
boundary-value problem (HBVP) [9]. In the case when the inequality path constraints
are inactive on the optimal solution, the HBVP is a two-point boundary value problem.
When the solution domain contains active/inactive switches in state inequality path
constraint activity, however, the HBVP will have interior-point constraints, resulting in a
multi-point boundary value problem [7].
A great deal of research has been done on solving optimal control problems with
state inequality path constraints using indirect methods [8, 10–13]. This research has
yielded a number of different ways to derive the necessary conditions for optimality,
each resulting in a different set of conditions. In the method of direct adjoining, the state
inequality path constraint is augmented to the Hamiltonian, and the first-order optimality
conditions are derived using the calculus of variations. This method results on a set
of “jump conditions” on the optimal costate which must be applied at the entrance and
exit of the constrained arc. In the aerospace engineering literature, state inequality
constraints have historically been handled through index-reduction of the high-index
differential-algebraic equation (DAE) system that results from the state constraint activity
[2]. The necessary conditions for optimality are derived from the calculus of variations
using an approach termed indirect adjoining in which the state inequality constraint is
differentiated before being adjoined to the Lagrangian [7]. Using this approach, Ref. [10]
develops a set of tangency conditions that are enforced at the entrance of a constrained
arc, often leading to discontinuities in the costate. The control along the constrained
arc is then defined by setting to zero the lowest derivative of the inequality constraint
that is an explicit function of the control variable. The costate discontinuities that arise
16
from the necessary conditions for optimality then become a function of the tangency
conditions. In Ref. [13] a modified problem is posed where the original path constraint
is augmented to the cost functional and the tangency conditions are applied at both
the entrance and exit of the constraint activity. The formulation of Ref. [13] leads to a
reduction in the dimension of the state space in the region of active constraint activity. In
[14] a numerical technique for dealing with these problems is developed using steepest
descent.
Another technique for solving state inequality path constrained optimal control
problems is the method of indirect adjoining approach with continuous multipliers
[15]. In this method, the discontinuity in the costate is “subtracted out,” leading to
a set of optimality conditions that yield a continuous costate even if if the solution
lies on a constrained arc [16–19]. Reference [15] summarizes the methods of direct
adjoining, indirect adjoining, and indirect adjoining with continuous multipliers used
in the derivation of the necessary conditions for optimality of a state inequality path
constrained optimal control problem.
Indirect methods are attractive because the solution of the HBVP is an extremal
and thus must satisfy the first-order optimality conditions from the calculus of variations.
Consequently, a solution obtained using an indirect method can be accurate. The
HBVP, however, generally does not have an analytic solution. Therefore, numerical
methods must be employed. Common numerical approaches for solving the HBVP are
shooting, multiple shooting, and collocation [20]. Numerical implementations of Indirect
methods pose a number of computational challenges. First, the first-order optimality
conditions are often difficult to derive. Second, the radius of convergence of the resulting
Hamiltonian boundary value problem can be notably small due to instabilities in the
Hamiltonian dynamics [21]. As a result, an indirect method often requires a good initial
guess for both the state and the costate [2, 7, 9]. However, providing an initial guess for
the costate is often difficult because the costate has no physical interpretation. Finally, in
17
the case when the optimal solution has constrained and unconstrained arcs, it becomes
necessary to estimate the constrained arc sequence [2]. Estimating switches in path
constraint activity is often difficult when no a-priori knowledge of the solution structure is
available.
The second class of numerical methods in optimal control are direct methods.
Different from indirect methods, direct methods parametrize the control and/or the state,
and the continuous-time problem is discretized and transcribed into a finite-dimensional
nonlinear programming problem (NLP). The resulting NLP can then be solved using
well developed optimization software [22–25]. Direct methods have gained a great deal
of popularity as they avoid a number of the pitfalls associated with indirect methods.
Specifically, because a direct method directly transcribes the optimal control problem
into a NLP, the lengthy derivations of the first-order optimality conditions are avoided.
Also, direct methods do not require an initial guess for the costate, and the problem
can be modified relatively easily without having to re-derive the optimality conditions
[2, 26, 27]. Many direct methods, however, are not as accurate as indirect methods and
they require further analysis to verify optimality once a solution is achieved.
Direct methods can employ either a sequential or a simultaneous optimization
approach. In a sequential approach the control is parametrized and the dynamics
are integrated over the trajectory domain. One example of a sequential optimization
method is the direct shooting method [28–30]. In a direct shooting method the
control is parametrized and the dynamics are integrated using numerical integration
methods. Direct shooting methods are useful when the control can be parametrized
using few parameters, keeping the problem size small. As the number of variables
needed to parametrize the control increases, however, convergence to a solution
using direct shooting methods becomes difficult. Direct multiple-shooting methods
improve convergence by subdividing the solution domain into multiple intervals [28]. The
shooting method is then applied in each interval, and continuity of the state is enforced
18
at the interval boundaries. Multiple-shooting methods have better convergence than
shooting methods because the integration of the state dynamics is done over shorter
intervals. Both direct shooting and direct multiple shooting methods, however, are not
computationally efficient due to the sequential numerical integration technique used to
integrate the dynamics. Furthermore, convergence still depends on a-priori knowledge
of the constrained and unconstrained arc sequence.
A particular direct methods known as a collocation method, employ a simultaneous
optimization approach [2, 30–36]. Collocation methods parametrize both the control and
the state, and the differential-algebraic equations are enforced at a set of discrete points
in the domain [2, 26, 37]. Direct collocation methods are attractive because they require
no a priori knowledge of the solution structure [38]. Furthermore, direct collocation
methods are less sensitive to the initial guess than the sequential approach of shooting
methods [2]. Well-known software implementations of direct collocation methods include
SOCS, DIDO, DIRCOL, and GPOPS [39–42].
Direct collocation methods can employ local h-method collocation, or global
p-method collocation. Often the class of Runge-Kutta methods is used to collocate
and integrate the system dynamics [2, 33, 43–45]. Runge-Kutta methods are usually
employed as h-methods in which the solution domain is subdivided into many intervals
and a fixed low-degree approximation is used in each interval. This type of scheme
is computationally efficient as it has a sparse structure that can be exploited by NLP
solvers [46]. Convergence of the numerical discretization using h-methods is then
achieved by increasing the number of intervals in the domain. Due to the polynomial
convergence rate of this kind of scheme, however, h-methods can lead to extremely
large NLP’s [45, 47, 48].
In contrast to local h-methods, global p-methods use a single polynomial to
approximate the state over the entire domain [26, 27, 49]. Convergence in a p-method
is then obtained by increasing the degree of the approximating polynomial. A p-method
19
has the advantage that it converges exponentially for problems for problems whose
solutions are smooth. In the case when the solution is not smooth (as often happens in
the presence of active inequality constraints) the convergence rate is significantly slower.
Furthermore, the NLP arising from a p-method is less sparse than the NLP arising from
an h-method.
This research will employ an hp-method using collocation at the LG and LGR points
[50, 51]. In an hp−method, or variable-order method, the solution domain is divided
into a mesh, and the degree of the approximating polynomial (that is, the number of LG
or LGR collocation points) in each interval is allowed to vary. Using an hp-method it is
possible to divide the problem into intervals such that the solution in each interval is
smooth. Thus convergence is achieved by increasing the degree of the approximating
polynomial in each interval. In this manner it is possible to achieve a high accuracy
solution solution while keeping the NLP smaller than what might be possible using an
h-method.
Over the last decade, one class of direct collocation methods which has risen
to prominence in the numerical solution of optimal control problems is the class of
(A) Approximation of the function given by Eq. (2–102) using 11Legendre-Gauss points.
00
0.1
0.2
0.2
0.3
0.4
0.4
0.5
0.6
0.6
0.7
0.8
0.8
0.9
1
1-0.8 -0.6 -0.4 -0.2-1τ
y(τ)
y(τ)
Y (τ)
(B) Approximation of the function given by Eq. (2–102) using 41Legendre-Gauss points.
Figure 2-4. Approximation of the function given by Eq. (2–102) using 11 and 41Legendre-Gauss points.
55
1.2
0
0
0.2
0.2
0.4
0.4
0.6
0.6
0.8
0.8
1
1-0.8 -0.6 -0.4-0.2
-0.2-1τ
y(τ)
y(τ)
Y (τ)
(A) Approximation of the function given by Eq. (2–102) using 11Legendre-Gauss-Radau points.
00
0.1
0.2
0.2
0.3
0.4
0.4
0.5
0.6
0.6
0.7
0.8
0.8
0.9
1
1-0.8 -0.6 -0.4 -0.2-1τ
y(τ)
y(τ)
Y (τ)
(B) Approximation of the function given by Eq. (2–102) using 41Legendre-Gauss-Radau points.
Figure 2-5. Approximation of the function given by Eq. (2–102) using 11 and 41Legendre-Gauss-Radau points.
56
0
0 30 40 50 60 70 80 90 100
-5
10
10-10
20
20
15
5
N
Ey
Uniform
LG
LGR
Figure 2-6. Base ten logarithm of infinity norm error as a function of number of supportpoints, N , for approximating the function given by Eq. (2–102).
where (t0, ... , tN) are the subinterval boundaries, or grid points, at which the function is being
evaluated. In order to demonstrate the use of the composite trapezoid rule, consider approximat-
ing the integral of the function f (τ) in the domain τ ∈ [−1,+1]:
f (τ) =
∫ +1
−1exp(τ) dτ . (2–106)
Figure (2-7) shows a graphical representation of the trapezoid rule approximation using three
intervals. Furthermore, Fig. (2-8) shows the base ten logarithm error of this integration as a
function of the base ten logarithm number of approximating intervals. It can be seen that the
convergence of this method is slow as 105 subintervals are necessary to reach an error of
O(10−10).
2.4.2.2 Gaussian quadrature
In contrast with low-order integrators such as the composite trapezoid rule, a Gaussian
quadrature is a high accuracy integrator which displays exponential convergence when approxi-
mating the integral of smooth functions. Gaussian quadrature rules approximate the integral of a
57
00 0.2 0.4
0.5
0.6 0.8
1
1-0.8 -0.6 -0.4 -0.2-1
2
3
2.5
1.5
f (τ)
f(τ)
Approx.
τ
Figure 2-7. Approximation of the integral of the function given by Eq. (2–106) using a 4interval Trapezoid rule.
−4
2 4 4.5 5
-5
32.5 3.5
-6
-7
-8
-9
-10
-11
log10
Err
or
log10 Number of Approximating Intervals
Figure 2-8. Base ten logarithm error of the integral of the function given by Eq. (2–106)as a function of the base ten logarithm number of approximating intervals.
58
function by evaluating the expression
∫ tf
t0
f (t) dt ≈N∑
i=1
wi f (τi), (2–107)
where wi are the quadrature weights associated with the set of points chosen to approximate the
integration. The three sets of points defined by Gaussian quadrature are the Legendre-Gauss-
Lobatto (LGL) points, the Legendre-Gauss-Radau (LGR) points, and the Legendre-Gauss (LG)
points. The LGL, LGR, and LG quadrature rules are exact for polynomials of degree at most
2N − 3, 2N − 2, and 2N − 1, respectively. In this research the LG and LGR points are used.
The N LG points are the roots of the Nth degree Legendre polynomial PN(τ), and the
corresponding LG quadrature weights are given as
wi =2
1− τ2i[PN(τi)]2
, (i = 1, ... ,N).
Similarly, the N LGR points are computed from the roots of PN(τ) + PN−1(τ), and the corre-
sponding LGR quadrature weights are given as
w1 =2
N2,
wi =1
(1− τi)[(PN−1(τi)]2, (i = 2, ... ,N).
Finally, the flipped LGR points and weights are simply the negative of the LGR points.
The accuracy of the LG and LGR quadrature methods can be seen from the function
f (τ) = exp(τ) in the domain τ ∈ [−1,+1]. Figure 2-9 shows the base ten log of the error
for the approximation of the integral in Eq. (2–106) as a function of N. It can be seen that the
convergence rate using Gaussian quadrature is exponential. Furthermore, it is seen that N = 6
LG or LGR points results in an error of less than O(10−10). For comparison, this same error
of O(10−10) required 105 subintervals using the composite Trapezoid rule. Thus, the benefits
of using Gaussian quadrature over low-order integrators when approximating the integral of a
smooth function is evident.
2.5 Orthogonal Collocation for the Solution of Optimal Control Problems
It is now possible to combine the concepts described in Section 2.4 in order to develop a
method to approximate the solution of the continuous-time optimal control problem of Section
59
6 7 8 9 10-16
-14
-4
-2
4 53
-6
-8
-10
-12
N
LG Points
LGR Points
log10
Err
or
Figure 2-9. Base ten logarithm of the error in the integration of the function given byEq. (2–106) as a function of number of LG and LGR Points Used.
2.1. In this section the methods of orthogonal collocation at both Legendre-Gauss (LG) and
Legendre-Gauss-Radau (LGR) points are described. Both these methods for approximating
solutions to optimal control problems can be used as either global collocation methods or
variable-order collocation methods. Global collocation methods use one single polynomial
approximation to collocate the differential-algebraic equations over the entire domain. Global
collocation at LG and LGR points is advantageous when solving problems whose solutions are
smooth, because the LG and LGR methods converge exponentially. When the solution is not
smooth, however, the convergence rate is significantly lower. In the case where the optimal
solution lies on a constrained arc for a subset of the solution domain, non-smooth features in the
solution state and/or control can occur. For such problems, it will thus be beneficial to employ
variable-order LG or LGR collocation. In a variable-order collocation scheme the solution domain
is divided into a mesh, and the degree of the approximating polynomial (that is, the number of
LG or LGR collocation points) in each mesh interval is allowed to vary. This method is useful
because it allows for capturing non-smoothness in the solution domain at interval boundaries.
Because both the LG and the LGR set of points are defined on the domain τ ∈ [−1,+1], the
following affine transformation will be used to map the time domain t ∈ [t0, tf ] to τ ∈ [−1,+1]
60
when using global collocation:
t =tf − t02
τ +tf + t02. (2–108)
Furthermore, it is noted that
d t
dτ=tf − t02
≡ h2, h = tf − t0.
When using variable-order collocation, the time domain t ∈ [t0, tf ] is divided into a mesh
consisting of K mesh intervals where the mesh points are t0 = T0 < T1 < · · · < TK−1 < TK =
tf , and the corresponding mesh intervals are [Tk−1,Tk ], (k = 1, ... ,K). Therefore each mesh
interval can be mapped to the domain τ ∈ [−1,+1] through the affine transformation
t =tk − tk−12
τ +tk + tk−12
, (k = 1, ... ,K).
It is also noted thatd t
dτ=tk − tk−12
≡ h(k)
2, h(k) = tk − tk−1.
The following notation and conventions will be used in the discussion that ensues. First, all
vector functions of time are denoted as row vectors, that is, if y(τ) ∈ Rn is a vector function of the
scalar variable τ , then y(τ) = [y1(τ), · · · , yn(τ)]. Next, any capital boldface character, Y, denotes
a matrix of size M × n, where each row of Yi corresponds to the evaluation of a function y(τ) at
a particular value τ = τi . Next, the notation Yi:j denotes rows i through j of the matrix Y, except
when referring to a differentiation matrix D, in which case Di refers to the i th column of D. Finally,
D⊤ denotes the transpose of matrix D, and D⊤i denotes the transpose of the i th column of D.
2.5.1 Global Collocation at Legendre-Gauss Points
The method for approximating solutions to optimal control problems using global orthogonal
collocation at Legendre-Gauss (LG) points is now described [27]. The LG points are defined
in the domain (−1,+1) which does not include either of the endpoints. However, the method
derived here for collocation at the LG points still approximates, but does not collocate, the state
at both endpoints τ0 = −1 and τN+1 = +1. Figure 2-10 shows the LG collocation points as well
as the endpoints at which the state is approximated but not collocated for various values of N.
61
0 0.2 0.4 0.6 0.8 1-0.8 -0.6 -0.4 -0.2-13
4
5
6
7
8
9
τ
Discretization Points
Collocation Points
Num
ber
ofLG
Poi
nts,N
Figure 2-10. Distribution of Legendre-Gauss discretization and collocation points in thedomain τ ∈ [−1,+1].
The state is then approximated using a Lagrange polynomial with support points at the N
LG points plus the noncollocated point τ0 = −1, such that
y(τ) ≈ Y(τ) =N∑
i=0
YiLi(τ), (2–109)
where the Lagrange polynomials Li(τ) are defined as
Li(τ) =
N∏
j=0
j 6=i
τ − τj
τi − τj; (i = 0, ... ,N). (2–110)
The state approximation is then differentiated at τ = τj , (j = 1, ... ,N) as
Y(τj) ≈N∑
i=0
Yi Li(τj) = [DY0:N ]j , (2–111)
where Dij = Li (τj) , (i = 1, ... ,N, j = 0, ... ,N) are the components of the N × (N + 1)
Legendre-Gauss (LG) differentiation matrix.
62
Next, the cost functional of Eq. (2–1) is approximated with a Gaussian quadrature. The
finite-dimensional transcription of the continuous-time optimal control problem of Eqs. (2–1)–
(2–4) then becomes to minimize the cost function
J = Φ(YN+1) +h
2
N∑
j=1
wjg(Yj ,Uj) (2–112)
subject to the algebraic constraints
DY0:N =h
2f(Y1:N,U1:N), (2–113)
YN+1 = Y0 +h
2
N∑
j=1
wj f(Yj ,Uj), (2–114)
φ(Y0) = 0, (2–115)
C(Y1:N ,U1:N) ≤ 0, (2–116)
where w = (w1, ... ,wN) are the LG quadrature weights. It is noted for LG collocation that
Eq. (3–24) provides an LG quadrature approximation of the state, YN+1, at the final noncollo-
cated point τN+1 = +1.
2.5.2 Global Collocation at Legendre-Gauss-Radau Points
The method for approximating solutions to optimal control problems using global orthogonal
collocation at Legendre-Gauss-Radau (LGR) points is now described [64]. The LGR points are
defined in the domain [−1,+1) such that τ1 = −1 is a LGR collocation point but τN+1 = +1 is
a noncollocated point. However, the method derived here for collocation at the LGR points still
approximates, but does not collocate, the state at the terminal point τN+1 = +1. Figure 2-11
shows the LGR collocation points as well as the noncollocated terminal point for various values
of N.
The state is then approximated using a Lagrange polynomial with support points at the N
LGR points plus the noncollocated point τN+1 = +1 such that
y(τ) ≈ Y(τ) =N+1∑
i=1
YiLi(τ), (2–117)
63
0 0.2 0.4 0.6 0.8 1-0.8 -0.6 -0.4 -0.2-13
4
5
6
7
8
9
τ
Discretization Points
Collocation Points
Num
ber
ofLG
RP
oint
s,N
Figure 2-11. Distribution of Legendre-Gauss discretization and collocation points in thedomain τ ∈ [−1,+1].
where the Lagrange polynomials Li(τ) are defined as
Li(τ) =
N+1∏
j=1
j 6=i
τ − τj
τi − τj; (i = 1, ... ,N + 1). (2–118)
The state approximation is then differentiated at τ = τj , (j = 1, ... ,N) as
Y(τj) ≈N+1∑
i=1
Yi Li(τj) = [DY1:N+1]j , (2–119)
where Dij = Li (τj) , (i = 1, ... ,N, j = 1, ... ,N + 1) are the components of the N × (N + 1)
Legendre-Gauss-Radau (LGR) differentiation matrix. Next, the cost functional of Eq. (2–1) is
approximated by an LGR quadrature. The finite-dimensional approximation of the continuous-
time optimal control problem of Eqs. (2–1)–(2–4) is then given as follows. Minimize the cost
function
J = Φ(YN+1) +h
2
N∑
j=1
wjg(Yj ,Uj) (2–120)
64
0 0.2 0.4 0.6 0.8 1-0.8 -0.6 -0.4 -0.2-13
4
5
6
7
8
9
τ
Discretization Points
Collocation Points
Num
ber
ofF
lippe
dLG
RP
oint
s,N
Figure 2-12. Distribution of Legendre-Gauss discretization and collocation points in thedomain τ ∈ [−1,+1].
subject to the algebraic constraints
DY1:N+1 =h
2f(Y1:N ,U1:N), (2–121)
φ(Y1) = 0, (2–122)
C(Y1:N ,U1:N) ≤ 0, (2–123)
where w = (w1, ... ,wN) are the LGR quadrature weights.
2.5.3 Global Collocation at Flipped Legendre-Gauss-Radau Points
The method for approximating solutions to optimal control problems using global orthogonal
collocation at the flipped Legendre-Gauss-Radau (LGR) points is now described [64]. The LGR
points are defined in the domain (−1,+1] such that τN = +1 is a LGR collocation point but
τ0 = −1 is a noncollocated point. However, the method derived here for collocation at the LGR
points still approximates, but does not collocate, the state at the initial point τ0 = −1. Figure
(2-12) shows the flipped LGR collocation points as well as the initial point at which the state is
approximated but not collocated for various values of N.
65
The state is then approximated using a Lagrange polynomial with support points at the N
LGR points plus the noncollocated point τ0 = −1, such that
y(τ) ≈ Y(τ) =N∑
i=0
YiLi(τ), (2–124)
where the Lagrange polynomials Li(τ) are defined as
Li(τ) =
N∏
j=0
j 6=i
τ − τj
τi − τj; (i = 0, ... ,N). (2–125)
The state approximation is then differentiated at τ = τj , (j = 1, ... ,N) as
Y(τj) ≈N∑
i=0
Yi Li(τj) = [DY0:N ]j , (2–126)
where Dij = Li (τj) , (i = 1, ... ,N, j = 0, ... ,N) are the components of the N × (N + 1)
Next, the cost functional of Eq. (2–1) is approximated by an LGR quadrature. The finite-
dimensional approximation of the continuous-time optimal control problem of Eqs. (2–1)–(2–4) is
then given as follows. Minimize the cost function
J = Φ(YN) +h
2
N∑
j=1
wjg(Yj ,Uj) (2–127)
subject to the algebraic constraints
DY0:N =h
2f(Y1:N ,U1:N), (2–128)
φ(Y0) = 0, (2–129)
C(Y1:N ,U1:N) ≤ 0, (2–130)
where w = (w1, ... ,wN) are the flipped LGR quadrature weights.
2.5.4 Variable-Order Collocation at Legendre-Gauss Points
The method for approximating solutions to optimal control problems using variable-order
collocation at the Legendre-Gauss (LG) points is now described. When implementing the
variable-order LG method, a single variable is used for the value of the state at the end of mesh
interval k and the start of mesh interval k + 1, that is, Y(k)Nk+1
≡ Y(k+1)0 , 1 ≤ k ≤ K − 1 such that
66
3
4
5
6
7
8
9
tT0 T1 T2 T3
Discretization Points
Collocation PointsNum
ber
ofLG
poin
tsP
erIn
terv
al,Nk
Figure 2-13. Distribution of multiple-interval Legendre-Gauss discretization andcollocation points for various values of Nk . The domain [t0, tf ] is split into K=3 intervalssuch that t0 = T0 and tf = T3.
continuity in the state is enforced. Hence, redundant variables defining the state at the interior
mesh points are eliminated. Figure 2-13 shows the LG collocation points as well as the mesh
points at which the state is approximated but not collocated for when K = 3 and for various
values of N.
In multiple-interval LG collocation, the state is approximated in each mesh interval k as
y(k)(τ) ≈ Y(k)(τ) =Nk∑
i=0
Y(k)i L
(k)i (τ), L
(k)i (τ) =
Nk∏
j=0
i 6=j
τ − τ(k)j
τi − τ(k)j
, (2–131)
Differentiating Y(k)(τ) in Eq. (2–131) with respect to τ , yields
Y(k)(τj) ≈Nk∑
i=0
Y(k)iL(k)i(τj) = [D
(k)Y(k)0:Nk]j , (2–132)
where D(k)ij= L
(k)i(τ)j , (i = 1, ... ,Nk , j = 0, ... ,Nk) are the components of the Nk × (Nk + 1)
Legendre-Gauss (LG) differentiation matrix in the kth mesh interval.
Next, the cost functional of Eq. (2–1) is approximated using a multiple-interval LG quadra-
ture. The finite-dimensional approximation of the continuous-time optimal control problem of
67
Eqs. (2–1)–(2–4) is then given as follows. Minimize the cost function
where w(k) = (w (k)1 , ... ,w(k)N k) are the LG quadrature weights in interval k . It is noted for LG
collocation that Eq. (2–135) provides an LG quadrature approximation, Y(k)0 , of the state at the
final noncollocated point τ (k)N+1 = +1 in interval (k = 1, ... ,K − 1), while Eq. (2–136) provides an
LG quadrature approximation, Y(K)N+1, of the state at the final noncollocated point of the domain,
tf = τ(k)N+1 = +1.
2.5.5 Variable-Order Collocation at Legendre-Gauss-Radau Points
The method for approximating solutions to optimal control problems using variable-order
collocation at the Legendre-Gauss-Radau (LGR) points is now described. When implementing
the variable-order LGR method, a single variable is used for the value of the state at the end of
mesh interval k and the start of mesh interval k + 1, that is, Y(k)Nk+1
≡ Y(k+1)1 , 1 ≤ k ≤ K − 1
such that continuity in the state is enforced. Hence, redundant variables defining the state at the
interior mesh points are eliminated. It is noted that the LGR points are particularly conducive
to this type of collocation; because only one of the domain endpoints is collocated, there is no
“double collocation” at the boundaries. Also, the only noncollocated point is the last point of the
final interval, tf = τ(K)NK+1
= +1. Figure (2-14) shows the LGR collocation points as well as the
terminal point at which the state is approximated but not collocated for when N = 3 and for
various values of K .
68
3
4
5
6
7
8
9
tT0 T1 T2 T3
Discretization Points
Collocation PointsNum
ber
ofLG
RP
oint
sP
erIn
terv
al,Nk
Figure 2-14. Distribution of multiple-interval Legendre-Gauss-Radau discretization andcollocation points for various values of Nk . The domain [t0, tf ] is split into K=3 intervalssuch that t0 = T0 and tf = T3.
In multiple-interval LGR collocation, the state is approximated in each mesh interval k as
y(k)(τ) ≈ Y(k)(τ) =Nk+1∑
i=1
Y(k)iL(k)i(τ), L
(k)i(τ) =
Nk+1∏
j=1
i 6=j
τ − τ(k)j
τi − τ(k)j
, (2–139)
Differentiating Y(k)(τ) in Eq. (2–139) with respect to τ , yields
Y(k)(τj) ≈Nk+1∑
i=1
Y(k)i L
(k)i (τj) = [D
(k)Y(k)1:Nk+1
]j , (2–140)
where D(k)ij = L(k)i (τ)j , (i = 1, ... ,Nk , j = 1, ... ,Nk + 1) are the components of the Nk × (Nk + 1)
Legendre-Gauss-Radau (LGR) differentiation matrix in the kth mesh interval.
Next, the cost functional of Eq. (2–1) is approximated using a multiple-interval LG quadra-
ture. The finite-dimensional approximation of the continuous-time optimal control problem of
Eqs. (2–1)–(2–4) is then given as follows. Minimize the cost function
where w(k) = (w (k)1 , ... ,w(k)N k) are the LGR quadrature weights in interval k .
2.5.6 Variable-Order Collocation at Flipped Legendre-Gauss-Radau Points
The method for approximating solutions to optimal control problems using variable-order
collocation at the flipped Legendre-Gauss-Radau (LGR) points is now described. When
implementing the flipped variable-order LGR method, a single variable is used for the value
of the state at the end of mesh interval k and the start of mesh interval k + 1, that is, Y(k)Nk ≡
Y(k+1)0 , 1 ≤ k ≤ K − 1 such that continuity in the state is enforced. Hence, redundant
variables defining the state at the interior mesh points are eliminated. It is noted that the flipped
LGR points are particularly conducive to this type of collocation; since only one of the domain
endpoints are collocated, there is no “double collocation” at the boundaries. Also, the only
noncollocated point is the first point of the first interval, t0 = τ(1)0 = −1. Figure (2-15) shows the
flipped LGR collocation points as well as the initial point at which the state is approximated but
not collocated for when N = 3 and for various values of K .
In multiple-interval flipped LGR collocation, the state is approximated in each mesh interval
k as
y(k)(τ) ≈ Y(k)(τ) =Nk∑
i=0
Y(k)i L
(k)i (τ), L
(k)i (τ) =
Nk∏
j=0
i 6=j
τ − τ(k)j
τi − τ(k)j
, (2–145)
Differentiating Y(k)(τ) in Eq. (4–46) with respect to τ , yields
Y(k)(τj) ≈Nk∑
i=0
Y(k)i L
(k)i (τj) = [D
(k)Y(k)0:Nk]j , (2–146)
where D(k)ij = L(k)i (τ)j , (i = 1, ... ,Nk , j = 0, ... ,Nk) are the components of the Nk × (Nk + 1)
flipped Legendre-Gauss-Radau (LGR) differentiation matrix in the kth mesh interval.
70
3
4
5
6
7
8
9
t
T0 T1 T2 T3
Discretization Points
Collocation Points
Num
ber
ofF
lippe
dLG
RP
oint
sP
erIn
terv
al,Nk
Figure 2-15. Distribution of multiple-interval Flipped Legendre-Gauss-Radaudiscretization and collocation points for various values of Nk . The domain [t0, tf ] is splitinto K=3 intervals such that t0 = T0 and tf = T3.
Next, the cost functional of Eq. (2–1) is approximated using a multiple-interval LG quadra-
ture. The finite-dimensional approximation of the continuous-time optimal control problem of
Eqs. (2–1)–(2–4) is then given as follows. Minimize the cost function
It is seen that Eq. (4–76) is a Legendre-Gauss-Radau quadrature of the costate
dynamics across interval K . Consequently, the right-hand side of Eq. (4–76) approximates
the costate at the final point in the domain. Eq. (4–76) is thus a subtle way of enforcing
the relationship λ(K)N = ∇YΦ(YNk) and it is expected that the last term of Eq. (4–75)
will be small while the remaining terms in Eq. (4–75) are a collocation collocation
scheme for the continuous adjoint equation in the final interval K . Similarly, the
112
right-hand side of Eq. (4–74) approximates the costate, λ(k)N , at the terminal point
in interval k via a Legendre-Gauss-Radau quadrature of the costate dynamics for
(k = 1, ... ,K − 1). Equation (4–74) is therefore a subtle way of enforcing the relationship
λ(k)N = λ
(k+1)0 , (k = 1, ... ,K − 1). If the costate is continuous across an interval
boundary, the relationship λ(k)N = λ(k+1)0 holds true and the last term of Eq. (4–73)
will be small while the remaining terms in Eq. (4–73) are a collocation scheme for the
continuous adjoint equation. Therefore in cases when the costate is continuous across
an interval boundary, the transformed optimality conditions of Eqs. (4–71)–(4–78) are
a discrete form of the first-order optimality conditions [given by Eqs. (4–7)–(4–12)] of
the continuous-time optimal control problem. However, it has previously been shown in
Section 2.3 that the presence of active state inequality path constraints in the solution
domain may cause discontinuities in the costate. Therefore, in the presence of active
state inequality path constraints in the solution domain, λ(k)N 6= λ(k+1)0 at the entrance
or exit of a constrained arc, and the last term of Eq. (4–73) will not be small. Therefore
Eq. (4–73) will not be a collocation scheme for the continuous adjoint equation using this
costate estimate.
4.4 Discussion
In this chapter a method first derived by Ref. [1] for obtaining costate estimates
from the KKT multipliers of the NLP was presented. This derivation showed that if
the costate is continuous, variable-order collocation at the LG and LGR points yields
a set of transformed optimality conditions of the KKT system which are a discrete
representation of the continuous-time first-order necessary conditions of the optimal
control problem, as can be seen in Fig. (4-1). If the costate is discontinuous, however,
variable-order collocation at the LG and LGR points yields a set of transformed
optimality conditions of the KKT system which are an inexact discrete representation
of the continuous-time first-order necessary conditions. This result was first shown by
113
Ref. [74], who suggested that the costate estimate must be modified in order to account
for the costate discontinuities:
“. . . high-accuracy approximations are achieved by the proposed
hp-method if the costate is continuous. If mesh points are at the location
of discontinuity in the costate, the transformed adjoint system is an inexact
discrete representation of the continuous-time first-order necessary
conditions. For a dynamic refinement algorithm that will exactly locate
the switch in activity of inequality path constraints, it is likely that the costate
may be discontinuous at mesh points. It is necessary to determine how
to make the transformed adjoint system a discrete representation of the
continuous-time first-order necessary conditions for a discontinuous costate
solution if mesh points are at the location of discontinuity in the costate”.
It was previously shown in Section 2.3 that discontinuities in the costate stem from
inequality path constraint activity in the solution domain. Therefore in the presence
of state inequality constraints, the costate becomes discontinuous, and the first-order
optimality conditions of the NLP are not a discrete approximation of the first-order
optimality conditions of the continuous-time problem. Therefore, in this research a new
method of costate estimation for variable-order collocation at LG and LGR points will be
derived using three different methods. Specifically, a costate estimate using the method
indirect adjoining with continuous multipliers will be presented. It will be shown that
this method for costate estimation using variable-order collocation at the LG and LGR
points leads to a transformed adjoint system which is a discrete representation of the
continuous-time first-order necessary conditions even in the presence of state inequality
path constraints.
114
Figure 4-1. Relationship between the direct and indirect methods for solving an optimalcontrol problem. In the indirect method, the problem is first optimized through thecalculus of variations, leading to a set of conditions which can then be discretized andsolved. In the direct method, the problem is first discretized and transcribed to an NLP,then it is optimized by solving the KKT system. The two systems are equivalent onlywhen the costate is continuous in the solution domain.
115
CHAPTER 5COSTATE ESTIMATION FOR STATE CONSTRAINED PROBLEMS
As was shown in Chapter 4, previous research has successfully derived a
high-accuracy estimate of the costate using collocation at the Legendre-Gauss and
Legendre-Gauss-Radau points for the case of a problem with no active state inequality
path constraints. However, Ref. [1] showed that in the case when the costate is
discontinuous (as is the case in the presence of active state inequality path constraints),
this previously derived costate estimate leads to a set of first-order optimality conditions
of the NLP that are not equivalent to the discrete form of the variational optimality
conditions. This non-equivalence leads to an inaccurate approximation of the costate. In
order to rectify this inacuracy, in this chapter a method for estimating the costate of state
inequality path constrained optimal control problems using collocation at LG and LGR
points is developed using the method of indirect adjoining with continuous multipliers.
The method of indirect adjoining with continuous multipliers was chosen to
develop a costate estimate over the direct and indirect adjoining methods for two
reasons. First, the method of indirect adjoining requires a modification of the original
problem formulation through index-reduction of the differential-algebraic equations. The
reformulation of the problem must be done analytically, and requires prior knowledge
of the solution structure. Thus, when using an automated solution process (such as
a mesh refinement technique), this procedure might be cumbersome to implement.
Second, both the methods of indirect and direct adjoining result in a discontinuous
costate. Because discontinuities are difficult to approximate numerically, both these
methods may yield large errors in the costate estimate if the location of the discontinuity
is not exact. Thus, because the method of indirect adjoining with continuous multipliers
yields a continuous costate, it offers an advantage over the methods of direct and
indirect adjoining which approximate a discontinuous costate.
116
Similar to the approach of Chapter 4, the following notation and conventions will
be used throughout this chapter to make the exposition more clear. First, all vector
functions of time are denoted as row vectors, that is, if y(τ) ∈ Rn is a vector function of
the scalar variable τ , then y(τ) = [y1(τ), · · · , yn(τ)]. Next, any capital boldface character,
Y, denotes a matrix of size M × n, where each row of Yi corresponds to the evaluation
of a function y(τ) at a particular value τ = τi . Next, the notation Yi:j denotes rows
i through j of the matrix Y, except when referring to a differentiation matrix D or the
integration matrix A, in which case Di and Ai refers to the i th column of D and A. Finally,
D⊤ denotes the transpose of matrix D, and D⊤i denotes the transpose of the i th column
of D. Given vectors x and y ∈ Rn, the notation 〈x, y〉 is used to denote the standard inner
product between x and y. Furthermore, if f : Rn −→ Rm, then ∇f is the m by n Jacobian
matrix whose i th row is ∇fi . In particular, the gradient of a scalar-valued function is a row
vector. If φ : Rm×n −→ R and Y is an m by n matrix, then ∇φ denotes the m by n matrix
whose (i , j) element is (∇φ(Y))ij = ∂φ(Y)/∂Yij .
The remainder of this chapter is organized as follows. First, Section 5.1 formulates
the continuous-time state inequality path constrained optimal control problem and
states the first order optimality conditions of the continuous problem. Next, in Sections
5.2 and 5.3 a new costate estimate is derived using variable-order collocation at the
Legendre-Gauss and flipped Legendre-Gauss-Radau points, respectively, through the
method of indirect adjoining with continuous multipliers. It is shown for each of these
derived costate estimates that the transformed first-order optimality conditions of the
NLP are a discrete form of the first-order optimality conditions of the continuous-time
optimal control problem. Finally, in Section 5.4 the derived costate estimates are
discussed, and conclusions are given.
5.1 Continuous-Time State Inequality Path Constrained Optimal Control Problem
The state inequality path constrained optimal control problem to be studied in
the remainder of this chapter is now presented. To simplify comparisons with the
117
transformed adjoint system, the domain t ∈ [t0, tf ] = I is divided into K intervals
If p(k)i were the continuous costate evaluated at τi in interval k , then by the continuous
adjoint equation, the sum in Eq. (5–102) approximates the integral of p(K) between −1
and +1. Eq. (5–102) amounts to an approximation to the relation
p(k)Nk= p
(k+1)0 . (5–103)
This condition shows that the costate will be continuous across mesh interval boundaries.
Consequently, it is expected that the eN term in Eq. (5–96) should be small, while the
remaining terms in Eq. (5–96) amount to a collocation scheme for the continuous adjoint
equation.
The connection between the transformed optimality conditions and the original
continuous optimality conditions is quite subtle. For example, the nonnegativity
conditions for the derivative of the state multiplier ν(k) and the complementary slackness
conditions in Eq. (5–12) are embedded in a very unusual way in the discrete optimality
conditions. As pointed out in Eq. (5–78), if the discrete multiplier νk)1:N associated with
the state constraint is interpolated by a polynomial ν(τ) of degree N − 1, then the
nonnegativity conditions in Eq. (5–98) only ensure nonnegativity of the polynomial
135
derivative at τ1 through τN−1. At τN , the discrete positivity condition amounts to
ν(K)(τN)−ν(K)(τN)
w(K)N
≥ 0, (5–104)
ν(K)(τN) +−ν(k)(τN) + ν(k+1)(τ0)
w(k)N
≥ 0, (k = 1, ... ,K). (5–105)
To illustrate how these conditions work, suppose that the state constraint is inactive over
the entire final interval [−1,+1] for the discrete problem. That is, S(Y(K)i ) < 0 for all i . In
this case, complementary slackness implies that
ν(K)(τi) = 0 for 1 ≤ i ≤ N − 1. (5–106)
Because the derivative of a polynomial of degree N − 1 is N − 2, the N − 1 conditions
Eq. (5–106) imply that the derivative is identically zero. Hence, ν(K)(τN) = 0 in
Eq. (5–104), and it is concluded that ν(K)(τN) = ν(+1) ≤ 0. Finally, from the
complementary slackness condition,
0 = S(Y(K)N )
T(ν(K)(τN)− ν(K)(τN)/wN)
= −S(Y(K)N )Tν(K)(τN)/wN ,
which implies that ν(K)(τN) = ν(K)N = 0 when S(Y(K)N ) < 0. Hence, the continuous
optimality conditions ν(K)(+1) ≤ 0 and 〈ν(K)(+1),S(y(K)(+1))〉 = 0 are satisfied
in the discrete problem. Furthermore, if complementary slackness holds in interval
K , then ν(K) is a non-decreasing function in interval K , and ν(K)0 ≥ ν(K−1)N , thus the
second term in Eq. (5–105) will be greater than zero. A similar argument can be made
over each interval, such that the condition ν(k) ≥ 0 is satisfied for (k = 1, ... ,K).
Thus, it has been shown that the conditions of the transformed adjoint system given
by Eqs. (5–93)–(5–99) are a discrete form of the first-order optimality conditions of the
continuous-time optimal control problem given by Eqs. (5–7)–(5–12).
The transformed adjoint system for variable-order collocation at the LGR points is
complex. For instance, the differentiation matrix associated with the state dynamics, D,
136
is a N × (N + 1) full-rank differentiation matrix associated with the space of polynomials
of degree at most N . Conversely, the differentiation matrix associated with the costate
dynamics, D† is a rank-defficient N × N differentiation matrix associated with the space
of polynomials of degree at most N − 1. Finally, the matrix associated with the state
constraint multiplier, D, is a full-rank N × N matrix that differentiates and evaluates the
terminal condition simultaneously.
5.4 Discussion
In this chapter costate estimates were derived for estimating the costate of state
inequality path constrained optimal control problems using orthogonal collocation
at the Legendre-Gauss and the flipped Legengre-Gauss-Radau Points. These
conditions result in a continuous approximation to the costate even in the presence
of state inequality path constraints. Furthermore, the costate estimate derived here
reduces to the costate estimate given by Ref. [1], presented in Chapter 4, when no
state inequality path constraints are present in the optimal control problem. Finally,
It was shown that the costate estimate using the method of indirect adjoining with
continuous multipliers resulted in a transformed adjoint system that is a discrete form of
the first-order optimality conditions of the continuous-time problem. Fig. (5-1) illustrates
the equivalence between the transformed adjoint system derived from the NLP and the
first-order optimality conditions of the continuous-time problem derived from the calculus
of variations.
137
Figure 5-1. Relationship between the direct and indirect methods for solving an optimalcontrol problem. In the indirect method, the problem is first optimized through thecalculus of variations, leading to a set of conditions which can then be discretizedand solved. In the direct method, the problem is first discretized and transcribed toan NLP, then it is optimized by solving the KKT system. The two systems are shown tobe equivalent even in the presence of a discontinuous costate.
138
CHAPTER 6EXAMPLES
In this chapter, four examples are studied using the methods developed in Chapters
3 and 5. The first two examples demonstrate the effectiveness of the costate estimation
methods derived in Chapter 3 using the integral form of LG and LGR collocation.
The first example is a single state nonlinear Mayer optimal control problem while
the second example is a single state nonlinear Lagrange optimal control problem.
Next, two state inequality path constrained optimal control problems are solved using
variable-order LG and LGR collocation as described by Chapter 5. The first state
inequality constrained example contains a first-order state inequality path constraint,
while the second state inequality constrained example contains a second-order state
inequality path constraint. The LG and LGR costate estimates derived in Ref. [1]
are shown to produce inaccurate estimates of the dual variables for both examples,
while the LG and LGR costate estimates using the method of indirect adjoining with
continuous multipliers (as described in Chapter 5) are shown to produce accurate
approximations of the dual variables.
Three main observations are made from the examples solved in this chapter. First,
it is shown that variable-order collocation at the LG and LGR points produces accurate
approximations to state inequality path constrained optimal control problems. Because
collocation at the LG points does not provide an approximation to the optimal control at
any of the mesh points whereas collocation at the LGR points provides an approximation
of the optimal control at all interior mesh points, collocation at the LGR method is found
to be the preferred method of solution. Second, it is shown that for state inequality path
constraints of at most order two, it is not necessary to reformulate the optimal control
problem by reducing the index of the DAE in order to obtain an accurate approximation.
Index-reduction requires an analytic reformulation of the optimal control problem which
may be cumbersome, if not impossible, to implement when using mesh refinement.
139
Third, because the method of indirect adjoining with continuous multipliers produces
a costate estimate that is continuous even in the presence of state inequality path
constraint, highly accurate estimates of the costate can be obtained even when using
low-order polynomial approximations in the state (that is, a small number of collocation
points).
6.1 Example 1: Mayer Optimal Control Problem
The first example considered is a nonlinear one-dimensional Mayer optimal control
where a(t) = 1 + 3 exp(5t/2), and b = exp(−5) + 6 + 9 exp(5). This example was solved
using the integral LG and LGR collocation methods using the NLP solver SNOPT [22],
where SNOPT was implemented using optimality and feasibility tolerances of 1 × 10−8
and 2 × 10−8, respectively. The initial guess used for the state and control was a linear
interpolation from the initial state value to zero. For collocation at either set of points,
the integral costate, py(τ), was estimated from the KKT multipliers of the NLP, and
the differential costate, λy(τ), was subsequently computed from the integral costate
approximation.
6.1.1 Solution Using Collocation at Legendre-Gauss Points
Example 1 was solved using integral collocation at LG points as described in
Chapter 3. Figure 6-1 shows the state and control approximation obtained using N = 20
140
00
0.1
0.2
0.2
0.3
0.4
0.4
0.5
0.6
0.6
0.7
0.8
0.8
0.9
1.2 1.4 1.6 1.8
1
1 2t
Sta
te
y∗(t)
y(t)
(A) State.
00
0.1
0.2
0.2
0.3
0.4
0.4
0.5
0.6 0.8 1.2 1.4 1.6 1.8
0.05
0.15
0.25
0.35
0.45
1 2t
Con
trol
u∗(t)
u(t)
(B) Control.
Figure 6-1. Primal solution for Example 1 obtained using integral collocation at LGpoints.
LG collocation points. It is seen that integral collocation at the LG points provides a
highly accurate approximation to the optimal solution.
Next, the integral costate, py (τ), was computed at the LG points using Eq. (3–62),
and the differential costate, λy(τ), was estimated at the LG points plus the noncollocated
endpoints τ0 and τN+1 using the results of Section 3.2.5. Figure 6-2 shows both the
141
Cos
tate
-0.5
0
0
0.5
0.5
1.5
1.5
2.5
-1
1
1
2
2t
p∗y
λ∗
y
py
λy
Figure 6-2. Integral and differential costate solutions for Example 1 obtained using LGcollocation.
integral and the differential costates obtained using N = 20 LG collocation points.
It is seen that the costate estimate is indistinguishable from the optimal costate.
Furthermore, Fig. 6-3 shows the base ten logarithm of the L∞-norm error for the
integral and differential costates when approximated using (N = 2, 4, 6, ... , 20) LG
collocation points. It is interesting to note that the differential costate estimate converges
exponentially as a function of N and reaches an accuracy of O(10−12) for N = 20,
whereas the integral costate estimate achieves an accuracy of approximately O(10−11)
for N = 20.
142
0
0
-8
-6
-4
-2
4 8 12-12 16 20
-10
N
log10
Infin
ity-N
orm
Err
or
∥
∥λy − λ∗y
∥
∥
∞
∥
∥py − p∗y∥
∥
∞
Figure 6-3. Integral and differential costate errors for Example 1 obtained using LGcollocation.
143
6.1.2 Solution Using Collocation at Legendre-Gauss-Radau Points
Next, Example 1 was solved using integral LGR collocation as described in Chapter
3. Figure 6-4 shows the state and control approximation obtained using N = 20
LGR collocation points. It is seen that, similar to collocation at the LG points, integral
collocation at the LGR points provides highly accurate approximation to the optimal
solution. However, unlike collocation at the LG points, collocation at the LGR points
provides an approximation of the control at the terminal boundary point, making
collocation at the LGR points more desirable than collocation at the LG points.
Next, the integral costate, py (τ), was computed at the LGR points using Eq. (3–116),
and the differential costate, λy(τ), was estimated at the LGR points plus the noncollocated
endpoint τN+1 using the results of Section 3.3.5. Note that the value py(τ1) (where
τ1 = −1 for the integral LGR collocation method) was found by extrapolating the
Lagrange interpolating polynomial as described by Eqs. (3–118) and (3–119). Figure
6-5 shows both the integral and the differential costates obtained using N = 20
LGR collocation points. It is seen that the costate estimate is indistinguishable from
the optimal solution. Furthermore, Fig. 6-6 shows the base ten logarithm of the
L∞-norm error for the integral and differential costates when approximated using
(N = 2, 4, 6, ... , 20) LGR collocation points. It is seen that the differential and integral
costate estimates converges exponentially as a function of N until the error reaches
approximately O(10−10) for N = 20.
144
00
0.1
0.2
0.2
0.3
0.4
0.4
0.5
0.6
0.6
0.7
0.8
0.8
0.9
1.2 1.4 1.6 1.8
1
1 2t
Sta
te
y∗(t)
y(t)
(A) State.
00
0.1
0.2
0.2
0.3
0.4
0.4
0.5
0.6 0.8 1.2 1.4 1.6 1.8
0.05
0.15
0.25
0.35
0.45
1 2t
Con
trol
u∗(t)
u(t)
(B) Control.
Figure 6-4. Primal solution for Example 1 obtained using integral collocation at LGRpoints.
145
Cos
tate
-0.5
0
0
0.5
0.5
1.5
1.5
2.5
-1
-1.5
1
1
2
2t
p∗y
λ∗
y
py
λy
Figure 6-5. Integral and differential costate solutions for Example 1 obtained using LGRcollocation.
0
0
-8
-6
-4
-2
4 8 12 16 20
-10
-12
N
log10
Infin
ity-N
orm
Err
or
∥
∥λy − λ∗y
∥
∥
∞
∥
∥py − p∗y∥
∥
∞
Figure 6-6. Integral and differential costate errors for Example 1 obtained using LGRcollocation.
146
6.2 Example 2: Lagrange Optimal Control Problem
This second example considered is a nonlinear one-dimensional Lagrange optimal
control problem given as follows.
Minimize J = 12
∫ tf
0
(log2 y + u2)dt subject to
y = y log y + yu,
y(0) = 5,
y(tf ) = 3.
The optimal solution to this example is given as
y ∗(t) = exp(x∗(t)),
λ∗y(t) = λ∗
x(t)/y∗(t),
p∗y (t) = − λ∗x(t)y
∗(t)− λ∗x(t)y
∗(t)
(y ∗(t))2,
(6–1)
where
x∗(t) = c1 exp(−t√2) + c2 exp(t
√2),
λ∗x(t) = c1(1 +
√2) exp(−t
√2) + c2(1−
√2) exp(t
√2),
(6–2)
and
c1
c2
=
1 1
exp(−tf√2) exp(tf
√2)
log y0
log yf
. (6–3)
The example was solved using the integral LG and LGR collocation methods using the
NLP solver SNOPT [22], where SNOPT was implemented using optimality and feasibility
tolerances of 1 × 10−8 and 2 × 10−8, respectively, with the exact state and control
evaluated at the discretization points as the initial guess. For collocation at either set of
points, the integral costate, py (t), was estimated from the KKT multipliers of the NLP,
and the differential costate, λy(t), was subsequently computed from the integral costate
approximation.
147
3.5
3.5
4.5
4.5
5.5
0 0.5
1.5
1.5
2.5
2.51 1
2
2
3
3
4
4
5
5t
Sta
te
y∗(t)
y(t)
(A) State.
-3.5
-2.5
-1.5
3.5 4.5
-0.5
0
0
0.5
0.5 1.5 2.5-4
-3
-2
-1
1 2 3 4 5t
Con
trol
u∗(t)
u(t)
(B) Control.
Figure 6-7. State and control for Example 2 obtained using integral LG collocation.
6.2.1 Solution Using Collocation at Legendre-Gauss Points
Example 2 was solved using integral collocation at LG points, as described in
Chapter 3. Figure 6-7 shows the primal solution (that is, the state and control) obtained
using N = 32 LG collocation points. It is seen that integral collocation at the LG points
provides highly accurate approximation to the optimal solution.
148
-0.8
-0.6
-0.4
-0.2
0
0
0.2
0.4
0.6
0.8
1
1 2 3 4 5t
Cos
tate
p∗y
λ∗
y
py
λy
Figure 6-8. Integral and differential costate for Example 2 obtained using LG collocation.
Next, the integral costate, py (τ), was estimated at the LG points using Eq. (3–62),
and the differential costate, λy(τ), was estimated at the LG points plus the noncollocated
endpoints τ0 and τN+1 using the results of Section 3.2.5. Figure 6-8 shows both the
integral and the differential costates obtained using N = 32 LG collocation points. It is
seen that both the differential and integral costate estimates are indistinguishable from
the optimal costates. Figure 6-9 shows the base ten logarithm of the L∞-norm error for
the integral and differential costates for (N = 4, 8, 12, ... , 32) LG collocation points. Both
the differential and integral costate estimates converges exponentially as a function of N
until the error reaches approximately O(10−12) for N = 32.
149
322824-12
0
0
-8
-6
-4
-2
4 8 12 16 20
-10
N
log10
Infin
ity-N
orm
Err
or
∥
∥λy − λ∗y
∥
∥
∞
∥
∥py − p∗y∥
∥
∞
Figure 6-9. Integral and differential costate errors for Example 2 obtained using integralLG collocation.
150
6.2.2 Solution Using Collocation at Legendre-Gauss-Radau Points
Next, Example 2 was solved using integral collocation at LGR points, as described
in Chapter 3. Figure 6-10 shows the state and control approximations obtained using
N = 32 LGR collocation points. It is seen that, similar to collocation at the LG points,
integral collocation at the LGR points provides highly accurate approximation to the
optimal solution. However, unlike collocation at the LG points, collocation at the LGR
points provides an approximation of the control at the terminal boundary point, making
collocation at the LGR points more desirable than collocation at the LG points.
The integral costate, py (τ), was computed at the LGR points using Eq. (3–116), and
the differential costate, λy(τ), was estimated at the LGR points plus the noncollocated
endpoint τN+1 using the results of Section 3.3.5. Note that the value py(τ1) (where
τ1 = −1 for the integral LGR collocation method) was found by extrapolating the
Lagrange interpolating polynomial as described by Eqs. (3–118) and (3–119). Figure
6-11 shows both the integral and the differential costates obtained using N = 32
LGR collocation points. It is seen that the costate estimate is indistinguishable from
the optimal solution. Furthermore, Fig. 6-12 shows the base ten logarithm of the
L∞-norm error for the integral and differential costates when approximated using
(N = 4, 8, 12, ... , 32) LGR collocation points. It is seen that the differential and integral
costate estimates converges exponentially as a function of N until the error reaches
approximately O(10−12) for N = 32.
151
3.5
3.5
4.5
4.5
5.5
0 0.5
1.5
1.5
2.5
2.51 1
2
2
3
3
4
4
5
5t
Sta
te
y∗(t)
y(t)
(A) State.
-3.5
-2.5
-1.5
3.5 4.5
-0.5
0
0
0.5
0.5 1.5 2.5-4
-3
-2
-1
1 2 3 4 5t
Con
trol
u∗(t)
u(t)
(B) Control.
Figure 6-10. State and control for Example 2 obtained using integral LGR.
152
-0.8
-0.6
-0.4
-0.2
0
0
0.2
0.4
0.6
0.8
1
1 2 3 4 5t
Cos
tate
p∗y
λ∗
y
py
λy
Figure 6-11. Integral and differential costate for Example 2 obtained using integral LGRcollocation with N = 32.
322824-12
0
0
-8
-6
-4
-2
4 8 12 16 20
-10
N
log10
Infin
ity-N
orm
Err
or
∥
∥λy − λ∗y
∥
∥
∞
∥
∥py − p∗y∥
∥
∞
Figure 6-12. Integral and differential costate errors for Example 2 obtained using integralLGR collocation.
153
6.3 Example 3: First-Order State Inequality Path Constraint Problem
Consider the following state inequality path constrained optimal control problem:
Minimize
∫ 3
0
e−tu dt subject to
y = u, y(0) = 0,
y − 1 + (t − 2)2 ≥ 0,
0 ≤ u ≤ 3.
(6–4)
The optimal state and control for this example are given as
y ∗ =
0 , t ∈ [0, 1),
1− (t − 2)2 , t ∈ [1, 2],
1 , t ∈ (2, 3],
u∗ =
0 , t ∈ [0, 1),
2(2− t) , t ∈ [1, 2],
0 , t ∈ (2, 3],
(6–5)
This example was solved using LG and flipped LGR collocation with the NLP solver
SNOPT, where SNOPT was implemented using default settings. The solution domain
was divided into three intervals with N collocation points in each interval. The boundaries
between the intervals were chosen to be the time instants where the state constraint
changes between active and inactive, namely, the interval boundaries were at t = 1 and
t = 2. Furthermore, a straight line initial guess between the initial state and unity was
used for the state and control.
6.3.1 Solution Using Collocation at Legendre-Gauss Points
Example 3 was solved using variable-order collocation at the Legendre-Gauss
points, as described in Chapter 2. Figure 6-13 shows the state and control approximations
obtained using N = 10 collocation points per interval. It is seen that variable-order LG
collocation provides highly accurate approximations to the optimal solution even though
index-reduction of the state inequality path constraint was not performed. Figure 6-14
154
0
0
0.2
0.4
0.5
0.6
0.8
1.2
1.5 2.5
1
1 2 3-0.2
t
Sta
te
y∗(t)
y(t)
(A) State.
00
0.5
0.5
1.5
1.5
2.5
2.5
1
1
2
2 3t
Con
trol
u∗(t)
u(t)
(B) Control.
Figure 6-13. Primal solution for Example 3 obtained using collocation at LG points.
shows the base ten logarithm of the L∞-norm error for the state and the control for
(N = 2, 4, 6, ... , 20) collocation points per interval. It is interesting to see that the LG
state and control approximations are highly accurate even using low-degree state
approximations (that is, using a small number of collocation points).
155
-8.5
-9.5
-10.5
-11
-11.5
12
-9
-8
2 4 6 8 10 12 14 16 18 20
-10
N
log10
Infin
ity-N
orm
Err
or
‖y − y∗‖∞
‖u − u∗‖∞
Figure 6-14. State and control errors for Example 3 obtained using LG collocation.
156
6.3.1.1 Previously derived costate estimate
The accuracy of the costate estimate of Ref. [1] is now compared to the costate
estimate derived in Chapter 5. The analytic optimal costate for Example 3 using
the method of direct adjoining can be found by applying the first-order optimality
conditions derived in Section 2.3.2. The costate and state constraint multipliers are
given, respectively, as
λ∗ =
−e−1 , t ∈ [0, 1),
−e−t , t ∈ [1, 2],
0 , t ∈ (2, 3],
µ∗ =
0 , t ∈ [0, 1),
−e−t , t ∈ [1, 2],
0 , t ∈ (2, 3].
(6–6)
Figure 6-15 shows the result of the costate approximation for N = 10 collocation
points per interval. It can be seen that the costate, λ(t), is not approximated correctly at
the interval boundaries where the discontinuities occur. Furthermore, the approximation
of the state constraint multiplier, µ, is quite poor. Figure 6-16 shows the base ten
logarithm of the L∞-norm error for the costate and the state constraint mutlipliers for
(N = 2, 4, 6, ... , 20) collocation points per interval. It is seen that the costate has large
errors near the known discontinuities in the optimal costate. Furthermore, the state
constraint multiplier estimate diverges.
157
0
0 0.5 1.5 2.5
0.05
1 2 3-0.4
-0.35
-0.3
-0.25
-0.15
-0.1
-0.05
-0.2
t
Cos
tate
λ∗(t)
λ(t)
(A) Costate.
-0.5
0
0 0.5 1.5 2.5-4
-3
-2
-1
1 2 3
-3.5
-2.5
-1.5
t
Sta
teC
onst
rain
tMul
tiplie
r
µ∗(t)
µ(t)
(B) State Constraint Multiplier.
Figure 6-15. Costate Estimate as derived by Ref. [1] for Example 3 obtained usingcollocation at LG points.
158
0
1.2
1
2 4 6 8 10 12 14 16 18 20
-0.2
-0.4
-0.6
0.2
0.4
0.6
0.8
N
log10
Infin
ity-N
orm
Err
or
‖λ− λ∗‖∞
‖µ− µ∗‖∞
Figure 6-16. Errors in costate estimate derived by Ref. [1] for Example 3 obtained usingLG collocation.
159
6.3.1.2 Costate estimate using method of indirect adjoining with continuousmultipliers
The costate estimate derived using the method of indirect adjoining with continuous
multipliers for collocation at the LG points, described in Chapter 5, is now analyzed. The
optimal costate for Example 3 using the method of indirect adjoining with continuous
multipliers can be found by applying the first-order optimality conditions derived in
Section 2.3.3. The costate and the state constraint multipliers are given, respectively, as
p∗ =
0 , t ∈ [0, 1),
0 , t ∈ [1, 2],
0 , t ∈ (2, 3],
ν∗ =
−e−1 , t ∈ [0, 1),
−e−t , t ∈ [1, 2],
0 , t ∈ (2, 3].
(6–7)
Figure 6-17 shows the costate approximation for N = 10. It can be seen that the
estimates presented in Chapter 5 provide an accurate costate approximation of the
continuous optimal control problem. Figure 6-18 shows the base ten logarithm of the
L∞-norm error for the costate and state constraint multiplier. It can be seen that the
error on the state inequality constraint multiplier decreases as the number of collocation
points is increased. Furthermore, the error on the costate approximation remains
approximately zero. Therefore, the costate estimate produces an accuracy of O(10−12)
even when inacuracies in the state constraint multiplier are present due to the use of
low-order polynomial approximations in the state.
160
0.5 1 1.5 2 2.5 3-0.4
-0.35
-0.3
-0.25
-0.2
-0.15
-0.1
-0.05
0
0
0.05
t
Dua
lVar
iabl
es
p∗(t)
ν∗(t)
p(t)
ν(t)
Figure 6-17. Dual variables for Example 3 obtained using collocation at LG points.
0
-2
-4
-6
-8
-10
-12
-142 4 6 8 10 12 14 16 18 20
N
log10
Abs
olut
eE
rror
ν(t)
p(t)
Figure 6-18. Costate errors for Example 3 obtained using collocation at LG points.
161
6.3.2 Solution Using Collocation at Flipped Legendre-Gauss-Radau Points
Example 3 is now solved using variable-order flipped LGR collocation as described
in Chapter 2. Figure 6-19 shows the state and control approximations obtained using
N = 10 collocation points per interval. It is seen that variable-order collocation at the
LG points provides highly accurate approximations to the optimal solution even though
index-reduction of the state inequality path constraint was not performed. Figure 6-20
shows the base ten logarithm of the L∞-norm error for the state and the control using
(N = 2, 4, 6, ... , 20) collocation points per interval. It is seen that the solution using LGR
collocation is less accurate than the solution obtained using collocation at the LG points.
This difference in accuracy is expected, as the LG points are known to have a higher
accuracy quadrature than the LGR points. Furthermore, it is seen that the state and
control reach accuracies of O(10−5) and O(10−8), respectively, for N = 20.
6.3.2.1 Previously derived costate estimate
The accuracy of the costate estimate of Ref. [1] using flipped LGR collocation is
now compared against the accuracy of the costate estimate derived in this research.
The optimal costate for Example 3 using the method of direct adjoining can be found by
applying the first-order optimality conditions derived in Section 2.3.2. The costate and
state constraint multipliers are given, respectively, as
λ∗ =
−e−1 , t ∈ [0, 1),
−e−t , t ∈ [1, 2],
0 , t ∈ (2, 3],
µ∗ =
0 , t ∈ [0, 1),
−e−t , t ∈ [1, 2],
0 , t ∈ (2, 3].
(6–8)
Figure 6-21 shows the result of the costate approximation when N = 10 collocation
points per interval were used. It can be seen that although the costate is approximated
162
0
0
0.2
0.4
0.5
0.6
0.8
1.2
1.5 2.5
1
1 2 3-0.2
t
Sta
te
y∗(t)
y(t)
(A) State.
00
0.2
0.4
0.5
0.6
0.8
1.2
1.4
1.5
1.6
1.8
2.5
1
1
2
2 3t
Con
trol
u∗(t)
u(t)
(B) Control.
Figure 6-19. Primal solution for Example 3 obtained using variable-order collocation atLGR points.
163
−16
−14
12
-8
-6
-4
-2
2 4 6 8 10 12 14 16 18 20
-10
N
log10
Infin
ity-N
orm
Err
or
‖y − y∗‖∞
‖u − u∗‖∞
Figure 6-20. State and control errors for Example 3 obtained using variable-ordercollocation at LGR points.
accurately, the state constraint multiplier, µ, is approximated very poorly where the
costate is discontinuous. Figure 6-22 shows the base ten logarithm of the L∞-norm
error for the costate and the state constraint mutliplier when approximated using
(N = 2, 4, 6, ... , 20) collocation points per interval. It is seen that the costate estimate
has large errors near the known discontinuities in the optimal costate. Furthermore, it is
seen that the state constraint multiplier estimate diverges.
6.3.2.2 Costate estimation using method of indirect adjoining with continuousmultipliers
The costate estimate derived using the method of indirect adjoining with continuous
multipliers for collocation at the flipped LGR points,described in Chapter 5, is now
analyzed. The analytic optimal costate for Example 3 using the method of indirect
adjoining with continuous multipliers can be found by applying the first-order optimality
conditions derived in Section 2.3.3. The costate and the state constraint multipliers are
164
Dua
l Sol
utio
n
0
0 0.5 1.5 2.51 2 3-0.4
-0.35
-0.3
-0.25
-0.15
-0.1
-0.05
-0.2
t
λ∗(t)
λ(t)
(A) Costate.
−14
−12
−10
−8
−6
Dua
l Sol
utio
n
0
0 0.5 1.5 2.5
-4
-2
1 2 3t
µ∗(t)
µ(t)
(B) State Constraint Multiplier.
Figure 6-21. Costate Estimate as derived by Ref. [1] for Example 3 obtained using LGRcollocation.
165
−0.5
0.5
0
1.5
1
2
2 4 6 8 10 12 14 16 18 20N
log10
Infin
ity-N
orm
Err
or‖λ− λ∗‖
∞
‖µ− µ∗‖∞
Figure 6-22. Errors in costate estimate derived by Ref. [1] for Example 3 obtained usingLGR collocation.
given, respectively, as
p∗ =
0 , t ∈ [0, 1),
0 , t ∈ [1, 2],
0 , t ∈ (2, 3],
ν∗ =
−e−1 , t ∈ [0, 1),
−e−t , t ∈ [1, 2],
0 , t ∈ (2, 3].
(6–9)
Figure 6-23 shows the costate approximation for N = 10 obtained by using the
method described in Chapter 5 for collocation at LGR points. It can be seen that the
estimate presented in this research provides an accurate approximation of the costate.
Figure 6-24 shows the base ten logarithm of the L∞-norm error for the costate and state
constraint multiplier approximations. It can be seen that the error on the state inequality
constraint multiplier decreases exponentially as the number of collocation points is
increased. Furthermore, the error on the costate approximation remains approximately
zero. Therefore, the costate estimate produces an accuracy of O(10−12) even when
166
0.5 1 1.5 2 2.5 3-0.4
-0.35
-0.3
-0.25
-0.2
-0.15
-0.1
-0.05
0
0
0.05
t
Dua
lVar
iabl
es
p∗(t)
ν∗(t)
p(t)
ν(t)
Figure 6-23. Costate estimate for Example 3 obtained using collocation at LGR points.
0
-2
-4
-6
-8
-10
-12
-142 4 6 8 10 12 14 16 18 20
N
log10
Abs
olut
eE
rror
ν(t)
p(t)
Figure 6-24. Costate errors for Example 3 obtained using collocation at LGR points.
inacuracies in the state constraint multiplier are present due to the use of low-order
polynomial approximations in the state.
167
6.4 Example 4: Second-Order State Inequality Path Constraint Example
Consider the following second-order state inequality constrained optimal control
problem from Ref. [10]:
minimize 12
∫ 1
0
u2dt subject to
x = v ,
v = u,
x(0) = 0,
x(1) = 0,
v(0) = 1
v(1) = −1,
x(t) ≤ ℓ.
It is known for this example that the inequality path constraint is inactive for ℓ > 1/4,
is active at only a single point for 1/6 < ℓ ≤ 1/4, and is active along a nonzero duration
arc for 0 < ℓ ≤ 1/6. In the case where 0 < ℓ ≤ 1/6, the optimal state and control are
given as
x∗(t) =
ℓ[
1−(
1− t3ℓ
)3]
,
ℓ,
ℓ[
1−(
1− 1−t3ℓ)3]
,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1],
v∗(t) =
(
1− t3ℓ
)2,
0,
−(
1− 1−t3ℓ)2,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1],
u∗(t) =
− 23ℓ(
1− t3ℓ
)
,
0,
− 23ℓ(
1− 1−t3ℓ)
,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1],
A value of ℓ = 1/10 was used in the analysis of this example. The solution domain was
divided into three intervals with N collocation points in each interval. The boundaries
between the intervals were chosen to be the time instants where the state constraint
changes between active and inactive, namely, t = 3/10 and t = 7/10. The solution was
168
approximated using N = 5 collocation points. All problems were solved using the NLP
solver SNOPT with default optimality and feasibility tolerances. [22]. The initial guess
used was the exact solution.
6.4.1 Solution Using Collocation at Legendre-Gauss Points
Example 4 was solved using variable-order collocation at the Legendre-Gauss
points, as described in Chapter 2. Figure 6-25 shows the state and control approximations
obtained using N = 5 collocation points per interval. It is seen that variable-order
collocation at the LG points provides highly accurate approximations to the optimal
solution even though index-reduction of the state inequality path constraint was not
performed. Figure 6-26 shows the base 10 logarithm of the L∞-norm error for the state
and the control when approximated using (N = 2, 3, ... , 10) collocation points per
interval. It is interesting to see that the errors in the primal solution for this example are
larger than the errors observed for the primal solution of Example 3. This difference
in accuracy can be attributted to the increase in the order of the state inequality path
constraint. Although the errors in this example are larger than for Example 3, it can be
seen that an accuracy of O(10−6) and O(10−5) for the state and control, respectively,
can be obtained using N = 3 collocation points per interval.
169
00
0.1
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.02
0.04
0.06
0.08
0.12
t
Sta
teC
ompo
nent
x∗(t)
x(t)
(A) x(t).
0
0 0.1
0.2
0.2 0.3
0.4
0.4 0.5
0.6
0.6 0.7
0.8
0.8 0.9
1
1
-0.2
-0.4
-0.6
-0.8
-1t
Sta
teC
ompo
nent
v∗(t)
v(t)
(B) v(t).
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
-4
-3
-2
1
1
-5
-6
-7
-1
t
Con
trol
u∗(t)
u(t)
(C) u(t).
Figure 6-25. Primal solution for Example 4 obtained using LG collocation.
170
N
0
-8
-7
-6
-5
-4
-3
-2
-1
2 3 4 5 6 7 8 9 10
log10
Infin
ity-N
orm
Err
or
x(t)
v(t)
u(t)
Figure 6-26. State and control errors for Example 4 using collocation at LG points.
171
6.4.1.1 Previously derived costate estimate
The accuracy of the costate estimate using LG collocation derived in Ref. [1] is
now compared against the accuracy of the costate estimate derived in this research.
The optimal costate for Example 4 using the method of direct adjoining can be found by
applying the first-order optimality conditions derived in Section 2.3.2. The costate and
state constraint multiplier are given, respectively, as
λ∗x =
29ℓ2
, t ∈ [0, 3ℓ],
0 , t ∈ [3ℓ, 1− 3ℓ],
− 29ℓ2
, t ∈ [1− 3ℓ, 1],
λ∗v =
23ℓ
(
1− t3ℓ
)
, t ∈ [0, 3ℓ],
0 , t ∈ [3ℓ, 1− 3ℓ],23ℓ
(
1− 1−t3ℓ
)
, t ∈ [1− 3ℓ, 1],
µ∗ =
0 , t ∈ [0, 3ℓ],
0 , t ∈ [3ℓ, 1− 3ℓ],
0 , t ∈ [1− 3ℓ, 1].
(6–10)
Figure 6-27 shows the costate approximation for N = 5 collocation points per
interval. It can be seen that the costate, λ(t), is not approximated correctly at the
interval boundaries where the discontinuities occur. Furthermore, the state constraint
multiplier, µ, is approximated very poorly. Figure 6-28 shows the base ten logarithm of
the L∞-norm error for the costate and the state constraint mutliplier when approximated
using (N = 2, 3, ... , 10) collocation points per interval. It can be seen that large errors
around the costate discontinuities prevent the costate estimate from converging to its
optimal solution.
172
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
25
20
15
10
5
-5
-10
-15
-20
-25t
Cos
tate
Com
pone
nt
λ∗
x (t)
λx (t)
(A) Costate.
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
6
7
-1
1
1
2
3
4
5
t
Cos
tate
Com
pone
nt
λ∗
v (t)
λv(t)
(B) Costate.
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-50
-100
-150
-200
-250
-300
-350
-400t
µ∗(t)
µ(t)
Sta
teC
onst
rain
tMul
tiplie
r
(C) State Constraint Multiplier.
Figure 6-27. Costate Estimate as derived by Ref. [1] for Example 4 obtained usingcollocation at LG points.
173
1.5
2.5
1
2
2
3
3 4 5 6 7 8 9 10
3.5
0.5
N
log10
Infin
ity-N
orm
Err
or
‖λx − λ∗x ‖∞
‖λv − λ∗v ‖∞
‖µ− µ∗‖∞
Figure 6-28. Errors in costate estimate derived by Ref. [1] for Example 4 obtained usingLG collocation.
174
6.4.1.2 Costate Estimation using method of indirect adjoining with continuousmultipliers
The costate estimate derived using the method of indirect adjoining with continuous
multipliers for collocation at the LG points, described in Chapter 5, is now analyzed. The
optimal costate for Example 4 using the method of indirect adjoining with continuous
multipliers can be found by applying the first-order optimality conditions derived in
Section 2.3.3. The costate and the state constraint multiplier are given, respectively, as
p∗x(t) =
{
− 29ℓ2, t ∈ [0, 1],
p∗v (t) =
23ℓ
(
1− t3ℓ
)
,
0,
23ℓ
(
1− 1−t3ℓ
)
,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1].
ν∗(t) =
− 49ℓ2,
− 29ℓ2,
0,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1].
Figure 6-29 shows the dual variable approximations for collocation at the LG points.
It can be seen that the mapping presented in Chapter 5 provides an accurate estimate
for the dual variables of the continuous optimal control problem. Figure 6-30 shows the
base ten logarithm of the L∞-norm error for the costate and state constraint multiplier
approximations obtained using Eqs. (5–32)–(5–38) and Eq. (5–79) for N collocation
points in each of the three mesh intervals. It can be seen that the error on the dual
variables reach an accuracy of O(10−5) for low-order polynomial approximations in the
state (that is, a small number of collocation numbers).
175
-22.5
-22.4
-22.3
-22.2
-22.1
-22
-21.9
-21.8
-21.7
-21.6
-21.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1t
p∗x (t)
px (t)
Cos
tate
Com
pone
nt
(A) px(t).
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
6
7
-1
1
1
2
3
4
5
t
p∗v (t)
pv (t)
Cos
tate
Com
pone
nt
(B) pv (t).
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-5
-10
-15
-20
-25
-30
-35
-40
-45t
ν∗(t)ν(t)
Sta
teC
onst
rain
tMul
tiplie
r
(C) ν(t).
Figure 6-29. Costate estimate for Example 4 obtained using collocation at LG points.
176
0
-6
-5
-4
-3
-2
-1
1
2
2 3 4 5 6 7 8 9 10
N
log10
Abs
olut
eE
rror
ν(t)
px(t)
pv (t)
Figure 6-30. Costate errors for Example 4 obtained using collocation at LG points.
177
6.4.2 Solution Using Collocation at Flipped Legendre-Gauss-Radau Points
Next, Example 4 was solved using variable-order collocation at the flipped
Legendre-Gauss-Radau points, as described in Chapter 2. Figure 6-31 shows the
state and control approximations obtained using N = 5 collocation points per interval. It
is seen that variable-order collocation at the LG points provides accurate approximations
to the optimal solution even though index-reduction of the state inequality path constraint
was not performed. Figure 6-32 shows the base 10 logarithm of the L∞-norm error for
the state and the control when approximated using (N = 2, 3, ... , 10) collocation points
per interval. It is seen that the state reaches an accuracy of O(10−5) for N = 3, whereas
the control only reaches an accuracy of O(10−2) for N = 10.
178
0.01
0.03
0.07
0.09
00
0.1
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
0.05
1
0.02
0.04
0.06
0.08
t
Sta
teC
ompo
nent
x∗(t)
x(t)
(A) x(t).
0
0 0.1
0.2
0.2 0.3
0.4
0.4 0.5
0.6
0.6 0.7
0.8
0.8 0.9
1
1
-0.2
-0.4
-0.6
-0.8
-1t
Sta
teC
ompo
nent
v∗(t)
v(t)
(B) v(t).
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
-4
-3
-2
1
-5
-6
-7
-1
t
Con
trol
u∗(t)
u(t)
(C) u(t).
Figure 6-31. Primal solution for Example 4 obtained using LGR collocation.
179
N
0
-5
2 3 4
5
5 6 7 8 9 10
-10
-20
-25
-30
-35
-15
log10
Infin
ity-N
orm
Err
or
x(t)
v(t)
u(t)
Figure 6-32. State and control errors for Example 4 using collocation at LGR points.
180
6.4.2.1 Previously derived costate estimate
he accuracy of the costate estimate using LGR collocation derived in Ref.[1] is now
compared against the accuracy of the costate estimate derived in this research. The
optimal costate for Example 4 using the method of direct adjoining can be found by
applying the first-order optimality conditions derived in Section 2.3.2. The costate and
state constraint multipliers are given, respectively, as
λ∗x =
29ℓ2
, t ∈ [0, 3ℓ],
0 , t ∈ [3ℓ, 1− 3ℓ],
− 29ℓ2
, t ∈ [1− 3ℓ, 1],
λ∗v =
23ℓ
(
1− t3ℓ
)
, t ∈ [0, 3ℓ],
0 , t ∈ [3ℓ, 1− 3ℓ],23ℓ
(
1− 1−t3ℓ
)
, t ∈ [1− 3ℓ, 1],
µ∗ =
0 , t ∈ [0, 3ℓ],
0 , t ∈ [3ℓ, 1− 3ℓ],
0 , t ∈ [1− 3ℓ, 1].
(6–11)
Figure 6-33 shows the costate approximation for N = 5 collocation points per
interval.. It can be seen that although the costate is approximated accurately, the state
constraint multiplier, µ, is approximated very poorly. Figure 6-34 shows the base ten
logarithm of the L∞-norm error for the costate and the state constraint mutliplier when
approximated using (N = 2, 3, ... , 10) collocation points per interval. It can be seen
that large errors around the costate discontinuities prevent the costate estimate from
converging to its optimal solution.
181
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
25
20
15
10
5
-5
-10
-15
-20
-25t
Cos
tate
Com
pone
nt
λ∗
x (t)
λx (t)
(A) Costate.
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
6
7
-1
1
1
2
3
4
5
t
Cos
tate
Com
pone
nt
λ∗
v (t)
λv(t)
(B) Costate.
−2000
−1500
−1000
−500
500
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1t
µ∗(t)
µ(t)
Sta
teC
onst
rain
tMul
tiplie
r
(C) State Constraint Multiplier.
Figure 6-33. Costate Estimate as derived by Ref. [1] for Example 4 obtained usingcollocation at LGR points.
182
0
-10
12
-8
-6
-4
-2
2
2 3 4 5 6 7 8 9 10N
log10
Infin
ity-N
orm
Err
or
‖λx − λ∗x ‖∞
‖λv − λ∗v ‖∞
‖µ− µ∗‖∞
Figure 6-34. Errors in costate estimate derived by Ref. [1] for Example 4 obtained usingLGR collocation.
183
6.4.2.2 Costate Estimation using method of indirect adjoining with continuousmultipliers
The costate estimate derived using the method of indirect adjoining with continuous
multipliers for collocation at the flipped LGR points, described in Chapter 5, is now
analyzed. The analytic optimal costate for Example 4 using the method of indirect
adjoining with continuous multipliers can be found by applying the first-order optimality
conditions derived in Section 2.3.3. The costate and the state constraint multiplier are
given, respectively, as
p∗x(t) =
{
− 29ℓ2, t ∈ [0, 1],
p∗v (t) =
23ℓ
(
1− t3ℓ
)
,
0,
23ℓ
(
1− 1−t3ℓ
)
,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1].
ν∗(t) =
− 49ℓ2,
− 29ℓ2,
0,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1].
Figure 6-35 shows the dual variable approximations obtained by using the method
described in this paper for collocation at LGR points. It can be seen that the mapping
presented in Chapter 5 provides an accurate estimate for the dual variables of the
continuous optimal control problem. Figure 6-36 shows the base 10 logarithm of the
L∞-norm error for the costate and state constraint multiplier approximations for N
collocation points in each of the three mesh intervals. It can be seen that the error
on the estimate of the state inequality constraint multiplier and the costate reach an
accuracy of O(10−5) for N larger than two.
184
-22.5
-22.4
-22.3
-22.2
-22.1
-22
-21.9
-21.8
-21.7
-21.6
-21.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1t
p∗x (t)
px (t)
Cos
tate
Com
pone
nt
(A) px(t).
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
6
7
-1
1
1
2
3
4
5
t
p∗v (t)
pv (t)
Cos
tate
Com
pone
nt
(B) pv (t).
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-5
-10
-15
-20
-25
-30
-35
-40
-45t
ν∗(t)ν(t)
Sta
teC
onst
rain
tMul
tiplie
r
(C) ν(t).
Figure 6-35. Costate estimate for Example 4 obtained using collocation at LGR points.
185
0
-6
-5
-4
-3
-2
-1
1
2
2 3 4 5 6 7 8 9 10
N
log10
Abs
olut
eE
rror
ν(t)
px(t)
pv (t)
Figure 6-36. Costate errors for Example 4 obtained using collocation at LGR points.
186
CHAPTER 7CONCLUSIONS
Solving an optimal control problem is not easy. For most engineering applications,
it is impossible to derive an analytic solution to an optimal control problem using the
first-order optimality conditions derived from the calculus of variations. Thus, numerical
methods must be used to approximate the solution to the continuous-time problem.
Applying the first-order optimality conditions of the continuous-time optimal control
problem result in a Hamiltonian boundary-value problem which must be solved.
Numerical methods that aproximate a solution to the Hamiltonian boundary-value
problem stemming from the first-order optimality conditions of the optimal control
problem are called indirect methods. Numerical methods that employ the indirect
method tend to result in a highly accurate solution because the first-order optimality
conditions of the optimal control problem are satisfied. Convergence using indirect
methods, however, can be very hard to achieve due to the unstable nature of the
Hamiltonian boundary-value problem. Thus, intuitive initial guesses are required to
achieve convergence using indirect methods.
Numerical methods for solving optimal control problems that do not formulate the
first-order optimality conditions of the continous-time problem are called direct methods.
Direct methods convert the infinite-dimensional continuous control problem into a
finite-dimensional discrete nonlinear programming problem (NLP). The resulting NLP
can then be solved by well-developed NLP algorithms. Direct methods are attractive
because the first-order optimality conditions need not be derived. Furthermore, because
the Hamiltonian boundary-value problem is not formulated, convergence using direct
methods is usually easier to obtain. In this research a direct orthogonal collocation
method using collocation at the Legendre-Gauss and Legendre-Gauss-Radau
points was analyzed. In particular, an estimate of the continuous-time costate of the
187
continuous-time optimal control problem was derived from the KKT multipliers of the
nonlinear programming problem of the discrete problem.
Costate estimation is an important step in the numerical solution of optimal control
problems. Mapping the dual variables of the numerical solution to the costate of the
continuous-time problem not only allows for a verification of the dual solution, but
also allows the first-order optimality conditions of the nonlinear programming problem
(NLP) to take a form that is equivalent to the first-order optimality conditions of the
continuous-time problem. Thus, having an accurate costate estimate shows that the
KKT conditions satisfied by the NLP are a discrete form of the first-order optimality
conditions of the continuous problem given by the calculus of variations, and will
converge to an optimal solution of the continuous-time problem if discretized correctly.
Costate estimates for direct methods using orthogonal collocation at the Legendre-Gauss
and Legendre-Gauss-Radau points have previously been derived for optimal control
problems with no active state inequality path constraints and when the dynamic
constraints are formulated in their differential form. In this research a gap of costate
estimation theory was closed by deriving a mapping for the costate estimate for the case
when the dynamic constraints are expressed in integral form and in the presence of
state inequality path constraints.
In the first part of this research a costate estimate was developed for problem stated
with integral constraints. While the differential and integral forms of the LG and LGR
methods are mathematically equivalent with regard to the primal variables (that is, the
state and control), the two formulations produce completely different dual variables. In
particular, the relationship between the Lagrange multipliers of the collocation conditions
of the dynamic constraints and the costate of the optimal control problem has been
well documented. On the other hand, the corresponding relationship between the
Lagrange multipliers associated with the integral forms of LG and LGR collocation and
the costate of the optimal control problem has not been established. When employing
188
the integral forms of LG and LGR collocation, however, it may be of interest to either
verify optimality or perform sensitivity analysis in a manner consistent with that which
would be performed when using variational methods. In such cases it is useful to
obtain a costate estimate when using the integral forms of the LG and LGR methods.
Thus, in this research, a costate estimate for collocation at Legendre-Gauss and
Legendre-Gauss-Radau points was derived for the case when the dynamic constraints
of the optimal control problem are formulated in integral form. It was demonstrated
that the costate mapping derived for collocation at the LG and LGR points leads to a
set of transformed optimality conditions of the NLP which were shown to be a discrete
representation of the necessary conditions for optimality of the continuous-time problem.
Finally, a relationship between the integral and the differential forms of the costate
estimate was given and it was shown that the two sets of optimality conditions are
equivalent.
The second part of this research focused on problems with active state inequality
path constraints. Although previous research has successfully derived a high-accuracy
estimate of the costate from the KKT multipliers of the NLP for the case of a problem
with no active state inequality path constraints, Ref. [1] subsequently showed that in the
case when the costate is discontinuous (as is the case in the presence of active state
inequality path constraints), this costate estimate leads to a set of first-order optimality
conditions of the NLP that are not equivalent to the discrete form of the variational
optimality conditions. This lack of equivalence leads to an inaccurate approximation
of the costate. Therefore, in this research costate estimates for collocation at LG and
LGR points were derived for problems with active state inequality path constraints.
The derived costate estimate was shown to lead to a transformed adjoint system of
the NLP which is a discrete approximation of the necessary conditions for optimality
of the continuous-time optimal control problem. This equivalence was not existent with
prior costate estimates using LG and LGR collocation. The costate estimates derived
189
in this dissertation were implemented in four problems to assess their accuracy. It was
shown that each discrete costate estimate led to an accurate approximation of the
continuous-time costate.
190
REFERENCES
[1] Darby, C. L., Garg, D., and Rao, A. V., “Costate Estimation Using Multiple-IntervalPseudospectral Methods,” Journal of Spacecraft and Rockets, Vol. 48, No. 5,September–October 2011, pp. 856–866.
[2] Betts, J., Practical Methods for Optimal Control and Estimation Using NonlinearProgramming, SIAM Press, Philadelphia, 2nd ed., 2009.
[3] Unger, J., Kroner, A., and Marquardt, W., “Structural analysis ofdifferential-algebraic equation systems - theory and applications,” Computersand Chemical Engineering, Vol. 19, No. 8, 1995, pp. 867 – 882.
[4] Feehery, W. F. and Barton, P. I., “Dynamic optimization with state variable pathconstraints,” Computers and Chemical Engineering, Vol. 22, No. 9, 1998, pp. 1241– 1256.
[5] Maurer, H. and Pesch, H. J., “Direct optimization methods for solving a complexstate-constrained optimal control problem in microeconomics,” Applied Mathematicsand Computation, Vol. 204, No. 2, 2008, pp. 568 – 579.
[6] Boskens, Christof, M. H., “SQP-methods for solving optimal control problems withcontrol and state constraints: adjoint variables, sensitivity analysis and real-timecontrol,” J. Comput. Appl. Math., Vol. 120, No. 1-2, Aug. 2000, pp. 85–108.
[7] Bryson, A. E. and Ho, Y.-C., Applied Optimal Control , Hemisphere Publishing, NewYork, 1975.
[8] Speyer, J. L., “Necessary Conditions for Optimality For Paths Lying on a Corner,”Management Science, Vol. 19, No. 11, July 1973.
[9] Kirk, D. E., Optimal Control Theory: An Introduction, Dover Publications, Mineola,New York, 2004.
[10] Bryson, A. E., Denham, W. F., and Dreyfus, S. E., “Optimal Programming Problemswith Inequality Constraints I: Necessary Conditions for Extremal Solutions,” AIAAJournal , Vol. 1, No. 11, 1962, pp. 2544–2550.
[11] Dreyfus, S. E., Variational Problems with State Variable Inequality Constraints,Ph.D. thesis, Harvard University, Cambridge, Massachussets, 1962.
[12] Jacobson, D. H., Lele, M. M., and Speyer, J. L., “New Necessary Conditions ofOptimality for Control Problems with State-Variable Inequality Constraints,” Tech.rep., Division of Engineering and Applied Physics - Harvard University, Cambridge,Massachussets, 1969.
[13] Speyer, J. L. and Bryson, A. E., “Optimal Programming Problems with a BoundedState Space,” AIAA Journal , Vol. 6, No. 8, August 1968, pp. 1488–1491.
191
[14] Denham, W. F. and Bryson, A. E., “Optimal Programming Problems with InequalityConstraints II: Solution by Steepest-Ascent,” AIAA Journal , Vol. 2, No. 1, January1964, pp. 25–34.
[15] Hartl, R. F., Sethi, S. P., and Vickson, R. G., “A Survey of the Maximum Principlesfor Optimal Control Problems with State Constraints,” SIAM Review , Vol. 37, No. 2,June 1995, pp. 181–218.
[16] Gollan, B., “On Optimal Control Problems with State Constraints,” Journal ofOptimization Theory and Applications, Vol. 32, 1980, pp. 75–80.
[17] Russak, I. B., “On Problems with Bounded State Variable,” Journal of OptimizationTheory and Applications, Vol. 5, 1970, pp. 114–157.
[18] Russak, I. B., “On General Problems with Bounded State Variable,” Journal ofOptimization Theory and Applications, Vol. 6, 1970, pp. 424–452.
[19] Vinter, R., Optimal Control (Systems & Control: Foundations and Applications),Birkhauser, Boston, 2000.
[20] Russel, R. and Shampine, L., “A Collocation Method for Boundary Value Problems,”Numerical Mathematics, Vol. 19, 1972, pp. 1–28.
[21] Rao, A. V. and Mease, K. D., “Dichotomic Basis Approach to solvingHyper-Sensitive Optimal Control Problems,” Automatica, Vol. 35, No. 4, April1999, pp. 633–642.
[22] Gill, P. E., Murray, W., and Saunders, M. A., “SNOPT: An SQP Algorithm forLarge-Scale Constrained Optimization,” SIAM Review , Vol. 47, No. 1, January2005, pp. 99–131.
[23] Biegler, L. T. and Zavala, V. M., “Large-Scale Nonlinear Programming Using IPOPT:An Integrating Framework for Enterprise-Wide Optimization,” Computers andChemical Engineering, Vol. 33, No. 3, March 2008, pp. 575–582.
[24] Wachter, A., An Interior Point Algorithm for Large-Scale Nonlinear Optimizationwith Applications in Process Engineering, Ph.D. thesis, Department of ChemicalEngineering, Carnegie-Mellon University, Pittsburgh, PA, 2002.
[25] Wachter, A. and Biegler, L. T., “On the Implementation of an Interior-Point FilterLine-Search Algorithm for Large-Scale Nonlinear Programming,” MathematicalProgramming, Vol. 106, No. 1, 2006, pp. 25–57.
[26] Fahroo, F. and Ross, I. M., “Costate Estimation by a Legendre PseudospectralMethod,” Journal of Guidance, Control, and Dynamics, Vol. 24, No. 2, March–April2001, pp. 270–277.
[27] Benson, D. A., Huntington, G. T., Thorvaldsen, T. P., and Rao, A. V., “DirectTrajectory Optimization and Costate Estimation via an Orthogonal Collocation
192
Method,” Journal of Guidance, Control, and Dynamics, Vol. 29, No. 6,November-December 2006, pp. 1435–1440.
[28] Bock, H. G. and Plitt, K. J., “A Multiple Shooting Algorithm for Direct Solution ofOptimal Control Problems,” IFAC 9th World Congress, Budapest, Hungary, 1984.
[29] Williamson, W. E., “Use of Polynomial Approximations to Calculate SuboptimalControls,” AIAA Journal , Vol. 9, No. 11, 1971, pp. 2271–2273.
[30] Kraft, D., On Converting Optimal Control Problems into Nonlinear ProgrammingCodes, Vol. F15 of NATO ASI Series, in Computational Mathematical Program-ming, Springer-Verlag, Berlin, 1985, pp. 261–280.
[31] Von-Stryk, O. and Bulirsch, R., “Direct and Indirect Methods for TrajectoryOptimization,” Annals of Operations Research, Vol. 37, 1992, pp. 357–373.
[32] Hargraves, C. R. and Paris, S. W., “Direct Trajectory Optimization Using NonlinearProgramming Techniques,” Journal of Guidance, Control, and Dynamics, Vol. 10,No. 4, July–August 1987, pp. 338–342.
[33] Enright, P. J. and Conway, B. A., “Discrete Approximations to Optimal TrajectoriesUsing Direct Transcription and Nonlinear Programming,” Journal of Guidance,Control, and Dynamics, Vol. 19, No. 4, July–August 1996, pp. 994–1002.
[34] Benson, D. A., A Gauss Pseudospectral Transcription for Optimal Control , Ph.D.thesis, Department of Aeronautics and Astronautics, Massachusetts Institute ofTechnology, Cambridge, Massachusetts, 2004.
[35] Elnagar, G., Kazemi, M., and Razzaghi, M., “The Pseudospectral Legendre Methodfor Discretizing Optimal Control Problems,” IEEE Transactions on AutomaticControl , Vol. 40, No. 10, 1995, pp. 1793–1796.
[36] Garg, D., Patterson, M. A., Darby, C. L., Francolin, C., Huntington, G. T., Hager,W. W., and Rao, A. V., “Direct Trajectory Optimization and Costate Estimationof Finite-Horizon and Infinite-Horizon Optimal Control Problems using a RadauPseudospectral Method,” Computational Optimization and Applications, Vol. 49,No. 2, June 2011, pp. 335–358.
[37] Biegler, L. T., Ghattas, O., Heinkenschloss, M., and van Bloemen Waanders,B., editors, Large-Scale PDE Constrained Optimization, Lecture Notes inComputational Science and Engineering, Vol. 30, Springer-Verlag, Berlin, 2003.
[38] Betts, J. T., Campbell, S. L., and Engelsone, A., “Direct Transcription Solution ofOptimal Control Problems with Higher Order State Constraints: Theory vs Practice,”Optimization and Engineering, Vol. 8, No. 1, 2007, pp. 1–19.
[39] Betts, J. T. and Huffman, W. P., “Sparse Optimal Control Software – SOCS,” Tech.Rep. MEA-LR-085, Boeing Information and Support Services, Seattle, Washington,July 1997.
193
[40] Ross, I. M. and Fahroo, F., User’s Manual for DIDO 2001 α: A MATLAB Applicationfor Solving Optimal Control Problems, Department of Aeronautics and Astronautics,Naval Postgraduate School, Technical Report AAS-01-03, Monterey, California,2001.
[41] Von-Stryk, O., User’s Guide for DIRCOL Version 2.1: A Direct Collocation Methodfor the Numerical Solution of Optimal Control Problems, Technische UniversitatDarmstadt, Darmstadt, Germany, 1999.
[42] Rao, A. V., Benson, D. A., Darby, C. L., Francolin, C., Patterson, M. A., Sanders,I., and Huntington, G. T., “Algorithm 902: GPOPS, A Matlab Software for SolvingMultiple-Phase Optimal Control Problems Using the Gauss PseudospectralMethod,” ACM Transactions on Mathematical Software, Vol. 37, No. 2, April–June2010, pp. 22:1–22:39.
[43] J.T.Betts, “Survey of Numerical Methods for Trajectory Optimization,” Journal ofGuidance, Control, and Dynamics, Vol. 21, No. 2, March–April 1998, pp. 193–207.
[44] Hager, W. W., “Runge-Kutta Methods in Optimal Control and the TransformedAdjoint System,” Numerische Mathematik , Vol. 87, 2000, pp. 247–282.
[45] Dontchev, A. L., Hager, W. W., and Veliov, V. M., “Second-Order Runge-KuttaApproximations In Constrained Optimal Control,” SIAM Journal on NumericalAnalysis, Vol. 38, 2000, pp. 202–226.
[46] Betts, J. T., “Sparse Jacobian Updates in the Collocation Method for OptimalControl Problems,” Journal of Guidance, Control, and Dynamics, Vol. 13, No. 3,May–June 1990, pp. 409–415.
[47] Dontchev, A. L., Hager, W. W., and Malanowski, K., “Error Bounds for the EulerApproximation and Control Constrained Optimal Control Problem,” NumericalFunctional Analysis and Applications, Vol. 21, 2000, pp. 653–682.
[48] Dontchev, A., Hager, W., and Veliov, V., “Uniform Convergence and Meshindependence of Newton’s method for Discretized Variational Problems,” SIAMJournal on Control and Optimization, Vol. 39, 2000, pp. 961–980.
[49] Kameswaran, S. and Biegler, L. T., “Convergence Rates for Direct Transcriptionof Optimal Control Problems Using Collocation at Radau Points,” ComputationalOptimization and Applications, Vol. 41, No. 1, 2008, pp. 81–126.
[50] Darby, C. L., Hager, W. W., and Rao, A. V., “An hp-Adaptive Pseudospectral Methodfor Solving Optimal Control Problems,” Optimal Control Applications and Methods,Vol. 32, No. 4, July–August 2011, pp. 476–502.
[51] Darby, C. L., Hager, W. W., and Rao, A. V., “Direct Trajectory Optimization Using aVariable Low-Order Adaptive Pseudospectral Method,” Journal of Spacecraft andRockets, Vol. 48, No. 3, May–June 2011, pp. 433–445.
194
[52] Vlassenbroeck, J. and Dooren, R. V., “A Chebyshev Technique for SolvingNonlinear Optimal Control Problems,” IEEE Transactions on Automatic Control ,Vol. 33, No. 4, 1988, pp. 333–340.
[53] Vlassenbroeck, J., “A Chebyshev Polynomial Method for Optimal Control with StateConstraints,” Automatica, Vol. 24, No. 4, 1988, pp. 499–506.
[54] Elnagar, G. and Razzaghi, M., “A Collocation-Type Method for Linear QuadraticOptimal Control Problems,” Optimal Control Applications and Methods, Vol. 18,No. 3, 1998, pp. 227–235.
[55] Elnagar, G. and Kazemi, M., “Pseudospectral Chebyshev Optimal Control ofConstrained Nonlinear Dynamical Systems,” Computational Optimization andApplications, Vol. 11, No. 2, 1998, pp. 195–217.
[56] Elnagar, G. N. and Razzaghi, M. A., “Pseudospectral Legendre-Based OptimalComputation of Nonlinear Constrained Variational Problems,” Journal of Computa-tional and Applied Mathematics, Vol. 88, No. 2, March 1998, pp. 363–375.
[57] Williams, P., “Jacobi Pseudospectral Method for Solving Optimal Control Problems,”Journal of Guidance, Control, and Dynamics, Vol. 27, No. 2, March–April 2004,pp. 293–297.
[58] Williams, P., “Application of Pseudospectral methods for Receding Horizon Control,”Journal of Guidance, Control, and Dynamics, Vol. 27, No. 2, 2004, pp. 310–314.
[59] Williams, P., “Hermite-Legendre-Gauss-Lobatto Direct Transcription Methods inTrajectory Optimization,” Americal Astronautical Society, Spaceflight MechanicsMeeting, August 2005.
[60] Huntington, G. T., Advancement and Analysis of a Gauss Pseudospectral Tran-scription for Optimal Control , Ph.D. thesis, Massachusetts Institute of Technology,Cambridge, Massachusetts, 2007.
[61] Huntington, G. T., Benson, D. A., and Rao, A. V., “Optimal Configuration ofTetrahedral Spacecraft Formations,” The Journal of the Astronautical Sciences,Vol. 55, No. 2, April-June 2007, pp. 141–169.
[62] Huntington, G. T. and Rao, A. V., “Optimal Reconfiguration of SpacecraftFormations Using the Gauss Pseudospectral Method,” Journal of Guidance,Control, and Dynamics, Vol. 31, No. 3, May-June 2008, pp. 689–698.
[63] Fahroo, F. and Ross, I. M., “Direct Trajectory Optimization by a ChebyshevPseudospectral Method,” Journal of Guidance, Control, and Dynamics, Vol. 25,No. 1, 2002, pp. 160–166.
[64] Garg, D., Patterson, M. A., Hager, W. W., Rao, A. V., Benson, D. A., and Huntington,G. T., “A Unified Framework for the Numerical Solution of Optimal Control Problems
195
Using Pseudospectral Methods,” Automatica, Vol. 46, No. 11, December 2010,pp. 1843–1851.
[65] Garg, D., Hager, W. W., and Rao, A. V., “Pseudospectral Methods for SolvingInfinite-Horizon Optimal Control Problems,” Automatica, Vol. 47, No. 4, April 2011,pp. 829–837.
[66] Dontchev, A. L. and Hager, W. W., “The Euler Approximation in State ConstrainedOptimal Control,” Mathematics of Computation, Vol. 70, 2001, pp. 173–203.
[67] Maurer, H. and Gillessen, W., “Application of multiple shooting to the numericalsolution of optimal control problems with bounded state variables,” Computing,Vol. 15, 1975, pp. 105–126.
[68] Garg, D., Advances In Global Pseudospectral Methods For Optimal Control , Ph.D.thesis, University of Florida, Gainesville, Florida, 2011.
[69] Patterson, M. A. and Rao, A. V., “GPOPS-II: A MATLAB Software for SolvingMultiple-Phase Optimal Control Problems Using hp-Adaptive Gaussian QuadratureCollocation Methods and Sparse Nonlinear Programming,” ACM Transactions onMathematical Software, February 2013, Submitted.
[70] Patterson, M. A., Hager, W. W., and Rao, A. V., “A ph-Collocation Scheme forOptimal Control,” Automatica, January 2013, Submitted.
[71] Vlases, W. G., Paris, S. W., Lajoie, R. M., Martens, M. J., and Hargraves, C. R.,“Optimal Trajectories by Implicit Simulation,” Tech. Rep. WRDC-TR-90-3056,Boeing Aerospace and Electronics, Wright-Patterson Air Force Base, Ohio, 1990.
[72] Falugi, P., Kerrigan, E., and van Wyk, E., Imperial College London Optimal ControlSoftware User Guide (ICLOCS), Department of Electrical Engineering, ImperialCollege London, London, UK, May 2010.
[73] Houska, B., Ferreau, H. J., and Diehl, M., “ACADO Toolkit–An Open-SourceFramework for Automatic Control and Dynamic Optimization,” Optimal ControlApplications and Methods, Vol. 32, No. 3, 2011, pp. 298–312.
[74] Darby, C. L., hp-Pseudospectral Method For Solving Continuous-Time NonlinearOptimal Control Problems, Ph.D. thesis, University of Florida, Gainesville, Florida,2011.
196
BIOGRAPHICAL SKETCH
Camila Francolin was born in Rio de Janeiro, Brazil. She received her dual Bachelor
of Science degrees in aerospace and mechanical engineering in December 2007
from the University of Florida. She then received her Master of Science degree in
aerospace engineering in May 2010, and her Doctor of Philosophy in aerospace
engineering in August 2013 from the University of Florida. Her research interests include
numerical approximations to differential equations, optimal control theory, and numerical
approximations to the solution of optimal control problems.