Nondegenerate and Normal forms of the Maximum Principle for Control Problems …faf/preprints/S2015-london.pdf · 2015-09-08 · Nondegenerate and Normal forms of the Maximum Principle

Nondegenerate and Normal forms of theMaximum Principle for Control Problems with

State Constraints

Fernando ACC Fontes,

Universidade do [email protected]

Workshop ’Nonlinear Analysis and Optimization’,On the 65th birthday of Francis Clarke and Richard Vinter,

The Royal Society, London, 7–8 September 2015

Nondegeneracy and Normality of the Maximum Principle (FACC Fontes)

Outline

Optimal Control in Critical Decision Making Scenarios

Abnormality in Mathematical ProgrammingNecessary Conditions of Optimality (NCO)Normal form of NCO

The Quest

Abnormality/Degeneracy in Optimal ControlMaximum Principle (MP)The abnormality phenomenonThe degeneracy phenomenon

Nondegenerate forms of the MPConstraint Qualification of Integral Type: CQI

Constraint Qualification at outward pointing velocities: CQout

Normal forms of the MP



Outline


Abnormality in Mathematical Programming

The Quest

Abnormality/Degeneracy in Optimal Control

Nondegenerate forms of the MP





I In the early years, optimal control was mainly used for planning, solvedoff-line (e.g. devising spaceship trajectories, economic growth, fishingpolicies, ...)

I Recently, it is increasingly being used in real-time to control processes(mainly within MPC, e.g. control of destillation columns in the petrolrefining industry, ...) .

I More recently, its is being used in real-time to make decisionsautonomously (e.g. autonomous underwater vehicles: minimum energyconsumption is a key factor in data-gathering missions, communication islimited; optimal storage strategies in electrical power systems: requiresfast, autonomous decisions for integration of variable and uncertainrenewable generation

⇒ When NCO are being used, we must guarantee that they help selecting the minimizers



Necessary Conditions of Optimality (NCO), Degeneracy and AbnormalityStructure: If x is a minimizer, then it satisfies the NCO.

(Example in unconstrained function optimization:If x is a minimizer for f , then∇f (x) = 0 )

Aim: To identify a (“small”) setcontaining all the minimizers.(The stricter the better. Ideally, if theNCO are also sufficient, then M = N.)

Degeneracy phenomenon: Alladmissible solutions satisfy the NCO:N = A.(NCO are useless in this case.)

Abnormality phenomenon: the NCOare merely state a relation between theconstraints and do not use theobjective function to select candidatesto minimizers.



Outline


Abnormality in Mathematical ProgrammingNecessary Conditions of Optimality (NCO)Normal form of NCO

The Quest






Necessary Conditions of Optimality (NCO)

Necessary Conditions of Optimality in Mathematical Programming

Nonlinear Programming problem with inequality constraints:

(MP) Minimizex∈IRn g(x)

subject to hi (x) ≤ 0 i = 1, 2, . . . , k .

Fritz-John Necessary Conditions of Optimality (1948): If x solves (MP) then∃(λ, µ) ∈ IR× IRk s.t.

(λ, µ) 6= 0

λ, µi ≥ 0 i = 1, 2, . . . , k

λ∇g(x) +∑

i=1,2,...,k

µi∇hi (x) = 0

∑i=1,2,...,k

µihi (x) = 0 .

Is this useful if λ = 0? Can we choose λ > 0? When?



Normal form of NCO

Can we always choose λ positive ?

Ans: Not alwaysA counter-example (Kuhn-Tucker 1951)

(MP1) Minimize −x1

subject to

x2 + (x1 − 1)3 ≤ 0

−x2 ≤ 0.

Solution is x = (x1, x2) = (1, 0).

With λ > 0 it is impossible to satisfy

λ∇g(x) + µ1∇h1(x) + µ2∇h2(x) = 0.



Normal form of NCO

Can we always choose λ positive ?

Ans: Not alwaysA counter-example (Kuhn-Tucker 1951)

(MP1) Minimize −x1

subject to

x2 + (x1 − 1)3 ≤ 0

−x2 ≤ 0.

Solution is x = (x1, x2) = (1, 0).

With λ > 0 it is impossible to satisfy

λ∇g(x) + µ1∇h1(x) + µ2∇h2(x) = 0.



Normal form of NCO

We can choose λ > 0, (λ = 1)for all problems satisfying a

Constraint Qualification[Kuhn Tucker 1951] (one of the most cited results in optimization)

For example

Constraint Qualification (Mangasarian-Fromovitz) There is a vector v ∈ IRk

satisfying∇hi (x) · v < 0

for all i such that hi (x) = 0.


The Quest

Outline



The Quest





The Quest

The Quest

In Optimal Control, the NCO - the Pontryangin Maximum Principle (PMP) -can also be abnormal or degenerate.

Our quest is to devise Nondegenerate and Normal forms of the PMP, validunder a constraint qualifications;

Obtain for Optimal Control a result analogous to what Kunh and Tucker havedone for Nonlinear Programming.



Outline



The Quest

Abnormality/Degeneracy in Optimal ControlMaximum Principle (MP)The abnormality phenomenonThe degeneracy phenomenon





The problem

Optimal control problem with (pathwise inequality) state constraints:

Minimize g(x(1))

subject to

x(t) = f (t, x(t), u(t)) a.e. t ∈ [0, 1]

x(0) = x0

x(1) ∈ C

u(t) ∈ Ω(t) a.e. t ∈ [0, 1]

h (t, x(t)) ≤ 0 for all t ∈ [0, 1],



Model Predictive Control

Consider a sequence tii≥0 s.t. ti+1 = ti + δ, δ > 0:

1. Measure state of the plant xti

2. Get u : [ti , ti +T ] 7→ IRm solution to the OCP:

Minimize

∫ ti+T

ti

L(t, x(t), u(t))dt + W (x(ti + T ))

subject to x(t) = f (t, x(t), u(t)) a.e. t ∈ [ti , ti + T ]

x(ti ) =xtiu(t) ∈ U(t) a.e. t ∈ [ti , ti + T ]

x(t) ∈ X a.e. t ∈ [ti , ti + T ]

x(ti + T ) ∈ S

3. Apply to the plant the control u∗(t) := u(t) inthe interval [ti , ti + δ]. (the remaining controlu(t), t > ti + δ is discarded)

4. Repeat for the next sampling time ti = ti + δ

ti Sequence of sampling instants

δ Sampling interval

xtiPlant state measured at time ti .

(x, u) OL state/control solving the OCP

(x∗, u∗) CL state/control of the MPC

T, L, W, S Design parameters



Maximum Principle (MP)

Necessary Conditions of Optimality: Maximum Principle (smooth version)

If (x , u) solves (P) then ∃λ ∈ IR, p ∈ AC , µ ∈ C∗ s.t.

µ[0, 1]+ ||p||L∞ + λ > 0,

−p(t) =

(p(t) +

∫[0,t)

hx(s, x(s))µ(ds)

)· fx (t, x(t), u(t)) a.e. t ∈ [0, 1],

−

(p(1) +

∫[0,1]

hx(s, x(s))µ(ds)

)∈ NC (x(1)) + λgx(x(1)),

suppµ ⊂ t ∈ [0, 1] : h (t, x(t)) = 0 ,

and for almost every t ∈ [0, 1], u(t) maximizes over Ω(t)

u 7→

(p(t) +

∫[0,t)

hx(s, x(s))µ(ds)

)· f (t, x(t), u) .



The abnormality phenomenon

The abnormality Phenomenon:When λ = 0, the objective function is not taken into account


µ[0, 1]+ ||p||L∞ + λ > 0,

−p(t) =

(p(t) +

∫[0,t)

hx(s, x(s))µ(ds)

)· fx (t, x(t), u(t)) a.e. t ∈ [0, 1],

−

(p(1) +

∫[0,1]

hx(s, x(s))µ(ds)

)∈ NC (x(1)) + λgx(x(1)),

suppµ ⊂ t ∈ [0, 1] : h (t, x(t)) = 0 ,


u 7→

(p(t) +

∫[0,t)

hx(s, x(s))µ(ds)

)· f (t, x(t), u) .



The degeneracy phenomenon

The Degeneracy Phenomenon

Suppose the trajectory starts on the boundary ofthe admissible region

h(0, x0) = 0,

(happens in problems of interest, e.g. ModelPredictive Control)

The Degenerate Multipliers

λ = 0, µ = δ0, p = −hx(0, x0).

(or scalar multiples of these) satisfy the MP for every pair (x , u) we might test.Note that as

λ = 0, and q(t) :=

(p(t) +

∫[0,t)

hx(s, x(s))µ(ds)

)= 0 a.e.

the Maximum Principle gives us no information.




The Maximum Principle trivially satisfied for the degenerate multipliers


µ[0, 1]+ ||p||L∞ + λ > 0,

−p(t) =

(p(t) +

∫[0,t)

hx(s, x(s))µ(ds)

)· fx (t, x(t), u(t)) a.e. t ∈ [0, 1],

−

(p(1) +

∫[0,1]

hx(s, x(s))µ(ds)

)∈ NC (x(1)) + λgx(x(1)),

suppµ ⊂ t ∈ [0, 1] : h (t, x(t)) = 0 ,


u 7→

(p(t) +

∫[0,t)

hx(s, x(s))µ(ds)

)· f (t, x(t), u) .




How to avoid Degeneracy?

Ans: Strengthening the MP

For example, strengthening the nontriviality condition to

µ(0, 1]+ λ+

∥∥∥∥∥p(t) +

∫[0,t)

hx(s, x(s))µ(ds)

∥∥∥∥∥∞

> 0,

I Eliminates only the Degenerate Multipliers

I But we have to guarantee that the Strengthened MP is still satisfied for alllocal minimizers.




How to avoid Degeneracy?

Ans: Strengthening the MP

For example, strengthening the nontriviality condition to

µ(0, 1]+ λ+

∥∥∥∥∥p(t) +

∫[0,t)

hx(s, x(s))µ(ds)

∥∥∥∥∥∞

> 0,

I Eliminates only the Degenerate Multipliers

I But we have to guarantee that the Strengthened MP is still satisfied for alllocal minimizers.




Is the Strengthened MP satisfied for every Problem?

Ans: NO!

Example:[Dubovitskii, in ArutyonovAseev97]

Minimize x2(1)

subject to

(x1(t), x2(t)) = (tu(t), u(t)) a.e. t ∈ [0, 1]

(x1(0), x2(0)) = (0, 0)

u(t) ∈ [−1, 1] a.e. t ∈ [0, 1]

x1(t) ≥ 0 for all t ∈ [0, 1],

I Here the degenerate multipliers are the only possible choice

I We need condition that identify the problems under which we canStrengthen the MP: a Constraint Qualification.




Is the Strengthened MP satisfied for every Problem?

Ans: NO!

Example:[Dubovitskii, in ArutyonovAseev97]

Minimize x2(1)

subject to

(x1(t), x2(t)) = (tu(t), u(t)) a.e. t ∈ [0, 1]

(x1(0), x2(0)) = (0, 0)

u(t) ∈ [−1, 1] a.e. t ∈ [0, 1]

x1(t) ≥ 0 for all t ∈ [0, 1],

I Here the degenerate multipliers are the only possible choice

I We need condition that identify the problems under which we canStrengthen the MP: a Constraint Qualification.



Outline



The Quest


Nondegenerate forms of the MPConstraint Qualification of Integral Type: CQI





Nondegenerate forms of the MP in the literature (< 2010)

I Early russian references: Dubovitskii,Dubovitskii 85,Arutyunov,Tynianskii85

I Ferreira,Vinter 94 (first reference in English)

I Arutyunov,Aseev 97

I Ferreira,Fontes,Vinter 99

I Arutyunov 2000

I Rampazzo, Vinter 2003

I dePinho,Ferreira,Fontes 2004

I Bettiol,Frankowska 2007

I Lopes, Fontes 2007

I ...



Two groups of Constraint Qualifications

[ArutyunovAseev97, RampazzoVin-

ter03, CerneaFrankowska05, Bettiol-

Frankowska07 ...]

(CQ1) ∃δ > 0, ∃u such that for tnear 0

hx(x0) · f (x0, u) < 0

+ Not involving the optimal control (eas-ier to verify)- Typically require more regularity.

[FerreiraVinter94, FerreiraFontesVinter99, dePin-

hoFerreiraFontes04, LopesFontes07, ...]

(CQ2) ∃δ > 0, ∃u such that for t near 0

hx(x0) · (f (x0, u)− f (x0, u(t))) < −δ

- Involving the optimal control (more difficult toverify except in special cases such as CV)+ applicable to wider/less regular class of problems



How do CQ1 and CQ2 relate?

Theorem1 Let (x , u) be a local minimizer in the class of piecewise continuous controls.Assume that hypotheses H1-H5 are satisfied. If the trajectory does not leavethe boundary immediately then CQ2 implies CQ1.

1Lopes, Sofia O., Fernando ACC Fontes, and Maria do RosA¡rio de Pinho. ”On constraintqualifications for nondegenerate necessary conditions of optimality applied to optimal controlproblems.” Discrete and Continuous Dynamical Systems (DCDS-A) 29.2 (2011): 559-575.



Constraint Qualification of Integral Type: CQI

Constraint Qualification of Integral TypeCQI : if h(0, x0) = 0, then there exist positive constants ε, ε1, δ and a

control function u(t) ∈ Ω(t) such that for all t ∈ [0, ε)∫ t

0

ζ · [f (τ, x0, u(τ))− f (τ, x0, u(τ))]dτ ≤ −δt,

for all ζ ∈ ∂>x h(s, x), s ∈ [0, ε), x ∈ x0 + ε1B.

Remark

In this constraint qualification, the inward pointing condition has to be satisfiedfor some, not all, instants of a neighbourhood of the initial time.

Theorem2 If constraint qualifications CQI is satified, then a stronger (nondegenerate)version of the maximum principle holds with

µ(0, 1]+ ‖q‖L∞ + λ > 0.

2S. O. Lopes, F. A. C. C. Fontes, M. d. R. de Pinho, In Integral-type Constraint Qualificationto Guarantee Nondegeneracy of the Maximum Principle for Optimal Control Problems with StateConstraints, Systems and Control Letters, vol. 62, pp. 686-692, 2013.




Comparison with previous CQ

CQI : if h(0, x0) = 0, then there exist positive constants ε, ε1, δ and acontrol function u ∈ U such that for all t ∈ [0, ε)∫ t

0

hx(s, x(s)) · [f (τ, x0, u(τ))− f (τ, x0, u(τ))]dτ ≤ −δt,

for all s ∈ [0, ε), x ∈ x0 + ε1B.

CQFFV99 : if h(0, x0) = 0, then there exist positive constants ε, ε1, δ and acontrol function u ∈ U such that for a.e. t ∈ [0, ε)

hx(s, x(s)) · [f (t, x0, u(t))− f (t, x0, u(t))] < −δ,

for all s ∈ [0, ε), x ∈ x0 + ε1B.

The inequality hx ·∆f < −δ does not have to be satisfied at a.e. pointst ∈ [0, ε), but just on a subset of t ∈ [0, ε) of positive measure!




Example (in which CQI is satisfied and CQ-ffv is not)

Minimize −x(1)

subject to x(t) = u(t) a.e. t ∈ [0, 1]

x(0) = 0, x(1) ∈ IR

u(t) ∈ Ω(t) a.e. t ∈ [0, 1]

x(t) ≤ 0 for all t ∈ [0, 1],

where Ω(t) = u ∈ IR : g(t) ≤ u ≤ 0and g is the function:

g(t) =

−1, t = 0

2− 42nt, t ∈

[122−n, 3

42−n), n ∈ N

42nt − 4, t ∈

[342−n, 2−n

), n ∈ N

The optimal solution is u(t) = 0 and x(t) = 0.





Minimize −x(1)


x(0) = 0, x(1) ∈ IR

u(t) ∈ Ω(t) a.e. t ∈ [0, 1]

x(t) ≤ 0 for all t ∈ [0, 1],


g(t) =

−1, t = 0

2− 42nt, t ∈

[122−n, 3

42−n), n ∈ N

42nt − 4, t ∈

[342−n, 2−n

), n ∈ N






Minimize −x(1)


x(0) = 0, x(1) ∈ IR

u(t) ∈ Ω(t) a.e. t ∈ [0, 1]

x(t) ≤ 0 for all t ∈ [0, 1],


g(t) =

−1, t = 0

2− 42nt, t ∈

[122−n, 3

42−n), n ∈ N

42nt − 4, t ∈

[342−n, 2−n

), n ∈ N





Example (cont.) I

For this example

hx(s, x(s)) · [f (t, x0, u(t))− f (t, x0, u(t))] = u(t),

Thus, the constraint qualifications reduce to:CQ-ffv: ∃δ, ε > 0 and a control function u such that for a.e. t ∈ [0, ε):

u(t) < −δ a.e. t ∈ [0, ε) (1)

CQI ∃δ, ε > 0 and a control function u such that for a.e. t ∈ [0, ε)∫ t

0

u(τ)dτ ≤ −δt ∀t ∈ [0, ε), (2)

But, for any ε > 0, g(t) = 0 and Ω(t) = 0 for an infinite number of pointst ∈ [0, ε).So CQFFV99 cannot be satisfied.




Example (cont.) II

On the other hand, considering u(t) = g(t) then CQI is satisfied.For any t ∈ (0, 1] we can find an (unique) k = 2n, n ∈ N such that

1

2k< t ≤ 1

k.

Now ∫ t

0

g(s)ds =

∫ 12k

0

g(s)ds +

∫ t

12k

g(s)ds.

It can be seen that the first term is equal to − 14k

and the second term isnegative. So, since ∫ t

0

g(s)ds ≤ − 1

4k≤ −1

4t,

CQI is satisfied.




The inequality in CQI just has to be satisfied on some, not all instants of timein the initial interval.

But, can we identify which are the instants of time in which it has to besatisfied?




Constraint Qualification at outward pointing velocities (F. Fontes, H.Frankowska 2015)

(CQd1out) ∃δ > 0, ∃u such that

hx(x0) · [f (t, x0, u)− f (t, x(t), u(t))] < −δ,

for a.e. t ∈ r ∈ [0, ε] ∪ [0, 1] : hx(x(r)) · f (t, x(r), u(r)) ≥ 0.

Remark

The inward pointing inequality just has to be satisfied for the times at which xhas an outward pointing velocity.




Constraint Qualification at outward pointing velocities(CQd1out) ∃δ > 0, ∃u such that

hx(x0) · f (t, x0, u) < −δ,

for a.e. t ∈ r ∈ [0, ε] ∪ [0, 1] :hx(x(r)) · f (t, x(r), u(r)) ≥ 0.

⇒(CQd1out) ∃δ > 0, ∃u such that

hx(x0)·[f (t, x0, u)− f (t, x(t), u(t))] < −δ,

for a.e. t ∈ r ∈ [0, ε]∪ [0, 1] : hx(x(r)) ·f (t, x(r), u(r)) ≥ 0.

Remark

When h is C 1, the constraint qualification CQ1out implies CQ2out .

Theorem3 If constraint qualifications CQout is satisfied, then a stronger (nondegenerate)version of the maximum principle holds with

µ(0, 1]+ ‖q‖L∞ + λ > 0.

3FACC Fontes, H Frankowska, ”Normality and Nondegeneracy for Optimal Control Problemswith State Constraints”, Journal of Optimization Theory and Applications: Volume 166, Issue 1(2015), Page 115-136



Outline



The Quest






Normality

CQn (Constraint Qualification for Normality)There exist a positive constants ε, δ, Ku, and a control u ∈ Usuch that

ζ · [f (t, x(t), u(t))− f (t, x(t), u(t))] < −δ, (3)

for all ζ ∈ ∂>x h(s, x(s)), all t, s ∈ (τ − ε, τ ] ∩ [0, 1] where τ is

defined as

τ = inf

t ∈ [0, 1] :

∫[t,1]

µ(ds) = 0

.

Theorem

Assume hypotheses ..., CQn and x(1) ∈ intC . Then, the maximum principle issatisfied with λ = 1. 4

4FACC Fontes, SO Lopes. ”Normal Forms of Necessary Conditions for Dynamic OptimizationProblems with Pathwise Inequality Constraints”. Journal of Mathematical Analysis andApplications, Vol.399 n. 1, pp.27-37, 2013



Constraint qualification for Normality

(CQnout) Define

τ = inf

t ∈ [0, 1] :

∫[t,1]

µ(ds) = 0

.

∃δ > 0, ∃u such that

hx(x(τ)) · [f (t, x(t), u)− f (t, x(t), u(t))] < −δ,

for a.e. t ∈ r ∈ [τ − ε, τ ] ∪ [0, 1] : hx(x(r)) · f (t, x(r), u(r)) ≥ 0.

Theorem5 If constraint qualifications CQnout is satisfied, then a stronger (normal)version of the maximum principle holds with λ+ |q(1)| 6= 0. In particular, ifx(1) ∈ intC , then λ = 1.

5FACC Fontes, H Frankowska, ”Normality and Nondegeneracy for Optimal Control Problemswith State Constraints”, Journal of Optimization Theory and Applications: Volume 166, Issue 1(2015), Page 115-136



Thank you!...and happy birthday

60th birthday, Porto



Some references

MMA Ferreira, FACC Fontes, and RB Vinter, Nondegenerate necessaryconditions for nonconvex optimal control problems with state constraints, J.Math. Anal. Appl. 233 (1999), no. 1, 116–129.

M. M. A. Ferreira and R. B. Vinter, When is the maximum principle for state

constrained problems nondegenerate?, J. Math. Anal. Appl. 187 (1994), no. 2,438–467.

SO Lopes, FACC Fontes, MdR de Pinho. ”On Constraint Qualifications forNondegenerate Necessary Conditions Of Optimality Applied to Optimal ControlProblems”. Discrete and Continuous Dynamical Systems series A, vol. 29, n. 2,2011.

FACC Fontes, SO Lopes. ”Normal Forms of Necessary Conditions for DynamicOptimization Problems with Pathwise Inequality Constraints”. Journal ofMathematical Analysis and Applications, Vol.399 n. 1, pp.27-37, 2013

SO Lopes, FACC Fontes, MdR de Pinho, An Integral-type ConstraintQualification to Guarantee Nondegeneracy of the Maximum Principle for OptimalControl Problems with State Constraints, Systems and Control Letters, vol. 62,pp. 686-692, 2013.

FACC Fontes, H Frankowska, ”Normality and Nondegeneracy for OptimalControl Problems with State Constraints”, Journal of Optimization Theory andApplications: Volume 166, Issue 1 (2015), Page 115-136.

Nondegenerate and Normal forms of the Maximum Principle for Control Problems …faf/preprints/S2015-london.pdf · 2015-09-08 · Nondegenerate and Normal forms of the Maximum Principle

Documents