Optimal Control, Guidance and Estimation...OPTIMAL CONTROL, GUIDANCE AND ESTIMATION Prof. Radhakant Padhi, AE Dept., IISc -Bangalore 7 Motivation Control energy is precious • Power

Lecture Lecture Lecture Lecture –––– 36363636

Constrained Optimal Control Constrained Optimal Control Constrained Optimal Control Constrained Optimal Control –––– III III III III

Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi

Dept. of Aerospace Engineering

Indian Institute of Science - Bangalore

Optimal Control, Guidance and Estimation

OPTIMAL CONTROL, GUIDANCE AND ESTIMATION

Prof. Radhakant Padhi, AE Dept., IISc-Bangalore2

Topics:

Constrained Optimal Control

� Motivation

� Pontryagin Minimum Principle

� Time Optimal Control of LTI Systems

• Time Optimal Control of Double-Integral System

� Fuel Optimal Control of LTI Systems

• Self-reading (Reference: D. S. Naidu, Optimal Control Systems, CRC Press, 2002, Chapter 7, pp.315-334).

� Energy Optimal Control of LTI Systems

� State Constrained Optimal Control

Summary of Summary of Summary of Summary of

PontryaginPontryaginPontryaginPontryagin Minimum PrincipleMinimum PrincipleMinimum PrincipleMinimum Principle






Objective

( )

( )

( )

0To find an "admissible" time history of control variable , , ,

where ( ) ( ) ,which:

1) Causes the system governed by , ,

to follow an admis

f

j j j

U t t t t

U t U u t U

X f t X U

− +

∈

≤ ≤ ≤

=

U

ɺ

or, component wise,

( ) ( )0

sible trajectory

2) Optimizes (minimizes/maximizes) a "meaningful" performance index

, , ,

3) Forces the system to satisfy "proper boundary conditions".

ft

f f

t

J t X L t X U dtϕ= + ∫



( ) ( ) ( )

( )

, , , ,

(i) State Equation:

Optimal Control Equation: Min

, ,

(ii) Costate Equation:

imize wi

(ii ) h i t

TH X U L X U f X U

HX f

H

t X U

H

X

λ λ

λ

λ

= +

∂ = =

∂

∂ = −

∂

Hamiltonian :

Necessary Conditions :

ɺ

ɺ

( ) ( )

( )

*

(iv) Boundary condi

repect to ( )

tions:

0

i.e. , , , ,

Specified, f

f

X

U t

H X U H X U

X

ϕλ

λ λ

∂= =

≤

∂

≤

U

Solution Procedure of a given Problem

Energy Optimal Control of LTI SystemsEnergy Optimal Control of LTI SystemsEnergy Optimal Control of LTI SystemsEnergy Optimal Control of LTI Systems






Motivation

� Control energy is precious

• Power in an electrical circuit: I2R = V2/R

� Control magnitude is restricted

• Control surface deflection should be as small as possible – to avoid saturation as well as to have sufficient margin for unexpected situations

• Rate of change of control is a function of energy drainage from the power source (battery) –smaller magnitudes usually lead to smaller rates



Problem Formulation

( ) ( )

( ) ( )0

0 : known, 0

where, and are state and control vectors

1 is either fixed or free

2

f

f

n m

t

T

f

t

X AX BU X t X t

X U

J U RU dt t

= + =

∈ ∈

= ∫

System Dynamics and Boundary Conditions :

Cost Functional :

Control Constraint :

ɺ

R R

1 or, component wise, 1, 1,2, , .j

U u j m≤ ≤ = ⋯



Problem Statement

( )0

The energy optimal control system is to transfer the

system from any initial state 0

towards the origin at time (either fixed or free) and,

at the same time, minimize the cost function

f

X AX BU X t

t

= + ≠ɺ

( )0

al

1

2

with the control constraints

1, 1,2, , .

ft

T

t

j

J U RU dt

u j m

=

≤ =

∫

⋯



( )

( )

( )

*

0 0

1, ,

2

(with Boundary Conditions)

,

, 0

Either is 'fixed' and is 'fre

T T T

T

f

f f

H X U U RU AX BU

HX AX BU X t X

HA t

X

t X

λ λ λ

λ

λ λ λ

= + +

∂ = = + =

∂

∂ = − = − =

∂

Step 1 : Hamiltonian

Step 2 : State and Costate Equations

Note :

ɺ

ɺ

e'

or, is 'free' and 0 f f

t X =

Energy Optimal Control System



Energy Optimal Control of LTI Systems

( ) ( ) ( )*

1

* *

, , , , min , ,

1

2

U

T T

H X U H X U H X U

U RU AX

λ λ λ

λ

≤≤ =

+

Step 3 : Optimal Condition

* 1

2

T T TBU U RU AXλ λ+ ≤ +

* * *

1

. .

1 1

2 2

1min

2

T

T T T T

T T

U

BU

i e

U RU BU U RU BU

U RU BU

λ

λ λ

λ≤

+

+ ≤ +

= +



( )

*

* 1

* * * 1 * *

* * * * *

* *

Let . Then

Hence, the optimality condition is

1 1

2 2

1Now, adding to both sides

2

T

T T T T T T

q

T T T T

T

q R B

BU U B U R R B U R q

U RU U Rq U RU U Rq

q Rq

λ

λ λ λ

−

−= = =

+ ≤ +

Step 4 : Optimal Control Computation

≜

��

* * * * * *

,

T T

U q R U q U q R U q + + ≤ + +





( )

( )

{ }

* 1

* * * * 1

* *

1

*

Then the optimality condition can be written as

min

. . attains its minimum value at .

T

T

T T T

U

W U q U R B

W U q U R B

W RW W RW W RW

i e W W

λ

λ

−

−

≤

+ = +

+ = +

≤ =

Define :

≜

≜




{ } { }

* *

* *

1 1

if and only if

provided is a positive definite matrix.

Hence, min is equivalent to min .

T T

T T

T T

U U

W RW W RW

W W W W

R

W RW W W≤ ≤

≤

≤

Fact from Convex Optimization :




{ } { }2

2 *

1 1 11 1

* *

* *

*

However,

min min min

Hence, the optimal control solution is:

if 1

1 if 1

1 if 1

j

m mT

j j jU U u

j j

j j

j j

j

W W w u q

q q

u q

q

≤ ≤ ≤= =

= = +

− ≤

= + < −

− > +

∑ ∑




( ){ }

{ }

{ } { }

* *

* * 1

if 1

sgn if 1

. .

i i

o i

i i

j j

T

f ff sat f

f f

u sat q

i e

U sat q sat R B λ−

≤=

>

= −

= − = −

Saturation function :

Control Solution :

≜



Energy Optimal Control System:

Some Observations

1. The energy-optimal control law described by

function, which is different from the function

for - control and - ( )

function for - control

saturation

signum

time optimal dead zone dez

fuel optimal

*

functions.

However, function is a " - function".

Hence, the minimum-energy control has no .

2. The optimal control is a function of time

which ag

sat well defined

singular solution

u continuous

ain is different from the -

functions of time for time-optimal and fuel-optimal problems.

piece wise constant




* 1 *

*

3. If there is on control,

then the control

Hence, if 1, the constrained and unconstrained

optimal control are the .

4. For the e

T

n

no constraint

unconstraint U R B q

q

same

constrained

λ−= − = −

≤

{ }* * 1 *

* *

nergy-optimal control system, using

optimal control, the state and costate system becomes

This is a set of 2 differential equations

and can only be so

T

T

X AX BSAT R B

A

n nonlinear

λ

λ λ

−= −

= −

ɺ

ɺ

lved by using numerical simulations.




Open-loop Implementation

0

*

Assume

Compute ( )

Evaluate

Solve for system trajectory ( )

Monitor ( ) and see if a : ( ) .

Then the control is Ti

j

f f

t

u

X t

X t t X t

λ

λ

•

•

•

•

• ∃ =

Adopt an iterative procedure

0

0

me-Optimal control.

If not, then change and repeat the procedure.λ•




Open-loop Implementation

Reference:

D. S. Naidu, Optimal Control Systems, CRC Press, 2002 (Chap. 7)




Closed-loop Implementation

{ }*Optimal control law ( )

However, an analytical/computational algorithm ( )

needs to be developed.

(Demonstrated through an example)

U sat h X

h X

=




A Scalar Example

( ) ( )

( )

2 2

0 0

2

System dynamics:

, 0

Performance index:

12

2

The final time is 'free' and 1

Hamiltonian:

, ,

f ft t

f

x ax u a

J u dt r u dt r

t u

H x u u ax uλ λ λ

= + <

= = =

≤

= + +

∫ ∫

ɺ




A Scalar Example

{ } ( ){ }

*

* 1

State equation:

, 0

Costate equation:

Optimal control:

/ 2

Hx a x u a

Ha

u sat r sat

λ

λ λλ

λ λ−

∂= = + <

∂

∂= − = −

∂

= − = −

ɺ

ɺSaturation function




A Scalar Example

( )

( )

( )

*

*

*

Hence,

1.0 if / 2 1 or 2,

1.0 if / 2 1 or 2,

0.5 if / 2 1 or 2,

1We note that the condition is also obtained from the results

2

of unconstrained control using th

u

u

λ λ

λ λ

λ λ λ

λ

+ ≤ − ≤ −

= − ≥ + ≥ +

− ≤ ≤ +

= −

* *

e Hamiltonian and the condition

10 2 0

2

Hu u

uλ λ

∂= → + = → = −

∂



( ) ( )

( )

{ } ( )*

0

0

0 , ( 0)

0 0 is not admissible because in that case

0 for 0, , and the state will

never reach the origin in finite time from the state 0.

In fact

at

at

f

f

t e a

u t t x t e x

t x

λ λ

λ

−= <

=

= ∈ =

≠

Solution of Costate :

Note :

( ) ( )

( )

( ) ( ) ( ) ( )( )

, in this case,

the costate 0 has in fact four possible

solutions, depending upon the initial value 0 ; . .

0 0 2, 0 2, 2 0 0, 0 2

att e

i e

λ λ

λ

λ λ λ λ

=

< < > − < < < −


A Scalar Example



( ) ( )

( ) ( )

*

1 0 0 2: for this case

1 1 or , 1

2 2

depending upon whether the system reaches the origin before

or after time , the function reaches the value of +2.

2 0 2: In this case, since 2,

a

u

t

λ

λ λ

λ

λ λ

< <

= − − −

> > + { }

( ) ( )

( )

*

*

the optimal control 1

3 2 0 0 : Depending on whether the state reaches the

origin before or after time , the function reaches the

1 1value 2, the optimal control is or , 1

2 2

4

c

u

t

u

λ

λ

λ λ

λ

= −

− < <

− = − − +

( ) { }*0 2 : Here, since 2, the optimal control 1uλ< − < − = +


A Scalar Example




A Scalar Example

Four possible solutions of costate

Reference:

D. S. Naidu,Optimal Control SystemsCRC Press, 2002 (Chap. 7)



( )

2

*

* *

* *

Here the Hamiltonian does not contain time explicitly.

Moreover, If the final time is , then:

, , 0 0,

This leads to:

0

1

This relationship can be analyze

f

f

t

t free

H x u t t

u ax u

u ux

a

λ

λ

λ

= ∀ ∈

+ + =

= +

−

d in detail to

come up with the closed form optimal control expression.

Scalar Example:

Closed Loop Implementation



( ) ( )

( )

( ) ( )

( )

*

*

(i) At time , 2, 1.

1Moreover,

2

(ii) At time , 2, 1.

1Moreover,

2

a a a

a

c c c

c

t t t u t

x ta

t t t u t

x ta

λ

λ

= = = −

=

= = − = +

= −

Saturated Region :

Scalar Example:




{ } ( )* 1

*2 *

/ 2

So, the Hamiltonian condition becomes

10

4

. . either 0 or 4

However, 0 is not admissible (shown before).

For 4 , the optimal co

u sat r

u ax u ax

i e ax

ax

λ λ

λ λ λ

λ λ

λ

λ

−= − = −

+ + = − =

= =

=

=

2. Unsaturated Region :

( )*

ntrol becomes

/ 2 2u axλ= − = −

Scalar Example:




( )

*

11, 0

2

11,

2

1 12 ,

2 2

: 0

if xa

u if xa

ax if xa a

Note a

− < + <

= + > −

− < < −

<

Scalar Example:


State Constrained Optimal Control:State Constrained Optimal Control:State Constrained Optimal Control:State Constrained Optimal Control:

Penalty Function Method






Problem Statement

( )

( )

( )

0

1 1 2

, , where, ,

, ,

, , , , 0

f

n m

t

t

n

X f X U t X U

J L X U t

g x x x t

= ∈ ∈

=

≥

∫

System Dynamics :

Performance Index :

Constraints :

ɺ

⋯

⋯⋯⋯

R R

( )1 2 , , , , 0,

The constraint functions have continuous

first and second partial derivatives with respect to .

p ng x x x t p n

X

≥ ≤

Assumption :

⋯




( ) ( ) ( ) ( ) ( )( )

( )( )

1

22

1 1 1 1

Convert inequality constraints to equality constraints

Define a new variable

, , ,

where, is a unit Heaviside step function

0, if ,. .

n

n n p p

i

i

i

x

x f X t g X t h g g X t h g

h g

g X ti e h g

+

+ + = + +

≥=

Idea :

ɺ ≜ ⋯

( )

( ) ( )1 0 1

0 for 1,2, ,

1, if , 0

Boundary conditions: 0.

i

n n f

i pg X t

x t x t+ +

=

≤

= =

⋯

This formulation makes it an infeasible problem,

unless all constraints are satisfied.

Note :




( ) ( ) ( ) ( )

( )

( )

1 1 1

1 1

1

, , , , , , , , ,

, ,

,

T

n n n

n n

n

H X U t L X U t f X U t f X t

HX f X U t

Hx f X t

H

X

λ λ λ λ

λ

λ

λ

+ + +

+ +

+

= + +

∂= =

∂

∂= =

∂

∂= −

∂

Hamiltonian :

State equation :

Costate equation :

ɺ

ɺ

ɺ

1

1

0n

n

H

xλ +

+

∂= − =

∂ɺ




Necessary Condition

( ) ( )

( ) ( ){ }

*

1 1

*

1 11

1

, , , , , , , ,

or,

, , , , min , , , ,

(1) 2 2 differential equations needs to be solved

to get the values for , ,

n n

n nU

n

H X U t H X U t

H X U t H X U t

n

X x

λ λ λ λ

λ λ λ λ

λ

+ +

+ +≤

+

≤

=

+

Optimal Control Condition :

Note :

( )

1, .

(2) Heaviside step functions are treated as 'constant functions'

in the costate equation while evaluating / .

n

H X

λ +

∂ ∂



Summary: Optimal control with

State Constraints

( )

( )

0

1 0

1

*

: Given

0

0

(obtained from Transversality condition)

f

n

n f

t

X

x t

x t

Ht

φ

+

+

=

=

∂ + ∂

Boundary Conditions for States :

Other Boundary Conditions :

i

i

i

i*

0

f

T

f f

t

t XX

φδ λ δ

∂ + − = ∂

State Constrained Optimal Control:State Constrained Optimal Control:State Constrained Optimal Control:State Constrained Optimal Control:

Slack Variable Method






Problem Statement

( )

( ) ( )

( )

0

th

, , where, ,

, , ,

, 0, where is of order

. . The c

f

n m

t

f ft

X f X U t X U

J X t L X U t dt

S X t S p

i e

ϕ

= ∈ ∈

= +

≤

∫

System Dynamics :

Performance Index :

Constraints :

ɺ R R

th

ontrol appears explicitly in the

order derivative of .

U

p S



Slack Variable Method(Also known as Valentine’s method; basic idea is due to F. A. Valentine)

( ) 2

Transforms Inequality constraint to equality constraint

by introducing a slack variable.

1. . , 0

2

Differentiating upto times with respect to

i e S X t

p t

α+ =

Idea :

( )

( )

( ) { }

1 1

2

2 1 2

1

, 0

, 0

, , terms involving , , , 0

where, subscripts on and denote the time derivatives.

p p

S X t

S X t

S X U t

S

αα

α αα

α α α

α

+ =

+ + =

+ =

…

⋯




( )

1

1

th

1

. .

.

Since is explicitly present in derivative,

one can solve for

, , , , ,p

dS S dX Si e S

dt X dt t

detc

dt

U p

U g X t

αα

α α α

∂ ∂ = = +

∂ ∂

=

= ⋯




( )

( ) ( )

( ) ( )

( ) ( )

1 1 0 0

ControlStates

1 0 0

1 2 1 0 1 0

1 1 0 1 0

Hence, the system dynamics can be written as:

, , , , , , , , ,

,

,

,

p p

p p p p

X f X g X t t X t t X

t t t

t t t

t t t

α α α α

α α α α

α α α α

α α α α

−

− − −

= = =

= = =

= = =

= = =

ɺ ⋯��

ɺ

ɺ

⋯⋯⋯⋯

ɺ

: Control variable p

α (unconstrained)




( )

( )

( )

( )

2

1 1

2

2 1 2

0

0 0

1We know: , 0

2

, 0

, 0

Hence, substituting ,

2 ,

S X t

S X t

S X t

t t

t g X

α

αα

α αα

α

+ =

+ =

+ + =

=

= ± −

Initial Conditions

⋯⋯

( )

( ) ( )

( ) ( ) ( )

0

1 0 1 0 0 0

2

2 0 2 0 0 1 0 0

,

,

t

t g X t

t g X t t

α α

α α α

= −

= − +

⋯⋯




( ) ( )( )0

1 1 , , , , , , , , ,ft

f f p pt

J X t L X g X t t dtϕ α α α α−= + ∫

Cost function :

⋯

( ) ( )

( ) ( )0

1 1

0 0

State Vector: , , , , , New Control:

System Dynamics:

, , , : Available

Cost Function:

, , ,f

T

p p

p

t

f f pt

n p

Z X

Z F Z t Z t Z

J Z t L Z t dt

α α α α

α

φ α

−

+

= =

= + ∫

New Problem (in dimension) :

≜ ⋯

ɺ



Slack Variable Method: Summary

( )

( ) ( )

1 1

0 0

, , , , ,

where, is -dimensional Lagrange multiplier

, , ,

T

p p

T

p

Z X

H L F

n p

Z F Z t Z t Z

α α α α

λ

λ

α

−

= +

+

= =

State Vector : Control :

Hamiltonian :

State Equation :

Costate Equation :

≜ ⋯

ɺ

( ) ( ) ( )

( )

( )

/ , /

/ 0

valid since is 'unconstrained'

f

T

f t

p

p

H Z t Z

H

λ λ ϕ

α

α

= − ∂ ∂ = ∂ ∂

∂ ∂ =

Optimal Control Equation :

ɺ



Thanks for the Attention….!!

Optimal Control, Guidance and Estimation...OPTIMAL CONTROL, GUIDANCE AND ESTIMATION Prof. Radhakant Padhi, AE Dept., IISc -Bangalore 7 Motivation Control energy is precious • Power

Documents