IntroductionMain results
ProofsFurther results
Optimal control of Hamilton-Jacobi-Bellmanequations
P. Jameson Graber
Commands (ENSTA ParisTech, INRIA Saclay)
17 January, 2014
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
MotivationBackground
Table of Contents
1 IntroductionMotivationBackground
2 Main resultsAssumptionsExistence of minimizersCharacterization of minimizers
3 ProofsDualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
4 Further resultsWell-posedness of weak solutionsLong time average behavior
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
MotivationBackground
Objective
We consider an optimal control problem:
Problem
(1) For a given cost functional K and finite measure dm0, find
inf
∫ T
0
∫K (t, x , f (t, x))dxdt −
∫u(0, x)dm0(x)
over functions u, f such that f ∈ C ([0,T ]× TN) and u solves
−∂tu(t, x) + H(x ,Du(t, x)) = f (t, x), u(T , x) = uT (x).
(2) Find a PDE characterization of (weak) minimizers.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
MotivationBackground
Motivation
Purpose: to steer an evolving set.Model: given a set A of admissible controls and a given regionΩT ⊂ RN , let Ωt be the backward reachable set at time t defined by
Ωt := y(t) : y = c(y , α), α ∈ A, y(T ) ∈ ΩT.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
MotivationBackground
Model
Strategy: use obstacles.Obstacle construction: Let f ≥ 0 be continuous. Define Kτ , τ ∈ [0,T ] by
Kτ := x : f (τ, x) > 0.
Consider the modified “blocked” evolving set
Ωt = y(t) : y = c(y , α), α ∈ A, y(T ) ∈ ΩT , y(τ) /∈ Kτ ∀ τ ∈ [t,T ].
Following [Bokanowski, Forcadel, Zidani 2010], we have acharacterization: let uT ≥ 0 (continuous) be such that uT = 0 preciselyon ΩT .
Ωt = y(t) : uT (y(T )) +
∫ T
t
f (s, y(s)) ≤ 0, y = c(y , α), α ∈ A.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
MotivationBackground
Model
Ωt = u(t, ·) ≤ 0 characterized by an optimal control problem:
u(t, x) := inf
uT (y(T )) +
∫ T
t
f (s, y(s))ds : y = c(y , α), α ∈ A, y(t) = x
.
u solves a Hamilton-Jacobi-Bellman equation:
−∂tu(t, x) + H(x ,Du(t, x)) = f (t, x), u(T , x) = uT (x).
Conclusion
Steering the front by obstacles is thus related to optimizing solutions toHJB equations.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
MotivationBackground
Related model
A related open problem:
Optimal control via velocity
Given a cost functional C, find
inf
C(c)−
∫u(T , x)dm0(x)
,
where the infimum is taken over u, c satisfying
ut + c(t, x)|Du| = 0, u(0, x) = u0(x).
Find a characterization of the minimizers.
Very little appears to be known.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
MotivationBackground
Table of Contents
1 IntroductionMotivationBackground
2 Main resultsAssumptionsExistence of minimizersCharacterization of minimizers
3 ProofsDualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
4 Further resultsWell-posedness of weak solutionsLong time average behavior
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
MotivationBackground
Mass transportation
In 2000, Benamou and Brenier considered a fluid mechanics formulationof the Monge-Kantorovich problem (MKP).For ρ0(x) ≥ 0 and ρT (x) ≥ 0 probability densities on Rd , we sayM : Rd → Rd transfers ρ0 to ρT if ∀A ⊂ Rd bounded,∫
x∈AρT (x)dx =
∫M(x)∈A
ρ0(x)dx .
The Lp MKP is to find
dp(ρ0, ρT )p = infM
∫|M(x)− x |pρ0(x)dx .
dp is called the Lp Kantorovich (or Wasserstein) distance.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
MotivationBackground
PDE characterization of MKP
Theorem (Benamou-Brenier, 2000)
The square of the L2 Kantorovich distance is equal to the infimum of
T
∫Rd
∫ T
0
ρ(t, x)|v(t, x)|2dxdt,
among all (ρ, v) satisfying
∂tρ+∇ · (ρv) = 0
for 0 < t < T and x ∈ Rd , and the initial-final condition
ρ(0, ·) = ρ0, ρ(T , ·) = ρT .
The optimality conditions are given by v(t, x) = ∇φ(t, x), where
∂tφ+1
2|∇φ|2 = 0.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
MotivationBackground
Dolbeault-Nazaret-Savare distances
In 2009, Dolbeault, Nazaret, and Savare, building on Benamou-Brenier,introduced a new class of transport distances between measures, given byminimizing ∫ 1
0
∫Rd
|v(t, x)|2m(ρ(t, x))dxdt
subject to
∂t +∇ · (m(ρ)v) = 0, ρ(0, ·) = ρ0, ρ(1, ·) = ρ1.
In 2013, Cardaliaguet, Carlier, and Nazaret studied geodesics for thisclass of distances when m(ρ) = ρα, getting optimality conditions ∂tρ+∇ · ( 1
2ρα∇φ) = 0,
ρ > 0⇒ ∂tφ+ α4 ρ
α−1|∇φ|2 = 0,ρ ≥ 0, ∂tφ ≤ 0, ρ(0, ·) = ρ0, ρ(1, ·) = ρ1.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
MotivationBackground
Mean Field Games
In 2007, Lasry and Lions give an overview of the following system ofequations:
∂u∂t − ν∆u + H(x ,∇u) = V [m] in Q × (0,T )
∂m∂t + ν∆m +∇ ·
(∂H∂p (x ,∇u)m
)= 0 in Q × (0,T )
u|t=0 = V0[m(0)] on Q, m|t=T = m0 on Q.
There are two characterizations of the system:
1 asymptotic behavior of differential games with large number ofplayers,
2 optimal control of Fokker-Planck equation, or of Hamilton-Jacobiequation (by duality).
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
AssumptionsExistence of minimizersCharacterization of minimizers
Table of Contents
1 IntroductionMotivationBackground
2 Main resultsAssumptionsExistence of minimizersCharacterization of minimizers
3 ProofsDualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
4 Further resultsWell-posedness of weak solutionsLong time average behavior
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
AssumptionsExistence of minimizersCharacterization of minimizers
Assumptions
Hamiltonian structure. H = H(x , p) is Lipschitz in the first variable:
|H(x , p)− H(y , p)| ≤ L|x − y ||p|.
H(x , ·) is subadditive and positively homogeneous, hence convex, withlinear bounds
c0|p| ≤ H(x , p) ≤ c1|p|.
H(x , p) := sup−c(x , a) · p : a ∈ ACost function. k = k(t, x ,m) continuous and strictly increasing with
1
C|m|q−1 − C ≤ k(t, x ,m) ≤ C |m|q−1 + C .
K∗(t, x , ·) a primitive of k, K(t, x , ·) its Fenchel conjugate. We willassume that K(t, x , 0) = 0 and that K(t, x , f ) is increasing in f for f ≥ 0.Note that
1
C|f |p − C ≤ K(t, x , f ) ≤ C |f |p + C .
Final-initial conditions. uT : TN → R is Lipschitz, m0 ∈ L∞(TN)
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
AssumptionsExistence of minimizersCharacterization of minimizers
Comparison with the literature
1 This is the first study with linearly bounded Hamiltonian.
2 In [Cardaliaguet, 2013] the Hamiltonian is supposed to satisfy
1
rC|p|r − C2 ≤ H(x , p) ≤ C
r|p|r + C
for some r > N(q − 1) ∨ 1 and, for some θ ∈ [0, rN+1 ),
|H(x , p)− H(y , p)| ≤ C |x − y |(|p| ∨ 1)θ.
The Hamiltonian H(x , p) = c(x)|p|r is forbidden by this restriction.By contrast, H(x , p) = c(x)|p| is permitted under our assumptions.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
AssumptionsExistence of minimizersCharacterization of minimizers
Existence theorem
Relaxed set
The set K will be defined as the set of all pairs (u, f ) ∈ BV × Lp suchthat u ∈ L∞, u(T , ·) = uT in the sense of traces, and−∂tu + H(x ,Du) ≤ f in the sense of measures, by which we mean thatf − H(x ,Du) + ∂tu is a non-negative Radon measure.
Relaxed problem: Find inf(u,f )∈KA(u, f ).
Lemma
The relaxed problem has the same infimum as the original.
Theorem (PJG)
There exists a minimizer to the relaxed problem.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
AssumptionsExistence of minimizersCharacterization of minimizers
Characterization of minimizers
Heuristically, use Lagrange multipliers to get system (MFG)
−∂tu + H(x ,Du) = k(t, x ,m(t, x)) (state equation)u(T , x) = uT (x)
∂tm − div (m∂pH(x ,Du)) = 0 (adjoint equation)m(0, x) = m0(x)
which characterizes minimizers (u, f ) of the relaxed problem (the optimalchoice is f = k(t, x ,m)).Problems:
H not differentiable,
weak solutions otherwise hard to define.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
AssumptionsExistence of minimizersCharacterization of minimizers
Definition of weak solutions
A pair (u,m) ∈ BV ((0,T )× TN)× Lq((0,T )× TN) is called a weaksolution to the MFG system if it satisfies the following conditions.
1 u satisfies∫TN
u(0)m0dx =
∫TN
u(t)m(t)dx +
∫ t
0
∫TN
k(s, x ,m)mdxds,∫TN
u(t)m(t)dx =
∫TN
uTm(T )dx +
∫ T
t
∫TN
k(s, x ,m)mdxds,
for almost all t ∈ [0,T ]. We also have−∂tu + H(x ,Du) ≤ k(t, x ,m) in the sense of measures.Moreover, u(T , ·) = uT in the sense of traces
2 m satisfies the continuity equation
∂tm + div (mv) = 0 in (0,T )× TN , m(0) = m0
in the sense of distributions, where v ∈ L∞((0,T )× TN ;RN) is avector field such that v(t, x) ∈ c(x ,A) a.e.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
AssumptionsExistence of minimizersCharacterization of minimizers
Well-posedness
Theorem (PJG)
(i) There exists a weak solution (u,m) of (MFG). Moreover, if u ∈W 1,p,then the Hamilton-Jacobi equation
−∂tu − v · Du = f
holds pointwise almost everywhere in m > 0, where v is the boundedvector field appearing in the definition.(ii) If (u,m) and (u′,m′) are both weak solutions (MFG), then m = m′
almost everywhere while u = u′ almost everywhere in the set m > 0.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Table of Contents
1 IntroductionMotivationBackground
2 Main resultsAssumptionsExistence of minimizersCharacterization of minimizers
3 ProofsDualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
4 Further resultsWell-posedness of weak solutionsLong time average behavior
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Smooth problem
Denote by K0 the set of maps u ∈ C 1([0,T ]× TN) such thatu(T , x) = uT (x). The “smooth problem” is to compute the infimumover K0 of the functional
A(u) =
∫ T
0
∫TN
K (t, x ,−∂tu + H(x ,Du))dxdt −∫TN
u(0, x)dm0(x).
Lemma
The “smooth problem” is equivalent to the original problem, i.e.
infu∈K0
A(u) = inff∈C([0,T ]×TN )
J (f ).
Proof: uses the superposition principle.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Dual problem
Define K1 to be the set of all pairs(m,w) ∈ L1((0,T )×TN)× L1((0,T )×TN ;RN) such that m ≥ 0 almosteverywhere, w(t, x) ∈ m(t, x)c(x ,A) almost everywhere, and
∂tm + div (w) = 0
m(0, ·) = m0(·)
in the sense of distributions. The “dual problem” is to compute theinfimum over K1 of
B(m,w) =
∫TN
uT (x)m(T , x)dx +
∫ T
0
∫TN
K∗(t, x ,m(t, x))dxdt.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Proof of duality
Lemma
The smooth and dual problems really are in duality, i.e.
infu∈K0
A(u) = − min(m,w)∈K1
B(m,w).
Moreover, the minimum on the right hand side is achieved by a pair(m,w) ∈ K1, of which m is unique, and which must satisfy the following:(t, x) 7→ K∗(t, x ,m(t, x)) ∈ L1((0,T )× TN) and m ∈ Lq((0,T )× TN).
Proof is an application of the Fenchel-Rockafellar duality Theorem. Let
X := C 1([0,T ]× TN ;R),
Y := C ([0,T ]× TN ;R)× C ([0,T ]× TN ;RN).
Let Λ : X → Y be given by Λ(u) = (∂tu,Du).
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Proof of duality (cont)
Lemma
infu∈K0 A(u) = −min(m,w)∈K1B(m,w).
Then define the functionals F : X → R and G : Y → R by
F (u) =
−∫TN u(0, x)dm0(x) if u(T , ·) = uT (·)
∞ otherwise
G (a,b) =∫ T
0
∫TN K (t, x ,−a + H(x ,b))dxdt
One can show the lemma is equivalent to
max(m,w)∈Y ′
−F ∗(Λ∗(m,w))− G ∗(−(m,w))
= − min(m,w)∈Y ′
F ∗(Λ∗(m,w)) + G ∗(−(m,w)).
Apply Fenchel-Rockafellar to finish.P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Table of Contents
1 IntroductionMotivationBackground
2 Main resultsAssumptionsExistence of minimizersCharacterization of minimizers
3 ProofsDualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
4 Further resultsWell-posedness of weak solutionsLong time average behavior
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Original and relaxed problems equivalent
Below we prove the following:
Proposition
inf(u,f )∈K
A(u, f ) = − min(m,w)∈K1
B(m,w).
Equivalently,inf
(u,f )∈KA(u, f ) = inf
u∈K0
A(u).
First part of the proof:Since for u ∈ K0 we have that (u,−ut + H(x ,Du)) ∈ K, it follows thatinf(u,f )∈KA(u, f ) ≤ infu∈K0 A(u).It remains to show other inequality.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
A technical lemma
Lemma
For u ∈ BV and f an integrable function, the following are equivalent:
−∂tu + H(x ,Du) ≤ f in the sense of measures, and
for every continuous vector field v = v(t, x) ∈ c(x ,A) we have
−∫ T
0
∫TN
φ∂tu + φv · Du ≤∫ T
0
∫TN
f φdtdx .
for every continuous function φ : [0,T ]× TN → [0,∞).
Remark: the crucial thing is that we only need to consider v continuous.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Proof of technical lemma
Lemma
−∂tu + H(x ,Du) ≤ f ⇔ −∫∫
φ(∂tu + v · Du) ≤∫∫
f φ ∀v cont.
One direction is obvious. For the other direction, let ε > 0. By Lusin’sTheorem, there is a compact set K with |Du|([0,T ]× TN \ K ) ≤ ε andDu/|Du| continuous on K . By continuity and convexity of c(·,A) weconstruct v(t, x) ∈ c(x ,A) by partition of unity so that−v · Du/|Du| ≥ H(x ,Du/|Du|)− ε. Use this to obtain the estimate∫ T
0
∫TN
φH(x ,Du) ≤ Cε+
∫ T
0
∫TN
φv · Du.
Then let ε→ 0.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Key lemma
Lemma
Suppose u ∈ BV ∩ L∞ satisfies −∂tu + H(x ,Du) ≤ f in the sense ofmeasures. Then for any m ∈ Lq satisfying the continuity equation∂tm + div(mv) = 0 for some vector field v with v(t, x) ∈ c(x ,A) a.e., wehave ∫
TN
u(0)m(0)dx ≤∫TN
u(t)m(t)dx +
∫ t
0
∫TN
fmdxds,∫TN
u(t)m(t)dx ≤∫TN
uTm(T )dx +
∫ T
t
∫TN
fmdxds,
for almost every t ∈ (0,T ).
Proof: obtain smooth approximations of m and mv through time scalingand convolution, apply previous lemma and integration by parts.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Proof of equivalence
Second half of the proof: Let (m,w) be a minimizer of the dual problem.It suffices to show that for (u, f ) ∈ K, we have A(u, f ) ≥ −B(m,w).This essentially follows from the last lemma. Indeed, we have
A(u, f ) =
∫ T
0
∫TN
K (t, x , f (t, x))dxdt −∫TN
u(0, x)m0(x)dx
≥∫ T
0
∫TN
fm − K∗(t, x ,m)dxdt −∫TN
u(0, x)m0(x)dx
≥ −∫ T
0
∫TN
K∗(t, x ,m)dxdt −∫TN
uT (x)m(T , x)dx = −B(m,w).
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Table of Contents
1 IntroductionMotivationBackground
2 Main resultsAssumptionsExistence of minimizersCharacterization of minimizers
3 ProofsDualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
4 Further resultsWell-posedness of weak solutionsLong time average behavior
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Upper bound lemma
Lemma
Suppose u is a continuous viscosity solution to the Hamilton-Jacobiequation
−∂tu + H(x ,Du) = f ≥ 0
in a region [0,T ]× U for some open set U in Euclidean space. Then forany 0 ≤ t < s ≤ T and for any 0 < β < 1, we have the followingestimate:
u(t, x)− u(s, y) ≤ C (1− β2)−N/2p‖f ‖p|t − s|α, ∀|x − y | ≤ βc0(s − t),
where α = 1− (N + 1)/p and the constant C depends on p,N, and c−10 .
Cf. [Cardaliaguet 2013]. This lemma gives us upper bounds.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Passing to the limit
Let (un, fn) be a minimizing sequence. WLOG, un and fn are Lipschitz,hence differentiable a.e., and fn ≥ 0. We have:
fn bounded in Lp by estimates on K , so fn → f weakly.
This implies un pointwise bounded.
The above implies H(x ,Dun) is bounded in L1, so Dun is as well.
The above implies ∂tun is bounded in L1, so un is bounded in BV .
(∂tun,Dun)→ (∂tu,Du) in the weak measure sense, while un → uin L1.
Conclusion: (u, f ) is the minimizer.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Table of Contents
1 IntroductionMotivationBackground
2 Main resultsAssumptionsExistence of minimizersCharacterization of minimizers
3 ProofsDualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
4 Further resultsWell-posedness of weak solutionsLong time average behavior
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Minimizers are weak solutions
Suppose (u, f ) ∈ K is a minimizer of relaxed problem and (m,w) ∈ K1 isa minimizer of dual problem.
We know that −∂tu + H(x ,Du) ≤ f in the sense of measures, so∫ t
0
∫fm +
∫u(t)m(t)− u(0)m0 ≥ 0,∫ T
t
∫fm +
∫uTm(T )− u(t)m(t) ≥ 0.
The main point is to get∫ T
0
∫fm +
∫uTm(T )− u(0)m0 = 0, so
that the above inequalities become equality.
Everything else comes directly from the definition of K and K1 andthe properties of minimizers already proved.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Minimizers are weak solutions, part 2
(u, f ) ∈ K is a minimizer of relaxed problem and (m,w) ∈ K1 is aminimizer of dual problem.By the duality theorem,
0 =
∫ T
0
∫K (t, x , f ) + K∗(t, x ,m) +
∫uTm(T )− u(0)m0
≥∫ T
0
∫fm +
∫uTm(T )− u(0)m0 ≥ 0,
which implies the desired result.It also implies K (t, x , f (t, x)) + K∗(t, x ,m(t, x)) = f (t, x)m(t, x)a.e. sof (t, x) = k(t, x ,m).
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Weak solutions are also minimizers
Suppose (u,m) is a weak solution. We set w = mv and f = k(·, ·,m).Then (u, f ) is a minimizer of the relaxed problem and (m,w) a minimizerof the dual problem. For example:
A(u′, f ′) =
∫∫K (t, x , f ′)−
∫u′(0)m0
≥∫∫
K (t, x , f ) + ∂f K (t, x , f )(f ′ − f )−∫
u′(0)m0
=
∫∫K (t, x , f ) + m(f ′ − f )−
∫u′(0)m0
=
∫∫K (t, x , f ) + mf ′ +
∫m(T )uT −m0u(0)− u′(0)m0
≥∫∫
K (t, x , f )−∫
m0u(0) = A(u, f ).
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
DualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
Weak solutions are unique
Suppose (u1,m1) and (u2,m2) are both weak solutions.
1 m1 and m2 are both minimizers of dual problem, so m1 = m2 =: m.
2 f := k(·, ·,m), then both (u1, f ) and (u2, f ) are minimizers ofrelaxed problem.
3 u := maxu1, u2; we show −∂tu + H(x ,Du) ≤ f in the sense ofmeasures.
4 Then (u, f ) is a minimizer of relaxed problem, so∫∫K (t, x , f )−
∫ui (0)m0 =
∫∫K (t, x , f )−
∫u(0)m0.
5 Combine with previous inequalities and use u ≥ ui to get u = ui inm > 0.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
Well-posedness of weak solutionsLong time average behavior
Table of Contents
1 IntroductionMotivationBackground
2 Main resultsAssumptionsExistence of minimizersCharacterization of minimizers
3 ProofsDualityEquivalence of original and relaxed problemsA priori boundsPassing to the limitExistence of weak solutionsUniqueness of weak solutions
4 Further resultsWell-posedness of weak solutionsLong time average behavior
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
Well-posedness of weak solutionsLong time average behavior
Recent work
The following is joint work with Pierre Cardaliaguet. We study (i) −∂tφ+ H(x ,Dφ) = f (x ,m)(ii) ∂tm − div (mDpH(x ,Dφ)) = 0(iii) φ(T , x) = φT (x),m(0, x) = m0(x)
and the corresponding ergodic problem (i) λ+ H(x ,Dφ) = f (x ,m(x))(ii) −div(mDpH(x ,Dφ)) = 0(iii) m ≥ 0,
∫Td m = 1
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
Well-posedness of weak solutionsLong time average behavior
Assumptions
1 (Initial-final conditions) m0 ∈ C (Td), m0 > 0, φT : Td → RLipschitz.
2 (Conditions on the Hamiltonian) H : Td × Rd → R is continuous inboth variables, convex and differentiable in the second variable, withDpH continuous in both variables. Also,
1
rC|p|r − C ≤ H(x , p) ≤ C
r|p|r + C .
3 (Conditions on the coupling) Let f be continuous on Td × (0,∞),strictly increasing in the second variable, satisfying
1
C|m|q−1 − C ≤ f (x ,m) ≤ C |m|q−1 + C ∀ m ≥ 1.
4 The relation holds between the growth rates of H and of F :
r > maxd(q − 1), 1.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
Well-posedness of weak solutionsLong time average behavior
Optimal control problemsOptimal control of HJ equation: find the infimum of
A(φ) =
∫ T
0
∫Td
F∗(x ,−∂tφ+ H(x ,Dφ))dxdt −∫Tdφ(0, x)dm0(x)
over the set of maps φ ∈ C1([0,T ]× Td ) such that φ(T , x) = φT (x).Optimal control of transport equation: find the infimum of
B(m,w) =
∫TdφTm(T )dx +
∫ T
0
∫Td
mH∗(x ,−
w
m
)+ F (x ,m)dxdt
over (m,w) ∈ L1((0,T )× Td )× L1((0,T )× Td ;Rd ), m ≥ 0 a.e.,∫Td m(t, x)dx = 1
a.e. t ∈ (0,T ), and∂tm + div (w) = 0, m(0, ·) = m0(·)
in the sense of distributions.
Theorem (Cardaliaguet, PJG)
The optimal control problems above are in duality, i.e.
infφ∈K0
A(φ) = − min(m,w)∈K1
B(m,w).
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
Well-posedness of weak solutionsLong time average behavior
Relaxed problem
K will be defined as the set of all pairs (φ, α) ∈ BV × L1 such thatDφ ∈ Lr ((0,T )× Td), φ(T , ·) ≤ φT in the sense of traces,α+ ∈ Lp((0,T )× Td), φ ∈ L∞((t,T )× Td) for every t ∈ (0,T ), and
−∂tφ+ H(x ,Dφ) ≤ α.
in the sense of distribution. Find
inf(φ,α)∈K
A(φ, α) = inf(φ,α)∈K
∫ T
0
∫Td
F ∗(x , α(t, x))dxdt−∫Td
φ(0, x)m0(x)dx .
Theorem (Cardaliaguet, PJG)
The relaxed problem has a minimizer. Moreover,
inf(φ,α)∈K
A(φ, α) = − min(m,w)∈K1
B(m,w) = infφ∈K0
A(φ).
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
Well-posedness of weak solutionsLong time average behavior
Definition of weak solution
A pair (φ,m) ∈ BV ((0,T )× Td)× Lq((0,T )× Td) is called a weaksolution if it satisfies the following conditions.
1 Dφ ∈ Lr and the maps mf (x ,m), mH∗ (x ,−DpH(x ,Dφ)) andmDpH(x ,Dφ) are integrable,
2 φ satisfies −∂tφ+ H(x ,Dφ) ≤ f (x ,m) in the sense of distributions,the boundary condition φ(T , ·) ≤ φT in the sense of trace and thefollowing equality∫∫
m (H(x ,Dφ)− 〈Dφ,DpH(x ,Dφ)〉 − f (x ,m)) dxdt
=
∫(φTm(T )− φ(0)m0)dx
3 m satisfies the continuity equation
∂tm − div (mDpH(x ,Dφ)) = 0 in (0,T )× Td , m(0) = m0
in the sense of distributions.P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
Well-posedness of weak solutionsLong time average behavior
Well-posedness
Theorem (Cardaliaguet, PJG)
(i) If (m,w) ∈ K1 is a minimizer of the dual problem and (φ, α) ∈ K is aminimizer of the relaxed problem, then (φ,m) is a weak solution andα(t, x) = f (x ,m(t, x)) almost everywhere.
(ii) Conversely, if (φ,m) is a weak solution, then there exist functionsw , α such that (φ, α) ∈ K is a minimizer of the relaxed problem and(m,w) ∈ K1 is a minimizer of the dual problem.
(iii) If (φ,m) and (φ′,m′) are both weak solutions, then m = m′ almosteverywhere while φ = φ′ almost everywhere in the set m > 0.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
Well-posedness of weak solutionsLong time average behavior
Main result
Let φf ∈ C 1 on Td .Let (φT ,mT ) be a “good” solution of −∂tφ+ H(x ,Dφ) = f (x ,m)
∂tm − div (mDpH(x ,Dφ)) = 0φ(T , x) = φf (x),m(0, x) = m0(x).
and (λ, φ, m) be a solution of the ergodic MFG system. Define
ψT (s, x) = φT (sT , x), µT (s, x) = mT (sT , x) ∀(s, x) ∈ (0, 1)×Td .
Theorem (Cardaliaguet, PJG)
As T → +∞,
(µT ) converges to m in Lθ((0, 1)× Td) for any θ ∈ [1, q),
ψT/T converges to the map s → λ(1− s) in Lθ((δ, 1)× Td) for anyθ ≥ 1 and any δ ∈ (0, 1).
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations
IntroductionMain results
ProofsFurther results
Well-posedness of weak solutionsLong time average behavior
Thank you!
P. Cardaliaguet and P. Graber, “Mean field games systems of firstorder,” submitted to ESAIM: Control, Optimization, and Calculus ofVariations. Available on the arXiv.
P. Graber, “Optimal control of first-order Hamilton-Jacobi equationswith linearly bounded Hamiltonians,” to appear in AppliedMathematics and Optimization. Also available on the arXiv.
P. Jameson Graber Optimal control of Hamilton-Jacobi-Bellman equations