Partial gradient flows in mean field games and statistical learning by GANs Gabriel Turinici CEREMADE, Universit´ e Paris Dauphine Institut Universitaire de France Groupe de Travail Statistique Numerique Paris Dauphine May 20, 2019 Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 1 / 30
30
Embed
Partial gradient flows in mean field games and statistical ... · Outline 1 Gradient flows General introduction Gradient flows examples 2 Vaccination (mean field) games 3 Computing
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Partial gradient flows in mean field games andstatistical learning by GANs
Gabriel Turinici
CEREMADE, Universite Paris DauphineInstitut Universitaire de France
Groupe de Travail Statistique NumeriqueParis Dauphine May 20, 2019
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 1 / 30
4 MFG numerical schemes on metric spaces: theoretical results
5 GAN and equilibrium flows
6 Perspectives
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 2 / 30
Gradient flows: theory• F : Rd → R = a smooth convex function, x ∈ Rd ; gradient flow from x= a curve (xt)t≥0: x ′t = −∇F (xt) for t > 0, x0 = x .• Polish metric space (X , d), functional F : (X , d)→ R ∪ +∞:non-trivial defintion, huge litterature (cf. books by Ambrosio et al. ,Villani, Santambroggio).
• Euclidian space (under some regularity assumptions):
ddt F (xt) = 〈∇F (xt), x ′t〉 ≥ −
∣∣x ′t ∣∣ · |∇F | (xt) ≥ −12∣∣x ′t ∣∣2 − 1
2 |∇F |2 (xt),
or equivalently ddt F (xt) + 1
2∣∣x ′t ∣∣2 + 1
2 |∇F |2 (xt) ≥ 0,
with equality only if x ′t = −∇F (xt).• Conclusion: d
dt F (xt) + 12 |x′t |
2 + 12 |∇F |2 (xt) ≤ 0 a.e. is equivalent with
x ′t = −∇F (xt).
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 3 / 30
Gradient flows: theory
• Euclidian space formulation: ddt F (xt) + 1
2 |x′t |
2 + 12 |∇F |2 (xt) ≤ 0 a.e.
• the (local metric) slope of F at x :|∇F | (x) = lim sup
z→x[F (x)−F (z)]+
d(x ,z) = max
lim supz→x
F (x)−F (z)d(x ,z) , 0
.
Definition: let I an interval in R. A function f : I → (X , d) is absolutelycontinuous on I if for any ε > 0, there exists δ > 0 such that whenever afinite sequence of pairwise disjoint sub-intervals (xk , yk) of I withxk , yk ∈ I satisfies
∑k(yk − xk) < δ then
∑k d(f (yk)− f (xk)) < ε.
• the metric derivative of x at t: |x ′t | = limh→0d(xt+h,xt )|h| , exists a.e. as
soon as t 7→ xt is absolutely continuous. Moreover |x ′| ∈ L1(0, 1).
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 4 / 30
Gradient flows: theory
• EDI ∇-flow (pointwise): ddt F (xt) + 1
2 |x′t |
2 + 12 |∇F |2 (xt) ≤ 0 a.e.
• EDI ∇-flow from x : an absolutely continuous curve such that:
∀s ≥ 0, F (xs) + 12
∫ s
0
∣∣x ′r ∣∣ dr + 12
∫ s
0|∇F |2 (xr ) dr ≤ F (x),
a.e. t > 0, ∀s ≥ t, F (xs) + 12
∫ s
t
∣∣x ′r ∣∣ dr + 12
∫ s
t|∇F |2 (xr )dr ≤ F (xt).
• EVI form for λ-convex (i.e., when smooth F ′′ ≥ λId ...) functionals:F (xt) + 1
2ddt d2(xt , y) + λ
2 d2(xt , y) ≤ F (y), ∀y , a.e. t ≥ 0.
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 5 / 30
X= P2(R) (the set of probability measures on (R,B(R)) with finitesecond-order moment, endowed with the Wasserstein distance W2)Consider for σ ∈ R F : P2(R)→ R ∪ +∞:F (ν) =
∫R V (x)ρ(x) + σ2
2∫R ρ(x) log(ρ(x))dx , if ν dx , ν = ρ(x)dx
F (ν) = +∞, if ν / dx .For smooth V , the gradient flow t 7→ ν(t) ∈ P2(R) of F satisfiesν(t) = ρ(t, ·)dx and:
∂ρ
∂t (t, x) = ∂
∂x [V ′(x)ρ(t, x)] + σ2
2∂2ρ
∂x2 (t, x), (1)
i.e., Fokker-Planck of the SDE: dX (t) = −V ′(X (t))dt + σdW (t).
Remark: also a L2 flow (term∫|∇ρ|2)...
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 6 / 30
Figure: Initial data for the heat flow (FP) model and its evolution (VIDEO).
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 7 / 30
Gradient flows examples : a 1D Patlak-Keller-Segel model
• the (modified) Patlak–Keller–Segel system (Perthame-Calvez-SharifiTabar 2007, Blanchet-Calvez-Carrillo 2008), is a PDE model fordiffusion-aggregation competition in biological applications (chemotaxis).
• Free energy functional:
G[ρ] =∫ρ(t, x) log(ρ(t, x)) dx + χ
π
∫ ∫ρ(t, x)ρ(t, y) log |x − y |dxdy
• the resulting Patlak-Keller-Segel equation:
∂ρ∂t = ∆ρ−∇(χρ∇c), t > O, x ∈ Ω ⊂ Rd (2)
c = − 1dπ log |z | ? ρ (3)
ρ = cell density, c = concentration of chemo-attractant, χ = sensitivityof the cells to the chemo-attractant.
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 8 / 30
Gradient flows examples: 1D Patlak-Keller-Segel model
-4 -2 0 2 4x
0
0.2
0.4
0.6
0.8
;E
VIE
(x)
t=0
;EVIE(x)
-5 0 5 10x
0
0.5
1
1.5
2
2.5
;E
VIE
(x)
t=0
;EVIE(x)
Figure: Initial data for the PKS model: χ = π (left), χ = 1.9π (right) and its evolution(VIDEO T = 2). Implementation : G. Legendre; ∇-flow JKO PKS code : courtesy A. Blanchet.
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 9 / 30
Gradient flows: the JKO scheme
• Jordan, Kinderlehrer and Otto ’98, (JKO) numerical scheme: time step= τ > 0, x τ0 = x ∈ X , by recurrence x τn+1 = a minimizer of the functional
x 7→ PJKOF (x ; x τn , τ) := 1
2τ d2(x τn , x) + F (x). (4)
• If X= Hilbert, F = smooth, JKO = implicit Euler (IE) scheme, i.e.,xτ
n+1−xτn
τ = −∇F (x τn+1).• JKO scheme was initially used theoretically to prove the existence of agradient flow
• similar schemes (VIM, EVIE) were proposed to compute numerically thesolution at second order in time (Legendre, G.T. 2017)
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 10 / 30
4 MFG numerical schemes on metric spaces: theoretical results
5 GAN and equilibrium flows
6 Perspectives
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 12 / 30
Disclaimer:
What follows is a THEORETICAL epidemiological investigation. It is notmeant to be used directly for health-related decisions; if in need to takesuch a decision please seek professional medical advice.
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 13 / 30
Vaccine scares: MFG modelsInfluenza A (H1N1) (flu) (2009-10)• At 15/06/2010 flu (H1N1): 18.156 deaths in 213 countries (WHO)• France: 1334 severe forms (out of 7.7M-14.7M people infected)
Countries Official target coverage Effective rate of vaccinationGermany 100 % 10%Belgium 100 % 6 %
Previous vaccine scares (some have been disproved since):• France: hepatitis B vaccines cause multiple sclerosis• US: mercury additives are responsible for the rise in autism• UK: the whooping cough (1970s), the measles-mumps-rubella (MMR)
(1990s).
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 14 / 30
Individual dynamics
Susceptible Infected Recovered
Vaccinated
−dV
−βSIdt −γIdt
Global dynamics : con-tinuous time determinis-tic ODE; is the masterequation of the individ-ual dynamics.
Susceptible Infected Recovered
Vaccinated
...
rate βI rate γ
Individual dynamics:continuous time Markovjumps between ’Sus-ceptible’, ’Infected’,’Recovered’ and’Vaccinated’ classes.
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 15 / 30
Summary for MFG vaccination models
• cost has the structure C(individual , societal); it is to be optimized withrespect to the ’individual ’ strategy, the ’societal ’ remains fixed, i.e.individual 7→ C(individual , societal);
• Nash / MFG equilibrium when ’individual ’ is unilaterally optimal and’societal ’= ’individual ’ (similar to a fixed point);
4 MFG numerical schemes on metric spaces: theoretical results
5 GAN and equilibrium flows
6 Perspectives
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 17 / 30
Computing the equilibrium: semi-explicit schemes onmetric spaces
• How to find and equilibrium ?• JKO converges to a ”benevolent planner” perspective.• semi-explicit numerical scheme for C(individual = ξI , global = ξG):Algorithm: set ξk = ξG
k = ξIk , and
ξk+1 ∈ argminη∈ΣN+1
dist(η, ξk)2
2∆τ + C(η, ξk).
• related to ”best reply” (MFG: cf. G. Carlier, A. Blanchet,...) and”fictitious play” (MFG: cf. P. Cardialiaguet et al.) learning methods ingame theory.
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 18 / 30
Figure: Notation: ξτ (t) is a time-dependent probability law over the possible vaccination timesindexed by variable t. Initial data ξτ=0(t) (uniform) and iterations (VIDEO) of the vaccinationMFG equilibrium strategy ξτ . Case: short persistence.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time
0
1
2
3
4
5
6
7
8
9
9
#10 -3
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 19 / 30
• give a meaning in a metric space to:∂τξ(τ, t) +∇1C(ξ(τ, t), ξ(τ, t)) = 0;• literature: ∇-flows for E (t, x): Ferreira-Valencia-Guevara ’15,Rossi-Mielke-Savare ’08, C. Jun ’12, Kopfer-Sturm ’16
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 21 / 30
• both are the limit of numerical schemes (under hyp.)
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 23 / 30
Theoretical results for the equilibrium flows
Bound when |∆| → 0 for∑
∆ C(ξk+1, ξk)− C(ξk , ξk) ?Finer division: what changes from x ,y when add z between ?C(y , x)− C(x , x)− [C(z , x)− C(x , x) + C(y , z)− C(z , z)]arg1
arg2
x
x
z
z
y
y
-
+
-
+ -
+
arg1
arg2
x
x
z
z
y
y
+
-
-
+
Requirement: C(x , y) + C(z , z)− C(z , y)− C(x , z) of order 2 ind(x , z) + d(z , y), formulated as:|C(x , y) + C(z , z)− C(z , y)− C(x , z)| ≤ CLd(x , z) · d(z , y), ok in bi-linearsetting.Right term only need to be of order O(d1+ε) or even an equivalent of uniform continuity ... asin Riemann sums.
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 24 / 30
4 MFG numerical schemes on metric spaces: theoretical results
5 GAN and equilibrium flows
6 Perspectives
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 25 / 30
Generative Adversarial Networks and equilibrium flows
~z ~xsyntheticG(~z)
generator / actor
pθ(~z)
~xrealpdata(~x)
~x real?D(~x)
discriminator / critic
Image credits : adapted from Petar Velickovic
• Notations: actor /generator θ → g(θ)→ Pθ• critic / discriminator (Wasserstein GAN formulation of Martin Arjovsky,Soumith Chintala, and Leon Bottou) : µ→ fµ• Functional 〈fµ,Pθ − Pr 〉 (1-Waserstein distance in dual form) to bemaximized with respect to µ in order to find (an approximation of)dW1(Pθ,Pr ). Then this is minimized with respect to θ.• WGAN formulation: nc steps of the SGD (or other stochastic descentlike Adam, RMSProp, ...) for µ and 1 step for θ.
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 26 / 30
Generative Adversarial Networks and equilibrium flows• F (µ, θ) = 〈fµ,Pθ − Pr 〉 =
∫fµ(x)Pθ(dx)−
∫fµ(x)Pr (dx)
• next critic µn+1 = arg minµ d(µ,µn)2
2τ − F (µ, θn)• next actor θn+1 = arg minθ d(θ,θn)2
2τ + F (µn+1, θ)• In particular d(µn+1,µn)2
2τ − F (µn+1, θn) ≤ −F (µn, θn)d(θn+1,θn)2
2τ + F (µn+1, θn+1) ≤ F (µn+1, θn)• We are accumulating the index:F (µn+1, θn)− F (µn, θn) + F (µn+1, θn)− F (µn+1, θn+1)• difference w/r to introducing a new point (µ′, θ′) between (µn, θn), and(µn+1, θn+1)
θ
µ
θn
µn
θn+1
µn+1
+- +
-
θ
µ
θn
µn
θn+1
µn+1
θ′
µ′
2- 2+
2+ 2-
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 27 / 30
Generative Adversarial Networks and equilibrium flows
• same if several steps are taken for the critic, as the same quantityaccumulates F (µn+1, θn)− F (µn, θn)• Assumption: similar to previously, i.e.,|F (µ2, θ1) + F (µ1, θ2)− F (µ1, θ1)− F (µ2, θ2)| ≤ CL · d(µ1, µ2) · d(θ1, θ2),ok in our bi-linear setting (distances Lip(·)× dW1).
Gabriel Turinici (CEREMADE & IUF) Equilibrium flows and GAN flows Paris Dauphine, May 20, 2019 28 / 30