Some Aspects of the Interplay Between Convex Analysis and Monotone Operator Theory Patrick L. Combettes Laboratoire Jacques-Louis Lions Facult · e de Math · ematiques Universit · e Pierre et Marie Curie Paris 6 75005 Paris, France Sestri Levante, 12 settembre 2014 Patrick L. Combettes Convex Functions and Monotone Operators 1/17
33
Embed
Some Aspects of the Interplay Between Convex Analysis …plc/sestri/combettes2014.pdf · Some Aspects of the Interplay Between Convex Analysis and Monotone Operator Theory Patrick
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Some Aspects of the Interplay BetweenConvex Analysis and Monotone Operator
Theory
Patrick L. Combettes
Laboratoire Jacques-Louis LionsFaculte de Mathematiques
Universite Pierre et Marie Curie – Paris 675005 Paris, France
Sestri Levante, 12 settembre 2014
Patrick L. Combettes Convex Functions and Monotone Operators 1/17
Overview
Our objective is to discuss certain aspects of the impor-tance of the theory of monotone operators and of non-expansive operators in the analysis and the numerical so-lution of problems in inverse problems and learning theory,even when those admit a purely variational formulation.
Special emphasis is placed on the role played by duality.
Patrick L. Combettes Convex Functions and Monotone Operators 2/17
Duality in Hilbert spaces
Duality-closed classes of objects:Closed vector subspaces: V → V⊥ (Frechet?)
Closed convex cones: K → K (Fenchel; ↑ : K = V )Lower semicontinuous proper convex functions:f → f ∗ (Fenchel, Moreau; ↑ : f = ιK )Maximally monotone operators: A→ A−1 (Moreau;↑ : A = ∂f )Firmly nonexpansive operators: J → Id− J ↑ : (Minty;l J = (Id + A)−1)Nonexpansive operators: R → −R (l R = 2J − Id)
Patrick L. Combettes Convex Functions and Monotone Operators 3/17
Duality in Hilbert spaces
Duality-closed classes of objects:Closed vector subspaces: V → V⊥ (Frechet?)Closed convex cones: K → K (Fenchel; ↑ : K = V )
Lower semicontinuous proper convex functions:f → f ∗ (Fenchel, Moreau; ↑ : f = ιK )Maximally monotone operators: A→ A−1 (Moreau;↑ : A = ∂f )Firmly nonexpansive operators: J → Id− J ↑ : (Minty;l J = (Id + A)−1)Nonexpansive operators: R → −R (l R = 2J − Id)
Patrick L. Combettes Convex Functions and Monotone Operators 3/17
Duality in Hilbert spaces
Duality-closed classes of objects:Closed vector subspaces: V → V⊥ (Frechet?)Closed convex cones: K → K (Fenchel; ↑ : K = V )Lower semicontinuous proper convex functions:f → f ∗ (Fenchel, Moreau; ↑ : f = ιK )
Maximally monotone operators: A→ A−1 (Moreau;↑ : A = ∂f )Firmly nonexpansive operators: J → Id− J ↑ : (Minty;l J = (Id + A)−1)Nonexpansive operators: R → −R (l R = 2J − Id)
Patrick L. Combettes Convex Functions and Monotone Operators 3/17
Duality in Hilbert spaces
Duality-closed classes of objects:Closed vector subspaces: V → V⊥ (Frechet?)Closed convex cones: K → K (Fenchel; ↑ : K = V )Lower semicontinuous proper convex functions:f → f ∗ (Fenchel, Moreau; ↑ : f = ιK )Maximally monotone operators: A→ A−1 (Moreau;↑ : A = ∂f )
Firmly nonexpansive operators: J → Id− J ↑ : (Minty;l J = (Id + A)−1)Nonexpansive operators: R → −R (l R = 2J − Id)
Patrick L. Combettes Convex Functions and Monotone Operators 3/17
Duality in Hilbert spaces
Duality-closed classes of objects:Closed vector subspaces: V → V⊥ (Frechet?)Closed convex cones: K → K (Fenchel; ↑ : K = V )Lower semicontinuous proper convex functions:f → f ∗ (Fenchel, Moreau; ↑ : f = ιK )Maximally monotone operators: A→ A−1 (Moreau;↑ : A = ∂f )Firmly nonexpansive operators: J → Id− J ↑ : (Minty;l J = (Id + A)−1)
Nonexpansive operators: R → −R (l R = 2J − Id)
Patrick L. Combettes Convex Functions and Monotone Operators 3/17
Duality in Hilbert spaces
Duality-closed classes of objects:Closed vector subspaces: V → V⊥ (Frechet?)Closed convex cones: K → K (Fenchel; ↑ : K = V )Lower semicontinuous proper convex functions:f → f ∗ (Fenchel, Moreau; ↑ : f = ιK )Maximally monotone operators: A→ A−1 (Moreau;↑ : A = ∂f )Firmly nonexpansive operators: J → Id− J ↑ : (Minty;l J = (Id + A)−1)Nonexpansive operators: R → −R (l R = 2J − Id)
Patrick L. Combettes Convex Functions and Monotone Operators 3/17
Problems in duality
Functional setting (Fenchel, Moreau): f + g → f ∗∨ + g∗
Until recently, general splitting methods lacked for thecomposite problem
0 ∈ Ax + L∗BLx
Patrick L. Combettes Convex Functions and Monotone Operators 7/17
Splitting methods: Composite problems
L : H → G linear and bounded, A : H → 2H, B : G → 2G
maximally monotoneSolve
0 ∈ Ax + L∗BLx
Main issue: 3 objects (A,B, L) to split... and a binaryrelation ∈ binding themWe need to reduce a problem to a 2-object problemin a larger spaceKey: recast the problem in the primal dual space
Patrick L. Combettes Convex Functions and Monotone Operators 8/17
Kuhn-Tucker set of a composite inclusion
Primal solutions: P ={
x ∈ H | 0 ∈ Ax + L∗BLx}
Dual solutions: D ={
v∗ ∈ G | 0 ∈ −L ◦ A−1(−L∗v∗) + B−1v∗}
Kuhn-Tucker set
Z ={
(x , v∗) ∈ H ⊕ G | −L∗v∗ ∈ Ax and Lx ∈ B−1v∗}
Z is a closed convex setZ 6= Ø⇔P 6= Ø⇔ D 6= ØZ ⊂P ×D
Patrick L. Combettes Convex Functions and Monotone Operators 9/17
Kuhn-Tucker set of a composite inclusion
Primal solutions: P ={
x ∈ H | 0 ∈ Ax + L∗BLx}
Dual solutions: D ={
v∗ ∈ G | 0 ∈ −L ◦ A−1(−L∗v∗) + B−1v∗}
Kuhn-Tucker set
Z ={
(x , v∗) ∈ H ⊕ G | −L∗v∗ ∈ Ax and Lx ∈ B−1v∗}
Z is a closed convex setZ 6= Ø⇔P 6= Ø⇔ D 6= ØZ ⊂P ×D
Patrick L. Combettes Convex Functions and Monotone Operators 9/17
Kuhn-Tucker set of a composite inclusion
Primal solutions: P ={
x ∈ H | 0 ∈ Ax + L∗BLx}
Dual solutions: D ={
v∗ ∈ G | 0 ∈ −L ◦ A−1(−L∗v∗) + B−1v∗}
Kuhn-Tucker set
Z ={
(x , v∗) ∈ H ⊕ G | −L∗v∗ ∈ Ax and Lx ∈ B−1v∗}
Z is a closed convex setZ 6= Ø⇔P 6= Ø⇔ D 6= ØZ ⊂P ×D
Patrick L. Combettes Convex Functions and Monotone Operators 9/17
Kuhn-Tucker set of a composite inclusion
Strategy: find a Kuhn-Tucker pair (x , v∗) by apply-ing a standard (e.g., Douglas-Rachford or forward-backward-forward) method to a monotone+skew de-composition of the problem in H⊕ G.[
00
]∈[A 00 B−1
]︸ ︷︷ ︸
M
[xv∗
]+
[0 L∗
−L 0
]︸ ︷︷ ︸
S
[xv∗
]
L. M. Briceno-Arias and PLC, A monotone+skew splittingmodel for composite monotone inclusions in duality, SIAM J.Optim., vol. 21, 2011.
Even in minimization problems, such a framework can-not be reduced to a functional setting: monotone op-erator splitting is required!Possible limitation: linear inversions (in DR) or knowl-edge of ‖L‖ (in FBF) are necessary
Patrick L. Combettes Convex Functions and Monotone Operators 10/17
Kuhn-Tucker set of a composite inclusion
Strategy: find a Kuhn-Tucker pair (x , v∗) by apply-ing a standard (e.g., Douglas-Rachford or forward-backward-forward) method to a monotone+skew de-composition of the problem in H⊕ G.[
00
]∈[A 00 B−1
]︸ ︷︷ ︸
M
[xv∗
]+
[0 L∗
−L 0
]︸ ︷︷ ︸
S
[xv∗
]
L. M. Briceno-Arias and PLC, A monotone+skew splittingmodel for composite monotone inclusions in duality, SIAM J.Optim., vol. 21, 2011.
Even in minimization problems, such a framework can-not be reduced to a functional setting: monotone op-erator splitting is required!Possible limitation: linear inversions (in DR) or knowl-edge of ‖L‖ (in FBF) are necessary
Patrick L. Combettes Convex Functions and Monotone Operators 10/17
Revisiting the proximal point algorithm
A maximally monotone, (γn)n∈N ∈ ]0,+∞[N, zer A 6= Ø,
xn+1 = JγnAxn
Classical results by Brezis&Lions (1978):If∑
n∈N γ2n = +∞, then xn ⇀ x ∈ zer A
If A = ∂f (f ∈ Γ0(H)) and∑
n∈N γn = +∞, then xn ⇀ x ∈zer A = Argmin f
Is the proximal point algorithm of any use?
Patrick L. Combettes Convex Functions and Monotone Operators 11/17
Revisiting the proximal point algorithm
A : H → 2H maximally monotone, V be a closed vec-tor subspace of HAV : partial inverse of A w.r.t. V (Spingarn, 1983)
gra AV ={
(PV x + PV⊥u,PV u + PV⊥x) | (x,u) ∈ gra A}
Then (Spingarn, 1983):AV is maximally monotonez ∈ zer AV ⇔ (PV z,PV⊥z) ∈ gra Ap = JAV z ⇔ PV p + PV⊥(z − p) = JAz
Aside – Open problem 4: Let f ∈ Γ0(H) and V = {0}.Then (∂f )V = ∂f ∗, but how to define a partial conju-gate of f ? (in general (∂f )V is not a subdifferential)
Patrick L. Combettes Convex Functions and Monotone Operators 12/17
Revisiting the proximal point algorithm
A : H → 2H maximally monotone, V be a closed vec-tor subspace of HAV : partial inverse of A w.r.t. V (Spingarn, 1983)
gra AV ={
(PV x + PV⊥u,PV u + PV⊥x) | (x,u) ∈ gra A}
Then (Spingarn, 1983):AV is maximally monotonez ∈ zer AV ⇔ (PV z,PV⊥z) ∈ gra Ap = JAV z ⇔ PV p + PV⊥(z − p) = JAz
Aside – Open problem 4: Let f ∈ Γ0(H) and V = {0}.Then (∂f )V = ∂f ∗, but how to define a partial conju-gate of f ? (in general (∂f )V is not a subdifferential)
Patrick L. Combettes Convex Functions and Monotone Operators 12/17
Revisiting the proximal point algorithm
Back to the Kuhn-Tucker set
Z ={
(x , v∗) ∈ H ⊕ G | −L∗v∗ ∈ Ax and Lx ∈ B−1v∗}
Define H = H⊕ G, A : (x , y) 7→ Ax × By , andV =
{(x , y) ∈ H ⊕ G | Lx = y
}Apply the (monotone operator) proximal point algo-rithm to AV
M. A. Alghamdi, A. Alotaibi, PLC, and N. Shahzad, A primal-dual method of partial inverses for composite inclusions, Op-tim. Lett., March 2014.
Patrick L. Combettes Convex Functions and Monotone Operators 13/17
Revisiting the proximal point algorithm
TheoremSet Q = (Id + L∗L)−1 and assume that zer(A + L∗BL) 6= Ø. Let(λn)n∈N ∈ ]0, 2[
N, such that∑
n∈N λn(2 − λn) = +∞, let x0 ∈ H,v∗0 ∈ G, and set y0 = Lx0, u0 = −L∗v∗0 , and
Then xn−wn → 0, yn−Lwn → 0, un−rn+tn → 0, and v∗n−sn+Ltn → 0,(xn, v∗n )⇀ (x , v∗) ∈ K .
Patrick L. Combettes Convex Functions and Monotone Operators 14/17
A strongly convergent splitting method
We have seen that the Kuhn-Tucker set
Z ={
(x , v∗) ∈ H ⊕ G | −L∗v∗ ∈ Ax and Lx ∈ B−1v∗}
is a closed and convexThe previous methods require efficient linear inversionsschmes or knowledge of ‖L‖, neither of which may beavailable in many situations (e.g., domain decompo-sition methods). In addition they require stringent ad-ditional hypotheses to guarantee strong convergence(to an unspecified Kuhn-Tucker point)We present a method which alleviates all these limita-tions.
Aside – Open problem 5: in infinite-dimensional prob-lems is weak convergence relevant?
Patrick L. Combettes Convex Functions and Monotone Operators 15/17
A strongly convergent splitting method
We have seen that the Kuhn-Tucker set
Z ={
(x , v∗) ∈ H ⊕ G | −L∗v∗ ∈ Ax and Lx ∈ B−1v∗}
is a closed and convexThe previous methods require efficient linear inversionsschmes or knowledge of ‖L‖, neither of which may beavailable in many situations (e.g., domain decompo-sition methods). In addition they require stringent ad-ditional hypotheses to guarantee strong convergence(to an unspecified Kuhn-Tucker point)We present a method which alleviates all these limita-tions.Aside – Open problem 5: in infinite-dimensional prob-lems is weak convergence relevant?
Patrick L. Combettes Convex Functions and Monotone Operators 15/17
A strongly convergent splitting method
An abstract Haugazeau scheme:Let C be a nonempty closed convex subset of H andlet x0 ∈H. Iterate
for n = 0, 1, . . .⌊take xn+1/2 ∈H such that C ⊂ H(xn,xn+1/2)xn+1 = P
H(x0,xn)∩H(xn,xn+1/2
)x0
Suppose that, for every x ∈ H and every strictly in-creasing sequence (kn)n∈N in N, xkn ⇀ x ⇒ x ∈ C . Then(xn)n∈N is well defined and xn → PCx0.Strategy: Apply this principle to C = Z , construct xn+1/2by suitably choosing points in gra A and gra B
A. Alotaibi, PLC, and N. Shahzad, Best approximation fromthe Kuhn-Tucker set of composite monotone inclusions, J.Nonlinear Convex Anal., to appear.
Patrick L. Combettes Convex Functions and Monotone Operators 16/17
A strongly convergent splitting method
TheoremFix (x0, v∗0 ) ∈ H× G and ε ∈ ]0, 1[, and iterate for every n ∈ N