Pergamon 0362-546X(94)00186-3 - Inria · 2019-09-16 · On a theorem of Danskin with an application to a theorem of Von Neumann-Sion 1165 A somewhat different issue was taken up by

Pergamon NonlinearAnatvsis, Theory, Methods & Applications, Vol. 24, No. 8, pp. 1163-1181, 1995

Copyright © 1995 Elsevier Science Ltd Printed in Great Britain. All rights reserved

0362-546X/95 $9.50 + .00

0362-546X(94)00186-3

O N A T H E O R E M O F D A N S K I N W I T H A N A P P L I C A T I O N T O A

T H E O R E M O F V O N N E U M A N N - S I O N

PIERRE BERNHARD and ALAIN RAPAPORT

INRIA-Sophia-Antipolis, 2004 route des lucioles, B.P.93, 06902 Sophia-Antipolis Cedex, France

(Received 23 November 1992; received in revised form 17 August 1993; received for publication 17 August 1994)

Key words and phrases: Minimax, differentration, convex analysis, saddlepoint.

1. I N T R O D U C T I O N

1.1. The problem considered and related work

In a book published in 1967 Danskin [1], proves the following theorem.

Hypotheses. Let V be a compact toplogical space, and J a map from A n x V into ~, assumed to be jointly continuous, and C 1 w.r.t, the first variable. Let

J(u) = max J(u ,v) , oEV

and

The theorem is as follows.

l~(u) = {v ~ V IJ(u, v) - J(u)}.

THEOREM 1 (Danskin). The function J has, for every u and h in R n a directional derivative at u in the direction h given by

DJ(u;h) = max ~ h i J i ( u , v ) , v~I~ i=1

where J~ stands for the partial derivative w.r.t, the component u i of u.

Let D~J(u, v; h) represent the directional partial derivative of J w.r.t, its first variable in the direction h, so that the above formula can be written

D](u; h) = max D1J(u, v; h). (1.1) v~l)

Since 1967, much work has been devoted to improve this result, or to related ones. There have been two main directions of research, one in the domain of convex analysis, and the other for nonconvex nondifferentiable functions.

Early work in the first area is described in Valadier's contribution [2]. A recent account can be found in Aubin and Ekeland [3] or Aubin [4]. This last reference, for instance, contains in the theorem 4.4, p. 53, exactly our theorem C1 below. Most of the literature has concentrated on the infinite dimension, as we do. However, there has been little work carried out on the

1163

1164 P. BERNHARD and A. RAPAPORT

compacity assumption. Valadier's work is a notable exception, and although rarely quoted, supersedes many later accounts. We applied his formula to the Von Neumann-Sion theorem. While Valadier needed to look at the subdifferential at neighboring points, as well as the generalized subdifferentials work quoted in the next paragraph, we propose instead a set of hypotheses with more regularity (mainly uniformity). (Furthermore, the simple "convex-concave" result we obtained, somewhat ad hoc for the application to the problem of Von Neumann, does not seem to have been pointed out before.)

Although early work needed the differentiable hypothesis (a form can be found, for instance, in [5, lemma 15.1, p. 53]) later work concentrated on the use of generalized (sub)differentials, such as Clarke's. A typical case of such results can be found in Rockafellar [6]. These results are always in finite dimensional spaces, and lead to estimations of a generalized subdifferential (generally Clarke's subdifferential) of the upper envelope, i.e. supersets, while we concentrate on exact expressions, in infinite dimension spaces. More importantly, this work proves the existence of a generalized subdifferential while we have results giving the existence of ordinary directional derivatives. The price to be paid is that our results require more regularity, (again, uniformity), and in particular do not deal in detail with infinite slopes, and singular subdifferentials. Again, we give results without compactness, particularly useful in the context of infinite dimensional spaces, that do not seem to have been considered before.

Related work on the so-called sensitivity optimization function, where what is sought is the derivative of a constrained max (or min) with respect to a variable occurring in the constraint of the optimization problem, should also be mentioned. Typically, the derivative or subdifferential of J(u) is defined by

or, more specifically

and

#(u) = {v ~ V I A(u , v) ~ 0},

#(u) = {v ~ V I A ( v ) <_ u},

J ( u ) = max J(v). v ~ #(u)

The two problems are very closely connected: one simple way of observing this is to rewrite the latter as follows. Let

Y(u, v) = J ( v ) - x¢~u) ( v ) ,

J(u) = max J(u, v) o E V

which is the form of problem considered. (Although this identification yields results in convex analysis it does not do so in differentiable analysis.)

Results on this problem most often fall into the second category above, typical examples being [7] and [8]. Let us quote also the earlier paper [9], which dealt with the more general problem where the variable u enters both the function to be maximized and the constraint. This paper falls into the category mentioned: finite dimension, and inclusions, very much in the spirit of those in the later papers quoted, which improve the results by weakening the regularity hypothesis, and by treating the singular subdifferential.

On a theorem of Danskin with an application to a theorem of Von N e u m a n n - S i o n 1165

A somewhat different issue was taken up by Bonnans [10], who was able to relax all qualification hypotheses for the constraint, at the cost of having directional derivatives in certain directions only. This is still in finite dimension.

Finally, let us quote the notable exception of [11], which deals with the same general problem as [9], obtaining estimations of generalized subdifferentials, including the singular case, but in infinite dimension. However, his results depended on a hypothesis on the regularity of the solution of the optimization problem, and the only sufficient conditions we know to satisfy this hypothesis have been stated in a finite dimensional space. (Although we could not check one of his references (Dolecki, "to appear").)

Finally, the link we show between these results and the Von Neumann-Sion theorems seems to be new.

This paper is based on a previous internal report [12]. A specialized version of the "differentiable" theorem can be found in [13, Chapter 9].

1.2. General framework

The following framework holds throughout the paper, and will not be repeated in the sequel.

U and V are subsets of a Banach space ~' and a topological space ~ , respectively. J is a mapping from U × V into R. The directional derivative of u ~ J(u, v) in a direction h of ~" is denoted by DiJ(u, v; h), and its subdifferential, in the case of a convex function, by O1J(u, v). Let

and, when it exists

] ( u ) = sup J(u,v) , (1.2) u E V

P(u) = {v ~ V lJ(u ,v) = ] ( u ) ) .

We will also need to consider

~ ( U ) = {{Vn} [J(u , Vn) -°ff(U) a s n ~ ~},

(1.3)

the set of maximizing sequences {v n} at u. Our aim is to characterize the directional derivatives DJ(u; h) of J, or, when it is convex, its

subdifferential OJ(u).

2. T H E D I F F E R E N T I A B L E C A S E

2.1. V compact

Firstly, a slightly improved version of theorem 1 above is stated.

Hypotheses D1. D1.0. V is compact. D1.1. Vv ~ V, the application (t, v) ~,J(u + th, v) is upper semi-continuous (u.s.c.) at

(0, v).


D1.2. Vv ~ V and Vt in a right neighborhood of 0, there exists a bounded directional derivative

D l ( u + t h , v ; h ) = lim l [ j ( u + ( t + ~ ) h , v ) - J ( u + t h , v ) ] . . r ~ O + T

D1.3. Moreover, the map (t, v) ~ D1J(u + th, v) is supper semi-continuous at (0, v).

THEOREM D1. Under hypotheses D.1, the function J has a directional derivative at u in the direction h, given by the formula

DJ(u;h)= max D1J(u,v;h) . u ~ t~(v)

Proof It should be noticed that by the assumption D1.3, the map v ~ DiJ(u, v; h) is u.s.c., so that, V being compact, the maximum is reached.

Let, for convenience,

A(t) = l [ j ( u + th) - ] ( u ) ] . (2.1) t

PROPOSITION 1. One has

lim inf A(t) >_ max D1J(u,v;h) . t~ 0 v ~ t~(u)

Proof of the proposition. Let 3 ~ l~(u). By definition, J ( u ) = J ( u , 3), and ](u + th)>_J(u + th, b ). Thus

A(t) > l [ j ( u + th,t3) - J (u , t3) ] .

Taking the lim inf,

lim inf A(t) > D1J(u, b), t~O

and since this holds for any b in l?(u), the proposition is proved.

PROPOSITION 2. Let {t~} be a sequence of real positive numbers, going to zero, and for all n, v~ ~ V(u + tnh). Then

v~ ~ l~(u), and J(u + t~h, Vn) -"~ ](U).

(The map t ~ ff'(u + th) is said to be u.s.c, at 0.)

Proof of the proposition. Proposition 1 implies that A(t n) is bounded below: 3a such that, for n large enough, A(t n) >_ a. Thus also

J(u + tnh) >__J(u) + at n.

Therefore, lim inf J(u + tnh) >_ ](u). Now, V is compact. Let, therefore, ~ be a cluster point of the sequence {vn}. One has

J(u) >_ J(u, F)) >_ lim sup J(u + t~ h, v~) >__ lim inf J(u + tn h, v,) >_ ] (u) .


The first inequality follows from the definition of ], the second one from the hypothesis DI.1, the last one from what has just been mentioned. Thus all inequalities are equalities, from which it can be concluded that ~ ~ 12(u), and the existence of the limit lim J(u + tnh, v n) = J(u).

PROPOSITION 3.

limsup A(t)_< max D1J(u,v;h). t~O v~D(u)

Proof of the proposition. With the same notations as in proposition 2, one has

1 [j(u,vn)_](u)] A(tn) = [J(u +t~h,v n) - J (u ,v . ) l + ~

By definition of ], the second term is nonpositive, hence

1 A(t,) < ~ [J(u + t,h,vn) -J(u,v , )] .

The function t ~J(u + th,%) having for all t ~ [0, t n] a bounded directional derivative, it is absolutely continuous, and there exists t', ~ [0, t n] such that

D1J(u + t',h, v n ; h) > ~ [J(u + t,h, v,) - J ( u , vn)],

hence A(t n) < D1J(u + t'nh, on; h). Due to the hypothesis D1.3, taking the lim sup

lim sup A(t n) < DtJ(u,-g; h),

where ~ is any cluster point of the sequence {vn}. Using Proposition 2, ~ ~ 12(u), and thus a fortiori the result claimed.

Finally, propositions 1 and 3 together prove the theorem. •

COROLARY If U ~J(u ,v) has a Ggteaux derivative J ' , and if the max is unique: l ) (u)= {b}, then J has a GSteaux derivative J'(u) given by the simple formula

]'(u) = J ' ( u , ~ ) .

Proof. It follows from Theorem 1 that, since D1J(u, v; h)=J'(u, v). h, then

D](u; h) =J~(u, b). h.

This equality proves the claim. •

2.2. Uniform case

The compacity hypothesis on V can be traded for more regularity on J, for instance in the following way (u and h are as in hypothesis D1).

Hypotheses D2. D2.1. The map u ~, J(u, v) is uniformly directionally differentiable in the following sense

W > O, 3~'> O: Vt ~ (0,~-), ¥ v e V ,

l [ j ( u + t h , v ) - J ( u , v ) ] - D 1 J ( u , v ; h ) <_E.


D2.2.

D2.3.

The directional derivative DiJ(u + th, v; h) is bounded in a right neighborhood of 0 in t, uniformly in v ~ V. The map t ~ D1J(u + th, v; h) is u.s.c, at 0, uniformly in v ~ V.

Remark. Hypothesis D2.1 and D2.3 may be lumped into any of the following two stronger hypotheses:

D2.a. The map u ~J(u ,v ) is uniformly directionally differentiable in the stronger following sense: for A > 0, write ux = u + Ah. The hypothesis reads

30>O: V~>O,37>O: V t~ (O,z), VA < O, V v ~ V,

l [ j (u~ - J(u~ DiJ(u ~, v; <_ E. + th 9V) h)

D2.b At point u, J has a second directional derivative with respect to its first variable in the direction h, uniformly bounded in v.

THEOREM D2. Under hypotheses D2, for all t in a left neighborhood of 0, there exits ](u + th) < 0% and ] has a directional derivative in the direction h, given by

D](u;h) = sup lim sup D1J(u,vk;h). {vk} ~ ~y/(u) k~oo

Remark. It could be agreed, with no ambiguity, to simply write the r.h.s, above as

D](u;h)= l imsup DaJ(U,Vk;h). {Vk}~r(u)

Proof. Let us call D the r.h.s, of the above equality, and let us define A(t) as in (1.1). In the sequel, it has been selected two sequences {t,} and {~n} of positive numbers such that tn -~ 0 and En/t, --* 0 as n ~ oo. (say, e.g. E, = t~).

PROPOSITION 1. lim inf A(t.) _> D.

(i)

and

Proof of the proposition. Let 6 be a positive integer. Choose N such that Vn > N,

en < t, 3 '

(ii) Vv ~ V, ~[J (u +t ,h ,v ) -J (u ,v ) ] >DiJ(u ,v;h) 3"

This is possible due to hypothesis D2.1. Let also {v k} ~ 7 f be a maximizing sequence at u,

V n , 3 K n : V k > K n, J ( u , v k ) > ] ( u ) - ~ ..

Hence, Vn > N, VK> Kn,

A(tn ) >_ ~-,1 [](u +tnh) -J (u ,vk ) ] - T~, > [J(u +tnh,V k) -J(u,vk)] 3

On a theorem of Danskin with an application to a theorem of Von Neumann-Sion 1169

By (ii), Vk > gn, A(t n) >_ D1J(u, v k) - 26/3. Let k go to infinity to conclude that

A(t,) > lira sup DiJ(u , Vk; h) - 26/3.

However, since {v k} is an arbitrary maximizing sequence, it may be chosen such that

iim sup D1J(U,Vk;h) > D - 6 /3 .

This way it gives Vn > N, A(t,) > D - 6, and this proves the proposition given that 6 was an arbitrary positive number.

PROPOSrrlON 2. Let {v,} be a sequence in V such that

V n > 0 , J ( u + t . h , v . ) > ] ( u + t . h ) - % . (2.2)

Then {v.} ~ 7f(u).

Proof of the proposition. One has

J(u) > J(u, %) > J(u + t .h , v n) - t .D lJ (u , %; h) - tnrl.,

where ft. ~ 0 by hypothesis D2.1. DIJ being bounded by the hypothesis D2.2, it also gives

J(u) > J ( u , v . ) > J(u + t . h , v . ) - 6., 6. ~ 0.

Making use of the definition (1.2) of the sequence {v.}, and of proposition 1 that implies that ](u + t .h) >_J(u) + 7. where 3'. ---' 0, finally gives

](u) > J ( u , v . ) >_](u) - 6. - % + 3". ~ ](u) ,

which proves the proposition.

PROPOSITION 3. lim sup A(t . )< D.

Proof of the proposition. The sequence {v.} is still as in (2.2). By definition, one has

En ~n fin A ( t . ) < ~ l [ J ( u + t . h , v . ) - J ( u ) ] + ~ < [ J ( u + t n h , v . ) - J ( u , v . ) ] + 7 . _ . (2.3)

As in the proof of theorem D1, by hypothesis D2.2, there exists t'. ~ [0, t.] such that

1 DIJ(u -t ' .h, v.;h) > ~ [J(u + t n h , v . ) -J(u,v . )] .

Moreover, making use of hypothesis D2.3, for n large enough

D l J ( u + t ' n h , V . ; h ) > D l J ( u , v . ; h ) + ~ ., ~1.~0,

so that, making further use of (2.3)

E n A(t.) <__DlJ(U,Vn;h) + ~ + rl. ,

and taking a lim sup

lira sup A(t,) < lim sup D l J ( u , v , ; h) <D.


This proves the proposition, because due to proposition 2, {v n} ~ 7f(u). Finally, propositions 1 and 3 together prove the theorem. •

3. THE CONVEX CASE

Versions in convex analysis of the preceding two theorems are now given. They are closely connected to them by the remark that for a convex function f, the map h ~ Df(u; h) is the support function of its subdifferential af(u). Thus the two theorems with compacity have identical conclusions. However, slight differences in the regularity requirements seem to prevent the "convex" theorems from being strict corollaries of the "differentiable" ones.

As it has been pointed out, the first theorem below is not new, (see [2, 4]). The proof given below is not as elegant as in these references. It has been chosen on the one hand to parallel the proofs in the differentiable case, and on the other hand to prepare the stage for the proof of the theorem without compacity, which seems to be original.

3.1. V compact

Hypotheses C. CO. V is (sequentially) compact in a topology for which Vu ~ U, the map v ~ J(u, v) is

U.S.C. C1. U is convex and Vv ~ V, the function u ~ J(u, v) is convex. Let us denote as O~lJ(U, u)

its subdifferential. C2. There exists u 0 e U, a neighborhood 0 of u 0 and a real number a such that

V(u,v)~ OX V, J(u,v)<a.

Let notice that hypothesis C2 implies that, Vfi ~ U, J(~) < a. Therefore, it will be introduced in the following definition.

Definition. Let U 0 be the interior of the subset of U, where J is finite.

LEMMA 1. In the presence of the hypothesis C1, the hypothesis C2 is equivalent to the following hypothesis C2a.

C2a Let u 0 ~ U. There exists a (bounded) neighborhood 0 of u 0 and a real number b such that

Vf i~O, V v~V , ~ /5~a l J ( f i , v ) with 11/511 _< b.

Proof of the lemma. Let us show that the hypothesis C2a and J (u 0) < ~ imply C2. Let 71 be such that fi ~ 0 imply I1~ -n i l _< 7. One has, V(fi, v ) ~ U x V, and with/5 E alJ(fi , v), chosen such that 11/511-< b,

hence

] ( u o) >_ J (uo , v ) > J( f i , v ) - ( p , ~ - u o)

J(fi, v) < J ( u o) + (/5, fi - u o) < J ( u o) + br/.

The converse is elementary, taking 0 in C2a strictly included in 0 in C2.


THEOREM C1. Under the hypothesis C, the function ] is convex continuous over U0, and its subdifferential at u ~ U 0 is given by the formula

M(u) =co I,J v e 9(u)

Proof. Let us first notice that being the upper envelope of a family of convex functions, ] is itself convex. According to C2, it is bounded in a neighborhood of u 0, and thus also over U 0, providing a uniform upperbound of J(u, v) in the neixghborhood of every point of U 0. Thus 3J and 01J exist over that set, and by compacity of V, V(u) exists, so that the above formula has a meaning.

Notice also that the classical proof of the continuity of a locally bounded convex function also proves the uniformity in v of the continuity of u ~ J ( u , v ) , since the upper bound is uniform. Then it easily follows, making use of CO, that the map (u, v ) ~ J ( u , v) is u.s.c.

PROPOSITION 1. One has

M(u) zco U a11(u,v). v ~ 9(u)

Proof of proposition 1. Let t3 ~ If(u), and p ~ 01J(u,b). Then

V w ~ U , J ( w , b ) > J ( u , b ) + ( p , w - u ) = J ( u ) + ( p , w - u ) ,

and thus

Vw U, J(w)>_J(u)+(p,w-u),

i.e. p ~ O](u). Since b was arbitrary in If, and p ~ 01J(u, ~)), we infer that OJ(u) includes the union of the subdifferentials OlJ. Finally, a subdifferential being convex, this proves the proposition.

PROPOSITION 2. Let h ~ U - u and t n ~ 0 + (or t, "~ 0) when n ~ oo, and v, ~ If(u + t,h). Then v, --, If(u).

Proof of, proposition 2. Since V is compact, the sequence v, has at least one cluster point ~. Let b ~ V(u). This gives

J (u ,~ ) > lim sup J(u + t , h , v , ) > lim inf J(u + t ,h ,b ) = J (u ,b ) = J (u) .

The first inequality because of the semicontinuity of J, the second by the definition of v,. The continuity of J in u and the definition of 3 give the two equalities. Therefore, J(u, ?))= J(u), and this proves the proposition.

PROPOSITION 3. Let h, t. and v. be as above, and p. ~ 01J(u + t .h, v~). There exists b ~ If(u) such that

lim sup(p~, h) _< sup ( p , h ) = D 1 J ( u , 3 ; h ) . p E C)l J ( u , t~)


Proof of the proposition. Let L = lim sup(p , ,h) , and Pm be a subsequence such that (pm,h )~L . Let also VmEff'(U+tmh), and again a subsequence with index k such that v k ~ b ~ if(u). Let us write D = D1J(u, b; h).

Let e > 0 be fixed. The slope [J(u+th ,b)-J(u ,b)] / t being, for a convex function, decreasing as t decreases to 0, this function has a directional derivative, and

:17> O:Vt < z, J ( u + t h , b ) < J ( u , b ) + t ( D + ~ ) . (3.1)

On the other hand, one always has

Vt, J (u+tkh+th ,vk)>_J(u+tkh ,Vk)+t(Pk ,h)>_J(u+tkh ,b)+t(pk ,h) .

Taking the lira sup, and taking into account the fact that J is u.s.c.,

J(u + th, b) >_J(u, ~) + t lim sup(pk, h) = J ( u , b) + tL.

Comparing this last inequality with (3.1), for t < T, it gives L < D + E, and this proves the proposition.

PROPOSITION 4.

sup (/~, h) < sup (p , h). p ~ aY(u) v ~ ~(u)

p ~ C~lJ(U,O)

Proof of the proposition. The subdifferential is a monotonous operator

Specifically,

Vfi, ~ O](u + t,h), Vp ~ J ] ( u ) , (fi , , ,h) > ( f i ,h ) .

inf (f in,h) > sup (/5,h). ~n E Of(u + tnh) ~E ~](u)

Moreover, making use of proposition 1 at u + tnh, it gives, with the same notations as above

inf (/3 n , h) < inf (pn, h). Pn E c~](u + tnh) Pn E c~lJ(u + tnh , Vn)

Regroup the two inequalities to obtain

sup ( p , h ) < ( p ~ , h ) Vp~ ~OlJ(u+t~h,v~). peM(u)

Making use of proposition 3, it can be infered that there exists b ~ l~(u) such that

sup (,b, h) < sup (p , h), f i e O](u) p E ~lJ(U, ~)

and afort ior i the inequality claimed. Now, proposition 4 implies the inclusion opposite to that proved in proposition 1, and the

two together prove the equality claimed in the theorem. •

3.2. The case without compacity

Let us adopt hypotheses C1, C2, and D2.1, where it is recalled that the directional derivative can be seen as the support of the subdifferential. It will be seen further what can be said without hypothesis D2.1, which is not very natural in this context.


The lemma 1 still holds, with the following precisions.

LEMMA 2. Hypothesis C2 is implied by C2a and the hypothesis that there exists ](u o) < ~. Moreover, the hypotheses C1 and C2 imply that, if u,-- . u ~ U 0, {vn} ~Tf(u) and pn 01J(u ~, v~), then lim sup IIp~ll < ~.

Proof of the lemma. The first claim has been proved in lemma 1. Let now u be fixed in U. Let us recall that by C2, J is continuous at u, uniformly in v. Let p > 0 be such that the ball B(u,2p) be included in/_7. Let h ~ U, with Llhll = p. For n large enough, u, + h ~ U, and thus

a >_J(u. + h, v.) >_J(u., v.) + (p . , h) >_J(u, vn) - i~. n +'(Pn, h),

where ~. goes to zero independently of v. due to the uniform continuity in u. Then, taking into account the fact that, by hypothesis, {v.} ~ 7f(u),

a >_](u) - 71~ - ~ + (pn,h),

where again, r/. - , O. Therefore,

( p . , h ) <_a - ] ( u ) + e. + rl~,

whence,

a - ] ( u ) lim sup Ilp~ll-< - -

P

In fact, otherwise, it could be chosen 8> 0 such that lim sup IIp~ll > ( a - ] ( u ) ) / p + 28, N such that for n >N, E, + ~% < 8p, and k > N such that IlPkll>(a - ] ( u ) ) / p + 8. Let then ~k ~ U of unit norm such that (pk,~k)= Ilpkll, taking h = P~k, one obtains a contradiction with the above inequality.

In order to simplify the statement of the next theorem, let us introduce the following natural definition.

Definition. Let 2/' be the topological dual space of ~', and {-~n} a sequence of subsets of ~". Let define lim sup ~n as the set of all limits in the weak-star topology of sequences {d~} of elements of -~n

oo oc

l i m s u p ~ = { d l 3 d ~ ~. .~:dn * ' d } = ['] U c 9 1 J ( u ' ° n ) " n~OC k= l n=k

(The closure operator in the last expression being in the sense of the weak-star topology.) It can now be stated the next theorem.

THEOREM C2. Under hypothesis C1 and C2, the function ] has at every u ~ U 0 a subdifferential given by the following formula

a Y ( u ) = c o

or, equivalently,

U lim sup a~J(u,v.) (v~} ~ ' ( u ) n ---,

ac oc

=co U N U {vn}ETf(u ) k = l n=k


(See another formulation after the proof.)

Proof. As in theorem C1, ] is convex continuous over U. Let us use the notation

_~ = [..J lim sup O1J(u , on). {v,} ~¢:(u) n--,

According to lemma 2, .~ is bounded.

PROPOSITION 1. OJ(u)z-~o.~.

Proof of the proposition. Let /3 ~ _~. By definition, there exist a maximizing sequence {v k}

and a sequence Pk ~ O1J(u, vk) such that Pk *'/5. It gives, Vh

](u + h) > J(u + h, v k) > J(u, v k) + (Pk, h) = ] ( u ) - e k + (Pk, h),

where e k ~ 0, whence, taking the limit

] (u + h) _>J(u) + (/5, h),

thus ~ c O](u), but as the latter is convex, the proposition is proved.

PROPOSITION 2. Let t. ~ 0, e. ~ 0, and v. such that J(u + t.h, v.) >](u + t.h) - e. and, finally, p. ~ OaJ(u + t.h, v.). Then {v.} ~ ~:¢:(u).

Proof of the proposition. One has

](u) >_J(u, v.) >_J(u + tnh, v n) - t . ( p . , h) >_](u + t.h ) - ~. - t . ( P n , h ) .

Let p ~ 0](u). (It has been seen that it is not empty.) Using it to upperbound the last occurrence of J above, it can be easily obtained

J(u) >_J(u, v.) > ](u) - t . (p , h) - e. - t . (p . , h).

By the lemma 2, p . is bounded, hence the proposition.

PROPOSITION 3. Let t., e., v. be as in proposition 2. Let, furthermore,

D = sup (/5,h) and D. = sup (Pn,h).

Then lira sup D. _< D.

Proof of the proposition. For all n, one can choose/5. ~ 01J(u,v .) such that D. >_ (/~., h ) > D. - e.. Thus

lim sup D n -- lim sup(/sn, h).

Extracting a subsequence {/~k} of {/5.} such that ( / s k , h ) ~ lim sup(/~.,h), and again a weak-star convergent subsequence converging to, say,/5 ~ _~, it gives

lim sup(/5., h) = (/5, h) >__ D.

which proves the proposition.


PROPOSITION 4. Let t n ~ O, and for each n, {v~} k ~ 7f(u + t,h). Let

_,On = lim sup 01J(u +t ,h ,v~) . k---~ oc

Then, if /5. ~-~., one has

lim sup( /5 , ,h) < D = sup (/5, h).

Proof of the proposition. Let Pn ~ "~n" Let us choose {v~} k ~ 7//'(u + t.h) and p~ ~ 01J(u + k) such that pn k --/5.. Let us also choose k n such that, for a fixed sequence En ~ 0, with t.h, v.

kn the notations v n = v n and pn k" = p . , the following holds

J ( u + t . h , v . ) > ] ( u + t . h ) - E . and I(pn - / 5 . , h ) l < e..

The sequence {v.} is as in proposition 2, and in particular is in ~ ( u ) . Moreover, for all a > 0,

J(u + t.h + ah,v~) > J(u + tnh,v~) + a ( p . , h ) > ](u + t.h) + t~( /5 . ,h) - 2e.,

holds, i.e. for/5 ~ 2 , hence .5 ~ O](u) according to proposition 1,

J(u + t.h + ah, vn) >](u) + t.(/5,h) - 2e. + a( /5 . , h).

On the other hand, let us set

D n = sup (~n,h). fine C)lJ(U, V.)

For every positive rl, there exists a positive a 0 such that, for every positive a smaller or equal to a 0, there holds

J(u + oth,v n) <J(u ,v n) + a(D, + ~7) <](u) + a(D, + 71).

Moreover, due to hypothesis D2.1, it may be picked a 0 independently of v,. (i.e. fixed as n ~ 2.) Since J is continuous in u, uniformly in v, for n large enough, one has

J(u + t,h + ah,vn) <J(u + ah ,v , ) + ~ . ,

whence, regrouping the last three inequalities,

3ao>O:Va~[O, ao], a (D ,+~?)>a( f~n ,h ) -3e~ .

Taking the limit, using proposition 3, to derive

D + r t> lim sup(/5,, h)

which proves the proposition, since ~ was arbitrary.

PROPOSITION 5. t3](u) C CO---~

Proof of the proposition. Let ~ c O](u). Since O] is a monotonous operator,

V ~ O ] ( u + t , h ) , (fi, h ) < ( ~ n , h ) .

Therefore, making use of proposition 1

(p ,h )_< inf ( p , , h ) < inf ( /5 , ,h) . ffn E O](u + tnh) Pn E-~n


Finally, taking the lim sup and making use of proposition 4,

V3 ~ O](u), ( p , h ) < sup (/~, h). p~__~

Thus,/5 ~ c o ~ , which proves the proposition. Finally, propositions 1 and 5 together prove the theorem. •

It is useful, at this point, to give an alternate form of the formula of theorem C2. Define the level sets at u, V,, in the following way.

Definition. Let E be a positive number, define

V, (u) = {v ~ V I J ( u , v ) > J ( u ) + ~}.

They are convex sets, increasing with ¢. When it exists, l~'(u) is just Vo(u). In terms of these sets, the formula of theorem C2 may be rewritten as follows

M(u)= 0 co t3 ad(u,v). E>0 v~V,

The above formulation is the natural one to state the result without the uniformity hypothesis D2.1. This theorem is proved in [2].

THEOREM C3 (Valdier). Under hypothesis C1 and C2, one has

M(u)= Nco U od(u,v), ~>0 vEV,

1) u~12

where 1-/ ranges over a complete set of neighborhoods of u.

4. THE CONVEX CONCAVE CASE

The following additional hypothesis will be made.

Hypothesis CC. V is a convex subset of a Banach space W, and V u ~ U, v ~ J (u , v) is concave.

Remark. In this case, if furthermore W is reflexive, in hypothesis CO the compacity of V may be replaced by V closed and bounded. Owing to v ~ J (u , v) being concave, its being u.s.c, is preserved in the weak topology.

The previous two theorems can be simplified in the following way.

THEOREM CC1. Under hypotheses C and CC, the subdifferential of J is given at any point u in U 0 by the formula

C~Y(U)'= U c91J(u,v)" v ~ P(u)

Proof. According to theorem C1, it suffices to prove the following proposition.

O n a t h e o r e m of D a n s k i n with an appl icat ion to a t h e o r e m of V o n N e u m a n n - S i o n 1177

PROPOSITION. -~ = U~ ~ ~(~) O~J(u, v) is convex.

Proof. For i = 1,2, let v~ ~ I~(u),and p~ ~ alJ(u , v). We know that l~(u) is convex, and thus VA ~ [0, 1], w = Av 1 + (1 - h)v 2 ~ V(u). Let us also set q = Apl + (1 - h)p2. Let h ~ U - u. Making use of hypothesis CC,

J(u + h,w) >_ AJ(u + h, v 1) + (1 - h ) J (u + h, O 2 ) ~ hJ(u, v 1) + (1 - h ) J (u , v 2) + (q, h).

In addition since, by definition, J(u, v~)= ](u),

J(u +h ,w) > J(u) + (q,h) =J(u ,w) + (q,h).

Thus q ~ aJ(u ,w) , where w ~ l~(u). •

THEOREM CC2. Under hypotheses C1, C2, and CC, the differential of ] is given at any point u in U0 by the formula

a](u) = [,.J lim sup alJ(u, v,). {v,,} ~ ( u ) n ~ : ~

Proof. Again, it suffices to prove that

. ~ = [,.J l imsup alJ(u,v .) (v.} e~¥r(u ) n - - , ~

is convex. For the sequel in this proof, the following notations will be used for i = 1, 2, let pi ~ ~ .

There exist {v~} k ~ ' ( u ) and pi k ~ 31J(u,v ~) such that p~ -- p ( For A ~ [0,1], let

w k = hv~ + (1 - h)v] , q = hp I + (1 - A)p 2, qk = hp~ + (1 -- h)p~.

P R O P O S I T I O N 1. {Wk} k ~Vf(U).

Proof of the proposition. By concavity, one has

J(u,wk ) >__ AJ(u,v~) + (1 - A)J(u,v~) --, ] ( u ) .

In addition since, by definition of J, J(u, w k) <_ ](u), J(u, w k) ~ J(u), which is the definition of {wAk ~ ~ ( u ) .

PROPOSITION 2. Let ~k be a sequence of convex subsets of ~/', and ~ = lim sup ~'k. Let D k = sup(pg, h) and D = sup(p, h), for Pk E ~'k and p ~ ~. Then:

(i) ~n is convex; (ii) l i m s u p D k_<D.

Proof of the proposition. The first item is elementary. The second one is the proposition 3 of the proof of theorem C2.

PROPOSITION 3. With the notations introduced for this proof, let ~k = t91J(U, Wk), and ~ = lim sup ~'k- Then q ~ ~.


Proof of the proposition. As in the previous theorem, Va > 0,

J(u + oth, w k) > AJ(u + ah,v~) + (1 - A)J(u + ah,v2) ,

i.e.

Vot, J(u + oth,w k) > AJ(u,v~) + (1 - A)J(u,v 2) + a(qk ,h) .

Making use of the uniform continuity of J, it can be infered that

Va, J ( u + a h , w k ) > J ( u ) - e k + a ( q k , h ) ,

where {E k} is a sequence decreasing to zero independently of a and h. On the other hand, Vh,V, /> 0, 3a 0 :Va ~ (0, a0),

J(u + olh,w k) < J ( u , w k) + a (D k + 7 l) <](u) + ot(D k + ~l).

Whence, comparing the last two inequalities, for ct < a0,

•k (qk,h) <Dk + rl+ - - Ol

and making use of proposition 2,

(q, h) = lim(qk, h) < D + ~.

Since r /was arbitrary, it can be concluded that (q, h) < D and, since, according to proposition 2, ~' is convex, the proposition is proved.

Finally, since ~ c ~ , q --- Apl + (1 - A)p2 ~ . ~ , and the theorem is proved. •

5. A P P L I C A T I O N T O T H E V O N N E U M A N N - S I O N T H E O R E M

It is shown here that classical theorems of the existence of a saddle point, or at least of a value, (inf sup = sup inf), to a convex-concave function, are simple consequences of the above theorems.

The first theorem below is often called "Von Neumann's theorem", although Von Neu- mann [16] himself only treated the case needed for matrix games, i.e. where U and V are simplices in Euclidean space, and J is linear. Sion [14] credits Shiffman for a more general form. The second theorem below is often called "Sion's theorem", although Sion credits Kneser and Fan for it. In [14], Sion gives a rather complete, and more general treatment of that question. An elegant theory can be found in [4].

Our hypotheses are similar to those of the previous section. We state them anew adapted to the present aim.

Hypotheses VN. VN1. U is convex compact, contained in an open subset U c ~', and V v ~ V, the function

u ~ J ( u , v ) is convex 1.s.c. from 0 into •. Furthermore, J is bounded above, uniformly in v, in a neighborhood of any point of U in U.

VN2. V is convex, and Vu ~ U, the function v ~ J ( u , v ) is concave. VN3. V is (sequentially) compact, and Vu ~ 0, the function v ~ J(u, v) is u.s.c.


THEOREM VN1. Under hypotheses VN1 to VN3, the function J has a saddle point over U x V, i.e. there exist fi ~ U and b ~ V such that

V(u ,v )~U×V, J(~,v)<J(~,b)<_J(u,b).

Remark. The existence of a saddle point implies that

rain max J(u,v) = max rain J(u,v) = J ( f i , b ) . u~U v~V v ~ V u~U

Proof. Theorem CC1 applies. In particular, hypothesis VN1 insures that U c U0, where ] is continuous, therefore, 1.s.c. (even in the weak topology if necessary). It reaches its minimum at a point fi ~ U. There exists thus /3 c 0](t~) such that

Vu~U, (~,u-~)>_O.

Making use of theorem CC1, there exists b E l?(fi) such that /3 ~ alJ(fi , b). Whence

J(fi,b) <J(u ,b) - (~,u - ~).

Remembering that b ~ l~(fi), one can take the left hand inequality of the saddlepoint, and with the above two inequalities, the right hand one. •

THEOREM VN2. Under hypotheses VN1 and VN2, there exists fi ~ U such that

sup J ( f i ,v ) = rain sup J(u,v) = sup rain J(u,v). v ~ V uEU v ~ V v ~ V uEU

Proof. The proof makes use of Valadier's formula. Let us first notice that again, hypothesis VN1 insures the existence of the minima in u. In

particular, ] has a minimum at a point ~ c U. Let

f = ] ( f i ) = min sup J ( u , v ) . u~U ~,~V

] being convex, there exists/) ~ O](fi) such that

Vu ~ U, ( / L u - fi) >_ 0.

It will be exhibited a sequence {w k} ~ V such that

VE>0, :iN, minJ(u,Wu)>f--e. u~U

Then, one can conclude

sup min J(u,v) > f. u ~ V u~U

Let e k be a decreasing sequence of positive numbers, and consider D~ c U and V k c V such that

V U E ~ ' ~ k , V v E V , J ( u , v ) > _ J ( u , v ) - f.k,

= = {v e v / 1 ( a , >_Y-


Such l)ks exist because u~ ,J (u ,v ) is l.s.c, uniformly in v, and any sequence { U k } ~ ' ~ k

converges to t~. Due to the definition of the level sets, any sequence {v h} ~ V k belongs to 7g(~). Let us also define the sequence of sets of subgradients

ek= U u ~ l ) k v ~ V k

According to Valadier's formula, there exists a sequence of finite barycenters over Pk (see [15 T. 2, XIX, 2; 2] for finitude even in infinite dimension) such that

nk

qk = ~., Ak,pk, *" i=O

and, of course, rt k

Vk >_ 0, hk, >_ 0, ][] Ak, = 1. i ~ 1

Then, for each k, by definition of Pk, we can define two maps: uk: Pk ~' l)k and Vk: Pk ~ Vk such that

Vp ~Pk, P ~ ~lJ(uk(P), vk(P))"

For all u ~ U, one has

J(u, Vk(p)) > J(uk(P), ok(p)) + (p, U --uk(p)).

Since uk(P) ~ Oh,

Vu ~ U, J(u, oh(p)) ~J(~ , vk(p)) + (p, u - u k ( p ) ) - E k.

Due to the concavity of v ~ J ( u , v ) and the convexity of 1/, one can take the convex combination of all the inequalities in Pki, and obtain

nk

i = 0

where nk

wk = Y'~ Ak, vh(Pk,) ~ V. i = 1

It has been seen that {vk(p), p ~ Pk} ~ gr(~), so n k

ff 3K1, Vk>_K1, aj(a,vk(pk,)) >-Y- 7"

i = 0

Due to uk(p) ~ g~,p ~Pk and lemma 2 which provides the fact that the elements of Pk are bounded,

ff =IK2, V u ~ U , Vk>_K2, V p ~ P k , ( p , u - u k ( p ) ) > ( p , u - ~ ) - -~.

For all u in U, there exists an open neighborhood ~f(u) and an integer n such that

V ~ N ( u ) , Vk>_n, ( q k , g ~ - - ~ ) > ( q k , u - - ~ ) - - g >(/3, U- -~ ) - -2 _> 4"


As U is compact, a finite covering from the ~¢(u) can be extracted. Let K 3 be the maximum of the corresponding ns. Then

Vu ~ U , 'qk >_ g 3 , ( q k , u - t~) > 4 '

for u belongs to one of the G(u) selected in the finite covering. ThUS,

E Vu~U, Vk>max(K1,K2,K3), J(u,wk)>f-3- 4 - e k .

Then, a N > max(K I, K 2, K 3) can be chosen such that IE u _~< ~//4, and the claim is proved. •

A trivial, but may be useful corollary is as follows.

COROLLARY. Under the hypotheses VN1 and VN2, i f / 3 c U,

sup inf J ( u , v ) = inf sup J ( u , v ) . v~V u ~P u~O v~V

Proof. It suffices to make use of the continuity of J and f and to apply the previous theorem to the closure U o f / ] . []

R E F E R E N C E S

1. DANSKIN J. M., The Theory of Max Min. Springer, Berlin, (1967). 2. VALADIER M., Contribution ~ l'analyse convexe, Th~se d'6tat, Paris (1970). 3. AUBIN J. P. & EKELAND I., Applied Nonlinear Analysis. Wiley-Interscience, New York (1984). 4. AUBIN J. P., L'analyse non lin~aire et ses motivations ~conomiques. Masson, Paris (1984). 5. FLEMING W. H. & RISHEL R. W., Deterministic and Stochastic Optimal Control. Springer, New York (1975). 6. ROCKAFELLAR R. T., Extensions of subgradients calculus with applications to optimization, Nonlinear Analysis

9, 665-698 (1985). 7. CLARKE F. H., Optimization and Nonsmooth Analysis. Wiley-Interscience, New York (1983). 8. GAUVIN J., The generalized gradient of a marginal function in mathematical programming, Math. Operat. Res.

4, 458-463 (1979). 9. HIRIART-URRUTY J. B., Gradients g6n6ralis6s de fonctions marginales, SIAM J. Control Optim. 16, 301-316

(1978). 10. BONNANS J. F., Directional derivatives of optimal solutions in smooth nonlinear programming, Rapport de

recherche INRIA 1006, Rocquencourt, France (1989). 11. THIBAULT L., On subdifferentials of optimal value functions, SlAM J. Control Optim. 29, 1019-1036 (1991). 12. BERHARD P., Variations sur un th~me de Danskin avec une coda sur un th~me de Von Neumann, Rapport de

recherche INRIA 1238. Sophia-Antipolis, France (1990). 13. BA~AR T. & BERHARD P., H=-Optimal Control and Related Minimax Design Problems, a Dynamic Game

Approach. Birkhaiiser, Boston (1991). 14. SION M., On General Minimax Theorems, Pacif. J. Math. 8, 171-176 (1958). 15. SCHWARTZ L., Analyse: topologie ggndrale et analyse fonctionnelle. Hermann, Paris (1970). 16. VON NEUMANN J. & MORGENSTERN O., Theory of Games and Economic Behaviour. Princeton University

Press, Princeton (1947).

Pergamon 0362-546X(94)00186-3 - Inria · 2019-09-16 · On a theorem of Danskin with an application to a theorem of Von Neumann-Sion 1165 A somewhat different issue was taken up by

Documents